• jaerie@fediverser.communick.devB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    They said multithreading can’t do parallel computing, what part of that is false?

    Besides, going to multiprocessing isn’t just “a little overhead” you need to switch from a shared data model to inter process communication, which isn’t always trivial

    • secretaliasname@fediverser.communick.devB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      There is a common dev story in python: Hrmm this is running slow, maybie I can use threads to make it go faster. Weird, not faster, discovers GIL. Maybe I can use multiprocessing. Hrmm this sucks I have to use IPC and serialize things to pass them. Hrmm faster but still weirdly slow. Proceeds to spend a ton of time optimizing IPC and figuring how to get code in multiple processes to communicate.

      • ajslater@fediverser.communick.devB
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        GIL removal solves the relatively small problem of, “I have a big workload but not so big that I need multiple nodes.”

        Small workloads are fine and don’t need free threading. Large workloads are going to use IPC anyway to coordinate across hundreds of nodes.

        Today you must use the IPC overhead approach for medium workloads and it is some extra work. But then if your application grows you’ve already done much of the scaling part.