(Posted for the whole Steering Council.) As we’ve announced before, the Steering Council has decided to accept PEP 703 (Making the Global Interpreter Lock Optional in CPython) . We want to make it clear why, and under what expectations we’re doing so. It is clear to the Steering Council that theoretically, a no-GIL (or free-threaded) Python would be of great benefit, and the majority of the community seems in agreement. Threads have significant downsides and caveats, but they are widely adopte...
Interesting, so this means that it could be used for high performance computing if the GIL became optional?
Pure python will likely never be used for that, but python already is used in HPC mostly as a DSL over native code.
There’s really no reason why you couldn’t write a bunch of python that produces a lazy compute graph that can be compiled or optimized under the hood for HPC right now.
The removal of the GIL just makes some stuff a lot easier to parallelize at the python level. Multiprocessing can have a lot of overhead and this would be a nice way to scale up a little.