This release includes the Beta version of the Ruff formatter — an extremely fast, Black-compatible Python formatter.
Try it today with ruff format.
Changes
Preview features
[pylint] Implement non-...
Tell that to Anaconda/Poetry. If I remember correctly, there’s some deficiency with Python packaging that makes dependency solving harder for than it is for other languages.
I believe it’s because when the metadata is missing, you need to download the entire package and try installing it to check compatibility.
Of course, this could still be mitigated by generating metadata / precomputing dependencies, or creating and hosting “mini” package proxies for dependency resolution only for the big packages, or many other engineering solutions, if the poetry devs were sufficiently motivated.
No one’s saying that we have to solve the Halting Problem (or similar) to get massive speedups in many cases.
I believe it’s because when the metadata is missing, you need to download each version of an entire massive package and try installing it to check compatibility. Repeated for every possible version, backwards until one works.
I think you’re right; that definitely seems to be poetry’s behaviour. So that’s clearly IO-bound. On the other hand, Anaconda got massive speedups by switching to a better dependency solver. Dependency resolution is basically SAT, after all.
The question isn’t a matter of creating a unified tool but of figuring out why past attempts (e.g. poetry) haven’t taken off more and if those problems can be avoided.
The only thing holding poetry back is still not supporting the standardized pyproject.toml. That’s the closest thing to cargo that we have. And issues related to various ML libraries
I see typechecking as the next natural extension to this. Won’t be easy, it’s probably the trickiest thing to do well, but it would be the ultimate python tool if it did that. It’s a natural extension point as well, since they have the parsing down.
For me, a big problem with poetry is the author’s insistence on being as strict as cargo on dependency resolution when the Python ecosystem doesn’t have the culture to go with it. You need to be able to override bad transitive dependencies.
I’m personally using pixi and ruff. Pixi is the first tool in the python ecosystem to solve all my package management woes (and runs fast). They’re conda based though (but they’re working on doing solves across pypi and conda).
Now make it do package management and I’m fully in
I don’t think that’s significantly CPU-bound, although you can certainly slow it down with a bad dependency-resolution algorithm (as pipenv did).
It’s not about the speed, which is trivial. It’s about having an unified tool like cargo
Tell that to Anaconda/Poetry. If I remember correctly, there’s some deficiency with Python packaging that makes dependency solving harder for than it is for other languages.
I believe it’s because when the metadata is missing, you need to download the entire package and try installing it to check compatibility.
Of course, this could still be mitigated by generating metadata / precomputing dependencies, or creating and hosting “mini” package proxies for dependency resolution only for the big packages, or many other engineering solutions, if the poetry devs were sufficiently motivated.
No one’s saying that we have to solve the Halting Problem (or similar) to get massive speedups in many cases.
I think you’re right; that definitely seems to be poetry’s behaviour. So that’s clearly IO-bound. On the other hand, Anaconda got massive speedups by switching to a better dependency solver. Dependency resolution is basically SAT, after all.
The question isn’t a matter of creating a unified tool but of figuring out why past attempts (e.g. poetry) haven’t taken off more and if those problems can be avoided.
The only thing holding poetry back is still not supporting the standardized pyproject.toml. That’s the closest thing to cargo that we have. And issues related to various ML libraries
I see typechecking as the next natural extension to this. Won’t be easy, it’s probably the trickiest thing to do well, but it would be the ultimate python tool if it did that. It’s a natural extension point as well, since they have the parsing down.
There’s pylyzer for a Rust-based type checking LSP alternative to Pyright, but I couldn’t really get it to work with Neovim.
For me, a big problem with poetry is the author’s insistence on being as strict as cargo on dependency resolution when the Python ecosystem doesn’t have the culture to go with it. You need to be able to override bad transitive dependencies.
It definitely is.
I have (or at least, have had) modest poetry projects that took 10 minutes to resolve on a top end CPU.
That’s because it is network bound. Python dependency resolution is a mess, poetry has to download a bunch of stuff to resolve things properly.
Judging by the tweets from Charlie Marsh, I think it’s being done 👀
While we are wishing, it should generate a single file executable.
I’m personally using pixi and ruff. Pixi is the first tool in the python ecosystem to solve all my package management woes (and runs fast). They’re conda based though (but they’re working on doing solves across pypi and conda).
Dependency resolution is mostly limited by network afaik. I really like pdm
I would like a fast type-checker, but it’s a really hard tool to do
Wtf no don’t do package management, lol.