This is an automated archive made by the Lemmit Bot.

The original was posted on /r/programminglanguages by /u/Inconstant_Moo on 2024-11-12 06:02:17+00:00.

Obviously pretty much any large software project gets nonlinearly more difficult as it gets bigger. But some of them, not very much. For example, the way your spreadsheet app renders pie charts doesn’t have to know about the bit that does bar graphs.

But when we do langdev, language features ought to be composable. Every core feature should work naturally with all the other core features. In fact, what makes them core features is that they have to be special-cased by hand to work together. Otherwise they can be turned into built-in functions or standard libraries, which you should do, and then you’re back in that happy O(n) space where your time library doesn’t have to know about your regex library, etc. For features to be core features means that you have to hand-code them to work together, or they wouldn’t have to be core.

I have sometimes jokingly stated it as a “law” of langdev that “for every two ‘orthogonal’ features there is a corner case”. But it could be turned into a real law just by changing that to “potential corner case”. That’s not a joke, that’s true by definition.

And so langdev is O(n²) when n is the number of features. I learned that much when I was writing my tree-walking prototype, and thought I was becoming a wise old man. Then I wrote the VM and discovered that n is not just the number of features but also the number of compiler optimizations. For example, every language feature may or may not fight with the constant folding. Every time I implement anything, I have to think about that, and I have to write tests to make sure that it doesn’t happen. This one extra implementation feature is in fact n implementation features, because at least potentially it might fight with everything I might do.

If anything I’ve understated the problem. Some things just can’t be done well. In the words of the old joke, “you can’t get there from here”. Golang, for example, has high overhead on its FFI. Why? Because of the way it does concurrency. Google has billions of dollars to hire the world’s best devs, but on that issue they’ve basically conceded defeat.

Or take adding dependent types to Haskell. If you look at this link, here’s a guy adding dependent types to his toy language in 80 lines of ML. And here’s the brave souls who’ve spent years trying to add it to Haskell.

But at the very least, langdev is O(n²). A core language feature is by definition a feature that can at least potententially fight with all the other core language features, which is the exact reason why you’re implementing it as a core language feature, rather than as a standard library or a built-in function.

When I first wrote this diatribe, it ended with the words “We’re all screwed. Have a nice day.” However, more optimistically, the reason I’ve survived this ordeal and that Pipefish still works is … here’s my big magical secret … have a big test suite.

I recently saw someone on this community saying something like “Because the language I was working on was so full of bugs, I abandoned it and now I’m working on a whole new language.” That new language will also be full of bugs, and will also be abandoned, because that person is a bad developer. The project should never have been full of bugs in the first place. It should have been full of verifiable successes, demonstrated by automated tests. There is no other way to maintain a project that is intrinsically O(n²).

So, maintain a large test suite … and have a nice day.