And for stuff like writing code, it turns out to be not as helpful as expected.

This is not a good judgment and is just taking the ChatGPT user interface at surface value.

It has simply been that LLMs like GPT4 have not been allowed to use tools to write programs outside of lab conditions, so it’s the equivalent of you running code in your head based on what is already in your memory.

Once an LLM has access to compilers or interpreters that run code, they can feedback their own mistakes into the next prompt and write working code. We already know that GPT4 can learn python, bash and other interpreted languages by simply allowing it to use the tools and allowing it to feed results back into new prompts. It can also tell which tool to use, based on the input.

The concept of tool use in LLMs is almost the same as for humans in amplifying a specific ability, such as using a calculator for numerical computations or using an SQL database to manage large tables of information.

The tool use that ChatGPT allows today is simply prompting search engines or Dall-E, reading some webpages as input prompts, but there is no feedback loop allowed for fact checking itself.

Alright thanks. This stuff is moving very fast, and I was only looking at the master branch.

Of note, TensorRT doesn’t support SDXL yet.

I realize that MacOS users will stay with Mac for a long time, but I wonder how much of a leap in performance a TR+4090 rig is versus an Apple Silicon Mac Studio on M2 Max, power consumption be damned, on apps that are common to both MacOS and Windows.

While there is a certain performance difference now between two such systems, when you go out and upgrade your PC to a 5090 in a couple of years and a 6090 in 4 years, the difference will be laughable.

If you want your Mac Studio to stay on the cutting edge of GPU power, the costs of constantly upgrading a whole Mac Studio vs. just the PC GPU is much worse than the initial cost of the first system of either type.

The Mac Studio just cannot function long term as a GPU powerhouse. You can gloat about it for 6-12 months and that’s it. It’s a machine that can work solidly for you for 10 years, if you don’t demand cutting edge GPU performance, but it will be relegated to “servicable performance” in 5-7 years.