> Everything else remains the same.

For now, which may be as little as two years, given the release cadence between GPT 3 and GPT 4, combined with the increased funding levels.

As several research papers have pointed out, GPT 3 may be just a "stochastic parrot", but GPT 4 has "sparks of AGI". I have no idea what GPT 5 will be capable of, but it will be... more.

Next, many people have already done the obvious thing and wired up LLMs in a loop, where code output is automatically compiled and/or tested, with the errors fed back into the LLM to make it automatically fix up any small mistakes.

This is nothing; it's just sticky-taping something on with external API calls. The real output of LLMs is a probability distribution over the output dictionary, which can be filtered so that instead of naively picking the "most likely output", the LLM will pick the "most likely syntactically valid output". For programming languages, this is relatively simple to add on. For non-programming output generation, this is a much harder problem to solve. In other words, expect to see output quality in SWE fields improve faster than any other field!

Even the existing GPT 4 has a (private-preview) ability of looking back 32K tokens, which is about 15KB of source code. This is significantly more than what can be processed now, and is amenable to further optimisation. For example, the required token budget decreases by 40% simply by converting 4x space characters to 1x tab character. Eliminating the leading white space entirely cuts that in half again.

Tooling is being integrated into ChatGPT and similar LLM-based web apps. Right now, these are just doing simple Google or Bing searches, but as the Wolfram Alpha integration shows, much more powerful integrations are possible. Just wait and see what happens when ChatGPT gets hooked up to a short-term memory scratchpad and a programming playground. And then see what happens when it gets specialised on this automatically, feeding back compiler errors or failed tests as negative reinforcement.

I can foresee a future LLM like GPT 5 replacing the need for junior developers almost entirely. A single senior developer can just give it instructions and have it bang out code at the same rate as 100 junior devs, but with the quality of average developers.

There's an industry metric that a typical developer in a large enterprise produces only 3-10 lines of code per day on average. Maybe we'll have a single senior developer + LLMs be able to bang out hundreds per day?

PS: Imagine even a simple IDE plugin where in the context of the line of code you're writing Chat GPT will go off in the background and read the entire source file, the associated documentation, KB articles, Stack overflow answers, and GitHub issues and make recommendations on the fly as you type!

Picture a little popup following your edit cursor around with little warnings like "this is actually buggy because of Issue #3512, you'll need a retry loop to work around", or "calling that in that way is a security hole mentioned in CVE-2023-####", etc...

PPS: LLMs like GPT are slow at producing output, but ridiculously fast at reading, able to process something like 10K-100K words per second.