> A shocking amount can be inferred from the internet by leveraging this straightforward (I won't say "simple"!) task.

Nobody sane would argue that. It is very visible that ChatGPT could do things.

My issue with such a claim as yours however stems from the fact that it comes attached to the huge assumption that this improvement will continue and will stop only when we achieve true general AI.

I and many others disagree with this very optimistic take. That's the crux of what I'm saying really.

> There is a lot of wisdom in there that LLMs evidently can distill out.

...And then we get nuggets like this. No LLM "understands" or is "wise", this is just modern mysticism, come on now. If you are a techie you really should know better. Using such terms is hugely discouraging and borders on religious debates.

> Now about high level engineering decisions: the parent comment said that high level experience is not spelled out in detail in the training data, e.g., on stack overflow. But that is not required.

How is it not required? ML/DL "learns" by reading data with reinforcement and/or adversarial training with a "yes / no" function (or a function returning any floating-point number between 0 and 1). How is it going to get things right?

> All that high level wisdom can probably also be inferred from the internet.

An assumption. Show me several examples and I'll believe it. And I really do mean big projects, no less than 2000 files with code.

Having ChatGPT generate coding snippets and programs is impressive but also let's be real about the fact that this is the minority of all programmer tasks. When I get to make a small focused purpose-made program I jump with joy. Wanna guess how often that happens? Twice a year... on a good year.

> It's not a bad bet that with these early LLMs we haven't seen the limits of what can be inferred.

Here we agree -- that's not even a bet, it's a fact. The surface has only been scratched. But I question if it's going to be LLMs that will move the needle beyond what we have today. I personally would bet not. They have to have something extra added to them for this to occur. At this point they will not be LLMs anymore.

> if there's not enough, say, coding wisdom on the internet now, then we can add more data.

Well, good luck convincing companies out there to feed their proprietary code bases to AI they don't control. Let us know how it goes when you start talking to them.

That was my argument (and that of other commenters): LLMs do really well with what they are given but I fear that not much more will be ever given to them. Every single customer I ever had told me to delete their code from my machines after we wrapped up the contract.

---

And you are basically more or less describing general AI, by the way. Not LLMs.

Look, I know we'll get to the point you are talking about. Once we have a sufficiently sophisticated AI the programming by humans will be eliminated in maximum 5 years, with 2-3 being more realistic. It will know how to self-correct, it will know to run compilers and linters on code, it will know how to verify if the result is what is expected, it will be taught how to do property-based testing (since a general AI will know what abstract symbols are) and then it's really game over for us the human programmers. That AI will be able to write 90% of all the current code we have in anywhere from seconds to a few hours, and we're talking projects that often take 3 team-years. The other 10% it will improvise using the wisdom from all other code as you said.

But... it's too early. Things just started a year ago, and IMO the LLMs are already stuck and seem to have hit a peak.

I am open to have my mind changed. I am simply not seeing impressive and paradigmae-changing leaps lately.