by ben_w | 2025-08-28 08:07:22+00:00
> Firstly, what is AGI? I've never heard a decent definition. Some say it's an AI that is as smart or as general as humans; some say it's an AI that's conscious.

Correct, but many have noticed this including Sam Altman in some interviews.

Everyone disagrees about all three initials of the initialism, plus if the whole even means something implied by those initials at all, plus treating each as a boolean rather than a number.

Also some people loudly reject this observation.

The definition I was using for "AGI" befor ChatGPT came out was met by ChatGPT-3.5: it's an AI that's general-purpose, it doesn't just e.g. play chess. But there's people who reject that it even counts as "AI" at all, despite, well, the Turing Test.

Anyway.

> I don't see how it could be a "step" by these definitions because nothing actually changes between a regular AI and an AGI. To make AI more general, you just make the input tokenization more granular, change the training method, and perhaps add some kind of iterative framework on top to make it do more stuff.

That's a lot of things.

> Also, AI is already more capable than humans in almost every way except scale.

No, not really. There's specific metrics where some AI can beat a lot of humans, like playing chess, or how many languages it speaks to the level of a competent adult learner, and they can pay attention to a lot more than we can, and they can process data faster than we can, but…

…but LLMs* are only giving the illusion of intelligence by using superhuman speed and superhuman attention to make up for having a mind with the complexity of a squirrel's that's had half a million years of experience reading everything it can lay its metaphorical hands on.

* not VLMs, they're not fast enough or smart-looking enough yet

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain, but that's because IQ tests compare competence to get relative intelligence; they don't test computation.

IQ tests are indeed bad, can be learned. This is demonstrable because the graph you may have seen for AI IQ ratings has two variants from the same people, one with a public IQ test and one with a private IQ test, and most of the AI do much much worse with the private test: https://trackingai.org/home

The problem you're pointing at is also why the ARC-AGI test exists, and indeed this shows that current AI aren't anything close to human performance.

> With that assumption, if AI can't be computationally stronger than humans, it's safe to say we won't have conscious computers for a while, but instead computers that act consciously instead.

If you think "AGI" is poorly defined, you'll be horrified to learn that "consciousness" has something like 40 definitions. Nobody knows where it comes from in our brain structures or how much structure we need to have it. Is a mouse conscious? If so, there's enough compute going on for many AI models to be also. But the quantity of compute is likely a red herring, and the compute exists even if it's not running an AI, just as our brain chemistry still goes on while we're sleeping.

> Does that mean that AI from hear on out is a waste that does nothing but take control from people while benefiting us the same?

Yes, but such people will take any opportunity or tech to do so, we've seen that since "automation" meant "steam engine".

> What is ChatGPT going to look like in 5 years? Am I just going to type in "do my taxes," and it's just going to do whatever it wants on my pc until my taxes are done?

If you're lucky. I think probably not, but 5 years is too long to rule out an architectural breakthrough that makes them much less error-prone; the Transformer models in 2020 were unimpressive for anything except translation.

> Why would I ever want that over a system designed to do my taxes correctly EVERY TIME by accountants?

$$$; plus accountants aren't perfect, they're only human.

> One thing I know about AI is it is slow as a mf, AI is great but you really have to think, we really just built a giant dictionary guy who's going to have the same problems human employees have.

If we're lucky.

Most likely it will continue to have new and exciting problems.