by ben_w | 2025-05-26 16:36:26+00:00
> You can program a computer to simulate, display and express all sorts of things.

Indeed. This is also my counter-argument to anyone who is too eager to say any given AI "must" have any given internal state like hope or fear — I was rather hoping my words were sufficient to point to "we don't know" rather than either "yes" or "no".

> But this doesn't make it truly self aware. A huge gap in functionality exists between expression and genuine awareness and understanding. I think AI fans tend to lose sight of this.

"Self awareness" is the easy part, even quite simple software like the C64 OS has that. Just have some of the sensory input point to the innards themselves rather than to the outside world.

As I say, 40 different definitions of the word "consciousness".

> An LLM is still just a programmed computer. Much of it's programming is derived from statistics and examples instead of explicit logic/code but it is still programmed and as such is still constrained to playback of some variation of it's input.

To say an LLM is "programmed" is like saying the knowledge in our brains comes from our genes. The programming is just the architecture.

And while the limitation you give here is definitely correct, it is so because that's the nature of information and also applies to humans. When it looks like I'm creative, I'm mixing existing ideas and modifying them a little here or there and seeing if the result works. I can create variations of what I learn and experience, and I can choose where I go and what I experience or allow others to do so for me, but I can't speak a word of Igbo language* because I've not experienced any. Ask me to invent a new language from scratch, and a linguist will likely say it's a trivial modification of English or German, because I don't know enough linguistics to even be as alien as Chinese or Latin.

Doing things bit by bit also applies to the species as a whole, which is why we didn't jump straight to the scientific method as soon as we had writing — we had to invent each rung on the ladder one meme (in the Dawkins' sense of the word) at a time, within the constraints of our own minds and societies, because too large a step is what led to Galileo being called a heretic and Ignaz Semmelweis' nervous breakdown.

> My favorite example of LLM understanding and "awareness" (of rather the lack thereof) --- a chatbot accused a pro basketball player of vandalism because it found references to him "throwing bricks" during play. In other words, any real understanding just isn't in there yet.

When I was a very young child, my dad was ill in bed, talking to my mum, and I was also in the room in the way small children often are. He said "the tissue was damaged", so I offered him my handkerchief.

I will quite happily say that LLMs are "stupid", because they need such a ridiculously large number of examples to learn from**, but this is not the same thing as "are they self-aware?" or "are they [insert any other definition of consciousness here]?"

* Except for any loan words that are the same as English or German words, the same way Welsh has "ambiwlans" and "teleffon", but even then I don't know that I know them.

** Caveat: they can be stupid very very quickly, and for some tasks (anything with self-play) that's good enough.