We are encountering the equivalent of a mirror-test, but one that says more about us than it does about the mirror ( https://www.youtube.com/watch?v=w6ChEmjsXCM | ). Many non-human animals when they encounter mirrors for the first time think they are looking at another individual with autonomy and agency.

We are feeling the same now. As of now, LLMs are still mirrors, a complex kaleidoscopic kind that retain all the light and shape of things reflected at them, remix them, and spit them out as reflections that look like other individuals, conscious individuals with shape-shifting personalities.

That’s a cocksure assertion isn’t it. To be able to say all this confidently, we'd first need to agree on a non-fuzzy definition of consciousness, and come up with a good computational model for consciousness that we'll be able to use to evaluate and grade the AIs. (IIT is not a good model)

Turns out we do have a great model. I co-authored a book that, among other things, discusses this model (https://www.goodreads.com/book/show/58085266-journey-of-the-...)

Here’s a summary where I discuss the book and how the things we discuss there can inform our current and increasingly urgent and important discussions about AI

https://saigaddam.medium.com/understanding-consciousness-is-...

I’ll summarize the summary here:

Consciousness is the disambiguation of sensory data into meaningful information. Data can become information only through a perspective. Who provides that perspective? The self, which is nothing but the totality of all our previous experiences. We are not our kidney or liver. We are our experiences stitched together into some strange web.

To put it another way: Consciousness is the constellation of past experiences experiencing the present, assimilating it to act and prepare for future opportunities.

Using this definition, we can try and understand what we are seeing with the likes of ChatGPT and Sidney (apparently that’s what Bing’s GPT calls itself)

The persona we seem to shine through in the chatbot’s reflection is nothing but some stable set of experiences it has had. Experiences here all the hundreds of billions of fragments of data they have been fed. As a result, they seem to have experience sets of every personality type or archetype. Why or how they seem to get steered towards the same archetypes is a fascinating question. Is it because of the new reinforcement learning methods (RLHF) that reward certain kinds of questions? Or is it that the we are self-selecting for the most unsettling encounters with the new mirror and putting them online? My guess is both.

To come back to the first question of consciousness. Are they conscious? No. A better way to think of LLMs is that they might have leapfrogged consciousness to become consciousness compilers. It is possible to simulate a conscious being and get it to play one, but it isn’t really conscious yet. The experience set does not get updated with every encounter with the world (at least for the ones we have now), and crucially, it does not have the idea or conception of a body that its consciousness is serving. This is the other point so many miss out when discussing consciousness and intelligence. Consciousness and intelligence took very little time on the evolutionary scale of things once autonomy was in place. Autonomy is the real hard problem. Consciousness and intelligence without autonomy will be great imitations but never truly seem like the real thing because that chatbot can’t really “do” anything that benefits “itself”.