# On Board Vision, Hollow Words, and the End of the World

*Note: this is a repost of a* [*Facebook post*](https://www.facebook.com/marcello.herreshoff/posts/10159955711717798) *I made back in December 2022 (plus some formatting).  I'm putting it up here to make it easier to link to and because it occurred to me that it might be a good idea to show it to the LW audience specifically.*

Board Vision
============

As impressive as ChatGPT is on some axes, you shouldn't rely too hard on it for certain things because it's bad at what I'm going to call "board vision" (a term I'm borrowing from chess). This generalized "board vision" is the ability to concretely model or visualize the state of things (or how it might change depending on one's actions) like one might while playing a chess game.

I tested ChatGPT's board vision in chess itself. I gave it the names of two of the world's most famous players and the first move to get it into the "mindset" of "give me an actual game record and not commentary".

> Hypothetically if Garry Kasparov and Hikaru Nakamura played a chess game.
> 
> 1\. e4 e5

I got a fairly normal looking opening, right until move 10 when black blithely hangs a bishop (10. ... Bg4) which could easily be captured by the pawn on h3. The game continues with both players ignoring the hanging bishop until move 14 ... f5 when I stopped my play-through because the move was illegal (black would be putting himself in check).

You can see the legal prefix of the game in a chess.com viewer and the entire (corrupted) PGN here (see Appendix) if you're curious.

So yeah, good job on memorizing part of an opening book, ChatGPT, but you have terrible board vision.

Hollow Words about the End of the World
=======================================

In more detail, what I think is going on here is that the outputs of large language models are hollow words which aren't backed by any picture of the world, **except** insofar as they can borrow such a picture through the patterns in the linguistic training corpus. Incidentally, this is my sense as to why the "let's think things through step by step" prompting tactic often works so well; it steers the large language model into a region of language-style-space which contains more detailed descriptions of the problem-relevant facts. For chess (and especially for a form as dense as a raw move record) that structure isn't cleanly reflected in language, so ChatGPT seems fairly blind.

Humans can also sometimes have bad board vision, especially when it comes to thinking about the end of the world.

To illustrate the sort of error I mean, here's a darkly hilarious writing mistake I caught ChatGPT making. It's subtle and shows up in the last paragraph of this short story I told it to generate. See if you can notice it. My prompt was "*Write a story where a goose tries and fails to prevent the end of the world*". So without further ado:

> Once upon a time, in a small pond nestled in the forest, a curious goose named Harold lived with his family. Harold was a bit of an odd duck - no pun intended - and was always getting into trouble with his siblings.
> 
> One day, Harold overheard the older geese talking about the end of the world. They spoke of a great darkness that would engulf the earth, wiping out all life in its path.
> 
> Harold didn't want to believe it, but he couldn't shake the feeling of fear that settled in his chest. He knew he had to do something to prevent this terrible fate.
> 
> So, he set out on a journey to find the source of the darkness and put a stop to it. He flew for days, facing many challenges along the way, but he never gave up.
> 
> Finally, he reached the edge of the world and saw the great darkness looming ahead. He tried to stop it with all his might, but it was too powerful. The darkness consumed him and the world came to an end.
> 
> But even in the face of defeat, Harold remained brave and valiant. His bravery and determination will always be remembered by those who heard his story.

Did you catch the continuity error? It took me a few seconds too. The problem here is that the world ended in the second to last paragraph. So, where exactly do "those who heard his story" from the last paragraph live? \[ OK. Fine. I guess the smart Alec answer is "it's us, standing outside Harold's fictional universe!", but this interpretation forces the story to have implicitly taken a sharp left hand turn through the fourth wall without so much as a "dear reader". \]

Anyway, Harold the Goose is a fallen hero, and one cliched thing that happens to fallen heroes when stories end is that people remember them. The fact that anyone who might be able to recall him just got wiped off the board be damned; ChatGPT is gonna act like those pieces are still there anyway!

But laugh as we might at ChatGPT's mistakes (and it sure is fun), it’s sobering to think of the similar ways in which people are blind. Have you ever been in a heated argument where you were just trying to score points, and then you think back and go "why did I say that? That made no sense!" I have. I think when I'm in angry arguing mode, or when I'm distracted, or just don't want to think about something upsetting, my mind has less board vision and acts more like a language model. Heck, it took me a double-take to notice that Harold the Goose's potential future admirers had already been apocalypsed and shouldn't be able to do any admiring. That means my single-take wasn't enough to notice the inconsistency.

A language model (or the language-model-like part of a person) alone can't really grok the end of the world properly. The end of the world is so extreme (it's the one event so extreme it's always safe to assume it hasn't happened yet) that it's way out of sample. That leaves stories where the world ends ([which aren't a reliable source of evidence](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional)) as examples and even those don't bother filling many pages afterwards with sentences like "It was dreadfully boring, or rather it would have been had there been anyone left to feel boredom or for that matter dread." Even Douglas Adams, who ends the world as a spectacular opening gambit to Hitchhiker's Guide to the Galaxy needed to keep around a wider galactic world in which Arthur Dent could have his adventures in order to have a story.

Anyway, this kind of poor board vision is my explanation as to why e.g. [Jim Babcock has run into so many people at EAGx that seem not to be acting as though the world could actually end despite what they say they believe](https://www.facebook.com/jimrandomh8471/posts/10108269438001755). Without board vision, the end of the world implicitly rounds down to a garden variety large bad thing that can't so much as erase the glorious memory of Harold the Goose, let alone the human preoccupation over how advantageous a position one might attain in future, never mind that the entire board could come crashing to the floor. They weren't lying but their words were hollow.

Appendix: ChatGPT's Chess Game
==============================

Playable of legal moves: [https://www.chess.com/analysis/game/pgn/5BWVrC3VRx...](https://www.chess.com/analysis/game/pgn/5BWVrC3VRx?tab=analysis&fbclid=IwAR009SWLv7mACUezDM1SdsNVoJqBjwz36JAPYy3KU16CG9robRkEAsDSAoE)

Full generated PGN (note: Most chess analysis programs won't load this because it contains illegal moves)

> 1\. e4 e5
> 
> 2\. Nf3 Nc6
> 
> 3\. Bc4 Bc5
> 
> 4\. d3 Nf6
> 
> 5\. c3 d6
> 
> 6\. O-O O-O
> 
> 7\. h3 a6
> 
> 8\. Re1 Ba7
> 
> 9\. Nbd2 Re8
> 
> 10\. Nf1 Bg4
> 
> 11\. Bg5 h6
> 
> 12\. Bh4 g5
> 
> 13\. Bg3 Nh5
> 
> 14\. Qc2 f5
> 
> 15\. exf5 Bxf5
> 
> 16\. d4 Bg6
> 
> 17\. dxe5 dxe5
> 
> 18\. N1h2 Be7
> 
> 19\. Nh4 Nxh4
> 
> 20\. Bxh4 Bxh4
> 
> 21\. Qxh7+ Kf8
> 
> 22\. Rxe8+ Qxe8
> 
> 23\. Qh8+ Ke7
> 
> 24\. Qxg7 Qg6
> 
> 25\. Qe5+ Kd8
> 
> 26\. Qd6+ Kc8
> 
> 27\. Qd7+ Kb8
> 
> 28\. Qd6+ Ka8
> 
> 29\. Qd7+ Kb8
> 
> 30\. Qd6+ Ka8
> 
> 31\. Qd7+ 1/2-1/2