Do AI language models dream of word token sheep?