The Geometry of Tokens in Internal Representations of Large Language
  Models
Authors: name: Karthik Viswanathan, name: Yuri Gardinazzi, name: Giada Panerai, name: Alberto Cazzaniga, name: Matteo Biagetti

We investigate the relationship between the geometry of token embeddings and
their role in the next token prediction within transformer models. An important
aspect of this connection uses the notion of empirical measure, which encodes
the distribution of token point clouds across transformer layers and drives the
evolution of token representations in the mean-field interacting picture. We
use metrics such as intrinsic dimension, neighborhood overlap, and cosine
similarity to observationally probe these empirical measures across layers. To
validate our approach, we compare these metrics to a dataset where the tokens
are shuffled, which disrupts the syntactic and semantic structure. Our findings
reveal a correlation between the geometric properties of token embeddings and
the cross-entropy loss of next token predictions, implying that prompts with
higher loss values have tokens represented in higher-dimensional spaces.