Anisotropy is Not Inherent to Transformers (2024)

Abstract: Isotropy is the property that embeddings are uniformly distributed around the origin. Previous work has shown that Transformer embedding spaces are anisotropic, which is called the representation degradation problem. This degradation has been assumed to be inherent to the standard language modeling tasks and to apply to all Transformer models regardless of their architecture. In this work we identify a set of Transformer models with isotropic embedding spaces, the large Pythia models. We examine the isotropy of Pythia models and explore how isotropy and anisotropy develop as a model is trained. We find that anisotropic models do not develop as previously theorized, using our own analysis to show that the large Pythia models optimize their final Layer Norm for isotropy, and provide reasoning why previous theoretical justifications for anisotropy were insufficient. The identification of a set of isotropic Transformer models calls previous assumptions into question, provides a set of models to contrast existing analysis, and should lead to deeper insight into isotropy.

## introduction much work has found that transformer models have globally anisotropic representations, which has been labeled the representation degradation problem @xcite . isotropy has two meanings: when using cosine similarity @xcite , it means the directions of representations are uniformly distributed, and when using a partition function @xcite distances must also be uniform. anisotropy has been shown to degrade downstream task performance @xcite , and an increase in isotropy correlates with better performance on some tasks. previous work has consisted of a set of theoretical justifications for the degradation and a large body of empirical experiments confirming global anisotropy. while no formal proof has been presented, due to the lack of any counterexamples, anisotropy is often taken as assumed for any transformer architecture. we identify the most globally isotropic models to date, the pythia models with at least 410m parameters @xcite , a strong counterexample to the assumption of anisotropy. these models are trained using cross-entropy loss, using autoregressive language modeling, and with a final layer norm. pythia model's most unique architecture feature is their untied embedding and unembedding matrices. pythia models have 143 evenly spaced checkpoints from training, allowing us to explore how isotropy changes during training. we explore the isotropy of pythia models using cosine similarity @xcite , a partition function @xcite , and our own analysis on the final layer norm of each model based on the theoretical work of @xcite . using multiple metrics allows us to present a more confident conclusion when all of our isotropy measures agree. contrary to previous work, which use token frequencies in the 1000s, we perform cosine analysis on 425m sentences from the actual training dataset, the pile @xcite . this allows us to include as many rare words as possible-standard methodology ignores words with frequency less than five, and to examine how isotropy might change across domains. in order to facilitate this analysis we reformulate average cosine similarity to a more computationally efficient form. our contributions are as follows: the representation degradation problem was introduced by @xcite for the unembedding matrix of transformers, with a similar result discovered in a model's hidden layers @xcite and later in sentence embeddings @xcite . many causes of anisotropy have been suggested, the optimal optimization solution of rare words @xcite , the gradient update of rare words @xcite , tying embedding and unembedding weights @xcite , linguistic biases @xcite , outlier neurons @xcite , or the loss function and attention mechanisms @xcite . most work has focused on the tied weights of the embedding (the matrix that maps tokens to input vectors) and unembedding (the matrix that maps output vectors to tokens) matrices, providing methods that increase isotropy and downstream task performance. these include token level methods focusing on the loss function @xcite @xcite , adjusting gradients @xcite , bias removal @xcite , mean centering, pca analysis, or clustering @xcite @xcite , and sentence level methods such as contrastive loss @xcite or normalizing the mean and variance of sentence embeddings @xcite . work that focuses on layers besides the unembedding layer includes cosine analysis @xcite , finding locally isotropic clusters @xcite , and "outlier neurons" found based on a dimension's contribution to cosine metrics @xcite , layer norm operations @xcite , and positional embeddings @xcite . these "outlier neurons" can correlate with token frequency @xcite and downstream task performance @xcite . we note, however, that the existence of outlier neurons can depend on the choice of orthonormal basis, and we could find no work linking this concept to principal component analysis, which could help standardize this methodology. recent work has shown that the existence of "outlier neurons" is not correlated with anisotropy @xcite , that increases in isotropy do not necessarily correlate with downstream task performance @xcite , that anisotropy does not degrade clustering tasks @xcite , that anisotropy causes models to rely on norm over direction @xcite , and that anisotropy should only degrade results when it is caused by linguistic biases @xcite . ## conclusions we have found strong evidence that the anisotropy of transformer models can not be assumed. we show that large pythia models are isotropic across all large model sizes using numerous metrics. we suspect having untied embedding and unembedding matrices leads to higher isotropy, and show that, contrary to previous assumptions, pythia models in fact optimize the final layer norm operation for isotropy. we have also explored how isotropy changes during training across different model scales. this work, providing a set of contrasting points, is a good first step into a deeper understanding of isotropy and its impacts. future work should consider an analysis of bias @xcite and clustering @xcite for these isotropic models, and a proper ablation study to confirm that untied embedding matrices is the root cause of this isotropy.