“Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models (2021)

Abstract: Contextualized representations based on neural language models have furthered the state of the art in various NLP tasks. Despite its great success, the nature of such representations remains a mystery. In this paper, we present an empirical property of these representations—”average” approximates “first principal component”. Specifically, experiments show that the average of these representations shares almost the same direction as the first principal component of the matrix whose columns are these representations. We believe this explains why the average representation is always a simple yet strong baseline. Our further examinations show that this property also holds in more challenging scenarios, for example, when the representations are from a model right after its random initialization. Therefore, we conjecture that this property is intrinsic to the distribution of representations and not necessarily related to the input structure. We realize that these representations empirically follow a normal distribution for each dimension, and by assuming this is true, we demonstrate that the empirical property can be in fact derived mathematically.

## introduction a large variety of state-of-the-art methods in nlp tasks nowadays are built upon contextualized representations from pre-trained neural language models, such as elmo @xcite , bert @xcite , and xlnet @xcite . despite the great success, we lack understandings about the nature of such representations. for example, @xcite have shown that averaging bert representations in a sentence can preserve its domain information. however, to our best knowledge, there is no analysis on what leads to the power of averaging representations. ## generality tests of the property different layers. to evaluate our discovered property's generality, we first investigate if this property only holds for the last-layer representations. for the four transformer-based language models, there are 13 possible layers (i.e., one after lookup table and 12 after encoder/decoder layers) to retrieve representations for tokens. therefore, we test the property based on representations from each layer and plot the average absolute cosine similarities in figure 2 . one can see that the property holds for the last few layers in all four models. random initialized models. we repeat the same test for randomly initialized models, i.e., not (pre-)trained at all. the results are in figure 2 . again, we can see that the property holds for the last few layers in all four models. random sentence. finally, we explore the case when the representations can even come from different sentences. specifically, we shuffle all the last-layer token representations of the 4,000 sentences and re-group them into 4,000 random lists of representations. with a high probability, each token representation in a list is generated independently of other tokens from the same list. we show the results in random sentences section in ## analysis in this section, we attempt to answer what could be a reason that the language models show this property. from section 4, we know that the property also holds for randomly initialized models. such models know nothing about natural languages. therefore, it is reasonable to believe that this property is intrinsic to the models and related to the distribution of these representations. ## related work simply averaging is a widely used, strong baseline to aggregate (contextualized) token representations @xcite @xcite @xcite . in this paper, we discover an empirical property of these representations ("average" ≈ "first principal component"), which can justify its effectiveness. there are other attempts to analyze properties of language models. @xcite analyze syntactic information that bert's attention maps capture. k et al. (2020) prune the causes for multilinguality of multilingual bert. @xcite show that position information are learned differently in different language models. different from these language-specific properties, we believe our newly discovered property relates more to the internal structure of neural language models. ## conclusion and future work this paper shows a common, insightful property of representations from neural language models-"average" ≈ "first principal component". this property is general and holds in many challenging scenarios. after analyzing the bert representations as a case study, we conjecture that these representations follow a normal distribution for each dimension, and this distribution leads to our discovered property. we believe that this work can shed light on future directions: (1) identifying the distributions that representations from language models follow, and (2) further implications or properties that representations have.