Reconstruction Probing (2023)

Abstract: We propose reconstruction probing, a new analysis method for contextualized representations based on reconstruction probabilities in masked language models (MLMs). This method relies on comparing the reconstruction probabilities of tokens in a given sequence when conditioned on the representation of a single token that has been fully contextualized and when conditioned on only the decontextualized lexical prior of the model. This comparison can be understood as quantifying the contribution of contextualization towards reconstruction—the difference in the reconstruction probabilities can only be attributed to the representational change of the single token induced by contextualization. We apply this analysis to three MLMs and find that contextualization boosts reconstructability of tokens that are close to the token being reconstructed in terms of linear and syntactic distance. Furthermore, we extend our analysis to finer-grained decomposition of contextualized representations, and we find that these boosts are largely attributable to static and positional embeddings at the input layer.

## introduction model building in contemporary natural language processing usually starts with a neural network pretrained on the objective of context reconstruction ("language modeling"). contextualized representations of complex linguistic expressions from such models have been shown to encode rich lexical and structural information @xcite , making these models an effective starting point for downstream applications. probing pretrained language models aims to understand the linguistic information they encode, and how well it aligns with our understanding of human language (see belinkov 2022 for a review). the methodologies employed include supervised classifiers targeting specific linguistic properties of interest @xcite @xcite , similarity-based analyses @xcite , clozetype tests @xcite , and causal intervention-based methods @xcite @xcite . this methodological diversity is beneficial given the high variability of conclusions that can be drawn from a study using a single method @xcite -converging evidence is necessary for a more general picture. we contribute to this line of research with a new analysis method that we name reconstruction probing, which relies on token probabilities obtained from context reconstruction, applicable to models pretrained on objectives of this kind. 1 our method is characterized by two core properties. first, it is causal: rather than asking "what features can we extract from the contextualized representations?", we ask "what effect does contextual information have on the model predictions?" through intervention at the input level. second, our method is behavioral: it relies on the context reconstruction objective that the model was trained on. this obviates the need to train specialized probes, which can be difficult to interpret due to the added confound of task-specific supervision. our method aims to probe how much information the contextualized representation of a single token contains about the other tokens that co-occur with it in a given sequence in masked language models. our approach is to measure the difference between the reconstruction probability of a co-occurring token in the sequence given the full contextualized representation being probed, and the reconstruction probability of the same co-occurring token only from the lexical priors of the model. this method can be generalized to compare two arbitrary representations where one representation is expected to contain strictly more features than the other (e.g., a static embedding of a token vs. an embedding of the same token created by summing the static embedding and its positional embedding in context). any difference between the reconstruction probabilities can be attributed to the presence/absence of those features. using this method, we find that the contextualized representation of a token contains more information about tokens that are closer in terms of linear and syntactic distance, but do not necessarily encode identities of those tokens. a follow-up analysis that decomposes contextualized representations furthermore shows that the gains in reconstructability we find are largely attributable to static and positional embeddings at the input layer. ## proposed approach pretrained transformer models such as bert @xcite learn to construct contextual representations through context reconstruction objectives like masked language modeling (mlm; e.g., predicting the token in place of @xcite in the [mask] sat on the mat). often, the models are also trained to reconstruct a randomly substituted token (e.g., predicting the token in place of door in the cat sat door the mat, created by randomly substituting a word in the cat sat on the mat). the classifier that makes these predictions can only make use of a single token representation from the final layer, meaning these representations are optimized to contain information about other tokens of the sequence and the position of the token itself insofar as this information can help to resolve the identity of the token. our approach aims to quantify how much the contextualization of these tokens contributes to changing the mlm predictions. ## decomposing contextualization while we examined the effect of contextualization compared to the lexical prior only baseline, our method allows for a finer-grained decomposition of the components of contextualization. in pretrained transformer models, the input representation of a token is a function of the static lexical embedding and a (context-specific) positional embedding. using our method, we can study the individual influence of the lexical embedding, positional embedding, and remaining sequence-specific contextualization (i.e., everything that happens beyond the input layer, full contextualization henceforth). we create various ablated versions of a fully contextualized sequence, as shown in the ablated sequence column of ## related work our research question is similar to @xcite which use supervised classifiers to investigate how much information about other tokens in context is contained in the contextualized representation(s) of a token. our approach addressed a similar question through reconstruction probability given more/less informative token representations. our findings about better reconstructability between tokens in a syntactic dependency relation echo prior work that show sensitivity of mlms to part-of-speech and other syntactic relations @xcite @xcite . a novel finding is that some of the syntactic dependency between tokens can be traced back to information in the input embeddings, complementing the dynamic layerwise analysis in work such as @xcite and @xcite . this result aligns with @xcite 's observation that syntactic dependency is reflected in the corpus distribution as encoded in static embeddings. existing work that analyzes static embeddings from contextualized models @xcite @xcite mostly concerns the distillation of static embeddings rather than isolating the contribution of static embeddings in contextualized prediction as in our work. more broadly, our work shares goals with intervention-based methods such as @xcite and @xcite , but we examine what the effect of our intervention is on masked language modeling probabilities rather than on separate downstream tasks. @xcite employs the most similar methodology to ours, in their use of predictions from the masked language modeling objective directly for probing. however, their primary analysis concerns the role we proposed reconstruction probing, a novel method that compares reconstruction probabilities of tokens in the original sequence given different amounts of contextual information. overall, reconstruction probing yields many intuitive results. we find that the information encoded in these representations tend to be a degree more abstract than token identities of the neighboring tokens-often, the exact identities of co-occurring tokens are not recoverable from the contexutalized representations. instead, reconstructability is correlated with the closeness of the syntactic relation, the linear distance, and the type of syntactic relation between the source and recon tokens. these findings add converging evidence to previous probing studies about the implicit syntactic information of contextual embeddings @xcite . furthermore, our method is generalizable to comparing reconstruction probabilities from any pair of representations that differ in the degree of informativeness. we extended our analysis to finer-grained decomposition of the components that constitute contextualized representations using this method, finding that most of the reconstruction gains we saw were attributable to information contained in static lexical and positional embeddings at the input layer. this calls for deeper investigations into the role of token representations at the input layer, complementing a large body of existing work on layerwise analysis of contextualized language models.