Probe-Less Probing of BERT ’s Layer-Wise Linguistic Knowledge with Masked Word Prediction (2022)

Abstract: The current study quantitatively (and qualitatively for an illustrative purpose) analyzes BERT’s layer-wise masked word prediction on an English corpus, and finds that (1) the layerwise localization of linguistic knowledge primarily shown in probing studies is replicated in a behavior-based design and (2) that syntactic and semantic information is encoded at different layers for words of different syntactic categories. Hypothesizing that the above results are correlated with the number of likely potential candidates of the masked word prediction, we also investigate how the results differ for tokens within multiword expressions.

## introduction the attention mechanism of transformers @xcite has enabled language models (lms) to effectively incorporate contextual information into word representation. one such model, bert @xcite , has been shown particularly useful in a wide range of downstream tasks, outperforming the state-of-the-art benchmarks in many cases. however, it is yet to be clear what exactly such lms learn, and what information is encoded in their contextual word representations (cwrs). for this reason, much work has been devoted to answering these questions, often referred to as bertology (see @xcite for a comprehensive review). among such studies, of particular interest is the localization of linguistic knowledge. as bert consists of multiple layers (12 layers for bert-base and 24 layers for bert-large), it is crucial to understand what information is encoded in each layer, and how it differs from one another. however, the methodologies employed in such studies differ substantially from each other ( §2): some directly utilize the internal structure of such models by training probing classifiers, while others study the behaviors of such models at inference time. structure-based probes have often been successful at assigning particular domains of linguistic knowledge to local regions, yet the reliance on probing classifiers (and the introduction of extra parameters) makes it unclear if such linguistic knowledge is just an artifact of the classifier or is truly encoded in the model. behavior-based probes do not rely on external classifiers, but tend to focus on qualitative analyses of the outputs from the final layer, whereas quantitative analysis of layer-wise output remains understudied in the behavioral paradigm. in this study, we explore layer localization with behavioral probing. specifically, we mask out tokens one at a time and check whether bert predicts the same word, another word with the same part of speech, or neither (3). by using different layers for the prediction, we can determine which parts of the network correspond to higher or lower rates of congruent predictions. along with generally confirming some of the main observations of the structure-based probing studies, we find considerable variation by part of speech and some effect of multiword expression status, and discuss possible interpretations of these findings ( §4). ## experimental setup we used streusle 4.4 @xcite , a corpus of web reviews written in english. this corpus contains 723 documents, 3,813 sentences, and 55,590 tokens in total with rich annotation of various syntactic and lexical-semantic information (e.g., annotation of 3,013 strong multiword expressions). the bert's prediction data were prepared in the following way: 1. for each sentence, create n variants, where n is the number of tokens in the sentence, by replacing one token by [mask] token. 2. for each variant (where one word is repalced with [mask] in step 1) of each sentence, run vanilla bert to generate a prediction from each layer ℓ ∈ l. 3. for each of the n variants of each sentence, where [mask] is now replaced by a predicted token in step 2, pos-tag the predicted token to identify its syntactic category. for the bert model, we use bert-base-uncased because bert-base and bert-large have similar distributions of layers, which @xcite call "stretch effect", although they do sometimes exhibit heterogeneous behaviors, such as responses to perturbation in word prediction @xcite . the model was retrieved from the pytorch implementation of bert by huggingface @xcite . for pos, the tag set of 17 poss from universal dependencies (ud) v2 @xcite was used, and stanza @xcite was used for the automatic tagging of predicted tokens. the above experiment resulted in the prediction of, and the tagging of, l × s × @xmath0,670 masked tokens, where l, s, and n are the number of layers, the number of sentences, and the (mean) length of the sentences, respectively. in addition to analyzing the descriptive statistics, in order to quantify the relative contribution of each layer to pos match and word match, differential scores at each task (pos match or word match) for each layer ∆ (ℓ) t were obtained by computing the incremental gain from the previous layer (equation 3 of @xcite : as a summary statistic of these scores, (pseudo) expectation of differential scores (equation 4 of @xcite was also calculated: this is an "expected layer", at which the gain scores are centered around. if the differential scores were uniformly distributed, the expected layer would simply be the middle layer, which is layer 6. if the contribution of lower layers were higher (i.e., differential scores were higher at lower layers), then the expected layer would be lower than 6, and vice versa. 4 results 4.1 quantitative results ## conclusion in this study, we set out to investigate if (1) the layer-wise linguistic knowledge found in structure studies can be replicated with a behavior-based design and if (2) the results vary by syntactic category. by analyzing bert's layer-wise masked word prediction, we have shown that the localization of linguistic knowledge found in various probing studies was indeed replicated; more specifically, syntactic knowledge was encoded primarily in lower layers, whereas semantic knowledge was spread across the 12 layers. we also observed that the contribution of particular layers on syntactic and semantic information varied substantially, depending on the syntactic category (i.e., upos) and on the syntactic class (i.e., open vs. closed class) more generally, of the original token. hypothesizing that the number of potential candidates is one of the contributing factors to this difference, we showed that, in general, the expected layers were higher for pos match and lower for word match for the tokens that are part of mwes (a supposedly more constrained environment). our contribution is twofold. first, by leveraging bert's layer-wise outputs, we confirmed the previous studies without relying on external probing classifiers or introducing extra parameters that can potentially obfuscate the locus of the observed linguistic knowledge (i.e., language model vs. probing classifier). second, by extending the analyses to all open and closed class categories rather than limiting the scope to popular content-words, such as verb, noun, and adjective, we show that the encoding of syntactic and semantic knowledge about words of different upos varies substantially. lastly, we acknowledge that this study has a few limitations. first, the layer-wise masked word prediction essentially feeds intermediate layers directly to the classification layer, thereby inferring the linguistic information encoded in the intermediate layers. however, this is not what bert is trained for; that is to say, arguably, only the final layer is optimized for the masked word prediction task, and other layers are not. hence, the intermediate layers' lower pos and word match accuracy may not be due to the "absence" of syntactic or semantic knowledge encoded in those layers; rather, they may simply suggest that those intermediate layers are not trained for such tasks. second, although we provided a possible explanation for our observations and showed a few examples that seem to support our hypotheses, these are highly speculative and not meant to prove anything. we consider this a limitation of our approach, and a more controlled experiment is needed to make stronger claims or to test our hypotheses, and this is left for future work.