Explaining Contextualization in Language Models using Visual Analytics (2021)

Abstract: Despite the success of contextualized language models on various NLP tasks, it is still unclear what these models really learn. In this paper, we contribute to the current efforts of explaining such models by exploring the continuum between function and content words with respect to contextualization in BERT, based on linguistically-informed insights. In particular, we utilize scoring and visual analytics techniques: we use an existing similarity-based score to measure contextualization and integrate it into a novel visual analytics technique, presenting the model’s layers simultaneously and highlighting intra-layer properties and inter-layer differences. We show that contextualization is neither driven by polysemy nor by pure context variation. We also provide insights on why BERT fails to model words in the middle of the functionality continuum.

## introduction the rise of contextualized language models (lm), i.e., contextualized word and sentence representations, such as elmo @xcite and bert @xcite , has brought many wellknown nlp tasks to a tremendous breakthrough. contextualized embeddings have replaced earlier static embeddings @xcite @xcite , creating new standards for the state-of-the-art. lms have learned highly transferable and task-agnostic properties of language (e.g., @xcite @xcite , even to a degree of imitating the classical nlp pipeline @xcite . despite these research efforts, it remains yet unclear as to what extent lms like bert capture complex linguistic phenomena and whether different linguistic properties are learned across the different layers of the model's architecture: the existing evidence is conflicting and in some cases even contradictory @xcite . one recent line of work @xcite explores the actual contextualization captured in these models, i.e., the degree to which a word is modeled as context-specific. this sheds light on the context-specificity of individual words and the degree of contextualization of different word groups. this paper contributes to this line of work by examining the degree of contextualization of function vs. content words. we treat functionality as a continuum, comparing and contrasting bert's @xcite modeling of categories of words within this continuum with the expected modeling according to the theoretical linguistic literature. it has been repeatedly shown that lms fail to generalize and capture the compositionality of language because they struggle with words of high functionality, e.g., quantifiers, prepositions, modals, conjunctions @xcite @xcite , to name only a few). thus, our linguistically-informed analysis sheds light on the peculiarities of these phenomena and contributes to our better understanding of bert. this paper utilizes the self-similarity contextualization score of @xcite for better comparability. the exploration of the scores and phenomena is enabled by lmexplorer, a visual analytics (va) technique for the layer-wise explanation of contextualized word embeddings. lmexplorer contributes a new perspective on the learned patterns of the model, and shows clusters and score developments in the model's layers simultaneously. overall, the contribution of this paper is twofold: (1) we generate insights as to how bert captures function vs. content words (sections 4 and 5), and (2) present a novel visual analytics technique that facilitates such insights by explaining lms through contextualization scoring (section 3). ## interpretability of language models research on the interpretability of lms has been pursued in two main directions, mainly focusing on bert. for one, probing tasks are used to investigate the linguistic properties learned by the lm by training a linear model on the basis of the corresponding contextualized embeddings for the prediction of specific linguistic properties. for another, the interpretability of lms has been explored via adversarial datasets to assess the performance of an lm with respect to challenging linguistic phenomena. to further explore the interpretablity of lms, we see work coming from the field of va as promising. va techniques have been used extensively for exploring and interpreting different deep learning models @xcite , incl. lms. probing -probing experiments have shown that bert's transformer architecture encodes semantic information such as word senses and semantic roles @xcite @xcite , syntactic information in the form of constituents and hierarchical structure @xcite @xcite , morphosyntactic and morphological features @xcite , and discourse-related information necessary for tasks such as coreference resolution @xcite . moreover, the traditional nlp pipeline sequence of pos tagging, syntactic parsing, named entity recognition, semantic role labeling and coreference resolution can be mapped onto bert's transformer layers from lower to higher @xcite . accordingly, several probing studies have shown that bert captures a hierarchy of linguistic information (e.g., @xcite @xcite : surface features are represented best in the lower layers, while syntactic features are captured best in the middle layers. the middle to higher layers represent morphological features best, and semantic information is captured best in the higher layers. adversarial testing -adversarial testing has shown that lms struggle in making generalizations on basic lexical relations @xcite , identifying ungrammaticality @xcite , efficiently capturing challenging linguistic phenomena, such as negation @xcite , modals, quantifiers and monotonicity @xcite , passives @xcite , conditionals @xcite , conjunctions @xcite , implicatives and factives @xcite , and modeling human reasoning patterns, such as numerical or common-sense reasoning @xcite . overall, the evidence from adversarial testing contradicts the results of the probing studies: if the lm indeed is able to acquire 'deep' linguistic knowledge (e.g., about syntactic hierarchies), it should be able to deal with the phenomena present in the adversarial test sets. contextualization -despite the conflicting evidence about the linguistic capacities of lms like bert, it is widely acknowledged that the word embeddings generated by such models are contextualized, i.e., there is no finite number of word sense representations and a word has different vector representations across different contexts. particularly, by assessing a word's contextualization on the basis of self-similarity scores, @xcite shows that the embeddings become more contextualized, i.e., more context-specific, in the upper layers of bert. moreover, it has been shown that contextualized embeddings generally cluster with one another with respect to word senses @xcite . visual lm explanations -approaches for visual lm explanations can be grouped into two main categories. one strand of research focuses on transformer-based lms and explains how they learn through visualizing attentions (e.g., nl-ize @xcite , seq2seq-vis @xcite , bertviz @xcite , exbert @xcite , sanvis @xcite , and attention flows @xcite ). another strand of research explains what the model learns by visualizing word embeddings. although most existing work on embedding explanation is based on probing tasks, visualization of embedding characteristics has emerged as an active research topic. the first tools were related to the exploration of static embeddings, e.g., by @xcite , who visualize word2vec and glove embeddings, focusing on analogy exploration. @xcite explain the same models and present visualizations that support analysis of multiple tasks, among others, the analysis of local word neighborhoods. also, @xcite explain static embeddings of word2vec, glove, and fasttext. their explanations focus on local neighborhoods visualized using small multiples by applying a dimensionality reduction. @xcite has recently presented a visual approach for exploring correlations between embedding clusters in bert for a single model's layer at a time. the novelty of our approach is the explanation of contextualized word embeddings through contextualization scores that are visualized for all of the model's layers simultaneously. ## lmexplorer: visual analytics technique to support the analysis of word contextualization within the functionality continuum, we have developed a va technique called lmexplorer. this technique discloses layer-wise spatial and score-based patterns in the learned embedding representations. using interlinked embedding projections, we show the spatial relations of the high-dimensional embedding space. to provide further insights into the word contextualization, the technique utilizes scoring functions (i.e., word self-similarity) as a contextualization explanation. the scores are used to explore and navigate the embedding space, which is facilitated by supporting views and interactions. the technique is integrated into the lingvis.io framework @xcite . ## exploring contextualization in bert while @xcite initially found that the increase in contextualization across the different bert layers (i.e., the decreasing self-similarity) seems to be driven by polysemy, 'stopwords' such as and, of, the and to seem to contradict this conclusion. stopwords, which in essence are function words, also become increasingly contextualized in the upper layers. thus, contextualization seems not to be entirely driven by polysemy, but rather the variety of contexts a word appears in @xcite . however, function words are not a homogeneous class, and some function words indeed have semantic content in addition to having a grammatical function. thus, we decided to investigate function and content words in more detail, using the lmexplorer to explore contextualization in bert with respect to the functionality continuum. ## insights: the functionality continuum during our exploration, we came across patterns that fit to the theory of the functionality continuum and others that were contrary to our expectations. above all, we observed that contextualization is neither triggered merely by polysemy nor by variation in context. to explain the observed patterns, a) we positioned the defined categories within the functionality continuum foot_2 based on the inherent linguistic properties of the words and on insights from lexical semantics, and b) we identified three criteria as potential triggers of contextualization, as shown in polysemy -polysemous words, mostly with 'content-like' properties, exhibit a low/high sense variation, depending on whether they are highly polysemous, i.e., have loosely related senses, or not, i.e., have semantically related senses. as it is typical of content words, polysemous words show high syntactic variation. concerning their semantic context variation, they are again in a 'grey' area depending on the degree of polysemy: highly polysemous words mostly appear in semantically different contexts, while plain polysemy is mostly found in semantically similar contexts since the senses are closely related. with this, the expected contextualization is respective to the degree of the polysemy. indeed, bert meets these expectations: highly polysemous words like field, home are in the min area across layers (high contextualization), while plain polysemous words are rather found in the max area (low contextualization). monosemy -monosemous words also seem to be correctly captured by bert. such words have low sense variation, high syntactic variation (as content words) and low semantic context variation (due to their low sense variation). according to this, they are also expected to have low contextualization. we find this low contextualization in bert as well, where monosemous words have max self-similarity across layers. temporal adverbials -at the middle of the functionality continuum, temporal adverbials have a low sense variation, e.g., yesterday has only one meaning, foot_3 as well as low syntactic variation. on the other hand, their semantic context variation is high because they can occur in semantically very different contexts. thus, the expected contextualization is high, i.e., their embeddings should be context-specific to match the semantically different contexts they can appear in. bert fails to learn this: temporal adverbials are either found within the mid area across all layers or end up in this range in the upper layers, contrary to the expected min. modals & quantifiers -bert also struggles in capturing the functionality continuum with modals and quantifiers. these are comparable to words with high 'sense' variation: modals can not only have a deontic or an epistemic flavor, but also express variation through their variable quantificational force; similarly, quantifiers exhibit variation via their variable scope interpretation (wide or narrow). both modals and quantifiers have low syntactic variation; they can only attach with specific syntactic phrases. the contexts they appear in can be semantically very different and thus they have a high semantic context variation. based on this halffunctional-half-content nature, modals and quantifiers are expected to have high contextualization, i.e., have context-specific embeddings based on the modal flavor they express, the quantificational force they capture, the scope resolution, etc. however, we can see that bert fails to meet this expectation. modals and quantifiers mostly occur in the mid range -instead of the expected min. prepositions -at the functional end of the continuum, we find prepositions and articles. prepositions are comparable to words with a high 'sense' variation, capturing the fact that the same preposition can, for example, be locative or temporal, depending on the context. prepositions have low syntactic variation, as most functional words. still, their semantic context variation matches their multiple 'senses.' therefore, we expect the preposition embeddings to be highly context-specific: this is indeed the case in bert, where prepositions are mostly found in the min area. articles -last, we investigate articles and particularly the determiners the and a. we take them to have no sense, foot_4 low syntactic variation and high semantic context variation -the contexts they appear in do not have any semantic similarity in most cases. thus, we expect them to demonstrate high contextualization with highly context-specific embeddings. bert is able to model this through low self-similarity, which is more prominent for the than for a, nonetheless consistent for both. discussion -summing up, we see that bert struggles to efficiently capture the functionality continuum. while bert manages to model the ends of the continuum, i.e., the mostly content and mostly functional words, it fails to create expressive embeddings for categories with content as well as functional properties. this finding is in line with previous literature that has shown that current lms cannot efficiently capture hard linguistic phenomena (e.g., dasgupta et al. (2018); mc-coy et al. (2019); richardson et al. (2020) ), with modals, quantifiers and temporal reasoning belonging to these phenomena. our work suggests that the bert embeddings are not specific enough to capture the inherent functionality of certain word types, i.e., bert does not learn the relevant generalizations. additionally, we show that contextualization is neither entirely driven by polysemy nor context variation. rather, contextualization can be explained via the harmonical combination of functionality, sense variation, syntactic variation and semantic context variation: bert can efficiently model polysemy, homonymy and mononymy, i.e., it can efficiently capture words that appear in semantic contexts of high variation and low variation and independently of their polysemy. what it cannot model are words that have a semi-functional/semicontent nature (models, quantifiers, temporal adverbials), see ## conclusion and future work this paper presented new insights on the contextualization of the functionality continuum, showing that bert fails to capture the nature of semifunctional-semi-content words. these insights were generated through a novel visual analytics technique for contextualized word embedding exploration and analysis. for a deeper understanding of the weaknesses of bert, our technique can be extended with scores that model common linguistic properties of words and their nearest neighbors, e.g., wordnet semantic similarity or pos similarity scores. hence, they could serve as means of explanation and bring added value to the explainable artificial intelligence (xai) research field. more information about the project can be found under: https://embeddings-explained.lingvis.io .