Putting Words in BERT ’s Mouth: Navigating Contextualized Vector Spaces with Pseudowords (2021)

Abstract: We present a method for exploring regions around individual points in a contextualized vector space (particularly, BERT space), as a way to investigate how these regions correspond to word senses. By inducing a contextualized “pseudoword” vector as a stand-in for a static embedding in the input layer, and then performing masked prediction of a word in the sentence, we are able to investigate the geometry of the BERT-space in a controlled manner around individual instances. Using our method on a set of carefully constructed sentences targeting highly ambiguous English words, we find substantial regularity in the contextualized space, with regions that correspond to distinct word senses; but between these regions there are occasionally “sense voids”—regions that do not correspond to any intelligible sense.

## introduction vector spaces defined over static word vectors are somewhat interpretable, as the points are limited to the vocabulary. contextualized representations (crs), by contrast, are mysterious because of the unbounded number of distinct contextualized embeddings, and no obvious way to discover the word and context that would correspond to an arbitrary point in the space. attempts have been made to characterize the information captured in contextualized representations @xcite @xcite , but some of the techniques used (e.g., probing classifiers) have been subject to criticism for their indirectness. we propose a new technique called masked pseudoword probing (mapp) that allows controlled exploration of the space of a contextualized masked lms (specifically, english bert; @xcite . mapp takes advantage of the static embedding at the first layer of bert and "hallu-z @xmath0 we focus on the contextualized meanings of ambiguous verbs and prepositions, which serve as ideal test cases, as they may often be disambiguated by their objects. for example, if a pseudoword interpreted as "in" induces a distribution over its argument slot that gives most of the probability mass to locations such as "london" or "paris", we may conclude that the pseudoword has a locative sense ( our experiments indicate a substantial regularity in the bert-space. we see regions in the space that correspond to distinct senses. these regions can be recovered using our technique; for example, by sampling points around a pseudoword and looking at the points in the bert-space which decode to it. moreover, we see that between sense-regions there are often "voids" in the space that do not correspond to any intelligible sense. ## analyzing contextual representations probing representations. deciphering the information encoded in contextualized representations like bert is widely investigated in recent nlp research. probing methods use crs as inputs to probing classifiers to see how well the crs may serve as features in predicting specific properties. the intuition is that if the cr can be used to predict a specific property, then knowledge about it is encoded in the representation. recent classifier-based probes have focused on various linguistic properties such as morphology, parts of speech, sentence length, and syntactic and semantic relations @xcite @xcite . closely related to ours is work that studied the extent to which the lexical semantic classes of nouns are disambiguated by @xcite , showing that bert fares well in this respect. beyond classifier-based probes, other approaches have also been explored, such as information theoretic probing @xcite , and structural probing @xcite , which evaluates whether syntax trees are embedded in a linear transformation of a cr's word representation space. an alternative approach to probing learned rep-resentations is directly analyzing the attention weights and activation patterns @xcite . criticism against some instances of this approach is found in @xcite , who claimed that attention weights are less transparent than is often stipulated. the shortcomings of probing. some recent work has taken a more critical view regarding probing techniques @xcite . @xcite argue that while probing methods might show that certain linguistic properties exist in a representation, they do not reveal how and if this information is being used by the probing model. this could be due to the disconnect between the representation itself and the probing model. relying on classifiers to interpret representations might be problematic; they add additional confounds to the interpretability of the results, and different representations may need different classifiers @xcite . another critique concerns the difference between correlation and causation @xcite : classifier-based probes may rely on shallow correlations in the training set, thus reflecting data artifacts that are irrelevant to the studied distinction. ## traversing cr spaces: a new probing methodology our motivating question is: are word senses encoded in bert's representations-and if so, how? as a test case, we look at highly ambiguous words, as they potentially offer complex geometric configurations of senses in the bert space. ## experimental research questions & the main hypothesis we study in this paper is: there are regular "nicely defined" regions in the bert-space around words that correspond to distinct senses. such regions may be variously interpreted: around a point with a particular sense, there is a ball which contains mostly points corresponding to that sense. or, for example, points that correspond to the same sense will lie on a high-dimensional manifold. cases that we consider as not "nicely defined", are, for example, points corresponding to different senses that are scattered in the space in an inseparable way (or at least inseparable by simple functions). we would like to be able to map the semantic concept of sense to the geometric properties of the bert-space. however, since little is known about the space, fully characterizing its geometry is out of the scope for this work. we present mapp to study this hypothesis, and in doing so introduce the concept of pseudowords. this concept opens additional research questions. specialization. let z * ∈ r d be a pseudoword obtained by solving eq. ( 1 ) for a sentence s with a focus token t and cue token at position j, holding a sense η. does z * yield a sense distribution (determined by its slot fillers in the j th position) that concentrates on η? that is, does a pseudoword decode to a specific sense of the focus token? generalization. is it possible to transplant a pseudoword into a sentence where the context around the focus token is different, and still obtain coherent results? for example, in the sentences: (a) "the pan is for cooking." and (b) "the fork is for eating?", the focus token "for" is the same, and in both cases has a purpose meaning. the context, however, is different. if we transplant the pseudoword for "for" induced with sentence (a), to the position of "for" in a masked version of sentence (b) (i.e, "the pan is for [mask]?"), will we get a coherent prediction with the same sense? or is the pseudoword obtained from one sentence limited to a specific context? if pseudowords obtained for one sentence do not generalize to another, we propose to induce a "generalized pseudoword" trained over multiple examples with the same sense. ## experiments we use mapp to empirically evaluate the hypotheses listed above. we conduct four types of experiments. first, we test specialization, the extent to which the induced pseudoword z * can be viewed as a sensedisambiguated version of the focus token's input embedding. second, we venture into the immediate regions around z * by perturbing it, and ex- ## discussion what is a pseudoword? the optimization problem defined in eq. ( 1 ) results in a pseudoword z * ∈ r d . we use the pseudowords as input vectors to the model, although they are not constrained to the 30k vectors in bert's vocabulary, but may be arbitrary vectors in r d . we discuss the validity of such an operation in appendix a.4. in practice, we find that many pseudowords behave as sensedisambiguated input vectors. while our goal is not to explore the pseudoword-space for its own sake-pseudowords are a tool to shed light on the geometry and behavior of the bert-space-our experiments with pseudowords and artificially perturbed pseudowords reveal that the pseudowordspace contains regions that are semantically coherent as inputs to bert. prospects for the mapp technique. our dataset is manually curated to control for specific linguistic phenomena. we expect that pseudoword may be less semantically targeted if learned with larger contexts that create more opportunities for confounds. we note also that senses are not necessarily discrete @xcite , and it would be worthwhile to explore how graded semantic distinctions are represented, as well as underspecified meanings. we are also interested in exploring how bert represents tokens in sentences that permit multiple plausible interpretations. the mapp technique can be applied to investigate the properties of other cr models as well, as it requires only that the model be a differentiable function from input token embeddings to contextualized embeddings. ## conclusion we have presented a novel methodology and a dataset for investigating the geometry of the bertspace, using a traversal technique which allows for a continuation of the input space. we showed that there is substantial regularity in the bert-space, with regions that correspond to distinct senses. moreover, we found evidence for "voids" in the space-regions that do not correspond to any intelligible sense. our technique gives rise to various types of analysis, creating avenues for future work. immediate directions that we plan to pursue are (a) examining sense representation in longer, naturally occurring sentences, and (b) extending our analysis to a multilingual setting. for computational linguistics: main volume, pages 3391-3401, online. association for computational linguistics. mengjie zhao, philipp dufter, yadollah yaghoobzadeh, and hinrich schütze. 2020. quantifying the contextualization of word representations with semantic class probing. in findings of the association for computational linguistics: emnlp 2020, pages 1219-1234, online. association for computational linguistics. yichu zhou and vivek srikumar. 2021. directprobe: studying representations without classifiers. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, online. association for computational linguistics.