Information-Theoretic Probing for Linguistic Structure (2020)

Abstract: The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually “know” about natural language. Probes are a natural way of assessing this. When probing, a researcher chooses a linguistic task and trains a supervised model to predict annotations in that linguistic task from the network’s learned representations. If the probe does well, the researcher may conclude that the representations encode knowledge related to the task. A commonly held belief is that using simpler models as probes is better; the logic is that simpler models will identify linguistic structure, but not learn the task itself. We propose an information-theoretic operationalization of probing as estimating mutual information that contradicts this received wisdom: one should always select the highest performing probe one can, even if it is more complex, since it will result in a tighter estimate, and thus reveal more of the linguistic information inherent in the representation. The experimental portion of our paper focuses on empirically estimating the mutual information between a linguistic property and BERT, comparing these estimates to several baselines. We evaluate on a set of ten typologically diverse languages often underrepresented in NLP research—plus English—totalling eleven languages. Our implementation is available in https://github.com/rycolab/info-theoretic-probing .

## introduction neural networks are the backbone of modern stateof-the-art natural language processing (nlp) systems. one inherent by-product of training a neural network is the production of real-valued representations. many speculate that these representations encode a continuous analogue of discrete linguistic properties, e.g., part-of-speech tags, due to the networks' impressive performance on many nlp tasks @xcite . as a result of this speculation, one common thread of research focuses on the construction of probes, i.e., supervised models that are trained to extract the linguistic properties directly @xcite @xcite @xcite . a syntactic probe, then, is a model for extracting syntactic properties, such as part of speech, from the representations @xcite . in this work, we question what the goal of probing for linguistic properties ought to be. informally, probing is often described as an attempt to discern how much information representations encode about a specific linguistic property. we make this statement more formal: we assert that the natural operationalization of probing is estimating the mutual information @xcite between a representation-valued random variable and a linguistic property-valued random variable. this operationalization gives probing a clean, information-theoretic foundation, and allows us to consider what "probing" actually means. our analysis also provides insight into how to choose a probe family: we show that choosing the highest-performing probe, independent of its complexity, is optimal for achieving the best estimate of mutual information (mi). this contradicts the received wisdom that one should always select simple probes over more complex ones @xcite @xcite . in this context, we also discuss the recent work of @xcite who proposes selectivity as a criterion for choosing families of probes. @xcite defines selectivity as the performance difference between a probe on the target task and a control task, writing "[t]he selectivity of a probe puts linguistic task accuracy in context with the probe's capacity to memorize from word types." they further ponder: "when a probe achieves high accuracy on a linguistic task using a representation, can we conclude that the representation encodes linguistic structure, or has the probe just learned the task?" information-theoretically, there is no difference between learning the task and probing for linguistic structure, as we will show; thus, it follows that one should always employ the best possible probe for the task without resorting to artificial constraints. in the experimental portion of the paper, we empirically analyze word-level part-of-speech labeling, a common syntactic probing task @xcite , within our mi operationalization. working on a typologically diverse set of languages (basque, czech, english, finnish, indonesian, korean, marathi, tamil, telugu, turkish and urdu), we show that only in five of these eleven languages do we recover higher estimates of mutual information between part-ofspeech tags and bert @xcite , a common contextualized embedder, than from a control. these modest improvements suggest that most of the information needed to tag part-of-speech well is encoded at the lexical level, and does not require sentential context. put more simply, words are not very ambiguous with respect to part of speech, a result known to practitioners of nlp @xcite . we interpret this to mean that part-ofspeech labeling is not a very informative probing task. we further investigate how bert fares in dependency labeling, as analysed by @xcite . in this task, estimates based on bert return more information than a type-level embedding in all analysed languages. however, our mi estimates still only show that bert contains at most 12% more information than the control. we also remark that operationalizing probing information-theoretically gives us a simple, but stunning result: contextual word embeddings, e.g., bert @xcite and elmo @xcite , contain the same amount of information about the linguistic property of interest as the original sentence. this follows from the data-processing inequality under a very mild assumption. what this suggests is that, in a certain sense, probing for linguistic properties in representations may not be a well grounded enterprise at all. it also highlights the need to more formally define ease of extraction. ## word-level syntactic probes for contextual embeddings following @xcite , we consider probes that examine syntactic knowledge in contextualized embeddings. these probes only consider a token's embedding in isolation, and try to perform the task using only that information. specifically, in this work, we consider part-of-speech (pos) and dependency labeling: determining a word's part of speech in a given sentence and the dependency relation for a pair of tokens joined by a dependency arc. say we wish to determine whether the word love is a noun or a verb. this task requires the sentential context for success. as an example, consider the utterance "love is blind" where, only with the context, is it clear that love is a noun. thus, to do well on this task, the contextualized embeddings need to encode enough about the surrounding context to correctly guess the pos. analogously, we need the whole sentence to know that love is the nominal subject. whereas in the sentence "greed can blind love", love is the direct object. ## control functions to place the performance of a probe in perspective, @xcite develops the notion of a control task. inspired by this, we develop an analogue we term control functions, which are functions of the representation-valued random variable r. similar to @xcite 's control tasks, the goal of a control function c(•) is to place the mutual information i(t ; r) in the context of a baseline that the control function encodes. control functions have their root in the data-processing inequality @xcite , which states that, for any function c(•), we have in other words, information can only be lost by processing data. a common adage associated with this inequality is "garbage in, garbage out." ## understanding probing information-theoretically in §3, we developed an information-theoretic framework for thinking about probing contextual word embeddings for linguistic structure. however, we now cast doubt on whether probing makes sense as a scientific endeavour. we prove in §4.1 that contextualized word embeddings, by construction, contain no more information about a wordlevel syntactic task than the original sentence itself. nevertheless, we do find a meaningful scientific interpretation of control functions. we expound upon this in §4.2, arguing that control functions are useful, not for understanding representations, but rather for understanding the influence of sentential context on word-level syntactic tasks, e.g., labeling words with their part of speech. ## a critique of control tasks we agree with @xcite -and with both zhang and bowman (2018) and tenney et al. (2019) -that we should have controlled baselines when probing for linguistic properties. however, we disagree with parts of their methodology for constructing control tasks. we present these disagreements here. ## experiments despite our discussion in §4, we still wish to empirically vet our estimation technique for the gain and we use this section to highlight the need to formally define ease of extraction (as argued in §4.3). we consider the tasks of pos and dependency labeling, using the universal pos tag @xcite and dependency label information from the universal dependencies 2.5 @xcite . we probe the multilingual release of bert foot_6 on eleven typologically diverse languages: basque, czech, english, finnish, indonesian, korean, marathi, tamil, telugu, turkish and urdu; and we compute the contextual representations of each sentence by feeding it into bert and averaging the output word piece representations for each word, as tokenized in the treebank. ## conclusion we propose an information-theoretic operationalization of probing that defines it as the task of estimating conditional mutual information. we introduce control functions, which put in context our mutual information estimates-how much more informative are contextual representations than some knowledge judged to be trivial? we further explored our operationalization and showed that, given perfect probes, probing can only yield insights into the language itself and cannot tell us anything about the representations under investigation. keeping this in mind, we suggest a change of focus-instead of concentrating on probe size or information, we should pursue ease of extraction going forward. on a final note, we apply our formalization to evaluate multilingual bert's syntactic knowledge on a set of eleven typologically diverse languages. although it does encode a large amount of information about syntax-more than 76% and 65%, respectively, about pos and dependency labels in all languages foot_8 -bert only encodes at most 12% more information than a simple baseline (a type-level representation). on pos labeling, more specifically, our mi estimates based on bert are higher than the control in less than half of the analyzed languages. this indicates that word-level pos labeling may not be ideal for contemplating the syntax contained in contextual word embeddings. accuracies base entropies random language bert fasttext one-hot random h(t ) h(t | id(r)) h(t | c(r)) g(t, r, c)