Information-Theoretic Probing with Minimum Description Length (2020)

Abstract: To measure how well pretrained representations encode some linguistic property, it is common to use accuracy of a probe, i.e. a classifier trained to predict the property from the representations. Despite widespread adoption of probes, differences in their accuracy fail to adequately reflect differences in representations. For example, they do not substantially favour pretrained representations over randomly initialized ones. Analogously, their accuracy can be similar when probing for genuine linguistic labels and probing for random synthetic tasks. To see reasonable differences in accuracy with respect to these random baselines, previous work had to constrain either the amount of probe training data or its model size. Instead, we propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL). With MDL probing, training a probe to predict labels is recast as teaching it to effectively transmit the data. Therefore, the measure of interest changes from probe accuracy to the description length of labels given representations. In addition to probe quality, the description length evaluates “the amount of effort” needed to achieve the quality. This amount of effort characterizes either (i) size of a probing model, or (ii) the amount of data needed to achieve the high quality. We consider two methods for estimating MDL which can be easily implemented on top of the standard probing pipelines: variational coding and online coding. We show that these methods agree in results and are more informative and stable than the standard probes.

## introduction to estimate to what extent representations (e.g., elmo @xcite or bert @xcite ) capture a linguistic property, most previous 1 we release code at https://github.com/ lena-voita/description-length-probing . work uses 'probing tasks' (aka 'probes' and 'diagnostic classifiers'); see @xcite for a comprehensive review. these classifiers are trained to predict a linguistic property from 'frozen' representations, and accuracy of the classifier is used to measure how well these representations encode the property. despite widespread adoption of such probes, they fail to adequately reflect differences in representations. this is clearly seen when using them to compare pretrained representations with randomly initialized ones @xcite . analogously, their accuracy can be similar when probing for genuine linguistic labels and probing for tags randomly associated to word types ('control tasks', @xcite ). to see differences in the accuracy with respect to these random baselines, previous work had to reduce the amount of a probe training data @xcite or use smaller models for probes @xcite . as an alternative to the standard probing, we take an information-theoretic view at the task of measuring relations between representations and labels. any regularity in representations with respect to labels can be exploited both to make predictions and to compress these labels, i.e., reduce length of the code needed to transmit them. formally, we recast learning a model of data (i.e., training a probing classifier) as training it to transmit the data (i.e., labels) in as few bits as possible. this naturally leads to a change of measure: instead of evaluating probe accuracy, we evaluate minimum description length (mdl) of labels given representations, i.e. the minimum number of bits needed to transmit the labels knowing the representations. note that since labels are transmitted using a model, the model has to be transmitted as well (directly or indirectly). thus, the overall codelength is a combination of the quality of fit of the model (compressed data length) with the cost of transmitting the model itself. intuitively, codelength characterizes not only the final quality of a probe, but also the 'amount of effort' needed achieve this quality (figure 1 ). if representations have some clear structure with respect to labels, the relation between the representations and the labels can be understood with less effort; for example, (i) the 'rule' predicting the label (i.e., the probing model) can be simple, and/or (ii) the amount of data needed to reveal this structure can be small. this is exactly how our vague (so far) notion of 'the amount of effort' is translated into codelength. we explain this more formally when describing the two methods for evaluating mdl we use: variational coding and online coding; they differ in a way they incorporate model cost: directly or indirectly. variational code explicitly incorporates cost of transmitting the model (probe weights) in addition to the cost of transmitting the labels; this joint cost is exactly the loss function of a variational learning algorithm @xcite . as we will see in the experiments, close probe accuracies often come at a very different model cost: the 'rule' (the probing model) explaining regularity in the data can be either simple (i.e., easy to communicate) or complicated (i.e., hard to communicate) depending on the strength of this regularity. online code provides a way to transmit data without directly transmitting the model. intuitively, it measures the ability to learn from different amounts of data. in this setting, the data is transmitted in a sequence of portions; at each step, the data transmitted so far is used to understand the regularity in this data and compress the following portion. if the regularity in the data is strong, it can be revealed using a small subset of the data, i.e., early in the transmission process, and can be exploited to efficiently transmit the rest of the dataset. the online code is related to the area under the learning curve, which plots quality as a function of the number of training examples. if we now recall that, to get reasonable differ-ences with random baselines, previous work manually tuned (i) model size and/or (ii) the amount of data, we will see that these were indirect ways of accounting for the 'amount of effort' component of (i) variational and (ii) online codes, respectively. interestingly, since variational and online codes are different methods to estimate the same quantity (and, as we will show, they agree in the results), we can conclude that the ability of a probe to achieve good quality using a small amount of data and its ability to achieve good quality using a small probe architecture reflect the same property: strength of the regularity in the data. in contrast to previous work, mdl incorporates this naturally in a theoretically justified way. moreover, our experiments show that, differently from accuracy, conclusions made by mdl probes are not affected by an underlying probe setting, thus no manual search for settings is required. we illustrate the effectiveness of mdl for different kinds of random baselines. for example, when considering control tasks @xcite , while probes have similar accuracies, these accuracies are achieved with a small probe model for the linguistic task and a large model for the random baseline (control task); these architectures are obtained as a byproduct of mdl optimization and not by manual search. our contributions are as follows: ## information-theoretic viewpoint let @xmath0, y 1 ), . . . , (x n , y n )} be a dataset, where x 1:@xmath1, x 2 , . . . , x n ) are representations from a model and y 1:@xmath2, y 2 , . . . , y n ) are labels for some linguistic task (we assume that y @xmath3, 2, . . . , k}, i.e. we consider classification tasks). as in standard probing task, we want to measure to what extent x 1:n encode y 1:n . differently from standard probes, we propose to look at this question from the information-theoretic perspective and define the goal of a probe as learning to effectively transmit the data. setting. following the standard information theory notation, let us imagine that alice has all (x i , y i ) pairs in d, bob has just the x i 's from d, and that alice wants to communicate the y i 's to bob. the task is to encode the labels y 1:n knowing the inputs x 1:n in an optimal way, i.e. with minimal codelength (in bits) needed to transmit y 1:n . transmission: data and model. alice can transmit the labels using some probabilistic model of data p(y|x) (e.g., it can be a trained probing classifier). since bob does not know the precise trained model that alice is using, some explicit or implicit transmission of the model itself is also required. in section 2.1, we explain how to transmit data using a model p. in section 2.2, we show direct and indirect ways of transmitting the model. interpretation: quality and 'amount of effort'. in section 2.3, we show that total codelength characterizes both probe quality and the 'amount of effort' needed to achieve it. we draw connections between different interpretations of this 'amount of effort' part of the code and manual search for probe settings done in previous work. foot_0 ## mdl and random models now, from random labels for word types, we come to another type of random baselines: randomly initialized models. probes using these representations show surprisingly strong performance for both token @xcite and sentence @xcite representations. this again confirms that accuracy alone does not reflect what a representation encodes. with mdl probes, we will see that codelength shows large difference between trained and randomly initialized representations. in this part, we also experiment with elmo and compare it with a version of the elmo model in which all weights above the lexical layer (layer 0) are replaced with random orthonormal matrices (but layer 0, is retained from trained elmo). we conduct a line of experiments using a suite of edge probing tasks @xcite . in these tasks, a probe can access only representations within given spans, such as a predicate-argument pair, and must predict properties, such as semantic roles. we build our experiments on top of the original code by @xcite and release our extended version. examples for each task are shown in ## related work probing classifiers are the most common approach for associating representations with linguistic properties (see @xcite for a survey). among the works highlighting limitations of standard probes (not mentioned above) is the work by @xcite , who show that probes are not suitable for analyzing learning dynamics. in addition to task performance, learning curves have also been used before by @xcite to evaluate how quickly a model learns a new task, and by @xcite to understand whether the performance of a lm on a task should be attributed to the pre-trained representations or to the process of fine-tuning on the task data. other methods for analyzing nlp models include (i) inspecting the mechanisms a model uses to encode information, e.g. attention weights @xcite @xcite @xcite or individual neurons @xcite @xcite , (ii) looking at model predictions using manually defined templates, either evaluating sensitivity to specific grammatical errors @xcite @xcite or understanding what language models know when applying them as knowledge bases or in qa settings @xcite @xcite ). an information-theoretic view on analysis of nlp models has been previously attempted in @xcite when explaining how representations in the transformer evolve between layers under different training objectives. in context of probing, @xcite attempted to formalize probing for linguistic structure from the information-theoretic perspective. the authors measure mutual information between representations and labels, and argue the importance of defining and taking into account "ease of extraction", though they do not formalize this notion. that work can serve as an additional motivation for using mdl. namely, minimum description length is the sum of (i) the data codelength, which is related to mutual information, and (ii) the model codelength, which measures "the amount of effort" needed to extract labels from representations; in @xcite , this is referred to as "ease of extraction". ## conclusions we propose information-theoretic probing which measures minimum description length (mdl) of labels given representations. we show that mdl naturally characterizes not only probe quality, but also 'the amount of effort' needed to achieve it (or, intuitively, strength of the regularity in representations with respect to the labels); this is done in a theoretically justified way without manual search for settings. we explain how to easily measure mdl on top of standard probe-training pipelines. we show that results of mdl probing are more sensible compared to standard probes. a description length and control tasks 4 . b mdl and random models b.1 experimental setting tasks and datasets. we focus on several core nlp tasks: pos tagging, syntactic constituent and dependency labeling, named entity recognition, semantic role labeling, coreference resolution, and relation classification. examples for each task are shown in we follow @xcite and use elmo @xcite trained on the billion word benchmark dataset @xcite . probes and optimization. probing architecture is illustrated in figure 5 . it takes a list of contextual vectors [e 0 , e 1 , . . . , e n ] and integer spans s 1 = [i 1 , j 1 ) and (optionally) s 2 = [i 2 , j 2 ) as inputs, and uses a projection layer followed by the self-attention pooling operator of @xcite to compute fixed-length span representations. the span representations are concatenated and fed into a two-layer mlp followed by a softmax output layer. as in the original paper, we use the standard cross-entropy loss, hidden layer size of 256 and dropout of 0.3. for further details on training, we refer the reader to the original paper by tenney et al. (2019). 7 for the variational code, the layers are replaced with that of bayesian compression by @xcite ; loss function changes to (2) and no dropout is used. similar to the experiments in the previous section, we do not anneal learning rate and train at least 200 validations to enable pruning. we build our experiments on top of the original code by @xcite and release our extended version.