Probing as Quantifying Inductive Bias (2022)

Abstract: Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. Such performance improvements have motivated researchers to quantify and understand the linguistic information encoded in these representations. In general, researchers quantify the amount of linguistic information through probing, an endeavor which consists of training a supervised model to predict a linguistic property directly from the contextual representations. Unfortunately, this definition of probing has been subject to extensive criticism in the literature, and has been observed to lead to paradoxical and counter-intuitive results. In the theoretical portion of this paper, we take the position that the goal of probing ought to be measuring the amount of inductive bias that the representations encode on a specific task. We further describe a Bayesian framework that operationalizes this goal and allows us to quantify the representations’ inductive bias. In the empirical portion of the paper, we apply our framework to a variety of NLP tasks. Our results suggest that our proposed framework alleviates many previous problems found in probing. Moreover, we are able to offer concrete evidence that—for some tasks—fastText can offer a better inductive bias than BERT.

## introduction improved pre-trained representations have led to new performance heights on nlp applications. this has prompted researchers to analyze these representations in an attempt to determine which linguistic properties they encode. probing is the primary method to perform such a quantification; typically, probing consists of training a supervised model, called a probe, to predict a linguistic property directly from the representations. it has been argued that the existence of a high-performing probe suggests that the representation encodes the property of interest @xcite . however, despite the apparent simplicity of probing and its wide-spread use, the community has yet to find consensus on several important problems about the endeavor. we enumerate several problems with the supervised probing framework in the following paragraphs. ## probing as quantifying inductive bias at the most fundamental level, the nlp community's interest in pre-trained representations is about reducing the sample complexity of models on downstream tasks. the community hopes that pre-trained representations are able to imbue nlp models with enough information about a given language that models can reach a higher performance with the same or even fewer training data. and, indeed, over and over again this has been shown to be the case @xcite @xcite . another way of phrasing this desire is that the nlp community hopes that pretrained representations have a suitable inductive bias for downstream tasks. this paper takes the position that, rather than probing the pre-trained representations for how much linguistic structure they contain-an endeavor that has received much attention @xcite @xcite , inter alia) but is still contentious @xcite @xcite )-we should directly ask how much they improve the inductive bias on tasks of interest. we propose to quantify the inductive bias of a model, i.e., a representation-probe pair, using the principle of occam's razor @xcite . occam's razor states that we representations that naturally separate the properties are preferred over random representations in terms of the evidence, since they have a better inductive bias. left: we compare an optimal representation that distinguishes both property classes (a) and a random representation (b). right: we compare a neural probe (c) to a linear probe (d) which is too simplistic. the evidence correctly prefers a neural probe since it better explains the data. in the following, we outline the probabilistic model for probing and the form of the evidence. this enables us to quantify the inductive bias of representations. crucially, part of the inference is to select the optimal probe for each representation so as to enable a fair comparison between representations. ## tackling probing with evidence as outlined in §1, current work in probing faces a series of problems. here we discuss how these problems are directly addressed by the evidence. ## experimental setup we evaluate our framework on a series of token, arc, and sentence tasks. our token-and arc-level tasks are multilingual, foot_6 whereas our sentence tasks only consider english. we remove any property values that have less than 20 examples in any of the splits. all our probes are trained using the adam (kingma and ba, 2015) optimizer. for details on hyperparameters, see app. b. token-level tasks. for our token-level probing tasks, we probe for part-of-speech (pos) tags, tense, number, and case. we use the setup in torroba hennigen et al. (2020), which consists of mapping the ud v2.5 @xcite treebanks to the unimorph schema @xcite using the converter by @xcite , and extracting examples of tokens tagged for the relevant properties. next, we obtain the representations for each of those tokens in their sentential context @xcite . finally, we split the resulting vocabulary using a 65-35 train-test split, such that no word appears in multiple splits. while the evidence does not require such a split, we use the split to validate results (cf. fig. 4 ). arc-level tasks. for our arc-level tasks, we conduct dependency arc labeling (dal). this consists of classifying the label for a dependency relation given only the representations for the head and dependent of that relation. these are extracted from the ud v2.5 treebanks using the approach in @xcite . we use the default ud splits. sentence-level tasks. for our sentence-level tasks, we consider four tasks. the first is multinli @xcite , a natural language inference task. the other three are the boolq @xcite , and recognizing textual entailment (rte; @xcite @xcite tasks, which are part of the superglue benchmark @xcite . if a task requires one or more passages as input, we first obtain a passage-level representations by averaging over all of its tokens. representations. in our token and arc tasks, we compare four different representations @xmath0: (i) m-bert @xcite , (ii) fasttext @xcite , (iii) a random representation (rand.), which offers no information, drawn i.i.d. from a gaussian distribution with zero mean and unit variance and the same dimensionality as bert for each data point, and (iv) a representation that assigns a unique random vector to every word in our vocabulary, so the only information it provides is the identity of the word (word ident.). the dimensionality of (iii) and (iv) is the same as that of the bert representation. for the sentence tasks, we consider (i) random, (ii) fasttext, (iii) bert, (iv) albert @xcite , (v) roberta @xcite , (vi) xl-net @xcite , and (vii) t5 @xcite . app. c lists details on the exact models and implementations used. probe family. in order to ensure fair comparisons, our framework requires us to define a suitably expressive set of priors p over probe families. in line with most of the probing literature, this includes linear and neural probes with 1 or 2 hidden layers, 100 hidden units, tanh activation, and varying weight decay parameter. ## results we find that our formulation of probing alleviates the problems that we identified in §3. firstly, the evidence suggests that random representations have an unsuitable inductive bias for linguistic tasks, which is in line with hypotheses from previous research @xcite . secondly, the automatic selection of the right probe architecture using the evidence shows that linear probes are seldom preferred, at least in our token-and arc-level experiments. that said, we also find evidence that even linear probes can overfit, and that the optimal linear probes may require many of their weights to be regularized to zero. clearly, allowing different probe architectures between representations is beneficial for a fair comparison: simpler representations can profit a ) p o s ( d e u ) p o s ( e n g ) p o s ( m a r ) p o s ( t u r ) p o s ( z h o ) t e n s e ( d e u ) t e n s e ( e n g ) t e n s e ( t u r ) n u m . ( a r a ) n u m . ( d e u ) n u m . ( e n g ) n u m . ( m a r ) n u m . ( t u r ) c a s e ( a r a ) c a s e ( d e u ) c a s e ( e n g ) n l i ( e n g ) b o o l q ( e n g ) r t e ( e n g ) c b ( e n g ) t5 xlnet roberta albert bert fasttext random -1 -0.8 -0.6 -0.4 1 2 1 1 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 2 0 2 0 0 figure 3: inductive biases of different representations (shown on the rows) for sentence-level tasks (shown on the columns), as measured by the log evidence averaged over the data set. the integers inside the cells denote the number of layers of the optimal probe, with 0 denoting a linear probe. the representations with the best inductive bias for each task are highlighted in red. from a more complex probe and demonstrate a superior inductive bias than more complex representations in some cases. specifically, we find that fasttext demonstrates a better inductive bias than bert on multiple morphosyntactic tasks, while t5 appears to offer the best inductive bias for all our sentence-level tasks. ## related work probing aims to provide insights into what linguistic information is encoded in pre-trained representations. since the introduction of probing for sentence representations @xcite , probing has also been applied to representations of words and tokens @xcite @xcite . nonetheless, comparison of representations, the choice of probe, and even probing tasks have been under scrutiny recently @xcite @xcite . ## conclusion previous approaches to linguistic probing are plagued by several key problems, namely the issues of nonsensical results, probe selection, and ill-defined goals. to overcome these issues, we have proposed a novel probing framework, which focuses on the inductive bias that pre-trained representations offer for different linguistic tasks. we have shown that the bayesian evidence, a natural measure for inductive bias, can be used in the context of probing. we have found that our framework empirically does not suffer from the aforementioned problems. we are hopeful that under this new paradigm, future work in probing will be more principled, comparable, and useful to the nlp community at large.