An information theoretic view on selecting linguistic probes (2020)

Abstract: There is increasing interest in assessing the linguistic knowledge encoded in neural representations. A popular approach is to attach a diagnostic classifier – or ”probe” – to perform supervised classification from internal representations. However, how to select a good probe is in debate. Hewitt and Liang (2019) showed that a high performance on diagnostic classification itself is insufficient, because it can be attributed to either ”the representation being rich in knowledge”, or ”the probe learning the task”, which Pimentel et al. (2020) challenged. We show this dichotomy is valid information-theoretically. In addition, we find that the ”good probe” criteria proposed by the two papers, *selectivity* (Hewitt and Liang, 2019) and *information gain* (Pimentel et al., 2020), are equivalent – the errors of their approaches are identical (modulo irrelevant terms). Empirically, these two selection criteria lead to results that highly agree with each other.

## introduction recently, neural networks have shown substantial progress in nlp tasks @xcite . to understand and explain their behavior, a natural question emerge: how much linguistic knowledge is encoded in these neural network systems? an efficient approach to reveal information encoded in internal representations uses diagnostic classifiers @xcite . referred to as "probes", diagnostic classifiers are trained on pre-computed intermediate representations of neural nlp systems. the performance on tasks they are trained to predict are used to evaluate the richness of the linguistic representation in encoding the probed tasks. such tasks include probing syntax @xcite @xcite , semantics @xcite , discourse features @xcite @xcite , and commonsense knowledge @xcite . however, appropriate criteria for selecting a good probe is under debate. the traditional view that high-accuracy probes are better is challenged by @xcite , who proposed that the high accuracy could be attributed to either (1) that the representation contains rich linguistic knowledge, or (2) that the probe learns the task. to circumvent this ambiguity, they proposed to use the improvement of probing task performance against a control task (predicting random labels from the same representations), i.e., the "selectivity" criterion. recently, @xcite , challenged this dichotomy from an information theoretic viewpoint. they proposed to use an "information gain" criterion, which empirically is the reduction in cross entropy from a "control function task" probing from randomized representation. in this paper, we show the "non-exclusive-or" dichotomy raised by @xcite is valid information-theoretically. there is a difference between the original nlp model learning the task and the probe learning the task. in addition, we show that the "selectivity" criterion and the "control function" criterion are comparably accurate. @xcite formulated their errors with the difference in a pair of kl divergences. we show that the error of the "selectivity" criterion @xcite , if measured from cross entropy loss, can be formulated in the difference in a pair of kl divergences as well. when randomizations are perfect, these two criteria differ by only constant terms. empirically, on a pos tag probing task on english, french and spanish translations, we show that the "selectivity" and the "control function" criteria highly agree with each other. we rank experi-ments with over 10,000 different hyperparameter settings using these criteria. the spearman correlation of the hewitt and liang (2019) vs. @xcite criteria are on par with the correlations of "accuracy vs. cross entropy loss" -two very strong baselines. overall, we recommend using control mechanisms to select probes, instead of relying on merely the probing task performance. when randomization is done well, controlling the target or representations are equivalent. ## related work diagnostic probes were originally intended to explain information encoded in intermediate representations @xcite @xcite . recently, various probing tasks have queried the representations of, e.g., contextualized word embeddings @xcite and sentence embeddings @xcite @xcite @xcite . the task of probing is usually formulated as a classifier problem, with the representations as input, and the features indicating information as output. a straightforward method to train a classifier is by minimizing cross entropy, which is the approach we follow. note that voita and titov (2020) derived training objectives from minimum description lengths, resulting in cross entropy loss and some variants. ## conclusion when selecting probes that better approximate i(t ; r), we recommend measuring with a control mechanism instead of relying on the traditional cross entropy on probing task. we show both information-theoretically and empirically, that controlling the targets and representations are equivalent, as long as the control mechanism is randomized. hyper-parameter ranges we sweep hyper parameters from the following ranges: early stopping could inhibit probe quality early stopping, if stopped before 24,000 gradient steps (approximately 4 epochs) may inhibit the quality of probes. in addition, we figure 1 shows high correlation between the "selectivity" @xcite and "information gain" @xcite criteria. small weight decay is beneficial we find that smaller weight decays (e.g., 0.01) are more beneficial for probes than larger weight decays. while the two criteria rank the capacity of probes similarly, the most simple probes tend to stand out more distinctively with the "selectivity" criterion @xcite , as are shown on figures 2 , 3 , and 4 . smaller probes are not necessarily better we find that while smaller probes have higher "selectivity" and "information gain" for mbert representations, probes with one hidden layer and 40-80 hidden neurons are better than more simplistic probes, as shown in figures 5 , 6 and 7 . the plots show consistency between the two criteria. for example, larger models and more layers do not necessary lead to better results. neither are the smallest probes with 0 hidden layers. note that we also swept hyperparameters for fasttext, where probes with less parameters do not always outperform more complex probes in either accuracy, loss, selectivity, or information gain. figures 8 and 9 illustrate these observations.