P areto Probing: T rading Off Accuracy for Complexity (2020)

Abstract: The question of how to probe contextual word representations in a way that is principled and useful has seen significant recent attention. In our contribution to this discussion, we argue, first, for a probe metric that reflects the trade-off between probe complexity and performance: the Pareto hypervolume. To measure complexity, we present a number of parametric and non-parametric metrics. Our experiments with such metrics show that probe’s performance curves often fail to align with widely accepted rankings between language representations (with, e.g., non-contextual representations outperforming contextual ones). These results lead us to argue, second, that common simplistic probe tasks such as POS labeling and dependency arc labeling, are inadequate to evaluate the properties encoded in contextual word representations. We propose full dependency parsing as an example probe task, and demonstrate it with the Pareto hypervolume. In support of our arguments, the results of this illustrative experiment conform closer to accepted rankings among contextual word representations.

## introduction neural networks are a pillar of modern nlp systems. however, their inner workings are poorly understood; indeed, for this reason, they are often referred to as black-box systems @xcite @xcite . this lack of understanding, coupled with the rising adoption of neural nlp systems in both industry and academia, has fomented a rapidly growing literature devoted to "cracking open the black box," as it were @xcite ; linzen the y-axis measures the probes performance on the task. probing the representations: albert, bert, roberta, fasttext, one-hot, and random. @xcite . one popular method for studying the linguistic content of neural networks is probing, which we define in this work as training a supervised classifier (known as a probe) on top of pretrained models' frozen representations @xcite . by analyzing the classifier's performance, one can assess how much 'knowledge' the representations contain about language. much work in probing advocates for the need for simple probes @xcite . indeed, on this point, @xcite write: "the task of a deep neural network classifier is to come up with a representation for the final layer that can be easily fed to a linear classifier (i.e., the most elementary form of useful classifier)." as a justification for their operationalization of complexity as the restriction of the probe to linear models (as opposed to deep neural networks). most saliently, @xcite attempts to operationalize complexity in terms of control tasks, which constrain a probe's capacity for memorization. 1 @xcite follow in this vein with an information-theoretic estimate of complexity: a model's minimum description length. in opposition to the complexity of a probe is its accuracy, i.e., its ability to perform the target probing task. from an information-theoretic perspective, @xcite argues for the use of more complex probes, since they better estimate the amount of mutual information between a representation and the target linguistic property. from a different perspective, @xcite also criticize the indiscriminate use of simple probes, because most neural representations are not estimated with the explicit aim of making information linearly separable; thus, it is unlikely that they will naturally do so, and foolish, perhaps, to expect them to. this paper proposes to directly acknowledge the existence of a trade-off between the two when considering the development of probes. we argue-in part based on experimental evidence-that naïvely selecting a family of probes either for its complexity or its performance leads to degenerate edgecases; see fig. 1 . we conclude that the nuanced trade-off between accuracy and complexity in probing should thus be treated as a bi-objective optimization problem: one objective encourages low complexity and another encourages high accuracy. we then propose a novel evaluation paradigm for probes. we advocate for pareto optimal probes, i.e., probes that are both simpler and more accurate than all others. the set of such optimal probes can then be taken in aggregate to form a pareto frontier, which allows for broader analysis and easier comparison between representations. we run a battery of probing experiments for partof-speech labeling and dependency-arc labeling, using both parametric and non-parametric complexity metrics. our experiments show that if we desire simple probes, then we are forced to conclude that one-hot encoding representations and randomly generated ones almost always encode more linguistic structure than those representations derived from bert-a nonsensical result. on the 1 @xcite define selectivity as the difference between a model's accuracy on a task versus its accuracy on a control version of that task. the control version of the tasks are built by randomly shuffling labels across word types and measures a probe's capacity for memorization. our nonparametric measures of complexity differ from control tasks; we describe these differences in § 5. to wrap up our paper, we levy a criticism at the probing tasks themselves; we argue that "toyish" probing tasks are not very useful for revealing how much more linguistic information bert manages to capture than standard baseline representations. with this in mind, we advocate for more challenging probing tasks, e.g., dependency parsing instead of its toyish cousin dependency arc labeling. we find that using actual nlp tasks as probing tasks reveals much more about the advantages bert provides over non-contextual representations. ## performance and complexity we argue in favor of treating probing for linguistic structure in neural representations as a two part optimization problem. on the one hand, we must optimize our probe for high accuracy on our chosen probing task: if we do not directly train the probe to accurately extract the linguistic features from the representation, how else can we determine whether they are implicitly encoded? on the other hand, the received wisdom in the probing community is that probes should be simple @xcite : if the probe is an overly complex model, we might ascribe high accuracy on the probing task to the probe itself, meaning the probe has "learned the task" to a large extent. in this section, we argue that a probing framework that does not explicitly take into account the accuracy-complexity trade-off may be easily gamed. indeed, we demonstrate how to game both accuracy and complexity respectively below. ## an invitation to pareto probing we now advocate for a probing evaluation metric that combines both accuracy and complexity. we argued in § 2 that probe accuracy and complexity exist in a trade-off. because of this trade-off, we should search for models that are pareto optimal. a probe is considered pareto optimal (with respect to a family of probes) if there is no competing probe where both the accuracy is higher and the complexity is lower on the task. the set of pareto optimal points may be called the pareto frontier and is generally connected, as is shown in fig. 2 . as can also be seen in we can also analyze each representation's frontier individually. this notion leads us to a very natural metric for evaluating probes: pareto hypervolume (ph; @xcite . 3 one important technical caveat involving evaluating the hypervolume is that it is undefined when the metric of model complexity for the experiment is unbounded. thus, it is necessary to restrict model complexity to a bounded interval so that the ph is always finite. ## parametric metrics of complexity we consider two types of probe complexity metrics. we term the first parametric complexity, which we discuss in this section. the second type is nonparametric complexity, which we discuss in § 5. for the parametric one, we first require a family of probes, e.g., the family of linear probes-which are all those that take the form of eq. ( 1 ), without restriction on the representation's dimension d. ## non-parametric metrics of complexity the parametric metrics of model complexity in § 4 have an explicit constraint that the models must belong to the same parametric family. specifically, it requires that we are able to define a penalty (generally dependent on the parameters) that enforces how complex each model should be. in this section, we move away from parametric notions of model complexity to non-parametric metrics. we opt to work with a notion of non-parametric complexity based on the ease with which a model can memorize training data. these non-parametric measures are rarely explicitly discussed as complexity metrics-although they are intuitive for that purpose-and have become common recently: @xcite originated this trend by shuffling outputs of image data so the images were no longer predictive of the labels, using this result to illustrate the effective memorization capacity of modern neural networks. the first of our two experiments to obtain non-parametric complexity measures is similar to theirs. we train our probe in a dataset with shuffled labels and get its accuracy in this training set. we will refer to this complexity metric as the label-shuffled scenario. 5 neural networks can take advantage of structured input (e.g., real images as opposed to noisy ones) to easily memorize their labels @xcite . these structured inputs may be easier to represent internally regardless of the outputs, given current theories that early stages of training are committed to memorizing inputs @xcite . as such, we may also want to analyze a probe's capacity to memorize unstructured inputin the case of language, we can easily remove structure by shuffling the word sequences themselves, creating random zipfian-distributed noise, which are harder for neural networks to exploit @xcite . by providing probes with unstructured input, we measure a more domain-independent sense of complexity than the ability to map structured inputs to random labels, because the model cannot rely on syntactic patterns when memorizing shuffled training data. we will refer to this second scenario, wherein both labels and inputs are shuffled, as fully shuffled. the distinction between memorization of real data and memorization of unstructured data is crucial, as experimenters choose the class of probes being learned. a comparison between label-shuffled and fully shuffled compression exposes the degree to which the class of probes employs a bias towards the true input structure in its compression. similarly, comparisons between different classes of probes can test the same assumed bias. we highlight that, while our non-parametric complexity metrics permit arbitrary classes of probes to be included in a probe hypothesis space, the selection criteria of possible probes may still reflect the assumed structure of the data, affecting compression and generalization. for example, linear probes reflect an assumption that the information lies in an euclidean (sub)space; however, this assumption may not be true: @xcite al. (2019) spiral manifold in bert, while the syntactic distances described by @xcite are pythagorean in nature. one advantage of these methods is the ability to compare between probe classes, which offers a test of the geometric assumptions behind model selection. 6 another advantage is that, unlike regularization-based parametric methods, they require no modification of the training procedure and can therefore run much faster. ## posl and dal experiments we present our experimental findings on the previously discussed part-of-speech labeling (posl) and dependency arc labeling (dal) probing tasks using both our parametric complexity metrics ( § 4) and the non-parametric ones ( § 5). when investigating posl, we take the target space t to be the set of universal part-of-speech tags for a specific language. we then train a classifier to predict these pos tags from word representations obtained from the analyzed model (e.g., bert). similarly, for dal, the target space t is defined as the set of arc dependency labels in the language, but we predict these labels from pairs of representations-the two words composing the arc. we analyze the contextual representations from bert @xcite , albert @xcite and roberta @xcite -noting that albert and roberta are trained in english alone, so we only evaluate their performance on that language. 7 for each of these models, we feed it a sentence and average the output word piece @xcite representations for each word, as tokenized in the treebank. we further analyze fast-text's non-contextual representations @xcite as well as one-hot and random representations such as those considered by @xcite . one-hot and random representations map each word type in the training data to a vector we sample from a standard normal distribution (zero mean and unit variance). new representations are sampled on the spot (untrained) for any out of vocabulary words. all representations are kept fixed during training, except for one-hot, which are learned with the other network parameters. ## a closer look at model complexity this work represents a new entry into a growing literature on taking the capabilities of probes into account when analyzing a model @xcite @xcite . the fundamental point we wish to espouse in this paper is that evaluating a probe for linguistic structure is fundamentally asking a question about a trade-off between accuracy and complexity. however, we wish to highlight that evaluating a probe's complexity is a very open problem. indeed, the larger question of model complexity has been treated for over 50 years in a number of disciplines. in statistics, model complexity is researched in the model selection literature, e.g., the classical techniques of bayesian information criterion @xcite and akaike information criterion @xcite . in computer science, learning theorists have introduced the vapnik-chervonenkis dimension @xcite ), pollard's pseudodimension @xcite , and rademacher complexity @xcite . algorithmic information theorists provide kolmogorov complexity @xcite -closely related to mdl-encoding the size of the model. a concrete discussion of complexity requires several distinctions regarding these measures. the first is the object of analysis of the complexity measure which can be either a model family, i.e the whole set of functions realizable by a choice of architecture and hyperparameters, or a learned model, which takes into account a specific set of learned parameters. the second is the aspect being analyzed by the used measure which could be, for example, the capacity of the function class (i.e., whether it is possible to set model weights so the architecture represents a specific target function-its hard constraints) or its bias (i.e., the soft constraints that influence whether the training process is likely to guide a model towards this target function). importantly, the measure of complexity the scientist employs will impact their scientific findings about how much linguistic structure they read into a neural network's hidden states. for instance, in some of our experiments we regularize the probe directly with a relaxation of a nuclear norm constraint, thus imposing a bias without modifying the total capacity of the model. meanwhile, our rank-based method controls the capacity of the probe directly, enforcing a model family's complexity. finally, our non-parametric methods, and selectivity, estimate the capacity of a model family (under a specific hyperparameter choice) by approximating a "hard" function in which labels are randomly assigned-differences in accuracy between these three complexity measures indicate a complex relationship between implicit regularization; input language structure; and model capacity. in comparison to our computable complexity measures, popular hypothetical notions also vary in how they explore the data's domain space to analyze a model family: rademacher complexity uses true observations as inputs, but vc dimension considers adversarially selected data-being defined according to the "worst" possible sample allowed in the input domain. as new techniques for considering the complexity of models emerge, it is critical to develop in parallel tools for reasoning about what aspect and object of "complexity" is really being measured. when one introduces a metric as modeling complexity, it can be explicitly situated within such a taxonomy; these considerations should be made explicit in the presentation. a sufficiently developed theory of probing will reveal not only the information contained in a representation, but the underlying geometry of the representation space, by comparing the performance of different model families. such developments are left to future work. ## conclusion in this paper, we argued for a new approach to probing, treating it as a bi-objective optimization problem. it has no single optimal solution, but can be analyzed-under the lens of pareto efficiencyto arrive on a set of optimal solutions. these pareto optimal solutions make explicit the trade-off between accuracy and complexity, also permitting for a deeper analysis of the probed representations. the second part of our paper argues that we need to select harder tasks for the purpose of probing representations for syntactic knowledge. for tasks such as posl or dal, which require only shallow notions of syntax, non-contextual representations can do almost as well as contextual onespretraining on large amounts of data, or encoding contextual knowledge in the representations, does not help much for these tasks. we then run a battery of experiments on the harder task of dependency parsing; these show that contextual representations indeed provide much more usable syntactic knowledge than non-contextual ones.