Test Harder than You Train: Probing with Extrapolation Splits (2021)

Abstract: Previous work on probing word representations for linguistic knowledge has focused on interpolation tasks. In this paper, we instead analyse probes in an extrapolation setting, where the inputs at test time are deliberately chosen to be ‘harder’ than the training examples. We argue that such an analysis can shed further light on the open question whether probes actually decode linguistic knowledge, or merely learn the diagnostic task from shallow features. To quantify the hardness of an example, we consider scoring functions based on linguistic, statistical, and learning-related criteria, all of which are applicable to a broad range of NLP tasks. We discuss the relative merits of these criteria in the context of two syntactic probing tasks, part-of-speech tagging and syntactic dependency labelling. From our theoretical and experimental analysis, we conclude that distance-based and hard statistical criteria show the clearest differences between interpolation and extrapolation settings, while at the same time being transparent, intuitive, and easy to control.

## introduction the use of contextualised language models such as elmo and bert has brought about remarkable performance gains on a wide range of downstream tasks @xcite ; but the question to what extent these models have acquired linguistic knowledge remains open. one way to investigate this question is through the use of probing classifiers trained to solve diagnostic prediction tasks that are considered to require linguistic information, such as parts-of-speech, syntactic structure, or semantic roles @xcite @xcite . however, what conclusions can be drawn from probing experiments is disputed. in particular, a central point of debate is how to know whether probes 'decode linguistic knowledge' or simply 'learn to solve the diagnostic task' @xcite . we suggest that new methods that define more rigorous and harder challenges are needed to get further insights into the capabilities and limitations of probes and probing methodology. in this paper, we propose to analyse probes in the context of an extrapolation setting, where the inputs at test time are deliberately chosen to be 'harder' than the training examples. while machine learning models and neural networks in particular have proved to be very effective learners in interpolation scenarios, where the examples at training time and those at test time are drawn from the same (idealised) underlying distribution, the ability of these models to extrapolate from the training data appears to be limited @xcite . at the same time, extrapolation has been proposed as a litmus test for abstract reasoning in neural networks @xcite . in the context of probing, we posit that the better the extrapolation capability of a probe, i.e. the higher its performance even in situations where the training and the test examples are substantially different, the more evidence we have for claiming that the probe actually uses abstract linguistic knowledge encoded in the input word representations. to construct extrapolation challenges, we propose a conceptually simple approach where we start from standard probing datasets, stratify them based on the 'hardness' of examples, and then use the 'easy' examples for training and the 'hard' ones for testing ( § 3). the central decision in this approach is how to measure 'hardness'. here we identify different scoring functions based on criteria grounded in linguistic theories, statistical properties of the base dataset, and learning behaviour. we apply these scoring functions to create extrapolation challenges from two standard probing tasks, part-ofspeech tagging and syntactic dependency labelling ( § 4), and use the results of our experiments to discuss the merits of our approach ( § 5). ## related work the method that we propose in this paper synthesises several strands of related work: ## experimental setup in this section we present our specific approach to creating extrapolation datasets, and the setup for our empirical evaluation. foot_0 ## results we now present our experimental results for each of the scoring functions. ## discussion while for all experiments, the accuracy in the extrapolation setup is substantially lower than in the standard interpolation setup, it is still above random guessing, which suggests that probes are able to extract some useful information from the word representations even under this experimental regime. however, the success of extrapolating from 'easy' to 'hard' examples varies depending on the choice of the scoring function. in this section we discuss these findings and the limitations of our method. ## conclusion we identified and suggested several ways to define the difficulty of training and validation examples based on linguistic, statistical, and learning-based criteria, to create extrapolation splits for natural language datasets. we demonstrated the usefulness of these measures for the analysis of two linguistic tasks, and proposed an evaluation protocol with baselines and metrics. our experimental results suggest that a probe trained on bert hidden representations is capable of applying patterns learned from easier examples to harder examples to some extent; but in well-motivated scenarios where the scoring function is an appropriate measure of difficulty of the examples, its competence is clearly limited compared to an interpolation probe. in our experiments, the most informative scoring functions are the distance-based arc length criterion that we applied to syntactic dependency labelling, and the word-specific most frequent tag criterion for partof-speech tagging. these functions allow for a clear and transparent extrapolation setup, while at the same time being simple and also computationally efficient. sentence length, as expected, did not turn out to be a strong indicator for hard examples, while learning-based criteria show a high margin between interpolation and extrapolation setups, but limited interpretability and qualitative insights. we conclude that enriching probing experiments with automated extrapolation setups can be a valuable supplement to standard probing methods, as it gives us an instrument to test the generalisation capability of the probe, and thereby the robustness of the features it uses. in addition to interpretation purposes, well-chosen extrapolation splits can provide a cheap but valuable extension of the evaluation of a model, testing its generalisation capabilities and verifying the progress made.