Benchmarking Long-tail Generalization with Likelihood Splits (2023)

Abstract: In order to reliably process natural language, NLP systems must generalize to the long tail of rare utterances. We propose a method to create challenging benchmarks that require generalizing to the tail of the distribution by re-splitting existing datasets. We create ‘Likelihood Splits’ where examples that are assigned lower likelihood by a pre-trained language model (LM) are placed in the test set, and more likely examples are in the training set. This simple approach can be customized to construct meaningful train-test splits for a wide range of tasks. Likelihood Splits surface more challenges than random splits: relative error rates of state-of-the-art models increase by 59% for semantic parsing on Spider, 93% for natural language inference on SNLI, and 33% for yes/no question answering on BoolQ, on our splits compared with the corresponding random splits. Moreover, Likelihood Splits create fairer benchmarks than adversarial filtering; when the LM used to create the splits is also employed as the task model, our splits do not unfairly penalize the LM.

## introduction success on in-distribution test data does not necessarily show that a system has solved the underlying task at hand. systems can achieve artificially high accuracy by exploiting dataset-specific shortcuts, such as spurious feature-label correlations that hold in the data but not in general @xcite . in many datasets, a large proportion of test examples are similar to training examples, further inflating in-distribution accuracy @xcite @xcite . outof-distribution (ood) evaluation paints a clearer picture of a system's ability to perform the task. prior work has proposed a variety of methods to test ood generalization, each with their own strengths and weaknesses. task-specific behavior tests @xcite @xcite give insights into model bethe high-likelihood "head" of the distribution becomes the training set while we evaluate generalization to the low-likelihood "tail" of the data. shown here are queries from the spider dataset in different likelihood buckets: one possible tail generalization could be the handling uncommon entities with known query types. in this work, we propose likelihood splits, a general-purpose method to create challenging ood splits for existing datasets. the principle behind likelihood splits is that any system that claims to reliably process natural language must be able to generalize from more common utterances seen during training to the long tail of rare utterances at test time. generalization, not merely memorization, is necessary because even a very large training dataset cannot exhaustively cover all possible longtail examples that may be encountered in the real world. moreover, standard annotation procedures tend to over-sample examples from the head of the distribution, further ignoring the challenge posed by infrequent examples. we identify tail examples using the likelihood under the gpt-2 language model @xcite . examples with low likelihood under gpt-2 are placed in the held-out evaluation sets and the high likelihood examples are used as the training set (see figure 1 ). likelihood splits are a novel, widely applicable strategy that can create interesting generalization benchmarks at no additional annotation cost. they are more challenging than a random split across a wide range of tasks: error rates relative to random splits increase by 59% for t5 @xcite on spider @xcite , 93% for elec-tra @xcite on @xcite , and 33% for roberta @xcite on boolq @xcite . moreover, the proposed splits do not unfairly penalize the gpt-2 model used to create the splits when it is used as a task model, thus avoiding one of the downsides of adversarial filtering. we identify many independent challenges required by likelihood splits, including generalizing to rare words, complex programs, and syntactically complex sentences. we encourage future benchmark creators to release likelihood splits as a complementary evaluation to the standard iid evaluation to better test out-of-distribution generalization performance. we will release the splits discussed in this work along with the code to easily create likelihood splits of other datasets. 1 2 related work generalizing to the long-tail. evaluating systems on long-tail phenomena is important, especially because many datasets over-sample the head of the distribution. for example, some questionanswering (qa) datasets limit their purview to pop-1 github.com/ameyagodbole/long-tail-likelihood-splits ular web-pages @xcite or frequent user queries @xcite . @xcite ; @xcite demonstrate that models trained on these datasets often fail on examples that do not match the most frequent training cases. similar observations have been made in entity linking to rare entities, @xcite , information retrieval for open-domain qa @xcite , relation extraction for rare relations @xcite and lexicon induction for rare senses in machine translation @xcite . zero-shot performance of large lms on numerical reasoning and factoid questions is also correlated with the frequency of occurence of the facts in the pre-training corpus @xcite @xcite . while we do not test whether models can memorize long-tail knowledge, we instead test whether models can process long-tail sentences. @xcite note that it is challenging to catalogue and evaluate generalization along micro-level dimensions and instead propose benchmarks that vary along macro-level dimensions (such as the language and domain) as a proxy. we hypothesize that lms learn which micro-level phenomena are rare, as this would improve their overall language modeling objective. in this work, we present a recipe that leverages lms to evaluate tail generalization for any language task. task-specific test sets. ribeiro et al. (2020) use templated queries to evaluate model performance under various linguistic perturbations. this method requires dataset designers to define phenomena of interest and axes of perturbation along which labels may be preserved or changed. naik et al. (2018) analyze model errors and instantiate tests that explicitly evaluate models on more examples from each error class. gardner et al. (2020) check for model consistency under local perturbations of test set examples. all of these approaches require annotators to create new examples, whereas we propose a method to resplit existing datasets. adversarial approaches. @xcite argue that random splits over-estimate model performance on new in-domain data and recommend the use of adversarial and heuristically challenging splits to estimate generalizability. adversarial data collection promotes the creation of difficult examples by encouraging annotators to fool a modelin-the-loop @xcite @xcite . similarly, adversarial filtering removes examples that are easy for a given task model in order to create more challenging benchmarks @xcite . however, @xcite and @xcite point out that adversarially collected or filtered examples may focus on a narrow set of skills that the "in-the-loop" model lacks, instead of covering all the abilities required for the underlying task. additionally, the "in-the-loop" task model is disproportionately penalized by the adversarial test sets @xcite . we show in §4.3 that likelihood splits do not suffer from this issue. ## capturing the tail of the distribution in order to find the tail within a dataset, we approximate likelihood of an utterance in the real distribution with its likelihood under a language model (lm). our method can be easily modified to create meaningful splits for any language task. we demonstrate this by creating likelihood splits for: • spider, a semantic parsing dataset @xcite consisting of natural language questions and corresponding sql programs; • snli, a natural language inference dataset @xcite consisting of premise and hypothesis sentences paired with labels denoting that the hypothesis is entailed by/neutral to/contradictory to the premise; • boolq, a question-answering dataset @xcite consisting of a passages, associated questions, and binary yes/no labels. ## experiments next, we benchmark task models on our likelihood splits. splits created using gpt2-medium will be the focus of our analysis. we will briefly study the effect of switching the lm to gpt2-large in §4.4. when creating likelihood splits, the number of folds k for fine-tuning the lm ( §3.2) can be chosen by the dataset curator. for results in §4 and §5, we set @xmath0. we analyse the effect of choosing a different value of k in appendix a.3. our results show that the trends and observations discussed here hold true for other values of k. ## analysis of data splits in order to highlight the challenges posed by our proposed splits, we analyze how the development sets (to ensure unseen test sets) differ from the training sets in each split. our splits require models to simultaneously excel at many different skills be-lieved to be important for language understanding. ## conclusion with the saturation of static, single-metric leaderboards, there is growing consensus for the development of holistic evaluation benchmarks. this includes evaluation of systems on aspects of performance beyond just single error rate on indistribution data; aspects such as performance on out-of-distribution data (linzen, 2020), and evaluating generalizability, robustness and fairness (ethayarajh and jurafsky, 2020). in this work, we describe an approach to benchmark long-tail generalization, a necessary skill for nlp systems that truly understand language. we demonstrate the challenge posed by our splits to state-of-the-art models on several tasks; standard evaluation overestimates model performance on long-tail utterances. instead of releasing a random split as the only metric on official benchmarks, our simple method can be used, for a wide range of tasks, to expose additional challenges in the collected data at no annotation cost. benchmarking long-tail generalization, in this manner, can test model behavior on a broad set of generalization challenges, which may be missed by evaluations that test specific skills in isolation.