We Need To Talk About Random Splits (2021)

Abstract: (CITATION) argued for using random splits rather than standard splits in NLP experiments. We argue that random splits, like standard splits, lead to overly optimistic performance estimates. We can also split data in biased or adversarial ways, e.g., training on short sentences and evaluating on long ones. Biased sampling has been used in domain adaptation to simulate real-world drift; this is known as the covariate shift assumption. In NLP, however, even worst-case splits, maximizing bias, often under-estimate the error observed on new samples of in-domain data, i.e., the data that models should minimally generalize to at test time. This invalidates the covariate shift assumption. Instead of using multiple random splits, future benchmarks should ideally include multiple, independent test sets instead; if infeasible, we argue that multiple biased splits leads to more realistic performance estimates than multiple random splits.

## introduction it is common practice in nlp to collect and annotate a text corpus -and split it into training, development and test data. these splits are often based on the order in which texts were published or sampled, and are referred to as 'standard splits'. @xcite recently showed that system ranking results based on standard splits differ from results based on random splits and used this to argue in favor of using random splits. while perhaps less common, random splits are already used in probing @xcite , interpretability @xcite , as well as core nlp tasks @xcite . 1 gorman and bedrick (2019) focus on whether there is a significant performance difference δ 1 see also many of the tasks in the semeval evaluation campaigns: http://alt.qcri.org/semeval2020/ between systems s 1 and s 2 ; m(g test , s 1 ) -m(g test , s 2 ), in their notation. they argue mc-nemar's test @xcite or bootstrap @xcite can establish that δ = 0, using random splits to sample from g test . this, of course, relies on the assumption that data is representative, i.e., was sampled i.i.d. @xcite . in reality, what @xcite call the true difference in system performance, i.e., δ = m(g test , s 1 ) -m(g test , s 2 ), is the system difference on data that users would expect the systems to work well on (see §2 for practical examples) -and not just on the corpus that we have annotations for. our corpus-based estimates of δ can in fact be very misleading, i.e., very different from performance on new samples of data. in this paper, we investigate how misleading our estimates can be: we show that random splits consistently over-estimate performance at test time. this favors systems that overfit. we investigate alternatives across a heterogeneous set of nlp tasks. based on our experiments, our answer to community-wide overfitting to standard splits is not to use random splits but to collect more diverse data with different biases -or if that is not feasible, split your data in adversarial, not random, ways. in general, we observe that estimates of test time error are worst for random splits, slightly better for standard splits (if those exist), better for heuristic and adversarial splits, but error still tends to be higher on new (in-domain) samples; see figure 1 . our results not only refute the hypothesis that δ can be estimated using random splits @xcite ), 2 but also the covariate shift hypothesis @xcite that δ can be estimated using reweightings of the data. while biased splits are useful in the absence of multiple held-out samples, and have been proposed before @xcite ), 3 they often overestimate performance in the wild. our code is made publicly available at https://github.com/ google-research/google-research/tree/ master/talk_about_random_splits . ## experiments we consider 7 different nlp tasks: pos tagging (like @xcite ), two sentence representation probing tasks, headline generation, translation quality estimation, emoji prediction, and news classification. we experiment with these tasks, because they a) are diverse, b) have not been subject to decades of community-wide overfitting (with the exception of pos tagging), and c) three of them enabled temporal splits (see appendix §a.5). ## results our results are presented in our main observations are the following: (a) random splits (and standard splits) consistently under-estimate error on new samples. the absolute differences between error reductions over random baselines for random splits and on new samples are often higher than 20%, and in the case of probing-bshift, for example, the bert model reduces 80% of the error of a random baseline when data is randomly split, but only 45% averaging over five samples of new data from the same domain. (b) heuristic splits sometimes under-estimate error on new samples. our heuristic splits in the above experiments are quite aggressive. we only evaluate our models on sentences that are longer than any of the sentences observed during training. nevertheless for 5/7 tasks, this leads to more optimistic performance estimates than evaluating on new samples! (c) the same story holds for adversarial splits based on approximate maximization of wasserstein distances between training and test data. while adversarial splits are very challenging, results on adversarial splits are more optimistic than on new samples in 4/7 cases. note the fact that random splits over-estimate real-life performance also leads to misleading system rankings. if, for example, we remove the crf inference layer from our pos tagger, performance on our random splits drops to 0.952; on the new sample, however, performance is 0.930, which is significantly better than with a crf layer.