Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020)

Abstract: Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture the abilities a dataset is intended to test. We propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. In particular, after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets. Contrast sets provide a local view of a model’s decision boundary, which can be used to more accurately evaluate a model’s true linguistic capabilities. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets (e.g., DROP reading comprehension, UD parsing, and IMDb sentiment analysis). Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets—up to 25% in some cases. We release our contrast sets as new evaluation benchmarks and encourage future dataset construction efforts to follow similar annotation processes.

## introduction progress in natural language processing (nlp) has long been measured with standard benchmark datasets (e.g., @xcite . these benchmarks help to provide a uniform evaluation of new modeling developments. however, recent work shows a problem with this standard evaluation paradigm based on i.i.d. test sets: datasets often matt gardner led the project. all other authors are listed in alphabetical order. two similarly-colored and similarly-posed chow dogs are face to face in one image. two similarly-colored and similarly-posed cats are face to face in one image. three similarly-colored and similarly-posed chow dogs are face to face in one image. two differently-colored but similarly-posed chow dogs are face to face in one image. ## how to create contrast sets here, we walk through our process for creating contrast sets for three datasets. examples are shown in figure 1 and drop drop @xcite is a reading comprehension dataset that is intended to cover compositional reasoning over numbers in a paragraph, including filtering, sorting, and counting sets, and doing numerical arithmetic. the data has three main sources of paragraphs, all from wikipedia articles: descriptions of american football games, descriptions of census results, and summaries of wars. there are many common patterns used by the crowd workers that make some questions artificially easy: 2 is the most frequent answer to how many. . . ? questions, questions asking about the ordering of events typically follow the linear order of the paragraph, and a large fraction of the questions do not require compositional reasoning. our strategy for constructing contrast sets for drop was three-fold. first, we added more compositional reasoning steps. the questions about american football passages in the original data very often had multiple reasoning steps (e.g., how many yards difference was there between the broncos' first touchdown and their last?), but the questions about the other passage types did not. we drew from common patterns in the training data and added additional reasoning steps to questions in our contrast sets. second, we inverted the semantics of various parts of the question. this includes perturbations such as changing shortest to longest, later to earlier, as well as changing questions asking for counts to questions asking for sets (how many countries. . . to which countries. . . ). finally, we changed the ordering of events. a large number of questions about war paragraphs ask which of two events happened first. we changed (1) the order the events were asked about in the question, (2) the order that the events showed up in the passage, and (3) the dates associated with each event to swap their temporal order. ## related work the fundamental idea of finding or creating data that is "minimally different" has a very long history. in linguistics, for instance, the term minimal pair is used to denote two words with different meaning that differ by a single sound change, thus demonstrating that the sound change is phonemic in that language @xcite . many people have used this idea in nlp (see below), creating challenge sets or providing training data that is "minimally different" in some sense, and we continue this tradition. our main contribution to this line of work, in addition to the resources that we have created, is giving a simple and intuitive geometric interpretation of "bias" in dataset collection, and showing that this long-standing idea of minimal data changes can be effectively used to solve this problem on a wide variety of nlp tasks. we additionally generalize the idea of a minimal pair to a set, and use a consistency metric, which we contend more closely aligns with what nlp researchers mean by "language understanding". training on perturbed examples many previous works have provided minimally contrastive examples on which to train models. @xcite , @xcite , and @xcite designed their data collection process to include contrastive examples. data augmentation methods have also been used to mitigate gender @xcite , racial @xcite , and other biases @xcite during training, or to introduce useful inductive biases @xcite . challenge sets the idea of creating challenging contrastive evaluation sets has a long history @xcite @xcite @xcite . challenge sets exist for various phenomena, including ones with "minimal" edits similar to our contrast sets, e.g., in image captioning @xcite , machine translation @xcite @xcite , and language modeling @xcite . minimal pairs of edits that perturb gender or racial attributes are also useful for evaluating social biases @xcite @xcite . our key contribution over this prior work is in grouping perturbed instances into a contrast set, for measuring local alignment of decision boundaries, along with our new, related resources. additionally, rather than creating new data from scratch, contrast sets augment existing test examples to fill in systematic gaps. thus contrast sets often require less effort to create, and they remain grounded in the original data distribution of some training set. since the initial publication of this paper, shmidman et al. have further demonstrated the utility of contrast sets by applying these ideas to the evaluation of morphological disambiguation in hebrew. @xcite create new test sets for cifar and imagenet by closely following the procedure used by the original datasets authors; yadav and bottou (2019) perform similar for mnist. this line of work looks to evaluate whether reusing the exact same test set in numerous research papers causes the community to adaptively "overfit" its techniques to that test set. our goal with contrast sets is different-we look to eliminate the biases in the original annotation process to better evaluate models. this cannot be accomplished by simply collecting more data because the new data will capture similar biases. ## conclusion we presented a new annotation paradigm, based on long-standing ideas around contrastive examples, for constructing more rigorous test sets for nlp. our procedure maintains most of the established processes for dataset creation but fills in some of the systematic gaps that are typically present in datasets. by shifting evaluations from accuracy on i.i.d. test sets to consistency on contrast sets, we can better examine whether models have learned the desired capabilities or simply captured the idiosyncrasies of a dataset. we created contrast sets for 10 nlp datasets and released this data as new evaluation benchmarks. we recommend that future data collection efforts create contrast sets to provide more comprehensive evaluations for both existing and new nlp datasets. while we have created thousands of new test examples across a wide variety of datasets, we have only taken small steps towards the rigorous evaluations we would like to see in nlp. the last several years have given us dramatic modeling advancements; our evaluation methodologies and datasets need to see similar improvements.