Adversarial NLI : A New Benchmark for Natural Language Understanding (2020)

Abstract: We introduce a new large-scale NLI benchmark dataset, collected via an iterative, adversarial human-and-model-in-the-loop procedure. We show that training models on this new dataset leads to state-of-the-art performance on a variety of popular NLI benchmarks, while posing a more difficult challenge with its new test set. Our analysis sheds light on the shortcomings of current state-of-the-art models, and shows that non-expert annotators are successful at finding their weaknesses. The data collection method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate.

## introduction progress in ai has been driven by, among other things, the development of challenging large-scale benchmarks like imagenet @xcite in computer vision, and snli @xcite , squad @xcite , and others in natural language processing (nlp). recently, for natural language understanding (nlu) in particular, the focus has shifted to combined benchmarks like senteval @xcite and glue @xcite , which track model performance on multiple tasks and provide a unified platform for analysis. with the rapid pace of advancement in ai, however, nlu benchmarks struggle to keep up with model improvement. whereas it took around 15 years to achieve "near-human performance" on mnist @xcite @xcite and approximately 7 years to surpass humans on imagenet @xcite @xcite , the glue benchmark did not last as long as we would have hoped after the advent of bert @xcite , and rapidly had to be extended into super-glue @xcite . this raises an important question: can we collect a large benchmark dataset that can last longer? the speed with which benchmarks become obsolete raises another important question: are current nlu models genuinely as good as their high performance on benchmarks suggests? a growing body of evidence shows that state-of-the-art models learn to exploit spurious statistical patterns in datasets @xcite @xcite @xcite , instead of learning meaning in the flexible and generalizable way that humans do. given this, human annotators-be they seasoned nlp researchers or non-expertsmight easily be able to construct examples that expose model brittleness. we propose an iterative, adversarial human-andmodel-in-the-loop solution for nlu dataset collection that addresses both benchmark longevity and robustness issues. in the first stage, human annotators devise examples that our current best models cannot determine the correct label for. these resulting hard examples-which should expose additional model weaknesses-can be added to the training set and used to train a stronger model. we then subject the strengthened model to the same procedure and collect weaknesses over several rounds. after each round, we train a new model and set aside a new test set. the process can be iteratively repeated in a never-ending learning @xcite setting, with the model getting stronger and the test set getting harder in each new round. thus, not only is the resultant dataset harder than existing benchmarks, but this process also yields a "moving post" dynamic target for nlu systems, rather than a static benchmark that will eventually saturate. our approach draws inspiration from recent efforts that gamify collaborative training of machine learning agents over multiple rounds @xcite and pit "builders" against "breakers" to learn better models @xcite . recently, @xcite showed that such an approach can be used to make dialogue safety classifiers more robust. here, we focus on natural language inference (nli), arguably the most canonical task in nlu. we collected three rounds of data, and call our new dataset adversarial nli (anli). our contributions are as follows: 1) we introduce a novel human-and-model-in-the-loop dataset, consisting of three rounds that progressively increase in difficulty and complexity, that includes annotator-provided explanations. 2) we show that training models on this new dataset leads to state-of-the-art performance on a variety of popular nli benchmarks. 3) we provide a detailed analysis of the collected data that sheds light on the shortcomings of current models, categorizes the data by inference type to examine weaknesses, and demonstrates good performance on nli stress tests. the anli dataset is available at github.com/facebookresearch/anli/. a demo is available at adversarialnli.com. ## dataset collection the primary aim of this work is to create a new large-scale nli benchmark on which current stateof-the-art models fail. this constitutes a new target for the field to work towards, and can elucidate model capabilities and limitations. as noted, however, static benchmarks do not last very long these days. if continuously deployed, the data collection procedure we introduce here can pose a dynamic challenge that allows for never-ending learning. ## dataset statistics the dataset statistics can be found in to over 103k examples for round 3. we collected more data for later rounds not only because that data is likely to be more interesting, but also simply because the base model is better and so annotation took longer to collect good, verified correct examples of model vulnerabilities. for each round, we report the model error rate, both on verified and unverified examples. the unverified model error rate captures the percentage of examples where the model disagreed with the writer's target label, but where we are not (yet) sure if the example is correct. the verified model error rate is the percentage of model errors from example pairs that other annotators confirmed the correct label for. note that error rate is a useful way to evaluate model quality: the lower the model error rate-assuming constant annotator quality and context-difficulty-the better the model. we observe that model error rates decrease as we progress through rounds. in round 3, where we included a more diverse range of contexts from various domains, the overall error rate went slightly up compared to the preceding round, but for wikipedia contexts the error rate decreased substantially. while for the first round roughly 1 in every 5 examples were verified model errors, this quickly dropped over consecutive rounds, and the overall model error rate is less than 1 in 10. on the one hand, this is impressive, and shows how far we have come with just three rounds. on the other hand, it shows that we still have a long way to go if even untrained annotators can fool ensembles of state-of-the-art models with relative ease. ## results base model performance is low. notice that the base model for each round performs very poorly on that round's test set. this is the expected outcome: for round 1, the base model gets the entire test set wrong, by design. for rounds 2 and 3, we used an ensemble, so performance is not necessarily zero. however, as it turns out, performance still falls well below chance foot_1 , indicating that workers did not find vulnerabilities specific to a single model, but generally applicable ones for that model class. rounds become increasingly more difficult. as already foreshadowed by the dataset statistics, round 3 is more difficult (yields lower performance) than round 2, and round 2 is more difficult than round 1. this is true for all model architectures. training on more rounds improves robustness. generally, our results indicate that training on more rounds improves model performance. this is true for all model architectures. simply training on more "normal nli" data would not help a model be robust to adversarial attacks, but our data actively helps mitigate these. roberta achieves state-of-the-art performance... we obtain state of the art performance on both snli and mnli with the roberta model finetuned on our new data. the roberta paper @xcite reports a score of 90.2 for both mnli-matched and -mismatched dev, while we obtain 91.0 and 90.7. the state of the art on snli is currently held by mt-dnn @xcite , which reports 91.6 compared to our 92.9. ...but is outperformed when it is base model. however, the base (roberta) models for rounds 2 and 3 are outperformed by both bert and xl-net (rows 5, 6 and 10). this shows that annotators found examples that roberta generally struggles with, which cannot be mitigated by more examples alone. it also implies that bert, xlnet, and roberta all have different weaknesses, possibly as a function of their training data (bert, xlnet and roberta were trained on different data sets, which might or might not have contained information relevant to the weaknesses). continuously augmenting training data does not downgrade performance. even though anli training data is different from snli and mnli, adding it to the training set does not harm performance on those tasks. our results (see also rows 2-3 of exclusive test subset difference is small. we included an exclusive test subset (anli-e) with examples from annotators never seen in training, and find negligible differences, indicating that our models do not over-rely on annotator's writing styles. ## linguistic analysis we explore the types of inferences that fooled models by manually annotating 500 examples from each round's development set. a dynamically evolving dataset offers the unique opportunity to track how model error rates change over time. since each round's development set contains only verified examples, we can investigate two interesting questions: which types of inference do writers employ to fool the models, and are base models differentially sensitive to different types of reasoning? the results are summarized in we observe that both round 1 and 2 writers rely heavily on numerical and quantitative reasoning in over 30% of the development set-the percentage in a2 (32%) dropped roughly 6% from a1 (38%)-while round 3 writers use numerical or quantitative reasoning for only 17%. the majority of numerical reasoning types were references to cardinal numbers that referred to dates and ages. inferences predicated on references and names were present in about 10% of rounds 1 & 3 development sets, and reached a high of 20% in round 2, with coreference featuring prominently. standard inference types increased in prevalence as the rounds increased, ranging from 18%-27%, as did 'lexical' inferences (increasing from 13%-31%). the percentage of sentences relying on reasoning and outside facts remains roughly the same, in the mid-50s, perhaps slightly increasing over the rounds. for round 3, we observe that the model used to collect it appears to be more susceptible to standard, lexical, and tricky inference types. this finding is compatible with the idea that models trained on adversarial data perform better, since annotators seem to have been encouraged to devise more creative examples containing harder types of inference in order to stump them. further analysis is provided in appendix b. ## related work bias in datasets machine learning methods are well-known to pick up on spurious statistical patterns. for instance, in the first visual question answering dataset @xcite , biases like "2" being the correct answer to 39% of the questions starting with "how many" allowed learning algorithms to perform well while ignoring the visual modality altogether @xcite . in nli, @xcite and tsuchiya (2018) showed that hypothesis-only baselines often perform far better than chance. nli systems can often be broken merely by performing simple lexical substitutions @xcite , and struggle with quantifiers @xcite and certain superficial syntactic properties @xcite . in question answering, @xcite showed that question-and passage-only models can perform surprisingly well, while @xcite added adversarially constructed sentences to passages to cause a drastic drop in performance. many tasks do not actually require sophisticated linguistic reasoning, as shown by the surprisingly good performance of random encoders @xcite . similar observations were made in machine translation @xcite and dialogue @xcite . machine learning also has a tendency to overfit on static targets, even if that does not happen deliberately @xcite . in short, the field is rife with dataset bias and papers trying to address this important problem. this work presents a potential solution: if such biases exist, they will allow humans to fool the models, resulting in valuable training examples until the bias is mitigated. ## discussion & conclusion in this work, we used a human-and-model-in-theloop training method to collect a new benchmark for natural language understanding. the benchmark is designed to be challenging to current stateof-the-art models. annotators were employed to act as adversaries, and encouraged to find vulnerabilities that fool the model into misclassifying, but that another person would correctly classify. we found that non-expert annotators, in this gamified setting and with appropriate incentives, are remarkably creative at finding and exploiting weaknesses. we collected three rounds, and as the rounds progressed, the models became more robust and the test sets for each round became more difficult. training on this new data yielded the state of the art on existing nli benchmarks. the anli benchmark presents a new challenge to the community. it was carefully constructed to mitigate issues with previous datasets, and was designed from first principles to last longer. the dataset also presents many opportunities for further study. for instance, we collected annotatorprovided explanations for each example that the model got wrong. we provided inference labels for the development set, opening up possibilities for interesting more fine-grained studies of nli model performance. while we verified the development and test examples, we did not verify the correctness of each training example, which means there is probably some room for improvement there. a concern might be that the static approach is probably cheaper, since dynamic adversarial data collection requires a verification step to ensure examples are correct. however, verifying examples is probably also a good idea in the static case, and adversarially collected examples can still prove useful even if they didn't fool the model and weren't verified. moreover, annotators were better incentivized to do a good job in the adversarial setting. our finding that adversarial data is more data-efficient corroborates this theory. future work could explore a detailed cost and time trade-off between adversarial and static collection. it is important to note that our approach is modelagnostic. hamlet was applied against an ensemble of models in rounds 2 and 3, and it would be straightforward to put more diverse ensembles in the loop to examine what happens when annotators are confronted with a wider variety of architectures. the proposed procedure can be extended to other classification tasks, as well as to ranking with hard negatives either generated (by adversarial models) or retrieved and verified by humans. it is less clear how the method can be applied in generative cases. adversarial nli is meant to be a challenge for measuring nlu progress, even for as yet undiscovered models and architectures. luckily, if the benchmark does turn out to saturate quickly, we will always be able to collect a new round.