Analyzing Dynamic Adversarial Training Data in the Limit (2022)

Abstract: To create models that are robust across a wide range of test inputs, training datasets should include diverse examples that span numerous phenomena. Dynamic adversarial data collection (DADC), where annotators craft examples that challenge continually improving models, holds promise as an approach for generating such diverse training sets. Prior work has shown that running DADC over 1-3 rounds can help models fix some error types, but it does not necessarily lead to better generalization beyond adversarial test data. We argue that running DADC over many rounds maximizes its training-time benefits, as the different rounds can together cover many of the task-relevant phenomena. We present the first study of longer-term DADC, where we collect 20 rounds of NLI examples for a small set of premise paragraphs, with both adversarial and non-adversarial approaches. Models trained on DADC examples make 26% fewer errors on our expert-curated test set compared to models trained on non-adversarial data. Our analysis shows that DADC yields examples that are more difficult, more lexically and syntactically diverse, and contain fewer annotation artifacts compared to non-adversarial examples.

## introduction traditional crowdsourcing methods often yield datasets that lack diversity, contain spurious correlations, and are easy for existing models @xcite @xcite @xcite . training on such examples can lead to models that reach deceptively high accuracy on in-distribution test data, yet fail on challenge sets @xcite @xcite , input perturbations @xcite , and distribution shifts @xcite . test accuracy (%) ## dynamic data collection in the limit the paradigm of dadc raises a natural but unanswered question: what would happen if we kept going? if we ran dadc for many years, how robust would the resulting models be? would models improve more quickly than if we had collected training data without a model-in-the-loop? answering these forward-looking questions is key to understanding whether researchers and practitioners should continue to collect data in an adversarial fashion. of course, we cannot practically run many years of data collection at once due to cost and time constraints. our key idea is to instead answer these questions for a more manageable test bed that still retains many of the key challenges associated with language understanding tasks. in particular, we scale down the natural language inference (nli) task to a small number of paragraphlength premises. in this setting, many rounds of smaller-scale data collection can tell us whether dadc or non-adversarial data collection leads to more robust model accuracy on test hypotheses for these same contexts. if dadc is indeed superior, this suggests that dadc can collect data that more effectively covers the challenging phenomena required for nli, and therefore scaling it up to (many) more contexts could yield models that are similarly robust for more general nli. ## dynamic adversarial data outperforms non-adversarial data here, we show that dadc outperforms both standard and static adversarial data collection in the long term. in particular, we train various models using the three different datasets and compare them on the validation and expert-curated test sets. ## analyzing adversarial data why is dynamic adversarial data superior to standard data in the long term? in syntax and sentence complexity. the dynamic adversarial data is more complex (ta-3 note that when computing each metric, we use a version of the no model and static model datasets that are randomly downsampled to be the same size as the dynamic model data (6,911 examples). this controls for any effect that dataset size would have on our analyses. 4 note that this diversity metric is effective because we collect hundreds of examples for a single context paragraph; otherwise, we would need to measure similarity between hypotheses for different premises, a more complicated problem. we also experimented with bertscore @xcite and found similar trends as bleu score. ble 4, middle). for each hypothesis, we measure its length in words, its flesch-kincaid readability @xcite , and its syntactic complexity using yngve scores @xcite . yngve scores roughly measure the deviation of a parse tree from a purely right-branching tree-it is the average number of left branches on the path from the root node to each word. to compute yngve scores, we parse sentences using the benepar parser @xcite based on t5 small @xcite . in all three metrics, the dynamic adversarial data scores highest, and it is statistically significantly higher than the static model data based on a t-test with @xmath1.05. we also show how the syntactic complexity evolves over the rounds in figure 4 . for the non-adversarial and static adversarial data, the syntactic complexity is relatively constant while the dadc examples become increasingly more complex. fewer artifacts. nli training datasets are known to suffer from spurious correlations. the dadc examples contain fewer instances of two known artifacts: hypothesis-only information @xcite @xcite and high-overlap entailment examples @xcite . to measure such artifacts, we first train a hypothesis-only model on the training set for each dataset using roberta large. we test ## related work post-hoc adversarial filtering. in adversarial filtering @xcite , one takes an existing dataset and trains a model on the most difficult subportion of the data. adversarial filtering shares motivations with adversarial data collection-difficult examples are more informative for learning-but it is focused on post-hoc data filtering rather than collection of new data. ## conclusion and future work we investigated dynamic adversarial data collection in the limit-over a large number of rounds until model performance starts plateauing-and demonstrated that data collected via this method is more valuable for training than alternatives, both on validation data and an expert-curated test set. we analyzed the collected data, showing that dadc yields examples that are more diverse, more complex, and contain fewer annotation artifacts compared to non-adversarial examples. our results show that when building large training sets for training nlp models, data collected in an adversarial fashion with a continually updating model-inthe-loop can be more useful than standard modelagnostic collection in the long term. in future work, it is vital to conduct similar experiments on different tasks, e.g., question answering and sentiment analysis, as well as on a larger number of contexts for nli. such experiments can provide insight into the generalizability of our findings. moreover, given that a core benefit of dadc is promoting diversity and complexity of examples, one could explore other diversity-promoting methods of data collection. lastly, our dadc setup is relatively simplistic in that we use a single target model and provide no other guides to the annotator; it would be interesting to provide generative models, model interpretations, or other methods to potentially further improve our dadc results.