Increasing Robustness to Spurious Correlations using Forgettable Examples (2021)

Abstract: Neural NLP models tend to rely on spurious correlations between labels and input features to perform their tasks. Minority examples, i.e., examples that contradict the spurious correlations present in the majority of data points, have been shown to increase the out-of-distribution generalization of pre-trained language models. In this paper, we first propose using example forgetting to find minority examples without prior knowledge of the spurious correlations present in the dataset. Forgettable examples are instances either learned and then forgotten during training or never learned. We show empirically how these examples are related to minorities in our training sets. Then, we introduce a new approach to robustify models by fine-tuning our models twice, first on the full training data and second on the minorities only. We obtain substantial improvements in out-of-distribution generalization when applying our approach to the MNLI, QQP and FEVER datasets.

## introduction despite the impressive performance of current nlp models, these models often exploit spurious correlations: they tend to capture prediction correlations that hold for most examples but do not hold in general. for instance, in natural language inference (nli) datasets, word-overlap between hypothesis and premise is highly correlated with the entailment label @xcite . therefore, these models are brittle when tested on examples that cannot be solved by recurring to these correlations, limiting their application in realworld scenarios. out-of-distribution or challenging sets are benchmarks carefully designed to break systems that rely on such correlations. the paradigm of fine-tuning pre-trained language models (plm) has pushed the state-of-theart in a large variety of tasks involving natural lan-guage understanding (nlu) @xcite . this is achieved by selfsupervised learning from an enormous amount of text. plms also show increased robustness on challenging datasets @xcite . this increase is attributed to an empirical finding that plms perform better on minority examples present in the training data @xcite . these minority examples violate the spurious correlations and therefore likely support the examples in challenging datasets. @xcite find minority examples by manually dividing the training data into two groups, according to the known spurious correlations (e.g., word-overlap in nli). they present an analysis of the robustness of plms and its connection to minority examples. in this work, we first introduce a systematic way to find minority examples that does not need prior knowledge of spurious correlations, a big limitation of the earlier work. we then present a simple approach that increases the robustness of plms further by tuning models more on these examples. to identify the set of minority examples, we adopt example forgetting @xcite . this statistic has been shown to relate to the hardness of examples, so we assume it is useful to find minorities in the training data. based on the definition presented in @xcite , we consider an example forgettable if during training it is either properly classified at some point and misclassified later, or if it is never properly classified. this method is model-and task-agnostic. we show in our datasets that minority examples w.r.t to spurious correlations, such as word-overlap in nli, are well represented in forgettable examples. after finding minorities through forgettable examples, we propose a simple method to increase the robustness of plms further. we perform an additional fine-tuning on the minorities ex-clusively, after fine-tuning on the whole training data. we find this strategy effective, as it increases robust accuracy, i.e., performance on outof-distribution data, while minimally impacting performance on in-distribution examples. we evaluate our proposed methods in three tasks, including nli (mnli, @xcite , paraphrase identification (qqp, @xcite and fact verification @xcite . for each task, recent work has introduced out-of-distribution test sets targetting specific spurious correlations. our contributions are the following: ## datasets we consider three sentence pair classification tasks, namely natural language inference, paraphrase identification, and fact verification. in the following, we describe the datasets we choose for each task following an introduction of the task. ## finding minorities with forgettables we first define example forgetting and how to compute it. we then show that it can be used to find minority examples in the training data. ## robustifying by fine-tuning on minority examples prior work shows that plms generalize to out-ofdistribution data because they generalize better on minority examples from the training set @xcite . here, we introduce a simple approach that exploits minority examples to increase robustness. in this approach, we fine-tune a plm in two successive phases, first on the full training set, and then on the minority examples only. our method does not need changes to the training objectives. an illustration is shown in fig 2 . ## related work a growing body of literature recently focused on out-of-distribution generalization, showing that it is far from being attained, even in seemingly simple cases @xcite @xcite . in particular, and in contrast with what @xcite recommend, nlp models do not seem to "embody the symmetries that allow the same meaning be expressed within multiple grammatical structures". supervised models seem to exhibit poor systematic generalization capabilities @xcite @xcite thus seemingly lacking compositional behavior @xcite . while this might seem at odds with the common belief that high-level semantic representations of the input data are formed @xcite , the reliance on highly predictive but brittle features is not confined to nlu tasks. it is also a perceived shortcoming of image classification models @xcite . to test systematically if machine learning models generalize beyond their training distribution, several challenging datasets have been introduced in nlp and other ml applications @xcite @xcite . those test sets are made automatically from designed grammars @xcite and/or by human annotators @xcite . dataset re-sampling and weighting these techniques have been studied in order to solve class imbalance problem @xcite or co-variate shift @xcite , notably by importance weighted empirical risk minimization. in nlp, clark et al. (2019); mahabadi and henderson (2019); he et al. (2019); utama et al. (2020) give evidence of the effectiveness of re-weighting training examples to increase robustness. they generally assume a priori knowledge of the heuristics present in the dataset and up/down-weight examples concerning those heuristics. aflite (sakaguchi et al., 2019) is an algorithmic method for bias removal in datasets without relying on prior knowledge about datasets. it filters out examples with a high average predictability score, relating them to points with biases or spurious correlations. bras et al. (2020) adopt aflite to build more robust models. this method harms the indistribution performance significantly. comparatively, we aim to increase robustness while maintaining in-distribution performance, so we do not filter out easy examples in our approach. our work also relates to distributionally robust optimization (duchi and namkoong, 2018; hu et al., 2018) and the more recent group-dro (sagawa et al., 2020a), which does not assume access to target data and optimizes the worst-case performance under an unknown, bounded distribution shift. recently, swayamdipta et al. (2020) introduced a two-dimensional criterion to identify hard and easy examples. they consider both the confidence of an example (the average of its loss during training epochs) and the variability (those for which the loss has high variance) and show that ambiguous examples (high-variance and highconfidence) can enhance ood accuracy. easy examples (low-variance and high-confidence) can instead help model optimization. although example forgetting is a coarser measure of variance of the loss, their results align with our findings: upweighting hard/ambiguous examples enhance ood generalization, but only training on those can harm optimization. curriculum learning dataset sampling is related to curriculum learning, where training proceeds along with a curriculum of samples with increasing difficulty @xcite . @xcite ; @xcite ; @xcite ; @xcite ; @xcite have shown the concept can be quite successful in a variety of areas. our robustifying method is related to this concept. however, our models are first trained using i.i.d spurious correlations in nlu datasets like mnli or fever are the subjects of many works. they include (i) the presence of specific words in the hypothesis or claim, for example, negation words like "not" are correlated with the contradiction label in entailment tasks @xcite , or bigrams like "did not" with the refute label, in fact, verification @xcite ; (ii) syntactic heuristics, like wordoverlap between premise and hypothesis; and (iii) sentence length @xcite , and its correlation with labels. hans @xcite and paws @xcite ) (which we evaluate on) generate plausible high word-overlap examples for both positive and negative classes. @xcite build a new test example by simple lexical inference rules and show the brittleness of models on this out-of-distribution dataset. they also show that having supporting examples in training data is key to predict a test example correctly. @xcite show that when datasets are long-tailed, rare and atypical instances make up a significant fraction of the data distribution and memorizing them leads to better indomain generalization. they find those rare and atypical examples using influence estimation. we instead study forgettable examples and their impact on out-of-distribution generalization. an interesting experiment would be to mine minority examples by influence estimation and compare with forgettable examples. ## conclusion we introduced a novel approach, based on example forgetting, to extract minority examples and build more robust models systematically. via example forgetting, we built a set of minority examples on which a pre-trained model is fine-tuned. we evaluated our method on large-scale models such as bert and xlnet and showed a consistent improvement in robustness on three challenging test sets. we also showed that the larger versions obtain higher out-of-distribution performance than the base ones but still benefit from our method.