End-to-End Bias Mitigation by Modelling Biases in Corpora (2020)

Abstract: Several recent studies have shown that strong natural language understanding (NLU) models are prone to relying on unwanted dataset biases without learning the underlying task, resulting in models that fail to generalize to out-of-domain datasets and are likely to perform poorly in real-world scenarios. We propose two learning strategies to train neural models, which are more robust to such biases and transfer better to out-of-domain datasets. The biases are specified in terms of one or more bias-only models, which learn to leverage the dataset biases. During training, the bias-only models’ predictions are used to adjust the loss of the base model to reduce its reliance on biases by down-weighting the biased examples and focusing the training on the hard examples. We experiment on large-scale natural language inference and fact verification benchmarks, evaluating on out-of-domain datasets that are specifically designed to assess the robustness of models against known biases in the training data. Results show that our debiasing methods greatly improve robustness in all settings and better transfer to other textual entailment datasets. Our code and data are publicly available in https://github.com/rabeehk/robust-nli .

## introduction recent neural models @xcite @xcite have achieved high and even near human-performance on several largescale natural language understanding benchmarks. however, it has been demonstrated that neural models tend to rely on existing idiosyncratic biases in the datasets, and leverage superficial correlations between the label and existing shortcuts in the training dataset to perform surprisingly well, foot_0 without learning the underlying task @xcite @xcite @xcite . for instance, natural language inference (nli) is supposed to test the ability of a model to determine whether a hypothesis sentence (there is no teacher in the room) can be inferred from a premise sentence (kids work at computers with a teacher's help) @xcite . 2 however, recent work has demonstrated that large-scale nli benchmarks contain annotation artifacts; certain words in the hypothesis that are highly indicative of inference class and allow models that do not consider the premise to perform unexpectedly well @xcite . as an example, in some nli benchmarks, negation words such as "nobody", "no", and "not" in the hypothesis are often highly correlated with the contradiction label. as a result of the existence of such biases, models exploiting statistical shortcuts during training often perform poorly on out-of-domain datasets, especially if the datasets are carefully designed to limit the spurious cues. to allow proper evaluation, recent studies have tried to create new evaluation datasets that do not contain such biases @xcite @xcite . unfortunately, it is hard to avoid spurious statistical cues in the construction of large-scale benchmarks, and collecting new datasets is costly @xcite . it is, therefore, crucial to develop techniques to reduce the reliance on biases during the training of the neural models. we propose two end-to-end debiasing techniques that can be used when the existing bias patterns are identified. these methods work by adjusting the crossentropy loss to reduce the biases learned from the training dataset, down-weighting the biased examples so that the model focuses on learning the hard examples. figure 1 illustrates an example of applying our strategy to prevent an nli model from predicting the labels using existing biases in the hypotheses, where the bias-only model only sees the hypothesis. our strat- ## related work to address dataset biases, researchers have proposed to augment datasets by balancing the existing cues @xcite or to create an adversarial dataset @xcite . however, collecting new datasets, especially at a large scale, is costly, and thus remains an unsatisfactory solution. it is, therefore, crucial to develop strategies to allow models to be trained on the existing biased datasets. @xcite propose to first compute the n-grams in the dataset's claims that are the most associated with each fact-verification label. they then solve an optimization problem to assign a balancing weight to each training sample to alleviate the biases. in contrast, we propose several end-to-end debiasing strategies. additionally, @xcite propose adversarial techniques to remove from the nli sentence encoder the features that allow a hypothesisonly model to succeed. however, we believe that in general, the features used by the hypothesis-only model can include some information necessary to perform the nli task, and removing such information from the sentence representation can hurt the performance of the full model. their approach consequently degrades the performance on the hard snli set, which is expected to be less biased. in contrast, we propose to train a bias-only model to use its predictions to dynamically adapt the classification loss to reduce the importance of the most biased examples. concurrently to our work, @xcite and @xcite have also proposed to use the product of experts (poe) models for avoiding biases. they train their models in two stages, first training a bias-only model and then using it to train a robust model. in contrast, our methods are trained in an end-to-end manner, which is convenient in practice. we additionally show that our proposed debiased focal loss model is an effective method to reduce biases, sometimes superior to poe. we have evaluated on new domains of nli hard sets and fact verification. moreover, we have included an analysis showing that our debiased models indeed have lower correlations with the bias-only models, and have extended our methods to guard against multiple bias patterns simultaneously. we furthermore study transfer performance to other nli datasets. ## reducing biases problem formulation we consider a general multi-class classification problem. given a dataset @xmath0,y i } n @xmath1, and labels y @xmath2, the goal of the base model is to learn a mapping f m parameterized by θ m that computes the predictions over the label space given the input data, shown as f m : @xmath3. our goal is to optimize θ m parameters such that we build a model that is more resistant to benchmark dataset biases, to improve its robustness to domain changes where the biases typically observed in the training data do not exist in the evaluation dataset. the key idea of our approach, depicted in figure 1 , is first to identify the dataset biases that the base model is susceptible to relying on, and define a biasonly model to capture them. we then propose two strategies to incorporate this bias-only knowledge into the training of the base model to make it robust against the biases. after training, we remove the bias-only model and use the predictions of the base model. ## evaluation on unbiased datasets we provide experiments on a fact verification (fever) and two large-scale nli datasets (snli and mnli). we evaluate the models' performance on recently-proposed challenging unbiased evaluation sets. we use the bert @xcite implementation of @xcite as our main baseline, known to work well for these tasks. in all the experiments, we use the default hyperparameters of the baselines. ## transfer performance to evaluate how well the baseline and proposed models generalize to solving textual entailment in domains that do not share the same annotation biases as the large nli training sets, we take trained nli models and test them on several nli datasets. datasets: we consider a total of 12 different nli datasets. we use the 11 datasets studied by @xcite . these datasets include mnli, snli, scitail @xcite , addonerte (add1) @xcite , johns hopkins ordinal commonsense inference (joci) @xcite , multiple premise entailment (mpe) @xcite , sentences involving compositional knowledge (sick) @xcite , and three datasets from @xcite which are automatically generated from existing datasets for other nlp tasks including: semantic proto-roles (spr) @xcite , definite pronoun resolution (dpr) @xcite , framenet plus (fn+) @xcite , and the glue benchmark's diagnostic test @xcite . we additionally consider the quora question pairs (qqp) dataset, where the task is to determine whether two given questions are semantically matching (duplicate) or not. as in @xcite , we interpret duplicate question pairs as an entailment relation and neutral otherwise. we use the same split ratio mentioned by @xcite . since the datasets considered have different label spaces, when evaluating on each target dataset, we map the model's labels to the corresponding target dataset's space. see appendix d for more details. we strictly refrained from using any out-of-domain data when evaluating on the unbiased split of the same benchmark in section 4. however, as shown by prior work @xcite , since different nli target datasets contain different amounts of the bias found in the large-scale nli dataset, we need to adjust the amount of debiasing according to each target dataset. we consequently introduce a hyperparameter α for poe to modulate the strength of the bias-only model in ensembling. we follow prior work @xcite and then report results on the test set. 4 we select hyper-parameters γ, α from {0.4,0.6,0.8,2,3,4,5}. results: ## conclusion we propose two novel techniques, product-of-experts and debiased focal loss, to reduce biases learned by neural models, which are applicable whenever one can specify the biases in the form of one or more bias-only models. the bias-only models are designed to leverage biases and shortcuts in the datasets. our debiasing strategies then work by adjusting the cross-entropy loss based on the performance of these bias-only models, to focus learning on the hard examples and downweight the importance of the biased examples. additionally, we extend our methods to combat multiple bias patterns simultaneously. our proposed debiasing techniques are model agnostic, simple, and highly effective. extensive experiments show that our methods substantially improve the model robustness to domainshift, including 9.8 points gain on fever symmetric test set, 7.4 on hans dataset, and 4.8 points on snli hard set. furthermore, we show that our debiasing techniques result in better generalization to other nli datasets. future work may include developing debiasing strategies that do not require prior knowledge of bias patterns and can automatically identify them.