Towards Debiasing NLU Models from Unknown Biases (2020)

Abstract: NLU models often exploit biases to achieve high dataset-specific performance without properly learning the intended task. Recently proposed debiasing methods are shown to be effective in mitigating this tendency. However, these methods rely on a major assumption that the types of bias should be known a-priori, which limits their application to many NLU tasks and datasets. In this work, we present the first step to bridge this gap by introducing a self-debiasing framework that prevents models from mainly utilizing biases without knowing them in advance. The proposed framework is general and complementary to the existing debiasing methods. We show that it allows these existing methods to retain the improvement on the challenge datasets (i.e., sets of examples designed to expose models’ reliance on biases) without specifically targeting certain biases. Furthermore, the evaluation suggests that applying the framework results in improved overall robustness.

## introduction neural models often achieve impressive performance on many natural language understanding tasks (nlu) by leveraging biased features, i.e., superficial surface patterns that are spuriously associated with the target labels @xcite ). 2 recently proposed debiasing methods are effective in mitigating the impact of this tendency, and the resulting models are shown to perform better beyond training distribution. they improved the performance on challenge test sets that are designed such that relying on the spurious association leads to incorrect predictions. prevailing debiasing methods, e.g., example reweighting @xcite , confidence regularization @xcite , and model ensembling @xcite @xcite , are agnostic to model's architecture as they operate by adjusting the training loss to account for biases. namely, they first identify biased examples in the training data and down-weight their importance in the training loss so that models focus on learning from harder examples. 3 while promising, these model agnostic methods rely on the assumption that the specific types of biased features (e.g., lexical overlap) are known a-priori. this assumption, however, is a limitation in various nlu tasks or datasets because it depends on researchers' intuition and task-specific insights to manually characterize the spurious biases, which may range from simple word/n-grams cooccurrence @xcite @xcite to more complex stylistic and lexico-syntactic patterns @xcite @xcite . the existing datasets or the newly created ones @xcite @xcite are, therefore, still very likely to contain biased patterns that remain unknown without an in-depth analysis of each individual dataset @xcite . in this paper, we propose a new strategy to enable the existing debiasing methods to be applicable in settings where there is minimum prior information about the biases. specifically, models should automatically identify potentially biased examples without being pinpointed at a specific bias in advance. our work makes the following novel contributions in this direction of automatic bias mitigation. first, we analyze the learning dynamics of a large pre-trained model such as bert @xcite on a dataset injected with a synthetic and controllable bias. we show that, in very small data settings, models exhibit a distinctive response to synthetically biased examples, where they rapidly increase the accuracy (→ 100%) on biased test set while performing poorly on other sets, indicating that they are mainly relying on biases. second, we present a self-debiasing framework within which two models of the same architecture are pipelined to address the unknown biases. using the insight from the synthetic dataset analysis, we train the first model to be a shallow model that is effective in automatically identifying potentially biased examples. the shallow model is then used to train the main model through the existing debiasing methods, which work by down-weighting the potentially biased examples. these methods present a caveat in that they may lose useful training signals from the down-weighted training examples. to account for this, we also propose an annealing mechanism which helps in retaining models' in-distribution performance (i.e., evaluation on the test split of the original dataset). third, we experiment on three nlu tasks and evaluate the models on their existing challenge datasets. we show that models obtained through our self-debiasing framework gain equally high improvement compared to models that are debiased using specific prior knowledge. furthermore, our cross-datasets evaluation suggests that our general framework that does not target only a particular type of bias results in better overall robustness. terminology this work relates to the growing number of research that addresses the effect of dataset biases on the resulting models. most research aims to mitigate different types of bias on varying parts of the training pipeline (e.g., dataset collection or modeling). without a shared definition and common terminology, it is quite often that the term "bias" discussed in one paper refers to a different kind of bias mentioned in the others. following the definition established in the recent survey paper by @xcite , the dataset bias that we address in this work falls into the category of label bias. this bias emerges when the conditional distribution of the target label given certain features in the training data diverges substantially at test time. these features that are associated with the label bias may differ from one classification setting to the others, and although they are predictive, mnli synthetic: premise: what's truly striking, though, is that jobs has never really let this idea go. figure 1 : synthetic bias datasets are created by appending an artificial feature to the input text that allows models to use it as a shortcut to the target label. for each example in mnli, a number-coded label (contradiction: 0 , entailment: 1 , neutral: 2 ) is appended to the hypothesis sentences. ## motivation and analysis debiasing nlu models recent nlu tasks are commonly formulated as multi-class classification problems @xcite , in which the goal is to predict the semantic relationship label @xmath0. for each example x, let b(x) be the biased features that happen to be predictive of label y in a specific dataset. the aim of a debiasing method for an nlu task is to learn a debiased classifier f d that does not mainly use b(x) when computing p(y|x). model-agnostic debiasing methods (e.g., product-of-expert @xcite ) achieve this by reducing the importance of biased examples in the learning objective. to identify whether an example is biased, they employ a shallow model f b , a simple model trained to directly compute p(y|b(x)), where the features b(x) are hand-crafted based on the task-specific knowledge of the biases. however, obtaining the prior information to design b(x) requires a dataset-specific analysis @xcite . given the ever-growing number of new datasets, it would be a time-consuming and figure 2 : the learning trajectory of a bert model on mnli datasets that are synthetically biased with different proportions: 0.9, 0.8, 0.7, and 0.6. all settings show models' tendency to rely on biases after seeing only a small number of training examples (accuracy goes up rapidly on "biased" while goes down on "anti-biased" after less than 10k training steps). in this work, we propose an alternative strategy to automatically obtain f b to enable existing debiasing methods to work with no precise prior knowledge. this strategy assumes a connection between large pre-trained models' reliance on biases with their tendency to operate as a rapid surface learner, i.e., they tend to quickly overfit to surface form information especially when they are finetuned in a small training data setting @xcite . this tendency of deep neural network to exploit simple patterns in the early stage of the training is also well-observed in other domains of artificial intelligence @xcite . since biases are commonly characterized as simple surface patterns, we expect that models' rapid performance gain is mostly attributed to their reliance on biases. namely, they are likely to operate similarly as f b after they are exposed to only a small number of training instances, i.e., achieving high accuracy on the biased examples while still performing poorly on the rest of the dataset. ## self-debiasing framework we propose a self-debiasing framework that enables existing debiasing methods to work without requiring a precise dataset-specific knowledge about the biases' characteristics. our framework consists of two stages: (1) automatically identifying biased examples using a shallow model; and ## related work the artifacts of large scale dataset collections result in dataset biases that allow models to perform well without learning the intended reasoning skills. in nli, models can perform better than chance by only using the partial input @xcite @xcite or by basing their predictions on whether the inputs are 9 although this may seem to be against the spirit of not using prior knowledge about the biases, we believe that this step is necessary to show the stability of the shallow models and to validate if they indeed capture the intended biases. there has been a flurry of work in dynamic dataset construction to systematically reduce dataset biases through adversarial filtering @xcite @xcite or human in the loop @xcite @xcite . while promising, researchers also show that newly constructed datasets may not be fully free of hidden biased patterns @xcite . it is thus crucial to complement the data collection efforts with learning algorithms that are more robust to biases, such as the recently proposed product-ofexpert @xcite @xcite , confidence regularization (utama et al., 2020), or other training strategies @xcite @xcite . despite their effectiveness, these methods are limited by their assumption on the availability of information about the task-specific biases. our framework aims to alleviate this limitation and enable them to address unknown biases. ## conclusion we present a general self-debiasing framework to address the impact of unknown dataset biases by omitting the need for thorough dataset-specific analysis to discover the types of biases for each new dataset. we adopt the existing debiasing methods into our framework and enable them to obtain equally high improvements on several challenge test sets without targeting a specific bias. the evaluation also suggests that our framework results in better overall robustness compared to the biasspecific counterparts. based on our analysis, future work in the direction of automatic bias mitigation may include identifying potentially biased examples in an online fashion and discouraging models from exploiting them throughout the training.