Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance (2020)

Abstract: Models for natural language understanding (NLU) tasks often rely on the idiosyncratic biases of the dataset, which make them brittle against test cases outside the training distribution. Recently, several proposed debiasing methods are shown to be very effective in improving out-of-distribution performance. However, their improvements come at the expense of performance drop when models are evaluated on the in-distribution data, which contain examples with higher diversity. This seemingly inevitable trade-off may not tell us much about the changes in the reasoning and understanding capabilities of the resulting models on broader types of examples beyond the small subset represented in the out-of-distribution data. In this paper, we address this trade-off by introducing a novel debiasing method, called confidence regularization, which discourage models from exploiting biases while enabling them to receive enough incentive to learn from all the training examples. We evaluate our method on three NLU tasks and show that, in contrast to its predecessors, it improves the performance on out-of-distribution datasets (e.g., 7pp gain on HANS dataset) while maintaining the original in-distribution accuracy.

## introduction despite the impressive performance on many natural language understanding (nlu) benchmarks @xcite , recent pre-trained language models (lm) such as bert @xcite are shown to rely heavily on idiosyncratic biases of datasets @xcite @xcite . these biases are commonly characterized as surface features of input examples that are strongly associated with the target labels, e.g., occurrences of negation words in 1 the code is available at https://github.com/ ukplab/acl2020-confidence-regularization natural language inference (nli) datasets which are biased towards the contradiction label @xcite . as a ramification of relying on biases, models break on the out-of-distribution data, in which such associative patterns between the surface features and the target labels are not present. this brittleness has, in turn, limited their practical applicability in some extrinsic use cases @xcite . this problem has sparked interest among researchers in building models that are robust against dataset biases. proposed methods in this direction build on previous works, which have largely explored the format of several prominent labelrevealing biases on certain datasets @xcite . two current prevailing methods, product-ofexpert @xcite and learned-mixin @xcite introduce several strategies to overcome the known biases by correcting the conditional distribution of the target labels given the presence of biased features. they achieve this by reducing the importance of examples that can be predicted correctly by using only biased features. as a result, models are forced to learn from harder examples in which utilizing solely superficial features is not sufficient to make correct predictions. while these two state-of-the-art debiasing methods provide a remarkable improvement on the targeted out-of-distribution test sets, they do so at the cost of degrading the model's performance on the in-distribution setting, i.e., evaluation on the original test data which contains more diverse inference phenomena. it raises a question on whether these debiasing methods truly help in capturing a better notion of language understanding or simply biasing models to other directions. ideally, if such an improvement is achieved for the right reasons (i.e., better reasoning capabilities by learning a more general feature representation), a debiased model product-ofexpert learnedmixin conf-reg (our) in this work, we address this shortcoming by introducing a novel debiasing method that improves models' performance on the out-of-distribution examples while preserves the in-distribution accuracy. the method, called confidence regularization, draws a connection between the robustness against dataset biases and the overconfidence prediction problem in neural network models @xcite . we show that by preventing models from being overconfident on biased examples, they are less likely to exploit the simple cues from these examples. the motivation of our proposed training objective is to explicitly encourage models to make predictions with lower confidence (i.e., assigning a lower probability to the predicted label) on examples that contain biased features. ## related work biases in datasets researchers have recently studied more closely the success of large fine-tuned lms in many nlu tasks and found that models are simply better in leveraging biased patterns instead of capturing a better notion of language understanding for the intended task @xcite . models' performance often drops to a random baseline when evaluated on out-of-distribution datasets which are carefully designed to be void of the biases found in the training data. using such targeted evaluation, @xcite observe that models trained on mnli dataset @xcite leverage syntactic patterns involving word overlap to blindly predict entailment. similarly, @xcite show that the predictions of fact verification models trained for the fever task @xcite are largely driven by the presence of indicative words in the input claim sentences. following similar observations across other tasks and domains, e.g., visual question-answering @xcite , paraphrase identification @xcite , and argument reasoning comprehension @xcite , researchers proposed improved data collection techniques to reduce the artifacts that result in dataset biases. while these approaches are promising, only applying them without additional efforts in the modeling part may still deliver an unsatisfactory outcome. for instance, collecting new examples by asking human annotators to conform to specific rules may be costly and thus limit the scale and diversity of the resulting data @xcite . recently proposed adversarial filtering methods @xcite are more cost effective but are not guaranteed to be artifacts-free. it is, therefore, crucial to develop learning methods that can overcome biases as a complement to the data collection efforts. debiasing models there exist several methods that aim to improve models' robustness and generalization by leveraging the insights from previous work about the datasets' artifacts. in the nli task, @xcite make use of the finding that partial input information from the hypothesis sentence is sufficient to achieve reasonable accuracy. they then remove this hypothesis-only bias from the input representation using an adversarial training technique. more recently, three concurrent works @xcite @xcite introduce a modelagnostic debiasing method for nlu tasks called product-of-expert. @xcite al. (2019a) also propose an adaptive variant of this method called learned-mixin. these two methods first identify examples that can be predicted correctly based only on biased features. this step is done by using a biased model foot_0 , which is a weak classifier that is trained using only features that are known to be insufficient to perform the task but work well due to biases. the output of this pre-trained biased model is then used to adjust the loss function such that it down-weights the importance of examples that the biased model can solve. while this approach prevents models from learning the task mainly using biased features, it also reduces model's ability to learn from examples that can be solved using these features. as a result, models are unable to optimize accuracy on the original training distribution, and they possibly become biased in some other ways. similar to these methods, our method also uses a biased model to identify examples that exhibit biased features. however, instead of using it to diminish the training signal from these examples, we use it to scale the confidence of models' predictions. this enables the model to receive enough incentive to learn from all of the training examples. confidence regularization methods for regularizing the output distribution of neural network models have been used to improve generalization. @xcite propose to penalize the entropy of the output distribution for encouraging models to be less confident in their predictions. previously, @xcite introduce a label smoothing mechanism to reduce overfitting by pre-venting the model from assigning a full probability to each training example. our method regularizes models' confidence differently: we first perform an adaptive label smoothing for the training using knowledge distillation @xcite , which, by itself, is known to improve the overall performance. however, our method involves an additional bias-weighted scaling mechanism within the distillation pipelines. as we will show, our proposed scaling mechanism is crucial in leveraging the knowledge distillation technique for the purpose of overcoming the targeted bias while maintaining high accuracy in the training distribution. similar to our work, feng et al. ( 2018 ) propose a regularization method that encourages the model to be uncertain on specific examples. however, the objective and the methodology are different: they apply an entropy penalty term on examples that appear nonsensical to humans with the goal of improving models' interpretability. on the contrary, we apply our confidence regularization on every training example with a varying strength (i.e., higher uncertainty on more biased examples) to improve models' performance on the out-ofdistribution data. the key idea of our method is to explicitly train f m to compute lower probability, i.e., less confidence, on the predicted label when the input example exhibits a bias. this form of confidence regularization can be done by computing the loss function with the "soft" target labels that are obtained through our proposed smoothing mechanism. the use of soft targets as the training objective is motivated by the observation that the probability distribution of labels for each sample provides valuable information about the underlying task @xcite . when the soft targets of certain examples have higher entropy, models can be explicitly taught that some labels are more likely to be correct than the others. based on this intuition, we argue that adjusting the confidence on soft labels can better inform the model about the true conditional distribution of the labels given the presence of the biased features. we first produce a meaningful softened target distribution for each training example by performing knowledge distillation @xcite . in this learning framework, a "teacher" model f t , which we parameterize identically to the main model f m , is trained on the dataset d using a standard classification loss. we then use f t to compute output probability distribution pi , where f t (x i ) = pi . in the original knowledge distillation approach, the output of the teacher model pi is then used to train f m . we extend this approach by adding a novel scaling procedure before we distill the teacher model into f m . we define a scaling function s that takes the probability distribution pi and scale it such that the probability assigned to its predicted label is lowered when the example can be predicted well by only relying on the biased features. training the biased model for several nlu tasks, biased features are known a-priori, e.g., the word overlapping features in nli datasets are highly correlated with the entailment label @xcite . we leverage this a-priori knowledge to design a measure of how well an example can be predicted given only the biased features. we refer to this measure as bias weight, denoted as β i for every example x i . similar to previous debiasing methods @xcite , we compute bias weights using a biased model. this biased model, denoted as f b , predicts the probability distribution b i , where we define the bias weight β i as the scalar value of the as-signed probability by f b to the ground truth label: bias-weighted scaling as illustrated in figure 1 , our method involves scaling the teacher output pi using β i . we do this by defining a scaling function for @xmath0, ..., k. the value of β i controls the strength of the scaling: as β @xmath1, the scaled probability assigned to each label approaches 1 k , which presents a minimum confidence. conversely, when β @xmath2, the teacher's probability distribution remains unchanged, i.e., s( pi , 0) = pi . training the main model the final step is to train f m by distilling from the scaled teacher model's outputs. since the main model is parameterized identically to the teacher model, we refer to this step as self-distillation @xcite . self-distillation is performed by training f m on pairs of input and the obtained soft target labels (x i , s( pi , β i )). specifically, f m is learned by minimizing a standard cross-entropy loss between the scaled teacher's output s( pi , β i ) and the current prediction of the main model: in practice, each s( pi , β i ) is computed only once as a preprocessing step. our method does not require hyperparameters, which can be an advantage since most out-of-distribution datasets do not provide a development set for tuning the hyperparameters. ## experimental setup in this section, we describe the datasets, models, and training details used in our experiments. ## results the results for the tasks of nli, fact verification, and paraphrase identification are reported in ## discussions and analysis ablation studies in this section, we show that the resulting improvements from our method come from the combination of both self-distillation and our scaling mechanism. we perform ablation studies to examine the impact of each of the components including (1) self-distillation: we train a model using the standard self-distillation without bias-weighted scaling, and (2) examplereweighting: we train a model with the standard cross-entropy loss with an example reweighting method to adjust the importance of individual examples to the loss. the weight of each example is obtained from the (scaled) probability that is assigned by the teacher model to the ground truth label. 8 the aim of the second setting is to exclude the effect of self-distillation while keeping the effect of our scaling mechanism. in-distribution performance drop of productof-expert the difference between our method with product-of-expert and its variants is the use of biased examples during training. product-ofexpert in practice scales down the gradients on the biased training examples to allow the model to focus on learning from the harder examples @xcite . as a result, models often receive little to no incentive to solve these examples throughout the training, which can effectively reduce the training data size. our further examination on a product-ofexpert model (trained on mnli for hans) shows that its degradation of in-distribution performance largely comes from the aforementioned examples. ensembling back the biased-model to the main model can indeed bring the in-distribution accuracy back to the bert baseline. however, this also leads to the original poor performance on hans, which is counterproductive to the goal of improving the out-of-distribution generalization. impact on models' calibration we expect the training objective used in our method to discourage models from making overconfident predictions, i.e., assigning high probability to the predicted labels even when they are incorrect. we investigate the changes in models' behavior in terms of their confidence using the measure of calibration, which quantifies how aligned the confidence of the predicted labels with their actual accuracy are @xcite . we compute the expected calibration error (ece) @xcite as a scalar summary statistic of calibration. results in ## conclusion existing debiasing methods improve the performance of nlu models on out-of-distribution datasets. however, this improvement comes at the cost of strongly diminishing the training signal from a subset of the original dataset, which in turn reduces the in-distribution accuracy. in this paper, we address this issue by introducing a novel method that regularizes models' confidence on biased examples. this method allows models to still learn from all training examples without exploiting the biases. our experiments on four out-of-distribution datasets across three nlu tasks show that our method provides a competitive outof-distribution performance while preserves the original accuracy. our debiasing framework is general and can be extended to other task setups where the biases leveraged by models are correctly identified. several challenges in this direction of research may include extending the debiasing methods to overcome multiple biases at once or to automatically identify the format of those biases which simulate a setting where the prior knowledge is unavailable.