Title: SIMPLE BUT EFFECTIVE TECHNIQUES TO REDUCE DATASET BIASES

Abstract: There have been several studies recently showing that strong natural language understanding (NLU) models are prone to relying on unwanted dataset biases without learning the underlying task, resulting in models which fail to generalize to out-of-domain datasets, and are likely to perform poorly in real-world scenarios. We propose several learning strategies to train neural models which are more robust to such biases and transfer better to out-of-domain datasets. We introduce an additional lightweight bias-only model which learns dataset biases and uses its prediction to adjust the loss of the base model to reduce the biases. In other words, our methods down-weight the importance of the biased examples, and focus training on hard examples, i.e. examples that cannot be correctly classified by only relying on biases. Our approaches are model agnostic and simple to implement. We experiment on large-scale natural language inference and fact verification datasets and their out-of-domain datasets and show that our debiased models significantly improve the robustness in all settings, including gaining 9.76 points on the FEVER symmetric evaluation dataset, 5.45 on the HANS dataset and 4.78 points on the SNLI hard set. These datasets are specifically designed to assess the robustness of models in the out-of-domain setting where typical biases in the training data do not exist in the evaluation set.

1. INTRODUCTION:

Recent neural models (Devlin et al., 2019; Radford et al., 2018; Chen et al., 2017) have achieved high and even near human-performance on several large-scale natural language understanding benchmarks. However, it has been demonstrated that neural models tend to rely on existing idiosyncratic biases in the datasets, and leverage superficial correlations between the label and existing shortcuts in the training dataset to perform surprisingly well1 , without learning the underlying task (Kaushik & Lipton, 2018; Gururangan et al., 2018; Poliak et al., 2018; Schuster et al., 2019; Niven & Kao, 2019; McCoy et al., 2019) . For instance, natural language inference (NLI) consists of determining whether a hypothesis sentence (There is no teacher in the room) can be inferred from a premise sentence (Kids work at computers with a teacher's help)2 (Dagan et al., 2006) . However, recent work has demonstrated that large-scale NLI benchmarks contain annotation artifacts; certain words in the hypothesis are highly indicative of inference class that allow models with poor premise grounding to perform unexpectedly well (Poliak et al., 2018; Gururangan et al., 2018) . As an example, in some NLI benchmarks, negation words such as "nobody", "no", and "not" in the hypothesis are often highly correlated with the contradiction label. As a consequence, NLI models do not need to learn the true relationship between the premise and hypothesis and instead can rely on statistical cues, such as learning to link negation words with the contradiction label.

As a result of the existence of such biases, models exploiting statistical shortcuts during training often perform poorly on out-of-domain datasets, especially if they are carefully designed to limit the spurious cues. To allow proper evaluation, recent studies have tried to create new evaluation datasets that do not contain such biases (Gururangan et al., 2018; Schuster et al., 2019) . Unfortunately, it is hard to avoid spurious statistical cues in the construction of large-scale benchmarks, and collecting new datasets is costly (Sharma et al., 2018) . It is therefore crucial to develop techniques to reduce the reliance on biases during the training of the neural models.

In this paper, we propose several end-to-end debiasing techniques to adjust the cross-entropy loss to reduce the biases learned from datasets, which work by down-weighting the biased examples so that the model focuses on learning hard examples. Figure 1 illustrates an example of applying our strategy to prevent an NLI model from predicting the labels using existing biases in the hypothesis. Our strategy involves adding a bias-only branch f B on top of the base model f M during training (In case of NLI, the bias-only model only uses the hypothesis). We then compute the combination of the two models f C in a way to motivate the base model to learn different strategies than the ones used by the bias-only branch f B . At the end of the training, we remove the bias-only classifier and use the predictions of the base model.

We propose three main debiasing strategies, detailed in Section 2.2. In our first two proposed methods, the combination is done with an ensemble method which combines the predictions of the base and the bias-only models. Our approaches are simple and highly effective. They require training a simple classifier on top of the base model. Furthermore, our methods are model agnostic and general enough to be applicable for addressing common biases seen in several datasets in different domains.

We evaluate our models on challenging benchmarks in textual entailment and fact verification. For entailment, we run extensive experiments on HANS (Heuristic Analysis for NLI Systems) (McCoy et al., 2019) , and hard NLI sets of Stanford Natural Language Inference (SNLI) (Bowman et al., 2015) and MultiNLI (MNLI) (Williams et al., 2018) datasets (Gururangan et al., 2018) . We additionally construct hard MNLI datasets from MNLI development sets to facilitate the out-of-domain evaluation on this dataset3 . Furthermore, we evaluate our fact verification models on FEVER Symmetric test set (Schuster et al., 2019) . The selected datasets are highly challenging and have been carefully designed to be unbiased to allow proper evaluation of the out-of-domain performance of the models. We show that including our strategies on training baseline models including BERT (Devlin et al., 2019) provide substantial gain on out-of-domain performance in all the experiments.

In summary, we make the following contributions: 1) Proposing several debiasing strategies to train neural models that make them more robust to existing biases in the dataset. 2) An empirical evaluation of the proposed methods on two large-scale NLI benchmarks and obtaining substantial gain on their challenging out-of-domain data, including 5.45 points on HANS and 4.78 points on SNLI hard set. 3) Evaluating our models on fact verification, obtaining 9.76 points gain on FEVER symmetric test set, improving the results of prior work by 4.65 points.

To facilitate future work, we release our datasets and code.

2. REDUCING BIASES:

Problem formulation We consider a general multi-class classification problem. Given a dataset D = {x i , y i } N i=1 consisting of the input data x i ∈ X , and labels y i ∈ Y, the goal of the base model is to learn a mapping f M parameterized by θ M which computes the predictions over the label space given the input data, shown as f M : X → R |Y| . Our goal is to optimize θ M parameters such that we build a model which is more resistant to benchmark biases to improve its robustness to domain changes when the typical biases observed in the training data do not exist in the evaluation dataset.

The key idea of our approach, depicted in Figure 1 is first to identify the dataset biases and heuristic patterns which the base model is susceptible to relying on. Then, we use a bias-only branch to capture these biases. We propose several strategies to incorporate the bias-only knowledge into the training of the base model to make a robust version of it. After training we remove the biasonly model and use the predictions of the base model. In this section, we explain each of these components.

2.1. BIAS-ONLY BRANCH:

We assume that we do not have access to any data from the out-of-domain dataset, so we need to know a priori about the possible types of shortcut patterns we would like the base model to avoid relying on them. Once these shortcut patterns are identified, we train a bias-only model designed to capture the identified biases which only uses the biased features. For instance, it has been shown that a hypothesis-only model in the large-scale NLI datasets can correctly classify the majority of samples using the artifacts (Poliak et al., 2018; Gururangan et al., 2018) . Therefore, our bias-only model for NLI only uses hypothesis sentences. But note that the bias-only model can, in general, have any form, and is not limited to models which are using only a part of input data. Let x b i ∈ X b be biased features of x i which are predictive of y i . We then formalize this bias-only model as a mapping f B : X b → R |Y| parameterized by θ B trained using cross-entropy loss L B :

EQUATION

where a i is the one-hot representation of the true label for the i th example. In the next section, we explain how we use the bias-only model to make a robust version of the base model.

2.2. PROPOSED DEBIASING STRATEGIES:

We propose several strategies to incorporate the bias-only f B knowledge into training of the base model f M and update its parameters θ M using the obtained loss L C of the combined classifier f C . All these strategies have the form illustrated in Figure 1 , where the predictions of the bias-only model are combined with either the predictions of the base model or its error to down-weight the loss from the biased examples, thereby affecting the error backpropagated into the base model.

As also illustrated in Figure 1 , it is often convenient for the bias-only model to share parameters with the base model, such as sharing a sentence encoder. To prevent the base model from learning the biases, the bias-only loss L B is not back-propagated to these shared parameters of the base model.

To accommodate this sharing, the bias-only and the base models are trained together. Next, we explain how the loss of the combined classifier, L C , is computed for each of our debiasing methods.

2.2.1. METHOD 1: PRODUCT OF EXPERTS:

Our first approach is based on the idea of the product of experts ensemble method (Hinton, 2002) : "It is possible to combine multiple probabilistic models of the same data by multiplying the probabilities together and then renormalizing.". Here, we use this notion to combine the bias-only and base model predictions by computing the element-wise product between their predictions as f B (x b i ) f M (x i ). We compute this combination in the logarithmic space, which works better in practice:

EQUATION

The key intuition behind this model is to combine the probability distributions of the bias-only and the base model to allow them to make predictions based on different characteristics of the input; the bias-only branch covers prediction based on biases, and the base model focuses learning the actual task. We then compute L C as the cross-entropy loss of the combined predictions f C . Then the base model parameters θ M are trained using the cross-entropy loss of the combined classifier f C : 2019)'s study is limited to alleviating biases in VQA benchmarks. We, however, evaluate the effectiveness of their formulation together with our newly proposed variations in the natural language understanding context on several challenging NLU datasets.

EQUATION

We first apply a sigmoid function to the bias-only model's predictions to obtain a mask containing an importance weight between 0 and 1 for each possible label. We then compute the element-wise product between the obtained mask and the base model's predictions:

EQUATION

The main intuition is to dynamically adjust the predictions of the base model to prevent the base model from leveraging the shortcuts. We note two properties of this loss. (1) When the bias-only model correctly classifies the example, the mask increases the value of the correct prediction while decreases the scores for other labels. As a result, the loss of biased examples is down-weighted. (2) For the hard examples that cannot be correctly classified using bias-only model, the obtained mask increases the score of the wrong answer. This, in turn, increases the contribution of hard examples and encourages the base model to learn the importance of correcting them. We additionally propose the following new variants of this model:

1. Computing the combination in logarithmic space, which we refer to it as RUBI + log space.

EQUATION

2. Normalizing the output of the bias-only model, followed by RUBI model, which we refer to it as RUBI + normalize:

EQUATION

As with our first method, we then update the parameters of the base model θ M by backpropagating the cross-entropy loss L C of the combined classifier.

2.2.3. METHOD 3: DEBIASED FOCAL LOSS:

Focal loss was originally proposed in Lin et al. (2017) to improve a single classifier by downweighting the well-classified points. We propose a novel variant of this loss, in which we leverage the bias-only branch's predictions to reduce the relative importance of the most biased examples and allow the model to focus on learning the hard examples. We define Debiased Focal Loss as:

EQUATION

where γ is the focusing parameter, which impacts the down-weighting rate. When γ is set to 0, our Debiased Focal Loss is equivalent to the normal cross-entropy loss. For γ > 0, as the value of γ is increased, the effect of down-weighting is increased. We set γ = 2 through all experiments, which works well in practice and avoid fine-tuning it further. We note the properties of the Debiased Focal Loss: (1) When the example x i is unbiased, and bias-only branch does not do well, softmax (f B (x b i )) is small, therefore the scaling factor is close to 1, and the loss remains unaffected. (2) As the sample is more biased and softmax (f B (x b i )) is closer to 1, the modulating factor approaches 0 and the loss for the most biased examples is down-weighted.

For this debiasing strategy, Debiased Focal Loss is then used to update the parameters of the base model θ M . Note that this loss has a different form from that used for the first two methods.

3. EXPERIMENTAL RESULTS:

We provide experiments on two large-scale NLI datasets, namely SNLI and MNLI, and FEVER dataset for our fact verification experiment and evaluate the models' performance on their challenging unbiased evaluation datasets proposed very recently. In most of our experiments, we consider BERT4 as our baseline which is known to work well for these tasks, and additionally, we have included other baselines used in the prior work to compare against them. In all the experiments, we kept the hyperparameters of baselines as the default. We include low-level details in the appendix.

3.1. FACT VERIFICATION:

Dataset: FEVER dataset contains claim-evidence pairs generated from Wikipedia. Schuster et al. (2019) collect a new evaluation set for FEVER dataset to avoid the idiosyncrasies observed in the claims of this benchmark. They make the original claim-evidence pairs of FEVER evaluation dataset symmetric, by augmenting the dataset and making each claim and evidence appear with each label. Therefore, by balancing the artifacts, relying on cues from claim to classify samples is equivalent to a random guess. The collected dataset is challenging and the performance of the models evaluated on this dataset drop significantly.

Base models: We consider BERT as the baseline, which works the best on this dataset (Schuster et al., 2019) , and predicts the relations based on the concatenation of the claim and the evidence with a delimiter token (see Appendix A).

Bias-only model::

The bias-only model predicts the labels using only claims as input.

Results: Table 1 shows the results. The obtained improvement of our debiasing methods varies between 1.11-9.76 absolute points. The Product of experts and Debiased Focal loss are highly effective, boosting the performance of the baseline model by 9.76 and 7.53 absolute points respectively, significantly surpassing the prior work (Schuster et al., 2019) .

3.2. TEXTUAL ENTAILMENT:

Datasets: We evaluate on hard SNLI and MNLI datasets (Gururangan et al., 2018) which are the split of these datasets where a hypothesis-only model cannot correctly predict the labels. Gururangan et al. (2018) show that the success of the recent textual entailment models is attributed to the biased examples, and the performance of these models are substantially lower on hard sets.

Base models: We consider InferSent (Conneau et al., 2017) , and BERT as our base models. We choose InferSent to be able to compare against the prior work (Belinkov et al., 2019b) .

The bias-only model only uses the hypothesis to predict the labels (see Appendix B). 

Results on MNLI::

We construct hard sets from MNLI development set for both MNLI Matched and MNLI Mismatched datasets. Following Gururangan et al. (2018) , we train a fastText classifier (Joulin et al., 2017) , to predict the labels using only the hypothesis and consider the subset of the samples on which our trained hypothesis-only classifier failed as hard examples. Table 3 shows the results on the development sets and their corresponding hard sets. For BERT baseline, on MNLI matched hard dataset, the product of experts and RUBI+normalize improve the results the most by 1.46 and 1.11 points. On MNLI mismatched hard, the Debiased Focal Loss and product of experts obtain 1.37, and 1.68 points gain respectively. For InferSent baseline, on MNLI matched hard, the product of experts and RUBI improve the results by 2.34 and 0.94 points. On MNLI mismatched hard, the Product of experts and Debiased Focal Loss improve the results by 2.61 and 2.52 points.

To comply with limited access to the submission system of MNLI, we evaluate only the best result of baseline and our models on the test sets. Table 4 shows the results on the MNLI test and hard sets.

Our product of expert model improves the performance on MNLI matched hard set by 0.93 points and 1.08 points on MNLI Mismatched hard set while maintaining the in-domain accuracy. 2019): 1) Whether all the words in the hypothesis are included in the premise. 2) If the hypothesis is the contiguous subsequence of the premise. 3) If the hypothesis is a subtree in the premise's parse tree 4) The number of tokens shared between premise and hypothesis normalized by the number of tokens in the premise. We additionally include some similarity features: 5) The cosine similarity between premise and hypothesis tokens followed by mean and max-pooling. We consider the same weight for contradiction and neutral labels in the bias-only loss to allow the model to recognize entailment from not-entailment. During the evaluation, we map the neutral and contradiction labels to not-entailment.

Results: As shown in Table 5 , the Product of experts and Debiased Focal loss improve the results the most by 5.45, 3.89 points. We provide the accuracy for each label on HANS dataset in Appendix C. As expected, improving the out-of-domain performance could come at the expense of the decreased in-domain performance, since the removed biases are useful for performing the in-domain task. This especially happens for Debiased Focal Loss, in which there is a trade-off between in-domain and out-of-domain performance as discussed depending on the parameter γ, and when the baseline model is not very powerful like InferSent. Our other models with BERT baseline consistently remain the in-domain performance. 

4. DISCUSSION:



Analysis of Debiased Focal Loss:



5. RELATED WORK:

Biases in NLU benchmarks and other domains Recent studies have shown that large-scale NLU benchmarks contain biases. Poliak et al. (2018) ; Gururangan et al. (2018); McCoy et al. (2019) demonstrate that textual entailment models can rely on annotation artifacts and heuristic patterns to perform unexpectedly well. On ROC Stories corpus (Mostafazadeh et al., 2016) , Schwartz et al. (2017) show that considering only sample endings without story contexts performs exceedingly well. A similar phenomenon is observed in fact verification (Schuster et al., 2019) , argument reasoning comprehension (Niven & Kao, 2019) , and reading comprehension (Kaushik & Lipton, 2018) . Finally, several studies confirm biases in VQA datasets, leading to accurate question-only models ignoring visual content (Goyal et al., 2017; Zhang et al., 2016) .

Existing techniques to alleviate biases The most common strategy to date to address biases is to augment the datasets by balancing the existing cues (Schuster et al., 2019; Niven & Kao, 2019) . In another line of work, to address the shortcoming in Stanford Question Answering dataset (Rajpurkar et al., 2016) , Jia & Liang (2017) propose to create an adversarial dataset in which they insert adversarial sentences to the input paragraphs. However, collecting new datasets especially in large-scale is costly and it remains an unsatisfactory solution. It is, therefore, crucial to develop strategies to allow training models on the existing biased datasets, while improving their out-of-domain performance. Schuster et al. (2019) propose to first compute the n-grams existing in the claims which are the most associated with each label. They then solve an optimization problem to assign a balancing weight to each training sample to alleviate the biases. In contrast, we propose several end-to-end debiasing strategies. Additionally, Belinkov et al. (2019a) propose adversarial techniques to remove from the sentence encoder the features which allow a hypothesis-only model to succeed. However, we believe that in general the features used by the hypothesis-only model can include some information necessary to perform the NLI task, and removing such information from the sentence representation can hurt the performance of the full model. Their approach consequently degrades the performance on hard SNLI dataset which is expected to be less biased. In contrast to their method, we propose to train a bias-only model to use its predictions to dynamically adapt the classification loss to reduce the importance of the most biased examples during training.

Concurrently to our own work, Clark et al. (2019) ; He et al. (2019) have also proposed to use the product of experts models. However, we have evaluated on new domains and datasets, and have proposed several different ensemble-based debiasing techniques.

6. CONCLUSION:

We propose several novel techniques to reduce biases learned by neural models. We introduce a bias-only model that is designed to capture biases and leverages the existing shortcuts in the datasets to succeed. Our debiasing strategies then work by adjusting the cross-entropy loss based on the performance of this bias-only model to focus learning on the hard examples and down-weight the importance of the biased examples. Our proposed debiasing techniques are model agnostic, simple and highly effective. Extensive experiments show that our methods substantially improve the model robustness to domain-shift, including 9.76 points gain on FEVER symmetric test set, 5.45 on HANS dataset and 4.78 points on SNLI hard set.

B TEXTUAL ENTAILMENT:

Base model: InferSent uses a separate BiLSTM encoder to learn sentence representations for premise and hypothesis, it then combines these embeddings following Mou et al. (2016) and feeds them to the default nonlinear classifier. For InferSent we train all models for 20 epochs as default without using early-stopping. We use the default hyper-parameters and following Wang et al. (2019) , we set BiLSTM dimension to 512. We use the default nonlinear classifier with 512 and 512 hidden neurons with Tanh nonlinearity. For Bert model, we finetune the models for 3 epochs.

Bias-only model For BERT model, we use the same shallow nonlinear classifier explained in Appendix A, and for the InferSent model, we use a shallow linear classifier with 512, and 512 hidden units.

C SYNTACTIC BIAS:

Base model: We finetune all the models for 3 epochs.

Bias-only model: We use a nonlinear classifier with 6 and 6 hidden units with Tanh nonlinearity.

Results: Table 6 shows the performance for each label (entailment and non entailment) on HANS dataset and its individual heuristics. 

:

We use biases, heuristic patterns or shortcuts interchangeably.

The given sentences are in the contradictory relation and the hypothesis cannot be inferred from the premise.

Removing the need of submitting to an online evaluation system for MNLI hard test sets.

https://github.com/huggingface/pytorch-pretrained-BERT

Figures and Tables:

Figure 1: An illustration of our debiasing strategies on NLI. Solid arrows show the flow of input information, and dotted arrows show the back-propagation flow of error. The combination is done in different ways for different models. Blue highlighted modules are removed after training. At test time, only the predictions of the base model f M are used.

Base model: We use BERT as our base model and train it on MNLI dataset. Bias-only model: We consider several features for the bias-only model. The first three features are based on the syntactic heuristics proposed in McCoy et al. (

Figure 2: Accuracy of InferSent model trained with Debiased Focal Loss on SNLI and SNLI hard sets for different values of γ. Increasing γ reduces the contribution of the biased examples to the loss and concentrates the model on learning hard examples.

Results on FEVER development (Dev) set and FEVER symmetric test set.

Table content: <table><tr><td>Debiasing method</td><td>Dev</td><td>Symmetric test set</td></tr><tr><td>None</td><td>85.99</td><td>56.49</td></tr><tr><td>RUBI</td><td>86.23</td><td>57.60</td></tr><tr><td>RUBI + log space</td><td>86.59</td><td>59.27</td></tr><tr><td>RUBI + normalize</td><td>86.16</td><td>60.11</td></tr><tr><td colspan="2">Debiased Focal Loss 83.07</td><td>64.02</td></tr><tr><td>Product of experts</td><td>86.46</td><td>66.25</td></tr><tr><td colspan="2">Schuster et al. (2019) 84.6</td><td>61.6</td></tr></table>

Results on SNLI and SNLI hard sets. Table2shows the results on SNLI dataset. For InferSent model, Debiased Focal Loss and Product of experts methods result in 4.14 and 4.78 points gain. Similarly, for the BERT model, Debiased Focal loss and Product of experts improve the results the most by 2.48 and 1.62 absolute points. Comparing to the results ofBelinkov et al. (2019b), our product of expert model obtains a 7.42 point gain, significantly surpassing the prior work.

Table content: <table><tr><td>Debiasing method</td><td colspan="3">BERT Test Hard Test InferSent Hard</td></tr><tr><td>None</td><td colspan="3">90.53 80.53 84.24 68.91</td></tr><tr><td>RUBI + log space</td><td colspan="3">90.74 81.32 83.67 69.0</td></tr><tr><td>RUBI</td><td colspan="3">90.69 80.62 83.93 69.64</td></tr><tr><td>RUBI + normalize</td><td colspan="3">90.70 80.83 83.6</td><td>69.24</td></tr><tr><td>Debiased Focal Loss</td><td colspan="3">89.57 83.01 73.54 73.05</td></tr><tr><td>Product of experts</td><td colspan="3">90.11 82.15 80.35 73.69</td></tr><tr><td colspan="2">AdvCls (Belinkov et al., 2019b) -</td><td>-</td><td>83.56 66.27</td></tr><tr><td colspan="2">AdvDat (Belinkov et al., 2019b) -</td><td>-</td><td>78.30 55.60</td></tr><tr><td>Results on SNLI:</td><td/><td/></tr></table>

Results on MNLI matched (MNLI) and mismatched (MNLI-M) dev and their hard sets.

Table content: <table><tr><td/><td/><td>BERT</td><td/><td/><td>InferSent</td></tr><tr><td>Debiasing method</td><td colspan="2">MNLI</td><td colspan="2">MNLI-M</td><td>MNLI</td><td>MNLI-M</td></tr><tr><td/><td>Dev</td><td colspan="2">Hard Dev</td><td colspan="2">Hard Dev</td><td>Hard Dev</td><td>Hard</td></tr><tr><td>None</td><td colspan="5">84.41 76.56 84.53 77.55 69.97 57.03 69.99 56.53</td></tr><tr><td>RUBI + log space</td><td colspan="5">84.46 76.80 84.86 78.04 69.70 56.57 69.95 56.56</td></tr><tr><td>RUBI</td><td colspan="5">84.48 77.13 85.17 78.63 70.51 57.97 70.53 58.08</td></tr><tr><td>RUBI + normalize</td><td colspan="5">84.80 77.67 84.77 78.54 70.16 57.53 70.09 57.62</td></tr><tr><td colspan="6">Debiased Focal Loss 83.72 77.37 84.85 78.92 60.78 57.88 61.12 59.05</td></tr><tr><td>Product of experts</td><td colspan="5">84.58 78.02 84.85 79.23 66.02 59.37 65.85 59.14</td></tr></table>

Results on MNLI matched (MNLI) and mismatched (MNLI-M) test and hard sets.

Table content: <table><tr><td>Debiasing Method</td><td>MNLI Test Hard</td><td>MNLI-M Test Hard</td></tr><tr><td>None</td><td colspan="2">84.11 75.88 83.51 75.75</td></tr><tr><td colspan="3">Product of experts 84.11 76.81 83.47 76.83</td></tr></table>

Results on MNLI matched dev and HANS datasets including its individual heuristic subsets.McCoy et al. (2019) show that NLI models can rely on superficial syntactic heuristics to perform the task. They introduce HANS dataset, which covers several examples on which the models employing the syntactic heuristics fail.

Table content: <table><tr><td>Debiasing Method</td><td colspan="5">MNLI HANS Constituent Lexical Subsequence</td></tr><tr><td>None</td><td>83.99</td><td>61.10</td><td>61.11</td><td>68.97</td><td>53.21</td></tr><tr><td>RUBI + log space</td><td>84.56</td><td>62.53</td><td>57.77</td><td>75.18</td><td>54.65</td></tr><tr><td>RUBI</td><td>83.93</td><td>60.35</td><td>56.51</td><td>71.09</td><td>53.44</td></tr><tr><td>RUBI + normalize</td><td>84.15</td><td>59.71</td><td>55.74</td><td>70.01</td><td>53.37</td></tr><tr><td colspan="2">Debiased Focal Loss 84.33</td><td>64.99</td><td>62.42</td><td>74.45</td><td>58.11</td></tr><tr><td>Product of experts</td><td>84.04</td><td>66.55</td><td>64.29</td><td>77.61</td><td>57.75</td></tr><tr><td>3.3 SYNTACTIC BIAS</td><td/><td/><td/><td/><td/></tr><tr><td>Dataset:</td><td/><td/><td/><td/><td/></tr></table>

To understand the impact of γ in Debiased Focal Loss, we train InferSent models with this loss for different values of γ on SNLI dataset and evaluate its performance on SNLI and SNLI hard sets. As illustrated in Figure2, increasing γ focuses the loss on learning hard examples, and reduces the attention on learning biased examples. Consequently, the in-domain accuracy on SNLI is dropped but out-of-domain accuracy on SNLI hard set is increased.Results: Through extensive experiments on different datasets, our methods improve out-of-domain performance in all settings. Debiased Focal Loss and Product of experts models consistently obtain the highest gains. Within RUBI variations, RUBI+log space outperforms the other variations on SNLI with BERT baseline and HANS dataset. RUBI+normalize does better than the rest on FEVER experiment and MNLI matched hard set with BERT baseline. RUBI performs the best on SNLI and MNLI experiments with InferSent baseline, and MNLI mismatched hard with BERT baseline.

Table content: <table/>

Accuracy for each label of entailment or non-entailment on HANS and its individual heuristics.

Table content: <table><tr><td>Debiasing Method</td><td/><td colspan="2">gold label: Entailment</td><td/><td colspan="4">gold label: Non-entailment</td></tr><tr><td/><td colspan="8">HANS Const. Lexical Subseq. HANS Const. Lexical Subseq.</td></tr><tr><td>None</td><td>98.37</td><td>98.98</td><td>96.76</td><td>99.38</td><td>23.82</td><td>23.24</td><td>41.18</td><td>7.04</td></tr><tr><td>RUBI+log space</td><td>97.51</td><td>98.56</td><td>95.44</td><td>98.54</td><td>27.55</td><td>16.98</td><td>54.92</td><td>10.76</td></tr><tr><td>RUBI</td><td>97.27</td><td>99.18</td><td>95.26</td><td>97.38</td><td>23.42</td><td>13.84</td><td>46.92</td><td>9.50</td></tr><tr><td>RUBI+normalize</td><td>97.87</td><td>98.48</td><td>96.32</td><td>98.80</td><td>21.55</td><td>13.00</td><td>43.70</td><td>7.94</td></tr><tr><td colspan="2">Debiased Focal loss 96.41</td><td>97.66</td><td>92.92</td><td>98.66</td><td>33.57</td><td>27.18</td><td>55.98</td><td>17.56</td></tr><tr><td>Product of experts</td><td>96.08</td><td>98.38</td><td>93.52</td><td>96.34</td><td>37.02</td><td>30.20</td><td>61.70</td><td>19.16</td></tr></table>

Peer review (iclr2020):
This manuscript introduces a general framework to deal with bias(es) in the datasets. The authors propose a bias-only model designed to capture biases in the data. The authors claim that performance of the bias only branch influences the overall cross-entropy loss which leads to learning bias-free models. The paper is on a very interesting topic, but it lacks proper methodological motivation, discussion, and backing up of the claims. Furthermore, the experiments only show accuracy results, while one expects to see a set of experiments demonstrating how the bias was actually removed from the target learned model. There are some major concerns on this paper: - The proposed methods all operate on the last loss component on how to combine the loss of the bias branch and the one coming from the actual model. The authors hypothetically mention that with this combination the model will learn to remove the bias-contaminated features from the model side because the bias branch already has them. This is very hypothetical and there is no guarantee to it. There is no regularization to make the model branch avoid the bias-contaminated features. There is absolutely no theoretical nor any experimental studies on this issue in the paper. Models are pretty much prone to cheating. They will cheat if they can. If there is a bias in the dataset, the model will capture it and cheat, even if it leads to redundant information coming from the bias side. - The second important shortcoming of the paper is lack of proper experiments to back up the claims in the paper. Experiments only show simple accuracy results with our without bias. But the authors keep claiming that their model learns to remove the bias. If that is the case, correlation analysis between the features learned from the model branch and bias terms/variables is missing. Also, there is no direct comparison on how each of the three proposed losses differ from each other in removing the effects of the bias. - The authors show in the experiments that their proposed model is better than the hypothesis-only model. This can only be attributed to the fact that info from the premise and in general the added meta-data help the learning framework to learn better features as it is regularized better (and constrained more). This does not mean that the bias is removed from them. There is no discussion or experiments on this issue. Minor: - The proposed models are all similar in terms of modifying the final loss function of L_C, how the outputs of B and M models are ensembled. It may not be a good idea to keep claiming that several different methods are proposed in this paper for dealing with bias. Nature of all methods is the same. - Figure 1 that is the introduced in the introduction and referred to ever after is not clear and self-contained. One cannot understand it until they read until the end of the method section. - There is no theoretical analysis or discussion on why the proposed loss functions should be able to remove bias!