Title: MITIGATING BIAS IN NATURAL LANGUAGE INFERENCE USING ADVERSARIAL LEARNING Anonymous authors Paper under double-blind review

Abstract: Recognizing the relationship between two texts is an important aspect of natural language understanding (NLU), and a variety of neural network models have been proposed for solving NLU tasks. Unfortunately, recent work showed that the datasets these models are trained on often contain biases that allow models to achieve non-trivial performance without possibly learning the relationship between the two texts. We propose a framework for building robust models by using adversarial learning to encourage models to learn latent, bias-free representations. We test our approach in a Natural Language Inference (NLI) scenario, and show that our adversarially-trained models learn robust representations that ignore known dataset-specific biases. Our experiments demonstrate that our models are more robust to new NLI datasets.

1. INTRODUCTION:

Recognizing the relationship between two texts is a significant aspect of general natural language understanding (NLU) (Allen, 1995) . Natural Language Inference (NLI) is often used to gauge a model's ability to understand such a relationship between two texts (Cooper et al., 1996; Dagan et al., 2006) . In NLI, a model is tasked with determining whether a hypothesis (the animal moved) would likely be inferred from a premise (a black cat ran). The development of new large-scale datasets has led to a flurry of various neural network architectures for solving NLI. However, recent work has found that many NLI datasets contain biases that enable hypothesis-only models -models that are given access to the hypothesis alone -to perform surprisingly well without possibly learning the relationship between two texts. For instance, annotation artifacts and statistical irregularities in the popular Stanford Natural Language Inference dataset (SNLI) (Bowman et al., 2015) allowed hypothesis-only models to perform at double the majority class baseline, and at least 5 other recent NLI datasets contain similar biases (Gururangan et al., 2018; Poliak et al., 2018c; Tsuchiya, 2018) . We will use the terms "artifacts" and "biases" interchangeably.

The existence of annotation artifacts in large-scale NLI datasets is detrimental for making progress in deep learning research for NLU. How can we trust the performance of top models if it is possible to infer the relationship without even looking at the premise? Solutions to this concern are so far unsatisfactory: constructing new datasets (Sharma et al., 2018) is costly and may still result in other artifacts; filtering "easy" examples and defining a harder subset is useful for evaluation purposes (Gururangan et al., 2018) , but difficult to do on a large scale that will enable training; and compiling adversarial examples (Glockner et al., 2018) is informative but again limited by scale or diversity. Furthermore, these solutions do not address a lingering question: can we develop models that will generalize well despite many NLI datasets containing specific hypothesis-only biases?

Inspired by domain-adversarial training of neural networks (Ganin & Lempitsky, 2015; Ganin et al., 2016) , we propose two architectures (Figure 1 ) that enable a model to perform well on other NLI datasets regardless of what annotation artifacts exist in the training corpus's hypotheses. While learning to classify the relationship between two texts, we simultaneously use adversarial learning to discourage our model from using dataset-specific biases.In this way, the resulting representations contain fewer biases, and the model is encouraged to learn the relationship between the two texts. Our experiments demonstrate that our architectures generate sentence representations that are more robust to annotation artifacts, and also transfer better: when trained on one dataset and evaluated on another, they perform better than a non-adversarial model in 9 out of 12 target datasets. The methodology can also be extended to other NLU tasks, and we outline the necessary changes to our architectures in the conclusion. To our knowledge, this is the first study that explores methods to ignore hypothesis-only biases when training NLI models.

2. RELATED WORK:



2.1. BIASES AND ARTIFACTS IN NLU DATASETS:

Annotation artifacts or biases were found in multiple NLU datasets and tasks. Early work on NLI, also known as recognizing textual entailment (RTE), found biases that allowed models to perform relatively well by focusing on syntactic clues alone (Snow et al., 2006; Vanderwende & Dolan, 2006) . More recently, Gururangan et al. (2018) found such artifacts in SNLI and its multi-genre extension, MNLI (Williams et al., 2018) . Tsuchiya (2018) observed similar findings in SNLI and SICK, a sentence similarity dataset, and Poliak et al. (2018c) reported similar results on six datasets, including those previous three. Some of the biased annotations that were found include the use of negation words ("not", "nothing") for cases of contradiction or approximate words ("some", "various") for entailment. Poliak et al. (2018c) discussed how faulty dataset-construction methods may be responsible for these biases.

Other NLU datasets also exhibit biases. In a story cloze completion setup, Schwartz et al. (2017b) obtained a high performance by only considering the candidate endings, without even looking at the story context. In the ROC stories dataset (Mostafazadeh et al., 2016) , stylistic features such as length or the use of certain words are predictive of the correct ending (Schwartz et al., 2017a; Cai et al., 2017) . A similar phenomenon was observed in reading comprehension, where systems performed non-trivially well by only using the final sentence in the passage, or by ignoring the passage altogether (Kaushik & Lipton, 2018) . Finally, multiple studies found non-trivial performance in visual question answering by using only the question, without any access to the image, due to biases in the question text (Zhang et al., 2016; Kafle & Kanan, 2016; Goyal et al., 2017; Kafle & Kanan, 2017; Agrawal et al., 2017) .

2.2. IMPROVING MODEL ROBUSTNESS:

Neural networks are notoriously sensitive to adversarial examples, primarily in the machine vision field (Szegedy et al., 2014; Goodfellow et al., 2015) , but also in NLP tasks like machine translation (Ebrahimi et al., 2018; Belinkov & Bisk, 2018; Heigold et al., 2018) and reading comprehension (Jia & Liang, 2017; Ribeiro et al., 2018; Mudrakarta et al., 2018) . A common approach to improving robustness is to train the model on data including adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015) . However, this method may not generalize well to new types of adversarial examples (Xiaoyong Yuan, 2017; Tramr et al., 2018; Belinkov & Bisk, 2018) . Domain-adversarial neural networks aim to increase robustness to domain change, by learning to be oblivious to the domain (Ganin et al., 2016) . This approach requires knowledge of the domain at training time, which makes transfer learning more difficult. Our method relies on a similar idea, but we learn to ignore latent annotation artifacts. Hence, we do not require direct supervision in the form of a domain label.

Others have attempted to remove biases from learned representations. Bolukbasi et al. (2016) successful removed gender biases from word embeddings. Li et al. (2018) removed sensitive information like sex and age from text representations, and obtained improved representations especially in out-of-domain scenarios. However, other work suggests that removing such attributes from text representations may be difficult (Elazar & Goldberg, 2018) . In contrast to this line of work, our final goal is not the removal of such attributes per se; instead, we strive for more robust representations that better transfer to other datasets, similar to Li et al. (2018) . Very recent work has focused on applying adversarial learning to NLI. Minervini & Riedel (2018) generate adversarial examples that do not conform to logical rules and then regularize models based on those examples. Similarly, Kang et al. (2018) incorporate external linguistic resources and use a GAN-style framework to adversarially train robust NLI models. Unlike these works, we do not use external resources and we are interested in removing specific biases that allow hypothesis-only models to perform well.

3. METHODOLOGY:

Let (P, H) denote a premise-hypothesis pair, and let f : S → v denote an encoder that maps a sentence S to a vector representation v, and g : v → y a classifier that maps a vector representation v to an output label y. Our baseline NLI architecture (Figure 1a ) contains the following components: Green or red arrows respectively mean that the gradient sign is kept as is or reversed. Gray arrow indicates that the gradient is blocked and not back-propagated.

• A premise encoder f P that maps the premise P to a vector representation p.

• A hypothesis encoder f H that maps the hypothesis H to a vector representation h.

• A classifier g NLI that maps a premise-hypothesis vector representation to an output y.

In this model, the premise and hypothesis are each encoded with separate encoders, f P and f H , into p and h respectively.1 Then their representations are combined into [p; h] 2 and fed to a classifier g NLI , which predicts an output y. If f P is not used, a model should no longer be able to successfully perform NLI. However, models without f P achieve non-trivial results, indicating the existence of biases in datasets' hypotheses (Gururangan et al., 2018; Poliak et al., 2018c; Tsuchiya, 2018) .

To overcome such biases that may limit the ability to transfer models across NLI datasets, we design two kinds of models for robust NLI, a single-classifier model and a double-classifier model. Both models aim to encourage the hypothesis representations to be free of biases via adversarial learning.

3.1. DOUBLE-CLASSIFIER MODEL:

The double-classifier model, illustrated in Figure 1b , is similar to our baseline model but includes an adversarial classifier g Hypoth that maps the hypothesis representation h to an output y. The crucial aspect of this model is the interaction between the NLI classifier, g NLI , and the hypothesis classifier, g Hypoth . The NLI classifier is trained to minimize the following objective:

EQUATION

where L(ỹ, y) is the cross-entropy loss. In the forward step, we feed the premise and hypothesis to their respective encoders, f P and f H , and forward their joint representation [p; h] to the classifier. In the backward step, gradients are back-propagated from the classifier into the premise and hypothesis encoders, f P and f H , in the normal fashion.

In contrast, the hypothesis classifier g Hypoth is adversary with respect to the input. It is also trained to minimize the cross-entropy loss, but its forward/backward propagation is different. In the forward step, we feed the hypothesis into its encoder f H and forward its representation h to the classifier.

In the backward step, we first back-propagate gradients through the classifier, as before. However, before back-propagating to the encoder, we reverse the gradients (Ganin & Lempitsky, 2015) . This simple step aims to discourage the model from learning patterns that may be useful for classification when considering only the hypothesis. The adversary minimizes the following objective:

L Adv = L(g Hypoth (GRL λ (f H (H)), y) (2)

where GRL λ is a gradient reversal layer. To control the interplay between g NLI and g Hypoth we set two hyper-parameters: λ Loss controls the importance of the adversarial loss function, and λ Enc controls the weight of the adversarial update by multiplying the gradients after reversing them. This is implemented by the scaled gradient reversal layer, GRL λ . The final loss function is defined by:

L = L NLI + λ Enc L Adv (3)

Limitation and a cryptographic perspective The double-classifier model is conceptually simple.

However, it has a potential limitation because of the separate adversarial classifier. In theory, it is possible that the adversary and the hypothesis encoder will co-adapt during training, such that the hypothesis representation still contains biased information but the adversary cannot utilize it. Thus we may be fooled to think that the biases were removed, while in fact they are encoded in a way that is accessible to the normal classifier but not to the adversary. A similar situation arises in neural cryptography (Abadi & Andersen, 2016) , where an encryptor Alice and a decryptor Bob communicate while an adversary Eve tries to eavesdrop on their communication. Alice and Bob are analogous to the hypothesis encoder f H and the normal classifier g NLI , while Eve is analogous to the adversary g Hypoth . Secret communication is analogous to solving NLI without using biases.

In their asymmetric encryption experiments, Abadi & Andersen observed seemingly secret communication, which on closer look the adversary was able to eavesdrop on. Here, if the adversarial classifier does not perform well, we might be tricked into thinking that the encoded representation does not contain any biases, while in fact they are still hidden in the representation. Our next architecture aims to prevent such a situation from possibly happening by folding the NLI and adversarial classifiers into a single network.

3.2. SINGLE-CLASSIFIER MODEL:

The single-classifier model also includes the premise and hypothesis encoders, f P and f H . However, it only has one classifier, g, which acts as both a normal NLI classifier and an adversarial hypothesis classifier. Having only one classifier aims at reducing the risk that a separate adversary would give a false impression of success, learning to not use hidden biases in the hypothesis representations. Since g must also do well on NLI with normal training, it is less likely to ignore hidden biases.

To achieve this, we consider two modes of operation. In the normal mode, we get a premisehypothesis pair from the training data, feed them through their encoders, forward to the classifier, and back-propagate in the normal fashion. In the adversarial mode, we get a premise-hypothesis from the training data and replace the premise with a random one. In the forward step, we feed the new pair to the encoders and classifier as before, predicting the entailment decision corresponding to the original premise-hypothesis pair. In the backward step, we first back-propagate through the classifier as usual. Then, we block the premise encoder and only back-propagate to the hypothesis encoder. In this case, we reverse the gradients going into the hypothesis encoder, as in the doubleclassifier. This procedure is shown in Figure 1c . The adversarial loss function is defined as:

EQUATION

Here GRL 0 implements gradient blocking on the premise encoder by using the identity function in the forward step and a zero gradient during the backward step. At the same time, GRL λ reverses the gradient going into the hypothesis encoder and scales it by λ Enc , as before.

The single-classifier model has the advantage of a simpler architecture, avoiding the need to train two different classifiers, which may result in failed adversarial learning as described before. (Williams et al., 2018) dataset. These datasets are human elicited: they were created by having human generate a corresponding hypothesis for a given premise and NLI label. Since SNLI is known to contain significant annotation artifacts/biases (Gururangan et al., 2018; Poliak et al., 2018c) , we will demonstrate the robustness of our methods by training our adversarial models on SNLI, and evaluating on all other datasets. We also evaluate on SNLI-hard, a subset of the test set that is thought to contain fewer biases (Gururangan et al., 2018) .

L = (1 -z)L NLI + zL RandAdv (5)

The second category of NLI datasets we consider are human-judged datasets that used automatic methods to pair context and hypothesis sentences and then relied on humans to label the pairs: Scitail (Khot et al., 2018) , ADD-ONE-RTE (Pavlick & Callison-Burch, 2016) , Johns Hopkins Ordinal Commonsense Inference (JOCI) (Zhang et al., 2017) , Multiple Premise Entailment (MPE) (Lai et al., 2017) , and Sentences Involving Compositional Knowledge (SICK) (Marelli et al., 2014) . We also consider datasets that are automatically recast from existing NLU datasets into NLI. We use the three datasets recast by White et al. (2017) to evaluate a number of semantic phenomena: FrameNet+ (FN+) (Pavlick et al., 2015) , Definite Pronoun Resolution (DPR) (Rahman & Ng, 2012) , Semantic Proto-Roles (SPR) (Reisinger et al., 2015) .

As many of these target datasets have different label spaces than SNLI, we define a mapping from our models' predictions to each target dataset's labels. These mappings are available in Appendix A.1.

Implementation Details:

We adopt InferSent's method (Conneau et al., 2017) for learning sentence representations from NLI data as our basic NLI architecture. In InferSent, each sentence is encoded by a bidirectional long short-term memory (LSTM) network whose hidden states are max-pooled. The premise and hypothesis representations are combined via a method introduced by Mou et al. (2016) and passed to a one hidden layer neural network. 3 We chose this model because it works well and its architecture is representative of many NLI models. However, our methodology can be applied to other models as well. We follow the InferSent training regime, using SGD with an initial learning rate of 0.1. See Appendix B.1 for hyper-parameter settings and more details.

5. RESULTS:

Table 1 reports the results of our proposed architectures compared to the non-adversarial baseline model. In each case, we tune the hyper-parameters on each target dataset's development set and evaluate on the corresponding test set. 4 The double-classifier model outperforms the baseline in 9 of the 12 target datasets (∆ > 0), though most of the improvements are small. The single-classifier only outperforms the baseline in 5 datasets, 4 of which are cases where the double-classifier also outperformed the baseline. These gains are much larger than the gains of the double-classifier. The fact that the two architectures agree to a large extent on which datasets benefit from adversarial training is a validation of our basic approach. As our results improve on the target datasets, we note that the double classifier models' performance on SNLI does not drastically decrease, even when the improvement on the target dataset is large (for example, the SPR case). For these models, the performance on SNLI drops by just an average of 1.11 (0.65 STDV). For the single-classifier, there is a large decrease on SNLI for many of the adversarial models as the models drop by an average of 11.19 (12.71 STDV). For these models, when we see large improvement on a target dataset, we often see a large drop on SNLI. For example, on ADD-ONE-RTE, the single classifier outperforms the baseline by roughly 17% but performs almost 50% lower than the baseline on SNLI.5 

A priori, we expect the adversarial models to benefit most when there are either no biases or biases that differ from ones in the training data. Indeed, both our adversarial architectures obtain improved results on the GLUE diagnostic test set, which was designed to be bias-free. We do not see improvements on the SNLI hard subset (Gururangan et al., 2018) , indicating that it may still have biases not identified by the authors, a possibility they also acknowledge.

To estimate the amount of bias that differ in the datasets, we compare the hypothesis-only results from Poliak et al. (2018c) with a hypothesis-only model trained on SNLI and tested on the target datasets. Since the results drop significantly below the majority class baseline (MAJ) on all but one dataset (Figure 4 , Appendix C.2), we believe that these target NLI datasets contain different biases than those in SNLI. The largest difference is on SPR where the hypothesis-only model trained on SNLI performs over 50% worse than when trained on SPR. On MNLI, this hypothesis-only model performs 10% above MAJ, compared to the roughly 20% when trained on MNLI, suggesting that MNLI contains similar biases as SNLI. This may explain why our adversarial models only slightly outperform our baseline on MNLI. The hypothesis-only model of Poliak et al. (2018c) did not outperform MAJ on DPR, ADD-ONE-RTE, SICK, and MPE. We observe improvements with adversarially trained models that are tested on all these datasets, to varying degrees (from 0.45 on MPE to 31.11 on SICK). On the other hand, we also see improvements on datasets with biases (high performance of hypothesis-only model), most noticeably on SPR. The only exception seems to be SCITAIL, where we do not improve although it has different biases than SNLI. However, when we strengthen the adversary (in the analysis below), the double-classifier outperforms the baseline. Our results demonstrates that our approach is robust to many datasets with different types of biases.

Fine-tuning on target datasets Our main goal is to determine whether adversarial learning can allow a model to perform well across multiple datasets by ignoring dataset-specific artifacts. In turn, we did not update the models' parameters on other datasets. However, what if we are given different amounts of training data for a new NLI dataset? Is our adversarial approach still helpful? To answer these questions, we updated four existing models, on increasing sizes of training data from different target datasets (namely, MNLI and SICK). The four models are (1) our baseline model trained on SNLI, (2) an adversarial double classifier trained on SNLI, (3) an adversarial single classifier trained on SNLI, and (4) our baseline model trained on the target dataset (MNLI/SICK). Both MNLI and SICK have the same label spaces as SNLI, allowing us to hold that variable constant. We use SICK because our adversarial models achieved good gains on it (Table 1 ). We also use MNLI, even though we saw small gains there, because MNLI's training set is large, allowing us to consider a wide range of different training set sizes.6 

Figure 2 shows the results on the dev sets. In MNLI, there is little to no gain from adversarial pre-training compared to non-adversarial pre-training. This is expected, as we saw relatively small gains with the adversarial models, and can be explained by SNLI and MNLI having similar biases.

In SICK, adversarial pre-training is better in most data regimes. The single-classifier is especially helpful, as it is the first to beat the model without pre-training (after using 25% of the training data).7 

6. ANALYSIS:

Stronger adversary Does a stronger adversary improve the quality of the transferred representations? Intuitively, we expect more adversarial training to hurt performance on the original data, where biases are useful, but improve on target data, which may have different or no biases. To test this, we trained adversarial models with larger hyper-parameter values on the adversary (see details in Appendix B.1). While there are other ways to make adversaries stronger, such as increasing the number or size of hidden layers (Elazar & Goldberg, 2018) , we are especially interested in the effect of these hyper-parameters as they control the trade-off between normal and adversarial training.

Table 2 shows the results of double-classifier models with a stronger adversary. As expected, performance on SNLI test sets decreases more, but many of the other datasets benefit from a stronger adversary (compared with Table 1 ). As for the single-classifier model, we found large drops in quality even in our basic configurations (Appendix C.3), so we do not increase its strength further.

Hidden biases in the representation Recall our motivation for the single-classifier model: we were concerned that the learned hypothesis representations may seem to be bias-free (low adversarial performance), while in fact there are still biases hidden in them. In the cryptographic analogy, it might appear that Alice and Bob communicate secretly (akin to solving NLI without biases), while in fact their communication can be decrypted (has biases). Indeed, Abadi & Andersen (2016) found that, "upon resetting and retraining Eve, the retrained adversary was able to decrypt messages nearly as well as Bob was". We perform an analogous experiment here: given a trained adversarial model, we freeze the hypothesis encoder f H and retrain a new, hypothesis-only classifier. We evaluate its quality to determine whether the (frozen) hypothesis representations have hidden biases.

Figure 3 shows the results on SNLI's dev set. A few trends can be noticed. First, we confirm that, in the double-classifier case (Figure 3a ), the adversary is indeed trained to perform poorly on the task (orange line), while the normal NLI classifier (blue line) performs much better. However, as suspected, retraining a classifier on frozen hypothesis representations (green line) leads to improved performance. In fact, the retrained classifier performs close to the fully trained hypothesis-only baseline from (Poliak et al., 2018c) , indicating that the hypothesis representation still contains biases.

Interestingly, we found that even a frozen random encoder captures biases in the hypothesis, as a classifier trained on it performs fairly well (63.26%), and far above the majority baseline (34.28%). One reason might be that just the word embeddings (which are pre-trained) contain significant information that propagates even through a random encoder. Others have also found that random encodings contain non-trivial information (Conneau et al., 2018; Zhang & Bowman, 2018) . Relatedly, Elazar & Goldberg found that adversarially trained text classification models still have demographic attributes in their hidden representations despite efforts to remove them.

Indicator words Certain words in SNLI are more correlated with specific entailment labels than others, especially with negation words ("not", "nobody", "no") correlated with contradiction (Gururangan et al., 2018; Poliak et al., 2018c) . Here we investigate whether our adversarial models make predictions that are less impacted by such biases. For each of the most biased contradiction words in SNLI, we computed the probability that an adversarial model (or the baseline model) predicts an example as contradiction, given that the hypothesis has the word. Table 3 shows the top 5 examples in the training set (Appendix C.4 has more examples). The single-classifier with λ Rand = 0.4, λ Enc = 1 predicts contradiction much less frequently than the baseline on examples with these words. This configuration was the strongest adversarial model that still performed reasonably well on SNLI (Appendix C.3). With these hyper-parameters, the single-classifier appears to remove some of the biases learned by the baseline. We also provide two other adversarial configurations that do not show such an effect, to illustrate that this behavior highly depends on the adversarial hyper-parameters.

7. CONCLUSION AND FUTURE WORK:

Biases in annotations are a major source of concern for the quality of NLI datasets and systems. In this paper, we presented a solution for combating annotation biases based on adversarial learning. We designed two architectures that discourage the hypothesis encoder from learning the biases, and instead obtain a more unbiased representation. We empirically evaluated our approach in a transfer learning scenario, where we found our models to perform better than a non-adversarial baseline on a range of datasets. We also investigated what biases remain in the latent representations.

The methodology developed in this work can be extended to deal with biases in other NLU tasks, where one is concerned with finding the relationship between two objects. For example, in Reading Comprehension, a question is being asked about a passage; in story cloze completion, an ending is judged with respect to a context; and in Visual Question Answering, a question is asked about an image. In all these cases, the second element (question, ending, and question, respectively) may contain biases. Our adversarial architectures naturally apply to any model that relies on encoding this biased element, and may help remove such biases from the latent representation. We hope to encourage such investigation in the broader research community. 

C ADDITIONAL RESULTS:



C.1 PERFORMANCE ON SNLI WITH ADVERSARIAL TRAINING:

Table 4 shows the results of transferring representations to new datsets. These results complement Table 1 . In each dataset, we tune the adversarial hyper-parameters on the dev set and report test set accuracies. We also give the performance on the SNLI test set. Figure 4 shows the results of several baselines in multiple datasets: the majority baseline and the hypothesis-only baselines trained on either SNLI or each dataset's training set, and evaluated on each target dataset's test set. Notice that the hypothesis-only model trained on SNLI drops in performance when evaluated on other models. When the hypothesis-only model trained on SNLI is tested on the target datasets, the model performs below the majority baseline (except for MNLI), indicating that the biases in SNLI's hypotheses do not occur in the other datasets' hypotheses. This is not surprising since these datasets contain different types of biases owing to noisy generation methods (FN+) (Poliak et al., 2018a) , social and cognitive biases (SNLI & MNLI) (Poliak et al., 2018c) , or even "non-uniform distributions from original dataset that have been recast" (SPR) (Poliak et al., 2018b) . 

C.3 HYPER-PARAMETER SWEEPS:

Here we provide cross-validation results with different settings of our hyper-parameters. Figure 5a shows the dev set results with different configurations of the single classifier. Notice that performance degrades quickly when we increase the fraction of random premises (large λ Rand ). In contrast, the results with the double classifier (Figure 5b ) are more stable.

:

This is common to many NLI models(Rocktäschel et al.,

2016;Mou et al., 2016;Cheng et al., 2016;Nie & Bansal, 2017;Chen et al., 2017), although some share information between the encoders via attention(Parikh et al., 2016;Duan et al., 2018). Extending our approach to this scenario would require some caution.2 For simplicity, we assume here that the two are concatenated. In the experiments we consider a more sophisticated method to represent sentence pairs.

Specifically, representations are concatenated, subtracted, and multiplied element-wise.

For MNLI, since the test sets are not available, we tune on the matched dev set and evaluate on the mismatched dev set, or vice versa. For GLUE, we tune on MNLI matched.

5 See Table4in Appendix C for the corresponding results on SNLI for each of the results listed in Table

1.6 We discuss necessary implementation details for these experiments in Appendix

B.2 7 Note that SICK is a small dataset (10K training examples), which explains why the model without pretraining does not benefit from more data, as it just achieves the majority class performance.

Figures and Tables:

Figure 1: Illustration of (a) the baseline NLI architecture, and two proposed adversarial architectures: (b) a double-classifier adds an adversarial hypothesis-only classifier, (c) a single-classifier is trained adversarially with a random premise, and otherwise in the normal manner. Upward and downward arrows correspond to forward and backward propagation. Green or red arrows respectively mean that the gradient sign is kept as is or reversed. Gray arrow indicates that the gradient is blocked and not back-propagated.

Figure 2: Effect of pre-training with adversarially trained models.

Figure 3: Results when retraining a classifier on a frozen hypothesis encoder compared to the adversarial NLI training.

Figure 4: Majority (Maj) and hypothesis-only baselines, trained on each dataset (Hyp Self) or on SNLI (Hyp SNLI) baselines. The differences determine whether each dataset has different types of biases in the hypothesis than the biases in SNLI's hypotheses.

Accuracies of the non-adversarial baseline and double/single classifiers models, trained on SNLI and evaluated on target datasets. ∆ = differences between baseline and adversarial models.Data To determine how well our proposed architectures enable a model to perform well on NLI datasets despite the high presence of the annotation artifacts in the training corpus, we use a total of 11 NLI datasets -the 10 datasets thatPoliak et al. (2018c) investigated in their hypothesis-only study plus GLUE's diagnostic test set that was carefully constructed to not contain hypothesisbiases(Wang et al., 2018). The most popular recent NLI datasets are arguably the Stanford Natural Language Inference (SNLI)(Bowman et al., 2015) dataset and its successor, the Multi-genre Natural Language Inference (MNLI)

Table content: <table><tr><td>Target Dataset</td><td colspan="2">Baseline Double</td><td>∆</td><td>Single</td><td>∆</td></tr><tr><td>SCITAIL</td><td>58.14</td><td colspan="2">57.67 -0.47</td><td>51.08</td><td>-7.06</td></tr><tr><td>ADD-ONE-RTE</td><td>66.15</td><td>66.15</td><td>0</td><td>83.46</td><td>17.31</td></tr><tr><td>JOCI</td><td>41.5</td><td>41.74</td><td>0.24</td><td>39.63</td><td>-1.87</td></tr><tr><td>MPE</td><td>57.65</td><td>58.1</td><td>0.45</td><td>52.35</td><td>-5.3</td></tr><tr><td>DPR</td><td>49.86</td><td>50.96</td><td>1.1</td><td>49.41</td><td>-0.45</td></tr><tr><td>MNLI matched</td><td>45.86</td><td>47.24</td><td>1.38</td><td>43.76</td><td>-2.1</td></tr><tr><td>FN+</td><td>50.87</td><td>52.48</td><td>1.61</td><td>57.03</td><td>6.16</td></tr><tr><td>MNLI mismatched</td><td>47.57</td><td>49.24</td><td>1.67</td><td>43.66</td><td>-3.91</td></tr><tr><td>SICK</td><td>25.64</td><td>27.44</td><td>1.8</td><td>56.75</td><td>31.11</td></tr><tr><td>GLUE</td><td>38.5</td><td>40.49</td><td>1.99</td><td>43.21</td><td>4.71</td></tr><tr><td>SPR</td><td>52.48</td><td>58.99</td><td>6.51</td><td>65.43</td><td>12.94</td></tr><tr><td>SNLI hard</td><td>68.02</td><td colspan="2">66.27 -1.75</td><td colspan="2">55.6 -12.42</td></tr><tr><td>4 EXPERIMENTAL SETUP</td><td/><td/><td/><td/><td/></tr></table>

Results with a stronger adversary in the double-classifier vs. a non-adversarial baseline.

Table content: <table><tr><td>Dataset</td><td colspan="2">Baseline Double</td><td>∆</td><td>Dataset</td><td colspan="2">Baseline Double</td><td>∆</td></tr><tr><td>SNLI hard</td><td>68.02</td><td colspan="3">63.81 -4.21 SCITAIL</td><td>58.14</td><td>60.82</td><td>2.68</td></tr><tr><td>JOCI</td><td>41.5</td><td colspan="3">39.29 -2.21 ADD-ONE</td><td>66.15</td><td>68.99</td><td>2.84</td></tr><tr><td>SNLI</td><td>84.22</td><td colspan="3">82.4 -1.82 GLUE</td><td>38.5</td><td>41.58</td><td>3.08</td></tr><tr><td>DPR</td><td>49.86</td><td colspan="3">49.41 -0.45 FN+</td><td>50.87</td><td>56.31</td><td>5.44</td></tr><tr><td>MNLI matched</td><td>45.86</td><td>46.12</td><td colspan="2">0.26 SPR</td><td>52.48</td><td>58.68</td><td>6.2</td></tr><tr><td>MNLI mismatched</td><td>47.57</td><td>48.19</td><td colspan="2">0.62 SICK</td><td>25.64</td><td colspan="2">36.59 10.95</td></tr><tr><td>MPE</td><td>57.65</td><td>58.6</td><td>0.95</td><td/><td/><td/></tr></table>

Indicator words and the % decrease in the correlation of double (D) and single (S) models with the "contradiction" label compared to the non-adversarial baseline.

Table content: <table><tr><td>Word</td><td colspan="3">D(1,1) S(0.4,1) S(1,1) Word</td><td colspan="3">D(1,1) S(0.4,1) S(1,1)</td></tr><tr><td>sleeping</td><td>15.63</td><td colspan="2">53.13 -81.25 driving</td><td>-8.33</td><td colspan="2">50 -66.67</td></tr><tr><td>Nobody</td><td>14.29</td><td>42.86</td><td>14.29 alone</td><td>0</td><td>83.33</td><td>0</td></tr><tr><td>cat</td><td>7.14</td><td colspan="2">57.14 -85.71 asleep</td><td>-18.75</td><td>50</td><td>12.5</td></tr><tr><td>no</td><td>0</td><td colspan="2">52.94 -52.94 empty</td><td>-16.67</td><td colspan="2">83.33 -16.67</td></tr><tr><td>eats</td><td>37.5</td><td>87.5</td><td>-25 naked</td><td>0</td><td colspan="2">83.33 -33.33</td></tr></table>

Each premise-hypothesis pair in SNLI is labeled as either ENTAILMENT, NEUTRAL, or CONTRA-DICTION. MNLI, SICK, JOCI, and MPE use the same label space. Since examples in SciTail are labeled as ENTAILMENT or NEUTRAL, when evaluating on SciTail, we convert the model's CON-TRADICTION to NEUTRAL. ADD-ONE-RTE and the recast datasets also model NLI as a binary prediction task. However, their label sets are ENTAILED and NOT-ENTAILED. In these cases, when the models predict ENTAILMENT, we convert the example to ENTAILED, and when the models predict NEUTRAL or CONTRADICTION, we map the label to NOT-ENTAILED. Loss , λ Enc , λ Rand by sweeping in the range {0.05, 0.1, 0.2, 0.4, 0.8, 1.0}. For the stronger adversary experiments (Table2) we consider the range {1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0}. In each dataset, we choose the best-performing model on the development set and report its quality on the test set.We follow the InferSent training regime, using SGD with an initial learning rate of 0.1. SeeConneau et al. (2017) for details.B.2 UPDATING TRAININGOne drawback of using MNLI is that gold labels for the test set are not publicly available. We remove 10K examples from the training set and treat them as our development set. Here, we use the MNLI matched development set as our test set since we assume that the new dataset, that we would like our robust NLI models to generalize well on, contains consistent domains and genres.

Table content: <table><tr><td>A DATA DETAILS</td></tr><tr><td>A.1 MAPPING LABELS</td></tr><tr><td>B TRAINING DETAILS</td></tr><tr><td>B.1 ADVERSARIAL TRAINING DETAILS</td></tr><tr><td>We tune the adversarial hyper-parameters λ</td></tr></table>

Results of transferring representations to new datasets. Scores are accuracies on the corresponding dataset. Left block: test results on target datasets. Right block: test results on SNLI with the models that performed best on each target dataset. ∆ are differences from the non-adversarial baseline. In all cases the models are trained on SNLI.

Table content: <table><tr><td/><td/><td colspan="2">On Target Dataset</td><td/><td/><td colspan="2">On SNLI</td><td/></tr><tr><td>Target Dataset</td><td>Double</td><td>∆</td><td>Single</td><td>∆</td><td>Double</td><td>∆</td><td>Single</td><td>∆</td></tr><tr><td>SCITAIL</td><td colspan="2">57.67 -0.47</td><td>51.08</td><td>-7.06</td><td colspan="2">84.04 -0.18</td><td>75.16</td><td>-9.06</td></tr><tr><td>ADD-ONE-RTE</td><td>66.15</td><td>0</td><td>83.46</td><td>17.31</td><td colspan="2">81.93 -2.29</td><td colspan="2">34.59 -49.63</td></tr><tr><td>JOCI</td><td>41.74</td><td>0.24</td><td>39.63</td><td>-1.87</td><td colspan="2">83.78 -0.44</td><td>78.3</td><td>-5.92</td></tr><tr><td>MPE</td><td>58.1</td><td>0.45</td><td>52.35</td><td>-5.3</td><td colspan="2">83.65 -0.57</td><td>83.68</td><td>-0.54</td></tr><tr><td>DPR</td><td>50.96</td><td>1.1</td><td>49.41</td><td>-0.45</td><td colspan="2">83.49 -0.73</td><td>76.41</td><td>-7.81</td></tr><tr><td>MNLI matched</td><td>47.24</td><td>1.38</td><td>43.76</td><td>-2.1</td><td colspan="2">82.97 -1.25</td><td>75.29</td><td>-8.93</td></tr><tr><td>FN+</td><td>52.48</td><td>1.61</td><td>57.03</td><td>6.16</td><td colspan="2">82.28 -1.94</td><td>83.78</td><td>-0.44</td></tr><tr><td>MNLI mismatched</td><td>49.24</td><td>1.67</td><td>43.66</td><td>-3.91</td><td colspan="2">82.97 -1.25</td><td>75.29</td><td>-8.93</td></tr><tr><td>SICK</td><td>27.44</td><td>1.8</td><td>56.75</td><td>31.11</td><td colspan="2">83.65 -0.57</td><td>75.29</td><td>-8.93</td></tr><tr><td>GLUE</td><td>40.49</td><td>1.99</td><td>43.21</td><td>4.71</td><td colspan="2">82.97 -1.25</td><td>75.29</td><td>-8.93</td></tr><tr><td>SPR</td><td>58.99</td><td>6.51</td><td>65.43</td><td>12.94</td><td colspan="2">82.46 -1.76</td><td colspan="2">70.21 -14.01</td></tr><tr><td>SNLI</td><td/><td/><td/><td/><td colspan="2">83.56 -0.66</td><td>78.3</td><td>-5.92</td></tr><tr><td>SNLI hard</td><td colspan="2">66.27 -1.75</td><td colspan="2">55.6 -12.42</td><td/><td/><td/><td/></tr></table>

Peer review (iclr2019):
 In this paper, the authors attempt to mitigate annotation artifacts in natural language inference data. The authors propose an adversarial setup to discourage the model from overfitting to examples that can be solved by only considering the hypothesis. They consider two variants of the adversarial setup: 1) an independent adversarial hypothesis-only classifier trained jointly with an NLI model (“double classifier”) and 2) a single classifier that is trained adversarially with randomized premises (“single classifier”). While the idea of this paper is appealing and well-motivated. The results are not super strong, but they are relatively consistent (at least for the double classifier method). I also liked the extensive analysis. My main concern is that I am not entirely convinced that the experimental results support the main claim made by the authors---that the proposed adversarial approach is a viable solution to mitigating annotation artifacts. If I understand correctly, the authors make two explicit assumptions: (a) that the biases in SNLI are somewhat responsible to its success. As a result, eliminating them also leads to performance degradation both in SNLI and on other datasets with similar biases (page 6: “This is expected, as we saw relatively small gains with the adversarial models, and can be explained by SNLI and MNLI having similar biases”). (b) that these biases are preventing the model from generalizing to other datasets that do not contain biases (or alternatively---contain other biases). As a result, eliminating bias is desired, as it will improve generalizability, at the expense of lower in-domain results, that are somewhat inflated to begin with. This is a nice and appealing story (though it could have been written more coherently). What I am concerned is whether the experimental gains observed actually result from mitigating biases. I can think of two alternative (though related) explanations for these gains: Were the pretrained SNLI baseline hyperparameters tuned on each corresponding dataset as well, or was the same model used in all cases? If the latter, this could partially explain the observed gains using their method: although the model has not seen any development example, selecting the best model based on the dev results gives it an unfair advantage over a model tuned on the original SNLI dataset. The adversarial models suggested by the authors can be seen as a type of regularization. Did the authors try more traditional regularization methods such as L1/L2/dropout on the original SNLI model? Using high regularization values could have a similar effect (reduced performance on SNLI, increased performance on other datasets). Assuming you tune the regularization values on the target dataset (as you do with your methods), this is expected to lead to similar phenomena. I suspect that most gains observed in Table 1 are small enough in most cases to be obtained by such methods. Other points: One experiment that I would like to see is whether biases are actually eliminated from SNLI. The authors could test the original SNLI model and the adversarial ones on a dataset containing the NULL string as premise (with the original hypothesis). Although the models are not trained on such data, based on the hypothesis-only results, I would expect the original model to do fairly well on it. If the authors’ claim is correct, we should see significantly lower performance by the adversarial ones. The results on testing the adversarial models on the SNLI test set (Table 4), as well as Figure 4 should be in the main text rather than the appendix. Minor: — Page 6: “The fact that the two architectures agree to a large extent on which datasets benefit from adversarial training is a validation of our basic approach.”: this claim is unconvincing: the double classifier improved on 9/12 cases (75%). Selecting 5 of them at random, it is likely that 4 of them will show improvement with double. -- Typos and such: - Last paragraph on page 1: "...biases.In this way..." (missing white space) - Scitail is inconsistently spelled throughout the paper