A Generative Approach for Mitigating Structural Biases in Natural Language Inference (2022)

Abstract: Many natural language inference (NLI) datasets contain biases that allow models to perform well by only using a biased subset of the input, without considering the remainder features. For instance, models are able to classify samples by only using the hypothesis, without learning the true relationship between it and the premise. These structural biases lead discriminative models to learn unintended superficial features and generalize poorly out of the training distribution. In this work, we reformulate the NLI task as a generative task, where a model is conditioned on the biased subset of the input and the label and generates the remaining subset of the input. We show that by imposing a uniform prior, we obtain a provably unbiased model. Through synthetic experiments, we find that this approach is highly robust to large amounts of bias. We then demonstrate empirically on two types of natural bias that this approach leads to fully unbiased models in practice. However, we find that generative models are difficult to train and generally perform worse than discriminative baselines. We highlight the difficulty of the generative modeling task in the context of NLI as a cause for this worse performance. Finally, by fine-tuning the generative model with a discriminative objective, we reduce the performance gap between the generative model and the discriminative baseline, while allowing for a small amount of bias.

## introduction natural language processing (nlp) datasets are plagued with artifacts and biases, which allow models to perform tasks without learning the desired underlying language capabilities. for instance, in natural language inference (nli) datasets, models can predict an entailment relationship y from the hypothesis text h alone, without considering the premise p at all @xcite . another identified source of bias is lexical overlap between p and h, which is associated with an entailment prediction @xcite . we refer to such biases as structural biases, cases where an undesired subset of the input alone incorrectly identifies the label. relying on such biases results in poor out-of-distribution (o.o.d) generalization when models are applied to data without bias. furthermore, models that contain such biases may make surprising predictions when the bias is present, causing problems in critical systems. a line of work has attempted to improve the performance on o.o.d datasets by proposing different objective functions (e.g., @xcite . however, these methods typically still result in a significant gap between the performance in and out of distribution, which indicates that the models are still biased. in this work, we reformulate classification as a generative task, where the model's task is to generate the remainder features r conditioned on the biased features b and the label y. using bayes' rule, we decompose the posterior p(y | b, r) into the likelihood p(r | y, b) and the prior p(y | b). this reformulation lets us control the amount of bias present in the final model. by setting a uniform prior we can obtain a provably unbiased model. we denote this generative model as gen.. to assess the extent to which a given model is biased w.r.t a specific structural bias, we consider two metrics: the o.o.d generalization gap and the correlation between a model and a biased model p(y | b), such as a hypothesis-only or overlap-only model. we first experiment with injecting synthetic bias into a fraction of the training set and evaluating on test sets with and without that bias. we find that the discriminative model's performance decreases as the amount of bias increases, while gen maintains similar performance at all bias levels. moreover, the biased-ness of the discriminative model increases, while gen remains unbiased. next, we experiment with two kinds of natural bias: hypothesis-only and overlap. we demonstrate that gen is unbiased compared to the discriminative baseline as measured by its low ∆ and low absolute correlation with a biased model (ρ). however, while our approach leads to unbiased models, it performs worse than the discriminative baseline even on o.o.d data. we then identify and quantify several causes for the poor performance of gen. we show that generative modeling is a more challenging task than discriminative modeling, and that it requires learning a large amount of spurious signal compared to the discriminative model. finally, to mitigate the difficulty of the generative modeling task, we fine-tune gen with a discriminative objective @xcite . while this leaks some bias into the model, the final model (denoted as gen-ft) matches or surpasses the discriminative baseline while maintaining a relatively small o.o.d generalization gap. to conclude, our contributions are as follows: ## structural bias consider the general case of a classification task, for which we wish to build a model p θ (y|x) where y is a low-dimensional label and x is an arbitrarily large set of features. the model is trained on an empirical training set @xmath0, y i )} n @xmath1. the dataset is constructed by humans, and inadvertently contains structural biases. we define a structural bias as a case where, if the input x is split into two disjoint sets @xmath2, @xmath3, the label y can be learned to be reliably predicted given only b. for most choices of b this is not a problem, but in some cases, the subset represents an externally imposed constraint that needs to be maintained or an externally imposed understanding of how the model should operate. this formulation comprises a broad set of commonly considered biases. for example, in the nli task, @xmath4, h) where p and h are the premise and hypothesis. if we choose the split @xmath5, we arrive at the hypothesis-only bias. this is an undesirable bias because as humans we know that nli is impossible if one is only given the hypothesis. taking different splits corresponds to different biases. for instance, we can model the lexical overlap bias under the structural bias framework with the subset @xmath6. nli models should perform no better than chance when given only the overlapping tokens between p and h. finally, this formulation extends beyond nli and nlp to broader biases. for example, if x is a vector of information about individuals and one of the features in x is a protected characteristic s (e.g., gender or race), @xmath7. foot_0 then, depending on the task, an undesirable structural bias may exist if a model can learn to predict y given s. we denote these biases as structural biases because they are defined through the structure b ⊂ x, rather than specific known patterns in the data. for example, in the hypothesis-only case, this formulation does not require knowledge about what aspects of the hypothesis allow a hypothesis-only model to predict the label (e.g., negation words), only that somehow the hypothesis alone incorrectly gives a signal about the label. thus, this type of bias is broader than specific known biases such as the presence of negation words, but narrower than unknown biases because it requires some knowledge of where the bias might be found. ## experiments in all experiments, we estimate p(r | y, b) with an encoder-decoder model, with inputs (y, b) and output r. to condition on y, we prefix a labelspecific token to b. we then train the model as a conditional generative model, by fine-tuning bert @xcite or bart @xcite with the standard auto-regressive cross-entropy loss. to use bert as an autoregressive decoder, the bidirectional self-attention mechanism of bert is masked, and a language modeling layer, which starts generating from the "cls" token, is added. a generative bert model is comprised of a regular bert model as an encoder and a bert decoder @xcite . all models are taken from the transformers library @xcite , and are fine-tuned with either the baseline discriminative objective or our proposed generative formulation. at test time, we attach all possible label tokens to each b and pick ŷ = arg max @xmath8, b). ## evaluating generated premises so far, we have only used gen to score existing examples (with teacher forcing), conditioned on the label and the biased features. in this section, we evaluate the quality of its generations when decoding without constraints. for the experiments here, we consider the hypothesis-only bias and evaluate the quality of gen in generating premises. we use a bart model trained on snli and generate premises for all hypotheses in the test set. to evaluate how well our model can generate premises, we used two metrics: bleu @xcite of the generated premises w.r.t gold premises, to measure the generation quality (higher is better), and self-bleu @xcite to measure the diversity of the generations (lower is more diverse). we report a bleu value of 0.1078, indicating that the model is not very good at generating premises. we report self-bleu of 0.8032 for the generated premises compared to 0.5875 for the original premises, suggesting that the generated premises are less diverse. nli models. by applying gradient attributions, 7 we found that more than 70% of the premise words have low attributions values (between -0.1 to 0.1), with fewer than 6% of the words having absolute values greater than 0.3. this shows that only a small number of words had any significant effect on the model predictions. ## discriminative fine-tuning the analysis in section 6 suggests that the central limitation of gen is that the purely generative task for which it is trained is challenging in its own right, but unaligned with the downstream classification task. the model is rewarded at training time for devoting significant capacity to modeling the full high-dimensional distribution of r, even when large parts of that distribution are unimportant for making downstream predictions. to help gen in such cases, we experiment with an additional fine-tuning step in which we directly optimize for predictive performance. specifically, for the fine-tuning step we construct the discriminative distribution using bayes' rule in equation 1 and use it at training time by minimizing the label cross-entropy loss: using this objective requires a choice of p θ (y | b). we explore the impact of different choices for this distribution in appendix a.2, but found that using a pretrained and frozen p θ (y | b) during the fine-tuning step works best. we hypothesize that this setup allows the generative compo- the adjusted training procedure is composed of the following steps: 1) train a discriminative prior model, p θ (y | b), freeze the weights. 2) train a generative model, p θ (r | y, b), as in section 4. 3) fine-tune the model using equation 4, using the pretrained p θ (y | b). 4) test the model using equation 1 with a uniform prior. ## scalability given that the generative approach consumes more compute than the discriminative baseline, it is natural to ask whether it can scale to larger models. to answer this, we experimented with the t5 model @xcite , an encoder-decoder available in five sizes, from 60m to 11b parameters. we focus on the hypothesis-only bias case in snli. we train the generative and discriminative models using regular fine-tuning (also called model-tuning), and also experiment with prompt-tuning @xcite , a faster and cheaper approach, which adds a small number of learnable tokens to the start of the input, and trains them end-to-end, while the model's original weights stay frozen. (memory and training statistics are found in figure 3 shows that both the generative and discriminative approaches scale with model size. prompt-tuning is effective, matching model-tuning performance at larger sizes. in larger models, the generative approach narrows the gap from the discriminative one, but cannot close it. ## conclusion structural biases are common in various nli datasets and are a major obstacle when trying to create robust systems for this task. we proposed a generative approach for nli, which leads to unbiased models. we demonstrated that our generative models are robust to large amounts of bias and perform equally well in and out of distribution. this comes, however, with a trade-off, where the generative models perform worse than discriminative baselines. we investigated reasons for the difficulty of training generative nli models, highlighting the large output space of generating sentences, as opposed to identifying a small subset of words that are often sufficient for solving the task. we showed how to mitigate this problem by fine-tuning gen with a discriminative objective. finally, we demonstrated that the method scales efficiently to large language models. our work lays down a novel formulation for the nli task, which may be applied to many other natural language understanding tasks. future work can examine other kinds of bias and different tasks. for instance, if the bias variables are constructed according to protected attributes like race or gender, our approach leads to unbiased models w.r.t the protected attributes.