Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference (2022)

Abstract: It has been shown that NLI models are usually biased with respect to the word-overlap between the premise and the hypothesis, as they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in the NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI systems are also highly biased towards the non-entailment label on instances with low overlap and that existing debiasing methods, which are reportedly successful on challenge datasets, are generally ineffective in addressing this category of bias.Through a set of analyses, we investigate the reasons for the emergence of the overlap bias and the role of minority examples in mitigating this bias.For the former, we find that the word overlap bias does not stem from pre-training, and in the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias.

## introduction natural language inference (nli) is one of the most commonly used nlp tasks, particularly in the scope of evaluating models for their language understanding capabilities. since their emergence, pre-trained language models (plms) have been highly successful on standard nli datasets, such as the multi-genre natural language inference @xcite . however, recent analytical studies have revealed that their success is partly due to their reliance on spurious correlations between superficial features of the input texts and gold labels in these datasets @xcite . as a result, performance usually drops on out-of-distribution datasets where such correlations do not hold. several proposals figure 1 : nli model's confidence on a randomly sampled subset of instances from the snli dataset across four different degrees of word overlap between premise and hypothesis. bert is biased towards the entailment label on instances with full overlap (denoted by the huge confidence gap with the non-entailment label). on the contrary, a reverse bias is seen for low and nonoverlapping instances, with a significant confidence lead on the non-entailment label. one of the well-known dataset biases in nli models is the spurious correlation of the entailment label and high word-overlap between premise and hypothesis. a number of challenging sets are designed to showcase the tendency of plms to predict entailment for most such cases. hans @xcite is arguably the most widely used dataset in this group. constructed based on human-made linguistic patterns, the dataset focuses on high-overlapping samples, the non-entailment subset of which is deemed as challenging for nli models. most current debiasing methods have considered the word-overlap bias as one of their main targets and have shown substantial improvements on hans @xcite . in this paper, we revisit the word-overlap bias in nli and the effectiveness of existing debiasing techniques. despite the popularity of this type of bias, we find that some of its aspects are generally ignored in the research community. if we consider word-overlap as a feature with values ranging from no to full overlap, and nli task with two labels of entailment and non-entailment, we show that there are other kinds of spurious correlation than the popular high word-overlap and entailment. specifically, as it is shown in figure 1 , we see a clear bias towards non-entailment for the low and no wordoverlap values (denoted by the high performance on the non-entailment label, which comes at the price of reduced performance on the entailment class). we will refer to this type of bias as reverse word-overlap throughout the paper. through a set of experiments, we demonstrate that the overlooked reverse word-overlap bias exists in popular nli datasets, such as mnli and snli, as well as in the predictions of plms. moreover, our results suggest that while existing debiasing methods can mitigate the overlap bias in nli models to some extent, they are ineffective in resolving the reverse bias. moreover, we analyze how nli models employ minority instances to enhance their generalization. focusing on the forgettable debiasing method @xcite , we realize that eliminating hans-like examples and the reverse ones do not hurt the generalization noticeably. in search of the origin of the bias, we employ prompt-based techniques to check whether the bias stems from pre-training. we also verify the robustness of plms in a few-shot learning experiment with controlled and balanced training sets. our results suggest that plms do not exhibit any bias towards a specific label. nevertheless, introducing a few samples triggers the bias toward the entailment label. furthermore, balancing the training examples with respect to their word-overlap prevents the emergence of bias to some extent. our contributions can be summarized as follows: ## natural language inference in nli, a model is provided with two input sentences, namely premise and hypothesis. the task for the model is to predict whether the hypothesis is true (entailment), false (contradiction), or undetermined (neutral) given the premise. ## reverse word-overlap considering the word-overlap bias as a spectrum, the existing studies have mainly focused on a small subset of the spectrum, i.e., the case with full wordoverlap and its spurious correlation with the entailment label. in this section, we evaluate the performance of nli models on other areas of the spectrum and with respect to both labels (entailment and non-entailment) to broaden our insights on the robustness of these models considering the word-overlap feature. ## conclusions in this work, we uncovered an unexplored aspect of the well-known word-overlap bias in the nli models. we showed a spurious correlation between the low overlap instances and the non-entailment label, namely the reverse word-overlap bias. we demonstrated that existing debiasing methods are not effective in mitigating the reverse bias. we found that the generalization power of debiasing methods (the forgettable approach in particular) does not stem from minority examples. we also showed that the word-overlap bias does not seem to come from the pre-training step of plms. as future work, we plan to focus on designing new debiasing methods for mitigating the reverse bias for nli and similar tasks. also, building specific challenging sets, similar to hans, for the reverse bias helps to expand this line of research.