NLI Data Sanity Check: Assessing the Effect of Data Corruption on Model Performance (2021)

Abstract: Pre-trained neural language models give high performance on natural language inference (NLI) tasks. But whether they actually understand the meaning of the processed sequences is still unclear. We propose a new diagnostics test suite which allows to assess whether a dataset constitutes a good testbed for evaluating the models’ meaning understanding capabilities. We specifically apply controlled corruption transformations to widely used benchmarks (MNLI and ANLI), which involve removing entire word classes and often lead to non-sensical sentence pairs. If model accuracy on the corrupted data remains high, then the dataset is likely to contain statistical biases and artefacts that guide prediction. Inversely, a large decrease in model accuracy indicates that the original dataset provides a proper challenge to the models’ reasoning capabilities. Hence, our proposed controls can serve as a crash test for developing high quality data for NLI tasks.

## introduction assessing the natural language inference (nli) and understanding (nlu) capabilities of a model poses numerous challenges, one of which is the quality and composition of the data used for evaluation. popular nli datasets @xcite contain annotation artefacts and statistical irregularities that can be easily grasped by a model during training and guide prediction, even if the model has not acquired the knowledge needed to perform this kind of reasoning. notably, recent work shows that major modifications such as word shuffling do not hurt bert's @xcite nlu capabilities ## related work a well-known problem of nlu evaluation benchmarks is that the proposed tasks are often solvable by simple heuristics @xcite . this is mainly due to the presence of linguistic biases in the datasets, which make prediction easy @xcite . notably, 90% of the hypotheses that denote a contradiction in the original snli dataset @xcite contain the verb sleep and its variants (sleeping, asleep) which serve to mark a contrast with an activity described in the premise (e.g., my sister is @xmath0; while contradictions in sick @xcite are often marked by explicit negation. this latter issue also exists in snli and mnli as spotted by @xcite , where negation is highly indicative of contradiction, and generic nouns (e.g., animal, something) of entailment. these grammatical or lexical cues are easily grasped by the models during training and help them correctly predict the relationship between two sentences, but this does not mean that the models are capable of performing this type of reasoning. notably, due to these annotation artefacts and statistical ir-regularities, it is possible even for hypothesis-only nli models (i.e. models that are fine-tuned only on the hypotheses without access to the premises) to make correct predictions @xcite . recent work shows that state-of-the-art nlu models are not very sensitive to word order which, however, is one of the most important characteristics of a sequence @xcite . specifically, performance of bert-based classifiers fine-tuned on glue tasks @xcite remains relatively high after randomly shuffling input words. this is mainly explained by the contribution of each individual word which remains unchanged after its context is shuffled. superficial cues such as the sentiment of keywords in sentiment analysis, or the word level similarity between sentence pairs in nli, allow bert-based models to make correct decisions even when tokens are arranged in random orders, suggesting that many glue tasks are not really challenging them to understand the meaning of a sentence. to the contrary, when simple heuristics do not suffice to solve the nli task, nli systems seem to be more prone to breaking. this is for example what happens when swapping the test and training datasets of different benchmarks (i.e. training on one nli dataset and testing on an other) @xcite . @xcite report problems in performance when the premise and the hypothesis are swapped. the idea is that the label of contradicting or neutral pairs should remain the same in the case of a swap, in contrast to entailment pairs where a different label should be proposed after the swap. this would be expected because entailment is a directional relationship, while contradiction is symmetric. 2 @xcite test various models with respect to this diagnostic and observe a a significant drop in performance (i.e. predicted labels change) when the contradicting and neutral pairs are swapped. the models' behaviour seems more reasonable when these are tested on the swapped entailment pairs, where all but one models correctly predict a different label. in the light of these results, the authors propose the swapping method as a sanity check for nli models. the low quality of existing datasets and the impressively high performance of nli systems, as measured on these benchmarks, have sparked a new research direction where the goal is to propose new more challenging and artefact-free datasets. the anli dataset, for example, was built precisely with the goal to eliminate annotation artefacts @xcite . the authors claim that this dataset is much less prone to annotation artefacts compared to previous benchmarks, as suggested by the lower prediction accuracy for models fine-tuned on the anli hypothesis-only dataset. although there still seems to be space for improvement (accuracy is around 0.5, i.e. well above chance), the reported findings are promising. specifically, the performance is lower than on the hypothesis-only snli/mnli datasets, showing that the dataset contains less artefacts that can guide prediction. anli is thus a natural candidate to further test our hypotheses, as it claims to remedy for a number of the shortcomings of earlier nli datasets. lessons learnt from previous work on designing reliable linguistic probing tasks @xcite and the overfitting problems of nli models discussed above, demonstrate the importance of systematic sanity checks like the ones we propose in this paper. our dedicated control tasks specifically allow to determine whether a dataset triggers the models' reasoning capabilities or, instead, allows them to rely on statistical biases and annotation artefacts for prediction. we use the quality of the predictions made by models finetuned and tested on corrupted data as a proxy to evaluate data quality. ## models we use google's original tensorflow implementation foot_4 of the uncased 768-dimensional bert model (bert-base), a transformer model that learns representations via a bidirectional encoder @xcite . bert was pre-trained using a masked language model (mlm or cloze) task where some percentage of the input tokens are masked at random, and the model needs to predict these masked tokens; and on a next sentence prediction (nsp) task, where it receives pairs of sentences(a, b) as input and learns to predict if b follows a in the original document. sentence b in (a, b) is 50% of the time the actual sentence that follows a, and 50% of the time it is a random sentence from the training corpus. nsp increases the model's ability to capture the relationship between two sentences, which is the core task in nli and question answering. variants of the bert model achieve very high performance on nlu tasks, surpassing the human baseline on glue @xcite and reaching near-human performance on the challenging superglue dataset @xcite . for each experiment, we fine-tune bert for ten epochs on the original mnli training dataset or its transformed versions described in section 3, using a batch size of 100 (unless explicitly stated). for the experiments on the anli benchmark, we apply the roberta-large model, a variant of bert which has much higher performance than bert on the glue and superglue benchmarks. 7 we use the training and evaluation scripts provided by @xcite . 8 we fine-tune the model for two epochs using a batch size of 16. ## discussion the question of whether current state-of-the-art neural network models that beat human performance in nlu tasks actually understand language is currently much debated. our proposed corruption transformations often lead to sentences that make very little sense. nevertheless, we observe that bert performs surprisingly well in these experiments. this indicates that rather than understanding the meaning of the sentences and the semantic relationship between them, the models are able to pick up on other cues from the data that allow them to make correct predictions. our proposed diagnostics tests are useful devices for assessing the quality of a dataset as a testbed for evaluating models' language understanding capabilities. in our experiments, they demonstrate the superior quality of a nli dataset (anli) over another (mnli). we test this finding in an additional experiment where we apply the word shuffling mechanism of @xcite on the anli data, which was shown to not deteriorate bert-based model performance on the glue tasks. our results in our test suite can be seen as an additional "crash test" for assessing the quality of bench- comparison to results obtained on the original anli dataset @xcite . the anli-shuffle-n1/n2/n3 test sets contain shuffled ngrams, with @xmath1, 2, 3} respectively. mark datasets that address common-sense reasoning. it falls in the same line as work that highlighted problems of earlier datasets and resulted in the creation of anli. our proposition can be part of a good methodology for building future nli datasets. the multi-faceted nature of the problems that exist in current nli datasets makes research that investigates these issues very important; the more the diagnostic tests we have, the more reliable the datasets will hopefully get. the fact that one type of testing (hypothesis only, word shuffling or word class dropping) does not eliminate all problems present in the datasets, highlights the need for a variety of diagnostic devices addressing different phenomena. we propose the following set of diagnostics as the minimum sanity check when developing new nli datasets: returning to the specific findings of this paper, we performed an additional set of analysis aimed at identifying what the observed, relatively small, impact of the proposed modifications is due to. we explore whether the drop in performance can be explained by the (smaller or larger) number of tokens pertaining to the word class being removed. as can be seen in figure 1 , where we compare the accuracy of bert and the number of tokens removed from the training data in each setting, this factor does not explain the obtained results. for example, there are only 492,895 occurrences of adverbs removed from the training set, but the delta to the original result is the highest (-3.53 points), whereas removing 886,966 determiners has only a small impact on accuracy (-0.59 points). this plot demonstrates the important role of content words in nli prediction. zhou and bansal (2020) have shown that high lexical overlap between premises and hypotheses can guide models' predictions. we thus explore the extent to which our results can be explained by the amount of lexical overlap in the corrupt-test sets. we measure lexical overlap by counting the tokens shared by the premise and the hypothesis in a sentence pair. the orange bars in the plot in figure 2 show the amount of lexical overlap between premises and hypotheses (% calculated over the total number of examples) in the original mnli and the corrupt-test test sets. the blue bars show the prediction accuracy obtained by bert fine-tuned on the original mnli data when evaluated on each test set. we observe that although there is a decrease in lexical overlap in some test sets (e.g., in mnli-noun), there is no clear correlation between lexical overlap and accuracy, which suggests that the model picks up on other cues that remain in the corrupted sentences for prediction. ## conclusion we propose a novel diagnostics suite for assessing the quality of datasets used for nli model training and evaluation. we show that data corruption is an efficient way to estimate dataset quality and their potential to reflect the real language understanding capabilities of the models. our results on the mnli and anli datasets show that our methodology can successfully identify datasets of high or low quality, i.e. whether a dataset triggers models' reasoning potential or rather allows them to rely on cues and other statistical biases for prediction. our proposed tests can be used for assessing the quality of existing benchmarks used by the community and interpreting the results accordingly, and also to guide the development of new datasets addressing reasoning tasks. in this latter case, data corruption would serve to identify whether a dataset construction methodology and the adopted annotation guidelines are on the correct track. lastly, although it would be interesting to compare a larger number of architectures, we leave this comparison for future work due to lack of space and also in order to not confuse the reader, given the large number of settings where experiments are conducted. we also focus in this paper on the mnli and anli datasets, since our main concern is to cover as many corruption settings as possible. extending the current work to other models and nlu datasets is a natural next step for future research. we have made our code available to promote research in this direction. 11 additionally, since the present work leaves open questions as regards the factors behind the high performance observed on the corrupted datasets, we plan to more thoroughly investigate the cues and artefacts on which the models rely and which allow them to label premise hypothesis contradiction the intends that with appropriate in developing this. the discourages to consult with any. the, or where the? the of saving. entailment in the original, is set up by his and then ambushed by a hostile named, and when he tries to answer with an eloquent ( is clenched. the other is retrospective and intended to help those who review to assess the of completed. it is made to help the assess the of the. the had been found in a in the. the that was in the was powdered. neutral in the other, the beat the. the are a better.