Investigating Reasons for Disagreement in Natural Language Inference (2022)

Abstract: We investigate how disagreement in natural language inference (NLI) annotation arises. We developed a taxonomy of disagreement sources with 10 categories spanning 3 high- level classes. We found that some disagreements are due to uncertainty in the sentence meaning, others to annotator biases and task artifacts, leading to different interpretations of the label distribution. We explore two modeling approaches for detecting items with potential disagreement: a 4-way classification with a “Complicated” label in addition to the three standard NLI labels, and a multilabel classification approach. We found that the multilabel classification is more expressive and gives better recall of the possible interpretations in the data.

## introduction natural language inference (nli) is the task of identifying whether a hypothesis sentence is true, false, or undetermined, given a premise. it is considered one of the most fundamental aspects of competent language understanding. in natural language processing, the nli task is widely used to evaluate models' semantic representations (i.a., @xcite , and to facilitate downstream tasks, for example, in natural language generation @xcite . large nli datasets have been built by collecting inference judgments for premise-hypothesis pairs and aggregating the judgments by simple methods such as majority voting. however, it has been pointed out that nli items do not all have a single ground truth and can exhibit systematic disagreement (i.a., @xcite . this questions the assumption of having a single ground truth for each item and the validity of measuring models' ability to produce such ground truth. for instance, in (1) from the mnli dataset @xcite , 3 out of 5 annotators labeled the item as ''entailment'' (the hypothesis is inferred from the premise), 0 labeled it as ''neutral'' (the hypothesis cannot be inferred from the premise), and 2 as ''contradiction'' (the hypothesis contradicts the premise). h: it doesn't hold many people. [e,n,c]: [3,0,2] people have indeed different judgments on which number is required to count as holding many people. the premise and hypothesis do not resolve explicitly what is being talked about, possibly a stadium. does 40,000 count as many for a stadium seating capacity? the premise states that it's not large enough and uses the term squeezing, leading some annotators to see the hypothesis it doesn't hold many people as being inferred from the premise. on the other hand, 40,000 people in a specific location is a large number, and some annotators therefore judge the hypothesis as contradictory to the premise. such disagreement is not captured when taking only one of the three standard nli labels as ground truth. recent work @xcite has thus explored approaches for building nli models that predict the entire annotation distribution, instead of the majority vote category, in an attempt to move away from assuming a single ground truth per item. however, little is understood about where the disagreement stems from, and whether modeling the distribution is the best way to handle disagreement in annotation. to investigate these questions, we created a taxonomy of different types of disagreement consisting of 10 categories, falling into 3 high-level classes based on the ''triangle of reference'' by @xcite . we manually annotated a subset of mnli with the 10 categories. our categorization shows that items leading to disagreement in annotation are highly heterogeneous. moreover, the interpretation of the nli label distribution differs across items. we thus explored alternative approaches for modeling disagreement items: a 4-way classification approach with an additional label (on top of the three nli labels) capturing disagreement items, and a multilabel classification approach of predicting one or more of the three nli labels. we found that the two models behave somewhat differently, with the multilabel model offering more interpretable outputs, and thus being more expressive. our findings deepen our understanding of disagreement in a widely used nli benchmark and contribute to the growing literature on disagreement in annotation. we hope they highlight directions to reduce disagreement when collecting annotations and to design models to handle the disagreement that persists. the annotations, the guidelines, and the code are available at https://github.com /njjiang/nli disagreement taxonomy . ## related work focusing on disagreement in annotation is not new: @xcite argued for embracing annotation disagreement, viewing it as signal, and not as noise. even for tasks with supposedly a unique correct answer, such as partof-speech tagging, there are items for which the right analysis is debatable @xcite : is social in social media a noun or an adjective? @xcite showed that incorporating such disagreement signals into the loss functions of part-of-speech taggers improves performance. previous work noted that disagreement in annotation exists in many semantic tasks: anaphora resolution @xcite @xcite , coreference @xcite , sentiment analysis @xcite , word sense disambiguation @xcite , among others. @xcite introduced the ''triangle of reference'' framework to conceptualize the annotation process and explain annotation disagreement. annotation differences can stem from the sentences to be annotated, the labels, or the annotators. indeed, annotators, who interpret the sentences, produce labels in a way that is defined by the annotation guidelines. underspecification in each of these three components can result in disagreement in the annotations. disagreement can arise from (1) uncertainty in the sentence meaning, (2) underspecification of the guidelines, or (3) annotator behavior. we use the triangle of reference to organize our taxonomy. @xcite and @xcite showed that disagreement was systematic in the older nli datasets. @xcite showed that real-valued nli annotations are better modeled as coming from a mixture of gaussians as opposed to a single gaussian distribution. @xcite collected categorical nli annotations and found disagreement to be widespread, corroborating @xcite findings. @xcite found that items involving entity/event coreference and ''loose definitions'' of inference (e.g., whether a hill covered by grass is the same as the side of a mountain) have lower interannotator agreement. however, there is not yet a systematic investigation of how disagreement in nli arises. ## disagreement taxonomy to investigate where disagreement stems from, we conduct a qualitative analysis of parts of the mnli dataset @xcite . we chose mnli because it is diverse in genre and inference types, compared with datasets based on image captions that only describe visual scenes (e.g., sick @xcite , @xcite ). ## modeling experiments now that we understand better how disagreement arises, we explore how to build models that provide disagreement information. as discussed in section 2, a distribution gives the most finegrained information but can be misleading to interpret, while categorical information is often needed in downstream applications. therefore, we experiment with models that provide two kinds of categorical information for disagreement: an additional ''complicated'' class for labeling low agreement items (section 4.3), and a multilabel classification approach, where each item is associated with one or more of the three standard nli labels (section 4.4). as baseline, we take the mixup approach in @xcite , which predicts a distribution over the three labels, and uses a threshold to obtain multilabels/4-way labels. these models can be useful in an annotation pipeline. one needs to collect multiple judgments for each item to cover the range of possible interpretations, but doing so may be prohibitively expensive at a large scale. the annotation budget could thus be prioritized by collecting annotations for items with potential for disagreement, as predicted by the model. therefore, our goal is not necessarily to maximize accuracy. a model that can recall the possible interpretations is preferred to a model that misses them. ## error analysis we analyze the model behavior with respect to the categories of disagreement sources. for each category, figure 3 gives the percentages of chaosnli items annotated with at least that category and having converging nli interpretations (>80 agree on the nli label) vs. percentages of items predicted to exhibit disagreement (complicated by the 4-way model or as having more than one label by the multilabel model). overall, a category with more agreement (higher majority vote) in the annotations tends to have fewer items predicted as exhibiting disagreement. this is expected given that an item with convergence corresponds to not having disagreement as gold label, and the model performs well overall. comparing the two models, we see that all categories, except [7] temporal reference, are farther to the right in the multilabel classification (bottom panel) whereas they are more spread out in the 4-way classification (top panel), meaning that the 4-way model predicts an agreement label (e/n/c) more often than the multilabel model. this suggests that the 4-way model is more strongly tied to the convergence statistics and fails to detect potentials of disagreement. it also aligns with the previous finding that the multilabel model has higher recall of the range of interpretations. items in [3] presupposition, [4] probabilistic enrichment, and [5] imperfection are often predicted in both setups to exhibit disagreement (they are to the right of both plots in figure 3 ). 1 oh, sorry, wrong church. he or she entered the wrong church. [82, 17, 1] complicated / en 2 there should be someone here who knew more of what was going on in this world than he did now. he knew things, but hoped someone else knew more. [82, 18, 0] complicated / en 3 what am i to do with them afterwards?" it is the narrator's responsibility to take care of them. @xcite @xcite n / nc 4 but they persevered, she said, firm and optimistic in their search, until they were finally allowed by a packed restaurant to eat their dinner off the floor. because all of the seats were stolen, they had to eat off the floor. the original wax models of the river gods are on display in the civic museum. they have models made out of clay. [5, 38, 57] c / c 6 indeed, said san'doro. indeed, they said. [52, 22, 26] e / en 7 this was built 15 years earlier by jahangir's wife, nur jahan, for her father, who served as mughal prime minister. nur jahan's husband jahangir served as mughal prime minister. [17, 17, 66] complicated / e 8 cruises are available from the bhansi ghat, which is near the citypalace. you can take cruises from phoenix arizona. [0, 51, 49] complicated /nc accommodating minimally added content 9 the key question may be not what hillary knew but when she knew it. according to current reports, the question is not if, but when did hillary know about it. [90, 9, 1] e / en 10 four or five from the town rode past, routed by their diminished numbers and the fury of the kal and thorn. kal and thorn were furious at the villagers. [50, 41, 9] n / en we thus take a closer look at the following categories, across all items annotated: probabilistic enrichment, coreference, and accommodating minimally added content (discussed in section 3.4). probabilistic enrichment the multilabel model predicts 68% of the items annotated with probabilistic enrichment to have more than one nli label. in particular, 36% are predicted as en and 27% as nc, corresponding the common patterns in probabilistic enrichment where the enriched (not explicitly stated) inference leads to entailment/contradiction, and the neutral label is warranted without enrichment. we found that the multilabel model often predicts labels when they are only slightly below the threshold of 20 that we used to count a label as present (items 1 and 2 in coreference for items annotated with coreference, both models predict entailment/ contradiction when the premise and hypothesis share the same argument structure or involve simple word substitutions (e.g., wax/clay in item 5 and san'doro/they in item 6, ## conclusion we examined why disagreement in nli annotations occurs and found that it arises out of all three components of the annotation process. we experimented with modeling nli disagreement as 4-way and multilabel classifications, and showed that the multilabel model gives a better recall of the range of interpretations. we hope our findings will shed light on how to improve the nli annotation process, for example, ways to specify the guidelines to reduce disagreement or introduce contexts that resolve underspecification, ways to gather enough annotations to cover the possible interpretations, as well as ways to model nli without the single ground truth assumption.