What Can We Learn from Collective Human Opinions on Natural Language Inference Data? (2020)

Abstract: Despite the subjective nature of many NLP tasks, most NLU evaluations have focused on using the majority label with presumably high agreement as the ground truth. Less attention has been paid to the distribution of human opinions. We collect ChaosNLI, a dataset with a total of 464,500 annotations to study Collective HumAn OpinionS in oft-used NLI evaluation sets. This dataset is created by collecting 100 annotations per example for 3,113 examples in SNLI and MNLI and 1,532 examples in αNLI. Analysis reveals that: (1) high human disagreement exists in a noticeable amount of examples in these datasets; (2) the state-of-the-art models lack the ability to recover the distribution over human labels; (3) models achieve near-perfect accuracy on the subset of data with a high level of human agreement, whereas they can barely beat a random guess on the data with low levels of human agreement, which compose most of the common errors made by state-of-the-art models on the evaluation sets. This questions the validity of improving model performance on old metrics for the low-agreement part of evaluation datasets. Hence, we argue for a detailed examination of human agreement in future data collection efforts, and evaluating model outputs against the distribution over collective human opinions.

## introduction natural language understanding (nlu) evaluation plays a key role in benchmarking progress in natural language processing (nlp) research. with the recent advance in language representative learning @xcite , results on previous benchmarks have rapidly saturated. this leads to an explosion of difficult, diverse proposals of tasks/datasets for nlu evaluation, including natural language inference (e.g., snli, mnli and anli) @xcite @xcite , grounded commonsense inference @xcite , commonsense qa @xcite , social interactions reasoning @xcite , abductive commonsense reasoning (αnli) @xcite , etc. one common practice followed by most of these recent works is to simplify the evaluation of various reasoning abilities as a classification task. this is analogous to asking objective questions to a human in educational testing. this simplification not only facilitates the data annotation but also gives interpretable evaluation results, based on which behaviors of the models are studied and then weaknesses are diagnosed @xcite . despite the straightforwardness of this formalization, one assumption behind most prior benchmark data sourcing is that there exists a single prescriptive ground truth label for each example. the assumption might be true in human educational settings where prescriptivism is preferred over descriptivism because the goal is to test humans with well-defined knowledge or norms @xcite . however, it is not true for many nlp tasks due to their pragmatic nature where the meaning of the same sentence might differ depending on the context or background knowledge. specifically for the nli task, manning (2006) advocate that annotation tasks should be "natural" for untrained annotators, and the role of nlp should be to model the inferences that humans make in practical settings. previous work @xcite that uses a graded labeling schema on nli, showed that there are inherent disagreements in inference tasks. all these discussions challenge the commonly used majority "gold-label" practice in most prior data collections and evaluations. intuitively, such disagreements among humans should be allowed because different annotators might have different subjective views of the world and might think differently when they encounter the same reasoning task. thus, from a descriptive perspective, evaluating the capacity of nlp models in predicting not only individual human opinions or the majority human opinion, but also the overall distribution over human judgments provides a more representative comparison between model capabilities and 'collective' human intelligence. therefore, we collect chaosnli, a large set of collective human opinions for examples in several existing (english) nli datasets, and comprehensively examine the factor of human agreement (measured by the entropy of the distribution over human annotations) on the state-of-the-art model performances. specifically, our contributions are: • we collect additional 100 annotations for over 4k examples in snli, mnli-matched, and αnli (a total of 464,500 annotations) and show that when the number of annotations is significantly increased: (1) a number of original majority labels fail to present the prevailing human opinion (in 10%, 20%, and 31% of the data we collected for αnli, snli, and mnli-matched, respectively), and (2) large human disagreements exist and persist in a noticeable amount of examples. the chaosnli dataset and experimental scripts are available at https://github.com/easonnie/ chaosnli 2 related work ## data collection our goal is to gather annotations from multiple annotators to estimate the distribution over human opinions. section 3.1 and 3.2 state some details of the collection. more importantly, section 3.3 explains the challenges of such data collection and how our designs ensure data quality. ## analysis of human judgements examples. ## analysis of model predictions in section 4, we discussed the statistics and some examples for the new annotations. the observation naturally raises two questions regarding the development of nlp models: (1) whether the stateof-the-art models are able to capture this distribution over human opinions; and (2) how the level of human agreements will affect the performance of the models. hence, we investigate these questions in this section. section 5.1 and 5.2 state our experimental choices. section 5.3 discusses the results regarding the extent to which the softmax distributions produced by state-of-the-art models trained on the dataset reflects similar distributions over human annotations. section 5.4 demonstrate the surprising influence of human agreements on the model performances. ## discussion & conclusion while common practice in natural language evaluation compares the model prediction to the majority label, section 5.4 questions the value of continuing such evaluation on current benchmarks as most of the unsolved examples are of low human agreement. to address this concern, we suggest nlp models be evaluated against the collective human opinion distribution rather than one opinion aggregated from a set of opinions, especially on tasks which take a descriptivist approach foot_5 to language and meaning, including nli and common sense reasoning. this not only complements prior evaluations by helping researchers understand whether model performance on a specific data point is reliable based on its human agreement, but also makes it possible to evaluate models' ability to capture the whole picture of human opinions. section 5.3 shows that such ability is missing from current models and potential room for improvement is huge. it is also important to note that the level of human agreement is an intrinsic property of a data point. section 5.4 demonstrates that such a property can be an indicator of the difficulty of the modeling. this hints at the connections between human agreements and uncertainty estimation or calibration @xcite where machine learning models are required to produce the confidence value of their predictions, leading to important benefits in real-world applications. in conclusion, we hope our data and analysis inspire future directions such as explicit modeling of collective human opinions; providing theoretical supports for the connection between human disagreement and the difficulty of acquiring language understanding in general; exploring potential usage of these human agreements; and studying the source of the human disagreements and its relations to different linguistic phenomena.