NOPE : A Corpus of Naturally-Occurring Presuppositions in E nglish (2021)

Abstract: Understanding language requires grasping not only the overtly stated content, but also making inferences about things that were left unsaid. These inferences include presuppositions, a phenomenon by which a listener learns about new information through reasoning about what a speaker takes as given. Presuppositions require complex understanding of the lexical and syntactic properties that trigger them as well as the broader conversational context. In this work, we introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus to investigate the context-sensitivity of 10 different types of presupposition triggers and to evaluate machine learning models’ ability to predict human inferences. We find that most of the triggers we investigate exhibit moderate variability. We further find that transformer-based models draw correct inferences in simple cases involving presuppositions, but they fail to capture the minority of exceptional cases in which human judgments reveal complex interactions between context and triggers.

## analysis of human judgments to investigate to what extent the different trigger types give rise to presuppositions, and to what extent they exhibit context-sensitivity and gradience, we analyzed the crowdsourced human judgements as described in the following sections. ## conclusion we presented nope, a dataset of naturally occurring presuppositions involving a diverse set of presupposition triggers. the human judgments in this dataset suggest that the examples we identifed give rise to presuppositions in most cases, and that presupposition cancellation and variability exist in a small but non-negligible minority of cases across contexts for presuppositions other than those triggered by clause-embedding verbs and implicatives. we used this dataset to evaluate the pretrained transformer-based models roberta and deberta. we found that the models were able to draw inferences involving presuppositions in the simple cases but failed to fully capture human-level context-sensitivity and gradience.