CREPE: Open-Domain Question Answering with False Presuppositions
Year: 2023

Abstract:
When asking about unfamiliar topics, information seeking users often pose questions with false presuppositions. Most existing question answering (QA) datasets, in contrast, assume all questions have well defined answers. We introduce CREPE, a QA dataset containing a natural distribution of presupposition failures from online information-seeking forums. We find that 25% of questions contain false presuppositions, and provide annotations for these presuppositions and their corrections. Through extensive baseline experiments, we show that adaptations of existing open-domain QA models can find presuppositions moderately well, but struggle when predicting whether a presupposition is factually correct. This is in large part due to difficulty in retrieving relevant evidence passages from a large text corpus. CREPE provides a benchmark to study question answering in the wild, and our analyses provide avenues for future work in better modeling and further studying the task.1

heading: 1 Introduction
text: When an information-seeking user poses a question about the topic they are unfamiliar with, they can often introduce false presuppositions (Kaplan, 1978; Duží and Číhalová, 2015) which are assumed but not directly stated. For instance, the question in Figure 1 incorrectly presupposes that the equal and opposite reactions in Newton’s law apply to the same object. Although such a question is unanswerable, we might still hope to identify the confusion and explain it to the user. This functionality goes well beyond prior open-domain QA task formulations, which focus on questions with a valid direct answer (Rajpurkar et al., 2016; Kwiatkowski et al., 2019) or that are unanswerable from lack of evidence (Rajpurkar et al., 2018; Choi et al., 2018; Asai and Choi, 2021). While
1The data, baselines, and the evaluation script are available at github.com/velocityCavalry/CREPE.
recent work studies unverifiable presuppositions in reading-comprehension-style questions given evidence context (Kim et al., 2021), there has been no work that identifies and corrects a presupposition that is false based on global, factoid knowledge.
In this paper, we introduce CREPE (CorREction of PrEsupposition), a new dataset consisting of 8,400 Reddit questions with (1) whether there is any false presuppositions, and (2) if any, the presupposition and its correction written. We find 25% of questions on Reddit (Fan et al., 2019) include false presuppositions, where the best response is to provide a correction of the presuppositions.
While the CREPE annotation task is challenging due to the need for extensive background knowl-
10457
edge and inherent debatability, we leverage the most upvoted comments written by community users to efficiently annotate the data. The most upvoted comments are likely to be factually correct, and typically also identify and correct any false presuppositions made in the question. By designing an annotation pipeline using these comments, we were able to collect high-quality data at a relatively low cost. Our data analysis (Table 2) shows that the types of false presuppositions are diverse, ranging from relatively explicit presuppositions (e.g., false clauses or false predicate) to subtle, nuanced presuppositions (e.g., false causality or false existential presuppositions).
We define two tracks with varying levels of difficulty, and introduce models to set baseline performance levels for each. A model is given either the question only (the main track) or the question and the comment (the GOLD-COMMENT track), and is supposed to perform two subtasks: identification of whether or not there is a false presupposition (the detection subtask) and generation of presuppositions and their corrections (the writing subtask). For the writing subtask, we propose a systematic human evaluation scheme based on Celikyilmaz et al. (2020) that considers fluency, correctness (precision of the information), adequacy (recall of the information) and consistency.
We include a range of baselines, including a question-only model, a nearest-neighbor model, and a competitive model based on the state-of-theart passage retrieval (Krishna et al., 2021) and pretrained language models (Liu et al., 2019; Raffel et al., 2020). Results and analyses indicate that (1) retrieval is very challenging since simply retrieving passages in the topic of the question is not enough; (2) models do moderately well in identifying explicit false presuppositions, and (3) models struggle with identifying implicit presuppositions and explaining how and why the presupposition is false. We also discuss open problems, such as an inherent ambiguity in the validity of presuppositions and inconsistency between different websites.

heading: 2 Background

heading: 2.1 Question Answering
text: There has been significant work on question answering, where the model receives a natural language, open-domain question and is required to return a short, concise answer (Voorhees and Tice, 2000; Lee et al., 2019). Most work focuses on
questions that have a short text span as a correct answer. Other work studies unanswerable questions, but they study questions that are either intentionally written to be unanswerable (Rajpurkar et al., 2018) or where there is a lack of evidence to find the answer (Choi et al., 2018; Asai and Choi, 2021). More recently, Kim et al. (2021) studies unverifiable presuppositions in questions under the given context, but using questions from Kwiatkowski et al. (2019) whose presuppositions are mostly not false based on global knowledge.2
In this work, we focus on open-domain questions with presuppositions that are false based on global knowledge, and argue that an adequate response to them is a correction of these presuppositions. We show that false presuppositions in questions are prevalent. 25% of questions contain false presuppositions in the domain of online forums (for which we collect annotations), research papers (Dasigi et al., 2021), scientific reviews (Kang et al., 2018), and social media (Sap et al., 2020).

heading: 2.2 Presupposition
text: Under a pragmatic point of view, a presupposition is a condition that a speaker would normally expect to hold in the common ground between discourse participants when that sentence is uttered (Beaver et al., 2014; Stalnaker et al., 1977). Unlike semantic presuppositions (Strawson, 1950), pragmatic presuppositions cannot easily be traced to specific words or phrases, but rather depend on the context and the expectations of the discourse participants (Potts, 2015). A key property of pragmatic presuppositions is that they are backgrounded—a pragmatic property of being a meaning that the speaker presumes to be mutual public knowledge. While whether or not one is a presupposition is inherently debatable, we define a false presupposition based on the most voted comment in the online forum, as we discuss further in Section 3.2.
False presuppositions in questions have been discussed in the linguistic literature (Kaplan, 1978; Duží and Číhalová, 2015). False presuppositions in the question make a question infelicitous because there is no direct answer. Kaplan (1978) claims that an adequate and unambiguous answer to such a question is a negated presupposition, referred to as corrective indirect response. We follow them in providing the negated presupposition as the response
2Less than 5% of questions from Kwiatkowski et al. (2019) contains false presupposition under our definition, likely because their questions are aggressively filtered.
to the question, and build the first benchmark based on questions written by information-seeking users.

heading: 3 Dataset: CREPE
text: Our task requires the model to be given a question q and to (a) identify whether q has any false presuppositions, and (b) if yes, generate the false presupposition as well as the correction.
We have three criteria in constructing the dataset:
C1. Naturalness of the questions: whether the questions in the data are written by real, information-seeking users.
C2. Validity of the presupposition: whether the identified presupposition is highly likely made by the question writer.
C3. Correctness and adequacy of the information: whether the information provided in the correction is factually correct, and adequate to convince the question writer.
We first describe the source of questions (Section 3.1) which address C1. We then describe the process of annotation (Section 3.2) that addresses C2 and C3. Finally, we present detailed, qualitative analysis of the data (Section 3.3). The formal definition of the task and metrics are in Section 4.

heading: 3.1 Data Source
text: Our highest priority is to study false presuppositions that naturally occur from information-seeking users. While it is significantly easier to manually write questions that would have false presuppositions, we think these questions will significantly be different from naturally occurring questions.
Following Fan et al. (2019), we use questions posted on the ELI5 subreddit.3 We made a few modifications to the procedure that Fan et al. (2019) took in order to improve the data quality. We first filter questions and comments based on upvotes with a higher threshold. We then split the training, the development and the test data based on the time of the posting: questions on the training set are posted in 2011–2018, questions on the development set are posted in Jan–Jun of 2019, and questions on the test set are posted in Jul–Dec of 2019. Appendix A provides more details.
Krishna et al. (2021) raised a concern that a significant amount of test are duplicates of those on the training set. We provide a detailed analysis in Appendix A. In summary, we think (1) the amount
3www.reddit.com/r/explainlikeimfive
of duplicated (or paraphrased) questions is significantly less than their estimate with respect to underlying presuppositions in the questions, and (2) even if there are paraphrased questions, data split based on the time frame is justified based on the real-world scenario.4

heading: 3.2 Data Annotation
text: Meeting the criteria C2 and C3 can be very difficult for the following reasons:
• For C2: The validity of presupposition is inherently debatable and largely depends on the background of individuals (Section 2.2).5
• For C3: The open-domain nature of the task requires the search of world knowledge on the web, which is extremely expensive and may not be exhaustive enough despite the best efforts made by annotators, as discussed in Kwiatkowski et al. (2019); Min et al. (2020).
In this work, we make use of the most upvoted comments written by community users. The comment, often written by domain experts, provides a response to the question in the ELI5 subreddit and has been used as a credible source in prior work (Fan et al., 2019). If the comment identifying a false presupposition has the most upvotes, it is likely that the presupposition is valid (made by a question writer) based on the background context shared by community users, thus satisfying C2. Moreover, the comment (1) is highly likely to contain information that is correct and adequate (satisfying C3), and (2) removes the need for exhaustively searching over the web (reducing the annotation cost).
Annotation task. Annotators are given a pair of the question and the most voted comment, and perform the following steps.
1. Filter out questions that are subjective, are uninformative, or rely on personal experience.
2. Judge whether there is a false presupposition in the question, identified by the comment.
3. If there is a false presupposition, write the presupposition and a correction as a concise, declarative sentence.
4We think having similar questions is an inherent property of questions on the web, and the model should be allowed to take whichever approach that is plausible, including the nearest neighbor approach (Lewis et al., 2021).
5This is also the case in previous work—for instance, the data annotated by experts in formal semantics and pragmatics can have low agreement (Jeretic et al., 2020).
Annotation pipeline. We maintain a pool of qualified annotators who passed our qualification task. We assign two annotators per question, where each annotators independently annotate the question. We filter out questions if either of the annotators mark them as such. If the annotators agreed to on the label (whether or not there is a false presupposition), their label as well as their writings are taken as gold references. When they disagreed, we assign a third annotator and take a majority vote over three workers. The percentage agreement in the initial stage of the annotation is 75%, and the Fleiss’ kappa is 43%, indicating moderate agreement.6 We find that the disagreements are mainly due to inherent ambiguities of the task due to the different interpretations of the question or the comment, or difference in individual background knowledge (discussion in Section 3.3).
More details in instructions and quality control are provided in Appendix B.

heading: 3.3 Data Analysis
text: The data statistics are provided in Table 1. We find that over 25% of questions posted on Reddit includes false presuppositions.7
Categorization of false presuppositions. We randomly sample 50 questions with false presuppositions and categorize them in Table 2. The five most frequent types of presuppositions include:
• False clauses are those where FP is made as a clause in Q, e.g., “the water has to be 100 to become steam” in Table 2. 6Kappa is a chance-corrected coefficient and is correlated
to the skewness of data distribution. 7Xu et al. (2022), who also used the ELI5 subreddit, identified 10% of questions have false presuppositions (rejected presuppositions) and excluded them in their data. Their estimate is significantly lower than ours because they did not include partial rejection while we do.
• False predicate are those where the predicate in Q is false, e.g., “current is stored in power plants”.
• False properties are those where certain properties or attributes are presupposed in Q, e.g., “cement blocks are too strong so that people who punch them are likely to break their hand.” They are often very implicit and may be deniable; however, the question writer would not have asked the question if they have not made the presupposition.
• False (causal) relationship between facts are those where Q makes a (causal) relationship between facts that are false. This is another implicit type of presuppositions, but again, the question writer would not have asked this question if they have not made such a presupposition. A large portion of questions involve scientific phenomenon on which the question writer has misunderstanding, e.g., in the example in Table 2, the question writer had a misconception about Newton’s Third Law of Motion.
• False existential presupposition indicates that Q includes an existential presupposition, one type of semantic presuppositions, that is false. For instance, the example Q in Table 2 presupposes an unused space in a hard disk, and C says there is no unused space.
Triggers in the comment. We analyze how comments point out the falsehood of the presupposition made in the question on the same set of 50 samples on the development data. In 68% of times, comments include specific lexical cues which we call triggers. 70% of such triggers are negations. Other word-level triggers include “actually”, “just”, “rare”, “really” and “though”. Sentence-level triggers include “You are mistaken”, “You’ve got some
and FP indicate the question, the comment, and the presupposition, respectively.
major confusion here”, “I don’t think you are fully comprehending ...” and “It should be noted that...”. 80% of triggers appear in the first sentence of the comment, and the rest of the sentences elaborate on how the presupposition is false or provide other relevant information that does not directly answer the question. The rest 32% do not include lexical triggers and requires more careful comprehension of the comment with respect to the question, e.g., the false existential example in Table 2.
Analysis of ambiguous cases. Even with our best efforts, there are still inherent disagreement between annotators. Some of them are due to inherent ambiguities in language, e.g., the first example in Table 9 where ‘the state of the water’ could either mean the molecule itself or the energy state of the molecule. Others are due to disagreement on the validity of the presupposition, e.g., in the second example in Table 9, it is debatable whether or not the question writer presupposes that the Board deals with day to day at a company. We revisit this issue in human performance estimation in Section 5.2.

heading: 4 Task Setup
text: The model is given a question q, and is required to perform the following subtasks: (a) Detection: assign a label to be FP or N; FP
means q has a false presupposition, and N means q has no false presuppositions. We use a macro-F1 score as an evaluation metric.
(b) Writing: if FP, write the false presupposition as well as the correction. We use sacreBLEU (Post, 2018) and unigram-F1 following Petroni et al. (2021) as well as SentBERT (Reimers and Gurevych, 2019) for evaluation. We also introduce a human evaluation scheme in Section 6.3.
We have two tracks: the main track and the GOLDCOMMENT track.
The main track provides q as the only input to the model. The model is expected to search necessary background knowledge to perform the task from any information source except for Reddit and Quora.8 This is the most realistic setting for the typical open-domain question answering problem.
The GOLD-COMMENT track provides the comment used for the annotation as an additional input to the model. This removes the need for retrieval, and guarantees that all necessary information to perform the task is given.
8This is because similar questions are being asked on these websites. The same decision has made in Nakano et al. (2021).

heading: 5 Experiments: Detection
text: This section discusses baseline experiments for the detection subtask; Section 6 discusses baseline experiments for the writing subtask.

heading: 5.1 Baselines

heading: 5.1.1 Trivial baselines
text: Random assigns FP or N randomly at uniform. FP only always assigns FP. N only always assigns N. Nearest Neighbor retrieves one of questions from the training set that is closest to the test question, based on c-REALM (Krishna et al., 2021), and returns its label as the prediction.

heading: 5.1.2 GOLD-COMMENT track baselines
text: Question only trains a RoBERTa-based (Liu et al., 2019) classifier that takes the question as the only input and classifies the label. It is often called closed-book model (Roberts et al., 2020). Comment only is a classifier based on RoBERTa-large that takes the comment as the only input and assigns the label. Question⊕Comment is a classifier based on RoBERTa-large that takes a concatenation of the question and the comment to the classifier, and assigns the label. We additionally experiment with the same model that is trained on either MNLI (Williams et al., 2018) or BoolQ (Clark et al., 2019), and tested on CREPE in a zero-shot fashion. This condition tests if training on similar, previously studied datasets helps.

heading: 5.1.3 Main track baselines
text: We design a model called c-REALM + MP (Multipassage) classifier that retrieves a set of paragraphs from Wikipedia and then assigns a label.
First, the model uses c-REALM (Krishna et al., 2021), a state-of-the-art retrieval model on ELI5, to retrieve a set of k passages from the English Wikipedia. Next, the model uses the multi-passage classifier based on RoBERTa in order to assign a label. Given a question q and a set of passages p1...pk, each pi (1 ≤ i ≤ k) is concatenated with q and is transformed into hi ∈ Rh through the Transformer model. We then obtain logits via p = FFN(MaxPool(h1...hk)) ∈ R2, where FFN is a feed-forward layer and MaxPool is an elementwise max operator. Finally, we use Softmax(p) to compute the likelihood of q having false presuppositions or not.
Self-labeling. Although our labeled data is small, there is large-scale unlabeled data (question and
comment pairs) available. We explore self-labeling to leverage this unlabeled data. Specifically, we use the Question⊕Comment to assign a silver label to the unlabeled training questions. We then train the classifier on the union of this silver data as well as the gold labeled data.

heading: 5.1.4 Human performance
text: We estimate human performance to better understand the model performance. We recruit two human workers who perform the task for 186 questions for each track.
We estimate two types of human performance. (1) Human with the most voted comment, where human workers assume the most voted comment as a ground truth in terms of factuality of the information and the validity of the presupposition. We think of it as an upperbound of model performance. (2) Human w/o the most voted comment, where human workers search over the web (except Quora and Reddit) to find information, and make the best judgment about the validity of the presupposition. We think it is likely to be worse than the upperbound of model performance, since only one worker, instead of multiple online users or domain experts, makes a decision.

heading: 5.2 Results
text: Results are reported in Table 3.
The GOLD-COMMENT track. All trivial baselines achieve poor performance. In particular, poor performance of the nearest neighbor model indicates that there is no significant train-test overlap on CREPE. Using both the question and the comment (Question⊕Comment) achieves the best performance, outperforming the best trivial baseline by 22% absolute. Zero-shot models trained on MNLI and BoolQ achieve poor performance, indicating that our problem is significantly different from existing tasks like NLI or binary question answering. The best model is 10% below human performance, indicating room for improvement, even in the easier track.
The main track. Using retrieved passages from c-REALM and multi-passage classifier achieves 66.3% on the test set, which is significantly better than all trivial baselines. The self-labeling technique leads to additional improvements, leading to an F1 of 67.1%. While these numbers are significantly better than trivial baselines, they are significantly worse than the model performance given
the comment. This strongly suggests a retrieval bottleneck—getting passages that provide evidence as strong as human-written comments is difficult even with the state-of-the-art retrieval model.
To further support the bottleneck in retrieval, we conduct a detailed error analysis in Appendix C.2. For instance, 86% of false negatives were due to retrieval misses, including failing to retrieve relevant topics (42%), retrieving evidence on the relevant topics but not related to the false presuppositions (32%), or retrieving evidence related to the presuppositions but is not direct enough (12%).9
Human performance. Humans given the most upvoted comment achieve performance that is significantly higher than all baseline numbers, indicating significant room for improvement.
Without the most upvoted comment, people achieve relatively poor performance (70.9%). To better understand this, we analyze 44 error cases, and categorize them in Table 4. Nearly half of the errors are due to an inherent disagreement in labels, either due to (1) ambiguity, either in language or whether the presupposition was made, or (2) whether it is critical to correct false presuppositions (especially cases in the exception category in Table 2). We think using the most upvoted com-
9This can be seen as a classification error—if the classification model can better capture implicit evidence, it could have made a correct prediction.
ment for a decision is reasonable since it is an aggregation of active community users and domain experts, but future work may take other approaches to consider ambiguities of the decision.

heading: 6 Experiments: Writing

heading: 6.1 Baselines
text: In the writing subtask, the system is given a question that is guaranteed to contain a false presupposition, and is required to generate the presupposition as well as the correction.

heading: 6.1.1 GOLD-COMMENT track baselines
text: Copy baseline. As a trivial baseline, we copy the given question as a presupposition and the given comment as a correction.
Question⊕Comment Dedicated. We train two generators separately to generate the presupposition and the correction, respectively, given a concatenation of the question and the comment. Both models are based on the pretrained T5-base model (Raffel et al., 2020).
Question⊕Comment Unified. We also design a unified model that can be used for both the presupposition and the correction, motivated by the intuition that generation of each can benefit from each other. We train one generator that is trained with a union of (1) annotated corrections, and (2) annotated presuppositions prepended with “It is not the case that” so that they look like corrections. At inference time, we use a standard, beam search decoding to generate the correction. To generate the presupposition, we first decode a sequence with a constraint (De Cao et al., 2021) that it should start with “It is not the case that”, and then take the sequence that comes next as a presupposition.

heading: 6.1.2 Main track baselines
text: We design c-REALM + MP (Multi-Passage) Dedicated and c-REALM + MP (Multi-Passage) Unified. They are similar to the dedicated and unified
models in Section 6.1.1. The only difference is that the model receives a question and a set of k passages from c-REALM instead of a questioncomment pair. In order for the T5 model to read multiple passages, we use the Fusion-in-Decoder architecture (Izacard and Grave, 2021). We refer to Appendix C.1 for more details.

heading: 6.2 Results: Automatic Evaluation
text: Table 5 reports the results in unigram F1 and BLEU. Examples of model outputs are provided in Appendix C.4. All models outperform the trivial copy baseline and perform better in the GOLDCOMMENT track than in the main track. Models are overall better at writing presuppositions than writing corrections, and the performance gap between the GOLD-COMMENT track and the main track is larger in presuppositions than in corrections. This is likely because the impact of evidence passages is more significant in correction writing than in presupposition writing since the presupposition can often be extracted from the question alone, while the correction requires information
beyond the question. It is also worth noting that the unified model is better than the dedicated model in the main track but not in the GOLD-COMMENT track. This indicates that, while multi-task learning improves the main track, it does not improve the GOLD-COMMENT track, possibly because performing two tasks in isolation is sufficient.
Table 6 reports results in SentBERT scores. The overall scores are high, likely because SentBERT considers entailment between a reference and a generation. For instance, the copy baseline achieves high scores since the question and the presupposition, and the comment and the correction entail each other by definition. It is important to note that entailment is a necessary but not a sufficient condition for a presupposition or a correction to be satisfactory in our task definition. The next section shows that models with high SentBERT scores obtain low human ratings.

heading: 6.3 Results: Human Evaluation
text: We conduct human evaluation of model generations on 200 randomly sampled test instances from the
following aspects (each in the 0–3 scale):
• Fluency: The generated text should be fluent (i.e., free of grammatical errors, spelling errors, and repetitions).
• Presupposition: The generated presupposition should be the valid one in the question, and is factually false.
• Correction: The correction should be made in the comment and provide reasonable amount of justification rather than being a simple negation of the presupposition.
• Consistency: The presupposition and correction should be on the same topic and negate each other.
We evaluate the output from all systems except the copying baseline, as well as the ground truth reference. Each question is assigned two raters in order to reduce noise and report inter-rater agreement on pairwise comparison. More details about the rating scheme are provided in Appendix C.4.
Based on results reported in Table 7, all models generate almost flawless fluent text and valid presuppositions. However, their outputs generated as false presuppositions are factually correct in half of the cases. These observations are relatively consistent across different systems.
Notable differences between systems are found in correction and consistency. The dedicated model generates better correction, likely because it is given a comment. All other models struggle: in particular, the unified models tend to generate the correction that starts with “It is not the case that” even the model is not restricted to do so at inference time. On the other hand, the unified model is better in consistency, likely because the dedicated model is more vulnerable in generating the presupposition and the correction in a totally different topic.

heading: 7 Conclusion
text: We introduced CREPE: the first benchmark for the identification and correction of false presuppositions in the open-domain setup. CREPE consists of 8,400 user questions, 25% of which contain false presuppositions and are paired with their corrections. Our detailed analysis highlights challenges in solving the task, including (1) retrieval of evidence that identifies false presupposition, (2) identification of implicit and subtle presuppositions, and (3) generating correction that is accurate and
adequately explains how and why the presupposition is false. We hope our benchmark adds to the problem of open-domain, open-ended question answering, inviting researchers to build models to study questions with false presuppositions. Further, we suggest future work to develop better models, explore approaches to address inherent debatability of the judgment, and evaluation of the model generation.
Limitations
Inherent debatability in false presuppositions. As discussed earlier, the validity of presupposition is inherently debatable and largely depends on the background context, i.e., even experts in formal semantics and pragmatics observe a high disagreement rate (Jeretic et al., 2020). Our proposal in using the most upvoted comments partially address the issue, but not perfectly, as discussed extensively in Section 5.2. One avenue for future work is to consider extra-linguistic context such as individuals background when judging the validity of presuppositions (Zhang and Choi, 2021).
Evaluating massive language models. Massive language models such as GPT-3 (Brown et al., 2020) have been shown impressive performance in open-ended question answering (Nakano et al., 2021). Our paper does not include large-scale, systematic evaluation of such models. Instead, we conduct a small-scale case study with GPT-3 text-davinci-002. See Appendix D for details. Most generations are roughly on the right topic, but often contain information that is factually false and do not precisely answer the question. Moreover, they rarely explicitly identify false presupposition and provide corrections, indicating that GPT-3 is far from solving our task. We think future work may explore larger-scale evaluation in a more systematic manner.
False presuppositions beyond online forums. The domain of CREPE is limited to online forums (Reddit). While this choice was made due to the availability of large data and its general domain, we argue that false presuppositions are not specific to such domains. For instance, we find that a similar portion (25%) have false presuppositions on information-seeking questions on NLP research papers posed by NLP experts; see Appendix E for details. We think future work can explore creating benchmarks on such domains, as well as studying
false presuppositions on a broader set of domains that require domain expertise.

heading: Acknowledgements
text: We thank Daniel Fried and Julian Michael for their feedback on an early stage of the project. We thank Zeqiu Wu and H2Lab members for comments on the paper. We thank Li Du for suggesting the name of the dataset. This research is supported by DARPA MCS program through NIWC Pacific (N66001-19-2- 4031), NSF IIS-2044660, an Allen Distinguished Award and gifts from AI2. SM is supported by a J.P. Morgan fellowship.

heading: A Details in Data Source
text: License The ELI5 (Fan et al., 2019) dataset uses the BSD License. Our usage of this dataset is consistent with its intended use, and the license will be included when downloading the data.
Filtering Data Sources. When the question has several comments, we choose the comment with the highest upvotes. Since our data is derived from Reddit, it may contain unintended social bias or harmful content. To remove toxic language on the data, we follow the toxicity word list from Cachola et al. (2018) to remove questions that contain any of the toxic words, except “hell” and “damn”, as these two words are commonly used as interjections. The original authors of the ELI5 dataset (Fan et al., 2019) and we have made our best efforts to remove such content; however, it is possible that some harmful context may remain on the data after the filtering process.
Analysis in Train-Test Overlap. Krishna et al. (2021) reported that 81% of the validation questions of the original ELI5 dataset are paraphrases of the question on the training set. We revisit this issue, and show with a careful assessment of underlying assumptions and a finer-grained definition of “paraphrases”, the proportion of paraphrased questions is significantly smaller.
We assess 104 randomly sampled questions in the validation set with their closest 7 training questions retrieved using c-REALM, as Krishna et al. (2021) did, but with a different rating scale (1–4), following Bhagat and Hovy (2013), Ganitkevitch et al. (2013), and Pavlick et al. (2015):
• 1: No Paraphrase; No similar intention: Two questions do not share the meaning nor the intention.
• 2: No Paraphrase; Similar intention: Two questions are likely to have the same intention, but they are not paraphrases, because their literal meanings are different and/or their underlying assumptions are different.
• 3: Non-trivial paraphrase: Most of the two questions’ meanings are the same; however, they do not belong to any of lexical paraphrase (single word to single word), phrasal paraphrase (multiword to single/multiword), or syntactic paraphrase (paraphrase rules containing nonterminal symbols),10 and require non-trivial
10Definition derived from Ganitkevitch et al. (2013).
background knowledge to identify whether they have the same meaning.
• 4: Paraphrases: Two questions fall into either lexical paraphrase, phrasal paraphrase, syntactic paraphrase, structured paraphrase, or other trivial paraphrase.
Table 8 presents the percentage and an example of each category. 76.9% questions have a rating of 2 or less, indicating that for most validation question, there are either no similar intention question in the training set, or there are questions with similar intention but either their literal meanings are different or their underlying assumptions are different. In particular, the latter indicates that whether or not there is a false presupposition can be different, even though they may share similar intention. Only 23.1% questions have a rating of 3 and above, indicating that relatively few questions in the validation set have a non-trivial or trivial paraphrase in the training set.
We also explored automatically filtering paraphrased questions using BLEU or TF-IDF. However, we find it is non-trivial to find the right threshold, thus include all questions and leave filtering to future work.

heading: B Details in Data Annotation
text: The annotation instruction is in Figure 2, and we show an example of the annotation interface in Figure 3. The data collection is approved by an Institutional Review Board.
Qualification Task. The qualification task contains 20 pre-annotated questions by the authors, and we provide examples as well as their explanation to workers to demonstrate our task. Based on both whether the worker can correctly identify false presupposition and the writing quality, we selected 30 qualified workers.
Generation Task. Qualified workers who passed the qualification task work on the main annotation task. Each question is assigned to two generators who independently annotate the label. We monitor the agreement of workers and send the disagreed cases to further validate. We revoke qualification of generators whose more than 10% of annotations marked to be invalid by the validators.
Validation Task. If two generators disagree on the label, their annotations are sent to two validators who judge their validity. We select a smaller
number of high qualified workers who have exceptional understanding of false presuppositions and are active users of Reddit. We find that for a small number of highly ambiguous cases, two validators have disagreement. In this case, we send the question to a third validator and take the majority vote.
Generators and validators are paid with reasonable hourly wage (13 USD/hour and 18 USD/hour, respectively).

heading: C Details in Experiments
text: C.1 Model details
Retrieval model. We use the English Wikipedia from 08/01/2019 provided by Petroni et al. (2021). We obtain c-REALM embeddings of Wikipedia passages as well as the questions. We use FAISS (Johnson et al., 2019) to do approximate maximum inner product search to retrieve the top-5 passages for the query.
Classifier model. For all the models in the GOLDCOMMENT track track, we use a per GPU batch size of 2, and for all the models in the main track,
Ambiguity in language Q When water boils, its bubbles are round, but when it freezes, its crystals are 6-sided. Why isn’t frozen water round or boiling water hexagonally shaped? Aren’t H2O molecules the same in either state? C Bubbles are round because (...) Ice crystals are shaped in such a way because (...) The water molecules are much slower and aren’t bouncing all over the place. Gaseous H2O is much higher energy and further apart so that the regular pattern of ice doesn’t come into effect.
Ambiguity in whether the presupposition is made Q How do executives hold board of director positions at multiple fortune 500 companies? C The Board only meets occasionally to vote on more important matters. They don’t really deal with day to day affairs like the CEO does.
Table 9: Two types of ambiguous cases. Q and C indicate the question and the comment, respectively.
we use the top-5 passages as the context and a per GPU batch size of 8. We train all our model with learning rate 10−5, weight decay of 0, and maximum sequence length of 256. We train all our models for 5000 steps and a gradient accumulation step of 5. We run the classifier models with 5 different random seeds and report the best result.
All experiments were done with two NvidiaRTX6000 GPUs (the detection subtask) or two Nvidia-A40 GPUs (the writing subtask). We use half-precision training (Micikevicius et al., 2018) when training for our detection task and gradient checkpointing from fairscale (authors, 2021), and choose the best checkpoint based on the performance on the validation set and report its test set performance. For the unified models for the writing subtask, we choose the best checkpoint that gives the best BLEU score for presupposition.
C.2 Error analysis on the detection subtask
We conduct error analysis of predictions from cREALM + MP classifier in the detection subtask. The authors annotate 50 false positive instances and 50 false negative instances on the validation set, sampled uniformly at random, and categorize the cause of errors.
Results are reported in in Table 10. Overall, most errors are due to failure in retrieval—although the model often successfully retrieves passages that are on right topic, it fails to retrieve passages that contain enough information about whether the backgrounded presuppositions are true or false. This issue is more prominent in false negatives, likely because finding that the presupposition is true requires significantly more exhaustive search than
noticing that the presupposition is false (since in
most cases, whether the presupposition is correct is not explicitly mentioned).
Secondly, there are 8% cases of labeling error in false positive cases. Note that for false positive cases, we did not distinguish between direct evidence and indirect evidence because there is no clear definition of “evidence” for non FP cases. For false negative cases, if there is a retrieved passage that can directly answer the question but the model fails in labeling, we consider this as a classification error rather than retrieving error, because the goal for retrieval is to retrieve passages that could give the answer to the question. The model also suffers more to classify given indirect evidence that requires reasoning, which matches our intuition.
Inherent disagreement and inconsistency issues contribute to the rest of the errors. We found that minor part (4%, 2% for false positive and false negative cases, respectively) of the error due to the inherent disagreements in labeling ambiguity. We also found that 2% of the false positive errors are due to whether the FP is critical to satisfy users information need based on the question. Furthermore, we also found that 14% of the errors are due to the comment and the retrieved passages are not inconsistent, causing label mismatch, however, they are not a labeling error because our ground truth annotator is not presented with additional passages when annotating. Example of these inconsistencies can be found in Table 11.
C.3 Discussion on inherent ambiguity and inconsistency on the web
Table 12 display examples for the category “Inherent disagreement: ambiguity”, “Inherent disagreement: criticalness”, and “Information on the web being inconsistent” from the human evaluation section of Section 5.2 and in Table 4.
9.1% of the errors is due to the inherent ambiguity of the criticalness of the FP, i.e., whether correcting the FP is critical to satisfy users’ information needed. For example, for the question “Why do things look darker when wet?” in Table 12, although our human rater found evidence on the internet that there exist things that look darker when dry, which would contradict the presupposition that (all) things look darker when wet, we believe that the question writer is mainly seeking an answer for the reason of the phenomenon that something looks darker when they are wet, and therefore, such FPis not critical to answer the users
original question, and therefore the comment writer does not point it out.
Note that this is different than the “exception” examples mentioned in Table 2, as the comment explicitly pointed out the falsehood of the presupposition, and therefore we consider the FP as critical to answer the user’s information seeking need.
Furthermore, 22.7% of the errors in human performance is due to information on the web being inconsistent. For the question “why do 78% of Americans live paycheque to paycheck?”, News on 06/07/2022 points out that 58% Americans lives paycheck to paycheck, while News on 01/11/2019 pointed out that 78% of Americans live paycheck to paycheck. For the question “why do pandas eat bamboo and when did they stop eating meat?”, creditable sources such as World Wide Fund(WWF) and Science says differently about whether panda eat meat or not.
34.1% of the errors are due to ambiguity, as analyzed in Section 3.3.
C.4 Details of the writing subtask
Example generations from each system are shown in Table 13, 14, and 15. As we discuss in Section 6.2, the unified model strongly prefers to generate the correction with prefix “It is not the case that”, or simply repeat or negate the presupposition, even though we do not restrict it to do so at inference time.
Details in Automatic Evaluation. We use sacreBLEU (Post, 2018) from the datasets library (Lhoest et al., 2021). We follow the preprocessing and evaluation script of unigram-F1 following Petroni et al. (2021), and the evaluation script of SentBERT following Reimers and Gurevych (2019).
Details in Human Evaluation. We conduct human evaluation for 200 questions on the test data. We recruit two student evaluators, who independently evaluate the presupposition and the correction given a question and its comment. They evalu-
Inputs given to the human raters Question: Why did scientists in the 1970s think that there was going to be a new ice age soon? Comment: They didn’t. Between 1965 and 1979, there was 7 papers talking about global cooling (not ice age and not necessarily soon). During the same period there was 44 papers about global warming. The media just liked the sensationalism, so there was some news article and a front page on the Times Magazine. They started with a minority of scientist talking about global cooling in a time period when there was still a lot of unknown in climate science and changed that to Scientific consensus that an Ice Age is coming soon. The 7 papers were the following : McComick and Ludwig 1967, Barrett 1971, Rasool and Schneider 1971, Hamilton and Seliga 1972, Chylek and Coakley 1974, Bryson and Dittbemer 1976 and Twomey 1977.
ated five outputs, including the reference in the data as well as generations from four systems in Section 6.1: the dedicated and the unified model from the GOLD-COMMENT track and the main track, re-
spectively.
We design detailed evaluation scheme, hold a 1 hour in-person tutorial session for human evaluators to be familiarized with the evaluation task.
Inputs given to the human raters Question: If the phases of the moon are due to the Earths shadow, why is the half moon a straight line/perfect semi circle, instead of a rounded shadow? Comment: Unless you’re looking at a lunar eclipse, the part of the moon that’s not lit up is not in the Earth’s shadow, it’s in the moon’s shadow. The bright portion is facing the sun, and is therefore illuminated, and the dark portion is facing away from the sun, and therefore not illuminated.
In particular, each output is rated based on four aspects as follows.
Fluency measures the fluency of the generated text, mainly whether it have repetitions, spelling errors or grammatical errors, or gibberish.
0: Generated text have fluency errors. 3: Generated text is free of fluency errors.
Presupposition evaluates whether the generated presupposition is the valid one in the question and whether it is factually false according to the comment.
0: The presupposition is invalid, i.e., does not exist in the question.
1: The presupposition is valid, e.g., exists in the question, but it is not factually false.
3: The presupposition is valid and is factually false.
Correction evaluates whether the generated correction provides a valid correction to the presupposition based on the comment with no hallucinated information, and provides enough justification (rather than simply being a negated presupposition). The former considers correctness (precision of the information), while the latter considers adequacy (recall of the information)
0: The correction is wrong based on the comment, or the correction is hallucinated.
1: The correction is correct based on the comment, but no additional information is provided to justify, or is a simple negation of the presupposition.
2: The correction is correct based on the comment, but misses some details to fully justify the falsehood of presupposition.
3: The correction is correct and provide enough information to justify the falsehood of presupposition.
Consistency requires the generated assumption and correction should be on the same topic and negate each other.
0: The presupposition and correction are not about the same topic.
1: The presupposition and correction are on the same topic, but they are not negating each other, or the negation is not explicit.
3: The presupposition and correction are consistent: are on the same topic and negate each other.
The evaluators are paid 17 USD/hour. See Section 6.3 for the results and discussion.

heading: D A case study with GPT-3
text: Large language models such as GPT-3 (Brown et al., 2020) have shown impressive performance
Q: How does flavored seltzer have 0 calories per serving but 10 calories per bottle? FP: If something has 0 calories per serving then it cannot have any calories in total. Corr: If a serving has under 5 calories, it can be rounded down to 0.
in generating a response to the question. We conduct small-scale evaluation of Instruct GPT3 (text-davinci-002), whose details are not public but is known as the best version of GPT-3. An example is depicted in Table 16. We find that most generations are roughly on the right topic,
e.g., all generations in Table 16 discuss Newton’s Law of Motion. However, they rarely correctly satisfy users information need:
• Most of them include information that is factually false, e.g., the equal and the opposite action are not instantaneous (1 4), their magnitude is not equal (2 6), or they do balance out (3).
• They are often not precisely about what the question is asking about. For instance, they discuss why an object may not move given a force, e.g., the wall does not move when you hit the wall (2 6). This is related to Newton’s Law of Motion, but not at all to the question.
• They do not explicitly identify false presuppositions. None of the generation mentions that the key misunderstanding is that the equal and opposite reaction apply to different objects, thus are not cancelled out. Sometimes the generation indicate some part of the question is wrong (indicates with ‘but’ or ‘However’) but does not precisely point out what is wrong, nor provide corrections.
It is possible performance could be improved with better prompting, but we leave this possibility to future work. We also experiment with ChatGPT in Mar 2023 and May 2023 in Table 17, but due to the closeness and the continuation of update, it is hard to evaluate and do an ablation study of the model, but we think the model is better than InstructGPT in generating answers that explicitly point out the false presupposition, but we leave the possibility to systematically evaluate this model on our task to future work.

heading: E False Presuppositions in Other Data
text: While we focus on questions from an online forum due to the availability of large unlabeled data and the domain being fairly general, we argue that false presuppositions are not specific to such domains. In fact, false presuppositions are more prevalent when the domain is specific and requires expertise.
We analyze 50 random samples of unanswerable questions from QASPER (Dasigi et al., 2021), a dataset consisting of information-seeking questions on NLP research papers, posed by NLP experts. We find that, out of 48 questions that are unanswerable (2 of them turn out to have valid answers), 25% of them has false presuppositions, because the question writer does not have sufficient background knowledge or misses facts in the research paper.
Table 18 shows a subset of such questions, along with false presuppositions and their correction annotated by the author. Identification of false presuppositions requires external knowledge beyond the specific paper the question writer is reading (1), or requires understanding of details of the paper which may not be explicitly written with the exact same terms in the paper (2 and 3).
It should be noted that this percentage is a strict lower bound of true percentage and may be significantly underestimated since identification of false presupposition requires knowledge in NLP; the estimate will be higher when annotated by multiple NLP experts. Moreover, presuppositions will significantly more prevalent when question writers are non-expertise, unlike in QASPER whose question writers are NLP experts.
We did not annotate and experiment with QASPER because the data is relatively small (272 and 62 unanswerable questions on the training set and on the validation set, respectively), but future work can investigate this data as well we domain transfer of models.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
In limitations, Appendix D.2, D.3
3 A2. Did you discuss any potential risks of your work? Appendix A
3 A3. Do the abstract and introduction summarize the paper’s main claims? Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3
3 B1. Did you cite the creators of artifacts you used? Section 3.1, Data source
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Appendix A. We will add license in our released github repository so that people will download the data with the license.
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Intend use of the data is clear from Section 1, 3, and 4.
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Appendix A: Details in Data Source includes how we remove toxic languages in the dataset. No personal information other than MTurk ID is collected in the data collection process.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Table 2, Section 1 L121-125.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Table 1
C 3 Did you run computational experiments? Section 5 and 6.
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Section 5.1, Section 5.2, Appendix D.1
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Appendix D.1
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Appendix D.1
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4, Appendix D.4
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 3, Appendix B
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Appendix D.4., Figure 2, Figure 3
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Appendix B, Appendix D.4
7 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? We recruit people and people need to opt-in to participate in annotation. All our annotators were aware of the intent of the annotations.
3 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Appendix B
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Appendix D.4

heading: Future Work
text: We did not annotate and experiment with QASPER because the data is relatively small (272 and 62 unanswerable questions on the training set and on the validation set, respectively), but future work can investigate this data as well we domain transfer of models. ACL 2023 Responsible NLP Checklist