P re W o M e: Exploiting Presuppositions as Working Memory for Long Form Question Answering (2023)

Abstract: Information-seeking questions in long-form question answering (LFQA) often prove misleading due to ambiguity or false presupposition in the question. While many existing approaches handle misleading questions, they are tailored to limited questions, which are insufficient in a real-world setting with unpredictable input characteristics. In this work, we propose PreWoMe, a unified approach capable of handling any type of information-seeking question. The key idea of PreWoMe involves extracting presuppositions in the question and exploiting them as working memory to generate feedback and action about the question. Our experiment shows that PreWoMe is effective not only in tackling misleading questions but also in handling normal ones, thereby demonstrating the effectiveness of leveraging presuppositions, feedback, and action for real-world QA settings.

## introduction answering information-seeking long-form questions has recently shown significant progress @xcite @xcite . however, users' questions in the real-world may often mislead language models (lms) to output misinformation. for example, users ask questions with false presuppositions (fp) @xcite which can induce hallucinations if lms believe the presuppositions are true @xcite . also, users often ask questions that do not have a single and clear answer, i.e., an ambiguous question, which is difficult for lms to identify @xcite . figure 1 shows examples of two types of misleading questions: (a) an ambiguous question that requires considering prior knowledge ("two different wars between italy and ethiopia") to answer properly, and (b) a question with fp ("somebody won the nobel peace prize for dna structure"), which should be corrected before answering the question @xcite . desirable answers to misleading questions require discerning and resolving misleadings. a previous work by amplayo et al. (2022) focuses on ambiguous questions, by introducing query refinement prompts that encourage lms to consider multiple facets of the question. another work @xcite tackles questions containing fp, by extracting and verifying presuppositions. while such works assume only one specific type of misleading factor, in a real scenario, the users ask various types of questions, which can be misleading or normal (i.e., non-misleading). 1 this makes prior singletype-tailored approaches hard to be deployed in real-world qa settings. in this paper, we emphasize the role of presuppositions in handling any type of information-seeking questions in a unified way. presuppositions are crucial building blocks of questions and play an essential role in understanding the meaning of the question @xcite . this suggests that exploiting presuppositions is promising in handling misleading questions, while generally applicable to any type of question. based on this, we propose a new approach, prewome (presupposition as working memory), that handles any type of information-seeking question in a unified way without any parameter updates. the main idea of prewome is to extract presuppositions and use them as working memory to generate analysis and directions for answering the question. our contributions can be summarized as follows: in this paper, we suggest using presuppositions to solve an information-seeking qa task. ## prewome we introduce prewome, a new approach that answers any type of long-form information-seeking question in a unified way. prewome follows three intermediate steps, as illustrated in figure 2 . ## experiments and results datasets we evaluate prewome on three distinct types of questions. for misleading questions with ambiguity, we utilize asqa @xcite , a lfqa dataset specifically curated for answering ambiguous questions. for misleading questions with false presupposition (fp), we use the subset of (qa) 2 @xcite that are labeled as harboring fp. for normal questions, we use bioasq @xcite , considering that questions in bioasq are unlikely to be ambiguous or have fp as biomedical experts have carefully curated the questions. detailed statistics for each dataset are in appendix c.1. ## conclusion we propose prewome, a new approach that handles any type of question in a unified way by exploiting presuppositions as working memory. prewome uses intermediate steps to guide the model to generate presuppositions along with useful feedback and action for itself. our experiments show that prewome boosts the performance of llms on different types of questions without parameter updates. this is achieved even though the model is given no information about the type of incoming question, demonstrating that prewome is effective for a real-world setting and will be a significant stepping stone to future works tackling real-world qa tasks.