Won’t Get Fooled Again: Answering Questions with False Premises (2023)

Abstract: Pre-trained language models (PLMs) have shown unprecedented potential in various fields, especially as the backbones for question-answering (QA) systems. However, they tend to be easily deceived by tricky questions such as “How many eyes does the sun have?”. Such frailties of PLMs often allude to the lack of knowledge within them. In this paper, we find that the PLMs already possess the knowledge required to rebut such questions, and the key is how to activate the knowledge. To systematize this observation, we investigate the PLMs’ responses to one kind of tricky questions, i.e., the false premises questions (FPQs). We annotate a FalseQA dataset containing 2365 human-written FPQs, with the corresponding explanations for the false premises and the revised true premise questions. Using FalseQA, we discover that PLMs are capable of discriminating FPQs by fine-tuning on moderate numbers (e.g., 256) of examples. PLMs also generate reasonable explanations for the false premise, which serve as rebuttals. Further replaying a few general questions during training allows PLMs to excel on FPQs and general questions simultaneously. Our work suggests that once the rebuttal ability is stimulated, knowledge inside the PLMs can be effectively utilized to handle FPQs, which incentivizes the research on PLM-based QA systems. The FalseQA dataset and code are available at https://github.com/thunlp/FalseQA .

## introduction recent advances in pre-trained language models (plms) @xcite @xcite @xcite have achieved significant performance gains for various types of tasks, even surpassing human levels on language ability benchmarks @xcite @xcite . the question answer sally's favorite cow died yesterday. when will the cow be alive again? 1 in a few days. how many eyes does the sun have? 2 the sun has one eye. without careful investigation, this paradox could easily lead to the conclusion that plms lack the world or commonsense knowledge to rebut these questions. although it's crucial for the plms to embed as much general knowledge as possible, we provide a pilot experiment to find out that the plms already possess the knowledge required for the tricky questions which they fail (see section 3.2). as a consequence, we hypothesize that the knowledge in current plms is enough for handling a large portion of tricky questions. however, this knowledge is not activated. to support our hypothesis, we take a close look at these tricky questions. most of these tricky questions contain false premises. for example, in the question "how many eyes does the sun have?", the questioner must presume that "the sun can have eyes" in order to make the query about the quantity meaningful. these questions are called false premise questions @xcite . such false premises always violate human knowledge or logic and rarely appear in the natural text, thus leading to an out-ofdistribution generalization gap for the plms. targeting to fill the gap between the natural text and fpqs, we present the first specialized dataset of fpqs, dubbed as falseqa dataset. specifically, we first systematically categorize the false premises to ensure the coverage of the dataset. then we ask human annotators to manually compose the fpqs, as well as explanations for the false premises. the annotators are also asked to edit the false premise questions into true premise questions (tpqs) using minimal modification, with which the plms are less prone to learn shortcuts from the format of fpqs. based on falseqa dataset, we first conduct systematic experiments on the plms' discrimination and rebuttal ability of fpqs. we reach three essential conclusions: (1) plms of different types and scales can distinguish the fpqs from tpqs, and scaling effect @xcite also holds for falseqa. (2) plms can give reasonable explanations for the false premises, which can serve as rebuttals. ## related work three groups of research are related to our work: direct question answering dataset, question unanswerability, and question premise verification. ## preliminaries in this section, we introduce the definition of fpq and the pilot experiment on plms about fpqs. ## dataset to build a dataset on fpqs, there are potentially two approaches. an approach is to collect them from natural corpora. however, false premise questions rarely appear in natural corpora, which makes the question collection process laborious. second, even if we collect false premise questions, the false premises are made by humans and thus are hard to be detected by humans, which doesn't fit with the motivation of this paper. in fact, @xcite have done pioneering work using this approach. on the contrary, our approach is to manually write such false premise questions. to ensure the quality of our dataset, we expect falseqa dataset to have the following key features: broad coverage, high quality, few shortcuts, and detailed explanations for the false premises. below we introduce the annotation steps that ensure these features. ## experiments our experiments are divided into two main parts. to begin with, we conducted extensive experiments to demonstrate that plms have the ability to discriminate and rebut fpqs with moderate training data. next, we propose a practical method to handle both fpqs and general questions well. ## conclusion in this paper, we investigate using plms to answer fpqs, which are simple for humans but deceive most plms. we present the first human-written dataset of fpqs. using the dataset, we successfully activate the discrimination and explanation ability of plms and produce plms that are both capable of general questions and robust to fpqs. for future directions, we think that more advanced techniques can be used together with falseqa to fully activate the model's ability, e.g., reinforcement learning with human feedbacks @xcite . incorporating more knowledge into plms is also beneficial for plms to answer fpqs.