A Zero-Shot Claim Detection Framework Using Question Answering (2022)

Abstract: In recent years, there has been an increasing interest in claim detection as an important building block for misinformation detection. This involves detecting more fine-grained attributes relating to the claim, such as the claimer, claim topic, claim object pertaining to the topic, etc. Yet, a notable bottleneck of existing claim detection approaches is their portability to emerging events and low-resource training data settings. In this regard, we propose a fine-grained claim detection framework that leverages zero-shot Question Answering (QA) using directed questions to solve a diverse set of sub-tasks such as topic filtering, claim object detection, and claimer detection. We show that our approach significantly outperforms various zero-shot, few-shot and task-specific baselines on the NewsClaims benchmark (Reddy et al., 2021).

## introduction claim detection over news involves identifying claims related to various topics in news articles. identifying such claims is a crucial first step for fighting misinformation and disinformation online. however, such harmful content can evolve rapidly, triggered by relatively new events which can gain extensive media coverage within a short time span. hence, claim detection in such scenarios requires systems that are able to adapt quickly, by working well under zero-shot or few-shot settings. towards this goal, @xcite propose a new benchmark, newsclaims, that evaluates claim detection for previously unseen topics without access to any training data. given a collection of news articles, the task involves identifying claims that are related to a pre-defined set of topics, along with extracting attributes for each claim, such as claim span, claim object, claimer figure 1 : an overview of our claim detection framework. we leverage zero-shot qa for claim topic filtering, claim object detection and claimer detection for the claims spotted by claimbuster @xcite . and stance. essentially, this benchmark extends the claim detection task to involve extracting additional background attributes relating to the claim, such as the claim object and claimer. furthermore, the claimer detection sub-task within newsclaims requires considerable document-level reasoning, making it harder than existing attribution tasks @xcite , which mainly involve sentence-level reasoning. @xcite propose various baselines for each sub-task in the newsclaims benchmark, which involve zeroshot and few-shot approaches in addition to baselines trained using task-specific data. to handle this scenario in a low-resource setting, we hypothesize that identifying claim topics and extracting corresponding claim attributes can be formulated as a question answering task. hence, we propose to leverage a question answering (qa) system for the claim detection task and show that the same qa model can be used to solve multiple sub-tasks within claim detection, without the need of any task-specific training data. this involves filtering claims relating to specific topics, identifying claim objects associated with such topics and attribution for identifying the claimers making these claims. we realise this by using directed questions to help solve connected sub-tasks such as topic filtering, claim object detection and claimer detection within the claim detection framework. an overview of this framework is shown in figure 1 . leveraging pre-trained language models for directly solving end-tasks has been explored in prompting @xcite , with promising performance in both zero-shot @xcite and few-shot (brown et al., 2020) settings. prior work uses prompts that are in the form of prefix (li and liang, 2021) or cloze-style tasks @xcite . in this work, we solve end-tasks by formulating them as directed questions, instead of prompts, that are fed into a qa model (see figure 1 1 for examples of directed questions). we will first briefly introduce the newsclaims benchmark (section 2) and then describe our zero-shot claim detection framework (section 3). our main contributions are: (1) we propose to use a single pre-trained question answering system in a zero-shot setting for various sub-tasks in claim detection, such as topic-filtering, claim object detection and claimer detection; (2) we show that, using directed questions, a qa model is able to outperform other attribution methods for claimer detection, which requires document-level reasoning; and (3) our proposed claim detection framework achieves state-of-the-art performance on multiple sub-tasks in the newsclaims benchmark, outperforming various zero-shot, few-shot and taskspecific baselines. ## newsclaims background newsclaims @xcite extends claim detection to extract additional background attributes relating to the claim, such as claim objects and claimers. newsclaims evaluates claim detection in the context of an emerging real-world scenario, by considering claims relating to various aspects of covid-19. specifically, the topics involved in the benchmark are about the origin of the virus, transmission of the virus, cure for the virus and protection from the virus. we refer the reader to @xcite for detailed definitions and sample claims. below, we briefly describe the different sub-tasks that we consider within this benchmark and their corresponding baselines, that were introduced in @xcite . ## method we first describe the qa model which we use as the pre-trained model to feed directed questions as input. next, we outline our zero-shot claim detection framework which leverages the above model for sub-tasks such as topic filtering, claim object detection and claimer detection. ## conclusions and future work we propose a new claim detection framework that leverages zero-shot qa with directed questions for various sub-tasks such as topic filtering, claim object detection and claimer detection. we show that these questions can be adept at identifying topic relevance for claims related to covid-19. we demonstrate that qa can be leveraged for claimer detection with document-level attribution, while considerably outperforming attribution systems that can be limited by sentence-level reasoning. future work involves building a unified model that can extract claims and corresponding attributes together, without the need for separate components for each individual attribute.