FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering
Year: 2023

Abstract:
Automatic fact verification has received significant attention recently. Contemporary automatic fact-checking systems focus on estimating truthfulness using numerical scores which are not human-interpretable. A human factchecker generally follows several logical steps to verify a verisimilitude claim and conclude whether it’s truthful or a mere masquerade. Popular fact-checking websites follow a common structure for fact categorization such as half true, half false, false, pants on fire, etc. Therefore, it is necessary to have an aspectbased (delineating which part(s) are true and which are false) explainable system that can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final verdict. In this paper, we propose a 5W framework (who, what, when, where, and why) for question-answer-based fact explainability. To that end, we present a semi-automatically generated dataset called FACTIFY-5WQA, which consists of 391, 041 facts along with relevant 5W QAs – underscoring our major contribution to this paper. A semantic role labeling system has been utilized to locate 5Ws, which generates QA pairs for claims using a masked language model. Finally, we report a baseline QA system to automatically locate those answers from evidence documents, which can serve as a baseline for future research in the field. Lastly, we propose a robust fact verification system that takes paraphrased claims and automatically validates them. The dataset and the baseline model are available at https: //github.com/ankuranii/acl-5W-QA 1 Fact checking demands aspect-based explainability Manual fact-checking is a time-consuming task. To assess the truthfulness of a claim, a journalist would either need to search online, offline, or both, browsWork does not relate to the position at Amazon. ing through a multitude of sources while also accounting for the perceived reliability of each source. The final verdict can then be obtained via assimilation and/or comparison of the facts derived from said sources. This process can take professional fact-checkers several hours or days (Hassan et al., 2019) (Adair et al., 2017), depending on the inherent complexity of the claim. There are several contemporary practices that journalists use for the manual verification of a claim. These methods can be categorized into four broad categories (Posetti et al., 2018): 1. Research and fact-checking: This involves carefully researching the claim and verifying its accuracy using reliable and credible sources such as news services, academic studies, and government data. 2. Interviews and expert opinions: This involves speaking with experts in the relevant field and asking for their opinions on the claim to see if it is supported by evidence and expertise. 3. Cross-checking with multiple sources: This involves comparing the claim with information from multiple sources to see if it is consistent or triangulates the facts obtained via multiple sources. 4. Verifying the credibility of sources: This involves checking the credibility of the sources used to support the claim, such as ensuring that they are reliable and unbiased. Overall, these methods can help journalists to carefully verify claims and ensure that they are accurate and supported by evidence. However, this process is tedious and hence time-consuming. A system that can generate relevant question-answer sets by dissecting the claim into its constituent components for a given verisimilitude claim could be a great catalyst in the fact-checking process. Research on automatic fact-checking has recently received intense attention (Yang et al.,

heading: Future Work
text: It is optimized through n-step ahead prediction, which predicts the next n tokens based on previous context tokens at each time step, encouraging the model to explicitly plan for future tokens. In this work, we employed the contextbased question generation approach to generate relevant and specific questions for the task of fact verification. This approach utilizes the claim information to ensure that the generated questions are appropriate for fact-checking. 5. 1 Human evaluation of QA generation
For the evaluation purpose, a random sample of 3000 data points was selected for annotation. The questions generated using the Prophetnet model were utilized for this purpose. The annotators were instructed to evaluate the question-answer pairs in three dimensions: the question is well formed,
which means it is syntactically correct, the question is correct which means it is semantically correct with respect to the given claim, and extracted answer from the model is correct. The evaluation results for the datasets are presented in the following analysis. 6 The 5W QA validation system
Finally, we propose a QA validation system, where the generated questions from the QG system and the evidence are passed through SoTA Question answering models (T5:3B) (Raffel et al. , 2020), T5:Large (Raffel et al. , 2020), Bert: Large (Devlin et al. , 2018)) demonstrated in figure 9