D e S e P tion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking (2020)

Abstract: The increased focus on misinformation has spurred development of data and systems for detecting the veracity of a claim as well as retrieving authoritative evidence. The Fact Extraction and VERification (FEVER) dataset provides such a resource for evaluating endto- end fact-checking, requiring retrieval of evidence from Wikipedia to validate a veracity prediction. We show that current systems for FEVER are vulnerable to three categories of realistic challenges for fact-checking – multiple propositions, temporal reasoning, and ambiguity and lexical variation – and introduce a resource with these types of claims. Then we present a system designed to be resilient to these “attacks” using multiple pointer networks for document selection and jointly modeling a sequence of evidence sentences and veracity relation predictions. We find that in handling these attacks we obtain state-of-the-art results on FEVER, largely due to improved evidence retrieval.

## introduction the growing presence of biased, one-sided, and often altered discourse, is posing a challenge to our media platforms from newswire to social media @xcite . to overcome this challenge, fact-checking has emerged as a necessary part of journalism, where experts examine "check-worthy" claims @xcite published by others for their "shades" of truth (e.g., factcheck.org or poli-tifact). however, this process is time-consuming, and thus building computational models for automatic fact-checking has become an active area of research @xcite . advances were made possible by new open source datasets and shared tasks: the fact extraction and verification shared task (fever) 1.0 and 2.0 @xcite ; thorne and vlachos, 2019 ), semeval 2019 shared task 8: fact-checking in community forums @xcite , and liar(+) datasets with claims from politifact @xcite . the fever 1.0 shared task dataset @xcite has enabled the development of endto-end fact-checking systems, requiring document retrieval and evidence sentence extraction to corroborate a veracity relation prediction (supports, refutes, not enough info). an example is given in figure 1 . since the claims in fever 1.0 were manually written using information from wikipedia, the dataset may lack linguistic challenges that occur in verifying naturally occurring check-worthy claims, such as temporal reasoning or lexical generalization/specification. @xcite designed a second shared task (fever 2.0) for participants to create adversarial claims ("attacks") to break state-of-the-art systems and then develop systems to resolve those attacks. we present a novel dataset of adversarial examples for fact extraction and verification in three challenging categories: 1) multiple propositions (claims that require multi-hop document or sentence retrieval); 2) temporal reasoning (date comparisons, ordering of events); and 3) named entity ambiguity and lexical variation (section 4). we show that state-of-the-art systems are vulnerable to adversarial attacks from this dataset (section 6). in addition, we take steps toward addressing these vulnerabilities, presenting a system for endto-end fact-checking that brings two novel contri-butions using pointer networks: 1) a document ranking model; and 2) a joint model for evidence sentence selection and veracity relation prediction framed as a sequence labeling task (section 5). our new system achieves state-of-the-art results for fever and we present an evaluation of our models including ablation studies (section 6). data and code will be released to the community. foot_0 2 related work approaches for predicting the veracity of naturallyoccurring claims have focused on statements factchecked by journalists or organizations such as politifact.org @xcite , news articles @xcite , or answers in community forums @xcite . however, those datasets are not suited for end-to-end fact-checking as they provide sources and evidence while fever @xcite requires retrieval. initial work on fever focused on a pipeline approach of retrieving documents, selecting sentences, and then using an entailment module @xcite @xcite ; the winning entry for the fever 1.0 shared task @xcite used three homogeneous neural models. other work has jointly learned either evidence extraction and question answering @xcite or sentence selection and relation prediction @xcite ; unlike these approaches, we use the same sequential evidence prediction architecture for both document and sentence selection, jointly predicting a sequence of labels in the latter step. more recently, @xcite proposed a graph-based framework for multi-hop retrieval, whereas we model evidence sequentially. language-based adversarial attacks have often involved transformations of the input such as phrase insertion to distract question answering systems @xcite or to force a model to always make the same prediction @xcite . other research has resulted in adversarial methods for paraphrasing with universal replacement rules @xcite or lexical substitution @xcite . while our strategies include insertion and replacement, we focus specifically on challenges in factchecking. the task of natural language inference @xcite provides similar challenges: examples for numerical reasoning and lexical inference have been shown to be difficult @xcite and improved models on these types are likely to be useful for fact-checking. finally, @xcite provided a baseline for the fever 2.0 shared task with entailment-based perturbations. other participants generated adversarial claims using implicative phrases such as "not clear" @xcite or gpt-2 @xcite . in comparison, we present a diverse set of attacks motivated by realistic, challenging categories and further develop models to address those attacks. ## problem formulation and datasets we address the end-to-end fact-checking problem in the context of fever @xcite , a task where a system is required to verify a claim by providing evidence from wikipedia. to be successful, a system needs to predict both the correct veracity relation-supported (s), refuted (r), or not enough information (nei)-and the correct set of evidence sentences (not applicable for nei). the fever 1.0 dataset @xcite was created by extracting sentences from popular wikipedia pages and mutating them with paraphrases or other edit operations to create a claim. then, each claim was labeled and paired with evidence or the empty set for nei. overall, there are 185,445 claims, of which 90,367 are s, 40,107 are r, and 45,971 are nei. @xcite introduced an adversarial set up for the fever 2.0 shared task -participants submitted claims to break existing systems and a system designed to withstand such attacks. the organizers provided a baseline of 1000 adversarial examples with negation and entailment-preserving/-altering transformations and this set was combined with examples from participants to form the fever 2.0 dataset. ## adversarial dataset for fact-checking while the fever dataset is a valuable resource, our goal is to evaluate complex adversarial claims which resemble check-worthy claims found in news articles, speeches, debates, and online discussions. we thus propose three types of attacks based on analysis of fv1 or prior literature: those using multiple propositions, requiring temporal and numerical reasoning, and involving lexical variation. for the multi-propositional type, graves (2018) notes that professional fact-checking organizations need to synthesise evidence from multiple sources; automated systems struggle with claims such as "lesotho is the smallest country in africa." in fv1dev, 83.18% of s and r claims require only a single piece of evidence and 89% require only a single wikipedia page. furthermore, our previous work on fever 1.0 found that our model can fully retrieve 86% of evidence sentences from wikipedia when only a single sentence is required, but the number drops to 17% when 2 sentences are required and 3% when 3 or more sentences are required @xcite . for the second type, check-worthy claims are often numerical @xcite and temporal reasoning is especially challenging @xcite . @xcite and @xcite showed that numbers and comparatives are indicative of truthful statements in news, but the presence of a date alone does not indicate its veracity. in fv1-dev, only 17.81% of the claims contain dates and 0.22% contain time information. 2 to understand how current systems perform on these types of claims, we evaluated three stateof-the-art systems from fever 1.0 @xcite @xcite , and examined the predictions where the systems disagreed. we found that in characterizing these predictions according to the named entities present in the claims, the most frequent types were numerical and temporal (such as percent, money, quantity, and date). finally, adversarial attacks for lexical variation, where words may be inserted or replaced or changed with some other edit operation, have been shown to be effective for similar tasks such as natural language inference @xcite and question answering @xcite , so we include these types of attacks as well. for the fact-checking task, models must match words and entities across claim and evidence to make a veracity prediction. as claims often contain ambiguous entities @xcite or lexical features indicative of credibility @xcite , we desire models resilient to minor changes in entities @xcite and words @xcite . we thus create an adversarial dataset of 1000 examples, with 417 multi-propositional, 313 temporal and 270 lexically variational. representative examples are provided in appendix a. ## methods verifying check-worthy claims such as those in section 4 requires a system to 1) make sequential decisions to handle multiple propositions, 2) support temporal reasoning, and 3) handle ambiguity and complex lexical relations. to address the first requirement we make use of a pointer network @xcite in two novel ways: i) to rerank candidate documents and ii) to jointly predict a sequence of evidence sentences and veracity relations in order to compose evidence (figure 3 ). to address the second we add a post-processing step for simple temporal reasoning. to address the third we use rich, contextualized representations. specifically, we fine-tune bert @xcite as this model has shown excellent performance on related tasks and was pre-trained on wikipedia. our full pipeline is presented in figure 2 . we first identify an initial candidate set of documents (1a) by combining the top m pages from a tf-idf search using drqa @xcite with pages from the approach of @xcite , which provides results from google search and predicted named entities and noun phrases. then, we perform document ranking by selecting the top @xmath0. next, an n -long sequence of evidence sentences (2) and veracity relation labels (3) are predicted jointly by another pointer network. prior to training, we fine-tune bert for document and sentence ranking on claim/title and claim/sentence pairs, respectively. each claim and evidence pair in the fever 1.0 dataset has both the title of the wikipedia article and at least one sentence associated with the evidence, so we can train on each of these pairs directly. for the claim "michelle obama's husband was born in kenya", shown in figure 3 , we obtain representations by pairing this claim with evidence sentences such as "obama was born in hawaii" and article titles such as [barack obama]. the core component of our approach is the pointer network, as seen in figure 3 . unlike our previous work @xcite , we use the pointer network to re-rank candidate documents and jointly predict a sequence of evidence sentences and relations. given a candidate set of evidence (as either document titles or sentences) and a respective fine-tuned bert model, we extract features for every claim c and evidence e p pair by summing the [cls] embedding for the top 4 layers (as recommended by @xcite ): next, to select the top k evidence, we use a pointer network over the evidence for claim c to extract evidence recurrently by computing the extraction probability p (p t |p 0 • • • p t-1 ) for evidence e p at time @xmath1. at time t, we update the hidden state z t of the pointer network decoder. then we compute the weighted average h q t of the entire evidence set using q hops over the evidence @xcite we concatenate m p and h q t and use a multi-layer perceptron (mlp) to predict p t . the loss is then: we train on gold evidence and perform inference with beam search for both document ranking (section 5.1) and joint sentence selection and relation prediction (section 5.2). ## experiments and results we evaluate our dataset and system as part of the fever 2.0 shared task in order to validate the vulnerabilities introduced by our adversarial claims (section 4) and the solutions proposed by our system (section 5). we train our system on fv1-train and evaluate on fv1/fv2-dev/test (section 3). we report accuracy (percentage of correct labels) and recall (whether the gold evidence is contained in selected evidence at @xmath2. we also report the fever score, the percentage of correct evidence sentences (for s and r) that also have correct labels, and potency, the inverse fever score (subtracted from one) for evaluating adversarial claims. our baseline-rl: for baseline experiments, to compare different loss functions, we use the approach of @xcite for document selection and ranking, the reinforcement learning (rl) method of chen and bansal (2018) for sentence selection, and bert @xcite for relation prediction. the rl approach using a pointer network is detailed by @xcite for extractive summarization, with the only difference that we use our fine-tuned bert on claim/gold sentence pairs to represent each evidence sentence in the pointer network (as with our full system) and use the fever score as a reward. the reward is obtained by selecting sentences with the pointer network and then predicting the relation using an mlp (updated during training) and the concatenation of all claim/predicted sentence representations with their maximum/minimum pooling. hyper-parameters and settings for all experiments are detailed in appendix b. ## conclusion we showed weaknesses in approaches to factchecking via novel adversarial claims. we took steps towards realistic fact-checking with targeted improvements to multi-hop reasoning (by a document pointer network and a pointer network for sequential joint sentence selection and relation pre-