Evidence Retrieval is almost All You Need for Fact Verification (2024)

Abstract: Current fact verification methods generally follow the two-stage training paradigm: evidence retrieval and claim verification. While existing works focus on developing sophisticated claim verification modules, the fundamental importance of evidence retrieval is largely ignored. Existing approaches usually adopt the heuristic semantic similarity-based retrieval strategy, resulting in the task-irrelevant evidence and undesirable performance. In this paper, we concentrate on evidence retrieval and propose a Retrieval-Augmented Verification framework RAV, consisting of two major modules: the hybrid evidence retrieval and the joint fact verification. Hybrid evidence retrieval module incorporates an efficient retriever for preliminary pruning of candidate evidence, succeeded by a ranker that generates more precise sorting results. Under this end-to-end training paradigm, gradients from the claim verification can be back-propagated to enhance evidence selection. Experimental results on FEVER dataset demonstrate the superiority of RAV.

## introduction fact verification endeavors to locate and incorporate credible evidence to autonomously assess the veracity of the target textual statements @xcite . existing works @xcite @xcite generally adhere to a two-stage learning paradigm: evidence retrieval to identify a set of key evidential sentences from a large corpus, and claim verification which determines the authenticity of a claim based on the semantic interactions between the claim and the retrieved evidence @xcite . considerable effort has been devoted to the claim verification stage @xcite @xcite . however, solely advancing the claim verification might not be the panacea. the * correspondence to: xi @xmath0.edu.cn>. efficacy of verification is significantly contingent upon the quality of retrieved evidence @xcite . if the refereed evidence fails to provide accurate knowledge relevant to the claim, it would be intractable to achieve the correct decisions. despite the significance of evidence retrieval, existing models @xcite generally utilize a trivial retrieval strategy. all candidates are embedded into low-dimensional embeddings alongside the target claim. subsequently, the k-nearest neighbor (knn) search is employed to identify the top-k evidence based on their cosine similarity to the claim. however, such knn-based strategies suffer from two limitations. firstly, evidence that is most semantically similar to the claim may not be the most desirable for claim verification. figure 1 illustrates two example claims along with their top-4 evidence retrieved by existing methods. although semantically similar, most top candidates fail to provide clues to verify the statement in the claim. for example, while the first and second evidence of case 1 exhibits a high degree of semantic correlation with the claim, they lack the pivotal evidence of "aired". thus, simply selecting evidence based on heuristic semantic similarity may not be the optimal solution @xcite . secondly, evidence retrieval is independent from the claim verification, resulting in a separated training paradigm. the training signals from the verification loss cannot guide the update of text encoder used in the evidence retrieval. thus, the embeddings utilized for evidence retrieval may be taskirrelevant, resulting in undesirable performance. in this paper, we concentrate on the crucial yet often overlooked evidence retrieval stage, with the goal of introducing a novel end-to-end training paradigm. gradients from the claim verification could be back-propagated to the evidence retrieval stage to enhance evidence selection. in this manner, retrieved evidence is ensured to be task-relevant rather than merely similar to the claim. a straightforward strategy involves concatenating the claim with each candidate evidence and feeding them into a scoring neural network to obtain a closeness score. however, this method suffers from low efficiency due to its computational complexity of o(m * n), where m and n denote the number of claims and candidate evidence, respectively. given that the number of candidate evidence n is comparatively large, this complexity becomes impractical. to address the aforementioned challenges, we propose a novel retrieval-augmented verification framework, dubbed rav. rav consists of two major modules: the hybrid evidence retrieval and the joint fact verification. the hybrid evidence retrieval involves a retriever for preliminary filtering of candidate evidence, followed by a ranker for more precise sorting of the remaining evidence. the retriever is implemented as a bi-encoder @xcite , wherein the claim and evidence are processed by separate encoders. with the inference time complexity of o(m + n), the retriever's efficiency can be further optimized using approximate nearest neighbor techniques, making it wellsuited for large-scale retrieval tasks. the ranker is implemented as a cross-encoder @xcite , which offers higher precision by considering semantics from both the claim and evidence sides. in the joint fact verification module, the signals derived from the fact verification loss can be back-propagated to guide the updates of both the retriever and ranker. rav can be easily adapted to existing fact verification models with minor modifications and boost the verification performance, demonstrating its generality and superiority. ## related works the mainstream paradigm of fact-checking is "retrieve-then-verify", and most of the existing methods focus on the claim verification phase. as for claim verification, current researches primarily adopt three distinct methodologies: sequence inference, graph reasoning, and knowledge-enhanced approaches. for sequence inference, early works concatenate all pieces of evidence for feature analysis @xcite . however, most current efforts adopt a nuanced strategy that integrates each evidence with the claim @xcite @xcite , generating semantically enhanced claim-evidence pairs to facilitate deep interactions. for graph-based fact verification, the interplay among pieces of evidence is achieved via the mechanism of information dissemination across graph structures @xcite @xcite . depending on the granularity of node attributes, evidence graph can be delineated into sentence-level, tokenlevel, and multi-level. furthermore, in recent years, some works achieve the augmentation of evidence information through the integration of text description @xcite or external knowledge @xcite , thereby effectively improving the precision of fact verification. despite the substantial advancements achieved in the phase of claim verification, the majority of existing studies fail to optimize the evidence retrieval module during the training process a pre-trained classifier is utilized to reassess evidence from optimized retriever in reread @xcite , undergoing fine-tuning through joint training @xcite . however, this approach needs manually annotated gold evidence for supervisory information and fails to address the efficiency optimization in large-scale retrieval @xcite @xcite . evidence retrieval provides a data foundation for claim verification to ensure the accuracy of verification results. existing methods are mostly knnbased, which can be subdivided into evidence retrieval based on feature engineering @xcite , neural ranking @xcite , and pre-trained models @xcite . these strategies depend on specific heuristic techniques, yet the evidence thus gathered may not benefit the process of claim verification well. gere @xcite introduces a generative evidence retrieval mechanism to reduce computational resource consumption, but it remains an independent retrieval process. hence, it can be concluded that there remains an absence of a genuinely universal, end-to-end framework. ## methodology as depicted in figure 2 , the proposed rav comprises two phases: hybrid evidence retrieval and joint fact verification. given a claim c and n pieces of candidate evidence @xmath1, e 2 , . . . , e n }, the hybrid evidence retrieval selects the top q pieces of evidence (q ≪ n), while the fact verification stage endeavors to categorize the claim. ## conclusion in this paper, we delves into the critical issue of evidence retrieval and puts forth a joint fact verification framework featuring hybrid evidence retrieval. experimental results show that integrating rav with established claim verification models markedly boosts overall performance.