CORRECT : Context- and Reference-Augmented Reasoning and Prompting for Fact-Checking (2025)

Abstract: Fact-checking the truthfulness of claims usually requires reasoning over multiple evidence sentences. Oftentimes, evidence sentences may not be always self-contained, and may require additional contexts and references from elsewhere to understand coreferential expressions, acronyms, and the scope of a reported finding. For example, evidence sentences from an academic paper may need contextual sentences in the paper and descriptions in its cited papers to determine the scope of a research discovery. However, most fact-checking models mainly focus on the reasoning within evidence sentences, and ignore the auxiliary contexts and references. To address this problem, we propose a novel method, Context- and Reference-augmented Reasoning and Prompting. For evidence reasoning, we construct a three-layer evidence graph with evidence, context, and reference layers. We design intra- and cross-layer reasoning to integrate three graph layers into a unified evidence embedding. For verdict prediction, we design evidence-conditioned prompt encoder, which produces unique prompt embeddings for each claim. These evidence-conditioned prompt embeddings and claims are unified for fact-checking. Experiments verify the strength of our model.

## introduction the proliferation of misinformation has posed growing challenge in the realm of information reliability. there is a need to develop automated factchecking methods @xcite to verify the truthfulness of real-world claims using evidence. existing fact-checking models @xcite have shown promise in aggregating and reasoning over multiple evidence sentences to verify a claim. however, the evidence sentences retrieved from a large corpus may contain incomplete information when they are taken claim: covid is the cause of long covid. claim: wearing masks prevents the spread of virus. evidence sentence from a paper: wearing mnc will effectively interrupt virus infections in the community. … contextual document from the same paper: most masks covering the mouth are named mouth nose covering (mnc) according to … long covid is an often debilitating illness that occurs in at least 10% of severe acute respiratory syndrome coronavirus 2 (sars-cov-2) infections. sars-cov-2, the virus that causes covid-19, replicates within the gut and acute covid-19 is associated with alteration of the gut microbiome. … evidence sentence from a paper referential document (e.g., cited paper, hyperlinked webpage) label: support (b) reference-dependent evidence label: support (a) context-dependent evidence out-of-corpus. we need to refer to additional contexts and references from elsewhere to understand coreferential expressions, acronyms, and the scope of a reported finding. for example, fig. 1 (a) illustrates context-dependent evidence, where undefined acronym "mnc" in evidence sentence from a paper abstract requires additional context from the abstract to jointly interpret the meaning of acronym "mnc". fig. 1(b ) presents reference-dependent evidence, where we need to check the cited paper to understand that "sars-cov-2 infection" and "covid-19 infection" are coreferential expressions, so that we could accurately fact-check the claim. such scenario also exists in general domain where evidence sentences from a wikipedia page may need contextual sentences in the same page and text in the hyperlinked pages to complement the insufficient information in the evidence. ## related work multi-hop fact-checking. complex claims usually require reasoning over multiple evidence sentences. many methods are based on language models @xcite and graph neural networks @xcite , such as gear @xcite , kgat @xcite , dream @xcite , sagp @xcite , decker @xcite , causalwalk @xcite , etc. however, they mainly focus on the reasoning within evidence sentences. they ignore the auxiliary contextual and referential documents. methods incorporating contextual documents are proposed, e.g., para-graphjoint @xcite , arsjoint @xcite , multivers @xcite , etc. some others integrating referential documents include transformer-xh @xcite and hesm @xcite . however, they incorporate either contextual or referential documents, but not both. in contrast, we construct a three-layer evidence graph to model evidence sentences, contexts, and references. there are fake news detection models where auxiliary graph with wikidata is used @xcite . fake news detection aims to detect the whole article with meta-data, while fact-checking focuses on claim sentences with retrieved evidence. some fact-checking works are based on retrievalaugmented generation @xcite . they unify evidence retrieval and claim verification as a joint approach, while our model mainly focuses on verification, and relies on external tool for evidence retrieval. our setting is consistent with existing works @xcite . prompt-based fact-checking. some models verify claims by prompting llms @xcite . protoco @xcite inputs both evidence sentences and claim to t5 @xcite . programfc @xcite decomposes complex claims into simpler sub-tasks and uses natural language to prompt llms. varifocal @xcite formulates fact-checking as question generation and answering. they rely on handcrafted natural language as prompt. the performance heavily relies on the choice of prompt, and it is difficult to design a prompt that produces a decent result, as shown in @xcite . our model is designed with learnable prompt embeddings where the prompting instruction is naturally learned by embeddings through optimization. prompt learning. prompting @xcite uses natural language as the input to language models to fulfill certain tasks. many prompting models have been proposed, including natural language prompt @xcite and prompt embeddings @xcite @xcite . prompting also benefits many tasks @xcite . however, no one has explored evidence sentence e's augmented embedding matrix π m,y the m-th prompt embedding for class y h m,y the m-th base prompt embedding for class y prompt embeddings for fact-checking. text-attributed graph. texts are usually connected in a graph structure, termed text-attributed graph @xcite . various methods have been developed to learn text embeddings in an unsupervised manner @xcite @xcite @xcite @xcite . though both our model and these works construct a text-attributed graph, our work is different from them, since our model is a supervised model for fact-checking. ## model architecture we introduce contextand reference-augmented reasoning and prompting for fact-checking (cor-rect). ## experiments we conduct extensive experiments and ablation analysis to evaluate the effectiveness of the proposed model correct. datasets. we use 4 datasets in baselines. we have 4 categories of baselines. i) multi-hop fact-checking, kgat @xcite , hesm (subramanian and lee, 2020), transformer-xh @xcite , multivers @xcite , and the recent causalwalk @xcite . multivers models contextual documents, and hesm and transformer-xh incorporate referential documents. by comparing to them, we highlight the advantage of threelayer graph for modeling both contextual and referential documents. since our model is built on transformer-xh, we further extend it by modeling both contextual and referential documents, and name it transformer-xh++. the comparison showcases the effect of evidence-conditioned prompting. implementation details. following @xcite , we set l to 12 and d to 768. number of prompt embeddings m is 8. temperature τ in eq. 10 is 100. for both our model and language model-based baselines, we initialize the model with pre-trained parameters in biomedical domain @xcite for scientific datasets, and in general domain @xcite for feverous-s. each result is obtained by 5 independent runs. experiments are done on 4 nvidia a100 80gb gpus. more details are in appendix c. we present two experimental settings below. fully supervised v.s. few-shot. for fully supervised setting, we train the model on the training set. if the dataset provides data split, we follow the split and obtain training and test sets. otherwise, we split the dataset into 80:20 for training and test, respectively. among training set, we further reserve 10% for validation. for few-shot setting, we report 5-shot experiments as the main results, i.e., for each class in the label set @xmath0, we randomly sample 5 instances from training set, obtaining 5 × |y| training instances. this setting is consistent with existing work @xcite . for a fair comparison, we sample instances using 5 random seeds. we keep the same sampling for our model and baselines. we report the results on test set. gold v.s. retrieved evidence. for gold evidence setting, we observe the ground-truth evidence sentences, and we verify the claim based on the gold sentences. for retrieved evidence setting, we do not observe any evidence sentences, and retrieve sentences from an evidence corpus, based on which we make prediction. we follow @xcite and use bm25 @xcite to retrieve top-3 evidence sentences for each claim. in the original check-covid dataset, if a claim is labeled as refute based on the evidence, this claim is reused in nei class with another random evidence. thus, there are two claims with the same content, but different evidence and labels. however, in our retrieved evidence setting, both claims will receive the same retrieved evidence, but they are labeled differently, making model training inconsistent. thus, for retrieved evidence setting, we remove claims in nei class for check-covid. ## conclusion we propose a context-and reference-augmented reasoning and prompting model for fact-checking. to model contextual and referential documents, we construct a three-layer graph with intra-and cross-layer reasoning. to integrate evidence into claims, we design evidence-conditioned prompting, which produces unique prompt embeddings for each claim. a future work is to extend three-layer graph to a multi-modal graph for fact-checking.