Fact Checking Beyond Training Set (2024)

Abstract: Evaluating the veracity of everyday claims is time consuming and in some cases requires domain expertise. We empirically demonstrate that the commonly used fact checking pipeline, known as the retriever-reader, suffers from performance deterioration when it is trained on the labeled data from one domain and used in another domain. Afterwards, we delve into each component of the pipeline and propose novel algorithms to address this problem. We propose an adversarial algorithm to make the retriever component robust against distribution shift. Our core idea is to initially train a bi-encoder on the labeled source data, and then, to adversarially train two separate document and claim encoders using unlabeled target data. We then focus on the reader component and propose to train it such that it is insensitive towards the order of claims and evidence documents. Our empirical evaluations support the hypothesis that such a reader shows a higher robustness against distribution shift. To our knowledge, there is no publicly available multi-topic fact checking dataset. Thus, we propose a simple automatic method to re-purpose two well-known fact checking datasets. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models, including recent domain adaptation models that use GPT4 for generating synthetic data.

## introduction fact checking is the process of applying a veracity rating to a particular statement or claim @xcite . automatic fact checking is beneficial for curbing misinformation, and also for supporting professional human fact-checkers @xcite . due to the growing size of the web, this task is increasingly becoming more challeng-ing. the performance of fact checking systems depends on the availability of evidence resources, and this makes the large commercial language models, in many cases, unsuitable for the task. we demonstrate this by reporting an experiment in figures 1a and 1b . we see that two existing large language models, i.e., chatgpt and gpt 4, are unable to verify a simple claim regarding the former president of the us. at the time of carrying out this experiment, the indictment of donald trump was widely being discussed on news outlets, such as the associate press, foot_1 and the social media websites. another factor that contributes to the difficulty of the task is the technological requirements. more specifically, existing automated fact checking systems rely on a pipeline of components to retrieve evidence documents and to infer the final verdict @xcite . as stated by @xcite , coordinating the components within such a pipeline presents an extra challenge. given these challenges, it is desirable to know how much a trained fact checking pipeline generalizes across domains. to our knowledge, little to no work has been done to investigate this area. there exist a few studies @xcite @xcite that report experiments on the transferability of the fact checking pipeline across various platforms, e.g., from wikipedia to scientific repositories, or from a fact checking website to another one. while these are valuable observations, these platforms, e.g., wikipedia and the scientific repositories, have shared domains and topics. 3 this can potentially distort certain reported conclusions, particularly, the conclusions on the generalizability of the pipeline to unseen scenarios. additionally, the solution proposed by these studies to enhance generalization is very limited. they primarily rely on pretraining the pipeline in one domain, and then, using it in another domain. in the next section, we provide a background on the fact checking pipeline. we then report a case study to show that a pipeline trained on outof-domain data is not as competitive as the one trained on in-domain data. we continue our study by focusing on the two primary components of the pipeline, i.e., the retriever and the reader, and propose two novel algorithms to enhance their performance. particularly, we use a bi-encoder dense retrieval model as the retriever, and propose an adversarial algorithm to enhance its robustness under distribution shift. we then exploit a previously unknown weakness of language models in detecting the reversal relationship between input statements, and propose an augmentation algorithm to provide the reader with more cues. to evaluate our pipeline, we use a public api to re-purpose the snopes @xcite and the multifc @xcite fact checking datasets. we extract eight fact checking scenarios out of these two datasets, and compare our proposed components individually to the stateof-the-art domain adaptation techniques, including the ones that exploit gpt 4. we also demonstrate that our entire fact checking pipeline outperforms the alternative pipelines that use these techniques. in summary, our contributions are threefold: ## preliminaries background. existing fact checking systems @xcite primarily rely on two components: 1) a document retrieval model, called "retriever", and 2) a veracity prediction model, called "reader". see figure 2a for an illustration. the retriever views the input claim as a query and returns the top evidence documents that are deemed relevant to the claim-the search is usually performed over a pre-indexed corpus. as the reader, existing studies usually train a classifier over the concatenation of the retrieved documents and the given claim @xcite . 5 as stated by @xcite and @xcite , the veracity prediction step resembles the natural language inference task (nli). the output of the veracity prediction component can be the word "support" or the word "refute"-depending on the system design, a third candidate output can be also added as "neutral". as it can be seen, developing, scaling up, and maintaining a fact checking system involves a lot of expertise, time, and budget. on the other hand, when such a system is deployed, even a small deterioration or improvement in performance can have profound impacts. detecting an unsupported claim early enough, and then, taking timely actions on the media can be invaluable. therefore, it is crucial to know if such a system is generalizable. in other words, if a model trained on the labeled data from one domain (i.e., source domain), demonstrates the same efficacy if it is used to verify the claims in another domain (i.e., target domain). in order to answer this question, below, we report a case study. setup. we compare the performance of indomain fact checking compared to out-of-domain fact checking across two domains of "miscellany" and "politics". each domain in this experiment has 7,900 claims, and each claim has two evidence documents. in each domain, 60% of data was used for training and 40% for testing. the claims are labeled either as support or refute. in this experiment, the retriever is a bi-encoder @xcite pretrained using the algorithm proposed by @xcite . the reader is a roberta-based model @xcite pretrained on the snli and multinli datasets @xcite . apart from these pretraining steps, all the models are finetuned in the source domains (using the labeled data), and then, evaluated on the target domain. we assume the target domain has no labeled data during the training. the target labels are used only for evaluation. we report macro f1 for the classification tasks and ndcg@10 for the ranking tasks. observations. figure 2b reports the performance of the pipeline in the in-domain scenarios compared to the out-of-domain scenarios. we see that the performance in both of the out-of-domain scenarios (i.e., @xmath0. this raises the question about the root of this performance deterioration. to reveal the cause, we report the performance of each underlying component in isolation. to evaluate the performance of the reader in isolation, we assume that the retriever perfectly returns all the relevant evidence documents. figures 2c and 2d report the results. we see the same trend in both experiments. both components suffer from distribution shift between the in-domain and out-of-domain training. in the next section, we formally describe the problem statement, and then, we propose solutions to enhance the performance of the pipeline. ## experimental setup we begin this section by providing an overview of the datasets used in the experiments. afterwards, we briefly discuss the baselines that we compare to, and finally, we present a summary of the setup. additional information about the baselines and the training setup can be found in appendix. datasets. we use two datasets in our experiments, the multifc dataset @xcite and the snopes dataset @xcite . the claims in these datasets are not categorized into domains, therefore, we automatically extract the domains. see appendix a for a description about the process, a sample set of the claims from each domain, and the top lda topics of the domains. baselines. a detailed description of each baseline and the setup can be found in appendix b. we compare our retriever with three baselines @xcite , @xcite . ## results and analysis in tables 3, we report the performance of the reader compared to the alternative methods individually. to evaluate the reader in isolation, we assume that the retriever returns all the relevant evidence documents. the first observation is that our model is able to offer a lot of improvement on top of the nli pretraining model-this model is pretrained on snli and multinli datasets. all the methods (including ours) uses this model as the starting checkpoint for training. we also observe that the gap between our model and the baselines is still present even if we finetune the pretrained model on the source dataset (nli-ft). this does not change even if we pretrain the model on the target domain using the masked language model task (nlimlm-ft). in we don't report the plain contriever model as it was unable to solve the task reasonably. however, all the methods (including ours) use this model as the starting point for training. a noteworthy observation from to better understand the properties of our model, we report a series of ablation studies in both components of the pipeline, as well as the entire pipeline itself. in in if used along existing large language models. to this end, we report an experiment in ## conclusions we studied automatic fact checking under domain shift. we showed that large language models are unable to do the task in certain cases. then we empirically showed that the common fact checking pipeline suffers from distribution shift, when it is trained in one domain and tested in another domain. we then proposed two novel algorithms to enhance the performance of the entire pipeline. we evaluated our model in eight scenarios and showed that in the majority of the cases our model is the top performing algorithm. and darpa kairos program no. fa8750-19-2-1004. the views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of darpa, or the u.s. government. the u.s. government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein. we thank the anonymous reviewers for their feedback.