How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models (2024)

Abstract: Given the growing influx of misinformation across news and social media, there is a critical need for systems that can provide effective real-time verification of news claims. Large language or multimodal model based verification has been proposed to scale up online policing mechanisms for mitigating spread of false and harmful content. While these can potentially reduce burden on human fact-checkers, such efforts may be hampered by foundation model training data becoming outdated. In this work, we test the limits of improving foundation model performance without continual updating through an initial study of knowledge transfer using either existing intra- and inter-domain benchmarks or explanations generated from large language models (LLMs).We evaluate on 12 public benchmarks for fact-checking and misinformation detection as well as two other tasks relevant to content moderation - toxicity and stance detection. Our results on two recent multi-modal fact-checking benchmarks, Mocheg and Fakeddit, indicate that knowledge transfer strategies can improve Fakeddit performance over the state-of-the-art by up to 1.7% and Mocheg performance by up to 2.9%. The code, model checkpoints, and dataset are available: https://github.com/given131/ fact-verifier-knowledge-transfer.

## introduction top news stories rapidly go out-of-date and are replaced, e.g. during political election cycles. a recent study of google trends 1 in 2018 found that popular news stories tend to stay relevant for a lifespan of only 7 days. the actual observed behavior of the general public seems incongruous with the current paradigm of automated fact-checking, which relies on static resources. @xcite correspondence can be sent to: jerry96@snu.ac.kr, yjy@yonsei.ac.kr or skgabrie@cs.ucla.edu. motivated by previous approaches that have shown transformer-based models can effectively learn task-specific and linguistic reasoning skills from unified pretraining, e.g. for qa @xcite and small-scale text-only misinformation detection @xcite , we conduct a largescale study of transfer learning across diverse textonly and multimodal datasets to boost reasoning capabilities of misinformation detection systems. we consider intra-domain transfer from 6 misinformation detection and fact-checking datasets, as well as inter-domain transfer, where a fact verification model is jointly trained across other content moderation tasks (e.g. hate speech detection) to overcome brittleness to biases of fact-checking datasets. we also consider the impact of parametric knowledge transfer from larger-scale closed models like gpt-3.5-turbo (openai, 2022) and gpt-4 (openai, 2023) through generated explanations for a specific veracity label. we also address how transfer learning impacts a model generalization issue known as covariate shift @xcite , when the test data distribution diverges from the training distribution despite consistency in labeling procedures. a very tangible example of this is when evaluated news topics change entirely from training data due to a major unforeseen event like the covid-19 pandemic. given a claim like "the fbi warned that smart tvs can 'spy' on their owners," figure 1 shows how our full verification pipeline can be used to predict the reliability of a social media claim. the system consists of (1) a unified fact verification model m f c that predicts the veracity or potential harm (e.g. toxicity) of multi-modal inputs, and (2) an explanation model m eg prompted to generate explanations of claim veracity. following from theories of human cognition and language interpretation, which relies not only on continuous learning of facts but on commonsense world understanding (e.g, @xcite , as well as recent work showing the effectiveness of llm explanations for countering misinformation @xcite @xcite , we seek to supervise verifier training with examples of correct and noisy fact verification reasoning using m eg . we evaluate our verification pipeline on 12 existing text-only and multi-modal fact verification, hate speech and stance detection benchmarks. our results confirm that fact verification models trained on common benchmarks are extremely brittle to distribution shift and indicate data diversity is an important factor in high-performing fact verifiers over scale alone. our best intra-domain mixture improves fakeddit results by 1.7% f1, leading to a performance of 93.42% f1. we also find that knowledge distillation through gpt-4o and gpt-3.5-turbo explanations can boost mocheg performance over the state-of-the-art by 2.9%. beyond fact-checking, our study has implications for other important content moderation domains like hate speech detection, where we find knowledge trans-fer from fact-checking data can boost performance by 13.65%. in summary, our contributions are as follows: to encourage further research on robust fact verification, we release our code, model checkpoints, and dataset at https://github.com/given131/ fact-verifier-knowledge-transfer . ## motivation & related work automated fact-checking. given the rapid proliferation of misinformation on social media, there is a critical need for automatic tools that can assist human fact-checkers @xcite . much of the earlier work in this area relied on linguistic cues or social media network features (e.g., @xcite @xcite . later work on detection of mis-and disinformation has considered transformer-based approaches, notably @xcite . most similar to our work is @xcite , which also does unified misinformation detection on a much smaller scale by training a model on various unimodal misinformation detection corpora. seperately, a body of prior work also explores fact-checking explanations (e.g., @xcite , and prior to us, @xcite draws a connection between claim verification and the natural logical inference underlying commonsense acquisition. robustness in fact-checking. prior work has highlighted challenges to automated fact-checking like insufficient evidence @xcite or spurious correlations used for evidence retrieval @xcite . we address these knowledge gaps by exploring the use of machine-generated explanations to aid in fact verification. this is similar to the motivation behind the vitamin-c dataset @xcite , however vitamin-c only considers wikipedia revisions and is not multi-modal. most recently, @xcite @xcite have delved into the limitations of llms for fact verification, noting that while the ability of llms to verify inputs surpasses their ability to generate factual content, they are still far less reliable than human fact-checkers. ## transfer learning strategies in this section, we first describe the basic experimental setup for multimodal fact-checking. we then describe transfer learning methodologies based on whether knowledge is being distilled from more diverse sources within the same domain, transfer of knowledge across domains (e.g. misinformation detection and toxicity detection), or from parametric knowledge through llm-generated explanations. ## evaluation details we evaluate claim verification, toxicity detection and stance detection performance using f1. for three-label dataset mixtures evaluated on two-label datasets, we map labels 0 and 1 between the datasets (e.g. @xmath0, and @xmath1. if the predicted label is 2, we determine ŷ for the two-label dataset from the model output probability distribution by finding the most probable label from 0,1. we consider 12 evaluation sets: fakeddit (fak), hatefulmemes (ham), hatexplain (hax), mmhs150k (mmh), mocheg (moc), misinfo reaction frames (mrf), 3 temporal nela-gt subsets (pre, pos, ukr) described in the next section, p-stance (pst), pubhealth (ph), and toxigen (tox). ## learning from dataset mixtures in figures 3 and 4 , 3 we show that the transfer learning mixture substantially impacts the in-distribution and out-of-distribution performance of clip-large-336 trained on fact-checking and toxicity detection benchmarks. while some mixtures (e.g. mocheg + fakeddit + pubhealth) actually decrease fact-checking benchmark performance of multimodal models, others (e.g. mocheg + pub-health for mocheg, mocheg + vitamin-c + fakeddit for fakeddit) improve by up to 1.56% f1 over single dataset baseline results. for fakeddit, we achieve the state-of-the-art performance with the mixture of mocheg + vitamin-c + fakeddit, resulting in 93.42% f1. ## learning from explanations in the next section, we explore the use of gpt-4o and gpt-3.5-turbo explanations as silver evidence. ## conclusion in conclusion, we test three different transfer learning strategies to measure impact on multimodal fact-checking performance and improve results on two widely used benchmarks. we also discuss future steps for robust fact verification, such as using closed llms to expand reasoning capabilities of open foundation models. we show that explanations generated from powerful llms like gpt-4o and gpt-3.5-turbo can boost performance of smaller models.