Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT -4 (2023)

Abstract: Misinformation poses a critical societal challenge, and current approaches have yet to produce an effective solution. We propose focusing on generalization, uncertainty, and how to leverage recent large language models, in order to create more practical tools to evaluate information veracity in contexts where perfect classification is impossible. We first demonstrate that GPT-4 can outperform prior methods in multiple settings and languages. Next, we explore generalization, revealing that GPT-4 and RoBERTa-large exhibit differences in failure modes. Third, we propose techniques to handle uncertainty that can detect impossible examples and strongly improve outcomes. We also discuss results on other language models, temperature, prompting, versioning, explainability, and web retrieval, each one providing practical insights and directions for future research. Finally, we publish the LIAR-New dataset with novel paired English and French misinformation data and Possibility labels that indicate if there is sufficient context for veracity evaluation. Overall, this research lays the groundwork for future tools that can drive real-world progress to combat misinformation.

## introduction misinformation represents a significant societal challenge, with detrimental effects observed in various domains spanning elections @xcite , public health @xcite , the economy @xcite , and more. recent developments in generative models have enabled the creation of increasingly sophisticated and convincing ai-generated misinformation, including deepfakes and bots using large language models, which further exacerbate the potential harm of "fake news" @xcite . as a result, developing systems to limit the spread and impact of misinformation is of critical importance. although there is a great deal of research in this area @xcite @xcite @xcite , practical solutions remain elusive. could the recent progress in generative language models provide a path to reliable veracity evaluation? in this work, we investigate the predictive capabilities of gpt-4 compared to previous approaches. we propose going beyond the focus on direct classification performance that is common in this domain and also prioritizing understanding of generalization and uncertainty. we believe this is critical since, despite extensive prior work on misinformation detection, existing solutions often fail to generalize and work in real-world settings, and there is no path to a perfect classifier that will solve this challenge with oracle-like answers alone. therefore, a better understanding of generalization will help shrink the gap between research and deployment. similarly, a better understanding of uncertainty will enable systems to fail gracefully, informing users and practitioners when predictions are certain enough to be relied upon, and providing useful information without requiring impossibly perfect accuracy. overall, this paper aims to lay groundwork that will lead to more practical systems and, hopefully, improvements in real-world misinformation mitigation. we examine three datasets: the widely-used liar @xcite , ct-fan-22 @xcite that contains both english and german corpora, and a new dataset liar-new. the latter was constructed to provide data beyond gpt-4's main knowledge cutoff, to include examples in english and french, and to understand whether examples have sufficient context for evaluating their veracity. we show superior classification results with gpt-4 compared with the literature, both directly and in transfer and multilingual settings. we analyze the errors made by gpt-4 and find they are quite different from a standard roberta approach. we also conduct extensive experiments on uncer-tainty and context, examining ways to productively quantify uncertainty and evaluate how results are affected by insufficient input information. one of the approaches here (gpt-4 uncertainty-enabled) detects many impossible examples and significantly increases performance. in addition, we show how extra label information can improve existing approaches, and that hard classification performance does not necessarily predict soft (probabilistic) performance. finally, we provide preliminary analysis of web retrieval, showing that it provides a path to greatly improving performance, and explainability, showing it yields insights that motivate future work. in summary, our key contributions are the following: reproducibility: we open-source our code and other contributions on github. 1 ## data we experiment on several misinformation datasets. first, we analyze the liar dataset @xcite , which is one of the most widely-used benchmarks for fake news detection. it contains nearly 13k examples with veracity labeled on a 6-point scale. we mainly follow the common approach in the literature of binarizing these labels, but also report 1 https://github.com/complexdata-mila/ mitigatemisinfo 6-way results. second, we use the ct-fan-22 dataset @xcite for additional tests, including transfer and multilingual settings. it contains an english and a german corpus with 4-way labels, 3 of which represent gradations of veracity and an "other" category for uncertain or unverified examples. finally, we provide a new dataset, liar-new. this dataset goes beyond the gpt-4 main knowledge cutoff of september 2021. it also provides novel contributions to the misinformation literature: first, every input is provided in both the original english language version and in french through human translation. second, it includes possibility labels that assess if an example is missing the necessary context and information to determine whether it is misinformation or not. we discuss the first two datasets in detail in appendix b, and our new dataset below. ## conclusions in conclusion, this study presented a multifaceted investigation to determine if we can use gpt-4, combined with careful analysis of generalization and uncertainty, to produce more practical misinformation detection. we found, first, that gpt-4 can give superior performance for information evaluation in multiple datasets and contexts. second, we found that gpt-4 has different failure modes from previous approaches like roberta. this could help produce better results for detecting misinformation and countering its spread. third, we investigated uncertainty quantification in this context, showing that gpt-4 can provide information linked to uncertainty and that that information can be leveraged towards significantly improved performance. fourth, it is not easy to guarantee inputs have sufficient context for a valid judgment to be made. we provide a new dataset and analysis that starts to address this issue. however, we believe that such explorations remain important to consider in future work. fifth and finally, we showed that both web retrieval and explainability offer promising avenues for future progress. overall, we hope this work will provide a starting point towards both better performance in general, and creating systems that will address key real-world factors that have so far prevented machine learning from reliably mitigating misinformation.