Adapting Fake News Detection to the Era of Large Language Models (2024)

Abstract: In the age of large language models (LLMs) and the widespread adoption of AI-driven content creation, the landscape of information dissemination has witnessed a paradigm shift. With the proliferation of both human-written and machine-generated real and fake news, robustly and effectively discerning the veracity of news articles has become an intricate challenge. While substantial research has been dedicated to fake news detection, it has either assumed that all news articles are human-written or has abruptly assumed that all machine-generated news was fake. Thus, a significant gap exists in understanding the interplay between machine-paraphrased real news, machine-generated fake news, human-written fake news, and human-written real news. In this paper, we study this gap by conducting a comprehensive evaluation of fake news detectors trained in various scenarios. Our primary objectives revolve around the following pivotal question: How can we adapt fake news detectors to the era of LLMs?Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa. Moreover, due to the bias of detectors against machine-generated texts (CITATION), they should be trained on datasets with a lower machine-generated news ratio than the test set. Building on our findings, we provide a practical strategy for the development of robust fake news detectors.

## introduction since brexit and the 2016 us presidential campaign, the proliferation of fake news has become a major societal concern @xcite . on the one hand, false information is easier to generate but harder to detect @xcite . previously, when generative ai was less prevalent, it was arguably reasonable to assume that most automatically generated news articles would be primarily used by malicious actors to craft fake news. however, with the remarkable advancement of generative ai in the last two years, and with their introduction in various aspects of our lives, these tools are now broadly adopted for legitimate purposes such as assisting journalists in content creation. reputable news agencies, for instance, use ai to draft or enhance their articles @xcite . nevertheless, the age-old problem of human-written fake news remains. this diverse blend of machine-generated genuine news, machine-generated fake articles, humanwritten fabrications, and human-written factual articles has shifted the way of news generation and the intricate intermingling of content sources is likely to endure in the foreseeable future. in order to adapt to the era of llms, the next generation of fake news detectors should be able to handle the mixed-content landscape of human/machine-generated real/fake news. while there exists a substantial body of research on fake news detection, it typically focuses exclusively on human-written fake news @xcite @xcite or on machine-generated fake news @xcite @xcite , essentially framing the problem as detection of machinegenerated text. however, robust fake news detectors should primarily assess the authenticity of the news articles, rather than relying on other confounding factors, such as whether the article was machine-generated. thus, there is a pressing need to understand fake news detectors on machineparaphrased real news (mr), machine-generated fake news (mf), human-written fake news (hf), and human-written real news (hr). here, we bridge this gap by evaluating fake news detectors trained with varying proportions of machine-generated and human-written fake news. our experiments yield the following key insights: our main contributions can be summarized as follows: ## related work fake news detection is the task of detecting potentially harmful news articles that make some false claims @xcite . the conventional solution for detecting fake news is to ask professionals such as journalists to perform manual factchecking @xcite , which is expensive and time-consuming. to reduce the time and the efforts for detecting fake news, researchers formulated this problem as a classification task and proposed various solutions for automatic fake news detection from a machine learning perspective @xcite @xcite . there are two main task formulations: one only consider human-written real vs. fake news, and the other one formulates this as detecting machinegenerated text, thus automatically categorizing any machine-generated news as fake news. ## methodology as the proportion of human-written vs. machinegenerated content shifts, it is crucial to study the impact on a model's proficiency in differentiating between real and fake news. here, we consider three distinct experimental setups, each representing different phases for news article generation due to the evolution of llms, as shown in figure 1 . we experiment with an llm as the news generator and we consider the news articles to contain only pure text without other modalities, as in previous fake news detection work @xcite . in the initial human legacy stage, the news was predominantly human-written. in this setting, we only use human-written real news articles as training data for the real news category. then, in order to see how the proportion of machinegenerated fake news in the training data affects the performance of the detector, we incrementally introduce machine-generated fake news articles, ranging from 0% to 100%. this setting mirrors a past era, where humans were the primary producers of real news. transitioning to the transitional coexistence stage, we reflect the current situation where language models collaboratively contribute to real news article generation. to simplify this setting, our training data in the real news class contain a human-written and a machine-generated part. this setting reflects the growing influence of llms in the news landscape. finally, in the machine dominance stage, we model a future where machine-generated texts surge for real news generation. for this, the training data for the real news class contains exclusively machine-generated real news articles. this reflects a future where llms become the primary and dominant way to produce the news. ## experimental results in this section, we describe our exhaustive experiments and exploration of the three stages that we described in section 3. specifically, we evaluate the above-mentioned five transformer-based models of two distinct sizes (base and large) across the three stages. coupled with the five different proportions of machine-generated fake news, this results in a total of 50 unique model configurations. we tested each of these configurations on the above-described in-domain test dataset gossipcop++ and on the out-of-domain dataset politifact++. as we show in appendix b, there are sizable differences between gossipcop++ and politifact++, and thus the latter can serve as a valuable out-of-domain dataset for assessing the robustness of fake news detectors that were trained on the former. ## discussion below, we offer some suggestions about the training data, i.e., how we should balance the machinegenerated (mf, mr) and the human-written training data (hf, hr). ## limitations one limitation of our work is that we used a coarsegrained proportion of machine-generated articles for training. our objective was to offer insights and to highlight potential adaptations in the training strategies during in the age of llms, thus raising awareness of responsible use of llms, and the three stages we outlined. note that it is easy to extend our framework to a more fine-grained study. the limitation in our paper as well as the observation from the experiments evoke several interesting future directions to address. from the perspective of fake news detection and misinformation research, there is a need for more nuanced evaluation and for combining different detectors to improve the detection accuracy for better fake news detection. moreover, our experiments inspire us to generalize the study of real/fake news distribution drift trends to macro contexts, particularly in light of how llms influence data distribution shifts. we elaborate more on this below. ## ethics and broader impact our research delves into fake news detectors and the dynamics of mis/disinformation, positing three hypothetical scenarios. while these scenarios are grounded in reason, they primarily serve to gauge detector performance and behavior. they should not be construed as predictions of the future landscape of fake and real news generation. our aim is to raise awareness of the potential risks that llms can pose, which goes beyond mis/disinformation and fake news detection, but to more subtle ways of influence related to the proportion of humanwritten texts online. we thus advocate for a responsible use of llms. although the dataset statistics show a distribution discrepancy between human-written and machinegenerated real and fake news, which might be a signal for the current fake news detection problem, from a broader data distribution standpoint, if journalists increasingly adopt llms in their writing, over time, the distribution of real news articles might gradually shift towards the distribution of the machine-generated articles (mf and mr). eventually, this shift could lead to a convergence where the distributions of real and fake news articles once again closely resemble each other.