Simulated Misinformation Susceptibility ( SMISTS ): Enhancing Misinformation Research with Large Language Model Simulations (2024)

Abstract: Psychological inoculation, a strategy designed to build resistance against persuasive misinformation, has shown efficacy in curbing its spread and mitigating its adverse effects at early stages. Despite its effectiveness, the design and optimization of these inoculations typically demand substantial human and financial resources, primarily due to the need for repeated experimental trials. To address these challenges, this paper introduces Simulated Misinformation Susceptibility Tests (SMISTs), leveraging Large Language Models (LLMs) to simulate participant responses in misinformation studies. SMIST employs a life experience-driven simulation methodology, which accounts for various aspects of participants’ backgrounds, to mitigate common issues of caricatures and stereotypes in LLM simulations and enhance response diversity. Our extensive experimentation demonstrates that SMIST, utilizing GPT-4 as the backend model, yields results that align closely with those obtained from human-subject studies in misinformation susceptibility. This alignment suggests that LLMs can effectively serve as proxies in evaluating the impact of psychological inoculations. Moreover, SMIST offers the critical benefit of being applicable to emerging or anticipated misinformation scenarios without exposing human participants to potentially harmful content. This characteristic of SMIST not only preserves participant safety but also expands the scope of misinformation research to include more sensitive or speculative topics.

## introduction misinformation poses significant, far-reaching harm, such as fueling vaccination hesitancy and exacerbating long-term mental health issues, notably during the covid-19 pandemic @xcite . differing from other social issues, early intervention is crucial in combating misinformation, as post-hoc corrections often fail to mitigate its detrimental effects @xcite . psychological inoculations, designed to weaken the belief in and spread of misinformation, are effective but require extensive misinformation susceptibility tests (mists) during development @xcite figure 1 : comparative illustration of the processes in human-subject mist (top) and smist (bottom). red and green boxes demonstrate cons of mist and pros of smist, respectively. q indicates the questionnaire. in smist, while all simulated participants within a group share specified demographic characteristics, they exhibit diversity in other uncontrolled aspects. @xcite @xcite . mists are designed to evaluate individual susceptibility to misinformation, shedding light on the factors that contribute to vulnerability. we follow prior studies in our definition of susceptibility @xcite , which use analyses of variance (anova) and regression analyses on participant responses to mists to measure susceptibility to misinformation as a predictor of belief in misinformation (we provide more detail in later sections). these tests usually involve subjecting participants to diverse misinformation forms to gauge their propensity to believe and disseminate such information. however, mists are hindered by significant labor and time demands, along with challenges in reproducibility and lim-ited generalizability across various misinformation types. additionally, they pose an ethical dilemma by potentially exposing the public to harmful misinformation before its widespread dissemination. leveraging recent advancements in large language models (llms) and their increasing application in computational social science @xcite , we propose the use of simulated mists (smists), utilizing llms in place of human subjects to address these challenges. although simulating llms as agents is a new and emerging topic, recent work has started to explore this concept in various scientific domains @xcite @xcite . these inquiries motivate our use of llms to study misinformation susceptibility. figure 1 illustrates the smist methodology compared to traditional human-subject mists. smist involves two main stages: participant profile generation and questionnaire response, both predominantly managed by llms. during the profile generation phase, we utilize an llm to fabricate participant life experiences based on a specific set of demographic features. to mitigate the tendency of llms to produce stereotypical or caricatured profiles, we limit the input to one demographic feature per profile and encourage the model to improvise on other demographic aspects (see appendix a for an example prompt and a generated life story). this approach has proven effective in generating a rich diversity in the uncontrolled dimensions of the profiles while ensuring that the controlled demographic features are represented accurately and without stereotyping, as confirmed by our manual reviews and validations. for the questionnaire response stage, we align with the methodology described in @xcite . here, the llm is instructed to answer misinformation questionnaires, adopting the perspectives of the participants with the created life experiences (see figure 4 for an example prompt). our initial application of the smist focused on covid-19 misinformation. in these tests, smist demonstrated robustness and reproducibility, akin to human participants in mists, with consistent results across repeated queries and various sample sizes. we employed statistical methods like anova and ordinary least squares (ols) regression to explore the relationship between demographic features of simulated participants and their susceptibility to misinformation. the outcomes corroborate findings from prior covid-19 misin-formation studies (e.g., @xcite ) and our human-subject experiments. further, we expanded our experiments to encompass different misinformation forms (satire, conspiracy theories), topics (russo-ukraine war, 2020 us election, gmo, climate change), and time frames (recent vs. past misinformation). in each of these contexts, smist's outputs remained in agreement with both our human-based studies and existing literature (e.g., uscinski et al. ( 2020) ). this consistency reinforces smist's capability to replicate humansubject mist results, its capability for controlled demographic representation, and its generalizability. significantly, smist effectively circumvents the ethical concern of exposing the public to misinformation prematurely, a notable issue in traditional mists. this positions smist as a valuable tool for developing rapid, effective psychological inoculations against misinformation. this paper's contributions are twofold: development of smist: we introduce smist, a novel approach aiming to enhance the efficiency, reproducibility, and generalizability of humansubject mists, facilitating the design of psychological inoculations against misinformation. life experience-based simulation with llms: we refine and extend the application of life experience-based simulations using llms to create more nuanced participant profiles. this method significantly reduces the risk of caricatures and stereotypes often present in llm simulations, presenting broader implications for simulation-intensive research domains. ## misinformation data preparation in our research, misinformation was sourced from three distinct reputable channels: (1) news articles from the new york times foot_0 and usa today foot_1 reporting on recent misinformation about various topics; (2) wikipedia entries detailing instances of misinformation, exemplified by the entry "disinformation in the russian invasion of ukraine" foot_2 ; and moreover, we incorporated five recent factchecked misinformation pieces from snopes foot_4 per topic. all these recent misinformation pieces have been fact-checked on snopes after 02/05/2024, less than a week before our human-subject mist questionnaires were distributed. this inclusion of recent misinformation aimed to test smist's robustness against unfamiliar misinformation (i.e., recent and not widely spread misinformation). moreover, using recent misinformation allows us to validate our human-subject mist, given the potential refusal of both humans and llms to trust or share widely recognized misinformation. figure 2 illustrates the construction and sizes of datasets utilized in our study. we refer to the three datasets as (a) original, (b) satire and conspiracy theory, and (c) recent misinformation datasets. ## methodology of smist this section discusses the methodology of smist, focusing on two key aspects. first, we describe the creation of participant profiles using an llm, ensuring a balanced representation of demographic features (section 3.2). second, we explain how these profiles guide an llm in responding to mist questionnaires, encapsulating the simulated participants' perspectives (section 3.1). the comprehensive smist process is visually summarized in figure 3 . we use openai's gpt-3.5 for generating participant life experiences and gpt-4 for answering the questionnaires. our choice of gpt-4 for questionnaire answering stems from preliminary analyses that evaluated several llms including gpt-3.5, gpt-4, and google bard. gpt-4 emerged as the superior model in simulating human responses to mists. the extensive time and resources required for experimenting with llms led us to focus our comprehensive set of experiments solely on gpt-4. ## smist results and analyses this section examines a few key research questions related to smist, around llms' ability to interpret participant profiles and respond to questionnaires, and how different demographic attributes affect misinformation susceptibility. unless otherwise specified, the experiments are conducted using the original dataset as introduced in section 2. ## conclusion and future work we introduced the smist, incorporating llm simulations into traditional mist frameworks for efficient, scalable, and generalizable analyses of misinformation susceptibility across various groups. smist offers a cost-effective method for studying factors influencing misinformation susceptibility, aiding the development of psychological inoculations. our findings validate smist's utility in misinformation research, showing alignment with prior studies and human-subject experiments. smist employs life stories to simulate participant profiles, addressing the caricature problem common in llm simulations and fostering diverse responses. this novel approach has the potential for adoption in other simulation-heavy research domains. to further aid the research community, we have made our simulated data and the analytical code publicly available at https://github.com/hikari-nyu/ smist . we encourage researchers to leverage these resources to advance their studies in misinformation and related fields. future work could focus on using smist for misinformation inoculations, leveraging its capabilities to assess inoculation effectiveness without needing repeated human-subject surveys.