Debunking with Dialogue? Exploring AI -Generated Counterspeech to Challenge Conspiracy Theories (2025)

Abstract: Counterspeech is a key strategy against harmful online content, but scaling expert-driven efforts is challenging. Large Language Models (LLMs) present a potential solution, though their use in countering conspiracy theories is under-researched. Unlike for hate speech, no datasets exist that pair conspiracy theory comments with expert-crafted counterspeech. We address this gap by evaluating the ability of GPT-4o, Llama 3, and Mistral to effectively apply counterspeech strategies derived from psychological research provided through structured prompts. Our results show that the models often generate generic, repetitive, or superficial results. Additionally, they over-acknowledge fear and frequently hallucinate facts, sources, or figures, making their prompt-based use in practical applications problematic.

## introduction conspiracy theories (cts) are ubiquitous, often emerging as attempts to identify underlying causes of unexpected or significant events. the resulting narratives allege clandestine machinations by powerful entities perceived as malevolent. cts are emotionally charged and compelling, forming closed belief systems characterized by internal coherence, which makes them difficult to refute @xcite . they are often associated with harmful social, health and political consequences @xcite . for example, cts can lead individuals to reject scientific consensus @xcite and influence societal attitudes about critical issues, such as climate change or vaccine policies @xcite . belief in cts can undermine trust in democratic institutions, amplify social tensions and lead to violence @xcite . counterspeech (cs) has emerged as an important strategy for mitigating the impact of harmful content in the online sphere. it describes communicative measures designed to refute undesirable content while respecting the principle of freedom of expression @xcite . many non-governmental organizations (ngos) employ it as a strategy to combat hate speech (hs), misinformation and cts @xcite . while shifting the beliefs of an ideologically committed user who is propagating cts might be rather difficult @xcite , counterspeech can positively affect the discourse norms of the audience @xcite . manual moderation for comprehensive counterspeech is challenging due to high volumes and the rapidly evolving social media landscape @xcite . in this context, supporting ngos by automatically generating counterspeech that requires little post-processing becomes increasingly vital @xcite . large language models (llms) could potentially assist @xcite . while research has explored using llms for counterspeech against hs, there is a notable gap in addressing cts. while cts can incite hatred and often perpetuate antisemitic or racist narratives @xcite , they typically differ linguistically from hs. this research gap motivates our study, which investigates the potential of llms to generate counterspeech for cts. we focus on the scenario of an ngo employee leveraging llm-generated suggestions to respond to comments promoting cts. since no expert-based datasets containing examples of counterspeech to cts could be identified, we rely on zero-shot prompting to evaluate the llms. our central research question is thus: @xcite to what extent can current llms guided by prompt-based strategy instructions generate effective counterspeech against conspiracy theories? the prompts are based on four strategies drawn from the literature, particularly in psychology: factcheck based refutation, providing alternative explanations, storytelling of a counter-narrative, and encouraging critical thinking. these strategies are considered promising for countering ct beliefs in audiences not yet fully absorbed by them. we applied three llms-gpt-4o, llama 3 (8b), and mistral (7b)-to a dataset of 152 comments from the platform x that promote two ct themes: (1) narratives involving the 'deep state', ' @xcite (2) claims related to 'geo-and bioengineering'. the generated cs is qualitatively evaluated according to the established criteria. in addition to assessing individual comments, we analyze the lexical and semantic diversity of the generated cs to evaluate its practical utility for civil society actors. our findings show that all three models struggled to produce cs with substantial depth, often defaulting to superficial statements and avoiding narrative storytelling as a strategy. responses frequently contained generic and repetitive phrases, resulting in limited linguistic variety. notably, approximately 10% of outputs contained confabulations which were often difficult to spot. differences between the models were low and mostly not significant. these findings suggest that current llms are not yet effective for the generation of cs to cts in a zero-shot setting, underscoring the need for further research on grounding llms in relevant domain knowledge. ## related work research on automated cs generation has gained traction, but has primarily focused on hs. to our knowledge, only @xcite have examined the use of llms to generate cs for cts. in lab experiments, participants engaged in multiturn dialogues with gpt-4 turbo, resulting in a statistically significant reduction in self-reported ct belief. this suggests general potential for ct belief revision through llm interaction, though in a direct, dialogue-based setting differing from the social media context. we found no publicly available datasets containing examples of effective cs to cts communicated online. existing datasets on online cts (e. g., @xcite ) were mainly created for classification tasks @xcite @xcite , and are mostly related to covid-19. in the remainder, we focus on llm-based cs generation in the context of hs, as we found no prior work addressing these aspects for cts. the majority of approaches in cs generation rely on fine-tuning methods @xcite , using datasets from platforms such as reddit and gab @xcite , compiled by ngo operators @xcite , or generated through models @xcite . more recently, zero-shot and few-shot prompting of various gpt-based models has shown promising results for generating cs against hs @xcite @xcite . some studies have explored tailored approaches, such as using typeprompts to specify predefined styles like humor or facts @xcite or personalizing cs based on author profiling (e. g., age and gender) @xcite to fit specific contexts. across these studies, the newer gpt models demonstrate strong performance. some studies have integrated automated cs generation into interactive tools. for instance, generation of contextualized cs was embedded within a content monitoring tool designed to address gdpr and dsa compliance @xcite , or incorporated into a dashboard enabling users to semiautomatically generate cs using text and memes @xcite . bonaldi et al. present a comprehensive guide to using nlp to generate cs. they emphasize that while research in this area is growing, there is currently a lack of standardized definitions and best practices to integrate the different approaches @xcite . ashida and komachi highlight that evaluating cs is challenging due to the lack of established standards @xcite . some studies use metrics like bleu, gleu, or me-teor to compare generated text to a 'ground truth'. however, without cs references, these metrics are not applicable. moreover, unlike machine translation, effective cs may include a wide range of diverse responses, making such metrics less suitable. nonetheless, these measures can assess the diversity and novelty of outputs @xcite . furthermore, classification models can be utilized to evaluate sentiment or argument characteristics @xcite . in addition to quantitative methods, most researchers rely on human annotators for more indepth evaluations with regard to 'informativeness' and 'offensiveness' @xcite , or 'suitableness', and 'intra-coherence' @xcite , 'grammaticality', 'specificity' @xcite , or 'diversity', 'relevance' and 'language quality' @xcite . ## counterspeech against conspiracy theories conspiracy theories are interpretative frameworks attempting to explain events as the covert actions of powerful, malicious groups or individuals pursuing self-interest at public expense @xcite @xcite , overlooking actual, more intricate causes (popper and @xcite . they thus usually need actors (e. g., 'globalists') with a malicious goal (e. g., destabilizing national economies) through an action or strategy (e. g., manipulating weather) carried out in secrecy @xcite . on social media, these narratives often appear in fragmented forms, with some components omitted or implied @xcite . while cts can overlap with hs @xcite , they have their own unique characteristics with regard to function and linguistic manifestation @xcite . interventions against cts can be categorized as preventive (pre) or harm-reducing (post), as well as targeting the sender or the audience @xcite . as individuals deeply invested in conspiracy beliefs are particularly resistant to change @xcite , we focus on addressing the recipients of cts, or 'cyber-bystanders', in a social media context, postexposure to a ct. study findings show that "there is no such thing as perfectly refuted misinformation" (krekó, 2020, p. 4), since part of the 'essence' of the conspiracy remains embedded in the mental model of an individual. nevertheless, research suggests that the design and formulation of cs can influence its impact @xcite . ## results we first evaluate how well the models followed core prompt instructions related to structure, tone, and issue-specific handling. ## conclusion and discussion we examined the capabilities of gpt-4o, llama 3, and mistral in generating counterspeech (cs) to social media comments containing conspiracy theories (cts), focusing on their ability to implement strategies predefined in a prompt. the findings reveal several critical insights into both the models' performance and the broader challenges of automating cs in online contexts. all models primarily attempted to counter cts using fact-checking (fact), alternative explanations (alt), and encouragment of critical thinking (crit). the use of narratives (narr), despite being explicitly defined and exemplified in the prompt, was exceedingly rare. this strategy being the only one in experiment 2 resulted in a ∼ 60% realization, though not necessarily effective. this suggests that narr is currently the least suitable strategy for our ngo employee scenario and warrants specific research attention if used in experiments. outputs often lacked depth and specificity, with generic responses and overuse of boilerplate text, aligning with recent findings indicating that llms tend to generate generic replies, e. g. by simply denouncing a statement @xcite , an effect reinforced through safety guardrails of models @xcite , leading to mediocre likert-scale scores for conciseness and clarity, and strategy effectiveness. the models struggled to adequately condemn hate speech (hs), when present, suggesting the need for robust hs detection models to filter or flag such content before generating cs. the models also over-acknowledged fear and anxiety. experiments with gpt-4o, in which references to fear were removed from the prompt as an example of emotion, did not mitigate this tendency. similarly, the tone of the responses often failed to align with the intent of the cs, despite detailed instruction in the prompt. these results confirm recent research showing social desirability biases across various models @xcite , that, however, might change in light of current shifts in llms' system prompt templates @xcite . comparatively, gpt-4o excelled in rule adherence, effectively managing hate speech (hs), minimizing the repetition of harmful content, and maintaining clarity. mistral offered the most diverse outputs, closely followed by gpt-4o, while llama 3 was more repetitive, limiting its usefulness for nuanced cs. however, overall performance differences were often not statistically significant, particularly in strategic efforts, indicating uniform limitations across models. this suggests that fine-tuning smaller open models could be as effective as with a large closed model like gpt-4o. despite explicit instructions in the prompt, all models made up factual information, including sources, publications, and quotes, sometimes in subtle ways that required detailed scrutiny to identify. with a confabulation rate of about 10%, our findings are not in line with @xcite , who reported that no response was made up and only 1 out of 128 claims was misleading. logical fallacies were also observed in our experiments, further questioning the reliability of current language models for counterspeech generation in practical settings. these issues, coupled with low diversity in responses, indicate that significant manual post-processing would be necessary to avoid sounding robotic or unreliable in real-world applications while providing factually correct and engaging cs. our findings suggest that prompt-based cs generation for cts is less effective in a social media setting targeting bystanders, contrasting with the positive results reported by @xcite in dialogue-based interactions. future work could ground models in a robust ct and countermeasures knowledge base, supported by dedicated datasets for fine-tuning and evaluation. as importantly, advances in understanding the relevance and effectiveness of different strategies (from the bystander perspective) are necessary, as current empirical research is inconclusive and in part contradictory. further empirical research, especially in online contexts, is required, bearing the potential to streamline prompt design and improve model performance. additionally, addressing diversity explicitly, e. g., through hyperparameter optimization, prompt design or contextualization @xcite , could enhance the practical usability of model-generated cs.