Is Explanation the Cure? Misinformation Mitigation in the Short Term and Long Term (2023)

Abstract: With advancements in natural language processing (NLP) models, automatic explanation generation has been proposed to mitigate misinformation on social media platforms in addition to adding warning labels to identified fake news. While many researchers have focused on generating good explanations, how these explanations can really help humans combat fake news is under-explored. In this study, we compare the effectiveness of a warning label and the state-of- the-art counterfactual explanations generated by GPT-4 in debunking misinformation. In a two-wave, online human-subject study, participants (N = 215) were randomly assigned to a control group in which false contents are shown without any intervention, a warning tag group in which the false claims were labeled, or an explanation group in which the false contents were accompanied by GPT-4 generated explanations. Our results show that both interventions significantly decrease participants’ self-reported belief in fake claims in an equivalent manner for the short-term and long-term. We discuss the implications of our findings and directions for future NLP-based misinformation debunking strategies.

## introduction misinformation (or fake news) refers to false statements or fabricated information that is spread on social media @xcite . the spread of misinformation has posed considerable threats to individuals and society, such as political elections @xcite @xcite , and public health @xcite , especially during the covid-19 pandemic @xcite . because of the negative impact, considering effort has been devoted to devising methods to mitigate misinformation, such as computationbased detection and prevention. traditional fact-checking approaches are labor-intensive and timeconsuming, often requiring domain expertise. given advances in natural language processing (nlp), there is an ongoing shift towards nlpbased solutions such as fake news detection @xcite @xcite and generation of fact-checked, counterfactual explanations using natural language generation (nlg) models @xcite . decision-aid methods have also been proposed and deployed to warn users when a piece of fake news has been identified. warning labels (or tags) have been widely adopted by social media platforms such as facebook foot_0 , twitter (x) foot_1 , and tiktok foot_2 to mitigate humans' belief in misinformation. tag-and explanation-based methods are both effective in debunking fake news @xcite @xcite . however, few studies have compared tag-based and machine-generated explanations. furthermore, studies on fake news debunking strategies have primarily focused on short-term effects instead of long-term effects @xcite @xcite . thus we propose a comprehensive evaluation of both the short-and long-term effectiveness of these debunking strategies, specifically in the context of real-world news in the united states. for the explanation-based debunking strategy, we focus on counterfactual explanations, which have demonstrated greater effectiveness than summary-based explanations in mitigating the spread of misinformation @xcite . with the improvements of large language models (llm) from openai, @xcite already generates explanations acceptable to humans @xcite . here we employ the gpt-4 model, the latest state-of-the-art llm from openai, to generate all counterfactual explanations @xcite . to investigate the effectiveness of debunking strategies, we focus on their impact over varied time frames. we aim to study the following research questions: 1. how are the effectiveness of tag-based and explanation-based interventions compare to conditions with no interventions in the short term and long term from readers' aspect? ## conclusion we demonstrate that both tag-based and explanation-based methods can effectively combat fake information in the short-and longterm. however, such findings suggest that an emphasis on generating improved explanations from the aspects of nlp is not sufficient to address the primary challenge in mitigating humans' belief in misinformation. to make the nlp-based debunking strategies more effective in the long-term and further reduce recurred misbeliefs, it is crucial to consider visual design choices to boost readers' motivation to engage with detailed yet accessible explanations, such as font size, text length, or even personalized explanations.