Narrative Style and the Spread of Health Misinformation on T witter (2023)

Abstract: Using a narrative style is an effective way to communicate health information both on and off social media. Given the amount of misinformation being spread online and its potential negative effects, it is crucial to investigate the interplay between narrative communication style and misinformative health content on user engagement on social media platforms. To explore this in the context of Twitter, we start with previously annotated health misinformation tweets (n \approx 15,000) and annotate a subset of the data (n=3,000) for the presence of narrative style. We then use these manually assigned labels to train text classifiers, experimenting with supervised fine-tuning and in-context learning for automatic narrative detection. We use our best model to label remaining portion of the dataset, then statistically analyze the relationship between narrative style, misinformation, and user-level features on engagement, finding that narrative use is connected to increased tweet engagement and can, in some cases, lead to increased engagement with misinformation. Finally, we analyze the general categories of language used in narratives and health misinformation in our dataset.

## introduction a narrative is a fundamental form of human communication. although colloquially used interchangeably with "story", the formal concept of narrative is defined more broadly-consisting of both story (connected events and characters) and narrative discourse (how the story is told) @xcite . in other words, a narrative represents the presentation of a sequence of events experienced by a character or characters @xcite . this study uses this definition to label and classify tweets into narrative and non-narrative based on whether or not they are written in a narrative style. an example * *equal contribution, listed in alphabetical order. narrative tweet is, "we got our first dose of the vaccine! can't tell you how excited and thankful we feel, and can't wait to get the next dose". considering the power of narrative information in influencing individuals' health-related beliefs, attitudes, and behaviors @xcite , it is important to understand and address narrative misinformation on social media. in particular, research has found that online health misinformation is often created using narratives @xcite . further, emerging research indicates that individuals are less likely to discern or verify misinformation when it is presented in a narrative (vs. non-narrative) format, possibly because narrative misinformation is perceived as more relatable and engaging @xcite . therefore, it is crucial to classify misinformation as narrative and non-narrative and examine how this affects user engagement on social media. misinformation is defined as false or misleading information that is contrary to the consensus of the scientific community based on the best available evidence at the time @xcite . the prevalence and diffusion of incorrect information and fabricated narratives on social media have been a growing concern @xcite . according to one study, 28% of the most viewed coronavirus videos included non-factual information, totaling 62 million views as of march 21, 2020 @xcite . us-based respondents of another survey estimate that about 33% of social media content that they view contains some misinformation @xcite . how individuals assess and act on misinformation can be affected by many factors, such as message features, source credibility, and individual motivations and emotions @xcite . existing research has shown that individuals are particularly susceptible to narrative misinformation that uses negative emotions such as anger or fear, incorrect personal stories, and made-up testimonials @xcite . yet, most studies were conducted in a laboratory setting, limiting their ecological validity and generalizability. further, prior studies have not quantitatively examined how social media misinformation in the narrative (vs. non-narrative) form affects user engagement at scale. therefore, in this study, we use a large set of health tweets to investigate the presence of narrative misinformation and its impact on user engagement, and explore the features of both narratives and misinformation on twitter. the contributions foot_0 of this work are (1) narrativity annotations for two existing misinformation datasets, which can be used to study the interaction between narrative and misinformation on twitter; ## related work prior work established the importance of narratives both on social media and in the context of sharing health-related information. on twitter, specifically, @xcite analyzed multi-tweet stories to study the importance of storytelling for identity construction. in the health domain, @xcite emphasized the importance of narrative research in developing healthcare services for patients, while @xcite studied how users on online health communities show support for one another. prior research suggests that narratives are more effective at communicating health risks than nonnarratives @xcite and promoting health behavior change @xcite . @xcite found that anti-vaccine pins on pinterest used more narrative information while pro-vaccine used more statistical information, and that the latter type had higher levels of user engagement. additionally, @xcite studied the influence of narratives on the perception of vaccination risks, and highlighted the effectiveness of narrative communication in influencing human decision making about vaccine risk. issues with the spread of misinformation on platforms like twitter are also widely researched. for example, @xcite studied covid-19 misinformation on twitter and the change in user perception of covid-19 vaccine content as a result of soft moderation of misinformation content on the platform. the spread of misinformation on social media is not just limited to the common public: a study done by @xcite reveals that verified twitter handles(organizations/celebrities) are also responsible for creating and spreading misinformation. the authors also observed that fake claims propagate faster than semi-fake claims, and tweets containing misinformation often attempt to discredit other information on social media. while most studies of narratives have relied on manual annotations, @xcite and @xcite used human-assigned labels to train text classification models to detect narrativity in social media posts in the health domain. both studies focused on bag-of-words models with ngram features, while more recent research found that deep learning models were more successful than classical machine learning models at detecting narratives in facebook posts @xcite . these studies suggest that automatic narrative classification could enable the labeling of even larger social media datasets. ## data collection and annotation we leveraged existing misinformation datasets and provided new annotations for the presence of narratives to create a single dataset that contained both sets of labels for each text: misinformation (or not) and narrative (or not). ## narrative detection given the annotated tweets, we then set out to train classification models that could be used to label the entirety of the misinformation datasets that we collected. ## narrative and misinformation in this section, we investigate the effect of narrativity and misinformation on user engagement metrics and variations in linguistic features. first, we analyze the distribution of each category of tweets in the full set of ∼ 14.5k tweets labeled by our annotators and the best roberta model from the previous section. second, we discuss how the dimensions of narrativity and misinformation affect user engagement (i.e., counts of retweets and likes). finally, we investigate how linguistic variations are significant between narratives and non-narratives and between informed and misinformed tweets. ## thematic analysis in order to understand the prevalent themes within the discourse surrounding covid-19 vaccinations on social media, we conducted a manual thematic analysis on a selected sample of 400 tweets, equally balanced across each of the following categories: narrative information, non-narrative information, narrative misinformation, and non-narrative misinformation (100 tweets from each category). we first examined the sampled tweets to uncover the recurring themes, and then labeled each tweet according to the predominant theme present. we find that a majority of the informative narratives described people's experiences with getting the vaccine, or expressing excitement that a loved one was vaccinated. this focus on vaccinationrelated stories can be partially attributed to the larger size of the antivax, which is specifically focused on vaccine-related information and misinformation, relative to the cmu-miscov19 dataset, which is more generally about covid-19. however, even across both datasets, we find that many users' stories centered around either vaccine experiences or experiences with covid-19 itself, as these were some the main ways that everyday users were personally affected by the pandemic. misinformative narratives typically centered around storylines that provided evidence for conspiracy theories or harmful side effects of the vaccine, and were less often written as first-person narratives. when users did talk about themselves in the misinformative narrative category, the stories described trying home remedies or hypothetical stories justifying why they would not get vaccinated. non-narrative informative (i.e., those not labeled as misinformation or narrative) tweets contained a wide range of topics, from sharing news and public health tips to memes in the form "if x, don't worry about what's in the vaccine", where x is something commonly done yet potentially unhealthy, such as eating fast food. these tweets are meant to downplay concerns others may be expressing about getting vaccinated. misinformative non-narratives presented similar ideas to the misinformative narratives, but these were presented in the form of "facts" about the dangers of vaccines or supporting conspiracy theories, rather than stories. ## conclusion in this paper, we study the relationship between narrative communication and health misinformation on social media, specifically in the context of twitter. we manually labeled a subset of tweets from misinformation datasets to add labels for narrative, and trained models to extend these labels to the entire datasets. then, using statistical modeling, we find that narrativity is connected to higher user engagement and misinformation to less engagement, but narrativity may help increase engagement of misinformation in some contexts such as for users with many followers. this finding may have implications for better promoting accurate health information by presenting and amplifying narrative messages. in line with studies of other domains, narratives showed more authenticity, had more mentions of people, time and many more expressions of positive emotion, while misinformation was less authentic used more analytic language, and had dramatically more references to death. many narratives focused on experiences of vaccines, illness, or home remedies, while nonnarratives focused more on news updates, public health information, and conspiracy theories. these findings shed light on how both the style and content of messages can relate to how users engage with then, suggesting that both are important when seeking to combat misinformation.