Qualitative Analysis of Persuasive Emotion Triggering in Online Content
Authors: Olga Uryupina (Università di Trento, 2023)
Topic: Fact Checking and Fake News Detection

Abstract
This paper presents a qualitative analysis of the emotional component in manipulative online content (fakes). We show that emotion triggering is a crucial persuasion technique widely employed by unscrupulous content generators. Based on a dataset of real-life fakes analyzed by fact-checking professionals, we identify the most common types of triggered emotions, which are then used as a taxonomy for further annotation.

Introduction
The manipulative content, ranging from propaganda to hate campaigns, fake news, trolling, and similar, is becoming more widespread, threatening our access to truthful and unbiased information and undermining our rights to make informed decisions as individuals and as members of society. While there is a growing body of multidisciplinary research on identifying untruthful content, there is still very limited understanding of the manipulative techniques employed by unscrupulous content writers to convince the reader and ultimately change their point of view. We believe that this manipulation occurs through multiple channels: careful selection of fact-checkable and non-fact-checkable claims, biased yet seemingly solid argumentation/analytics, multimedia support, and, most importantly, the emotional component. Our current study focuses on emotion triggering – a technique widely used by content writers. When the reader experiences a strong feeling, they become less critical, easily overlooking deficiencies in the argumentation, making them more prone to manipulation. Fig. 1 shows examples of manipulative textual content with strong emotional triggering. In (1a), the message appeals strongly to fear by mentioning HIV, intensified further by mentioning "children." The fact-checking report informs the reader that the COVID-19 vaccines do not contain any HIV material, but do contain other lipids to protect the mRNA. However, distressed users might not trust this information fully, due to the strong emotion of fear for their children’s health. Example (1b) shows a typical manipulative message not addressed properly by the state-of-the-art verification-oriented technology. The message combines a verifiable true claim ("Murkowski, Collins, and Romney voted for Ketanji Brown Jackson") with a statement that seems like a similarly factual claim ("Murkowski, Collins, and Romney are pro-pro-pedophile"), but is actually an explanation/opinion offered by the writer. This triggers anger toward the authorities under scrutiny, their presumed hypocrisy, and their presumed (lack of) values. Again, the triggering is intensified by bringing up a topic related to children. The fact-checking report debunks this claim, stating that "Sens. Murkowski, Collins, and Romney have clear track records of acting against child exploitation, whether online or in person," and that the implied accusations about Judge Johnson are "misleading." Nevertheless, an emotionally driven reader might still remain manipulated ("no smoke without fire"), even if only partially. These examples show that fake news is much more complex than simply untrue messages. They may combine true facts with partially false or impossible-to-check statements, provide biased analytics on top, and add very strong emotional messages to manipulate readers. We believe that while the NLP community is making impressive progress on the fact verification task, our understanding of other phenomena related to manipulative content is still rather limited. The goal of our study is to gain a deeper and more realistic insight into the emotional component of fakes. As a first step, we provide a qualitative, data-driven analysis of emotion triggering. The contributions of this study are as follows: (i) we provide data-driven analysis, focusing on real data, combining original (source) fakes and high-quality reports by professional fact-checkers to improve our insight; (ii) we aim at a taxonomy of triggered emotions covering a majority of real-life fakes, departing from more theory-oriented labels; and (iii) we analyze perceived (i.e., triggered) emotions, as opposed to the common focus on expressed emotions, because we believe that induced sentiment plays a more important role in manipulation/persuasion.

Conclusion
This study focuses on the emotional component of manipulative online content. By analyzing real-life fake content from PolitiFact, we have observed a variety of emotional triggers used to promote unscrupulous content, agitating users and making them less critical of deficiencies in the fact selection and argumentation of the manipulative discourse. We have found that emotions play a crucial role in advancing different kinds of manipulative agendas. It is therefore extremely important for the scientific community to extend state-of-the-art verification-based approaches to fact-checking and incorporate models for emotion triggering and fallacious argumentation. Our study identifies the most common types of emotions triggered by manipulative content. However, defining these emotions accurately is not a trivial task, as we have already observed with fear. Our current work focuses on refining the definitions of the most common triggers to provide reliable annotation guidelines and create a dataset of emotional appeals. Triggered emotions (reactions) have so far mostly been out of the scope of the NLP community, where the vast majority of research focuses on emotions expressed in the document. We believe that our research can contribute to a better understanding of perceived emotions, which are crucial for modeling a text’s impact on the reader. In particular, we plan to study the relation between expressed and triggered emotions and investigate possibilities of transferring high-performing state-of-the-art (expressed) emotion recognition models to account for triggered emotions. Finally, we believe that a multi-factor understanding of manipulative content is essential to generate an adequate response and thwart misinformation. Emotionally-loaded fakes are particularly hard to debunk since they make the user less receptive to the rational argumentation of fact-checkers. As part of our future work, we want to investigate strategies for automatic response generation that take into account the emotional component and try to produce an adequate reaction, regaining the users’ trust.