Multimodal Online Manipulation: Empirical Analysis of Fact-Checking Reports
Authors: Olga Uryupina (Università di Trento, 2024)
Topic: Fact Checking and Fake News Detection

Abstract
This paper presents an in-depth exploratory quantitative study of the interaction between multimedia and textual components in online manipulative content. We discuss relations between content layers (such as proof or support) as well as unscrupulous techniques compromising visual content. The study is based on fakes reported and analyzed by PolitiFact and comprises documents from Facebook, Twitter and Instagram. We identify several pervasive phenomena currently, affecting the impact of manipulative content on the reader and the possible strategies for effective de-bunking actions, and discuss possible research directions.

Introduction
Manipulative online content (fake news, propaganda, among others) is growing at an alarming rate, hindering our access to truthful and unbiased information and thus threatening principles of the democratic society. The problem has been addressed by professional journalists, who -- with the help of crowd-workers -- fight a never-ending battle to prevent information contamination. To enable a large-scale response to the misinformation threat, the AI community has invested a considerable effort into building competitive models for identifying non-transparent content, such as false claims or altered videos (deep fakes). However, we still lack a thorough understanding of the manipulative content and multiple aspects affecting its perception and impact on the reader. This paper aims at an in-depth analysis of one of such aspects, namely, the interaction between different (multimedia) layers of the manipulative message. More specifically, we study the semantics underlying the relation between multimedia and textual parts of the fake news. Our study is based on around 800 fakes from January till September 2022, as identified and analysed by PolitiFact.) is an independent journalistic agency and one of the most experienced fact-checking organizations, providing detailed analytics for non-transparent online content since 2007. Multimedia content, such as videos, reels, photos, screenshots or images is becoming increasingly popular in social media: it is an appealing and powerful way of expressing and/or enhancing one's message. Nevertheless, as a scientific community, we still have little understanding of the way the authors integrate multimedia into their content: most research so far has focused on a specific component and not on their interplay. Our study aims at identifying the role of multimedia part of manipulative messages. [] \centering \subfloat[Biden to teachers: “They're not somebody else's children. They're yours when you're in the classroom." (VIDEO)]{\includegraphics[width=60mm]{video-proof-cropped.png}} %% proof, cropped \hspace{1cm \subfloat[Now you know why there's suddenly "a formula shortage". The new age robber barrons have conveniently invested in some unholy breast milk made from human organs.]{\includegraphics[width=60mm]{screenshot-support.png}} %% miscaption --old headline \\ \subfloat[In honor of \#TaxDay, I remind you that Governor Evers wanted to increase your taxes by \1 billion just for heating your homes. Instead, Republicans cut your taxes by more than \2 billion. ] {\includegraphics[width=60mm]{infographics-paraphrase.png} \hspace{1cm \subfloat[Italian football agent Mino Raiola has died after suffering from an illness. RIP]{\includegraphics[width=60mm]{photo-illustration.png} %% illustration, ok Figure example:fig shows some examples from potential fakes analyzed by PolitiFact. We observe different relations between the text and the image. In particular, in (example:figa), the video is supposed to prove the claim by providing direct evidence, whereas in (example:figb), the image provides a support (appeal to authority). In (example:figc), the image is a visual paraphrase of the claim, enhancing its appeal but not providing extra proof, support or informational material. Finally, in (example:figd), the photo is an illustration that, while depicting the discussed person, does not aim at being relevant to the claim's veracity or impact. While understanding the relation between the image and the text is interesting from the scientific perspective, it is also a crucial prerequisite for efficient and meaningful fact-checking response. For example, if a supposed proof is a compromised photo, the response should highlight this fact (e.g., the video in (example:figa) has been cropped misrepresenting the quote, which should be highlighted in the fact-checking report). On the contrary, if a compromised photo is used as a mere illustration, the effective fact-checking report should focus on the textual claim per se. Another important angle is the issue with the multimedia part. In our example, the video in (example:figa) is cropped. On the contrary, (example:figb) represents an authentic screenshot, yet, it has been miscaptioned by the claim: an older content, irrelevant for the current events/topics, has been repurposed. The current paper focuses on these two aspects to analyze empirically the interplay between multimedia and textual components in fake news, as identified by PolitiFact. To this end, we reannotate the PolyFake dataset AUTHOR with fine-grained labels reflecting multimedia aspects.

Conclusion
We have presented an in-depth analysis of the interaction between textual and multimedia components of compromised social media documents. We have identified several high-impact issues, insufficiently studied by the community at the moment. These include the interaction between different modalities, the role of the multimedia part and its impact on selecting the successful fact-checking strategy, the difference between platforms and media types (current NLP studies predominantly focus on Twitter and images) and the importance of a more principled approach to content re-use. We hope that this study, motivated by human fact-checking expertise, can sparkle a meaningful discussion and improve automatic modeling. We thank the Autonomous Province of Trento for the financial support of our project via the AI@TN initiative. \appendix