A Survey on Multimodal Disinformation Detection (2022)

Abstract: Recent years have witnessed the proliferation of offensive content online such as fake news, propaganda, misinformation, and disinformation. While initially this was mostly about textual content, over time images and videos gained popularity, as they are much easier to consume, attract more attention, and spread further than text. As a result, researchers started leveraging different modalities and combinations thereof to tackle online multimodal offensive content. In this study, we offer a survey on the state-of-the-art on multimodal disinformation detection covering various combinations of modalities: text, images, speech, video, social media network structure, and temporal information. Moreover, while some studies focused on factuality, others investigated how harmful the content is. While these two components in the definition of disinformation – (i) factuality, and (ii) harmfulness –, are equally important, they are typically studied in isolation. Thus, we argue for the need to tackle disinformation detection by taking into account multiple modalities as well as both factuality and harmfulness, in the same framework. Finally, we discuss current challenges and future research directions.

## introduction the proliferation of online social media has encouraged individuals to freely express their opinions and emotions. on one hand, the freedom of speech has led to a massive growth of online content which, if systematically mined, can be used for citizen journalism, public awareness, political campaigning, etc. on the other hand, its misuse has given rise to the proliferation of hostility online @xcite , resulting in offensive content in the form of fake news, hate speech @xcite , propaganda @xcite , cyberbullying @xcite , etc. indeed, researchers have argued that this situation has set the dawn of the post-truth era, dominated by emotions and "alternative facts" @xcite @xcite . more recently, with the emergence of the covid-19 pandemic, a new blending of medical and political false information has given rise to the first global infodemic @xcite @xcite . 1 the term "fake news" is commonly used, although it is very generic, and misleads people to focus only on veracity. that is why international organizations such as the un, who, eu, and nato prefer the term disinformation @xcite , which refers to information that is (i) fake and also (ii) spreads deliberately to deceive and harm others. the latter aspect of the disinformation (i.e., harmfulness) is often ignored, but it is equally important. a related term is misinformation, which also refers to the spreading of false content, but lacks the underlying intention to do harm. this is illustrated by the definitions of these notions by first draft @xcite where misinformation is defined as "unintentional mistakes such as inaccurate photo captions, dates, statistics, translations, or when satire is taken seriously", while disinformation is "fabricated or deliberately manipulated text/speech/visual context, and also intentionally created conspiracy theories or rumors". in our survey, we will focus on disinformation, and we will study both the factuality and harmfulness aspects of the problem, with focus on different modalities. note that there are posts that can be harmful but factually true or non-factual but harmful (e.g., hate speech); our study also covers some related work on them. the term factuality refers to automatically evaluating the solidity of the reporting/social media statements in terms of facts and figures @xcite . the harmfulness or harmful content typically refers to "anything online which causes a person distress or harm". 2 figure 2 , in appendix, gives examples of such content. @xcite addressed both aspects of disinformation using social media content related to the covid-19 infodemic. they demonstrated a correlation between factuality and harmfulness, which varies across languages even in the same country, e.g., for arabic, 56% of the false content was harmful, while for english, it was 24%. disinformation often spreads as text. however, internet and social media allow the use of different modalities, which can make a disinformation message attractive as well as impactful, e.g., a meme or a video is much easier to consume, attracts much more attention, is perceived as more credible @xcite , spreads further than simple text @xcite , and can be weaponized @xcite . notably, multimodality remains under-explored in disinformation detection. @xcite performed a meta-review of 23 fake news models and the data modality they leveraged, and found that 91.3% used text, 47.8% looked into social media network structure, 26% relied on temporal data, and only a handful made use of images or videos. moreover, while there has been research to detect whether an image or a video has been manipulated, the attempt is less in a truly multimodal setting @xcite @xcite @xcite . here we survey research on multimodal disinformation detection covering various combinations of modalities: text, images, speech, video, social media network structure, and temporal information. the data sources include social media (e.g., twitter), news, video (e.g., courtroom trials), and tv shows. we further argue for the need to cover multiple modalities in the same framework, while taking both factuality and harmfulness into account. while there have been a number of surveys on "fake news" @xcite @xcite , misinformation @xcite , fact-checking @xcite , truth discovery @xcite , rumour detection @xcite , harmful memes @xcite ) and 2 https://swgfl.org.uk/services/report-harmful-content/ propaganda detection @xcite , none of them had multimodality as the main focus. moreover, they targeted either factuality (most surveys above), or harmfulness (the latter survey), but not both. here, we aim to bridge this gap. therefore, in the present survey, we analyze the literature covering various aspects of multimodality (text, image, speech, video, network, and temporal), with a focus on the two aspects of disinformation: factuality and harmfulness, as shown in figure 1 . ## multimodal factuality prediction in this section, we focus on the first aspect of disinformation -factuality. automatic detection of factual claims is important to debunk the spread of misleading information, as it is crucial to detect the factuality of statements that can mislead people. a large body of work has been devoted to fact-checking textual claims but such claims are often expressed and disseminated together with other modalities such as images, speech, and video, and are further propagated through social networks. we summarize relevant studies in ## multimodal harmful content detection in this section, we focus on the second aspect of disinformation: harmfulness. it is essential to filter or to flag online harmful content. the harmful content includes child abuse material, violent and extreme content, hate speech, graphic content, sexual content, and spam content @xcite . 3 in recent years, the ability to recognize harmful content within online communities has received a lot of attention by researchers @xcite and policymakers that aim to keep users safe in the digital world. studies in this direction include detecting harmful contents in network science @xcite , natural language processing @xcite @xcite and computer vision @xcite @xcite . in ## modeling techniques in this section, we discuss modeling techniques for both factuality and harmfulness. to combine multiple modalities, there have been several approaches: (i) early-fusion, where low-level features from different modalities are learned, fused, and fed into a single prediction model @xcite @xcite @xcite ; (ii) late-fusion, where unimodal decisions are fused with some mechanisms such as averaging and voting @xcite , and (iii) hybrid-fusion, where a subset of learned features are passed to the final classifier (early-fusion), and the remaining modalities are fed to the classifier later (late-fusion) @xcite . within these fusion strategies, the learning setup can also be divided into unsupervised, semi-supervised, supervised and self-supervised methods. @xcite investigated different fusion strategies (e.g., earlyand late-fusion and self-supervised models) for propaganda detection using visualbert @xcite , mmbt @xcite , and vilbert @xcite . their findings suggest that self-supervised joint learning models, such as mmbt, vilbert, and visual-bert perform better in increasing order, respectively, compared to the other fusion methods. as 4 https://medium.com/1st-draft/how-to-improve-ouranalysis-of-coordinated-inauthentic-behavior-a4ec62ce9bff a part of "hateful memes challenge" to classify hateful vs. non-hateful memes, several such models have been investigated by @xcite . attempts to design unsupervised models are limited. müller-budack et al. (2020) introduced crossmodal consistency verification tool (ccvt) to check the coherence between images and associated texts. @xcite defined trust of news and credibility of users who spread the news and used bayesian learning to iteratively update these quantities. news with low trustworthiness is returned as fake news. @xcite proposed gtut, a graph-based approach that exploits the underlying bipartite network of users and news articles to detect the dense communities of fake news and fraud users. due to the scarcity of labeled data, a few studies attempted to design semi-supervised methods by leveraging an ample amount of unlabelled data. helmstetter and paulheim (2018 within a supervised learning setup, two other types of learning method have also been explored for disinformation detection such as adversarial learning and autoencoder based. adversarial learning models for fake news detection include eann @xcite , an event adversarial neural network to detect emerging and timecritical fake news, and same @xcite , a sentiment-aware multimodal embedding method which leverages multiple modalities with the sentiment expressed by readers in their comments. ## major challenges recently, several initiatives were undertaken by major companies and government entities to combat disinformation in social media (digi, 2021). 5 however, automatic detection of misleading and harmful content poses a number of challenges as discussed below and in appendix (section d). models combining multiple modalities. the major challenge is to devise a mechanism to combine multiple modalities in a systematic way so that one modality complements the others. current stateof-the-art primarily adopts early and late fusion, which are limited and do not always yield strong results @xcite . very recently, jointly trained multimodal transformer-based models (e.g., vilbert @xcite , visual bert @xcite and multimodal bitransformers (mmbt) @xcite ) have shown strong potential @xcite . however, such models are trained considering only two modalities (textual and visual), while fact-checking or disinformation-related content consists of more than two modalities e.g., text, speech, video, network, etc. @xcite . hence, there is a room for improvement in developing multimodal models that involve additional, and potentially more than two modalities. another important problem is cross-modal inconsistency in social media content, as shown in figure 2(c ), which poses a challenge in a multimodal setting @xcite . datasets. one of the major challenges when working with such diverse modalities, i.e., text, image, speech, video, and network, is to get access to an appropriate dataset, and moreover to one that considers both factuality and harmfulness. furthermore, there is a need to integrate data from multiple platforms (e.g., news, posts from twitter, reddit and instagram) as different data sources present different styles and focus on different topics. ## future directions based on the aforementioned challenges, we forecast the following research directions: explainability. model interpretation remains largely unexplored. this can be addressed in future studies to understand the general capability of the models. providing evidence of why certain claims are false is also important. there has been work in this direction such as tabfact @xcite and fever @xcite . however, such approaches rely on existing knowledge bases (e.g., wikipedia) and may fail for a new problem such as disinformation about covid-19. it is also important to understand what models learn, e.g., lexical or semantic concepts or a set of neurons may learn one aspect better than the others. moreover, while current studies on explainable fact-checking focus on explaining the predictions, very few focus on model explanations @xcite . beyond content and network signals. state-ofthe-art methods for multimodal factuality prediction and harmful content detection are primarily based on content signals and network structure. however, the information in these signals is limited and does not include personal preferences or cultural aspects. in the future, we envision multimodal techniques for disinformation detection that would go beyond content and network and would include signals like common sense and information about the user. moreover, multimodal models will become larger with more heterogeneous signals as input, and they would be pre-trained on a wider variety of tasks to shelter both aspects of disinformation: factuality and harmfulness. ## conclusion we surveyed the state-of-the-art in multimodal disinformation detection based on prior work on different modalities, focusing on disinformation, i.e., information that is both false and intents to do harm. we covered the major research topics of factuality and disinformation. our survey brought several interesting research challenges for multimodal disinformation detection, such as combining various modalities, which are often not aligned and are in different representations (e.g., text vs. speech vs. network structure), and the lack of such datasets to foster future research. in addition to highlighting the challenges, we also pointed to several research directions. while doing so, we argued for the need to tackle disinformation detection by taking into account multiple modalities as well as both factuality and harmfulness in the same framework. looks at both aspects of disinformation: factuality and harmfulness. while alam et al. (2021) did address both aspects, they only covered the text modality.