MOMENTA : A Multimodal Framework for Detecting Harmful Memes and Their Targets (2021)

Abstract: Internet memes have become powerful means to transmit political, psychological, and socio-cultural ideas. Although memes are typically humorous, recent days have witnessed an escalation of harmful memes used for trolling, cyberbullying, and abuse. Detecting such memes is challenging as they can be highly satirical and cryptic. Moreover, while previous work has focused on specific aspects of memes such as hate speech and propaganda, there has been little work on harm in general. Here, we aim to bridge this gap. In particular, we focus on two tasks: (i)detecting harmful memes, and (ii) identifying the social entities they target. We further extend the recently released HarMeme dataset, which covered COVID-19, with additional memes and a new topic: US politics. To solve these tasks, we propose MOMENTA (MultimOdal framework for detecting harmful MemEs aNd Their tArgets), a novel multimodal deep neural network that uses global and local perspectives to detect harmful memes. MOMENTA systematically analyzes the local and the global perspective of the input meme (in both modalities) and relates it to the background context. MOMENTA is interpretable and generalizable, and our experiments show that it outperforms several strong rivaling approaches.

## introduction the growing popularity of social media platforms has given rise to a new form of multimodal entity: the meme, which is an image, embedded with a short piece of text. memes are easily shared and can spread fast on the internet, especially in social media. they are typically humorous and amusing in nature; however, by using an adroit combination of images and texts in the context of contemporary political and socio-cultural divisions, a seemingly harmless meme can easily become a multimodal source of harm. such harmful memes can be dangerous as they can easily damage the reputation of individuals, renowned celebrities, political entities, companies, or social groups, e.g., minorities. despite memes being so influential, their multimodal nature and camouflaged semantics makes them very challenging to analyze. the abundant quantity, fecundity and escalating diversity of online memes has led to a growing body of research on meme analysis, which has focused on tasks such as meme emotion analysis @xcite , sarcastic meme detection @xcite , and hateful meme detection @xcite @xcite . research on these problems has shown that off-theshelf multimodal systems, which often perform well on a range of visual-linguistic tasks, struggle when applied to memes. there are a number of reasons for that. first, memes are context-dependent, and thus focusing only on the image and on the text without background knowledge about the context in which the meme was generated, as well as some background information about people, companies and events, often is not enough to understand it. second, unlike other multimodal tasks, the image and the textual content in the meme are often uncorrelated, and its overall semantics is presented holistically. finally, real-world memes can be noisy, and the text embedded in them can be hard to extract using standard ocr tools. the proliferation of virulent memes has stimulated research focusing on their dark sides: hate @xcite and offensiveness @xcite . recently, @xcite defined the notion of harmful meme and demonstrated its dependency on the background context. for example, the meme in figure 1a is somewhat harmful to joe biden in the context of an election, but it is arguably neither hateful nor offensive. moreover, the notion of harm is often apparent only when the two modalities are combined. for example, in figure 1b , the unimodal cues are not harmful, but the meme as a whole is harmful to donald trump. moreover, identifying the target of harmful memes (e.g., joe biden and donald trump) requires separate analysis, which is not prevalent for hateful or offensive memes. with the above motivation in mind, here we aim to explore the role of background context for detecting harmful memes and for identifying the social entities they target. in particular, we make the following contributions: finally, recent research and a shared task focused on propaganda in memes @xcite , but did not target harmfulness per se. ## defining harmful meme following @xcite , we abridge the definition of harmful meme as follows: a multimodal unit consisting of an image and an embedded text that has the potential to cause harm to an individual, an organization, a community, or society. here, harm includes mental abuse, defamation, psycho-physiological injury, socio-economic damages, proprietary damage, emotional disturbance, compensated public image, etc. offensive and hateful memes are harmful, but not the other way round. offensive memes typically aim to mock or to bully a social entity, usually by using abusive words. a hateful meme contains derogatory content, influenced by utmost bias towards an entity (e.g., an individual a community, or an organization). the harmful content in a harmful meme is often camouflaged and might require critical judgment to detect. furthermore, the social entities attacked or targeted by harmful memes can be any individual, organization, or community, as opposed to hateful memes, where entities are attacked based on personal attributes. ## data here, we describe our two datasets, harm-c and harm-p, which consist of memes related to covid-19 and to us politics, respectively. ## momenta: our proposed system here, we describe our system, momenta for harmful meme detection and target identification. it takes a meme as input, and extracts the embedded text using google's ocr vision api foot_4 . we encode each text-image pair using clip @xcite , a pre-trained visual-linguistic model, leveraging its representations to capture the strong invariance and the overall semantics of the meme. in addition to the clip features, we also identify faces and object proposals 6 (ren et al., 2015), and we extract various attributes (see figure 3 ), which define high-level topics or entities, such as joe biden and the republican party, in an image. then, on the visual side, we encode the rois using the pre-trained vgg-19 model @xcite , and on the textual side, we encode the topics/entities using distilbert @xcite . as we mentioned earlier, the analysis of harmful memes is challenging because of their abstruse nature without specific context. thus, we hypothesize that adding object proposals and attributes would enable the model to understand the high-level concepts in the meme. our analysis below shows that it indeed captures the appropriate background context reasonably well. afterwards, we fuse the proposal and the attribute features together with the clip representations, separately for the image and for the textual representations, and we add first intramodel attentions, and then a crossmodal attention on top of them, in a hierarchical attention architecture. finally, we use the resulting multimodal context-aware representation to predict the meme's harmfulness and its target. figure 2 shows the overall architecture of momenta; below, we explain each of its components in detail. ## experiments we train momenta and all baselines using pytorch on nvidia tesla v100 gpu, with 32 gb dedicated memory, with cuda-11.2 and cudnn-8.1.1 installed. the hyper-parameter values for all models are given in appendix a. we experiment with harm-c and harm-p using a variety of state-of-the-art unimodal textual models, unimodal visual models, and multimodal models that were pre-trained on both modalities. we use three measures for evaluation: accuracy, macro-f1, and macro-averaged mean absolute error (mmae) @xcite . for the first two, higher values are better, while for mmae, lower values are better. £ mmbt: this is a multimodal bitransformer @xcite , capturing the intra-modal and the inter-modal dynamics of the two modalities. £ vilbert cc: vision and language bert @xcite , trained on an intermediate multimodal objective (conceptual captions) @xcite , is a strong model with task-agnostic joint representation of image and text. £ visual bert coco: this is visual bert @xcite , pre-trained on the coco dataset @xcite , another strong multimodal model. ## experimental results we compare momenta to unimodal textual models, unimodal visual models, and multimodal models pre-trained on both modalities. except for the unimodal visual models, we use the mmf framework. 10 we further explore the generalizability and the interpretability of momenta. ## conclusion and future work we introduced two large-scale datasets, harm-c and harm-p, for detecting harmful memes and their targets. we further proposed momenta, a novel multimodal deep neural network that systematically analyzes the local and the global perspective of the input meme (in both modalities) and relates it to the background context. extensive experiments on the two datasets showed the efficacy of momenta, which outperforms ten baselines for both tasks. we further demonstrated its transferability and interpretability. in future work, we plan to extend the datasets with more domains and languages.