Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models (2023)

Abstract: The age of social media is rife with memes. Understanding and detecting harmful memes pose a significant challenge due to their implicit meaning that is not explicitly conveyed through the surface text and image. However, existing harmful meme detection approaches only recognize superficial harm-indicative signals in an end-to-end classification manner but ignore in-depth cognition of the meme text and image. In this paper, we attempt to detect harmful memes based on advanced reasoning over the interplay of multimodal information in memes. Inspired by the success of Large Language Models (LLMs) on complex reasoning, we first conduct abductive reasoning with LLMs. Then we propose a novel generative framework to learn reasonable thoughts from LLMs for better multimodal fusion and lightweight fine-tuning, which consists of two training stages: 1) Distill multimodal reasoning knowledge from LLMs; and 2) Fine-tune the generative framework to infer harmfulness. Extensive experiments conducted on three meme datasets demonstrate that our proposed approach achieves superior performance than state-of-the-art methods on the harmful meme detection task.

## introduction the development of social media platforms has given rise to a new form of multimodal content known as: meme. a meme typically comprises a picture that is combined with a concise text component. memes possess the capacity to quickly spread across the internet, especially on social media platforms, due to their ease of dissemination. while memes are often seen as humorous, there is a potential for harm when the combination of images and texts is strategically used to promote political and sociocultural divisions. for instance as in @xcite , during the covid-19 pandemic, a widely circulated meme falsely claimed that the mrna vaccine would alter human genetic code (dna) 1 . such multimodal disinformation spread caused vaccine safety and effectiveness concerns, hindering the formation of strong immune defenses in impacted areas globally @xcite . besides, another meme example shown in figure 1 (b) perpetuates harmful stereotypes and generalizations about asians. therefore, it is necessary to develop automatic approaches to facilitate harmful meme detection for unveiling the dark side of memes. harmful memes foot_1 are generally defined as "multimodal units consisting of an image and accompanying text that has the potential to cause harm to an individual, an organization, a community, or the whole society" @xcite . previous studies @xcite attempted to straightforwardly utilize pretrained vision-language models @xcite for harmful meme detection by training additional task-specific classification layers. more recently, @xcite proposed a prompt-tuning method with the meme text and image caption as the prompt for masked language modeling @xcite . however, existing harmful meme detection approaches oversimplified the problem as an end-toend classification paradigm, which only recognizes the superficial signals conveyed through the surface text and image. but more in-depth investigation and cognition on the implicit meaning is required especially when the image and text are not obviously correlated @xcite . intuitively, the key to harmful meme detection is to excavate rich correlations beneath the surface of the seemly uncorrelated text and image in the meme: 1) for example as in figure 1 (b), the image and the text are not harmful when considered in isolation, but are harmful when taken as a whole. a human checker should cognize that, the "biting" action in the image of a young woman with her pet dog ridicules asians' "dog-eating" behavior, which corresponds to the "asian" word in the text. because the text is a quote from a popular movie and is often used as a philosophical statement about the choices people make in life. and the image further adds a celebratory and joyful tone to the overall message. in comparison, conventional detection methods just focused on recognizing shallow harmindicative signals without such multimodal reasoning and essential background knowledge consideration, so the social dynamics of different races or the origin of the meme text from classical movie lines may not be well-cognized. unlike such recognitionlevel detection models, we argue that establishing reasonable thought between textual and visual information can further improve meme understanding with background knowledge for better harmful meme detection. inspired by the success of llms for reasoning at the cognition level with contextual background knowledge @xcite @xcite , we propose a novel approach: mr.harm, by leveraging the multimodal reasoning knowledge distilled from llms for harmful meme detection. to this end, we first prompt llms for abductive reasoning, and then propose a two-stage generative framework based on smaller language models to learn reasonable thoughts from llms for better multimodal fusion and lightweight fine-tuning. more specifically, we incorporate the meme text and image into a twostage training paradigm: 1) reasoning distillation: in the first stage, we fine-tune our smaller language models with the interaction of language and vision features to distill multimodal reasoning knowledge from llms, which empowers our framework with the ability to conduct cognitive reasoning for the harmfulness prediction. 2) harmfulness inference: in the second stage, we exploit the fine-tuned small language models to infer the final harmfulness prediction. in this manner, we augment the harmful meme detection model with multimodal reasoning knowledge to unmask the implicit meaning hidden in holistic multimodal information from memes. we evaluate our proposed approach based on three public meme datasets. the results not only show that our method outperforms strong harmful meme detection baselines by a large margin, but also provide fine-grained analysis for interpreting how our approach works. our contributions are summarized as follows in three folds: ## our approach problem statement we define a harmful meme detection dataset as a set of memes where each meme @xmath0, i, t } is a triplet representing an image i that is associated with a text t , and a ground-truth harmfulness label @xmath1, harmless}. in this work, to investigate multimodal reasoning distilled from llms, we convert the harmful meme detection task into a natural language generation paradigm, where our model takes the text t and image i as the input and generates a text sequence that contains the label y to clearly express whether the meme is harmful. our core idea is to reason and evolve with the cognition-level rationale beyond the recognitionlevel perception @xcite by capturing the inter-relationship between visual and textual elements in memes. the overview of our framework is shown in figure 2 , which consists of abductive reasoning with llms (see sec. ## conclusion and future work in this paper, we propose to capture implicit meaning that is not explicitly conveyed through the surface of the text and image in memes for harmful meme detection. we first conduct abductive reasoning with llms. then we present a novel generative framework to distill multimodal reasoning knowledge from llms, which includes two training stages: 1) reasoning distillation and 2) harmfulness inference. results on three meme benchmarks confirm the advantages of our proposed framework. for future work, since it is harder to judge the quality of the intermediate reasoning, where the evaluation is necessarily qualitative, we plan to do some sort of systematic study towards explainable harmful meme detection to claim explainability through a human subjects study for evaluation.