Detecting the Role of an Entity in Harmful Memes: Techniques and their Limitations (2022)

Abstract: Harmful or abusive online content has been increasing over time and it has been raising concerns among social media platforms, government agencies, and policymakers. Such harmful or abusive content has a significant negative impact on society such as cyberbullying led to suicides, COVID-19 related rumors led to hundreds of deaths. The content that is posted and shared online can be textual, visual, a combination of both, or a meme. In this paper, we provide our study on detecting the roles of entities in harmful memes, which is part of the CONSTRAINT-2022 shared task. We report the results on the participated system. We further provide a comparative analysis on different experimental settings (i.e., unimodal, multimodal, attention, and augmentation).

## introduction social media have become one of the main communication channels for sharing information online. unfortunately, they have been abused by malicious actors to promote their agenda using manipulative content, thus continuously plaguing political events, and the public debate, e.g., regarding the ongoing covid-19 infodemic @xcite . such type of content includes harm and hostility @xcite , hate speech @xcite , offensive language @xcite , abusive language @xcite , propaganda @xcite , cyberbullying @xcite , cyberaggression @xcite , and other kinds of harmful content @xcite . the propagation of such content is often done by coordinated groups @xcite using automated tools and targeting specific individuals, communities, and companies. there have been many research efforts to develop automated tools to detect such kind of content. several recent surveys have highlighted these aspects, which include fake news @xcite , misinformation and disinformation @xcite @xcite , rumours @xcite , propaganda @xcite , hate speech @xcite , cyberbullying @xcite , offensive @xcite and harmful content @xcite . the content shared on social media comes in different forms: textual, visual, or audio-visual. among other social media content, recently, internet memes became popular. memes are defined as "a group of digital items sharing common characteristics of content, form, or stance, which were created by associating them and were circulated, imitated, or transformed via the internet by many users" @xcite . memes typically consist of images containing some text @xcite . they are often shared for the purpose of having fun. however, memes can also be created and shared with bad intentions. this includes attacks on people based on characteristics such as ethnicity, race, sex, gender identity, disability, disease, nationality, and immigration status @xcite . there has been research effort to develop computational methods to detect such memes, such as detecting hateful memes @xcite , propaganda @xcite , offensiveness @xcite , sexist memes @xcite , troll memes @xcite , and generally harmful memes @xcite . harmful memes often target individuals, organizations, or social entities. @xcite developed a dataset where the annotation consists of (i) whether a meme is harmful or not, and (ii) whether it targets an individual, an organization, a community, or society. the constraint-2022 shared task follows a similar line of research @xcite . the entities in a meme are first identified and then the task asks participants to predict which entities are glorified, vilified, or victimized in the meme. the task is formulated as "given a meme and an entity, determine the role of the entity in the meme: hero vs. villain vs. victim vs. other." more details are given in section 3. memes are multimodal in nature, but the textual and the visual content in a meme are sometimes unrelated, which can make them hard to analyze for traditional multimodal approaches. moreover, context (e.g., where the meme was posted) plays an important role for understanding its content. another important factor is that since the text in the meme is overlaid on top of the image, the text needs to be extracted using ocr, which can result in errors that require additional manual post-editing @xcite . here, we address a task about entity role labeling for harmful memes based on the dataset released in the constraint-2022 shared task; see the task overview paper for more detail @xcite . this task is different from traditional semantic role labeling in nlp @xcite , where understanding who did what to whom, when, where, and why is typically addressed as a sequence labeling problem @xcite . recently, this has also been studied for visual content @xcite , i.e., situation recognition @xcite , visual semantic role labeling @xcite @xcite , and human-object interaction @xcite . to address the entity role labeling for a potentially harmful meme, we investigate textual, visual, and multimodal content using different pretrained models such as bert @xcite , vgg16 @xcite , and other visionlanguage models @xcite . we further explore different textual data augmentation techniques and attention methods. for the shared task participation, we used only the image modality, which resulted in an underperforming system in the leaderboard. further studies using other modalities and approaches improved the performance of our system, but it is still lower (0.464 macro f1) than the best system (0.586). yet, our investigation might be useful to understand which approaches are useful for detecting the role of an entity in harmful memes. our contributions can be summarized as follows: the rest of the paper is organized as follows: section 2 presents previous work, section 3 describes the task and the dataset, section 4 formulates our experiments, section 5 discusses the evaluation results. finally, section 6 concludes and points to possible directions for future work. ## task and dataset below, we describe the constraint 2022 shared task and the corresponding dataset provided by the task organizers. more detail can be found in the shared task report @xcite . ## results and discussion below, we first discuss our sequence labeling and classification experiments. we then perform some analysis, and finally, we put our results in a broader perspective in the context of the shared task. ## conclusion and future work we addressed the problem of understanding the role of the entities in harmful memes, as part of the constraint-2022 shared task. we presented a comparative analysis of the importance of different modalities: the text and the image. we further experimented with two task reformulations -sequence labeling and classification-, and we found the latter to work better. overall, we obtained improvements when using block fusion, attention between the image and the text representations, and data augmentation. in future work, we plan to combine the sequence and the classification formulations in a joint multimodal setting. we further want to experiment with multi-task learning using other meme analysis tasks and datasets. last but not least, we plan to develop better data augmentation techniques to improve the performance on the low-frequency roles.