“Female Astronaut: Because sandwiches won’t make themselves up there”: Towards Multimodal misogyny detection in memes (2023)

Abstract: A rise in the circulation of memes has led to the spread of a new form of multimodal hateful content. Unfortunately, the degree of hate women receive on the internet is disproportionately skewed against them. This, combined with the fact that multimodal misogyny is more challenging to detect as opposed to traditional text-based misogyny, signifies that the task of identifying misogynistic memes online is one of utmost importance. To this end, the MAMI dataset was released, consisting of 12000 memes annotated for misogyny and four sub-classes of misogyny - shame, objectification, violence and stereotype. While this balanced dataset is widely cited, we find that the task itself remains largely unsolved. Thus, in our work, we investigate the performance of multiple models in an effort to analyse whether domain specific pretraining helps model performance. We also investigate why even state of the art models find this task so challenging, and whether domain-specific pretraining can help. Our results show that pretraining BERT on hateful memes and leveraging an attention based approach with ViT outperforms state of the art models by more than 10%. Further, we provide insight into why these models may be struggling with this task with an extensive qualitative analysis of random samples from the test set.

## introduction with a rise in social media usage, memes have become an important part of expression, and communication today. multiple research studies have found that memes play a role in shaping a wide range of beliefs, such as climate change, use as bonding icons, political discussion, and social development. this new form of media, however, is still host to old-school offensive content that was previously seen in non-multimodal settings. this 1 /* denotes equal contribution includes hate speech of different forms, such as sexism and racism. the emergence of this popular media format has brought along the need to detect hateful content in multimodal formats, to ensure that the internet remains a safe space for all groups. further, there has been evidence to show that women are disproportionately targeted on the internet. for example, 33% of women under 35 say they have been sexually harassed online, while 11% of men under 35 say the same foot_0 . it has also been shown through many psychological and social science-based studies that the effects of online hate speech are observed well beyond the boundaries of the cyber world @xcite . yet, traditional, language-based misogyny detection techniques are no longer fully effective when it comes to multimodal misogyny. this is because, unlike textbased misogyny, identifying multimodal misogyny involves picking up on visual cues combined with sarcasm and linguistic nuances. to try and bridge this challenge, @xcite developed, licensed, and released mami: multimedia automatic misogyny detection, a dataset of 12000 memes, labeled for misogyny and four subclasses -shaming, objectification, violence, and stereotypes. the dataset is balanced across all classes and was released as a part of the semeval task in 2022. while this dataset is widely cited, and there have been multiple approaches developed to leverage this dataset for misogyny detection, we find that this challenging task remains unsolved to a large extent. to the best of our knowledge, no research aims to understand exactly why even the best models are unable to succeed at this task. moreover, there is no research (to the best of our knowledge) that showcases the potential benefit (or lack thereof) of using models pre-trained on other hate-speech data. therefore, the focus of our work is two-fold: thus, in our work, instead of solely focusing on developing a model that outperforms the current state-of-the-art architectures, we focus on the following broader research questions: our contributions, per the aforementioned research questions, are as follows: the rest of this paper is organized as follows: section 2 describes the related work, section 3 elaborates on the experiments we conduct as a part of our methodology and section 4 summarizes the results obtained. ## related work one of the first large-scale challenges that involved detecting hateful memes is the 'hateful memes challenge' organized by facebook ai @xcite . to quote the authors, "memes pose an interesting multimodal fusion problem: consider a sentence like "love the way you smell today" or "look how many people love you". unimodally, these sentences are harmless, but combine them with an equally harmless image of a skunk or a tumbleweed, and suddenly they become mean." they release the hateful memes dataset, consisting of 10,000 memes annotated for unimodal hate, multimodal hate, benign text, benign image, and random non-hateful examples. this was followed by many research efforts to categorize memes beyond hateful, such as zia et al. @xcite , who looked at classifying memes as racist and/or sexist, nafiah and prasetyo (2021) who focused on analyzing and identifying sexist memes during the covid-19 pandemic, and @xcite who used the presidential election to develop a dataset of memes consisting of racism, sexism and homophobia. the mami dataset @xcite was the first of its kind to motivate the sub-classification of misogynistic memes. this task, as a part of the semeval 2022 contest, showcased many noteworthy methodologies for the proposed problem. for example, @xcite proposed an r2d2 architecture that used pre-trained models as feature extractors for text and images. they used these features to learn multimodal representation using methods like concatenation and scaled dot product attention. this methodology achieved an f1 score of 0.757 and was ranked 3rd in subtask, and 10th on subtask b, with an f1 score of 0.690. in another study, @xcite develop an ensemble model consisting of xlm-roberta, distil-bert, resnext, and data-efficient image transformer to achieve an average f1 of 0.71 on task a and 0.69 on task b. however, these authors established an svm as their baseline. our goal is to explore a wider range of similar models, using such models as a baseline, and hopefully develop a better one. for now, we plan to use precision, recall, and f1 score as our evaluation metrics, along with a manual qualitative analysis that can provide insight into how to better direct future model improvements. another interesting approach is that proposed by @xcite , which combines bert and clip, achieving an f1 of 0.727 on sub task a, and an f1 of 0.710 on sub task b. @xcite extends a similar approach by using bert and fcnn, and testing it on the aforementioned facebook ai's hateful meme challenge dataset for generalisability. an approach that is similar to ours to an extent is that of @xcite , who test a variety of language models on the text part of the mami dataset. our approach involves using such models to establish a comparative baseline of language-only models and combine them with vision-based models to analyze how that affects model performance. finally, in another noteworthy experiment, @xcite proposes a clip text encoder and an lstm for the text encoding part of the model. this model attains an f1 score of 0.834 on subtask a and an f1 score of 0.731 on subtask b. in our work, we want to look beyond merely training a classifier that outperforms these methods. we are more interested in analyzing the finer details, and understanding what these models are doing wrong. further, we are interested in establishing comparison baselines through text models and vision models to gain insight into which feature is more important, and to what extent. to the best of our knowledge, prior to this, there has been no experimentation to show the benefits of using pretrained models for multimodal misogyny detection. ## methodology as discussed in section 2, following the semeval task itself, we will refer to task a as the classification of a meme as misogynistic and task b as the subclassification of a misogynistic meme. further, all models are finetuned on this dataset. as described later in the paper, bert* benefits from pretraining on the hateful meme dataset. ## conclusion in this research, we have delved into the challenging task of identifying misogynistic memes online. by utilizing the mami dataset with 12,000 annotated memes, we have established baselines and conducted experiments with various models, including text-only, vision-only, and multimodal models. our findings indicate that pretraining bert on hateful memes and utilizing an attentionbased approach with vit performs better than the state-of-the-art models by more than 10% for subtask a, and by 2% on subtask b. this highlights the importance of domain-specific pretraining in identifying multimodal misogyny. further, we have provided a comprehensive qualitative analysis of random samples from the test set, which provided insight into the challenges of detecting multimodal misogyny. our research emphasizes that identifying misogynistic memes online is a complex task that necessitates a thorough consideration of both visual and linguistic cues, and the significance of domain-specific pretraining in this area. future work includes extending the dataset to multiple languages to evaluate the generalizability of the proposed approach beyond english. additionally, a similar analysis could be performed on multimodal media such as reels and tiktoks to assess the effectiveness of the proposed approach on these platforms. further research is also needed to reduce the computational complexity of training and deploying these models for downstream tasks. finally, investigating the interpretability of the proposed approach could shed light on which multimodal cues are most indicative of misogyny, thereby helping to better understand the underlying mechanisms of this phenomenon.