H ate U at S em E val-2022 Task 5: Multimedia Automatic Misogyny Identification (2022)

Abstract: Hate speech expressions in social media are not limited to textual messages; they can appear in videos, images, or multimodal formats like memes. Existing work towards detecting such expressions has been conducted almost exclusively over textual content, and the analysis of pictures and videos has been very scarce. This paper describes our team proposal in the Multimedia Automatic Misogyny Identification (MAMI) task at SemEval 2022. The challenge consisted of identifying misogynous memes from a dataset where images and text transcriptions were provided. We reported a 71% of F-score using a multimodal system based on the CLIP model.

## introduction expressions of hate are common in online environments, and they can appear in different types of multimedia content @xcite . however, the related work on hate-speech and offensive language detection is primarily focused on textual english content @xcite . but, even for the english language, the task is still not solved. evidence of that is recent reports of the increasing amount of hateful content in social media foot_0 following the occurrence of social or political events. recent events like the covid pandemic have brought a new wave of hate @xcite , with new targets and expressions including hateful memes @xcite . therefore, the techniques for hate speech detection need to evolve towards new types of hate, representations, and languages. the lack of generality of existing resources along with the emergence of new nets of hate makes current systems quickly outdated foot_1 . most of the available datasets contain tweets @xcite , facebook and youtube comments @xcite and, in general, textual content. similar to the hateful memes challenge foot_2 hosted by facebook in 2020, the multimedia automatic misogyny identification (mami) challenge (elisabetta fersini, 2022) is an excellent opportunity for covering the hate speech detection task beyond written expressions. in this competition, the organizers provided a training set of 10,000 memes labeled as hate speech in two different forms: binary (misogynous, not misogynous) and multi-class (stereotype, shaming, objectification, and violence). the competition comprises two tasks: task a, for binary identification of misogyny, and task b, for fine-grained classification of misogynous memes. for final system evaluation, the organizers published a set of 1000 extra unlabeled memes. each meme in training and testing sets consists of an image with an overlay text. each object in the dataset consists of an image and a transcription of the overlay text. this paper describes our team participation in task a of the mami challenge. we encoded images and texts using a pre-trained multi-modal model based on the clip model @xcite . we combined the encoded vectors in different ways to obtain a final classification output. our best result reported was 71% of f-score. in section 2 we describe the work that has been done on hate speech detection using multi-modal content. in section 3 we described the training dataset provided in the competition. then, in section 4 we describe our system and experiments. our conclusions can be read in section 6. ## background most of the research in hate speech detection has been conducted over textual datasets @xcite @xcite . several strategies based on machine learning models and natural language processing have been used to solve the task, though without success. on the other hand, the identification of hate other than text formats of multimedia content has been treated only in a few works. @xcite 2017 have took advantages of the multi-modal information they could extract from instagram foot_3 for they work on cyberbullying detection. while perez-martin et al. (2020) used the multi-modal representations for retrieving twitter memes from textual queries. fortunately, in recent years the multi-modal detection of hateful content has gained popularity due to competitions like "the hateful memes challenge" (velioglu and rose, 2020) hosted by facebook where different models were proposed to detect hateful content on memes. the proposed approaches encompass different visual state of the art models like visualbert @xcite , lxmert @xcite , vilbert @xcite among others. the winning system combined some of these models with predefined rules @xcite for improving the classification accuracy of difficult samples. another recent result on multimodal detection of offensive content has addressed the detection of harmful memes related to the covid pandemic, also contributing with a new meme dataset @xcite . there is much to do in the multimedia offensive language detection in images and video, considering the popularity of social networks like instagram and tik tok foot_4 . ## dataset description the dataset is composed of 10 000 memes, 5000 of which are labeled as misogynous and 5000 as not misogynous. for each meme is provided the corresponding image in jpg format and meme text transcription. all texts are in english; the most extensive text transcription found in the dataset contains 252 words, while the shortest contains one word. a characteristic of this dataset is that in some examples, only the text is enough for determining the nature of the comment (see figure 1 ). we do not have evidence of an example where system f-score text_only 69.23 image_only 65.37 clip_concat 70.50 clip_sum 71.20 ## experiments and results though we experimented with several models for texts and images, our best result was obtained using the clip model as the core of our system. clip model: the clip model proposed by radford et al. 2021 exploits the state-of-the-art textual and visual approaches for learning about images from texts. the general idea of the clip training strategy is to jointly learn image and text representations and predict the most similar pairs (image, text). according to the authors, the model can competitive transfer to different vision tasks. clip based systems: we use a pre-trained clip model 6 for learning text (text_clip) and image (text_clip) representations from the texts transcriptions and images provided for the competition. we combined these outputs in different ways to obtained a vector x used as input for a classification final classification. the different results can be found in image_only: similar to the text_only system, we investigated if only the images were enough for successfully detecting misogyny using this dataset. the f-score obtained after five epochs is 65%, a lower result than the text_only system. the results improved by using both representations to a 70% of f-score. clip_sum_system: in this variant of the system, we sum both image and text representation in one single vector. this sum was pondered by a trainable parameter of the model a. the idea of this combination is to give the possibility to the model of using the necessary weights for image and text. with this combination we obtained our best reported result for the competition. ## error analysis we observed the memes miss classified by our best model (clip_sum). the most common type of error was the false negative error, examples wrongly classified as not misogynist, we noticed that most of them represent male figures or inanimate objects. only a small number of memes picture a woman as the central figure (see figure 2 ). on the other hand, the female figures in the false positive examples is very common (see figure 3 ). figure 3: meme example 15977 from the testing set. our model clip_sum wrongly classified it as a misogynist meme. the text transcription is: "2020 before and after" this phenomenon could be caused by a particular bias in the training set that relates misogyny memes with the images of women. but a deeper analysis has to be conducted in this regard. ## conclusions this paper describes our team participation in task 1 of the multimedia automatic misogyny identification (mami) at semeval 2022. the purpose of this task was to identify memes as misogynists or not. images and texts were provided in a training set of 10000 examples. our team implemented a system based on the pre-trained clip approach and reported a 71% of f-score. the multimodal hate speech detection has been under-addressed through the years and recently is gaining popularity, though there is still much for research in this regard. moreover, other types of multimedia, like videos, need to be analyzed since they are popular ways of communicating on social networks.