VL - BERT +: Detecting Protected Groups in Hateful Multimodal Memes (2021)

Abstract: This paper describes our submission (winning solution for Task A) to the Shared Task on Hateful Meme Detection at WOAH 2021. We build our system on top of a state-of-the-art system for binary hateful meme classification that already uses image tags such as race, gender, and web entities. We add further metadata such as emotions and experiment with data augmentation techniques, as hateful instances are underrepresented in the data set.

## introduction in this work, we present our submission to the shared task on hateful memes at woah 2021: workshop on online abuse and harms. 1 detecting hateful memes that combine visual and textual elements is a relatively new task @xcite . however, research can build on earlier work on the classification of hateful, abusive, or offending textual statements targeting individuals or groups based on gender, nationality, or sexual orientation @xcite . ## system description in this paper, we exploit the analysis proposed by @xcite for the fine-grained categorization of hateful memes. ## results and discussion when the model is trained on the train set along with augmented data, hardly any significant performance improvement is encountered. this is 3 ). from the confusion matrices ( for the final submission, we generate predictions on the test set using our two best-performing models based on their auroc score -vl-bert large +w,rg,e (winning solution) and +w,rg (2 nd rank) (see ## summary in this paper, we presented our approach to identify and categorize attacked protected groups in hateful memes. we performed experiments using a visual-linguistic pre-trained model called vl-bert large along with metadata information extracted from the meme image and text. results