Findings of the WOAH 5 Shared Task on Fine Grained Hateful Memes Detection (2021)

Abstract: We present the results and main findings of the shared task at WOAH 5 on hateful memes detection. The task include two subtasks relating to distinct challenges in the fine-grained detection of hateful memes: (1) the protected category attacked by the meme and (2) the attack type. 3 teams submitted system description papers. This shared task builds on the hateful memes detection task created by Facebook AI Research in 2020.

## introduction the spread and impact of online hate is a growing concern across societies, and increasingly there is consensus that social media companies must do more to counter such content @xcite . at the same time, any interventions must be balanced with protecting people's freedom of expression and ability to engage in open discussions. ensuring that online spaces are both open and safe requires being able to reliably and accurately find, rate and remove harmful content such as hate. scalable machine learning based solutions offer a powerful way of solving this problem, reducing the burden on human moderators. to date, detecting online hate has proven remarkably difficult and concerns have been raised about the performance, robustness, generalizability and fairness of even state-of-the-art models @xcite @xcite @xcite . to advance the field, and develop models which can be used in real-world settings, research needs to go beyond simple binary classifications of textual content. to this end, we have used trained professional moderators to reannotate the hateful memes dataset from @xcite foot_0 . it contains two sets of labels, which correspond to our two sub-tasks: the protected category that has been attacked (e.g., women, black people, immigrants) as well as the type of attack (e.g., inciting violence, dehumanizing, mocking the group). detecting hateful memes is a particularly challenging task because the content is multi-modal rather than uni-modal, such as text or images alone. when humans look at memes they do not think about the words and photos independently but, instead, combine the two together. in contrast, most ai detection systems analyze text and image separately and do not learn a joint representation. this is inefficient and limits the performance of systems. they are likely to fail when an image that by itself is non-hateful is combined with nonhateful text to produce content that expresses hate through the interaction of the image and text. for ai to detect hate communicated through multiple modalities, it must learn to understand content the way that people do: holistically. in this paper we present the results of the woah 5 shared task on fine-grained hateful memes detection. ## conclusion detecting hate remains technically difficult, with many unaddressed or unsolved challenges and frontiers. hateful memes are one issue that has received little attention, despite the ubiquity of such media online. the shared task at woah 5, with two subtasks for fine-grained detection of the protected category and the attack type, is another step forward in this still-nascent research area. for future work, we hope to scale the finegrained annotations to other hate speech datasets, as we think it is important toi develop classifiers that can detect the nuances of hate speech. meanwhile, the annotated datasets are publicly available and we welcome researchers to make use of them.