Unmasking the Hidden Meaning: Bridging Implicit and Explicit Hate Speech Embedding Representations (2023)

Abstract: Research on automatic hate speech (HS) detection has mainly focused on identifying explicit forms of hateful expressions on user-generated content. Recently, a few works have started to investigate methods to address more implicit and subtle abusive content. However, despite these efforts, automated systems still struggle to correctly recognize implicit and more veiled forms of HS. As these systems heavily rely on proper textual representations for classification, it is crucial to investigate the differences in embedding implicit and explicit messages. Our contribution to address this challenging task is fourfold. First, we present a comparative analysis of transformer-based models, evaluating their performance across five datasets containing implicit HS messages. Second, we examine the embedding representations of implicit messages across different targets, gaining insight into how veiled cases are encoded. Third, we compare and link explicit and implicit hateful messages across these datasets through their targets, enforcing the relation between explicitness and implicitness and obtaining more meaningful embedding representations. Lastly, we show how these newer representation maintains high performance on HS labels, while improving classification in borderline cases.

## introduction the proliferation of hate speech (hs) on social media platforms has become a pressing concern in online social communities. while significant progress has been made in the development of hs detection methods, current sota models focus on detecting explicit hs, leaving implicit hate cases undetected @xcite . this issue is aggravated by the sheer volume of implicit hate speech content being spread across various online platforms, necessitating automated approaches to detect them effectively. implicit hs detection poses unique challenges compared to its explicit counterpart: it contains coded, ambiguous or indirect language that does not immediately denote hate, but still disparages a person or a group based on protected characteristics such as race, gender, cultural identity, or religion (e.g., "i think it is a bit late to think to look after the safety and the future of white people in south-africa" -the white supremacy forum dataset @xcite ). the performance of current hs systems heavily relies on how coded language is represented and how well classifiers can capture the underlying semantic meaning of messages through embeddings. hence, obtaining better text representation becomes crucial in effectively identifying implicit hs messages. in this direction, the goal of our work is to bridge the gap between explicit and implicit messages, aiming to enhance the embedding representations of sota models. our contribution is fourfold: i) we analyze the embedding representations of five benchmark datasets with veiled hateful content, examining the levels of explicitness and implicitness, through cross-evaluation using state-of-theart transformer models. ii) we examine the embedding representations of implicit messages across different target groups. through this analysis, we gain insights into how implicit hs messages are encoded based on their target groups. iii) we propose a novel approach to link explicit and implicit hs messages in the representation space. iv) we illustrate that the newer representation space preserves strong efficacy for hs labels, while also refining classification in borderline instances. using contrastive learning techniques @xcite @xcite , we aim to push explicit and implicit messages effectively enforcing the uncovered relation between these two notions and thereby obtaining more meaningful representations than those obtained through fine-tuning learning methods. 1 note: this paper contains examples of language which may be offensive to some readers. they do not represent the views of the authors. ## related work hs detection has been extensively studied by the research community providing multiple resources, such as lexicons @xcite , datasets @xcite @xcite , and supervised methods @xcite @xcite (for a survey, see @xcite ). these studies provide a solid starting point to examine the problem of abusive language, especially in social media messages. lately, there has been growing interest in the detection of implicit hs, which provides additional challenges. datasets specifically designed for implicit hs @xcite @xcite @xcite , more solid veiled detectors @xcite , guided augmentation strategies @xcite , and theoretical analysis @xcite @xcite have been recently proposed to advance in this direction. however, little attention has been dedicated to effectively represent implicit messages through embeddings on these benchmarks. embeddings play a crucial role in the performance of classifiers @xcite @xcite , yet their application to capture the implicit nature of hs has been underinvestigated. in this direction, @xcite tackles cross-dataset underperforming issues on hs classifiers and proposes a contrastive learning method that encodes a hateful post and its corresponding implication close in representation space, closely depending on the annotated implications and without contra positioning explicitness with implicitness. @xcite captures topic-generic and topic-specific knowledge when trained on different data to improve generalization. ## conclusions our contribution in this study is fourfold: i) we studied how models' embeddings capture hs w.r.t. explicitness and implicitness, ii) we showed how explicit and implicit hs messages result in similar encodings if grouped by their protected target, iii) we analyzed a contrastive learning method to force this property when representing implicit text. we prove our research hypothesis on 5 hs benchmarks, moving a step forward in bridging the gap between explicitness and implicitness, and iv) we show how the newer representation space maintains high performance on hs labels while improving classification in borderline cases. in future work, we'll refine contrastive learning, delving into contextual pairing based on other semantic dependencies between explicit and implicit cues, aiming to sharpen nuanced hate speech detection.