Identifying Slurs and Lexical Hate Speech via Light-Weight Dimension Projection in Embedding Space (2023)

Abstract: The prevalence of hate speech on online platforms has become a pressing concern for society, leading to increased attention towards detecting hate speech. Prior work in this area has primarily focused on identifying hate speech at the utterance level that reflects the complex nature of hate speech. In this paper, we propose a targeted and efficient approach to identifying hate speech by detecting slurs at the lexical level using contextualized word embeddings. We hypothesize that slurs have a systematically different representation than their neutral counterparts, making them identifiable through existing methods for discovering semantic dimensions in word embeddings. The results demonstrate the effectiveness of our approach in predicting slurs, confirming linguistic theory that the meaning of slurs is stable across contexts. Our robust hate dimension approach for slur identification offers a promising solution to tackle a smaller yet crucial piece of the complex puzzle of hate speech detection.

## introduction recent years have seen an increase in attention towards hate speech detection due to the rising prevalence of online hate speech and its negative impact on society @xcite . current approaches to hate speech detection focus on identifying hate speech at the utterance level @xcite , which remains a challenging task due to the nuanced and complex nature of hate speech. hate speech can take many different forms, especially in the context of social media platforms where language use is dynamic and constantly evolving @xcite . this paper aims to tackle one aspect of hate speech detection by identifying hate speech at the lexical level, specifically through the identification of slurs based on their contextualized representations. breaking down the problem into smaller pieces allows us to focus on specific aspects of hate speech, such as slurs, and understand how "hatefulness" is encoded as a dimension of meaning in the embedding space of language models. this, in turn, can inform the development of more robust hate speech detection methods. slurs can be defined as terms referencing a specific social group, and generally carry derogatory connotations, regardless of the situation in which they are used. thus, the meanings of slurs remain relatively consistent across different contexts @xcite . in contrast, more subtle forms of hate speech such as dog whistles or expressions that depend on the speaker or audience can vary widely in their interpretation. for instance, the word "bitch" may be used as a derogatory term towards women, but among women, it can also be used casually in certain contexts @xcite . slurs are commonly included in hate speech lexicons, yet systematic study into automatically identifying them has barely been touched upon. nevertheless, their semantic consistency across contexts makes slurs a promising target for identification based on their embedding representation. word embeddings have demonstrated their effectiveness in capturing various aspects of meaning, including relations like synonymy and antonymy, but also more abstract concepts like cultural or social bias. as a prominent example, previous studies have shown that gender bias can be detected by extracting the relevant semantic dimension from the embedding model @xcite . these "semantic dimension identification" techniques typically rely on a small set of carefully selected words or word pairs that only differ with respect to the semantic dimension of interest. although this approach has demonstrated its capability to generalize to different dimensions of meaning, there are still open questions and chal-lenges. for example, the task of selecting the right set of words or word pairs to capture a specific dimension of meaning is still indistinct. building on linguistic theories on slurs and the findings of dimension identification studies, we hypothesize that slurs have a systematically different representation than their neutral counterparts in the embedding space, and that this difference can be identified using existing methods for discovering semantic dimensions within word embedding models @xcite . specifically, our study addresses the following research questions: 1) can we identify slurs based on their contextualized word embeddings? 2) how do we leverage dimension-based methods for slur identification? 3) can we confirm existing work in linguistics which suggest that the meaning of slurs is stable across contexts? 4) can we use the hate dimension identified based on slurs for detecting other lexical units pertaining to hate speech? in addressing these questions, we focus on methodological aspects such as the selection of lexical pairs and leveraging a pre-trained contextualized language model and incorporating multi-word expressions. in addition, our research puts emphasis on the robustness of the proposed methods across various hate speech domains and datasets without the need of big annotated data. to sum up, this paper presents a more targeted and efficient method for detecting hate speech, that aims to identify and gain more insight into the use of slurs in online discourse. 2 ## related work in this section, we review existing research on two key areas related to our study. first, we address hate speech at the lexical semantic level, with a particular focus on slurs. second, we discuss previous work on semantic dimension identification and its applications in computational semantics. ## methodology our approach can be divided into 1) the selection of pairs and extraction of their lexical representations from a language model, 2) the creation of a dimension vector based on those representations, and 3) the projection of lexical representations onto this dimension. we outline each of these components in sections 3.1 through 3.3, respectively. ## data & experiments the ingredients for our semantic dimension approach using a contextualized language model are lexical units and contexts in which they occur. the subsequent sections will introduce the data that we utilize to construct dimensions (4.1) and to assess them (4.2), respectively. ## results in the following, we present the results of our default set-up on two test sets, as well as experiments examining the impact of the selection of pairs (5.1). furthermore, we analyze the effect of the number of contexts included and the domain they are sourced from (5.2). finally, we discuss our findings on our third test set that includes other categories besides persons (5.3). ## discussion & future directions the results indicate that our dimension approach is effective in predicting slurs based on their contextualized embedding, with the importance of selecting pairs carefully to create a robust hate dimension. due to the lack of a universally agreed-upon definition of hate speech, the creation of hate speech datasets is difficult and prone to bias @xcite . as a result, datasets are often limited in size and scope, making it challenging to train models that can effectively detect a wide range of hate speech in different domains. our results demonstrated that the effectiveness of the dimension is not significantly influenced by the size of the context sample. this implies that our dimension approach is a promising cost-effective and domain-agnostic method for identifying slurs with low-data requirements. ## conclusion this paper addresses the complex puzzle of hate speech detection by breaking it down and concentrate on a smaller but crucial piece, the identification of slurs. we propose a novel approach that applies semantic dimension identification with contextualized embeddings to the detection of slurs. in this study, we set out to address several key research questions concerning the identification of slurs. first, we investigated whether slurs can be identified based on their contextualized word embeddings. our experimental results demonstrated the effectiveness of our method in predicting slurs by leveraging contextual representations, thereby affirming their effectiveness. simultaneously, we explored the application of dimension-based methods for slur identification. our findings highlight the significance of carefully selecting lexical pairs while demonstrating that extensive data is not necessarily required. additionally, we aimed to confirm existing work in linguistics, which suggests that the meaning of slurs is stable across contexts. findings on our experiments across different domains and datasets align with linguistic theory, as evidenced by consistently strong prediction performance. lastly, we explored the potential of utilizing the hate dimension identified based on slurs for detecting other lexical units related to hate speech. our method exhibited promising results in detecting other categories of lexical hate speech, showcasing its broader applicability potential beyond slurs. in conclusion, our approach contributes to a more targeted and efficient method for detecting hate speech and sheds light on the use of slurs in online discourse.