An In-depth Analysis of Implicit and Subtle Hate Speech Messages (2023)

Abstract: The research carried out so far in detecting abusive content in social media has primarily focused on overt forms of hate speech. While explicit hate speech (HS) is more easily identifiable by recognizing hateful words, messages containing linguistically subtle and implicit forms of HS (as circumlocution, metaphors and sarcasm) constitute a real challenge for automatic systems. While the sneaky and tricky nature of subtle messages might be perceived as less hurtful with respect to the same content expressed clearly, such abuse is at least as harmful as overt abuse. In this paper, we first provide an in-depth and systematic analysis of 7 standard benchmarks for HS detection, relying on a fine-grained and linguistically-grounded definition of implicit and subtle messages. Then, we experiment with state-of-the-art neural network architectures on two supervised tasks, namely implicit HS and subtle HS message classification. We show that while such models perform satisfactory on explicit messages, they fail to detect implicit and subtle content, highlighting the fact that HS detection is not a solved problem and deserves further investigation.

## introduction the rising mass of communication through social media further exacerbates harmful consequences of online hate speech. as a result, social media have faced mounting pressure from civil rights groups demanding to ramp up their enforcement of antihate speech policies, so that to monitor and limit this kind of content. in the latest years, numerous methods have been developed to automatically identify this type of utterances expressing hateful or abusive content on social media using natural language processing methods. a variety of datasets have also been built, exemplifying various manifestations of this harmful content @xcite . however, most of the research carried out so far on this topic has focused on overt forms of hate speech. explicit hate speech is more easily identifiable by recognizing a clearly hateful word or phrase. only recently, a few works @xcite @xcite @xcite have started to focus on implicitness, where circumlocution, metaphor, or stereotypes are used to intentionally convey hatred towards a particular group. in those messages, hatefulness can be captured only by understanding their global meaning, as well as contextual information. in this paper, we carry out an in-depth analysis of implicit hs in standard benchmarks for hs detection. additionally, we define the notion of subtle hs that puts forward hateful meanings elusively relying on human perception and through the use of complex syntactic structures. in our study, we collect messages from 7 available datasets for hs detection that cover different topics and are extracted from different social media platforms, and we enrich them with the following three-layer annotation: hs/non hs, explicit/implicit and subtle/non subtle. we also provide a fine-grained annotation for implicit hs messages with 18 implicit properties such as irony, exaggeration, metaphor, and rhetorical question, among others. the newly created resource named ishate (implicit and subtle hate speech) provides a rich and variegate benchmark for pushing forward research on implicit and subtle hateful messages, and constitutes a challenging test-bed to evaluate computational approaches. 1 additionally, we evaluate sota and competitive baseline classifiers to detect both implicit and subtle hs in ishate, showing that current methods fail to effectively detect implicit and subtle hs messages due to their peculiar nature. note: this paper contains examples of language which may be offensive to some readers. they do not represent the views of the authors. ## related work in the latest years, there has been significant research on abusive language and hate speech detection using natural language processing (nlp) methods (e.g., @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ). a few works focus on subtypes of hs, such as @xcite that tackles the recognition of antisemitism, or @xcite ; @xcite ; @xcite that investigate predictive features to identify hs in the form of racism and sexism. in this context, several challenges and shared tasks have also been organized over the years, that made datasets and resources for multiple languages available (for a survey, see @xcite ). research studies carried out so far have mostly focused on overt forms of hate speech, while very few works address the issue of implicit and subtle hs @xcite . however, several works show awareness of the problem. for instance, @xcite and @xcite discuss systems' limitations in identifying hs messages which are ambiguous, have patterns of emotional speech or lack context. @xcite and @xcite highlight the complexity of recognizing hateful messages when the meaning is conveyed through sarcasm, stereotypes, complex syntactic structure, or non-explicit lexical patterns. among the few studies that attempted to address the issues of implicit and subtle detection, @xcite defines a shared task to detect implicit and explicit abusive messages from abuseval, a reannotated dataset based on olid/offenseval @xcite . @xcite paraphrases german hs tweets obtaining implicit and explicit messages to study classification methods. @xcite shows how taking user context improves cyberbullying detection with neither explicit profanities nor apparent neutral emotions. @xcite and @xcite explain why explicitness, implicitness, and subtlety are typologies of abusiveness and encourage researchers to develop proactive technologies in this area. @xcite introduces a taxonomy of implicit hate speech and a benchmark corpus with fine-grained labels for each message. @xcite proposes a large-scale approach to automatically generate benign and implicit hs statements through the language model gpt3. @xcite proposes resources to tackle implicitly abusive comparisons and abusive remarks, which are two subtypes of implicitness. @xcite explains the key issues in the implicit abuse detection, as well as possible future directions to explore. even though these studies set the basis to explore and model the issue of implicit hs, there is still large room for improvement, both in terms of creating adequate resources to investigate these aspects and in terms of computational approaches to address them. ## hs, implicitness and subtlety hate speech is defined as a direct attack against people -rather than concepts or institutionsbased on protected characteristics (pc): race, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity, and severe disease @xcite . we encompass in the concept refugees, migrants, immigrants, asylum seekers from the most severe attacks, and occupations when they are referenced along with pc, though commentary and criticism of immigration policies are excluded from hs. following this definition, hs differs from related concepts mainly since it is specifically oriented to groups of people with pc as the main target @xcite . in the following, we provide clear and operational definitions of explicit, implicit and subtle hs. reported examples are extracted from the white supremacy forum (wsf) @xcite and the conan @xcite datasets, described in section 4. ## the ishate dataset relying on the fine-grained definitions of hs provided in the previous section (explicit, implicit and subtle hs), we collect and enrich 7 available standard datasets for hs detection. as a result, we create the first benchmark for implicit and subtle hs detection on social media messages extracted from different sources. ## evaluation to show that implicit and subtle hs detection is still a very challenging task, we evaluate a set of state-of-the-art models for hs detection on the ishate dataset. we propose two 3-label classification tasks: to this goal, we consider the following models: universal sentence encoder (use) + svm @xcite . first-ranked model on the hateval benchmark @xcite . the use @xcite is a sentence embedding that encodes text into high dimensional vectors of 512 dimensions, trained on large data sources to provide an encoding method that works for various nlp tasks. an svm classifier with rbf kernel and default parameters is then used for classification. deberta v3 (hate_speech18). sota model on the wsf dataset @xcite . for classification, a default huggingface implementation of a one-layer feed forward network is used on top of deberta @xcite , a transformer-based model. the model is later fine-tuned for 4 epochs (learning rate of 2e-5, batch size of 32). bert @xcite . we use this language model to encode text sequences and classify them by adding a feed-forward neural network on top. hatebert. a re-trained bert model using over 1 million posts from banned communities on reddit @xcite and then fine-tuned on our dataset. hatebert obtained very promising results in the benchmarks hateval, offenseval @xcite , and abuseval @xcite . as for preprocessing, we replace long non-space character chains for only one occurrence, and delete digits, special symbols, and urls. ## conclusions in this paper, we have presented ishate, the first benchmark dataset annotated with both implicit and subtle hs labels, which represents a challenging test-bed to evaluate computational approaches. we also provide a fine-grained annotation for implicit hs messages with 18 implicit properties which represent the relevant features that hs classifiers should possess to improve implicit hs detection. it has been created enriching 7 existing datasets for hs detection over different topics and from different social media. we have shown that current sota models fail to properly detect implicit and subtle hs messages as peculiar features connected to sentiment, inference, context and irony, as well as complex syntactic structure, cannot be properly understood. we also investigated data augmentation strategies to increase the number of instances for the minority classes. we show that -while they cannot be the ultimate solution to the lack of implicit and subtle examples -they still play a role in improving the systems' performances, in line with @xcite . as for future work, we plan to propose alternative large-scale methods to collect implicit and subtle messages by targeting "hateful" users, manual creation @xcite or refining human-in-the-loop generative methods as in @xcite . also, we will investigate features modeling implicit properties @xcite @xcite and new model architectures for hs detection @xcite .