What Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection (2023)

Abstract: Hate speech has unfortunately become a significant phenomenon on social media platforms, and it can cover various topics (misogyny, sexism, racism, xenophobia, etc.) and targets (e.g., black people, women). Various hate speech detection datasets have been proposed, some annotated for specific topics, and others for hateful speech in general. In either case, they often employ different annotation guidelines, which can lead to inconsistencies, even in datasets focusing on the same topics. This can cause issues in models trying to generalize across more data and more topics in order to improve detection accuracy. In this paper, we propose, for the first time, a topic-oriented approach to study generalization across popular hate speech datasets. We first perform a comparative analysis of the performances of Transformer-based models in capturing topic-generic and topic-specific knowledge when trained on different datasets. We then propose a novel, simple yet effective approach to study more precisely which topics are best captured in implicit manifestations of hate, showing that selecting combinations of datasets with better out-of-domain topical coverage improves the reliability of automatic hate speech detection.

## introduction on social media and other online communication platforms, hate speech (hs hereafter) is found in many forms, from textual harassment to threats, targeting an individual or group (e.g., black people, women), based on some characteristics, such as race, color, ethnicity, gender, sexual orientation, nationality, religion, etc, which we refer to as topics. while the broad nature of these topics of hs is generally understood @xcite , determining whether a social media post is a manifestation of hs (and if so, to which topic(s) it belongs to) is not a trivial task for humans and automated machine-learning systems. indeed, the latter often require large quantities of annotated data, which in the case of hs, is made difficult due to (1) the absence of a comprehensive definition of these topics; but also, (2) the potential internal biases and subjectivity present in annotators and/or annotation guidelines employed to construct annotated datasets @xcite @xcite . to palliate these issues, cross-dataset evaluation has become an active line of research aiming at studying the generalization capabilities of hs detection systems to unseen data during training @xcite @xcite . as data collection and annotation is an expensive and time-consuming process, current approaches use mixtures of existing hs detection datasets to study generalization across different social media platforms @xcite @xcite or across different manifestations of hs, often relying on one-to-many (train on one dataset/mixture, test on others) experimental settings @xcite @xcite @xcite . in most of these, a unification scheme is proposed to adapt the original datasets' potentially fine-grained annotations to a set of binary labels, which can often fail to take into account the heterogeneity inherently present across different encodings of hs @xcite . in addition, some datasets intentionally focus only on some specific kinds of manifestations of hs covering for example gender (e.g., misogyny, sexism), ethnicity, religion or race (e.g., xenophobia, anti-immigrants/refugees hs), i.e., they are topic-specific, whereas others attempt to cast a wider net without specific sets of topics in mind, i.e., they are topic-generic. in either case, variations will always exist, either due to the reasons mentioned above, or due to shifts in the content found on social media through time. all of these may cause significant differences in the topics of hs that classification models might learn to recognize, which ultimately might make them less reliable, particularly in out-domain contexts @xcite . acknowledging these issues and being aware of the noise that will invariably be introduced by mixing together different datasets, we propose for the first time to empirically study what modern transformer architectures can effectively learn to generalize from existing datasets, in a unified setting, with a focus on their topical nature. more precisely: do models learn similarly from topicgeneric datasets as from mixtures of topic-specific datasets? does this acquired knowledge generalize to implicit expressions of hs, and if so, which finergrained hs topics are learned by these models? our contributions are: ## related work a number of previous works have studied generalizability of hs detection datasets and models, with different focuses: for example, @xcite and @xcite have analyzed the compatibility of many hs datasets (including some used in this study), both in terms of their properties (origin of data, annotated phenomena, class definitions, etc.) and empirically with intra-and inter-dataset generalization experiments. they conclude that model choice, intra-dataset performance, and the type of phenomenon being classified, are the most important factor that determine generalization. in particular, they conclude that phenomena like toxicity, abuse, or offensiveness that are often lexicalized are easier to generalize than hate speech, something which was also confirmed and discussed in other works @xcite . in this paper, we are interested in the topic-oriented nature of hs specifically, and thus discard these former classes from our study. alternative solutions have recently been proposed to handle unseen topics by building new target-oriented datasets from scratch, such as the hatexplain dataset @xcite , with an additional labeling of the target topics of hate (race, religion, and origin). while such linguistic resources are valuable for the research community, we believe existing larger datasets could be successfully exploited for generalization, as mixtures of datasets, as shown for example by @xcite , @xcite , or, in a different fashion, by multitask hs detection systems or domain adaptation techniques @xcite @xcite . in this paper, we continue this line of research by proposing for the first time, as far as we know, models able to generalize across topic generic vs. specific datasets as well as predict fine-grained manifestations in implicit hate messages. this is particularity challenging as these manifestations are more difficult to generalize due to their limited lexical features @xcite . to this end, we rely on transformer models like bert that have been shown to be able to generalize better overall than previous architectures @xcite . in particular, we experiment with both existing bert-like models, some preadapted to the domain of hs, as well as t5 @xcite , a text-to-text architecture that has never been used in a generalizability study. ## datasets we experiment with six popular and freely available english tweets corpora (or english subsets thereof) from previous studies. we first present the datasets and how they were used in a common experimental setting. we then provide some discussions on the compatibility of these datasets, and how this may impact the generalization performances in our experiments. ## models to study how different pre-trained models and architectures may differ in how they capture hs in various settings, we choose five pre-trained transformers from the literature. among them, the last three have been adapted to the domain of hs detection, either through pretraining or pre-finetuning, on data related directly or indirectly to hs detection. for the experiments, we relied on the hugging face transformers library @xcite . roberta-base @xcite is an optimized bert-like @xcite encoder transformer commonly used for various nlp classification tasks, including hs detection. t5-base is the 220 million parameters pretrained variant of the t5 architecture, initially proposed by @xcite . it differs from bert-like encoder transformers in that it is a text-to-text model, for which classification tasks are reframed as text generation, with the output labels used in their textual form (in our case, "normal" and "hateful"), and as this particular variant was also pretrained on various supervised tasks (sentiment analysis, natural language inference, and question answering, etc.), a task prefix is traditionally prepended to the input text, which for our fine-tuning, we fix to "hate speech: ". fbert @xcite and hatebert @xcite are two models derived from bert, retrained with a masked language modelling (mlm) objective on over 1.4 million social media offensive posts from the solid dataset, and reddit abusive language english dataset (ral-e), respectively. toxdectroberta was proposed by @xcite , and is a roberta-large model, finetuned on founta, with the hateful and abusive classes merged into a single toxic class. the authors use the methods proposed by @xcite and @xcite to attempt to avoid dataset bias issues, such as spurious correlations between particular lexical and dialectical markers (such as those found in african american english) with the toxic class. ## cross-topics generalization we first study the differences in generalizability between topic-generic and topic-specific datasets and their potential mixtures, in a cross-dataset/mixture setting: in each individual experiment, one of the previously described models is trained on one dataset/mixture and tested on all individual test sets. ## finer-grained topics analysis to better understand which manifestations of hs can be generalized to out-of-domain data, we then perform a finer-grained analysis of the topics learned by our models, by relying on an implicit hate speech dataset, for which fine-grained target annotations are available. ## conclusion and perspectives in this work, we have proposed a set of topicoriented analyses of the generalizability of hs datasets. we have shown how topic-generic and topic-specific datasets yield different degrees and nature of generalization, both when used individually, or as mixtures of datasets. with the former, we found that, while not very successful at generalizing in-domain to topic-specific datasets, the use of mixtures allows smoothing out individual weaknesses. with topic-specific datasets, we found that generalization is possible, both for single-topic and multi-topic mixtures. through a finer-grain out-ofdomain generalization analysis, we showed how a priori somewhat similar datasets can vary wildly in the forms of implicit hs that can be learned from them. implicit expressions of hate, with few lexical features, are more difficult to generalize, as models can struggle to capture underlying hateful intents in messages. notably, one barrier to understanding more implicit manifestations of hate is the lack of context for individual social media posts: more conversational datasets could represent an interesting avenue of research in that regard. these are all important considerations for the purpose of constructing more reliable automatic hs detection systems, intended to function on raw, potentially noisy, and never-seen-before data. in particular, we found that topic-generic datasets like founta @xcite appear promising for future research on generalization-optimized mixtures of datasets. finally, concerning hs detection architectures, we found that, for bert-like models, predomain adapted variants generalize slightly better than a more generic roberta classifier, but yield similar results to a generic text-to-text t5 architecture, which seems promising for future research in hs detection. in future work, we will explore the use of these approaches to guide data selection, by for example employing the method proposed by @xcite , in order to construct mixtures of hs datasets better suited to out-of-domain generalization. alternatively, active learning methods have been shown to be successful in nlp @xcite , and could be used to palliate annotated scarcity in hs detection: by using topic-oriented analysis methods to detect difficult-to-predict topics, better targeted additional annotations could be acquired, to help improve the topical coverage of automated systems.