Introduction
The surge of interest in combating online hate
led to increased efforts in creation of benchmark datasets and organization of shared tasks, and, as a consequence, rapid development of hate speech (HS) detection models (Caselli et al., 2020 ; Poletto et al., 2021 ) . However, state-of-the-art HS detectors do not perform equally well across different datasets (Fortuna et al., 2021 ) and different target identities (Ludwig et al., 2022 ) .
These performance discrepancies have been attributed to diverging dataset annotations (Fortuna et al., 2020 ) , out-of-domain distribution (Jin et al., 2023 ) , spurious correlation between specific target identities and the labels (Ramponi and Tonelli, 2022 ) , and specific topical focuses (Bourgeade et al., 2023 ) .

Unfortunately, none of these diagnoses resulted so far in a revision of the datasets or models, and, as a matter of fact, this is not surprising. Instead of treating HS detectors as black boxes and HS datasets as given commandments, the research community should understand what the datasets entail and how the models behave under different circumstances. Such insights can help us make practical progress in building more robust and fair classifiers (Chen et al., 2024 ) beyond pushing a single metric such as accuracy or F 1 score.

Some work has already been done in this direction.
To provide more diagnostic insights, Röttger et al. ( 2021 ) introduced HateCheck , a comprehensive suite of functional tests that covers 29 model “functionalities” across seven target identities. Each functionality tests the models’ behavior on a specific kind of hateful or contrastive non-hateful content (e.g., “denouncements of hate.”). To generate examples at scale, handcrafted templates (Ribeiro et al., 2020 ) for each functionality (e.g., “ [identity] belong in a zoo.”) have been used. Jin et al. ( 2024 ) further improved HateCheck by substituting its simplistic and boilerplate examples by a new dataset GPT-HateCheck , with LLMs-generated test cases. They demonstrate that the new dataset has better lexical diversity and naturalness than HateCheck .

In this work, we aim to disentangle the impact of different factors on HS prediction by tapping on the two aforementioned functionality test datasets. Specifically, we study the difference in predicted scores among minimal sets from HateCheck , where the only variable is the mention of the target identity (Section 3.1 ). We then identify fine-grained emotions (Section 3.2 ) and stereotypes (Section 3.3 ) from GPT-HateCheck and analyze their impact on classifiers’ performance; Figure 1 illustrates the analyses we conduct. The experiments reveal that HS detection models possess a systematic bias based on specific target identity mentions. Models predict accurately in case of intense stereotypes but struggle when the stereotype is mild. What is more concerning is that models tend to misclassify non-hateful posts expressing negative emotions as hateful, such as counter-speech or posts expressing sadness towards HS. Our contributions are threefold:

We quantitatively measure the impact of different factors on HS prediction.

We conduct emotion and stereotype analyses of the recently-introduced GPT-HateCheck dataset. To the best of our knowledge, it is the first systematic analysis of the impact of emotions and stereotypes on HS prediction.

We highlight critical model weaknesses, such as the confusion of hate with negative emotions. These findings shed light on new directions for improving the robustness and fairness of HS detectors.