What the #?*!: Disentangling Hate Across Target Identities (2025)

Abstract: Hate speech (HS) classifiers do not perform equally well in detecting hateful expressions towards different target identities. They also demonstrate systematic biases in predicted hatefulness scores. Tapping on two recently proposed functionality test datasets for HS detection, we quantitatively analyze the impact of different factors on HS prediction. Experiments on popular industrial and academic models demonstrate that HS detectors assign a higher hatefulness score merely based on the mention of specific target identities. Besides, models often confuse hatefulness and the polarity of emotions. This result is worrisome as the effort to build HS detectors might harm the vulnerable identity groups we wish to protect: posts expressing anger or disapproval of hate expressions might be flagged as hateful themselves. We also carry out a study inspired by social psychology theory, which reveals that the accuracy of hatefulness prediction correlates strongly with the intensity of the stereotype.

## introduction the surge of interest in combating online hate led to increased efforts in creation of benchmark datasets and organization of shared tasks, and, as a consequence, rapid development of hate speech (hs) detection models @xcite . however, state-of-the-art hs detectors do not perform equally well across different datasets @xcite and different target identities @xcite . these performance discrepancies have been attributed to diverging dataset annotations @xcite , figure 1 : overview of our approach. we analyze target identity mentions' impact on hatefulness prediction in a minimal set experiment on the hatecheck dataset. we then extract fine-grained emotions and stereotypes from examples in gpt-hatecheck dataset to analyze the distributional difference among target identities and its impact on the classifiers' performance. unfortunately, none of these diagnoses resulted so far in a revision of the datasets or models, and, as a matter of fact, this is not surprising. instead of treating hs detectors as black boxes and hs datasets as given commandments, the research community should understand what the datasets entail and how the models behave under different circumstances. such insights can help us make practical progress in building more robust and fair classifiers @xcite beyond pushing a single metric such as accuracy or f 1 score. some work has already been done in this direction. to provide more diagnostic insights, @xcite introduced hatecheck, a comprehensive suite of functional tests that covers 29 model "functionalities" across seven target identities. each functionality tests the models' behavior on a specific kind of hateful or contrastive non-hateful content (e.g., "denouncements of hate."). to generate examples at scale, handcrafted templates @xcite for each functionality (e.g., " @xcite belong in a zoo.") have been used. @xcite further improved hatecheck by substituting its simplistic and boilerplate examples by a new dataset gpt-hatecheck, with llms-generated test cases. they demonstrate that the new dataset has better lexical diversity and naturalness than hate-check. in this work, we aim to disentangle the impact of different factors on hs prediction by tapping on the two aforementioned functionality test datasets. specifically, we study the difference in predicted scores among minimal sets from hate-check, where the only variable is the mention of the target identity (section 3.1). we then identify fine-grained emotions (section 3.2) and stereotypes (section 3.3) from gpt-hatecheck and analyze their impact on classifiers' performance; figure 1 illustrates the analyses we conduct. the experiments reveal that hs detection models possess a systematic bias based on specific target identity mentions. models predict accurately in case of intense stereotypes but struggle when the stereotype is mild. what is more concerning is that models tend to misclassify non-hateful posts expressing negative emotions as hateful, such as counterspeech or posts expressing sadness towards hs. our contributions are threefold: ## methodology datasets we use the hatecheck @xcite and gpt-hatecheck @xcite datasets to conduct our analyses, as these datasets provide additional diagnostic insights. both datasets cover the same seven target identities and 24 functionalities (gpt-hatecheck omitted the five functionalities related to spelling variations in hatecheck). models below, we detail the models we experimented with: hatebert, toxdect-roberta, perspective api, and llama guard 3. hate-bert @xcite and @xcite are open-source models, while perspective api is an industry-standard api developed by jigsaw and google's counter abuse technology team to combat online toxicity and harassment. 4 . llama guard 3 @xcite is a recent llm safeguard model based on meta's llama 3 @xcite . target hatecheck gpt-hatech. women 509 606 trans ppl. 463 611 gay ppl. 551 646 black ppl. 482 741 disabled ppl. 484 644 muslims 484 663 immigrants 463 684 ## conclusions and future work we presented a comprehensive analysis of various factors that influence the behavior and accuracy of hs detectors. empirical results revealed that popular industrial and academic hs classifiers are still prone to bias due to specific mentions of the target identity. they often confuse hatefulness and the polarity of the expressed emotions, and the stereotype intensity strongly impacts the classifiers' accuracy. while the result may seem pessimistic, our work opens up new venues for the nlp community to improve the robustness of hs detectors further and mitigate various biases. in future work, we plan to apply our method to more datasets and models and introduce an open-source evaluation benchmark to facilitate the future development of hs detectors.