STATE T oxi CN : A Benchmark for Span-level Target-Aware Toxicity Extraction in C hinese Hate Speech Detection (2025)

Abstract: The proliferation of hate speech has caused significant harm to society. The intensity and directionality of hate are closely tied to the target and argument it is associated with. However, research on hate speech detection in Chinese has lagged behind, and existing datasets lack span-level fine-grained annotations. Furthermore, the lack of research on Chinese hateful slang poses a significant challenge. In this paper, we provide two valuable fine-grained Chinese hate speech detection research resources. First, we construct a Span-level Target-Aware Toxicity Extraction dataset (STATE ToxiCN), which is the first span-level Chinese hate speech dataset. Secondly, we evaluate the span-level hate speech detection performance of existing models using STATE ToxiCN. Finally, we conduct the first study on Chinese hateful slang and evaluate the ability of LLMs to understand hate semantics. Our work contributes valuable resources and insights to advance span-level hate speech detection in Chinese.

## introduction with the popularity of social media, user-generated content has experienced explosive growth, and hate speech has also flourished. hate speech refers to harmful statements that express hatred or incite harm against specific groups or individuals based on race, religion, gender, region, sexual orientation, physical characteristics, and other factors @xcite . due to its damaging impact on individuals and society, hate language is now widely considered as a problem of increasing importance @xcite . recently, researchers have been actively engaged in the study of hate speech detection, and this research has gradually shifted from post-level @xcite to span-level @xcite @xcite . approximately 941 million people, or 12% of the global population, speak mandarin chinese as their first language @xcite . however, research on chinese hate speech detection lags significantly behind. there are two key issues that remain unresolved. firstly, existing research @xcite @xcite is limited to the post-level, leaving span-level chinese hate speech detection unexplored. the intensity and directionality of hate speech are closely tied to the target and argument it is associated with @xcite . in the chinese linguistic context, challenges in span-level hate speech detection arise due to flexible word order and the absence of word delimiters like spaces. these issues complicate the identification of targets and arguments. for example, as shown in exp.1 in secondly, although chinese hate lexicons have been developed @xcite , the absence of interpretable annotations hinders a deeper understanding of hate speech. as an ideographic language, chinese has a wealth of synonyms and near-synonyms @xcite , making the forms and types of hateful slang diverse and difficult to capture. chinese hateful slang often evades model detection through techniques such as homophonic substitution, character splitting and merging, and historical allusions @xcite . for example, in exp. detailed annotations to construct the first annotated lexicon for chinese hateful slang. this lexicon serves as a foundational resource for research into the understanding of chinese hate semantics. to address the lack of resources for span-level chinese hate speech research, we introduce a spanlevel target-aware toxicity extraction dataset (state toxicn), a novel dataset containing 8029 posts and 9533 target-argument-hateful-group quadruples addressing sexism, racism, regional bias, and anti-lgbtq sentiments. using this dataset, we evaluate the performance of llms in span-level chinese hate speech detection. specially, we annotate: 1) extraction of the targets and arguments from the post, 2) determination of whether each target-argument pair constitutes hate speech, and 3) classification of the groups for hateful target-argument pairs. moreover, we summarize and annotate commonly used hateful slang from real-world online forums to address the challenges of identifying hateful slang. we compile a comprehensive annotated lexicon, labeling each hateful slang with its frequent groups and providing explanations of its usage and contextual meaning. this is the first annotated lexicon of chinese hateful slang with interpretative annotations. this resource not only helps to understand how hateful slang is used to disguise hate speech, but also provides valuable annotated data to evaluate and improve the ability of llms to detect hateful slang. the main contributions of this work are summarized as follows: chinese hate speech dataset. while some chinese hate speech datasets exist, these efforts remain limited to the post-level. tocp and tocab, derived from taiwan's ptt platform, focus on detecting profanity and abusive language @xcite . the sina weibo sexism review (swsr) centers on identifying sexism @xcite . the chinese offensive language dataset (cold) categorizes sentences into types such as individual attacks and anti-bias @xcite . zhou et al. (2022) introduce cdial-bias, the first annotated dataset specifically addressing social bias in chinese dialogues. lu et al. (2023) present tox-icn, a dataset encompassing both explicit and implicit toxic language samples. xiao et al. (2024) introduce a dataset to evaluate llms' robustness against cloaking perturbations. work platforms language #posts span tar. arg. group lex. davidson et al. (2017) twitter english 24,802 ✓ founta et al. (2018) twitter english 80,000 toxic spans @xcite civil comments english 10,629 ✓ hatexplain @xcite twitter, gab english 20,148 ✓ ✓ tbo @xcite twitter english 4,673 ✓ ✓ ✓ cold @xcite zhihu, weibo chinese 37,480 ✓ swsr @xcite weibo chinese 8,969 ✓ ✓ cdial-bias-utt @xcite zhihu chinese 13,394 ✓ cdial-bias-ctx @xcite zhihu chinese 15,013 ✓ toxicn @xcite zhihu, tieba chinese 12,011 ✓ ✓ toxicloakcn @xcite zhihu, tieba chinese 4,582 ✓ state toxicn (ours) zhihu, tieba chinese 8,029 ✓ ✓ ✓ ✓ ✓ while previous works offer quality corpora, span-level research remains unexplored. state toxicn is the first such dataset, annotating target-argument-hateful-group quadruples. unlike existing lexicons, we offer interpretative annotations and targeted group labels, creating a comprehensive chinese hateful slang lexicon. ## conclusions with the advancement of hate speech detection, recent research has shifted from post-level detection to more fine-grained span-level detection. in this work, we focus on building resources for finegrained chinese hate speech detection. firstly, we present the first span-level target-aware toxicity extraction dataset (state toxicn). based on this dataset, we evaluate the performance of llms in span-level chinese hate speech detection. secondly, we provide the first chinese hateful slang lexicon with interpretable annotations. all the hateful slang terms are sourced from real-world online platforms. using this lexicon, we evaluate the impact of hateful slang on model detection capabilities and assess llms' understanding of hateful slang. experimental results show that current llms still face challenges in effectively addressing both span-level hate speech detection and the understanding of hate semantics. we hope that our resources and benchmarks will be valuable for researchers in this field.