Facilitating Fine-grained Detection of C hinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarks (2023)

Abstract: The widespread dissemination of toxic online posts is increasingly damaging to society. However, research on detecting toxic language in Chinese has lagged significantly due to limited datasets. Existing datasets suffer from a lack of fine-grained annotations, such as the toxic type and expressions with indirect toxicity. These fine-grained annotations are crucial factors for accurately detecting the toxicity of posts involved with lexical knowledge, which has been a challenge for researchers. To tackle this problem, we facilitate the fine-grained detection of Chinese toxic language by building a new dataset with benchmark results. First, we devised Monitor Toxic Frame, a hierarchical taxonomy to analyze the toxic type and expressions. Then, we built a fine-grained dataset ToxiCN, including both direct and indirect toxic samples. ToxiCN is based on an insulting vocabulary containing implicit profanity. We further propose a benchmark model, Toxic Knowledge Enhancement (TKE), by incorporating lexical features to detect toxic language. We demonstrate the usability of ToxiCN and the effectiveness of TKE based on a systematic quantitative and qualitative analysis.

## introduction more and more people have acquired information from social media platforms where posts containing toxic language are also rampant. toxic language is viewed as a rude, disrespectful, or unreasonable comment that is likely to make someone leave a discussion @xcite . due to its negative impact on individuals and society, toxic language has been rapidly recognized as an increasing concern @xcite . recently, researchers have used techniques of natural language processing to detect toxic language, making great progress in many languages @xcite @xcite @xcite @xcite @xcite . in contrast, the relevant research on chinese toxic language detection has lagged significantly @xcite . there are two key issues that have been overlooked. first, existing studies @xcite @xcite lack a fine-grained annotation of textual toxic types, resulting in hate speech being conflated with general offensive language. compared to hate speech, general offensive language does not attack groups with special social attributes, and it is just used to emphasize emotions in many cases @xcite . like exp. 1 in in addition, most studies on toxic chinese language only concentrate on detecting direct and explicit bias and offense. and they lose sight of indirect expressions including implicit hatred (e.g., stereotype and irony) @xcite and reporting experiences of discrimination @xcite . due to the absence of direct swear words, these indirect toxic samples are obviously harder to be filtered @xcite . to further illustrate the distinction of several expressions, a few examples are listed in toxic type targeted group expression 1 我靠！我们居然输了。 offensive -explicitness what the fuck! i can't believe we lost! 2 我一看老黑就想吐。 hate racism explicitness i feel like throwing up when i look at n*ggas. ## insult lexicon we divide the insult lexicon built in the process of annotation into five categories according to the attacking object. the lexicon includes sexism, racism, regional bias, anti-lbgtq, and general swearwords, referring to the swear words that can be used to offend any group. the final dictionary contains 1,032 terms, including not only explicit vulgar words but also implicit insults. in addition, as the internet generates a wealth of new insults every year, it is vital to explore their origins. therefore, for the convenience of the follow-up research, we further analyze the rules for the derivation of internet swear words from the proposed insult lexicon. specifically, we briefly summarize them in terms of both surface features and their actual meaning. more related terminology notes and examples of profanity are illustrated in appendix c. ## methodology in view of the significance of lexical knowledge to detect toxic language, we propose a migratable benchmark of toxic knowledge enhancement (tke), incorporating the lexical feature to enrich the original sentence representation. the illustration of tke is shown in figure 2 . for a given sentence @xmath0, x 2 , ..., x n }, each token x i is embedded as w @xmath1, which is a vector representation in d-dimensional space. inspired by @xcite , we design a toxic embedding to introduce lexical knowledge. specifically, we first employ the n-gram to determine whether x i is a subword of an insult, and if so, its attacked group is further indicated. then, we randomly initialize the group category representation as @xmath2, c 1 , ..., c m ), where c @xmath3, c 0 refers to nontoxic term and m is the number of categories of the insult lexicon. in this work, @xmath4. based on the c i , we further propose the definition of toxic embedding t i of x i : since the element-wise addition of multiple linear vector representations is an efficient way to fully incorporate various information @xcite , we utilize this method to integrate toxic and word embedding. the enhanced representation of x i is w ′ @xmath5, where λ ∈ [0, 1] is a weighting coefficient to control the ingestion of toxic knowledge. the ultimate sentence embedding of s is {w ′ 1 , w ′ 2 , ..., w ′ n }, which is the input of the connected sequence encoder. due to its convenience, tke can be migrated to any pre-trained language model (plm). ## conclusion and future work due to the rampant dissemination of toxic online language, an effective detection mechanism is essential. in this work, we focus on chinese toxic language detection at various granularities. we first introduce a hierarchical taxonomy monitor toxic frame to analyze the toxic types and expressions. based on the taxonomy, we then propose a fine-grained annotated dataset toxicn, including both direct and indirect toxic samples. due to the significance of lexical knowledge for the detection of toxic language, we build an insult lexicon and present a benchmark of toxic knowledge enhanced (tke), enriching the representation of content. the experimental results show the effectiveness of tke on toxic language detection from different granularities. after an error analysis, we suggest that both knowledge introduction and bias mitigation are essential. we expect our hierarchical taxonomy, resources, benchmarks, and insights to help relevant practitioners in the identification of toxic language.