Exploring Multimodal Challenges in Toxic C hinese Detection: Taxonomy, Benchmark, and Findings (2025)

Abstract: Detecting toxic content using language models is important but challenging. While large language models (LLMs) have demonstrated strong performance in understanding Chinese, recent studies show that simple character substitutions in toxic Chinese text can easily confuse the state-of-the-art (SOTA) LLMs. In this paper, we highlight the multimodal nature of Chinese language as a key challenge for deploying LLMs in toxic Chinese detection. First, we propose a taxonomy of 3 perturbation strategies and 8 specific approaches in toxic Chinese content. Then, we curate a dataset based on this taxonomy, and benchmark 9 SOTA LLMs (from both the US and China) to assess if they can detect perturbed toxic Chinese text. Additionally, we explore cost-effective enhancement solutions like in-context learning (ICL) and supervised fine-tuning (SFT). Our results reveal two important findings. (1) LLMs are less capable of detecting perturbed multimodal Chinese toxic contents. (2) ICL or SFT with a small number of perturbed examples may cause the LLMs “overcorrect”: misidentify many normal Chinese contents as toxic.

## introduction detecting toxic content, broadly defined as rude, disrespectful, or discriminating material @xcite , has emerged as a critical challenge. previous studies @xcite show that perturbing language contents can easily bypass toxic content detectors. despite that llms bring great advancements in detecting toxic contents of many languages @xcite ; figure 1 : an example of one toxic chinese content with 8 possible perturbations from a multimodal perspective. similarity scores are computed using text-embedding-3-small model via the openai platform. token ids are generated by gpt-4o tokenizer. @xcite , identifying the toxic chinese, especially perturbed toxic chinese, remains a significant challenge @xcite . for instance, @xcite show that sota llms are less capable of detecting "cloaked" offensive chinese, where toxic characters are simply replaced by homophones and emojis. the main reason is that chinese is a more complex language system than english, with glyph, phonetic, and semantic modals for presentation @xcite . on the one hand, this gives malicious entities more opportunities to revise toxic text in different modalities to bypass detectors. on the other hand, there is a clear culture trend that chinese netizens use more "perturbed" chinese (e.g., internet slang, abbreviations, emojis) on social media platforms for efficiency, expressiveness and group identify foot_0 @xcite @xcite . therefore, as shown in figure 1 , there exist many modalities to design and embed perturbations into toxic chinese contents, allowing them to bypass the detection while maintaining comprehensibility to chinese netizens. therefore, we identify the chinese multimodal language nature as the key challenge of leveraging llms to detect perturbed toxic chinese contents. unfortunately, existing studies all overlook this fundamental nature, significantly compromising the robustness of the designed toxic content detectors. classic detection solutions like adversarial training rely on the complete collection and knowledge of all possible perturbations. however, currently there lacks such a comprehensive taxonomy to guarantee the effectiveness of these methods. while recent llms have demonstrated impressive abilities of language understanding, it is still unknown how accurately these llms can detect perturbed toxic chinese contents, particularly when considering the unique chinese multimodal feature. to address the above challenges, this paper introduces a novel study towards toxic chinese detection. our contributions are threefold. @xcite we present a comprehensive taxonomy of chinese toxicity perturbation methods, encompassing three main strategies and eight specific kinds of approaches (see examples in figure 1 ). this taxonomy can fully capture the chinese multimodal language characteristics in a systematic way. (2) based on this taxonomy, we design a generationvalidation pipeline to construct a large-scale labeled dataset, cntp, consisting of about 2,500 perturbed toxic chinese contents for each approach. we further benchmark 9 sota llms developed in usa (e.g. o3-mini from openai) and china (e.g. deepseek-v3) to understand if these llms are capable of detecting the perturbed chinese. (3) using cntp, we explore cost-effective enhancement strategies like in-context learning (icl) and supervised finetuning (sft) with a small amount of samples. we draw two interesting findings from our evaluations. first, even sota llms can fail in detecting certain kinds of perturbed toxic chinese. llms developed in china do not have clear advantages over the ones from usa. second, we find that even a very small amount of samples can significantly change llms' detection behaviors, despite that these llms still do not understand the semantics behind toxic chinese content. for instance, finetuning gpt-4o-mini with only 10 samples from cntp can cause it to "overcorrect". although its detection rate for toxic content increases from less than 60% to over 98% across two perturbations, its error rate (i.e., normal chinese content being misclassified as toxic) also rises from 2% to more than 30%. human checks by native chinese speakers confirm that the fine-tuned llm does not understand the semantics of the perturbed chinese. ## taxonomy of chinese perturbation chinese, unlike alphabetic languages such as english, employs characters as its minimal semantic units. words (or phrases) are typically formed by combining multiple chinese characters. such linguistic features pose unique multimodal challenges for language models to detect toxicity, as there are more unexpected approaches to perturb the chinese toxic content while maintaining its comprehensibility to native speakers. in this section, we provide a comprehensive taxonomy of possible solutions to bypass toxicity detection via content perturbation. it includes 3 main strategies and 8 specific methods. this taxonomy will serve as a cornerstone to curate our perturbed dataset and benchmark llms in the following sections. ## dataset construction based on the above taxonomy, we design a pipeline to construct a dataset of chinese toxic content with diverse multimodal perturbations (cntp). as shown in figure 3 , we first sample contents from a base dataset toxi_cn @xcite , and filter out the base dataset. then, we carry out 2 major stages: toxic entity extraction and perturbation embedding. human validation foot_1 is also involved throughout the pipeline. we follow three key principles: (1) linguistic diversity (covering 8 specific kinds of glyph, phonetic, and semantic perturbations), ( 2 ) human readability and comprehensibility verification, and ## discussion and future works we discuss and list our future works in two aspects. from a chinese linguistic perspective, we hope this work raises awareness about the impact of perturbations driven by chinese popular culture trends on the internet. for our first future work, we aim to continue improving the taxonomy to better understand how attackers manipulate toxic chinese to bypass detection. for mitigation solutions, our findings suggest that advanced llms may not fully grasp perturbed chinese during their training stages. therefore, our second future work is to explore more effective ways to help llms better understand perturbed chinese content. we believe that understanding how to perturb chinese is the foundation of designing mitigation strategies. ## conclusion in this study, we introduced a taxonomy of 8 perturbation methods based on the chinese multimodal language nature, which facilitates the creation of a perturbed toxic chinese dataset, cntp. by benchmarking 9 sota llms, we revealed that even advanced models like deepseek-v3 or o3-mini are less capable of detecting perturbed toxic chinese. additionally, we explored cost-effective enhancements like in-context learning and fine-tuning. however, they fail to enable models like 4o-mini to fully understand the perturbed content, often resulting in overcorrection: a clear increase in misclassification of normal content as toxic.