T o XCL : A Unified Framework for Toxic Speech Detection and Explanation (2024)

Abstract: The proliferation of online toxic speech is a pertinent problem posing threats to demographic groups. While explicit toxic speech contains offensive lexical signals, implicit one consists of coded or indirect language. Therefore, it is crucial for models not only to detect implicit toxic speech but also to explain its toxicity. This draws a unique need for unified frameworks that can effectively detect and explain implicit toxic speech. Prior works mainly formulated the task of toxic speech detection and explanation as a text generation problem. Nonetheless, models trained using this strategy can be prone to suffer from the consequent error propagation problem. Moreover, our experiments reveal that the detection results of such models are much lower than those that focus only on the detection task. To bridge these gaps, we introduce ToXCL, a unified framework for the detection and explanation of implicit toxic speech. Our model consists of three modules: a (i) Target Group Generator to generate the targeted demographic group(s) of a given post; an (ii) Encoder-Decoder Model in which the encoder focuses on detecting implicit toxic speech and is boosted by a (iii) Teacher Classifier via knowledge distillation, and the decoder generates the necessary explanation. ToXCL achieves new state-of-the-art effectiveness, and outperforms baselines significantly.

## introduction warning: this paper discusses and contains content that can be offensive or upsetting. while social media has dramatically expanded democratic participation in public discourse, they have also become a widely recognized platform for the dissemination of toxic speech @xcite @xcite . online toxic speech, therefore, is prevalent and can @xcite . the input post is fed into both models. the baseline model, roberta, fails to detect the implicit toxic speech while our proposed toxcl model successfully detects it and generates a toxic explanation that closely matches the ground truth explanation. toxic speech can generally be categorized as either explicit or implicit. explicit toxic speech contains direct offensive language targeting individuals or groups @xcite and has been extensively studied @xcite . on the other hand, implicit toxic speech presents a more challenging detection task as it relies on stereotypes and indirect language (elsherief et al., 2021) (see fig. 1 ) and has received limited attention. moreover, given the absence of explicit offensive words or cues, it is crucial for ai models not only to detect implicit toxic speech but also to provide explanations for its toxic nature @xcite . the act of explanation serves practical purposes in various real-life applications, including improving humanmachine interactions and building trustworthy ai systems @xcite @xcite . these applications, therefore, pose a need for arxiv:2403.16685v2 [cs.cl] 20 may 2024 unified systems that can effectively detect implicit toxic speech and explain its toxicity. however, previous works have mainly focused on a hybrid approach that combines detection and explanation tasks into a single text generation problem. for example, @xcite proposed concatenating the toxic speech label and explanation as the target output, @xcite and @xcite extended this approach by incorporating additional data such as target group(s) or social norms. unfortunately, these hybrid approaches can introduce error propagation problems @xcite , possibly due to differences in training objectives (see sec. 3.2.3). consequently, models formulated in this manner tend to have much lower detection scores compared to models that focus solely on detection, as evidenced by our experimentation results (sec. 4). another simple approach is building a modular-based system separating the detection module and the explanation generation module. however, in reality, this kind of framework is computationally expensive to train, store and deploy as it has multiple components. to bridge these gaps in detecting and explaining implicit toxic speech, we propose a unified framework toxcl consisting of three modules (fig. 2 ). our approach is motivated by the findings that modeling the minority target groups associated with toxic speech can potentially improve the performance of both implicit toxic detection and explanation tasks @xcite @xcite . to achieve this, we build a target group generator as our first module to generate the target minority group(s) based on the input post. the generated target group(s) and the post are then input into an encoder-decoder model whose encoder detects the speech, and the decoder outputs the necessary toxic explanation. to enhance the detection performance of our encoder, we incorporate a strong teacher classifier that utilizes the teacher forcing technique during training to distill knowledge to our encoder classifier. finally, we introduce a conditional decoding constraint to enhance the explanation ability of the decoder during inference. our model achieves state-of-theart performance on the implicit hate corpus (ihc) @xcite and social bias inference corpus (sbic) @xcite in the task of implicit hate speech detection and explanation, outperforming baselines. our contributions are as follows: ## conclusion we present toxcl, a unified framework for implicit toxic speech detection and explanation (sec. 3.2). it consists of three components: a target group generator, an encoder-decoder model, and a teacher classifier. our findings show that the target group generator effectively identifies target groups, improving both accuracy and f1 scores for detecting implicit toxic speech. the novel encoderdecoder architecture successfully performs both tasks of detection and explanation without harming each other. the integration of the teacher classifier and the conditional decoding constraint further enhances the performance of toxcl, achieving state-of-the-art results in the task of toxic speech detection and explanation on two widely-used benchmarks. in the future, we will focus on addressing several limitations faced by our framework and baselines as specified in section 5.3 to further enhance the performance of our model.