D etox LLM : A Framework for Detoxification with Explanations (2024)

Abstract: Prior works on detoxification are scattered in the sense that they do not cover all aspects of detoxification needed in a real-world scenario. Notably, prior works restrict the task of developing detoxification models to only a seen subset of platforms, leaving the question of how the models would perform on unseen platforms unexplored. Additionally, these works do not address non-detoxifiability, a phenomenon whereby the toxic text cannot be detoxified without altering the meaning. We propose DetoxLLM, the first comprehensive end-to-end detoxification framework, which attempts to alleviate the aforementioned limitations. We first introduce a cross-platform pseudo-parallel corpus applying multi-step data processing and generation strategies leveraging ChatGPT. We then train a suite of detoxification models with our cross-platform corpus. We show that our detoxification models outperform the SoTA model trained with human-annotated parallel corpus. We further introduce explanation to promote transparency and trustworthiness. DetoxLLM additionally offers a unique paraphrase detector especially dedicated for the detoxification task to tackle the non-detoxifiable cases. Through experimental analysis, we demonstrate the effectiveness of our cross-platform corpus and the robustness of DetoxLLM against adversarial toxicity.

## introduction the term toxic language is usually used to refer to any form of offensive or hateful speech @xcite ; specifically, toxic or abusive language is defined as any form of microaggression, condescension, harassment, hate speech, trolling, and the like @xcite . use of toxic language online has been a significant issue over the years. although a plethora 1 ubc-nlp/detoxllm-7b don't defend the tsa. f**kin thieving retards. don't support the tsa. they are incredibly frustrating and unprofessional. ## related works over the years, several works have studied abusive language detection @xcite @xcite . the task of text style transfer (tst) has also been explored in the field of nlp due to its then, we generate texts of opposite classes ( §3.2). we filter out ambiguous data ( §3.3). after that, we generate explanation and paraphrase labels ( §3.4). finally, we train the detoxification and the paraphrase detection models ( §3.5). detoxification is formulated as style transfer from toxic to neutral and non-toxic style @xcite . prior works like nogueira dos @xcite and @xcite create their own detoxification corpus from @xcite , respectively. @xcite employ style-trained language models to guide a paraphraser preserve the content and remove toxicity. the authors further use the masked language modeling strategy of @xcite to replace the toxic tokens with its non-toxic alternatives. @xcite develop a human-annotated parallel corpus from jigsaw, x (formerly known as twitter), and reddit. the authors train a bart @xcite model on this parallel corpus and achieve the sota performance on detoxification, showing the importance of high quality parallel data. @xcite propose crosslingual detoxification through simultaneous text translation and detoxification. however, none of the prior works explore the idea of cross-platform detoxification potentially due to the scarcity of parallel data. this research gap motivates our work on this particular subproblem. ## proposed methodology we present our methodology in figure 2 (please see the caption for the overview). now, we describe each component of our cross-platform detoxification framework. ## results overview. we present the performance of the models on cross-platform detoxification in ## analyses 6.1 effectiveness of cross-platform data we further analyze how our cross-platform dataset improves models trained on human-annotated data. hence, we take the finetuned paradetox model and continue training it on our cross-platform dataset with varying sample sizes (100 -1000 samples). then, we evaluate the models' performance on the human-annotated paradetox test set. figure 4 shows the relative difference in accuracy and bleu between the paradetox model trained on different sample sizes of our crossplatform dataset and the original paradetox model. as is evident, the finetuned models (up to sample size 700) tend to maintain higher bleu score. importantly, the model's accuracy tends to increase with the increase in the sample size. the higher accuracy and bleu score signify the models' capability to detoxify input text while producing humanlike non-toxic output, which consequently indicates the effectiveness of our cross-platform dataset. we report the detailed results in appendix g. we further present analysis on multilingual transfer of detoxification in appendix i. ## human evaluation of explanation to assess the quality of the toxicity explanation, we conduct another human evaluation similar to the detoxification evaluation. we implement a fourlevel (a, b, c, d) rating system to measure the quality of the explanation generated by the models. we randomly sample 100 test cases and pass it to two human annotators for evaluating the explanation. we assess the quality of the explanations based on the following metrics: we provide a detailed description of the evaluation framework in appendix o. figure 7 : human evaluation on the explanations (generated by chatgpt) for the toxic inputs from training dataset (inter-annotator @xmath0.78). a is the best, and d is the worst rating for explanation of the toxic input. quality of training data. we first analyze the quality of the training data (explanation) generated by chatgpt (figure 7 ). through human evalution, we find that chatgpt produces mostly relevant, comprehensive, and convincing explanations. this human evaluation further demonstrates the high quality of our training data. results. we present the evaluation results in figure 8. as noticed, detoxllm (llama-ce) (figure 8a ) produces better explanations according to (a) human evaluation on detoxllm (llama-ce) generated explanation for the toxic input. figure 8 : human evaluation of the models' generated explanation for the toxic inputs (inter-annotator @xmath1.65). a is the best, and d is the worst rating for explanation of the toxic input. the human annotators. we further find that the majority of the explanations from detoxllm (figure 8a ) are relevant (86% of rating-a) and comprehensive (69% of rating a). importantly, 70% (rating-a) of the responses from detoxllm are found convincing, signifying that the user would be motivated enough to alter the input. ## conclusion in this work, we propose detoxllm, a comprehensive end-to-end detoxification framework to tackle toxic language across multiple platforms. we generate a novel cross-platform pseudo-parallel corpus through multi-step data processing and generation with chatgpt. we train a suite of detoxification models. especially, our corss-platform detoxification model trained with cot explanation (cot-expl llama) outperforms sota detoxification models. we additionally introduce explanation into the detoxllm framework for promoting trustworthiness. we also develop a dedicated paraphrase detector to handle the cases of non-detoxifiability. through an extensive experimental analysis, we further show the effectiveness of our cross-platform data as well as the robustness of detoxllm against implicit and token-level adversarial toxicity.