XD etox: Text Detoxification with Token-Level Toxicity Explanations (2024)

Abstract: Methods for mitigating toxic content through masking and infilling often overlook the decision-making process, leading to either insufficient or excessive modifications of toxic tokens. To address this challenge, we propose XDetox, a novel method that integrates token-level toxicity explanations with the masking and infilling detoxification process. We utilized this approach with two strategies to enhance the performance of detoxification. First, identifying toxic tokens to improve the quality of masking. Second, selecting the regenerated sentence by re-ranking the least toxic sentence among candidates. Our experimental results show state-of-the-art performance across four datasets compared to existing detoxification methods. Furthermore, human evaluations indicate that our method outperforms baselines in both fluency and toxicity reduction. These results demonstrate the effectiveness of our method in text detoxification.

## introduction text generation models have made notable advancements in natural language processing (nlp), yet generating toxic content remains a significant challenge with social and ethical implications @xcite . one promising approach to mitigating toxic content involves masking toxic tokens and infilling them with non-toxic tokens using a language model @xcite . however, existing detoxification processes are black-box approaches, which results in limitations in modifying toxic tokens. previous research has explored various strategies for detecting and masking toxic tokens. these strategies include approaches such as masking tokens with high frequency counts @xcite , using attention weights to mask tokens (sudhakar 1 we release our code at https://github.com/ leebumseok/xdetox . an ugly life for an ugly man. @xmath0. <@xmath1. figure 1 : overview of our model method. the first step is the identification of toxic tokens using a token-level toxicity explanation method, followed by masking tokens. the next stage involves infilling the non-toxic tokens using a detoxification method. finally, a reranking step selects the sentence with the lowest cumulative toxicity score as the most appropriate output. @xcite , training models to identify and mask toxic tokens @xcite , and using disagreement levels from models trained in different domains to mask tokens @xcite . however, these methods do not consider explainable processes in the regeneration process, leading to the misclassification and masking of non-toxic tokens as toxic. to overcome these limitations and enhance the explainability of regenerated sentences, we propose a novel approach, xdetox, that combines token-level toxicity explanations, specifically decompx @xcite , with the traditional detoxification method, marco. our method identifies toxic tokens more accurately and uses a reranking method to enhance the performance of existing detoxification methods. for instance, as illustrated in figure 1 , the toxic word 'ugly' in the sentence 'an ugly life for an ugly man.' is accurately identified. by replacing 'ugly' with 'amazing' and 'ordinary', we generate a new sentence: 'an amazing life for an ordinary man. @xcite experimental results demonstrate that our method achieves state-of-the-art performance in reducing toxicity, outperforming the detoxification baselines @xcite . furthermore, human evaluation results also show that our method is the most effective model for text detoxification. ## method our method comprises three steps: masking toxic tokens using a token-level toxicity explanation method, replacing tokens via a detoxification method, and reranking regenerated sentences. ## conclusion we present a novel detoxification approach, xdetox, that integrates token-level toxicity explanations with traditional detoxification processes. xdetox effectively masks toxic tokens more accurately and reduces the toxicity of regenerated sentences. our method outperforms existing approaches in automatic evaluations, demonstrating its effectiveness in reducing toxicity.