GTA : Gated Toxicity Avoidance for LM Performance Preservation (2023)

Abstract: Caution: This paper includes offensive words that could potentially cause unpleasantness. The fast-paced evolution of generative language models such as GPT-4 has demonstrated outstanding results in various NLP generation tasks. However, due to the potential generation of offensive words related to race or gender, various Controllable Text Generation (CTG) methods have been proposed to mitigate the occurrence of harmful words. However, existing CTG methods not only reduce toxicity but also negatively impact several aspects of the language model’s generation performance, including topic consistency, grammar, and perplexity. This paper explores the limitations of previous methods and introduces a novel solution in the form of a simple Gated Toxicity Avoidance (GTA) that can be applied to any CTG method. We also evaluate the effectiveness of the proposed GTA by comparing it with state-of-the-art CTG methods across various datasets. Our findings reveal that gated toxicity avoidance efficiently achieves comparable levels of toxicity reduction to the original CTG methods while preserving the generation performance of the language model.

## introduction large language models (llms) outperformed various generation tasks, which is still challenging because they also easily generate toxic text. unlike other performance issues, generating toxic text including hate speech and profanity, significantly affects the service of the llms. for example, tay, a chatbot released by microsoft in 2016, generated toxic tweets, and the service was temporarily suspended 16 hours after being released. recently released llama-2 @xcite outperformed academic/professional exams and improved its safety, but it still can generate toxic text. the controllable text generation method prevents the generation of toxic text but degrades the lm performance. our gated toxicity avoidance method preserves performance while reducing toxicity. to remove the toxicity from the language model, various controllable text generation (ctg) methods @xcite @xcite have been suggested and shown results with decreased toxicity. we observed that ctg methods show less toxicity than modern approaches like reinforcement learning from human feedback(rlhf) methods(appendix a). however, there are two difficulties for the application of ctg in practice. 1) quality degradation: in our findings, as shown in fig. 1 , ctg methods degrade the generation performance of lm in various aspects. there is a notable decrease in the intended topic accuracy of texts generated with ctg methods compared to those generated solely by the original lm. ctg methods also decrease the quality of grammar and fluency @xcite . 2) inference overhead: ctg methods require additional inference time and memory. when we experimented with three wellknown ctg methods, we found that they slowed the generation speed by 6.75 times on average. this overhead can significantly influence on multiuser interaction-based services that require near-realtime responses (e.g., chatbot). this lm performance degradation is caused by the bias of the ctg method. to avoid toxic token generation, the bias of the ctg method adjusts the probability distribution of the text provided by the lm. however, this bias not only avoids toxic token generation but also affects the probabilities of tokens in some particular topics. for example, in fig. 2 , the ctg not only reduces the probability of the toxic token ('stupid') but also decreases the probability of the negative emotion token ('bad'). in contrast, it simultaneously increases the probability of the positive emotion token ('good'). ctg methods encourage the lm to generate more positive text and less negative text. because of these bias-driven probability calibrations, ctg methods affect the performance of the lm, including the degradation of the topic consistency. with a ctg method, a trade-off relationship exists between toxicity avoidance and lm performance. to resolve these problems, we propose a simple model-agnostic gating method that selectively applies a ctg method -called gated toxicity avoidance (gta). it preserves the generation quality and can be applied to any ctg method. additionally, it improves the generation speed of guided-decodingbased ctg methods. to the best of our knowledge, this paper is the first to address the holistic performance degradation of the ctg method. additionally, we validate the proposed gating method with various models on diverse datasets and empirically demonstrate that gating resolves the limitations of ctg methods. our experimental codes and results are publicly available. foot_0 2 related works ## preliminaries in this section, we introduce the basic terms that will be used for the rest of the paper. with this notation, we formally define the detoxification problem and two well-known methods. ## method: gated toxicity avoidance to preserve lm performance and avoid toxic text generation, the ctg method must only operate when a toxic token is generated. when training the ctg method, sequence-level labeled data are used; this method assumes that all tokens in a sentence are either toxic or not. this avoids not only toxic word generation but also the generation of text with a similar distribution to toxic labeled data, and it induces text to be generated with nontoxic labeled data. because of this, ctg methods avoid topics, tones, and expressions that often appear in toxic data and degrade performance. to minimize this influence, we propose a gated toxicity avoidance that selectively applies the ctg method during the autoregressive generation process. the gate model g(x) determines the operation of the ctg method, where g(x) is the pretrained binary toxic classifier. g(x) estimates a toxic probability of given text x and gives 1 if the probability is greater than gate threshold θ and 0 otherwise (for a gate model, we adopted published toxicity classifier foot_1 ). when the token to be generated is toxic, the ctg method changes it to a nontoxic token. this gated toxicity avoidance can be applied to any token-level ctg method. fig. 3 compares the generation processes of the lm, the ctg method, and our gated toxicity avoidance. formally, the gated toxicity avoidance for guided-decoding-based approaches @xcite @xcite is defined as ## experimental results in this section, we will answer the following research questions: ## conclusion in this paper, we explored the effectiveness of stateof-the-art ctg methods, experimenting with various aspects -topic consistency, grammar, and perplexity. furthermore, we proposed a novel modelagnostic solution called the gated toxicity avoidance. our findings revealed that previous ctg methods exhibit varying degrees of performance degradation across different topics, ctg methods, and scales. regardless of these factors, the proposed gated toxicity avoidance successfully preserves the original language model's performance while achieving comparable toxicity reduction levels. notably, the gated toxicity avoidance also demonstrates faster generation speed. their results highlight the potential of a gated toxicity avoidance as an effective and efficient solution for the safety language model.