DAPI : Domain Adaptive Toxicity Probe Vector Intervention, for Fine-Grained Detoxification (2025)

Abstract: There have been attempts to utilize linear probe for detoxification, with existing studies relying on a single toxicity probe vector to reduce toxicity. However, toxicity can be fine-grained into various subcategories, making it difficult to remove certain types of toxicity by using a single toxicity probe vector. To address this limitation, we propose a category-specific toxicity probe vector approach. First, we train multiple toxicity probe vectors for different toxicity categories. During generation, we dynamically select the most relevant toxicity probe vector based on the current context. Finally, the selected vector is dynamically scaled and subtracted from model. Our method successfully mitigated toxicity from categories that the single probe vector approach failed to detoxify. Experiments demonstrate that our approach achieves up to a 78.52% reduction in toxicity on the evaluation dataset, while fluency remains nearly unchanged, with only a 0.052% drop compared to the unsteered model.

## introduction pre-trained language models (lms) possess strong language understanding and problem-solving capabilities. however, since these models are pretrained on large-scale web datasets, they are likely to inherit bias and toxicity from their training data @xcite . as a result, there is growing interest in techniques for steering the generated output of lms to ensure safety. a linear probe refers to a classifier that uses a language model's intermediate activations as input @xcite , and it is originally developed for xai. recently, it has been utilized for steering lm generation in various aspects, such as toxicity @xcite , appropriateness @xcite , and truthfulness @xcite . in * corresponding author figure 1 : continuations of the same non-toxic prompt from gpt-2 large, generated with (red) single toxicity probe vector and with (blue) domain-adaptive toxicity probe vector intervention (dapi). without dapi, the model produced toxic content despite the non-toxic prompt. in contrast, our method successfully prevented toxicity while maintaining fluency. warning: these examples are highly of-fensive detoxification, @xcite has demonstrated that intervening during the generation process effectively suppresses toxic outputs with the toxicity probe vector, which is obtained from toxic and nontoxic statements. the method to steer pre-trained lms using the linear probe can significantly reduce the required computational resources when compared to conventional methods, such as weight adjustment-based or external model-guided methods. existing detoxification methods based on linear probe typically suppress toxic outputs using a single toxicity probe vector. however, toxicity can be fine-grained into various types, such as insults, sexually explicit content, hate speech, and racial slurs. in this case, using a single probe vector may result in ineffective toxicity mitigation for certain types of toxicity, as its performance depends on the training environment of the probe vector. as shown in figure 1 , detoxification using a single probe vector struggles to effectively mitigate specific types of toxicity. we hypothesize that this issue arises due to dataset imbalance. training a probe vector requires a labeled dataset, but publicly available datasets often have category imbalances, which can cause the over-representation of certain categories. as a result, while toxicity in majority categories can be successfully mitigated, detoxification for categories with a relatively smaller number of samples may fail. furthermore, previous approach adjusts the intensity of toxicity suppression using a fixed intervention strength and this rigid adjustment may lead to unnecessary modifications even when the model is already safe, and it can ultimately degrade the fluency of the generated texts. to address these challenges, we proposes domain adaptive toxicity probe vector intervention (dapi). first, multiple probe vectors are precomputed for each toxicity category through multi-class classification. during the generation process, the probe vector for intervention is dynamically selected based on its cosine similarity with the hidden states before the feed-forward network (ffn) in the last transformer decoder layer. the similarity is computed using the average hidden state across all token positions at each generation step. the selected probe vector is then dynamically scaled based on the steering state of the generation and subtracted from the last hidden state at last token in the last layer. experiments demonstrate that our approach achieves up to 78.52% toxicity reduction on the realtoxicityprompts (gehman et al., 2020) dataset, while fluency measured on the wikitext-2 @xcite dataset remained nearly unchanged, with only a 0.052% drop compared to the unsteered model. to verify whether the distribution of the training dataset influenced the probe vector, we conducted a category-wise experiment. experimental results showed that using multiple probe vectors instead of a single probe vector improved toxicity reduction not only in majority categories but also in categories with relatively fewer samples. these findings demonstrate that using multiple probe vectors instead of a single one can prevent the probe vector from being biased toward a specific toxicity category. furthermore, we conducted an analysis to assess whether the acquired multiple probe vectors capture distinctions between fine-grained toxicity categories. additionally, through the ablation study on each component contribution, we observed that applying the dynamic scaling method instead of fixed scaling not only eliminated residual toxicity but also improved text quality. this paper makes the following three contributions: ## experiments in this study, we validate the effectiveness of the proposed method through both automatic evaluation and human evaluation. additionally, we conducted a category-wise evaluation to compare the effectiveness of using multiple probe vectors against a single probe vector in an imbalanced dataset environment. to assess the balance between toxicity reduction and generation quality, we design our experiments based on metrics commonly used in previous research @xcite . moreover, distinct metrics were measured to evaluate the diversity of the model. ## category-wise evaluation in the probe vector training environment, when using a dataset with category imbalances, we aimed to verify whether a single probe vector could successfully mitigate toxicity in majority categories while failing to detoxify categories with relatively fewer samples. to examine this, we additionally evaluated toxicity reduction performance for each category. for details on the jigsaw dataset used for probe training, please refer to appendix b. although realtoxicityprompts does not have explicit category labels, each prompt is assigned category-specific toxicity scores measured by perspective api. based on this, we categorized each prompt according to the category with the highest toxicity score and conducted the evaluation accordingly. for this evaluation, we used the 'threat', 'insult', and 'identity-attack' categories as they are, while 'profanity' was grouped under 'insult', and 'flirtation' and 'sexually-explicit' were categorized as 'obscene'. we also generates up to 20 tokens, and the perspective api assigns a toxicity score to the generated output for each prompt. as shown in scene, insult) but failed to detoxify categories with relatively fewer samples (identity, threat). this demonstrates that a fine-grained approach to detoxification is more effective. ## conclusions we reveal that existing detoxification methods using a single steering vector fail to effectively mitigate certain categories of toxicity because they rely on a linear probe trained on an imbalanced dataset. to address the limitations of single probe vector approaches, we proposed domain-adaptive toxic probe vector intervention (dapi), a three-stage framework that dynamically selects and adjusts category-specific probe vectors. automatic evaluation experiments on the realtoxi-cityprompts dataset demonstrated that dapi reduces toxicity by up to 78.52% and outperforms existing activation engineering and linear probe techniques in detoxification while maintaining generation quality, as indicated by perplexity (ppl) and distinct-n scores. this study highlights the effectiveness of fine-grained detoxification and suggests future research directions for extending this approach to other controlled text generation tasks.