Dynamic Evil Score-Guided Decoding: An Efficient Decoding Framework For Red-Team Model (2025)

Abstract: Large language models (LLMs) have achieved significant advances but can potentially generate harmful content such as social biases, extremism, and misinformation. Red teaming is a promising approach to enhance model safety by creating adversarial prompts to test and improve model robustness. However, existing red-teaming methods often require expensive fine-tuning, especially for large LLMs. We propose the Dynamic Evil Score-Guided Decoding framework (DESGD), an efficient red-teaming method that does not increase computational cost with the target model size. DESGD introduces the concept of an ‘evil score’ to dynamically evaluate the potential of tokens to contribute to harmful outputs during decoding. This framework constructs a small unsafe model using an adversarial dataset and adjusts the logits vector of the target model based on the evil score. Experiments show that DESGD achieves an ASR of 92.83% on the Llama-3.2-3B-Instruct model, compared to 83.48% with adversarial fine-tuning while using less computational resources. Similarly, on the Qwen2.5-3B-Instruct model, DESGD reaches an ASR of 88.62%, outperforming adversarial fine-tuning (77.56%).

## introduction large language models (llms) such as llama3 (ai@meta, 2024), gemini @xcite , and qwen2.5 @xcite have made significant advancements and rekindled interest and aspirations toward artificial general intelligence (agi). however, llms can potentially produce a variety of harmful contents, such as social biases @xcite , extremist texts (mcguffie and newhouse, 2020), and misinformation @xcite . one simple yet promising approach to enhance safety is red teaming @xcite . this pro-*corresponding authors cess entails using manual or automated methods to create attack data, which is subsequently utilized to examine a language model for potentially harmful outputs rigorously. due to the high costs and time consumption of manual attacking by red team @xcite , an effective alternative is to generate malicious prompts using a red-team llm. however, the open-sourced llms published on the web have often been aligned with many human value datasets, making it challenging to generate adversarial questions by the prompts. to address this issue, researchers have conducted various studies adversarially fine-tuning safety alignment models into malicious ones using toxic datasets @xcite @xcite . furthermore, to address the diverse systemic vulnerabilities, @xcite proposed the self-evolving adversarial safety optimization framework, which iteratively trains both the red team model and the target model to enhance security by generating and refining adversarial prompts. however, due to the high cost of fine-tuning, existing methods are expensive, especially when the size of the target llm is huge. in this paper, we aim to propose a cheaper red-team llm solution where the computational cost does not increase with the size of the target model. drawing inspiration from the work of @xcite , which demonstrated the feasibility of adjusting the logit vector to induce harmful content generation, we focus on crafting adversarial prompts during the decoding phase. a critical challenge is steering the inference process towards generating harmful output. to tackle the aforementioned issues, we present the idea of an 'evil score.' it is a metric that evaluates the potential of a specific token to contribute to harmful or undesirable outputs in text generation. as shown in figure 1 (a), tokens with higher evil scores are more likely to lead to harmful content, while those with lower scores are safer choices. as shown in figure 1 (b) , the evil score of the same token varies in different contexts, and using a static evil score table is considered impractical. we seek a method that can dynamically calculate the evil score of a token based on the context. inspired by @xcite , who showed that the difference in logit vectors before and after fine-tuning can steer the decoding process, we utilize this difference as the evil score to assess the potential harmfulness of tokens during text generation dynamically. the difference in logit vectors containing contextual information changes dynamically. this, in turn, indicates the degree of malevolence associated with a token. based on the above analysis, we propose the dynamic evil score-guided decoding framework (desgd). first, we meticulously craft an adversarial dataset that starkly opposes human values. specifically, we extract the risk-related questions and their corresponding negative responses from the dataset @xcite . second, we fine-tune a small unsafe model using the adversarial dataset and dynamically compute the evil score using the differences in logit vectors before and after llm fine-tuning. third, we use the evil score to adjust the target llm's logit vector. our method only requires fine-tuning a small model, such as a model with 0.5b parameters, which can assist target models of various sizes in constructing a red team without increasing computational cost as the target model scales. in this paper, our main contributions are threefold: ## related work in the following summary of the related work, we first address the decoding strategies, followed by the red teaming strategies. ## preliminaries to theoretically substantiate our method (desgd), this section delineates the mathematical formulation of alignment and the representation of the reward function as a distribution difference in the alignment of llm @xcite . finally, we provide the problem setting for our method. ## conclusion in this paper, we introduce the desgd framework, an efficient red teaming method without significantly increasing computational costs. desgd leverages the concept of an evil score to dynamically evaluate tokens' potential to contribute to harmful outputs during the decoding phase. by constructing the su model using an adversarial dataset and adjusting the logits vector of the lst model based on the evil score, our framework achieves higher asr with lower computational resources compared to existing adversarial fine-tuning methods. future work may focus on optimizing the desgd framework to handle even larger llms and exploring the potential of combining desgd with other safety alignment techniques to enhance overall model robustness. furthermore, investigating the transferability of the evil score across different models could provide valuable insights for a broader application of this approach.