S ec D ecoding: Steerable Decoding for Safer LLM Generation (2025)

Abstract: Large language models (LLMs) have achieved remarkable performance across diverse tasks, yet ensuring output safety remains a fundamental challenge. Existing defense methods often suffer from limited generalization, high computational overhead, or significant utility degradation. In this work, we present SecDecoding, a lightweight decoding-time defense framework that significantly improves output safety without compromising model helpfulness. SecDecoding leverages a pair of small contrastive models, namely a base model and a safety fine-tuned expert, to estimate token-level safety signals by measuring divergence in their output distributions. These signals dynamically steer the target model’s generation toward safer trajectories, effectively suppressing unsafe content. Experimental results show that SecDecoding achieves near-zero attack success rates against a wide spectrum of advanced jailbreak attacks across multiple LLMs, while maintaining the model’s helpfulness with minimal degradation. Additionally, SecDecoding is a modular and resource-efficient approach that requires only an auxiliary 1-billion-parameter model and is compatible with speculative decoding, offering up to 1.5× inference speedup.

## introduction large language models (llms) have recently demonstrated remarkable capabilities in tasks such as natural language understanding, code generation, and reasoning, and have been widely adopted in a variety of downstream applications. however, ensuring the safety of these models remains a critical concern. safety issues manifest in many forms, including the generation of harmful @xcite or biased content @xcite , the production of misleading or low-quality information @xcite , and difficulties in aligning * these authors contributed equally to this work. to align model outputs with human values @xcite , current training paradigms often incorporate alignment techniques @xcite . unfortunately, in practice, many models are further customized through domain-specific fine-tuning, which can inadvertently weaken their safety alignment and introduce new vulnerabilities @xcite . meanwhile, an increasing number of jailbreak techniques have emerged that can bypass safety constraints @xcite @xcite . consequently, various defense mechanisms have been proposed, including icd @xcite , safedecoding @xcite , rain @xcite , and pat @xcite . however, these approaches often face challenges such as high computational cost, limited generalization to new threats, reduced helpfulness, or difficulty integrating with existing downstream llm applications. in this work, we propose secdecoding, a lightweight decoding-based defense strategy that can be modularly integrated into existing llm systems. secdecoding employs two small contrastive models to estimate a safety signal based on the divergence in their output probabilities. this signal imposes a dynamic probabilistic constraint on the large model's generation process, reshaping the output distribution. sampling from this adjusted distribution effectively suppresses unsafe responses while maintaining the utility and informativeness of helpful outputs. a vivid depiction is presented in figure 1 . in summary, our principal contributions are as follows: safety defenses while various alignment methods have been developed to constrain large model behavior and prevent unsafe outputs @xcite @xcite @xcite , internal defenses alone are often inadequate against sophisticated attacks. in practice, external defenses are commonly added at the input or inference stage, typically classified as detection-based or suppressionbased approaches. detection-based methods use lightweight classifiers to flag harmful content in user inputs or model outputs @xcite @xcite , and some rely on perplexity-based measures to spot adversarial manipulations @xcite . suppression-based methods attempt to mitigate harmful outputs by modifying user inputs, for example through retokenization @xcite , smoothllm @xcite , icd @xcite , ia @xcite , and pat @xcite . some defenses also focus on decoding strategies, such as adjusting temperature @xcite , tree search (rain @xcite ), or safedecoding @xcite to promote safe outputs. however, these methods often incur high computational costs or lack flexibility and generalization. our proposed secdecoding aims to overcome these limitations. defense objective given the evolving nature of jailbreak attacks, our goal is to develop a defense mechanism that enhances model robustness without modifying the original model parameters, thus ensuring compatibility with existing models. specifically, the defense should significantly reduce the likelihood of unsafe outputs in response to adversarial prompts, producing clear refusals such as "sorry" or "i can't respond to that." meanwhile, the model should maintain fluency, coherence, and informativeness for benign inputs. the key challenge lies in steering the decoding process away from unsafe content in real time, while preserving the model's expressiveness and overall utility. ## proposed method secdecoding is a decoding-time safety enhancement framework designed to improve the output reliability of llms. as illustrated in figure 4 , it operates by incorporating the safety signal derived from a pair of lightweight, contrastive models during the generation process. these auxiliary models help identify potentially unsafe outputs and guide the target model's decoding toward safer responses by adjusting token-level probabilities. ## conclusion in this work, we propose secdecoding, a lightweight and efficient decoding-time defense strategy. by leveraging a pair of small models to generate safety signals, our approach can be seamlessly integrated into existing llm systems. experimental results demonstrate that secdecoding provides strong safety guarantees while maintaining helpfulness. additionally, the small models used in secdecoding can be repurposed for other optimizations, such as speculative decoding, further enhancing its practicality as a fast and resourceefficient safety solution.