Beyond Surface Alignment: Rebuilding LLM s Safety Mechanism via Probabilistically Ablating Refusal Direction (2025)

Abstract: Jailbreak attacks pose persistent threats to large language models (LLMs). Current safety alignment methods have attempted to address these issues, but they experience two significant limitations: insufficient safety alignment depth and unrobust internal defense mechanisms. These limitations make them vulnerable to adversarial attacks such as prefilling and refusal direction manipulation. We introduce DeepRefusal, a robust safety alignment framework that overcomes these issues. DeepRefusal forces the model to dynamically rebuild its refusal mechanisms from jailbreak states. This is achieved by probabilistically ablating the refusal direction across layers and token depths during fine-tuning. Our method not only defends against prefilling and refusal direction attacks but also demonstrates strong resilience against other unseen jailbreak strategies. Extensive evaluations on four open-source LLM families and six representative attacks show that DeepRefusal reduces attack success rates by approximately 95%, while maintaining model capabilities with minimal performance degradation.

## introduction large language models (llms) @xcite @xcite have demonstrated impressive performance across various natural language tasks, but ensuring them behave safely and reliably remains a significant challenge @xcite . saftey alignment efforts, such as refusal training, have endowed llms with the ability to provide refusal responses to inappropriate and toxic prompts, as shown in figure 1a . however, these safeguards frequently fail when confronted with adversarial jailbreak attacks that disguise harmful requests to induce llms to circumvent refusal behaviors and * corresponding author. trap in jailbreak states @xcite @xcite , as shown in figure 1b . at present, numerous defense strategies have been proposed to defend against jailbreak attacks. adversarial training @xcite tries to enhance refusal robustness of llms against such attacks by training on jailbreak samples. @xcite suppresses harmful activations within the model's hidden states (figure 1c ) to bolster resilience. however, our experiments show that existing defense strategies, even stateof-the-art ones, can be easily bypassed by prefilling attacks and refusal direction attacks, and are susceptible to transfer attacks. specifically, we find that existing defenses remain predominantly surface-level and exhibit two significant limitations. to mitigate the above limitations inherent in current safety alignment methods, we present deeprefusal, an innovative framework designed for deep and robust safety alignment. unlike traditional approaches that focus merely on surface-level finetuning, deeprefusal introduces adversarial pressure directly into the model's representation space during the fine-tuning process, as illustrated in figure 1d . by probabilistically ablating refusal directions across multiple layers and token depths, deeprefusal effectively simulates potential jailbreak scenarios internally. the refusal direction is a direction in the model's internal representation space that is strongly associated with producing refusal responses. this unique approach compels the model to develop a more robust refusal mechanism, ensuring it can resist harmful outputs at every token depth and layer depth, thereby enhancing its safety alignment capabilities significantly. our contributions are summarized as follows: ## our deeprefusal method drawing on these observations, we introduce deep-refusal, a novel approach designed to achieve deep and robust safety alignment for llms. deep-refusal simulates jailbreak scenarios within the model's internal structure without undermining its language modeling capabilities or utility. by doing so, it compels the model to rebuild and reinforce its refusal mechanism across layer depth and token depth. our goal is to expose the model to these simulated jailbreak scenarios during training. this ensures the model learns to maintain robust refusal behaviors even under adversarial conditions. in practice, deeprefusal utilizes two main strategies to accomplish this goal: layer-wise and tokewise probabilistic activation ablation (paa). the former involves probabilistically removing activation patterns across different layers of the model, and the latter focuses on selectively ablating activations at specific token positions within the model's output sequence. by integrating these strategies, we ensure the model encounters a di-verse array of simulated adversarial attacks. this all-encompassing method compels the model to develop robust defenses against various jailbreak techniques, thereby maintaining safe and reliable performance even in challenging scenarios.