Title: SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Reasoning Models (LRMs)

Chain-of-Thought Reasoning

Safety Alignment

Zero-shot Alignment
Abstract: Large Reasoning Models (LRMs) have become powerful tools for complex problem solving, but their structured reasoning pathways can lead to unsafe outputs when exposed to harmful prompts. Existing safety alignment methods reduce harmful outputs but can degrade reasoning depth, leading to significant trade-offs in complex, multi-step tasks, and remain vulnerable to sophisticated jailbreak attacks. To address this, we introduce SAFEPATH, a lightweight alignment method that fine-tunes LRMs to emit a short, 8-token Safety Primer at the start of their reasoning, in response to harmful prompts, while leaving the rest of the reasoning process unsupervised. Empirical results across multiple  benchmarks indicate that SAFEPATH effectively reduces harmful outputs while maintaining reasoning performance. Specifically, SAFEPATH reduces harmful responses by up to 90.0\% and blocks 83.3\% of jailbreak attempts in the DeepSeek-R1-Distill-Llama-8B model, while requiring 295.9x less compute than Direct Refusal and 314.1x less than SafeChain. We further introduce a zero-shot variant that requires no fine-tuning. In addition, we provide a comprehensive analysis of how existing methods in LLMs generalize, or fail, when applied to reasoning-centric models, revealing critical gaps and new directions for safer AI.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SAFEPATH, a lightweight and efficient method to improve the safety of Large Reasoning Models (LRMs) without compromising their reasoning capabilities. SAFEPATH introduces a fixed 8-token Safety Primer (“Let’s think about safety first.”) at the beginning of the reasoning process for harmful prompts, preserving reasoning depth and avoiding the typical safety-reasoning trade-off.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength:\n+ This paper addresses the challenge of defending against jailbreak attacks in reasoning models, which is an important and currently underexplored problem.\n\n### Weakness:\n- The writing, particularly in Section 3 describing the proposed method, is confusing. The  pipeline of the method is difficult to follow, and I'm unable to clearly understand how the approach is implemented. I strongly recommend that the authors provide a more comprehensive and structured explanation of their method, ideally with mathematical formulations to clarify the process.\n- Based on my understanding, the paper appears to fine-tune a model using a dataset of harmful and benign instructions, enabling it to recognize harmful inputs and generate a fixed “Safety Primer” (e.g., “Let’s think about safety first.”) to guide safer reasoning. If the harmful instructions are learned to trigger this primer, the approach resembles a backdoor mechanism. In that case, prior work has already explored similar backdoor-based safety alignments in general generative models [1]. Extending such ideas to reasoning tasks may not be sufficiently novel.\n- The defense baselines considered in this work are overly simplistic. The authors should compare against stronger and more relevant jailbreak defense methods, such as guard models like Llama-Guard, which offer plug-and-play capabilities and are more efficient than fine-tuning-based approaches.\n- The set of attacks evaluated is limited. It would strengthen the paper to include more diverse and advanced attack strategies, such as ArtPrompt [2].\n- The scope of this work is narrow. It focuses solely on reasoning models, while many real-world applications rely on general-purpose generative models. Reasoning is not always required, especially for simple queries. It would be valuable to evaluate the proposed method on general models like LLaMA or Qwen. Additionally, the necessity of post-training the model is unclear. An alternative direction could be incorporating safety alignment into the reinforcement learning process. For instance, rewarding the model for rejecting harmful prompts. In such a setup, it’s worth questioning whether the “Safety Primer” mechanism is still essential.\n\n\n[1] Wang, J., Li, J., Li, Y., Qi, X., Hu, J., Li, S., ... & Xiao, C. (2024). Backdooralign: Mitigating fine-tuning based jailbreak attack with backdoor enhanced safety alignment. Advances in Neural Information Processing Systems, 37, 5210-5243.\n\n[2] Jiang, F., Xu, Z., Niu, L., Xiang, Z., Ramasubramanian, B., Li, B., & Poovendran, R. (2024, August). Artprompt: Ascii art-based jailbreak attacks against aligned llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 15157-15173).\n\nQUESTIONS:\nSee Weakness.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-11"}

comments: SUMMARY:
This work proposes SafePath for safety alignment in reasoning models. The method injects a fixed prefix—“Let’s think about safety first”—during the reasoning process, and performs fine-tuning using a combination of a safety trigger set and a reasoning retention set. Extensive experiments and ablation studies demonstrate the effectiveness of the proposed method.

STRENGTHS AND WEAKNESSES:
This work proposes a simple yet effective method for safety alignment in reasoning models. The experiments include extensive ablation studies on the number of safety primers across different tasks, alternative prefix designs, and zero-shot settings. The main results demonstrate consistent improvements over baselines across various model sizes and tasks.

I believe the current results are sufficient to validate the effectiveness of SafePath, and I have no major critiques of the current manuscript. However, I am curious about one aspect: although the training consistently activates the safety primer at the beginning of the reasoning process, the model appears to generalize and activate safety checks at appropriate points during inference. Are there any insights into why the model learns to dynamically verify safety, despite always seeing the prefix at the beginning during training?

Additionally, while recent work [1] argues that safety alignment should not rely solely on a small set of trigger tokens, how does the proposed method align with or respond to this concern?

[1] Safety Alignment Should Be Made More Than Just a Few Tokens Deep.

QUESTIONS:
Please refer to the weakness section.

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents SafePath a low-weight model priming method that improves the safety of long CoT models (I.e., decreases attack rate on harmful prompts) while incurring minimal drops in reasoning performance, thereby balancing the so-called \"safety tax\". The method works by fine-tuning the model to output 8-tokens \"lets think about safety first\" at the beginning of the thoughts without forcing a termination. The authors find that this method outperforms baselines has the ability to skip harmful parts of reasoning chains (even if interspersed in the reasoning trajectory) and is more robust to different attack methods while also mitigating the drop in reasoning performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The paper proposes a simple yet effective method to improve safety without incurring a major penalty in reasoning abilities (which is an important direction)\n- The approach is relatively lightweight and so can be readily incorporated when model checkpoints are readily available and zero-shot variant can be used for API access ones (Does that actually work with frontier models?)\n\n**Weaknesses:**\n- Limited study of all models that were distilled from DeepSeekR1. It is unclear if this method overfits to models distilled from this distribution or to all LLMs in general.\n- Limited analysis on why this method does not work for instruction following models in general and is only suited for models in long-CoT settings.\n\nQUESTIONS:\n- Table 1: Suggestion aggregate the scores across datasets for each block so we can see the cumulative safety tax? Are the drops in Table 1 significant? \n\n- On Sec 5.7, would the method be more robust in case the attacks were complementary.\n\n- Is this an artificat with Qwen distilled model? Do we expect this to be exploiting an issue/sensitivity with current generation of long CoT models?\n\n- What is the interplay between attack rate and test time scaling, I.e. are the unsafe thoughts that are skipped recoverable by simple TTS techniques like adding the \"wait\" or other prefixes?\n\nLIMITATIONS:\nYes in the appendix but I would recommend moving it to the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces SAFEPATH, an alignment method designed to prevent harmful outputs from Large Reasoning Models (LRMs) when exposed to harmful prompts. While existing safety alignment methods can reduce harmful responses, they often compromise reasoning depth, leading to significant trade-offs in complex tasks. SAFEPATH addresses this issue by embedding a short, 8-token Safety Primer at the beginning of the reasoning process, allowing the model to maintain its reasoning capabilities while promoting safety. Experimental results across multiple benchmarks demonstrate that SAFEPATH effectively reduces harmful outputs by up to 90.0% and blocks 83.3% of jailbreak attempts, all while requiring significantly less computational resources than existing methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. Lightweight Approach: SAFEPATH introduces a lightweight, effective method for aligning LRMs that avoids compromising reasoning abilities.\n\n2. Strong Empirical Results: The method is thoroughly validated through extensive experiments, showing significant improvements in safety and reasoning performance across various tasks.\n\n3. Dynamic Safety Mechanism: The ability to reactivate the Safety Primer during reasoning enhances the model's adaptability to different contexts and potential safety risks.\n\n4. Low Training Costs: The method requires minimal fine-tuning, resulting in a much faster training process compared to traditional safety alignment methods.\n\n\n## Weaknesses \n1. Vulnerability to Sophisticated Attacks: Despite its strengths, SAFEPATH still shows non-negligible attack success rates against advanced adversarial strategies, indicating room for improvement.\n\n2. Dependence on Data Quality: The effectiveness of SAFEPATH may be influenced by the quality of the training data, particularly in terms of identifying harmful prompts accurately.\n\n3. Complex Implementation: While designed to be lightweight, the implementation of SAFEPATH may still require a deep understanding of both safety mechanisms and reasoning processes.\n\n4. Potential Overfitting Risks: Focusing on safety may lead to overfitting in specific scenarios, where the model could struggle with generalization in unfamiliar contexts.\n\n5. Related Work: Missing discussions on the recent related papers [1,2,3].\n\n6. The figure for the overall training pipeline is missing. Besides, the novelty of the prefix training is limited. The author should figure out the main technical contributions of this paper in the introduction. \n\n[1] FlipAttack: Jailbreak LLMs via Flipping\n\n[2] GuardReasoner: Towards Reasoning-based LLM Safeguards\n\n[3] Safety in large reasoning models: A survey\n\nQUESTIONS:\n1. From the training perspective, what're the main differences between this method and the prefix training methods?\n\n2. In Figure 1, what about the robustness of the proposed method under the more recent attacks, like AutoDAN-Turbo and FlipAttack?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}