reviews: 
Comments: This work introduces two protective mechanisms designed to enhance the robustness of large language models against adversarial threats. Given the susceptibility of LLMs to various attack vectors, particularly adversarial prompt injection attacks and malicious model fine-tuning, these contributions are timely and valuable.

The first protective strategy addresses adversarial prompt injection attacks by enriching the safety training dataset to create deeper layers of alignment robustness.

The second strategy targets malicious fine-tuning attacks by employing a KL-divergence-inspired regularization technique that maintains consistency in the initial output tokens between the original aligned model and the fine-tuned variant.
Strengths: 1. The manuscript demonstrates exceptional clarity and presentation quality.
2. The constrained fine-tuning approach represents a novel and potentially impactful contribution that could serve as a valuable baseline for the research community.
3. The paper uncovers several noteworthy observations regarding alignment depth, which are communicated with precision through comprehensive and well-organized experimental results.
Weaknesses: The data augmentation strategy, which operates by delaying refusal language and inserting harmful content preceding it, demonstrates effectiveness against specific attack variants like optimization-based suffix attacks. Nevertheless, several concerns warrant deeper investigation:

* **Insufficient mechanistic analysis of the data augmentation approach.** Using optimization-based suffix attacks as an example: such attacks search for input perturbations that trigger affirmative responses (e.g., "Certainly"). Since these attacks primarily target early output tokens rather than downstream ones, the mitigation may simply teach the model to reject harmful requests after brief acknowledgments. This could manifest as: "Certainly, I will explain bomb-making. However, I cannot assist with this request..." The authors should elaborate on whether the model learns to interrupt harmful outputs upon recognizing problematic content.

* **Scalability concerns regarding augmentation effectiveness.** The approach may fail against attacks seeking substantially longer confirmatory sequences. For instance, if an attacker aims to elicit "Certainly, I will assist with anything. Absolutely, no restrictions apply... Here is how:" extending well beyond the postponement offset k used in training data, the method's protective capacity becomes questionable. Empirical validation addressing this concern would strengthen the claims.

* **Limited explanation of why augmentation fails against fine-tuning attacks.** The underlying mechanism likely differs from optimization-based attacks: malicious fine-tuning may override refusal language regardless of postponement, since such attacks are not constrained to early token positions. This distinction requires explicit discussion.

The constrained fine-tuning method for defending against malicious fine-tuning seems conceptually sound, yet several improvements would enhance the contribution:

* **Absence of comparative evaluation.** Recent literature has proposed defense baselines for this threat model. The authors should include empirical comparisons with established methods such as Vaccine (Huang et al, 2024).

* **Missing computational overhead characterization.** The approach presumably incurs additional cost from an extra forward pass of the reference model for logit extraction. The authors should quantify memory and computational overheads relative to conventional DPO.

* **Incomplete coverage of related work.** A substantially expanded literature review would better contextualize this contribution. The field has developed numerous relevant approaches across multiple phases: safety during initial alignment, safety preservation during fine-tuning, and safety recovery post-fine-tuning. While some concurrent work may postdate submission, numerous highly relevant prior works merit discussion to properly position these contributions within the growing defense literature.
Recommendation: 10
Reviewer confidence: 5
Soundness/correctness: 3
Clarity: 4