reviews: 
Comments: This work identifies and examines the issue of shallow safety alignment across multiple illustrative examples. The authors reveal that numerous alignment-bypassing techniques succeed due to a shared vulnerability in safety-trained LLMs: the alignment mechanism primarily affects the initial tokens of generated text. The paper subsequently proposes remedial approaches, including a data augmentation technique and a constrained optimization-based loss function.
Strengths: The paper tackles a critical vulnerability in LLM safety mechanisms with significant practical implications for real-world deployment. The work synthesizes existing literature in a way that clarifies the underlying commonality across successful attacks (i.e., the exploitation of shallow safety alignment), making prior research more accessible. The proposed findings establish a foundation upon which subsequent safety alignment improvements can build. While the paper offers specific mitigation techniques, the identification of the shallow alignment phenomenon itself could catalyze numerous alternative solutions and provide insight into both attack efficacy and the performance of current defenses. The experimental evaluation is comprehensive, spanning multiple architectures, benchmarks, and attack methodologies, supported by both empirical evidence and theoretical grounding. The manuscript is well-structured and logically presented, facilitating clear comprehension of the primary contributions.
Weaknesses: The discussion of related work lacks depth. While existing work is cited, the paper provides insufficient clarification regarding how the current contribution distinguishes itself from prior efforts. Statements such as 'some works have also noted asymmetries...' would benefit from explicit comparison to the current findings. Many assertions regarding prior work are overly general and require greater specificity. For example, does the work subsume all known alignment failure modes, or are there existing failures not addressed? The paper also fails to discuss whether the proposed mitigation strategies (data augmentation and constrained optimization) represent the first solutions to this problem or whether alternative mitigation approaches exist in the literature. Regarding the empirical results, the post-mitigation attack success rates remain non-negligible, yet this limitation receives insufficient explanation. The paper briefly mentions that augmented models retain vulnerability but does not clarify whether this residual vulnerability stems from distinct failure modes or from incomplete remediation of the shallow alignment issue. The conceptual contribution requires clearer exposition. Phrases such as 'the first work to consolidate the unified notion' are ambiguous—if shallow alignment has been previously identified, what precisely does consolidation entail? If shallow safety alignment is a novel term being introduced, this should be made explicit. Finally, the paper would be strengthened by concrete real-world motivation. The introduction could include a concrete scenario demonstrating how an attacker might exploit jailbreaks in a deployed system.
Recommendation: 10
Reviewer confidence: 2
Soundness/correctness: 3
Clarity: 3