Title: Aligning at the Source: Steering Corrective to the Origins of Harmfulness in LLMs
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Safety Alignment

Abstract: Persistent vulnerabilities in safety alignment hinder the deployment of large language models (LLMs). Existing methods remain susceptible to jailbreak attacks, suggesting a fundamental, unaddressed flaw in current safety paradigms. In this work, we diagnose the root cause of this fragility, identifying a systemic issue we term Depth-wise Alignment Discrepancy. We find a fundamental misalignment: harmful vectors—latent representations predisposed to unsafe content—predominantly originate in the model's lower layers, yet conventional alignment training concentrates its corrective gradients disproportionately on the top-most layers. This creates a brittle, "end-of-pipe" defense that is easily bypassed. To address this discrepancy, we propose SAGA, a framework that achieves robust safety through two synergistic innovations. First, it leverages high-entropy Chain-of-Thought (CoT) augmented data to provide the deep semantic signals necessary to reach the source of harmfulness. Second, it introduces a novel Synergistic Gradient Scaling (SGS) mechanism to explicitly reshape the gradient flow, ensuring these corrective signals are precisely applied to the identified vulnerable layers. Extensive experiments on five LLMs against six distinct jailbreak attacks demonstrate SAGA's superiority, reducing attack success rates (ASR) by 21\%–63\% compared to state-of-the-art baselines. Our method preserves downstream task accuracy while introducing minimal computational overhead (<3\%).