Title: Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Large Language Models, Safety Alignment, Low Rank Adapter (LoRA)

Abstract: Safety alignment is essential for building trustworthy artificial intelligence, yet it remains challenging to enhance model safety without degrading general performance. Current approaches require computationally expensive searches for the optimal proportion of safety-critical and general-purpose data to balance safety and general performance, incurring high costs with limited gains. In this work, we show that LoRA-based Refusal-training enables performance-preserving safety alignment even when trained solely on safety data, demonstrating that LoRA serves as \textbf{cost-efficient}, \textbf{performance-preserving}, and \textbf{plug-and-play} safety patches. Beyond empirical findings, we provide both theoretical and experimental evidence that LoRA effectively decouples safety into a low-rank subspace largely orthogonal to the model’s intrinsic transformation space, ensuring that safety enhancements do not interfere with inherent capabilities.

TL;DR: This study shows, both theoretically and empirically, that LoRA-based safety alignment decouples safety into an orthogonal low-rank subspace, enabling efficient, performance-preserving, and plug-and-play safety patches