Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defense

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• Almost Surely Safe Alignment of Large Language Models at Inference-Time (2025)
• Picky LLMs and Unreliable RMs: An Empirical Study on Safety Alignment after Instruction Tuning (2025)
• Safety Alignment Depth in Large Language Models: A Markov Chain Perspective (2025)
• CASE-Bench: Context-Aware Safety Evaluation Benchmark for Large Language Models (2025)
• Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare (2025)
• Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models (2025)
• Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning (2025)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Summary of "Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs"

Objective

The paper investigates the safety implications of activation approximation techniques in Large Language Models (LLMs). While these approximations enhance inference efficiency, their impact on model safety remains largely unexplored.

Methods

The study systematically evaluates seven activation approximation techniques across three categories:

1. Activation Polynomialization – Replacing nonlinear activation functions with polynomial approximations.
2. Activation Sparsification – Truncating small activation values to zero for computational efficiency.
3. Activation Quantization – Reducing the bit-width of activations to lower precision.

The research assesses the safety degradation introduced by these techniques across ten safety-aligned LLMs, including models like Llama, Gemma, and Mistral.

Findings

• Safety Degradation Before Utility Loss: Activation approximations cause LLMs to generate harmful yet coherent responses even before utility metrics are significantly affected.
• Early Layers Are Critical: Safety degradation is most pronounced when approximations are applied to the first few layers of an LLM, whereas later layers have minimal impact.
• Harmful Prompts Become Easier to Evade Detection: Activation approximations shift harmful activations into benign regions, making them harder to filter using standard safety mechanisms.
• Attack Success Rate (ASR) Spikes: The attack success rate (ASR) on models like Llama-3.1-8B-Instruct increases from 0.19% to 69.23% due to activation approximations, highlighting severe safety vulnerabilities.

Proposed Solution: QuadA (Activation Approximation-Aware Alignment)

To counteract safety vulnerabilities, the authors propose QuadA, a lightweight safety enhancement method that integrates into existing LLM alignment procedures. Key features include:

• Minimal Computational Overhead: Can be implemented with 2-3 additional lines of code without significantly increasing inference costs.
• Robustness Against Jailbreak Attacks: Improves resilience against adaptive jailbreak techniques like GCG and AutoDAN.
• Cross-Technique Effectiveness: Works across multiple activation approximation methods and magnitudes.

Implications

• For Model Developers: Activation approximation methods should be carefully evaluated for safety implications before deployment.
• For AI Practitioners: Awareness of safety trade-offs is critical when optimizing inference efficiency.
• For Researchers: Further work is needed to develop universal safety mitigation techniques that function across different approximation approaches.

Conclusion

This study is the first comprehensive analysis of how activation approximations can undermine safety in aligned LLMs. It provides crucial insights into the hidden risks of efficiency-focused optimizations and proposes QuadA as a potential countermeasure.