Investigating the Impact of Quantization Methods on the Safety and
  Reliability of Large Language Models
Authors: name: Artyom Kharinaev; name: Viktor Moskvoretskii; name: Egor Shvetsov; name: Kseniia Studenikina; name: Bykov Mikhail; name: Evgeny Burnaev
Published: 2025-02-18 20:32:05

Abstract: Large Language Models (LLMs) have emerged as powerful tools for addressing
modern challenges and enabling practical applications. However, their
computational expense remains a significant barrier to widespread adoption.
Quantization has emerged as a promising technique to democratize access and
enable low resource device deployment. Despite these advancements, the safety
and trustworthiness of quantized models remain underexplored, as prior studies
often overlook contemporary architectures and rely on overly simplistic
benchmarks and evaluations. To address this gap, we introduce OpenSafetyMini, a
novel open-ended safety dataset designed to better distinguish between models.
We evaluate 4 state-of-the-art quantization techniques across LLaMA and Mistral
models using 4 benchmarks, including human evaluations. Our findings reveal
that the optimal quantization method varies for 4-bit precision, while vector
quantization techniques deliver the best safety and trustworthiness performance
at 2-bit precision, providing foundation for future research.