Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning
  Models
Authors: name: Ruikang Liu, name: Yuxuan Sun, name: Manyi Zhang, name: Haoli Bai, name: Xianzhi Yu, name: Tiezheng Yu, name: Chun Yuan, name: Lu Hou

Recent advancements in reasoning language models have demonstrated remarkable
performance in complex tasks, but their extended chain-of-thought reasoning
process increases inference overhead. While quantization has been widely
adopted to reduce the inference cost of large language models, its impact on
reasoning models remains understudied. In this study, we conduct the first
systematic study on quantized reasoning models, evaluating the open-sourced
DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B
parameters, and QwQ-32B. Our investigation covers weight, KV cache, and
activation quantization using state-of-the-art algorithms at varying
bit-widths, with extensive evaluation across mathematical (AIME, MATH-500),
scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our
findings reveal that while lossless quantization can be achieved with W8A8 or
W4A16 quantization, lower bit-widths introduce significant accuracy risks. We
further identify model size, model origin, and task difficulty as critical
determinants of performance. Contrary to expectations, quantized models do not
exhibit increased output lengths. In addition, strategically scaling the model
sizes or reasoning steps can effectively enhance the performance. All quantized
models and codes will be open-sourced in
https://github.com/ruikangliu/Quantized-Reasoning-Models.