Title: BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Quantization; Large language Model

Abstract: Rotations are essential for state-of-the-art LLM quantization, effectively smoothing outliers in weights and activations. A prevailing trend is to pursue marginal accuracy gains by optimizing the global rotation matrix, an approach that incurs prohibitive optimization costs due to full-model backpropagation and offers limited practical utility.
In this work, we challenge this paradigm. We identify two fundamental error sources that persist even with optimal rotations: (i) channel mean misalignment, which inflates rounding errors, and (ii) clipping-induced energy loss, exacerbated by the rotation-induced Gaussian-like distributions. We argue that addressing these issues directly is a more effective path to high accuracy.
Consequently, we introduce BASE-Q, a lightweight framework that sidesteps expensive global rotation learning. Instead, it employs simple yet powerful blockwise corrections: bias correction to nullify channel mean variance and asymmetric scaling to compensate for clipping energy loss. This blockwise strategy drastically reduces optimization overhead, enabling 70B model quantization on a single GPU.
Extensive experiments on various LLMs and benchmarks demonstrate the effectiveness of BASE-Q, narrowing the accuracy gap to full-precision models by 50.5\%, 42.9\%, and 29.2\% compared to previous rotation method QuaRot, SpinQuant, and OSTQuant, demonstrating the superiority of our lightweight paradigm.