SINQ: Sinkhorn-Normalized Quantization for Calibration-Free
  Low-Precision LLM Weights
Authors: name: Lorenz K. Müller; name: Philippe Bich; name: Jiawei Zhuang; name: Ahmet Çelik; name: Luca Benfenati; name: Lukas Cavigelli
Published: 2025-09-26 21:22:54

Abstract: Post-training quantization has emerged as the most widely used strategy for
deploying large language models at low precision. Still, current methods show
perplexity degradation at bit-widths less than or equal to 4, partly because
representing outliers causes precision issues in parameters that share the same
scales as these outliers. This problem is especially pronounced for
calibration-free, uniform quantization methods. We introduce SINQ to augment
existing post-training quantizers with an additional second-axis scale factor
and a fast Sinkhorn-Knopp-style algorithm that finds scales to normalize
per-row and per-column variances, thereby minimizing a novel per-matrix proxy
target for quantization: the matrix imbalance. Our method has no interactions
between layers and can be trivially applied to new architectures to quantize
any linear layers. We evaluate our method on the Qwen3 model family and
DeepSeek-V2.5. SINQ improves WikiText2 and C4 perplexity significantly against
uncalibrated uniform quantization baselines and can be further enhanced by
combining it with calibration and non-uniform quantization levels. Code to
reproduce the results of this work and to easily quantize models using SINQ is
available at https://github.com/huawei-csl/SINQ.