G en PTQ : Green Post-Training Quantization for Large-Scale ASR Models with Mixed-Precision Bit Allocation (2025)

Abstract: Large-scale models have achieved state-of-the-art performance in automatic speech recognition (ASR), but their high memory and computation demands pose significant challenges for deployment. To address these challenges, weight-only quantization is widely adopted in large-scale models, where weights dominate memory usage, as it enables efficient compression with minimal accuracy degradation compared to activation quantization. Accordingly, most prior quantization studies for ASR models have focused on weights and employed quantization-aware training (QAT) to restore accuracy. However, QAT incurs substantial additional training costs, posing clear limitations for practical application to large-scale models. Moreover, despite the varying quantization sensitivity across layers, mixed-precision quantization (MPQ) remains underexplored in ASR. In this paper, we propose GenPTQ, a mixed-precision post-training quantization method that optimizes the trade-off among accuracy, model size, and optimization cost by leveraging gradient-based sensitivity measurement and transforming the search space into a continuous domain for efficient numerical optimization. Applied to Whisper and Conformer models across multiple speech datasets, GenPTQ achieves up to 89.1% model size reduction (2.5-bit average precision) with only a 0.8% increase in WER, and completes optimization in just 15 seconds. These results demonstrate its effectiveness for low-resource ASR deployment.

## introduction recently, large-scale end-to-end models @xcite @xcite based on the transformer architecture @xcite have achieved state-of-the-art (sota) recognition accuracy in automatic speech recognition (asr), driving their widespread adop-* † corresponding author tion across various speech processing applications @xcite . however, as model size increases, memory usage and computational cost grow substantially, leading to excessive power consumption and inference latency in real-world deployment scenarios @xcite . these issues extend beyond efficiency concerns, presenting a critical challenge in terms of environmental impact and degradation of user experience @xcite . accordingly, various model compression methods have been actively explored to reduce the size and computational cost of asr models, while preserving recognition accuracy @xcite @xcite . quantization reduces the bitprecision of weights or activations-originally represented in 32-bit floating-point format-thereby effectively reducing the overall model size @xcite . in particular, for large-scale models such as whisper @xcite , memory bandwidth often emerges as the primary bottleneck rather than computational cost. consequently, reducing weight bit-precision is crucial for improving practical efficiency @xcite . moreover, as weights consume over 3.4× more memory than activations, applying lowerbit quantization to weights is more efficient, especially given that activations tend to be more sensitive to accuracy degradation. accordingly, most asr quantization studies have primarily focused on weight-only quantization, reflecting this characteristic @xcite . quantization methods are generally categorized into post-training quantization (ptq) and quantization-aware training (qat) @xcite . ptq applies quantization directly to pre-trained weights without requiring retraining, making it attractive for its simplicity and efficiency. however, the rounding involved in converting floating-point values to lower-precision formats can introduce quantization noise, leading to potential performance degradation. to address this, qat jointly optimizes model weights and quantization parameters (e.g., scaling factors) during training, which allows the model to better adapt to quantization and typically results in higher accuracy @xcite . as a result, most prior asr quantization studies have focused on qat. nevertheless, qat imposes substantial training overhead, often requiring tens of gpu-hours. for example, applying qat to the conformer-large model demands over 84 hours of training @xcite , which presents a major barrier for practical deployment, especially in user-adaptive or low-resource settings. furthermore, as model sizes continue to grow, the training cost of qat is expected to scale accordingly @xcite . these limitations highlight the need for ptq methods that offer a better trade-off between model size and accuracy, without incurring the high cost of full retraining. meanwhile, mixed-precision quantization (mpq) mitigates accuracy degradation by assigning optimal bit-precision to each layer based on its quantization sensitivity @xcite @xcite . while mpq has shown strong potential, its application is limited by the prohibitively large search space involved in identifying layer-wise precision configurations. this results in significant computational and optimization overhead, posing a major barrier to practical adoption, especially in large-scale asr models. consequently, these challenges have slowed the progress of mpq research in the asr domain, despite its theoretical advantages. to address the limitations of prior asr quantization methods, we propose genptq, a novel mixed-precision ptq approach that achieves an improved trade-off among accuracy, model size, and optimization efficiency. genptq efficiently allocates per-layer bit precision with minimal optimization overhead, achieving robust quantization performance across various asr models and datasetsall without requiring additional training. the contributions of this work are summarized as follows: ## conclusion we presented genptq, a mixed-precision posttraining quantization framework that effectively balances recognition accuracy, model size, and op-