Title: Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization
Venue: NeurIPS 2025
Accepted: yes
Keywords: quantization

post-training quantization

layer-wise post-training quantization
Abstract: Layer-wise PTQ is a promising technique for compressing large language models (LLMs), due to its simplicity and effectiveness without requiring retraining. However, recent progress in this area is saturating, underscoring the need to revisit its core limitations and explore further improvements. We address this challenge by identifying a key limitation of existing layer-wise PTQ methods: the growth of quantization errors across layers significantly degrades performance, particularly in low-bit regimes. To address this fundamental issue, we propose Quantization Error Propagation (QEP), a general, lightweight, and scalable framework that enhances layer-wise PTQ by explicitly propagating quantization errors and compensating for accumulated errors. QEP also offers a tunable propagation mechanism that prevents overfitting and controls computational overhead, enabling the framework to adapt to various architectures and resource budgets. Extensive experiments on several LLMs demonstrate that QEP-enhanced layer-wise PTQ achieves substantially higher accuracy than existing methods. Notably, the gains are most pronounced in the extremely low-bit quantization regime.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- Summary:\n> - The paper addresses the challenge of quantization error accumulation in layer-wise post-training quantization (PTQ) for large language models (LLMs), which degrades performance in low-bit quantization scenarios.\n> - The proposed QEP uses a closed-form weight correction from full-precision/quantized activation differences to address error propagation. A tunable α controls correction strength as regularization, which prevents overfitting.\n> - Extensive experiments validate that QEP enhances popular layer-wise PTQ methods (e.g., GPTQ, AWQ, QuIP) on datasets such as WikiText-2, and \"ArcE, PiQA, and SC datasets\". \n> - Ablation studies confirm the effectiveness of the tunable propagation mechanism, and runtime analyses show minimal computational overhead, making QEP suitable for large-scale LLM deployment. The framework demonstrates robustness to calibration data variations and outperforms baselines across different model sizes and bit-widths.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths:\n> - (1)  **QEP addresses the core limitation of existing layer-wise PTQ by explicitly modeling and compensating for quantization error accumulation**, a problem that causes performance degradation in low-bit quantization. The framework’s weight correction term and tunable propagation parameter α prevents overfitting.\n> - (2)  The paper provides formal proofs (appendix A) showing that QEP bounds quantization error better than baseline methods. This theoretical foundation validates the approach’s effectiveness in principle.\n> - (3) QEP demonstrates significant improvements across popular PTQ methods (GPTQ, AWQ, QuIP) on Llama-2 and Mistral-7B models. \n> - (4) Despite introducing error correction, QEP maintains computational overhead comparable to existing methods. Tunable α​ allows adaptive control over complexity as shown in 5.3.\n> - (5) QEP shows stability across calibration datasets (C4, PTB, WikiText-2) and model sizes (7B–70B), outperforming baselines in zero-shot tasks like ARC-Easy and PIQA. \n>- (6) **Source codes are available**\n\n- Weaknesses:\n> - (1) QEP’s performance might rely on representative calibration data, as overfitting to small or biased datasets can degrade results. \n> - (2) the optimal α​ requires manual adjustment (e.g., α​ = 1/2), which may limit usability for practitioners. Automated hyperparameter optimization is suggested but not explored.\n> - (3) Although the paper provides a runtime comparison in Table 3, the analysis of computational cost is insufficient. For instance, it could present the comparison between GPTQ and QEP + GPTQ.\n> - (4) The paper compares the runtime of the quantization process (Table 3), but does not provide the memory footprint data of QEP during the training phase.\n\nQUESTIONS:\n- Does the proposed method QEP rely on high-quality calibration datasets? How to remove such dependency?\n- The proof of Theorem 5.2 relies on the Lipschitz continuity assumption of activation functions. Do activation functions in practical models strictly satisfy this condition? Does it lead to a discrepancy between the theoretical upper bound and the actual error control effect?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of quantization error propagation in post-training quantization (PTQ)—a crucial yet underexplored challenge. To tackle this, the authors propose Quantization Error Propagation (QEP), a method that enhances layer-wise PTQ by explicitly propagating and compensating for accumulated quantization errors. Experiments on several large language models (LLMs) demonstrate that the proposed approach improves the performance of existing layer-wise PTQ methods across various datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper focuses on a crucial yet underexplored challenge: the accumulation and amplification of quantization errors across layers, which can significantly degrade performance. The paper is well organized and easy to follow. The proposed method is simple yet effective, and the authors conduct extensive experiments across various datasets to demonstrate its effectiveness.\n\n\nWeaknesses:\n\nThe reformulation of Eq. (3) has been discussed in previous work, such as [1], and should be properly acknowledged and discussed in the paper. Additionally, an important baseline is missing—namely, directly solving Eq. (3) using SGD as in Omniquant [Shao et al., 2023]. Including this baseline would provide a more comprehensive comparison and help to better contextualize the effectiveness of the proposed method relative to existing methods.\n\nRefernece:\n\n[1] Up or Down? Adaptive Rounding for Post-Training Quantization. ICML 2020.\n\nQUESTIONS:\n1. The authors reformulate the layer-wise quantization problem and introduce weight correction to address accumulated errors, resulting in a simple yet effective approach. However, it is unclear how the proposed method is applied to the baseline round-to-nearest (RTN) quantization. Since RTN involves simple rounding and does not consider layer-wise reconstruction error, it would be helpful if the authors could provide more detailed explanations regarding its integration with the proposed method.\n\n2. The authors introduce a scaling hyperparameter $\\alpha$ in Eq. (6) to control the strength of weight correction. However, in the experiments on Llama-2 70B, weight correction is excluded for the MLP layers by setting $\\alpha=0$. Could the authors clarify the reasoning behind this decision? As I understand it, quantization error propagation typically becomes more severe as network depth increases. Therefore, for deeper models, including weight correction should help mitigate this issue. This experimental setup appears to contradict the motivation of the paper.\n\n3. An important result is missing from Table 1. The authors should include the performance of the unquantized models to better illustrate the impact of quantization and to facilitate comparison of performance degradation.\n\n4. In Table 3, the authors only report results for QEP+RTN, which is insufficient, especially since QEP+RTN performs worse than GPTQ and AWQ. It would be more informative to also include the results for QEP combined with GPTQ and AWQ.\n\nLIMITATIONS:\nThe authors do not discuss the limitations of their work or its potential negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

comments: SUMMARY:
This work presented analytical results on quantization error propagation along network depth, and proposes a new formulation of local objective for PTQ optimization.  It also introduced a regularizer that interpolates toward the traditional local loss, to prevent overfitting.  Experiments of GPTQ, AWQ and QuIP that minimized the proposed objectives were presented.

STRENGTHS AND WEAKNESSES:
Strengths: 
+ The main idea is clearly presented.  
+ The experiments and ablations were generally comprehensive. 

Weaknesses: 
- Originality of the proposal is undermined by similar concurrent work.  
- The proposed regularizer introduced further complexity in hyperparameter tuning in practice.

QUESTIONS:
Major: 
* A recent, concurrent work, GPTAQ (Li et al. 2025, arXiv:2504.02692) proposed the same local objectives, though they only presented the application in the case of GPTQ, whereas this work combined it with multiple methods.  

Minor: 
* Though the proposed regularizer $\alpha$ might be useful, it introduced substantial complexity into hyperparameter tuning.  Also, could you demonstrate with a concrete example the _overfitting_ phenomenon and how certain $\alpha$ schedules mitigated the overfitting?   
* How does QEP interact with other orthogonal techniques such as rotationally invariant transformers for quantization?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a Quantization Error Propagation training method aiming to alleviate the performance degradation problem of large language models caused by layer-by-layer error accumulation during the quantization process. The authors model the propagation path of the quantization error and control the cumulative effect of the error by introducing a specially designed optimization objective in training. The paper includes a theoretical explanation for the effectiveness of QEP, which is a commendable attempt. Nevertheless, the paper is providing solid motivation and the necessity of some theoretical arguments, and the comprehensiveness of the experimental comparison.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The core contribution of this paper is that QEP is proposed as a plug-and-play module. The experimental part (as shown in Tables 1 and 2) clearly shows that it can be combined with a variety of existing quantization methods (such as RTN, GPTQ, etc.) and consistently reduces the perplexity of the model on multiple benchmarks. This versatility is very valuable. QEP mainly plays a role in the training or calibration phase. Once the model is quantized, there is almost no additional computational load introduced in the inference phase, which is crucial for LLM deployment scenarios that pursue extreme efficiency.\n\n2. The authors try to explain theoretically \"why controlling error propagation is effective\". By establishing a mathematical model of quantization error propagation and deriving the corresponding optimization strategy, the authors provide a more structured perspective on the problem, which helps inspire more fundamental research in the future.\n\nWeaknesses:\n\n1. One of the main weaknesses of this paper is the lack of robust motivation. The motivation of this paper is to control the accumulation and amplification effect of errors in the quantization process, which has strong similarities with the earlier work \"Defensive Quantization: When Efficiency Meets Robustness\". Although the latter focuses on adversarial robustness rather than general quantization performance, its core idea is also to suppress the propagation of perturbations (i.e., errors) by constraining the network. Given this ideological connection, the author must discuss \"Defensive Quantization\" in the introduction or related work section, and clearly explain the similarities, differences, and uniqueness of your QEP method in motivation, technical implementation, and target scenarios.\n\n2. Theorem 5.2 is also confusing. The author has defined $\\theta_{QEP} as the optimal parameter that minimizes the specific quantization error propagation loss function in Eq (3). The inequality to be proved in Theorem 5.2 seems to be a direct or even equivalent statement of the optimal solution to the optimization problem. In other words, when \\theta_{QEP}$ is already the solution to Eq (3), the conclusion of Theorem 5.2 seems to be trivial. I hope the author can further clarify the contribution and insights of the theorem if anything is missing.\n\n3. The author verified the effectiveness of QEP on multiple quantization methods, which proves its universality. But a key issue is that the baseline method you selected (such as RTN, GPTQ, etc.) is not the most powerful SOTA method in the current LLM quantization field. In recent years, many more advanced quantization schemes have emerged (such as DB-LLM for 2-bit and Omni for 2-4-bit). It is recommended that authors conduct experiments on at least one or two currently recognized SOTA quantization methods to verify whether QEP can still bring significant benefits.\n\nQUESTIONS:\nCan you add a comparative discussion with related work, such as \"Defensive Quantization\" to clarify the uniqueness and innovation of your work?\n\n2. Please elaborate on the theoretical contribution of Theorem 5.2. What profound properties does it reveal that cannot be directly seen from the optimization definition of Eq (3)?\n\n3. How do you view the potential effect of QEP on the current more advanced SOTA quantization methods?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}