Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization
Authors: Yamato Arai
Yuma Ichikawa
Venue: NeurIPS 2025 poster ()
Keywords: quantization
post-training quantization
layer-wise post-training quantization
Primary area: deep_learning
TL;DR: Revisiting layer-wise PTQ, propagating quantization errors enhances existing methods.

Abstract: Layer-wise PTQ is a promising technique for compressing large language models (LLMs), due to its simplicity and effectiveness without requiring retraining. However, recent progress in this area is saturating, underscoring the need to revisit its core limitations and explore further improvements. We address this challenge by identifying a key limitation of existing layer-wise PTQ methods: the growth of quantization errors across layers significantly degrades performance, particularly in low-bit regimes. To address this fundamental issue, we propose Quantization Error Propagation (QEP), a general, lightweight, and scalable framework that enhances layer-wise PTQ by explicitly propagating quantization errors and compensating for accumulated errors. QEP also offers a tunable propagation mechanism that prevents overfitting and controls computational overhead, enabling the framework to adapt to various architectures and resource budgets. Extensive experiments on several LLMs demonstrate that QEP-enhanced layer-wise PTQ achieves substantially higher accuracy than existing methods. Notably, the gains are most pronounced in the extremely low-bit quantization regime.

Quantization Error Propagation: Revisiting Layer-Wir Wise Post-Training Quantization
Yamato Arai
Yuma Ichikawa
Fujitsu Limited
Fujitsu Limited
Department of Basic Science
RIKEN center for AIP
The University of Tokyo
O Code: https: http:://gitttb.cc///ititt
Abstract
Layer-wise PTQ is a promising technique for compressing large language models (LLMs), due its simplicity and effectiveness without requiring retraining. How- ever, recent progress in this area is saturating, underscoring the need to revisit its core limitations and explore further improvements. We address this challenge by identifying a key limitation of existing layer-wise PTQ methods: the growth of quantization errors across layers significantly degrades performance, particularly low-bit regimes. To address this fundamental issue, we propose Quantization Error Propagation (QEP), a general, lightweight, and scalable framework that enhances layer-wise PTQ by explicitly propagating quantization errors and compensating for accumulated errors. QEP also offers a tunable propagation mechanism that prevents overfitting and controls computational overhead, enabling the framework to adapt to various architectures and resource budgets. Extensive experiments several LLMs demonstrate that QEP-enhanced layer-wise PTQ achieves sub- stantially higher accuracy than existing methods. Notably, the gains are most pronounced in the extremely low-bit quantization regime.
Introduction
Large Language Models (LLMs) have achieved impressive performance in various natural language processing tasks, including open-ended text generation, multi-step reasoning, and dialogue modeling. Notable examples include ChatGPT [Achiam et al., 2023] and the Ilama family ['ouurron al., 2023, Grattafiori et al., 2024]. However, deploying LLMs cost-effectively remains difficult because of their substantial memory usage and computational demands [Chen et al., 2023]. This limitation is especially critical for edge computing and latency-sensitive applications. To address these challenges, a wide range of model compression techniques, such as quantization [Lang et al., 2024, Gong et al., 2024], pruning [Wang et al., 2024, Cheng et al., 2024], low-rank approximation [Yang al., 2024a, Hu et al., 2022], and knowledge distillation [Xu et al., 2024a, Yang et al., 2024b], have been explored.
Among these methods, layer-wise post-tran teattrraining quantization (PTQ) has emerged as practical and widely used approach for large-scale I.LMs [Frantar et al., 2022, Lin al., 2024, Yao al., 2022, Chee et al., 2023]. Unlike biock-wise PTQ [Tseng et al., 2024, Shao et al., 2023], global fine-tuning training (QAT) [Xu et al., 2024b,
[Egiazarian al., 2024, Tseng et al., 2024], quantization-awaan Wang et al., 2023, Liu et al., 2023]. and all of which require heavy retraining and backpropagation. layer-wise PTQ quantizes model parameters layer-by-layer without retraining or backpropagation. resulting in significantly lower computational and memory demands. Despite its simplicity. layer-wise PTQ effectively preserves model quality even at lower bit widths [Frantar et al., 2022, Lin et al., 2024, Chee et al., 2023]. As result, layer-wise PTQ is increasingly adopted in real-world applications due to its efficient quantization, reduced computational cost, and broader compatibility with large-scale LLMs. varying bit widths, and diverse quantization strategies.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

INT4 Quantization
INT3 Quantization
INT2 Quantization
6.5
20.0
55.0
Perplexity on WikcText-2
RTN
ANO
QUINITN
QUP-WOO
GPTO
QuP
5.5
QEPOPPTQ
QUP
15.0
40.0
25.0
4.5
10.0
10.0
3.5
5.0
7B
13B
70B
7B
13B
70B
7B
13B
70B
Model Size
Model Size
Model Size
Figure I:
WikiText-2 perplexity comparison across Llama-2 models (7B-70B) quantized to INT-4, INT-3, and INT-2, employing RTN, GPTQ, AWQ. and QulP methods. Solid bars indicate PTQ with QEP: border bars represent PTQ without QEP. Truncated bars indicate perplexities exceeding axis limits. QEP consistently reduces perplexity, with greater improvements observed at lower bitwidths and smaller model sizes. See Section 6 for detailed settings and results.
Despite significant progress layer-wise PTQ, advancements in this area are saturating [Malinovskii et al., 2024]. This study aims to push the performance boundaries of layer-wise PTQ by revisiting its core design strategy. This study begins by identifying a fundamental limitation of existing layer-wise PTQ approaches. These approaches do not adequately account for the propagation of quantization errors across layers. Quantization errors accumulate significantly, leading to a degradation overall model performance, especially in low-bit settings. This represents a key bottleneck for the practical deployment of layer-wise PTQ in large-scale LLMs.
To address this issue, we propose Quantization Error Propagation (QEP). a general and computation- ally efficient framework that enhances the performance of layer-wise PTQ methods. QEP modifies the layer-wise optimization objective to propagate and compensate for accumulated quantization errors, while maintaining computational complexity comparable to existing layer-wise PTQ methods. Furthermore, we introduce a tunable propagation mechanism whose adjustable propagation strength prevents overfitting, known issue previously observed in GPTQ [Lin al., 2024]. This mechanism also enables adaptive control over computational overhead, especially in parameter- heavy components such as MLP blocks. Notably, the enhancement of QEP is orthogonal to existing PTQ methods and can be seamlessly integrated with any layer-wise PTQ pipeline.
Extensive experiments several LLMs across various bit-width settings show that QEP significantly enhances layer-wise PTQ methods, including GPTQ [Frantar et al.. 2022], AWQ [Lin et al., 2024], QulP [Chee et al., 2023], as shown Figure These improvements are particularly pronounced in extreme low-bit regimes, such as 2-bit quantization, where standard layer-wise PTQ methods typically degrade significantly.
2 Related Work
Quantization techniques primarily include data-free PTQ [Dettmers and Zettlemoyer, 2023], layer- wise PTQ [Frantar et al., 2022, Lin et al., 2024, Chee et al., 2023], block-wise PTQ [Tseng et al., 2024, Shao et al., 2023], global fine-tuning PTQ [Egiazarian et al., 2024, Tseng et al., 2024], and QAT [Xu et al., 2024b, Wang et al., 2023, Liu et al., 2023]. Among these methods, weight-only layer-wise PTQ has become especially popular for large-scabe scale models because of its computational efficiency and strong performance [Frantar et al., 2022, Lin et al., 2024, Chee et al., 2023]. Recent benchmarking further highlights that most PTQ advances specifically target layer-wise methods [Zhao et al., 2025]. Following the taxonomy [Zhao et al., 2025), we outline three distinct approaches and recent developments.
Compensation-bast layer-wise PTQ This category, pioneered by GPTQ [Frantar al., 2022], uses sequential quantization strategy, in which model weights are quantized based on the Hessian computed from a calibration dataset, while compensating for subsequent unquantized weights. Several studies refined the compensation mechanism by improving update rules [Behdin et al., 2023], integrating nonlinear quantization schemes [L.iu et al., 2024a], employing adaptive grid selection [Zhang and Shrivastava, 2024], and using block-wisc optimization [Guan et al., 2024].

---

Rotation-based layer-wise PTQ A second promising direction, advanced by QulP [Chee et al., 2023], involves preprocessing weights through structured rotation matrices more uniformly re- distribute weight magnitudes. This approach was improved by randomized Hadamard transforms and block-wise and global fine-tuning optimization [Tseng et al., 2024]. Learning-based methods to determine rotation matrices have also been introduced [Liu et al., 2024b]. This rotation-baa based strategy has also been extended to activation quantization [Ashkboos et al., 2024].
Salience-based layer-wise PTQ Other approaches focus on identifying and preserving salient weights, often using mixed-precision quantization frameworks [Dettmers et al., 2022, 2023, Shang et al., 2023]. Although mixed-precision methods usually add complexity due to various data types, AWQ [Lin et al., 2024] mitigates these implementation difficultics. AWQ strategically employs global scaling mechanism to align salient weights with the quantization grid better, simplifying deployment while maintaining high accuracy.
Recent advances in layer-wise PTQ have mainly focused on nonlinear quantization and block -wiee and global fine-tuning extensions; however, the fundamental layer-wise optimization has remained largely unchanged since GPTQ [Frantar et al., 2022]. This study revisits this foundational strategy, identifies its key limitations, and proposes improvements, demonstrating performance gains on the fundamental benchmarks such as GPTQ [Frantar et al., 2022], QulP [Chee et al., 2023], and AWQ [L.in et al., 2024]. Therefore, our contributions complement and are orthogonal to recent advancements, such as nonlinear quantization and structured extensions.
3 Background
Post-training quantization Post-training quantization (PTQ) is a technique that converts the parameters of pre-trained models into discrete quantized representations. Formally, let W; E R= denote the pre-trained weight matrix associated with the I-th linear operation. Note that the index I specifically refers to individual linear transformations rather than entire transformer blocks. The objective of PTQ is to find quantized approximation Wi E Quixdi that closely approximates the behavior of the original model, preserving performance while reducing computational costs and memory usage. The set Q C denotes the discrete quantization domain, which is represented as a finite set of 2h distinct quantization levels, referred to as b-bit quantization scheme. To achieve accurate quantization, many approaches leverage a small calibration dataset. Specifically, given calibration dataset X [dd xm consisting of m samples, these methods aim to find optimal quantized parameters Wi that minimizes the deviation from the performance of the original model.
Layer-wise PTQ Layer-wise PTQ has emerged as a promising framework [Frantar et al., 2022, Frantar and Alistarh, 2022] for compressing large-scale LLMs. Recent advancements in this area have significantly reduced the computational overhead and memory requirements of deploying LLMs. Despite methodological differences, existing layer-wist PTQ approaches typically follow shared sequential quamtization scheme, processing each layer independently and sequentially from the input layer toward the output layer.
Formally, these techniques quantize the model parameters (Wi)ez by solving the following layer- wise independent optimization problem:
min W.
WiX: WIX F.
(1)
where Xx denotes the input activations to the 1-th layer. This quantization proceeds sequentially from / 1 toward the output layers. Due to the quadratic form of the reconstruction objective, the associated Hessian, H₃ = XXX, can be efficiently precomputed and cached for reuse in subsequent optimization steps. improving computational efficiency practice.
Existing PTQ methods typically use one of two possible forms for activation inputs X₁: Either quantized activations X,, obtained by forward propagating the calibration dataset through previously quantized weights (Wi,.. ......, Wi-1). or full-precision activations X,, resulting from forward propa- gation through the original, unquantized weights { (W,... W,-).. There is no consensus among existing PTQ methods [Frantar et al., 2022, Lin et al., 2024, Chee et al., 2023] regarding whether quantized or full- full-precision activations produce better quantization outcomes.
3

---

2-bit Quantization
3-bit Quantization
4-bit Quantization
0.3-
BASE
15.0
0.6
With QEP
0.2
10.0
0.4
0.1
5.0
0.2
0.0
0.0
0.0
10
20
30
10
20 (Block)
30
10
20
(Block)
(Block)
Figure 2: Accumulation and growth of quantization errors across layers in a partially quantized 1lama2-7B model [Touvron et al., 2023]. The first 10 Transformer blocks are quantized using standard RTN (BASE) and QEP-enhanced RTN (With QEP), while the remaining Transformer blocks after the 10th remain at full precision. The plot shows the squared Frobenius norm Am. defined in Eq. (2), between the original and partially quantized outputs at each Transformer block m.
Leading layer-wise PTQ methods use distinct optimization strategies to approximate the behavior of the original model while adhering to the foundational sequential layer-wise framework in Eq. (1). GPTQ [Frantar et al., 2022], for example, uses quantized activations, X = X,, and quantizes parameters row-wise by sequentially minimizing reconstruction error and correcting residuals in the remaining unquantized entries until each row is fully quantized. AWQ [Lin et al., 2024] uses original activations, X X,, and identifies small subset of salient weights whose magnitudes significantly influence the layer outputs, subsequently rescaling these weights before quantization.
4 Bottleneck: Quantization Error Accumulation and Growth
To motivate our proposed approach, this section first revisits the core layer- lation given by Eq. (1), emphasizing its key limitation: The accumulation and growth of quantization errors across layers significantly degrade the performance. We investigate this phenomenon using experiments conducted on the pre-trained Llama-2-7B model [Touvron al., 2023]. Specifically, quantize only the first 10 Transformer blocks (Vaswani et al., 2017]. while keeping all subsequent blocks in full precision. To quantify the propagation and accumulation of the errors, measure the discrepancy between fully precise and partially quantized outputs at each block using calibra- tion dataset. Let TransBlock. occk (-) denote the original full-precision m-th Transformer block, and TransBlock,, (-) denote quantized counterpart. We evaluate the following metric at the m-th block:
layer-wise wise optimization formu-
Am (x) Fm (X)
(2)
(x) = TransBlock,, .... TransBlock TransBlock, ....o TransBlock;( (X), (X) TransBlock,,
TransBlock,+1 TransBlock, ....o TransBlock, (X).
This experiment sets 10. Figure 2 shows an approximately exponential accumulation or errors within the quantized layer, as well as an error growth that persists the unquantized layers. This growth occurs due to the layer-wise independent quantization approach described in Eq. (1), which neither accounts for quantization error propagated from previous layers nor corrects previously accumulated errors, thus exacerbating error growth in subsequent unquantized layers. The exponential accumulation of quantization errors observed empirically can also be theoretically explained under mild conditions, detailed in Appendix B.2. Therefore, instead of treating layer-wiss quantization as a series independent optimization problems, it essential to reformulate the original layer-wise optimization presented in Eq. (1) to mitigate accumulation and growth.
5 QEP: Quantization Error Propagation
Existing layer-wise independent PTQ has inherent limitations, particularly the accumulation and growth of quantization errors discussed Section4. To address these limitations, we introduce Quantization Eroo Propagation (QEP), general, lightweight, and scalable framework that improves

---

layer-wise PTQ by propagating quantization errors. In subsequent sections, we provide theoretical evidence showing that QEP effectively reduces quantization errors.
5.1 Problem Reformulation
We reformulate the layer-wise independent optimization strategy presented in Eq. (1) to propagate quantization errors across layers effectively. Instead of minimizing output differences based on shared input activations X,, our reformulation directly minimizes the discrepancy between pelecpriinionn and quantized outputs, each computed using their respective upstream inputs. Formally, for each layer 1, we optimize the discrete quantized weight matrix Wi as follows:
min EiEQ
WiXi
WiXi
(3)
This objective ensures that the quantized weights W, are optimized not only to independently approximate the full-precision weights W; but also to counteract and compensate for the cumulative quantization errors introduced by previous layers. In contrast to the existing objective in Eq. (1), where the trivial optimal solution is Wi Wi if Wi Qmix4,, the optimal solution under the formulation Eq. (3) is generally Wi + Wi, explicitly enabling error correction and accounting for accumulated quantization errors.
Although the modification from Eq. (1) seems straightforward, Eq. (3) inherently breaks the key structural simplification that facilitates efficient quantization existing PTQ frameworks. Specifically, the optimization in Eq. (3) longer solely depends on the Hessian matrix H₁, thereby preventing the direct use of existing Hessian-bart -ased acceleration methods for quantization. In the following section, we address this challenge by proposing a practical and efficient weight correction scheme that overcomes this limitation while retaining the advantages of our error-propagation approach.
5.2 Weight Correction
To efficiently perform quantization by the objective in Eq. (3) in existing layer-wise PTQ methods, we relax the discrete feasible set to a continuous domain, leading to the following proposition.
Proposition 5.1. Assume that the matrix Hi is invertible. Then, after relaxing the discrete feasible set X into the continuous domain xx,,d the optimal solution W is given by the following closed-form expression:
Wi = Wi + W H
argmin W.,
| - WIX
(4)
where 8 = Xi X represents the accumulated quantization error from proceeding layers, H. = X denotes the empirical Hessian constructed from quantized activations.
The proof of Proposition 5.1 is provided in Appendix B.1. Proposition 5.1 highlights important distinction from the existing formulation given by Eq. (1). Specifically, when upstream quantization introduces non-negligible errors, i.e., 8,_ j O, the optimal quantized weights differ from straightfor- ward approximations of the original weights W, Instead, the optimal solution explicitly includes a correction term that compensating for accumulated quantization errors.
This corrected weight enables us to reformulate the equivalent optimization objective within the
original discrete set W, E Quixd as follows:
min W.e
X W.X
(5)
This objective shares the same structure Eq. (1), with W; replaced by its corrected counterpart W This reformulation restores the quadratic structure found in Eq. (1), facilitating efficient optimization through the Hessian matrix H; = H The structure of Eq. (5) allows for seamless integration with various existing layer-wise PTQ methods, as discussed in Section2. Furthermore, the proposed layer-wise quantization formulation in Eq. (3) formally guarantees improved quantization accuracy compared to the existing layer-wise independent PTQ defined in Eq. (1). Specifically, we establish the following theoretical result:
5

---

Theorem 5.2 (Informal). Consider an L-layer neural network defined by:
(W) ...))..
where each activation function is Lipschitz continuous and ô denotes the set of all full-precision parameters (Wil--1- The output quantization error of the proposed quantization method defined in Eq. (3) is bounded by that of the existing layer-wise PTQ defined in Eq. (I):
[o(X) SÂEER (x)1 S <|160(x) - -SōSASE (x)
where PQEP and ÓBASE denote the sets of parameters quantized by the objective in Eq. (3) and the base PTQ method by the objective in Eq. (1), respectively.
Explicit conditions and detailed proof are provided in Appendix B.3. The additional computational overhead arises solely from computing the correction term 8; since computing the Hessian inverse H remains unchanged from existing layer-wise independent PTQ. As empirically demonstrated in Section 6.1, this additional computation requires significantly less runtime compared to the quantization processes of layer-wise PTQ methods, even for large-scale LLMs, due to the tunable mechanism described in the next section.
5.3 Controlling Propagation Strength
Although solving Eq. (5) effectively reduces the accumulation of quantization error, it can lead to overfitting. This issue is particularly pronounced when the calibration dataset is small and insufficiently representative of the target task, or when the model includes blocks with large number of parameters such as the MLP blocks commonly found in transformer architectures, causing the correction to overfit the calibration dataset.
To address this issue, introduce tunable propagation mechanism that generalizes the correction term using a scaling parameter og E [o, 1]:
W (az) Wi a Woo,X, H.
Here, setting a = recovers original fully-corrected case presented Eq. (4). whereas setting
ai = 0 corresponds to the existing approach in Eq. (1) under the setting that X X1. This tunable correction mechanism relates the following regularization optimization:
Proposition 5.3. The parameter og corresponds to the regularization parameter X in the following optimization problem:
min WiEQ Xd
WiX: - WiXi + W - Will. AER.
Specifically, as oy increases from 0 to 1, the corresponding parameter À decreases too to O.
The derivation is provided in Appendix B.4. Additionally, the following proposition is established.
Proposition 5.4. Under the same assumptions in Theorem 5.2, the oulput quantization error of the method employing QEP with parameter { decreases monotonically as euch ai approuches
Explicit conditions and comprehensive proofs of this proposition are provided in Appendix B.3. Consequently, the parameter on effectively controls overfitting, analogous to regularization techniques, and importantly provides a systematic way to balance overfitting and underfitting in layer-wise PTQ methods. Indeed, this parameter is crucial for preventing overfitting, especially in MLP blocks, which contain more parameters than other blocks.
Furthermore, in large-scale LLMs, the high-dimensional activations in MLP layers often result in computationally expensive correction terms. In these cases, selectively setting a₁ = 0 for specific layers eliminates the computational cost of the correction term and acts implicit regularization. potentially improving generalization. Therefore, appropriately setting a; correction time by approximately one-third and one-half. Developing adaptive strategies for layer- wise, data-aware, resource-efficient tuning of a; is a promising direction for future research. In the following, we refer to the overall approach, including the tunable mechanism controlled by {or)e Quantization Error Propagation (QEP).
can reduce the

---

Experiments
We conduct experiments to validate the effectivvnesss of QEP in improving the performance of layer-wise PTQ relative to existing methods.
Baselines We use representative layer-wise PTQ methods based on linear quantization such as round-to-nearest (RTN) [Frantar et al., 2022, Dettmers and Zettlemoyer, 2023], GPTQ [Frantar et al., 2022], AWQ [Lin et al., 2024], and QulP [Chee et al., 2023]. Although previous studies have explored extensions, such as non-linear and block-wise quantization, as discussed in Section 2, these techniques are orthogonal to the core improvement introduced by QEP. Therefore, to isolate and emphasize the impact QEP, we focus on these representative layer-wise PTQ methods.
Quantization This study focuses on weight-only quantization schemes, specifically per-channel and group-wise quantization, which have recently shown superior trade-offs between efficiency and accuracy [Dettmers and Zettlemoyer, 2023, Frantar al., 2022, Lin al.. 2024]. The main text evaluates per-channel quantization under INT4, INT3, and INT2 precision settings. Due to space constraints, detailed results for group-wise quantization are presented in Appendix D. For the propagation strength parameter og, we adopt a representative default value of except for the MI.P layers in the Llama-2 70B model, for which we set os = 0. Tuning oj can further improve performance but is beyond the scope of this study and is left for future work.
1/2 for all layers,
Datasets Following previous studies, we evaluate the Hessian matrix using the same default calibration datasets used in their original implementations. Specifically, GPTQ and QulP use the C4 dataset [Frantar et al., 2022] for calibration, while AWQ uses the Pile dataset [Gao et al.. 2020]. Following Frantar et [2022], we evaluate the correction term in Eq. (4) using 128 randomly sampled segments of 2048 tokens each from the C4 dataset[Raffe et al., 2020], which consists of web-crawled text excerpts.
Models Following Lin al. [2024], Frantar al. [2022]. we evaluate our method on recent popular LLMs, namely the Llama-2 and Llama-3 model families [Touvron et al., 2023], with size ranging from 7B 70B parameters, well as Mistral-7 B [Jiang, 2024]. These models demonstrate superior performance compared to other open-source LLMs [Zhang et al., 2022, Workshop et al., 2022] and have become widely adopted as foundational models for numerous derivative open-source models [Taori et al., 2023, Chiang et al., 2023].
Evaluations Following established evaluation protocols from prior studies [Dettmers et al., 2022, Xiao al., 2023, Frantar et al., 2022, Dettmers and Zetllmoyer, 2023, Yao et al., 2022], we evaluate the quantized LLMs using the perplexity (PPL) on WikiText2 [Merity et al., 2016], Penn Treebank (PTB) [Mareus al., 1994], and C4 [Raffel al., 2020], and zero-shot accuracy on benchmarks including ARC Easy (ArcE) [Boratko et al., 20181, PiQA [Bisk et al., 2020], and StoryCloze (SC) [Mostafazadeh et al., 2016]. Due to space limitations, detailed results for each dataset are provided in Appendix D. All experiments are conducted using single NVIDIA V100 GPU.
6.1 Results
Perplexity Table summarizes PPL results of various quantized models evaluated on WikiText2, comparing several bit-widths and different layer-wise PTQ methods, both with and without QEP. Additional C4 and PTB dataset results are provided in Appendix D. demonstrating consistent trends in the following. Our results indicate that incorporating QEP significantly enhances the performance of layer-wise PTQ. substantially reducing perplexity across nearly all tested methods and quantization levels. In medium-bit scenarios such INT4 and INT3, where AWQ already exhibits strong performance, applying QEP yields further improvements. At 2-bit quantization, existing layer- wise PTQ methods based on línear quantization typically suffer severe PPL degradation, rendering deployment infeasible. However, QEP effectively mitigates this issue, making INT2 quantization achievable with practical perplexity levels. Notably, QE-Peenhanced QulP achieves state-of-the-a perplexity results among all tested layer-wise PTQ methods. Similar significant improvements are observed for RTN, GPTQ, and AWQ at INT2g32, INT2g64, and INT2g 28 quantization levels; see Appendix D.I for details.

---

Table 1: Evaluation of perplexities (1) for Llama models on WikiText-2 under various layer-wise PTQ methods and bitwidths.
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
Llama-3-8B
Mistral-7B
FP16
5.472
4.883
3.319
6.137
5.255
X
6.1 16
5,206
3.672
8,540
5.997
RTN
6.017
5.165
3.621
8.021
5.877
X
6.083
5.167
3.594
147.912
5.643
GPTQ
5,933
5.127
3.576
9.509
5,528
INT4
5.831
5.064
3.484
7.108
5.716
AWQ
5.756
5.041
3,479
6,981
5.636
8,434
5.137
3.826
6.998
11.109
QulP
5.753
5.034
3.485
6.650
5,479
539.866
10.688
7.530
2276.227
29.390
RTN
17.309
7.458
5.648
86.430
10.241
10.881
6.632
4.860
64.457
8.247
GPTQ
7.898
6.245
4.102
18.845
7.347
INT3
15 15.299
6,448
4.362
11.802
7.902
AWQ
11.131
6.092
4.103
10.713
7.169
12.048
5,503
4.135
8.288
7.108
QulP
6.154
5.347
3.813
7.703
5,842
17783.918
51152.832
26077.172
1437176.750
78488,328
RTN
97153.266
158.5555
26063.672
554142.313
50540.059
13051,460
1301.395
107.458
236596.891
3543.708
GPTQ
7214.328
2782.353
52.472
282245.
1665.287
INT2
199448.797
93036.517
81834.344
1044956.250
31391.543
AWQ
729888.406
74735.836
88684.156
639158.313
32668.666
65.593
11.232
6.536
70.518
26.632
QulP
11.972
8.417
5.869
27.326
9.586
Zero-shot tasks We evaluate the zero-shot accuracy of quantized models several tasks. Table summarizes the average accuracy for the ArcF, PiQA, and SC datasets. Detailed results for each dataset are provided in Appendix D.2. Consistent with the perplexity results, QEP effectively improves existing layer-wise PTQ methods. Notably, the performance gains from QEP are especially pronounced with INT2 quantization. For Llama-2-70B, the QEP-enhanced QulP INT2 achieves performance comparable to RTN and GPTQ at INT3 quantization.
Runtime We examine the impact of computation time required for the correction term. Table shows the process- ing time of each layer-wise PTQ. Since the quantization processing time for RTN is only few seconds and thus negligible, the measured time for QEP+RTN primar- ily due to computing the preprocessing of the correction term. This result indicates that calculating the QEP correc- tion term requires significantly less computation time than other existing layer-wise PTQ quantization processes. Moreover, using the same calibration dataset for weight correction and quantization reduces preprocessing overhead by approximately one-half to one-third by reusing computational steps.
Table 3: Runtime comparison of the quantization process.
Runtime
Llama-2
7B
13B
70B
GPTQ AWQ QEP RTN
14.9m
26.4m
2.9h
13.6m
25.4m
2.4h
10.9m
19.6m
1.7h
Table 4: Perplexity relative to RTN on Wiki- Text2, comparing GPTQ and QEP+RTN cali- brated with C4, PTB, and WikiText2.
Robustness As discussed in Section 5.3, our method adaptively controls propagation strength in Eq. (6) to mitigate overfitting to the calibration dataset. this section, we empirically validate this approach. Table 4 compares the perplexity differ- ence among QEP-enhanced RTN, GPTQ, and RTN when quantizing Llama-2-77 evaluated Wiki-
Calibration Dataset
PPL RTN (L)
C4
PTB
WikiTex12
GPTQ
-0.25
+0.07
-0.46
QEP RTN
-0.33
-0.30
-0.49

---

Table 2: shoo average accuracy (f) on ARC-Easy, PIQA, and StoryCloze for Llama models across three quantization settings.
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
Llama-3-8B
Mistral-7B
FP16
0.7601
0.7840
0.8014
0.7920
0.8056
X
0,6802
0.7160
0.7325
0.7643
0.7831
RTN
0.6844
0.7131
0.7343
0.7686
0.7921
0.6817
0.7134
0.7306
0,4812
0.7906
GPTQ
0.6795
0.7104
0.7308
0.7531
0.7904
INT4
0.6832
0.7120
0.7257
0.7821
0.7956
AWQ
0.6870
0.7126
0.7331
0.7879
0.7967
0.6500
0.7248
0.7285
0.7872
0.7204
QulP
0.6920
0.7167
0.7311
0.7800
0.8012
0.4770
0.6082
0.6402
0.4560
0.6448
RTN
0,5802
0,6550
0.6939
0.5388
0.6963
0.6367
0.6747
0.7043
0.4891
0.7305
GPTQ
0.6549
0.6853
0.7078
0.5901
0.7422
INT3
0.5840
0.6886
0.7209
0.7074
0.7534
AWQ
0.6264
0.6916
0.7283
0.7216
0.7675
0.6232
0.7034
0.7246
0.7433
0.7422
QulP
0.6804
0.7128
0.7273
0.7549
0.7933
0.4139
0.4283
0.4147
0.4183
0.4130
RTN
0.4199
0.4191
0.4145
0.4108
0.4084
0.4162
0,4222
0.4356
0.4116
0.4159
GPTQ
0.4263
0.4283
0.4714
0.4228
0.4148
INT2
0.4213
0,4176
0.4129
0.4164
0.4177
AWQ
0.4162
0,4165
0.4140
0.4150
0.4181
X
0.4667
0,5945
0.6628
0.4600
0.5422
QulP
0.5926
0.6404
0.6998
0.5121
0.6858
Text2 across various calibration datasets. Consistent with prior findings [Lin et al., 2024]. GPTQ exhibits significant sensitivity to the calibration dataset: it outperforms RTN C4 and WikiText2 but experiences notable performance degradation on PTB. In contrast, QEP-enhanced RTN consistently improves performance across all calibration datasets, demonstrating robustness to distributional shifts. This highlights the effectiveness of propagation control in preventing overfitting to the calibration dataset.
Conclusion
We revisit the core design of layer-wise PTQ and identify critical limitation: the exponential accumulation and growth of quantization errors across network layers. To address this issue, we propose QEP, general framework that explicitly propagates and compensates for accumulated quantization errors. Extensive experiments demonstrate that QEP substantially improves performance, especially in low-bit quantization scenarios. These findings underscore that meaningful progress in layer-wise PTQ can still be made by revisiting fundamental strategies, complementing recent trends primarily centered around non-linear and block-wise quantization techniques. Integrating QEP with these advanced quantization methods in the future presents a promising approach toward achieving extreme compression, potentially exceeding QAT performance.
Limitations QEP relies on small calibration set, as in other layer-wise PTQ approaches, which makes performance sensitive data quality: however, overfits less than comparable methods such as GPTQ and AWQ. The method also introduces per-layer propagation-strenge parameter og: Although this parameter is tunable, fixed value, e.g., ai = 1/2, works well in most cases, and automatic learning of oę is left for future work.

---

References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Alemán, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. GPT-4 technical report. arXiv preprint XXXxx:3333007744 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302. 2023.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The LLaMA 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305. 00000.0.517750 /66 2023.
Jiedong Lang, Zhehao Guo, and Shuyu Huang. A comprehensive study on quantization techniques for large language models. In Proceedings the 4th International Conference on Artificial Intelligence, Robotics, and Communication IantiiionnIAAAAA (ICAIRC), pages 224-231. IEEE, 2024.
Ruihao Gong, Yifu Ding, Zining Wang, Chengtao Lv, Xingyu Zheng, Jinyang Du, Haotong Qin, Jinyang Guo, Michele Magno, and Xianglong Liu. A survey of low-bit large language models: Basics, systems, and algorithms. arXiv preprint arXiv:2409. 1664:44 2024.
Wenxiao Wang, Wei Chen, Yicong Luo, Yongliu Long, Zhengkai Lin, Liye Zhang, Binbin Lin, Deng Cai, and Xiaofei He. Model compression and efficient inference for large language models: survey. arXiv preprint 2rXiv:2402.09746 2024.
Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Menglin Yang. Jialin Chen, Yifei Zhang, Jiahong Liu, Jiasheng Zhang. Qiyao Ma, Harshit Verma, Qianru Zhang, Min Zhou, Irwin King, et al. Low-rank adaptation for foundation models: A comprehensive review. arXiv preprint arXiv:2501.00366 2024a.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. International Confer- ence on Learning Representations (ICLR), 2022.
Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116 2024a.
Chuanpeng Yang, Yao Zhu, Wang Lu, Yidong Wang, Qian Chen, Chenlong Gao, Bingjie Yan, and Yiqiang Chen. Survey knowledge distillation for large language models: Methods, evaluation, and application. ACM Transactions on Intelligent Systems and Technology. 2024b.
Elias Frantar, Salch Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.173233 2022.
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. Proceedings of Machine Learning and Systems, 6: 87-100, 2024.
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35:27168 01277168-27111 27183, 2022.
Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M. De Sa. Quip: 2-bit quantization of large language models with guarantees. Advances in Neural Information Processing Systems, 36: 4396-4429 2023.
10

---

Albert Tseng, Jerry Chee, Qingyao Sun. Volodymyr Kuleshov, and Christopher De Sa. Quip#: Even better LLM quantization with hadamard incoherence and lattice codebooks. arXiv preprint arXiv:2402.04399 2024.
Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models. arXiv preprint arXiv:2308. 13137, 2023.
Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, and Dan Alistarh. Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06/ 18, 2024.
Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. Onebit: Towards extremely low-bit large language models. arXiv preprint arXiv:2402./1299 2024b.
Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma. Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. Bitnet: Scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310. O./ 1453, 2023.
Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mchdad, Yangyang Shi, Raghuraman Krishnamoorthhi, and Vikas Chandra. LLM-QAT: Data-free quantization aware training for large language models. arXiv preprint arXiv:2305. / 2023.
Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtárik. PV-Tuning: Beyond straight-through estimation for extreme LLM compression. Advances in Neural Information Processing Systems, 37:5074-5121, 2024.
Tim Dettmers and Luke Zettlemoyer The case for 4-bit precision: k-bit inference scaling laws. In Proceedings of the International Conference on Machine Learning (ICML), pages 7750-7774, 2023.
Jiaqi Zhao, Ming Wang, Miao Zhang, Yuzhang Shang, Xuebo Liu, Yaowei Wang, Min Zhang, and Liqiang Nie. Benchmarking post-training quantization in LLMs: Comprehensive taxonomy, unified evaluation, and comparative analysis. arXiv preprint 1rXiv:2502.13178, 2025.
Kayhan Behdin, Ayan Acharya, Sathiya Keerthi Aman Gupta, and Rahul Mazumder. Quantease: Optimization- based quantization for language models an efficient and intuitive algorithm. stat, 1050:5, 2023.
Yifei Liu, Jicheng Wen, Yang Wang. Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, and Mao Yang. VPTQ: Extreme low-bit vector post-training quantization for large language models. arXiv preprint arX1v:2409.17666, 2024a.
Tianyi Zhang and Anshumali Shrivastava. Leanquant: Accurate and scalable large language model quantization with loss-enor--wwre grid. arXiv preprint arXiv:2407.1003: 2024.
Ziyi Guan, Hantao Huang, Yupeng Su, Hong Huang, Ngai Wong, and Hao Yu. APTQ: Attention- aware post-training mixed-precision quantization for large language models. In Proceedings of the 61 ACM/IEEE Design Automation Conference, pages 1-6, 2024.
Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krish- namoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. Spinquant: LLM quantization with learned rotations. arXiv preprint 4rXiv:2405.16406, 2024b.
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference in rotated LLMs. Advances in Neural Information Processing Systems, 37:100213-100240 2024.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3. 8-bit matrix multiplication for transformers at scale, Advances in Neural Information Processing Systems, 35: 00318-03332, 2022.
11

---

Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashk- boos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. Spqr: A sparse-quantized represen- tation for near-lossless L.M weight compression. arXiv preprint arXiv:2306.030772 2023.
Yuzhang Shang, Zhihang Yuan, Qiang Wu, and Zhen Dong. Pb-llm: Partially binarized large language models. arXiv preprini urXiv:2310.0003: 2023.
Elias Frantar and Dan Alistarh. Optimal brain compression: A framework for accurate post-training quantization and pruning. Advances in Neural Information Processing Systems, 35:4475 4488, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800 gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101 0000070.002..... 2020.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020.
Fengqing Jiang. Identifying and mitigating vulnerabilities in Ilm-integrated applications. Master's s thesis, University of Washington, 2024.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, al. OPT: Open pre-trained transformer language hwtgg/ab//2205... 01068, 2022.
models. https://araa
BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush. Stella Biderman, et al. BLOOM: A 176--parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 2022. v4.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-foolooingg LLaMA model. GitHub repository, 2023. URL https: / thub.
wttpiiaaaaattttddddaeeeee _alaca..
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, et al. Vicuna: An open-source chatbot impressing GPT-4 with 90%* * chatgpt quality. https: //lmsys.////m..m/ org/blog/ (ealllgg/22230022 2023-03-22222 30- vicuna/, March 2023.
Guangxuan Xiao, Ji Lin, Mickael Seznec. Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. Proceedings of the International Conference on Machine Learning (ICML), pages 38087-38099 2023.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. https://arxiv wwgp///iiooooooooooooooooooo aa//a/as////0 07843, 2016.
Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. The Penn Treehank: Annotating predicate argument structure. In Human Language Technology Workshop (HIT), 1994.
Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj. Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue-Nkoutche, Pavan Kapanipathi, Nicholas Mattei, et al. A systematic classification of knowledge, reasoning, and context within the ARC dataset. https: //arxiv. org/ abs/ 1806. 00358, 2018
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. PIQA: Reasoning about physical commonsense natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7432-7439, 2020.
12

---

Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He. Devi Parikh, Dhruv Batra, Lucy Vander- wende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of NAACI-HILT, pages 839-849, 2016.
Ji Lin, Chuang Gan, and Song Han. Defensive quantization: When efficiency meets robustness. arXiv preprint arX4v:I904.000
2019.
Yuhang Li, Ruokai Yin, Donghyun Lee, Shiting Xiao, and Priyadarshini Panda. Gptaq: Efficient finetuning- free quantization for asymmetric calibration. In Forry-second International Conference on Machine Learning, 2025. URL
http///errreeeeeeeeeerrrrrrreeee pett/forumiid-qdELyylo...
Dongwon Jo. Taesu Kim, Yulhwa Kim, and Jac-Joon Kim. Mixture of scales; Memory-efficient token- adaptive binarization for large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL htwps:tt/sseett /atpps org/ ab/////b//2.... 12311.
Peijie Dong, Lujun Li, Yuedong Zhong, Dayou Du, Ruibo Fan, Yuhan Chen, Zhenheng Tang, Qiang Wang. Wei Xue, Yike Guo, and Xiaowen Chu. Sthlim: Breaking the 1-bit barrier with structured binary Ilms. In International Conference Learning Representations (ICLR), 2025,
URL https:tt//// pttttp://openreviieee wttprrviwweeeeeentneeeeeeeeeeeee aett//forumm/AddsXSSVVVBR
Vladimír Boža and Vladimír Macko. Addition is almost all you need: Compressing neural networks with double binary factorization. CoRR, abs/2505.1 1076, 2025. doi: 10.48550/arXi..2555. 1076.
URL https:tt//// //tp/v/awvvvv////////// wwwgg/abs/5555.. 11076.
Banseok Lee, Dongkyu Kim, Youngcheon You, and Youngmin Kim. Littlebit: Ultra low-bit quantiza- tion via latent factorization. Advances Neural Information Processing Systems (NeurIPS). 2025. URL /tttss:/oppnreee
net/formm/id==2aaueeeeeee
Ziang Long, Penghang Yin, and Jack Xin. Learning quantized neural nets by coarse gradient method for nonlinear classification. Research in the Mathematical Sciences, 8(3):48, 2021.
Yuma Ichikawa, Shuhei Kashiwamura, and Ayaka Sakata. High-dimensional learning dynamics of quantized models with traight-through estimator. arXiv preprint wrXhv:2510.10665 2025.
Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, and Jack Xin. Under- standing straight-through estimator in training activation quantized neural nets. arXiv preprins arXiv: 056662, 2019.
Yuma Ichikawa and Hiroaki Iwashita. Continuous parallel relaxation for finding diverse solutions in combinatorial optimization problems. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https:///penreriie..tte wwtperrrrrwe.etttooooooe.eeeedad
Yuma Ichikawa and Yamato Arai. Optimization by parallel quasi- annealing with gradient- based sampling. In The Thirteenth International Conference on Learning Representations, 2025.
URL https: /ttp:////ppereeieeee/ net//foruumIdd--BBBeee....
Haoran Sun, Katayoon Goshvadi, Azade Nova, Dale Schuurmans, and Hanjun Dai. Revisiting sampling for combinatorial optimization. In International Conference on Machine Learning, pages 32859-32874. PMLR, 2023.
Yuma Ichikawa. Controlling continuous relaxation for combinatorial optimization. Advances in Neural Information Processing Systems, 01144789--471166 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes].
Justification: The abstract and Introduction explicitly state (i) the limitation of existing layer-wise PTQ, (ii) the proposal of QEP, and (iii) the empirical and theoretical gains; all are substantiated by proofs in Appendix B and experiments in Section 6.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Section 5.3 and Section 6.1 acknowledge over-fitting risks, calibration-sel bias, and the need for layer-wise tuning of ay; computational trade-offs for large MLP blocks are also discussed.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes].
Justification: All assumptions are stated before Proposition 5.1, Theorem 5.2, etc., and full proof' are given in Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 6 and Appendix C detail the datasets, bit-widths, calibration sizes, and default values of ox; Table reports the runtimes for the quantization process.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: All datasets publicly available, and the supplementary material includes an anonymized ZIP file containing the code, execution scripts, and a README file with a list of required packages.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/ public/guides/
Stw.iideee/ooeeumiiisssssssss onPolicy) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: httpa///egiiieee//eeeetiiiiiieee
/ //nips.com
http//uddes///ee /at////ooeSSeeaeeeeeeeeee wwwssssiicciiiiiiioi for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Datasets in Section 6, calibration procedure, number of tokens, and hyper- parameters (group size, og defaults) are listed; no additional training optimiser is used because PTQ is post-training.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The impact of random seeds on QulP and QEP-QuIP evaluated using error bars and reported in Appendix D.3.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Table reports wall-clock time on single NVIDIA V100 for 7B-70B models.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics attps://neurri: http://ccc......cee.cccce.aaaaaa
Answer: [Yes]
Justification: The work only processes publicly released models and text corpora; no personal, sensitive, protected data used.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
17

---

This work focuses an algorithmic advance in model quantization; given space constraints we prioritized technical details and empirical results, full societal- impact discussion was omitted.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper does not release new generative models; it only provides a compres- sion method and therefore poses minimal additional misuse risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Every external asset (Llama-2/3. Mistral- C4, PTB, WikiText-2, ARC-E, PIQA, StoryCloze) is fully cited, publicly available, where the respective download pages state their open-source licenses.
Guidelines:
18

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA].
Justification: The paper introduces new dataset or model; only supplies an algorithmic framework.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No hww.n-uuujepeee data crowdsourcing is involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
19

---

Justification:
Not applicable, no human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality the research, declaration is required.
Answer: [NA]
Justification: An LLM was used only for writing assistance, proofreading, and minor code refactoring; played no role in the core methodology or experiments.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //tpur/eeipp//eeeeee cc/oonferrncces22255 2222)L for what should or should be described.
20

---

A Additional Related Work
Quantization error mitigation Defensive Quantization (DQ) [Lin al., 2019] mitigates accumulation by adding an orthogonality penalty to weights to reduce the correlation- drivenn amplifi- cation of quantization noise and by employing gradient-bast based quantization to position quantization levels that further suppress propagated error, thereby enhancing robustness. In contrast, the QEP module is plug-and-play component for general layer-wise quantization that explicitly modulates propagation strength under standard linear, gradient-free quantization methods, such as GPTQ, AWQ, and QuIP.
Relation to GPTAQ GPTAQ [Li et al., 2025] optimizes the same local objective but is closely related to GPTQ and therefore does not easily generalize to other layer-wise PTQ methods, such as AWQ and QuIP. In contrast, QEP adds the correction term in Eq. (4) directly to the pre-trained weights, enabling plug-and-play use with diverse PTQ algorithms and strong performance in low- bit regimes. Morcover, whereas GPTAQ offers guarantee that its local optimization reduces global quantization error, our analysis provides such guarantee. QEP also introduces a per-layer propagation-strennn parameter, ay, which mitigates the overfitting often observed with GPTQ and GPTQA.
Extreme low-bit layer-wise PTQ In the ultra-low precision regime, the layer-wise PTQ pipeline described in the main text, such as GPTQ and AWQ with standard linear quantizers, often becomes inadequate. As a result, many methods adopt alternative formalisms that introduce additional degrees of freedom beyond naive rounding to avoid catastrophic quality degradation. Representative examples include SVID-bbase 1-bit parameterizations [Xu et al., 2024b), token-adaptive mixtures of scaling factors [Jo et al., 2024], and structured sparsity designed for extreme quantization [Dong et al., 2025]. In parallel, binary-factor formats decompose each weight matrix into bit-packed sign factors with lightweight diagonal scaling, allowing inference to be largely driven by efficient 1-bit kernels [Boža and Macko, 2025, Lee et al., 2025]. these formats, optimization also more challenging: strictly binary variables and highly non-smooth objectives can render et al., 2021, Ichikawa et al., 2025, Yin et al., 2019], motivating the use of discrete optimization or robust relaxation- based solvers. Recent advances in controlled continuous relaxations and annealing- style objectives have emphasized promising techniques such as QQA and iSCO [Ichikawa and Twashita, 2025, Ichikawa and Arai, 2025, Sun et al., 2023, Ichikawa, 2024]. A natural direction for future work is to examine how these extreme low-bit approaches interact with our error-propagation perspective. For instance, sparsity-basee methods can further suppress cross-layer error growth while maintaining the strong INT2 performance observed here and yielding larger gains as precision approaches bit.
estimators brittle [Long
should investigate whether combining QEP with binary-factor
B Additional Thcoretical Results
This section presents proofs of the propositions stated in the main text along with additional theoretical analyzes. To avoid ambiguity, we will fix the following notation throughout this section:
Symbol
Deseription
X e Rdim
Calibration dataset (input activations at layer 1)
Wi Rux
Full-precision weight matrix at layer Quantized weight matrix at layer I
Wi Quixd
a)())
Activation function at layer Full-precision activations at layer (2 + 1) X 07 (W)
X1+1 = 01 (.Xi)
Quantized activations at layer (1 + 1)
6, X. X,
Quantization matrix at layer
H. = X
Empirical Hessian of a full-precision model at layer / Empirical Hessian of quantized model layer
H. X
21

---

In the following section, we assume that Hi and Wi are invertible. This assumption is standard in existing layer-wise PTQ methods, which also use these inverse matrices. To ensure numerical stability, a diagonal matrix pI, > 0 is commonly added to the Hessian when its inversion becomes numerically unstable. The subsequent analysis remains consistent and valid even when applying this stabilization procedure, which simply involves adding pl the Hessian in the following derivations.
Throughout this section, examine the first-order linear term in the weight perturbations {Er). Concretely, we define each quantity A A0) + A (1) R. where A (1) collects all terms linear in } and the remainder satisfies | = O (max, Ex (13) as max, | - 0. This matches practice: with INT8 rounding E / Will= 10
10 so quadratic terms are one order of magnitude smaller than any first-order contribution. Furthermore, The baseline PTQ and the QEP pipeline use the same quantiser configuration, hence they induce errors of the same order. We therefore write the same symbol E, for the error matrix in either scheme; any difference is at most few percent and does not affect first-order bounds.
B.1 Derivation of Proposition 5.1
This section presents detailed proof' of Proposition 5.1 stated in the main text.
Proof. First, we rewrite the residual inside the Frobenius norm by using the following relationship:
WiX: W, + Wid. Thus, the objective can be expressed as follows:
W.X: wix
(w:
W X W1.
Since the objective a strictly convex quadratic function of Wi when Hi is invertible, the stationary point is the unique minimizer. To find the minimizer Wi, we set the gradient of the expression with respect to W. equal to zero. Using standard matrix calculus, we find that the calculus for stationary point is
(W, WT X Wi8 X 0.
By defining H - the above condition can be rewritten
(Wi Wi -
Assuming Hi is invertible, we multiply both sides on the right by H. obtaining
W W. Wi H
and hence
W. W + Wio,X, H
This closed-form expression is indeed the unique minimizer of the Frobenius norm objective, thus completing the proof.
B.2 Quantization Error Accumulation
This section demonstrates that, under standard layer-wise PTQ, where each layer is quantized independently without considering downstream effects, the activation difference at the output layer, defined as d, := X,. _x,, grows exponentially with depth, to first order in the quantization noise, under mild conditions.
Proposition B.1. For each layer I 1,..., L 1, the activation error can be expressed as follows:
1-1
[-1
- Ri
II J.W. J.E.X,+O Rc++
1
max
AS1-1
where the empty product I' is defined to be the identity matrix, and E = We WE represents the weight quantization error at layer k.
22

---

Proof. Consider explicitly the activations at layer in both full-precision and quantized forms:
X [
X; 0 (
By recursively applying this relation back the first layer, we derive the activation difference o as
6, = Xi X
-0-- Xi - -Wi
WW - W - + +0 max Ej 87-11)
= 1
J1-1 - Wi -
max 1 8%-1))
+ O maxx ( 1
By explicitly expanding 81-1, obtain
_ - 2 Jr 2 O (max E E 8()
Substituting this expression into the previous equation yields
6, = E
+ J.--WW 1 -.-
+ O (max E 82-21
= - X Ji- Wi-1J1 2
+
+ +0 (max E Ei 82-2))
By recursively repeating this explicit expansion down to the first layer, obtain the fully expanded form as follows, noting 6 0:
= Σ k=1
II J.W.
JnEx +0
max
Ex
ASI-1
6=k+1
where the empty product for l,...,l- is defined as the identity matrix.
Proposition B.2. Assume each activation σ1 R + Ruxx is n-Lipschitz with respect to the Frobenius norm and satisfies (0) 0:
|[o(U)) (V)ll S millU Vlle, m> 0.
Let Xi = X; for 1 I,...,L-
L - define
X = (Wix)), X (W)X), W. W E
and 6, =x- X1, Assume Willa for all 1= 11 and set
L-1
| 1 [ l1
GL-1
II
n Wilk,
1=1
Then the final activation mismatch satisfies the explicit bound
||,2|| S ((1+r) 1)G2-1|XX|1:
Proof. Since (0) and 0; are -iiischhit
| || = ||a (W))) (0)lee S nl W.Xi|F S sa Will2 Xilee
By induction,
1-1
X <
II
n Wall
G 121
(8)
Using Lipschitz continuity again,
1 1lo0=lo,, (W,
WnlWW W.
23

---

Since W. = Wi + E, and ò = Xi X,,
WiX: Wx W,X; (Wi E E
E,X, Wid!
hence, by the triangle inequality and || S ||
101+1 - (|E + |
(9)
By definition of r. Er S 2/ Additionally, W 1l & Will + Edllz s (1 + (1+r)][Wl)) W/ Combining these with (8) in (9) yields
a Willa ( F (1 (1+r)
Define the normalized quantity
| G |F
a1 =
121
Note d1 0 because ō = 0. Dividing the previous inequality by G:l X|| (where Gt =
G1-m7 Wi (12) yields
a++ S + (1 +1+r) +r)a.
Let by = + 1. Then bi+1 I (1 + bb and bj 1; hence by S (1 (1+r therefore
a (1 + +r)
Taking = L gives Eq. (7).
Proposition B.3. Assume each activation ai R - Roix is n-Lipschitz with respect to the
Frobenius norm and satisfies (0) 0:
|o, - (V)ll S n V|
Assume moreover that is Fréchet differentiable at the full- pre-activation Y = WiX for each = 1,...,L-111 1.
Let X X X and for I...., L define
X o: (i) X = (WX W: Wi + E..
Assume W f :: 1 ,,..,, L and define
T-1
|
G = II ml Willa, r== max 1SkSL-1
[
Define the first-order component ( as follows. For E ler Wi(t) = Wi + tEE and define
Xi(t) X, (t) or (t)) Let di(t) -i Xi)... Then 8%) is defined by the derivative
6 = dt t=0
(t)
Under these assumptions,
_6 || S (T. 1)r |
(10)
In particular, if will2 < 1 + E for all I, then
(1 || S (I. - 1)r(1 + 1 IF-
(11)
Proof. Since (0) 0 and o; is LLLiscchitz
|X = ||a XX) or (0)lle S al WiXiIF S w/lWa Xills-
By induction,
|X S
7 Willz |X||F 121
(12)
24

---

Fix E [1....,- --) By assumption, 9i is Fréchet differentiable at Y = WiX, Let J: = Doy(Y) denote its Fréchet derivative. Because of is -y-Lipschitz, the operator norm Jú, induced by the Frobenius norm, satisfies
1}.
| S 7--
(13)
Indeed, for any H.
oo (H) oz (Y))
H|H [t]
|| H
lim t0
lim 60
mi
H |p-
[8]
Now consider (1) or tiable Y,, the chain rule gives
( (i(())X((1) Since matrix multiplication is smooth and or is differen-
d
x0(0)
(t)
E +
Wi
dt
1=
1=
Because 6((t) X - (t)),, we have $ $1 Xi(t) -0 1(21 Therefore,
/+1 = - W Jj (E) J((1 (()),, 8 O.
Taking Frobenius norms, using (13) and |AB S A 2 B|| p.
| || S n
Ex ||
X F + ア W | Ir.
(14)
Define a;
===1] [[.. From (14) and @1 = 0, straightforward induction yields
) The Ex 112 & Woll2
ag Σ
II
%
Apply Eq. (12) to | r and factor out GL-1:
18 |r GL-1
T-1
Σ
Exll2
|X (L- (L-1))GL-
W
which is Eq. (10). If additionally nil Will + for all I, then GL-1 $ (1 -) proving Eq. (11).
Proposition B.4. Consider the I-dimensional network (d; ni = with (12) 2. Let Wi 1+s for all = 1,..... L
with e >> O, and let the input be X = C O. Choose quantized weights
W. = Wi+ E,, E = > 0, 1 1.1
Then, for all L 2 2, the exact activation mismatch at layer L satisfies
1 = X 81, (T. ()c=0(1 6)
(15)
In particular,
C
18 2 +
(1 +E)
(16)
Proof. Since or the identity map, we have
X (1 (1++))--C X, (1 + tc)
Hence
b = C (1 (1+ +1 + (1 (1+ +))-111
Apply the mean value theorem to f(t) = +1-11 on the interval [1 + E, + E + cgl: there exists
E E -,, + Cg) such that
(1 + cg)
(g) L-1 (1 + --11 (E) = (L- 1) -2g-
Since & 2 1 + E, we obtain
82 2 (L (L- 1) (I-1)c
2-2,
which proves Eq. (15). Finally, Eq. (16) follows from (1 e) (1 + +e) and 1 for L > 2.
D
25

---

B.3 Derivation of Theorem 5.2 and Corollary 5,4
This section presents a rigorous statement and proof of Theorem 5.2 and Corollary 5.4. We first formally restate Theorem 5.2 below.
Theorem B.5. Consider an L-layer network
=, X1-1 = oi(())))) 1=1,...,, L.
Assume each ot n-Lipschitz with respect to || || and satisfies oi(0) Let the quantized forward pass be
X1= X, X 0; (W.Xi).
Define the activation mismatch 6i = X: Xi. Fix any matrices E1 and set
W BASE = W + E..
For QEP, define each! 1
= =; a H. aq [0.1].
and set
WORT = WWi(ar) + E
Define the per-layer pre-activation residuals
W M (BASE, QEP),
the global Lipschitz upper bound the output mismatch
MM Σ
II
Yell TellWalla)
k=1
10=4+1
Then for every choice of {ai) C 10,
UQEP S BASE
Consequently.
QEP || S. SUEEP < BASE BASE
|F S. BASE
Proof. Fix method M E (BASE, QEP). By Lipschitz continuity,
1 |r = (W; an W. X S 7 WiX, WM M m/ MM
Iterating this inequality through the remaining layers yields
L
II
7 Woll
R || UMM
k=1
18=++1
remains show QEQ < BASE, We prove a stronger per-layer inequality:
R QEP
|
R. BASE F, Vl.
(17)
Fix / and write X = X QFP
and 8 = X1 X,. Define the orthogonal projection
P =
-
which satisfies P? Pt and P.T = P,.
By the construction of (az), have the exact identity
(i WiX + Wio.P.
26

---

Therefore, using W
(ai) + E,,
R
W,X, (Wi E,)X; Wi(X; 6;) Wi X1 E,X,
Wis: a E,X, W Pi) E,X,
Consider the BASE residual at the same layer evaluated on the same X R
WiX: (W, + E Wid: E,
Hence
R R BRSSE aj Wi Since Pt an orthogonal projection and S an 1, Lemma B.8 implies
|| - 0z Pi)lI - |
Therefore, by the triangle inequality,
| |E = [I oz X & BRASE
Finally, since X BASE is the BASE activation produced by the BASE recursion, REBSSE is exactly RBASE evaluated at XTASE, Thus, taking X; X in the above inequality yields Eq. (17). Summing with nonnegative weights (II 7s W. 112) yields Q/QE S L/BESE, thus completing the proof.
We further demonstrate that the final quantization error decreases monotonically each propagation strength parameter on approaches 1.
Corollary B.6. Fix the layer index I {1,...; L.} and assume that Hi X,Xi is invertible. Let
the activation mismatch be 6 = X X,. Define the orthogonal projection
P For [0, 1), define the QEP corrected weight the continuous domain as
(X ) X m
Wi' = W a H
and let Wi(a) Wi((a) + E, be for some fixed matrix Et. Then, for any 0 < a' S a S 1,
| - a.)i)l S [[ - a' P.)
(18)
Moreover, the pre-activation residual satisfies the exact identity
W,X, - Wi Wid,(I - aP). E,X;,
(19)
and hence the following upper bound is also monotone in a::
WiX: Wi
( Exlla Xilrr.
(20)
Proof. First, Pt is an orthogonal projection. Indeed,
P =
X Pt:
and
p
XiX
X,X,
XiX ) 1 = Pt.
Hi.
Hence, Lemma B.8 applies with Z = Wid: and P = Pt, yielding
[ - aP))lF [I a' P,)
which proves Eq. (18).
27

---

Next, using the definitions of W (a) and H, = -X(x,))
(a) = W,X; + aiii (XIX)) - =x +aWipp. Therefore, since Wi(a) (a) Ex and X; = =XI+
WiX; - Wi = Wi(Xi + 6:) - (Wi((a) + E Wi6i a. - = -
PP) -
E,X,,
which is (19). Finally, (20) follows from the triangle inequality and || Xillr s
| Xill..
LI
B.4 Relationship of QEP Correction and Ridge Regularization
We formally establish a rigorous mathematical connection between the Quantization Error Propagation (QEP) correction parameter oj and the ridge regularization parameter X. We show that tuning the QEP parameter a is equivalent to adjusting the strength of ridge regularization with parameter X. We prove the monotone inverse relationship between these two parameters.
Proposition B.7. The QEP update with mixing factor ag [0,1] is
(an) W1(I + ad, H, 1
The unique minimizer of the ridge objective
min I(Wi), S(W)) 1|, Wi Ar WT WIl 220
Wi
equals
WT() - (1 (H -
(21)
Let the positive definite matrices be
R(A:) (H + +
Then
On S 02 =(a)) G(22), d A = R(A1) = R()2). and the scalar mapping as follows:
1 ((H 1 a(A)
Σ
K 7/8
d
A
with 2 ... 2 7/d > 0 the eigenvalues of Hi, is strictly decreasing, satisfies a(0) = and limx a(A)= = O, and obeys
TrH(G(a(A)) = TTHH,
Thus, decreasing Afrom too to 0 corresponds to increasing on from 0 to
Proof. A standard differrnnial identity A| 2A gives
W (Wi) 2 W. - - + +2A(W,
Wi).
Setting this gradient 0 yields
W + WXXXX +XI),
and right multiplication by inverse of Hi + AI produces Eq. (21). Convexity of f ensures uniqueness.
Diagonalise H, = UTUT with I = diag(?1.... ). Then G(au) = Ua--'UT has eigenval- ues ez/Ti, which increase strictly with on, while R(Ar) = UT XT) A - has eigenvalues 1/ A, which decrease with Ar. which means Loewner relations follow directly.
28

---

Furthermore, the following equation holds:
d1
1 (T((HiR(A)))
1
%
a(A)
di
-
di ==1
vi +
Each summand has derivative
0 %
ax 7s +
(74 + X)
which means (A < 2A O. Thus, a(.) is strictly decreasing on [0, oo). One has
7/s
lim %/4 A 1, lim
0. 7/4
a(-) is strictly decreasing from 0 and smooth o0) Because is continous, strictly decreasing, it is a bijection from [0, +20) onto (0, 1]. By construction
IrH()(((A)) = R(H,
Thus, decreasing À from +00 to 0 corresponds to increasing or from 0 to 1.
B.5 Technical Lemma
Lemma B.8. Let Z Rmxx be arbitrary, and let P E Ruxa be an orthogonal projection, i.e.
p== and PT = P. For every pair 0 S a' S S 1,
| - aaP) S [(I a' [(P)]l < 1
(22)
Proof. Write f(a) = || aP) Because PT_ and p2 P.
f(a) Tr [[I a z(I -PP]] || 2a(1 a) Tr(Z ZP).
120
Thus, = a)t 0 on [0. 1] indicates that f(a) is non-increasing. Taking square roots
yields the first incquality Eq. (22). Setting a' yields the second inequality: | aP) S |Z|r..
C Additional Implementation Details
C.1 Damping for Hessian
A standard numerical issue in PTQ arises when the Hessian matrix Hi is ill-conditioned or singular, rendering inversion unstable or undefined. Following GPTQ [Frantar et al., 2022], we resolve this issue by employing a damping strategy that adds small scalar value À to the diagonal elements of H, to ensure positive definiteness. In our implementation, set À to the mean of the diagonal elements of Hi, providing a straightforward yet effective method stabilize the inversion process.
D Additional Experiments
D.1 Additional Perplexity Results
Due to space constraints, the main text reports perplexity results solely for the WikiText-2 dataset. Here, we provide additional results for PTB (Table 6) and C4 (Table 7), along with extended results for WikiText-2 (Table 5). These supplementary results further validate that QEP consistently enhances PTQ performance, particularly in low-bit quantization scenarios.
29

---

Perplexity on WilTexx-22
INT4 Quantization
INT3 Quantization
INT2 Quantization
30
QEI-QUIP
30
QuiP
20
10
10
138
70B
136
708
78
138
70B
Model Size
Model Size
Model Size
INT4 Quantization
INT3 Quantization
INT2 Quantization
Average accuracy
0.7
0.70
0.70
0.655
0.65
0.6
0.60
0.60
0.5
78
138 Model Size
708
78
138
708
78
138 Model Size
708
Model Size
Figure 3: Results averaged over 5 random seeds comparing QulP with and without QEP across different quantization levels. Each subplot shows results for INT4, INT3, and INT2 quantization, respectively. with the horizontal axis indicating model size (7B, 13B, 70B). The top row reports perplexity on WikiText-2 (lower better), while the bottom row shows the average of normalized accuracy scores on ARC (casy), PIQA, and StoryCloze benchmarks (higher is better), representing generalization capability. Error bars represent the standard error of the mean (SEM). Models using QEP-QuIP consistently outperform or match the performance of baseline QulP, especially under more aggressive quantization (INT3 and INT2).
D.2 Detailed Accuracy Results for Individual Tasks
Due space limitations, the main text reports only the average accuracy across three tasks. Here, we provide task-specific accuracies for PIQA (Table 8), StoryCloze (Table 9), and ARC-Easy (Table 10), further confirming that QEP consistently improves layer-wise PTQ.
D.3 Stability of QulP Results Across Random Seeds
We assess the stability of QulP-only experiments by averaging five independent runs per configuration. Model sizes, quantization levels, and benchmarks align with the main Experiments section. Figure 3 plots QulP with or without QEP at three quantization levels. Each marker the mean of five seeds, and the error bars show the standard error of the mean. The top row gives perplexity on WikiText 2; the bottom row reports mean normalized accuracy on ARC easy, PIQA, and StoryCloze. Seed-to-seed variation is small and does change the main conclusions. QEP-QuIP keeps its advantage, especially at INT3 and INT2. The main text lists the best seed per configuration for consistency with past work. This appendix confirms that the gains are not seed-specine but robust and reproducible, supporting using QEP.
D.4 Comparison with OmniQuant Baseline
Table 11: WikiText-2 perplexity for LLaMA-
2-7B at different bit-widths. NaN denotes di- vergence.
For completeness, we compare QEP-enhanced layer- wise PTQ with block-wise OmniQuant [Shao et al., 2023] LLaMA-2-7B using WikiText-2 perplex- ity: lower values indicate better performance. As shown in Table 11, QuIP+QEP achieves the low- est perplexity at INT4/INT3 and remains stable at INT2, whereas OmniQuant diverges. These findings align with recent PTQ benchmarks that indicate Om- niQuant's underperformance relative layer-wise PTQ [Zhao et al., 2025].
Method
INT4
INT3
INT2
RTN+QEP GPTQ+QEP
6.017
17.309
97153.266
5.933
7.898
7214.328
AWQ+QEP
5.756
11.131
229888.406
QuiP+QEP
5.753
6.154
11.972
OmniQuant
5.880
7.065
NaN
30

---

Table 5: Perplexities ()) on WikiText-2 for Llama-2 (7B, 13B, 70B) under eight quantization settings.
Bits
Method
QEP
Llama-2-7B
Llama-2-70B Llama-2-13B
5.726
4.984
3.463
RTN
5.687
4.966
3,431
5.698
4.987
3.419
1NT4
GPTQ
5.609
4.969
3,416
X
5.599
4.987
3.408
AWQ
5,580
4.969
3.404
X
6.116
5.206
3.672
RTN
6.017
5.165
3.621
6.083
5.167
3.594
INT4
GPTQ
5.933
5.127
3,576
5.831
5.064
3.484
AWQ
5.756
5.041
3.479
6.662
5.518
3.978
RTN
6.330
5,412
3.882
6.411
5,459
3.880
INT3g128
GPTQ
6.160
5,358
3,838
6.247
5.315
3.740
AWQ
6.108
5,295
3.724
539.866
10,688
7.530
RTN
17.309
7.458
5.648
10.881
6.632
4.860
INT3
GPTQ
7.898
6.245
4.102
15.299
6.448
4.362
AWQ
11.131
6.092
4.103
90.692
10.563
6.802
RTN
12.249
7.920
5,869
12.023
8,394
5.621
INT2g32
GPTQ
9.245
7.362
5.445
15887.200
106933.227
63663,707
AWQ
51.874
80654.797
37096.516
431,595
26,220
10.312
RTN
19.371
9.917
6.992
278.302
11.584
6.546
INT2g64
GPTQ
14.737
8.685
6.030
217111.860
121737.148
71703.781
AWQ
241136.594
126944.578
74227.539
4270.828
122.063
27.268
RTN
35.291
12.779
8.799
43.915
16.653
8.123
INT2g128
GPTQ
17.886
19.952
6.825
222344,250
122795.898
72446.680
AWQ
247751,203
126813. 126813.172
74192.570
17783.918
51152.832
26077.17
RTN
97153.266
61158.555
26063.672
X
1305 1,969
1301.395
107.458
INT2
GPTQ
7214.328
2782.353
52.472
199448.797
93036.517
81834.344
AWQ
229888.406
74735.836
88684.156
31

---

Table 6: Perplexities (1) on PTB for Llama-2 (7B, 13B, 70B) under eight quantization settings. "N/A" denotes numerical overflow (NaN).
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
X
61.750
53.835
24.146
RTN
47.798
49.503
24.604
X
N/A
51.133
24.101
INT4g128
GPTQ
N/A
50.072
24.243
43.894
53.863
24.525
AWQ
40.445
55.345
24.554
82.641
60.749
23.545
RTN
50.168
53.117
23,346
N/A
53.561
24.720
INT4
GPTQ
124291.96
53.537
24.149
60.261
56.152
25.542
AWQ
46.937
57.445
24.411
55.467
64.638
23.586
RTN
48.576
54,866
24.776
N/A
57.079
24.091
1283g128
GPTQ
N/A
62.083
24.092
X
64.932
57.273
24.668
AWQ
52.356
61.479
26.309
X
37167.801
294.802
64.002
RTN
5514.820
113.856
34.212
X
44807.99 926
106.715
27.839
INT3
GPTQ
N/A
81.117
27.469
130.308
121.698
26.887
AWQ
81.606
93.260
25.592
20280.412
262.244
63.428
RTN
1685.683
96.913
36.677
18292.635
152.169
29.163
INT2g32
GPTQ
N/A
110.507
30.465
47850.137
60977.195
48520.398
AWQ
3741.642
47591.414
20185.246
9252.538
551.510
153.528
RTN
1096.720
158,306
42.991
N/A
275.949
37.024
INT2g64
GPTQ
N/A
187.477
37.384
202939.484
113584.867
79866.031
AWQ
220728.234
117658.867
82598.511
9685.755
1213.282
767.896
RTN
4462.478
207.651
63.806
10694.694
395.689
56.685
I2T22128
GPTQ
N/A
325.407
45.569
X
202164.484
113784.242
80543.727
AWQ
222388.375
117059.742
82493.251
31824.279
42619.883
26063.672
RTN
10824.680
55286.305
26077.172
X
N/A
3868.426
2438.034
INT2
GPTQ
N/A
3850.578
4050.844
183984.766
87673.695
90442.352
AWQ
198744.750
62160.063
91939.883
32

---

Table 7: Perplexities (4) on C4 for Llama-2 (7B, 13B, 70B) under eight quantization settings.
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
7.584
6.869
5.826
RTN
V
7,513
6.839
5.786
7.522
6.860
5.778
INT4g128
GPTQ
7.421
6,828
5.770
X
7,443
6.840
5.772
AWQ
7.416
6.829
5.767
8.165
7.146
6.012
RTN
7.945
7.067
5.947
7.866
7.069
5.905
GPTQ
7.719
6.998
5.880
7.721
6.962
5.842
AWQ
7.634
6.932
5.828
8.977
7.582
6.266
RTN
8,510
7.402
6.150
8.502
7.463
6.105
INT3g128
GPTQ
8.185
7.316
6.072
8.300
7.310
6.036
AWQ
8.105
7.264
6.019
524.279
13.883
10.886
RTN
21.436
10.284
8.202
11,780
8.826
7.067
INT3
GPTQ
9.950
8,429
6.869
17.418
9.049
6.631
AWQ
13.934
8.257
6.353
225.440
13.879
9.720
RTN
16.148
10.561
8,459
14.365
10.719
7.932
INT2g32
GPTQ
11.839
9.685
7.717
9028.133
76591.883
57596.215
AWQ
51.811
49645.738
33026.816
553.766
30.445
15.155
RTN
22.089
12.762
9.850
20,860
13.394
8.981
INT2g64
GPTQ
14.084
11.039
8.508
X
164447.442
95241.625
64913.477
AWQ
181582.719
98917.820
67203.359
1811
131.665
47.878
RTN
34.022
15.398
12.081
X
33.370
18.008
10.535
INT2g 128
GPTQ
18.184
12.704
9.433
168465.266
95617.305
65646.594
AWQ
187329.625
98457.031
67248.492
28258.385
52642.387
24912.074
RTN
108424.680
71050.250
29042.623
3048.671
299.684
56.719
INT2
GPTQ
276.638
629.527
30.874
156266.797
81233.602
73251.945
AWQ
177576.750
640988.504
75607.21
33

---

Table 8: Accuracy (1)
PIQA for Llama-2 (7B, 13B, 70B) under eight quantization settings.
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
0.773
0.792
0.804
RTN
V
0.773
0.790
0.806
0.770
0.789
0.807
INT4g128
GPTQ
0.771
0.792
0.806
X
0.768
0.790
0.807
AWQ
0.764
0.791
0.810
0.763
0.789
0.811
RTN
0.767
0.788
0.812
0.755
0.789
0.804
INT4
GPTQ
0.761
0.787
0.811
0.760
0.789
0.807
AWQ
0.763
0.784
0.814
0.757
0.770
0.793
RTN
0.761
0.779
0.806
0.758
0.778
0.806
INT3g128
GPTQ
0.764
0.782
0.807
0.760
0.780
0.805
AWQ
0.765
0.780
0.805
0.563
0.705
0.724
RTN
0.677
0.752
0.764
0.720
0.757
0.783
INT3
GPTQ
0.745
0.770
0.791
0.647
0.760
0.787
AWQ
0.725
0.770
0.801
0.588
0.696
0.760
RTN
0.693
0.735
0.771
0.690
0.732
0.772
INT2g32
GPTQ
0.714
0.748
0.776
0.568
0.505
0.503
AWQ
0.702
0.514
0.501
0.597
0.614
0.714
RTN
0.676
0.710
0.748
0.647
0.705
0.745
INT2g64
GPTQ
0.677
0.713
0.765
0.502
0.506
0.502
AWQ
0.702
0.506
0.504
0,511
0.566
0.635
RTN
0.652
0.678
0.721
0.581
0.639
0.715
INT2g 128
GPTQ
0.659
0.683
0.747
0.501
0.505
0.503
AWQ
0.501
0.507
0.503
0.509
0.493
0.499
RTN
0.510
0.506
0.510
0.500
0.509
0.511
INT2
GPTQ
0.493
0.507
0.544
0.507
0.504
0.502
AWQ
0.505
0.504
0.504
34

---

Table 9: Accuracy (f) on StoryCloze for Llama-2 (7B, 13B, 70B) under eight quantization settings.
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
0.765
0.785
0.791
RTN
V
0.770
0.788
0.794
0.768
0.784
0.793
INT4g128
GPTQ
0.771
0.789
0.798
X
0.777
0.782
0.792
AWQ
0.777
0.785
0.798
0.756
0.777
0.796
RTN
0.763
0.777
0.798
0.765
0.776
0.794
GPTQ
0.766
0.775
0.792
0.760
0.774
0.789
AWQ
0.766
0.777
0.794
0.749
0.766
0.790
RTN
0.756
0.773
0.789
0.763
0.776
0.793
INT3g128
GPTQ
0.759
0.770
0.796
0.761
0.767
0.795
AWQ
0.761
0.782
0.795
0.546
0.669
0.738
RTN
0.672
0.728
0.776
0.722
0.752
0.780
INT3
GPTQ
0.745
0.766
0.782
0.689
0.767
0.787
AWQ
0.702
0.764
0.782
0.645
0.668
0.745
RTN
0.704
0.721
0.776
0.758
0.715
0.724
INT2g32
GPTQ
0.763
0.748
0.766
0.660
0.511
0.516
AWQ
0.703
0.570
0.569
0.607
0.617
0.718
RTN
0.670
0.696
0.766
0.654
0.686
0.756
INT2g64
GPTQ
0.712
0.720
0.758
0.476
0.479
0.476
AWQ
0.474
0.479
0.475
0,509
0.577
0.647
RTN
0.651
0.677
0.741
0.588
0.634
0.724
INT2g 128
GPTQ
0.649
0.690
0.753
0.475
0.478
0.476
AWQ
0.475
0.478
0.476
0.468
0.491
0.482
RTN
0.488
0.487
0.482
0.485
0.501
0.539
INT2
GPTQ
0.514
0.513
0.589
0.489
0.478
0.475
AWQ
0.482
0.476
0.477
35

---

Table 10: Accuracy (1) ARC-Easy for Llama-2 (7B, 13B, 70B) under eight quantization settings.
Bits
Method
QEP
Llama-2-7B
Llama-2-13B
Llama-2-70B
0.554
0,367
0.596
RTN
V
0.540
0.572
0.596
0.531
0,573
0,586
INT4g128
GPTQ
0.521
0.579
0.592
X
0.537
0.577
0.585
AWQ
0.526
0,580
0.592
0.521
0.582
0.590
RTN
0.524
0.574
0.593
0.525
0.575
0.594
GPTQ
0.512
0,570
0.589
0.529
0.572
0.580
AWQ
0.532
0.577
0.591
0.528
0.569
0.575
RTN
0.517
0.556
0.572
0.521
0.568
0.580
INT3g128
GPTQ
0,515
0,568
0,569
0.534
0.561
0.597
AWQ
0.527
0.561
0.592
X
0.322
0.450
0.459
RTN
0,391
0.485
0,541
0.468
0.514
0.550
INT3
GPTQ
0.474
0.520
0.551
0.416
0.539
0.588
AWQ
0.452
0.540
0.602
0.339
0.445
0.533
RTN
0.426
0.474
0.557
0.421
0.481
0.506
INT2g32
GPTQ
0.441
0.486
0.547
0.352
0.272
0.263
AWQ
0.449
0.280
0.263
0.332
0.371
0.467
RTN
0.390
0.430
0.557
0.377
0.455
0.485
INT2g64
GPTQ
0.404
0.458
0.548
0.266
0.270
0.262
AWQ
0.265
0.270
0.263
0.269
0.253
0.395
RTN
0.376
0.407
0.479
0.338
0.383
0.443
INT2g 128
GPTQ
0.367
0.418
0.508
0.266
0.269
0.260
AWQ
0.265
0.269
0.261
0.265
0.253
0.263
RTN
0.262
0.264
0.261
0.263
0.256
0.257
INT2
GPTQ
0.272
0.265
0.281
0.267
0.270
0.262
AWQ
0.262
0.270
0.261
36


Reviews:
[{"review_id": "FmnZZ8lmwa", "reviewer": "Reviewer_jKGZ", "summary": "This paper proposes a Quantization Error Propagation training method aiming to alleviate the performance degradation problem of large language models caused by layer-by-layer error accumulation during the quantization process. The authors model the propagation path of the quantization error and control the cumulative effect of the error by introducing a specially designed optimization objective in training. The paper includes a theoretical explanation for the effectiveness of QEP, which is a commendable attempt. Nevertheless, the paper is providing solid motivation and the necessity of some theoretical arguments, and the comprehensiveness of the experimental comparison.", "questions": "Can you add a comparative discussion with related work, such as \"Defensive Quantization\" to clarify the uniqueness and innovation of your work?\n\n2. Please elaborate on the theoretical contribution of Theorem 5.2. What profound properties does it reveal that cannot be directly seen from the optimization definition of Eq (3)?\n\n3. How do you view the potential effect of QEP on the current more advanced SOTA quantization methods?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The core contribution of this paper is that QEP is proposed as a plug-and-play module. The experimental part (as shown in Tables 1 and 2) clearly shows that it can be combined with a variety of existing quantization methods (such as RTN, GPTQ, etc.) and consistently reduces the perplexity of the model on multiple benchmarks. This versatility is very valuable. QEP mainly plays a role in the training or calibration phase. Once the model is quantized, there is almost no additional computational load introduced in the inference phase, which is crucial for LLM deployment scenarios that pursue extreme efficiency.\n\n2. The authors try to explain theoretically \"why controlling error propagation is effective\". By establishing a mathematical model of quantization error propagation and deriving the corresponding optimization strategy, the authors provide a more structured perspective on the problem, which helps inspire more fundamental research in the future.\n\nWeaknesses:\n\n1. One of the main weaknesses of this paper is the lack of robust motivation. The motivation of this paper is to control the accumulation and amplification effect of errors in the quantization process, which has strong similarities with the earlier work \"Defensive Quantization: When Efficiency Meets Robustness\". Although the latter focuses on adversarial robustness rather than general quantization performance, its core idea is also to suppress the propagation of perturbations (i.e., errors) by constraining the network. Given this ideological connection, the author must discuss \"Defensive Quantization\" in the introduction or related work section, and clearly explain the similarities, differences, and uniqueness of your QEP method in motivation, technical implementation, and target scenarios.\n\n2. Theorem 5.2 is also confusing. The author has defined $\\theta_{QEP} as the optimal parameter that minimizes the specific quantization error propagation loss function in Eq (3). The inequality to be proved in Theorem 5.2 seems to be a direct or even equivalent statement of the optimal solution to the optimization problem. In other words, when \\theta_{QEP}$ is already the solution to Eq (3), the conclusion of Theorem 5.2 seems to be trivial. I hope the author can further clarify the contribution and insights of the theorem if anything is missing.\n\n3. The author verified the effectiveness of QEP on multiple quantization methods, which proves its universality. But a key issue is that the baseline method you selected (such as RTN, GPTQ, etc.) is not the most powerful SOTA method in the current LLM quantization field. In recent years, many more advanced quantization schemes have emerged (such as DB-LLM for 2-bit and Omni for 2-4-bit). It is recommended that authors conduct experiments on at least one or two currently recognized SOTA quantization methods to verify whether QEP can still bring significant benefits.", "was_revised": true, "final_justification": "Due to the rebuttal I will go a bit higher with the scores", "rebuttal": "We thank the reviewer for recognizing the strengths of our work, particularly that **QEP functions as a *plug-and-play module* compatible with various layer-wise quantization schemes**, and for highlighting several other strong points. Below, we respond to each of the concerns and questions you raised.\n\n## **Weakness and Question 1**\n\nAs you correctly pointed out, our paper should emphasize the motivation for controlling the quantization error propagation.\nIn the camera-ready version, we will make this motivation explicit in the Introduction section of the main text.\n\nWe sincerely appreciate your recommendation of the additional reference, \"*Defensive Quantization: When Efficiency Meets Robustness.*''  As you pointed out, Defensive Quantization (DQ) and our method, QEP, share conceptual similarities, as both aim to control the propagation of quantization error. However, their target scenarios and technical implementations differ significantly, so we believe the originality and contribution of our method remain substantial. In the following, we explain these details:\n\n- **Target Scenarios**: As you noted, DQ is motivated by enhancing the robustness of quantized models against adversarial attacks. In contrast, our method aims to enhance a layer-wise quantization approach as a *plug-and-play module.* By adjusting the propagation strength, we aim to improve quantization performance even in low-bit regimes.\n- **Technical Implementation**: DQ and QEP take fundamentally different approaches to controlling quantization error accumulation. DQ introduces an orthogonality penalty to the weights and employs gradient-based quantization to mitigate the accumulation of quantization error, thereby improving adversarial robustness. In contrast, QEP does not use a penalty-based gradient quantization scheme. Instead, it explicitly controls the propagation using the formulations described in Equations (5) and (6) of the main text. Furthermore, QEP performs quantization using linear quantization as in GPTQ, AWQ, and QuIP, without relying on gradients. \n\nIn summary, although the two approaches are conceptually related, **they differ in purpose, target scenarios, and technical implementation**. We will explicitly discuss their connection in the camera-ready version of the paper.\n\n## **Weakness and Question 2**\n\nThank you for your comments on Theorem 5.2. Some portions of your Markdown appear to be garbled, so we are not entirely sure what you were asking. Could you kindly clarify your question and indicate precisely which part of Theorem 5.2 you consider to be trivial?\n\nTo address your question, we briefly explain the meaning of Theorem 5.2 and why it is included in the main text. \n**Theorem 5.2 provides a rigorous guarantee that solving the layer-wise *local* optimization problem defined in Equation (3) of the main text reduces the final quantization error.** Although, as you pointed out, it may seem intuitive that optimizing Equation (3) reduces the global quantization error, we believe it is still important to prove this intuition formally. \nAs shown in Appendix A (Additional Theoretical Results), the proof requires the use of properties of projection matrices and involves a nontrivial mathematical derivation. Therefore, we believe that it is not immediately obvious that local optimization of Equation (3) leads to a reduction in global quantization error.\n\nMoreover, **we do not believe that such a mathematical guarantee can be considered a weakness.** Theorem 5.2 can be interpreted as showing that the local optimization approach reduces the global quantization error compared to previous layer-wise methods, thereby enhancing the reliability of the technique. However, if it is still considered less central to the paper, we are open to relocating the results to the Appendix.\n\n## **Weakness and Question 3**\n\nAs noted in Section 6 (Experiments, Lines 243–248), **our primary objective is to improve the performance of layer-wise PTQ**. Therefore, we consider benchmarking QEP against RTN, GPTQ, AWQ, and QuIP to be sufficient.\nWe outline the key reasons for this choice below.\nRecent literature (see Section 2, Related Work, and standard PTQ benchmark papers [1]) classifies layer-wise PTQ methods into three principal categories: GPTQ, AWQ, and QuIP. These approaches are among the most widely adopted and practically effective for layer-wise, backpropagation-free, linear quantization.\nMost SOTA methods build directly on these foundational techniques, seeking further improvements by incorporating additional computational strategies, as described below:\n\n- Block-wise quantization following warm-up through layer-wise PTQ.\n- Gradient-based fine-tuning of continuous quantization parameters.\n- Advanced vector quantization replacing standard linear quantizers.\n\nFor instance, the high-performing SOTA method QuIP\\# [2] begins with standard QuIP layer-wise quantization and subsequently applies block-wise refinements via backpropagation.\nSince such methods are built upon a layer-wise foundation, we believe it is essential to benchmark QEP against the three core layer-wise PTQ approaches: GPTQ, AWQ, and QuIP.\nEvaluating every derivative method is beyond the scope of this paper, as our focus is on enhancing these core layer-wise PTQ techniques.\nThat said, we agree that evaluating QEP on prominent extensions (e.g., QuIP\\#) would be valuable and view this as a promising direction for future work. If there are specific SOTA methods you believe warrant inclusion, we are open to conducting additional experiments during the discussion or camera-ready phase.\n\n- [1] Zhao Jiaqi, et al., *\"Benchmarking post-training quantization in llms: Comprehensive taxonomy, unified evaluation, and comparative analysis.\"*, arXiv preprint arXiv:2502.13178 (2025).\n- [2] Albert Tseng, et al., \"QuIP\\#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks\", Forty-first International Conference on Machine Learning, 2024.\n\n\n## **Closing**\n\nIn summary, regarding **Weakness 1**, we clarified the relationship between the related works you cited and our contributions. For **Weakness 2**, we explained the significance of Theorem 5.2 and why it should not be viewed as a weakness. Regarding **Weakness 3**, we justified our choice of benchmark settings as appropriate for demonstrating the effectiveness of our proposed method, which primarily aims to improve core layer-wise PTQ techniques.\nWe hope that, in light of these clarifications, you will reconsider your evaluation."}, {"review_id": "WAJFRa3gMr", "reviewer": "Reviewer_pLsN", "summary": "This work presented analytical results on quantization error propagation along network depth, and proposes a new formulation of local objective for PTQ optimization.  It also introduced a regularizer that interpolates toward the traditional local loss, to prevent overfitting.  Experiments of GPTQ, AWQ and QuIP that minimized the proposed objectives were presented.", "questions": "Major: \n* A recent, concurrent work, GPTAQ (Li et al. 2025, arXiv:2504.02692) proposed the same local objectives, though they only presented the application in the case of GPTQ, whereas this work combined it with multiple methods.  \n\nMinor: \n* Though the proposed regularizer $\\alpha$ might be useful, it introduced substantial complexity into hyperparameter tuning.  Also, could you demonstrate with a concrete example the _overfitting_ phenomenon and how certain $\\alpha$ schedules mitigated the overfitting?   \n* How does QEP interact with other orthogonal techniques such as rotationally invariant transformers for quantization?", "limitations": "Yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n+ The main idea is clearly presented.  \n+ The experiments and ablations were generally comprehensive. \n\nWeaknesses: \n- Originality of the proposal is undermined by similar concurrent work.  \n- The proposed regularizer introduced further complexity in hyperparameter tuning in practice.", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's positive remarks on the **comprehensiveness of our numerical experiments**. Below, we address each weakness and question in turn.\n\n## **Major Weakness: Relation to GPTAQ**\n\nWe appreciate the reviewer for highlighting the recent GPTAQ (Li et al. 2025, *arXiv:2504.02692*) . \nAlthough GPTAQ and our method share the same core local optimization objective, **GPTAQ approach is specifically designed for GPTQ and does not generalize to other layer-wise PTQ methods, such as AWQ or QuIP.**\nWe elaborate on this distinction in the following.\nMoreover, our method offers **additional theoretical guarantees** and **control over quantization error propagation strength**, as detailed below.\n\n- **Main Difference**: GPTAQ is an algorithm specifically tailored to GPTQ. **It is tightly integrated with GPTQ: its pseudo-code (Algorithm 1, highlighted in blue in GPTAQ paper) inserts the compensation term directly into GPTQ's internal routine**. Because these steps rely on GPTQ-specific structures, applying GPTAQ directly to other layer-wise PTQ methods such as AWQ or QuIP is non-trivial. In contrast, our method is based on the reformulation presented in Proposition 5.1. **It adds a correction term from Equation. (6) to the pre-trained weights, enabling *seamless* integration with any layer-wise PTQ algorithm.**  This versatility allows us to achieve, for example, 2-bit quantization using QuIP+QEP, an outcome that GPTAQ cannot produce.\n- **Theoretical Guarantee**: While GPTAQ does not guarantee that minimizing its local objective reduces the global quantization error, Theorem 5.2 in our work provides precisely this guarantee, offering a reliable and theoretically grounded foundation for practical use.\n- **Propagation Strength $\\alpha\\_{l}$**: **GPTAQ does not address the overfitting commonly observed in GPTQ.** In contrast, our generalized formulation introduces the parameter $\\alpha\\_{l}$ in Eq. (6), which controls the propagation strength of quantization error and effectively mitigates overfitting. As demonstrated in the table below for LLaMA‑2‑7B, 70B (INT3), setting $\\alpha\\_{l} = 1$ (equivalent to GPTAQ) leads to overfitting, while $\\alpha\\_{l} = 0$ results in underfitting.\n\n| Method               | $\\alpha\\_l$  | Llama2 7B (INT3, PPL) | Llama2 70B  (INT3, PPL) |\n|----------------------|--------:|-----------:|-----------:|\n| GPTQ (baseline)      |    0.0    | 10.881  | 4.860 |\n| **GPTQ + QEP**          |    0.5   | **7.898**  | **4.102** |\n| GPTQ + QEP           |      1.0  |    8.169 | 4.771 |\n\nThese distinctions underscore the unique and meaningful contributions of our QEP. \nWe will elaborate on these similarities and differences in the Related Work section of the camera-ready version.\n\n## **Minor Question 1**\n\nWe are grateful for your insightful comment highlighting the importance of $\\\\{\\alpha\\_l \\\\}_{l}$.\nTo begin, we provide a concrete example that demonstrates how $\\alpha\\_{l}$ mitigates overfitting:\n\n| Method               | $\\alpha_l$ (down\\_proj) | $\\alpha_l$ (others)  | Llama2 70B  (INT3, PPL) |\n|----------------------|--------:|--------:|-----------:|\n| GPTQ (baseline)      |    0.0    | 0.0   | 4.860 |\n| **GPTQ + QEP**      | 0.0    |    0.5    | **4.102** |\n| GPTQ + QEP      | 0.5    |    0.5    | 4.743 |\n| GPTQ + QEP           |      1.0 | 1.0  |   4.771 |\n\nWhen $\\alpha\\_{l} = 1$ (equivalent to GPTAQ), the large MLP matrix is prone to overfitting, which degrades generalization performance. Reducing $\\alpha\\_{l}$ alleviates this problem. Moreover, setting $\\alpha\\_{l}=0$ for large MLP layers (Lines 232–236) allows us to omit the weight correction defined in Equation (6) of the main text, thereby reducing memory usage and runtime, which is a crucial benefit when computational resources are limited.\n\nAs you correctly noted, hyperparameter tuning remains a concern. **Nevertheless, as the results indicate, using a representative value of $\\alpha\\_{l} = 1/2$, even without extensive tuning, yields robust improvements in the performance of GPTQ, AWQ, and QuIP.** Furthermore, even better results may be achieved through layer-wise tuning of $\\alpha\\_{l}$. In such cases, one can leverage the lightweight nature of layer-wise PTQ to determine $\\alpha\\_{l}$ using techniques such as cross-validation.\n\n## **Minor Question 2**\n\nIt is not entirely clear which specific techniques you are referring to. Could you please clarify the method or provide a reference, so that we can offer a more precise and relevant response?\n\n## **Closing**\n\nWe believe that the primary concerns regarding GPTAQ have now been thoroughly addressed. We would greatly appreciate it if you could kindly consider this clarification and reconsider your overall evaluation."}, {"review_id": "PUitPsOUsb", "reviewer": "Reviewer_q1UX", "summary": "This paper addresses the problem of quantization error propagation in post-training quantization (PTQ)—a crucial yet underexplored challenge. To tackle this, the authors propose Quantization Error Propagation (QEP), a method that enhances layer-wise PTQ by explicitly propagating and compensating for accumulated quantization errors. Experiments on several large language models (LLMs) demonstrate that the proposed approach improves the performance of existing layer-wise PTQ methods across various datasets.", "questions": "1. The authors reformulate the layer-wise quantization problem and introduce weight correction to address accumulated errors, resulting in a simple yet effective approach. However, it is unclear how the proposed method is applied to the baseline round-to-nearest (RTN) quantization. Since RTN involves simple rounding and does not consider layer-wise reconstruction error, it would be helpful if the authors could provide more detailed explanations regarding its integration with the proposed method.\n\n2. The authors introduce a scaling hyperparameter $\\alpha$ in Eq. (6) to control the strength of weight correction. However, in the experiments on Llama-2 70B, weight correction is excluded for the MLP layers by setting $\\alpha=0$. Could the authors clarify the reasoning behind this decision? As I understand it, quantization error propagation typically becomes more severe as network depth increases. Therefore, for deeper models, including weight correction should help mitigate this issue. This experimental setup appears to contradict the motivation of the paper.\n\n3. An important result is missing from Table 1. The authors should include the performance of the unquantized models to better illustrate the impact of quantization and to facilitate comparison of performance degradation.\n\n4. In Table 3, the authors only report results for QEP+RTN, which is insufficient, especially since QEP+RTN performs worse than GPTQ and AWQ. It would be more informative to also include the results for QEP combined with GPTQ and AWQ.", "limitations": "The authors do not discuss the limitations of their work or its potential negative societal impacts.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper focuses on a crucial yet underexplored challenge: the accumulation and amplification of quantization errors across layers, which can significantly degrade performance. The paper is well organized and easy to follow. The proposed method is simple yet effective, and the authors conduct extensive experiments across various datasets to demonstrate its effectiveness.\n\n\nWeaknesses:\n\nThe reformulation of Eq. (3) has been discussed in previous work, such as [1], and should be properly acknowledged and discussed in the paper. Additionally, an important baseline is missing—namely, directly solving Eq. (3) using SGD as in Omniquant [Shao et al., 2023]. Including this baseline would provide a more comprehensive comparison and help to better contextualize the effectiveness of the proposed method relative to existing methods.\n\nRefernece:\n\n[1] Up or Down? Adaptive Rounding for Post-Training Quantization. ICML 2020.", "was_revised": true, "final_justification": "I appreciate the authors’ response. As the rebuttal has satisfactorily resolved my earlier concerns, I will increase my score.", "rebuttal": "We sincerely appreciate the reviewer's positive feedback on the clarity of our manuscript and **the comprehensiveness of our experiments**. Below, we respond to each of the reviewer's concerns and questions in turn.\n\n## **Weakness: Relation to Prior Work (AdaRound)**\n\nThank you for pointing us to AdaRound (ICML 2020). Although AdaRound (see Eq. 25 in the original paper) also emphasizes minimizing the accumulation of quantization error, it does so through block-wise quantization and does not propagate quantization errors across layers as QEP does. Therefore, we believe that QEP remains novel and offers a distinct contribution.\nOur primary objective is to improve the performance of layer-wise PTQ by revisiting the core design of existing methods and introducing quantization error propagation across layers.\nIn the camera-ready version, we will include a statement in the Related Work section comparing AdaRound with QEP, highlighting both their similarities and key differences.\n\n## **Weakness: OmniQuant Baseline**\n\nAs noted by Reviewer pLsN, \"*The experiments and ablations were generally comprehensive*,\" and in Section 6 (Experiments, Lines 243–248),  **our primary objective is to enhance the performance of layer-wise PTQ by revisiting its core algorithm.**\nTherefore, we consider benchmarking QEP against RTN, GPTQ, AWQ, and QuIP to be appropriate, as OmniQuant performs block-wise rather than layer-wise PTQ and thus falls outside the scope of this study, which focuses specifically on layer-wise PTQ.\n\nThat said, as you pointed out, comparing QEP-enhanced layer-wise PTQ with OmniQuant, despite their methodological differences, is indeed an interesting direction. Accordingly, we provide a representative comparison between QEP-enhanced layer-wise PTQ and OmniQuant using LLaMA 2 7B, as shown below.\n\n| Method       | PPL(Wiki) INT4   | PPL(Wiki)  INT3   | PPL(Wiki) INT2      |\n|--------------|-------:|-------:|----------:|\n| RTN + QEP    | 6.017  | 17.309 | 97 153.266 |\n| GPTQ + QEP   | 5.933  | 7.898  | 7 214.328  |\n| AWQ + QEP    | 5.756  | 11.131 | 229 888.406|\n| QuIP + QEP   | **5.753**  | **6.154**  | **11.972**    |\n| OmniQuant    | 5.880  | 7.065  | NaN       |\n\nNaN indicates that the value has diverged as a result of becoming excessively large.\nAs shown in the table above, the QEP-enhanced layer-wise PTQ outperforms OmniQuant.\n**This result aligns with the findings reported in the PTQ benchmarking paper [1], which also shows that OmniQuant underperforms compared to layer-wise PTQ without QEP enhancement (see Tables 1–4 in [1]).**\nWe will explicitly incorporate this observation in the Related Work and Appendix sections of the camera-ready version.\n\n[1] Zhao Jiaqi, et al., *\"Benchmarking post-training quantization in llms: Comprehensive taxonomy, unified evaluation, and comparative analysis.\"*, arXiv preprint arXiv:2502.13178 (2025).\n\n## **Question 1**\n\nRTN quantizes each FP16 weight $W\\_l$ using simple round-to-nearest. To combine RTN with QEP, we first compute the corrected weight $W^{\\ast}\\_l(\\alpha\\_l)$ using Equation (6) in the main text, and then apply the same round-to-nearest rule. We will include a brief note describing this procedure in the appendix of the camera-ready version.\n\n## **Question 2**\n\nThank you for your question regarding the propagation strength $\\alpha\\_{l}$. We begin by outlining the role of propagation strength.\nThe propagation strength $\\alpha\\_{l}$ (see Lines 227–236) serves two primary purposes: (i) mitigating overfitting and (ii) reducing computational and quantization-time overhead.\nNote also that although $\\alpha\\_{l}$ values closer to 1 tend to minimize quantization error on calibration data, as shown in Theorem 5.2, this does not necessarily lead to better generalization to unseen data.\nAs shown in the table below for Llama‑2‑70B (INT3), setting $\\alpha\\_{l} = 1$ leads to overfitting, whereas setting $\\alpha\\_{l} = 0$ leads to underfitting.\n\n| Method               | $\\alpha_l$ (down\\_proj) | $\\alpha_l$ (others)  | Llama2 70B  (INT3, PPL) |\n|----------------------|--------:|--------:|-----------:|\n| GPTQ (baseline)      |    0.0    | 0.0   | 4.860 |\n| **GPTQ + QEP**      | 0.0    |    0.5    | **4.102** |\n| GPTQ + QEP      | 0.5    |    0.5    | 4.743 |\n| GPTQ + QEP           |      1.0 | 1.0  |   4.771 |\n\n## **Question 3**\n\nWe agree that including the FP16 results is essential. The table below presents the perplexity and accuracy metrics of the unquantized models.\n\n| Metric (FP16)         | Llama2(7 B)   | Llama2(13 B)  | Llama2(70 B)  |\n|-----------------|-------|-------|-------|\n| Perplexity (PPL) | 5.472 | 4.883 | 3.319 |\n| Average Accuracy        | 0.7601| 0.7840| 0.8014|\n\nThese results will be incorporated into the camera-ready version.\n\n## **Question 4**\n\nAs you suggested, we agree that the results for GPTQ+QEP and AWQ+QEP should be included. The table below shows the results for GPTQ+QEP and AWQ+QEP.\n\n| Method        | 7 B  | 13 B | 70 B |\n|---------------|------|------|------|\n| GPTQ          | 14.9 m | 26.4 m | 2.9 h |\n| AWQ           | 13.6 m | 25.4 m | 2.4 h |\n| QEP + GPTQ    | 19.1 m | 34.3 m | 3.2 h |\n| QEP + AWQ     | 21.8 m | 41.4 m | 4.2 h |\n| QEP + RTN | 10.9 m | 19.6 m | 1.7 h |\n\nSince the quantization time of RTN is negligible, the additional cost of incorporating QEP with RTN is approximately equal to the inherent overhead of QEP itself. Including QEP does not significantly increase the overall quantization time, and its overhead remains small enough to maintain the simplicity and efficiency of layer-wise PTQ methods.\n\n## **Limitations**\n\nThank you for pointing out the lack of discussion regarding the limitations of our method.\nWe acknowledge this oversight and will address it in the camera-ready version by explicitly outlining the following limitations:\nFirst, like other layer-wise PTQ methods, QEP depends on calibration data. Consequently, its performance is affected by the quality of that data. As long as calibration is required, fully resolving this issue remains challenging. Nevertheless, as shown in Table 4 of the main text, QEP shows less overfitting to calibration data compared to other layer-wise PTQ approaches.\nSecond, although a key contribution of QEP is its mechanism for controlling propagation strength, it introduces a new hyperparameter, the strength factor $\\alpha\\_l$, which requires tuning. Nonetheless, we demonstrate that using a representative fixed value (e.g., $\\alpha\\_{l} = 1/2$) results in substantial improvements over existing methods in most cases. Developing a method that adaptively determines $\\alpha_l$ for each layer is a promising future direction.\nRegarding potential negative societal impacts, our work is a technical contribution focused on model quantization and does not directly involve end users or societal applications. \n\n\n## **Closing**\n\nWe believe we have addressed your concerns regarding **the relationship with Adaround** and **the benchmarking with respect to OmniQuant.** We would greatly appreciate it if you could kindly reconsider the score in light of these clarifications."}, {"review_id": "RFAhixxreV", "reviewer": "Reviewer_ffnz", "summary": "- Summary:\n> - The paper addresses the challenge of quantization error accumulation in layer-wise post-training quantization (PTQ) for large language models (LLMs), which degrades performance in low-bit quantization scenarios.\n> - The proposed QEP uses a closed-form weight correction from full-precision/quantized activation differences to address error propagation. A tunable α controls correction strength as regularization, which prevents overfitting.\n> - Extensive experiments validate that QEP enhances popular layer-wise PTQ methods (e.g., GPTQ, AWQ, QuIP) on datasets such as WikiText-2, and \"ArcE, PiQA, and SC datasets\". \n> - Ablation studies confirm the effectiveness of the tunable propagation mechanism, and runtime analyses show minimal computational overhead, making QEP suitable for large-scale LLM deployment. The framework demonstrates robustness to calibration data variations and outperforms baselines across different model sizes and bit-widths.", "questions": "- Does the proposed method QEP rely on high-quality calibration datasets? How to remove such dependency?\n- The proof of Theorem 5.2 relies on the Lipschitz continuity assumption of activation functions. Do activation functions in practical models strictly satisfy this condition? Does it lead to a discrepancy between the theoretical upper bound and the actual error control effect?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths:\n> - (1)  **QEP addresses the core limitation of existing layer-wise PTQ by explicitly modeling and compensating for quantization error accumulation**, a problem that causes performance degradation in low-bit quantization. The framework’s weight correction term and tunable propagation parameter α prevents overfitting.\n> - (2)  The paper provides formal proofs (appendix A) showing that QEP bounds quantization error better than baseline methods. This theoretical foundation validates the approach’s effectiveness in principle.\n> - (3) QEP demonstrates significant improvements across popular PTQ methods (GPTQ, AWQ, QuIP) on Llama-2 and Mistral-7B models. \n> - (4) Despite introducing error correction, QEP maintains computational overhead comparable to existing methods. Tunable α​ allows adaptive control over complexity as shown in 5.3.\n> - (5) QEP shows stability across calibration datasets (C4, PTB, WikiText-2) and model sizes (7B–70B), outperforming baselines in zero-shot tasks like ARC-Easy and PIQA. \n>- (6) **Source codes are available**\n\n- Weaknesses:\n> - (1) QEP’s performance might rely on representative calibration data, as overfitting to small or biased datasets can degrade results. \n> - (2) the optimal α​ requires manual adjustment (e.g., α​ = 1/2), which may limit usability for practitioners. Automated hyperparameter optimization is suggested but not explored.\n> - (3) Although the paper provides a runtime comparison in Table 3, the analysis of computational cost is insufficient. For instance, it could present the comparison between GPTQ and QEP + GPTQ.\n> - (4) The paper compares the runtime of the quantization process (Table 3), but does not provide the memory footprint data of QEP during the training phase.", "was_revised": true, "final_justification": "The authors resolved my concerns during the rebuttal, so I raised my score to Accept.", "rebuttal": "We appreciate the reviewer's acknowledgment of (i) **the novelty and theoretical soundness of QEP**, and (ii) **our commitment to releasing the source code**. Below, we respond to each weakness and question in turn.\n\n## **Weakness and Question (1)**\n\n**We acknowledge that PTQ results can be sensitive to the choice of calibration data. However, this issue is not unique to QEP. it affects all calibration-based PTQ methods.** To ensure a fair comparison, we used the same calibration data as in prior work, as detailed in the Experiments (Datasets) section of the main text.\nIn response to your question, *\"Does the proposed method QEP rely on high-quality calibration datasets?\"*,  we emphasize that we used the same calibration datasets as benchmark methods such as GPTQ, AWQ, and QuIP. Therefore, **QEP does not rely on calibration data of particularly higher quality than these baselines**, and we do not view this as a limitation specific to our approach.\n\nNote that **QEP is less sensitive to the choice of calibration data compared to other methods**. As demonstrated in lines 307–319 of the main text, QEP avoids the overfitting commonly observed with GPTQ. While a comprehensive analysis of all possible calibration subsets is beyond the scope of this paper, we regard this as a promising direction for future work and will highlight it in the revised manuscript.\n\n## **Weakness (2)**\n\nAs noted in Lines 253–256, we did not fine-tune $\\alpha\\_l$; instead, we selected a representative value of $\\alpha\\_{l} = 1/2$ uniformly across all layers. \nNote that, **despite using this single representative setting, QEP consistently outperforms GPTQ, AWQ, and QuIP across all experiments.**\n\nAs shown in Proposition 5.3 of the main text, the parameter $\\alpha\\_{l}$ serves a role similar to regularization, helping to prevent overfitting. Simply setting alpha $\\alpha\\_l$ for the down projection, which contains a large number of parameters and is more prone to overfitting, to zero leads to noticeable performance gains. As shown in the table below for Llama‑2‑70B (INT3), setting $\\alpha\\_{l} = 1$ results in overfitting, while setting $\\alpha\\_l = 0$ results in underfitting.\n\n| Method               | $\\alpha_l$ (down\\_proj) | $\\alpha_l$ (others)  | Llama2 70B  (INT3, PPL) |\n|----------------------|--------:|--------:|-----------:|\n| GPTQ (baseline)      |    0.0    | 0.0   | 4.860 |\n| **GPTQ + QEP**      | 0.0    |    0.5    | **4.102** |\n| GPTQ + QEP      | 0.5    |    0.5    | 4.743 |\n| GPTQ + QEP           |      1.0 | 1.0  |   4.771 |\n\n\nIf necessary, additional results can be provided during the discussion phase or included in the camera-ready version.\n\nIn practical deployments, users can select layer-specific $\\\\{\\alpha\\_l \\\\}\\_{l}$ values via cross-validation, potentially improving performance at a modest cost, as layer-wise PTQ remains substantially faster than block-wise quantization.\nAlthough automatic tuning of $\\\\{\\alpha\\_l\\\\}\\_{l}$ is a promising avenue for future work, it lies outside the scope of our main contribution: revisiting layer-wise PTQ and introducing quantization error propagation.\n\n\n## **Weakness (3)**\n\nAs you suggested, we agree that results for GPTQ+QEP and AWQ+QEP should be included. The table below presents the results for GPTQ+QEP and AWQ+QEP.\n\n| Method        | 7 B  | 13 B | 70 B |\n|---------------|------|------|------|\n| GPTQ          | 14.9 m | 26.4 m | 2.9 h |\n| AWQ           | 13.6 m | 25.4 m | 2.4 h |\n| QEP + GPTQ    | 19.1 m | 34.3 m | 3.2 h |\n| QEP + AWQ     | 21.8 m | 41.4 m | 4.2 h |\n| QEP + RTN | 10.9 m | 19.6 m | 1.7 h |\n\nSince the quantization time of RTN is negligible, the additional cost of incorporating QEP with RTN is roughly equivalent to the inherent overhead of QEP itself. Including QEP does not significantly increase the total quantization time, and its overhead is sufficiently small to preserve the simplicity and efficiency of the layer-wise PTQ methods.\n\n## **Weakness (4)**\n\nExperiments were conducted on a 32GB V100 GPU to perform layer-wise PTQ, enabling successful quantization of LLaMA 2 70B. These findings suggest that large-scale quantization is feasible even with practical consumer-grade hardware.\nThe QEP correction, defined in Equation (6) of the main text, is carried out as a distinct step from the quantization process.\nDuring quantization, **GPU memory is released after the correction step, ensuring that no additional memory is consumed.**\n\n## **Question (2)**\n\nThank you for your insightful comments on our theoretical analysis.\nFrom the standpoint of training stability, **many widely used activation functions in deep learning, including ReLU, satisfy Lipschitz continuity**.\nTherefore, this assumption is expected to provide robust results in practical deep learning models.\nIndeed, the perplexity and accuracy results in Tables 2 and 3 of the main text show that employing QEP leads to smaller final quantization errors, thereby supporting the robustness of the theoretical analysis.\nAs you noted, we acknowledge the importance of validating the assumptions underlying the theoretical analysis through comprehensive experimental verification, and we consider this an important direction for future work.\n\n## **Closing**\n\nAs outlined above, we believe we have adequately addressed the concerns you raised, especially those related to the calibration data. We would appreciate it if you could take these improvements into account when reevaluating our score."}]

Author rebuttal:

Meta-review: The paper proposes Quantization Error Propagation (QEP), a lightweight and theoretically grounded framework that mitigates error accumulation in layer-wise post-training quantization for LLMs. It introduces a tunable error-propagation mechanism and closed-form correction, showing consistent accuracy gains across GPTQ, AWQ, and QuIP, with minimal overhead. Strengths include clear motivation, broad applicability, strong empirical results on large models and tasks, and theoretical guarantees. Weaknesses lie in limited comparisons to some concurrent methods (e.g., GPTAQ, QuIP#) and the need for manual hyperparameter tuning, though these do not undermine the main contribution. I recommend acceptance because QEP provides a simple yet impactful improvement to PTQ, addressing a fundamental challenge with practical and scalable benefits. During rebuttal, the authors added OmniQuant baselines, clarified differences from AdaRound and GPTAQ, and provided calibration analyses; most reviewers acknowledged these efforts, with scores raised or maintained positively. Overall, this is a solid and timely contribution to efficient model quantization.
Decision: Accept (poster)