TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation
Authors: Junhan Kim
Yeo Jeong Park
Seungwoo Son
Chungman Lee
Ho-young Kim
Joonyoung Kim
Yongkweon Jeon
Venue: ICLR 2026 Poster (2026)
Keywords: Quantization
LLMs
Hessian
Attention
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We propose TurboBoA, a backpropagation-free post-training quantization algorithm that significantly accelerates attention-aware quantization while achieving state-of-the-art LLM accuracy.

Abstract: The rapid growth of large language models (LLMs) has heightened the importance of post-training quantization (PTQ) for reducing memory and computation costs. Among PTQ methods, GPTQ has gained significant attention for its efficiency, enabling billion-scale LLMs to be quantized within a few GPU hours. However, GPTQ's assumption of layer-wise independence leads to severe accuracy drops in low-bit regimes. Recently, BoA improved upon GPTQ by incorporating inter-layer dependencies within attention modules, but its reliance on sequential quantization across all out-channels makes it substantially less efficient. In this paper, we propose TurboBoA, a new backpropagation-free PTQ algorithm that preserves the accuracy benefits of BoA while significantly accelerating the process. The proposed TurboBoA introduces three key innovations: (i) joint quantization of multiple out-channels with a closed-form error compensation rule, which reduces sequential bottlenecks and yields more than a three-fold speedup; (ii) a correction mechanism for errors propagated from preceding quantized layers; and (iii) adaptive grid computation with coordinate descent refinement to maintain alignment during iterative updates. Extensive experiments demonstrate that TurboBoA delivers substantial acceleration over BoA while consistently improving accuracy. When combined with outlier suppression techniques, it achieves state-of-the-art results in both weight-only and weight-activation quantization. The code will be available at https://github.com/SamsungLabs/TurboBoA.

Published as a conference paper at ICLR 2026
TURBOBOA: FASTER AND EXACT ATTENTION-AWARE QUANTIZATION WITHOUT BACKPROPAGATION
Junhan Kim, Yeo Jeong Park, Seungwoo Son, Chungman Lee, Ho-young Kim, Joonyoung Kim, Yongkweon Jeon" Samsung Research, Scoul, Korea {jun_one one kim, yeo.j.park, dragwon. jeon) sron)esamsung.com
ABSTRACT
The rapid growth of large language models (LLMs) has heightened the importance of post-training quantization (PTQ) for reducing memory and computation costs. Among PTQ methods, GPTQ has gained significant attention for its efficiency, enabling billion-scale LLMs to be quantized within few GPU hours. However, GPTQ's ' assumption of layer-wise independence leads severe accuracy drops in low-bit regimes. Recently, BoA improved upon GPTQ by incorporating inter- layer dependencies within attention modules, but its reliance sequential quanti- zation across all out-channels makes it substantially less efficient. In this paper, we propose TURBOBOA, the accuracy benefits of BOA while significantly accelerating the process. The proposed TURBOBOA introduces three key innovations: (i) joint quantization of multiple out- out-channels with a closed-for sequential bottlenecks and yields more than three-fold speedup; (ii) correction mechanism for errors propagated from preceding quantized layers; and (iii) adap- tive grid computation with coordinate descent refinement to maintain alignment during iterative updates. Extensive experiments demonstrate that TURBOBOA de- livers substantial acceleration over BoA while consistently improving accuracy. When combined with outlier suppression techniques, it achieves state-of-the-art results both onlght-only and weight- quantization. The code will be available httttps:: //t::::///ttttt com/ Samsammmsssaaas Sawsungaabs........o//o..a.aa... Tarass/uubbbbba.
backpropagation-fii free PTQ algorithm that preserves
error compensation rule, which reduces
INTRODUCTION
The rapid scaling of large language models (LLMs) (Touvron et al. 2023a b has dramatically in- creased their memory footprint and computational requirements, making deployment on resource- constrained hardware challenging. As practical solution to reduce memory usage and accelerate inference, post-training quantization (PTQ), which reduces the precision of weights and activations using only small calibration dataset, has received considerable attention.
The PTQ pipeline for LLMs typically involves two major stages. First, the model is transformed more robust to quantization by suppressing outliers weights and activations through scal- ing (e.g., SmoothQuant (Xiao et al.) (023)) or rotation (e.g., QuaRot (Ashkboos et al.) 20244). Next, the transformed model is quantized under specific bit-width constraints. For weight quan- tization, backpropagation- frree methods exploiting besssian-guided error compensation have been widely adopted (Frantar et al., 22033] Kim et al. 2025 Li al. (025), as they facilitate efficient optimization of quantized weights without gradient-based training.
Among backpropagation-free -free methods, GPTQ is representative approach known for its efficiency, enabling the quantization of billion-scale L.MSs within few GPU hours (Frantar et al. 2023). How- ever, GPTQ assumes layer-wise independence, which leads to severe accuracy degradation in tow-bit regimes (e.g., INT2). Recently, BoA addressed this by exploiting attention reconstruction errors in the Hessian approximation (Kim et al. 2025). By capturing cross-layer dependencies within at- tention modules, BoA yields substantial accuracy gains GPTQ. However, BoA introduces
"Corresponding Author

---

Published as a conference paper at ICLR 2026
significant computational bottleneck: it performs quantization sequentially across out-channels to compensate for the quantization error each out-channel (see Fig. D. Such sequential process, although necessary for precise enror compensation, severely slows down the overall process and makes BoA substantially less efficient than GPTQ.
The primary goal of this paper is to accelerate BoA without sacrificing accuracy and even to achieve further performance improvements. Our main contributions are as follows:
We propose TURBOBOA, which significantly accelerates BoA (Section 3.1
Our key idea is to quantize multiple out-channele simultaneously, thereby reducing the number of sequential operations while explicitly incorporating their dependencies into the error com- pensation (Proposition 3.1). Our timing measurements demonstrate that the proposed joint quantization leads to more than a three-fold speedup over BoA (Table
We incorporate features into TURBOBOA enhance its performance (Sections 3.2 and (.3)) First, TURBOBOA compensates for errors propagated from preceding quantized layers, mitigating error accumulation across layer depths (Proposition 3.2). Second, TUR- BOBOA adaptively determines quantization grids to align them with weights iteratively updated for the error compensation and further refines grids reduce attention reconstruc- tion errors (Proposition 3.3).
From extensive experiments, we demonstrate that TURBOBOO delivers substantial acceler- ation over BoA while achieving superior accuracy (Table Furthermore, when integrated with outlier suppression techniques, TURBOBOA achieves (late-of-the-Be results for both weight-only and weight-activation quantization (Tables [aoblssaand] and
Notations We use lowercase letters to denote vectors (e.g., w) and uppercase letters for matrices (e.g., W). wN denotes the i-th element w, and Wa, is the (i.j)-th entry in W. We denote the i-th row of W, which corresponds to the i-th out-channel, by W,, i,; and the j-th column of W by W..j- The submatrix of W consisting of the rows indexed by the index set B is denoted by W B., Similarly, W. denotes the submatrix of W with the columns indexed by B. e, is the vector with a in the i-th coordinate and O's elsewhere, and I denotes the identity matrix. Od, x and 1d, xd, are (dg dimenssosll matrices with entries being zeros and ones, respectively.
2 RELATED WORKS
2.1 LLM QUANTIZATION
The main goal of PTQ is to minimize the degradation in task loss induced by quantization, which can be relaxed to the layer-wise reconstruction problem (LeCun al. 1989 Nagel et al. 2020)
min II( XI QEQ
where W E Rodu is weight matrix for one layer, X Rd.x X is its input of length L, and Q is the set of discrete quantized weights Q. channel-wise quantization adopted, Q can be expressed as
Q = diag(s) W int, Wine (O....,."
2 1]
1]dous dous x
(2)
where s E BRottt is scale vector and b is the target bit-width.
Early PTQ approaches aimed reformulate the assignment of discrete quantized values into a con- tinuous optimization problem, enabling quantized weights to be learned through gradient-based training. Representative algorithms include AdaRound, which introduced differentiable approxi- mations for the rounding operation (Nagel et al., (020), and BRECQ, which further extended this idea to the block-wise reconstruction problem to consider cross-layer dependencies (Li et al. I) Although these methods have been successful for small-scale models (e.g., ResNet), they depend on time-consuming gradient-bart based training, which renders them impractical for LLMs with billions of parameters.
Recent research has therefore focused on developing cost-effective altermatives for LLM quantiza-
tion (Frantar al. 2023 Jeon et al. 2023 Kim al. 2024 2025). These works can be categorized
2

---

Published as a conference paper at ICLR 2026
into two orthogonal classes. The first is backpropagattio free methods, which resort to Hessian-
guided error compensation (e.g., GPTQ (Frantar et al. 2023) and BoA (Kim et al. 2025)). The second is transformation-baat based methods, which suppress outliers via scaling rotation, thereby transforming LLMs into a more quantization-frienlly form (e.g., SmoothQuant (Xiao et al. 2023
and QuaRot (Ashkboos al. 2024)).
Our approach belongs to the backpropagation-fie class and further improves BoA by enhancing both efficiency and accuracy. Furthermore, similar GPTQ and BoA, our method can be effectively combined with transformation-baase methods, demonstrating strong complementarity between the two classes.
2.2 BACKPROPAGATION--FRE. WEIGHT QUANTIZATION
Backpropagatton--in free PTQ algorithms, which rely on the Hessian- Hessian-guided error compensation, have been widely adopted for efficient LLM quantization (Frantar al. 2023 Li et al. [025] Kim al.]
2025). These algorithms rapidly quantize LLMs by iteratively conducting quantization and error correction, which is given as (Frantar al. 2023)
wp 4p UP. where U -Chol Up.P
Aw
(3)
where 9p is the quantized version of the weight up, H is the Hessian matrix, and Chol(-) denotes Cholesky decomposition, that is, U is upper triangular such that H U.U.
The first algorithm that successfully scaled this principle to LLMs was GPTQ (Frantar et al. 2023
However, GPTQ approximates the Hessian based layer-wise reconstruction errors, fai failing to account for inter-layer dependencies and resulting in suboptimal performance, particularly at low bit-widths (e.g., INT2). Recently, BoA addressed this issue by exploiting attention reconstruction errors in the Hessian approximation (Kim et al.) 2025). The resulting Hessians explicitly model dependencies between out-channels (see Hout Table [ enabling the error compensation for each out-channel and yielding substantial accuracy gains over GPTQ. Nevertheless, such improved Hessians make sequential processing across out-channels unavoidable; the second out-channel can be quantized after compensating for the quantization error induced by the first out-channel, which differs from GPTQ that quantizes all out-channels simultaneously (see Fig. [ To alleviate this bottleneck, BOA parallelizes quantization across different attention heads (e.g.. quantizing the first out-channel of all heads concurrently) by assuming head-wise independence. While this strategy provides some acceleration, BoA still remains substantially more time-consuming than GPTQ, highlighting a trade-off between accuracy and efficiency.
Table 1: Loss used approximate Hessians and the corresponding Hessians in GPTQ and BoA.
Method
Layer
Loss (|) ([GG)X)))))
H = H.n @ Hout
GPTQ
W (QQ.KV
JAWX 2
xx T &1
W
K, A5,,W, Q, X
XX* & K,
BoA
W KA
JQ, K,A K X
XX & Q
W v.A
W out, AW V.A.WWW.. XATIIR
X & WT
outhh Wouth
h denotes the index of the attention head.
TURBOBOA
We now introduce the proposed TURBOBOA. To enhance both the efficiency and accuracy of BoA, we introduce three key innovations, each of which will described in the following subsections in detail.
3.1 SIMULTANBOUS QUANTIZATION OF MULTIPLE OUT-CHANNELS
As described earlier, BoA sequentially quantizes out-channels one by one. This means that when quantizing weight matrix with 128 out-channels (e.g., query, key, and value projection weights in Llama3-8B), BoA requires 128 sequential operations. Consequently, BoA substantially more time-consuming than GPTQ, in which all out-channels quantized in parallel.
3

---

Published as a conference paper at ICLR 2026
out-channels
(a) GPTQ
(b) BoA
(c) TURBOBOA (N 4)
Figure I: Quantization orders in GPTQ, BoA, and the proposed TURBOBoA, (a) GPTQ quantizes all out-channels jointly but without error correction. (b) BoA compensates for the quantization error but requires fully sequential processing across out-channels. (c) TURBOBOA reduces sequential operations by quantizing multiple N out-channels jointly while still applying emror compensation.
To accelerate the quantization process, TURBOBOA quantizes multiple N out-channels simultane- ously, thereby reducing the number of sequential operations (see Fig. [T. In the previous example, when . 16, the number of sequential operations decreases from 128 8. We note that while N out-channela are quantized together as if they were mutually independent (as in GPTQ), we explic- itly incorporate their dependencies into the error compensation step. To do so, instead of naïvely adding weight compensation for each out-channel, we formulate the problem of compensating for the errors of multiple out-channels as
min AW
||AAAX 12:
(4)
S.L. e AW Q W., 10si< N),
where Q is the quantized version of W i and we use the unified notation GAWX | to denote the attention reconstruction errors in Table [ G K2, for the query projection weight W W..)). In the following proposition, we present closed-form solution to
Proposition 3.1. Let W be matrix whose Hessian is given as H Hin @ Hout- Suppose the first N out-channels of W have been quantized simultaneously and the other out-channels are updated to minimize the attention reconstruction error
Then, the updute [AW]Ns; satisfies
AWW] N.. = -UU N. U U B. (WB. -QB.),
(5)
where B (0,..., 1) and Uout Chol(H "T.
Proof. See Appendix C
pertinent question whether such joint quantization inevitably leads accuracy degradation. In the conventional BoA, the quantization error of the first out-channel can be compensated by all subsequent out-channels; e.g., 127 out-channels participate the error compensation in the previous example. Whereas, in our approach, multiple out-channels of out-channels available for the error correction decreases (e.g.. 127-+11 112 when N 16). In Section 4.2 will empirically show that the degradation arising from the reduced error correction flexibility is negligible, even in the low-bi bit regime (see Table 2).
quantized at once, so the number
3.2 ERROR COMPENSATION FOR PRE-QUANTIZED LAYERS
Another limitation of BoA is that it does not account for quantization emors propagated from previ- ously quantized layers. During quantization, errors produced in one layer propagate to subsequent layers by perturbing their input distributions, and these deviations accumulate as the network depth increases, as reported in GPTAQ (Li et al. 2025).
Let X be the original full-precision (FP) input. We observe that the input deviation AX == X introduces additional distortion in the attention output as follows:
X
GQX GWX G(Q w)x GW(X- - X) GAWX + GWAX.
(6)
Here, GAWX corresponds to the error introduced by quantizing the current layer, while GWAX captures the output deviation induced by the perturbed input. We explicitly incorporate the addi-
tional distortion GWAY into the error compensation. Specifically, after quantizing N out-channels
4

---

Published as a conference paper at ICLR 2026
Algorithm TURBOBOA
Input: weights W (Q.,.V) HRd xd target wit-width b, inputs X E R FP representation X R
number N of out-channels quantized simultaneously, and stabilization coefficient Output: quantized weights (Q.K.V)
1: for W E (Wq. W k. Wv) ) do
2: Initialize quantized outputs and integer weights: Qn, Wint.h < Oá, xd
Initialize out-channel scales: St. 1d,x1
4: Compute attention-aware Hessians Hin, and Hout,
5: Compute U = Chol(H )², Uouth = Chol(H out,h D², and R = -=a(x-x)x
6: Initialize updated weights: W W
for i 0, 2N do
8: Take N out-channels to be quantized jointly: W [Wa]B, (B (i,
N 13)
9: Set scales: [S^]B <- min, tr( (r(AW)) Hin, T))
10: Quantize W ([Qu]R,:. (.. (W..:.(Wm..) (Wnn,a),.) <- GPTAQ(W()) Un R. [s)]
b see Algorithm
11:
Update remaining out-channels:
R [W [W.l+NNs + [W WWll+N N mu AE (Wa,], [Qn]B,)
U A [W] RH.
Refine seales: Ss, 4- min, |G (diag(s) Wine, W.)X + G. AX
b see Algorithm
13:
Update quantized weights: Qa + diag(s)) Wenth
W WB.. we compensate for both the error introduced by the weight perturbation (i.e., G. AW B. and the error incurred by the input deviation AX (i.e., G. WB, B. AX), which reformulates the error compensation problem in 4 as
min AW
||AAWX G. W B AX
S.I. e AW -W (0 <i<iN)
The following proposition provides closed-form solution to the above problem.
Proposition 3.2. Lei W be a matrix whose Hessian given as H Hin & Hout- Suppose the first N out-channels of W have been quantized simultaneously, where the input X is distorted from the FP representation X due the quantization errors produced in earlier layers. Then, the update [AW).. of the other out-channels to minimize the attention reconstruction error in 7 is
[AW] - B J ((W B.: -B.))-V WBB. WB.RH..)
(8)
where (0...., --13, 1), ont L and -XXT.
Proof. See Appendix D
Compared to the update rule in (5) for the first layer, the update rule in (8) involves the additional term related the input deviation AX, which explicitly compensates for errors propagated across quantized layers and ensures the quantized model to replicate the behavior of the FP model more faithfully across all layers.
We note that our approach of incorporating the input deviation AX is motivated by the error com- pensation framework of GPTAQ (Li et 2023). However, a key technical distinction lies in the structure of the Hessian Hout- While GPTAQ assumes Hout = I, which decouples out-channels and simplifies the optimization into set of independent vector equations, our framework addresses general (potentially dense) Hout to incorporate dependencies within out-channels. This transition from a separable vector-wise optimization to coupled matrix- wise formulation requires a more sophisticated derivation of the closed-form update rule, detailed in Appendix
3.3
ADAPTIVE GRID SELECTION WITH ATTENTION- REFINEMENT
remaining limitation of BoA lies its grid computation. First, once initialized, the quantization grid remains fixed throughout the iterative process (Kim et al. 2025). However, since out-channels
5

---

Published as a conference paper at ICLR 2026
continuously updated due to the enror compensation, the initial grid becomes misaligned with the updated weights. This misaligmment would particularly be severe in low-bit regimes, where large weight perturbations (W B.: a. result in large updates [AW] N:.: (sce ( Second, BoA computes the grid in a way to minimize the layer-wise reconstruction loss, which is not aligned with the goal of minimizing the attention reconstruction error.
To address these issues, TURBOBOA determines the quantization grid immediately before each out- channel is quantized (line in Algorithm D. which ensures that every quantization step uses grid aligned to the previously updated weights. To reduce unnecessary overhead, the grid is computed exclusively for the aut-channels to be quantized each quantization step. Furthermore, we introduce a grid refinement step (line 12 in Algorithm| []. At this stage, we freeze the integer weights Wine E (0. ..... 2 }] dout X di.. assigned through the iterative process (lines 7-11 in Algorithm and refine only scales E IRdout to further reduce the attention reconstruction error in 66
min [G[diag(ss W int W) + GWAX
To solve this problem, we adopt coordinate descent (CD), which iteratively updates one scale at time while fixing the others. The following proposition presents the closed-form update rule for each CD step, which facilitates each scale update without costly numerical optimization.
Proposition 3.3. Let W be a matrix whose Hessian is given as H Hin Hout- Suppose W has been quantized to Q = tiag(s)W: ina where is the scale vector and W int is the fixed integer weights. Suppose the scales s are refined to minimize the attention reconstruction error via CD. Then, the update rule for each CD step is given as
W int HH (W- Q)" R R²' W
5 = 83 +
[ Hin WT Houulss
where R = AXXT.
Proof. See Appendix E
After refining the scales, we update the quantized weights (line 13 in Algorithm D. which yields the final output the proposed TURBOBOA.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We evaluate the performance of TURBOBOA using Llama models (Touvron al. 2023a b). Fol- lowing prior works (Ashkboos et al. 2024 Liu et al. 2024 Kim et al. 2025). we use 128 sequences of length 2048 randomly sampled from the WikiText-2 (Wiki2) dataset Merity et al. 2016) as cali- bration data for quantization. As performance metric, we use perplexity (PPL) on the Wiki2 and C4 (Raffel et 2020) test sets and the average accuracy across eight zero-shot commonsense rea- soning tasks All experiments were conducted using NVIDIA H100 GPUs (80 GB). While single GPU was sufficient for most models, we utilized two GPUs for the 70B model accommodate its larger memory requirements.
Hessian We adopted Hessians derived in BoA (see Table D. since they are currently the most ac- curate closed-form Hessians available in the literature. However, we emphasize that our main results in Propositions 3.1 3.3 are not specific to BoA's Hessians and can be applied to any Kronecker- structured Hessians H = H Haut- Consequently, our method can directly leverage more ad- vanced Hessian formulations once they become available.
ERC-chhaleengeeasy (Clark et al. 2018), BoolQ (Clark et al. 2019 HellaSwag Zellers et al. 2019
LAMBADA (Paperno et al. 2016), OpenbnokQA (Mihaylov al. 2018), PIQA (Bisk et al. 2020). and Wino-
Grande (Sakaguchi al. 2021

---

Published as a conference paper at ICLR 2026
Table 2: Ablation of multiple-row processing (INT2 quantization)
Llama3.2-1B
L-33
Llama3-81
Llama3.1-708
Method
NV
Time (min)
Wik12 (})
Time (min)
Wika2 (U)
Time (min)
Wiki2 (U)
Time (hr)
Wiki2 (L)
13.32
59.94
32:26
94.75
16.99
7.726
4
6,255
41.09
22.68
32.21
39.46
15.27
7,683
7.721
8
5,001
41.53
16.01
31,66
30.55
15.30
6.274
7.714
BOA F1
16
4.363
41.85
12.70
31.99
25.30
15.41
5.633
7.758
32
3,985
22.95
5,060
7.746
64
10.29
32.31
21.56
15.44
4,885
7.774
Following BoA [Kim et al.
2025
QuaRot has been applied hefore quantizing weights. We note that TURBOBOA reduces to GPTQ
under N 64 for Llama3.2 1B and N 128 for other models.
Table 3: Ablation of features targeting performance enhancement (INT2 quantization)
Method
F2
F3
L1ama3.2-1
L-ama3.2-38
Llama3-88
WikiZ (4)
Wiki2 (4)
C4 (3)
Time
(4)
(U)
BoA
40.40
104.9
13.32
32.26
79.17
59.94
15.20
36.95
94.75
41.85
108.1
4.363
31.99
80.09
12.70
15.41
38.96
25.30
TURBOBOA (N 16)
37.15
92.58
6.253
25.92
63.48
17.51
14.21
34.67
40.16
107.3
15.01
33.33
85.55
6.263
24.10
54.20
17.71
13.54
32.99
40.20
Time in minutes. QuaRol has been applied before quantizing weights.
Joint Quantization Hyperparameter N Our ablation study the number N of jointly quantized out out-channels indicates that significant speedups are achievable up to N 16, beyond which further increases (e.g., N 32 64) yield only marginal gains. To ensure stability, conservatively set N 16 for all main experiments. A detailed analysis is provided in Section 4.2
CD-based Scale Refinement We set the number of CD iterations to one (i.e., Pitter in Al- gorithm 2 see Appendix E additional iterations yield only marginal improvements. The corre- sponding ablation study can be found in Appendix F.4
Stabilization Coefficient Following the implementation of GPTAQ (Li et al. 2025), we intro- duce stabilization coefficient modulate the impact of the input deviation AX arising from the quantization eтrоrs of preceding layers (see line in Algorithm D This coefficient acts regularization parameter that prevents the compensation term from over adjusting to accumulated distortions, which could otherwise lead to numerical instability. In our experiments, we evaluated (0.05, 0.125, 0.25) and reported the best-performing result for each model.
4.2 ABLATION STUDIES
Recall that we incorporated three key features into the conventional BoA to accelerate the overall process and enhance the quantization performance. To validate the effectiveness of each feature, conduct ablation studies.
Speedup We first investigate the efficacy of the joint quantization of multiple N out-channels (F1), introduced mitigate the sequential bottleneck of BoA. Specifically, we measure the processing time by varying N E (4, 8, 16, 32, 64). As expected, the processing time decreases significantly as more out-channels
quantized simultaneously (see Table 2); for example, when N 16, TUR- BOBOA achieves more than a three-fold speedup. In particular, for the 70B model, this translates saving of 9~ hours in absolute terms, demonstrating substantial gain that becomes more impactful as model scale increases.
Intuitively, increasing N reduces the degrees of freedom available for error compensation. How- ever, our empirical results reveal that performance degradation remains negligible up to N 64, suggesting that the remaining out-channels provide sufficient capacity and the proposed update rule in ()) effectively captures inter-channel correlations to compensate for joint quantization errors. We leave a formal theoretical characterization of the error bounds with respect to as an interesting open question. While this robustness a large N allows for aggressive parallelization, that the speedup gain diminishes beyond N 16. Therefore, we conservatively set N = 16 for the remaining experiments to retain a higher margin of flexibility for emor compensation.
observe
7

---

Published as a conference paper at ICLR 2026
Table 4: Weight-only quantization performance on transformed Llama2 and Llama3 models
(a) PPL (C)
Precision
Transform
Quandieer
L1ama3.2-18
Llama322-18
Liama--8
Lluma2-7B
Llama2-13B
FP16
Daseline
13.16
21.31
11.05
16.49
6.139
9.444
5.473
7.266
4.885
6,730
OmniQQuatt I DuQuant
RIN
6.640
9.383
5.593
7.989
KTN
1864
22.51
17.90
7,598
RTN GPTQ
31.06
14:55
5.576
8.595
SpinQuant
16.21
27.60
12.87
20.47
7.438
12.73
6.001
8.619
5.299
7.682
RTN
139.0
101.1
111.9
48.79
QuaRat
GPTQ
16.56
27.28
13.58
20.45
7,490
12.92
6.122
8,688
5.382
7.706
BoA
26.15
19.95
7.145
12:25
5,874
8.268
5.202
7.436
TURROBOA
OmniKQuant 1 DuQuant
RIN
21.85
39.34
12.92
19.99
RTN
9.363
1.604
770.9
905.7
2.664
1.8c5
46.27
09.02
10.40
15.35
INT2
SpinQuunt
GPTQ
KIN
68.30
127.1
144.1
33.91
73.09
21.52
44.30
16:95
29.21
9.742
16.25
RTN
2665
2.5e5
2.3c6
1.1el
3.5e5
3,665
1.1el
1.lef
7,963
6.2c3
QuaRoi
GPIQ
118.5
128.8
22,05
41.92
9,293
19.47
BoA
40886
107.9
33.40
79.21
15.24
36.82
10422
19.17
8.237
14.66
TURBoBOA
33,33
85.55
24.10
54.20
13,54
32.99
9,108
16.64
7.337
13,04
(b) Zero-sho Accuracy (1)
Precision
Transform
Quantizer
Llama3.2-1B
Llamu3.2-38
Llam3-88B
Lluma2-7B
Llamx2-13B
FP16
Beseline
56,82
63.01
70.34
67.28
69.83
OmniQuant DoQuant
65
RTN
31.12
$1.59
52.42
63.24
67.04
SpinQuani
KIN
48.68
56,09
64.32
60.40
65.32
INT3
GPTQ
$1.33
59.39
67.05
64.34
67.62
RTN
38.05
35,99
42,80
31.71
36,
QuaRot
$1.13
67.79
68.
TURBOBOA
53.30
61.26
68.57
65.21
69.07
OmniQuant DuQuant
RTN
37.92
44. 14
30.42
35.97
37.94
38.95
47.41
SpinQuant
GPTQ
36.50
39.71
46.78
43.03
49.50
31.04
29.91
Qualtor
GPTQ
BoA
38.67
36-41
43.86
39.17
50.29
45.02
51.00
38.98
49. 01.51
56.92
TUNBOBOA
The official code does support models exploiting grouped query attention.
Performance Enhancement We next examine the effectiveness of two features introduced to en- hance the performance of TURBOBoA: error compensation for pre-quantized layers (F2) and adap- tive grid computation with CD-based refinement (F3). As shown in Table 3 incorporating either feature individually leads to consistent improvements. For example, on Llama3.2-1B, F2 reduces PPL from 41.85 37.15 on Wiki2 and from 108.1 to 92.58 on C4, highlighting the benefit of mitigating error accumulation across layer depths. Similarly, F3 improves alignment with the up- dated weight distribution, lowering PPL. to 39.45 and 107.3 Wiki2 and C4, respectively. Notably, the combination of both features yields the best performance, demonstrating their complementary roles; TURBOBOA achieves PPLs of 33.33 on Wiki2 and 85.55 on C4 for Llama3.2- IB, represent- ing substantial reductions over the baseline BOA. Consistent trends are observed in larger models, confirming that these enhancements generalize effectively across scales.
Finally, we analyze the runtime overhead introduced by these features. While F3 adds only a marginal cost (e.g., approximately one minute for Llama3-8B) the overhead of F2 is more no- ticeable as it requires an additional forward pass of the FP model to compute the input deviation AX. However, we emphasize that this is fixed, one-time cost, the FP activation X is inde- pendent of the quantization process. Despite this overhead, TURBOBOA still completes the entire quantization process substantially faster than BoA, which confirms that the efficiency gains from ducing sequential operations via more than compensate for the additional computations required for accuracy enhancement.
8

---

Published as a conference paper at ICLR 2026
Table 5: Weight-activation quantization performance on transformed Llama2 and Llama3 models
(a) PPL (C)
Precision
Transform
Quantizer
Llama3.2-18
C4
L.ama332-3 38
C4
Llamal-88
Llama2-7B
Llama2-138
FP16
Daseline
13.16
21.31
11.05
16.19
6.139
9.440
5.473
7,266
4.885
6.730
OmnoQQuaat DuQqant
KIN
2.363
3.2c3
2863
4.4c3
RTN
1.504
1.683
2.5c5
514.3
20:12
GPTQ
235.3
173.7
76.71
W2A4KV16
SpinQuant
BoA
29.95
136.5
34.24
110.0
17.31
48.04
11:27
19.86
8.682
15.33
TURROBOA
49.74
132.0
27.01
92.01
15.43
37.68
9.905
17.52
7,862
13.64
GPTQ
71.49
154.6
51.60
145.6
21.73
60.39
23.55
47.79
10.73
22.21
OSTQuuat
BoA
44.90
107.7
29,90
74.04
15.16
37.49
18.22
7.894
13.96
TURBOBOA
35.61
15.77
12.78
OmniQuuatt
RIN
1,0c5
1.945
3,5c3
5.4e3
DuQuant
RTN
8.5e5
1.504
1.763
2,583
4.164
2.5c5
405.9
753.3
16.35
24.83
GPTQ
143.8
330.0
65.09
194.6
29.57
83.07
24.29
49.45
15.54
38.13
W2A4KV4
SpieQuant
RoA
167.0
120.0
48.52
15.96
TURBOBOA
18.95
GPTQ
214.6
22.47
OSTQuuatt
RoA
57.27
141.3
31.74
84 64.68
16.05
30.93
10.19
18.37
8.073
14.51
TURBOBOA
46,10
111.7
24.53
72.72
14.51
38.12
9.142
16.59
7,508
13.25
(b) Zero- shot Accuracy (t)
Precision
Transform
Quantizer
Lamm332-18
Llama3.2-38
Llamu3-81
Llama2-7B
Llama2-136
FP16
Baseline
56.82
63.01
70.34
67.28
69.83
OmniQuant DuQuuat
RTN
30.58
30.47
30.77
30.45
41.72
GPTQ
34.03
33.59
39.29
36.83
42.91
W2A4KV16
SpinQquant
BnA
$6.56
39.56
44.53
48.25
$4.35
GPTV
35.28
35.42
45.08
OSTQuant
TURBOBOA
39.47
45.80
50.49
52.14
$8.77
OmniQqunt DuQquat
RTN
30.29
29.78
37.26
W2A-KV4
SpinQuant
BoA
36.13
39.53
45.02
47.14
$2.50
TURBOBOA
37.28
42.44
47.75
49.89
$5.86
GPTQ
33.90
35.32
41.70
36.82
46.54
OSTQuant
BoA
36.82
41.87
46.01
49.22
55.78
The official code does support models explocting grouped query attention.
4.3 COMPARISON WITH PRIOR ARTS
We compare the performance of TURBOBOA against existing LLM quantization methods. Our comparison includes BOA, which serves as the primary baseline, and transformation-ben approaches that improve performance by suppressing outliers via scaling and/or rotation (e.g.. Om- niQuant (Shao et al.) 2023), DuQuant (Lin et al.) 2024), QuaRot (Ashkboos et al.] 2022)) Spin- Quant (Liu et 2024), and OSTQuant (Hu 2025 ); see Appendix B for the details of each method.
Weight-only Quantization We first evaluate the performance of weight-only quantization. Fol- lowing BoA (Kim al., 2025). we integrate TURBOBoA with QuaRot, which requires no training and incurs no additional inference costs. The complementarity with other transfornation-basee ap- proaches (e.g., SpinQuant and OSTQuant) will be investigated the weight-activation quantization setting (see Table 5). For results without any transformation, please refer to Appendix [.]] where we demonstrate the intrinsic effectiveness of the proposed error correction and grid selection. No- tably, in Appendix F2 provide direct comparison with GPTAQ (Li et al.) 2025) highlight the importance of incorporating dependencies between out-channels in low-bit regimes.
Table 4 summarizes the results under INT2 and INT3 quantization. Overall, BoA and the proposed TURBO outperform other methods because they explicitly account for cross-layer dependencies within the attention module during weight quantization. In contrast, OmniQuant, SpinQuant-RTT

---

Published as a conference paper at ICLR 2026
and SpinQuant-GPTT consider cross-layer dependencies only when learning transformation matri- ces and rely on naive nearest rounding or GPTQ with layer-wise objectives, thereby failing to capture such dependencies during weight quamtization. As shown, TURBOBOA consistently achieves the best results. For example, on 2-bit quantization of I Llama3.2-18, TURBOBoA improves Wiki2 PPI. from 40.86 (BoA) to 33.33. The benefits extend to zero-shot evaluation as well, where TURBOBOA achieves at least 2%p accuracy gain over other methods across all model scales. Notably, under 3- bit quantization, TURBOBoA BAA nearly preserves the FP performance. For instance, on Llama2-13B, TURBOBOA achieves 69.07%, which is very close to the FP baseline of 69.83%.
Weight-Activation Quantization We next evaluate the performance of weight- activation quanti-
zation. Following prior works (Ashkboos et al. 2024 Liu et al. 2024 Kim et al.) (025). we quantize input activations to all linear layers and KV caches using the Min-Max quantizer, where quantization parameters dynamically computed for each token. For outlier suppression,
integrate GPTQ, BoA, and TURBOBOA with either SpinQuant or OSTQuant. Unlike QuaRot, which relies on a fixed Hadamard matrix, Spin and OSTQuant optimize rotation matrices by explicitly incorporating activation quantization effects during training (Liu et al.) 2024 Hu et al. 2025
Table summarizes the results under W2A4KV4 and W2A4KV16 settings. Across both configura- tions and all model scales, TURBOBOA consistently outperforms BoA and other baselines. For ex- ample, with SuinQuant applied under W2A4KV4 on Jlama3.2-1B. TURBOBoA reduces Wiki2 PPL from 77.05 (BoA) to 3..07. When combined with OSTQuant under W2A4KV16 on Llama3.2-3B, TURBOBoA lowers C4 PPL from 74.04 (BoA) 63.75, while GPTQ and DuQuant exhibit sub- stantially higher PPLs. Consistent gains are also observed for larger models such as Llama3-8B and Llama2-13B, confirming the scalability of the proposed approach. Beyond PPL. TURBOBOA deliv- ers clear improvements in zero-shot accuracy. On Llama3-8B under W2A4KV16. TURBOBOA with SpinQuant achieves 49.22%, surpassing BoA by 5%p. On Llama2-133 13B under W2A4KV4, TUR- BoBOA with SpinQuant attains 55.86%, yielding absolute gain of more than 3%p over BoA and over 15%p compared to GPTQ. These results demonstrate that TURBOBOA not only accelcrates quantization but also achieves state-of-the-art performance in weight-activation quantization.
CONCLUSION
In this work, we proposed TURBOBOA, backpropagation-fie free PTQ algorithm that addresses the key efficiency and accuracy bottlenecks of the conventional BoA. By quantizing multiple out- channels simultaneously, TURBOBoA significantly reduces sequential operations, accelerating the quantization process by more than three-fold. Furthermore, by extending error compensation to in- corporate errors of previously quantized layers and adaptively determining quantization grids with a further CD-based refinement, TURBOBoA effectively mitigates error accumulation and misalign- ment, which could be critical in the low-bit regime. Our experimental results demonstrate that TURBOBOA delivers substantial speedup over BoA while achieving superior accuracy, and when combined with transformation based outlier suppression methods, it establishes new state-of-the-a results in both weight-only and weight-activation quantization. We believe TURBOBOA paves the way for broader deployment LLMs on resource-constrainet hardware, offering practical balance between computational efficiency and model fidelity.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian I. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot: Outlier-free 4-bit inference in rotated LLMs. arXiv:2404.00456, 2024.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. PIQA: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference artificial intelli- gence, volume 34, pp. 7432-7439, 2020.
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty natural yes/no questions. 1rXiv:1905.10044 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? Try ARC, the A12 reasoning chal- lenge. arXiv: 180300555vv,, 2018.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quantization for generative pre- pre-trained Transformers. In The Eleventh International Conference on Learning Representations, 2023.
Xing Hu, Yuan Cheng. Dawei Yang, Zukang Xu, Zhihang Yuan, Jiangyong Yu, Chen Xu, Zhe Jiang. and Sifan Zhou. OSTQuant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting. arXiv:2501. 13987, 2025.
Yongkweon Jeon, Chungman Lee, Kyungphil Park, and Ho-young Kim. frustratingly easy post- training quantization scheme for I.LMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 14446-14461, 2023.
Junhan Kim, Chungman Lee, Eulrang Cho. Kyungphil Park, Ho-young Kim, Joonyoung Kim, and Yongkweon Jeon. Towards next-level post-training quantization of hyper-scale Transformers. Advances in Neural Information Processing Systems, 37:94292-94337 2024.
Junhan Kim, Ho young Kim, Eulrang Cho, Chungman Lee, Joonyoung Kim, and Yongkweon Jeon. BoA: Attention-awan post-training quantization without backpropagation. In Forty-second In- ternational Conference on Machine Learning (ICML), 2025.
Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances in neural information processing systems, 2, 1989.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. BRECQ: Pushing the limit of post-training quantization by block reconstruction. In International Conference on Learning Representations (ICLR), 2021.
Yuhang Li, Ruokai Yin, Donghyun Lee, Shiting Xiao, and Priyadarshini Panda. GPTAQ: Efficient finetuning free quantization for asymmetric calibration. arXiv:2504.00 026692, 2025.
Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Lingi Song, Zhenan Sun, and Ying Wei. DuQuant: Distributing outliers via dual transformation makes stronger quan- tized LLMs. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krish- namoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. SpinQuant: LLM quantiza- tion with learned rotations. 245. 2024.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. 17X.0:3609.07843, 2016.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? A new dataset for open book question answering. 1009vv8009.00 2018.

---

Published as a conference paper at ICLR 2026
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? Adaptive rounding for post-training quantization. In International Conference on Machine Learning (ICML), pp. 7197-7206, 2020.
Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoe Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring broad discourse context. arXiv:1606.06033 2016.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu, Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 21(1))5485-555 2020.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. WinoGrande: An ad- versarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li. Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. OmniQuant: Omnidirectionally calibrated quantization for large language models. arXiv:2308. / 13/37, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro. Faisal Azhar, al. LLaMA: Open and efficient foundation language models. arXiv:2302.1397) 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv:2307.0920 2023b.
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp. 38087-38099. PMLR, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can ma- chine really finish your sentence? In Proceedings of the 57th Anmual Meeting of the Association for Computational Linguistics, pp. 4791-4800 2019.
12

---

Published as a conference paper at ICLR 2026
A USE OF LLMS
In our work, LLMs were used solely to assist with paper writing, specifically for improving gram- mar, polishing phrasing, and enhancing readability We have not used LLMs for developing research ideas, designing the methodology, conducting experiments, analyzing results, or drawing conclu- sions.
B TRANSFORMATION-BASHH PTQ METHODS
As noted, transformation-baee based methods aim suppress outliers within weights or activations by ap- plying a certain type of transformation, such as scaling, rotation, and permutation. Transformation- based methods have often been used to improve the quantization robustness of models before con- ducting quantization (Ashkboos et al. 2024] Liu et al. 2024, Kim et al. (025). If we denote transformation matrix by T, then the transformation one layer can be expressed as
WX (T)))
(10)
Under this formulation, the main goal of transformation-basee based methods is to construct a "good" T that makes WT and T -- easier to quantized.
Over the years, various transformation-metthn ferent strategy for constructing T to better suppress outliers and improve quantization robustness further. For example, some methods adopt lightweight deterministic transformations, while others learn T through optimization guided by calibration data. Below, we briefly summarize the contri- butions of each transformation-basee method used in comparison.
have been proposed. Each algorithm adopts a dif-
OmniQuant adopts a diagonal transformation matrix (Le., T = diag c)) to mitigate activation out- liers that persist in several channels across all tokens (Shao et al. 2023). The scaling factor is jointly optimized with the quantization parameters (scale and zero-point) of each layer via gradient- based training. The learned scaling factor can be seamlessly merged into existing components (e.g., normalization layers), thereby incurring no additional inference overhead. When measuring perfor- mance, we activated both learnable equivalent transformation (LET) and learnable weight clipping (LWC) options.
QuaRot/Spinnuant adopt orthogonal (rotation) matrices R (i.e., RR. = R I), redistribut- ing extremely large activation outliers that are present in few tokens (Ashkboos et al.] 2024 Liu ctt al. 2024). While QuaRot employs Hadamard matrices as the orthogonal transformation and thus requires training, SpinQuant learns R guided by calibration data. By applying the same or- thogonal matrix across different Transformer layers, both methods can integrate R seamlessly into existing components, thereby incurring inference overhead.
DuQuant integrates scaling diag(c), rotations R1, R2, and permutation P into a single transfor- mation (i.e., T diag(c) RR PR2) (Li et al. 2024). It provides an efficient backpropagation algorithm to compute the transformation parameters c, RRR, R2, P for each layer. Unlike QuaRot and Quant DuQuant learns distinct parameters for different Transformation blocks. While this design incurs additional inference costs, the authors demonstrate through empirical timing measure- ments that the overhead remains manageable. When measuring performance, we activated the LWC option, which leads to the better result (Lin et al. 2024).
OSTQuant combines scaling and rotation for transformation (i.e., T Riag(c) (Hu et al., 2025). In addition, it introduces a new metric, termed quantization space utilization rate (QSUR), to evaluate the quantizability of transformed data and provides a theoretical justification that the joint use of scaling and rotation improves QSUR. The scaling factors and rotation matrices are learned through gradient-baset training and then fused into the original inference graph.
13

---

Published as a conference paper at ICLR 2026
C UPDATE RULE FOR ERROR COMPENSATION OF MULTIPLE OUT-CHANNELS
We solve the constrained optimization problem in (4
by exploiting its Lagrangian:
N-1
I(WW, Ao 1 ||AAWX
AW (Q.: W.))
N-1 Σ c AW + W.. Qi.)):.
tr(H AWH
+
where Ao..... AN-1 Rd are Lagrange multipliers. Specifically, the update rule can be ob- tained by taking derivatives of the Lagrangian L(AW, Ay.... Av-1) and then setting these deriva- tives
OL
N-
2H out AWH,
Σ
Odout Xdsn?
(11a)
DAW
1=0
(OL/0Aa))
e/ AW + Wo.:
Qo::
[AW]n, + (Wn.: - QB.) = Nxdd
(OL/0AN-1)*
WW WN-1. QN-1.
(11b)
As result, the solution is attained when
X
AW 2 o,
H in
(12a)
AW B,
W B.: QB.).
(12b)
combining which yields
X
2 H
B.B W B,:
H
(13)
Finally, by combining
12ab
and
we obtain the desired update rule in (5
out N: - W QR.))
(a)
UT
Uoud N U out Unui 111 B.B W - QB.))
out
UoulN: U U B.B (IU (U B, (Uuud -1 (WB, B.: QB.)
- B UT T 1 B,B WB. B.: QB.).
(14)
where (a) because U Uout Chol(H 7 and (b) is because U out is upper triangular.
14

---

Published as a conference paper at ICLR 2026
D UPDATE RULE INCORPORATING QUANTIZATION ERRORS OF EARLIER TRANSFORMER BLOCKS
We note that the objective function (7) can be expressed
|GAWX + G. B | ||AAWX]
/ 2tr(G., B. AXX" AW" to
||AAA 2 AXX AW") +c =
||AAAX + 2tr([Hout]:. W B, RAW") +c,
where [. W B,:
AX is constant with respect to AW and the last equality holds because Hout G G and R AXX". Thus, the optimization problem in is equivalent to
min
|GAWX / 2tr( Houd W W...RAY RAW!)
(15)
s.t.
AW = -Q. W., E B).
Compared to the optimization problem in (4) for the first layer, involves an additional term in the objective whose derivative is
2tr Hou) H W R. RAW")) 2[H W..RR.
(16)
Using this together with 11 the solution attained when
aL
N-1
2H AWH,, Σ
C,
+ H
I. W R = Odouee Xdin
dAw
2=0
which is equivalent to Combining this with 12b yields
X2
AW
H I RH
(17)
XT
AW B.:
H out B,B
H
W B. RH -W B.: QB.),
which leads to
2 nut B.B W B. QB. 2
B.B
B.B
WB.R.
(18)
AT
Finally, by combining
17
and
18
we obtain the desired update rule in 8
[AW]N.: =
H
N
W B.: QB.) (IN. out
RH
out
out
B,B
out B.B
m
H ou N
(Wr B.:
QB.) + H H out
W B. RH
B.B
B.B
= [UT out N U out
W B.: QB.) + out N out
UT
WB. RH in
B.B
B.B
where the last equality holds because out Chol(H (see (a) and (b) Appendix
15

---

Published as a conference paper at ICLR 2026
E ATTENTION-AWARE SCALE REFINEMENT VIA CD
Let G go ... gd....-1 and W int emror in expressed as
= Wiitt ... Wint,dow-- 1I", then the attention reconstruction
((s) = G diag(s) W int X
2(G W X, N,nxX,GGW. GW
done
dous-
8J8, W m X
-
Σ
w GWX
1=
f=0
daut
daut
tr tr(g, int,)
XX" W Wont. 88y8k
tr(g, int,
XX W G' J8y +
3.==0
1=0
where F. denotes the Frobenius inner product (i.e. (A, = tr(AB") and is constant with respect to scales s. Using this together with Hin = XX" and XX" - (H R)", we have
- AX)
dane -1
daut
[(s) Σ
tr(g: wine. Han Win 18s8k
Σ
tr(g, w int,y W (Hin R)² r G" +
1.k=0
3=0
doul-1
dowl
(Went, Hnn Wan 8)) - 2
(w int,
(Hin R g,)s +c
j.km0
==0
dout
done
[W WinHH, Hi (Honell 888k - 2
[ Wiin - R) W Houtis 83 + e,
f.k=0
1=0
where the last equality holds because Hout = G. To minimize L(s), we adopt the CD algorithm, i.e., we iteratively update one scale at time while keeping the others fixed. Since the loss is quadratic 8j, the update formula for 8j can be obtained by setting 0c/0s,
97 Wint in HH R)
T W Houdlys EkØ, Wint Hin WT. (Ho.l..HHouull...ss.
[W sntt Hin W... Houtis
8, + Wind Wine HH, R)² Hontiss En [W Hin WT Winlii Hou
W Wint Hin W HHoulis
=8 int (H R...T' TTT'Hell Houtlis [W.. Hin. S...QT HHHolsij
[W ins H., WT Whali Houlis
W int H [ Q) RTW Hout
8, +
Wiin Hi, WT. W..elisi (Hurllis
which completes the proof. In Algorithm we summarize the pseudocode for the CD-based scale refinement.
Algorithm 2 Coordinate Descent-based Scale Refinement
Input: FP weights W, integer weights W W.n.. initial scales Hessians Hout and Hin. and deviation correla- tion R = AXX
Output: refined scales
I: Initialize quantized weights: Q <- diag(s)Wa W Witt
2: for 0, ... Mater
do
3: for j = 0, ... dout - do
4:
Update seale for the j-th out-channel:
W and (Han W Q) R R Houd
- s; +
W Winch WT WhalsssHouuly
5: Update quantized weights: Q <- diag(s)aWWa). Wa..
16

---

Published as a conference paper at ICLR 2026
F ADDITIONAL EXPERIMENTAL RESULTS
In this appendix, we present supplementary experimental results that were omitted from the main text due to page constraints. Specifically, we provide (i) weight-only quantization results without applying any transformations (e.g., scaling or rotation), (ii) a direct comparison with GPTAQ. (iii) weight- activatiii quantization results under higher weight bit- bit-widths, and (iv) an ablation study on the number of CD iterations.
F.1 WEIGHT-ONLY QUANTIZATION PERFORMANCE WITHOUT TRANSFORMATION
Table 6 reports the weight-only quantization performance GPTQ, BoA, and the proposed TUR- BOBOÅ without additional transformation. Across both 2-bit and 3-bit settings, TURBOBoA con- sistently outperforms both GPTQ and BOA. For instance, on Llama3.2 2-1B (INT2), TURBOBOA significantly reduces Wiki2 PPL from 538.9 (GPTQ) and 312.2 (BOA) 111.3, while simultane- ously improving zero-shot accuracy by 2.5%p. Even under the INT3 setting, TURBOBOA achieves clear improvements over BoA, demonstrating that the proposed enhancements remain highly effec- tive even in the absence of transformation-base outlier suppression.
Table 6: Weight-only quantization performance on Llama2 and Llama3 models
(a) Wiki2 PPL (L)
Precision
Method
Llama3-2-18
Llama3,2-38
Llama3-8B
Llama2-7B
Llama2-138
FP16
Baseline
13.16
6.139
RTN
1.9c3
882.6
129.1
342.4
227.2
GPTQ
112.0
46.14
8.226
6,719
9,790
INT3
BoA
26.43
13.64
7,782
6.007
5.833
TURBOBOA
19.73
13.12
7,523
5.958
5.288
RTN
6.3e4
2.0e4
6.6e4
7.7c3
5.7e3
GPTQ
538.9
98.19
24.54
30.85
35.08
INT2
BoA
312.2
TURBOBOA
111.3
17.83
C4 PPI. (1)
Precision
Melhod
Llama3.2-1B
Llamm3.2-3B
Lluma3-8B
Llama2-715
Llama2-13B
FP16
Baseline
21.31
16.49
9.444
7.266
6.730
RTN
1.6e3
119.8
GPTQ
201.2
150.8
20.05
92.15
20.
INT3
BoA
37.98
24.05
14.10
8.686
7.634
TURBoBOA
36.43
23.79
13.59
8.554
7.587
RTN
4.6c4
1.1c4
8.204
8.2c3
4.8c3
GPTQ
1.c3
BOA
571.9
214.0
92.69
26.42
28.36
TURBOBOA
313.8
166.6
81.24
17.66
19.89
(c) Zero-shot Accuracy (1)
Precision
Method
Llama3.2-18
Llama3.2-38
Llama3-8B
Llama2-715
Llama2-138
FP16
Baseline
56.82
63.01
70,34
67.28
69.83
RTN
33,19
33,37
36.01
33.18
32.92
GPTQ
37.44
39.19
61.72
58.38
54.84
BoA
47.05
59.38
65.37
63.70
63.35
TURBOBOA
47.46
67.07
RTN
31.08
32.79
GPTQ
30.48
34.39
36.01
42.50
39.08
INT2
BOA
31.33
38.53
42.16
45.81
45.06
TURBOBOA
33,91
42.03
44.87
51.41
47.35
17

---

Published as a conference paper at ICLR 2026
F.2 COMPARISON WITH GPTAQ
To further validate the importance of incorporating inter-channel dependencies, we provide direct comparison between GPTAQ (Li al. 2025) and TURBOBOA. While both algorithms aim com- pensate for quantization errors from preceding layers, they differ fundamentally in their treatment of the out-channel-wis. Hessian Hout- Specifically, while GPTAQ assumes Hoat I. thereby ig- noring the correlations between out-channels, the proposed TURBOBoA explicitly incorporates the attention-aware Hessian ouu (see T to capture these dependencies.
Table reports the performance on Llama3 models under the 2-bit weight-only quantization setting without additional transformations. Across all model scales, TURBOBOA consistently outperforms GPTAQ in both PPL and zero-shot accuracy. Notably, TURBOBOA achieves 7.7%p accuracy gain on Llama-3.2-3B and a 10.5%p improvement on Llama-3-8B compared to GPTAQ. These results highlight that accounting for inter-channel dependencies is crucial for mitigating accuracy degradation aggressive low-bit regimes.
Table 7: Evaluation on Llama3 models (INT2 quantization)
L-mm
Llama3.2-3b
Llama3-8b
Method
Wiki2 (1)
0-shot (1)
Wiki2 (3)
0 sho (f)
Wiki2 (1)
O-shot (f)
GPTAQ
200.5
31.73
47.90
34.31
19.29
34.36
TURBOBOA
111.3
33,91
33.42
42.03
17,83
44.87
18

---

Published as a conference paper at ICLR 2026
F.3 WEIGHT-ACTIVATIN QUANTIZATION PERFORMANCE UNDER HIGHER BIT- WIDTHS
We further report weight-activation quantization results under higher weight bit-widths in Table g In this table, results for OmniQuant excluded because its official implementation does not sup- port models utilizing grouped query attention. As expected, the performance gap among different algorithms narrows this regime, as 4-bit quantization preserves most of the original FP accuracy. Nevertheless, TURBOBOA consistently provides robust improvements in almost all cases, confirm- ing the effectiveness of our method even when quantization is less challenging.
Table 8: Weight-activation quantization performance on transformed Llama3 models
PPL
(L)
L1ama3.2-18
L-3ma3
Llama3-8B
Precision
Transform
Quantizer
Wiki2
C4
Wiki2
C4
Wiki2
C44
FP16
Baseline
13.16
21.31
11.05
16.49
6.139
9.444
DuQuant
RTN
19c4
1.8c4
13.32
19:49
8.066
13:24
GPTQ
16.68
26.87
11.87
19.47
7.636
12.59
SpinQuant
11.57
19.04
7.496
12.35
W4A4KV16
TURBOBOA
16.09
26.12
11.55
19.11
7.474
12.32
GPTQ
16.02
25.26
11.88
18.60
7.349
12.04
OSTQuant
BoA
15.60
24.81
11.74
18.45
7.224
11.82
TURBOBOA
DuQuant
RIN
1.704
1.4c4
13.84
20.52
8.402
13.59
GPTQ
18.31
29.46
12.24
20.21
7,869
12.99
SpinQuant
19.84
7,705
W4A4KV4
TURBOBOA
17.77
28.56
11.88
19.73
7,680
12.70
GPTQ
17.29
28.19
12.64
20.00
7,540
12.42
OSTQuant
27.30
12.43
19.58
7,428
TURBOBOA
16.86
27.10
12.39
19.45
7.416
12.20
(b) Zero-shot Accuracy (t)
Precision
Transform
Quantizer
L.ama3.2-1B
1lama3.2-3B
Llamm3-88
FP16
Baseline
56.82
63.01
70.34
DuQuant
RIN
30.33
57.93
63.15
GPTQ
50.89
58.71
64.79
SpinQuant
BoA
51.76
59.17
65.31
W4A4KV16
59.42
GPTQ
OSTQuant
BoA
53.24
60.94
67.43
TURBOBOA
53.67
61.65
67.88
DuQuant
RTN
30.71
56.53
62,76
GPTQ
48.86
57.54
64.05
SpinQuant
58.90
W4A4KV4
TURBOBOA
50.73
58.77
65.64
GPTQ
50.44
59.34
65.25
OSTQuant
BoA
50.94
$9.66
66.47
TURBOBOA
59.86
19

---

Published as a conference paper at ICLR 2026
F.4 ABLATION THE NUMBER OF CD ITERATIONS
In this subsection, we investigate the impact of the number Thtter of CD iterations (see Algorithm 2 on quantization quality. We focus the attention reconstruction loss || j measured the first Transformer block to avoid confounding effects from previous blocks. The results in Table 9 indicate that the first CD iteration accounts for nearly all the reduction in loss, with additional itera- tions yielding diminishing returns. Accordingly, the end-to-end PPL performance remains virtually unchanged between and 2 iterations. To maintain optimal computational efficiency, we set the CD) iteration count to 1 for all main experiments.
Table 9: Ablation on the number of CD iterations
Model
niter
Loss (Query)
Loss (Key)
Wiki2 (4)
(4)
66.97
92.58
1..ma3,2.18
1
315.9
66.68
33.33
85.55
315.8
66.67
32.28
88.03
0
170.1
70.72
25.92
63.48
Llamm332-338
2
168.7
70.16
24.07
54.53
0
126.1
43.36
14.21
34.67
Llama--88
125.6
43.20
13.54
32.99
2
125.5
43.17
13.46
33.23
20

---

Published as a conference paper at ICLR 2026
G PSEUDOCODE FOR GPTAQ
In this appendix, we provide the pseudocode of the conventional GPTAQ (Li et al. 2025), which is omitted in the main manuscript due the page limitation.
Algorithm 3 GPTAQ
Input: weights W. Hessian information U... deviation correlation R AXX". and scale
1: Initialize quantized and integer weights: Q, Wint <- Odoutxdi..
2: Compute P = (RU,, @ Mu) Un. (Mu: strictly upper triangular masking matrix with ones above the diagonal)
3: for = 0,... ,dn - do
4: Quantize the j-th in-channel:
"wo,] [ineles + clamp (ittgg))--W.)]1....
Q (Weia))
5: Estimate quantization error: E < (W., Q.))/Uumnis
Update remaining in-channels:
W .. Qud [U [Umm]is:
P
Unbs
Output: quantized weights Q, integer weights Wine
21


Reviews:
[{"review_id": "MaMOJty1AG", "reviewer": "Reviewer_Nt2A", "summary": "The paper proposes TurboBoA, an improved PTQ method that accelerates the BoA algorithm while enhancing accuracy. The key innovation is jointly quantizing multiple out-channels simultaneously using a closed-form error compensation rule, achieving 4-6× speedup. The method demonstrates state-of-the-art results on Llama models in both weight-only and weight-activation quantization.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper provides three propositions with closed-form solutions for joint quantization, cross-block error compensation, and adaptive grid selection. The mathematical derivations are rigorous and elegant, enabling efficient backpropagation-free optimization. (Note that I have not checked the math very carefully.)\n\nTurboBoA achieves 4-6× speedup over BoA while improving accuracy. For INT2 quantization on Llama3.2-1B, it reduces Wiki2 PPL from 40.86 to 33.33 while cutting processing time from 13.33 to 5.33 minutes.", "weaknesses": "There's no theoretical bound on accuracy degradation as a function of $N$ (number of jointly quantized channels) and model properties.\n\nThe improvement compared with BoA is not practical. Though it reduces Wiki2 PPL from 40.86 to 33.33 while cutting processing time from 13.33 to 5.33 minutes, but in my opinion, for a LLM PTQ method, the calibration time reduced from 13 to 5 mins, is not a practical improvement. \n\nImportant design choices lack ablation studies, including the number of coordinate descent iterations, the optimal choice of N across different models/bit-widths, and evaluation is limited to only the Llama family of models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r1ADUaS8mx", "reviewer": "Reviewer_W4R5", "summary": "This paper proposes TurboBOA, a faster variant of BOA for LLM PTQ.\n\nThe main contributions are:\n\n1. Quantizing N rows jointly in closed-form instead of row-by-row (BOA)\n\n2. a ΔX-aware error accumulation formulation that incorporates propagated quantization residuals into the next step\n\n3. “adaptive grid selection” = re-selecting quantization scale (hence grid points) conditioned on the updated (error-polluted) input, instead of using the original FP activations. a small closed-form refinement CD update for attention blocks\n\n4. Experiments are on LLaMA variants, reporting 4–6× speedup vs BOA with similar or better PPL.", "questions": "please see weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "the motivation (reduce BOA’s sequential dependency bottleneck, accumulation error, fixed grid) is legitimate and practically important\n\n\nThe methods has good empirical improvement over BOA.\n\nwriting quality is mostly clean", "weaknesses": "Motivation: the method does not completely consider cross-layer dependency, as the objective follows BOA in Table 1. As I understand, instead of cross layer dependency, it seems like sequential optimisation taking into account the interaction of attention mask, especially for the objective of W_q and W_k\n\nTechnical novelty:\n1. the ΔX-aware accumulation term and how the author solve this problem is conceptually very close to GPTAQ-style accumulated loss [1]. Without explicit comparison, this make the contribution less strong\n\n2. the multi-row closed-form step is the mathematically natural extension of the BOA 1-row closed-form, while the adaptive grid selection is very similar to re-estimating scale in GPTQ+finetune regimes so while it is naturally, the technical part is not  too strong for me; is the only purpose of multi-row closed-form to reduce the overhead?\n\nEmpirical evaluation: how is the results and efficiency compared to method like GPTAQ [1]?\n\n\n[1] Li et al. (2025). **GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration**. arXiv:2504.02692.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fBljrSyBxN", "reviewer": "Reviewer_MW4o", "summary": "This paper proposes TurboBoA, a new backpropagation-free, post-training quantization (PTQ) algorithm for large language models (LLMs). The work aims to resolve the critical trade-off between fast but less-accurate PTQ methods like GPTQ, and more accurate but computationally slow methods like BoA. TurboBoA introduces three main innovations: (i) a joint quantization scheme that processes $N$ out-channels simultaneously, enabled by a closed-form error compensation rule (Proposition 3.1), to accelerate the sequential process of BoA; (ii) a correction mechanism to compensate for quantization errors propagated from preceding Transformer blocks (Proposition 3.2), mitigating error accumulation across the model's depth; and (iii) an adaptive grid selection strategy combined with an attention-wise scale refinement step to better align quantization grids with the iteratively updated weights (Proposition 3.3). Experimental results demonstrate that TurboBoA achieves a 4-6x speedup over the original BoA (Table 2) while also consistently improving accuracy. When combined with outlier suppression techniques, the method achieves state-of-the-art results on Llama models in both weight-only (Table 4) and weight-activation quantization (Table 5) settings.", "questions": "* **Expanding the Analysis of $N$.**\n    * Could the authors please provide a justification for the choice of $N=16$ in the main experiments?\n    * It would significantly strengthen the paper to include a brief sensitivity analysis of $N$ on a representative model (e.g., Llama3-8B) for the final TurboBoA method (F1+F2+F3), not just for F1 as in Table 2.\n    * A discussion on the performance/speed trade-off for larger values, such as $N=32$ or $N=64$, would be valuable to understand the practical upper bound of the joint quantization speedup.\n\n* **Clarifying the Runtime of Feature F2.**\n    * To improve clarity, please provide a more direct breakdown of the runtime cost of each component. A small table or paragraph detailing [Time(BoA), Time(BoA+F1), Time(BoA+F1+F2), Time(BoA+F1+F3), Time(TurboBoA)] for one model would make the individual overheads of F2 and F3 explicit.\n    * Please also clarify *why* F2 is computationally expensive. Does computing $\\tilde{X}$ require a full forward pass of the FP model on the calibration data for each Transformer block being quantized?\n\n* **Improving Mathematical Readability.**\n    * For Proposition 3.1 and 3.2, please consider adding a brief note to explicitly define the block matrix notations (e.g., $[U]_{:,B}$ as the submatrix of $U$ taking columns indexed by $B$) directly within the proposition statement to aid the reader.\n    * Please elaborate on the grid initialization step in Algorithm 1, Line 9. How is this $\\min_{s}$ operation performed? Is it a standard min-max clipping based on $W^{(i)}_{update}$, and if so, how is the trace objective minimized?\n    * In Appendix D, please expand the derivation from (Eq 21) to (Eq 22). Showing the specific matrix identity being used (e.g., related to the inverse of a matrix after a rank-1 update, or how $H_{in,-j}^{-1}$ is derived) would make the proof much more self-contained and easier to follow.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* **Clear Motivation and Strong Problem Definition.**\n    * The paper clearly articulates the limitations of existing backpropagation-free PTQ methods, situating the work in a well-understood context.\n    * It correctly identifies a critical trade-off: GPTQ's layer-wise independence assumption leads to high speed but poor accuracy in low-bit regimes, whereas BoA's attention-aware dependency modeling is accurate but suffers from a severe bottleneck due to its sequential processing of out-channels.\n    * TurboBoA is well-positioned as a direct solution to this trade-off, explicitly designed to retain the accuracy benefits of BoA while recovering the efficiency of methods like GPTQ.\n\n* **Novel and Technically Sound Methodological Contributions.**\n    * The primary idea of jointly quantizing $N$ out-channels simultaneously (Section 3.1) is a practical and effective method for acceleration, which is well-visualized in Figure 1.\n    * This acceleration is supported by a non-trivial, closed-form error compensation rule (Proposition 3.1, Appendix C) that correctly incorporates dependencies for the jointly quantized block, which is critical for preserving accuracy.\n    * The introduction of error compensation for *preceding* quantized Transformer blocks (Proposition 3.2, Section 3.2) is a key novelty. It directly addresses the problem of error accumulation across model depth by accounting for the input deviation $\\Delta X = X - \\tilde{X}$.\n    * The two-part adaptive grid selection mechanism (Section 3.3) is also a strong contribution. It first aligns grids to updated weights *before* quantization (Line 9, Algorithm 1) and then refines scales *after* quantization using coordinate descent to minimize the true attention-wise loss (Proposition 3.3, Line 13, Algorithm 1).\n\n* **Comprehensive and Rigorous Experimental Evaluation.**\n    * The paper features an excellent ablation study (Section 4.2) that validates each of the three contributions (termed F1, F2, F3).\n    * Table 2 provides clear evidence for the 4-6x speedup from joint quantization (F1) and confirms that the accuracy degradation from this step alone is negligible.\n    * Table 3 successfully isolates the accuracy gains from F2 (pre-block error) and F3 (adaptive grid), demonstrating their complementary and significant benefits over the baseline.\n    * State-of-the-art comparisons are extensive, covering both weight-only (Table 4) and weight-activation (Table 5) settings across a wide range of Llama models and bit-widths (INT2, INT3, W2A4, W4A4).\n    * The inclusion of results *without* any transformation-based methods (Table 6, Appendix F.1) is a valuable addition, as it isolates the performance of the core quantizer itself, showing TurboBoA still significantly outperforms BoA and GPTQ.\n    * The evaluation metrics are comprehensive, including PPL on two datasets (Wiki2, C4) and average accuracy on eight zero-shot commonsense reasoning tasks.\n\n---", "weaknesses": "* **Analysis of Hyperparameter $N$.**\n    * The paper introduces $N$, the number of jointly quantized out-channels, as a new and important hyperparameter governing the speed/accuracy trade-off.\n    * While Table 2 ablates $N$ for $N \\in \\{1, 4, 8, 16\\}$, all subsequent experiments (Tables 3, 4, 5, 6, 7) appear to use a fixed $N=16$.\n    * The manuscript does not provide a discussion on how $N=16$ was chosen as the default, or how sensitive the final state-of-the-art results are to this specific value.\n    * The trade-off for $N > 16$ (e.g., $N=32, 64$) is not explored. This leaves the full behavior of this parameter uncharacterized, and it is unclear if $N=16$ is optimal across all model sizes and bit-widths.\n\n* **Clarity of Computational Overhead.**\n    * The paper states that the overhead of F2 (error compensation for pre-quantized blocks) is \"not negligible\" because it requires computing the FP representation $\\tilde{X}$ to find the input deviation $\\Delta X$.\n    * While the data in Table 3 supports this (e.g., for Llama3-8B, adding F2 increases runtime from 24.59 min to 37.83 min, an increase of ~13.2 min), the text does not explicitly analyze *why* this cost is so high.\n    * It is unclear if this computation is a one-time cost per model or must be performed per-block, and how this cost scales with model size or sequence length. A more direct breakdown of the runtime cost of each component (F1, F2, F3) would be beneficial.\n\n* **Mathematical Presentation and Notation.**\n    * The notation in the core propositions is dense. For instance, in Proposition 3.1 (Eq 5) and 3.2 (Eq 8), block matrix notations like $([U_{out}^{T}]_{B,B})^{-1}$ are used. While a general notation guide is provided (Section 1), these specific expressions are complex and could benefit from an explicit definition in the context of the proposition for easier readability.\n    * The scale-setting step is given in Algorithm 1, Line 9. This is part of the \"adaptive grid selection\" (F3), but the manuscript provides no derivation or explanation for how this $\\min_{s}$ operation is performed, in contrast to the detailed derivation for the \"attention-wise refinement\" part of F3 (Proposition 3.3, Appendix E).\n    * The proof for Proposition 3.2 (Appendix D) involves a significant leap from (Eq 21) to (Eq 22). This step relies on replacing a term with $H_{in}^{-1} - H_{in,-j}^{-1}$ and \"exploiting the properties of Cholesky decomposition\". This derivation is non-trivial and would be much easier to verify if the intermediate steps or the specific matrix identities (e.g., Sherman-Morrison) were provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kr5Mg2mBEi", "reviewer": "Reviewer_EjSS", "summary": "This paper presents TurboBoA, a post-training quantization (PTQ) framework that enhances the BOA algorithm by introducing attention-aware optimizations that accelerate quantization while preserving accuracy. The key idea is to eliminate sequential dependencies in channel-wise quantization through a multi-channel parallelization strategy, extend error compensation across transformer blocks, and introduce adaptive grid selection for attention-wise refinement. The method is designed to operate without backpropagation which allows faster quantization on large-scale language models such as Llama3 and Llama2 variants. Experiments on WikiText2 and C4 datasets under different precision settings (INT2, INT3, W4A4KV4, and W4A4KV16) demonstrate that TurboBoA consistently achieves lower perplexity and higher zero-shot accuracy compared to GPTQ, SpinQuant, DuQuant, and BOA.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.This paper addresses a practically relevant problem in LLM quantization, focusing on bridging the accuracy-efficiency trade-off.  \n\n2.The algorithmic improvements are well-motivated, clearly described, and supported by mathematical derivations and pseudocode for reproducibility. \n\n3.The evaluation is extensive. It covers multiple model scales and both weight-only and weight-activation quantization. \n\n4.TURBOBOA achieves notable runtime improvements while maintaining accuracy. \n\n5.The combination with outlier suppression methods (QuaRot, SpinQuant, OSTQuant) highlights good modularity and compatibility with transformation-based approaches.", "weaknesses": "1.The paper lacks a rigorous theoretical justification for why the proposed joint quantization maintains accuracy despite reduced error-compensation flexibility. The reasoning remains heuristic without curvature analysis or theoretical bounds on error propagation. \n\n2.The novelty is limited. The method largely builds upon established concepts in Hessian-guided PTQ, extending BOA with more efficient update rules and adaptive scaling. The work feels more like a refined engineering optimization than a fundamentally new quantization paradigm. \n\n3.The experimental section is comprehensive but remains narrow in scope. The work focused almost entirely on the LLaMA family. There is no evidence the approach generalizes to architectures with different attention patterns. \n\n4.The discussion on computational overhead is insufficient. The algorithm introduces multiple Cholesky decompositions and iterative grid refinements, but the additional cost versus BOA or GPTQ is not systematically quantified. \n\n5.Although results show improvement in perplexity and zero-shot accuracy, the absolute performance gains are modest in some configurations. That could be due to diminishing returns relative to complexity. \n\n6.The broader claims of “state-of-the-art quantization” (p9, l470-471) could be better substantiated through comparisons with more recent PTQ or hybrid fine-tuning approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "44aLRdJMJ8", "reviewer": "Reviewer_MvrE", "summary": "This paper addresses the challenge of efficiently performing post-training quantization for Transformers, especially attention layers, without backpropagation, while maintaining high accuracy at extremely low bit-widths. It proposes TURBOBOA, which jointly quantizes multiple output channels with closed-form error compensation, incorporates cross-layer error correction, and adaptively refines quantization grids. Experiments on LLaMA models show that TURBOBOA achieves 4–6× speedup over BOA while improving perplexity and zero-shot accuracy, particularly in 2-bit and 3-bit settings. The results demonstrate that TURBOBOA effectively balances efficiency and precision, making low-bit quantization more practical for large-scale Transformers.", "questions": "Please refer to the weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper clearly identifies the limitations of existing post-training quantization methods, particularly the trade-off between accuracy and efficiency in attention-aware quantization.\n\n- TURBOBOA introduces joint channel quantization with closed-form error compensation, cross-layer error correction, and adaptive grid refinement, effectively addressing accuracy loss in low-bit quantization.\n\n- The method is evaluated on multiple LLaMA models with various bit-widths and settings, showing significant speedup and improved perplexity and zero-shot accuracy.\n\n- The paper successfully demonstrates that TURBOBOA achieves a practical trade-off between computational efficiency and quantization accuracy for large-scale Transformer models.", "weaknesses": "- The method involves multiple closed-form derivations and joint updates, which may increase implementation and debugging difficulty.\n\n- Cross-layer error compensation requires extra intermediate computations, which could be non-negligible on very large models or resource-constrained hardware.\n\n- The choice of the number of jointly quantized channels (N) can affect both speed and accuracy, and may require tuning for different models or layers.\n\n- This paper emphasizes that backpropagation is not required, but in practice it still involves a considerable amount of computation. In contrast, EfficientQAT[1] achieves significant performance improvements with minimal cost. Could you elaborate again on the advantages of being backpropagation-free? Generally, we still aim to obtain models with higher accuracy benefits.\n\n[1] Chen, Mengzhao, et al. \"Efficientqat: Efficient quantization-aware training for large language models.\" arXiv preprint arXiv:2407.11062 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)