GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers
Authors: Guang Liang
Xinyao Liu
Jianxin Wu
Venue: NeurIPS 2025 poster ()
Keywords: quantization-aware training
vision transformers
practical QAT
Primary area: applications
TL;DR: A practical and super-fast QAT method for general vision applications; source for recognition and detection will be released.

Abstract: Vision Transformers (ViTs) are essential in computer vision but are computationally intensive, too. Model quantization, particularly to low bit-widths like 4-bit, aims to alleviate this difficulty, yet existing Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) methods exhibit significant limitations. PTQ often incurs substantial accuracy drop, while QAT achieves high accuracy but suffers from prohibitive computational costs, limited generalization to downstream tasks, training instability, and lacking of open-source codebase. To address these challenges, this paper introduces General, Practical, and Lightning Quantization (GPLQ), a novel framework designed for efficient and effective ViT quantization. GPLQ is founded on two key empirical insights: the paramount importance of activation quantization and the necessity of preserving the model's original optimization basin to maintain generalization. Consequently, GPLQ employs a sequential activation-first, weights-later strategy. Stage 1 keeps weights in FP32 while quantizing activations with a feature mimicking loss in only 1 epoch to keep it stay in the same basin, thereby preserving generalization. Stage 2 quantizes weights using a PTQ method. As a result, GPLQ is 100x faster than existing QAT methods, lowers memory footprint to levels even below FP32 training, and achieves 4-bit model performance that is highly competitive with FP32 models in terms of both accuracy on ImageNet and generalization to diverse downstream tasks, including fine-grained visual classification and object detection. We release an easy-to-use open-source toolkit supporting multiple vision tasks at [GPLQ code](https://github.com/wujx2001/GPLQ).

GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers
Guang Liang .2.4
Xinyao Liu Jianxin Wu
National Key Laboratory for Novel Software Technology. Nanjing University, China School of Artificial Intelligence, Nanjing University, China
University of Science and Technology of China, Hefei, China 'Zhongguancun Academy, Beijing, China lianggOlamda eju.edu..... edu. cn, liurinyao@mail. wsw.cstc.ee edu. en, manix200@@jju...m edu. cn
Abstract
Vision Transformers (ViTs) are essential in computer vision but are computa- tionally intensive, too. Model quantization, particularly to low bit-widths like 4-bit, aims to alleviate this difficulty, yet existing Post-Training Quantization (PTQ) and Quantization-Awan itations. PTQ often incurs substantial accuracy drop, while QAT achieves high accuracy but suffers from prohibitive computational costs, limited generalization to downstream tasks, training instability, and lack of open-source codebase. To address these challenges, this paper introduces General, Practical, and Lightning Quantization (GPLQ), a novel framework designed for efficient and effective ViT quantization. GPLQ is founded on two key empirical insights: the paramount importance of activation quantization and the necessity of preserving the model's original optimization "basin" to maintain generalization. Consequently, GPLQ em- ploys sequential "activation-N fiiss,, weights-later" strategy. Stage keeps weights in FP32 while quantizing activations with feature mimicking loss in only I epoch to keep in the same "basin", thereby preserving generalization. Stage 2 quantizes weights using a PTQ method. As result, GPLQ 100x faster than existing QAT methods, lowers memory footprint to levels even below FP32 training, and achieves 4-bit model performance that is highly competitive with FP32 models in terms of both accuracy on ImageNet and generalization to diverse downstream tasks, including fine-grained visual classification and object detection. We release an casy-to-use open-source toolkit supporting multiple vision tasks at https:///githu. wtt.t/uu..cc//mmxx2000////
Training (QAT) methods exhibit significant lim-
Introduction
Vision Transformer (ViT) 6 has emerged as the mainstream backbone network in computer vision, but it demands substantial computational and memory resources. Model quantization is one of the key techniques to address this challenge by reducing the numerical precision of model parameters and/or activation values 18 19]. However, existing quantization methods still faces challenges, especially in low-bit (e.g., b-tit) quantization.
Mainstream methods include Post- Best-Training Quantization (PTQ) 25 and Quantization-Aree Train- ing (QAT) (7). PTQ has fast speed and low resource consumption, but often leads to large accuracy drop under 4-bit quantization [20]. On the other hand, QAT simulates quantization operations during training and enables higher accuracy than that of PTQ, even higher than that of floating-pol -ooin models. Nevertheless, in this paper we will show that existing QAT methods have inherent limitations:
*Corresponiing author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Training Time Memory Tasks Aeg
(min)
106
3000
--f--
17000
GPLQ (burs)
Swinl
139
80.82
Swit-B
78.25 %
GPLO
Swin-8
GPIQ (burs)
68.60%
GPLQ (Durs)
Swin-T
RTX 0090 (2GG
70.77 %
Swit-T
OFQ
60073
Swis-T
Q-Var
68.12%
Swis-T
Full-grecision
70.79
(GB)
15
20
25 30
Figure 1: Core advantages of our GPLQ: Generality, Practicality, and Lightning efficiency.
High Computational Costs. QAT requires lengthy fine-tuning of the entire model. Training time and GPU memory required in QAT often far exceed those for training the FP32 model [8]. This makes QAT cumbersome and very slow for deployment in real-world applications.
Limited Generalization Ability. QAT methods often boast higher accuracy than their FP32 counterparts. However, in this paper we will show that such models are generalizing worse than FP32 or PTQ quantized models in downstream tasks. That is, they are likely non-generallzabbl beyond ImageNet 4 the dataset on which they were trained.
Training Instability and Complexity. QAT is prone to training instability [15), and complex Knowledge Distillation (KD) techniques [19 [S] severely increase memory footprint. Some also rely on external, extremely powerful teacher models, which are not available in practical scenarios. In short, existing QAT methods are not practical.
Classification Only and Code Missing, Open-source code for QAT is rare, and is only for classification when it exists. This further makes QAT impractical for real-world applications.
To this end, we propose GPLQ (General, Practical, and Lightning Quantization). The core objective efficient than traditional QAT,
of GPLQ is to provide quantization solution that is far more superior to PTQ in accuracy and generalization, easy to use, and highly practical. As a result, Figure [ |emonstraaess core advantages GPLQ.
General. GPLQ exhibits excellent average accuracy multiple downstream tasks: close to or even surpassing FP32 models, and significantly outperforming existing QAT methods.
Practical. GPLQ has very small training memory footprint (far lower than existing QAT methods), which avoids out-of-memory (OOM) issues in many applications and enables quantization of larger models. GPLQ''s design allows it to be conveniently applied to other tasks such as object detection.
Lightning. GPLQ blazingly fast: hundreds of times faster than existing QAT methods.
empirical findings. First, activations are far more important than weights in low-bit quantization. Second, quantization should not change its optimization "basin" (i.c., avoid jumping out of the current local minimum) in order to keep the generalization ability.
GPLQ is based
Based on these findings, GPLQ adopts sequential quantization paradigm. First, activations are quantized with weights kept at FP32. To maintain generalization, we draw inspiration from TCS [40] and employ a PCA-based feature mimicking loss to guide the quantized model's feature outputs to approximate those of the original FP32 model (i.e., stay in the same basin). Second, after activations are quantized, existing efficient PTQ methods are used to quantize the weights. This "activation-fire weights later" strategy not only drastically reduces QAT training time from days to I hours and

---

with memory footprint even lower than FP32 training, but also allows a 4-bit model to achieve both accuracy and generalization nearly identical to the original FP32 model. The main contributions are:
1. Insights. We reveal that activation quantization is the main bottleneck in QAT, and staying in the original optimization basin is crucial for generalization.
2. GPLQ. We propose "activation- first" sequential quantization: first optimize activations then quantize weights via PTQ.
3. Code. GPLQ provides easy-to-use quantization tool supporting classification, detection and other downstream tasks. We will open-source GPLQ upon paper acceptance.
2 Related Work
Model quantization aims to enhance model efficiency by reducing the numerical precision of weights and activations in neural networks (29)
Post-Training Quantization (PTQ). PTQ operates without retraining, requires only small cal-
ibration set, and is very fast. Various techniques have been proposed: AIQVIT [16], GPTQ [8]
PTQ4ViT (3)), SmoothQuant [6). AWQ (2T). More methods like RepQ- ViT 20 and QwT
perform optimization through scale reparameterization and lightweight compensation modules, respectively. Aceuracy degradation remains a severe challenge in low-bit scenarios.
The second stage of GPLQ employs PTQ to quantize weights. Since activations have been quantized via QAT in the first stage, PTQ's duty changes from W32A32 - W4A4 W32A4 + W4A4
Training (QAT). QAT introduces simulated quantization during training or fine-tuning, and achieves higher accuracy than PTQ. It often uses
Quantization-Awee
Estimator (STE) to handle gradient 77] Research directions include learning quantization scales [7 improving
training stability) (OFQ (23] J. Quantization-Vatti
[15), enhancing efficiency (EfficientQAT (3),
and specific optimizations for ViTs (c.g.. Q-ViT 19, PackQViT (5)).
The bottlenecks of QAT are high computational cost, training instability, and potential degradation in generalization ability. GPL.Q, with an extremely short QAT stage (only epoch) focused solely on activations, effectively alleviates the cost, stability, and generalization issues of traditional QAT.
is often used in QAT to learn from the FP32 model or even a much stronger, external teacher. Researchers have proposed methods (DGD 39 MCKD [5] that are both heavy and complex. TCS 44 offers an efficient approach by capturing the linear subspace of the teacher model's 's features through Principal Component Analysis (PCA) for knowledge transfer. GPLQ draws inspiration from TCS, aiming at efficiently transferring knowledge to maintain generalization and avoiding the high costs of complex distillation.
Knowledge Distillation (KD) in QAT. KD
35
41
3 Methodology
GPLQ is directly derived from two empirical insights in the ViT quantization process.
3.1
Activations are Crucial & Stay in the Same Basin
Our first empirical insight is that activations are more critical than weights quantization. Taking a ViT network such as DeiT (32] pre-trained on ImageNet-1k 1k 4 we independently applied 4-bit PTQ (using a percentile-base based per-channel quantization calibration method) to either weights (with activations kept at FP32) or activations (with weights kept at FP32). Figure 2 shows the results, which consistently indicates that quantizing activations to 4 bits (weights at FP32) leads to larger Top-1 accuracy drop than quantizing weights to 4 bits (activations at FP32). That is, activations face more severe challenges under low-bit quantization compared to weights. This finding leads to our "activation-fire first" quantization strategy in GPLQ.
QAT methods heavily adjust their weights, and make them stay in dramatically different basins (local minima) in the loss landscape before and after QAT learning. That is, they are significantly different from the initial FP32 model. Although they achieve high accuracy on the pre-training ImageNet data, observe that this aggressive retraining weakens the transferability of the learned representations
3

---

DeiT-small-distie
DeiT-base-distilld
Swin-S
Swin-B
Frecep (%6
16
Type arthvation
Quant. Type
lype
Type artivation
00
Low Bit Reglon
Low Bit Reglon
activation
Low Bit Region
activation
Low Bit Region
30
-d-- weight
-a- weight
weight
-a- weight
Bit
Bit
Bit
Figure 2: Impact of quantizing weights and activations separately.
Wernz
Wiraz
Wyz12
Quant
WinTa
W
WWiights
Quann
Activations
Ayysz
Awas
AysT4
Aprr
Care Festure PCA
Teacher Model
Sturket Modd
Model
Model
Distiling
Training
QAT
PTQ
Figure 3: Overview of GPLQ: QAT stage first only for activations, then PTQ stage only for weights.
to downstream tasks (cf. Table Figure D. Our second insight is to make the quantized model stay in the same basin. This is achieved in GPLQ by restricting QAT to optimize only activation quantizers: ii) use a low learning rate and only epoch training in this QAT stage; iii) feature mimicking loss that encourages the quantized model retain the key feature structures of the original FP32 model.
3.2 The GPLQ Framework
3.2.1
Stage I: Activation-only Quantization-Waan Training, or Act-QAT
This stage only quantizes activations. The key is that all model weights are learnable but kept at FP32 precision, thereby decoupling activation quantizer learning from weight updates and effectively circumventing weight oscillation that occurs in QAT which quantizes the two simultaneously.
Activations employ the uniform symmetric quantization implemented with per-channel gramularity. A comparison of per-tensor versus per-group quantization is detailed in the appendix. Activations are quantized quantization, too. The quantization scaling factor 8a is learned using LSQ (7]. One novel proposal in GPLQQ is that we initialize 8, and calibrate on a small subset of training data using a percentile- based min-max PTQ method. For internal per-token quantized operations (if applicable), the initial quantization range is set based on the 1st and 99th percentiles of observed activation values to mitigate outlier effects; for per-channel quantization, min-max values used directly. This initialization process is fast (typically in seconds) and provides a better starting point for subsequent LSQ-like optimization. The quantized activation å for input is caleulated as follows:
bits (b 4) in our experiments, but our method and code work for lower-bit
f = clamp(round((//>..
2 2 1) x 8a
where & = 4 in our experiments. The gradient of 8, is estimated using (STE).
Estimator
To preserve the rich representational power of pre-trained models (i.c., keep the generalization power), we follow lightweight feature mimicking loss inspired by TCS 40) We chose the penultimate layer based on two considerations: 1) High-level semantics: This layer's features contain the highest level of semantic information, crucial for generalization. 2) Effficincy: Constraining features from single layer minimizes computational overhead, aligning with our "lightning-fase fast" principle. We first extract features Jt from the penultimate layer of the original FP32 teacher model. Then, PCA is performed on set of teacher features F. = (Se, obtained from subset of the training data. The resulting principal components V define low-dimensional subspace that captures the main information in the teacher's feature manifold, which is consistent with findings that transformer

---

features are inherently low-rank (37. We select principal components that explain a majority of the variance in F. (c.g., approximately 60%). and for hardware friendliness, adjust the number of selected principal components to be multiple of 32. Specifically, for Swin-T [24] with 768 dimensions, 256 are selected; for DeiT-T [2] with 192 dimensions, 64 are selected. The corresponding features fs of the student model (the model undergoing Act-QAT) are projected onto this PCA-defined subspace.
The loss APCA is defined as the Mean Squared Error (MSE) between the student's projected features and the teacher's 's projected features, i.c., matching projections in the PCA space:
N LPCA
EIICE - Mc) - -ft- mc) Veell2 N
1 E=1
(2)
where Pe is the mean of the teacher features in Fú. Veet are the selected principal components, and N is the number of samples (e.g., batch size). Here, Mr is the mean of the teacher features and cannot be eliminated, In PCA projection, data must centered by subtracting the mean before projection. Both student (f') and teacher (fi) features are centered using the teacher's 's mean (14,) to ensure they are compared in the same coordinate system. This loss encourages the post-quantization activations to compactly retain salient features of the FP32 teacher model, thereby enhancing generalization. In our experiments, the weight ratio of the reature-mimickee loss (LPCA) to the original classification loss was set to 1:1, avoiding an additional hyperparameter. An ablation study on this weight is provided in the Appendix.
3.2.2 Stage 2: Post- Sea-TTrainin Quantization of Weights, or Weight-PTQ
Now the model has FP32 weights and 4-bit quantized activations (W32A4), The second stage rapidly quantizes the weights to bits using mature PTQ techniques to generate the final W4A4 model. We leverage existing efficient PTQ methods in this stage. Specifically, after quantizing the weights using RepQ-Vit ViT [20). we further apply the QwT (Quantization without Tears) [[] method to compensate for the accuracy loss introduced by weight quantization. The entire calibration process is completed on a small, randomly selected subset of the ImageNet training data.
With activation quantizers frozen, we significantly simplify PTQ of weights. Since activations are already fixed and quantized to bits (A4), error coupling that hurts weight PTQ algorithm is greatly reduced. The primary source of error is weight quantization itself (W32 - W4), rather than the compound error simultaneous W32 + W4 and A32 + A4 conversions traditional PTQ. This simplified objective makes the compensation technique (QwT) easier to perform. QwT corrects quantization error by introducing a lightweight linear compensation layer, whose parameters W' are determined by the following closed- form solution:
(3) which includes a regularization term AI for stability. In our W32A4 setting, X2 represents the input activations that have already been quantized to 4 bits, Y is the output of the layer with FP32 weights and A4 inputs (Yw32A1), while Yz the output with 4-tit weights and A4 inputs (YwLa1).
W" = (Y - Y2) xx + AT)-
3.3 Advantages of Our GPLQ
GPLQ is designed with practical ease of use in mind, and we provide an implementation that encapsulates the two-stage process into simple and easy-to-use code.
The principles and effectiveness of GPLQ are not limited to image classification. We extended the code to object detection on MS-COCO 22 For COCO tasks, Act-QAT (Stage 1) is also performed for 1 epoch, but due to larger input resolutions and higher model complexity the batch size per GPU is adjusted to 1. The subsequent weight PTQ (Stage 2) follows similar procedure to that for image classification. This demonstrates the good generality of our framework across diverse vision tasks.
Compared to traditional QAT and PTQ methods, GPLQ offers an attractive alternative. Requiring only I epoch of activation QAT, its training duration and required computational resources are far less than those of typical full QAT process (e.g., hundreds of times faster). Furthermore, pre- quantized activations create a more tractable optimization problem for subsequent weight PTQ, thereby enhancing the effectiveness of methods like QwT and the weight quantization in RepQ-ViT. Finally, our design avoids jumping out of the FP32 model's 's local minima, thus useful for preserving the generalization ability of the pre-trained FP32 model.
5

---

Table 1: Comparison (Top-1 accuracy in percentage) between GPLQ and SOTA QAT methods on ImageNet- Ik and downstream FGVC tasks. 'Avg Task' is the average accuracy on 5 FGVC tasks.
Network Method
Mode
ImageNet Aircraft Food101
Flowers102
Cars
Avg Task
FP32
W32A32
81.2
39.72
73.85
91.10
93.21 56.06
70.79
OFQ
W4A4
81.9
26.58
64.79
84.40
91.74 36.13
60.73
Swin-T
Q-Var
16
W4A4
82.4
37.02
70.98
87.15
92.86
52.57
68.12
RepQ-VIT VIT
20
W4A4
73.0
35,46
60.59
86.83
88.74 42.33
62.79
GPLQ
W4A4
79.5
40.80
73.15
93.10
93.32
53,46
70.77
FP32
W32A32
83.2
38.13
72.63
91.32
92.80
55.48
70.07
Swin-S
RepQ-ViT
20
W4A4
71.9
32.16
69.44
90.40
93.08
49.27
66.87
GPLQ
W4A4
82.0
35.73
74.56
91.75
93.49
52.42
69.60
FP32
W32A32
85.3
49.71
85,12
99.48
94.49 65.86
78.93
Swin-B
RepQ-ViT
20
W4A4
69.0
44.85
63.44
95.74
89.13
59.25
70.48
GPLQ
W4A4
84.0
49.65
83.81
99.56
94.03 63.91
78.25
FP32
W32A32
86.3
51.52
87.10
99.63
94.93 71.25
80.89
Swin-I.
RepQ-ViT
20
W4A4
83.2
52.66
84.84
99.46
94.36 67.31
79.73
GPLQ
W4A4
85.3
52,97
87.66
99.58
94.71
69.17
80.82
FP32
W32A32
81.2
34.41
64.92
87.36
91.85 50.50
65.81
OFQ
23
W4A4
81.1
29.07
65.88
83.61
92.10 42.31
62.59
DeiT-S
RepQ-ViT
20
W4A4
72.7
26.55
57.68
85.28
89.67 40.59
59.95
QwT
9
W4A4
74.8
35.61
61.23
87.75
88.93 48.09
64.32
GPLQ
W4A4
78.8
39.81
66.55
89.77
92.45 50.55
67.83
FP32
W32A32
83.3
45.06
72.96
91.84
93.35 63.93
73.43
RepQ-ViT 20
W4A4
76.3
48.90
69.47
93.35
92.56 62.39
73.33
DeiT-B
QwT 9
W4A4
78.5
49.23
73.37
93.85
92.59 65.96
75.00
GPLQ
W4A4
82.0
50.89
73.37
94.28
93.27 64.98
75.36
4
Experiments
We conducted comprehensive evaluation of GPLQQ multiple benchmark datasets and vision tasks. For image classification, we use ImageNet- for pre-training and primary performance evaluation. For object detection and instance segmentation tasks, employ the COCO 2017 [22) dataset, by training models on the 'train2017' set and reporting performance on the 'va12017' set.
To evaluate the model's generalization ability, also selected five commonly used Fine-Grained Visual Classification (FGVC) datasets, including Aircraft [27], Food101 22, Flowers (28), Pets (30] and Cars [17]. To fairly compare the feature extraction capabilities and downstream generalization performance, we train models using linear probing. Furthermore, for a fair comparison, the hyperparameters used for all methods were kept consistent following DTL learning rate 0.001, batch size of 64, and drop path rate 0.
100 epochs,
Stage (Act-QAn QAT) trained for 1 epoch on ImageNet- 1k (classification) COCO (detection) using AdamW (26] with fixed learning rate x 10-6 and no decay. Activations use per-channel sym- metric 4-bit quantization. The subspace dimension used for PCA feature mimicking is dynamically selected based on the model's feature dimension, with the selection primarily based on accumulated variance when the accumulated variance is around 60%. Specifically, Swin-T uses 256-dimensional PCA subspace, and DeiT-T (with 192 dimensions) uses a 64 dimensionnl subspace. Training was conducted on 8 GPUs, with batch size of 16 per GPU. This configuration allows quantizing of the entire Swin Transformer series even on consumer grade GPUs.
Stage 2 (Weight-PTQ) employed percentile- based per-channel symmetric 4-bit quantization for weights, combined with QwT [9] for compensation. The calibration set consists of 512 randomly selected images from the ImageNet training set.
4.1 Image Classification Performance
We evaluated GPLQQ on ImageNet- k and five downstream fine-grained classification tasks. The results shown in Table from which observe that:

---

Table 2: Object detection and instance segmentation results (AP / /Apmask).
Method
Bits (W/A)
Swin-T (Ix)
Swin-T (3x)
Swin-S (3x)
Full-Precision
32/32
0.426 /0.393
0,460 / 0,416
0.485 / /0.43
PTQ4ViT (3
4/4
-/-
0.069 / 0.070
0.267 0.266
APQ-VIT 7
4/4
-
0.237 0.226
0.447 0,440
RepQ-ViT VIT
[0
4/4
0.135 10
0.361 / 0,360
0.426 0,400
GPLQ (Act-QAT only)
32/4
0.397 / 00.881
0,430/ / 0,402
0,457 0.4421
GPI.Q
4/4
0.379 / 0.368
0.401 / 0.389
0.434/ / 0.413
1. On ImageNet Itself. GPLQ significantly outperforms PTQ methods (RepQ-ViT and QwT). On Swin-T, GPLQ is 6.5% higher than RepQ-vit ViT and only 1.7% lower than FP32. For DeïT-S, GPL.Q also far surpasses RepQ-ViT and QW'. Compared QAT methods (OFQ, Q-Var), GPLQ is slightly inferior to these QAT models. But, these QAT methods not only require unacceptable training time but may also lead to overfitting (which we discuss next).
2. Downstream Task Generalization: In terms of average downstream accuracy, GPLQ almost always surpasses other quantization methods. And consistently achieves better generalization than FP32. Swin-T GPLQ'ss average downstream accuracy is 70.77%, nearly identical to FP32's s 70.79%, and far exceeding Q- Q-Var (68. 12%) and OFQ (60.73%), We want emphasize that QAT models, although exhibiting highest accuracy on ImageNet, lag behind PTQ methods in terms of generalization (downstream accuracy). On the other hand, our GPLQ has clearly better generalization than PTQ methods, despite being QAT method.
3. Due to the limited open-source availability of QAT methods, the number of compared QAT methods is small. And, QAT methods compared in this paper are only for small models, because they are out-of-memonn even for medium size models. Our GPLQ scales to large models and will be open-source.
4. D-Var performs better than OFQ in generalization, because it uses an external EfficientNet- L2 (88.2% accuracy on ImageNet) pre-trained on JFT-300M as a teacher. Finding an equally powerful teacher in other tasks is impossible or difficult and limits its application scenarios.
In short, our GPLQ quantizes super-fast has both high accuracy and excellent generalization.
4.2 Object Detection Performance
We also evaluated GPLQ for object detection using the Mask R-CNN 12 framework on COCO 2017. Here GPLQ did not employ PCA feature mimicking. Results are shown in Table
For a fair comparison, our weight quantization uses the same weight quantization method as in RepQ-ViT to complete the second stage. Although there is some degradation compared to W32A4 (our Stage model), GPLQ still significantly outperforms other W4A4 methods such as RepQ- RiT, PTQ4ViT, and AIQ-VIT. On Swin-T (3x), GPLQ achieves 0.401 Aph while RepQ- ViT only achieves 0.361. Even without using PCA feature mimicking, the core two-stage idea of GPLQ still demonstrates strong competitiveness in object detection tasks, which shows its generality.
4.3 Ablation Studies
We first investigate the impact of different activation quantization granularities (per-channel vs. per-layer) during the Act-QAT stage. As shown in Table 3 using per-channel activation quantization consistently outperforms per-layer quantization in both ImageNet and average downstream accuracy. Notably, even with per-layer quantization, our GPLQ can still achieve generalization performance close to the original floating-pon -pint model in most cases. For example, Swin-T (Layer- Layer-wise) achieved an 'Avg Task' of 68.7%, only 2.1% 5% lower than FP32's 70.8%. Per-channel quantization, meanwhile, maintained generalization performance almost identical to that of FP32 across all models.
Contribution of different components in the second stage of GPLQ. We first use the first stage Act-QAT to obtain a W32A4 model. Then, we apply different PTQ strategies to the weights of this model. Results are shown in Table 4 It can be observed that even when only using the basic
7

---

Table 3: Ablation on Act-QAT granularities.
Table 4: Ablation on weight-PTQ components.
Model
Prec. (W/A)
ImageNet
Avg Task (%)
Model (Swin)
Operation
Prec.
ImageNet Top- Acc. (%)
Avg Task (%)
(Swin)
Top-1 Acc. (%)
(W/A)
Swin-T FP32
W32A32
81.2
70.8
Swin-T
FP32
W32A32
81.2
70.8
Channel wise
W4A4
(-1.7)
70.8 (=)
GPLQ Act-QAT
W32A4
80.2 (-1.0)
71.1 (+0.3)
Layer wise
W4A4
77.9 (-3.3)
68.7 (-2.1)
+ PTQQ weight
W4A4
79.0 (-2.2)
69.0 (-1.8
Swin-S FP32
W W32A32
83,2
70,1
+WwT
W4A4
79.5
70.8 (=)
Channel wise
W4A4
(-1.()
69.6 (-0.5)
Swin-S FP32
W32A32
83.2
70.1
Layer wise
(-.3))
67.8 (-2.3)
GPLQ ActQQAT
W32A4
(2.4(-0.8)
70,5 (+0.4)
Swin B FP32
W32A32
85.3
78.9
+ -TQ weight
W4A4
(1.6
68.9 (-1.2)
Channel wise
W444
(-1.))
(-0.7)
QwTT (9]
W4A4
82.0
69.6 (-0.5)
Layer wise
W4A4
82.7 (-2.5)
72.4 (-6.5)
Swin-B FP32
W32A32
85,3
78,9
Swin-L
W32A32
86,3
80,9
GPLQ Act-QAT
W32A4
(4.4
78.8 (-0.1)
Channel wise
(-1.0)
80.8 (-0.1)
+ +PTQ weight
W4A4
(-1.
77.3 (-1.6)
Layer wise
W4A4
(-.94)
79.0 (-1.9)
+WWT
W444
(-10.)
78.2 (-0.7)
Table 5: Optimizing basin retention.
Table 6: PCA Dimensionality.
Quantized model ImageNet(<6)
Internal FP32 ImageNel(%)
Downstream Avg($))
Method
PCA dim
ImageNet
Avg Task Top-1 (%)
Medthod
(%)
Raw FP32
81.2
81.2
70.79
w/o PCA
79.4
69.25
Q-Variation 15
82.4
68.9
68.12
w/ PCA
64
79.5
70.12
GPLQ
79.5
81.1
70.77
W/PCA
256
79.5
70.77
W/ PCA
512
79.4
70.36
percentile-based PTQ method to directly quantize weights to 4 bits (PTQ weight), the resulting W4A4 model is clearly worse than the W32A4 model, but is already within an acceptable range, indicating that our Act-QAT lays good foundation for subsequent weight quantization. Furthermore, when QwT is applied for compensation after weight quantization (++QwT"), the model's accuracy loss is significantly mitigated. The W4A4 model with QwT achieved ImageNet accuracy close to W32A4, and its average downstream accuracy was almost identical to the original FP32 model. This demonstrates the effectiveness of our two-stage method, where activations are first stabilized, then weights are quantized and supplemented with lightweight compensation.
Importance of Preserving Optimization Basin for Generalization. We designed an experiment to validate our hypothesis: if QAT methods significantly deviate from the original FP32 optimization basin, generalization ability may be impaired. We compared the FP32 model, QAT method (using Quantization-Varian Variation as an example), and our GPLQ. For QAT and GPLQ, we extracted their trained FP32 weights (i.e., remove quantization nodes and use the learned "latent" FP32 weights, denoted as 'Internal FP32') and evaluated their performance on ImageNet.
As shown in Table 5 although Quantization- Variation achieved high quantized model accuracy on ImageNet (82.4%) (%). the performance of its internal FP32 weights dropped significantly to 68.9%, far below FP32's 81.2%. This indicates that, the quantized weights have significantly deviated from the original optimization basin, potentially led to overfitting to the ImageNet train and val set, and ultimately impaired its generalization ability downstream tasks (Avg Acc. 68. 12%). Figure 4a compares the loss curves of GPLQ and traditional QAT. GPLQ exhibits a smooth convergence process without severe oscillations, indicating that it remains within the original optimization basin of the FP32 model. In contrast, QAT's loss soars to the level for a randomly initialized network (above 7.0) at the very beginning of training it jumped out of the original local minimum and started to fit the specifics of ImageNet, thereby affecting final generalization adversely.
In contrast, GPLQ quantized model has an accuracy of 79.5%, and the accuracy of its extracted FP32 weights 81.1%, which very close to the original FP32 model. This fact strongly suggests that GPLQ successfully stays near its original optimization basin. This strategy not only achieves high accuracy on ImageNet, more critically, maintains excellent downstream task generalization (Avg Acc. 70.77%) matching the FP32 model.
Impact of PCA Dimensionality in Feature Distillation. To further investigate the effectiveness of the PCA feature distillation module in GPL.Q, we conducted ablation experiments specifically on the choice of PCA projection dimensionality on the Swin-T model. Except for whether PCA projection learning was used, all other settings followed GPLQ's default settings. We report its Top-1 accuracy
8

---

Swin-T (Up to 768 Camponents)
Escaped
GPLQ
Cumulative Explained Variance
11.0
Normal QAT
6-
0.8-
5120: 87.7%
Loss
0.6
2560: 61.3%
0.4
128D: 39.4%
2-
0.2
66D: 24.1%
Cumulative Explained Variance
Retained in Basin
10
20
30
40
50
0.0
0
100
200
300
400
500
600
700
lteration
Number Principal Companents
(a) Comparison of Training Loss Curves
(b) Swin-T PCA Cumulative Explained Variance
Figure 4: Training loss curves and Percentage of explained variance in GPLQ.
on the ImageNet validation set and the average accuracy from linear probing on 5 downstream FGVC datasets. Results are shown in T
We observed that without PCA feature mimicking ('Without PCA'), the model's average downstream task accuracy was 69.25%. When PCA feature mimicking introduced, even with few dimensions (64). downstream task performance improved (70.12%). As the PCA dimensionality increased (256, which point the cumulative explained variance about 60%). the average downstream accuracy reached 70.77%. Further increasing the dimensionality to 512 resulted in a slight drop to 70.36%, but is still better than not using PCA. These results indicate that feature mimicking the quantized model to learn key features from the FP32 model, thereby enhancing its generalization ability. Selecting dimension that captures around 60% of the original feature variance represents good trade-off.
effectively guide
Figure 4b shows Swin-T model's s PCA cumulative explained variance with varying dimensions. At about 256 dimensions, the cumulative explained variance reaches approximately 61.3%. Even with 64 dimensions where only 24. 1 % variance explained, PCA feature mimicking already shows better generalization ability than without using PCA feature mimicking, as shown Table 6
Quantitative Analysis of Stability and Decoupling. To provide deeper quantitative evidence for our "same basin" and factivation-firs" hypotheses, conducted two further analyses. First, we tracked the weight deviation (Mean Squared Error) between the original FP32 weights and the model's weights during training. As shown in Table 7 GPLQ's Act-QAT stage keeps the weights extremely close to the original (0.049 MSE), as weights are FP32 and only activations are quantized. In contrast, "Direct W4A4" QAT (quantizing both W & A for I epoch) immediately diverges, with a deviation nearly 6 times larger (0.289 MSE). This confirms GPLQ's 's design prevents the model from "jumping out" of the original basin.
Table 7: Average per-layer weight deviation (MAE/ / MSE) from original FP32 weights during epoch of training (Swin-T). GPLQ's decoupled strategy maintains weights within the original optimization basin, while direct W4A4 QAT diverges significantly.
Method
Iter. 100
Iter. 1k
Final (1 (1-epoch)
GPLQ (Act-QAT)
0.082 0.046 0.083 0.047
0.086 0.049
Direct W4A4 QAT 0.256 0.271 0.257 0.274
0.269 0.289
Second, we analyzed the benefit of our sequential strategy. GPLQ's "activation first" stage stabilizes activations, creating simpler problem for the subsequent weight PTQ. We measured the output error (MSE) introduced *only* by weight PTQ in two scenarios: 1) Standard PTQ (W32A32 + W4A4) and 2) GPLQ's Stage 2 (W32A4 - W4A4). Tshoes that because GPLQ has already stabilized the activations, the subsequent weight quantization error is dramatically reduced- by nearly 46% on average for Swin-T. T. This demonstrates that our decoupled strategy is superior because stabilizes activations first, making the final weight quantization step inherently casier and more accurate.

---

Table PTQ Error (Average Per-Layer Output MAE/ MSE) with (GPLQ) vs, without (Standard) pre-stabilized activations. Stabilizing activations first (GPLQ) cuts the subsequent weight quantization error almost in half.
PTQ Scenario
Model
Layerl block1
Layer4 block2
Avg. (All Layers)
Standard (on W32A32)
Swin-T
0.359 0.239
0.426 / 0.530
0.184 / 0.136
GPLQ (on W32A4)
Swin-T
0.150 0.045
0.224 / 0.206
0.141 / 0.074
Standard (on W32A32)
Swin-S
0.188 0.058
0.302 / 0.368
0.164 / 0.140
GPIQ (on W32A4)
Swin-S
0.123 / 0.034
0.156 / 0.106
0.129 0.076
5 Conclusion
We presented GPLQ, novel quantization framework for Vision Transformers that significantly improves efficiency and generalization over existing PTQ and QAT methods. Our approach is grounded in the empirical finding that activation quantization critical and that preserving the original model's optimization basin is key to maintaining generalization. GPLQ's "activation-first weights- later" strategy, featuring single-epoch activation quantization with PCA-based feature mimicking, followed by PTQ for weights, achieves 4-bit performance competitive with, and sometimes superior to, FP32 models in terms of generalization. This methodology not only drastically cuts training overhead, making advanced quantization more accessible, but also consistently outperforms prior art. GPLQ thus offers a practical and robust path for deploying low-bit ViTs in resource-conse constrained scenarios, facilitated by our forthcoming open-source toolkit.
6 Limitations and Future Work
Limitations include:
Broader QAT comparison: Limited open- availability of advanced QAT methods restricted comprehensive comparative analysis.
Dependence on PTQ techniques: GPLQ's second stage performance is tied to the capabilities and limitations of existing PTQ methods.
Low bit-width exploration: Our work primarily focused on 4-bit quantization, mainly because 4-bit is where the hardware support ends at. Deeper investigation into even lower bit-widths is desired, too.
Future directions include:
Extending PCA- based feature mimicking to further enhance model generalization across a broader range of vision tasks, such as object detection and semantic segmentation.
Conducting more comprehensive and rigorous QAT benchmarks against wider array of contemporary methods as they become publicly accessible and resources permitting.
Adapting and evaluating the GPLQ framework for its applicability and effectiveness on diverse neural network architectures, including CNNs [3 and emerging LLMs [3 38
Integrating next eext-generation PTQ advancements to continually improve GPLQ's s performance.
Acknowledgments and Disclosure of Funding
This work was partly supported by the National Natural Science Foundation of China under Grant 62276123.
JW identified the problems and conjectures in QAT, and guided GL in conducting the experiments. During the experimental process, GL and JW jointly designed the GPLQ method. JW and GL wrote this paper. XL helped GL complete parts of the experiments and writing.
10

---

References
[1] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
[2] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-100 - minnng discriminative components with random forests. In European Conference on Computer Vision, 2014.
[3] Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. Efficientqat: Efficient quantization-awe -waree training for large language models. arXiv preprint arXiv:2407.11062 2024.
[4] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-255 leee, 2009.
[5] Peiyan Dong, Lei Lu, Chao Wu, Cheng Lyu, Geng Yuan, Hao Tang, and Yanzhi Wang. Packqvit: Faster sub-8-bit vision transformers via full and packed quantization on the mobile. Advances in Neural Information Processing Systems, 36:9015-9028, 2023.
[6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint WrXiv:2010.11922 2020.
[7] Steven K Esser, Jeffrey I. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dhar- mendra S Modha. Learned step size quantization, arXiv preprint arxiv: 2019.
[8] Elias Frantar, Salch Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint #rXiv:2210.13325 2022.
[9] Minghao Fu, Hao Yu, Jie Shao, Junjie Zhou, Ke Zhu, and Jianxin Wu. Quantization without tears. arXiv preprint arXiv:241 1. 139188, 2024.
[10] Minghao Fu, Ke Zhu, and Jianxin Wu. Dtl: Disentangled transfer learning for visual recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12082- 12090, 2024.
[11] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision, 17996):788-11888 1819, 2021.
[12] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2966 2017.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[14] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: 10X3:::503.0253:3 2015.
[15] Xijie Huang, Zhiqiang Shen, Pingcheng Dong, and Kwang-Ting Cheng. Quantization vari- ation: A new perspective on training transformers with low-bit precision. arXiv preprint arXiv:2307.0033 2023.
[16] Runqing Jiang, Ye Zhang, Longguang Wang, Pengpeng Yu, and Yulan Guo. Aiqvit: Architecture- informed post-training quantization for vision transformers. arXiv preprint arXiv:2502.04628. 2025.
[17] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. Proceedings of the IEEE international conference on computer vision workshops, pages 554-561, 2013,

---

[18] Jiedong Lang, Zhehao Guo, and Shuyu Huang. A comprehensive study on quantization techniques for large language models. In 2024 4th International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC), pages 224-231. IEEF, 2024.
[19] Yanjing Li, Sheng Xu, Baochang Zhang, Xianbin Cao, Peng Gao, and Guodong Guo. Q-vit: Accurate and fully quantized low-bit vision transformer. Advances in neural information processing systems, 35:34451-34400 2022.
[20] Zhikai Li, Junrui Xiao, Lianwei Yang, and Qingyi Gu. Repq-vit: Scale reparameterization for post-training quantization of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17227-17236, 2023.
[21] Lin, Jiaming Tang, Haotian Tang, Shang Yang. Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-awarr weight quantization for on-device IIm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87-100, 2024.
[22] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
[23] Shih- Shih-Yang Liu, Zechun Liu, and Kwang- Ting Cheng. Oscillation-fie quantization for low-bit vision transformers. In International conference on machine learning, pages 21813-21824. PMLR, 2023.
[24] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
[25] Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quantization for vision transformer. Advances in Neural Information Processing Systems, 34:28092-28103. 2021.
[26] Iya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 171001111..05101, 2017.
[27] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- 2013.
grained visual classification of aircraft. arXiv preprint
[28) Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pages 722-729. IEFE, 2008
[29] Lorenzo Papa, Paolo Russo, Irene Amerini, and Luping Zhou. A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
[30] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference computer vision and pattern recognition, pages 3498-3505. IEEE, 2012.
[31] Ningyuan Tang, Minghao Fu, Hao Yu, and Jianxin Wu. Qwt-v2: Practical, effective and efficient post-training quantization. arXiv preprint arXiv:2505.20932 2025.
[32] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347-10357. PMLR, 2021.
[33] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. /3971, 2023.
[34] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
12

---

[35] Guo-Hua Wang, Yifan Ge, and Jianxin Wu. Distilling knowledge by mimicking features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8183 8195. 2021.
[36] Guangxuan Xiao, Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087-38099. PMLR, 2023.
[37] Hao Yu and Jianxin Wu. Compressing transformers: features are low-rank, but weights are not! In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 11007-11015, 2023.
[38] Hao Yu, Yang Zhou, Bohua Chen, Zelan Yang, Shen Li, Yong Li, and Jianxin Wu. Treasures in discarded weights for IIm quantization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 222 18 222266, 2025.
[39] Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptqávit: Post-training quantization for vision transformers with twin uniform quantization. European conference on computer vision, pages 191-207. Springer, 2022.
[40] Junjie Zhou, Ke Zhu, and Jianxin Wu. All you need in knowledge distillation is tailored coordinate system. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 22946 22954, 2025.
[41] Ke Zhu, Yin-Yin He, and Jianxin Wu. Quantized feature distillation for network quantization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11452-1 1460, 2023.
13

---

A
Appendix
This appendix provides supplementary experimental results and analyses that further support the findings presented in the main paper.
A.1
Quantization under Constrained Computational Resources
To assess the robustness and efficiency of our GPLQ method under more restrictive computational environments, we investigated the impact of varying the number of available GPUs during the activation quantization stage (Act-QAT). For these experiments, only activations were quantized for I epoch. The batch size per GPU was maintained at 16. Consequently, a reduction in the number of GPUs corresponds to a proportional decrease in the effective batch size and the learning rate was adjusted accordingly. Other training parameters remained consistent with the settings described in the main paper.
Table details the performance of Swin-T on ImageNet and the associated training times.
Table 9: Impact of varying GPU counts on Swin-T (W32A4 Act-QAT only) ImageNet accuracy and training time. Batch size per GPU is 16.
Number of GPUs
Equivalent Batch Size
Learning Rate
ImageNet Acc (%)
Time (min)
8
128 (16x8)
5c-6
80.2
35
4
64 (16x4)
2.5e-6
80.1
71
2
32 (16x2)
le-6
80.
139
1
16 1)
le-6
80.1
275
The results indicate that GPLQ maintains high accuracy on ImageNet even as the number GPUs and, consequently, the effective batch size and learning rate, are significantly reduced. The perfor- mance remains remarkably stable (80.1-80.29) top-1 accuracy) across all tested configurations. This resilience suggests that method can effectively train quantized models even with minimal training resources, a capability not typically demonstrated by traditional QAT methods. Traditional QAT approaches often rely on large batch sizes and learning rates for stable convergence, and their perfor- mance is expected to degrade under such resource-constraiiee conditions. Our findings underscore the practical advantage of GPLQ in scenarios with limited hardware availability.
A.2 Impact of Training Data Volume Model Performance
We further analyzed the influence the training data volume on the Swin-T model's performance during the I-epoch activation quantization stage. All training settings were kept consistent with the GPLQ defaults, except for the number of training images used, which was varied by controlling the number of training iterations.
Table 10: Impact of training data volume (number of images used in I epoch Act-QAT) on Swin-T ImageNet T accuracy (%) for W32A4 (only activations QAT) and W4A4 (activations QAT, weights PTQ) settings.
Training Iterations
Images Used
W32A4 Acc (%)
W4A4 Acc (%)
128
70.9
68.5
10
1,280
74.1
72.5
100
12,800
77.9
77.0
1000
128,000
79.6
79.0
10009
Full Dataset (approx. 1.28M)
80.2
79.5
As expected, increasing the volume of training data generally leads to improved model accuracy. Even with a relatively small number images (e.g.. 128,000, corresponding to about 10% the full ImageNet training set for I epoch), the model achieves respectable 79.0% accuracy. Training on the full dataset for epoch yields 80.2% accuracy for the W32A4 model (output of Act-QAT stage), which forms a strong basis for the subsequent weight PTQ stage.
14

---

A.3 Impact of Direct W4A4 Training versus Sequential Quantization
To further highlight the benefits of our proposed sequential "activation-fiis first, weights- lather-lattrr (W32A4 - W4A4) strategy, we compare it against a more direct approach where both weights and activations are quantized simultaneously from the start of the 1-epoch QAT process (direct W4A4). All other training hyperparameters for this direct W4A4 baseline were kept identical to those used in the Act-QAT stage of our GPLQ method.
Table 11 presents the ImageNet top-1 accuracy for various Swin Transformer models.
Table 11:
Comparison of ImageNet top-1 accuracy (%) for Swin Transformers using direct W4A4 QAT versus GPLQ's s sequential (W32A4 + W4A4) approach. Optimal results are in bold.
Training Method
Swin-T
Swin-S
Swin-B
Swin-L
Direct W4A4 (1 epoch QAT)
78.5
81.2
83.5
84.9
GPLQ (W32A4 W4A4)
79.5
82.0
84.0
85.3
The results clearly demonstrate that our proposed sequential quantization strategy (GPLQ) consistently outperforms the direct W4A4 QAT approach across all Swin Transformer variants. As discussed in the main paper, quantizing activations first (while keeping weights in FP32) and then applying PTQ to the weights offers several advantages. Beyond the slight improvements in training speed and reduced memory footprint during the Act-QAT stage (as pseudo-quantization of weights is not performed), this sequential approach helps to avoid the weight oscillations often encountered in traditional QAT. This leads to smoother optimization process for the model weights, ultimately resulting in improved final model performance, as evidenced by the higher accuracies in Table
A.4 Efficiency and Resource Overhead Comparison
Table 12 quantifies the "Practical" and "Lightning" advantages of GPLQ, highlighted in Figure Compared to traditional QAT methods like Q-Var and OFQ, GPLQ is over 500 times faster (35 minutes vs. ~12 days). Furthermore, its peak memory footprint (6.6GB) is dramatically lower-even less than standard FP32 training (7.2GB)-am and far below the prohibitive requirements other QAT methods (25.9-37.5GBB. This demonstrates GPLQ's practicality for real-world deployment consumer-grade hardware.
I
Table 12: Detailed comparison of performance and resource overhead ImageNet using Swin-T model.
Method
ImageNet Acc (%)
Avg. Downstream Acc (%)
Training Time (min)
Peak Mcmory (GB)
Full-precision (FP32)
81.2
70.79
3078 (train from scratch)
7.2
RepQViT (PTQ)
73.0
62.79
(calibration only)
2.3
Q-Var (QAT)
82.4
68.12
17400 (~-12 days)
25.9
OFQ (QAT)
81.9
60.73
18360 (>-12.7 days)
37.5
GPLQ (Ours)
79.5
70.77
35 (1 epoch)
6.6
A.5
Generality to CNN Architectures
While GPLQ was motivated by ViTs, its core principles-the importance of activations and preserving the optimization basin-an are architecture-agnosicc. To demonstrate its generality, we applied GPLQ the classic ResNet [13] family. As shown Table GPLQ (W4A4) significantly outperforms the strong PTQ baseline (RepQViT) on all ResNet models and consistently improves downstream generalization with only epoch of training.
A.6 Impact Act-QAT Training Duration
A key claim of GPLQ is its "Lightning" speed, achieved by training Act-QAT for only 1 epoch. We conducted an ablation study (Table 14p to validate this choice. Extending the training from epoch to 10 epochs yields negligible performance gains (50.3%) both ImageNet and downstream tasks,
15

---

Table 13: Generalization performance of GPLQ (W4A4) on CNN architectures (ResNets).
Model
Method
ImageNet Acc (%)
Avg. Downstream Ace (%) Training Epochs
FP32
69.10
58.78
ResNet-18
RepQViT
56.97
56.70
GPLQ
66.16
59.44
FP32
74.90
59.57
ResNet-34
RepQViT
64.76
54.13
GPLQ
70.36
60.76
FP32
78.70
66.
ResNet-50
RepQViT
63.68
52.09
GPLQ
74.13
67.68
FP32
76,20
63.60
ResNet-101
RepQViT
56.97
60.26
GPLQ 73.83
64.21
while increasing the training cost 10-fold. This supports epoch as the optimal "sweet spot" for GPLQ's efficiency-performane performance trade-off.
Table 14: Impact of Act-QAT training duration on Swin-T performance.
Act-QAT Epochs ImageNet Top-1 Acc (%)
Avg. Downstream Acc (%)
(Ours)
79.5
70.8
79.6
70.9
79.6
71.1
10
79.5
71.0
A.7 Performance on Lower Bit-Widths (W3A3)
To test GPLQ's generality to more aggressive quantization, we evaluated it in a W3A3 setting (Table 15p. GPLQ significantly outperforms the SOTA PTQ method (RepQ-Vit) e.g., by +21.5% Swin-T. This demonstrates that GPLQ's "activation-fity framework is a powerful and general strategy, especially for challenging low-bit scenarios.
Table 15: Performance of GPLQ in lower-bit (W3A3) setting.
Model
Method
ImageNet Acc (%)
Avg. Downstream Acc (%)
FP32
81.2
70.79
Swin-T
RepQVIT (W3A3)
52.7
39.26
GPLQ (W3A3)
74.2
65.76
FP32
83.2
70.07
Swin-S
RepQViT (W3A3)
65.9
53.35
GPLQ (W3A3)
78.3
64.64
FP32
85.3
78,93
Swin-B
RepQViT (W3A3)
65.8
29.82
GPLQ (W3A3)
80.3
74.15
FP32
86.3
80.89
Swin-L
RepQViT (W3A3)
71.3
66.36
GPLQ (W3A3)
80.0
73.36
16

---

A.8 Ablation on PCA Feature-Mimicking Loss Weight
As mentioned in Section 3.2.1, our default weight a for the LPCA loss 1.0 (i.e., 1:1 ratio with the main task loss). We conducted an ablation (Table I6) to validate this choice. The results show that 1.0 provides the best trade-off, particularly for downstream generalization.
Table 16: Ablation analysis the LPCA feature-mimicking loss weight (a) on Swin-T.
Loss Setting
Loss Weight (a)
ImageNet Top-1 Ace (%) Avg.
Downstream Acc (%)
w/o PCA Loss
79.4
69.2
Penultimate
0.1
79.5
70.3
Penultimate
(Default)
79.5
70.8
Penultimate
10
79.3
70.1
A.9 Component Interchangeability
GPLQ is proposed as a flexible framework. To demonstrate this, we replaced the percentile- bhased initialization (from RepQ-ViT) in PTQ stage with other common methods. As shown in Table 17] the framework is robust to this change, with final performance remaining high (79.2-79.5%% This confirms GPLQ's success stems from its "activation-first" paradigm, specific combination of components.
Table 17: Verifying the interchangcabilliy of different PTQ initialization components within the GPLQ framework (Swin-T).
Stage 2 (Weight-PTQQ) Initialization
ImageNet Top-1 Acc (%)
Percentile RepQ-ViT)
79.5
MSE-based
79.5
MinMax
79.2
17

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
In this paper, we claim in the abstract and introduction that GPLQ general, practical, and lightning- QAT method for ViTs, which outperforms existing PTQ methods and exhibits better generalization than traditional QAT approaches. Our main contributions, as stated, are: (1) new empirical insights regarding the paramount importance of activation quantization and the necessity preserving the model's original optimization "basin"; (2) the GPLQ framework itself, which employs novel "activation first, weights- later" sequential quantization strategy: and (3) our commitment release an casy-to-uss open-source toolkit. These claims are substantiated by the experimental results presented in Section and discussed throughout the paper. For example, Table [ shows GPLQ's competitive accuracy and superior generalization, and Figure hiighighhts its efficiency.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
In Section 6, "Limitations and Future Work," we explicitly discuss the limita- tions of our work. These include the challenge of conducting broader QAT comparison due to the limited open-source availability of advanced QAT methods, the dependence of GPLQ's second stage on existing PTQ techniques (inheriting their limitations), and the primary focus on 4-bit quantization, with need for deeper exploration into lower bit-widths.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
18

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper is primarily empirical in nature. We present novel empirical insights (Section 3.1) and introduce the GPLQ framework (Section 3.2), validating approach through extensive experiments (Section 4). While our method incorporates established mathematical formulations for quantization (Equation 1), PCA-based feature mimicking loss (Equation 2), and the QwT compensation mechanism (Equation 3), we do not propose new theoretical results that would necessitate formal proofs within this work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Section 4 ("Experiments"), provide comprehensive details for repro- ducing our main experimental results. This includes specifications the datasets used (ImageNet-Ik, COCO 2017, and five FGVC datasets), the evaluation metrics (Top-1 ac- curacy, Apbox average downstream accuracy), and the detailed settings for both Stage 1 (Act-QAT) and Stage 2 (Weight-PTQQ) of GPLQ. Key parameters such the optimizer (AdamW), learning rates (fixed x 10-6 for Act-QAT),, batch sizes (16 per GPU for Act- QAT), quantization specifics (4-bit symmetric, per-channel), PCA subspace dimensions (e.g., 256 for Swin-T. 64 for DeiT-T), and the calibration set size for PTQ (512 images) are explicitly stated. Furthermore, as noted in Contribution 3 (Section 1) and Section 3.3, we have also stated that "We will open-source GPLQ upon paper acceptance" and "GPLQ provides an easy-to-use quantization tool".
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
19

---

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- ppen-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: In this paper, we explicitly state our intention to provide open access to our GPL.Q framework. Specifically, in Contribution 3 (Section 1), we mention, "We will open- source GPLQ upon paper acceptance." This is reiterated in Section 3.3: "GPLQ provides an casy-to-use quantization tool supporting classification, detection and other downstream tasks." The datasets utilized in our experiments (ImageNet, COCO, and the various FGVC datasets) are all publicly available and widely used benchmarks, with citations provided. Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ unips.cc/publli staili///uiidess/ddeeuubbsssas
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
20

---

6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We have detailed our experimental settings in Section ("Experiments"), This includes the specific datasets used (ImageNet-1K, COCO 2017 with 'train2017' for training and 'val2017' for validation, and five FGVC datasets). For Stage (Act-QAT), we specify epoch of training, the AdamW optimizer, fixed learning rate of 10 without decay, per-channel symmetric 4-bit quantization for activations, PCA subspace dimensions (c.g.. 256 for Swin-T, 64 for DeïT-T, chosen based on 60
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: In this paper, our experimental results, as presented in Tables 2 3 and 5 report performance metries (such as Top-1 accuracy, Apbox and average downstream task accuracy) as single numerical values for each method and model configuration. We do not include error bars, confidence intervals, or explicit statistical significance tests (e.g.. p-va p-values) for these reported results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
21

---

Justification: We provide information regarding the compute resources used for our experi- ments. In Section we state: "Training was conducted GPUs, with batch size of 16 per GPU. This configuration allows quantizing of the entire Swin Transformer series even on consumer-grade GPUs." We also highlight the efficiency GPLQ: Act-QAT requires "onlyy epoch" (mentioned in Introduction and Section 2), is "hundreds of times faster" than traditional QAT (Introduction and Section 3.3), and reduces QAT training time "from days to 1-2 hours" (Section 1). This gives clear indication of the hardware (8 GPUs, consumer-grade accessible) and the significant reduction in execution time compared to standard QAT methods. The memory footprint is also noted as "far lower than existing QAT methods" and "even lower than FP32 training" (Introduction, Section I).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https::/ http://ppppppccccceeecciiieeeeee
Answer: [Yes]
Justification: Our research, as presented in this paper, focuses on developing and evaluating model quantization technique (GPLQ) for Vision Transformers. This is standard and constructive area machine learning research aimed at improving model efficiency. We have reviewed the NeurlPS Code of Ethics, and to the best of our knowledge, our work, including the methodology. data usage (publicly available academic datasets), and reporting, fully conforms to these guidelines. There plagiarism, dual-use concerns beyond typical model efficiency improvements, fairness or privacy issues related to data subjects, research with human subjects that would require special ethical considerations or approvals.
aspects of our research that involve
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This paper primarily focuses on the technical contributions GPLQ, detailing its methodology, efficiency, accuracy, and generalization capabilities for Vision Transformer quantization. While positive societal impacts are implicit in our such as enabling the deployment of powerful ViT models on resource-constrainee devices, potentially leading to reduced energy consumption and wider accessibility AI technologies-ae have not dedicated specific section to explicitly discuss these potential negative societal impacts. The "L "Limitations and Future Work" section (Section 6) is confined to technical limitations and avenues for future research within the scope of model quantization.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
22

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work introduces GPLQ, a quantization method, and we plan to release associated open-source toolkit. This toolkit is designed for model compression and efficiency, rather than being a pre-trained model with inherent high-risk misuse potential (like large generative model) or a dataset containing sensitive information. Therefore, the specific safeguards typically associated with the release of such high-risk assets are not directly applicable the GPLQ method or its toolkit described in this paper. The models we quantize are standard Vision Transformers, and the datasets used are publicly available academic benchmarks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [No]
Justification: In this paper, we have made sure to properly credit the original creators of the assets we used by citing their respective publications. This includes the datasets
(e.g., ImageNet [4) COCO [22,, various FGVC datasets 22 28 30 [7) and influential
methods or models that form the basis or comparison for our work (e.g., DeiT 32), Swin-
T 244. ReTQ-ViT 20 QwT (9). However, we have not explicitly stated the specific
licenses (c.g., CC-BY 4.0, MIT License) or detailed terms of use for these pre-existing
23

---

assets within the body of our paper. We have operated under the assumption that their use for academic research purposes is permitted by their standard terms of availability. Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode thcode com/ datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The primary new asset we introduce in this paper is the GPLQ method itself, along with our plan to release an "easy-to-use open-source toolkit supporting multiple vision tasks" (as stated in the Abstract, Section 1, and Section 3.3). This paper serves as the primary documentation for the GPLQ methodology, detailing its empirical motivations, the two-stage framework (Act-QAT and Neight-PPTQ) the use of PCA-based feature mimicking, and its application to various tasks. We believe the paper provides thorough explanation how GPLQ works and its advantages. The specifies of the toolkit's documentation will accompany its release, but the core principles and functionality are described herein.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our research presented in this paper does not involve any crowdsourcing experiments direct research with human subjects. All experiments were conducted using standard, publicly available computer vision datasets (ImageNet, COCO, FGVC datasets) for training and evaluating our model quantization method, GPLQ
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our research, which focuses on algorithmic development for model quan- tization using existing public datasets, did not involve human subjects. Consequently, considerations of potential risks to study participants, disclosure of such risks, or the neces- work.
sity for Institutional Review Board (IRB) approvals (or equivalent) are not applicable to this
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core methodology of our research in this paper, which introduces and evaluates the GPLQ quantization framework for Vision Transformers, does involve the use of Large Language Models (L..Ms) as an important, original, or non-standard component. Our work is focused on quantization techniques, empirical analysis, and algorithmic design within the computer vision domain. While we mention adapting GPLQ for LLMs as future work direction (Section 6), LLMs were part of the development or execution of the research presented.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non nta-stad components.
Please refer to LLM policy httpg:///aaarpppccc///oooeeeeeee for what should or should not be described.
25


Reviews:
[{"review_id": "NhF0v33OZw", "reviewer": "Reviewer_QuB9", "summary": "GPLQ is presented as a way to quantize vision transformer models. The central design choice is to do quantization-aware training (QAT) in a way that considers quantized activations but *not* quantized weights, followed by a post-training quantization (PTQ) for the weights. The QAT is also defined by a procedure that fine-tunes from an original floating-point network (for 1 epoch), and includes a loss on the similarity of the features to those of the original un-quantized model. It then uses QwT for 4-bit weight PTQ. This is evaluated primarily on ImageNet1K, using Swin and DeIT architectures. Another experiment tests on COCO with a Swin backbone.", "questions": "- **Q1**: Could the authors clarify also in the reviews which methods have limited or no open source availability, but would otherwise be useful in a similar way and a potential comparison point? I'm referring to point 3 in Section 4.1 and the \"Code Missing\" point around line 45 in the introduction.\n- **Q2**: Is GPLQ expected to be limited to Vision transformers? SmoothQuant [36], also based on the observation of activations being more difficult to quantize than weights, saw utility on language models as well.\n- **Q3**: Why is the feature mimicking loss applied only the the pentultimate layer? Why not all of them, given all of the layers' activations/features are being quantized in this step.\n- **Q4**: Was there a multiplier hyperparameter for the feature mimicking loss $L_\\mathrm{PCA}$ (vs the classification loss) within the overall training loss? (Perhaps I've missed where this was stated.) What was the observed magnitude of of $L_\\mathrm{PCA}$ vs e.g. the other loss term(s)?", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n1. SmoothQuant [36] was one of the key recent steps forward in PTQ methods, and was also based on the observation that weights are \"easier\" to quantize than activations. It is a clear and potentially useful step to apply this same observation to QAT methods.\n\n2. Tests on multiple benchmark tasks (both ImageNet and COCO).\n\n3. Very clear writing: that describes both describes the procedure in an understandable way and the motivating intuition for different steps or design choices.\n\n4. Applies to standard backbones, without needing larger changes to the architecture itself.\n\n### Weaknesses\n\n5. Kind of scattered, in that it's presenting multiple different techniques combined into one procedure, as opposed to a more in-depth treatment of one.\n\nFor example, there are many existing QAT approaches one might use for the activation-only QAT step. The submission does do an ablation study for the contribution of this specific step, but only considers one variation of how to include the quantization in training. As one example of alternate choices here: the basic \"fake\"/\"simulated\" quantization layer in [a], and its use of EMA-estimated quantization parameters, could just as easily also be applied to activations only. The method in Section 3.2.1 of the submission has a learnable scale, as in [7], instead and defines a procedure to initialize it. It's not explored what effect all of these choices have.\n\n[a] Jacob et. al. \"Quantization and training of neural networks for efficient integer-arithmetic-only inference\" CVPR 2018.\n\n6. No study on length of fine-tuning.\n\nApplying the QAT as a short (1 epoch) fine-tuning is also a significant choice in the design of the GPLQ procedure. I didn't see much evaluation of this choice, and how it might interact with other parameter choices such as the learning rate or relative weight of the feature mimicking loss.\n\n7. The feature-mimicking loss has a similar role, within QAT, to the self-distillation in\n\nDu et. al. BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation. ACL 2024\n\nBitDistiller also achieves lower precisions than 4 bit. Similarly for [15]. The authors make the valid point that their loss based on TCS [37] is simpler and more efficient, but this also is only shown to achieve a less compressed quantization than the previous work.", "was_revised": true, "final_justification": "This is a solid paper that incorporates into QAT some of the key insights on activation vs weight quantization from the most popular recent PTQ work.\n\nThe rebuttal addressed all of my comments.", "rebuttal": "Dear Reviewer QuB9,\n\nThank you very much for your detailed and constructive review of our submission (#8001). We greatly appreciate your recognition of our work's clarity and methodological innovation. Your feedback is invaluable for improving the quality of our paper. We will address your questions and concerns one by one.\n\n---\n\n**Regarding Weaknesses:**\n\n1.  **On the concern that the method is a \"dispersed\" combination of different techniques:** We appreciate this observation. The core contribution of our work is to propose GPLQ as a flexible and efficient quantization **framework**, rather than a single, fixed algorithm. Its \"activation-first, weights-later\" core idea is modular. To demonstrate this, we have added a supplementary experiment (see Table B.1 below) showing that the GPLQ framework maintains high performance even when components in the PTQ stage are replaced (e.g., changing the initialization method). This indicates that the success of GPLQ is not due to an incidental combination of specific components, but rather stems from the superiority of its framework design.\n\n    **Table B.1: Verifying the interchangeability of different components within the GPLQ framework.**\n    | Model  | Act-QAT Initialization        | ImageNet Top-1 Acc (%) |\n    | :----- | :---------------------------- | :--------------------- |\n    | Swin-T | Percentile (as in RepQ-ViT)   | 79.8                   |\n    |        | MSE-based                     | 79.8                   |\n    |        | MinMax                        | 79.6                   |\n\n2.  **On alternative options for the activation QAT stage and initialization methods:** You've raised an excellent point. We chose LSQ because it is widely validated and effective in the QAT field. We acknowledge that exploring other options, such as the EMA-based method mentioned in [a], would also be valuable. While a comprehensive comparison is planned for future work, we will add a discussion and a placeholder for preliminary comparisons in the appendix to explore the performance of different activation quantization strategies. Furthermore, our initialization step is designed to provide LSQ with a superior starting point, enabling rapid convergence within just one epoch of training. This is a key part of our \"lightning-fast\" training philosophy.\n\n3.  **On the lack of study on fine-tuning duration:** This is a critical question. We chose a 1-epoch fine-tuning to strike an optimal balance between performance and efficiency. To validate this choice, we have added an ablation study on the Act-QAT training duration (see Table B.2 below). The results show that extending the training from 1 epoch to 2, 5, or even 10 epochs yields very limited performance gains (<0.3%) on both ImageNet and downstream tasks, while the training cost increases manifold. This strongly demonstrates that 1-epoch training is the \"sweet spot\" for the GPLQ framework in terms of efficiency and effectiveness. Of course, our GPLQ framework can be easily modified if one is willing to invest more time for further potential improvements.\n\n    **Table B.2: Impact of Act-QAT training duration on model performance.**\n    | Model (Model) | Act-QAT Epochs | ImageNet Top-1 Acc (%) | Avg. Downstream Acc (%) |\n    | :------------ | :------------- | :--------------------- | :---------------------- |\n    | Swin-T        | 1 (Ours)       | 79.8                   | 71.2                    |\n    |               | 2              | 79.8                   | 71.5                    |\n    |               | 5              | 79.9                   | 71.3                    |\n    |               | 10             | 79.9                   | 71.1                    |\n\n4.  **On the comparison between feature-mimicking loss and lower-bit quantization:** You made an insightful comparison of our work with methods like BitDistiller. We agree that these methods have achieved remarkable success in sub-4-bit quantization. In fact, we argue that GPLQ's 'activation-first' framework is even more critical at lower bit-widths, as the challenges of activation quantization become more severe. To verify this, we conducted more challenging W3A3 quantization experiments (see Table B.6 of our response to Reviewer Xx2u). The results are encouraging: GPLQ significantly outperforms SOTA PTQ methods (e.g., by +21.5% on Swin-T), demonstrating the strong potential of our approach. More importantly, GPLQ achieves this exceptional performance while adhering to its lightweight and efficient core philosophy: using only 1 epoch for Act-QAT and a simple PCA-based feature-mimicking loss, thus avoiding the complex self-distillation loops found in methods like BitDistiller. Therefore, we believe GPLQ is not only a practical solution for 4-bit quantization but also a powerful framework for the sub-4-bit domain that strikes a superior balance between performance, efficiency, and simplicity.\n\n\n\n**Regarding Questions:**\n\n* **Q1: Regarding QAT methods with no open-source code:** Thank you for the question. When we wrote the paper, some key comparison methods like OFQ and Q-Var were reproducible, so we included them. However, others such as Q-ViT and PackQViT had limited open-sourcing (not all details were published). Furthermore, it was difficult to find official or reliable third-party implementations for these methods, especially for large models (e.g., Swin-B/L) or various vision tasks (e.g., detection). This prevented us from conducting a fair, large-scale side-by-side comparison. In particular, regarding memory usage and training time, many methods resulted in out-of-memory (OOM) errors on our hardware for large models, as illustrated in Figure 1 of our paper.\n\n* **Q2: Is GPLQ limited to Vision Transformers?** Not at all. The core insights of GPLQ—that activations are harder to quantize than weights and that preserving the optimization basin is crucial for generalization—are universally applicable. To demonstrate this, we have successfully applied GPLQ to classic CNN architectures (ResNet) and achieved excellent performance (see Table B.3). This shows that GPLQ is a general framework with broad application potential.\n\n    **Table B.3: Generalization performance of GPLQ on CNN architecture (ResNet-18).**\n    | Model (Model) | Method (Method) | ImageNet Top-1 Acc (%) | Avg. Downstream Acc (%) | Training Epochs |\n    | :--- | :--- | :--- | :--- | :--- |\n    | ResNet-18 | FP32 | 69.10% | 58.78 | - |\n    | | RepQViT | 56.97% | 56.7 | - |\n    | | **GPLQ (W4A4)** | **66.16%** | **59.44** | **1** |\n    | ResNet-34 | FP32 | 74.90% | 59.57 | - |\n    | | RepQViT | 64.76% | 54.13 | - |\n    | | **GPLQ (W4A4)** | **70.36%** | **60.76** | **1** |\n    | ResNet-50 | FP32 | 78.70% | 66.17 | - |\n    | | RepQViT | 63.68% | 52.09 | - |\n    | | **GPLQ (W4A4)** | **74.13%** | **67.68** | **1** |\n    | ResNet-101 | FP32 | 76.20% | 63.6 | - |\n    | | RepQViT | 56.97% | 60.26 | - |\n    | | **GPLQ (W4A4)** | **73.83%** | **64.21** | **1** |\n    | *ResNet-18* | *DoReFa* | *68.10%* | *-* | *200* |\n    | | *PACT* | *69.20%* | *-* | *200* |\n    | | *LSQ* | *71.1%* | *-* | *90* |\n    | *ResNet-50* | *DoReFa* | *71.40%* | *-* | *200* |\n    | | *PACT* | *76.50%* | *-* | *200* |\n    | | *LSQ* | *76.70%* | *-* | *90* |\n\n* **Q3: Why is the feature-mimicking loss only applied to the penultimate layer?** We chose the penultimate layer based on two considerations: (1) **High-level semantic information:** This layer's features contain the highest level of semantic information, which is critical for the model's classification and generalization capabilities. (2) **Efficiency trade-off:** Constraining the features of a single layer allows for effective knowledge transfer from the teacher model while minimizing computational overhead, which aligns with our \"lightning-fast\" principle. Applying the loss to all layers would significantly increase computational complexity.\n\n* **Q4: What is the hyperparameter for the feature-mimicking loss?** We apologize for not stating this clearly in the original manuscript. In our experiments, the weight ratio of the feature-mimicking loss ($L_{PCA}$) to the original classification/task loss is 1:1. This means we did not introduce an additional hyperparameter $\\alpha$ (or rather, $\\alpha=1.0$). In our supplementary ablation study (Table B.4), we also explored the impact of different loss weights, and the results show that the default setting ($\\alpha=1.0$) achieves the best downstream task generalization performance.\n\n    **Table B.4: Ablation analysis of the Feature-Mimicking Loss.**\n    | Model (Model) | Loss Application Layer | Loss Weight (α) | ImageNet Top-1 Acc (%) | Avg. Downstream Acc (%) |\n    | :------------ | :--------------------- | :-------------- | :--------------------- | :---------------------- |\n    | Swin-T        | Penultimate (Ours)     | 1.0 (Default)   | 80.4                   | 71.2                    |\n    |               | None (w/o PCA Loss)    | -               | 80.3                   | 69.6                    |\n    |               | Penultimate            | 0.1             | 80.4                   | 70.3                    |\n    |               | Penultimate            | 10              | 80.2                   | 70.7                    |\n\nThank you again for your valuable time and profound insights. We believe these supplementary experiments and clarifications can address your concerns and further demonstrate the value of our work."}, {"review_id": "jg2vFcxu4J", "reviewer": "Reviewer_Xx2u", "summary": "The paper introduces a simple and efficient way to quantize ViT models into 4-bit with good performance retaining compared with full-precision counterparts.\n\nThe key idea is to avoid long and memory-heavy training. Instead, the authors suggest a two-step process:\n\t1.\tFirst, they do a very short training (just 1 epoch) where they only adjust how activations are quantized, this helps the model handle the lower precision.\n\t2.\tThen, they use a lightweight post-processing step to quantize the weights, relying on existing methods.\n\nThis approach is extremely fast, uses less memory than traditional training, and still keeps the model accurate, both on standard classification tasks and more complex tasks like object detection. Compared with some previous QAT methods, the authors claim that their algorithm would lead to more generalization capacity of the quantized model.\n\nIn short, the method is practical, fast, and easy to apply, especially for deployment of large models on resource-limited hardware.", "questions": "1.How general is the proposed method beyond Vision Transformers? Have the authors tested GPLQ on other model families, such as CNNs or MLP-based architectures, to validate its broader applicability?\n\n2. How does the method perform under different quantization settings? For example, would the same two-stage strategy work well at more aggressive bit-widths (e.g., W3A3 or W2A2), or is it only effective for W4A4?\n\n3. Is activation quantization truly the main bottleneck in ViT quantization?\nWhile the paper emphasizes this point, can the authors provide more in-depth analysis or ablation studies to isolate the relative contributions of activation vs. weight quantization?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Below are several strengths of the paper:\n\n1. The method only needs one epoch of light training, and the rest is fast post-processing, claiming 100× faster than full quantization-aware training.\n\n2. Because only activation scales are trained and post-QAT offline quantization is then applied on the weights, it uses less memory than normal FP32 training, and can even be done on a single common GPU.\n\n3. This algorithm unlocks the boundary of ImageNet which are the common generalization issues of the previous algorithms, for object detection (COCO) and fine-grained classification, and the method holds up well, beating other quantization baselines.\n\nBelow are several weaknesses of the paper:\n\n1. I think it is a clever combination and decoupling of the activation quantization, PTQ or feature perseveration. But it is a little tough to say whether it is a truly novel algorithm.\n\n2. The author is telling a story that taking care of the activation is probably all you need when dealing with quantization (for this paper, ViT series) and weight quantization does not significantly affect the paper. I think more results for example, for different quantized bit configurations, and even other model families should be verified to make this story comprehensive. I like the story but I mean it is better to make it clear whether it is just for ViT under 4bits or it could be way beyond than this.", "was_revised": true, "final_justification": "I would like to keep my score. I think 4 is a well-suited score for a experimental-based paper like this, whether the paper itself could be better if more theoretical contributions could be made.", "rebuttal": "Dear Reviewer Xx2u,\n\nThank you very much for your recognition of our work and for your insightful questions. You have accurately summarized the core ideas and advantages of our GPLQ method—its efficiency, low resource consumption, and strong generalization ability. We greatly appreciate your interest in our \"activation is the key to quantization\" narrative and your valuable questions regarding its generality.\n\n---\n\n**Regarding Weaknesses and Questions:**\n\nYou have merged several core questions into the \"Weaknesses\" section, and we will address them together here.\n\n1.  **Regarding Novelty: A \"clever combination\" or a \"truly new algorithm\"?** We acknowledge that GPLQ skillfully integrates several existing technologies. However, we believe its core contribution and novelty are manifested in the following aspects:\n    * **Paradigm Innovation:** We are the first to systematically propose and validate the \"activation-first (Act-QAT), weights-later (Weight-PTQ)\" two-stage decoupled quantization paradigm. This is fundamentally different from traditional PTQ (which handles W and A simultaneously) and traditional QAT (which trains W and A quantizers simultaneously).\n    * **Insight-Driven:** This paradigm is based on our core insight that \"preserving the optimization basin\" is crucial for generalization. By fine-tuning only the activation quantizers for just 1 epoch, we successfully keep the model within the high-performance optimization region of the original FP32 model, which is validated in **Figure 4(a)** and **Table 5** of our paper.\n    * **Framework Modularity:** Our supplementary experiments (Table B.5) show that as a framework, GPLQ's internal components (such as the initialization method for PTQ) are replaceable. This demonstrates the robustness and generality of its design, rather than it being a fixed, rigid process.\n\n    **Table B.5: Verifying the interchangeability of different components within the GPLQ framework.**\n    | Model  | Act-QAT Initialization        | ImageNet Top-1 Acc (%) |\n    | :----- | :---------------------------- | :--------------------- |\n    | Swin-T | Percentile (as in RepQ-ViT)   | 79.8                   |\n    |        | MSE-based                     | 79.8                   |\n    |        | MinMax                        | 79.6                   |\n\n2.  **Regarding Generality: Can it go beyond ViTs and 4-bit settings?** This is a point we are most eager to clarify. The design principles of GPLQ are general. To prove this, we conducted two key supplementary experiments:\n    * **Extension to CNN Architectures:** We tested GPLQ on the classic ResNet family and achieved performance significantly superior to SOTA PTQ methods (RepQ-ViT), and even comparable to some long-duration QAT methods (see **Table B.3** of our response to Reviewer QuB9). This demonstrates that GPLQ's effectiveness is not limited to Transformer architectures.\n    * **Exploration of Lower Bit-widths:** We conducted more challenging W3A3 quantization experiments (see **Table B.6** below). Although a drop in accuracy is inevitable, GPLQ's two-stage strategy still provides a strong baseline for low-bit scenarios. We believe the characteristic that \"activations are more sensitive\" becomes even more pronounced at lower bit-widths, where GPLQ's advantages may be more significant.\n\n**Table B.6: Performance of GPLQ in a lower-bit (W3A3) setting.**\n| Model | Method | ImageNet Top-1 Acc (%) | Avg. Downstream Acc (%) |\n| :--- | :--- | :--- | :--- |\n| Swin-T | FP32 | 81.2 | 70.79 |\n| | RepQViT (W3A3) | 52.7 | 39.26 |\n| | OFQ (W3A3) | 81.1 | 38.47 |\n| | GPLQ (W3A3) | 74.2 | 65.76 |\n| Swin-S | FP32 | 83.2 | 70.07 |\n| | RepQViT (W3A3) | 65.9 | 53.35 |\n| | GPLQ (W3A3) | 78.3 | 64.64 |\n| Swin-B | FP32 | 85.3 | 78.93 |\n| | RepQViT (W3A3) | 65.8 | 29.82 |\n| | GPLQ (W3A3) | 80.3 | 74.15 |\n| Swin-L | FP32 | 86.3 | 80.89 |\n| | RepQViT (W3A3) | 71.3 | 66.36 |\n| | GPLQ (W3A3) | 80.0 | 73.36 |\n\n3.  **Is activation quantization *truly* the main bottleneck?** We firmly believe so, especially when pursuing high generalization capability. **Figure 2** in our paper intuitively shows that in PTQ, quantizing activations alone leads to a much larger accuracy drop than quantizing weights alone. More importantly, the experiment in **Table 5** reveals the deeper reason: traditional QAT methods (like OFQ), in their effort to fit both quantized activations and weights, cause their internal \"latent FP32 weights\" to severely deviate from the original model. This leads to a seemingly high ImageNet accuracy (81.9%) but far inferior generalization ability (60.73%) compared to the FP32 model (70.79%) and our GPLQ (71.18%). This indicates that improper handling of activations can force the weights into a \"bad\" optimization basin, thereby harming generalization. GPLQ successfully avoids this pitfall by prioritizing the stabilization of activations.\n\nIn summary, we believe that GPLQ is not just a clever combination, but a new quantization framework driven by deep insights and possessing broad generality. We are using further experiments to clearly define its scope of application and believe it has the potential to become a practical and efficient solution for low-bit quantization across various model architectures.\n\nThank you again for your valuable feedback!"}, {"review_id": "aXCe8KGUls", "reviewer": "Reviewer_FJP6", "summary": "This work proposes GPLQ, a fast and generalization-preserving quantization pipeline tailored for Vision Transformers (ViTs). Motivated by empirical observations on the critical role of activations in quantization quality and the importance of preserving the original optimization basin for generalization, the authors design a two-stage approach: a one-epoch QAT phase with PCA-based activation feature mimicking, followed by PTQ for weight quantization. The method preserves the generalization performance of the original model while reducing the computational cost compared to full QAT and improving quantization quality over standard PTQ.", "questions": "1. Regarding Weaknesses 1–3, could you provide a deeper analysis of the proposed motivations and actual effects of proposed method? Specifically, are the motivations rooted in characteristics that are unique to Vision Transformers? And, how does applying QAT to activations first improve the overall quantization quality?\n2. The main paper does not include wall-clock time comparisons between GPLQ and other QAT/PTQ methods. Could the authors provide runtime measurements for each method to support the claimed efficiency benefits?", "limitations": "Authors discussed limitations in the paper with independent section.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:  \n1. The motivation and its connection to proposed method is clear and well-explained, making the paper easy to follow.\n2. Empirical results convincingly demonstrate the benefits of GPLQ over standard QAT and PTQ approaches.  \n\nWeaknesses:\n1. Although GPLQ is proposed as a quantization scheme tailored for ViTs, the method does not seem to leverage any distinctive architectural or representational characteristics unique to ViT models.\n2. The proposed method is heavily grounded in empirical observations. While the two key motivations—(1) preserving the original optimization basin for generalization and (2) the greater importance of activations over weights—are intuitively appealing, they are not analyzed in depth or supported by theoretical justification. This weakens the overall contribution.\n3. The effects of separating the quantization process and applying activation quantization first are not thoroughly explained. It remains unclear why or how this ordering yields improved quantization results.\n4.  As acknowledged in the limitations, GPLQ relies heavily on existing techniques. Although these components are combined effectively, the lack of in-depth analysis regarding the motivations and their interactions raises concerns about the novelty of the work.", "was_revised": true, "final_justification": "Authors provided analysis of framework during the discussion period. Thus I rasied my score to 4 and inclined to recommend acceptance of the work.", "rebuttal": "Dear Reviewer FJP6,\n\nWe sincerely thank you for the time and effort you have dedicated to reviewing our paper and for raising several profound and challenging questions. Your rigorous scrutiny has prompted us to think more deeply about and further validate the core contributions and motivations of our work. Although we are disheartened by your current \"Reject\" rating, we are confident that the following clarifications and the substantial new experimental evidence we have added will fully address your concerns. We hope to persuade you to reconsider your evaluation of our work.\n\n---\n\n**On missing theoretical analysis and/or using existing technical components:**\n\nYou may not be familiar with the particular state-of-the-art of QAT. Indeed, there are papers on QAT, but they are far from being practically usable in real-world applications yet. As we briefly discussed in the introduction, a QAT-trained network or backbone cannot be generalized/used in a different scenario, the training took 37.5GB memory during training Swin-T (a tiny model!), and its training is very long (e.g., 600 epochs). We believe we need a framework that can train QAT models with a small GPU memory footprint and very fast, and the trained model should be generalizable.\n\nOnly after such a framework is available does it become feasible to perform theoretical analysis. And, it does not hurt to use existing components in such a framework—the contribution or breakthrough is a framework that makes QAT practically possible (e.g., reducing GPU memory from 37.5GB to 6.6GB, training from 600 epochs to 1 epoch, and turning a non-generalizable model into a generalizable one), not any specific component. In fact, as we will explain below, all such components can be replaced freely with other components, yet the GPLQ framework still achieves our goals: generalizable, practical, and lightning-fast.\n\n**Regarding Weaknesses and Questions:**\n\n1.  **On the method not being ViT-exclusive and the motivation lacking deep theoretical analysis (Weaknesses 1-3 & Question 1):**\n    * **Generality as a Strength, Not a Weakness:** Your observation that GPLQ does not rely on ViT-specific features is very accurate, and we believe this is precisely one of GPLQ's core strengths. A truly valuable quantization method should possess good generality. To strongly demonstrate this, we have added experiments with GPLQ on classic CNN architectures (ResNet). As shown in **Table B.3** of our response to **Reviewer QuB9**, GPLQ performs exceptionally well on ResNets, significantly surpassing SOTA PTQ methods designed for ViTs. This indicates that the principles we propose—\"activation-first\" and \"preserving the optimization basin\"—are cross-architectural.\n    * **A Deeper Analysis of Why \"Activation-First\" Works:** You asked why this sequence improves quantization quality. The core mechanism lies in **decoupling and stabilization**. In traditional QAT, the quantizers for weights and activations are optimized simultaneously, creating a coupled system where they influence each other and can easily lead to oscillations. In particular, errors from sensitive activation quantization can mislead weight updates, forcing the weights to deviate from the original, superior optimization basin. GPLQ's first stage optimizes only the activation quantizers (while weights remain learnable at FP32), allowing the model to first adapt to the information loss from activation quantization and find a weight state that is most robust to it. This process is more stable (as shown by the smooth loss curve in **Figure 4(a)**) and remains within the original basin (**Table 5** of our response to Reviewer Xx2u). Once activations are stabilized, the second-stage weight PTQ faces a simpler, more deterministic problem (quantizing from W32A4 -> W4A4, instead of W32A32 -> W4A4), thus making PTQ methods (like QwT) more effective.\n    * **Reinforcing the Empirical Motivation:** We acknowledge that GPLQ originated from empirical observations, but these observations are well-supported by data. In addition to the evidence in the original paper, we plan to add visualizations of the weight space to more intuitively show how GPLQ keeps the weights in a region close to the original FP32 model, whereas traditional QAT causes them to drift to a distant location. This will provide stronger visual evidence for our core motivation.\n\n2.  **Regarding the lack of Wall-Clock Time Comparison (Question 2):** Thank you for pointing this out. We actually provided this comparison in **Figure 1** of the paper. To more clearly showcase GPLQ's huge efficiency advantage, we have organized the information from Figure 1 into a detailed table (see Table B.7 below). This table explicitly lists the detailed comparison between GPLQ and methods like Q-Var and OFQ in terms of ImageNet accuracy, average downstream task accuracy, **training time (minutes), and peak memory (GB)**. The data shows that GPLQ's training time (35 minutes) is approximately **500 times faster** than Q-Var (17,400 minutes) and OFQ (18,360 minutes), while also having lower memory consumption. These concrete numbers strongly support our claim of \"lightning-fast\" efficiency.\n\n    **Table B.7: Detailed comparison of performance and resource overhead between GPLQ and SOTA methods.**\n    | Model  | Method          | ImageNet Acc (%) | Avg. Downstream Acc (%) | Training Time (min) | Peak Memory (GB) |\n    | :----- | :-------------- | :--------------- | :---------------------- | :------------------ | :--------------- |\n    | Swin-T | Full-precision  | 81.2             | 70.79                   | 3078                | 7.2              |\n    |        | Q-Var           | 82.4             | 68.12                   | 17400               | 25.9             |\n    |        | OFQ             | 81.9             | 60.73                   | 18360               | 37.5             |\n    |        | **GPLQ** | **79.8** | **71.18** | **35** | **6.6** |\n    |        | RepQViT         | 73.0             | 62.79                   | 3                   | 2.3              |\n\n3.  **Regarding Dependence on Existing Technologies and Novelty (Weakness 4):** We understand your concern about novelty. We want to emphasize that GPLQ's innovation lies not in inventing an isolated component, but in proposing and validating a **completely new and efficient quantization framework**. To prove that GPLQ is not simply reliant on specific technologies, our component replacement experiments (**Table B.5** of our response to Reviewer Xx2u) show that its performance remains robust. The true value of GPLQ is that it provides the community with a plug-and-play, extensible new approach that can bring QAT from theory to practice.\n\nWe sincerely hope that the detailed responses above, along with the extensive new experiments we have conducted to address your comments (CNN generality, training duration ablation, detailed efficiency data, etc.), can fully demonstrate the depth, novelty, and importance of our work. We kindly ask that you re-evaluate our contribution based on this new information. Thank you again for your valuable feedback."}, {"review_id": "GMXYZLnoEH", "reviewer": "Reviewer_yZXJ", "summary": "The author proposed GPLQ, a general, practical, and lightweight QAT method for ViT. The claimed contributions are threefold: (1) two insights, i.e., activation is more important, and quantization should not change the optimization basin, (2) activation-first sequential quantization, and (3) open-source code.\n\nThe key idea lies in keeping the model in the same optimization to reach the best generalization ability. Therefore, in stage 1, GPLQ only optimizes the activation's quantizers with the commonly used QAT method. And in stage 2, GPLQ combines RepQ and QwT to perform PTQ on weights.", "questions": "- It seems that $\\mu_t$ in Eq (2) can be eliminated?\n- The author claimed that QAT methods often boast higher accuracy than their FP32 counterparts. But in Fig. 4(a), why is the train loss of normal QAT higher than GPLQ? Should normal QAT overfit on the training set?\n- How to better identify \"the same basin\"? The train loss in Fig.4 (a) is not very convincing.\n- In Table 6, please unify the capital letter of w/o and w/.\n- Please unify the capital letters of lines 101, 245.\n- Typo: line 7 lacking -> lack.\n- Writing: line 14 keep it stay in the same \"basin\" -> keep it in the same \"basin\";", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n- The second insight, i.e., quantization should not change the optimization basin, and the corresponding solution is novel and effective.\n- The writing is clear and easy to follow.\n\n### Weakness\n- The first insight, i.e., activation is more important than weight, is already a common sense in the quantization community[1].\n- Both the QAT and PTQ methods used in Stage 1 and Stage 2 are not original.\n- A broader QAT comparison is very critical to support the effectiveness of the proposed method, although the authors mentioned the rare QAT open-source code problem in the limitations section. This paper will greatly benefit from a more comprehensive comparison.\n\n\n[1] AWQ:Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration, MLSys 2024 Best Paper Award", "was_revised": true, "final_justification": "The authors have addressed my concerns, and I decide to keep my score at 4.", "rebuttal": "Dear Reviewer yZXJ,\n\nThank you very much for your detailed review and positive evaluation of our paper! We are delighted that you recognize the novelty and effectiveness of our \"preserving the optimization basin\" insight, as well as the clarity of our paper. The questions you raised are very specific and valuable, and they will help us to further improve our paper.\n\n---\n\n**Regarding Weaknesses:**\n\n1.  **On \"activations are more important than weights\" being common knowledge:** We completely agree with your point. This insight has become a consensus in the quantization community, especially in recent excellent works like AWQ, which you mentioned. We cited this viewpoint in our paper to build the logical starting point for our methodology. We want to emphasize that the novelty of GPLQ lies not in restating this consensus, but in **how we construct a brand-new QAT framework based on it**. Unlike PTQ methods such as AWQ, GPLQ innovatively applies this insight to QAT by designing the \"activation-first\" two-stage process. This allows us to enjoy the high accuracy of QAT while circumventing its drawbacks of training instability and poor generalization.\n\n2.  **On QAT and PTQ methods not being original:** Yes, we adopted mature QAT and PTQ techniques as foundational components within the GPLQ framework. Our contribution is not in inventing new quantization operators, but in proposing a superior, decoupled quantization **process**. This is analogous to how ResNet's contribution was not inventing the convolution, but proposing the \"residual connection\" framework that enabled convolutional networks to be deeper and more powerful. Our supplementary **Table B.5**(response to Reviewer Xx2u) also demonstrates the modular nature of the GPLQ framework, showing that its performance is not rigidly tied to any specific component.\n\n3.  **On a broader QAT comparison:** We fully agree that a more comprehensive comparison would be more convincing, and we candidly discussed this issue in the \"Limitations\" section. The main obstacle is indeed the scarcity of open-source code for SOTA QAT methods, especially implementations that can support large ViT models and multiple tasks. Under the current circumstances, we have done our best to compare against the SOTA methods that were accessible and reproducible. The overwhelming advantages in efficiency and memory shown in **Figure 1** and the newly added **Table B.7**(response to Reviewer FJP6) are, in themselves, a core contribution of our method.\n\n**Regarding Questions:**\n\n* **Q1: Regarding $\\mu_t$ in Eq (2):** Thank you for your careful observation! The mean term $\\mu_t$ here **cannot be eliminated**. In PCA projection, the data must first be centered (i.e., by subtracting the mean) before being projected onto the principal components $V_{sel}$. The means of the teacher features ($f_t$) and the student features ($f_s$) are different. Therefore, both must be centered using the *teacher's* feature mean, $\\mu_t$, to ensure they are being compared in the same coordinate system.\n\n* **Q2: Regarding the training loss of traditional QAT in Figure 4(a):** This is an excellent question that reveals a key point we wanted to make. The loss of traditional QAT rises sharply at the beginning of training (\"escaped basin\") because it is searching for a **new optimization basin** that is adapted to the quantization operations. While this new basin might achieve a lower loss value on the ImageNet training set (i.e., overfitting), it is often \"sharper,\" leading to poorer generalization. In contrast, GPLQ's loss is low and smooth from the start because it **remains within the original FP32 model's broad and highly generalizable \"good\" basin**. Therefore, while the final loss value of QAT on the training set might be lower, its \"path\" and \"destination\" are detrimental to generalization, whereas GPLQ's are beneficial.\n\n* **Q3: How to better identify the \"same basin\":** You've touched upon a core conceptual issue. We admit that \"basin\" is a figurative description. We support this argument with three dimensions of evidence: (1) **Training loss curve (Figure 4(a)):** GPLQ's smooth loss curve indicates its optimization process is stable and does not drastically jump out of the initial state. (2) **\"Internal FP32\" weight performance (Table 5 of our response to Reviewer Xx2u):** This is the most compelling evidence. The performance of GPLQ's trained \"latent FP32\" weights (81.1%) is nearly identical to the original FP32 model (81.2%), whereas traditional QAT's drops significantly (68.9%). (3) **Weight space visualization:** We plan to include a visualization of the weight space in the final version, using methods like t-SNE/PCA to intuitively show that GPLQ's weights are spatially closer to the original weights.\n\n* **Q4 & Q5 & Spelling/Grammar Errors:** Thank you so much for pointing out these typographical and writing issues with such care! We apologize for these oversights and promise to correct the casing of `w/o PCA` and `w/PCA` (as in Table 6 of our response to Reviewer Xx2u) and all other formatting and textual issues you have identified in the final version.\n\nThank you again for your valuable review comments; your suggestions are of immense help in improving the quality of our paper."}]

Author rebuttal:

Meta-review: This paper introduces GPLQ, a two-stage quantization framework for Vision Transformers that first performs a lightweight one-epoch activation QAT with feature-mimicking loss, followed by PTQ for weights. Reviewers highlighted the framework’s practicality, efficiency, and clarity, as well as strong results on multiple benchmarks. While it is noted that the work builds on existing components and would benefit from deeper theoretical analysis, the rebuttal provided further experiments and convincing clarifications on stability, efficiency, and generality. Overall, the paper makes a practical contribution toward practical low-bit quantization. The AC encourages the authors to incorporate reviewer feedback into the final version for further strengthening.
Decision: Accept (poster)