2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution
Authors: Kai Liu
Haotong Qin
Yong Guo
Xin Yuan
Linghe Kong
Guihai Chen
Yulun Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Quantization
Image super resolution
Low bit
Post-training quantization.
Primary area: optimization_for_deep_networks
TL;DR: A dual-stage low-bit post-training quantization algorithm for image super-resolution

Abstract: Low-bit quantization has become widespread for compressing image super-resolution (SR) models for edge deployment, which allows advanced SR models to enjoy compact low-bit parameters and efficient integer/bitwise constructions for storage compression and inference acceleration, respectively. However, it is notorious that low-bit quantization degrades the accuracy of SR models compared to their full-precision (FP) counterparts. Despite several efforts to alleviate the degradation, the transformer-based SR model still suffers severe degradation due to its distinctive activation distribution. In this work, we present a dual-stage low-bit post-training quantization (PTQ) method for image super-resolution, namely 2DQuant, which achieves efficient and accurate SR under low-bit quantization. The proposed method first investigates the weight and activation and finds that the distribution is characterized by coexisting symmetry and asymmetry, long tails. Specifically, we propose Distribution-Oriented Bound Initialization (DOBI), using different searching strategies to search a coarse bound for quantizers. To obtain refined quantizer parameters, we further propose Distillation Quantization Calibration (DQC), which employs a distillation approach to make the quantized model learn from its FP counterpart. Through extensive experiments on different bits and scaling factors, the performance of DOBI can reach the state-of-the-art (SOTA) while after stage two, our method surpasses existing PTQ in both metrics and visual effects. 2DQuant gains an increase in PSNR as high as 4.52dB on Set5 (x2) compared with SOTA when quantized to 2-bit and enjoys a 3.60x compression ratio and 5.08x speedup ratio. The code and models are available at https://github.com/Kai-Liu001/2DQuant.

2DQuant: Low-bit Image Super-Resolution
Quantization for
Kai Liu', Haotong Qin', Yong Guo Xin Yuan',
Linghe Kong"*, Guihai Chen', Yulun Zhang
'Shanghai Jiao Tong University, 2ETH Zürich, 'Max Planck Institute for Informatics, *Westlake University
Abstract
Low-bit quantization has become widespread for compressing image super- resolution (SR) models for edge deployment, which allows advanced SR models enjoy compact low-bit parameters and efficient integer/bitwise constructions for storage compression and inference acceleration, respectively. However, it is notorious that low-bit quantization degrades the accuracy of SR models compared to their full-precision (FP) counterparts. Despite several efforts to alleviate the degradation, the transformer- baseed SR model still suffers severe degradation due to its distinctive activation distribution. In this work, we present a dual-stage low- bit post-training quantization (PTQ) method for image super-resolution, namely 2DQuant, which achieves efficient and accurate SR under low-bit quantization. The proposed method first investigates the weight and activation and finds that the distribution characterized by coexisting symmetry and asymmetry, long tails. Specifically, we propose Eestiibution-Orinnne using different searching strategies to search a coarse bound for quantizers. To obtain refined quantizer parameters, further propose Distillation Quantization Calibration (DQC), which employs distillation approach to make the quantized model learn from its FP counterpart. Through extensive experiments on different bits and scaling factors, the performance of DOBI can reach the state-of-the (SOTA) while after stage two, our method surpasses existing PTQ in both met- rics and visual effects. 2DQuant gains an increase in PSNR as high as 4.52dB on Set5 (x2) compared with SOTA when quantized to 2-bit and enjoys a 3.60 compression ratio and 5.08x speedup ratio. The code and models are available at https://github.com www.a///oooo//aoooOoO/aA Lii-Liu001/2Quaan.
Bound Initialization (DOBI),
Introduction
As one of the most classical low-level computer vision tasks, image super-resolution (SR) has been widely studied with the significant development of deep neural networks. With the abil- ity to reconstruct high-resolution (HR) image from the corresponding low-resolution (LR) image, SR has been widely used in many real-world scenarios, including medical imaging [13, 21, 19], surveillance [46, 39], remote sensing [1], and mobile phone photography. With massive parameters, DNN-based SR models always require expensive storage and computation in the actual application. Some works have been proposed to reduce the demand for computational power of SE models, like lightweight architecture design and compression. One kind of approach investigates lightweight and efficient models as the backbone for image SR. This progression has moved from the earliest convolutional neural network (CNNs) [10, 11, 25, 49] to Transformers [48, 32, 44, 42, 4, 3] and their combinations. The parameter number significantly decreased while maintaining or even enhancing performance. The other kind of approach is compression, which focuses on reducing the parameter (e.g., pruning and distillation) or bit-width (quantization) of existing SR models.
Model quantization [7, 9, 20, 31] a technology that compresses the floating-pon ppini parameters of neural network into lower bit-width. The discretized parameters are homogenized into restricted
Cooresponding authors: Yulun Zhang, yulun100@ gmail.com, Linghe Kong, linghe.kong @ @stu.eduu.cn
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

candidate values and cause heterogenization between the FP and quantized models, leading to severe performance degradation. Considering the process, quantization approaches can be divided into quantization-awar training (QAT) and post-training quantization (PTQ). QAT simultaneously optimizes the model parameters and the quantizer parameters [6, 16, 26, 50], allowing them to adapt mutually, thereby more effectively alleviating the degradation caused by quantization. However, QAT often suffers from a heavy training cost and a long training time, and the burden is even much heavier than the training process of the FP counterparts, which necessitates large amount of compatibility and makes it still far from practical in maining-resoure--le
scenarios.
Fortunately, post-training quantization emerges as a promising way to quantize models at a low training cost. PTQ fixes the model parameters and only determines the quantizer parameters through search or optimization. Previous researches [41, 26] on PTQ for SR has primarily focused on
IRR
Perranik (2)
L'han IoC: iny. not
[(]
Figure 1: Existing methods suffer from blurring artifacts.
CNN-based models such as EDSR [33] and SRResNet [24]. However, these quantization methods are not practical for deployment for two reasons. Firstly, these CNN-based models themself require huge space and calculation resources. Their poor starting point makes them inferior to advanced models in terms of parameters and computational cost, even after quantization. As shown in Table 1, the light version of SwinIR needs only 16.2% parameters and 15.9% FLOPs compared with quantized EDSR. But its PSNR metric is close to that of the FP EDSR. While the previous PTQ algorithm, DBDC+Pac, suffers from unacceptable degradation in both visual and metrics. Secondly, most of these methods can not adapt well to Transformer-basse quantized transformers. As shown in Figure I, when applied on SwinlR, the existing methods still suffer from distorted artifacts compared with FP or HR.
models because of the deterioration of self-attention in
Therefore, we conducted a post-training quantization analysis on super-resolution with a classical Transformer-bast SwinIR [32]. The weight and activation distribution is characterized by coexisting symmetry and asymmetry, long tails. Firstly. if the previous symmetric quantization method is applied for asymmetric distribution, at least half of the candidates are completely ineffective. Besides, the long tail effect causes the vast majority of floating- point numbers to be compressed into one homogenization. Furthermore, with such a small number of parameters, SwinIR's 's information has been highly compressed, and quantizing the model often results in significant performance degradation. Nevertheless, the excellent performance and extremely low computational requirements of Transformer based models are precisely what needed for deployment in real- cal-world scenarios.
Table I: Complexity and performance (x4).
Mosld
BO5K (3g ERIS (180)
Swaatt-iphh [3]] DaII--S (188) Oan cles)
model
Parazze (3BB)
37233
21.35
3
137
1.77
Ope (0)
$23.34
PNSR Librart00
25.55
peadingg-pinttttt
candidates, leading worse parameter
In this paper, we propose 2DQuant, a two-stage PTQ algorithm for image super-resolution tasks. To enhance the representational capacity in asymmetry scenarios, employ a quantization method with two bounds. The bounds decide the candidate for numbers out of range and the interval of candidates in range. First, we propose distribution-oriente Bound Initialization (DOBI), fast MSE-based searching method. It is designed to minimize the value heterogenization between quantized and FP models. Two different MSE [5] search strategies are applied for different distributions to avoid nonsense traversal. This guarantees minimum value shift while maintaining high speed and efficiency in the search process. Second, we propose Distillation Quantization Calibration (DQC), training- based method. It is designed to adjust each bound to its best position finely. This ensures that the outputs and intermediate feature layers of the quantized model and that of the FP model should remain as consistent as possible. Thereby DQC allows the quantizer parameters to be finely optimized toward the task goal. The contributions of this paper can be summarized follows:
(1) To the best of our knowledge, we are the first to explore PTQ with Transformer- based model in SR thoroughly. We design 2DQuant, unique and efficient two-stage PTQ method (see Figure 2) for image super-resolution, which utilizes DOBI and DQC to optimize the bound from coarse to fine.
(2) In the first stage of post-quantization, customized search strategies for different distributions to balance speed and accuracy. the second stage, we design DQC, a more fine-gra -raiggr optimization- based training strategy, for the quantized model, ensuring it aligns with the FP model on the calibration set.
use DOBI to search for quantizer parameters, employing
2

---

24
FP Model
Moving Simutaneously
[(l, u arg min
1/ Mella
X
FP weights and activations
Quant Model
4
Distillation Quantization Calibration
Full-precision
Quantized Layer
Searching Direction
Layer
Moving Upper Bound Only
Distribution Oriented Bound Initialization
Initial Bound
DOC Loss
Forward
Figure 2: The overall pipeline of our proposed 2DQuant method. The whole pipeline contains two stages, optimizing the clipping bound from coarse to fine. In stage 1, we design DOBI efficiently obtain the coarse bound. In stage 2, DQC is performed to finetune clipping bounds and guarantee the quantized model learns the full full-precision (FP) model's s feature and output information.
(3) Our 2DQuant can compress Transformer- based model 4,3,2 bits with the compression ratio being 3.07x, x, 3.31 x, and 3.60x and speedup ratio being 3.99 x, 4.47x, x, and 5.08 x. No additional module is added so 2DQuant enjoys the theoretical upper limit of compression and speedup.
(4) Through extensive experiments, our 2DQuant surpasses existing SOTA on all benchmarks. We gain an increase in PSNR by as high as 4.52dB in Set5 (x2) when compressed to 2 bits, and our method has a more significant increase when compressed to lower bits.
2 Related work
Image super-resolution. Deep CNN networks have shown excellent performance in the field of image super-resolution The earliest SR-CNN [10, 11] method adopted CNN architecture. surpassed previous methods in the image super-resolution domain. In 2017, EDSR [33] won the NTIRE2017 [40] championship, becoming a representative work of CNNs in the SR by its excellent performance. Thereafter, with the continuous development of Vision Transformers (ViT) [12], models based on the ViT architecture have surpassed many CNN networks. These Transformer- based models achieve significant performance improvements and they have fewer parameters and lower computational costs. Many works have modified the VIT architecture, achieving continuous improvements. A notable example is SwinIR [32]. With simple structure, it outperforms many CNN- based models. However, previous explorations of post-quantization in the super-resolution domain have been limited to CNN-based models. They focus models like EDSR [33] SRResNet [24]. is a far cry from advanced models no matter in parameters, FLOPs, or performance. Currently, there is still a research gap in post-quantization for Transformer architectures.
Model quantization. Model quantization is used to compress large models [30, 28, 29] and can be divided into QAT and PTQ. QAT is widely accepted due to its minimal performance degradation. PAMS [26] utilizes a trainable truncated parameter to dynamically determine the upper limit of the quantization range. DAQ [17] proposed a channel-wise distribution-aware quantization scheme. CADyQ [16] is designed for SR networks and optimizes the bit allocation for local regions and layers in the input image. However, QAT usually requires training for as long as or even longer than the original model, which becomes a barrier for real scenarios deployment. Instead of training the model from scratch, existing PTQ methods use the pre-trained models. PTQ algorithms only find the clipping bound for quantizers, saving time and costs. DBDC+Pac [41] ]] the first to optimize the post-training quantization for image super-resolution task. It outperforms other existing PTQ algorithms. Whereas, they only focus on EDSR [33] and SRResNet [24]. Their 4-bit quantized version is inferior to advanced models in terms of parameters and computational cost, let alone performance. reveals a promising result for PTQ on SR, and using more advanced model could bridge the gap between nigh-peeffrmance models and limited calculation resource scenarios.
3 Methodology
To simulate the precision loss caused by quantization, we use fake- lake-quantize 222lieeqquanizzation dequantization, for activations and weights. and the process can be written as
te Clip(u, L,u), u), e, Round( 2 - 1
(ve 1)), 0y
Ur
+ I
(1)
2 1
3

---

Wa
SoftMax
Wx
BMM
Scale
LayerNorm
K
Projection
+
FC1
BELU
BMM
W,
FP32
INT4
Figure 3: Quantization scheme for SwinIR Transformer blocks. Fake quantization and INT arithmetic are performed in all compute-intensive operators including all linear layers and batch matmul. Lower bits such as 3 or even 2 are also permitted. Dropout of attention and projection is ignored
where o denotes the value being fake quantized, which can be weight or activation. 1 and u are the lower and upper bounds for clipping, respectively. Clip(v, u) max (aximmn, (v, u), Z). and Round rounds the imput value to the nearest integer. ee denotes the value after clipping, and denotes the integer representation of v, and Vy denotes the value after fake quantization. The Clip and Round operations contribute to reducing the parameters and FLOPs but also introduce quantization errors.
Figure 3 shows the basic structure of the Transformer block. We have quantized all the modules with a significant computational load within them, effectively reducing the model's FLOPs. Table 2 shows the FLOPs needed for each module. The Linear layers and matrix multiplication account for approximately 86% of the compu- tation load, which are all transformed into integer arithmetic. When performing gradient backpropagation, we follow the Brraigh-Thhoouge Estimator [8] (STE) style:
Table 2: FLOPs distribution.
Module
FLOPs (G)
Ratio (%)
Linear & BMM
14.34
85.66
Cony
2.33
13.90
Other
0.07
0.44
Total
16.74
100.00
Btg dv
Vc -1 Oug Ou
0/ -6
Ou du 2" Dr
2 u- al al 2" 2
u 1
dr. lu v) and J. where Du
(-) denotes Heaviside step function [47]. This formula approximates the direction of gradient backpropagation, allowing training-basse optimization to proceed. The derivation of the formula can be found in the supplementary material.
Figure 2 shows the whole pipeline of 2DQuant, which is two-stage coarse-to- to-fine post-training quantization method. The first stage is DOBI, using two strategies to minimize the value shift while the second stage is DQC, optimizing two bound of each quantizer towards the task goal.
3.1 Analysis of data distribution
To achieve better quantization results, we need to analyze the distribution of the model's weights and activations in detail. We notice that the data distribution shows significantly different pattern from previous explorations, invalidating many of the previous methods. The weights and activations distri- bution of SwinlR is shown in Figure 4. More can be found in supplemental material. Specifically, the weights and activations of SwinIR exhibit noticeable long-tail, coexisting symmetry and asymmetry.
Weight. The weights of all linear layers are symmetrically distributed around zero, showing clear symmetry, and are generally similar to normal distribution. This is attributed the weight decay applied to weights, which provides muantization-friendee distributions. From the value shift perspective. both symmetric and asymmetric quantization are tolerable. Whereas, from the vantage point of task objectives, asymmetric quantization possesses the potential to offer markedly enhanced information density, thus elevating the overall precision of the computational processes involved.
Activations. As for activations, they exhibit obvious periodicity in different Transformer Blocks. For V or the input of FCI, the obtained activation values are symmetrically distributed around However, for the attention map or the input of FC2 in each Transformer Block, due to the Softmax calculation or the GELU [14] activation function, the minimum value is almost fixed, and the overall distribution is similar to an exponential distribution. Therefore, the data in SwinIR's weights and activations exhibit two distinctly different distribution characteristics. Setting asymmetric quantization and different search strategies for both can make the search rapid and accurate.
3.2 Distribution-oriente bound initialization
Because the data distribution exhibits significant long- low effective bits. Common clipping methods include density-based, ratio-based, and MSE-based approaches. The first two require manually specifying the clipping ratio, which significantly affects the clipping outcome and necessitates numerous experiments to determine the optimal ratio. Thus we proposed the Distribution- Arriibtttio--Oiinteed Bound Initialization (DOBI) to search the bound for weight and
tail tail effect, we must first clip the range to avoid

---

Actlayer. block. 0 Attention
Actlayer, 1, bllock.1 V
Actilayer2.blocck FC1
Actilayer.3 3 FC2
[0000]1:0]
[ [1,373777
[:5,8177311]
[3.17,8.811
0.2
0.4
0.6
0.5
1.0
2
-5.0
-2.5
0.0
5.0
7.5
Weighttlayer 0.t.ye........... Projection
Weightlayer1 block. 1 QKV
Weightlayer block.2 2loch
FC1
Weightlayer. block.1 FC2
(0.35,0.29)
[-0.00,0.61)
[-0.69,0.66]
[-0.82,0.84]
-0.2 0.0 0.2
-0.50 -0.25 9.00 0.25 0.50 -0.75 -0.50 -0.25 0.00 0.25 0.50
-0.5
0.0 0.5
Figure 4: The selected representative distribution of activations (Row 1) and weights (Row 2). The
range of data is marked in the figure. All weights obey symmetric distribution. The attention map and the input of FC2 are asymmetrie due to softmax function and GELU function.
activation, avoiding manually adjusting hyperparameters. The global optimizing goal is follows
{li, u arg min L.
Σ 11: - -
1=1
The collection of all quantizers' bounds {(1,, u,) w,)} the linchpin quantized model performance it indicates the candidate value for weights and activations. We note that the data distribution falls into two categories: one resembling a bell-shaped distribution and the other resembling an exponential distribution. For the bell-shaped distribution, Whereas, for the exponential distribution, we fix the lower bound to the minimum value of the data and only traverse the right bound. The specific search method is shown in Algorithm 1. The time complexity of Algorithm is O(M where M the number elements in data and K the number of search points. The condition is symmetrical is obtained by observing the visualization of e and the activations are from the statistics on a small calibration set.
symmetric noundary-narrowing search method.
3.3 Distillation quantization calibration
To further fine-tune the clipping range, we propose distillation quantization calibration (DQC) to transfer the knowledge from the FP model to the quantized model. It leverages the knowl- edge distillation [15] where the FP model acts as the teacher while the quantized model is the student. Specifically, for the same input image, the student model needs to continu- ously minimize the discrepancy with the teacher model on the final super-resolution output. The loss for the final output can be written
Algorithm DOBI pipeline
1
Lo = CoHoWo 10-0-111-
(4)
Data: Data to be quantized the
number of search point K, bit
where and O, are the final outputs of the teacher and student models, Co. Ho. and Wo represent the number of output channels, height, and width, respec- tively. adopt the LI loss for the final output, as tends to converge more easily compared to the L2 loss [33]. As the quantized model shares the same structure with the FP model and is quantized from the FP model, the student model also need to learn extract the same feature of the teacher model, which be written
Result: Clip bound I,
+ min(v),u + max(v);
mán.
< +oc;
if v symmetrical then Al + (max(u) min(v))/2K:
else
Al 0;
end
Au + (max(v) min( (a)in(())/23
while i < K do
F.
Fat 1
4 - 1-Itix A, u, u u Au; get 11 based on Eq. (1);
Cr= Cr=E Er=Ec,H.W C.H.W.
(5)
[F
mse - v2llzi
where F; and Fgi are the intermediate features of the teacher and student models respectively and is the in- dex of the layer. In the field of super-resolution, there is a clear correspondence between the feature maps and the final reconstructed images, making training on feature maps crucial. since the quantized network
if msc E mii_msse then
min_mse +- mse;; best + 4, u best u:;
end
end
5

---

and the full- full-precision network have identical structures, we do not need to add extra adaptation layers for feature distillation. The final loss function can be written as
L Lo + XCF;
(6)
where À is the co-efficient of Lr. In the second stage, based on training optimization methods, the gap between the quantized model and the full-precision model will gradually decrease. The performance of the quantized model will progressively improve and eventually converge to the optimal range.
Experiments
4.1 Experimental settings
Data and evaluation. We use DF2K [40, 34] as the training data, which combines DIV2K [40] and Flickr2K [34], as utilized by most SR models. During training, since we employ a distillation training method, we do not need to use the high-resolution parts of the DF2K images. For validation, we use
the Set5 [2] as the validation set. After selecting the best model, we tested it on five commonly used benchmarks in the SR field: Set5 [2]. 4et14 [45], B100 [36], Urban100 [18], and Mangal09 [37]. On
the benchmarks, we input low-resolution images into the quantized model to obtain reconstructed images, which we then compared with the high-resolution images to calculate the metrics. We do not use self-ensemble in the test stage as it increases the computational load eightfold, but the improvement in metrics is minimal The evaluation metrics we used are the most common metrics PSNR and SSIM [43], which are calculated on the Y channel (iee, luminance) of the YCbCr space.
Implementation details. We use SwinIR-light [32] as the backbone and provide its structure in the supplementary materials. We conduct comprchensive experiments with scale factors of 2, 3, and and with 2, 3, and 4 bits, where Our hyperparameter settings remain consistent. During DOBI, use a search step number of K=100, and the statistics of activations are obtained from 32 images in DF2K being randomly cropped retain only xx64 x 64. During DQC, we use the Adam [23] optimizer with learning rate of x 10-2, betas set to (0.9, 0.999), and weight decay of O. We employ Conealnnn [35] as the learning rate scheduler to stabilize the training process. Data augmentation is also performed. We randomly utilize rotation of 90", 180", and 270" and horizontal flips to augment the input image. The total iteration for training 3,000 with batch size of 32. Our code is written with Python and Py [38] and runs on an NVIDIA A800-80G GPU.
4.2
Comparison with state-of-the-art methods
The methods we compared include MinMax [22], Percentile [27], and the current SOTA post- quantization method in the super-resolution field, DBDC+Pac [41]. For a fair comparison, we report the performance of DBDC+Pac [41] on EDSR [33], the authors performed detailed parameter adjustments and model training on EDSR. We directly used the results reported by the authors, recorded in the table as EDSR It should be noted that the EDSR method uses self-ensembl in the final test, which can improve performance to some extent but comes at the cost of 8 times the computational load. Additionally, hyperparameters as those set by the authors for EDSR, recorded in the table as DBDC+Pac [41]. The following are the quantitative and qualitative results of the comparison.
applied DBDC+Pac [41] to SwinlR-light [32], using the same
Quantitative results. Table 3 shows the extensive results of comparing different quantization methods with bit depths of 2, 3, and 4, as well as different scaling factors of x2, x3, and x4.
DBDC+Pac [41] performs
Quantizer lower bound percentile.
Quantizer upper bound percentile.
poorly mainly because 1. The DBDC process requires
0.30
DOBBi
1.00
0.25
DOBI-DQC
0.95
manually specifying the clipping ratio, which signif- icantly affects performance.
0.20
0.15
0.85
0.10
0.80
2. DBDC does not prune weights, and the learning rate in the Pac process is too low, causing slow con- vergence of weight quantizer parameters. However, both adverse factors are eliminated in our 2DQuant algorithm. When using only DOBI algorithm, our performance has already reached a level comparable to that of DBDC+Pac algorithms. Upon applying DQC, our performance experienced a remarkable and discernible enhancement, elevating to new heights. the case of x2, 4-bit on Set5 and Urban 100, DOBI has an improvement of 1. IdB and 0.39 dB compared to EDSR, while
0.05
0.75
DOBI
DOBI+D0C
0.00
0.70
140
20 40 60
80
100
120 140
Figure 5: The bound percentile of DOBI and DQC.

---

Table 3: Quantitative comparison with SOTA methods. EDSR means applying DBDC+Pac [41] on CNN-based backbone EDSR [34]. Its results are eited from the paper [41].
Method
Bit
Set5 (x2)
(x2)
B100
Urban 100 (x2)
Mangal (x2)
PSNRT
SSIM
PSNRT
SSIMT
PSNRT
SSIMT
PSNRT
SSIM
PSNR1
SSIM
SwinIR-light [32]
32
38.15
0.9611
33.86
0,9206
32.31
0,9012
32.76
0,9340
39.11
0.9781
Bicubic
32
32.25
0.9118
29.25
0.8406
28.68
0.8104
25.96
0.8088
29.17
0,9128
MinMax [22]
34.39
0.9202
30.55
0,8512
29.72
0.8409
28.40
0.8520
33,70
0.9411
Percentile [27]
37.37
0,9568
32.96
0.9113
31.61
0.8917
31.17
0,9180
37.19
0.9714
EDSR 41]
36.33
0.9420
32.75
0,9040
31.48
0.8840
30.90
0.9130
N/A
N/A
DBDC+Pac [41]
4
37.18
0.9550
32.86
0,9106
31,56
0.8908
30.66
0.9110
36.76
0,9692
DOBI (Ours)
37.44
0,9568
33.15
0,9132
31.75
0.8937
31.29
0,9193
37.93
0.9743
2DQuant (Ours)
37.87
0.9594
33.41
0.9161
32.02
0.8971
31.84
0.925
38.31
0.9761
MinMax [22]
18.19
0.6961
26.40
0.6478
25.83
0.6225
25.19
0.6773
28.97
0.7740
Percentile [27)
34.37
0.9170
31.04
0.8646
29.82
0.8339
28.25
0.8417
33.43
0.9214
DBDC+Pac [41]
35.07
0,9350
31.52
0.8873
30.47
0.8665
28.44
0.8709
34.01
0.9487
DOBI (Ours)
36.37
0.9496
32.33
0,9041
31.12
0.8836
29.65
0.8967
36.18
0,9661
2Qanil (Ours)
37.32
0.9567
32.85
0.9106
31.60
0,8911
30.45
0.9086
37.24
0,9722
MinMax [22]
33.88
0.9185
30.81
0.8748
29.99
0.8535
27 48
0.8501
31,86
0.9306
Percentile [27]
30.82
0.8016
28.80
0,7616
27.95
0.7232
26.30
0.7378
30.37
0.8351
DBDC+Pac [41]
34.55
0.9386
31.12
0,8912
30.27
0.8706
27.63
0,8649
32.15
0,9467
DOBI (Ours)
35.25
0.9361
31.72
0.8917
30.62
0.8699
28. .2.52
0.8727
34.65
0,9529
2DQuant (Ours)
36.00
0:9497
31.98
0,9012
30.91
0,8810
28.62
0,8819
34.40
0,9602
Set5 (x3)
(x33)
(x3)
Urban
(x:)
Mangal
(x3)
Method
Bit
PSNR
SSIM
PSNR1
SSIMI
PSNRT
SSIM
PSNR1
SSIM
PSNR1
SSIM
SwinlR-ligh light [32]
32
34.63
0.9290
30.54
0.8464
29.20
0.8082
28. 66
0.8624
33,99
0,9478
Bicubic
32
29.54
0.8516
27.04
0.7551
26.78
0.7187
24.00
0.7144
26.16
0.8384
MinMax [22]
4
31.66
0.8784
28.17
0.7641
27.19
0.7257
25.60
0.7485
29.98
0.8854
Percentile [27]
4
33.34
0,9137
29.61
0.8275
28.49
0.7899
27.06
0.8242
32.10
0.9303
DBDC+Pac [41]
4
33.42
0.9143
29.69
0.8261
28.51
0.7869
27.05
0,8217
31,89
0.9274
DOBI (Ours)
4
33.78
0.9200
29.87
0,8338
28.72
0.7970
27.53
0.8391
32.57
0,9367
2DQuant (Ours)
34.06
0.923
30.12
0,8374
28.89
0,7988
27.69
0.8405
32.88
0,9389
MinMax [22]
26.01
0.6260
23.41
114444
22.46
0.4182
21.70
0.4730
24.68
0.6224
Percentile [27]
3
30.91
0.8426
28.02
0,7545
27.23
0.7183
25.32
0.7349
29.43
0,8537
DBDC+Pac [41]
3
30.91
0.8445
28.02
0.7538
26.99
0.6937
25.10
0.7122
28.84
0.8403
DOBI (Ours)
32.85
0,9075
29.33
0,8200
28.27
0.7820
26.36
0.8036
31.14
0.9178
2DQuant (Ours)
3
33.24
0.9135
29.56
0.8255
28.50
0,7873
26.65
0.8116
31,46
0.9235
MinMax [22]
2
26.05
0,5827
24.74
0,5302
24.42
0,4973
22.87
0,5155
24.66
0.5652
Percentile [27]
2
25.30
0.5677
23.60
0.4890
23.77
0.4751
22.33
0.4965
24.65
0.5882
DBDC+Pac [41]
29.96
0.8254
27.53
0.7507
27.05
0.7136
24.57
0.7117
27.23
0.8213
DOBI (Ours)
2
30.54
0.8321
27.74
0,7312
26.69
0.6643
24. 80
0.6797
28.18
0.7993
2DQuant (Ours)
2
31.62
0.8887
28.54
0,8038
27.85
0.7679
25.30
0.7685
28.46
0.8814
Set5 (x4)
4et14
B100
(x-4)
Urbun
00 (x-))
Mangal 099 PSNR1
(x-4)
Method
Bit
PSNRT
SSIM
PSNR1
SSIMI
PSNRT
SSIM
PSNR1
SSIM
SSIM
SwinIR-light [32]
32
32.45
0.8976
28.77
0.7858
27.69
0.7406
26.48
0.7980
30,92
0.9150
Bicubic
32
27.56
0.7896
25.51
0.6820
25.54
0.6466
22.68
0.6352
24.19
0.7670
MinMax [22]
4
28.63
0.7891
25.73
0.6657
25.10
0.6061
23.07
0.6216
26.97
0,8104
Percentile [27]
4
30.64
0.8679
27.61
0.7563
26.96
0.7151
24.96
0.7479
28.78
0,8803
EDSR [33, 41J
31.20
0.8670
27.98
0.7600
27.09
0.7140
25.56
0.7640
N/A
N/A
PBDC+Pac [41]
30.74
0.8609
27.66
0.7526
26.97
0.7104
24.94
0.7369
28.52
0.8697
DOBI (Ours)
4
31.10
0.8770
28.03
0,7672
27.18
0.7237
25.43
0.7631
29.31
0,8916
2DQuant (Ours)
31.77
0.8867
28.30
0,7733
27.37
0.7278
25.71
0.7712
29.71
0.8972
MinMax [22]
19.41
0.3385
18.35
0.2549
18.79
0.2434
17.88
0.2825
19.13
0.3097
Percentile [27]
27.55
0.7270
25.15
0.6043
24.45
0.5333
22. 80
0.5833
26.15
0.7569
DBDC+Pac [41]
27.91
0.7250
25.86
0.6451
25.65
0.6239
23.45
0.6249
26.03
0.7321
DOBI (Ours)
29.59
0.8237
26.87
0.7156
26.24
0.6735
24.17
0.6880
27.62
0,8349
2DQuant (Ours)
30.90
0.8704
27.75
0.7571
26.99
0.7126
24.85
0.7355
28.21
0.8683
MinMax [22]
23.96
0.4950
22.92
0.4407
22.70
0.3943
21.16
0.4053
22.94
0.5178
Percentile [27]
23.03
0.4772
22.12
0.4059
21.83
0,3816
20.45
0.3951
20.88
0.3948
DBDC+Pac [41]
25.01
0.5554
23.82
0.4995
23.64
0.4544
21.84
0.4631
23.63
0.5854
DOBI (Ours)
28.82
0.7699
26.46
0.6804
25.97
0.6319
23.67
0.6407
26.32
0.7718
2DQuant (Ours)
29.53
0.8372
26. 86
0.7322
26.46
0.6927
23.84
0.6912
26.07
0.8163
2DQuant has an improvement of 0.69 dB and 1.18 dB compared to the SOTA method. All these results indicate that our two-stage PTQ method can effectively mitigate the degradation caused by quantization and ensure the quality of the reconstructed images.
Figure 5 shows the bound percentile of DOBI searching and DQC. Overall, the bound of DQC is tighter as the values around the zero point enjoy greater importance. Besides, the shallow layers'
7

---

HR
Bicubic
MinMax [22]
Percentile [27]
Urban100: ing 004 (x4)
DBDC+Pac [41]
DOBI (Ours)
2DQuamt (Ours)
FP
HR
Bicubic
MinMax [22)
Percentile [27]
Urban100: img 046 (x4)
DBDC+Pac [41]
DOBI (Ours)
2DQuant (Ours)
FP
HR
Bicubic
MinMax [22]
Percentile [27]
Urban100: img_023 023 (x4)
DBDC+Pac [41]
DOBI (Ours) 2DQuant (Ours)
Figure 6:
Visual comparison for image SR (x4) in challenging cases.
Learning rate
PSNR1
SSIM
Batch size
PSNR1
SSIM1
DOBI DQC
PSNRT
SSIM
10-
37.82
0.9594
37.82
0.9594
34.39
0,9202
10 2
37.87
0.9594
0.9594
37.44
0,9568
10
37.78
0.9592
37.84
0.9593
37.32
0,9563
10-4
37.74
0.9587
32
37.87
0.9594
37.87
0.9594
Learning rate
(b) Batch size
(c) DOBI and DQC
Table 4: Ablation studies. The models are trained on DIV2K and Flickr2K, and tested on SetS (x2).
bounds vary more significantly due to the elevated significance of these layers within the neural network. Detailedly, the bound for the second MLP fully connected layer's weight in Layer 0 Block only remains 46% data its range. It has the second-highest lower bound percentile and the smallest upper bound percentile among the network. Its percentiles are 0.2401 and 0.7035 respectively while its bound values are -0.062 and 0.047 and its distribution visualized in Figure 4. conclusion, only through task-oriented optimization of each bound fine-grained level can redundant information be maximally excluded and useful information be maximally retained.
Qualitative results. We show the visual comparison results for x4 in Figure 6. Since quantized models are derived from full-pree full-precision models with information loss, their global performance will rarely exceed that of full-precision models. As seen in the three images for Minmax, after quantization, if no clipping is performed, the long tail effect will lead to large number of useless bits, resulting in a significant amount noise and repeated distorted patterns in the reconstructed images. In these challenging cases, training method allows the model retain edge information of objects better, preventing blurring and distorted effects. For example, in img_046 (446 and mg_023, have the highest similarity the full-precision model, while other methods show varying degrees of edge diffusion, significantly affecting image quality. Compared to the DBDC+Pac method, our DOBI and DQC allow for better representation of edge and texture information in the images and effectively avoid distortions and misalignments in the graphics. The visual results demonstrate that our proposed DQC is essential for improving performance both metric and visual comparisons.
4.3 Ablation study
Learning rate and batchsize. We first study the performance variations of the model under different hyperparameters. From Tables 4a and 4b, it can be seen that DQC enables the model to

---

converge within a range of outstanding performance for most learning rates and batch sizes. Due to the non-smooth impact of quantization parameters on the model, the quantized model is more prone to local optima compared to the full-precision model, resulting in a noticeable performance drop when the learning rate is too low. Additionally, as shown in Table 4h, the larger the batch size, the better the model's performance, and the smoother the convergence process. However, even with a smaller batch size, we can still achieve a performance of 37.82dB on Set5, indicating that our two-stage method has good robustness to different hyperparameters.
DOBI and DQC. Moreover, we also study the impact of different stages on performance, with the results shown in Table 4c, from which we can draw the following conclusions: Firstly, the goal of DOBI is to minimize the value shift for weights and activations. Although it is not the task goal, can still enjoy significant enhancement due to better bit representational ability, Secondly, DQC alone cannot achieve the optimization effect of DOBI. This is because the impact of quantizer parameters on model performance is oscillatory, and training alone is prone to converge to local optima. In contrast, ssurch-based methods can naturally avoid local optima. So it's necessary to use results from the search-based method to initialize training-basee based method in PTQ. Thirdly, when DOBI and DQC are combined, namely our 2DQuant, the 4-bit quantized model has only a 0.28dB decrease on Set5 compared to the FP model, which maximally mitigates the accuracy loss caused by quantization.
5 Discussion
Why our results surpass FP outcomes. While method's performance metrics do not yet fully match those of full-precision models, visual results reveal a compelling advantage. As observed in image img_ _09_02 of Figure of Urban100, our approach correctly identifies the direction of the stripes in the image. Whereas the full-precision model erroneously selects the wrong direction. This discrepancy arises because the lower-resolution image, affected by aliasing, creates illusion of slanted stripes, misleading the FP model's reconstruction. This phenomenon demonstrates that our PTQ algorithm allows more accurate restored results in certain localized and challenging tasks without being misled. More examples are in the supplementary materials.
suggests that full-precision models contain not only redundant knowledge but also incorrect information. The latter is hard to get rid of by training the FP model. Our quantization method can effectively reduce model parameters and computational demands while eliminating erroneous information, achieving multiple benefits simultaneously. This also suggests that the FP model doesn't t represent the pinnacle of what a quantized model can achieve.
Limitations. Despite achieving excellent results, this study still has some limitations. During the DOBT process, the data distribution of activations and weights required to approximate a bell curve or exponential distribution; otherwise, the DOBI method cannot find the most suitable positions. Additionally, increasing the number of search points for a single tensor in MSE does not necessarily guarantee better performance. However, the second-stage training can somewhat alleviate this issue. Moreover, our method requires a calibration set; without which, the first-stage DOBI and the second-stage DQC cannot be carried out at all.
Societal impacts. Our super-resolution quantization method effectively saves computational re- sources, facilitating the deployment of super-resolution models the cutting edge
6 Conclusion
This paper studies the post-training quantization in the field of image super-resolution. We first conducted a detailed analysis of the data distribution of Transformer-based model in SR. These data exhibit clear long-tail effect and symmetry and asymmetry coexisting effect. We designed 2DQuant, a dual-stage PTQ algorithms. In the first stage DOBI, we designed two different search strategies for the two different distributions. In the second stage DQC, we designed a distillation- based training method that let the quantized model learn from the FP model, minimizing the accuracy loss caused by quantization. Our 2DQuant can compress Transformer-based model to 4.3.2 bits with the compression ratio being 3.07 x, 3.31 x, and 3.60x and speedup ratio being 3.99x, 4.47 x, and 5.08 x. No additional module is added so 2DQuant enjoys the theoretical upper limit of compression and speedup. Extensive experiments demonstrate that 2DQuant surpasses all existing PTQ methods in the field of SR and even surpasses the FP model in some challenging cases. In the future, recognizing the significant impact of the model on performance, we will conduct PTQ research on more advanced super-resolution models and attempt to deploy quantized super-resolution algorithms to actual photography tasks, providing more detailed evaluation of the performance of PTQ algorithms.

---

Acknowledgments and Disclosure of Funding
This work is supported by the National Natural Science Foundation of China (62141220, 62271414), Shanghai Municipal Science and Technology Major Project (2021SH2DZ20112,, the Fundamental Research Funds for the Central Universities, Zhejiang Provincial Distinguish Young Science Founda- tion (LR23F01 (LR23,010001), Zhejiang "Pioneer" and "Leading Goose" R&D Program (2024SDHHDX0006 2024C03 182), the Key Project of Westlake Institute for Optoelectronics (2023GD007) and Ningbo Science and Technology Bureau, "Science and Technology Yongjiang 2035" Key Technology Break- through Program (2024Z126).
References
[1] Wele Gedara Chaminda Bandara and Vishal M. Patel. Hypertransformer: A textural and spectral feature fusion transformer for pansharpening. In CVPR, 2022. 1
Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. BMVC, 2012.
[3] Zheng Chen, Yulun Zhang, Jinjin Gu, Linghe Kong, Xiaokang Yang, and Fisher Yu. Dual aggregation transformer for image super-resolution. In CVPR, 2023.
[4] Zheng Chen, Yulun Zhang, Jinjin Gu, Yonghing Zhang, Linghe Kong, and Xin Yuan. Cross aggregation transformer for image restoration. In NeurlPS, 2022.
[5] Jungwook Choi, Pierce I-Jen Chuang, Zhuo Wang, Swagath Venkataramani, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Bridging the accuracy gap for 2-bit quantized neural networks (qnn). arXiv, 2018.
[6] Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang. Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv: 805. 06085, 2018. 2
[7] Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural networks for efficient inference. In ICCVW, 2019.
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-iani,, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ or-1. arXiv preprint arviv: /tfi::/602.2288800 2016. 4
[9] Yifu Ding, Haotong Qin, Qinghua Yan, Zhenhua Chai, Junjie Liu, Xiaolin Wei, and Xianglong Liu. Towards accurate post-training quantization for vision transformer. In ACM MM, 2022. 1
[10] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Learning deep convolutional network for image super-resolution. ECCV, 2014. 1,3
[11] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. TPAMI, 2016. 1,3
[12] Alexcy Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv, 2020. 3
[13] Hayit Greenspan. Super-resolution in medical imaging. The Computer Journal, 2008.
[14] Dan Hendrycks and Kevin Gimpel. Bridging nonlincarities and stochastic regularizers with gaussian error linear units. CORR, 2016.
[15] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling knowledge a neural network. NeurlPS Workshop, 2014. 5
[16] Cheeun Hong, Sungyong Baik, Heewon Kim, Scungjun Nah, and Kyoung Mu Lec. Cadyq: Content-aware dynamic quantization for image super-rcsolution. In ECCV, 2022. 2,
[17] Cheeun Hong. Heewon Kim, Sungyong Baik, Junghun Oh, and Kyoung Mu Lee. Daq: Channel-wist distribution-wware quantization for deep image super-resolution networks. In WACV, 2022. 3
10

---

[18] Jia-Bin Huang. Abhishek Singh, and Narendra Ahuja. Single image super-resolution from transformed self-exemplars. In CVPR, 2015. 6
[19] Yawen Huang, Ling Shao, and Alejandro F Frangi. Simultaneous super-resolution and cross-modality synthesis of 3d medical images using seakly-supervised joint convolutional sparse coding. In CVPR, 2017.
[20] Ita Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets, In ICML, 2021.
[21] Jithin Saji Isaac and Ramesh Kulkarni. Super resolution techniques for medical image processing. In ICTSD, 2015.
[22] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training neural networks for efficient integer- arithmetic-only inference. In CVPR, 2018. 3, 6, 7,8
[23] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[24] Christian Ledig, Lucas Theis, Ferene Huszár, Jose Cahallero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, 2017. 2,
[25] Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew P Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, 2017. 1
[26] Huixia Li, Chenqian Yan, Shaohui Lin, Xiawu Zheng, Baochang Zhang, Fan Yang, and Rongrong Ji. Pams: Quantized super-resolution via parameterized max scale. In ECCV, 2020. 2,
[27] Rundong Li, Yan Wang. Feng Liang, Hongwei Qin, Junjie Yan, and Rui Fan. Fully quantized network for object detection. In CVPR, 2019. 2, 6, 7,8
[28] Yuchen Li, Haoyi Xiong, Linghe Kong, Qingzhong Wang. Shuaiqiang Wang, Guihai Chen, and Dawei Yin. S2phere: Semi-supervised pre-training for web search over heterogeneous learning to rank data. In ACM SIGKDD, 2023. 3
[29] Yuchen Li, Haoyi Xiong, Linghe Kong, Rui Zhang, Fanqin Xu, Guihai Chen, and Minglu Li. Mhrr: Mooes recommender service with meta hierarchical reinforced ranking. IEEE TSC, 2023. 3
[30] Yuchen Li, Haoyi Xiong, Qingzhong Wang, Linghe Kong, Hao Liu, Haifang Li, Jiang Bian, Shuaiqiang Wang, Guihai Chen, Dejing Dou, et al. Coltr: Semi-supervised learning to rank with co-training and over-parameterization for web search. JEEE TKDE, 2023. 3
[31] Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang. Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Breeq: Pushing the limit of post-training quantization by block reconstruction. In ICLR, 2021. 1
[32] Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. ICCVW, 2021. 1, 2, 3, 6,
[33] Bee Lim, Sanghyun Son, Heewon Kim. Seungjun Nah, Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. CVPRW, 2017. 2, 6,
[34] Bee Lim, Sanghyun Son, Heewon Kim, Scungjun Nah, and Kyoung Mu Lec. Enhanced deep residual networks for single image super-rcsolution. CVPRW, 2017. 6,
[35] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv, 2016.
[36] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, 2001. 6
[37] Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa. Sketch-based manga retrieval using mangal09 dataset. Multimedia Tools and Applications. 2017.
[38] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, performarre deep learning library. NewrIPS, 2019. 6

---

[39] Pejman Rasti, Tönis Uiboupin, Sergio Escalera, and Gholamreza Anbarjafari. Convolutional neural network super resolution for face recognition in surveillance monitoring. In AMDO, 2016.
[40] Radu Timofte, Eirikur Agustsson, Lue Van Gool, Ming- Huuan Yang, Lei Zhang, Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, Kyoung Mu Lee, et al. Ntire 2017 challenge single image super-resolution: Methods and results. In CVPRW, 2017. 3,
[41] Zhijun Tu, Jie Hu, Hanting Chen, and Yunhe Wang. Toward accurate post-training quantization for image super resolution. In CVPR, 2023. 2, 6, 7,8
[42] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: A general u-shaped transformer for image restoration. In CVPR, 2022.
[43] Zhou Wang. Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP, 2004. 6
[44] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan -ssun Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022.
[45] Roman Zeyde, Michael Elad, and Matan Protter. single image scale-up using sparse-represeenttiion.. In Proc. 7th Int. Conf. Curves Surf., 2010.
[46] Liangpei Zhang, Hongyan Zhang, Huanfeng Shen, and Pingxiang Li. A super-resolution reconstruction algorithm for surveillance images. Elsevier Signal Processing, 2010.
[47] Weihong Zhang and Ying Zhou. Chapter 2- level-set functions and parametric functions. In The Feature- Driven Method for Structural Optimization. Elsevier. 4
[48] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. In ECCV, 2018. 1
[49] Yulun Zhang, Yapeng Tian. Yu Kong, Bineng Zhong, and Yun Fu. Residual dense network for image super-resolution. In CVPR, 2018.
[50] Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint wxXv::600.006160 2016. 2
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to our abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
12

---

The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specifcatton, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't I acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Question:
Answer: [NA]
Justification: The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of paper should be complemented by
formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided implementation details in the experiments section. We will also release all the code and models.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
13

---

While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(h) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may he tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may he that access the model is limited in some way (e.g., registered users), but should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: Firstly, all data in the paper can be accessed publicly. Secondly, we provide very detailed instructions (e.g., method descriptions and implementation details) reproduce our results. Thirdly, we promise to release code and all models.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: //nips//iipp//.. http://iipsspcppuuue//// cc/publcc/ quides/CodeSuban asiinnolicyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g.. for a open-soure benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (httpa: //nipa/ cc/ public/ guides Saalee//Coeeubmiiii swssssssiiipiyi for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer. etc.) necessary understand results?
Answer: [Yes]
Justification: We have provided implementation details, which cover the above questions.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please refer to the experiment part.
14

---

Guidelines:
answer means the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should he explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean,
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures rates).
symmetric error bars that would yield results that are of range (e.g. negative If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Please refer to experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory
storage.
The paper should provide the amount of compute required for cach of the individual experimental runs well as estimate the total compute.
The paper should disclose whether the full research project required compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, every respect, with NeurIPS Code of Ethics https::/ htapp:://neurps....
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., if there is a special considcration due laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to the supplementary file.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
15

---

Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance). fairness considerations (e.g., deploy- ment technologies that could make decisions that unfairly impact specific groups), privacy considcrations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible relcase of data or models that have a high risk for misuse (e.g. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of model, for example by requiring users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We have credited most previous works in the paper. The license and terms are respected properly.
Justification:
Guidelines:
The answer NA mcans that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset is used and, possible, include a URL..
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should he provided. For popular datasets, paperswi thcode. cattatta has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
this information not available online, the authors are encouraged reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
16

---

Justification: We will release code and models. In the paper, we have provided implementation details and other contents to reproduce results.
Guidelines:
The answer NA means that the paper does not release assets.
Rescarchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing rescarch with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human Subjeets
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review.
17


Reviews:
[{"review_id": "Jz6eLz9eha", "reviewer": "Reviewer_6qcS", "summary": "The authors propose a low-bit post-training quantization (PTQ) method, 2DQuant, for image super-resolution. 2DQuant is a dual-stage low-bit PTQ method. They first investigate the weight and activations. They propose Distribution-Oriented Bound Initialization (DOBI) by using different searching strategies to get a coarse bound for quantizers. They further propose Distillation Quantization Calibration (DQC) to refine the quantizer parameters with a distillation approach. They provide extensive experiments on different bits and scaling factors. When compared with SOTA methods, there proposed methods achieve superior performance with 3.6X compression ratio and 5.08X speedup ratio in the 2-bit case.", "questions": "In Figure 1, the proposed method performs even better than the full-precision (FP) version. What are the potential reasons behind this observation?\n\nCan this method be applied to diffusion based image SR models? If so, please give some key ideas. It will be much better if the authors can provide some preliminary results.\n\nIn Table 3, what is the model size for the original EDSR? I do not which version of EDSR did the authors use in the paper.\n\nWhat is the motivation by providing two versions, i.e., DOBI (ours) and 2DQuant (ours), in Table 3?\n\n------------After Rebuttal and  Discussions----------\nAfter rebuttal and disscussions, my concerns have been well solved.", "limitations": "The authors have discussion the limitation in their paper.\n\nFlag For Ethics Review: No ethics review needed.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1)The authors explore PTQ to Transformer based image super-resolution (SR), which is one of the first works in this research field. This topic is also very practical and is favored in real-world applications.\n\n2)The authors investigate this topic with some observations and visualizations. For example, the visualizations in Figures 4, 5, 7-11, are inspiring.\n\n3)They propose distribution-oriented Bound Initialization (DOBI) to minimize the value heterogenization between quantized and the FP models. They DOBI to search for quantizer parameters by employing customized search strategies for different distributions to balance speed and accuracy.\n\n4)They propose Distillation Quantization Calibration (DQC) to adjust each bound to its best position finely. DQC is used to ensure the quantized model align with the FP model on the calibration set.\n\n5)In the ablation study, the authors investigate the effect of some key components, like DOBI and DQC.\n\n6)In the main comparisons with other SOTA methods, the proposed method obtain the best performance and visual results. I believe this low-bit quantization work in Transformer achieves excellent permance and has good potential in future research.\n\n7)The authors give detailed derivation of their backward gradient propagation formula, which makes the method more convincing.", "weaknesses": "The authors did not give results about inference time. The quatization can reduce params and ops largely. Then, how about the acceleration when deploying the quantized models into device?\n\nSome parts in the experiments are confusing. For example, in Table 3, the results of EDSR confuse me somehow. I am not very sure how the EDSR results are obtained. The original EDSR is full-precision model. Here, the authors report its quantized version. The baseline in Table 3 is not very clear. Please clarify them.\n\nIn Table 4 (b), when the batch size becomes larger, the performance changes slightly. This is not very consistent with the common observation, where larger batch size usually improves performance.\n\nThe writing can be further improved. Some details should be given attention and revised. For example, Section 3.2, the first letter should be capital.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: ... how about the acceleration when deploying the quantized models into device?\n\nA1: In the field of quantization research, when we apply the same quantization method to the same neural network module, the speedup ratio of the model does not change due to variations in the quantizer parameters. Therefore, the algorithm typically focuses on how to better optimize the quantizer parameters to improve model performance after applying such quantization methods. \n\nBesides, the speedup ratio reported in lines 20-22 of our paper is the actual deployment speedup ratio——2DQuant achieves an increase in PSNR of up to 4.52 dB on Set5 (×2) compared to SOTA when quantized to 2 bits, along with a 3.60× compression ratio and a 5.08× speedup ratio.\n\n> Q2:.... I am not very sure how the EDSR results are obtained. The baseline in Table 3 is not very clear. Please clarify them.\n\nA2: We clarify that the EDSR in Table 3 does not refer to the original FP model. Its specific meaning is explained in lines 237-239 – it is the test result of the previous SOTA method DBDC+Pac on EDSR. It should be noted that the parameter size of EDSR is 172.36MB, while the parameter size of SwinIR-light is 3.42MB, which is about 2% of the former. As for the baseline in Table 3, it is the full-precision version of SwinIR-light.\n\nOur comparison with EDSR aims to illustrate that: \n1. The best result in [1] was achieved on EDSR, and applying our quantization method on SwinIR-light performs better than their method on EDSR; \n2. The choice of the quantized model itself is crucial. To achieve better quantization results, we cannot only experiment on classic models (as done in the [1]), because in terms of parameter size, cutting-edge models before quantization already outperform quantized classic models.\n\n> Q3:In Table 4 (b), when the batch size becomes larger, the performance changes slightly. This is not very consistent with the common observation, that larger batch size usually improves performance.\n\nA3:Thank you for your keen observation. The main reason for this issue lies in the fact that DOBI provides excellent quantizer parameters. In the SR field, the numerical evaluation metric usually adopted is PSNR. If we only look at PSNR, with the increase of batch size, our performance shows a slight improvement. The reason for this slight improvement is that our one-stage search algorithm, DOBI, has already provided quite optimal quantizer parameters, resulting in a very high starting performance for the model. At the same time, the initialization parameters provided by DOBI ensure that the model can achieve almost the same excellent results regardless of changes in batch size during subsequent training. \n\nI believe this phenomenon is beneficial because, even when GPU memory is limited, we can still achieve results comparable to those obtained with a large batch size by reducing the batch size and extending the training time for DQC.\n\n> Q4: The writing can be further improved. Some details should be given attention and revised. For example, Section 3.2, the first letter should be capital.\n\nA4: Thank you for your suggestions on writing details. We have revised in the Arxiv version.\n\n> Q5:In Figure 1, the proposed method performs even better than the full-precision (FP) version. What are the potential reasons behind this observation?\n\nA5:We clarify that the main reason is that FP models tend to overfit the training set, leading to weaker generalization performance on **some** test data. Model quantization might alleviate this overfitting. Although there might be an overall performance drop, the performance on certain data can be better than that of the FP model.\n\n> Can this method be applied to diffusion based image SR models? If so, please give some key ideas. It will be much better if the authors can provide some preliminary results.\n\nA6:Yes! The core components of diffusion are the denoising network and the sampler, with UNet being a common form of the denoising network. The computationally intensive parts of the UNet are the convolutional layers. We can use the DOBI and DQC methods mentioned in the article to perform two-stage quantization of the convolutional layer parameters. \n\nHowever, it is important to note that the generation process of diffusion models typically involves several steps, and the distribution of activations is usually associated with the time steps. Therefore, designing algorithms tailored to each time step might yield better results. For example, we can divide the time steps into different segments and apply different quantizers to the activations in different segments.\n\n> Q7:In Table 3, what is the model size for the original EDSR? I do not which version of EDSR did the authors use in the paper.\n\nA7: As mentioned in A2, the EDSR in Table 3 refers to the data in Table 3 of [1]. Details and parameters of the model used can be found in the following link at line 11 in the blob/master/src/model/edsr.py file in the github repo \"sanghyun-son/EDSR-PyTorch\". To our knowledge, this is the largest version of EDSR.\n\n> Q8: What is the motivation by providing two versions, i.e., DOBI (ours) and 2DQuant (ours), in Table 3?\n\nA8: DOBI is our one-stage algorithm while 2DQuant represents the final results of both the one-stage and two-stage processes. The reason we included both in Table 3 is to demonstrate that even just using DOBI, the model can exceed the SOTA performance on most datasets, and the two-stage algorithm can further enhance the model's performance.\n\n[1] Tu, Zhaopeng et al. “Toward Accurate Post-Training Quantization for Image Super Resolution.” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023): 5856-5865.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RPnLx8xirG", "reviewer": "Reviewer_Xp5V", "summary": "This paper present a practical post-training quantization method for SR transformer, namely 2DQuant. The 2dquant mainly rely one two techniques, the first is Distribution-Oriented Bound Initialization (DOBI) determine the quantization range initially, and the second one is distillation Quantization calibration (DQC) to further finetune the quantizer for accurate quantization. Both of the proposed two techniques significantly improving the performance of PTQ for SR transformer, and the quantization can even be pushed to as low as 2-bit without retraining.", "questions": "Please respond for the issues raised in weaknesses part.", "limitations": "The authors have discussed the limitation in their paper.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The proposed method is a PTQ method without full retraining instead of the usually studied quantization-aware training methods for SR tasks, which can be seen as resource-saving in real applications. Considering the targeted architectures are SR transformers is always used as well-pretrained models, the PTQ pipeline is significant and practical.\n\n2. The proposed quantization method for SR transformer are clear and effective. DOBI allows the optimization of quantization ranges start from a statistical search-based optimal, and then use DQC to fully optimized the quantizer. It’s brings improvements of both weight and activation quantization, especially for the activation with dynamic distribution that requires more robust and powerful quantizer.\n\n3. Experiments show that the proposed method achieve SOTA results on ultra-low 2-4 bit Sr networks, which allows SR transformer enjoys very good efficiency with little accuracy loss. And the visualization also show the good performance of the networks quantized by the proposed methods.\n\n4. The paper is easy to follow and with good writing and presentation, and with comprehensive visualization for the distribution of both weights and distributions.", "weaknesses": "1. As presented in section 3, the authors use fake quantization during the design of the method, but didn’t mention if it can be replaced by the real quant, which can be implemented and bring real acceleration in hardware. The author should discuss if the proposed method can achieve real quantized inference when deployment.\n\n2. The algorithm of DQC is missed and make the description not very clear, I suggest to add it or merge it to the algorithm of DOBI. And Figure 3 not highlight the application of the proposed techniques, which is also suggested to revise.\n\n3. Some details of writing should be improved and polished carefully, e.g., the subtitles of section 2 and 3, some explain of the proposed equations are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1:As presented in section 3, the authors use fake quantization during the design of the method, but didn’t mention if it can be replaced by the real quant, which can be implemented and bring real acceleration in hardware. The author should discuss if the proposed method can achieve real quantized inference when deployment.\n\nA1: Fake quantization is a widely used technique in model quantization. Its purpose is to simulate the precision loss caused by quantization. Using fake quantization, we can obtain the model parameters of the quantizer. With the quantizer's parameters, we can deploy and test the model using common quantization techniques. We omitted this part in the article because the quantization method we adopted is already very mature in this regard and our speedup ratios are obtained from real deployment.\n\nBesides, the speedup ratio reported in lines 20-22 of our paper is the actual （real quant） deployment speedup ratio——2DQuant achieves an increase in PSNR of up to 4.52 dB on Set5 (×2) compared to SOTA when quantized to 2 bits, along with a 3.60× compression ratio and a 5.08× speedup ratio.\n\n> Q2a:The algorithm of DQC is missed and make the description not very clear, I suggest to add it or merge it to the algorithm of DOBI. \n\nA2a: Thank you for your suggestion. The nature of DQC is distillation between the FP model and the quantized model, which optimizes the parameters of the quantizers. We will merge this part of the algorithm into the DOBI algorithm.\n\n> Q2b: And Figure 3 not highlight the application of the proposed techniques, which is also suggested to revise.\n\nA2b: You are correct. \n\nFigure 3 is not closely related to the proposed techniques. However, it is still very necessary to present Figure 3. We included Figure 3 to show that in the Transformer block, we have quantized all the computationally intensive parts, such as BMM, FC, etc, some of which are often forgotten to be quantized. Other parts, such as Softmax and Layernorm, have relatively small computational loads, but quantizing these parts would have a significant negative impact on the model. \n\n> Q3: Some details of writing should be improved and polished carefully, e.g., the subtitles of section 2 and 3, some explain of the proposed equations are missing.\n\nA3: Thank you very much for your suggestions on writing details. We have already made the changes you mentioned in the arXiv version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q7NjodFaXK", "reviewer": "Reviewer_DoGX", "summary": "This paper introduces a novel two-stage post-training quantization (PTQ) method aimed at compressing image super-resolution (SR) models for efficient deployment on edge devices. The authors address the challenge of accuracy degradation in low-bit quantization by proposing the 2DQuant method, which combines Distribution-Oriented Bound Initialization (DOBI) and Distillation Quantization Calibration (DQC) to achieve efficient and accurate super-resolution under low-bit quantization.", "questions": "See weaknesses.", "limitations": "Yes.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1. The writing of this paper is very clear.\n2. The figures in the paper are simple and easy to understand.\n3. This paper conducted some experiments to verify the effectiveness of the proposed method.", "weaknesses": "1. The contribution lacks novelty.\nThis paper does not bring any new insights. For example, the distribution of weights and activations is well studied by previous works[1][2][3]. Some papers have used searchable upper/lower bounds for quantization [2] (although these methods are based on QAT, the essence of QAT and PTQ is not different, with only slight differences in training strategies) and NO references were given in this manuscript. The proposed distillation approach is a widely used routine operation in quantization tasks [1][2][3][4][5][6].\n\n2.  Experiments are insufficient.\nThe PTQ for transformers has been extensively studied in many other papers (e.g. [4][5]), but it seems that the authors have not compared it with these methods.\n\n3.  Experimental settings are not consistent.\nOnly 100 images are used in [7], which is inconsistent with this paper.\n\n\nReferences:\n[1] Li, Huixia et al. “PAMS: Quantized Super-Resolution via Parameterized Max Scale.” ArXiv abs/2011.04212 (2020): n. pag.\n\n[2] Zhong, Yunshan et al. “Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks.” European Conference on Computer Vision (2022).\n\n[3] Hong, Chee and Kyoung Mu Lee. “Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks.” ArXiv abs/2307.13337 (2023): n. pag.\n\n[4] Li, Yanjing et al. “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” ArXiv abs/2210.06707 (2022): n. pag.\n\n[5] Liu, Shi et al. “Oscillation-free Quantization for Low-bit Vision Transformers.” International Conference on Machine Learning (2023).\n\n[6] Tu, Zhaopeng et al. “Toward Accurate Post-Training Quantization for Image Super Resolution.” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023): 5856-5865.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1a: The contribution lacks novelty... For example, the distribution of weights and activations is well studied by previous works. \n\nA1a: First, the study of distribution is not our core contribution or innovation, it is one of our experimental observations. While our main contributions are 1) the proposed 2DQuant, which utilizes DOBI and DQC to quantize Transformer-based model. 2) 2DQuant achieves 3.60x compression ration, 5.08x speedup ratio, and 4.52dB increasement in Set5 compared with SOTA[6]. Second, [1][2][3] focus on CNN-based models, such as ResNet, EDSR, and RDN networks, but our paper targets SwinIR, a Transformer-based model. There are significant differences between CNN networks and Transformer-based networks in terms of both architecture and model performance, so the direct application of these CNN-based insights/techniques on Transformer-based networks is not straightforward and even impossible.\n\n> Q1b: Some papers have used searchable upper/lower bounds for quantization (although these methods are based on QAT, the essence of QAT and PTQ is no different, with only slight differences in training strategies) and NO references were given in this manuscript. \n\nA1b:**First**, PTQ and QAT are two different concepts. PTQ stands for Post-Training Quantization, while QAT stands for Quantization-Aware Training. The former uses a pre-trained neural network without needing to update its parameters, only updating the quantizer parameters. In contrast, the latter typically uses an untrained neural network and updates both the neural network and quantizer parameters. From the perspective of memory usage, PTQ mainly requires storing the parameter values and their gradients (for backpropagation). The space required to store gradients is consistent with the space required for parameters. In addition to this, QAT needs to store the optimizer states for all parameters due to the necessity of updating the neural network's parameters. For example, Adam (used in [2]) requires saving momentum and squared gradients for each parameter, which has the same shape as the parameters. Therefore, for the same model, the memory requirement for QAT is typically twice that of PTQ, which is not a slight difference.\n\n**Second**, we are not a survey paper, so we do not need to cite all relevant articles. We have already cited similar works (like [6]), and not citing [2] is not a serious issue. To further enhance the quality of the paper, we will include a discussion of this article in the Related Work section.\n\n> Q1c: The proposed distillation approach is a widely used routine operation in quantization tasks.\n\nA1c: Although we both use a distillation method, there are significant differences in our specific implementations.\n1. We only distill the quantizer parameters and do not need to distill the model parameters, which greatly reduces memory requirements and speeds up the algorithm.\n2. The granularity of distillation is different. We target the largest substructure of the Transformer, which is the Transformer Layer. In SwinIR-light, there are a total of four Transformer layers, each containing six Transformer blocks. In contrast, [1-6] distills the residual blocks, of which there are 32 in EDSR—eight times more than ours. Therefore, our loss function computation will be faster and the distillation approach is different.\n\n> Q2:Experiments are insufficient...\n\nA2: Both [4] and [5] are QAT methods, training the model from scratch, while our proposed algorithm is a PTQ method. Comparing between PTQ method with QAT method is unfair as QAT often performs better than PTQ because of the fine-tuning of model parameters. A fairer comparison between [4][5] and our paper is applying their methods to optimize the quantizer parameters but freeze the model parameters. The result is shown below and more is in Author Rebuttal's attachment file. The additional experiments show our robustness and advantages.\n\n|Method|Bit|Set5(x4)||Set14(x4)||B100(x4)||Urban100(x4)||Manga109(x4)||\n|-|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|||PSNR|SSIM|PSNR|SSIM|PSNR|SSIM|PSNR|SSIM|PSNR|SSIM|\n|OFQ*|3|30.16|0.854|27.26|0.746|26.73|0.705|24.25|0.714|26.93|0.843|\n|Ours|3|30.91|0.870|27.75|0.757|26.99|0.713|24.85|0.736|28.21|0.868|\n|OFQ*|2|29.15|0.827|26.59|0.725|26.33|0.688|23.59|0.680|25.61|0.803|\n|Ours|2|29.53|0.837|26.86|0.732|26.47|0.693|23.84|0.691|26.07|0.816|\n\n> Q3:Experimental settings are not consistent. Only 100 images are used in [7], which is inconsistent with this paper.\n\nA3: Our paper mentions using 32 images in line 228. **First**, we and [6] are two independent papers, so there is no need to adopt the same settings. **Second**, we achieved better performance with fewer images, which is our advantage instead of weakness. **Finally**, the results of 100 images are shown below, more can be seen in Author Rebuttal.\n\n|Set Size|Bit|Set5(x4)||Set14(x4)||B100(x4)||Urban100(x4)||Manga109(x4)||\n|-|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|||PSNR|SSIM|PSNR|SSIM|PSNR|SSIM|PSNR|SSIM|PSNR|SSIM|\n|32|3|30.91|0.870|27.75|0.757|26.99|0.713|24.85|0.736|28.21|0.868|\n|100|3|30.94|0.870|27.79|0.757|27.02|0.713|24.90|0.737|28.22|0.867|\n|32|2|29.53|0.837|26.86|0.732|26.47|0.693|23.84|0.691|26.07|0.816|\n|100|2|29.82|0.843|27.03|0.736|26.57|0.696|24.01|0.698|26.40|0.823|\n\nReferences: [1] Li, Huixia et al. “PAMS: Quantized Super-Resolution via Parameterized Max Scale.” ECCV (2020).\n\n[2] Zhong, Yunshan et al. “Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks.” ECCV (2022).\n\n[3] Hong, Chee and Kyoung Mu Lee. “Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks.” ECCV (2024).\n\n[4] Li, Yanjing et al. “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” NIPS (2022)\n\n[5] Liu, Shi et al. “Oscillation-free Quantization for Low-bit Vision Transformers.” ICML (2023).\n\n[6] Tu, Zhaopeng et al. “Toward Accurate Post-Training Quantization for Image Super Resolution.” CVPR (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3oJoaQmKVr", "reviewer": "Reviewer_EXmU", "summary": "The paper presents 2DQuant, an innovative low-bit post-training quantization technique for transformer-based image super-resolution that significantly advances the state-of-the-art by introducing a two-stage optimization process. This process includes a novel Distribution-Oriented Bound Initialization strategy and a Distillation Quantization Calibration method, resulting in exceptional performance with minimal loss of accuracy for SwinIR-light. The 2DQuant approach demonstrates the potential to compress and accelerate transformer-based super-resolution models effectively for edge deployment.", "questions": "Please refer to the weaknesses above.", "limitations": "The authors have discussed the limitations in this paper.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and easy to understand.\n\n2. Transformer-based SR is much popular in recent year, this method is much helpful for obtaining accurate quantized transformer-based SR models. \n\n3. The experimental results and visualizations are sufficient.", "weaknesses": "1. The proposed DOB and DQC are much similar with DBDC and PaC, but only change weight compression to symmetric quantization, could you give more details about the differences?\n\n2. The motivation in the introduction are confusing. Usually, the deterioration of self-attention in quantized transformer causes sever performance degradation instead of the unadaptable changes in weight and activation distributions, which is also appeared in CNN-based SR networks. \n  \n3. The selected baselines in this paper are rough and the experimental results are not convinced. This paper does not compare with transformer-based post-training quantization methods, such as PTQ4ViT[1], FQ-ViT[2], NoiseQuant[3] and RepQ-ViT[4], these methods are open-source, the author should apply these PTQ methods for super resolution for that the basic transformer blocks are not different for super resolution and image classification.\n\n[1] Z. Liu et al. Post-training quantization for vision transformer. Advances in Neural Information Processing Systems, 34:28092–28103, 2021.\n\n[2] Y. Lin et al. Fq-vit: Post-training quantization for fully quantized vision transformer. arXiv preprint arXiv:2111.13824, 2021.\n\n[3] Y. Liu et al. Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20321– 20330, 2023.\n\n[4] Z. Li et al. Repq-vit: Scale reparameterization for post-training quantization of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17227–17236, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Reviewer4 EXmU\n\n> Q1:The proposed DOB and DQC are much similar with DBDC and PaC, but only change weight compression to symmetric quantization, could you give more details about the differences?\n\nA1:\nIn fact, our DOBI and DQC are quite different from DBDC and Pac. The specific differences are as follows:\n\n- **DOBI vs. DODB:**\n    1. We perform DOBI on both weights and activations, while DODB only on weights.\n    2. We use a one-sided search for activations, while DODB does not involve this aspect.\n    3. Our search objective is to find the minimum MSE, which can be accelerated by GPU, whereas DODB aims to find the minimum interval containing the T-th parameter size, where T is a hyper-parameter.\n\n- **DQC vs. Pac:**\n    1. Granularity is different. **DQC** is performed on the four Transformer layers in the SwinIR architecture, which is the largest substructure. We have tried finer granularity, such as layer-wise (24 in total) or linear-wise (96 in total), and the results indicate that this granularity is the most suitable. Limited by the number of pages, this part of the content was not included in the paper. In contrast, **DODB** is performed on each residual block, totaling 32 blocks, slower and worse than DQC.\n\n> Q2:The motivation in the introduction are confusing. ...\n\nA2:We assume that you are referring to lines 60-62 in the original text. Our statement is closer to the surface, while your statement is indeed the essence. We will modify this part to be \"Secondly, most of these methods can not adapt well to Transformer-based models because of the deterioration of self-attention in quantized transformer.\".\n\n> Q3:The selected baselines in this paper are rough and the experimental results are not convinced. This paper does not compare with transformer-based post-training quantization methods, such as PTQ4ViT[1], FQ-ViT[2], NoiseQuant[3] and RepQ-ViT[4]...\n\nA3: Thank you for your suggestions! For the four articles you mentioned, we have completed comparative experiments with them. The results of the experiments are shown below and more results can be found in the attachment file in Author Rebuttal.\n\n| Method | Bit | Set5($\\times 2$) |  | Set14($\\times 2$) |  | B100($\\times 2$) |  | Urban100($\\times 2$) |  | Manga109($\\times 2$) |  |\n|---|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  |  | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ |\n| Baseline | 32 | 38.15  | 0.961  | 33.86  | 0.921  | 32.31  | 0.901  | 32.76  | 0.934  | 39.11  | 0.978  |\n| Bicubic | 32 | 32.25  | 0.912  | 29.25  | 0.841  | 28.68  | 0.810  | 25.96  | 0.809  | 29.17  | 0.913  |\n| PTQ4ViT | 2 | 33.25  | 0.892  | 30.22  | 0.840  | 29.21  | 0.807  | 27.31  | 0.811  | 32.75  | 0.909  |\n| RepQ | 2 | 31.65  | 0.833  | 29.19  | 0.779  | 28.27  | 0.741  | 26.56  | 0.746  | 30.46  | 0.827  |\n| NoisyQuant | 2 | 30.13  | 0.762  | 28.80  | 0.756  | 28.26  | 0.742  | 26.68  | 0.763  | 30.40  | 0.820  |\n| Ours | 2 | 36.00  | 0.950  | 31.98  | 0.901  | 30.91  | 0.881  | 28.62  | 0.882  | 34.40  | 0.960  |\n\n\n| Method | Bit | Set5($\\times 3$) |  | Set14($\\times 3$) |  | B100($\\times 3$) |  | Urban100($\\times 3$) |  | Manga109($\\times 3$) |  |\n|---|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  |  | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ |\n| Baseline | 32 | 34.63  | 0.929  | 30.54  | 0.846  | 29.20  | 0.808  | 28.66  | 0.862  | 33.99  | 0.948  |\n| Bicubic | 32 | 29.54  | 0.852  | 27.04  | 0.755  | 26.78  | 0.719  | 24.00  | 0.714  | 26.16  | 0.838  |\n| PTQ4ViT | 2 | 29.96  | 0.790  | 27.36  | 0.700  | 26.74  | 0.659  | 24.56  | 0.646  | 27.37  | 0.739  |\n| RepQ | 2 | 27.32  | 0.648  | 25.63  | 0.592  | 25.44  | 0.565  | 23.42  | 0.558  | 24.51  | 0.572  |\n| NoisyQuant | 2 | 27.53  | 0.664  | 25.77  | 0.595  | 25.37  | 0.561  | 23.59  | 0.574  | 26.03  | 0.663  |\n| Ours | 2 | 31.62  | 0.889  | 28.54  | 0.804  | 27.85  | 0.768  | 25.30  | 0.768  | 28.46  | 0.881  |\n\n\n| Method | Bit | Set5($\\times 4$) |  | Set14($\\times 4$) |  | B100($\\times 4$) |  | Urban100($\\times 4$) |  | Manga109($\\times 4$) |  |\n|---|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  |  | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ | PSNR$\\uparrow$ | SSIM$\\uparrow$ |\n| Baseline | 32 | 32.45  | 0.898  | 28.77  | 0.786  | 27.69  | 0.741  | 26.48  | 0.798  | 30.92  | 0.915  |\n| Bicubic | 32 | 27.56  | 0.790  | 25.51  | 0.682  | 25.54  | 0.647  | 22.68  | 0.635  | 24.19  | 0.767  |\n| PTQ4ViT | 2 | 27.23  | 0.670  | 25.38  | 0.591  | 25.15  | 0.562  | 22.94  | 0.559  | 24.66  | 0.613  |\n| RepQ | 2 | 25.55  | 0.583  | 23.54  | 0.475  | 23.30  | 0.430  | 21.62  | 0.449  | 23.60  | 0.556  |\n| NoisyQuant | 2 | 25.94  | 0.586  | 24.33  | 0.507  | 24.16  | 0.472  | 22.32  | 0.484  | 23.82  | 0.540  |\n| Ours | 2 | 29.53  | 0.837  | 26.86  | 0.732  | 26.47  | 0.693  | 23.84  | 0.691  | 26.07  | 0.816  |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers and Area Chairs,

We appreciate all reviewers (R1-6qcS, R2-Xp5V, R3-DoGx, R4-EXmU) for the constructive reviews and positive feedback to our 2DQuant. Your expertise and insightful comments help us to further improve our paper.

We are pleased that:

- R1 and R2 acknowledge the practicality of our techniques.
- R1, R2, and R4 think our visualizations are inspiring, comprehensive, or sufficient.
- R1 and R2 recognize the impressive performance of our proposed DOBI and DQC.
- R2, R3, and R4 appreciate our writing as easy to follow or clear.

We have responded individually to each reviewer to address any concerns. Here, we offer a summary:

- We clarify the acceleration, deploying problem, fake quantization, and EDSR related problems.
- We discuss the reason that the Quant model outperforms FP model, the application of 2DQuant on the Diffusion model, the batch size ablation, and the motivation of table design.
- We explain the difference between our method and [7].
- We improve the writing, including motivation in the introduction, the subtitle of Section 2 and 3, and merging DQC into DOBI.
- We compare our approach with four PTQ methods, and one QAT method and provide analysis. The results are in the **attached PDF**.

Thanks again to all the reviewers and area chairs. We appreciate you taking the time to review our responses and hope to discuss further whether the issues have been resolved. If you need any clarification, please let us know.

Best Regards,

Authors

---

The attached PDF includes:

1. Table 1: Quantitative comparison with PTQ and QAT methods.
2. Table 2: Ablation on size of calibration set.

## Discussion of Table 1

Reviewer EXmU requests us to compare with more PTQ methods, such as PTQ4ViT[1], FQ-ViT[2], NoiseQuant[3], and RepQ-ViT[4]. Reviewer DoGX requests us to compare with more QAT methods, such as Q-ViT[5] and OFQ[6]. We have compared our methods with OFQ[6] in the PTQ fashion and all the PTQ methods. The results are shown in **Table 1 in the attached PDF file**. We fail to compare with Q-ViT[5] as their code assigns inappropriate initial values (see line 162-165 and line 188-193 in  \_quan\_base.py file in YanjingLi0202/Q-ViT) to the quantizers and the pre-trained model fails to perform Forward Propagation, let alone optimization.

For a fair comparison, we made necessary adjustments to their codes.

- FQ-ViT quantizes all modules in SwinIR, which certainly leads to worse performance compared with partly-quantized models. So we align its quantization scheme with ours, only quantize the linear and BMM modules in the Transformer block.
- NoiseQuant doesn't quantize the BMM part in the Transformer block and we fixed this bug.
- OFQ is a QAT method. For a fair comparison, we apply OFQ in our task but freeze the model parameters, only optimize the quantizer parameters. 

We mark the highest with red and the second highest with blue. It's worth noting that our method **gains SOTA performance in 2bit and 3bit situations** and is _slightly_ lower than RepQ. But RepQ uses **per-channel** quantization while ours is per-tensor. And in one transformer block, RepQ needs **128** quantizers but 2DQuant only takes **16** quantizers. If switched to per-tensor quantization, RepQ's performance is lower than ours in all situations, which is not shown in Table 1 due to page limitation. Besides, FP-ViT adopts minmax optimizer, which leads to model collapse in 4bit. So it's not necessary to compare it with lower bits.


## Discussion of Table 2

Table 2 shows the result. With a larger calibration set, 2DQuant gains higher performance in all situations.

Reference:
[1] Z. Liu et al. “Post-training quantization for vision transformer.” NIPS (2021).

[2] Y. Lin et al. “Fq-vit: Post-training quantization for fully quantized vision transformer.” IJCAI (2022).

[3] Y. Liu et al. “Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers.” CVPR (2023).

[4] Z. Li et al. “Repq-vit: Scale reparameterization for post-training quantization of vision transformers.” ICCV (2023).

[5] Li, Yanjing et al. “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” NIPS (2022).

[6] Liu, Shi et al. “Oscillation-free Quantization for Low-bit Vision Transformers.” ICML (2023).

[7] Tu, Zhaopeng et al. “Toward Accurate Post-Training Quantization for Image Super Resolution.” CVPR (2023).

Meta-review: This paper proposed a PTQ-based SR transformer. Relying on Distribution-Oriented Bound Initialization (DOBI) & distillation Quantization calibration (DQC), the method is able to achieve better compression(around 4x) and speed acceleration(around 5x).

Most reviewers appreciated the technical advancements leaning toward acceptance although one reviewer tends to reject it. 3 reviewers agreed the novelty of the work, but reviewer DoGX still has different opinion on that. Reviewer DoGX raised concerns about the novelty  (similarity between [ref1]), lack of baselines (i.e. PTQ-based Transformer) and need for demonstration about scalability. In my opinion, the authors resolved most of raised concerns in rebuttal phase. Reviewer DoGX is still unsatisfied especially on the novelty of the work, but this AC agreed the other 3 reviwers' perspective which are on consensus about certain differences from [ref1]. In my point of view, the paper demonstrated SR transformer with simulated quantization besides trainable ranges and successful results within PTQ schematic whereas [ref1] is based on QAT and demonstrated mostly on CNNs. Further, two reviewers (Xp5V & 6qcS) highlighted the importance of the paper (8 strong accept & 7 accept).

Therefore, I recommend the acceptance of the paper for publication.

[ref1] Zhong, Yunshan et al."Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks." ECCV (2022).
Decision: Accept (poster)