2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution

Authors: kai  liu
haotong  qin
yong  guo
xin  yuan
linghe  kong
guihai  chen
yulun  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-24 17:06:14

Abstract: Low-bit quantization has become widespread for compressing image super-resolution (SR) models for edge deployment, which allows advanced SR models to enjoy compact low-bit parameters and efficient integer/bitwise constructions for storage compression and inference acceleration, respectively. However, it is notorious that low-bit quantization degrades the accuracy of SR models compared to their full-precision (FP) counterparts. Despite several efforts to alleviate the degradation, the transformer-based SR model still suffers severe degradation due to its distinctive activation distribution. In this work, we present a dual-stage low-bit post-training quantization (PTQ) method for image super-resolution, namely 2DQuant, which achieves efficient and accurate SR under low-bit quantization. The proposed method first investigates the weight and activation and finds that the distribution is characterized by coexisting symmetry and asymmetry, long tails. Specifically, we propose Distribution-Oriented Bound Initialization (DOBI), using different searching strategies to search a coarse bound for quantizers. To obtain refined quantizer parameters, we further propose Distillation Quantization Calibration (DQC), which employs a distillation approach to make the quantized model learn from its FP counterpart. Through extensive experiments on different bits and scaling factors, the performance of DOBI can reach the state-of-the-art (SOTA) while after stage two, our method surpasses existing PTQ in both metrics and visual effects. 2DQuant gains an increase in PSNR as high as 4.52dB on Set5 (x2) compared with SOTA when quantized to 2-bit and enjoys a 3.60x compression ratio and 5.08x speedup ratio. The code and models are available at https://github.com/Kai-Liu001/2DQuant.

Main paper: 
 introduction: 
 As one of the most classical low-level computer vision tasks, image super-resolution (SR) has been widely studied with the significant development of deep neural networks. With the ability to reconstruct high-resolution (HR) image from the corresponding low-resolution (LR) image, SR has been widely used in many real-world scenarios, including medical imaging [13, 21, 19] , surveillance [46, 39] , remote sensing [1] , and mobile phone photography. With massive parameters, DNN-based SR models always require expensive storage and computation in the actual application. Some works have been proposed to reduce the demand for computational power of SE models, like lightweight architecture design and compression. One kind of approach investigates lightweight and efficient models as the backbone for image SR. This progression has moved from the earliest convolutional neural network (CNNs) [10, 11, 25, 49] to Transformers [48, 32, 44, 42, 4, 3] and their combinations. The parameter number significantly decreased while maintaining or even enhancing performance. The other kind of approach is compression, which focuses on reducing the parameter (e.g., pruning and distillation) or bit-width (quantization) of existing SR models.Model quantization [7, 9, 20, 31] is a technology that compresses the floating-point parameters of a neural network into lower bit-width. The discretized parameters are homogenized into restricted candidate values and cause heterogenization between the FP and quantized models, leading to severe performance degradation. Considering the process, quantization approaches can be divided into quantization-aware training (QAT) and post-training quantization (PTQ). QAT simultaneously optimizes the model parameters and the quantizer parameters [6, 16, 26, 50] , allowing them to adapt mutually, thereby more effectively alleviating the degradation caused by quantization. However, QAT often suffers from a heavy training cost and a long training time, and the burden is even much heavier than the training process of the FP counterparts, which necessitates a large amount of compatibility and makes it still far from practical in training-resource-limited scenarios. Urban100: img_092 HR Bicubic Percentile [27] DBDC+Pac [41] Ours FP Fortunately, post-training quantization emerges as a promising way to quantize models at a low training cost. PTQ fixes the model parameters and only determines the quantizer parameters through search or optimization. Previous researches [41, 26] on PTQ for SR has primarily focused on CNN-based models such as EDSR [33] and SRResNet [24] . However, these quantization methods are not practical for deployment for two reasons. Firstly, these CNN-based models themself require huge space and calculation resources. Their poor starting point makes them inferior to advanced models in terms of parameters and computational cost, even after quantization. As shown in Table 1 , the light version of SwinIR needs only 16.2% parameters and 15.9% FLOPs compared with quantized EDSR. But its PSNR metric is close to that of the FP EDSR. While the previous PTQ algorithm, DBDC+Pac, suffers from unacceptable degradation in both visual and metrics. Secondly, most of these methods can not adapt well to Transformer-based models because of the deterioration of self-attention in quantized transformers. As shown in Figure 1 , when applied on SwinIR, the existing methods still suffer from distorted artifacts compared with FP or HR.Table 1 : Complexity and performance (×4).Model EDSR [33] EDSR (4bit) [41] SwinIR-light [32] DBDC+Pac (4bit) [41] Therefore, we conducted a post-training quantization analysis on super-resolution with a classical Transformer-based model SwinIR [32] . The weight and activation distribution is characterized by coexisting symmetry and asymmetry, long tails. Firstly, if the previous symmetric quantization method is applied for asymmetric distribution, at least half of the candidates are completely ineffective. Besides, the long tail effect causes the vast majority of floating-point numbers to be compressed into one or two candidates, leading to worse parameter homogenization. Furthermore, with such a small number of parameters, SwinIR's information has been highly compressed, and quantizing the model often results in significant performance degradation. Nevertheless, the excellent performance and extremely low computational requirements of Transformer-based models are precisely what is needed for deployment in real-world scenarios.In this paper, we propose 2DQuant, a two-stage PTQ algorithm for image super-resolution tasks. To enhance the representational capacity in asymmetry scenarios, we employ a quantization method with two bounds. The bounds decide the candidate for numbers out of range and the interval of candidates in range. First, we propose distribution-oriented Bound Initialization (DOBI), a fast MSE-based searching method. It is designed to minimize the value heterogenization between quantized and FP models. Two different MSE [5] search strategies are applied for different distributions to avoid nonsense traversal. This guarantees minimum value shift while maintaining high speed and efficiency in the search process. Second, we propose Distillation Quantization Calibration (DQC), a trainingbased method. It is designed to adjust each bound to its best position finely. This ensures that the outputs and intermediate feature layers of the quantized model and that of the FP model should remain as consistent as possible. Thereby DQC allows the quantizer parameters to be finely optimized toward the task goal. The contributions of this paper can be summarized as follows:(1) To the best of our knowledge, we are the first to explore PTQ with Transformer-based model in SR thoroughly. We design 2DQuant, a unique and efficient two-stage PTQ method (see Figure 2 ) for image super-resolution, which utilizes DOBI and DQC to optimize the bound from coarse to fine.(2) In the first stage of post-quantization, we use DOBI to search for quantizer parameters, employing customized search strategies for different distributions to balance speed and accuracy. In the second stage, we design DQC, a more fine-grained optimization-based training strategy, for the quantized model, ensuring it aligns with the FP model on the calibration set. (3) Our 2DQuant can compress Transformer-based model to 4,3,2 bits with the compression ratio being 3.07×, 3.31×, and 3.60× and speedup ratio being 3.99×, 4.47×, and 5.08×. No additional module is added so 2DQuant enjoys the theoretical upper limit of compression and speedup.(4) Through extensive experiments, our 2DQuant surpasses existing SOTA on all benchmarks. We gain an increase in PSNR by as high as 4.52dB in Set5 (×2) when compressed to 2 bits, and our method has a more significant increase when compressed to lower bits. 
related work: 
 Image super-resolution. Deep CNN networks have shown excellent performance in the field of image super-resolution. The earliest SR-CNN [10, 11] method adopted a CNN architecture. It surpassed previous methods in the image super-resolution domain. In 2017, EDSR [33] won the NTIRE2017 [40] championship, becoming a representative work of CNNs in the SR by its excellent performance. Thereafter, with the continuous development of Vision Transformers (ViT) [12] , models based on the ViT architecture have surpassed many CNN networks. These Transformerbased models achieve significant performance improvements and they have fewer parameters and lower computational costs. Many works have modified the ViT architecture, achieving continuous improvements. A notable example is SwinIR [32] . With a simple structure, it outperforms many CNNbased models. However, previous explorations of post-quantization in the super-resolution domain have been limited to CNN-based models. They focus on models like EDSR [33] or SRResNet [24] . It is a far cry from advanced models no matter in parameters, FLOPs, or performance. Currently, there is still a research gap in post-quantization for Transformer architectures.Model quantization. Model quantization is used to compress large models [30, 28, 29] and can be divided into QAT and PTQ. QAT is widely accepted due to its minimal performance degradation. PAMS [26] utilizes a trainable truncated parameter to dynamically determine the upper limit of the quantization range. DAQ [17] proposed a channel-wise distribution-aware quantization scheme. CADyQ [16] is designed for SR networks and optimizes the bit allocation for local regions and layers in the input image. However, QAT usually requires training for as long as or even longer than the original model, which becomes a barrier for real scenarios deployment. Instead of training the model from scratch, existing PTQ methods use the pre-trained models. PTQ algorithms only find the clipping bound for quantizers, saving time and costs. DBDC+Pac [41] is the first to optimize the post-training quantization for image super-resolution task. It outperforms other existing PTQ algorithms. Whereas, they only focus on EDSR [33] and SRResNet [24] . Their 4-bit quantized version is inferior to advanced models in terms of parameters and computational cost, let alone performance. It reveals a promising result for PTQ on SR, and using a more advanced model could bridge the gap between high-performance models and limited calculation resource scenarios. 
methodology: 
 To simulate the precision loss caused by quantization, we use fake-quantize [22] ,i.e.quantizationdequantization, for activations and weights. and the process can be written asEQUATIONX W Q W K W V Q K V BMM Scale SoftMax BMM Projection LayerNorm FC2 GELU FC1 FP32 INT4Figure 3 : Quantization scheme for SwinIR Transformer blocks. Fake quantization and INT arithmetic are performed in all compute-intensive operators including all linear layers and batch matmul. Lower bits such as 3 or even 2 are also permitted. Dropout of attention and projection is ignored where v denotes the value being fake quantized, which can be weight or activation. l and u are the lower and upper bounds for clipping, respectively. Clip(v, l, u) = max(min(v, u), l), and Round rounds the input value to the nearest integer. v c denotes the value after clipping, and v r denotes the integer representation of v, and v q denotes the value after fake quantization. The Clip and Round operations contribute to reducing the parameters and FLOPs but also introduce quantization errors. Figure 3 shows the basic structure of the Transformer block. We have quantized all the modules with a significant computational load within them, effectively reducing the model's FLOPs. Table 2 shows the FLOPs needed for each module. The Linear layers and matrix multiplication account for approximately 86% of the computation load, which are all transformed into integer arithmetic. When performing gradient backpropagation, we follow the Straight-Through Estimator [8] (STE) style:EQUATIONwhere ∂vc ∂u = H(u -v) and vc ∂l = H(l -v), H(•) denotes Heaviside step function [47] . This formula approximates the direction of gradient backpropagation, allowing training-based optimization to proceed. The derivation of the formula can be found in the supplementary material.Figure 2 shows the whole pipeline of 2DQuant, which is a two-stage coarse-to-fine post-training quantization method. The first stage is DOBI, using two strategies to minimize the value shift while the second stage is DQC, optimizing two bound of each quantizer towards the task goal.To achieve better quantization results, we need to analyze the distribution of the model's weights and activations in detail. We notice that the data distribution shows a significantly different pattern from previous explorations, invalidating many of the previous methods. The weights and activations distribution of SwinIR is shown in Figure 4 . More can be found in supplemental material. Specifically, the weights and activations of SwinIR exhibit noticeable long-tail, coexisting symmetry and asymmetry.Weight. The weights of all linear layers are symmetrically distributed around zero, showing clear symmetry, and are generally similar to a normal distribution. This is attributed to the weight decay applied to weights, which provides quantization-friendly distributions. From the value shift perspective, both symmetric and asymmetric quantization are tolerable. Whereas, from the vantage point of task objectives, asymmetric quantization possesses the potential to offer a markedly enhanced information density, thus elevating the overall precision of the computational processes involved.Activations. As for activations, they exhibit obvious periodicity in different Transformer Blocks. For V or the input of FC1, the obtained activation values are symmetrically distributed around 0. However, for the attention map or the input of FC2 in each Transformer Block, due to the Softmax calculation or the GELU [14] activation function, the minimum value is almost fixed, and the overall distribution is similar to an exponential distribution. Therefore, the data in SwinIR's weights and activations exhibit two distinctly different distribution characteristics. Setting asymmetric quantization and different search strategies for both can make the search rapid and accurate.Because the data distribution exhibits a significant long-tail effect, we must first clip the range to avoid low effective bits. Common clipping methods include density-based, ratio-based, and MSE-based approaches. The first two require manually specifying the clipping ratio, which significantly affects the clipping outcome and necessitates numerous experiments to determine the optimal ratio. Thus we proposed the Distribution-Oriented Bound Initialization (DOBI) to search the bound for weight and 0.0 0.2 0.4 0.6 0.8 1.0>@ $FWOD\HUEORFN$WWHQWLRQ 2 0 2 >@ $FWOD\HUEORFN95.0 2.5 0.0 2.5 5.0 7.5 >@ $FWOD\HUEORFN)& 0 2 4 6 >@ $FWOD\HUEORFN)& 0.2 0.0 0.2EQUATIONThe collection of all quantizers' bounds {(l i , u i )} N i=1 is the linchpin of quantized model performance as it indicates the candidate value for weights and activations. We note that the data distribution falls into two categories: one resembling a bell-shaped distribution and the other resembling an exponential distribution. For the bell-shaped distribution, we use a symmetric boundary-narrowing search method. Whereas, for the exponential distribution, we fix the lower bound to the minimum value of the data and only traverse the right bound. The specific search method is shown in Algorithm 1. The time complexity of Algorithm 1 is O(M K), where M is the number of elements in data v and K is the number of search points. The condition v is symmetrical is obtained by observing the visualization of v and the activations are from the statistics on a small calibration set.To further fine-tune the clipping range, we propose distillation quantization calibration (DQC) to transfer the knowledge from the FP model to the quantized model. It leverages the knowledge distillation [15] where the FP model acts as the teacher while the quantized model is the student. Specifically, for the same input image, the student model needs to continuously minimize the discrepancy with the teacher model on the final super-resolution output.Algorithm 1: DOBI pipeline Data: Data to be quantized v, the number of search pointK, bit b Result: Clip bound l, u l ← min(v),u ← max(v); min_mse ← +∞; if v is symmetrical then ∆l ← (max(v) -min(v))/2K; else ∆l ← 0; end ∆u ← (max(v) -min(v))/2K; while i ≤ K do l i ← l + i × ∆l, u i ← u + i × ∆u; get v q based on Eq. (1); mse ← ∥v -v q ∥ 2 ; if mse ≤ min_mse then min_mse ← mse; l_best ← l i , u_best ← u i ; end endThe loss for the final output can be written asEQUATIONwhere O and O q are the final outputs of the teacher and student models, C O , H O , and W O represent the number of output channels, height, and width, respectively. we adopt the L1 loss for the final output, as it tends to converge more easily compared to the L2 loss [33] . As the quantized model shares the same structure with the FP model and is quantized from the FP model, the student model also need to learn to extract the same feature of the teacher model, which can be written asEQUATIONwhere F i and F qi are the intermediate features of the teacher and student models respectively and i is the index of the layer. In the field of super-resolution, there is a clear correspondence between the feature maps and the final reconstructed images, making training on feature maps crucial. since the quantized network and the full-precision network have identical structures, we do not need to add extra adaptation layers for feature distillation. The final loss function can be written asEQUATIONwhere λ is the co-efficient of L F . In the second stage, based on training optimization methods, the gap between the quantized model and the full-precision model will gradually decrease. The performance of the quantized model will progressively improve and eventually converge to the optimal range.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: The paper does not include theoretical results. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments: 
 4.1 Experimental settings Data and evaluation. We use DF2K [40, 34] as the training data, which combines DIV2K [40] and Flickr2K [34] , as utilized by most SR models. During training, since we employ a distillation training method, we do not need to use the high-resolution parts of the DF2K images. For validation, we use the Set5 [2] as the validation set. After selecting the best model, we tested it on five commonly used benchmarks in the SR field: Set5 [2] , Set14 [45] , B100 [36] , Urban100 [18] , and Manga109 [37] . On the benchmarks, we input low-resolution images into the quantized model to obtain reconstructed images, which we then compared with the high-resolution images to calculate the metrics. We do not use self-ensemble in the test stage as it increases the computational load eightfold, but the improvement in metrics is minimal The evaluation metrics we used are the most common metrics PSNR and SSIM [43] , which are calculated on the Y channel (i.e., luminance) of the YCbCr space.Implementation details. We use SwinIR-light [32] as the backbone and provide its structure in the supplementary materials. We conduct comprehensive experiments with scale factors of 2, 3, and 4 and with 2, 3, and 4 bits, where Our hyperparameter settings remain consistent. During DOBI, we use a search step number of K=100, and the statistics of activations are obtained from 32 images in DF2K being randomly cropped to retain only 3×64×64. During DQC, we use the Adam [23] optimizer with a learning rate of 1×10 -2 , betas set to (0.9, 0.999), and a weight decay of 0. We employ CosineAnnealing [35] as the learning rate scheduler to stabilize the training process. Data augmentation is also performed. We randomly utilize rotation of 90°, 180°, and 270°and horizontal flips to augment the input image. The total iteration for training is 3,000 with batch size of 32. Our code is written with Python and PyTorch [38] and runs on an NVIDIA A800-80G GPU.The methods we compared include MinMax [22] , Percentile [27] , and the current SOTA postquantization method in the super-resolution field, DBDC+Pac [41] . For a fair comparison, we report the performance of DBDC+Pac [41] on EDSR [33] , as the authors performed detailed parameter adjustments and model training on EDSR. We directly used the results reported by the authors, recorded in the table as EDSR † . It should be noted that the EDSR method uses self-ensemble in the final test, which can improve performance to some extent but comes at the cost of 8 times the computational load. Additionally, we applied DBDC+Pac [41] to SwinIR-light [32] , using the same hyperparameters as those set by the authors for EDSR, recorded in the table as DBDC+Pac [41] . The following are the quantitative and qualitative results of the comparison. Quantitative results. Table 3 shows the extensive results of comparing different quantization methods with bit depths of 2, 3, and 4, as well as different scaling factors of ×2, ×3, and ×4. DBDC+Pac [41] performs poorly mainly because 1. The DBDC process requires manually specifying the clipping ratio, which significantly affects performance. 2. DBDC does not prune weights, and the learning rate in the Pac process is too low, causing slow convergence of weight quantizer parameters. However, both adverse factors are eliminated in our 2DQuant algorithm. When using only DOBI algorithm, our performance has already reached a level comparable to that of DBDC+Pac algorithms. Upon applying DQC, our performance experienced a remarkable and discernible enhancement, elevating it to new heights. In the case of ×2, 4-bit on Set5 and Urban100, DOBI has an improvement of 1.11dB and 0.39 dB compared to EDSR, while bounds vary more significantly due to the elevated significance of these layers within the neural network. Detailedly, the bound for the second MLP fully connected layer's weight in Layer 0 Block 1 only remains 46% data in its range. It has the second-highest lower bound percentile and the smallest upper bound percentile among the network. Its percentiles are 0.2401 and 0.7035 respectively while its bound values are -0.062 and 0.047 and its distribution is visualized in Figure 4 . In conclusion, only through task-oriented optimization of each bound at a fine-grained level can redundant information be maximally excluded and useful information be maximally retained.Qualitative results. We show the visual comparison results for ×4 in Figure 6 . Since quantized models are derived from full-precision models with information loss, their global performance will rarely exceed that of full-precision models. As seen in the three images for Minmax, after quantization, if no clipping is performed, the long tail effect will lead to a large number of useless bits, resulting in a significant amount of noise and repeated distorted patterns in the reconstructed images. In these challenging cases, our training method allows the model to retain edge information of objects better, preventing blurring and distorted effects. For example, in img_046 and img_023, we have the highest similarity to the full-precision model, while other methods show varying degrees of edge diffusion, significantly affecting image quality. Compared to the DBDC+Pac method, our DOBI and DQC allow for better representation of edge and texture information in the images and effectively avoid distortions and misalignments in the graphics. The visual results demonstrate that our proposed DQC is essential for improving performance in both metric and visual comparisons.Learning rate and batchsize. We first study the performance variations of the model under different hyperparameters. From Tables 4a and 4b , it can be seen that our DQC enables the model to converge within a range of outstanding performance for most learning rates and batch sizes. Due to the non-smooth impact of quantization parameters on the model, the quantized model is more prone to local optima compared to the full-precision model, resulting in a noticeable performance drop when the learning rate is too low. Additionally, as shown in Table 4b , the larger the batch size, the better the model's performance, and the smoother the convergence process. However, even with a smaller batch size, we can still achieve a performance of 37.82dB on Set5, indicating that our two-stage method has good robustness to different hyperparameters.DOBI and DQC. Moreover, we also study the impact of different stages on performance, with the results shown in Table 4c , from which we can draw the following conclusions: Firstly, the goal of DOBI is to minimize the value shift for weights and activations. Although it is not the task goal, it can still enjoy significant enhancement due to better bit representational ability. Secondly, DQC alone cannot achieve the optimization effect of DOBI. This is because the impact of quantizer parameters on model performance is oscillatory, and training alone is prone to converge to local optima. In contrast, search-based methods can naturally avoid local optima. So it's necessary to use results from the search-based method to initialize training-based method in PTQ. Thirdly, when DOBI and DQC are combined, namely our 2DQuant, the 4-bit quantized model has only a 0.28dB decrease on Set5 compared to the FP model, which maximally mitigates the accuracy loss caused by quantization.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: We have provided implementation details in the experiments section. We will also release all the code and models. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
discussion: 
 Why our results surpass FP outcomes. While our method's performance metrics do not yet fully match those of full-precision models, visual results reveal a compelling advantage. As observed in image img_092 of Figure 1 of Urban100, our approach correctly identifies the direction of the stripes in the image. Whereas the full-precision model erroneously selects the wrong direction. This discrepancy arises because the lower-resolution image, affected by aliasing, creates an illusion of slanted stripes, misleading the FP model's reconstruction. This phenomenon demonstrates that our PTQ algorithm allows more accurate restored results in certain localized and challenging tasks without being misled. More examples are in the supplementary materials. It suggests that full-precision models contain not only redundant knowledge but also incorrect information. The latter is hard to get rid of by training the FP model. Our quantization method can effectively reduce model parameters and computational demands while eliminating erroneous information, achieving multiple benefits simultaneously. This also suggests that the FP model doesn't represent the pinnacle of what a quantized model can achieve.Limitations. Despite achieving excellent results, this study still has some limitations. During the DOBI process, the data distribution of activations and weights is required to approximate a bell curve or exponential distribution; otherwise, the DOBI method cannot find the most suitable positions. Additionally, increasing the number of search points for a single tensor in MSE does not necessarily guarantee better performance. However, the second-stage training can somewhat alleviate this issue. Moreover, our method requires a calibration set; without which, the first-stage DOBI and the second-stage DQC cannot be carried out at all.Societal impacts. Our super-resolution quantization method effectively saves computational resources, facilitating the deployment of super-resolution models at the cutting edgeQuestion: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: Firstly, all data in the paper can be accessed publicly. Secondly, we provide very detailed instructions (e.g., method descriptions and implementation details) to reproduce our results. Thirdly, we promise to release code and all models.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details. • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 This paper studies the post-training quantization in the field of image super-resolution. We first conducted a detailed analysis of the data distribution of Transformer-based model in SR. These data exhibit a clear long-tail effect and symmetry and asymmetry coexisting effect. We designed 2DQuant, a dual-stage PTQ algorithms. In the first stage DOBI, we designed two different search strategies for the two different distributions. In the second stage DQC, we designed a distillationbased training method that let the quantized model learn from the FP model, minimizing the accuracy loss caused by quantization. Our 2DQuant can compress Transformer-based model to 4,3,2 bits with the compression ratio being 3.07×, 3.31×, and 3.60× and speedup ratio being 3.99×, 4.47×, and 5.08×. No additional module is added so 2DQuant enjoys the theoretical upper limit of compression and speedup. Extensive experiments demonstrate that 2DQuant surpasses all existing PTQ methods in the field of SR and even surpasses the FP model in some challenging cases. In the future, recognizing the significant impact of the model on performance, we will conduct PTQ research on more advanced super-resolution models and attempt to deploy quantized super-resolution algorithms to actual photography tasks, providing a more detailed evaluation of the performance of PTQ algorithms.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We have provided implementation details, which cover the above questions.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: Please refer to the experiment part.• The answer NA means that the paper does not include experiments. • The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Please refer to the supplementary file.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This work poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We have credited most previous works in the paper. The license and terms are respected properly.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: We will release code and models. In the paper, we have provided implementation details and other contents to reproduce our results.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: This paper proposed a PTQ-based SR transformer. Relying on Distribution-Oriented Bound Initialization (DOBI) & distillation Quantization calibration (DQC), the method is able to achieve better compression(around 4x) and speed acceleration(around 5x).

Most reviewers appreciated the technical advancements leaning toward acceptance although one reviewer tends to reject it. 3 reviewers agreed the novelty of the work, but reviewer DoGX still has different opinion on that. Reviewer DoGX raised concerns about the novelty  (similarity between [ref1]), lack of baselines (i.e. PTQ-based Transformer) and need for demonstration about scalability. In my opinion, the authors resolved most of raised concerns in rebuttal phase. Reviewer DoGX is still unsatisfied especially on the novelty of the work, but this AC agreed the other 3 reviwers' perspective which are on consensus about certain differences from [ref1]. In my point of view, the paper demonstrated SR transformer with simulated quantization besides trainable ranges and successful results within PTQ schematic whereas [ref1] is based on QAT and demonstrated mostly on CNNs. Further, two reviewers (Xp5V & 6qcS) highlighted the importance of the paper (8 strong accept & 7 accept).

Therefore, I recommend the acceptance of the paper for publication.

[ref1] Zhong, Yunshan et al."Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks." ECCV (2022).

Reviews: review: paper_summary: paper_summary: The authors propose a low-bit post-training quantization (PTQ) method, 2DQuant, for image super-resolution. 2DQuant is a dual-stage low-bit PTQ method. They first investigate the weight and activations. They propose Distribution-Oriented Bound Initialization (DOBI) by using different searching strategies to get a coarse bound for quantizers. They further propose Distillation Quantization Calibration (DQC) to refine the quantizer parameters with a distillation approach. They provide extensive experiments on different bits and scaling factors. When compared with SOTA methods, there proposed methods achieve superior performance with 3.6X compression ratio and 5.08X speedup ratio in the 2-bit case.
strength_weakness: strength_weakness: The authors did not give results about inference time. The quatization can reduce params and ops largely. Then, how about the acceleration when deploying the quantized models into device?

Some parts in the experiments are confusing. For example, in Table 3, the results of EDSR confuse me somehow. I am not very sure how the EDSR results are obtained. The original EDSR is full-precision model. Here, the authors report its quantized version. The baseline in Table 3 is not very clear. Please clarify them.

In Table 4 (b), when the batch size becomes larger, the performance changes slightly. This is not very consistent with the common observation, where larger batch size usually improves performance.

The writing can be further improved. Some details should be given attention and revised. For example, Section 3.2, the first letter should be capital.
questions: questions: In Figure 1, the proposed method performs even better than the full-precision (FP) version. What are the potential reasons behind this observation?

Can this method be applied to diffusion based image SR models? If so, please give some key ideas. It will be much better if the authors can provide some preliminary results.

In Table 3, what is the model size for the original EDSR? I do not which version of EDSR did the authors use in the paper.

What is the motivation by providing two versions, i.e., DOBI (ours) and 2DQuant (ours), in Table 3?

------------After Rebuttal and  Discussions----------
After rebuttal and disscussions, my concerns have been well solved.
limitations: limitations: The authors have discussion the limitation in their paper.

Flag For Ethics Review: No ethics review needed.
paper_summary: paper_summary: This paper present a practical post-training quantization method for SR transformer, namely 2DQuant. The 2dquant mainly rely one two techniques, the first is Distribution-Oriented Bound Initialization (DOBI) determine the quantization range initially, and the second one is distillation Quantization calibration (DQC) to further finetune the quantizer for accurate quantization. Both of the proposed two techniques significantly improving the performance of PTQ for SR transformer, and the quantization can even be pushed to as low as 2-bit without retraining.
strength_weakness: strength_weakness: 1. As presented in section 3, the authors use fake quantization during the design of the method, but didn’t mention if it can be replaced by the real quant, which can be implemented and bring real acceleration in hardware. The author should discuss if the proposed method can achieve real quantized inference when deployment.

2. The algorithm of DQC is missed and make the description not very clear, I suggest to add it or merge it to the algorithm of DOBI. And Figure 3 not highlight the application of the proposed techniques, which is also suggested to revise.

3. Some details of writing should be improved and polished carefully, e.g., the subtitles of section 2 and 3, some explain of the proposed equations are missing.
questions: questions: Please respond for the issues raised in weaknesses part.
limitations: limitations: The authors have discussed the limitation in their paper.
paper_summary: paper_summary: This paper introduces a novel two-stage post-training quantization (PTQ) method aimed at compressing image super-resolution (SR) models for efficient deployment on edge devices. The authors address the challenge of accuracy degradation in low-bit quantization by proposing the 2DQuant method, which combines Distribution-Oriented Bound Initialization (DOBI) and Distillation Quantization Calibration (DQC) to achieve efficient and accurate super-resolution under low-bit quantization.
strength_weakness: strength_weakness: 1. The contribution lacks novelty.
This paper does not bring any new insights. For example, the distribution of weights and activations is well studied by previous works[1][2][3]. Some papers have used searchable upper/lower bounds for quantization [2] (although these methods are based on QAT, the essence of QAT and PTQ is not different, with only slight differences in training strategies) and NO references were given in this manuscript. The proposed distillation approach is a widely used routine operation in quantization tasks [1][2][3][4][5][6].

2.  Experiments are insufficient.
The PTQ for transformers has been extensively studied in many other papers (e.g. [4][5]), but it seems that the authors have not compared it with these methods.

3.  Experimental settings are not consistent.
Only 100 images are used in [7], which is inconsistent with this paper.


References:
[1] Li, Huixia et al. “PAMS: Quantized Super-Resolution via Parameterized Max Scale.” ArXiv abs/2011.04212 (2020): n. pag.

[2] Zhong, Yunshan et al. “Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks.” European Conference on Computer Vision (2022).

[3] Hong, Chee and Kyoung Mu Lee. “Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks.” ArXiv abs/2307.13337 (2023): n. pag.

[4] Li, Yanjing et al. “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” ArXiv abs/2210.06707 (2022): n. pag.

[5] Liu, Shi et al. “Oscillation-free Quantization for Low-bit Vision Transformers.” International Conference on Machine Learning (2023).

[6] Tu, Zhaopeng et al. “Toward Accurate Post-Training Quantization for Image Super Resolution.” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023): 5856-5865.
questions: questions: See weaknesses.
limitations: limitations: Yes.
paper_summary: paper_summary: The paper presents 2DQuant, an innovative low-bit post-training quantization technique for transformer-based image super-resolution that significantly advances the state-of-the-art by introducing a two-stage optimization process. This process includes a novel Distribution-Oriented Bound Initialization strategy and a Distillation Quantization Calibration method, resulting in exceptional performance with minimal loss of accuracy for SwinIR-light. The 2DQuant approach demonstrates the potential to compress and accelerate transformer-based super-resolution models effectively for edge deployment.
strength_weakness: strength_weakness: 1. The proposed DOB and DQC are much similar with DBDC and PaC, but only change weight compression to symmetric quantization, could you give more details about the differences?

2. The motivation in the introduction are confusing. Usually, the deterioration of self-attention in quantized transformer causes sever performance degradation instead of the unadaptable changes in weight and activation distributions, which is also appeared in CNN-based SR networks. 
  
3. The selected baselines in this paper are rough and the experimental results are not convinced. This paper does not compare with transformer-based post-training quantization methods, such as PTQ4ViT[1], FQ-ViT[2], NoiseQuant[3] and RepQ-ViT[4], these methods are open-source, the author should apply these PTQ methods for super resolution for that the basic transformer blocks are not different for super resolution and image classification.

[1] Z. Liu et al. Post-training quantization for vision transformer. Advances in Neural Information Processing Systems, 34:28092–28103, 2021.

[2] Y. Lin et al. Fq-vit: Post-training quantization for fully quantized vision transformer. arXiv preprint arXiv:2111.13824, 2021.

[3] Y. Liu et al. Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20321– 20330, 2023.

[4] Z. Li et al. Repq-vit: Scale reparameterization for post-training quantization of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17227–17236, 2023.
questions: questions: Please refer to the weaknesses above.
limitations: limitations: The authors have discussed the limitations in this paper.

Comments: title: After rebuttal
comment: Thanks for your detailed responses, which have well addressed my concerns. I also go through other reviewer's comments, and recognize the contributions of  low-bit post-training quantization (PTQ) for image super-resolution with using different searching strategies to get a coarse bound for quantizers. Overall, I tend to keep my original score as 6.
Dear Reviewer DoGX,

Thank you for your quick response and for outlining the concern that remains unresolved.

> Q1: My concerns still revolve around the similarity between the core idea of this article and [1]

A1: The main contributions of [1] are 1) A layer-wise quantizer with trainable upper and lower bounds, 2) A dynamic gate controller to adaptively adjust the upper and lower bounds at runtime. We differentiate ourselves from [1] in the following aspects:

- **Quantizer.** We have indeed utilized the same quantize. But this is widely used and is not our main contribution.
- **Different loss functions.** The granularity of the loss functions during training differs. Our DQC is conducted on the four Transformer layers within the SwinIR architecture, which represents the largest substructure. Experimentation with finer granularity, such as layer-wise (24 in total) or linear-wise (96 in total), has shown that this level of granularity is most optimal. Due to page limitations, this part of the content was omitted from the paper. In contrast, [1] applies quantization to each residual block, totaling 32 blocks in EDSR, which is slower and less effective than DQC.
- **No additional modules.** [1] employs a __Dynamic gate controller__, necessitating additional floating-point (FP) modules to obtain dynamic bounds, thereby increasing the inference cost. We, on the other hand, use entirely static quantizers and do not require any additional modules, achieving the theoretical optimal speedup ratio for quantization.
- **Initial values from DOBI.** [1] uses the percentile method to assign initial values, which is highly sensitive to the hyperparameters and thus requires a time-consuming search for the best parameters. Failure to do so can lead to **model collapse**, as reported in their GitHub repository issue 3. This is particularly problematic for Transformer-based models, where the self-attention modules contain many more activations. The consequence is that all elements in the attention map become the same value after quantization. Therefore, it is imperative to apply our efficient DOBI method for initial value assignment. The experimental results in Table 3 demonstrate that DOBI alone can achieve SOTA performance.

Additionally, we would like to bring to your attention a **critical issue** in [1]'s implementation. They report the quantization method as per-layer quantization. But, in their github repo files(line 327-349 in model/quant\_ops.py in github repo \"zysxmu/DDTB\"), they actually uses different bounds for different channels in one activation tensor. This implement is a typical error for per-channel quantization. This is not appliable in real world GPU and will greatly **increase** the model performance. And the correct approach to quantization involves using a single pair of clipping bounds for activations and different pairs for different convolutional kernels, which is also noted in [2]. We hope you can focus on **correct** quantization approaches instead of one with implement error.


We trust these clarifications address your concerns and further highlight the unique contributions of our work.

Sincerely,

Authors

References:

[1] Zhong, Yunshan et al. "Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks." ECCV (2022).

[2] Markus Nagel et al. "A White Paper on Neural Network Quantization." arXiv:2106.08295.
Dear Authors,

I hope this message finds you well.

My concerns still revolve around the similarity between the core idea of this article and [1]. Although you mentioned the differences in training details between PTQ and QAT (such as the parameters updating during backward pass) and the contribution of your method to SwinIR (which I have never denied), the academic contribution of this paper is still not convincing. The difference between SwinIR and CNN models lies in the model structure, rather than the constituent modules of the model. Therefore, applying existing quantization methods to new model architectures in this paper has a certain amount of engineering contribution rather than academic contribution. If the paper considers contribution from the perspective of engineering quantity, should deployment on practical reasoning frameworks become a necessary factor? In conclusion, I choose to maintain my score.

Sincerely,

Reviewer DoGX

References: [1] Zhong, Yunshan et al. “Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks.” ECCV (2022).
Dear Reviewer DoGX,

We hope this message finds you well.

In our preceding rebuttal, we made every effort to address your concerns and dispel any misunderstandings. Below is a summary of the issues we have resolved:

- We have restated our primary contributions and clarified the difference between the observed distribution and those of prior studies.
- We have elucidated the reasons for our initial omission of reference [1] and have now included it in our revised manuscript.
- We have delineated the differences between DQC and previous distillation methodologies.
- We have conducted additional experiments, comparing our approach with five other quantization methods, thereby demonstrating our superior performance in the 2-bit scenario.
- We have explained the reason behind our choice of different calibration set sizes and have provided evidence of even better results when using the same set size.
- We have expounded on the divergences between our proposed PTQ method and other QAT approaches.
- We have clarified that we have neither avoided citations nor exaggerated our contributions.

As the deadline for the rebuttal approaches, we have noticed that **Reviewer EXmU has acknowledged the merits of our work**, and we are eager to learn if there are any remaining concerns from your side that have not yet been addressed, to which we will provide further detailed explanations and clarification.

We would like to express our gratitude once again for the time you have invested in reviewing our work and for the constructive feedback you have offered.

Sincerely,

Authors

References:
[1] Zhong, Yunshan et al. “Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks.” ECCV (2022).
Dear Reviewer EXmU,

We extend our sincerest gratitude for your thoughtful and constructive feedback on our 2DQuant. Your recognition of the innovative aspects of our work, particularly the utility of 2DQuant in enhancing the deployment of transformer-based image super-resolution models, is **greatly appreciated**. We are honored by your decision to raise your score to a "weak accept", reflecting your positive view of our research's contribution to model quantization and SR fields.

**Thank you again for your time and the positive impact your suggestions have had on our work.**

Sincerely,

Authors
Thanks for the detailed explanation, I think I misunderstood a bit about the contributions of this paper. 2DQuant is much useful for accelerating the deployment of transformer-based image super resolution. This method is particularly beneficial to reduce the quantization error of  post-softmax and post-GELU feature maps in transformer architectures. After reviewing the latest response, I decided to raise my score to 6 (weak accept).
Dear Reviewer EXmU,

We appreciate your constructive reviews on the additional experiments and the positive feedback on our performance. We would like to continue addressing your concerns and resolving any misunderstandings.

> Q1: The specific differences about 2DQuant and PTQ4SR ... are marginal. DOBI on both weights and activations, one-side searching，new search object and different granularity are just engineering optimisations, the core distributions are still based on PTQ4SR. 

A1: We clarify that the only similarity between our DOBI and PTQ4SR is the use of a search algorithm, while all other aspects are different. If these differences are merely considered engineering optimizations, then under this assumption, most search-based PTQ methods would lack any innovative aspects, which is clearly unrealistic.

> Q2: Although 2DQuant gets much better performance than baselines, the academic contributions of this paper are negligible. 

Our 2DQuant shows an increase in PSNR of up to 4.52 dB on Set5 (×2) compared with PTQ4SR. This increase is significant in the SR field. How can such a substantial improvement be considered negligible? The SR field urgently requires lightweight and high-performance DNN-based SR models. Our method demonstrates that a DNN-based SR model with 2-bit quantization can surpass Bicubic, which is the standard for practical use. Therefore, our method can greatly accelerate the deployment of SR models in the real world, representing a significant contribution to the SR field.

> Q3: ...this paper uses the A+B approach to exaggerate the contribution.

Could you please specify how we have exaggerated our contributions? We believe that your request for additional experiments is intended to further highlight and strengthen our contributions. The additional experiment results show that 2DQuant surpasses all other PTQ methods you requested for comparison, as well as the state-of-the-art PTQ4SR, confirming that we have not exaggerated our contributions.

We value your feedback and have addressed each of your concerns in detail. The significant performance improvements demonstrated by 2DQuant, combined with its novel approach and impact on real-world deployment, affirm its substantial contribution to the SR field. We hope this clarification resolves any misunderstandings and further underscores the value of our work.

Thank you for your consideration.

Sincerely,  

Authors
Thank you for the response. After reviewing the rebuttal, I can confirm that all of my concerns have been fully addressed.
Thank you for the response and additional experiments.  The specific differences about 2DQuant and PTQ4SR are clearly stated, which further shows that the differences between these two methods are marginal. DOBI on both weights and activations, one-side searching，new search object and different granularity are just engineering optimisations, the core distributions are still based on PTQ4SR. Although 2DQuant gets much better performance than baselines, the academic contributions of this paper are negligible.  And I agree with Reviewer DoGX that this paper use the A+B approach to exaggerate the contribution. So I decided to keep my score.
Dear Reviewer DoGX,

We appreciate your constructive reviews and positive feedback on our work, 2DQuant. We would like to continue addressing your concerns and resolving any misunderstandings.

> Q1: The author's rebuttal partially solved my problem.

First, you mentioned that we partially solved your problem. We would like to know what remains unresolved. We will do our best to address it. Please let us know the specifics.

> Q2: In fact, PTQ and QAT are only different in the training process, and there is no difference in quantization essence.

Second, you emphasize once again that QAT shares the same essence with PTQ. We do not agree with this point due to the differences in training cost, performance degradation, and task-specific designs. Nevertheless, we have provided detailed experimental results of [2]. We also mentioned in the rebuttal that we will cite [1] in our reversion. Our method outperforms [2] when following the PTQ settings. We think we have reached an agreement in this issue.

> Q3: Since there are already highly similar quantization approaches in previous works...

Third, we do not think the previous works are "highly similar" to ours. **IF so, how could our method significantly outperform other methods in the 2-bit senario?** The experimental results demonstrate that our approach is distinct from previous ones and **certainly not a simple combination of A and B**.

> Q4: ...rather than attempting to use the A+B approach to avoid citation and exaggerate the contribution.

Additionally, we must clarify that **we do not avoid citation or exaggerate the contribution**. As mentioned earlier, we will cite [1] in the reversion. Furthermore, the experimental results, not we, demonstrate the contributions, leaving no room for exaggeration.

> Q5: So the final rating would be 3.

We noticed that you scored **3** at first, but shortly after the beginning of the rebuttal, you revised it to **2** without explanation. After addressing your concerns and making efforts to resolve them, you reverted your rating back to **3**. We'd like to know why you decided to make the initial change. Could you please clarify the reason behind your initial change? Kindly list any unresolved issues so we can further improve the work.

Thank you for your continued feedback and constructive engagement with our work. We look forward to resolving any remaining concerns and continuing the discussion to further improve our research.

Best Regards,

Authors

[1] Zhong, Yunshan et al. “Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks.” ECCV (2022).

[2] Liu, Shi et al. “Oscillation-free Quantization for Low-bit Vision Transformers.” ICML (2023).
The author's rebuttal partially solved my problem. The authors mentioned that 'not urbanization [2] is not a serious issue.' In fact, PTQ and QAT are only different in the training process, and there is no difference in quantization essence. Since there are already highly similar quantization approaches in previous works, the authors should first consider clarifying the contribution of this paper during their writing, rather than attempting to use the A+B approach to avoid citation and exaggerate the contribution. So the final rating would be 3.