Rate-Distortion Optimized Post-Training Quantization for Learned Image Compression
Authors: Junqi Shi
Ming Lu
fangdong chen
Shiliang Pu
Zhan Ma
Venue: Submitted to ICLR 2023 (2023)
Keywords: learned image compression
post-training quantization
rate-distortion optimization

Abstract: Quantizing floating-point neural network to its fixed-point representation is crucial for Learned Image Compression (LIC) because it ensures the decoding consistency for interoperability and reduces space-time complexity for implementation. Existing solutions often have to retrain the network for model quantization which is time consuming and impractical. This work suggests the use of Post-Training Quantization (PTQ) to directly process pretrained, off-the-shelf LIC models. We theoretically prove that minimizing the mean squared error (MSE) in PTQ is sub-optimal for compression task and thus develop a novel Rate-Distortion (R-D) Optimized PTQ (RDO-PTQ) to best retain the compression performance. Such RDO-PTQ just needs to compress few images (e.g., 10) to optimize the transformation of weight, bias, and activation of underlying LIC model from its native 32-bit floating-point (FP32) format to 8-bit fixed-point (INT8) precision for fixed-point inference onwards. Experiments reveal outstanding efficiency of the proposed method on different LICs, showing the closest coding performance to their floating-point counterparts. And, our method is a lightweight and plug-and-play approach without any need of model retraining which is attractive to practitioners. 

Under review as a conference paper at ICLR 2023
RATE-DISTORTION OPTIMIZED POST-TRAINING QUANTIZATION FOR LEARNED IMAGE COMPRESSION
Anonymous authors Paper under double-bling blind review
ABSTRACT
Quantizing floating-por paating-point neural network to its ixed-point representation cru- cial for Learned Image Compression (LIC) because it ensures the decoding consis- tency for interoperability and reduces space-time complexity for implementation. Existing solutions often have retrain the network for model quantization which is time consuming and impractical. This work suggests the use of Post- Training Quantization (PTQ) to directly process pretrained, off-the-sheil LIC models. We theoretically prove that minimizing the mean squared error (MSE) in PTQ is sub- optimal for compression task and thus develop novel Rate-Distortion (R-D) Optimized PTQ (RDO-PTQ) to best retain the compression performance. Such RDO-PTQ just needs to compress few images (e.g., 10) to optimize the transfor- mation of weight, bias, and activation of underlying LIC model from its native 32-bit floating-pois (FP32) format to 8-bit fixed exed-point (INT8) precision for fixed- point inference onwards. Experiments reveal outstanding efficiency of the pro- posed method on different LICs, showing the closest coding performance to their floating-poin counterparts. And, method is lightweight and approach without any need of model retraining which is attractive to practitioners.
INTRODUCTION
Compressed images are used vastly in networked applications for efficient information sharing, which continuously drives the pursuit of better compression technologies for the past decades (Wal-
lace Sullivan et al. 2012: Bross et al.) (202T). Built upon the advances of deep neural networks
(DNNs), recent years have witnessed the explosive growth of image compression solutions (Ballé
et al. 2018 Minnen et al.) 2018 Chen et al. 2021 Cheng et al. 2020 Hu et al.) [021 Lu et al.
2022 with superior efficiency to well-known rules- based JPEG [Wallace 1992). HEVC Intra (BPG)
SSullvan et al. 2012), and even Versatile Video Coding Based Intra Profile (VVC Intra) (Bross et al.
2021).
Nevertheless, existing learned image compression (LIC) approaches typically adopt the floating- point format for data representation (e.g., weight, bias, activation), which not only consumes exces- sive amount of space-time complexity but also brings up the platform inconsistency and decoding failures (He et al., 2022). To tackle these for practical application, model quantization is usually applied to generate fixed-point (or integer) LICs (Ballé et al. 2018 2021.
Hong al. 2020 Sun al.
Popular Quantization-Aware Training (QAT) (Bhalgat et 2020 Le 2022 was mainly used in (Ballé et al.) 2018 Hong et al. 2020 Sun el al. 2020) 2021) to transform floating-poin LIC to its txed-point representation. Such methods requires model re-training with the full access of labels which is expensive and impractical.
Sun et al. 2021)
Recently, Post-Training Quantization (PTQ) (Nagel al.) 2020 2021) offered a lightweight and
blug-and-play solution to directly quantize pretrained, thf--the-sheff network models without requir- ing model retraining. However, such PTQ scheme was mostly dedicated for high-level vision tasks as studied in (Choukroun et al. 2019 Liu et al. 2021). This work therefore extends the use of PTQ to image compression model quantization.

---

Under review as a conference paper at ICLR 2023
We theoretically prove that only minimiz- ing the quantization error (e.g., MSE) PTQ for LIC model quantization vision tasks (Choukroun et al., 2019; Liu et al. 202T) is sub-optimal from the pression perspective because of localized non-monotonic relation between the quan- tization eπrоr and rate-distortion perfor- mance of the image compression task. We thus propose the Rate-Distortion Op- timized PTQ (RDO-PTQ) for LIC model quantization.
other
AE
9a
X
010010...
Py
AD
Os
Considering the optimization complexity, the RDO-PTQ is executed from one net- work layer to anther (e.g., layerwisely) to process weight, bias and activation in ei- ther convolutional or self-attention com- putation to respectively determine their proper ranges for quantization. In current implementation, just compresses a tiny calibration image set (c.g., less than 10 images) to optimize relevant quantization factors like range, offset, etc for fixed- point model.
Figure I: Learned Image Compression (LIC). Ia (gs) is main encoder (decoder); AE/AD arithmetic en- coding/decoding using Py ( Either convolution or self-attention is used to derive Y of input X. Model quantization Q is applied at every layer (convolutional self-attention) to transform weight w. bias b and ac- tivation æ in native FP32 to INT8 precision.
Given that the distribution of both weight and activation varies across channels at each network layer, the range is adapted channel-wisely besides the layer-wise adaptation (see 33: Besides the range determination of the bias, a bias rescaling is applied to ensure the computation using INT8 data tensor strictly.
Contribution. 1) We suggest the use of PTQ to quantize LIC model for lightweight, plug-and- play solution by just compressing fewer image samples to derive the fixed-point model without any model retraining; 2) Both rate and distortion metrics are optimized jointly compression task in- ference stage to determine proper ranges of weight, bias and activation for quantization proposed RDO-PTQ; 3) Our method is generalized to variety of LIC models, demonstrating the closest compression efficiency between native FP32 and corresponding quantized INT8 model.
RELATED WORK
Learned Image Compression (LIC). As shown in Fig. popular LICs mainly built upon the Variational Auto-Encoder (VAE) architecture to find rate-distortion optimized compact representa- tion of input image. In Ballé et al. (2018), on top of the GDN (Generalized Divisive Normalization) based nonlinear transform. a hyper prior modeled by factorized distribution was introduced to bet- ter capture the distribution latent features. Shortly, the use of joint hyper prior and autoregressive neighbors for entropy context modeling was developed in Minnen et al. (2018). demonstrating better efficiency than the BPG (e.g., a HEVC Intra implementation).
Later then, stacked convolution with simple ReLU was used in (Cheng et al. 2020 Chen et al., 2021) to replace GDN and the attention mechanism was augmented for better information embed- ding, which, for the first time, outperformed the VVC Intra. Recalling that the principle behind the image coding is to find content-dependent models (c.g., transform, statistical distribution) for more compact representation, apparently, solutions simply stacking convolutions ciently characterizing the content dynamics because of the fixed receptive field and fixed network parameters of trained convolutional neural network (CNN). To enable the content-adaptive dy- namic embedding, self-attention mechanism was extended in (Qian et al.) 2021 Lu et al.] [002]] Lu & Ma 2022). As extensively studied in (Lu et al. 2022 Lu & Ma 2022
not capable of effi-
integrated convolution
For simplicity, we refer the "network layer" to the "layer".
2

---

Under review as a conference paper at ICLR 2023
and self-attention could not only provide performance improvements to the VVC intra various datasets, but also reduces the space-time complexity significantly.
As shown subsequently, three popular open-resource LICs are used to exemplify the efficiency of our RDO-PTQ. including the Minnen2018 (Minnen et al., 2018), Cheng2020 (Cheng et al., (2000) and Lu2022 (Lu et al. 2022).
LIC Model Quantization. Although learning-bass based solutions remarkably improved the perfor- mance for various tasks, their native floating-pon paating--oitt representation incurred serial problems for practical enabling, including the excessive amount of space-time complexity consumption, nonde- terministic inconsistency across heterogeneous platforms, etc. These issues more severe for image compression task because the guarantee of interoperability across a variety of devices is the vital functionality of LIC solution. As reported. small numerical rounding-off error in floating- point computation may lead to decoding failures or incorrect reconstructions (Ballé et al. 2018).
pioneer exploration was made Ballé et al. (2018) to train an integer LIC to resolve platform inconsistency and decoding failures. Subsequently, the quantization of convolutional weights was specifically treated with in Sun al. (2020) 202T). Almost at the same time, Hong et al. (Hong et al.) 2020) suggested layer-wise range-adaptive quantization (RAQ) for both weights and bias, and linear scaling for feature activation. Recently, [Le al. (2022) quantized the transformer- based learned codec and realized the real-time decoding on mobile device. Note that these methods require the full access of labels for model retraining when performing the quantization, which is impractical and expensive some extent (Nagel et al. 2021).
Recently, PTQ attracts intensive attentions (Nagel et al. 2021) because is a push-buttorn approach without model retraining, which is presumably applicable to any off the-shelf, pretrained neural networks. Unfortunately, a majority of PTO studies were still devoted to high-level vision tasks as in (Chourroun et al. 2019 Nagel al. 2020 Liu et al. (021), where, more importantly, they mainly minimized quantization induced MSE (Mean Square Error). Given the advantages of PTQ, this work extends the image compression task; Since image compression pursues the optimal rate- distortion performance, a novel rate-distortion optimized PTQ is developed to fulfill the purpose.
3 QUANTIZATION FUNDAMENTAL
For an arbitrary floating-poin value flont (e.g., weight, bias activation of either convolutional or self-attention layer), it quantized using:
Fint = clip(1 Flont
+ Zn, 220 + 2, 1),
8x
where |-] rounds the input to its nearest integer. 2n is the offset and s, is the linear scaling factor.
Each Tint is mapped to a fixed-point å for inference (Hong et al. 2020) or simulating the effect of integer quantization avoid gradient vanishing (Dai al. 2021 training. Thus, the mapping is often formulated as
8z Fint Zzr-
Such mapping is also used in our RDO-PTQ for optimizing quantization factors by examining the compression performance of a tiny set of images in task inference stage to derive the corresponding fixed-point model.
As pointed out in Nagel al. (2021). z typically makes integer computation more complicated. Therefore, practical accelerators usually apply the symmetric quantization assuming ž, = 0. this way, we only need to define the scale factor 8 and bit width precision be. In general, 8 can be predefined, and s is deduced using
To
[(
2b-1 my. b,,bb))
(3)
where yn represent the dynamic range of ny is reserved number of decimal digits and r(-) is defined below
T(s; ny, b²) clip(s 2"r -2 11 20 1) x
In practice, these functions can be implemented using bit shift operation.
3

---

Under review as a conference paper at ICLR 2023
4 RATE-DISTORTION OPTIMIZED PTQ
4.1 FROM QUANTIZATION ERROR TO R-D Loss
To derive proper s. for model quantization in Eq. (3) existing works mostly minimize the square error between vectorized elements æ FP32 and quantized ã in INT8, e.g.
arg min a 21
(5)
The use Eq. (5) generally assumes the monotonic relation between quantization enror induced dis- tortion D (e.g.. MSE) and compression efficiency measured by the R-D metric J = R++ XD (Davis-) son 1972 Apparently, such assumption does not hold because of inevitable rate contribution to J.
Theoretical Justification. Assuming a trained compression task model J (æ, w) with floating-point activation and weight w, its fixed-point model used for inference can be formulated as J(x + Ax, w + Aw). Aw and Ax are quantization noises for w and æ respectively. To simplify the deduction, the bias as studied in Botev et al. (2017).
neglected since it can be absorbed into by appending unit term to
Then the model quantization induced performance loss of underlying task model is: AJ = E [[(x + Ar, w + Aw) J(x w)]
[[
OJ OJ + |A Aw] Hæ,
1
Aa
E
(6)
dx dw
Aw
where Ha,0 is the Hessian matrix, and high-order terms are ignored for simple derivation. For converged model, its gradient is close to 0, c.g., 1 1 == yielding
Ax
Aæ Aw
AJ
H H
Aw
Subsequently, Eq. 7 can be further expanded
1
20PJ
02.J
(8) As seen, the R-D loss not only depends on the quantization error, e.g. Ax and Aw?, but also related the second-order derivatives of J. Particularly, even having all positive second-order derivatives, the different sign of Ax and Aw would lead to negative cross term AwAæ, suggesting that a larger absolute error of quantization may not lead to a larger R-D loss. Similar observation was also discussed in Nagel et al. (020)).
2 - Dx Ax + fxow ArAu +
Aw3J.
Ow
As toy example, suppose symmetric Hessian matrix as
1
0.5
Ha
(9)
0.5
Then, the R-D loss can be rewritten as
AJ x E AxAw Aw"]
2 A
(10)
If we happen to have a case with a larger quantization error, c.g., [Ax Aw] = [0.4, -0.4]. and the other scenario with a smaller quantization error [0.3, 0.3], the R-D loss 0.08 and 0.135 respectively. As seen, larger quantization eтror gives smaller R-D loss.
Localized Non-monotonic Behavior. the meantime, we visualize the absolute quantization error of weight, e.g., Aw with compression task loss measured by the AJ, which further confirms the theoretical justification aforementioned.. As seen in Fig. 2 although over wide range of Awr it is monotonically related with the R-D loss AJ in compression task; it presents localized non- monotonic behavior where the minimization of quantization error does not lead to the minimum of AJ, which, consequently, degrades the overall compression performance of underlying LIC model we pursue the minimization of quantization error for model quantization.
As result, to best retain the identical compression efficiency of quantized LIC as its floating-pons counterpart, we propose to optimize the PTQ from the rate-distortion perspective for underlying compression task.
4

---

Under review as a conference paper at ICLR 2023
80-
0.05
60-
100
0.02
40
50
20-
0.01
00.02
0.00
(a) weight distribution
0.008
0.000
0.015
15
10
5-
0.2
0.4
0.8
1.0
1.2
Aw
-1
0
0.2
0.1
0.0
0.1
(b) activation distribution
Figure 2: Compress Task Loss vs. Quanti- zation Error. Localized non-monotonic be- havior is presented where the minimization of quantization error Aw does not lead to the minimal loss of rate-distortion metric (AJ).
Figure Exemplified weight and activation distribution of two channels for given layer. Other layers exhibit similar distribution from channel to another.
4.2
OPTIMIZED SCALING
As for the compression task, the optimization target in PTQ to minimize the R-D loss by consid- ering the distortion D and bite rate R jointly (Davisson 1972). For typical VAE structure with conditional context modeling, its R-D loss is
J= J- Ry Rg D(X, + E loga (Y12)) + E[- log2 El
(11)
where R- Ry + Rg is the total bit rate consumed by the latent feature Y and hyper prior 2, and D is the distortion between the original X and its reconstruction X, which typically computed using MSE or MS-SSIM. Besides, Py- and P are the probability distribution of respective Y and Z for entropy coding. We adapt Lagrange multiplier À for various bit rates as they do in (Minnen al. 2018 Cheng et al. 2020
et
2022p.
For any pretrained floating-poin LIC model Jo. the examination of the proposed RDO-PTQ is defined below derive proper scaling factor for the quantization of activation, weight and bias, e.g.,
8pg San, 86 arg min || Jall2 arg min [I X- D) - (Ro + A Do)11
(12)
990 98
It is impractical to optimize all layers of LIC model once because the accumulation of quanti- zation errors from layer to layer (e.g., forward and backprop) makes extremely difficult for such a great amount of parameters (e.g., weight, bias, activation) converge fastly and reliably. We there- fore suggest to apply the layer-wise quantization to progressively optimize the parameters. Usually, parameters former layer are fixed for the optimization of successive layers. As for the parameter quantization at (-th layer, all parameters from the very first to (l - 1)-th layer are already quantized and fixed for optimization; while floating-pont parameters from (1 1)-th layer the last layer are kept without change. Thus, Eq.
12) is reformulated as:
/ ur s/ arg min [a i.. b²) Jo(æ w/, ,')b 16.6
(13)
where / A(w 1 0 with A(-) as a nonlinear activation function like ReLU. Besides, inspired by AdaQuant (Hubara et al. 2020 AdaRound (Nagel et al.) (020) and BRECQ et 2020), continuous variable 2t added the weight parameter for better rounding.
w Q(w +
(14)
where Q(-) is the quantization function. Note that the same operation for the weight is applied to the bias well. Further details provided in the supplementary material.

---

Under review as a conference paper at ICLR 2023
4.3 DYNAMIC RANGE DETERMINATION
In Eq. (3), when the bit width precision is given, scaling factor, e.g., Snn syg or s' is only related to the dynamic range r' of corresponding data tensor, e.g., w', b', or æ'. For the uniform quantization used mostly, the derivation of scaling factor is equivalent the determination of the dynamic range. In general, weight, bias and activation assume the similar Gaussian distribution (Hong et al. 2020), and their ranges are approximated below.
Weight. As shown in Fig. weight distribution not only varies from one layer to another of a given LIC model, but also from one channel to another given layer. This suggests us to model the range of weight tensor using
N1.k max
(15)
with (kk))) directly computed from k-th channel weights 1-th layer. As seen, method mainly attempts to determine the proper Ni.k channel-wisely for weight quantization. Although Sun al. (Sun al., 202I) did the similar channel-wise grouping to clip weights for fixed-point pro- cessing, model retraining was required for finetuning, which largely differs from our PTQ solution.
Activation. Feature activations closely related the original image input. However, Hong et al. (Hong et al., 2020) enforced a fixed dynamic range per layer, e.g., [μ 14-30 30, + 30] to normalize the activation, which apparently sub-optimal without thoroughly considering the dynamics of input content. After carefully examining the activation distribution, the channel-wise variations in activation tensor are also considered, leading to
L.K 2
No. k maxr (1).
(16)
Bias. First, the range of bias is approximated layerwisely using
r No- maz((b')),
(17)
determine the scaling factor sf, using Eq. (
Typically, bias term augmented with the product of weight and activation to output feature activa- tion of current (-th layer as the input next (1+ 1) layer. Even though we enforce INT8 precision for all data tensor, in practice, a 32-bit accumulator is often used to host intermediate data @! avoid potentially data overflow. This suggests:
and to
& it x² b'.
(18)
Here, we wish to simply scale 32-bit Lnn to have 8-bit +++1 to input the (1 + 1)-th layer.
By combining Eq. long we have
s/ blnel.
(19)
s/ otz
we then arrive at #nnnd s wint
s'y æ'ne + s/ bl,t = s 5/4 (wint aint bl
(20)
which shows that we can simply scale 32-bit Linnd to derive 8-bit input o i.e., 1+1
at
in Eq. (19) which would not affect the task performance due to rounding operation. We call the operation defined in Eq. (19 bias rescaling which is used to enforce strict INT8 computation layer by layer.
Often time,
On the contrary, many existing works either simply assume the INT32 precision, or brutally enforce zeros, for bias. Such inappropriate processing of bias may lead to catastrophic results. For example, having the bias INT32 precision may cause data overflow of the INT32 accumulator; while setting bias as zero would degrade the model performance significantly. More details can be found in supplementary materials.

---

Under review as a conference paper at ICLR 2023
Algorithm RRDO-PQQ for Learned Image Compression
Input: Floating-poin model; Calibration image set
Output: The quantized model; Optimized scaling factors
for in { do
2: repeat
3: repeat
4:
Initial s.., o
5:
Quantize 1-th layer and forward propagation
6:
Update , No. by SGD (Hinton et al. 2012 Kingma & Ba 2014
7:
until s. s/ arg ( X D) Jull
8:
repeat
9:
Initial 6'
10:
Forward propagation
Update NZL by SGD
12:
until s/.x = arg min [ + A- D) - Joll?
until Convergence or excess limitation
14:
end for
4.4 SUMMARY
The overall RDO-PTQ process is summarized Algorithm ! We optimize the model from one network layer to another where dynamic range and scale factor for weight and bias are processed first, and then the activation. When the rate-distortion loss reaches its minimum, the optimization for 1-th layer is completed and all settings are fixed for the optimization of (I + 1)-th layer. Such process continues until the last layer of the model. Note that except for the bias that we apply the layer-wise processing with rescaling, we perform channel-wise quantization for both weight and activation parameters.
EXPERIMENTS
Extensive experiments are conducted report the efficiency of the proposed RDO-PTQ for LIC model quantization.
5.1 COMPARISON SETUP
Pretrained FP32 LICs, We choose three popular LIC models in their native FP32 format for evalu- ation, namely Minnen2018 (Minnen al. 2018), Cheng2020 (Cheng al. 2020 and Lu2022 (Lu et al. 2022). For instance, Minnen2018 is a seminal work first introducing the joint exploration of hyperprior and autoregressive neighbors for entropy coding of latent features; and Cheng2020 is one of several works which first proposed to apply attention mechanism to better aggregate in- formation. Both Minnen2018 and Cheng2020 rely on stacked convolutions while Lu2022 one of several earlier attempts that uses both convolution and self-attention. Subsequent results report the generalization of our RDO-PTQ these representative models regardless of its key unit (e.g., convolution or self-attention, ReLU GDN, etc), which is attractive to practical applications.
Pretrained models are directly obtained from their open resource website 2 respectively. Models used in comparison are trained using MSE loss for distortion measurement in R-D metric. Results for MS-SSIM loss trained models are provided supplemental material. For each model, six bitrates are experimented by setting six different λs, e.g., {0.0018, 0.0035, 0.0067, 0.013, 0.025, 0.0483) directly without any finetuning.
Alternative PTQs. Given that Lu2022 currently demonstrates the leading compression efficiency, we mainly use it as the baseline to implement other model quantization schemes. Unfortunately, to the best of our knowledge, are not aware of any open-resource PTQ method specifically for LIC models. For fair comparison, we implement the Range-Adaptive Quantization (RAQ) (Hong et al.
httpas///iithbb..omm/mmm http://iiiaaiiionnnnooeeeeeee
7

---

Under review as a conference paper at ICLR 2023
Table 1: BD-rate loss over floating-poimt models
Model
Lu2022
Cheng2020
Minnen2018
Method
Ours
FQ-VIT
RAQ
Ours
RAQ
Ours
RAQ
Kodak
3.70%
7.06%
29.40%
4.88%
27.84%
5.84%
30.41% %
Tecnick
6.21 %
12.13%
31.04%
6.86%
29.95%
8.23%
31.55%
2020)
originally requiring model retraining as PTQ approach; On the other hand, we also include FQ-ViT ViT (Lin al. [022) for comparative study. is a PTQ method originally designed for image classification and objective detection using Transformer backbone. Here extend its main idea to support the compression task. Besides, we compared MSE optimization and two methods (Sun et al. 2020 2021) with retraining in the ablation experiment, and the more details are provided in the supplementary material.
Testing Datasets. Two popular datasets that contain diverse images are used for evaluation, i.e., the Kodak dataset with the image size of 768 x 512, Tecnick dataset with the image size of 1200 x 1 1200. The Peak Signal-to-Noise Ratio (PSNR) measures the image quality and bits per pixel (bpp) reports the consumption of compressed bitrate.
5.2 EVALUATION
Quantitative Performance. We plot R-D curves in Fig. 8 of supplementary material for various floating-point LIC models and their quantized INT8 counterparts, and further report the BD-rate performance (Bjontegaard] (00T) over 32-bit floating-point models for different PTQ approaches in Table In the meantime, compression efficiency using both VVC Intra (VTM) and HEVC Intra (BPG) are also exemplified to help understand the relative performance gaps between quantized LIC and traditional image coder.
As seen, quantized INT8 LICs using our RDO-PTQ provide the least BD-rate loss to corresponding floating-point LIC models, greatly outperforming other PTQ alternatives (see averaged results in Table and curves in Fig. 8). More importantly, having the anchor of Lu2022 [FP32], Lu2022 [INT8] using our RDO-PTQ provides similar performance as the VVC Intra on Kodak dataset, and largely outperforms it on Tecnick dataset.
As visualized Fig. 8, the BD-rate loss enlarges at higher bitrates for the proposed RDO-PTQ on various floating-poin LICs. One potential reason is the increase of channels for high bitrate LIC models (c.g., from 192 to 320). Note that we optimize the PTQ channel -iisel,, accumulated quantization error is typical larger for the model with more channels. Such problem can be possibly resolved by optimizing the PTQ for all layers at once which however may need excessive amount of computations. This is interesting topic for further study.
We notice that Sun et al. (Sun et al. 2022 also introduced PTQ method to quantize LICs recently. They demonstrated that the influence of quantization error the final reconstruction was different across channels. As a result, they manually categorized channels whose quantization may lead larger reconstruction error into different groups. Although they included the reconstruction error instead of the quantization error into the discussion, they did not connect reconstruction performance with the weight, bias, and activation following the R-D optimization means, which is very different from our proposed RDO-PTQ. Unfortunately, we do not find any publicly accessible material for us to perform the comprehensive comparison.
5.3 ABLATION STUDIES
Dynamic Range. Previous methods usually tend to clip large parameter values to control the dy- namic range, while our proposed RDO-PTQ, by contrast, optimizes the dynamics of tensor for better R-D performance. Without any publicly accessible resources, we manually reproduced the
two SOTA methods (Sun et al. 2020 2021) for comparison. More details of the experiment can be found in the supplementary materials. Fig shows the result of weight only quantization. Fig. [ shows the result of quantizing weight and activation.
8

---

Under review as a conference paper at ICLR 2023
37
36
PSNR (8R)
35
34
Lu2022 [FP32]
Lu2022
Ours [INTS]
Ours (INT8)
53
MSE [INTM]
[INTS]
(2021) [INTS]
52
[[NTS]
sun al. (20201 [NNN
sun al. (2020) (NN15)
0.3
0.4
0.3
0.7
0.8
Bibenti (hpp)
Bilente (hpp)
Figure 4: Weight only quantization.
Figure 5: Weight and activation quantization.
0.09
1.0
w/o rescaling [JNT32]
0.07
0.5
w/ bins rescaling [INT8]
0.06
0.00
0.01
0.02
0.03
0.04
0.05
10
Calibration set size
Figure 6: Effect of INT8 bias compared with INT32 bias.
Figure 7: Comparison of RD as calibration set size increases.
Noted that [Sun et al. (2000 202I) only quantize weights their work, as result, we add the quantization of activations channel-wisely for fair comparison. The BD-rate losses over Lu2022 are 4.43% and 6. 7.78% respectively which are higher than ours. Besides, the results illustrate the sub-optimality of MSE optimization.
Bias Rescaling. Taking INT32 bias without rescaling as the reference, we test the effect of bias rescaling Lu2022. The common Min-Max quantization is adopted without quantizing activation, which does not require the calibration set and does not introduce quantization error of activation, which generally simplifies the problem for discussion. We test performance Kodak dataset. As shown in Fig. 6. bias rescaling for fully 8-bit processing has no negative impact on performance.
The Size of Calibration Set. We randomly select images from ImageNet to formulate the calibra- tion set for quantization factor determination in our RDO-PTQ. As shown Fig. 7 when the size of calibration increases to 10, there is basically loss the R-D performance compared with the floating point model. Therefore, we adopt 10 images for calibration in this work.
CONCLUSION
This paper studied the PTQ directly quantize off-the-shelf, pretrained floating point image com- pression models for their fixed-poin counterparts, To retain the compression efficiency as the native floating-poiin LICs, we suggested the R-D optimized PTQ which was first justified theoretically and then proved experimentally. In RDO-PTQ, we determined the proper ranges of weight and activation channel-wisely from one layer to another, and layer-wise ranges of bias for subsequent rescaling, to which tiny set of images (e.g., about 10) were sufficient to calibrate these model quantization related factors. Results revealed the superior efficiency of the proposed method, presenting large performance improvements existing approaches. More importantly, any model retraining, and offered a push-bottom solution for all existing LICs. An interesting topic for further study is reduce the quantization loss high bitrates.
method did not require

---

Under review as a conference paper at ICLR 2023
REFERENCES
Johannes Ballé, Nick Johnston, and David Minnen. Integer networks for data compression with latent-variable models. International Conference on Learning Representations, 2018.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with scale hyperprior. In 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. Lsq+: Improving low-bit quantization through learnable offsets and better initialization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 696-697, 2020.
Gisle Bjontegaard. Calculation of average psnr differences between rd-curves. VCEG-M33, 2001.
Aleksandar Botev, Hippolyt Ritter, and David Barber. Practical gauss-newton optimisation for deep learning. International Conference on Machine Learning, pp. 557-565. PMLR, 2017.
Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens- Rainer Ohm. Overview of the versatile video coding (vve) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, 31(10):37363766 2021. doi: 10. N0.1109/TCSVT.22013110995
Tong Chen, Haojie Liu, Zhan Ma, Qiu Shen, Xun Cao, and Yao Wang. End-to-end learnt image compression via non-local attention optimization and improved context modeling. IEEE Trans- actions on Image Processing, 30:3179-3191, 2021.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the IЕEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7939-7948, 2020.
Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural net- works for efficient inference. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3009-3018, 2019. doi: 10. 1.11199C/..22009.000
Steve Dai, Rangha Venkatesan, Mark Ren, Brian Zimmer, William Dally, and Brucek Khailany. Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference. Proceedings of Machine Learning and Systems, 3:873-884, 2021.
Davisson. Rate distortion theory: mathematical basis for data compression. JEEE Transactions on Communications, 20(6):202- 1972. doi: 10.1 19110//T0M..197.1091111
Dailan He, Ziming Yang. Yuan Chen, Qi Zhang, Hongwei Qin, and Yan Wang. Post-training quan- tization for cross-platform learned image compression. arXiv preprint arXiv:2202.07513. 2022.
Geoffrey Hinton, Nitsh Srivastava, and Kevin Swersky. Neural networks for machine learning.
Coursera, video lectures, 264(1):21 (14(1)2114622552 2012.
Weixin Hong, Tong Chen, Ming Lu, Shiliang Pu, and Zhan Ma. Efficient neural image decoding via fixed-point inference. IEEE Transactions on Circuits and Systems for Video Technology, 31(9): 3618-3630 2020.
Yueyu Hu, Wenhan Yang. Zhan Ma, and Jiaying Liu. Learning end-to-end lossy image compression: A benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Improving post training neural quantization: Layer-wise calibration and integer programming. arXiv preprint arXiv:2006.105/1 2020.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.690, 2014.
10

---

Under review as a conference paper at ICLR 2023
Hoang Le, Liang Zhang, Amir Said, Guillaume Sautiere, Yang Yang, Pranav Shrestha, Fei Yin, Reza Pourreza, and Auke Wiggers. Mobilecodec: neural inter-frame video compression on mobile devices. In Proceedings of the 13th ACM Multimedia Systems Conference, pp. 324-330, 2022.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Brecq: Pushing the limit of post-training quantization by block reconstruction. In Interna- tional Conference on Learning Representations, 2020.
Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, and Shuchang Zhou. Fq-vit: Post-training quantiza- tion for fully quantized vision transformer. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCA1-22, pp. 1173-1179, 2022.
Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quanti- zation for vision transformer. Advances in Neural Information Processing Systems, 34:28092- 28103, 2021.
Ming Lu and Zhan Ma. High-efficiency lossy image coding through adaptive neighborhood infor- mation aggregation. arXiv preprint arXiv:2204. 1448, 2022.
Ming Lu, Peiyao Guo, Huiqing Shi, Chuntong Cao, and Zhan Ma. Transformer-based im- age compression. In 2022 Data Compression Conference (DCC), pp. 469-469, 2022. doi: 10.11099DCC2220.22222 00080.
David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for learned image compression. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kris- ten Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, 3-8 December 2018, Montréal, Canada, pp. 10794- 10803, 2018.
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. op down? adaptive rounding for post-training quantization. In International Conference on Machine Learming, pp. 7197-7206. PMLR, 2020.
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Baalen, and Tijmen Blankevoort. A white paper on neural network quantization. arXiv preprint arXiv:2106.08225. 2021.
Yichen Qian, Xiuyu Sun, Ming Lin, Zhiyu Tan, and Rong Jin. Entroformer: A transformer-bee entropy model for learned image compression. In International Conference Learning Repre- sentations, 2021.
Gary J. Sullivan, Bens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (heve) standard. IEEE Transactions Circuits and Systems for Video Technology, (2112):16991166 2012. doi: 10.1 1..109/TCSVT..212222111
Heming Sun, Zhengxue Cheng, Masaru Takeuchi, and Jiro Katto. End-to-end learned image com- pression with fixed point weight quantization. 2020 IEEE International Conference on Image Processing (ICIP), pp. 3359-3363. IEEE, 2020.
Heming Sun, Lu Yu, and Jiro Katto. Learned image compression with fixed-point arithmetic. In 2021 Picture Coding Symposium (PCS), pp. 1-5. IEEE, 2021.
Heming Sun, Lu Yu, and Jiro Katto. Q-lic: Quantizing learned image compression with channel splitting. arXiv preprint arXiv: /4510, 2022.
G.K. Wallace. The jpeg still picture compression standard. JEEE Transactions on Consumer Elec- tronics, 38(():xviii-xx) -xxivy, 1992. doi: 12502..
11


Reviews:
[{"review_id": "v1Q5lPf5D4", "reviewer": "Reviewer_6kPp", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a method for quantizing floating-point representation to fixed-point precision in a learned image compression framework. The main advantage of the proposed method is, that it can be added to pretrained off-the-shelf compression models, making it useful in practice. The paper adapts an existing post-training quantization method to image compression task by considering the rate-distortion during optimization instead of only mean squared error. Results are shown by adding the approach to 3 existing methods.", "summary_of_the_review": "The paper presents a simple, but apparently effective method, however the main weaknesses are in the experimental section and performance analysis. Without additional insights and description of the effect of the calibration dataset it is difficult to judge how useful the method is in practice.\n\n**Post-rebuttal**  \nThe additional experiments including re-training methods are appreciated. However, for me the paper remains a borderline submission. The method works in practice but the analytical analysis/insights are limited.", "strength_and_weaknesses": "*Strengths*\n\n- The loss due to the quantization is with the proposed method smaller than with other post-processing methods. \n- The method is simple and works better than other methods shown.\n- The motivation, to use the loss function during optimization, which is also used in evaluation, makes sense.\n\n*Weaknesses*\n- The main technical contribution of the work is considering the compression rate in addition to the MSE loss. However, the comparison to the baseline using only MSE is only shown in the supplement. In my opinion, this comparison and discussion should be part of the main paper.\n- Some details related to the experimental setup are missing (e.g., algorithm runtime, needed to get the quantized model and the optimized scaling factors, used hardware, calibration set selection).\n- The quantization parameters are obtained by training on a small calibration dataset. From the paper it is unclear how this set has been selected and how sensitive the results are to the selection, e.g., by measuring the variance of performance over different image sets. I am wondering whether the calibration set is dataset specific in the experiments, and if that is the case, how the performance would be with an unsuitable calibration set. This clarification is important to fulfill the claim, that is truly a plug-and-play method without model retraining.\n- For completeness it would be interesting to compare also to a method which retrains the compression model, as the paper claims to be more efficient in terms of training time and practicability, e.g., by reporting the time needed.\n- The algorithm needs to be run for every desired bit rate.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "pWF1d-3Y15", "reviewer": "Reviewer_8wCS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper applies PTQ in learned image compression. Two methods are exploited. One is AdaRound (ICML'20) for weight, and the other is a learnable dynamic range for both weight and activation.\nMy main concern is about the novelty of this paper. AdaRound was presented in ICML'20, and learnable dynamic (clipping) range has also been studied in quite a few of literatures such as PACT.\n\nPACT: Parameterized Clipping Activation for Quantized Neural Networks", "summary_of_the_review": "As described in the above weaknesses, the proposal looks like just a combination of some previous methods and some mathematical derivations are also quite close to the previous paper. Besides, the experimental results should be improved, there is no ablation study and the comparison with previous works can also be more comprehensive.", "strength_and_weaknesses": "Strength\n\nThe paper is one of very few works which studied the network quantization for learned codec.\n\nWeaknesses\n\n1. The novelty is really margin. AdaRound and dynamic range determination have been proposed in previous works. I wonder what is new when you utilize AdaRound in the specific task (i.e. learned codec)?\n2. The writing can be improved. In Section 4.1, the equations are quite similar with the ones in AdaRound (Section 2. Motivation). Section 4.2 looks like some common equations for rate-distortion loss. Section 4.3 just introduces the N (dynamic range scaling) for weight/bias/activation.\n3. The experiments are not enough. There is no ablation studies for the final results. Now that you have three methods in detail, AdaRound for weight, dynamic range for weight and dynamic range for activation, which is the individual contribution to the final result?\n4. There is no running time evaluation.\n5. This paper applied the network quantization into a transformer-based learned codec. There is a related work which also quantized the transformer-based learned codec (MobileCodec), it is better to cite the paper and make a comparison.\n\n    MMSys '22: MobileCodec: neural inter-frame video compression on mobile devices\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RCguxSoXuBd", "reviewer": "Reviewer_ra2Y", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the use of Post-Training Quantization (PTQ) to directly process pretrained, off-the-shelf Learned Image Compression  (LIC) models.  The proposed study proves theoretically that minimizing the mean squared error (MSE) in PTQ is suboptimal for compression task, and proposes a novel Rate-Distortion (R-D) Optimized PTQ (RDO-PTQ) to best retain the compression performance. ", "summary_of_the_review": "The paper is clear and well organized. The proposed method is well described and the experimental results have shown their effectiveness. The proposed contribution is interesting.", "strength_and_weaknesses": "Strength :\nThe paper is clear and well organized. The idea of the proposed contribution seems interesting. Indeed, the proposed method needs to compress few images to optimize the transformation of weight, bias, and activation of underlying LIC model from its native 32-bit floating-point (FP32) format to 8-bit fixed-point (INT8) precision for fixed-point inference onwards. In addition, the experimental results have shown the effectiveness of the proposed method.\n\nWeaknesses :\nOther state-of-the-art methods could be considered in the comparison.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KxR_xcYhEsz", "reviewer": "Reviewer_LeDd", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method for post-training quantization (PTQ) for learned image compression models. Quantization offers three main benefits: (1) fixed-point math is deterministic, which ensures reliable decoding (this is not guaranteed when different hardware is used to decode compressed representations using floating-point operations), (2) it reduces space complexity if int8 weights are used instead of float32, and (3) it reduces time complexity since typical parallel hardware can perform int8 operations more quickly than float32 or float16. Furthermore, PTQ occurs after training, as the name implies, and does NOT require additional fine-tuning to achieve good rate-distortion performance relative to the original float32 model. This simplifies application and reduces total training time.\n\nThe authors show, both theoretically and empirically, that selecting quantization parameters that minimize mean-squared error (mse) to the float parameters does NOT maximize rate-distortion performance. To address this, they propose rate-distortion optimization (RDO) for the quantization parameters and describe an algorithm for doing this in detail.\n\nThe empirical evaluation shows a relatively small drop in RD performance using RDO-PTQ. In particular, the drop is smaller (i.e. the resulting RD performance is better) than two previous methods (range-adaptive quantization (RAQ) and FQ-ViT). The authors run the evaluation using two image datasets and three different image compression models.\n\n", "summary_of_the_review": "This paper addresses a practical problem for learned image compression, and the proposed algorithm outperforms the two baseline methods included in the empirical evaluation. The method, RDO-PTQ, is well-motivated and has many nice properties: it is generally applicable, relatively simple, does not require additional fine-tuning, and leads to only a relatively small drop in rate-distortion performance.\n\nAs discussed earlier, understanding the potential benefit of fine-tuning would strengthen the paper. I also think the outcome is more or less expected (RDO should always outperform a proxy loss for compression models unless it can be shown that the proxy is identical), and the applied nature of the result is interesting to learned image compression practitioners but is not a great fit for ICLR in general.", "strength_and_weaknesses": "Strengths:\n1. The paper addresses an important, practical challenge for learned image compression: decode reliability and reducing space/time complexity.\n2. The method is general (can be applied to any model) and appears to outperform other PTQ approaches.\n3. The evaluation uses multiple datasets and multiple models, which minimizes concerns that it may not be widely applicable in practice. Also, one of the models is quite recent (Lu 2022).\n\nWeaknesses:\n1. Although post-training quantization is convenient, fine-tuning is not a major barrier. So additional evaluation showing the benefit (or lack thereof) of fine-tuning would strengthen the paper. In practical scenarios, additional training time is negligible compared to the lifetime of the codec and potential RD benefits.\n2. The central insight is not very surprising or novel. Although RDO-PTQ outperforms the baseline methods, the basic expectation in compression is that RDO should always be performed and will always be best when it is possible/practical to apply.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "70N6eGLKQx", "reviewer": "Reviewer_AqaT", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper suggests the use of post-training quantization to quantize neural image compression models, where the model parameters are compressed from Float32 format to Int8 precision. The weight, bias and activation parameters are processed with post-training optimization. Some theoretical justifications are provided to prove that the quantization error is not monotonically related to the rate-distortion performance. The experimental results show the proposed model quantization method can convert the network precision with only 3.70% BD-rate increases. ", "summary_of_the_review": "The PTQ method has been applied on other tasks for many times. This paper simply applies the PTQ method into learned image compression models. The theoretical analyses in this paper also follow previous papers without new conclusions.", "strength_and_weaknesses": "Strength:\n1.\tThe proposed post-training quantization (PTQ) method can transform the model from Float32 format to Int8 precision without large-scale model retraining.\n2.\tSome theoretical discussions are provided to illustrate the motivation of the proposed method.\n3.\tThe experimental results demonstrate the effectiveness of the PTQ method in neural compression models, i.e., tolerable performance drop but much more light-weighted neural network.\nWeakness:\n1.\tThe only difference between using PTQ in LIC models and conventional neural networks is the format of task loss. This paper simply applies the PTQ method for the task of LIC, thus makes somewhat trivial technical contributions. \n2.\tThe theoretical derivations and the relevant conclusions in this paper have been given in the paper of AdaRound [1].\n3.\tThe setting of this paper is to transform the neural network from FP32 to INT8. But actually this setting is defined by the author without any explanations. Some other choices can be adopted, e.g., INT4 or unsigned INT16.\n4.\tThere are some vague claims without evidence, such as, ‘Such inappropriate processing of bias may lead to catastrophic results. For example, having the bias in INT32 precision may cause data overflow of the INT32 accumulator; while setting bias as zero would degrade the model performance significantly.’ \nReferences:\n[1] Up or down? adaptive rounding for post-training quantization. Nagel et al., ICML 2020.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "2eQv-N6W5i", "reviewer": "Reviewer_soh4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes a framework for applying off-the-shelf post-training quantization (PTQ) methods for learned image compression (LIC) task. The authors show that the optimum quantization scaling factor may not be discovered by only minimizing the MSE loss for LIC task and provide theoretical insights on this. To alleviate this, they propose to use a rate-distortion (RD) loss as their objective function where both the distortion and bit-rate are coupled by a Lagrange multiplier. Their experiments on 2 benchmark datasets show that INT8 quantized LIC (using their PTQ method) can achieve similar RD performance compared to the original 32-bit counterparts while outperforming the other quantization baselines. ", "summary_of_the_review": "Overall, due to the limited novelty, lack of proper discussion about the related works, lack of proper experiments and justification on why R-D loss is a better objective function compared to the MSE loss, I can not recommend for acceptance. However, I am willing to change my opinion if the authors can provide solid answers to my above concerns. ", "strength_and_weaknesses": "Pros:\nThe idea of utilizing the RD term as an objective function to find the optimum scaling factor in the context of LIC task seems novel and the main contribution of the paper. \n\nCons:\nI have several concerns about the papers as outlined below:\n\nNovelty and Prior-works: The authors claim that applying PTQ for LIC task is one of their main contributions. However, there have been several prior works that have already applied PTQ on LIC tasks. See [1-4]. The authors refer [1], and [2] in the introduction section as a quantization aware training (QAT) method, which I believe, is not a correct statement. Rather these methods are PTQ followed by fine-tuning (which is very common in computer vision tasks, specially for bit-width lower than 8), similar to [4]. Other than that, the work in [3] have directly applied existing PTQ method on LIC task without requiring any fine-tuning, similar to this work. Although the authors refer [3] in the introduction section as a part of their motivation (cross-platform inconsistency issue of the floating-point LIC methods), they do not discuss how the proposed work is different from theirs. I believe the main claim of this work should be 'PTQ on LIC without requiring any fine-tuning' (if we do not consider [3], of course).\n\nSignificance: The authors claim that the quantization error and compression metric do not have any monotonic relation (which is the main motivation behind their proposed loss). However, from Fig. 2, this claim seems to be valid only at the edge of minimum R-D loss(i.e., $\\Delta$J ~= 0) while they follow a monotonic relationship everywhere else. There was no theoretical justification or any insightful discussion behind such observation. This also questions the significance of using the R-D loss as the objective function rather than simple MSE loss as the authors in [3] showed pretty impressive results on the same benchmark (Fig 5(a) in [3]) with MSE loss only.\n\nExperiments: Comparison with other PTQ methods are missing. While [3] and [4] might be considered as pretty recent works, at least comparison with [1] and [2] is desirable. Also, in the ablation study, there could be an additional experiment (a graph similar to Fig 4) that clearly shows the performance gain of R-D loss over MSE loss where everything else (except the objective function) is same for fair comparison. \n\n1. Sun, Heming, et al. \"End-to-end learned image compression with fixed point weight quantization.\" 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020.\n2. Sun, Heming, Lu Yu, and Jiro Katto. \"Learned Image Compression with Fixed-point Arithmetic.\" 2021 Picture Coding Symposium (PCS). IEEE, 2021.\n3. He, Dailan, et al. \"Post-Training Quantization for Cross-Platform Learned Image Compression.\" arXiv preprint arXiv:2202.07513 (2022).\n4. Sun, Heming, Lu Yu, and Jiro Katto. \"Q-LIC: Quantizing Learned Image Compression with Channel Splitting.\" arXiv preprint arXiv:2205.14510 (2022). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject