Post-Training Quantization Is All You Need to Perform Cross-Platform Learned Image Compression
Authors: Dailan He
Ziming Yang
Yan Wang
Yuan Chen
Qi Zhang
Hongwei Qin
Venue: ICLR 2022 Submitted (2022)

Abstract: It has been witnessed that learned image compression has outperformed conventional image coding techniques and tends to be practical in industrial applications. One of the most critical issues preventing it from being practical is the non-deterministic calculation, which makes the probability prediction cross-platform inconsistent and frustrates successful decoding. We propose to solve this problem by introducing well-developed post-training quantization and making the model inference integer-arithmetic-only, which is much simpler than presently existing training and fine-tuning based approaches yet still keeps the superior rate-distortion performance of learned image compression. Based on that, we further improve the discretization of the entropy parameters and extend the deterministic inference to fit Gaussian mixture models. With our proposed methods, the current state-of-the-art image compression models can infer in a cross-platform consistent manner, which makes the further development and practice of learned image compression more promising.

Under review as a conference paper at ICLR 2022
POST-TRAINING QUANTIZATION Is ALL YOU NEED TO PERFORM CROSS-PLATFORM LEARNED IMAGE COMPRESSION
Anonymous authors Paper under double-bling review
ABSTRACT
It has been witnessed that learned image compression has outperformed conven- tional image coding techniques and tends be practical industrial applications. One of the most critical issues that need to be considered is the non-deterministic calculation, which makes the probability prediction cross-platform inconsistent and frustrates successful decoding. We propose solve this problem by intro- ducing well-developed post-training quantization and making model inference sateger-arihhmetic-onlg, which is much simpler than presently existing training and fine-tuning based approaches yet still keeps the superior rate-distortion per- formance of learned image compression. Based that, further improve the discretization of the entropy parameters and extend the deterministic inference fit Gaussian mixture models. With our proposed methods, the current state-of- the-art image compression models can infer in a cross-platform consistent manner, which makes the further development and practice of learned image compression more promising.
INTRODUCTION
(a) original image
(b) failed decoding
Figure The cross- gross-platform inconsistency caused by non-deterministic model inference. This inconsistency is catastrophe to establish ceneral-purposs image compression systems, and it is almost inevitable when practicing learned image compression with floating -oiiin arithmetic.
In recent years, learned data compression techniques have attracted lot of attention and achieved remarkable progress (Ballé et al. 2017 2018; Minnen et al. 2018 Cheng et al.) [2020] He et al.)
2021 Guo et al., 2021). Recently, few learned lossy image compression approaches (Minnen et al. 2018 Cheng et al. 2020) He et al. 2021 Guo et al.] (021) have outperformed BPG (Bel- lard 2015)) and the intra-frame coding of VVC (vtm 2020), which are the state-of-the-an manually designed image compression algorithms. As lossy image compression is one of the most fundamen- tal techniques of visual data encoding, these results promise possibility of transmitting and storing images or frames lower bit rate, which benefit almost all industrial applications dealing with visual data. Therefore, it is highly important to study how to make these rapidly developing bamiing-bssed methods practical.

---

Under review as a conference paper at ICLR 2022
cod- ing scheme proposed in [Bllé et al. (2018). The model will firstly transform the image to a major representation and minor representation ž. Then ý will be encoded by an arithmetic coder (Ris-) sanen & Langdon 1981 Martin ) with independently saved å as side-information. Later this method is further delved ved by introducing hackward adaption with context modeling (Minnen al.) He ct al. [202]] Guo et al., 2021). However, this line of models suffer from a non-determinism issue [Ballé et al. 2019), which makes the image hard to be decoded when the encoder and the de- coder run on heterogeneous platforms. The non-determinism is caused by floating-pone arithmetic, which can let the calculation results, from the same input operands but on different software or hardware platforms, different. This issue makes calculating prior from side-information and context model pross-platform inconsistent so that the decoding fails (See Figure mission is a vital requirement of establishing practical image coding systems, this inconsistency is critical.
Most currently state-of-the-ar learned image compression approaches adopt
2018
[]. As cross-pla pross--plaffom trans-
To make the computation deterministic for cross-platform consistency, Ballé al. 2019 proposes to train an integer network specially designed for learned image compression, where the inference stage is nntege--rithmetiioonly. It performs well on earlier proposed models like Ballé et al. (018)). After determinizing [Ballé et al. (2018) with integer network, the compression performance has a marginal reduction compared with the floating-point version. However, we find that on more com- plex models with context modeling like Minnen et al.] (2018) and Cheng et al.] (2020), adopting this integer network approach cannot keep the performance loss negligible. Another previous work (Sun) iet al. (202T)) proposes to quantize model parameters and activation to fix fix-point, enabling cross- platform decoding of the mean-scale hyperprior-only network in Minnen et al. (2018) without hurt- ing compression performance. However, the determinism of state-of-ther oof-thh-art joint autoregressive and hyperprior methods (Minnen et al., 2018 Cheng al. 20200 He et al.] [202]] Guo et al.) (202T) are still not considered. As the slow serial decoding problem of autoregressive context model has been addressed in He et al. (202T). this joint modeling architecture is very promising and valuable for practical application and its cross- pross-platform decoding issue needs to be solved.
We notice that, these existing approaches are similar to general model quantization techniques in spirit, L.e. quantization-awwr
training Jacob et al. (2018); 8); Krishnamoorthi 2018); Esser et al.
(2019: ; |halgat et al. (2020)) and post-training quantization (PTQ, Nagel et al. (2019) 2020); Li
et al. 20200). Instead of proposing another approach to provide
ad-hoc solution, we advocate to solve this cross-pla pross--patffom consistency issue based on those well-developed model quantization techniques in a more flexible and extendable manner. Thus, we investigate cross-platform consistent
inference with PTQ. Usually, it requires much less calibration data (Nagel et al.] 2020; 2021 Li
al. 2020) and costs much less time to quantize the model than QAT, yet for several computer vision tasks it can achieve almost the same accuracy as QAT when the target bit-width is 8 (Nagel ct al. 2019 2020 Li et al. 2020). It does not demand re-training or fine-tuning the trained floating- point models, which is very friendly to industrial deployment. By applying intege--rithmmetioonll operators (Jacob et al., 2018 Zhao et al. 2020: Yao et al. (02T) after quantization, we can achieve deterministic models which get rid of the inconsistency issue.
In this paper, we contribute to the community from following perspectives:
1. We evaluate and prove that, the deterministic computing issue of learned data compression can be reduced to a general model quantization problem. After applying standard post- training quantization (PTQ) technique, we obtain cross-platform consistent image com- pression models with marginal compression performance loss.
2.
We successfully determinize presently ofate-of-the-ard learned image compression models with context modeling and Gaussian mixture models. To the best of our knowledge, this is the first work investigating and accessing cross-pland clos--lltform consistent inference on those models. As we use standard quantization techniques, it is also promising to adopt our method to determinize future compression models.
3. We propose a novel approach to discretize the entropy parameters, which can be computed directly in deterministic manner. Compared with the existing method based on searching algorithm, our method significantly speeds up the parameter discretization and eliminates the bottleneck.
2

---

Under review as a conference paper at ICLR 2022
Hyper Analysis
y
Analysis
Quant
-Py 0
Context Model
AEIAD
AEIAD
Bitstream
Synthesis
y
Parameter
Network
Hyper Syrtteels
AD
-Py
Figure 2: Diagram of joint autoregressive and hyperprior architecture for learned image compres- sion (Mnnee et al. 2018 Cheng et al. (020). AE/AD denote arithmetic en/de-coder. 0 is the set of predicted entropy parameters (i.e. i. p. σ), modeling the distribution blue, red and orange arrows denote encoding, decoding and shared data flows, respectively. The highlighted orange networks are shared by both encoding and decoding to estimate code probability for AE/AD, demanding cross-platform consistency.
element-wisely The
We emphasize that goal is not propose method for quantizing neural networks, but to call attention to the relation between the deterministic issue and model quantization techniques. Benefiting from the maturely developed quantization techniques, show that the cross-platform consistency issue in learned image compression can be better solved.
PRELIMINARY
Notations.
We represent matrices with capital bold letters (e.g. W) and denote vectors as small bold letters (e.g. v). We use ceiling [-1 and floor |.] notations represent round-up and round-down operators, respectively. And the blended [.] denotes guud-to-nearess: By default, we denote the inputs and activated outputs of a linear layer as and respectively. And the activation function is represented by l(-). We use the capital B as the quantization bit-width, which by default is set to 8 as we prefer an 8-bit quantization. We scalar with subscript to denote the corresponding quantization step, e.g. Sy means the guantization step of v. We may introduce a bracketed superscript to distinguish variables different layers, e.g. (f) is the input of the L-th layer while (!+1) the input of the next layer. For simplicity, we use the notation clip (-) value clipping introduced by uniform affine quantization, omitting the clipping bounds. By default we quantize the tensors to B-bit signed integers, with the upper and lower clipping bounds 2 and -
Learned image compression and non-determinism issue.
In Appendix D we describe several popular deep-leaming-biie based image compression methods. Briefly speaking, almost all the cutting-edge learned image compression techniques adopt the joint autoregressive and hyperprior modeling illustrated in Figure To encode the major symbols ői an entropy model Pyla.y;<: (ūs: 2, f)<s) is adopted to predict the probability density of S condi- tioned on iideinformation à and already decoded symbols ÿ The predicted density functions will then be accumulated to obtain the cunulative distribution functions (CDF) and fed to arithmetic en/de-coders.
Generally, we input & to the hyper synthesizer to obtain the intermediate feature क, and use the context models to summarize the context representation V. Then, we use parameter network to calculate the element-wiss distribution parameters from &b and y to generate the predicted distri- butions. To ensure correct decompression, we should make sure the inference of bbove-mentioned models (hyper synthesizer, context models and parameter network) deterministic. It means that, with the same input å and yjeú, the predicted entropy parameters should always be the same no matter what platform the encoder or decoder is running on. The calculation should be platform-independent
3

---

Under review as a conference paper at ICLR 2022
and cross-plattonn consistent. However, if the models infer with floating- point numbers, the deter- minism is hard to satisfy.
Uniform afline guantization (UAQ). UAQ is widely adopted by model quantization rescarches (Ja- cob et al. 2018: Nagel et al. 2019 2020 Li et al. 2020). UAQ, both activation values and weights should be quantized to fixed point numbers allow the use of integer matrix multiplica- tions. Usually, UAQ maps floating-pon point values to B-bit integers. For a given vector v, UAQ with a quantization step Sy is formulated as:
Qv = clip
+
The term 2v is an integer representing the zero-point shifting the center of quantization range, which applied in asymmetric uniform affine quantization and omitted in symmetric ones. In recent QAT, the step 8v and zero-point Ev are usually learned during training/fie-tuningg (Esser et al.) 2019). And in PTQ they are normally determined by the value range of given weights activation vectors Nagel et al. 2019 2021).
Therefore, the floating point vector v is discretized an integer vector qv, where each integer value actually represents a fixed point value. We can remap qv to corresponding fix-point vector through dequantization:
= 9v Sv
(2)
When v is the activation output, the second term can be pre-computed and absorbed by the convo- lution bias, so will not introduce extra calculation inference. Therefore, it is recommended by Nagel et al. (22TT) to apply asymmetric quantization on activation and symmetric quantization on weights. Nagel al. (202T) also suggests to adopt a per-channel quantization on weights (Krish-
namoorthi 2018 et al. 2019), which we introduce in Appendix A.I
Integer-arithmeticoonly requantization. Several previous works on QAT for integer-arin arithmetic- only inference have been proposed (Jacob et al.] 2018 Zhao et al., 2020 Yao et al. (021). One of the most important techniques is the dyadic requantization, Between two convolution layers, the dequantized fixed point activation will get quantized again, this can be fused to one operation called requantization. A requantization after the L-th layer is:
(f+1) clip ([m()0) +21(
where ( the int32 activation accumulation of the l-th layer, 21 is the zero point, and m! is the requantization scale factor. Please refer to Appendix [A. detailed derivation of this equation.
Ideally, all floating-poit pponnt operator should be removed from model inference but m still floating- point, which makes the requantization non-deterministic. Therefore, a dyadic number mo is intro- duced in Jacob al. (208)) to approximate m:
me = [2
(4)
where my and are integers. Now we have = 2 Obviously, the approximation error becomes smaller when gets larger. With this approximation, the requantization described in eq. can be rewritten as:
(E) m qu
(t)
(E+1) clip
(0) = clip ([(m)
a ( 1/2m(0) + (e)
(5)
2
where the operator // is integer division with mouning-to--eaare (round-int-div, RID). And this RID calculation can be further implemented as bit shift operation with rounding-to-nut ne-areesst (round- int-shift, RIS). Since both mo and 32-bit activation q are integers now, the calculation of q (++1) requires only an integer multiplication, a RID or RIS and a clipping. Hence the requantization becomes anteger-arithmetie
3 INTEGER-ONLY PTQ FOR DETERMINISTIC CODING
Different from works on general model quantization whose major purpose to compress the model size and improve inference efficiency, we introduce quantization eliminate the inconsistent arith- metic. Thus, we should strictly prevent using Dlafform-specinn operators. So we restrict all inference calculations to 32-bit integer arithmetic, it is the most widely supported set of integer arithmetic by hardware devices. Particularly, leveraging execution efficiency and compatibility, we only con- sider adopting matrix multiplication on 8-bit integer operands to perform convolution.
4

---

Under review as a conference paper at ICLR 2022
Int8
Int32
Qw
4b
INtO
Conv MatMul
Accumulation
Ciiping
Requant
In12
my
Int2
qu
4v
Int8
Int32
Int8
Requant
Int8 to Int32
Int32 to Int8
(a) Integer-arillmeetic-onny inference
(b) Requantization
Figure 3: The
inference. An offline-constrained pnogger-rrthmmeti-onfj re- quantization will be adopted during inference, which will discuss in Section 3.2
3.1 PTQ BASELINE
We adopt relatively simple and standard quantization pipeline described in Nagel et al. (202 We apply symmetric per-channel quantization on weights and asymmetric per-tensor quantization activation except the output of the last layer in parameter net. Instead, symmetrically quantize the activation from the last layer to 16 bits, with fixed quantization step 2-6. This benefits the CDF indexing and calculation which will discuss later in Section 4.1 As recommended in Nagel al. (202I), we adopt grid search minimizing the reconstruction error to obtain the weight quantization step sw of each layer. We figure the activation quantization step sn with Min-Max method little calibration data. We detailedly describe the algorithms in Appendix C.3
After setting the quantizers, we apply a per-block (or per-network) adaptive rounding reconstruction (Brecq, [i et al. (2020)), though the quantized models without this reconstruction have already achieved acceptable rate-distortion performance (Appendix C.4
3.2
OFFLINE-CONSTRAINN ENTEGER-AITTMMETICCOOO -NLY REQUANTIZATION
Previous use of aateger-arithmett
requantization is based QAT (Jacob al. 2018 Yao
ct al. 2021 Zhao et al.) 2020). It is first proposed in Jacob et al. (2018) where my in eq is always integer bigger than 230 which should be represented with 32 bits. We find it risky, this 32- bit integer my will later multiply with the 32-bit integer accumulation. Assuming that no specific hardware support available, we tend to believe that the requantization should be implemented with standard 32-bit integer arithmetic (i.e. we cannot conduct the bit-shift on 64-bit multiplication result registers). Thus, the 32-bit integer multiplication may overflow and result in error we use unconstrained ma- Another issue is. in Jacob et al. (2018) the requantization scale factor m is empirically assumed as always less than 1, which is hard satisfy in PTQ.
To address this problem, we slightly modify the requantization by moving the zero-point adding operation eq. forward:
(8) ml())
(f+1) clip ((m(()
(1 + + clip (mm()())
where g0(() qu (6) + Pu
(6) is the 32- bit integer activation biased with the 32-bit pre-scaling zero-point
Pu By doing so, the asymmetric quantization of qu changes to the symmetric quantization of q'..
As the requantized value will be clipped to B-bit integer within range 1-2 2- 1). the
maximal and minimal valid value of q (f) are computable:
2B
-
B-1
(2)
(8)
Tmax
ml())
ml())
5

---

Under review as a conference paper at ICLR 2022
with these bounds, earlier conduct the clipping operation before the rescaling (see Figure in Appendix B.I restricting the biased activation values qu, in the range [qmin 9max] Thus, we can figure out the largest subject to b4n keeps precision as much as possible:
qu: -231 mohy < 231, which avoids overflow while
n = 32 B,
(2) mg = 2
m
(f)
Please refer Appendix [B. for detailed explanation of eq. Therefore, after setting the quantiz- ers, we offline calculate the dyadic numbers and replace the requantization operations layer by layer. Empirically, the proposed pileger-arithmetic-only requantization with constrained magnitude of mo still keeps the model performance, though it introduces slightly more numerical error than the orig- inal version proposed Jacob et (2018). We can safely adopt it across platforms with standard 32-bir bit integer multiplications and RISs/RIDs. Figure 33 (right) shows this offline-conss ceftineconsttained form of requantization.
FROM 16-BIT OUTPUT TO DISCRETIZED CUMULATIVE DISTRIBUTION
4.1 BINARY LOGARITHM STD DISCRETIZATION FOR DETERMINISTIC ENTROPY MODELING
The network-output parameters should be discretized, so that the CDFs can be stored as limited look-up-tabel tableslles (LUTss. as discussed in Appendix D.2] In previously proposed approaches (Ballé al. 2019: Sun al. 2021), the standard deviations (STD) are logarithmically discretized with computing natural logarithm log(:):
i log(o) llo((mmi)))
(10)
Ag
which is hard to determinize. Sun et al. (021) obtains is from parameter network output in deterministic way by comparing the fix-point a value with pre-computed sampling points ô. This is moderately efficient there are 64 levels of â for each predicted STD to compare. We detailedly introduce this discretization approach in Appendix D.3 (eq. and the following paragraph).
Empirically, the STD is long-tail distributed most of the latents predicted to have entropy close to zero. So the logarithmic discretization of a is reasonable to suppress error and has been proved effective. To keep this favor and be more hardware-friendee Hence, we modify eq. [0 eq. 51 in Appendix D.3 to:
we propose to use binary logarithm.
1
loga mmax
A
1
Ømin
2 [log2(0) -log2(0 (
A A-1
(11)
(exp2 (1))
= Tiiu
where exp2(x) = 2 denotes the power of Ballé et al. (2019) and Sun et al. (202 adopt the same bounds Omin 0.11 and Fmax 256 with level L 64. To omit netedetermmnism while to simplify the calculation, we instead adopt mmin = 0.125, max = 32 and L = 9. Therefore, we obtain simplified formula with A and log2(amin) long-tail distribution, using a smaller upper bound Tmax is painless. Adopting less levels, however, enlarges error because the sampling points get sparser. Thus, we uniformly interpolate additional minor levels between each two adjacent major levels of &. Considering the input a is the dequantized fix- fix-point value with corresponding quantized integer and scale 2-6 (i.e. sq), the updated formula is:
-3. Since the STD has an extremely
[ogz(4)] + log (s) +3
j q exp2 ( [og (1)gg(((())) (4)l)
exp2 (llog, (q) - 3)
(12)
i, = si +
eep2 + jexp2 (i (i-3) (3)
④ = Tmin

---

Under review as a conference paper at ICLR 2022
32
minor major
16
16
24
32 Index
40
48
56
64
Figure 4: Visualization of the proposed 65-1evel STD parameter discretization. The orange dots denote logrithiaaa distributed major levels and the blue ones are linearly interpolated minor levels.
where log2(s) = 6 is constant integer. E: 10.1..... 8) and j {0, 10.... 8) are the major and minor indexes, respectively. Note that i = 8 iff. 2 when = 0. Thus, the summary index
to {0, 1.... 64} corresponds to 65 values of o, which generates 65 CDFs to be stored as LUTs. We further describe the derivation of this discretization in Appendix B.2 Figure shows the value distribution of this discretization.
Thus, the calculation of i, reduces to calculating the round-down integer binary logarithm of 16-bit integer q. To compute the like BSR on x86 or CL2 on ARM. can also use a platform-indepe ingeem-dddeeeddent bit-wise algorithm to count the leading zeros to obtain result of integer binary logarithm, which is easy to vectorize. The Algorithm in Appendix E describes the detailed process converting the 16-bit output to the indexes.
of an integer, we can adopt platform-specife instructions
4.2 DETERMINISTIC ENTROPY CODING WITH GAUSSIAN MIXTURE MODEL
For models using GMM with & Gaussian components, we can calculate CDF of the i-th component with given μi, and d; and accumulate with multiplier n; The predicted multiplier is dequantized 16-bit fix-poin number 8
C (0: Σ 9n, Cy (0;
(13)
where 8x is the quantization scale of 9x,- We can omit s in eq. 13 because it is constant scalar normalizer and can be merged into the normalization factor of frequency based CDF table. The remaining part only involves integer arithmetic.
To obtain the CDF the i-th Gaussian component Cg (úr: (4, dz), we can query two-level pre- computed LUTs with combined i, (eq. 12 and in (eq. 53 in Appendix D.3) as outer index and ÿ - [ eq. 122 as inner index. However, indexing the inner level LUT with y [ua] suffers from risk of out-of-bound error. We cannot directly restrict the range of ij to [[m]] - [4]] R] like Sun al. (202T) because multiple Gaussian components are involved. 7m, subject to 14-14 > R. the direct restriction may result in significant error. Instead,
set the cumulative distribution value to and CDF max when ÿ [M, < -R and 9 - [u,] Z R respectively. We restrict the symbol ý bigger than (max [μ]) R where the CDF value is CDF max- Also we restrict it larger than situations the symbol ÿ is out of these upper and lower bounds, we will adopt Golomb coding encode ÿ. inspired by
(min [₂]) - R where the CDF value is zero.
implementation! This situation is quite rare as its corresponding probability mass is close to zero, and adopting Golomb coding will not damage the overall bit rate. Thus, the CDF of GMM in eq. [3] is still monotonically increasing which can be reversed with searching algorithms during decoding.
With pre-computed CDFs of single mu-scale Gaussian entropy model. probability of ú in each Gaussian component can be checked from shared LUTs. Then the aggregated CDF value of 9 can
oangaa/ssooooaaaroooooooooooa.ar fow

---

Under review as a conference paper at ICLR 2022
42-
Minnnn at (2010)
30-
82-
BPG (6-64)
Cheeg at (2322)
Zhang etal (SDRS) PTO
Unibim Affin PTO (oung)
et al (2018)
sannen et (2058) PTO
Sun et al, (2021)
Ballo et (2018)
Burld ot (2013) wt PTO
Sun st at, (2021) Par-laaa
WGVVTM
BPQ (6-4-))
28-
Colld et (2010)
0.2
24
0.5
10 0.00
0.28
0.50
0.75
1,00
1.25
1.50
-36
2.00
Bio Per Pheal
Biou Pert Post
Quantized models
full-precision models.
Comparison with prior works.
RD curves evaluated Kodak. (a) RD performance of different models w/ w/o quan- tization. The solid lines with cross markers denote original model inference with floating point numbers. The dots represent quantized 8-bit antggr--rithmmeticonly models, sharing the same color as its floating point version. (b) We test Sun et al. (2021) by quantizing the activations in both per-channel and per-tensor manner. We also test Ballé et (2019).
Figure 5:
be calculated with given 4u. In Appendix coding for reference. Algorithm describes the whole encoding process and Algorithm describes the decoding.
1
we provide pseudo-code of this GMM involving
EXPERIMENTS
We train various presently state-of-thh-arr learned image compression architectures on floating point obtain the full-precision models. After the training,
PTQ algorithms quantize sub- networks involved in the entropy estimation to 8-bit, as above-mentioned. Then we replace all requantization and Leaky ReLU (Appendix (A.3) with corresponding deterministic adaptions and insert additional layers for LUT-index calculation. The detailed experiment setting is described in Appendix C
5.1 COMPRESSION PERFORMANCE
Method
ours per-tenso 0.35
Sun et al. 2222
Act. Quant. BD-Rate (%)
per-channel
per- per-tensor
1.22
4.04
Table 1: BD-rates over full- full-precision model. The RD data is the same as that in Figure 5(b)
We quantize sub-networks of Ballé al. (2018). Minnen et al. (2018). and Cheng et al. (2020) involving entropy prediction to 8-bit and compare their rate-distortion (RD) performance with orig- inal full-precision version, shown in Figure ((a) The results indicate that, existing learned im- age compression techniques are compatible with standard PTQ. All tested models can infer with anteger--rrthmeett
computations with negligible reduction on RD performance, which guaran- tees painless cross-platform consistency. It is not necessary any more for us to particularly develop new training techniques nor network structures to address the inconsistency issue.
We compare our approach with existing ones (Sun et al. 2021 Ballé al. 2019). We present the RD results in Figure 5(b) and further report the corresponding BD-rates (Bjontegaard (200I) in approach or (Sun et al. (202l) on Minnen al. (2018) model, the RD performance marginally deteriorates. However, Sun et al. (2021) adopts a per-channel activation quantization which is unfriendly to hardware implementation and rarely supported (Nagel et al.) 2021). When adopting Sun et al.) (202I) with per-tensor activation quantization, the performance, especially higher bit-rates, gets hurt significantly. We have successfully reproduced the good
Table I When adopting
8

---

Under review as a conference paper at ICLR 2022
performance reported in Ballé et al. (2019) for determinizing Ballé et al. (2018) (not shown here), although we try hard, we find this method cannot keep marginal performance deterioration for de- terminizing context-modellinvee model-involeed architectures like Minnen al. (2018). Note that [Bllé et al. 2019) is dedicated QAT method, we find it hard to train when applied on Minnen et al. (2018).
5.2
RATE OF DECODING ERROR
PTQ
w/o PTQ (FP32)
w/ PTQ (Int8)
Encoding Platform
GPU
CPU
GPU
CPU
Error Rate on Kodak
12/24 (50.0%)
12/24 (50.0%)
0/24 (0.0%)
0/24 (0.0%)
Error Rate Tecnick
72/100 (72.0%)
84/100 (84.0%)
0/100 (0.0%)
0/100 (0.0%)
2018 when inference with or without PTQ. The results are tested on NVIDIA GTX 1060 (marked as GPU) and Intel Core 17-7700 (marked as CPU). The decoding is cross-evaluated the two platforms, iee. encoding one and decoding the other.
Table 2:
Decoding error rates of Minnen et al.
Following Ballé et al. 2019) and [Snn et al. (2021), report the rate of decoding error in Table for completeness, As the networks have been strictly restricted to only perform 8-bit and 32-bit integer arithmetic, the inference is strictly deterministic.
5.3 LATENCY OF BINARY LOGARITHM BASED STD DISCRETIZATION
Comparison (CompressAl)
Comparison (vectorized)
Calculation (ours)
Method
Hyper Synthesis
Latency Kodak
17.32
9.52
4.35
22.26
Latency on Tecnick
61.01
33.81
9.48
47.74
Table Discretization latency with different approaches (unit: microsecond).
Table
shows the inference latency when adopting comparison-based discretization used by Sun
al.
202T) and our proposed calculation-based discretization. All the results are tested on NVIDIA
GTX
1060.
To evaluate the comparison-bast based approach, we refer to popular CompressAl (Bégaint
2020 implementation and also test another more efficient vectorized algorithm (described in Appendix E.3). The inference latency of hyper synthesis is also reported as reference. The results on Kodak (resolution: 512 768 px) prove that the directly calculated binary logarithm is more efficient, strongly suppressing the speed bottleneck of STD discretization. And the results on larger 1200 x 1200 px Tecnick images indicate that this improvement can more significant when compressing high-resolution images.
al.
DISCUSSION
The mature investigation on general model quantization provides free lunch for establish- ing a cross-platform consistent entropy estimation approach, which essential to practical learned image compression. In this paper, we experimentally prove that the non-consistency issue of state- of-the-art learned image compression architectures can reduce to an integer-arithmeticoon) model quantization problem. With standard PTQ scheme, we achieve deterministic compression models which have almost the same compression performance as their rrre-uuattzzed full-precision versions. This result is encouraging. Furthermore, we improve the parameter discretization and extend it fit GMM entropy model. In the future, we will further delve into practical learned image compression by extending the rate-distortion tradeoff to rate-distortion-speed tradeoff.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Versatile video coding reference software version 11.0 (vum-11.0) 2020. URL https: /vggit. hhi. www.nhofer..r..om /e/ / VTM/-/tags /VTM 11.0
Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution:
Dataset and study. 2017 IEEE Conference Computer Vision and Pattern Recognition Work- shops (CVPRW), pp. 1122-1131, 2017.
N. Asuni and A. Giachetti. Testimages: a large-scale archive for testing visual devices and basic image processing algorithms. In STAG: Smart Tools & Appsfor Graphics (2014), 2014.
Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compression. In Int. Conf. on Learning Representutions, 2017.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with scale hyperprior. In Inl. Conf. on Learning Representations, 2018.
Johannes Ballé, Nick Johnston, and David Minnen. Integer networks for data compression with latent-variable models. Int. Conf. on Learning Representations, 2019.
Ballé. Efficient nonlinear transforms for lossy image compression. In 2018 Picture Coding Symposium (PCS), pp. 248-2522 June 2018. doi: 10.1109/PC3.201884456777
Jean Bégaint, Fabien Racapé, Simon Feltman, and Akshay Pushparaja. Compressai: py- torch library and evaluation platform for end-to-end compression research. arXiv preprint
arXiv:2011.03022 2020.
Fabrice Bellard. Bpg image format, 2015. URL http:/pp://elllarr..rrr/////////
Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. Lsq+: Improving low-bit quantization through learnable offsets and better initialization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 696-697, 2020.
Gisle Bjontegaard. Calculation of average psnr differences between rd-curves. VCEG-M33, 2001.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. leee, 2009.
Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmen- dra S Modha. Learned step size quantization. In International Conference on Learning Repre- sentations, 2019.
V. K. Goyal. Theoretical foundations of transform coding. IEEE Signal Processing Magazine, 18 (5):9-21, Sep. 2001. ISSN 1558-0792. doi: 10. 109/799992800
Zongyu Guo, Zhizheng Zhang, Runsen Feng, and Zhibo Chen. Causal contextual prediction for learned image compression. IEEE Transactions on Circuits and Systems for Video Technology, 2021.
Dailan He, Yaoyan Zheng, Baocheng Sun, Yan Wang, and Hongwei Qin. Checkerboard context model for efficient learned image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14771-14780, 2021.
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arinn arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2704-2713, 2018
10

---

Under review as a conference paper at ICLR 2022
Eastman Kodak. Kodak lossless true color image suite (photocd ped0992), 1993. URL http:
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.0852 2018.
Rundong Li, Yan Wang. Feng Liang, Hongwei Qin, Junjie Yan, and Rui Fan. Fully quantized network for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2810-2819, 2019.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Breeq: Pushing the limit of post-training quantization by block reconstruction. In Interna- tional Conference on Learning Representations, 2020.
G Martin. Range encoding: an algorithm for removing redundancy from digitised message. In Video and Data Recording Conference, Southampton, 1979. pp. 24-27, 1979.
David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for learned image compression. In Advances in Neural Information Processing Systems, pp- 10771-10780, 2018.
Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. Data-free quantization through weight equalization and bias correction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1325-1334,2011
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort, Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learming. pp. 7197-7206. PMLR, 2020.
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization. arXiv preprint 200.:22006.02255, 2021.
Jorma Rissanen and Glen Langdon. Universal modeling and coding. IEEE Transactions Infor- mation Theory, 27(1):12-23, 1981.
Heming Sun, Lu Yu, and Jiro Katto. Learned image compression with fixed-point arithmetic. In 2021 Picture Coding Symposium (PCS), pp. 1-5. IEEE, 2021.
Zhou Wang, Eero Simoncelli,, and Alan C Bovik. Multiscale structural similarity for image quality assessment. In The Tereth-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, volume 2, pp. 1398-1402. Icce, 2003.
Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang, Qi- jing Huang, Yida Wang, Michael Mahoney, al. Hawq-v3: Dyadic neural network quantization. In International Conference on Machine Learning, pp. 11875-11886. PMLR, 2021.
Hengrui Zhao, Dong Liu, and Houqiang Li. Efficient infeger-arihhmeticoonly convolutional neural networks, arXiv preprint arXiv:2006. / 1735, 2020.

---

Under review as a conference paper at ICLR 2022
A DETAILED DESCRIBTION OF PTQ
A.1 PER-CHANNEL. WEIGHT QUANTIZATION
Weights of convolution filters may have various value ranges. Because of this imbalance, using the same quantization scale Sw for all filters may cause large rounding error in filters with small ranges and clipping error in filters with large ranges. Therefore, per-channel weight quantization is proposed (Krishnamoorthi 2018 Li et al., 2019). To per-channel quantize a weight tensor W = [Wi, Wa....; Wo] with e filter channels, we introduce scales Sw.....p Sw. and separately quantize each filter:
W; Sw. qw i= 1.
(14)
where the scale factors Sw; can be merged into later requantization operation.
Note that, we do not recommend to conduct this per-channel quantization on activation, as it will introduce extra calculation to rescale the multiplication results before convolution accumulation. Therefore, we only per-channel quantize the weights while keep activation per-tensor quantized, as Nagel et (2021) suggests.
A.2 REQUANTIZATION
Consider a convolutional or fully connected layer with input v, weights W and activation function h(-). The output activated vector is: h (WV)
(15)
Provided that we have quantized the weights and imput vector with scale factors and respec- tively, the corresponding quantized integer representation Qw and qv are:
Qw = clip ([(sw
(16)
qv = clip ([(,,'v))
(17)
and the corresponding dequantization results
W sw Qw
(18)
9v
(19)
Thus, the activation ú calculated from quantized weights and input is:
ú = h( WV) = h.sws, Qwqv)
(20)
Usually, we adopt ReLU or Leaky ReI.U as the activation h(.). They are segmented linear functions with scaling invariance, i.e. scaling the input of ht-) with a factor is equivalence to scaling the output with
h(v) s h(vv
(21)
Therefore, we can move the scalar swav eq. 20 outside the activation function:
á sw h (Qwqv)
(22)
Now Qw is integer matrix multiplication, outputting a 32-bit integer vector. Also, the activa- tion function h(-)) can be formulated as imtege-ariitmmeti-onnl. Thus, the activation l(QQqu)) is a 32-bit integer vector, named qu by us. Now ú is a 32-bit fix-point number with integer representa- tion qu and quantization scale factor swsv:
qu = hwQQqv)
(23)
Notice that, after each layer, the output should be quantized to 8 bits again, or the matrix multipli- cation in the next layer cannot be conducted using 8-bit integer arithmetic. Therefore, we quantize (0+1
( with scale factor 8y (note that (f+1) = (())
q
(
(++)
=lip
(quantize ((+1)
((+1)
Sv
(e+1) (())) )) (v(ctt)=((()) i(())
=clip
(24)
(8)
sw
=clip
sv (e+1) (
(eq. 23
12

---

Under review as a conference paper at ICLR 2022
Following Jacob al. (2018), fuse the division of scale factors m, which is called the requan- tization scale factor:
m
(25)
Introducing m to eq.
24 finally we obtain the requantization formula:
q.- (++1) = clip ([m² q!!¹)
(26)
A.3 FOLDING LEAKY RELU TO CONDITIONED DYADIC REQUANTIZATION
Leaky ReLU is frequently adopted in presently state-of-the of-ee---tterttt compression models. For instance, Minnen al. (2018) introduces it in hyper analyzer and synthesizer, Cheng et al. (2020) uses it as activation functions of the parameter network. Different from ReLU which can be simply seen as a conditioned clipping operator which is deterministic, Leaky ReLU involves floating point multipli-
cation.
Considering Leaky ReLU with negative slope applied each quantized scalar du qu. pro- vided the requantization factor is m, the requantization result yv is:
clip
(gu))
(8)
+
Pu
(6)
2 0
(++1)
(27)
clip
omm
10
(m)
Pa
0
where qu signed integer and checking sign painless. For non-negative and negative cu- mulative values, the above operation can be separately performed as two branches of requantization with different factors m and am. Hence the Leaky ReLUU can be fused into the requantization in vectorized manner. We implement the fused activation layer as conditioned dyadic multiplication, with scaling factor for non-negative Zu and am for negative fu.
B EQUATION DERIVATIONS
B. DERIVATIONS OF EQUATION
q' 10
scaling dioping
4, (1-1)
q'u
10
dipping scaling
9. (++3)
max
Vtnx
127
Gaaz
valid
clipping
disping ranga
-220
Gain
-129
min c
Figure 6: Comparison between cealing-clipping requantization and clipping-scall requantization. By computing the bounds of valid range, we can conduct the clipping operation before scaling with factor m.
Revisit eq.
8 in the main body:
2
(8)
mmxx
=
(f)
- B--1
(8)
Amin
ml()
The bounds qmax qmin describe the valid value range the activation q', biased by the zero-poin Pu- After the clipping operation, all of the clipped value q", will be in this range:
q' a" = clip (q' n. qmin 4max)
(28)
s.l. by E q".. - B- < mqmin S mq S mqmax < 2
13

---

Under review as a conference paper at ICLR 2022
To perform dyadic requantization, want to find out proper integers ma, subject to
Vg E eq -2 <
mow
22B-1-1
(29)
2
We let my become function of n:
mo = 12
(30)
Notice that my is non-negative and n, m is positive. For each 9 q" n.
mog
Maggax
m
Umax
I [mmgmax]]
S [2B-1 -16-1-1 11]=28-14 -1
2
2
2
(31)
mog
Moqmin
1
Umin
mmmmin]
2 2 -2B11
2
2
2
which satisfies the condition in eq. 29
Therefore, we only need to determine proper value n following two conditions:
1. The multiplication overflow from myg",. u should be avoided. As my is a function driven by n now, this condition implicitly define upper bound of n.
2. n should be set as large as possible, to suppress the rounding error introduced by requanti- zation.
To avoid the 32-bit multiplication overflow, constrain the magnitude of moq" subject to:
Vg E q" -231 I mog S 231
(32)
Introducing mo
m]. obtain the conditions:
12 m_ quax S <231_
(33)
m2"m] Imin
-231
(34)
Further introducing the definition of 4max and gmin in eq. 8 we have:
231
1
231
2 <
Jmax
2
Y70
(35)
231
211111
231
-231
231
2 ml <
Jwin
2-1
28-1
(36)
231
20-1
Since m is positive, we have m S 2mm and the condition let subject to:
231
291
2
Hence we relax
28
231
2nm m S
(37)
2
We get:
2"
< 2 32-B
(38)
so that we let = 32 B. It is almost the largest value n subject to the condition in eq. 29 and
eq. 32 avoiding the multiplication moq" overflow.
14

---

Under review as a conference paper at ICLR 2022
100%
64-level natural (Ballé et al. 2019)
75%
0%
65-level binary (ours)
50%
25%
0%
24
40
48
56
Index
Figure 7: Comparison of our binary logarithm discretization and natural logarithm discretization proposed in inBallé al. (2019). The Y-axis is the percent of discretized standard deviation introduced for meaningful comparison: o 0% and max 32 for ours.
X 100% with max 256 for the natural logarithm one
maax
B.2 DERIVATIONS OF EQUATION
12
The 65 level STD discretization is derived from linear interpolation. Revisit the definition of major levels in eq. 11 and consider sq:
[l)g((a] - 3
major Tmin (exp2(i))
Since has been clipped to [8, 2048] (corresponding to dequantized a E [0.125,32] with quantiza- 8} tion step 2 "), à {0, 8 and mmajor E (2-3,2-2 25..
The discretization will map all E [0.125, 0.25) 0.125 and [0.25, 0.5) to 0.25, which is sparse and will result in large error, significantly hurting the compression performance. Thus, we interpolate minor values between two adjacent major values i and i +. 1.
Let o; = mmin (e)p2(i)) denotes the STD corresponding to index i. And similarly let 6 corre- sponds i 1. We linearly insert minor levels in [ i 1] with the step size Aminori
6 ++1 0;
A minor
1
8 min (exp2(1 +1 exp2(i))
(39)
o min exp2(i)
and the minor index is:
3 = A A (o - 8:31
_ (xp2(-i)(o o:)
Bexp2( 3)1 -D)) eexp((i)
(40)
(q exp2(i + 3))
exp2(i)
where 3; < a < ا:a1 and a = Introduce the above equation, we have:
2 6 q. So i < [[gg(q)] - < i 1. thus, [log((g] - 3.
(g - exp2
i=
(41)
exp2([logg/b 3)
Thus, the STD a will be discretized to:
à + +
(42)
And the interpolated levels are न; + A Amiorr 0 + minors ...,, , 4747mminos When å + 7 a < 0 + 8Aminor . the STD will be mapped to the next major level 0:41- When occurs, the corresponding minor index 8.
15

---

Under review as a conference paper at ICLR 2022
C EXPERIMENT: DETAILED SETTINGS AND MORE RESULTS
C.I TRAINING FLOATING-POINT MODELS
To evaluate the proposed PTQ-based scheme, we should first obtain the full-precision models trained on doating-point numbers. this section we will report our detailed training settings for repro- ducibility.
We draw 8000 images with largest resolution from ImageNet (Deng et al. 2009) to construct the training dataset. The training set is shared to train all reported models. Before training apply the same preprocessing as Ballé et al. (2017) on all data by downsamping and disturbing the input images. During training,
randomly crop each image to 256 x 256 px in each iteration. We adopt batch-size of 16 and a initial learning rate of le-4 all training. To optimize the parameters, we use Adam optimizer with 31 0.9, 82 0.999. To cover range of rate- rate-distortion tradeoffs,, 6 different values of À are chosen from {0.0016, 6006600000 0.0032, 0.0075, 0.015, 0.03, 0.45] for MST-optimization, fol- lowing previous works (Cheng et al. 2020 He et al. 202T). On all models, adopt spectral Adam optimization (Sadam, Ballé (2018) to improve the training stability. For each model architecture, we particularly adjust the training setting according to previous suggestions:
Ballé et al. (2018). For each we train the corresponding model 2000 epochs. As sug- gested by the authors, we set a so-called bottleneck with N = 128, M 192 when using the lower and N 192, M 320 when using the others.
Minnen et al. 2018 Following the suggestion from the authors We train 6000 epochs. We choose N and .M dependent on A, with N 128 and M 192 for the lower and N = 192 and M = 320 for the 3 higher ones. We decay the learning rate to 5e-5 after training 3000 epochs.
Cheng et al. 2020). As recommended by He et al. (2021), we train 6000 epochs to achieve 128 for the lower-rate models, and set as 192 for the 3 higher-rate models. We decay the learning rate to 5e-5 after training 3000 epochs.
well-optimized performance of Cheng et al. (2020). N
C.2 EVALUATION
Following prior works, we use Kodak (Kodak our test dataset. The results of image compression are shown as rate-distortion curves. Following previous works, we use PSNR and MS-SSIM (Wang et al.) (203)) the distortion metric to mea- sure the reconstruction quality. We calculate bits-per-pixel (BPP) as the rate metric to measure the compressed file size.
1993) and Tecnick (Asuni & Giachetti 2014 as
C.3 CALIBRATION
We use a calibration dataset to calculate the activation quantization steps. To avoid data leakage, DIV2K (Agustsson & Timofte (2017) as the calibration dataset. We center-crop each one of 900 images in DIV2K to 256 x 256 pixels and feed them into the quantization pipeline with a batch-size of 32.
Recommended by Nagel et al. 2021), we search the weight quantization step using a grid search method. Given full-precision weight vector w, which is represented by floating point, we minimize its reconstruction mean-square-error to find the optimal quantization step 8w:
arg min || w|l2
s.t. Sw {si, 82, .2...., 8N
(43)
where N is the volume of the search space and w is the reconstructed vector w which is quantized with factor Sw. This approach can be directly applied on each convolution filter w; conduct the per-channel weight quantization.
httpp///wwr....oo.o..ooooooooeer 26U/m/ cD4%./mJLAgAJ
https://groups.cr
16

---

Under review as a conference paper at ICLR 2022
38
Minnen al. (2018)
(2018)
36
Minnen al. (2018) PTO (AdaRound)
Minnen et al. (2018) wt PTQ (no recon.)
PSNR
32
28
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Bits Per Pixel
Figure 8: Comparison of quantized Minnen et al. (2018) w/ and w/o Brecq weight rounding recon- struction. The results are evaluated on Kodak.
To obtain the full-precision activation, feed batch of calibration data to the model before quanti- zation and store the floating-point activation outputs. And the range of saved full- pele-ppeeiioon activation will be used to obtain step Su and zero-point zn with Min-Max approach:
(max min u)
Su
2
(44)
Zu sn
min u
(45)
C.4 EFFECTS OF WEIGHT ROUNDING RECONSTRUCTION.
AdaRound (Nggel et al. 2020)
Breeg
Method
w/o reconstruction
Li ct al.
2020
Minnen et al. 2018
1.329%
0.741%
0.663%
Cheng et al.
2020
0.607%
0.415%
0.416%
Table 4: Relative BPP increment on Kodak when using (or not) different weight rounding recon- struction approaches, compared with full-precision model.
We evaluate quantization results with and without weight rounding reconstruction, i.e. AdaRound (Nagel et al. (020) Brecq (Li et al., 2020). Shown in Figure and Table 4 the direct quantization without per-layer reconstruction has achieved almost no performance loss at lower bit rates (BPP< 0.4). The rate increment at higher bit rates can be compensated by Breeq. Brecq is directly performed on weight offline, it is relatively costless and we adopt it to establish our PTQ baseline.
We implement Brecq according to the open- code provided by the authors. We view the hyper synthesizer, context model and parameter network as three blocks perform weight reconstruction, iee. we adopt the per-block reconstruction mentioned in Li et al. (2020). We use MSE as the object instead of FIM for simplicity. We batch-size of 32, and learning of 1e-3 with Adam to perform the gradient descent, tuning each block for 20000 iterations.
C.5 MORE RATE-DISTORTION RESULTS
For completeness, following existing work (Ballé et al. 2018 Minnen et al. 2018 Cheng et al. 2020 He et al. 2021), we further report RD performance evaluated on Teenick (Figure 10) and RD performance of models optimized on MS-SSIM (Figure 9 After the quantization, the models keep comparable performance.
17

---

Under review as a conference paper at ICLR 2022
20
18
MSSSIM dB
16
12
Cheng (2020) [opt.me-ssim]
Minnen at (2018) [spt.ms-ssim)
10
Chang (2020) [opt.ms-ssim] w/ PTQ
Minnen et al. (2018) [spt.me-esin] wi PTO
WCNTM
BPG (4:4:4)
0.1
0.2
0.3
0.4
0.5
0.6
Bits Per Pixel
Figure 9: Influence of quantization to models with MS-SSIM optimization. Evaluated on Kodak.
38
Chang (2020)
Minnen et el. (2018)
Cheng et al. (2020) wl PTQ
36
Minnen et al. (2018) w/ PTO
PSNR
34
30
Bils Per Pixel
Figure 10: The same models as Figure 5(a) evaluated on Tecnick.
Chang at (2020) in parallel
Minnen et al. (2016) paraliol
36
Cheng et (2020) in parallel, PTQ
Minnen
parallel,
PSNR
30
0.1
Bits Per Pixel
Figure 11: RD curves of models with parallel context models, quantized or Evaluated on Kodak.
18

---

Under review as a conference paper at ICLR 2022
C.6 ORTHOGONALITY WITH PARALLEL CONTEXT MODEL
issue of learned image compression with context models the inefficiency of serial decoding. He al. (2021) proposes to address this problem by developing checkerboard-shape context modeling scheme, instead the original serial autoregressive method. We also investigate the quantization Minnen al. (2018) and Cheng al. (2020) with this parallel context adaption. As shown Figure I1 PTQ still performs well without hurting the RD performance, making the learned image compression pragmatic.
parallel
D LEARNED IMAGE COMPRESSION
D.1 OVERVIEW
010
AC
%g
P(P)-MAL
014
01:
U1Q
96
71F
719
A1R
(a) Auto-encoder
(b) Hyperprior
(c) Joint autoregressive and hyperprior
Operation diagrams of different learned image compression architectures. (a) an auto- encoder like model (Ballé al., (017) (b) Scale hyperprior model. ha and h2 represent hyper analysis and synthesis while z denotes the hyperprior (Ballé et al.) (018). (c) Joint autoregressive and hyperprior model. denotes the context model (Minnen et al. 2018).
Figure 12:
As shown in Figure 12 a), the widely adopted framework of auto-encoder like learned image com- pression proposed by Ballé et al. (2017) is similar to transform coding (Goyal 2001). During en- coding, the input image x is mapped the latent representation y through parametric analysis transform ga (-). After the round-to-neares. quantization, the quantized ý is used as coding symbols, which will be encoded into the bitstream later. To losslessly encode ý into a short enough bitstream, an entropy model Py (ý) is introduced to fit the probability mass of y, so that we can entropy encoders. With the same entropy model and an entropy decoder, ý is decoded from the compressed data stream, and fed learned synthesis transform (-()) to produce the reconstructed image x.
To train the auto-encoder like model with stochastic gradient descent, a uniform noise estimator introduced to replace the rounding operation (whose gradient is zero almost everywhere) during training. It produces noisy vector ý = + where r ~ (-55, 0.5). to approximate the quantized fit a continuous distribution Py (y), which is modeled segmented x|x. y y and ž for simplicity.
during training. Thus, linear parametric model in Ballé et al. (001)). Hereinafter we always use x, y and z to represent
By supervising the entropy, the training of learned image compression model is formulated as rate- distortion optimization. The loss function can be expressed as:
C = R + A- D
(46)
E logappy ( + X- x)]
where À is the coefficient trade-off rate and distortion. d(x, denotes the distortion between the original image x and the reconstructed image & where mean squared error (MSE) is the most common choice.
19

---

Under review as a conference paper at ICLR 2022
Shown in Figure 12],.. in Ballé al. (2018), the entropy of is estimated using conditioned Gaussian scale model (GSM) by introducing hyper latent z as side information:
(0) = [N(0,, a;) U-00.5, 0.5)] (0))
f g
NOO, ((),
(47)
0.5
P91a II Py a (8.1z)
where z is the hyperprior calculated from y with a hyper analyzer (a(+)) and each o, is inferred from å by hyper synthesizer (-)-) By encoding and decoding 2 independently, this hyperprior is used as side-information to perform forward-adaptive coding, conditioning on which the entropy of each i can be better modeled.
Shown in Figure 12 c), in Minnen et al. 2018) aackward-adaption is also introduced, forming a joint autoregressive and hyperprior coding scheme. Provided that the currently en/de-coding symbol is D:, we can predict it with the already visible symbols ÿz<ı which have been en/de-coded before di- Further adopting mean-scale Gaussian entropy model which extends GSM, we can achieve more flexible entropy model:
Pala (1:l2,
[N of) (U.5 (0:)
+0.5
(48)
0.-0.5
where element-wise entropy parameters μ4 and σi are jointly predicted from context model, hyper synthesizer and parameter network as highlighted in Figure Cheng et al. (2020) further introduces Gaussian mixture model (GMM) with K components the entropy model:
K
0: +0.5
Pyl2,, (0,
Σ
T (())
(K) , 2(1) (x)dy
(49)
91 0.5
subject to Σx= n (k) = 1. The entropy parameters n. M4 a are still jointly predicted from context model, hyper synthesizer and parameter network. Recently, this GMM- optimization is further promoted by introducing global context modeling and grouped context modeling (Guo al. 202I), which outperforms VVC/VTM on both PSNR and MS-SSIM
D.2 ENTROPY CODING WITH ARITHMETIC CODERS AND THE INCONSISTENCY ISSUE
In learned image compression, we usually adopt arithmetic en/de-coder (AE/AD) or its variants, range coder (Martin) 1979) and asymmetric numeral systems (ANS), to compress/decompress the symbols ý to/from bitstream. Using AE as example, during encoding AE requires both ý and its cumulative distribution function (CDF) Cy as input. AE will allocate bits for each latent D; according to the CDF. The same CDF should be fed to AD during decoding to correctly decompress y. This is exactly what we are investigating in this work: making the CDF estimated by entropy model consistent across different platforms to ensure the correct decoding of entropy coding.
Calculating CDF relies on the output of parameter network, which is continuous Boating-point rep- resentation. Thus, the calculation of CDF is usually expensive and non-deterministic doating-point point arithmetic. For instance, CDF for a discrete mean-scale Gaussian distribution (Ballé et al. 2018) is:
Cglii,,)) = = Σ/
N (2, (0) + /
g+..5
0 1 0.5
(50)
y- 0-55
where ý is one element in the quantized integer latent representations ý (we omit the subscript for simplicity). (b-) is the CDF of standard Gaussian distribution and is often calculated from the stan- dard Gaussian probability density function ndt or the error function erfe which is often approx-
htt.e//ss/////bb////// blll/llll.... /
ent ee.ooyyy..ddddd//mrrroooooooooo.
20

---

Under review as a conference paper at ICLR 2022
imated by interpolation algorithms. Therefore, after quantizing the parameter network output: the mean μl and the standard deviation (STD) 0, the calculation of CDF is still non non-deterministic.
Existing implementations pre-compute and store limited number of CDFs into look-up-tablee (LUTs) to address this problem. By directly distributing the saved CDF tables across various plat- forms, the inconsistency in calculating CDF is eliminated. To conduct the LUT-bbaed CDF query, the mean and STD should be discretized to limited sets values } and {ō}. Otherwise, the non- constrained parameters will generate infinite CDFs, which cannot be stored. We will discuss this discretization in the following section, Appendix D.3
Encoding (Platform Alice)
ParamNet
0j+1
0z+2
Ao
Decoding (Platform Bob)
ParamNet
Ao
discretization
floating-point
discretized value
1-11
0i+1
0s+2
Figure 13:
When the cross-platform inconsistency occurs, the discretized å (and also ji) may become different between the sender Alice and the receiver Bob. Thus, the corresponding CDF index and i + differ, which ends with failed decoding.
When the calculation inconsistency occurs, the discretized parameters and their corresponding in- dexes may be different across encoder and decoder (Figure 13). When encoding symbol ý with the inferred STD a. for instance, Alice discretizes a to the i-th sampling point o,- Then Alice will the i-th CDF allocate bits for and encode it. After all the symbols have been encoded, Alice send the bitstream to Bob who will decode the symbols on another platform. Because of the non-deterninism, Bob infers the entropy parameter of ý Alice. Thus, the discretized parameter is ō,+1 with corresponding index i i 1. Then Bob tries to incorrectly decode ÿ with the (i + 1)-th CDF instead of the i-th. Because the arithmetic coding is of recursion form, the failed decoding of symbol ý results in failed decoding of all the following symbols. It corrupts the decoded image, as shown Figure]]
Ao with error Aa different from
D.3 PARAMETER DISCRETIZATION
CDF INDEXING
As mentioned in Appendix D.2 we should discretize the entropy parameters for LUT query. Ballé al. (219) proposes constrain each predicted values in limited discrete set:
A log( @ max) log minu
L 1
log(a) log(amin)
(51)
A
exp ii log(omin)
where the lower and upper bounds o min Fmax are constant values. The input or should get clipped into range [min max and the clipping omitted in above formula for simplicity. The discretizing
21

---

Under review as a conference paper at ICLR 2022
level L is set 64. Since each index in corresponds to particular & which will generate CDF, we need to obtain the indexes order to perform LUT-baased CDF query. models using integer networks proposed in Ballé al. (2019), the parameter network outputs the 6-bit integer index io directly. But in uniform quantization- based approaches, the parameter network instead outputs dequantized fix-point a. Thus, we need extra map the fix-bit output to the integer index of LUT. Sun et al. 2021) obtains is by comparing the fix-point value with pre-computed discretized ō.
The discretization of means has been addressed in Sun al. (2021), with observation that:
Cglús μ4 o) Cg [u] iu [u],
(52)
where μ [ E [0, ) is the decimal part of Jr. Thus, we can just uniformly discretize the decimal part of ja to M levels and store limited number of CDFs:
" [( [n])M i = i= + Luj
(53)
Therefore, we can obtain LM pairs of (i )g), corresponding to LM CDFs. For all CDFs, we pre-compute their function values the input range { R, - +1 ,, ..... R 1, R) and save them as LUTs with lengths of 2R + 2, where each table includes an extra element representing ending-of-bitstrean
E ALGORITHMS
E. PARAMETER DISCRETIZATION
The discretization method described in eq. 12 is used to convert the 16-bit network output to CDF indexes. We give its clement-wise description with pseudo-code in Algorithm [
Algorithm 1: Binary logarithm discretization with interpolation
Input: 16-bit integer
Output: Corresponding CDF index i
/* Clip with lower and upper bounds
*/
q < 8 then
L q==8
// amin 0.125 quantized by step size 2-6
else if > 2048 then
*Lq q: 2048
// 0max = 32 quantized by step size 2-6
s = Int.og2(())
i = - 3
é1 = << b
E2 = << 3)
j = (q - ei e1+62 1)/e2 // round-up integer-divisien by adding e2 -
10 1 = 8 x xi+j
E.2 DETERMINISTIC EN/DE-CODING WITH GAUSSIAN MIXTURE MODEL
For simplicity, in this section we use to represent the joint outer index T + i, for ken Gaussian component. We use CDF to denote the outer query by ig and C& [-] to denote the inner query by [ua]. The output CDF value cg in Algorithm is the frequency cumulate below ÿ.
The encoding is described in Algorithm 3 where Line 2 denotes symbol 인 bigger than smaller than (min [m]] R where the CDF value is zero. In this case, Line and 5 encode symbol - -R as a placeholder in AE AarttmmeticEne(symbbee
(max [n]) + It where the CDF value is CDF
power_cumulate, aoper_cummlate, state)), while Line 4 encodes ing Golomb coding (oooombennsyybboa state). The decoding process is given in Algorithm where the reverse process is performed.
22

---

Under review as a conference paper at ICLR 2022
E.3 OUR FASTER VECTORIZED IMPLEMENTATION OF COMPARISON BASED DISCRETIZATION
Existing implementation of comparison based discretization adopted by Sun et al. (2021) is for- loop manner, which is somewhat inefficient. For fair comparison, we further provide vectorized implementation. Its PyTorch code is like:
initialization
scale_table exp linspace (np. log (0.11), np. log (256), 64)) scale_table torch tensor (cale table [:-1]). cuda ()
scale scale-table [None. None, None, None, :]
running
sigma_expand sigma. unsqueeze(-1)
lut-index (sigma.expand scale.table sum (-1)
Algorithm 2: Deterministic GMM CDF indexing (CDFIndex)
Input: symbol ý. CDF index i f1...... ik round-down means [ ... [Px] quantized weights
4m ... 9TT Output: CDF value
Data: CDF LUTs CDF CDF(64]. upper bound of frequency cumulate CDF max: range
bound R
Cy =0
for k in 1. ... K do
p= y [px]
Cs CDF
if 2 R then
C = CDF max
else i <
R then
L Ch : 0
else
L Ck Cklp + R
// shift R to obtain non-negative query index
= fr, X
12
C3 = Cÿ Ck
Algorithm 3: Encoding with deterministic GMM
Input: symbol ÿ. presently coder state t, CDF index iK. round-down means [+]]... Ju quantized weights (x, ... Output: updated state t
Data: upper bound frequency cumulate CDF max ik. [14]] [PK]] 9m, ... 9xK
Cg CDFIndex(j,
Cg = > 4n CDF or = 0 then
((1-R) == CDFIndex( R 1, 1,ir..ii, Len ... [ux] 1.. (xxx
GolombEnc(j, t t= ArithmeticEnce R, ((:
else
C++1 CDFIndex(i 1, ... ik. [un] ... [Mx] 9m. ... 9TK)
t = ArithmeticEnc (ū. Cg. c,+1,t)
23

---

Under review as a conference paper at ICLR 2022
Algorithm 4: Decoding with deterministic GMM
Input:
presently decoder state CDF index i ik. round-down means [ي]] ... [ux] quantized weights -
Output:
decoded symbol ÿ. updated state f
Data: upper bound of frequency cumulate CDF max
/* Search-based ń decoding.
This can be replaced by binary
search
Go = GetDecodderDDFLoo/)
9 R
for y in - R.... R do
Cy == CDFIndex (y, (y, i .... ik. [n]]... ... [uk] J. 97 ... (xx)
Cy+1 CDFIndex(y 1, ik. [u]] ... [Kx] qmz ... Ix
cy S %o < G,41 then
==
/* Update the decoder state
g - R then
/* The Golomb coding is adopted
C(1 R) == CDFIndex(-RR R 1,1. [Ma]... ... [ux] 1x, ... 1xxx
t = ArithneticDDeCppa(... (())),))))))
ÿ = GolombDec(t)
12 else
13
Cj = CDFIndex( (. i ik. [u]].. ... [u [..].c. 9m, ... gxK)
14
Cs+1 == CDFIndex(j 1, ... ik, [u ... [ux] 4m ... (Tx)
t ArithhncticDec(hpp(lc...pppp Cx+1,)
15
24


Reviews:
[{"review_id": "tZNEPQekyYL", "reviewer": "Reviewer_L7dn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1.  It is interesting to see the model compression techniques could be beneficial for the learned image compression in terms of cross-platform development. Instead of designing new model components, the proposed approach is more flexible. \n2. The experimental results are encouraging. It seems that this quantization method does not introduce too much performance degradation. \n\nWeaknesses:\nSince the major techniques are borrowed from the quantization techniques in model compression, so the technical contribution may be limited.  The authors are suggested to make a clear description for the difference between the techniques used in this paper and that in model compression.\n\nBesides, the authors only provide the RD curve below ~1bpp. How about the compression performance at high bitrate?", "summary_of_the_paper": "The paper tried to solve the non-deterministic issue for the learned image compression and reduce the inconsistent cross-platform probability prediction. It is a practical problem in the development of learned image compression. The authors utilize some off-the-shelf techniques in the model compression and revisit the non-deterministic issue from the perspective of model compression. The experimental results are convincing. ", "summary_of_the_review": "I tend to accept this paper though I have some concerns about the technical novelty. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "o16v6EE2mvE", "reviewer": "Reviewer_2XDr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n   - In my view, the most important contribution of the paper is to demonstrate that state-of-the-art image compression methods can be appropriately quantized to retain essentially the same compression performance as their unquantized counterparts\n   - The extension of the look-up table technique of [1] to Gaussian mixture models is interesting and clearly useful\n\n## Weaknesses\n   - I am not convinced that the authors can claim the application of post-training quantization (PTQ) to achieve cross-platform consistency for image compression models as a contribution. As I understand, this idea was first proposed in [2] and further developed in [1], and in both papers, the authors demonstrate the efficiency of PTQ. In fact, the authors seem to acknowledge this a couple of paragraphs above! I believe that what the authors can claim is the application of PTQ to models using autoregressive context models. Could the authors please clarify if my understanding is correct? In case it is, the claim should be changed to better reflect the contribution of the paper.\n   - The results of the paper are not compared with relevant methods, most notably [1] and [2]. In particular, the authors present the performance of their method on the method of [3] but do not show the performance of e.g. [2] on the same model.\n   - In the introduction, the authors make the following claim \"However, we find that on more complex models with context modelling like Minnen et al. (2018) and Cheng et al. (2020), adopting this integer network approach cannot keep the performance loss negligible.\" - I could not find any reference to support this claim in the main text or the appendix. Could the authors point to any reference that would support this?\n\n### Minor issues\n   - The 1st and 2nd sentences of the abstract seem to be inconsistent with each other, the 1st implying that LIC is practical, while the 2nd implying that it is not.\n   - There are many grammatical mistakes in the text that need to be fixed\n   - The abbreviation \"LUT\" (which I believe stands for look-up table) is undefined in the text\n     \n## Questions\n   - The authors claim that they use quantization not for the purpose of compression but as a way to switch calculations to integer arithmetic for cross-platform consistency. In this case, why do the authors still prefer 8-bit quantized representations for the activations and weights?\n   - While the authors method to quantize the log-standard deviations is sensible, could they elaborate on why the method of [1] is not adequate? In the paper the authors claim that the method of [1] is moderately efficient, so I am unsure what benefit their method brings. \n\n## After Rebuttal\nThe authors have addressed my main concerns, and I, therefore, decided to raise my score.\n   \n\n[1] H. Sun, L. Yu, and J. Katto. Learned image compression with fixed-point arithmetic. 2021 IEEE Picture Coding Symposium\n\n[2] H. Sun, Z. Cheng, M. Takeuchi, and J. Katto, End-to-end learned image compression with fixed point weight quantization, 2020 IEEE International Conference on Image Processing \n\n[3] J. Balle, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston. Variational image compression with a scale hyperprior. ICLR 2018\n", "summary_of_the_paper": "A well-known issue in learned compression is that due to inconsistencies in the implementations of low-level floating-point arithmetic across different hardware architectures, catastrophic decoding errors might occur for the compressed data. The authors propose several different ways to quantize the weights and activations of the models to make the encoding and decoding procedures use integer arithmetic only. Since the implementation of integer arithmetic is consistently implemented across different hardware, this solves the decoding issues, while also providing some speed-ups. The authors demonstrate the efficiency of their method by quantizing some state-of-the-art image compression methods and show that the performance of the quantized models does not degrade much.\n\n\n", "summary_of_the_review": "I currently believe that while the work is useful to the community (see 1st point in \"Strengths\"), I think in its current form the paper makes a few considerable claims that are not supported. \n\nIf the authors can address my main concerns, I will reconsider raising my score.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_-Zprn8U9N", "reviewer": "Reviewer_oV3R", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "It is interesting that post-training quantization is enough for the model robustness of learned image compression. \n\nHowever, this paper requires polishing for the presentation and comparisons and the writing clarity should be improved.\nFor example, the term 'cross-platform inconsistency' is hard to understand until reading the Appendix and the paper of Balle et al. (2019).\nFigure 1 and 2 present the problem statement and the methods in previous works which limit to emphasize the contributions of this paper.\nExperiments do not analyze the contributions of the proposed method described in Section 3.\nMore importantly, this paper misses the comparisons to the method of Balle et al. (2019) for the model robustness across devices.\n\nOn the other hand, the proposed method described in Section 4 seems to be an alternative to post-training quantization since it is a predetermined CDF for different devices that minimizes the cross-platform inconsistency.\nBased on the alternative, I'm curious about the importance of the cross-platform inconsistency which can easily be solved by saving the same CDF as metadata and using it across devices.\n\nWhile quantizing weight with per-channel scaling factors, which granularity is used for the feature map quantization?\nDoes the proposed quantized model have actual latency acceleration with the per-channel weight quantization?\n\n", "summary_of_the_paper": "This paper presents a network quantization method for robust learned image compression.\nRecent deep-learning-based approaches for image compression usually adopt the cumulative distribution functions (CDF) for both encoding and decoding processes.\nHowever, the CDFs are usually implemented in devices (e.g., CPU, GPU) with different approximations.\nThe different values from approximated CDFs incur significant decoding errors.\nThis paper alleviates this problem by quantizing the feature maps and weights of the encoder and decoder to 8-bit.", "summary_of_the_review": "This paper presents a simple practical solution, post-training quantization, for robust learned image compression using neural networks.\nHowever, the contributions of this paper are marginal since the problem statement and solution are overlapped with the previous work (Balle et al., 2019).\nMoreover, this paper presents limited comparisons to the work of Balle et al. (2019) while Section 1 describes its failure cases.\nThe authors should thoroughly state the differences and improvements of the proposed method to the work of Balle et al. (2019).\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NGh6GEA9wSp", "reviewer": "Reviewer_eyVf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n1. Authors experimentally showed that PTQ can be used to prevent data corruption (Table 1). The effect on RD performance is also shown in Figure 6.\n2. I think cross-platform support for learned image compression is an important issue for practical applications.\n\n\nCons:\n\n1. Issues to be addressed are unclear:\n- The authors state in chapter 1: \"However, we find that on more complex models with context modeling like Minnen et al. (2018) and Cheng et al. (2020), adopting this integer network approach cannot keep the performance loss negligible. \", it is interesting but I could not find evidence for this; if the RD rate performance of the integer network is degraded, it should be shown in experiments.\n\n2. Comparisons with previous methods (e.g. Balle ́ et al. (2019) and Sun et al. (2021)) are not sufficient:\n- If there are any advantages of this research compared to the previous methods (e.g., if previous methods are not able to prevent data corruption in some image compression models), I think you should clearly show that comparison.\n- This paper proposes a new method (Binary Logarithm based STD discretization) to aim for more hardware-friendly. It is interesting but unfortunately, the experiments in Table 2 do not seem to show a comparison with Sun et al. (2021), which is written like a baseline in chapter 4.\n\n3. Other unclear points:\n- Is Table 1 the result of applying the LUT (proposed method in chapter 4)? If the error rate is 0% with only PTQ without applying the LUT, why is LUT necessary? Or, if you were applying the LUT, what happens if you do not apply the LUT but apply the PTQ? I am curious about the contribution for prevent data corruption of each of the PTQ and LUT.\n- Is the time shown in Table 2 the average time for the entire data set?", "summary_of_the_paper": "Learned image compression has a known issue that the nondeterminism of the floating point operation makes it impossible to recover the compressed data on different platforms.\n\nThis paper experimentally shows in compression models using context modeling (Minnen et al. (2018)) that it is possible to recover the data on different platforms even when using the known quantization method called PTQ. (Table.1 )\n\nReferring to the method of computing STD using pre-computed sampling points σˆ by Sun et al. (2021), the authors proposed a method called Binary Logarithm based STD discretization.", "summary_of_the_review": "The authors experimentally showed that PTQ can be used to prevent data corruption of learned image compression, but the value of this research is not clear due to the unclear target issues, insufficient comparison with previous methods (e.g. Balle ́ et al. (2019) and Sun et al. (2021)), and some unclear points, so I considered that currently this paper did not meet the conditions for acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5asiGmIOVS6", "reviewer": "Reviewer_GrpS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \nThe authors do a great job of immediately conveying the problems in image compression (designing deterministic decoders and the speed performance of those decoders).\n\nThe results are strong, they show very little to no loss until high bitrates due to this quantization technique.\n\nThe reforming of the entropy parameter discretization to be more computationally efficient shows impressive runtime improvements on this quantization process with no RD loss.\n\nAdditionally, the appendices were well written and contain many additional derivations and detailed descriptions of the quantization algorithms applied.\n\n\nWeaknesses:\nRate-distortion performance is rarely shown outside the 1.0 bpp range. Since the losses appear to be more evident at higher bitrate, it would be instructive to know if PTQ is as successful outside of the low to medium quality range.\n\nHow much of a difference does one LUT make? The baselines are compared against a 64 level table and the new method proposes a much faster 65 level table. It is an obvious improvement in runtime due the formulation, but are there any significant RD gains for having another level.\n\n\nQuestions/Feedback:\nTable 2: I'm not 100% sure what is being compared here. Is this a hyper synthesis with scale, scale-mean, context, GMM? Is there a reference for the popularly vectorized version? Was your implementation done in C++ or inline Assembly to guarantee the fast assembly instructions were properly generated?\n\nIn Section 3.1: \"a grid search minimizing the reconstruction error\", how much does this grid search help in the quantization process? Does this need to be done on a per loss function basis or per model? i.e. if I train a model for a specific lambda with a specific architecture, will I generally use the same quantization step, or is this more sensitive to individual runs in compression models?", "summary_of_the_paper": "Quantization issues around entropy coding in neural image compression can lead to non-determinism across platforms and runtimes (i.e. CPU to GPU or even different drivers on the same GPU). Many works in the are perform quantization during training, which can make for more complex or longer training setups. This paper shows the effectiveness of post-training quantization techniques, shows the results are successful even on context and GMM models, and introduces a faster entropy parameter discretization method.", "summary_of_the_review": "This is a well written paper that shows the effectiveness of existing post training quantization techniques when applied to multiple neural image architectures. In addition to showing how well this PTQ techniques perform, a more computationally efficient entropy parameter discretization technique is introduced.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers quantization issues for learned neural-network-based image compression methods. 
Many works on the topic incorporate quantization into the training of the method. The paper provides evidence that post-training quantization is effective. Specifically, the paper demonstrates that state-of-the-art learned image compression methods can be quantized post-training and retain a very similar level of compression performance. The paper argues that this is important in particular for cross-platform applications, where an image is decompressed on different architectures. Finally, the paper proposes an approach to discretize entropy parameters.

The reviewers raised the following concerns. 
- Reviewer 2XDr is concerned about the application of post-training quantization being a contribution, since post-training quantization has already been studied in [1] (and in the recent paper [2] that can be considered as concurrent work). The authors response is that the methods in [1] has extra overhead and clarify how the prior work is in fact different. This addressed the reviewer's concern, and the reviewer raised their score.

- Reviewer eyVf finds the comparisons with previous methods to be insufficient, and in general find the value of the research unclear, as the goals are not sufficiently specified. The authors clarified, and the reviewer was satisfied with the response and raised the rating to marginal above the threshold. 

- Reviewer L7dn tends towards acceptance, but has concerns about the technical novelty, that are unspecified unfortunately.

- Reviewer oV3R argues that the solution is marginal relative to prior work, and votes to reject the paper. The authors responded why they think it isn't, and also wrote a private letter to the area chair in which they explain why they think that reviewer's oV3R should not be taken into account. I agree with the authors that the paper under review provides a step relative to Balle et al (2019), and that the writing of the paper is not an issue; however, the reviewer's overarching point is that the overall contribution is marginally significant when taking the prior work by Balle et al (2019) into account and this is the sentiment of other reviewers as well.

- Reviewer GrpS, an expert on image compression, leans towards acceptance and argues that the results are strong as they show little to no loss due to the quantization technique, but also rates the contribution to only be marginally significant and novel, and raises a few questions and issues, to which the reviewers responded. 

This paper is really borderline. Four out of five reviewers rate this paper as marginally above the acceptance threshold. The consensus is that while the experiments and claims are correct, the contribution is only marginally significant or novel, in particular, relative to prior work, and therefore I recommend rejecting the paper. I would, however, not be upset if it would be accepted.
Decision: Reject