Compression without Quantization
Authors: Gergely Flamich
Marton Havasi
José Miguel Hernández-Lobato
Venue:  (2020)
Keywords: Image Compression
Bits-back efficient
Quantization

Abstract:  Standard compression algorithms work by mapping an image to discrete code using an encoder from which the original image can be reconstructed through a decoder. This process, due to the quantization step, is inherently non-differentiable so these algorithms must rely on approximate methods to train the encoder and decoder end-to-end. In this paper, we present an innovative framework for lossy image compression which is able to circumvent the quantization step by relying on a non-deterministic compression codec. The decoder maps the input image to a distribution in continuous space from which a sample can be encoded with expected code length being the relative entropy to the encoding distribution, i.e. it is bits-back efficient. The result is a principled, end-to-end differentiable compression framework that can be straight-forwardly trained using standard gradient-based optimizers. To showcase the efficiency of our method, we apply it to lossy image compression by training Probabilistic Ladder Networks (PLNs) on the CLIC 2018 dataset and show that their rate-distortion curves on the Kodak dataset are competitive with the state-of-the-art on low bitrates.

Under review as a conference paper at ICLR 2020
COMPRESSION WITHOUT QUANTIZATION
Anonymous authors Paper under double-bling blind review
ABSTRACT
Standard compression algorithms work by mapping an image to discrete code us- ing an encoder from which the original image can be reconstructed through a de- coder. This process, due to the quantization step, is inherently con-differentiable so these algorithms must rely coder end-to-end. In this paper, we present an innovative framework for lossy im- age compression which is able to circumvent the quantization step by relying on -on-determiiistic compression codec. The decoder maps the input image to dis- tribution in continuous space from which sample be encoded with expected code length being the relative entropy to the encoding distribution, i.e. it is bits- back efficient. The result principled, end-to-end differentiable compression framework that can be straight-forwardly trained using standard gradient- optimizers. To showcase the efficiency of our method, we apply it lossy im- age compression by training Probabilistic Ladder Networks (PLNs) on the CLIC 2018 dataset and show that their rate-distortion curves on the Kodak dataset are competitive with the state-of-the-ar on low bitrates.
approximate methods to train the encoder and de-
INTRODUCTION
The recent development of powerful generative models, such as Variational Auto-Encoders (VAEs) and their hierarchical extensions, such as Probabilistic Ladder Networks (PLNs) (Kingma & WWeling 2014] SSondeby et al.] 2016 Higgins et al.) (017)) has caused great deal of interest in their application to lossy compression, notably Ballé al. (2016); Theis et al. (2017): Rippel & Bourdeey (2017); Ballé et al.] (2018); Mentzer et al. (2018); Johnston et al. (2018). The benefit of using these models as opposed to hand-crafted methods is that they can adapt to the statistics of their inputs much better, and hence allow significant gains in compression rate at given quality setting, and in quality at given compression rate. A second advantage is their casier adaptability to new media formats, such fielt-feeld cameras, 360 images, Virtual Reality (VR), video streaming, etc. for which classical methods are not currently applicable, or would perform pathologically badly.
Lossy compression codecs usually perform lossy transformation on the continuous representation of the data, after which it is mapped to a discrete domain through an operation known as quan- tization, such that entropy coding may be used to encode this transformed representation. This continuous- contnnous-to-disereee -o-discrete mapping will mean that quantization will necessarily have zero derivative almost everywhere. Since the above mentioned generative models are usually trained using some gradient-based technique, this means that they cannot be directly applied as compression models, as quantization will destroy the learning signal, and hence a workaround is needed. nice remedy to this problem is to replace the quantities affected by quantization (or at least their derivatives) by some smooth approximation during training, as done recent approaches, e.g. [Bllé et al. (2016); Theis et al. 2017); Ballé et al. (2018); Mentzer et al. (2018); Johnston et al. (2018).
Similarly to Ballé et al. (016); Theis et al. (2017); Ballé et al. (208)) we aim to minimize the rate-distortion of model directly, but unlike them replace entropy coding with more gen- cral coding technique that allows us to use probability densities for coding instead of probability masses. This then allows to forgo the quantization operation altogether and we get end-to-end differentiable model that can be optimized with standard gradient- based optimizers.
We select appropriate generative model p(x, z) p(x z)p(z) with approximate posterior qzz xx), where the latents z are going to serve as the transformed representation of x that we wish to compress. We show that -lggp(x corresponds the distortion and KL,[ x) || p(z)]

---

Under review as a conference paper at ICLR 2020
corresponds the rate of the model. Hence, optimizing model for the rate-distortion is as simple as minimizing - lox z) + BKL [ x) || p(z) where / is a scalar hyperparameter
controlling the compression rate.
To compress some data a with our trained model, instead of selecting = deterministically and then coding it, map to the posterior g(z æ)) and use it code sample from it, using adapted version of the importance sampling algorithm proposed by Havasi et al.) (2018). First, set K KL æ) || p(z) and stochastically draw independent samples (Z1,.... [[exp(K)]) from p(z). Next, seleet ze- for some 1 c² {X We can the index c* as the compressed representation of z,-. Clearly, c* can be coded in K nats, and it can also be shown that Z,- will be a low-bias sample from q(z 2²). As the efficiency of this algorithm is KT. [q(z) æ:) || p(z)], will refer to it as relative entropy coding (REC), and note that REC algorithms achieve bits-back efficiency (Hinton & Van Camp 1993).
The contributions of this paper are as follows:
We show how any generative model p(x, x) p(x 22)(2)) with approximate latent pos- terior q(z x) can be used for bits-back efficient lossy transform coding, using an adapted version of Minimal Random Code Learning (MIRACLE, Havasi et al. (2018)).
We present the concept of relative entropy coding (REC) algorithms and attainable bounds on their worst-case performance. We note their current intractability in high-dimensional settings and propose approximate solution that is tractable, though the rate-distortion quality is impacted.
We demonstrate the effectiveness by applying our method to lossy image compression. Concretely, train a Probabilistic Ladder Network (Sønderby al.) (016) the CLIC (2018) dataset and show that the rate-distortion curve of our method is competitive with the
on the Kodak dataset (Eastman Kodak Company 1999) on low bitrates.
LITERATURE REVIEW
2.1 TRANSFORM CODING
Neural network- based approaches lend themselves easiest to transform coding as realisation of lossy compression. In transform coding (Goyal 2001), given some data it is first transformed us- ing the analysis transform z = fa (x) and then quantized á = [=]. where [-] could denote rounding. any other quantizer in general. The distribution of this quantized representation is modelled by some probability mass function P(ž), which is used to encode & using some entropy code. Note, that this is different from the true distribution of &, P(2). which is entirely determined by the dis- tribution of [x)], where x p(x). During decompression, once is decoded, the synthesis transform x' = fz(z) ) is applied to it to obtain a reconstruction x. Given metric ",
), the distortion of the codec is
Exxp(x) [d(x, x')],
and its rate is the expected codelength of the entropy code, which is
E Ex (())
[- log P(t) Ex<p(x) log P(z)
The equality holds, since is deterministic transformation of x.
2.2 BITS-BACE EFFICIENCY
instedd mapping some data we wish to compress æ to a single deterministic point and entropy coding it by using some prior p(z). we map it to approximate posterior q(z x)), then we may select a random representative z from gz x)) in the sense that z" has short Minimum Description Length (MDL, Grünwald al. (2007). Bits-back efficient methods
such that this MDL is
approximately KL [Z[] x) p(z An important distinction should made whether we wish to communicate the distribution (z) m) or sample 2" drawn from as the communication problem
2

---

Under review as a conference paper at ICLR 2020
from which the MDL can be obtained, its precise value and the method required to obtain it are different.
The concept originates in the work of Hinton & Van Camp (993). Given data D and model weights w, they derive the MDL of the posterior weight distribution g(w D) of a Bayesian Neural Network (BNN) with prior p(w) be K = KL [q(w] D) p(w)] However, they did not give method for achieving this, in fact, in their argument they merely show that in a longer message only K nats are used to code the posterior.
In our setting, we are only interested in communicating a single sample =" from g(z) æ:. We assume that we have access generative model p(x | p))(z) 2)) and posterior q(2 x).. Then, the problem we must answer is what is the minimum expected number of nats T'x z] that is required to be communicated, such that we could draw sample ~ q(z x). This question was answered by
Harsha et al. 2007), who proved the following theorem:
Theorem Harsha et al. 2 Let the setting be as described above, and assume that the coder and the decoder share a common source of randomness (e.g. by using the same pseudo-randon number generator with the same seed), then the minimum expected number of nats T : z] that
need to be communicated in order for the decoder to be able to sample a 2 q(z x) can be bounded in terms of the mutual information [[x z] as
1[x: z) - T(x: z) < [[x: z] + 2 log (1[x : z] + +1 + 011)
(3)
They prove the above claim constructively, by giving rejection sampler that achieves this. This, to our knowledge, is the first example of what we call a single-shot relative entropy coding (REC) algorithm, and we will examine it in further detail in Section[4] Bits-back efficiency comes from the fact that / [x z Ex~p(xx) [KL [z x) || p(z)]] [].
Relative entropy coding subsumes entropy coding in the sense that any entropy code is relative entropy code as well, since for a discrete 2 ~ P(2) and the point-mass &g on E,
log P(2) KT. [& || P[2)].
See Havasi et al. (018)) for details on the derivation. In this sense, methods that use quantization and then entropy coding are also bits-back efficient, but are restricted to point-masses for their latent posteriors. Thus, one of the advantages of our method lies in that allows much wider class of posteriors to be used.
2.3 RELATED WORKS
Minimal Random Code Learning (Havasi et al. 2018) is a powerful coding technique that allows us to code random sample from continuous distribution, rather than coding deterministic sample, which requires quantization, as any fixed sample would have density. They use their technique to compress the weights of BNNs. The structure of the weight space is much different from the structure of the latent space of a generative model trained for the compression of some medium. Hence, we argue that their practical compression procedure is infeasible for our setting, the random fixed-size block partitioning as well as the retraining between the coding of blocks is incompatible with our setting as well computationally infeasible.
Townsend et al. (2019) were the first to develop a bits-back efficient image compression algorithm using VAES, called Bits-back with ANS (BB-ANS). However, they develop a lossless compression algorithm, and hence the performance their approach is not comparable to ours. The sense in which their method is "bits-back" also differs from ours. Townsend et al. (2019) stick to the original bits- back setting. Concretely, they show a clever way of utilizing the overhead of the method of Hinton Van Camp) (1993) for efficient coding, our work on the other hand, avoids sending it in the first place. Their coding efficiency collapses onto entropy efficiency when coding single image and bits-back efficiency is only achieved when coding larger batches.
Lately, there has been an explosion of interest in using powerful deep generative models for lossy im- age compression, and thus all previous methods had to deal with the issue of the non-differentiability of quantization. These approaches how they avoid this problem. There are approaches that waive the paradigm of end-to-end training, e.g. by not incorporating the feedback from the gradient of the rate term into the optimization of the
generally be put into three different categories. depending
3

---

Under review as a conference paper at ICLR 2020
entropy coder, either by using inflexible distributions without any parameters (Toderici al. 2017
or by optimizing them post-training (Johnston et al. 2018: Rippel & Bourdey 2017). The issue with this, pointed out in Ballé al. (2018) 18 that not trainin end-to-end might severly limit the efficiency of these methods.
Another approach of getting aroud this issue has been learn generate samples from the image distribution p(x), using a Generative Adversarial Network (GAN, Goodfellow et al. (2014), by learning the synthesis transform first and using an auxiliary discriminator. Then, fixing the discrimi- nator, the analysis transform is trained in order to minimize the reconstruction error d( fa (Ja (3)))) The noise provided fa during the initial GAN training should be continous relaxation of quantiza- tion noise, such as uniform distribution (Ballé et al. 2016).
Finally, there are methods that stick Variational Auto-Encoders (VAEs, Kingma Welling 20144; Higgins et al. (2017)), optimize the weighted rate-distortion directly, and replace the non- differentiable quantities with smooth approximations during training. (Thiis al. (2017) introduce compressive auto-encoders, and replace the derivative of the quantization operation with the iden- tity function, and the rate of the discrete by differentiable upper bound which used in the loss. Closest to our work are the works of [aalé et al. (2016) 2018); Minnen et al. (2018). who all model the quantization noise as uniform noise centered on the quantized representation. [Ball ct al. (2016) are the first to derive the connection between the weighted rate-distortion loss and the 3-Evidence Lower Bound (8-ELBB, Higgins et al. (2177), ), though they only derive for smaller class of distributions than do, well through different route,
arrive at it through the MDL principle (Grünwald et al., (207). Crucially, these methods are still only equivalent to VAEs during training, and the continuous relaxations are switched back the original discrete operations thereafter.
3 COMPRESSION WITHOUT QUANTIZATION
Our method relies on two observations. First, we note that the sole purpose of quantization is that once we have obtained a deterministic representation 2, has non-7ero probability mass, and hence entropy coding may be used to compress it. Hence, if we could replace entropy coding with method that could use probability density p(z) instead of the mass P(2), then we could forgo the quantization step and thus our training objective becomes end-to-end differentiable.
Second, as we have seen in Section 2.2 if in addition have access to an approximate latent posterior q(2) x), then it is enough to communicate approximately
[[x
z] hlz]
A Alz
x]
Ex=p(xx [KL [g xx) || p(z)]]
(5)
nats such that the decoder may obtain a sample ~ q(> .). Applying Sa to z could then be used to reconstruct
These two could be combined by noting that our setting fulfils all premises required for both obser- vations to hold. This enables the following development of a lossy transform codec:
Training After fixing an appropriate architecture for fa( = q(2 a:) and f. such that the like- lihood is p(x [=(z)]2), we train our model using the 3-Evidence Lower Bound (Higgins al. 2017|
Ex loggllg(px z)] [Ex=p(x [KL q(z x) || []
(6)
Note, that if the distortion metric d(:,.) is such, that expí d(x x')} can be normalized, then the negative log-likelihood can be identified with the distortion D. Exxxp(x) [TL[[4] x) p(z) = [[x z] can also be identified with the rate R. Hence, in this case the negative 3-ELBO is equivalent to training for the rate-distortion loss
L= D= 3R,
way to interpret 3 in Equation 6 it allows the model to converge on different points on its rate-distortion curve (see Ballé et al. (216)) for the full argument).
which was our aim to train for in the first place. This gives
Note, that even if normalizing exp(-d(a- x')} is intractable 3 can be adjusted in Equation 6 to compensate for this unknown constant during training.
4

---

Under review as a conference paper at ICLR 2020
Compression and Decompression For given that we wish to compress, calculate g(z x) using the analysis transform. Then, we use relative entropy coding (REC) to code a sample ~ q(z) æ) The decoder can then re-obtain and apply the synthesis transform to it to reconstruct
4 RELATIVE ENTROPY CODING
As mentioned in the previous section, our method relies on the existence of relative entropy cod- ing (REC) in order to be able to code the latent representation of a given imput æ with bits-back efficiency. To knowledge, the first such algorithm was developed by Harsha et al. (2007), variant rejection sampling. This algorithm is unfortunately intractable in more than a few dimen- sions, as it requires to keep track of probabilities across the entire range of 2.
The simplest way to extend the univariate algorithm to the multivariate setting is to note the inde- pendence assumption for the dimensions of and thus concatenating univariate samples for each dimension will given an exact multivariate sample. Hence, we could use REC to encode the di- mensions of 2 individually, i.e. for each 1; obtain some code C₁. and build an empirical distribution P(c) over these. Then, we can encode each using entropy coding. The issue with this approach that while the upper bound in Theorem T is quite tight if we apply to the whole multivariate distribution over 2, when applied to each of its dimensions 2, the ((1) cost is incurred for each dimension, which makes this approach very inefficient. Hence, we focus on multivariate REC.
4.1 ADAPTIVELY GROUPED IMPORTANCE SAMPLING
Algorithm Importance sampling algorithm proposed by Havasi et al. (0188
Inputs:
P Proposal distribution
Q Target Distribution
(z: ~ Qli N) Shared sequence of random draws from Q
procedure IMPORTANCE-SE SONORTTACE-SSMPLERRCC SAMPLER(PP Q, (z; ~ Q N))
K f- (KL Q || P}}
w; + log P(a.)
Q(x.)
Vi 1, K
j f arg max; Wi
return J.
end procedure
The basis of our approach is the approximate importance sampler proposed by Havasi et al.
2018
For given æ; it proceeds by letting K KL [g(z æ)) p(z) ] and drawing e samples
(21; ; Es) from p(z). Then the index of the maximal importance weight
(Ze (
c" = arg max <E (1.2...88) p(zc)
(8)
used as the encoding of x. The decoder can recover 2e- by drawing e* samples from p(z) and then can use it to reconstruct æ assuming it can generate the same sequence of samples. Clearly, by virtue of the choice for S, c* can be coded in KL[g(2 æ)) p(z)] nats, and Havasi al. (2018) also show that Ze- has negligible bias. The algorithm is depicted in Algorithm I note that Havasi al, (0188) originally sample the softmax distribution of the ú, We found better to select samples biased towards the mode by always selecting the most likely sample. Though now we do not need to keep track of the probabilities over the entire range of z, the run-time is still going to be exponential in K, and hence becomes prohibitive for K larger than 15 20. In our experiments we had to deal with KLs well above 10a, and hence solving this issue is critical for the viability of our method.
To solve this issue, Havasi et al. (2018) randomly partition the vector of interest into fixed size blocks, in the hope that on average each individual block will have approximately some C nats of total KL. They then code each the blocks sequentially. The concern is that the next block to be coded a suboptimal choice, i.c its total KL is significantly higher or lower than the desired nats. To circumvent this problem, they introduce blockwise KI. penalty to the loss that forces each block
5

---

Under review as a conference paper at ICLR 2020
to have equal KL, and they intersperse coding block with several rounds of retraining. They show that this solves the suboptimal choice problem, and their coding procedure also does not impact the quality of the coded w.
Unfortunately, this fix is not applicable in our case. Havasi et al. (018)) can use the retraining trick to compensate for suboptimal block choices. However, if we hope obtain useful compression codec, we should avoid extremely expensive operations to code a single item, such as performing gradient descent for each individual input æ to our algorithm.
To remedy these issues, instead of selecting fixed-size blocks randomly we impose a group size constraint of G bits and total group KI. constraint of B bits. Then, partition the dimensions of z such that each set in the partition (what we call a group) has size at most 2 and each group has total KL most K bits. The details are depicted in Algorithm 2
Algorithm Adaptively grouped importance sampler
Inputs:
K Maximum individual KL allowed
G Maximum group size
B Bit budget per group
P Proposal distribution
Q Target Distribution
(r; Q E N) Shared sequence of random draws from Q
procedure
G, B. ( QliEN))
F ()
D Initialize empty list of group sizes Get KLs for each dimension
k; KL[Q: || P
P' REMOOEEO
y 0,+
> Current group size and group KL
for + 1, 1,....iii)( do
if k: +k; kl; B o >>>>>>> then Append to
k k4, 1
else
h: k: kl,,7 4 7
end if
end for Append to r
Append the last group size
S (), 0,9+ for in do
Samples, sample codes and current group index Now importance sample each group i, IMPORTANCF-SAMPLERR g:gtt: Qyigtry (x Q i E N))
Append to I, append to S
g f g + end for return I, S, end procedure
While the concern of intractability due high KL. resolved this way but we are faced with two inefficiencies. First, for groups whose size is much smaller than 2 dedicating G bits to coding the group size is wasteful. Second, many groups will have size 20 but total KL much less then K, and hence dedicating K bits code the sample is wasteful. To solve these issues, we built an empirical distribution over group sizes using our training dataset, which tractable since we chose to be not too large (2 < G 6), and then used arithmetic coding (Rissanen & Langdon [981) to code the group sizes. This worked very effectively, to the extent that for appropriately selected the cost of communicating the group sizes is effectively negligible.

---

Under review as a conference paper at ICLR 2020
Output
Input
Sigmoid
Conv2D 5x6x3 / u2
Conv2D 5x5xN
IGDN
GDN
Conv2D SxdxN u2
Gonv2D 5x5xN d2
First Level
IGDN
Conv2D
Conv2D 5x5xN d2
IGDN
GDN
Conv2D 5x5xN u2
Conv2D 5x5xF d2
Conv2D 5x5xF d2
Exp
Exp
Conv2D
Conv2D 5x5xF
Conv2D 5x5xF
Leaky ReLU
Second Level
Leaky ReLU
Conv2D 5x5xM
Leaky ReLU
Lesky ReLU
Conv2D
Conv2D 5x5xM u2
Conv2D
Sigmold
Blocks Layers
Arrows
Distribtions
Input Output
Shard for posterior and prior predicioon
Output of torget
Chriut scurce predicte mean of larget
Prions
Powieriar prediction
raconstruction
Sample from source fod inpul of target
Output predicts
Postariors
Figure 1: PLN network architecture. The blocks signal data transformations, the arrows signal the flow information. Block descriptions: Conv2D: 2D convolutions along the spatial dimensions, where the W x H C/S implies W H convolution kemnel, with C' target channels and S gives the downsampling rate (given a preceding letter "d") or the upsampling rate (given a preceding letter "u")) If the slash is missing, it means that there is no up/downsampling. All convolutions operate in same mode with zero-padding. GDN IGDN: these are the non-linearities described in Ballé et al. (2016). Leaky ReLU: elementwise con-linearity defined as max a.z), where Sigmoid: Elementwise non non-linearis linearity defined as I+exp[-2] We ran all experiments presented here
a 0.2.
with N = 196, M 128, F 128, G 24.
7

---

Under review as a conference paper at ICLR 2020
The relative entropy codes of the groups were also further compressed using Elias d-coding (Elias 1973). This the main point of inefficiency of this method, as the empirical distribution of sample indices has much heavier tail than the distribution implied by ô-coding. Concretely, that using 5-coding the resulting code lengths are approximately 1.6 1.8 times the theoretically possible REC code lengths. However, building empirical distribution over the sample indices and using it for entropy coding is more challenging, as the number of possible indices for a "useful" K between 2 2 < 220, This means that obtain good estimate of the tail mass, i.e. of indices above 2 2 we need lot of data, or some reasonable smoothing should be used, and leave this for future work.
found
EXPERIMENTS
Figure 2: Demonstration of the effectiveness of the PLN's conditional independence assumption p(z) = ()z )2 a) Original image from the Kodak dataset. b) Randomly selected channel of latent sample 21) ((2(2) 22(2), x). clearly there is lot of structure present in the sample. c) The same channel of the latent sample standardized according to p(z(2) 2(2)), i.e. we
1 is displayed for each dimension, where the n4; and Ø; the means and standard deviations 5, of the dimensions of p(z()) x(()), respectively. We see that relative to the conditional prior. the latent structure effectively captured.
We applied our framework to lossy image compression. We followed the work of Ballé et al. (2016) and [Bllé al.] (2018). Not only do their results represent the current state-of-the-art, but their work is also the closest to ours. In particular, as mentioned before, to our knowledge they were the first derive the connection of the weighted rate-distortion loss with the 3-ELBO as the training objective for VAE. However, we stress that they derived this for VAE whose latent distribution was continuous relaxation of the quantization error. Therefore, their results were restricted narrow set of distributions, and neither did they make the connection to bits-back efficientcy. Most notably though, they only used the continuous relaxation during the training of the model, thereafter switching back to quantization and entropy coding, which, they show does not impact the predicted performance, and hence confirming that their relaxation during training is reasonable. This in striking contrast with our method, in which not only do we allow any latent distributions for which the KL. can be calculated, but our architecture also remains the same after training.
Nonetheless, based architecture entirely the one they present in Ballé et al. (2018). appro- priately modified for our setting. Most importantly, where they use i..dd uniform distributions for the posterior and complicated non-parametric priors, opted for Gaussians, much standard
8

---

Under review as a conference paper at ICLR 2020
choice for VAEs. We also opted for similar hierarchical VAE structure, which in case is Probabilistic Ladder Network (PLN, Sønderby et al. (2016)) The reason for this is as shown by Ballé et al. 2018) and also confirmed by us empiricall,, the full independence assumption in the la- tent distribution of a VAE is very limiting, as there is lot of topological structure even on the latent level. One solution introduce rich dependence structure is introduce further latent variables, conditioned upon which the rest are assumed to be independent (Bishop, 1998). Concretely, we z zt pp() and our approximate latent posteriors now become ((z(2) x) and q( (1) x, z 2()) x q(z()) x)p(z |z(2)). Further, our archi- tecture also only uses convolutions and deconvolutions non-linearities. This allows us compress arbitrary sized images, and the latent space can naturally grow with the dimensions of the images. confirmation that the PLN's conditional independence structure efficiently removes the topological structure in the latent space on the first stochastic level is shown in Figure Note, that order for us code latent sample q(z x),
break up our generative model p(x, z) p(x ( (p(z())
Full details of the architecture can be seen in Figure [
we first code (2) (z( x) using p(z(2)) and then code (1) (2) q(z (1) xt, (2)).
MS-SSIM camparison for kodim11
PSNR comparisnn kodim11
31
M. SSIM (dB)
21
PSNR
Theoretical
Theoreticll
Actual
Actual
Ballé et
Ballé
JPEG
JPEG
Bits / Pixel
Bits / Pixel
MS-SSIM comparison for kodim21
PSNR comparison kodim21
225
202
MS SSIMSSM (dB)
21
003
Theoretical
Theoretical
Actual
Actual
Ballá et
Ball et
.PEG
JPEG
Bits / Pixel
Bits / Pixel
Figure 3: Comparison of rate-distortion curves on kodimll png andkodim21 png from the Kodak dataset. MS-SSIM results are presented decibels, where the conversion done using the formula - 10 1og (1 MS-SSIM(z, x')) is done by Ballé et al. (2018). For the description of the labels, see Section
In our experiments, we compare the performance of our method with the method of Ballé et al. (018) and JPEG as baseline on their rate-distortion curves, using the Peak Signal-to-Noise Ra- tio (PSNR) and Multiscale Structural Similarity Index (MS-SSIM) perceptual metrics (Huynh- Thu Ghanbari 2008 Wang et al.
2003) as distortion distances. Our models were trained using and the Mean Absolute Error (MAE) loss. The reason for the MAE loss is due to Zhao et al. 2013).. who show that using it greatly improves image recon- struction quality at no significant extra cost. The model of Ballé et al. (2018) was trained using {0.001, 0.003, 0.01, 0.03, 0.1, 0.3) and the Mean Squared Error (MSE) loss. Both were trained on the CL.C 2018) training dataset, which allows better comparison in performance. All models
e {300, 600, 1200, 2400, 4800

---

Under review as a conference paper at ICLR 2020
trained for 2 x 10' iterations with Adam (Kingma & Ba 2014), using a learning rate of 10 We stress, that for comparability, the architectures were made completely identical, except for the latent distributions, which needed adaptation to our setting.
We report the rate-distortion curves on two images taken from the Kodak dataset in Figure 3 Further comparison plots be found in Appendix A For models, report the theoretically optimal
rate-distortion, the rate of which for an image a can be calculated as KL[g(2 x==) p(z) 1og 2, and
the actual rate-distortion achieved by our proposed REC algorithm from Section 4.1
For the models
of Ballé et al. (2018). we report the actual rate-distortion.
We that our method is competitive with the method of Ballé al. (2018) low bitrates and starts tailing off more on higher bitrates.
CONCLUSION
We develop a lossy transform coding framework based on generative models equipped with an approximate latent posterior and show that is bits-back efficient (Hinton & Van Camp 1993) and can also be optimized end-to-end. We propose a relative entropy coding algorithm to achieves bits- back efficiency, and we demonstrate the efficiency our method by training Probabilistic Ladder Network on the CLIC (2018) dataset and show that it is competitive with the current state-of-the-an on the Kodak dataset Eastman Kodak Company (1999).
While allowing for a much wider class of latent distributions, our methods relies on the existence of efficient relative entropy coding algorithms for the selected distributions. Our REC algorithm based the importance sampler proposed by Havasi al (2018), and while it is tractable and works reasonably well on low bitrates, its rates are approximately 1.6 1.8 times higher than the theoretically possible ones. Currently the run-time is also slower than other methods, it takes around 1-5 minutes to compress a reasonably sized image. Improving the rate factor and the run- time does not seem too difficult task, but since the focus of work demonstrate the efficiency of relative entropy coding, it is left for future work.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compression. International Conference on Learning Representations, 2016.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with a scale hyperprior. In Interattinnal Conference on Learning Represen- tations, 2018. https::t:////eeeet
net/ teee//oorummLGGrkccFFWW
Christopher M Bishop. Latent variable models. In Learning in graphical models, pp. 371-403. Springer, 1998.
CLIC. Workshop and challenge on learned image compression. httpps//ww..compresssll cc 2018. Accessed: 2019-03-25.
Eastman Kodak Company, Kodak lossless true color image suite. http://zkk.u//rapphiin
kodak/ 1999.
Peter Elias. Universal codeword sets and representations of the integers. IEEE transactions on information theory, 21(2):194 203, 1975.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances neural infor- mation processing systems, pp. 2672-2680, 2014.
Vivek K Goyal. Theoretical foundations of transform coding. IEEE Signal Processing Magazine, (815))9211, 2001.
P.D. Grünwald, A. Grunwald, and J. Rissanen. The Minimum Description Length Principle. Adaptive computation and machine learning. MIT Press, 2007. ISBN 9780262072816. URL htttp::/bbokss goog le. co. uk www.u///ooaoe bookkokk ? 6 ToUrBgC
Prahladh Harsha, Rahul Jain, David McAllester, and Jaikumar Radhakrishnan. The communica- tion complexity of correlation. In Twenty-Secone Annual IEEE Conference on Computational Complexity (CCC'07). pp. 10-23. IEEE, 2007.
Marton Havasi, Robert Peharz, and José Miguel Hermández-Lobato. Minimal random code learning: Getting bits back from compressed model parameters. NIPS workshop on Compact Deep Neural Networks with industrial application, 2018.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017.
Geoffrey Hinton and Drew Van Camp. Keeping neural networks simple by minimizing the descrip- tion length of the weights. In in Proc. of the 6th Ann, ACM Conf. Computational Learning Theory. Citeseer, 1993.
Quan Huynh-Thu and Mohammed Ghanbari. Scope of validity of psnr in image/video quality sessment. Electronics letters, 44(13):800-88) 8010,-00,22000 2008.
Nick Johnston, Damien Vincent, David Minnen, Michele Covell, Saurabh Singh, Troy Chinen, Sung Jin Hwang. Joel Shor, and George Toderici. Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 06 2018.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference Learning Representations, 12 2014.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. International Conference Learning Representations, 2014.

---

Under review as a conference paper at ICLR 2020
Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Con- ditional probability models for deep image compression. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 06 2018.
David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for learned image compression. In Advances in Neural Information Processing Systems, pp- 10771-10780, 2018.
Oren Rippel and Lubomir Bourdev. Real-time adaptive image compression. In Proceedings of the 34th International Conference on Machine Learning Voluue 70, pp. 2922-2930. JMLR. org, 2017.
Jorma Rissanen and Glen Langdon. Universal modeling and coding. JEEE Transactions on Infor- mation Theory, 27(1):12-25 1981.
Casper Kaac Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. How to train deep variational autoencoders and probabilistic ladder networks. In 33rd International Conference on Machine Learning (ICML 2016), 2016.
Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár. Lossy image compression with compressive autoencoders. International Conference Learning Representations, 2017.
George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, and Michele Covell. Full resolution image compression with recurrent neural networks. Proceed- ings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5306-5314, 2017.
James Townsend, Tom Bird, and David Barber. Practical lossless compression with latent variables using bits back coding. arXiv preprint 1rXiv:1901.04666 2019.
Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality assessment. In The Thirty-Seventh Asilomar Conference on Signals, Systems Computers, 2003, volume 2. pp. 398--102 Icee, 2003.
Hang Zhao, Orazio Gallo, luri Frosio, and Jan Kautz. Loss functions for neural networks for image processing, arXiv preprint urXiv; 10Xx::51/1008660 1,08611, 2015.
12

---

Under review as a conference paper at ICLR 2020
A FURTHER COMPARISONS
Figure 4: Reconstruction comparison for kodim05, .pg.. Top: original image. Bottom: Recon- struction using our PLN with 3 = 600.
13

---

Under review as a conference paper at ICLR 2020
lived
(S Gelienfobriiaation
lived
eiifinjobriiaation
Worlk
Figure 5: Reconstruction comparison for kodi 10 8 -png Top: original image. Bottom: Recon- struction using our PLN with 3 1200.
14

---

Under review as a conference paper at ICLR 2020
Figure 6: Reconstruction comparison for kodim13 3.g.. Top: original image. Bottom: Recon- struction using our PLN with 5 2400.
15

---

Under review as a conference paper at ICLR 2020
MS-SSIM comparison for kodim005
PSNR comparison for kodim05
21
203
21
MS. SS-.SSMM (dB)
Theoretical
Thearetical
Actual
Actual
Ballé el
Ballé et
JPEG
JPEG
28
23
Bits / Pixel
Bits / Poxel
Figure 7: Rate-distortion curves for kodim05
MS-SSIM comparison for kodim08
PSNR comparison for kodim08
MS. SSS-SSMMO0 (dB)
PSNR
Thooretical
Thearetical
Actual
Actual
Ballé et a
Ballé
JPEG
JPEG
19
25
05
10
28
15
Bits / Pixel
Bits / Poxel
Figure 8: Rate-distortion curves for kodimO8
MS-S5IM comparison for k3dim13
PSNR comparison for kodim13
Iheoretical
Actual
202
Ballé al.
JPEG
MS. SSIM (dB)
875
PSNR 321
Thooretical
Actual
Ballé et a
JPEG
15
25
40
68
10
25
35
Bits / Pixal
Bits / Poxel
Figure 9; Rate-distortion curves for kodiml1
16


Reviews:
[{"review_id": "SkxaSBcaFH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper aims to circumvent the quantization step and associated gradient approximations for compression algorithms that make use of entropy coding for compression. Entropy coding requires a probability mass function over discrete symbols. As an alternative approach the authors adapt the MIRACLE algorithm by Havasi et al. (2018), which was originally used to compress Bayesian neural networks, to work for lossy source compression with probability density functions over continuous latent variables.\nThe algorithm is based on taking several importance samples from the prior p(z) with the encoding distribution q(z|x) as the target distribution. The number of samples is equal to the exponent of the KL between the encoding and prior distribution, which can quickly grow to an uncontrollably large number. The index of the sample with the maximum importance weight is used as the code for the original image. If both sender and receiver have access to the same random sampler, the receiver can reproduce the sample by drawing a number of samples equal to this index, and decoding the last sample. Similar in spirit to Havasi et al, the authors take several steps to ensure that the KL divergence (and therefore the number of samples) does not become prohibitively large. The compression performance is evaluated by training the proposed model and the competing neural network-based method [1] on the Clic dataset, and evaluating it on a subset of the images of the Kodak dataset. JPEG is also used as a baseline. The authors show results achievable if coding using the relative entropy was perfect (denoted with ‘theoretical’), and the practically achieved compression performance (‘actual’). \n\nDecision:\nWeak reject: although the idea of circumventing the quantization step required by the use of entropy coders is certainly valid and interesting, the results in the paper show that the resulting compression performance is worse than the competing method that does quantize. Moreover, the encoding time is significantly longer than this same baseline. \n\nSupporting arguments for decision:\nAlthough the motivation for circumventing the quantization step seems plausible, the authors show no evidence that the competing method [1], which does perform a post-training quantization step, actually suffers from it. The authors even state on page 8 that “ Most notably though, they only used the continuous relaxation during the training of the model, thereafter switching back to quantization and entropy coding, which, they show does not impact the predicted performance, and hence confirming that their relaxation during training is reasonable.“ If post-training quantization is reasonable, then this overthrows the entire motivation. More importantly, the “theoretically” achievable results of the proposed method in seem only competitive in the low-bit rate regime and worse in the higher bit rate regimes. Even more, the “actual” practically achieved compression results are worse than [1] and also considerably worse than the “theoretically” achievable compression results. The authors do provide a reason for why the “theoretical” and “actual” results are so far apart, but are unfortunately not able to overcome this issue.\nIn the conclusion the authors honestly admit that the runtime of their method is much slower than the competitors (1-5 min for proposed method vs ~0.5 s  for [1] for encoding times). I appreciate that the authors mention this. The authors then state “improving the rate factor and the run-time does not seem too difficult a task, but since the focus of our work was to demonstrate the efficiency of relative entropy coding, it is left for future work.” I do not think this paper demonstrates the efficiency of relative entropy coding, the results simply don’t support this claim, and I therefore think that stating that the issues seem not too difficult to overcome is insufficiently convincing. \n\nThe quality of the empirical study can be improved. Another neural network-based compression baseline would make the empirical evaluation of the proposed method more insightful. Now we only see that the result is worse than [1], but it would be good to know how it compares to other baselines such as [2]. Furthermore, the paper does not show compression results aggregated over the entire Kodak dataset, but rather picks 2 images for the main part of the paper, and shows 3 in the appendix. Showing aggregate results gives a more robust estimate of the performance. Individual image results can just be put in the appendix. \n \n\nAdditional feedback to improve paper (not part of decision assessment):\n- In section 5 on the dependency structure of latents in the ladder VAE: is it really necessary to indicate the dependency structure with “topological structure”? Seems unnecessary to me as dependency structure is a clear enough description already without making it sound overly complicated.\n- Page 9: “Further, our architecture also only uses convolutions and deconvolutions as non-linearities.” Convolutions and deconvolutions are not non-linearities.\n- Fig 2: I’m not sure if this figure is relevant enough for such a prominent placement in the paper. It doesn’t discuss anything relevant to the contributions claimed in this paper. \n- I can’t find a definition of O in line 3 of “procedure” in algorithm 2 and in the return statement.\n\n\n[1] Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with a scale hyperprior. ICLR 2018.\n[2] Lucas Theis et al.  Lossy Image Compression with Compressive Autoencoders. ICLR 2017\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJgHF8Qt5B", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studied the image compression problem. Specially, the authors proposed to use neural networks to act as encoder / decoder. The training consists of minimizing the distortion of reconstruction and the difference between approximated posterior and true distribution. The output of encoder is sampled by the proposed relative entropy coding, which extended a previous method by introducing adaptive grouping for acceleration.\n\nIn summary, this paper gets rid of commonly used quantization techniques in image compression by using an approximate importance sampler which produces the encoding of images in a non-deterministic manner. With the construction of parameterized encoder / decoder, end-to-end training is conducted by popular gradient descent.\n\nHere is a question:\nIn experiments, the authors mentioned that the architecture is borrowed from another work. My question is how neural network architecture affects the performance? In other words, how to ensure that the performance is not obtained from the power of the backbone but from the proposed method itself. Are there any possible experiments which can be conducted to show the effectiveness by using different architectures?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HkgqED8k5H", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a new image compression method that does not require quantizing the encoded bits in an auto-encoding style image compression model. The method builds on a VAE with image x, code z, and posterior q(z | x). Instead of directly encoding z, the proposed method samples z_{c^\\star} from posterior and store c^\\star as the compressed representation. A decoder then reconstruct an approximation of x from z_{c^\\star}. The authors show that this framework is bits-back efficient and draw connections to prior theoretical results. Experiments were conducted on the Kodak dataset based on the model of Balle et al. The proposed method works only slightly worse than Balle et al. at low bit-rate region, but the gap becomes larger in higher bit rate regions. \n\nThe method is technically sound and the paper is clearly written. My main concerns fall in practical aspects. In Figure 3, for 3 out of 4 images, the theoretical upper bound of the proposed method still do not outperform Balle et al. This suggests limitations of the proposed method. Discussion on the limitations of the method is limited. The results also beg the question: How much does quantization in existing methods impact performance, and how much will fixing this benefit the overall system. Finally, in my opinions, in some sense the sampling of z_{c^\\star} is also a form of quantization. Does drawing different samples from posterior leads to different reconstructed images? If it does, doesn't it also suffer from similar limitations as existing \"quantization\" methods? \n\nOverall, I think the direction of the proposed method has good potential, but it also leaves important questions unanswered. I think this paper will benefit from additional revisions. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1x_DUuhcr", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for lossy image compression. Based on the encoder-decoder framework, it replaces the discrete codes by continuous ones, so that the learning can be performed in an end-to-end way.\n\nOverall, I think the current version should not be accepted. \n\nThe detailed comments are as follows.\n\n1. The novelty is not clear. Using continuous latent space has been analyzed for years. So does the negative beta-ELBO . Section 4 starts with importance sampling, then some implementation compromises, which makes the efficacy of resultant method unclear.\n\n2. You mention that REC achieves bits-back efficiency according to (Hinton 1993). However, how is c* selected in their paper? Now that you use importance sampling, does this still hold?\n\n3. The experiments are very insufficient. It only compares with one method, which is not enough. From the main content, the proposed method is improved upon (Havasi 2018). But it is not compared.  Let alone the other methods mentioned in the related works. PNG and JPEG should also be compared. \n\n4. For the reconstruction, it is better to measure the quality quantitively by PSNR, and mark the Bits/Pixel. \n\n5. Ablation study is also needed. PLN without your contributed part should be evaluated alone. So far, I cannot tell how your method works and which part works. \n\n6. There are many broken sentences and typos.\n\n7. Some statements such as the application parts should be properly cited.\n\nLast, the paper may be entitled as 'image compression without quantization' instead. ", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for lossy image compression. Based on the encoder-decoder framework, it replaces the discrete codes by continuous ones, so that the learning can be performed in an end-to-end way. The idea is interesting, but the motivation is based on a quantization "problem" that the authors show no evidence the competing method is actually suffering from. It is thus unclear how much does quantization in existing methods impact performance, and how much will fixing this benefit the overall system. Also, the authors may add some discussions on whether the proposed sampling of z_{c^\star} is indeed also a form of quantization.

Experimental results are not convincing. The proposed method is only compared with one method. While it works only slightly worse at low bit-rate region, the gap becomes larger in higher bit rate regions. Another major concern is that the encoding time is significantly longer. Ablation study is also needed. Finally, the writing can be improved.
Decision: Reject