Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The paper proposes a method for lossy image compression. Based on the encoder-decoder framework, it replaces the discrete codes by continuous ones, so that the learning can be performed in an end-to-end way.

Overall, I think the current version should not be accepted. 

The detailed comments are as follows.

1. The novelty is not clear. Using continuous latent space has been analyzed for years. So does the negative beta-ELBO . Section 4 starts with importance sampling, then some implementation compromises, which makes the efficacy of resultant method unclear.

2. You mention that REC achieves bits-back efficiency according to (Hinton 1993). However, how is c* selected in their paper? Now that you use importance sampling, does this still hold?

3. The experiments are very insufficient. It only compares with one method, which is not enough. From the main content, the proposed method is improved upon (Havasi 2018). But it is not compared.  Let alone the other methods mentioned in the related works. PNG and JPEG should also be compared. 

4. For the reconstruction, it is better to measure the quality quantitively by PSNR, and mark the Bits/Pixel. 

5. Ablation study is also needed. PLN without your contributed part should be evaluated alone. So far, I cannot tell how your method works and which part works. 

6. There are many broken sentences and typos.

7. Some statements such as the application parts should be properly cited.

Last, the paper may be entitled as 'image compression without quantization' instead. 
review_title: Official Blind Review #5

final_score: rating: 6: Weak Accept
review_content: This paper studied the image compression problem. Specially, the authors proposed to use neural networks to act as encoder / decoder. The training consists of minimizing the distortion of reconstruction and the difference between approximated posterior and true distribution. The output of encoder is sampled by the proposed relative entropy coding, which extended a previous method by introducing adaptive grouping for acceleration.

In summary, this paper gets rid of commonly used quantization techniques in image compression by using an approximate importance sampler which produces the encoding of images in a non-deterministic manner. With the construction of parameterized encoder / decoder, end-to-end training is conducted by popular gradient descent.

Here is a question:
In experiments, the authors mentioned that the architecture is borrowed from another work. My question is how neural network architecture affects the performance? In other words, how to ensure that the performance is not obtained from the power of the backbone but from the proposed method itself. Are there any possible experiments which can be conducted to show the effectiveness by using different architectures?

review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: The authors propose a new image compression method that does not require quantizing the encoded bits in an auto-encoding style image compression model. The method builds on a VAE with image x, code z, and posterior q(z | x). Instead of directly encoding z, the proposed method samples z_{c^\star} from posterior and store c^\star as the compressed representation. A decoder then reconstruct an approximation of x from z_{c^\star}. The authors show that this framework is bits-back efficient and draw connections to prior theoretical results. Experiments were conducted on the Kodak dataset based on the model of Balle et al. The proposed method works only slightly worse than Balle et al. at low bit-rate region, but the gap becomes larger in higher bit rate regions. 

The method is technically sound and the paper is clearly written. My main concerns fall in practical aspects. In Figure 3, for 3 out of 4 images, the theoretical upper bound of the proposed method still do not outperform Balle et al. This suggests limitations of the proposed method. Discussion on the limitations of the method is limited. The results also beg the question: How much does quantization in existing methods impact performance, and how much will fixing this benefit the overall system. Finally, in my opinions, in some sense the sampling of z_{c^\star} is also a form of quantization. Does drawing different samples from posterior leads to different reconstructed images? If it does, doesn't it also suffer from similar limitations as existing "quantization" methods? 

Overall, I think the direction of the proposed method has good potential, but it also leaves important questions unanswered. I think this paper will benefit from additional revisions. 
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: Summary:
This paper aims to circumvent the quantization step and associated gradient approximations for compression algorithms that make use of entropy coding for compression. Entropy coding requires a probability mass function over discrete symbols. As an alternative approach the authors adapt the MIRACLE algorithm by Havasi et al. (2018), which was originally used to compress Bayesian neural networks, to work for lossy source compression with probability density functions over continuous latent variables.
The algorithm is based on taking several importance samples from the prior p(z) with the encoding distribution q(z|x) as the target distribution. The number of samples is equal to the exponent of the KL between the encoding and prior distribution, which can quickly grow to an uncontrollably large number. The index of the sample with the maximum importance weight is used as the code for the original image. If both sender and receiver have access to the same random sampler, the receiver can reproduce the sample by drawing a number of samples equal to this index, and decoding the last sample. Similar in spirit to Havasi et al, the authors take several steps to ensure that the KL divergence (and therefore the number of samples) does not become prohibitively large. The compression performance is evaluated by training the proposed model and the competing neural network-based method [1] on the Clic dataset, and evaluating it on a subset of the images of the Kodak dataset. JPEG is also used as a baseline. The authors show results achievable if coding using the relative entropy was perfect (denoted with ‘theoretical’), and the practically achieved compression performance (‘actual’). 

Decision:
Weak reject: although the idea of circumventing the quantization step required by the use of entropy coders is certainly valid and interesting, the results in the paper show that the resulting compression performance is worse than the competing method that does quantize. Moreover, the encoding time is significantly longer than this same baseline. 

Supporting arguments for decision:
Although the motivation for circumventing the quantization step seems plausible, the authors show no evidence that the competing method [1], which does perform a post-training quantization step, actually suffers from it. The authors even state on page 8 that “ Most notably though, they only used the continuous relaxation during the training of the model, thereafter switching back to quantization and entropy coding, which, they show does not impact the predicted performance, and hence confirming that their relaxation during training is reasonable.“ If post-training quantization is reasonable, then this overthrows the entire motivation. More importantly, the “theoretically” achievable results of the proposed method in seem only competitive in the low-bit rate regime and worse in the higher bit rate regimes. Even more, the “actual” practically achieved compression results are worse than [1] and also considerably worse than the “theoretically” achievable compression results. The authors do provide a reason for why the “theoretical” and “actual” results are so far apart, but are unfortunately not able to overcome this issue.
In the conclusion the authors honestly admit that the runtime of their method is much slower than the competitors (1-5 min for proposed method vs ~0.5 s  for [1] for encoding times). I appreciate that the authors mention this. The authors then state “improving the rate factor and the run-time does not seem too difficult a task, but since the focus of our work was to demonstrate the efficiency of relative entropy coding, it is left for future work.” I do not think this paper demonstrates the efficiency of relative entropy coding, the results simply don’t support this claim, and I therefore think that stating that the issues seem not too difficult to overcome is insufficiently convincing. 

The quality of the empirical study can be improved. Another neural network-based compression baseline would make the empirical evaluation of the proposed method more insightful. Now we only see that the result is worse than [1], but it would be good to know how it compares to other baselines such as [2]. Furthermore, the paper does not show compression results aggregated over the entire Kodak dataset, but rather picks 2 images for the main part of the paper, and shows 3 in the appendix. Showing aggregate results gives a more robust estimate of the performance. Individual image results can just be put in the appendix. 
 

Additional feedback to improve paper (not part of decision assessment):
- In section 5 on the dependency structure of latents in the ladder VAE: is it really necessary to indicate the dependency structure with “topological structure”? Seems unnecessary to me as dependency structure is a clear enough description already without making it sound overly complicated.
- Page 9: “Further, our architecture also only uses convolutions and deconvolutions as non-linearities.” Convolutions and deconvolutions are not non-linearities.
- Fig 2: I’m not sure if this figure is relevant enough for such a prominent placement in the paper. It doesn’t discuss anything relevant to the contributions claimed in this paper. 
- I can’t find a definition of O in line 3 of “procedure” in algorithm 2 and in the return statement.


[1] Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with a scale hyperprior. ICLR 2018.
[2] Lucas Theis et al.  Lossy Image Compression with Compressive Autoencoders. ICLR 2017

review_title: Official Blind Review #3

Meta-review: The paper proposes a method for lossy image compression. Based on the encoder-decoder framework, it replaces the discrete codes by continuous ones, so that the learning can be performed in an end-to-end way. The idea is interesting, but the motivation is based on a quantization "problem" that the authors show no evidence the competing method is actually suffering from. It is thus unclear how much does quantization in existing methods impact performance, and how much will fixing this benefit the overall system. Also, the authors may add some discussions on whether the proposed sampling of z_{c^\star} is indeed also a form of quantization.

Experimental results are not convincing. The proposed method is only compared with one method. While it works only slightly worse at low bit-rate region, the gap becomes larger in higher bit rate regions. Another major concern is that the encoding time is significantly longer. Ablation study is also needed. Finally, the writing can be improved.
Decision: Reject