Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: A well-known issue in learned compression is that due to inconsistencies in the implementations of low-level floating-point arithmetic across different hardware architectures, catastrophic decoding errors might occur for the compressed data. The authors propose several different ways to quantize the weights and activations of the models to make the encoding and decoding procedures use integer arithmetic only. Since the implementation of integer arithmetic is consistently implemented across different hardware, this solves the decoding issues, while also providing some speed-ups. The authors demonstrate the efficiency of their method by quantizing some state-of-the-art image compression methods and show that the performance of the quantized models does not degrade much.



main_review: ## Strengths
   - In my view, the most important contribution of the paper is to demonstrate that state-of-the-art image compression methods can be appropriately quantized to retain essentially the same compression performance as their unquantized counterparts
   - The extension of the look-up table technique of [1] to Gaussian mixture models is interesting and clearly useful

## Weaknesses
   - I am not convinced that the authors can claim the application of post-training quantization (PTQ) to achieve cross-platform consistency for image compression models as a contribution. As I understand, this idea was first proposed in [2] and further developed in [1], and in both papers, the authors demonstrate the efficiency of PTQ. In fact, the authors seem to acknowledge this a couple of paragraphs above! I believe that what the authors can claim is the application of PTQ to models using autoregressive context models. Could the authors please clarify if my understanding is correct? In case it is, the claim should be changed to better reflect the contribution of the paper.
   - The results of the paper are not compared with relevant methods, most notably [1] and [2]. In particular, the authors present the performance of their method on the method of [3] but do not show the performance of e.g. [2] on the same model.
   - In the introduction, the authors make the following claim "However, we find that on more complex models with context modelling like Minnen et al. (2018) and Cheng et al. (2020), adopting this integer network approach cannot keep the performance loss negligible." - I could not find any reference to support this claim in the main text or the appendix. Could the authors point to any reference that would support this?

### Minor issues
   - The 1st and 2nd sentences of the abstract seem to be inconsistent with each other, the 1st implying that LIC is practical, while the 2nd implying that it is not.
   - There are many grammatical mistakes in the text that need to be fixed
   - The abbreviation "LUT" (which I believe stands for look-up table) is undefined in the text
     
## Questions
   - The authors claim that they use quantization not for the purpose of compression but as a way to switch calculations to integer arithmetic for cross-platform consistency. In this case, why do the authors still prefer 8-bit quantized representations for the activations and weights?
   - While the authors method to quantize the log-standard deviations is sensible, could they elaborate on why the method of [1] is not adequate? In the paper the authors claim that the method of [1] is moderately efficient, so I am unsure what benefit their method brings. 

## After Rebuttal
The authors have addressed my main concerns, and I, therefore, decided to raise my score.
   

[1] H. Sun, L. Yu, and J. Katto. Learned image compression with fixed-point arithmetic. 2021 IEEE Picture Coding Symposium

[2] H. Sun, Z. Cheng, M. Takeuchi, and J. Katto, End-to-end learned image compression with fixed point weight quantization, 2020 IEEE International Conference on Image Processing 

[3] J. Balle, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston. Variational image compression with a scale hyperprior. ICLR 2018

summary_of_the_review: I currently believe that while the work is useful to the community (see 1st point in "Strengths"), I think in its current form the paper makes a few considerable claims that are not supported. 

If the authors can address my main concerns, I will reconsider raising my score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Learned image compression has a known issue that the nondeterminism of the floating point operation makes it impossible to recover the compressed data on different platforms.

This paper experimentally shows in compression models using context modeling (Minnen et al. (2018)) that it is possible to recover the data on different platforms even when using the known quantization method called PTQ. (Table.1 )

Referring to the method of computing STD using pre-computed sampling points σˆ by Sun et al. (2021), the authors proposed a method called Binary Logarithm based STD discretization.
main_review: Pros:

1. Authors experimentally showed that PTQ can be used to prevent data corruption (Table 1). The effect on RD performance is also shown in Figure 6.
2. I think cross-platform support for learned image compression is an important issue for practical applications.


Cons:

1. Issues to be addressed are unclear:
- The authors state in chapter 1: "However, we find that on more complex models with context modeling like Minnen et al. (2018) and Cheng et al. (2020), adopting this integer network approach cannot keep the performance loss negligible. ", it is interesting but I could not find evidence for this; if the RD rate performance of the integer network is degraded, it should be shown in experiments.

2. Comparisons with previous methods (e.g. Balle ́ et al. (2019) and Sun et al. (2021)) are not sufficient:
- If there are any advantages of this research compared to the previous methods (e.g., if previous methods are not able to prevent data corruption in some image compression models), I think you should clearly show that comparison.
- This paper proposes a new method (Binary Logarithm based STD discretization) to aim for more hardware-friendly. It is interesting but unfortunately, the experiments in Table 2 do not seem to show a comparison with Sun et al. (2021), which is written like a baseline in chapter 4.

3. Other unclear points:
- Is Table 1 the result of applying the LUT (proposed method in chapter 4)? If the error rate is 0% with only PTQ without applying the LUT, why is LUT necessary? Or, if you were applying the LUT, what happens if you do not apply the LUT but apply the PTQ? I am curious about the contribution for prevent data corruption of each of the PTQ and LUT.
- Is the time shown in Table 2 the average time for the entire data set?
summary_of_the_review: The authors experimentally showed that PTQ can be used to prevent data corruption of learned image compression, but the value of this research is not clear due to the unclear target issues, insufficient comparison with previous methods (e.g. Balle ́ et al. (2019) and Sun et al. (2021)), and some unclear points, so I considered that currently this paper did not meet the conditions for acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper tried to solve the non-deterministic issue for the learned image compression and reduce the inconsistent cross-platform probability prediction. It is a practical problem in the development of learned image compression. The authors utilize some off-the-shelf techniques in the model compression and revisit the non-deterministic issue from the perspective of model compression. The experimental results are convincing. 
main_review: Strengths:
1.  It is interesting to see the model compression techniques could be beneficial for the learned image compression in terms of cross-platform development. Instead of designing new model components, the proposed approach is more flexible. 
2. The experimental results are encouraging. It seems that this quantization method does not introduce too much performance degradation. 

Weaknesses:
Since the major techniques are borrowed from the quantization techniques in model compression, so the technical contribution may be limited.  The authors are suggested to make a clear description for the difference between the techniques used in this paper and that in model compression.

Besides, the authors only provide the RD curve below ~1bpp. How about the compression performance at high bitrate?
summary_of_the_review: I tend to accept this paper though I have some concerns about the technical novelty.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a network quantization method for robust learned image compression.
Recent deep-learning-based approaches for image compression usually adopt the cumulative distribution functions (CDF) for both encoding and decoding processes.
However, the CDFs are usually implemented in devices (e.g., CPU, GPU) with different approximations.
The different values from approximated CDFs incur significant decoding errors.
This paper alleviates this problem by quantizing the feature maps and weights of the encoder and decoder to 8-bit.
main_review: It is interesting that post-training quantization is enough for the model robustness of learned image compression. 

However, this paper requires polishing for the presentation and comparisons and the writing clarity should be improved.
For example, the term 'cross-platform inconsistency' is hard to understand until reading the Appendix and the paper of Balle et al. (2019).
Figure 1 and 2 present the problem statement and the methods in previous works which limit to emphasize the contributions of this paper.
Experiments do not analyze the contributions of the proposed method described in Section 3.
More importantly, this paper misses the comparisons to the method of Balle et al. (2019) for the model robustness across devices.

On the other hand, the proposed method described in Section 4 seems to be an alternative to post-training quantization since it is a predetermined CDF for different devices that minimizes the cross-platform inconsistency.
Based on the alternative, I'm curious about the importance of the cross-platform inconsistency which can easily be solved by saving the same CDF as metadata and using it across devices.

While quantizing weight with per-channel scaling factors, which granularity is used for the feature map quantization?
Does the proposed quantized model have actual latency acceleration with the per-channel weight quantization?


summary_of_the_review: This paper presents a simple practical solution, post-training quantization, for robust learned image compression using neural networks.
However, the contributions of this paper are marginal since the problem statement and solution are overlapped with the previous work (Balle et al., 2019).
Moreover, this paper presents limited comparisons to the work of Balle et al. (2019) while Section 1 describes its failure cases.
The authors should thoroughly state the differences and improvements of the proposed method to the work of Balle et al. (2019).
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Quantization issues around entropy coding in neural image compression can lead to non-determinism across platforms and runtimes (i.e. CPU to GPU or even different drivers on the same GPU). Many works in the are perform quantization during training, which can make for more complex or longer training setups. This paper shows the effectiveness of post-training quantization techniques, shows the results are successful even on context and GMM models, and introduces a faster entropy parameter discretization method.
main_review: Strengths: 
The authors do a great job of immediately conveying the problems in image compression (designing deterministic decoders and the speed performance of those decoders).

The results are strong, they show very little to no loss until high bitrates due to this quantization technique.

The reforming of the entropy parameter discretization to be more computationally efficient shows impressive runtime improvements on this quantization process with no RD loss.

Additionally, the appendices were well written and contain many additional derivations and detailed descriptions of the quantization algorithms applied.


Weaknesses:
Rate-distortion performance is rarely shown outside the 1.0 bpp range. Since the losses appear to be more evident at higher bitrate, it would be instructive to know if PTQ is as successful outside of the low to medium quality range.

How much of a difference does one LUT make? The baselines are compared against a 64 level table and the new method proposes a much faster 65 level table. It is an obvious improvement in runtime due the formulation, but are there any significant RD gains for having another level.


Questions/Feedback:
Table 2: I'm not 100% sure what is being compared here. Is this a hyper synthesis with scale, scale-mean, context, GMM? Is there a reference for the popularly vectorized version? Was your implementation done in C++ or inline Assembly to guarantee the fast assembly instructions were properly generated?

In Section 3.1: "a grid search minimizing the reconstruction error", how much does this grid search help in the quantization process? Does this need to be done on a per loss function basis or per model? i.e. if I train a model for a specific lambda with a specific architecture, will I generally use the same quantization step, or is this more sensitive to individual runs in compression models?
summary_of_the_review: This is a well written paper that shows the effectiveness of existing post training quantization techniques when applied to multiple neural image architectures. In addition to showing how well this PTQ techniques perform, a more computationally efficient entropy parameter discretization technique is introduced.
review_title: null

Meta-review: The paper considers quantization issues for learned neural-network-based image compression methods. 
Many works on the topic incorporate quantization into the training of the method. The paper provides evidence that post-training quantization is effective. Specifically, the paper demonstrates that state-of-the-art learned image compression methods can be quantized post-training and retain a very similar level of compression performance. The paper argues that this is important in particular for cross-platform applications, where an image is decompressed on different architectures. Finally, the paper proposes an approach to discretize entropy parameters.

The reviewers raised the following concerns. 
- Reviewer 2XDr is concerned about the application of post-training quantization being a contribution, since post-training quantization has already been studied in [1] (and in the recent paper [2] that can be considered as concurrent work). The authors response is that the methods in [1] has extra overhead and clarify how the prior work is in fact different. This addressed the reviewer's concern, and the reviewer raised their score.

- Reviewer eyVf finds the comparisons with previous methods to be insufficient, and in general find the value of the research unclear, as the goals are not sufficiently specified. The authors clarified, and the reviewer was satisfied with the response and raised the rating to marginal above the threshold. 

- Reviewer L7dn tends towards acceptance, but has concerns about the technical novelty, that are unspecified unfortunately.

- Reviewer oV3R argues that the solution is marginal relative to prior work, and votes to reject the paper. The authors responded why they think it isn't, and also wrote a private letter to the area chair in which they explain why they think that reviewer's oV3R should not be taken into account. I agree with the authors that the paper under review provides a step relative to Balle et al (2019), and that the writing of the paper is not an issue; however, the reviewer's overarching point is that the overall contribution is marginally significant when taking the prior work by Balle et al (2019) into account and this is the sentiment of other reviewers as well.

- Reviewer GrpS, an expert on image compression, leans towards acceptance and argues that the results are strong as they show little to no loss due to the quantization technique, but also rates the contribution to only be marginally significant and novel, and raises a few questions and issues, to which the reviewers responded. 

This paper is really borderline. Four out of five reviewers rate this paper as marginally above the acceptance threshold. The consensus is that while the experiments and claims are correct, the contribution is only marginally significant or novel, in particular, relative to prior work, and therefore I recommend rejecting the paper. I would, however, not be upset if it would be accepted.
Decision: Reject