Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a centered symmetric quantizer (CSQ) that can map floating point tensors into zero-centered quantized integer values. The CSQ method works well for ultra-low bit quantization such as 2-bit.
The authors also propose a binary coding method to run efficiently on hardware.
main_review: I think the paper is generally well written.

The authors show that CSQ can be efficiently expressed by binary representations, and then be calculated using XNOR operations. I appreciate the details and analysis provided in the main text and supplementary materials.

CSQ has quantization results for mobilenets, which are compact models that generally are harder for quantization. Experiments on large datasets (such as ImageNet) are also helpful to validate the effectiveness of CSQ.

In addition, some ablation study are meaningful, such as the BRECQ-CSQ experiments in the supplementary material.

Despite the merits, I think the paper still has the following weaknesses:
1. The scope of application is quite limited. From the paper we can see that CSQ can consistently outperform CLQ on 2-bit, but is not guaranteed to be better on 3-bit or 4-bit. Plus, there are other orthogonal methods that can alleviate the quantization degradation on 2-bit. Although the authors claim the difference between CSQ and previous methods (Section 2.3, Stochastic precision ensemble etc.), the novelty of CSQ is not particularly strong.
2. In Table 5, the ESQ results mentioned on the caption are missing.
3. The experimental results of CSQ didn't show great improvement over CLQ. For example, on ImageNet with 2-bit ResNet34, the gain is 0.26%. I wonder if the accuracy gain will remain if advanced quantization-aware training methods (with or without distillation) or mixed-precision methods are applied together with CLQ/CSQ.
4. Some state-of-the-art results need to be compared with, for example:
[1] Distribution-Aware Adaptive Multi-Bit Quantization.
5. The ResNet34 baseline (73.3%) used in the experiment is not state-of-the-art (>74%). I suggest using ResNet50 which is a more common and standard choice.
summary_of_the_review: I think CSQ is a simple but effective method that can be helpful for ultra-low bit quantization. However, the novelty and application scope are limited and the experiments have room for improvement. Consequently, I think the paper is marginally below the acceptance threshold.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a centred symmetric quantizer(CSQ), a new symmetrical quantization scheme.
main_review: Strength: A new idea of applying centered symmetric quantizer for model quantization.

Weakness:

1. Speedup setting: The authors only compare the single matrix multiplication runtime, which does not necessarily represent the speedup and the accuracy of the whole network [1]. 

2. Due to the new quantization level, the authors are encouraged to conduct experiments on other tasks (e.g., detection and NLP tasks.), which can present the generalization and robustness of CSQ.

3. The accuracy improvement is marginal than CLQ on the ImageNet dataset.

4. If people optimize sparse mand quantized models jointly [2, 3], is the CLQ with sparsity equal to CSQ? 


[1]. MQBench: Towards Reproducible and Deployable Model Quantization Benchmark

[2]. Compressing Deep Neural Networks with Pruning, Trained

[3]. A unified framework of dnn weight pruning and weight clustering/quantization using admm
summary_of_the_review: The analysis and experiments have some issues (see weakness). I will give a borderline due to those concerns and change it accordingly based on rebuttal.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper analyzed the difference in quantization methods between the conventional linear quantizer (CLQ) and centered symmetric quantization (CSQ). The authors explained that CSQ might exploit the limited quantization states better to represent the data than CLQ, with supporting analysis on representation capacity. The authors further proposed a bitwise implementation of CLQ and CSQ. The evaluation in terms of the quantization accuracy and GPU implementation speed is given for their analysis.
main_review: This paper seems to be the first to deep dive into the difference between CLQ and CSQ, although these quantization techniques are not new. However, the resulting analysis seems to be straightforward for the following reasons:

- The conclusion that CSQ could be (marginally) better seems to be as expected, since CLQ might not be efficient for representing the signed (or centered) distribution. Although this paper explicitly revealed this fact, it does not seem to provide new insights.

- The analysis based on the representation capacity seems to be weak. Table 3 reveals that the number of distinct output states varies for different combinations of quantizers. Therefore, it does not necessarily prioritize CSQ to CLQ. Furthermore, the importance of the representation capacity is not clear; although CSQ-CLQs has a 50% larger representation capacity for the signed activation, its accuracy gain seems to be marginal (~0.4% from Table 5,6). 

- The accuracy results reported in Table 6 seem to be misleading. 1) The authors did not put the accuracy results of LSQ (= CLQ case) for ResNet18 and ResNet34 (67.6% and 71.6%, respectively), which are significantly higher than CSQ. 2) In the case of MobileNetV2, the reported accuracy results CSQ seem to be significantly lower than the state-of-the-art (e.g., PROFIT(ECCV2020) 4-bit MobileNet-V2 = 69.06% compared to CSQ=66.98%). 


summary_of_the_review: This paper seems to provide a better understanding of the low-precision (= 2-bit) quantizers, but the discovery itself seems to be marginally significant. Therefore, I am inclined toward rejection.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper points out that conventional uniform quantization methods do not allocate the same quantization level in the positive and negative areas and proposes Centered Symmetric Quantizer (CSQ), a zero-centered symmetric quantizer, to address the resulting degradation at very low precision. It also presents a method of mapping the proposed CSQ to BNN hardware through bit-wise operations. This paper claims that no specific analysis of CSQ has been made in previous studies and that low-precision quantization results in poor performance due to the allocation of asymmetric quantization levels in the positive and negative sides of the distribution.
main_review: This work's main idea is well-written and pretty straightforward by comparing the proposed CSQ with CLQ (Conventional Linear Quantizer) and explaining that some quantization representations are wasted in the conventional quantization methods. The idea of using a zero-centered quantizer is ok, but it might need more novelty because it has already been used in several previous works. That's why the authors claim that this paper has other contributions in that they propose an explicit quantizer function for CSQ and provide some analysis on the effectiveness of such a scheme at very low precision. However, it is not clear to the reviewer if their claim is technically sound because theoretical analysis or empirical evidence does not support their main contributions strongly. Moreover, there are some cons to demonstrating its effectiveness in the experimental results. Here are a few details regarding this concern. 

1. Apart from explaining the problem of CLQ and the expressive power of existing quantization methods, the reviewer is unsure about the necessity of allocating symmetric quantization levels. The learned weight distribution as described in Figure 2 of DFQ [1] may not be symmetrical. Because the weight distribution being learned may be similar to the normal distribution but not a perfectly symmetric structure, allocating symmetric quantization levels need not be assigned to the positive and negative regions. Previous QAT studies such as clipping methods [2], QIL [3], or LSQ [4] have been attempted to reduce the dynamic range of non-uniform quantization or to learn the quantization intervals for this reason. It seems that applying CSQ is not always better than the previous approaches in the above situation. The author should explain first the necessity of allocating symmetric quantization levels by comparing these methods to claim the motivation of this study. Is there any particular reason why the authors excluded non-uniform quantization methods or other QAT approaches to solve the same problem from the discussion?

2. In the experiment, it is explained that CSQ is applied only to weight quantization. However, it doesn't seem easy to conclude that the results presented in Table 5, Table 6, and Figure 3 are entirely caused by symmetrical quantization of CSQ. Nevertheless, it should be possible to clearly explain that CSQ majorly contributes to these results, excluding the effects of activation quantization and learning weights. 

3. Overall, the improvement due to CSQ seems to be minor and insignificant for both CIFAR-10 and ImageNet because there is little performance gap when looking at the results presented in Table 5 and Table 6 and results of applying PTQ covered in the appendix. Therefore, it seems not convincing to validate the effectiveness of the proposed CSQ scheme.

4. Some recent works (Boo et al.(2021), Lee et al.(2021), Chen et al.(2021)) have used a quantizer that results in zero-centered quantization levels similar to the one proposed in this paper. So why not compare the experimental results with them?

Minor Comments: 
1. The second equality in equation (12) is not intuitive, so it needs to be proved. 
2. Equation (30) on page 6 should be replaced with equation (14). 
3. There are a few typos.

References: 
[1] M. Nagel, M. V. Baalen, T. Blankevoort, and M. Welling, "Data-free quantization through weight equalization and bias correction," ICCV 2019. 
[2] R. Banner, Y. Nahshan, E. Hoffer, and D. Soudry, "ACIQ: Analytical Clipping for Integer Quantization of Neural Networks," arXiv e-print, arXiv:1810.05723, Oct 2018.
[3] S. Jung, C. Son, S. Lee, J. Son, J. Han, Y. Kwak, S. Hwang, and C. Choi, "Learning to quantize deep networks by optimizing quantization intervals with task loss," CVPR 2019.
[4] S. Esser, J. McKinstry, D. Bablani, R. Appuswamy, and D. Modha, "Learned step size quantization," ICLR 2019.

summary_of_the_review: To sum up, this paper seems to have several technical flaws for the claims to be well supported.
review_title: null

Meta-review: ### Description
The paper investigates the choice of a fixed quantization grid for weights. Namly, the paper observes that symmetric uniform quantization levels such as {-1.5,-0.5,0.5,1.5} lead to better results than non-symmetric ones, e.g. {-2,-1,0,1}. While it is a small thing, it can be appreciated that it is investigated systematically and pedantically, proposing an explanation and showing experimentally that the effect is constantly present in favour of symmetric quantization. While the improvement is small, it comes almost at no cost. A part of the contribution proposes an efficient implementation. 

### Decision
Reviewers and AC came to a consensus that the contribution of the paper is marginal. Symmetric quantization schemes themseleves were already employed by many models, albeit without analysis or even a discussion of such choice. The analysis presented in the paper was found unconvincing by the reviewers (see below). The efficient implementation follows from basic linear algebra (see below). The potential impact of the work was considered as limited due to a rather marginal observed improvement. The average rating of the paper was 4.5. Therefore must reject.

### Details
Regarding the proposed analysis of CSQ, it is not clear, why the number of quantization levels of an elementary product matters, given that these numbers are then summed over all corresponding input channels and spatial dimensions of a convolution kernel applied at a single location. It is questionable whether the number of these quantization levels indeed corresponds to the representation capacity. Finally, the paper misses to demonstrate the effect on binary (1 bit) networks. In this case the standard approach is to use {-1,1} weights and {-1,1} activatinos. The paper could investigate the case of {0,1} activations, where there would be 50% more unique possible outputs from the product, namely {-1,0,1} to validate their hypothesis. If the hypothesis holds, an improvement in the binary case would be observed. This is important since the binary case is know to be the hardest and since the respective recommendation of representations would be non-standard. It could be further questioned why the distribution of real-valued weights has any relevance (such as in the arguments in appendix E) if the model is trained from scratch? A training method need not keep any real-valued latent weights in the first place.

The technical part in section 5 "efficient realization" adds very little, if anything, to the paper's contribution. A simple linear algebra suffices to see that 

$(W-0.5) \ast x = W* x - 0.5 I \ast x,$

where $I$ is the kernel of ones of the same shape as $W$. It is clear that the convolution $I \ast x$ can be implemented efficiently (e.g. it is just a sum over channels followed by a separable spatial only convolution) and is not a bottleneck and. The final detail such as whether to slice by bits and use popcount for it or to use 8-bit addition, depend very much on the choice of the bit-packed representation and the hardware available. It would be known to engineers in the field how to implement it efficiently.
Decision: Reject