Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: In this paper, the authors proposed a novel quantization method, additive powers-of-two quantization and show both the uniform quantization and powers-of-two quantization are the special case of this method. To train such a quantized network, new clipping function, and weight normalization are proposed. The numerical results show that the proposed method only introduces a small accuracy loss and sometimes even improve accuracy. Compared to the other methods, it also shows better performance. Overall, I think this work is valuable and may be considered for publication. But the reviewer is completely out of this neural network quantization area, and thus not very familiar with the related works.

The following are some more detailed comments:

1. On page 3, "when we increase the bit-width from b to b+1 ... be split into 2^b subintervals." Based on Equation 3, I think it might be 2^(b-1) +1 subintervals.

2. From Figure 1 (c), the introduced quantization method introduces non-monotonic interval steps, which is a little unintuitive. Can the authors explain if this can be further improved?

3. A very important theme of this paper is on hardware, however, I feel the paper doesn't have a satisfying discussion on the hardware implementation. In order to make the argument more solid, the authors may want to provide more discussion on the hardware implementation trade-offs. Also, a detailed comparison of the quantization size comparison should also be provided.

4. The authors claim the quantized network can sometimes achieve better performance. This statement needs to be further checked. Since the performance provided in the baseline Resnet models can be slightly improved with further training and training schedule tuning. Without a thorough optimization, such a claim might be misleading.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: 
The paper presents an approach based on power-two quantization to compress the weights of neural networks. The authors elaborate on a coding scheme that adds several quantized values with different log scales (typically 2^(-2k) and 2^(-2k+1) for varying k, for the sum of two log scales). More importantly, they emphasize on the importance of weight scaling and learning the clipping threshold. experiments are carried out with several versions of ResNet on CIFAR-10 and ImageNet, and the authors show better performances than competing baseline for a fixed quantization budget per float.

The results are better than the non-compressed baseline (CIFAR with 3 and 5 bit/flotat, ImageNet with 5bit/float.) This seems surprising. Is there any clear reason why? Shouldn't the performance decrease as a result of the compression?

While the results are good, a large part of the paper is dedicated to the idea of summing powers-of-two quantizers. The motivation comes from the sub optimality of uniform quantizers. Some ablation studies demonstrate that the scheme power-of-two is better than uniform. Nonetheless, the results in Table 2 indicate that uniform already achieves very good performances (already better than the non-compressed network for top-1 accuracy), so that most of the gain seems to come from weight normalization and learning the clipping threshold.  

Thus, the advantage of the exact scheme that is proposed compared to other methods is unclear to me. There already are well-known algorithms for non-uniform quantization (e.g., Lloyd). Lin et al. ("Towards Accurate Binary Convolutional Neural Network", cited in the paper) already addressed the non-uniform quantization by learning different binary bases. While the results of the paper are better than a number of previous results, it seems to me that most of the gain does not come from the quantizer but rather from the other elements of the method. 

The contribution then seems a bit incremental, as it is mostly weight normalization +  straight through estimator to learn the clipping constant by minimizing directly the loss function. One way to make the contribution stronger is to isolate this part to see its effect more generally on several non-uniform quantization schemes.

other comments:
- is there any (possibly intuitive) justification for the fact that the results of the compressed networks are better than the non-compressed ones?

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Summary:
The authors propose to compress Neural Networks (NNs) by quantizing their weights to sums of powers-of-two. This allows both to take the non-uniform distribution of the weights into account and to perform fast inference on dedicated hardware. 

Strengths of the paper:
- The problem is clearly stated (in particular the two questions about the clipping operation and the quantization levels are clearly presented in the Introduction). Similarly, the authors alleviate what they call the "rigid resolution" problem by using sums of powers-of-two and clearly explain their choice. 
- The paper addresses the inference time, which is an important metric. Indeed, researchers tend to focus on the size of the compressed weights as this is easily measurable and not questionable. Inference time however depends on the hardware but is crucial as compressed models often that run on embedded devices are often required to run in real-time (say image detection models). The paragraph "Computation" in Section 2.1 is therefore useful for the reader.

Weaknesses of the paper:
- Both the reparameterization of the clipping function and the weight normalization before quantization approaches seem not novel to me, see for instance: "Weight Normalization based Quantization for Deep Neural Network Compression", Cai et al. 
- The experiments are lacking the widely used ResNet-50 baseline (and Table 4 should be in the main paper instead of the appendix). Other tasks such as Image Detection (Mask R-CNN) could also strengthen the impact of the paper. 
- Moreover, unless I am mistaken, the authors "do not quantize the first and last layers". While not quantizing the first layer has no impact on the compressed size (weights of size 7x7x3x64 = 36 KB for a ResNet-18), not quantizing the last layer (ie the classifier, weights of size 512x1000 = 1.95 MB) seems problematic. It is indeed challenging to quantize the classifier, however this adds an overhead of 1.95 MB. Assuming the ResNet18 is compressed with 2 bits/weight, the compressed part has a size of  44.6 MB/16 = 2.8 MB. Thus, with the classifier, its size is 2.8+1.95 = 4.75 MB, which is roughly a x10 compression factor.  

Justification of rating:
The paper presents an interesting idea that accounts both for the inference time and the bell-shaped distribution of weights in NNs. However, the results must be pushed further (see "weaknesses"). 


=== After the rebuttal ===

I thank the authors for their rebuttal. They answer to all the raised concerns with clarity and concision and performed additional experiments. 

In particular, the authors:
- perform additional ResNet50 experiments
- quantize the input layer and the classifier to 8 bits
- clarifying the positioning and in particular with respect to weight normalization

The paper is substantially better in its updated version. Therefore, I am happy to update my rating to "weak accept".

review_title: Official Blind Review #3

Meta-review: This paper presents a quantization scheme with the advantage of high computational efﬁciency. The experimental results show that the proposed scheme outperforms SOTA methods and is competitive with the full-precision models. The reviewers initially raised some concerns including baseline ResNet performance,  detailed comparison of the quantization size, and comparison with ResNet50. Authors addressed these concerns in the rebuttal and revised the draft to accommodate the requested items. The reviewers appreciated the revision and find it highly improved. Their overall recommendation is toward accept, which I also support.
Decision: Accept (Poster)