Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This works presents a method for inferring the optimal bit allocation for quantization of weights and activations in CNNs. The formulation is sound and the experiments are complete. My main concern is regarding the related work and experimental validation being incomplete, as they don't mention a very recent and similar work published in ICIP19 https://ieeexplore.ieee.org/document/8803498: "Optimizing the bit allocation for compression of weights and activations of deep neural networks". A reference in related work as well as a comparison in experimental validation would be necessary  and the novelty of this work is rather weak given the above mentioned 2019 publication.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This work nicely proposes a new theoretically-sound unequal bit allocation algorithm, which is based on the Lagrangian rate-distortion formulation. Surprisingly, the simple Lagrange multiplier on the constraints leads us to the convenient conclusion that the rate distortion curves for the weight quantization and the activation quantization have to match. Based on this conclusion, the authors claim that their search for the best bit allocation strategy is with a less complexity.

I found this paper interesting and enjoyed reading it. However, I wish the paper could address some issues that are a little bit confusing to me. 

First of all, the paper is not about 2bit quantization. It seeks an “average” 2bit quantization. It means that some weights in some layers can be quantized with higher or lower bits per weight.  Same story goes on for the activation quantization. I don’t exactly know the implication of this, but it seems that the hardware implementation of a convolution layer could be either too complicated to benefit from this quantization scheme, or doesn’t really improve the efficiency of, say 4bit quantization for all layers. Is it really more efficient to do multiplication-and-addition between 2 bit weights and 5 bit input (the output of the previous activation) than between 4bit weights and 4bit input? I’m not a hardware person, but this part needs to be clearly addressed. Storage-wise, lowering the bitrate might be a clear benefit (I guess)

I wish the actual optimization part briefly mentioned in section 4.2 could be elaborated more. It is a crucial part but somewhat understated. 

I also wonder what’s the effect or limitation of using MSE for this optimization, where cross-entropy is a more suitable choice. I know that the objective function in eq 5 is just to find the best combination of bit allocations per layer, but still, the error space might not be the best for this classification problem. 
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: Very good paper that studies the error rate of low-bit quantized networks and uses Pareto condition for optimization to find the best allocation of weights over all layers of a network. The theoretical claims are strongly supported by experiments, and the experimental analysis covers state-of-the-art architectures and demonstrates competitive results. The paper in addition also analyzes the inference cost of their approach (in addition to the accuracy results), and shows positive results on ResNet and MobileNet architectures.

The paper primarily shows that the mean squared error of the final output of a quantized network has the additive property of being equal to the sum of squared errors of the outputs obtained by quantizing each layer individually. Although there is no reason why this should be case, experimental results from the authors on AlexNet and VGG-16 validate this. Based on this assumption, the authors then use a Lagrangian based constrained optimization to minimize the sum of squared errors of outputs when individual weights/activations are quantized, with the constraint being the total bit budget for weights and activations. The authors show that this can be optimized under the Pareto condition easily.

The experimental section is quite detailed and covers the popular architectures instead of toy ones. The accuracy results compared to other 2-bit and 4-bit approaches are competitive. It's also nice to see analysis of inference cost where unequal bitrate allocation performs better than other methods.

The authors show that given the constrained optimization, layers that have a large number of weights receive lower bitrates and vice-versa. While it makes sense that this would contribute to stronger inference speedup compared to methods with either equal bitrate allocation across layers or those that allocate higher bitrate to layers with large number of weights, it's not entirely clear why the optimization would produce this allocation in the first place. Do the authors mean to conclude that layers with large number of weights hold a lot of redundancy and don't have a significant impact on the overall accuracy of the model? This needs to be clarified further.
review_title: Official Blind Review #3

Meta-review: This works presents a method for inferring the optimal bit allocation for quantization of weights and activations in CNNs. The formulation is sound and the experiments are complete. However, the main concern is that the paper is very similar to a recent work by the authors, which is not cited.
Decision: Reject