Venue: ICLR 2017 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: There is a great deal of ongoing interest in compressing neural network models. One line of work has focused on using low-precision representations of the model weights, even down to 1 or 2 bits. However, so far these approaches have been accompanied by a significant impact on accuracy. The paper proposes an iterative quantization scheme, in which the network weights are quantized in stages---the largest weights (in absolute value) are quantized and fixed, while unquantized weights can adapt to compensate for any resulting error. The experimental results show this is extremely effective, yielding models with 4 bit or 3 bit weights with essentially no reduction in accuracy. While at 2 bits the accuracy decreases slightly, the results are substantially better than those achieved with other quantization approaches.

Overall this paper is clear, the technique is as far as I am aware novel, the experiments are thorough and the results are very compelling, so I recommend acceptance. The paper could use another second pass for writing style and grammar. Also, the description of the pruning-inspired partitioning strategy could be clarified somewhat... e.g., the chosen splitting ratio of 50% only seems to be referenced in a figure caption and not the main text.
review_title: Great idea, very impressive results.

final_score: rating: 7: Good paper, accept
review_content: The idea of this paper is reasonable - gradually go from original weights to compressed weights by compressing a part of them and fine-tuning the rest. Everything seems fine, results look good, and my questions have been addressed.

To improve the paper:

1) It would be good to incorporate some of the answers into the paper, mainly the results with pruning + this method as that can be compared fairly to Han et al. and outperforms it.

2) It would be good to better explain the encoding method (my question 4) as it is not that clear from the paper (e.g. made me make a mistake in question 5 for the computation of n2). The "5 bits" is misleading as in fact what is used is variable length encoding (which is on average close to 5 bits) where:
- 0 is represented with 1 bit, e.g. 0
- other values are represented with 5 bits, where the first bit is needed to distinguish from 0, and the remaining 4 bits represent the 16 different values for the powers of 2.

review_title: Reasonable idea

final_score: rating: null
review_content: 1) Can you compare to more existing methods than just to BWN and TWN in Table 4? For example, a very relevant one is Han et al. 2016. Looking at your Table 1, AlexNet ref 32-bit versus AlexNet 5-bit, the relative size of the networks is 5/32=16%. Comparing this to their figure 6, it seems that at 16% they get 0% accuracy loss. So, is this method really worth it?

2) Why did you stick to weights as powers of 2, why not simply do k-means?

3) Regarding weight partitioning - you tried only random partitioning and partitioning based on the absolute value. Doesn't it make sense to also partition based on quantization error? You can just find the set of weights that are most similar to values they would be quantized to, and tune the rest.

4) I'm not completely clear about the formulation that is repeated a few times "5-bit (4 bits for powers of two plus 1 bit for zero)" ? Does it mean that out of the possible 2^5 values that can be expressed with 5 bits, one is a zero, others are +- powers of 2, and one is unused? Or are you talking about some variable length encoding (though in this case the representation wouldn't be 5 bit)?

5) I think the region between equation 1 the end of section 2.1 needs some clarifications, though I think I understand it, it required a bit of thinking to understand what is going on. Firt, "bounding the elements in P_l" below equation 1 is a bit confusing, as the elements are bounded just by [-2^n1, +2^n1] (so no dependence on n2). Second, where does equation 2 come from, it's not explained apart from saying that it is "tricky". I would guess n1=round(log2(s)), so where does 4/3 come from? From later on (eq 4) 3/2 is also feasible, but 4/3 just seems random. Then, first paragraph of page 4, shouldn't the last term of the equation for n2 be (2^b - 1)/2, i.e. -1 should be out of the exponent, especially since 2^(b-1)/2 is just 2^(b-2)? Again this is not explained, but my thinking is that:
2^b bits are available for representing these many values: 1 for 0, 2*(n1-n2+1) for both signs of the n1-n2+1 whole numbers that are in the range [n2, n1]
So: 2^b = 2*(n1-n2+1)+1, thus n2=n1+1-(2^b-1)/2
Below equation 4, it might be worth saying that 2*beta is the following number in the sequence alpha, beta, 2 beta (apart from when beta=0), in order to understand equation 4 easier.
Also, doesn't it make sense to perform this quantization in the log scale instead of in the linear scale? The elements in P_l are distributed uniformly in the log scale (apart from 0) so it seems more natural to quantize that way i.e. something like w_l = 2^b sign(w_l) when .. for alpha and beta being the consecutive exponents.


review_title: Various

final_score: rating: 7: Good paper, accept
review_content: Nice idea but not complete, model size is not reduced by the large factors found in one of your references (Song 2016), where they go to 5 bits, but this is ontop of pruning which gives overall 49X reduction in model size of VGG (without loss of accuracy). You may achieve similar reductions with inclusion of pruning (or better since you go to 4 bits with no loss) but we should see this in the paper, so at the moment it is difficult to compare
review_title: Quantize a fully trained network with an iterative 3 step process of partition/hard quantize/retrain, repeated on the retrained partition until fully quantized. Achieves nice results on ImageNet tasks down to 4 bits, but is missing pruning steps which is needed for large competitive compression.

final_score: rating: null
review_content: 1. Could you comment on the distribution of weights after quantization? For instance, for 5 bit quantization, how often did the weights assume the most extreme values?
2. Did you compare quantization to powers of two vs. scalar quantization (uniform spacing in some interval) at the same bit-rate?
review_title: Questions

Meta-review: The paper presents a method for iterative quantization of neural networks weights to powers of 2. The technique is simple, but novel and effective, with thorough evaluation on a variety of ImageNet classification models.
Decision: Accept (Poster)