Venue: ICLR 2018 conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This is a well-written paper with good comparisons to a number of earlier approaches. It focuses on an approach to get similar accuracy at lower precision, in addition to cutting down the compute costs. Results with 2-bit activations and 4-bit weights seem to match baseline accuracy across the models listed in the paper.

Originality
This seems to be first paper that consistently matches baseline results below int-8 accuracy, and shows a promising future direction.

Significance
Going down to below 8-bits and potentially all the way down to binary (1-bit weights and activations) is a promising direction for future hardware design. It has the potential to give good results at lower compute and more significantly in providing a lower power option, which is the biggest constraint for higher compute today. 

Pros:
- Positive results with low precision (4-bit, 2-bit and even 1-bit)
- Moving the state of the art in low precision forward
- Strong potential impact, especially on constrained power environments (but not limited to them)
- Uses same hyperparameters as original training, making the process of using this much simpler.

Cons/Questions
- They mention not quantizing the first and last layer of every network. How much does that impact the overall compute? 
- Is there a certain width where 1-bit activation and weights would match the accuracy of the baseline model? This could be interesting for low power case, even if the "effective compute" is larger than the baseline.

review_title: High accuracy at low-precision

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper studies the effect of reduced precision weights and activations on the performance, memory and computation cost of deep networks and proposes a quantization scheme and wide filters to offset the accuracy lost due to the reduced precision. The study is performed on AlexNet, ResNet and Inception on the Imagenet datasets and results show that accuracy matching the full precision baselines can be obtained by widening the filters on the networks. 

Positives
- Using lower precision activations to save memory and compute seems new and widening the filter sizes seems to recover the accuracy lost due to the lower precision.

Negatives
- While the exhaustive analysis is extremely useful the overall technical contribution of the paper that of widening the networks is fairly small. 
- The paper motivates the need for reduced precision weights from the perspective of saving memory footprint when using large batches. However, the results are more focused on compute cost. Also large batches are used mainly during training where memory is generally not a huge issue. Memory critical situations such as inference on mobile phones can be largely mitigated by using smaller batch sizes. It might help to emphasize the speed-up in compute more in the contributions.  
review_title: WRPN: Wide Reduced-Precision Networks

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper presents an simple and interesting idea to improve the performance for neural nets. The idea is we can reduce the precision for activations and increase the number of filters, and is able to achieve better memory usage (reduced). The paper is aiming to solve a practical problem, and has done some solid research work to validate that.  In particular, this paper has also presented a indepth study on AlexNet with very comprehensive results and has validated the usefulness of this approach.   

In addition, in their experiments, they have demonstrated pretty solid experimental results, on AlexNet and even deeper nets such as the state of the art Resnet. The results are convincing to me. 

On the other side, the idea of this paper does not seem extremely interesting to me, especially many decisions are quite natural to me, and it looks more like a very empirical practical study. So the novelty is limited.

So overall given limited novelty but the paper presents useful results, I would recommend borderline leaning towards reject.
review_title: Review of WRPN

Meta-review: This paper explores the training of CNNs which have reduced-precision activations. By widening layers, it shows less of an accuracy hit on ILSVRC-12 compared to other recent reduced-precision networks. R1 was extremely positive on the paper, impressed by its readability and the quality of comparison to previous approaches (noting that results with 2-bit activations and 4-bit weights matched FP baselines). This seems very significant to me. R1 also pointed out that the technique used the same hyperparameters as the original training scheme, improving reproducibility/accessibility. R1 asked about application to MobileNets, and the authors reported some early results showing that the technique also worked with smaller network/architectures designed for low-memory hardware. R2 was less positive on the paper, with the main criticism being that the overall technical contribution of the paper was limited. They also were concerned that the paper seemed to motivate based on reducing memory footprint, but the results were focused on reducing computation. R3 liked the simplicity of the idea and comprehensiveness of the results. Like R2, they thought the paper was limited novelty. In their response to R3, the authors defended the novelty of the paper. I tend to side with the authors that very few papers target quantization at no accuracy loss. Moreover, the paper targets training, which also receives much less attention in the model compression / reduced precision literature. Is the architecture really novel? No. But does the experimental work investigate an important tradeoff? Yes.
Decision: Accept (Poster)