Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper trains wide ResNets for 1-bit per weight deployment.
The experiments are conducted on CIFAR-10, CIFAR-100, SVHN and ImageNet32.

+the paper reads well
+the reported performance is compelling 

Perhaps the authors should make it clear in the abstract by replacing:
"Here, we report methodological innovations that result in large reductions in error rates across multiple datasets for deep convolutional neural networks deployed using a single bit for each weight"
with
"Here, we report methodological innovations that result in large reductions in error rates across multiple datasets for wide ResNets deployed using a single bit for each weight"

I am curious how the proposed approach compares with SqueezeNet (Iandola et al.,2016) in performance and memory savings.


review_title: a single bit for each weight

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper introduces several ideas: scaling, warm-restarting learning rate, cutout augmentation. 

I would like to see detailed ablation studies: how the performance is influenced by the warm-restarting learning rates, how the performance is influenced by cutout. Is the scaling scheme helpful for existing single-bit algorithms?

Question for Table 3: 1-bit WRN 20-10 (this paper) outperforms WRN 22-10 with the same #parameters on C100. I would like to see more explanations. 

review_title: Mixed ideas

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose to train neural networks with 1bit weights by storing and updating full precision weights in training, but using the reduced 1bit version of the network to compute predictions and gradients in training. They add a few tricks to keep the optimization numerically efficient. Since right now more and more neural networks are deployed to end users, the authors make an interesting contribution to a very relevant question.

The approach is precisely described although the text sometimes could be a bit clearer (for example, the text contains many important references to later sections).

The authors include a few other methods for comparision, but I think it would be very helpful to include also some methods that use a completely different approach to reduce the memory footprint. For example, weight pruning methods sometimes can give compression rates of around 100 while the 1bit methods by definition are limited to a compression rate of 32. Additionally, for practical applications, methods like weight pruning might be more promising since they reduce both the memory load and the computational load.

Side mark: the manuscript has quite a few typos.

review_title: Solid work

Meta-review: The paper presents a way of training 1bit wide resnet to reduce the model footprint while maintaining good performance. The revisions added more comparisons and discussions, which make it much better. Overall, the committee feels this work will bring value to the conference.
Decision: Accept (Poster)