Paper:
Convolutional Neural Networks (CNN) are very popular in many fields including computer vision, speech recognition, natural language processing, to name a few. Though deep learning leads to ground breaking performance in these domains, the networks used are very demanding computationally and are far from real-time even on a GPU, which is not power efficient and therefore does not suit low power systems such as mobile devices. To overcome this challenge, some solutions have been proposed for quantizing the weights and activations of these networks, which accelerate the runtime significantly. Yet, this acceleration comes at the cost of a larger error. The NICE method proposed in this work trains quantized neural networks by noise injection and a learned clamping, which improve the accuracy. This leads to state-of-the-art results on various regression and classification tasks, e.g., ImageNet classification with architectures such as ResNet-18/34/50 with low as 3-bit weights and 3 -bit activations. We implement the proposed solution on an FPGA to demonstrate its applicability for low power real-time applications.

Review:
The authors present a method for fine-tuning neural networks so inference can be performed in a quantized low bit data format down to 3 bits. The authors achieve this through a combination of three techniques:
1. Noise injection to fine-tune the weights before quantization. The effect of noise injection can model that of quantization, but rather than being stuck in a quantization bin, fine grained weight updates are still possible
2. A schedule that quantizes layer by layer, rather than all layers at the same time
3. Clipping weights and activations within a learned range to obtain finer grained bins within that range. 
The main contribution is a novel combination of mostly existing techniques. Clipping (or clamping as the authors call it) has been proposed by Zhang et al. 2018, but it's an interesting contribution to have the clipping learned directly via backpropagation with a straight-through estimator. Treating the quantization as noise has been proposed in a different form in McKinstry et al. 2018. Gradual quantization appears novel, but is also the least interesting of the techniques. Therefore, novelty on ideas/methods is somewhat limited, and the contribution is mostly the in the impressive experimental results, which appear to be outperforming previous methods. The main weaknesses are poor writing, and that some details of the implementation required to reproduce the results are missing. For example, the training schedule is not given, e.g. how many epochs to train the clean model, how many with noise, how many quantized. Details on the gradual quantization are also missing. Block based quantization is completely heuristic and not well motivated. If this is the main novel ingredient, more details on the mechanics would be needed. Is both the noise injection and the quantization done in blocks? If the motivation is in "the opportunity to adapt”, then what does the adaptation look like? 
As above, my other main issue is with the writing, there are many examples where I would suggest improvements:
This work could be improved greatly by copy editing for English grammar. There are many typos (including ones that can be caught by autocorrect, missing punctuation, or using similar but unrelated words, e.g. "token" instead of "taken"). The manuscript appears hastily put together and not ready for publication. 
The acronym NICE already has a meaning in the DL literature: Dinh, L., Krueger, D., & Bengio, Y. (2014). NICE: Non-linear Independent Components Estimation. It confusing to reuse it. 
The term clamping is only explained on page 4 but used since the abstract. It’s used in a nonstandard way to mean “constrained to lie within a range” which should be explained earlier. I think “clipped” would be a better term, following the related Choi et al. 2018. Clamping usually means "constrained to a fixed value" (not a range), so it is not a good term to use in this context. 
Are the results shown in table 2 and table 3 from a single trial or averaged across reruns? If single trial, it's misleading to have 2 figures after the decimal. Even non-quantized ResNet tends to have 0.5% or so run to run variability, which is much larger than the differences between some of the methods shown here. In fact, a lot of the results could just be due to picking a lucky random seed. 
Comparisons are shown against methods JOINT (Jung et al), LQ-Nets (Zhang et al), FAQ (McKinstry et al). It would be helpful to present them with the same names in the "related work" section, and explain why they were picked out for the comparison. For someone not familiar with the literature it's hard to see why these 3 would be the obvious picks. 
Readability would increase if table 2 and 3 were moved to section 4 where they are referenced, rather than after the discussion. Fig 2 font size too small and hard to read.