Paper:
Recent efforts on training light-weight binary neural networks offer promising execution/memory efficiency. This paper introduces ResBinNet, which is a composition of two interlinked methodologies aiming to address the slow convergence speed and limited accuracy of binary convolutional neural networks. The first method, called residual binarization, learns a multi-level binary representation for the features within a certain neural network layer. The second method, called temperature adjustment, gradually binarizes the weights of a particular layer. The two methods jointly learn a set of soft-binarized parameters that improve the convergence rate and accuracy of binary neural networks. We corroborate the applicability and scalability of ResBinNet by implementing a prototype hardware accelerator. The accelerator is reconfigurable in terms of the numerical precision of the binarized features, offering a trade-off between runtime and inference accuracy.

Review:
This paper proposes ResBinNet, with residual binarization, and temperature adjustment. It is a reconfigurable binarization method for neural networks. It improves the convergence rate during training. 
I appreciate a lot that the authors were able to validate their idea by building a prototype of an actual hardware accelerator.
I am wondering what are the values of \gamma’s in the residual binarization after learning? What is its advantage over having only one \gamma, and then the rest are just 1/2*\gamma, 1/4* \gamma, … , etc.? The latter is an important baseline for residual binarization because that corresponds to the widely used fixed point format for real numbers. If you can show some results that residual encoding is better than having {\gamma, 1/2*\gamma, 1/4* \gamma, …, } (which contains only one \gamma), it would validate the need of using this relatively complex binarization scheme. Otherwise, we can just use the l-bit fixed point multiplications, which is off-the-shelf and already highly optimized in many hardwares. 
For the temperature adjustment, modifying the tanh() scale has already had a long history, for example, http://yann.lecun.com/exdb/publis/pdf/lecun-89.pdf page 7, which is exactly the same form as in this paper. Adjusting the slope during training has also been explored in some straight-through estimator approaches, such as https://arxiv.org/pdf/1609.01704.pdf. In addition, having this residual binarization and adjustable tanh(), is already adding extra computations for training. Could you provide some data for comparing the computations before and after adding residual binarization and temperature adjustment? 
The authors claimed that ResBinNet converges faster during training, and in table 2 it shows that ResBinNet just needs 1/10 of the training epochs needed by BinaryNet. However, I don’t find it very fair. Given that the accuracy RBN gets is much lower than Binary Net, the readers might suspect that maybe the other two models already reach ResBinNet’s accuracy at an earlier training epochs (like epoch 50), and just take all the remaining epochs to reach a higher accuracy. On the other hand, this comparison is not fair for ResBinNet as well. The model size was much larger in BinaryNet than in ResBinNet. So it makes sense to train a BinaryNet or FINN, in the same size, and then compare the training curves. Lastly, in CIFAR-10 1-level case, it didn’t outperform FINN, which has the same size. Given these experiments, I can’t draw any convincing conclusion.
Apart from that, There is an error in Figure 7 (b), where the baseline has an accuracy of 80.1% but its corresponding bar is lower than RBN1, which has an accuracy of 76%.