Paper:
Towards Accurate Binary Convolutional Neural Network
We introduce a novel scheme to train binary convolutional neural networks (CNNs) -CNNs with weights and activations constrained to {-1,+1} at run-time. It has been known that using binary weights and activations drastically reduce memory size and accesses, and can replace arithmetic operations with more efficient bitwise operations, leading to much faster test-time inference and lower power consumption. However, previous works on binarizing CNNs usually result in severe prediction accuracy degradation. In this paper, we address this issue with two major innovations: (1) approximating full-precision weights with the linear combination of multiple binary weight bases; (2) employing multiple binary activations to alleviate information loss. The implementation of the resulting binary CNN, denoted as ABC-Net, is shown to achieve much closer performance to its full-precision counterpart, and even reach the comparable prediction accuracy on ImageNet and forest trail datasets, given adequate binary weight bases and activations.

Review:
The paper describes a new method for training binary convolutional neural networks. The scheme addresses both using binary weights and activations. The key insight here is to approximate the real valued weights via a linear combination of M binary basis weights. The coefficients for reconstructing the real weights can be found using least squares in the forward pass, and then pulled outside the convolution to allow for fast binary convolution at test time. A similar approach is taken for the activations, but in this case the weights and shifts are trained as normal during backpropagation. The result is a network that requires M more binary convolutions than a straightforward binary neural network, but it is expected that these will be significantly more hardware friendly. Experiments on ImageNet show that the approach outperforms competing approaches.
Binary convolutional networks are an important topic with with obvious commercial applications. The idea in the paper is good, and the results on ImageNet are encouraging.
Comments
1. Given that you approximate a real valued convolution with M binary convolutions, it seems to me that the approximate cost would be similar to using M binary CNNs like XNOR nets. With this in mind, I think the approach should really be compared to an ensemble of M XNOR nets trained with different initial weights, both in terms of total train time and prediction accuracy.
2. The paper suggests the v_i shifts are trainable. Why are values for these given in table 4 and the supplementary material. Are these the initial values?
3. The paper uses fixed values for the u_i's but mentions that these could also be trained. Why not train them too? Have you tried this?
4. It's not clear from the paper if solving regression problems for the alphas during training adds much to the train time.
5. It would be nice to have an estimation of the total cost of inference for the approach on a typical net (e.g. ResNet-18) given some realistic assumptions about the hardware, and compare this with other methods (perhaps in the supplementary material)