From: NIPS.cc/2018/Workshop/CDNNRIA/Paper5/Authors
comment: We thank you for your comments and address your most pressing concerns as follows.

1.The quantization itself is questionable. For example, by quantizing x = 1 into 2 bits. Following Eq. 6, it will result in encoding [1,1], which is not easy to interpret.

RESPONSE: To address your concern, we will make the description clearer in the final paper. For the example you pointed out, when we use 2-bit to encode the data x and constrain to [-1，1], there are 4 encoded states, as shown below. The affine mapping between quantized real numbers and their encoded states is given in the following table. 

Quantized real numbers | -1 | -1/3 | 1/3 | 1
Encoded states | {-1, -1} | {-1, 1} | {1, -1} | {1, 1}

From the above results, we can see that there is a linear factor \alpha between quantized real numbers and encoded states (e.g., \alpha=3 for the above table). When we use Eq. (5) to compute the multiplication of two encoded vectors, the value will be expanded \alpha*\alpha times. Therefore, the result of Eq. (5) can multiply its scale factor to get the final result.  

2.In addition, the acceleration ratio reported in this paper is not correct. All the calculations are based on 64x reported in XNOR-Net, which is very misleading. "As the binary multiplication operates in 1 bit, whereas the full precision multiplication operates in 32 bit, despite the feasible implementations, the acceleration should be 32x in theory" (Also see the comments in https://openreview.net/forum?id=S19dR9x0b)

RESPONSE: To address your concern, we analyzed the theoretical performance of the acceleration ratio as follows. 

Suppose there are two vectors x=[x^1,x^2,...,x^64] and w=[w^1,w^2,...,w^64], composed by 64 real numbers. If we use the 32-bit floating-point data type to represent each element of the two vectors, the element-wise vector multiplication should require 64 multiplications. However, after each element of the two vectors is quantized to a binary state, we use 1-bit to denote their elements, and can use two 64-bit data type to represent those two vectors. Then the element-wise multiplication of those two vectors can be computed at once. The general computing platform (i.e., CPU, GPU andARM) can perform a 64-bit binary operation in one clock cycle. In addition, an Intel SSE (e.g., AVX and AVX-512) instruction can perform 128 (or 256, 512) bits binary operation [3]. 

More details are given as follows: We use S_{MK} to denote the speedup ratio of our method by constraining activation values to M-bit and the values of weights to K-bit, where M, K \in {1,2,...,8}. Following [1, 2], S_{MK} can be formulated as follows: 

S_{MK} = (N\gamma)/[MK(\gamma+2\lceil N/L\rceil)+(MK-1)\beta],

where \gamma denotes the speedup ratio of multiply-accumulate operation (MAC) compared with bitwise operation, and \beta denotes the speedup ratio of addition compared with bitwise operation. Following [2], we can know that \beta=\gamma/2, \gamma=1.91, L=64 and N=2304 for the majority of convolutional layer in the ResNet architecture. Thus, our method can obtain at most ~59x speedup and ~32x memory saving over its full-precision counterpart. Thus, users can easily achieve different encoding precisions arbitrarily according to their requirements (e.g., accuracy and speed) and hardware resources (e.g., memory). 

Besides the theoretical analysis, we implemented the matrix multiplication via our {-1, +1} encoding scheme on GPU (TITAN X), the speedup ratios are given in the following table. 

N | M=K=1 | M=K=2 | M=K=3 | Full-Precision
2048 | 6.17x | 3.00x | 1.80x | 1x
4096 | 14.30x | 6.51x | 3.75x | 1x
8192 | 23.28x | 10.13x | 5.44x | 1x

The result shows that as the length of the vector (i.e., N) increases, the speedup ratio becomes larger. We can see that our method with different encoding precisions (e.g., M=K=2) is significantly faster than full-precision counterparts. 

[1]R. Mohammad, et al., Xnor-net: Imagenet classification using binary convolutional neural networks.
[2] W. Diwen, et al., TBN: Convolutional Neural Network with Ternary Inputs and Binary Weights.
[3] https://software.intel.com/sites/landingpage/IntrinsicsGuide/

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper5/Authors
comment: Thank you for your positive comments.

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper5/Authors
comment: We thank you for your comments and address your most pressing concerns as follows.

1. Experiments of the proposed method as shown in Table 2 demonstrated that the performance of 8-bit quantization is acceptable, while the 1-bit quantization result of the proposed method is far inferior to the baseline method, i.e. BWN [17] and XNORNet [18], especially, the results on ImageNet.

RESPONSE: Different from BWN and TWN, whose weights are only quantized rather than activation values, our method quantifies both weights and activation values simultaneously. Although BWN and TWN can obtain little higher accuracies than our method with 1-bit quantization model, our method obtains ~59x speedup and the speedup ratio of existing methods such as BWN and TWN is limited to ~2x. Due to limited and fixed expression ability, existing methods (such as BWN, TWN, BNN, XNOR-Net) can not satisfy higher precision requirements. In particular, our method can provide 64 available encoding choices, and hence our encoded networks with different encoding precisions have different calculation speed, memory requirements and experimental precisions. 

2.Fcn.6 proposes a new method for quantizing input data x and parameters w. However, we also need to quantize input data x in order to match the quantized w in the inference stage, which will definitely increase the running time. Therefore, the reported ~59x speed up seems to be inaccurate.

RESPONSE: To address your concern, we analyzed the theoretical performance of our encoding scheme described in Section 2 as follows. We use S_{MK} to denote the speedup ratio of our method by constraining activation values to M-bit and the values of weights to K-bit, where M, K \in {1,2,...,8}. Following [1, 2], S_{MK} can be formulated as follows: 

S_{MK} = (N\gamma)/[MK(\gamma+2\lceil N/L\rceil)+(MK-1)\beta],

where \gamma denotes the speedup ratio of multiply-accumulate operation (MAC) compared with bitwise operation, and \beta denotes the speedup ratio of addition compared with bitwise operation. Following [2], we can know that \beta=\gamma/2, \gamma=1.91, L=64 and N=2304 for the majority of convolutional layer in the ResNet architecture. The general computing platform (i.e., CPU, GPU andARM) can perform a 64-bit binary operation in one clock cycle. In addition, an Intel SSE (e.g., AVX and AVX-512) instruction can perform 128 (or 256, 512) bits binary operation. 
Thus, our method can obtain at most ~59x speedup and ~32x memory saving over its full-precision counterpart. Thus, users can easily achieve different encoding precisions arbitrarily according to their requirements (e.g., accuracy and speed) and hardware resources (e.g., memory). 

Besides theoretical analysis, we implemented the matrix multiplication via our {-1, +1} encoding scheme on GPU (TITAN X), the speedup ratios are given in the following table. 

N | M=K=1 | M=K=2 | M=K=3 | Full-Precision
2048 | 6.17x | 3.00x | 1.80x | 1x
4096 | 14.30x | 6.51x | 3.75x | 1x
8192 | 23.28x | 10.13x | 5.44x | 1x

The result shows that as the length of the vector (i.e., N) increases, the speedup ratio becomes larger. We can see that our method with different encoding precisions (e.g., M=K=2) is significantly faster than full-precision counterparts. 

[1] R. Mohammad, et al., Xnor-net: Imagenet classification using binary convolutional neural networks.
[2] W. Diwen, et al., TBN: Convolutional Neural Network with Ternary Inputs and Binary Weights.

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper5/AnonReviewer1
title: On the quantization method and acceleration ratio
review: This paper presents a new multi-bit quantization method. By applying to CIFAR-10, ImageNet, and VOC tasks, the method is reported to achieve plausible performance. However, it is unclear why does the new quantization method, i.e., Eq. 6,  is better than existing ones (see all the mentioned multi-bit quantization methods in https://openreview.net/forum?id=S19dR9x0b). 


The quantization itself is questionable. For example, by quantizing x = 1 into 2 bits. Following Eq. 6, it will result in encoding [1,1], which is not easy to interpret.

In addition, the acceleration ratio reported in this paper is not correct. All the calculations are based on 64x reported in XNOR-Net, which is very misleading. "As the binary multiplication operates in 1 bit, whereas the full precision multiplication operates in 32 bit, despite the feasible implementations, the acceleration should be 32x in theory" (Also see the comments in https://openreview.net/forum?id=S19dR9x0b)
rating: 2: Marginally below acceptance threshold
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper5/AnonReviewer2
title:    
review: 
Summary: 
The authors propose a novel encoding scheme by using {−1, +1} to decompose quantized neural networks (QNNs) into multi-branch binary networks. The model achieves better results in terms of runtime and memory over its full-precision counterparts.

The results in experiment part are detailed. The authors conducted plenty of experiments on 3 datasets. The results also outperform previous methods.

rating: 3: Marginally above acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper5/AnonReviewer3
title: Overall rating: 2/5 
review: - This paper try to represent convolution operation in a new fashion and then propose a variable method for quantizing existing deep neural networks.

- Experiments of the proposed method as shown in Table 2 demonstrated that the performance of 8-bit quantization is acceptable, while the 1-bit quantization result of the proposed method is far inferior to the baseline method, i.e. BWN [17] and XNORNet [18], especially, the results on ImageNet.

- Fcn.6 proposes a new method for quantizing input data x and parameters w. However, we also need to quantize input data x in order to match the quantized w in the inference stage, which will definitely increase the running time. Therefore, the reported ~59x speed up seems to be inaccurate.

rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: ~Lixin_Fan1
comment: Authors please send author information to workshop chairs (lixin.fan@nokia.com), so that you submission can be assigned to reviewers without conflict of interests.
title: Authors please send author information to workshop chairs (lixin.fan@nokia.com)