And the Bit Goes Down: Revisiting the Quantization of Neural Networks
Authors: Pierre Stock
Armand Joulin
Rémi Gribonval
Benjamin Graham
Hervé Jégou
Venue:  (2020)
Keywords: compression
quantization

Abstract: In this paper, we address the problem of reducing the memory footprint of convolutional network architectures. We introduce a vector quantization method that aims at preserving the quality of the reconstruction of the network outputs rather than its weights. The principle of our approach is that it minimizes the loss reconstruction error for in-domain inputs. Our method only requires a set of unlabelled data at quantization time and allows for efficient inference on CPU by using byte-aligned codebooks to store the compressed weights. We validate our approach by quantizing a high performing ResNet-50 model to a memory size of 5MB (20x compression factor) while preserving a top-1 accuracy of 76.1% on ImageNet object classification and by compressing a Mask R-CNN with a 26x factor.

Published as a conference paper at ICLR 2020
AND THE BIT GOES DOWN: REVISITING THE QUAN- TIZATION OF NEURAL NETWORKS
Pierre Stock 12, Armand Joulin', Rémi Gribonval', Benjamin Graham', Hervé Jégou'
Facebook AI Research, Univ Rennes, Inria, CNRS, IRISA
ABSTRACT
In this paper, we address the problem of reducing the memory footprint of con- volutional network architectures. We introduce vector quantization method that aims at preserving the quality of the reconstruction of the network outputs rather than its weights. The principle of our approach is that it minimizes the loss recon- struction error for in-domain inputs. Our method only requires a set of unlabelled data at quantization time and allows for efficient inference on CPU by using byte- aligned codebooks to store the compressed weights. We validate our approach by quantizing high performing ResNet-50 model to memory size of MB (20x compression factor) while preserving a top-1 accuracy of 76.1% on ImageNet ob- ject classification and by compressing Mask R-CNN with 26x factor
INTRODUCTION
There is a growing need for compressing the best convolutional networks (or ConvNets) to sup- port embedded devices for applications like robotics and virtual/angmented reality. Indeed, the performance of ConvNets on image classification has steadily improved since the introduction of AlexNet (Krizhevsky et al.] 2012).. This progress has been fueled by deeper and richer ar- chitectures such as the ResNets (He et al. 2015) and their variants ResNeXts (Xie al., 2017) or DenseNets (Huang et al. 2017). Those models particularly benefit from the recent progress made with weak supervision (Mahajan et al./ 2018 Yalniz et al., 2019 Berthelot al.. 2019). Compres- sion of ConvNets has been an active research topic in the recent years, leading to networks with a 71% top-1 accuracy on ImageNet object classification that fit in 1 MB (Wang et al. 2018b).
In this work, we propose a compression method particularly adapted to ResNet-like architectures. Our approach takes advantage of the high correlation in the convolutions by the use of a structured quantization algorithm, Product Quantization (PQ) (Tégou al. 201I). More precisely, we exploit the spatial redundancy of information inherent to standard convolution filters (Denton et al. 2014).
Besides reducing the memory footprint, we also produce compressed networks allowing e efficient inference on CPU by using byte-aligned indexes, as opposed to entropy decoders (Han et al. 2016))
Our approach departs from traditional scalar quantizers (Han et al.] 2016) and vector quantiz- ers (Gong et al., 2014] Carreira- & Idelbayev] (017) by focusing on the accuracy of the activations rather than the weights. This is achieved by leveraging weighted k-means technique. To our knowledge this strategy (see Section (3) is novel in this context. The closest work we are aware of is the one by Choi et al. (2016), but the authors use a different objective (their weighted term is derived from second-order information) along with a different quantization technique (scalar quantization). Our method targets better in-domain reconstruction, as depicted by Figure
Finally, we compress the network sequentially to account for the dependency of our method to the activations each layer. To prevent the accumulation of errors across layers, we guide this compression with the activations of the uncompressed network on unlabelled data: training by dis- tillation (Hinton et al.) 2014) allows for both an efficient layer-by-laye compression procedure and global fine-tuning of the codewords. Thus, we only need set of unlabelled images to adjust the codewords. As opposed to recent works by Mishra Marr (2017) or Lopes et al. (2017), our distillation scheme is sequential and the underlying compression method is different (PQ vs. scalar).
"ooee and compressed httpp://ggicchb...m///accaoooot

---

Published as a conference paper at ICLR 2020
in-domain
Pstandard
Pactivations
out-of-domain
Figure 1: Illustration of method. We approximate binary classifier yp that labels images as dogs or cats by quantizing its weights. Standard method: quantizing & with the standard objective function (I) promotes a classifier Pstandard that tries approximate sp over the entire input space and can thus perform badly for in-domain inputs. Our method: quantizing with our objective function (2) promotes a classifier Pactivations that performs well for in-domain inputs. Images lying in the hatched area of the input space are correctly classified by Pactivations but incorrectly by Pstandard.
We show that applying approach the semi-supervised ResNet-50 of Yalniz et al. (Yalniz jet al. 2019) leads to a MB memory footprint and 76.1% top-1 accuracy on ImageNet object classification (hence x compression the original model). Moreover, approach generalizes to other tasks such as image detection. As shown in Section 4.3] we compress a Mask R- R-CNN (He al.) 2017) with size budget around MB (26x compression factor) while maintaining competitive performance.
RELATED WORK
There is large body of literature network compression. We review the works closest to ours and refer the reader to two recent surveys (Guo) 2018 Cheng et al. 2017) for comprehensive overview.
Low-precision training. Since early works like those of Courbariaux et al.
(2015). researchers
have developed various approaches to train networks with low precision weights. Those approaches include training with binary or ternary weights (Shayer et al. 2017 Zhu al.
2016 Li & Liu
2016 Rastegari et al., 2016 McDonnel] 2018). learning a combination of binary bases (Lin et al,
2017 and quantizing the activations (Zhou al. 2016 2017 Mishra et al. (201). Some of these
methods assume the possibility to employ specialized hardware that speed up inference and improve power efficiency by replacing most arithmetic operations with bit-wise operations. However, the tack-propagation has to be adapted to the case where the weights are discrete.
Quantization. Vector Quantization (VQ) and Product Quantization (PQ) have been extensively studied in the context of nearest--eighhoo search (Jegou et al. 2011 Ge et al. 2014 Norouzi & Fleet 2013). The idea is decompose the original high-dimensional space into cartesian product of subspaces that are quantized separately with joint codebook. To our knowledge, Gong et al. (2014) were the first to introduce these stronger quantizers for neural network quantization, followed by Carreira-Perpiñán & Idelbayev (017)) As we will see in the remainder of this paper, employing this discretization off-the-shelf does not optimize the right objective function, and leads catastrophic drift of performance for deep networks.
Pruning. Network pruning amounts to removing connections according to an importance criteria (typically the magnitude of the weight associated with this connection) until the desired model size/aceuracy tradeoff is reached (LeCun et al., 1990). A natural extension this work is to prune structural components of the network, for instance by enforcing channel-level (Liu et al. 2017) or filter-level (Luo et al. 2017) sparsity. However, these methods alternate between pruning and re-training steps and thus typically require long training time.
2

---

Published as a conference paper at ICLR 2020
Dedicated architectures, Architectures such as SqueezeNet (landola et al. 2016), NASNet (ooh et al. 2017), ShuffleNet (Zhang et al.) 2017 Ma et al. 2018), MobileNets (Sandler et al. 2018) and EfficientNets (Tan & Le 2019)
designed to be memory efficient. As they typically rely on a combination of depth-wise and point-wise convolutional filters, sometimes along with channel shuffling, they are less prone than ResNets structured quantization techniques such PQ. These architectures are either designed by hand or using the framework of architecture search (Howard al. 2019 For instance. the respective model size and test top-1 accuracy of ImageNet of MobileNet are 13.4 MB for 71.9%, to be compared with vanilla ResNet-50 with size 97.5 MB for a top-1 of 76. 2.. Moreover, larger models such as ResNets can benefit from lcage-scale weakly- or semi-supervised learning to reach better performance (Mahajan et al. 2018 Yalniz et al.) 2019).
Combining some of the mentioned approaches yields high compression factors as demonstrated by Han et al. with Deep Compression (DCC (Han et al. 2016) or more recently by Tung & Mori (Tung Mori 2018). Moreover and from a practical point of view, the process of compressing networks depends
the type of hardware on which the networks will run. Recent work directly quantizes to optimize enery-efffciency and latency time on specific hardware (Wang et al. 8a). Finally, the memory overhead of storing the full activations is negligible compared to the storage of the weights for two reasons. First, in realistic real-time inference setups, the batch size is almost always equal to one. Second, a forward pass only requires to store the activations of the current layer -which are often smaller than the size of the input- and not the whole activations of the network.
3 OUR APPROACH
In this section, we describe our strategy for network compression and we show how to extend our ap- proach to quantize a modern ConvNet architecture. The specificity of our approach is that it aims at a small reconstruction error for the outputs of the layer rather than the layer weights themselves. We first describe how we quantize a single fully connected and convolutional layer. Then we describe how we quantize a full pre-trained network and finetune it.
3.1 QUANTIZATION OF A FULLY-CONNECTED LAYER
We consider fully- conneced layer with weights W RCxxCau and, without loss of generality, we omit the bias since it does not impact reconstruction error.
Product Quantization (PQ). Applying the PQ algorithm to the columns of W consists in evenly splitting each column into m contiguous subvectors and learning a codebook on the resulting mCost subvectors. Then, column of W is quantized by mapping each of its subvector its nearest codeword in the codebook. For simplicity, we assume that Cn is a multiple of m, i.e., that all the subvectors have the same dimension d = Cin/m.
More formally, the codebook C = {ci,...., c contains codewords of dimension d. Any col- umn Wj of W is mapped to its quantized version ((w,) (c.,...., C...) where é denotes the index of the codeword assigned to the first subvector of wj. and so forth. The codebook is then learned by minimizing the following objective function:
W - W 2
Σ q(w,))
where W denotes the quantized weights. This objective can be efficiently minimized with k-means. When m is set to 1, PQ is equivalent to vector quantization (VQ) and when m is equal to C.. it is the scalar k-means algorithm. The main benefit of PQ is its expressivity: each column w, is mapped to vector in the product C = C x ... x C, thus PQ generates an implicit codebook of size km
Our algorithm. PQ quantizes the weight matrix of the fully-connected layer. However, prac- tice, we are interested in preserving the output of the layer, not its weights. This is illustrated in the case of a non-linear classifier in Figure [ preserving the weights a layer does not necessarily guar- antee preserving its output. In other words, the Frobenius approximation of the weights of layer is not guaranteed to be the best approximation of the output over some arbitrary domain (in particular for in in-domain inputs). We thus propose alternative to PQ that directly minimizes the reconstruc- tion error on the output activations obtained by applying the layer to in-domain inputs. More pre- cisely, given batch of B input activations x E BxC,, we interested in learning codebook C
3

---

Published as a conference paper at ICLR 2020
that minimizes the difference between the output activations and their reconstructions:
lly - - = Σ | - q(w,))ll
(2)
where xW is the output and y xW its reconstruction. Our objective is re-weighting of the objective in Equation (T). We can thus learn our codebook with a weighted k-means algorithm. First, we unroll x of size B x Cin into & of size (B x m) x d Lee we split each row of into m subvectors of size d and stack these subvectors, Next, we adapt the EM algorithm as follows.
E-step (cluster assignment). Recall that every column w is divided into m subvectors of dimension d. Each subvector v is assigned to the codeword cy such that
Cj argmin EEC
||x(c- -
This step is performed by exhaustive exploration. Our implementation relies on broadcast- ing to be computationally efficient.
M-step (codeword update). Let consider codeword E C. We denote (Vp)per, the subvectors that are currently assigned to c. Then, we update e <- c*, where
aargmin [[ cER'
BEL.
This step explicitly computes the solution of the least-squares problem Our implemen- tation performs the computation of the pseudo-invers of x before alternating between the Expectation and Minimization steps it does not depend the learned codebook C.
We initialize the codebook C by uniformly sampling k vectors among those we wish to quantize. After performing the E-step, some clusters may be empty. To resolve this issue, we iteratively perform the following additional steps for each empty cluster of index i. (1) Find codeword co corresponding to the most populated cluster; : (2) define new codewords c = co e and c₁ = Co e, where e N (0, e1) and (3) perform again the E-step. We proceed to the M-step after all the empty clusters are resolved. We set = 1e-8 and we observe that its generally takes less than 1 or 2 E-M iterations to resolve all the empty clusters. Note that the quality of the resulting compression is sensitive to the choice of x.
3.2 CONVOLUTIONAL LAYERS
Despite being presented in the case of fully-connected layer, our approach works on any set of vectors. As consequence, our apporoach can be applied to a convolutional layer if we split the associated 4D weight matrix into set of vectors. There are many ways to split 4D matrix set of vectors and we are aiming for one that maximizes the correlation between the vectors since vector quantization based methods work the best when the vectors are highly correlated.
Given a convolutional layer, have Cout filters of size K K x Cin. leading to overall 4D weight matrix W E no particular reason to be correlated. On the other hand, the spatial dimensions related to the filter size are by nature very correlated: nearby patches or pixels likely share information. As depicted in Figure 2 we thus reshape the weight matrix in way that lead to spatially coherent quantization. More precisely, we quantize W spatially into subvectors of size d procedure. We first reshape W into 2D matrix of size (Cin x K x K) Cout- Column j of the reshaped matrix W, corresponds to the jth filter of W and is divided into Cin subvectors of size K K. Similarly, we reshape the input activations accordingly X, so that reshaping back the matrix x, W, yields the same result as * W. In other words, we adopt a dual approach to the one using bi-level Toeplitz matrices to represent the weights. Then, we apply our method exposed in Section 3.1 to quantize each column of W, into m = Cn subvectors of size d = K x K with codewords, using X, as input activations in 2 As a natural extension, we also quantize with larger subvectors, for example subvectors of size d x K x K, see Section for details.
The dimensions along the output and input coordinate have
x K using the following
Denoting x the Moore-Penren pscudoinverse of x, obtain c" =/1 (Epele vp)

---

Published as a conference paper at ICLR 2020
Filters
Reshaped filters
Codebook
Cout
Cin
Cout
Figure 2: We quantize Cout filters size On x K x using subvector size of d = K K. In other words, spatially quantize the convolutional filters to take advantage of the redundancy of information the network. Similar colors denote subvectors assigned to the same codewords.
In our implementation, adapt the reshaping of W and x to various types of convolutions. We count for the padding, the stride, the number groups (for depthwise convolutions and in particular for pointwise convolutions) and the kernel size. We refer the reader to the code for more details.
3.3 NETWORK QUANTIZATION
In this section, we describe our approach for quantizing a neural network. We quantize the network sequentially starting from the lowest layer the highest layer. We guide the compression of the student network by the non-compressed teacher network, as detailled below.
Learning the codebook. We recover the current input activations of the layer, i.e. the input activa- tions obtained by forwarding a batch of images through the quantized lower layers, and we quantize the current layer using those activations. Experimentally, we observed drift in both the reconstruc- tion and classification errors when using the activations of the non-compressed network rather than the current activations.
Finetuning the codebook. We finetune the codewords by distillation (Hinton et al. 2014) using the non-compressed network as the teacher network and the compressed network (up the cur- rent layer) the student network. Denoting Y1 (resp. y.) the output probabilities of the teacher (resp. student) network, the loss we optimize the Kullhack-Leibler divergence = KI, Finetuning codewords is done by averaging the gradients of each subvector assigned to given codeword. More formally, after the quantization step, we fix the assignments once for all. Then, denoting (b,) pp the subvectors that assigned to codeword c, perform the SGD update with a learning rate
Σ Oc
c --c-ni [[o
(5)
pEte ab,
Experimentally, find the approach perform better than finetuning the target of the images as demonstrated in Table Moreover, this approach does not require any labelled data.
3.4 GLOBAL FINETUNING
In a final step, we globally finetune the codebooks of all the layers reduce any residual drifts and we update the running statistics of the BatchNorm layers: We empirically find it beneficial to finetune all the centroids after the whole network is quantized. The finetuning procedure is exactly the same as described in Section 3.3 except that we additionally switch the BatchNorms to the training mode, meaning that the learnt coefficients are still fixed but that the batch statisties (running mean and variance) are still being updated with the standard moving average procedure.
We perform the global finetuning using the standard ImageNet training set for epochs with an initial learning rate of 0.01, weight decay of 10-4 and momentum of 0.9. The learning rate is decayed by a factor 10 every 3 epochs. As demonstrated in the ablation study in Table finetuning on the true labels performs worse than finetuning by distillation. A possible explanation is that the supervision signal coming from the teacher network is richer than the one-hot vector used as a traditional learning signal in supervised learning (Hinton al. 2014).
5

---

Published as a conference paper at ICLR 2020
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We quantize vanilla ResNet-18 and ResNet-50 architectures pretrained on the ImageNet dataset (Deng al. 2009). Unless explicit mention of the contrary, the pretrained models are taken from the Py Torch model zoc We run our method on a 16 GB Volta V100 GPU. Quantizing a ResNet- 50 with our method (including all finetuning steps) takes about one day on GPU. We detail our experimental setup below. Our code and the compressed models are open-sourced.
Compression regimes. We explore large block sizes (resp.small block sizes) compression regime by setting the subvector size of regular 3x3 convolutions to d d=9 (resp.d = 18) and the sub- vector size of pointwise convolutions to of pointwise convolutions is always equal to 4. The number of codewords k E {256, 512, 1024, 2048) for each compression regime. Note that we clamp the number of cen- troids to min(k, Com x m/4) for stability. For instance, the first layer of the first stage of the ResNet-50 has size 64x 64x 1 x1, thus we always use l = 128 centroids with a block size d ===0 8. For given number of centroids k, small blocks lead to lower compression ratio than large blocks.
(resp.d = 8). For ResNet-18, the block size
centroids is set to
Sampling the input activations. Before quantizing each layer, we randomly sample a batch of 1024 training images to obtain the input activations of the current layer and reshape it as described in Section 3.2 Then, before each iteration (E+M step) of our method, we randomly sample 10, 000 rows from those reshaped input activations.
Hyperparameters. We quantize each layer while performing 100 steps of our method (sufficient for convergence in practice). We finetune the centroids of each layer on the standard ImageNet training set during 2,500 iterations with a batch size of 128 (resp 64) for the ResNet-18 18 (resp. Nest 50) with a learning rate of 0.01, a weight decay of 10-4 and a momentum of 0.9. For accuracy and memory reasons, the classifier is always quantized with block size d = 4 and k = 2048 (resp. = 1024) centroids for the ResNet-18 18 (resp., ResNet-50). Moreover, the first convolutional layer of size 7 x 7 is not quantized, as it represents less than 0.1% (resp., 0.05%) of the weights of ResNet-18 18 Respp.ResNet-5O...
Metrics. We focus the tradeoff between accuracy and memory. The accuracy is the top-1 error on the standard validation set of ImageNet. The memory footprint is calculated as the indexing cost (number of bits per weight) plus the overhead of storing the centroids in float16. As example, quantizing a layer of size 128 x 128 x 3 x 3 with = 256 centroids (1 byte per subvector) and a block size of d = leads an indexing cost of kB for m 16, 384 blocks plus the cost of storing the centroids of 4.5 kB.
4.2 IMAGE CLASSIFICATION RESULTS
We report below the results of our method applied to various ResNet models. First, compare method with the state of the art on the standard ResNer-18 and ResNet-50 architecture. Next, we show the potential of our approach the pertinence of our method.
competitive ResNet-50. Finally, ablation study validates
Vanilla R8 and ResNet-50. We evaluate our method on the ImageNet benchmark for R8 and ResNet-50 architectures and compare our results to the following methods: Trained
Ternary Quantization (TTQ) (Zhu al., 2016), LR-Net (Shayer al., 2017). ABC-Net (Lin al. 2017). Binary Weight Network (XNOR-Ne or BWN) (Rastegari et al. 2016). Deep Compression
(DC) (Han et al. 2016) and Hardware-Aware Automated Quantization (HAQ) WWngg et al. 2018a) 8a).
We report the accuracies and compression factors in the original papers and/or in the two surveys 2018 Cheng et al.] (017)) for given architecture when the result is available. We do not compare our method to DoReFa-Net (Zhou et al. 2016) and WRPN (Mishra et al. 2017) as those approaches also use low-precision activations and hence get lower accuracies, e.g., 51.2% top-1 accuracy for a XNOR -et with ResNet- 18. The results are presented in Figure4.22 For better read- ability, some results for our method are also displayed in Table []]
(Guoo) 2018
We report the average accuracy and standard deviation over runs. Our method significantly outperforms state of the art papers for

---

Published as a conference paper at ICLR 2020
ResNet-18 on ImageNet
ResNet-50 on ImageNet
76
HAQ bite) DC (4 bis)
ADC-Net (M-5)
2048
76
HAQ biss)
k-2048
k-1024
k-1024
75-
DC bist
kw512
5-112
k-20-8
Test top-1 1 (%)
FTQ
2049
74
k-256
6c
ABC-Net
Ogk 1024 1024
355
73
1-1021
Test top-
64
ARC-Nat (M-2)
5-512
72-
k-512
LRtNet bits)
71
HAQ (2 bits
62-
Ours, small blocks
Ours, large blocks
Aec-Net (M-1)
BWN
k-256
Ours, small blocks
Ours, large blocks
60
Original model
Reference methods
JRRNeet (1 bit)
69
Original model
Reference methods
(2 bis))
k-256
0
10
20
30
40
0
3
10
15
20
25
30
Compression factorr (original network size: 44.6M18)
Compression factor (original network size: 97.5MB)
Figure 3: Compression results for ResNet-18 and ResNet-50 architectures. We explore two com- pression regimes as defined in Section 4.1 small block sizes (block sizes of 4=4 and 9) and large block sizes (block sizes 8 and 18). The results of our method for k 256 centroids are of practical interest as they correspond to byte-compatible compression scheme.
Table I: Results for vanilla ResNet-18 and ResNet-50 architectures for k = 256 centroids.
Model (original top-1)
Compression
Size ratio
Model size
Top-1 1 (%)
Small blocks
29x
1.54 MB
65.81 #0.04
ResNet-18 (69.76%)
Large blocks
43x
1.03 MB
61.10 +0.03
Small blocks
19x
5.09 MB
73.79
00.05
ResNet-50 (76.15%)
Large blocks
31x
3.19 MB
68.21 #0.04
various operating points. For instance, for ResNet- 18, 18, our method with large blocks and k = 512 centroids reaches a larger accuracy than ABC-Net (M 2) with compression ratio that is 2x larger. Similarly, on the ResNet-50, our compressed model with k = 256 centroids in the large blocks setup yields comparable accuracy to DC (2 bits) with a compression ratio that is 2x larger.
The work by Tung & Mori (Tug & Mori 2018) is likely the only one that remains competitive with ours with a 6.8 MB network after compression, with a technique that prunes the network and therefore implicitly changes the architecture. The authors report the delta accuracy for which have no direct comparable top-1 accuracy, but their method is arguably complementary to ours.
Semi-supervised ResNet-50. Recent works (Mahajan et al. 2018 Yalniz et al., 2019) have demonstrated the possibility to leverage a large collection of unlabelled images to improve the per- formance of given architecture. In particular, Yalniz et al. (Yalniz et al. 2019] use the publicly available YFCC-100M dataset (Thomee et al., 2015) to train a ResNet-50 that reaches 79.1% top-1 accuracy on the standard validation set of Net. In the following, we use this particular model and refer to it as semi-supervised ResNet-50. In the low compression regime (block sizes of 4 and 9), with k 256 centroids (practical for implementation), our compressed cunisvupr ResNet-50 reaches 76.12% % top-1 accuracy. In other words, the model compressed to 5 MB attains the perfor- mance of a vanilla, non con-compeessed ResNet50 (vs.97. 97.5MB for the non-compressed ResNet-50).
Comparison for a given size budget. To ensure a fair comparison, we compare our method for a given model size budget against the reference methods in Table 2 It should be noted that our method can further benefit from advances in semi-supervised learning to boosts the performance of the non-compressed and hence of the compressed network.
Ablation study. We perform an ablation study on the vanilla ResNet-18 to study the respective effects of quantizing using the activations and finetuning by distillation (here, finetuning refers both to the per-layer finetuning and to the global finetuning after the quantization described in Section 3 We refer to our method as Act + Distill. First, we still finetune by distillation but change the quamti- zation: instead of quantizing using our method (see Equation (2). we quantizing using the standard PQ algorithm and do not take the activations into account, see Equation (I). We refer to this method as No act Distill. Second, quantize using our method but perform standard finetuning using
7

---

Published as a conference paper at ICLR 2020
Table 2: Best top-1 accuracy ImageNet for given size budget (no architecture constraint).
Size budget
Best previous published method
Ours
~ MB
70.90% % (HAQ (Wang et al.
2018a
MobileNet v2)
64.01% % (vanilla ResNet-188
~55 MB
71.74% (HAQ Wang et al.
2018a
MobileNet vl)
76.12% (semi-sup.Resile ResNet-50)
10 MB
75.30% (HAQ Wang et al.
2018a
ResNet-50)
77.85 % Ceemisupp.ResNet-S5
Table 3: Ablation study on ResNet- 18 (test top-1 aceuracy ImageNet).
Compression
Centroids A
No act Distill
Act Labels
Act + Distill (ours)
256
64.76
65.55
65.81
512
66.31
66.82
67.15
Small blocks
1024
67.28
67.53
67.87
2048
67.88
67.99
68.26
256
60.46
61.01
61.18
512
63.21
63.67
63.99
Large blocks
1024
64.74
65,48
65.72
2048
65.94
66.21
66.50
the image labels (Act Labels). The results are displayed in Table 3 Our approach consistently yields significantly better results. As side note, quantizing all the layers of ResNet-18 with the standard PQ algorithm and without any finetuning leads to top-1 accuracies below 25% for all oper- ating points, which illustrates the drift accuracy occurring when compressing deep networks with standard methods (as opposed to our method).
4.3 IMAGE DETECTION RESULTS
To demonstrate the generality of our method, we compress the Mask R-CNN architecture used for image detection many real-life applications (He et al. (017). We compress the backbone (ResNet- 50 FPN) in the small blocks compression regime and refer the reader to the open-sourced compressed model for the block sizes used in the various heads of the network. We use = 256 centroids for every layer. We perform the fine-tuning (layer-wise and global) using distributed training V100 GPUs. Results are displayed Table We argue that this provides an interesting point of comparison for future work aiming compressing such architectures for various applications.
CONCLUSION
We presented a quantization method based on Product Quantization that gives state of the art re- sults ResNet architectures and that generalizes other architectures such Mask R-CNN. Our compression scheme does not require labeled data and the resulting models are byte-aligned, al- lowing for efficient inference on CPU. Further research directions include testing our method on wider variety of architectures. In particular, our method can be readily adapted to simultaneously compress and transfer ResNets trained on ImageNet to other domains. Finally, we plan to take the non-linearity into account to improve our reconstruction error.
Table Compression results for Mask R-CNN (backbone ResNet-50 FPN) for 256 centroids (compression factor 26x).
Model
Size
Box AP
Mask AP
Non-compressed
170 MB
37.9
34.6
Compressed
6.51 MB
33.9
30.8
8

---

Published as a conference paper at ICLR 2020
REFERENCES
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning.
arXiv preprint
1rX.vv:905.02249, 2019.
Miguel Á. Carreira-Perpiñaee and Yerlan Idelhayev. Model compression as constrained optimization, with application neural nets. part ii: quantization, 2017.
Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang. survey of model compression and acceleration for deep neural networks. CoRR, 2017.
Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. Towards the limit of network quantization. CoRR, 2016.
Matthieu Courbariaux, Yoshua Bengio, and Yean-Piere David. Binaryconnect: Training deep neural networks with binary weights during propagations. CoRR, 2015.
Deng, W. Dong, R. Socher, L.-J. Li. Li, and L. Fei-Fei. ImageNet: A Large-Scall Hierarchical Image Database. In Conference on Computer Vision and Pattern Recognition, 2009.
Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. Exploiting linear structure within convolutional networks for efficient evaluation. Advances in Neural Informa- tion Processing Systems 27. 2014.
Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. Optimized product quantization. IEEE Trans. Patlern Anal. Mach. Intell., 2014.
Yunchao Gong, Liu Liu, Ming Yang, and Lubomir Bourdev. Compressing deep convolutional net- works using vector quantization. arXiv preprint arXiv:1412.6115, 2014.
Yunhui Guo. A survey on methods and theories of quantized neural networks. CoRR, 2018
Song Han, Huizi Mao, and William Dally, Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. International Conference on Learning Representations, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CORR, 2015.
Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. Mask r-cnn. International Confer- ence on Computer Vision (ICCV), 2017.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. NIPS Deep Learning Workshop, 2014.
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang. Vijay Vasudevan, Quoc V. Le, and Hartwig Adam. Searching for mobilenetv3. arXiv e-prints, 2019.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. Conference on Computer Vision and Pattern Recognition, 2017.
Forrest landola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 00.5mb model size. CoRR, 2016.
Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEBE Transactions Pattern Analysis and Machine Intelligence, 2011.
Hervé Jégou, Matthijs Douze, and Cordelia Schmid. Product Quantization for Nearest Neighbor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2.1
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in Neural Information Processing Systems. 2012.

---

Published as a conference paper at ICLR 2020
Yann LeCun, John S. Denker, and Sara A. Solla. Optimal brain damage. In Advances in Neural Information Processing Systems, 1990.
Fengfu Li and Bin Liu. Ternary weight networks. CoRR, 2016.
Xiaofan Lin, Cong Zhao, and Wei Pan. Towards accurate binary convolutional neural network. CoRR, 2017.
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learn- ing efficient convolutional networks through network slimming. International Conference on Computer Vision, 2017.
Raphael Gontijo Lopes, Stefano Fenu, and Thad Starner. Data-free knowledge distillation for deep neural networks, 2017.
Jian-Hao Luo, Jianxin Wu, and Weiyao Lin. Thinet: A filter level pruning method for deep neural network compression. CoRR, 2017.
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet V2: practical guidelines for efficient CNN architecture design. CoRR, 2018.
Dhruv Mahajan, Ross B. Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. CoRR, 2018.
Mark D. McDonnell. Training wide residual networks for deployment using a single bit for each weight, 2018.
Asit K. Mishra and Debbic Marr. Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy. CoRR, 2017.
Asit K. Mishra, Eriko Nurvitadhi, Jeffrey J. Cook, and Debbie Marr. WRPN: wide reduced-precision networks. CoRR, 2017.
Mohammad Norouzi and David Fleet. Cartesian k-means. Conference on Computer Vision and Pattern Recognition, 2013.
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xeor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision, 2016.
Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chich Chen. Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and seg- mentation. CoRR, 2018.
Oran Shayer, Dan Levi, and Ethan Fetaya. Learning discrete weights using the local reparameteri- zation trick. CoRR, 2017.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks, 2019.
Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. The new data and new challenges in multimedia research. CoRR, 2015.
Frederick Tung and Greg Mori. Deep neural network compression by in-parallel pruning- quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018.
Kuan Wang, Zhijian Liu, Yujun Lin andx Ji Lin, and Song Han. HAQ: hardware-aware automated quantization. CoRR, 2018a.
Kuan Wang, Zhijian Liu, Yujum Lin, Ji Lin, and Song Han. Haq: hardware-aware automated quan- tization. arXiv preprint arXiv: 1. 1.08886, 2018b.
10

---

Published as a conference paper at ICLR 2020
Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Conference on Computer Vision and Pattern Recognition, 2017.
I. Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale semi- supervised learning for image classification. arXiv e-prints, 2019.
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. CoRR, 2017.
Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, and Yurong Chen. Incremental network quantiza- tion: Towards lossless cnns with low-precision weights. CoRR, 2017.
Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. CoRR, 2016.
Chenzhuo Zhu, Song Han, Huizi Mao, and William J. Dally. Trained temnrry quantization. CoRR, 2016.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. Learning transferable architectures for scalable image recognition. CoRR, 2017.


Reviews:
[{"review_id": "ryegpfeaYS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper suggests a quantization approach for neural networks, based on the Product Quantization (PQ) algorithm which has been successful in quantization for similarity search. The basic idea is to quantize the weights of a neuron/single layer with a variant of PQ, which is modified to optimize the quantization error of inner products of sample inputs with the weights, rather than the weights themselves. This is cast as a weighted variant of k-means. The inner product is more directly related to the network output (though still does not account for non-linear neuron activations) and thus is expected to yield better downstream performance, and only requires introducing unlabeled input samples into the quantization process. This approach is built into a pipeline that gradually quantizes the entire network.\n\nOverall, I support the paper and recommend acceptance. PQ is known to be successful for quantization in other contexts, and the specialization suggested here for neural networks is natural and well-motivated. The method can be expected to perform well empirically, which the experiments verify, and to have potential impact.\n\nQuestions:\n1. Can you comment on the quantization time of the suggested method? Repeatedly solving the EM steps can add up to quite an overhead. Does it pose a difficulty? How does it compare to other methods?\n2. Can you elaborate on the issue of non-linearity? It is mentioned only briefly in the conclusion. What is the difficulty in incorporating it? Is it in solving equation (4)? And perhaps, how do you expect it to effect the results?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "rklaWryJoH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The suggested method proposes a technique to compress neural networks bases on PQ quantization. The algorithm quantizes matrices of linear operations, and, by generalization, also works on convolutional networks. Rather than trying to compress weights (i.e. to minimize distance between original and quantized weights), the algorithm considers a distribution of unlabeled inputs and looks for such quantization which would affect output activations as little as possible over that distribution of data. The algorithm works by splitting each column of W_ij into m equal subvectors, learning a codebook for those subvectors, and encoding each of those subvectors as one of the words from the codebook.\n\nThe method provides impressive compression ratios (in the order of x20-30) but at the cost of a lower performance. Whether this is a valuable trade-off is highly application dependent.\n\nOverall I find the paper interesting and enjoyable. However, as I am not an expert in the research area, I can not assess how state of the art the suggested method is.\n\nThere are a few other questions that I think would be nice to answer. I will try to describe them below:\n\nSuppose we have a matric W_{ij} with dimensions NxM where changing i for a given j defines a column. By definition, linear operation is defined \ny_i = sum_j W_ij x_j . Now say each column of matrix W is quantized into m subvectors. We can express W_ij in the following way:\nW_ij = (V^1_ij + V^2_ij + ... V^m_ij)x_j where V^m_ij is zero everywhere except for the rows covering a given quantized vector.\nFor example, if W had dimensions of 8x16 and m=4, \nV^2_{3,j}=0, for all j, V^2_{4,j}=non_zero, V^2_{7,j}=non_zero, V^2_{8,j}=0, V^2_{i=4:8,j}=one_of_the_quantized_vectors.\n\ny_i = sum_j W_ij x_j = sum_k sum_j (V^k_ij) x_j =def= sum_k z^k_i where z^k are partial products: z^k_i=0 for i<k*N/m and i>(k+1)N/m\n\nThus, the suggested solution effectively splits the output vector y_i into m sections, defines sparse matrices V^k_{ij} 1<=k<=m, and performs column-wise vector quantization for these matrices separately.\n\nGenerally, it is not ovious or given that the current method would be able to compress general matrices well, as it implicitly assumes that weight W_{ij} has a high \"correlation\" with weights W_{i+kN/m,j} (which I call \"vertical\" correlation), W_{i,k+some_number} (which I call \"horizontal\" correlation) and W_{i+kN/m,k+some_number} (which I call \"other\" correlation). It is not given that those kind of redundancies would exist in arbitrary weight matrices.\n\nNaturally, the method will work well when weight matrices have a lot of structure and then quantized vectors can be reused. Matrices can have either \"horizontal\" or \"vertical\" redundancy (or \"other\" or neither). It would be very interesting to see which kind of redundancy their method managed to caprture.\n\nIn the 'horizontal' case, it should work well when inputs have a lot of redundancy (say x_j' and x_j'' are highly correlated making it possible to reuse code-words horizontally within any given V^k: V^k_ij'=V^k_ij''). However, if thise was the case, it would make more sense to simply remove redundancy by prunning input vector x_j by removing either x_j' or x_j'' from it. This can be dome by removing one of the outputs from the previous layer. This can be a symptom of a redundant input.\n\nAnother option is exploiting \"vertical\" redundancy: this happens when output y_i' is correlated with output y_{i'+N/m}. This allows the same code-word to be reused vertically. This can be a symptom of a redundant output. It could also be the case that compressibility could be further subtantially improved by trying different matrix row permutations. Also, if one notices that y_i' ir correlated with y_i'', it might make sense to permute matrix rows in such a way that both rows would end up a multiple N/m apart. It would be interesting to see how this would affect compressibility.\n\nThe third case is when code words are reused in arbitrary cases. \n\nGenerally, I think that answering the following questions would be interesting and could guide further research:\n1. It would be very interesting to know what kind of code-word reusa patterns the algorithm was able to capture, as this may guide further research.\n2. How invariance copressibility is under random permutations of matrix rows (thus also output vectors)?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "S1e82d0HqB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use codes and codebooks to compress the weights. The authors also try minimizing the layer reconstruction error instead of weight approximation error for better quantization results.\nDistillation loss is also used for fine-tuning the quantized weight. Empirical results on resnets show that the proposed method has a good compression ratio while maintaining competitive accuracy.\n\nThis paper is overall easy to follow. My main concern comes from the novelty of this paper. The two main contributions of the paper: \n(1) using codes and codebooks to compress weights; and \n(2) minimizing layer reconstruction error instead of weight approximation error\nare both not new. For instance, using codes and codebooks to compress the weights has already been used in [1,2].  A weighted k-means solver is also used in [2], though the \"weighted\" in [2] comes from second-order information instead of minimizing reconstruction error. In addition, minimizing reconstruction error has already been used in low-rank approximation[3] and network pruning[4]. \nClarification of the connections/differences, and comparison with these related methods should be made to show the efficacy of the proposed method.\n\nIt is not clear how the compression ratio in table 1 is obtained. Say for block size d=4, an index is required for each block, and the resulting compression ratio is at most 4 (correct me if I understand it wrong).\nCan the authors provide an example to explain how to compute the compression ratio? \n\n[1]. Model compression as constrained optimization, with application to neural nets. part ii: quantization. \n[2]. Towards the limit of network quantization.\n[3]. Efficient and Accurate Approximations of Nonlinear Convolutional Networks.\n[4]. ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression. \n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "H1gAi6GdtH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses to compress the network weights by quantizing their values to some fixed codeword vectors. The authors aim to reduce the distortion of each layer rather than the weight distortion. The proposed algorithm first selects the candidate codeword vectors using k-means clustering and fine-tune them via knowledge distillation. The authors verify the proposed algorithm by comparing it with existing algorithms for ResNet-18 and ResNet-50.\n\nOverall, I think that the proposed algorithm is easy to apply and the draft is relatively well written. Some questions and doubts are listed below.\n\n-In k-means clustering (E-step and M-step), is it correct to multiply \\tilde x to (c-v)? I think that the error arising from quantizing v into c is only affected by a subset of rows of \\tilde x. For example, if v is the first subvector of w_j, then I think that only 1-st, m+1-th, 2m+1-th, … rows of \\tilde x affect to the error.\n\n-Does minimizing reconstruction error minimizes the training loss (before any further fine-tuning) compared to naïve PQ? If not, \n\n-Is there any guideline for choosing the optimal number of centroids and the optimal block size given a target compression rate?\n\n-Is there any reason not comparing the proposed algorithm with other compression schemes? (e.g., network pruning and low-rank approximation) \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses to compress the network weights by quantizing their values to some fixed codeword vectors. The paper is well written, and is overall easy to follow. The proposed algorithm is well-motivated, and easy to apply. The method can be expected to perform well empirically, which the experiments verify, and to have potential impact. On the other hand, the novelty is not very high, though this paper uses these existing techniques in a different setting.
Decision: Accept (Spotlight)