Network quantization is one of network compression techniques employed to reduce the redundancy of deep neural networks. It compresses the size of the storage for a large number of network parameters in a neural network by quantizing them and encoding the quantized values into binary codewords of smaller sizes. In this paper, we aim to design network quantization schemes that minimize the expected loss due to quantization while maximizing the compression ratio. To this end, we analyze the quantitative relation of quantization errors to the loss function of a neural network and identify that the Hessian-weighted distortion measure is locally the right objective function that we need to optimize for minimizing the loss due to quantization. As a result, Hessian-weighted k-means clustering is proposed for clustering network parameters to quantize when fixed-length binary encoding follows. Hessian-weighting properly handles the different impact of quantization errors not only within layers but also across layers and thus it can be employed for quantizing all layers of a network together at once; it is beneficial since one can avoid layer-by-layer compression rate optimization. When optimal variablelength binary codes, e.g., Huffman codes, are employed for further compression of quantized values after clustering, we derive that the network quantization problem can be related to the entropy-constrained scalar quantization (ECSQ) problem in information theory and consequently propose two solutions of ECSQ for network quantization, i.e., uniform quantization and an iterative algorithm similar to Lloyd's algorithm for k-means clustering. Finally, using the simple uniform quantization followed by Huffman coding, our experiment results show that the compression ratios of 51.25, 22.17 and 40.65 are achievable (i.e., the sizes of the compressed models are 1.95%, 4.51% and 2.46% of the original model sizes) for LeNet, ResNet and AlexNet, respectively, at no or marginal performance loss.
REF proposes entropyconstrained network quantization schemes, further improving network compression ratio.
Towards the Limit of Network Quantization
ArXiv
ArXiv
Computer Science