# Optimization of Linear Quantization for General and Effective Low Bit-Width Network Compression
## 1 Introduction
Quantization-aware training can more effectively ensure accuracy, but it may not be practicable because it requires full datasets, and retraining and adjusting the model is time-consuming. On the other hand, post-training quantization only needs part of the training set. Post-training quantization initially focused on 8-bit quantization, among which linear quantization is more commonly used. In 2017, TensorFlow-Lite, an automatic quantification tool that enabled on-device machine learning developed by Google, used linear quantization to automatically convert floating-point weights to 8-bit weights [13]. In the same year, NVIDIA developed the TensorRT tool which also applied linear quantization to create lightweight models. Due to the redundancy of network parameters and relatively large data representation space, 8-bit linear quantization usually does not cause a significant drop in model accuracy [13]. Currently, numerous edge devices may contain customizable parts such as FPGA, CPLD, and ASIC, to support adders and multipliers to do calculations on low bit-width weights. Using fewer bits to represent weights, model inference can directly simplify implementation and speed up execution in a hardware-friendly way [14]. However, post  
Figure 1: Synapses before and after pruning. training quantization with low bit-width usually results in a significant drop in accuracy. With traditional linear quantization, when the bit-width is under five, the model accuracy may drop to 1%. To deal with the above problem, researchers proposed to clip outliers because they generally believed that outliers in the network weights affect the results of quantization at low bit-widths [9]. For example, NVIDIA uses KL (Kullback-Leibler) divergence to reduce weight outliers in its TensorRT [15]. Figure 2 shows that quantization work based on the cropping method can reduce the accuracy drop, but still performs poorly on some models. Different from previous studies, this paper looks deeply into the process of linear quantization of network weights. Linear quantization can be considered as two stages, clustering and mapping. In the clustering stage, the weights are divided into different categories through a uniform partition. In the mapping stage, the weights in each clustered category are mapped to one fixed-point number corresponding to its maximum and minimum. The fundamental reason for the reduction of the network accuracy by traditional linear quantization is that it combines the two stages into one stage, and at low bit-widths, it cannot accurately distinguish the categories of weights according to their true distribution. To overcome the challenges, this paper proposes a two-stage quantization method to optimize the clustering function of linear quantization thus promoting the effect of low bit-width network compression. The motivation for this paper comes from a new understanding of the quantization process. To our best knowledge, this paper is the first to split the quantization process into two stages, clustering and mapping. To present a comprehensive study on post-training quantization, we also evaluate the result between clip threshold techniques and ours. The main contributions of this paper are:  
* This paper optimizes the linear quantization method with a two-stage technique. The clustering function is separated before mapping from the traditional linear quantization. Specifically, the optimized method applies a modified K-means algorithm to cluster the weights and then uses the uniform partition to map the centroids to fixed-point numbers. * The results of the K-means algorithm are greatly affected by the initial cluster centroids, which may cause non-convergence. In neural network quantization, the number of cluster centroids can be determined by the bit-width. This paper selects the particle swarm algorithm to obtain the initial cluster centroids to facilitate the convergence of clustering.