# Advances in the Neural Network Quantization: A Comprehensive Review
## 2 Quantization Fundamentals  
Neural network models use floating-point data types. To achieve high-speed computations, it is necessary to quantize the floating-point neural network models into integer neural networks. As shown in Figure 1, quantization reduces computational cost by decreasing the precision of the data type, which minimizes the bit-width of data storage and the amount of data passing through the deep neural network. Computing and storing data at lower bit-widths enable fast inference and reduces energy consumption. The quantization process of a neural network model involves choosing a quantization mapping strategy, deciding whether to train the model, computing the quantization parameters, generating the inverse quantization parameters, and deploying the inference on resource-limited devices. When performing the quantization process on a neural network model, the activation and weight values of the model are restricted to a discrete set of numbers that can have different distributions: uniform or non-uniform. The non-uniform quantization method is a logarithmic distribution [24]. Inference of neural network models in hardware platforms requires processing of quantized computational results through certain inverse quantization computational steps to obtain a true floating-point type output. Non-uniform quantization methods have complex inverse quantization computation processes and are not friendly to hardware platforms. Most current hardware supports only uniform quantization methods. Therefore, the most widely used quantization scheme is the uniform distribution [25] method with uniform step size. The key issue of quantization is to design a proper quantization mapping function and a proper method to calculate quantization parameters. For uniform quantization, most existing approaches use either asymmetric or symmetric quantization mapping functions [26]. The asymmetric quantization mapping function is as follows:  
\[r=f(Q)=s\cdot Q+D \tag{1}\]  
\[Q=f^{-1}(r)=round\bigg{(}\frac{r-D}{s}\bigg{)} \tag{2}\]  
where \(f\) and \(f^{-1}\) are the quantization mapping function, \(f^{-1}\) is the inverse function of \(f\), round is the rounding operation, \(r\) is the floating-point real value, \(Q\) is the integer value after quantization, \(s\) and \(D\) are quantization parameters, \(s\) is the scaling factor, and \(D\) is the zero point, chosen such that the 0 value would exactly map to quantized values. Symmetric quantization is a simplified version of the general asymmetric case [27]. The symmetric quantizer restricts the quantization parameter \(D\) to 0 [28]. In Figure 2, we take symmetric quantization to 8-bit as an example. We can see that quantization converts continuous floating-point data into discrete integers, which brings accuracy loss. The quantization parameters are very important for both asymmetric and symmetric quantization, and affect the performance of the quantized neural network. The quantization parameters depend on the clipping range, and the scaling factors divide the clipping range into a number of partitions. Figure 1: Quantization: Turning the 32-bit floating-point neural network to an 8-bit or even lower bit integer network. The optimal clipping range for the input is \([min_{in},max_{in}]\), the optimal clipping range for the output is \([min_{out},max_{out}]\), and the threshold of the weights is \(th_{w}\). The method to compute the quantization parameters according to the clipping thresholds is as follows: The quantization parameters \(s_{in}\), \(D_{in},s_{out}\), \(D_{out}\), \(s_{w}\) of a layer are computed according to Equations (3)-(7).