## 3 Preliminary Knowledge
### Post-Training Quantization  
Post-training quantization refers to the quantization of model parameters after training when the weights of the model are fixed. Generally, the quantization parameters that minimize the quantization loss of each layer of the network are obtained through the validation set. The quantization process could be generally divided into dynamic quantization and static quantization.  
Figure 2: STE diagram.  
Post-training dynamic quantification  
This method is to quantize the operations after the convergence of the floating-point model training, where the weights are quantized in advance. The activation output needs to be dynamically quantified in the forward propagation process during inference. During quantization, each layer calculates a quantization parameter to obtain the maximum and minimum input and then quantizes them. In the actual calculation process, each layer needs to be traversed, which increases the calculation cost.  
Post-training static quantification  
The weights are quantized in advance in this method. Moreover, the activation output is quantized based on the fixed quantization parameters recorded in the small sample calibration process (the maximum and minimum values of each layer). There is no recalculation of quantization parameters in the whole process, so the computational cost is low However, the risk of accuracy drop is relatively high because small samples often cannot accurately reflect the data distribution and data range of the whole dataset.  
Taking our commonly used uniform linear quantization as an example, we briefly describe the traditional post-training quantization method. Assuming that the weights of convolution are w, the input is x, and the weight quantization parameters obtained in advance are \(w_{max}\) and \(w_{min}\), the input quantization parameters are \(x_{max}\). Formulas (5) and (6) show the process that input and weights are linearly adjusted to \((-1,1)\):  
\[x^{\prime}=\frac{2(x-x_{min})}{x_{max}-x_{min}}-1 \tag{5}\]  
\[w^{\prime}=\frac{2(w-w_{min})}{w_{max}-w_{min}}-1 \tag{6}\]  
The normalized input \(x^{\prime}\) and weights \(w^{\prime}\) are multiplied by a magnification S and rounded to an integer (if \(S\) is 128, then the new \(x^{\prime\prime}\) and \(w^{\prime\prime}\) are converted to int8):  
\[x^{\prime\prime}=round\big{(}Sx^{\prime}\big{)} \tag{7}\]  
\[w^{\prime\prime}=round\big{(}Sw^{\prime}\big{)} \tag{8}\]  
The core of the convolutional neural network is the matrix multiplication of \(w\) and \(x\). In the following, we use a series of dequantization operations to simplify this multiplication. Formulas (7) and (8) are obtained from formulas (5) and (6):  
\[x=x_{min}+\frac{(x^{\prime}+1)(x_{max}-x_{min})}{2} \tag{9}\]  
\[w=w_{min}+\frac{(w^{\prime}+1)(w_{max}-w_{min})}{2} \tag{10}\]  
Multiply formulas (7) and (8):  
\[w\cdot x=\frac{(x_{max}-x_{min})(w_{max}-w_{min})}{4}w^{\prime}x^{\prime}+\frac {(x_{max}-x_{min})(w_{max}+w_{min})}{4}x^{\prime}\]  
\[+\frac{(x_{max}+x_{min})(w_{max}-w_{min})}{4}w^{\prime}+\frac{(x_{max}+x_{min} )(w_{max}+w_{min})}{4} \tag{11}\]  
Since \(x_{max}\), \(x_{min}\), \(w_{max}\), \(w_{min}\) are known constants, the formula is simplified with \(k_{1}\), \(k_{2}\), \(k_{3}\), \(k_{4}\):  
\[w\cdot x=k_{1}w^{\prime}x^{\prime}+k_{2}x^{\prime}+k_{3}w^{\prime}+k_{4} \tag{12}\]  
\[w\cdot x\approx\frac{k_{1}w^{\prime\prime}x^{\prime\prime}}{S^{2}}+\frac{k_{2} x^{\prime\prime}}{S}+\frac{k_{3}w^{\prime\prime}}{S}+k_{4} \tag{13}\]  
In short statistical input is needed, the amount of calculation is large, and the computational complexity of inverse quantization is high. For static post-quantization, the clip value obtained by small samples cannot fully reflect the data distribution of the whole dataset, which inevitably leads to accuracy loss. Therefore, in the next section, we propose a simple and efficient quantitative method with little accuracy loss.