# Sequential Characteristics Based Operators Disassembly Quantization Method for LSTM Layers
## 4 Experiments
### Implementation Details
#### 4.1.2 Experimental Details  
The methodology described in Section 3 was implemented as two software tools based on the Caffe. These tools are quantization and simulation software, which are designed to complete the generation of quantization parameters and the NPU calculation simulation [24]. The quantization software pre-analyzes the distribution and range of data at each layer of the network model, calculates the quantization parameters, and generates a guide file for the NPU to map floating-point data to low-precision data. Then,  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Experiment Type** & **Application/Platform** & \begin{tabular}{c} **Benchmark** \\ **Datasets** \\ \end{tabular} & **Evaluation Metric** \\ \hline \multirow{4}{*}{\begin{tabular}{c} Comparison \\ experiments \\ \end{tabular} } & Character-level language prediction & Penn Treebank/War and Peace & BPC/Variation/Size \\ \cline{2-4}  & Word-level language prediction & Penn Treebank & \begin{tabular}{c} PPW/Variation/ \\ Relative MSE \\ \end{tabular} \\ \cline{2-4}  & Image classification & UCF101 & Acc Loss (Top1) \\ \hline \multirow{4}{*}{\
\begin{tabular}{c} Ablation \\ experiments \\ \end{tabular} } & \(\text{W}/\text{ and W}/\text{O}\) sequential quantization calibration & UCF101 & Acc Loss (Top1) \\ \cline{2-4}  & Comparison between clipping methods & UCF101 & Acc Loss (Top1) \\ \hline \multirow{2}{*}{
\begin{tabular}{c} Performance \\ experiments \\ \end{tabular} } & PC(CPU/GPU) & / & Running speed \\ \cline{2-4}  & NPU & / & Running speed \\ \hline \hline \end{tabular}
\end{table}
Table 2: Experimental settings and evaluation metric.  
the simulation software simulates the runtime and NPU calculation process to verify the correctness of the hardware calculation results.  
The inputs of the quantization software shown in Figure 4 include the structure file (protobuf) and the parameter file (caffemodel) of the model, and the quantization calibration images. The output is the structure file with the quantization parameters added. The quantization parameters are used to guide the NPU to map floating-point data into integer data. Therefore, the structure file with the quantization parameters added, the parameter file, and the test image dataset make up the inputs to the simulation software. Its output is the accuracy calculation results.  
In detail, this is accomplished by taking two steps for each layer: one for the weights and one for the activations. For the weight, count the weight threshold \(c\) of the current layer according to the value range, truncate the weight of the current layer to the \([-c,c]\) interval, and then use the uniform-symmetric quantization method to quantize the weight, as shown in the following equation:  
\[quantize(w,n,c)=round(clamp(w,c)/s)\times s \tag{18}\]  
where the scaling factor \(s\) is obtained by the weight threshold \(c\) and the quantization bit width \(n\):  
\[s=c/(2^{n-1}-1) \tag{19}\]  
For activation values, the quantization method is similar, except for one point, where the activation threshold \(t\) is chosen by finding the optimal value \(x\) that minimizes the KL divergence in the original activation value distribution and the quantized activation value distribution:  
\[t=\underset{x}{\text{argmin}}D_{KL}(a\bigg{|}quantize(a,n,x)) \tag{20}\]  
Therefore, the quantization algorithm is deployed on the embedded neural network accelerator according to the weights and activation thresholds generated above, namely quantization parameters. According to the quantization parameter threshold and bit width of the current layer, the quantization coefficient is calculated according to Equation (19). Then, the signed integer weight is calculated according to Equation (18) and the activation thresholds are generated according to Equation (20). Finally, the output of the last layer of NPU calculation is inverse-quantized into floating-point values.  
Figure 4: Inputs and outputs of quantization and simulation software.