# Sequential Characteristics Based Operators Disassembly Quantization Method for LSTM Layers
## 4 Experiments
### Ablation Experiments
#### 4.3.2 Comparision between Clipping Methods  
The goal of a quantization algorithm is to generate quantization parameters. The activation threshold for each layer is a quantization parameter that is selected according to the dynamic range of the activation value. After investigation, LSTM layer activation quantization clipping methods include KL-divergence, MIN-MAX, AVG-MAX, Easy Quant, ADMM, etc. Here, the first three were selected for the ablation experiments on the UCF101 dataset.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & **Model** & \begin{tabular}{c} **Sequential** \\ **Quantization** \\ \end{tabular} & **Bit-Width** & **Origin** & \begin{tabular}{c} **ConvNet** \\ **Proportion** \\ \end{tabular} & \
\begin{tabular}{c} **FCNet** \\ **Proportion** \\ \end{tabular} &
\begin{tabular}{c} **RNN** \\ **Proportion** \\ \end{tabular} & **Acc Loss** \\ \hline
[37] & CLDNN & none & All 8 & All 16 & 20\% & 60\% & 20\% & <2.0\% \\ \hline
[11] & LRCN & none & 9/8/8/8/9/8/8/8/8/8/8 & All 16 & 50\% & 30\% & 20\% & <1.8\% \\ \hline
[38] & AlexNet-LSTM & none &
\begin{tabular}{c} AlexNet9/8/8/6/6/8/8/8, \\ others:12 \\ All 8 \\ \end{tabular} & All 16 & 50\% & 30\% & 20\% & <1.8\% \\ \hline
[11] & Vgg16-LSTM & none & Vgg16:8,others:12 & All 16 & 76.19\% & 14.29\% & 9.52\% & <2.0\% \\ \hline
[11] & Vgg16-LSTM & none & All 8 & All 32 & 76.19\% & 14.29\% & 9.52\% & <1.7\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Quantization accuracy loss on LSTM-based models.  
Table 8 shows the accuracy calculation results of different test models under different activation clipping methods, where \" indicates that the corresponding activation clipping method was added. It can be seen from the accuracy loss results that no matter which test model it is based on, and no matter which activation quantization clipping method is used, the quantization loss using the proposed method only differs by 0.803% in the best accuracy and 2.812% in the worst.  
In order to show the specific values of the inference accuracy on PC and the calculation accuracy on NPU in more detail, we have plotted the following comparative experiment based on different test models. The quantization accuracy experiments in Figure 8 show that, compared with the inference accuracy on PC, the model deployed on the NPU using the proposed method reflects high accuracy loss and the practical value of the proposed quantization method.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Sequential Quantization** & **Inference Acc** & **Quantization** & \multirow{2}{*}{**Acc Loss**} \\  & **Quantization** & **on PC** & **Acc on NPU** & \\ \hline \multirow{2}{*}{AlexNet-LSTM} & none & \multirow{2}{*}{52.808\%} & 51.264\% & \(-\)1.544\% \\  & & & 52.005\% & \(\mathbf{-0.803\%}\) \\ \hline \multirow{2}{*}{Vgg16-LSTM} & none & \multirow{2}{*}{56.889\%} & 54.182\% & \(-\)2.707\% \\  & & & 55.209\% & \(\mathbf{-1.68\%}\) \\ \hline \hline \end{tabular}
\end{table}
Table 7: Experiment on sequential quantization calibration.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Sequential Quantization** & **KL-** & \multirow{2}{*}{**MIN-MAX**} & **AVG-MAX** & **Acc Loss** \\  & **Quantization** & **Divergence** & & & \(-\)0.803\% \\ \hline \multirow{2}{*}{
\begin{tabular}{c} AlexNet- \\ LSTM \\ \end{tabular} } & \multirow{2}{*}{} & \multirow{2}{*}{} & \multirow{2}{*}{} & \(-\)1.104\% \\  & & & & \(-\)0.892\% \\ \hline \multirow{2}{*}{Vgg16-LSTM} & \multirow{2}{*}{} & \multirow{2}{*}{} & \multirow{2}{*}{} & \(-\)1.68\% \\  & & & & \(-\)2.075\% \\ \end{tabular}
\end{table}
Table 8: Experiment on activation quantization clipping.  
Figure 8: Detailed comparative experiment on different activation quantization clipping methods.  
Therefore, these two ablation experiments show that the operator disassembly operation of the LSTM layer can solve the problem that the existing NPU cannot directly compute the LSTM layer.