# Sequential Characteristics Based Operators Disassembly Quantization Method for LSTM Layers
## 2 Related Works  
The main acceleration targets of the current neural network accelerators on the embedded Field Programming Gate Arrays (FPGA) platform [5] are typical neural network operators, such as convolution, full connection, pooling, etc. However, with the increasing complexity of the problems solved by deep learning technology, some new types of complex operators have gradually emerged, which are mainly represented by variant form models of RNN [6], such as LSTM [7] and GRU [8]. The RNN neural network model is widely used in the field of natural language processing, but it has problems with vanishing or exploding gradients when processing large amounts of text. The improved form of RNN is LSTM, which introduces three gated structures, the forget gate, the input gate, and the output gate, in the unit, which can effectively solve this problem. The forget gate is responsible for the selective deletion of the input information and the hidden state information output by the previous unit, the input gate is responsible for selectively storing the retained new information into the unit state, and the output gate is responsible for selectively outputting the information in the unit state to the hidden layer. LSTM removes or adds information to the cell state, called gates: input gate (\(i_{t}\)), forget gate (\(f_{t}\)), output gate (\(o_{t}\)), cell input vector (\(g_{t}\)), cell state vector (\(c_{t}\)), and hidden state vector (\(h_{t}\)) can be defined as follows:  
\[i_{t}=sigmoid(W_{hi}*h_{t-1}+W_{xi}*x_{t}+b_{i}) \tag{1}\]  
\[f_{t}=sigmoid(W_{hf}*h_{t-1}+W_{xf}*x_{t}+b_{f}) \tag{2}\]  
\[o_{t}=sigmoid(W_{ho}*h_{t-1}+W_{xo}*x_{t}+b_{o}) \tag{3}\]  
\[g_{t}=\tanh(W_{hg}*h_{t-1}+W_{xg}*x_{t}+b_{g}) \tag{4}\]  
\[c_{t}=(f_{t}.*c_{t-1})+(i_{t}.*g_{t}) \tag{5}\]  
\[h_{t}=o_{t}.*\tanh(c_{t}) \tag{6}\]  
where \(W_{xi}\), \(W_{xf}\), \(W_{xo}\), \(W_{xg}\) is the weight matrix of the input node; \(W_{hi}\), \(W_{hf}\), \(W_{ho}\), \(W_{hg}\) is the weight matrix of the hidden state; and \(b_{i}\), \(b_{f}\), \(b_{o}\), \(b_{g}\) is the bias term. LSTM has been widely used in various sequential data predictions [9]. In the field of text processing, LSTM can perform text classification, text sequence annotation, text translation, image generation text, text-generated speech, speech recognition [10], etc. In the field of computer vision, LSTM is primarily used for video classification, image annotation, video annotation [11], and recently popular visual Q&A. In the field of engineering practice, computation offloading in mobile edge computing can also be predicted by LSTM [12]. The quantization for LSTM compresses the original LSTM network by reducing the numerical bit width of the model and reducing the calculation and inference time, and it has a direct relationship with the hardware. The specific quantization process for most quantization algorithms is as follows. It uses the layers as the basic processing unit. First, the distribution and range of weight and output data of each layer of the neural network model are analyzed in advance. Then, each floating-point value is represented by a low-bit integer based on the uniform-symmetric [13] quantization strategy. Finally, the quantization-calibrated weights and activation thresholds, namely quantization parameters, are calculated to generate a guide file for the neural network accelerator to map floating-point data into integer data. The quantization strategy of mapping floating-point data to integer data is to select a reasonable threshold based on the dynamic range of the data to ensure that the loss of information is minimized. Therefore, the original information is truncated before mapping through the threshold, and it is mapped to a low bit-precision range. This avoids the problem of wasted dynamic range resources. There are many quantization strategies for selecting thresholds, such as the minimum-maximum (MIN-MAX) quantization strategy used by Tengine, which makes the absolute maximum value the threshold value.