## 2 Theoretical Background Study
### Lstm  
Extended short-term memory models (LSTM) [5; 6; 7] are a type of recurrent neural network (RNN) and is an algorithm that has recently been in the spotlight along with convolutional neural networks (CNN).  
RNN is an artificial neural network in which directed edges connect hidden nodes to form a directed cycle. This algorithm is a model suitable for processing data that appears sequentially, such as voice and text.  
It is known that the RNN's learning ability is significantly reduced as the gradient during backpropagation gradually decreases when the distance between the relevant information and the point where the information is used is long. This is called a vanishing gradient problem. To address this issue, LSTM was created. LSTM is a structure that adds a cell state to the hidden form of RNN.  
LSTM recurrent neural network is a model that has emerged to overcome the gradient loss and gradient explosion problems of RNN. On the other hand, LSTM makes erase gates, input gates, and output gates to continuously add inputs to long-term memory so they can be remembered for a long time. In addition, LSTM is a model created to compensate for the shortcomings of RNN. The RNN has a problem because the font information cannot be transmitted backward as the time step extends, so the LSTM supplements this by adding a cell state. In addition, LSTM has the characteristic of erasing bulky memories by adding input gates, forgetting gates, and output gates to the memory cells of the hidden layer and storing things to be remembered.  
A characteristic of LSTM is the ability to perform learning that requires a long-term dependence period while maintaining past data by introducing the cell state concept to solve the vanishing gradient problem of RNN. In addition, LSTM has the advantage that it is easy to process time series and sequence data by storing information for a long time. However, there is a disadvantage in that the operation speed is slow.  
In addition, LSTM has a chain-like structure similar to RNN, but unlike only hidden states, LSTM is designed to exchange information uniquely called cell states. As can be seen from the LSTM structure in Figure 1, the cell state includes a forget gate that determines which of the past information about the input data passed from the previous cell to reflect, an input gate that determines how much of the current input information will be remembered, It consists of an output gate that determines how much data will be reflected in the data to be transmitted to the next cell [8].  
Once again, the above structure is described as follows. In the LSTM structure, ht is a short-term state, and ct is a long-term state. The memory cell ct stores the LSTM memory at time t. It is assumed that the memory required from the past to time t is stored. Based on this memory containing all the necessary information, the hidden state ht is output to the outer layer. At this time, the output ht is the value converted by the tanh function, as shown in the following figure. Here, tanh means to apply the tanh function to each element. Additionally, the core of this structure is that the hidden state ht is calculated using ct obtained using three inputs (xt, ht\({}_{-1}\), ct\({}_{-1}\)).  
Figure 1: The basic structure of LSTM.