# Stock Price Prediction Using a Frequency Decomposition Based GRU Transformer Neural Network
## 3 Methods
### Hybrid GRU-Transformer
#### 3.4.2 LSTM and GRU  
Long Short-Term Memory (LSTM) network [18] is a specially designed version of RNN to handle sequential dataset. Previous RNN models only use the past information based on the time line, while gates have been proposed in LSTM to realize the function of capturing the long time important information. In the LSTM unit, there are 1 input gate, 1 forget gate and 1 output gate.  
Gate Recurrent Unit (GRU) network [19] is similar to LSTM but with simpler structure. In the GRU unit, there are 1 update gate and 1 reset gate. The structures of LSTM and GRU unit can be visualized in Figure 3.  
Figure 3: The structure of LSTM and GRU unit.  
In the LSTM unit in Figure 3, \(h_{l-1}\) and \(c_{t-1}\) can be regarded as the short and long term memories, respectively. The forget gate (\(f_{t}\)) determines the importance of information stored in the long term memory after obtaining new data at time step t. The input gate (\(i_{t}\)) indicates how the long term memory should be updated. The output gate calculates the new short term memory, taking the new long term memory into consideration. Mathematically, it can be formulated as:  
\[i_{t}=\sigma(W_{ii}x_{t}+b_{ii}+W_{hi}h_{t-1}+b_{hi}) \tag{4}\]  
\[f_{t}=\sigma(W_{if}x_{t}+b_{if}+W_{hf}h_{t-1}+b_{hf}) \tag{5}\]  
\[g_{t}=tanh(W_{ig}x_{t}+b_{ig}+W_{hg}h_{t-1}+b_{hg}) \tag{6}\]  
\[c_{t}=f_{t}\bigodot c_{t-1}+i_{t}\bigodot g_{t} \tag{7}\]  
\[h_{t}=o_{t}\bigodot tanh(c_{t}) \tag{8}\]  
* \(i_{t}\): the input gate at time t
* \(f_{t}\): the forget gate at time t
* \(g_{t}\): the candidate memory state at time t
* \(c_{t}\): the memory state at time t
* \(h_{t}\): the hidden state at time t
* \(o_{t}\): the output gate at time t
* \(\sigma\): the activation function
* \(W_{x,y}\): the weight matrix from state x to state y, where i, g, f, h, n mean the input state, the candidate memory state, the forget state, the hidden state, the new memory state, respectively
* \(b_{x,y}\): the bias term from state x to state y  
In the GRU unit, compared to LSTM, \(h_{t-1}\) contains integrate information of both long and short terms. The reset gate is similar to the function of the input gate in LSTM, while the update gate realizes the functions of the forget and output gates simultaneously. In practice, it is widely accepted that GRU performs more or less the same as LSTM with less computational burden [50]. The mathematical update process is:  
\[z_{t}=\sigma(W_{iz}x_{t}+b_{iz}+W_{hz}h_{t-1}+b_{hr}) \tag{9}\]  
\[r_{t}=\sigma(W_{ir}x_{t}+b_{ir}+W_{hr}h_{t-1}+b_{hr}) \tag{10}\]  
\[n_{t}=tanh(W_{in}x_{t}+b_{in}+r_{t}(W_{hn}h_{t-1}+b_{hn})) \tag{11}\]  
\[h_{t}=(1-z_{t})n_{t}+z_{t}*h_{t-1} \tag{12}\]  
* \(z_{t}\): the update gate at time t
* \(r_{t}\): the reset gate at time t
* \(n_{t}\): the new memory state at time t
* \(h_{t}\): the hidden state at time t
* \(\sigma\): the activation function
* \(W_{x,y}\): the weight matrix from state x to state y, where i, z, h, n mean the input state, the update state, the hidden state, the new memory state, respectively.
* \(b_{x,y}\): the bias term from state x to state y