# A Hybrid DenseNet-LSTM Model for Epileptic Seizure Prediction
## 3 Proposed Method: DenseNet-LSTM
### Deep Learning Architecture
#### 3.3.2 Lstm  
LSTM is a special structure of RNN, a field of deep learning, and solves the long-term dependency problem. The long-term dependency problem says when past information is not delivered to the end. By solving these problems, LSTM shows good performance in analyzing and predicting not only short sentences but also long data such as voice and video and time-series data.  
Figure 7 shows the structure of the LSTM. The top line in Figure 7 is the cell state, which is the core of the LSTM. The cell state flows like a conveyor belt, adding and subtracting information through the gate and sending the information to the next level. It's also makes the previous information directly influence the future output. LSTM basically goes through four steps. The first step is the forget gate layer, expressed by Equation (1). In this step, it is used to decide what information to forget by the sigmoid layer. \(x_{t}\in\mathbb{R}^{d}\) is the input vector to the LSTM unit, \(h_{t-1}\in(0,1)\) is the previous hidden state vector which can be seen as the output vector of the previous LSTM unit. \(W_{f}\in\mathbb{R}^{d}\), \(b_{f}\in\mathbb{R}^{d}\) also means the weight matrices and bias vector parameters for forget layer which need to be optimized during model training. \(\sigma\) is a sigmoid function, the sigmoid functions return values (\(y\) axis) in the range 0 to 1, and the LSTM unit will select which value in the range of 0 to 1 to forget. The second step is the input gate layer of Equation (2) and the tanh layer of Equation (3). The input gate layer determines which values to update through the sigmoid layer, and the tanh layer creates a new candidate value of \(\widetilde{C}_{t}\) which is a cell input activation vector. Finally, the values of the two layers are added and appended to the cell state. The third step is to create a new cell state by updating the past state as shown in Equation (4). First, the information decided to be dropped through the forget gate is discarded, and the information decided to be added is appended next. The last step is the part to decide which value to output with  
Figure 6: Bottleneck layer and transition layer. Batch normalization is performed independently for each feature map, the ReLU is a piecewise linear function that will output directly if the input is positive; otherwise, it will output zero. As for the \(1\times 1\) convolution is used to reduce the number of feature maps to improve the computational efficiency.  
the output gate layer of Equation (5). First, determine which part of the cell state is to be exported through sigmoid for input data, and determine the final output by multiplying the value obtained through the tanh layer in the cell state as shown in Equation (6).  
\[f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}+b_{f}]) \tag{1}\]  
\[i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}+b_{i}]) \tag{2}\]  
\[\widetilde{C}_{t}=\tanh(W_{c}\cdot[h_{t-1},x_{t}]+b_{c}) \tag{3}\]  
\[C_{t}=(f_{t}\times C_{t-1})+(i_{t}\times\widetilde{C}_{t}) \tag{4}\]  
\[O_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{5}\]  
\[h_{t}=o_{t}\times tanh(C_{t}) \tag{6}\]