# Storm Surge Prediction Based on Long Short-Term Memory Neural Network in the East China Sea
## 2 Methods  
The forerunner of LSTM neural network is Recurrent Neural Network (RNN) [91], which is the evolution of Multi-Layer Perception and is capable of processing sequential data due to its short-term memory ability. The memory ability is realized by the so-called hidden state \(h\) which is transmitted from the former hidden layer cell to the next which is the major improvement to Feedforward Neural Network [92]. A traditional RNN structure whose hidden layer is unfolded into a full network is shown in Figure 1a, and the formulas correspondingly are shown as follows:  
\[h_{t}=f(Uh_{t-1}+Wx_{t}+b), \tag{1}\]  
\[y_{t}=Vh_{t}, \tag{2}\]  
where \(x_{t}\) is the input, \(y_{t}\) is the output, \(h_{t}\) is the hidden state which is transmitted to the next hidden layer cell. The subscript \(t\) denotes the time step. \(f\) is the nonlinear activation function which is typically applied with tanh. \(U\), \(W\), \(b\) and \(V\) are the hyperparameters to be calibrated. The cycle of the hidden state can store the information of the previous step to keep the dependency between the hidden layer cells, and it can improve the ability of learning and extracting characteristics from the sequential data. However, the long-term dependence problem [93], e.g., the vanishing or exploring of the gradient during the back-propagation calculation, cannot be well-solved. Therefore, LSTM neural network is proposed to improve RNN with the Gating Mechanism [81], selectively adding new information and forgetting previously accumulated information. The structure of LSTM is more complicated compared with RNN and is shown in Figure 1b. Compared with RNN, LSTM neural network [82; 83], as shown in Figure 2, introduces a new internal state \(c_{t}\), which deliveries information linearly to the next hidden layer cell and outputs information nonlinearly to the hidden layer's external state \(h_{t}\) (which is analogous with the hidden state \(h_{t}\) in RNN). \(c_{t}\) and \(h_{t}\) are expressed as follows:  
\[c_{t}=f_{t}\cdot c_{t-1}+i_{t}\cdot\widetilde{c}_{t}, \tag{3}\]  
\[h_{t}=o_{t}\cdot\tanh(c_{t}), \tag{4}\]  
\[\widetilde{c}_{t}=\tanh(W_{c}x_{t}+U_{c}h_{t-1}+b_{c}) \tag{5}\]  
where \(c_{t}\) is the internal state. \(f_{t}\), \(i_{t}\), \(o_{t}\) are the three gates to control the path of information transmission. The subscript \(t\) denotes the time step. \(\widetilde{c}_{t}\) is the candidate state through the nonlinear activation function tanh. \(U\), \(W\), \(b\) and \(V\) are the hyperparameters to be calibrated and the subscript \(c\) represents \(\widetilde{c}_{t}\). LSTM neural network introduces Gating Mechanism to control the path of information transmission, i.e., forget gate, input gate, and output gate. The formulas of these three gates are expressed as follows:  
\[f_{t}=\sigma\Big{(}W_{f}x_{t}+U_{f}h_{t-1}+b_{f}\Big{)}, \tag{6}\]  
\[i_{t}=\sigma(W_{i}x_{t}+U_{i}h_{t-1}+b_{i}), \tag{7}\]  
\[o_{t}=\sigma(W_{o}x_{t}+U_{o}h_{t-1}+b_{o}) \tag{8}\]  
where \(f_{t}\) is the forget gate to control how much information needs to be forgotten about the internal state \(c_{t-1}\) of the previous time step. \(i_{t}\) is the input gate to control how much information needs to be saved about the candidate state \(\widetilde{c}_{t}\) of the current time step. \(o_{t}\) is the output gate to control how much information about the internal state \(c_{t}\) needs to be output to the external state \(h_{t}\) at the current time step. \(U\), \(W\), \(b\), and \(V\) are the hyperparameters to be calibrated and the subscripts \(f\), \(i\) and \(o\) represent forget gate, input gate and output gate,  
Figure 1: The typical structure of (**a**) traditional RNN and (**b**) LSTM. respectively.