## 2 Methodology
### LSTM: Long-Short-Term Memory  
In DL tasks, the input and output are usually specified based on the real-world application. DL models differ in the architecture of hidden layers, with the temporal relations in time series data being difficult to model using conventional DNNs. To address this issue, recurrent connections that map outputs of earlier steps to later steps are introduced to the hidden layers, resulting in RNN, as illustrated in Figure 3a, where the red arrow represents the recurrent connections. The neural network of each step \(t\) is a conventional DNN with multiple hidden layers and can usually be illustrated in the folded expression in Figure 3b for efficiency. Where the rectangular blocks represent the hidden layers and the cyclic arrows are the recurrent connections.  
Long short-term memory (LSTM) is proven to be a SOTA model of RNN in the sequence data learning tasks [44; 45; 46], thus it is employed in this study. In the LSTM model, the hidden layers are LSTM cells with gate units, as illustrated in Figure 3c and Equation (6).  
\[f^{\mathrm{(t)}}=\sigma\left(\mathbf{W}_{t}\mathbf{h}^{\mathrm{(t-1)}}+ \mathbf{U}_{t}\mathbf{x}^{\mathrm{(t)}}+\mathbf{b}_{t}\right)\] (6) \[i^{\mathrm{(t)}}=\sigma\left(\mathbf{W}_{t}\  
where \(f^{(t)},i^{(i)},o^{(i)}\) are the forget gate, input gate, and output gate, respectively.  
Two hidden states are contained in LSTM, i.e., the short-term state \(\mathbf{h}^{(\mathrm{t})}\) and the long-term state \(C^{(\mathrm{t})}\). The forward propagation of LSTM units is illustrated in Equations (6)-(9). The updating of these states at step \(t\) writes:  
\[\widetilde{\mathbf{C}}^{(\mathrm{t})}=\tanh\Bigl{(}\mathbf{W}_{\mathrm{C}}\mathbf{h}^{( \mathrm{t}-1)}+\mathbf{U}_{\mathrm{C}}\mathbf{x}^{(\mathrm{t})}+\mathbf{b}_{\mathrm{C}} \Bigr{)} \tag{7}\]  
\[\mathbf{C}^{(\mathrm{t})}=f^{(\mathrm{t})}*\mathbf{C}^{(\mathrm{t}-1)}+i^{(\mathrm{t}) }*\widetilde{\mathbf{C}}^{(\mathrm{t})} \tag{8}\]  
\[\mathbf{h}^{(\mathrm{t})}=o^{(\mathrm{t})}*\tanh\Bigl{(}\mathbf{C}^{(\mathrm{t})} \Bigr{)} \tag{9}\]  
where \(\mathbf{W}_{\mathrm{f}},\mathbf{W}_{i},\mathbf{W}_{o},\mathbf{W}_{c}\) are weights for states of the forget/input/output gate and the long-term memory state; \(\mathbf{U}_{\mathrm{f}},\mathbf{U}_{\mathrm{i}},\mathbf{U}_{o},\mathbf{U}_{c}\) are weights for inputs of the forget/input/output gate and the long-term state, \(\mathbf{b}_{\mathrm{f}},\mathbf{b}_{\mathrm{i}},\mathbf{b}_{\mathrm{o}},\mathbf{b}_{c}\) are biases of the forget/input/output gate and the long-term state. \(*\) represents the Hadamard product of matrices (the elementwise production), and \(\sigma(\cdot)\) and \(\tanh(\cdot)\) are the nonlinear activation functions in sigmoid and \(\tanh\) form, respectively. Note that in Equation (8), the state of the former step \(\mathbf{C}^{(\mathrm{t}-1)}\) propagates to the current step state \(\mathbf{C}^{(\mathrm{t})}\) in a linear style \(f^{(\mathrm{t})}*\mathbf{C}^{(\mathrm{t}-1)}\), and the linearity can be propagated in the whole length of the sequence. This part of the update keeps the long-term memory, \(C^{(\mathrm{t})}\) is therefore named the long-term memory state. Comparably, \(\mathbf{h}^{(\mathrm{t})}\) is the short-term memory state.