## 4 Recurrent Neural Network Modeling
### Recurrent Neural Network  
The LSTM network consists of a stack of LSTM units, each equipped with a cell and three gates--a forget gate, an input gate, and an output gate that regulate information flow across the cells. An LSTM unit typically processes information in accordance with the following equations [38]. Thus, the model output at time-step is given by \(h_{\text{rf}t=1,2,\ldots,k}\) as in Equation (1). Where \(\sigma\) is the logistic function with a range of zero and one that determines the ratio of information flow through each gate. The forget gate (\(f^{(i)}\)) controls what information is retained from the previous cell state (C\({}^{(t-1)}\)), whereas the input gate (\(i^{(t)}\)) combines with the input-modulating gate (C\({}_{\text{i}}^{(t)}\))--which uses a hyperbolic tangent activation (tanh) function to update the cell state to (C\({}^{(t)}\)). In addition, the output gate (\(o^{(t)}\)) sieves the information within the new cell state (C\({}^{(t)}\)), using both the logistic and tanh functions to regulate its output, and is transferred to the hidden state (\(h^{(t)}\)). W\({}_{*}\) and b\({}_{*}\) represent the weight and bias parameter matrix in each gate, respectively. Combining the gating mechanisms enables the LSTM to extract relevant information from an input sequence.  
\[\begin{split} f^{(t)}&=\ \sigma\left(\text{W}_{\text{f}} \left[h^{(t-1)},\ \ x^{(t)}\ \right]+\text{b}_{\text{f}}\ \right)\\ i^{(t)}&=\ \sigma\left(\text{W}_{\text{i}}\left[h^{(t-1)}, \ \ x^{(t)}\ \right]+\text{b}_{\text{i}}\ \right)\\ C_{\text{i}}^{(t)}&=\ \tanh\!\left(\text{W}_{ \text{c}}\left[h^{(t-1)},\ \ x^{(t)}\ \right]+\text{b}_{\text{c}}\right)\\ C^{(t)}&=f^{(t)}*C^{(t-1)}+i^{(t)}*C_{\text{i}}^{(t)} \\ o^{(t)}&=\ \sigma\left(\text{W}_{\text{o}}\left[h^{(t-1)}, x^{(t)}\ \right]+\text{b}_{\text{o}}\right)\\ h^{(t)}&=o^{(t)}\ *\ \tanh\!\left(C^{(t)}\right) \end{split} \tag{1}\]  
Figure 9: Training and test data split for (**a**) master position (as part of the input variable) and (**b**) the slave position (target variable).  
In contrast, the stacked GRU units comprise a structure where each unit is equipped with two gates--a reset gate \(\left(r^{(t)}\right)\) and update gate \(\left(z^{(t)}\right)\). The update gate regulates what information flows into the memory using logistic function within the interval (0, 1), similar to LSTM. This controls the similarity between the previous state and the new state. The reset gate determines what percentage of the previous state is retained and how to compute the candidate hidden state ( \(\mathrm{h}^{~{}~{}\left(t\right)}\)). An elementwise operation of the update gate with the previous state (\(\mathrm{h}^{(t-1)}\)) and the candidate hidden state result in the final output of the model \(\mathrm{h}^{(t)}\) is calculated as shown in the following equations below [39].  
\[\begin{split} r^{(t)}&=~{}\sigma\left(\mathrm{W}_{ \mathrm{r}}\left[h^{(t-1)},~{}~{}x^{(t)}\right]+\mathrm{b}_{\mathrm{r}}\right) \\ z^{(t)}&=~{}\sigma\left(\mathrm{W}_{\mathrm{z}} \left[h^{(t-1)},~{}~{}x^{(t)}\right]+\mathrm{b}_{\mathrm{z}}\right)\\ \widetilde{h}^{(t)}&=\mathrm{tanh}\left(\mathrm{W}_ {h}\left[r^{(t)}\,\ast h^{(t-1)},~{}~{}x^{(t)}\right]+b_{h}\right)\\ h^{(t)}&=z^{(t)}\,\ast h^{(t-1)}+\left(1-z^{(t)} \right)\ast\widetilde{h}^{(t)}\end{split} \tag{2}\]