# Pore Water Pressure Prediction Based on Machine Learning Methods--Application to an Earth Dam Case
## 2 Materials and Methods
### Artificial Neural Networks
#### 2.2.2 Dynamic Neural Network  
The Standard Recurrent Neural Networks  
Recurrent neural networks (RNNs) represent a category of ANNs characterized by their dynamic behavior over time, owing to the interconnections between nodes forming either a directed or undirected graph across a time series [44]. These networks stem from feedback neural networks and possess the capability to handle input sequences of varying lengths by utilizing their internal state (memory), making them a kind of dynamic neural network. The basic architecture of the RNNs and the transfer processing are illustrated in Figure 3.  
At each time step denoted as \(t\), the activation \(a^{<t>}\) and the output \(y^{<t>}\) are defined as follows:  
\[a^{<t>}=f_{1}\Big{(}W_{aa}a^{<t-1>}+W_{ax}x^{<t>}+b_{a}\Big{)} \tag{2}\]  
\[y^{<t>}=f_{2}\big{(}W_{yaa}a^{<t>}+b_{y}\big{)} \tag{3}\]  
Here, \(W_{ax}\), \(W_{aa}\), \(W_{ya}\), \(b_{a}\), and \(b_{y}\) are coefficients that are shared temporally, while \(f_{1}\) and \(f_{2}\) represent activation functions.  
RNNs are commonly utilized to process sequential data. However, training RNNs can be challenging due to well-documented issues such as the vanishing and exploding gradient problems arising from the repeated multiplication of the recurrent weight matrix [45]. Additionally, they can struggle to learn long-term patterns [46]. Despite these drawbacks, RNNs possess feedback within their network architecture, which gives them the ability to represent and learn state processes. Due to the drawbacks of the RNNs, in this paper, advanced RNNs will be introduced to improve the prediction performance.  
The gated recurrent unit (GRU) represents an advanced type of RNN. Figure 4 displays the general structure of a GRU, depicting inputs for both the reset and update gates within the GRU architecture. The reset gate and update gate are used as vectors, each with entries in the range of \((0,1)\) to enable convex combinations. For example, the reset gate controls the extent to which the previous state is preserved, while the update gate governs how much of the new state replicates the old state. The learning rate, a parameter ranging from zero to one, determines the speed at which the weights are updated [47].  
Figure 3: The standard recurrent neural networks architecture.  
The input vector at time \(t\) is denoted as \(x_{t}\). The output vectors of hidden units at time \(t\) are represented by \(h_{t}\). These inputs are determined by the current time step's input and the hidden state from the preceding time step. The outputs of these gates are produced through two fully connected layers with a sigmoid activation function, and \(y_{t}\) depicts the outputs of the GRU.  
The GRU [48] is a gate variant that is more streamlined and often offers comparable performance while being significantly faster to compute. Unlike MLP, in GRU, adjacent hidden neurons are connected. The sliding window allows for the sequential transfer of time-dependent input information through hidden units, enabling consideration of temporal correlations between events occurring at considerable temporal distances.