## 3 Machine Learning Approaches for Slope Deformation Prediction
### Employed Statistical and Machine Learning Approaches
#### 3.4.3 Deep Learning Approaches  
(1) Conventional DNN  
The DNN is a neural network that contains multiple hidden layers. DNNs are divided into three types of neural network layers: the input layer, hidden layers, and the output layer. Generally, the first layer is the input layer, the middle layers are hidden layers, and the last layer is the output layer. In this paper, we established a conventional DNN model as a comparison baseline, and the layers of the model were fully connected to each other.  
(2) CNN  
The CNN is type of feedforward neural network. Common CNNs consist of a convolution layer, a pooling layer, and a fully connected layer. CNNs play an important role in the field of computer vision; furthermore, CNNs can also be adapted to time-series problems. The difference between them is that a two-dimensional convolution kernel is applied to an image, while a one-dimensional convolution kernel is applied to a time series.  
(3) Conventional RNN, LSTM, GRU, BiLSTM and Attention-LSTM  
The RNN is a recurrent neural network that is mainly used to model sequence data. Common RNNs consist of an input layer, hidden layers, and an output layer. In contrast to the DNN and CNN, the RNN is quite effective for data with sequential characteristics and  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Parameter** & **FJ003** & **FJ004** & **FJ005** & **FJ006** & **FJ009** & **FJ010** & **FJ011** \\ \hline \(p\) & 1 & 3 & 1 & 1 & 0 & 1 & 1 \\ \(d\) & 0 & 1 & 0 & 1 & 1 & 2 & 2 \\ \(q\) & 0 & 0 & 2 & 1 & 0 & 0 & 0 \\ \hline \end{tabular}
\end{table}
Table 1: The optimal parameters of the ARIMA model for the displacement data of the Huanglianshu landslide.  
possesses a strong model fitting ability, which can mine information between time-series data.  
The LSTM and GRU are two variants that improve the limitation of RNNs for long-term dependence. The LSTM is a special network structure with three \"gates\" that can be used to remove or increase the information of the \"cell state\". The gate used to increase information is called the \"update gate\", the gate used for forgetting is called the \"forget gate\", and the \"output gate\" is used for the output. The GRU is a variant of the LSTM model that combines the input gate and forget gate into an update gate. The internal structures of the RNN, LSTM, and GRU are shown in Figure 7.  
BiLSTM [37] is a combination of the forward LSTM and backward LSTM. Modeling time series with an LSTM cannot encode back-to-back information, but a BiLSTM can better capture the relationships between sequence data in both directions.  
The Attention-LSTM model is an LSTM model with an attention mechanism. In recent years, the attention mechanism, which can focus on the key part of time-series data to reduce the impact of monkey data for prediction and is considered a fully connected layer and softmax function, has been widely used.  
(4) Transformer  
The transformer is a sequence-to-sequence (seq2 seq) structure [38] with encoder and decoder networks that use a self-attention mechanism. Compared with the traditional RNN and CNN, the transformer model proposes a new position encoding mechanism to capture the time-series information between input data; thus, the transformer model can fully capture the time-series information with richer dimensions. In the time-series prediction problem, the encoder takes the history of the time-series as the input, while the decoder predicts the future value via an autoregression. The employed transformer model in this paper included the encoder part of the transformer and finally connected a fully connected layer.