## 3 Materials and Methods
### The Framework of Semi-Supervised Deep Learning
#### 3.3.2 Pre-Training DNN Architecture  
Deep learning, through a non-linear complex model, is a feature learning method that transforms the original data into a higher level and more abstract expression. Deep neural network (DNN) is the basic algorithm of deep learning [59]. The network structure usually consists of an input layer, several hidden layers, and an output layer. DNN discovers complex structures in large datasets by changing the internal parameters, which are used to calculate each layer's representation from the previous layer. So far, various DL architectures (for example, auto-encoders, Restricted Boltzmann Machine (RBM), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN) have been proposed and perform well in many fields [60; 61]. Landslide susceptibility is essentially a nonlinear logistic regression problem. The classification probability in the model can be derived by sigmoid function, that is, the susceptibility of landslide at a certain point. In this study, the DNN model was applied to LSM. Impact factor (IF) became the input signal received in the first layer, and analyzed in the hidden layer. Finally, the prediction category was displayed in the output layer, and two possible labels could be exported: landslide and non-landslide. During the development of the DNN model, the main characteristics set were the number of layers and nodes, which define the depth of the architecture, and the activation and transfer functions. The hidden layers and processing elements were determined by the characteristics of the dataset and the number of training sets. Based on the above datasets, according to many tests of trial and error, the structure of the DNN (Figure 7), which consisted of a model of three hidden layers including 16 neurons and two output neurons, was established in supervised learning. The network was applied as a pre-training DNN in semi-supervised learning and was also regarded as a supervised learning model for comparison. After the introduction of pseudo labels, the number of neurons was mainly modified to 64. The activation function introduces nonlinear factors into the neural network, then fits various curves through the activation function. Meanwhile, it converts the input signal of the node into an output signal and stacks it as the input for the next layer. Rectified linear unit (ReLU), as an acclaimed activation function, has been used successfully in a wide range of applications [62; 63]. The problem of gradient vanishing can be solved (in a positive interval), and the convergence speed is fast. The analytic expression of the function is:  
\[f(x)=\left\{ \begin{array}{ll}xi\,x\\ 0\,i\,f\,x\end{array} \right.=\max(x,0) \tag{1}\]  
Figure 7: Illustration of the DNN for landslide susceptibility. Sigmoid function is a commonly used nonlinear activation function, and is given by the following equation:  
\[f(x)=\frac{1}{1+e^{-x}} \tag{2}\]  
The adaptive moment optimization (Adam) algorithm is employed in the framework as it requires less memory and is efficient in calculation, mainly by combining the advantages of adaptive gradient algorithm (AdaGrad) and root mean square propagation (RMSProp) [64]. As an alternative optimization algorithm, Adam can perfectly replace stochastic gradient descent, and the default parameters are competent to handle most of the problems. First, the gradient and the square of the gradient are calculated for the moving average \(m_{t}\) and \(v_{t}\):  
\[m_{t}=\beta_{1}m_{t-1}+(1-\beta_{1})g_{t} \tag{3}\]  
\[v_{t}=\beta_{2}v_{t-1}+(1-\beta_{2})g_{t}^{2} \tag{4}\]  
where \(\beta_{1}\) and \(\beta_{2}\) are the exponential decay rate, which default to 0.9 and 0.999, respectively. The exponential moving mean of the gradient \(m_{0}\) and the exponential moving mean of the gradient squared \(v_{0}\) are initialized to 0.