# Landslide Displacement Prediction during the Sliding Process Using XGBoost, SVR and RNNs
## 2 eXtreme Gradient Boosting (XGBoost)
### Methodology  
Given a training sample data set \((x_{1},y_{1}),(x_{2},y_{2}),\cdots,(x_{M},y_{M})\), \(x_{i}\in\mathbb{X}\in\mathbb{R}^{m}\), \(\mathbb{X}\) is sample input space, \(y_{i}\in\mathbb{Y}\in\mathbb{R}\), \(\mathbb{Y}\) is output space, \(m\) is input-space dimension. If \(\mathbb{X}\) is divided into nonintersecting regions \(\mathcal{R}_{1},\mathcal{R}_{2},\cdots,\mathcal{R}_{J}\), and \(c_{j}\) is the constant output determined in each region. The tree may then be expressed by Formula (1):  
\[\mathcal{T}(x;\mathcal{Q}_{k})=\sum_{j=1}^{J}c_{j}I\big{(}x\in\mathcal{R}_{j} \big{)} \tag{1}\]  
where \(\mathcal{Q}=\big{\{}(\mathcal{R}_{1},c_{1}),(\mathcal{R}_{2},c_{2}),\cdots,( \mathcal{R}_{J},c_{J})\big{\}}\) represents tree region division and the constants in each region; \(J\) is the number of leaf nodes. The boosting-tree model of XGBoost may be denoted by Formula (2) as an additive model of decision trees:  
\[f_{D}(x)=\sum\limits_{d=1}^{D}\mathcal{T}(x;\mathcal{Q}_{d}) \tag{2}\]  
where \(\mathcal{T}(x;\mathcal{Q}_{k})\) is decision tree; \(\mathcal{Q}_{d}\) is decision-tree parameters; \(D\) is the number of trees. The forward step-by-step boosting tree algorithm in XGBoost is used as follows. (1) The initial boosting tree is determined as \(f_{0}(x)=0\). (2) The model at the \(d\)-th step is expressed by Formula (3):  
\[f_{d}(x)=f_{d-1}(x)+\mathcal{T}(x;\mathcal{Q}_{d})\,,\;k=1,2,\cdots,K \tag{3}\]  
where \(f_{d-1}(x)\) is the current model; \(\mathcal{Q}_{d}\) is decision-tree parameters determined by Formula (4):  
\[\hat{\mathcal{Q}}_{d}=\underset{\mathcal{Q}_{d}}{\text{argmin}}\sum\limits_{i =1}^{D}\mathcal{L}(y_{i},f_{d-1}(x_{i})+\mathcal{T}(x_{i};\mathcal{Q}_{d})) \tag{4}\]  
where \(\hat{\mathcal{Q}}_{d}\) is the \(d\)-th tree parameters; \(\mathcal{L}(y_{i},f_{d-1}(x_{i})+\mathcal{T}(x_{i};\mathcal{Q}_{d}))\) is the loss value between observed value \(y\) and predicted value \(f_{d}(x)\) for the \(i\)-th sample point. The loss function can be denoted as \(\mathcal{L}(y,f(x))=(y-f(x))^{2}\) by the least-square-error criterion. \[l\big{(}y,y^{\prime}\big{)}=\mathcal{L}(y,f_{d-1}(x)+\mathcal{T}(x;\mathcal{Q }_{d}))=[y-f_{d-1}(x)-\mathcal{T}(x;\mathcal{Q}_{d})]^{2}=[r-\mathcal{T}(x; \mathcal{Q}_{d})]^{2} \tag{5}\]  
where \(r=y-f_{d-1}(x)\) is fitting residual of the current model; \(f(x)\) is the prediction value obtained from the fitted model, \(y^{\prime}=f(x)\); \(f_{d-1}(x)\) is the fitted model at the \(d\)-1-th step. (3) The fitting residual is calculated by Formula (6). \[r_{d_{i}}=y_{i}-f_{d-1}(x_{i}),\;i=1,2,\cdots,D \tag{6}\]  
A regression tree is learned according to the fitting residual \(r_{d_{i}}\), and \(\mathcal{T}(x;\mathcal{Q}_{d})\) is obtained. (4) \(f_{d}(x)=f_{d-1}(x)+\mathcal{T}(x;\mathcal{Q}_{d})\) is updated. (5) The boosting tree of regression problem \(f_{D}(x)=\sum\limits_{d=1}^{D}\mathcal{T}(x;\mathcal{Q}_{d})\) is solved. The loss function of XGBoost \(\mathcal{L}(\phi)\) is denoted by Formula (7). \[\mathcal{L}(\phi)=\sum\limits_{i=1}^{n}l(y_{i},y^{\prime}_{i})+\sum\limits_{d }\mathcal{W}(f_{d})= l\big{(}y,y^{\prime}\big{)}+\sum\limits_{d}\mathcal{W}(f_{d})= [r-\mathcal{T}(x;\mathcal{Q}_{d})]^{2}+\sum\limits_{d}\mathcal{W}(f_{d}) \tag{7}\]  
where \(n\) is the number of training samples; \(\sum\limits_{i=1}^{n}l(y_{i},y^{\prime}_{i})\) is loss value caused using the gradient-boosting algorithm, calculated by Formula (5), assuming it is a differentiable convex function; \(\mathcal{W}(f_{d})\) is regular terms; \(f_{d}\) is weak learner function.