# NSGA-III-XGBoost-Based Stochastic Reliability Analysis of Deep Soft Rock Tunnel
## 2 EXtreme Gradient Boosting (XGBoost)  
XGBoost is a scalable end-to-end tree boosting system, and data scientists have widely used it in many machine learning challenges and achieved state-of-the-art results [26; 27]. XGBoost is an improved version of the gradient-boosting decision tree (GBDT). The basic component of XGBoost is the decision tree. These decision trees are referred to as \"weak learners\", which together make up XGBoost. There is a sequential order between the decision trees that make up XGBoost. The generation of the latter decision tree will consider the prediction results of the previous decision tree, taking into account the deviation of the previous decision tree (as reflected in the objective function). Each decision tree is generated using the entire dataset. So the generation of each decision tree can be seen as a complete process of decision tree generation. XGBoost is excellent in parallel computing efficiency, missing value processing, control overfitting, and prediction generalization ability. The specific algorithm process of XGBoost and its calculation steps are detailed in reference [28]. The steps and key technologies of XGBoost algorithm are as follows [26]:  
1. Construct the objective function: additive model, forward step-by-step algorithm, objective function derivation and optimal solution. Define \(q\) as the mapping from the input vector \(\mathbf{x}\) to the leaf node number of the decision tree. Let the sample set on each leaf \(p\) be \(I_{p}=\{i|q(\mathbf{x}_{i})=p\}\). Set the objective function as \(Obj(\Theta)\), and \(\Theta\) represents the region division of the tree and the constants on each region. Given the training sample dataset \(M=\{(\mathbf{x}_{1},\mathbf{y}_{1}),(\mathbf{x}_{2},\mathbf{y}_{2}),\cdots,(\mathbf{x}_{N},\mathbf{y}_ {N})\},\mathbf{x}_{i}\in\mathbf{\chi}\in\mathbf{R}^{N}\), \(\mathbf{\chi}\) is the input space of the training sample, \(\mathbf{y}_{i}\in y\in R\), \(y\) is the output space, and \(N\) is the dimension of input space. The objective function \(Obj(\Theta)\) can be denoted by Equation (2). \[Obj(\Theta)=L(\Theta)+\Omega(\Theta) \tag{2}\]  
where \(L(\Theta)\) represents the training loss function, indicating how well the model fits the training data; \(\Omega(\Theta)\) is the regularization term, which measures the complexity of the model. Simplify the objective function \(Obj(\Theta)\) as  
\[Obj(\Theta)\cong\sum_{p=1}^{M}\left[G_{p}w_{p}+\frac{1}{2}(H_{t}+\lambda)w_{p }^{2}\right]+\gamma M \tag{3}\]  
where \(M\) is the decision tree; \(\gamma\) and \(\lambda\) are two artificially set coefficients; \(\omega_{p}\) is the weight of leaf \(p\). \[\left\{ \begin{array}{l}g_{pi}=\frac{\partial L(y_{i}y_{j})}{\partial y_{i}^{ \prime}}\bigg{|}_{y_{i}^{\prime}=y_{i,p-1}^{\prime}},h_{pi}=\frac{\partial^{2} L(y_{i}y_{j}^{\prime}y_{j}^{\prime})}{\partial y_{i}^{\prime 2}}\bigg{|}_{y_{i}^{ \prime}=y_{i,p-1}^{\prime}}\\ G_{p}=\sum_{i\in I_{p}}g_{pi},H_{p}=\sum_{i\in I_{p}}h_{pi}\end{array} \right. \tag{4}\]where \(\mathbf{y^{\prime}}=f(\mathbf{x})\), \(f(\mathbf{x})\) is the prediction value obtained using the fitting model. 2. The methods for constructing trees (splitting nodes) include precise greedy algorithms, approximate search algorithms, automatic processing of missing values, and setting learning rates. In XGBoost, using the greedy algorithm to generate a tree will greatly improve computational efficiency. The greedy algorithm, in simple terms, is a method that ensures that each step is the optimal solution, thereby achieving the global optimal solution. In the generation of decision trees, the greedy algorithm can ensure that the new tree generated by each node split is the one with the minimum objective function value. The pseudocode for the exact greedy algorithm for split finding (see Algorithm 1) is as follows [26].