# Prediction of Horizontal in Situ Stress in Shale Reservoirs Based on Machine Learning Models
## 2 Experimental Datasets and Methods
### Experimental Methodology
#### 2.2.2 XGBoost  
XGBoost is an efficient implementation of Gradient Boosting Decision Trees (GBDT) and is widely used for regression, classification, and ranking tasks [36; 37]. Its high efficiency, flexibility, and excellent predictive performance make it a popular tool in machine learning, especially for complex problem-solving. XGBoost aims to reduce model error by combining multiple decision trees. In each iteration, it adjusts the tree structure based on gradient information to minimize the loss function. This is achieved by optimizing an objective function, which is typically a weighted sum of a loss function and a regularization term, as shown in equation:  
\[L(\theta)=\sum_{i=1}^{n}l(y_{i},\hat{y}_{i})+\sum_{k=1}^{K}\Omega(f_{k}) \tag{1}\]  
The loss function (\(\ell(y_{i},\hat{y}_{i})\) measures the difference between the model's predicted values (\(\hat{y}_{i}\)) and the true values (\(y_{i}\)). Common loss functions include Mean Squared Error (MSE) and log loss. The regularization term (\(\Omega(f_{k})\)) controls model complexity to prevent overfitting and is related to tree depth and the number of leaf nodes. K represents the number of trees. In regression tasks, XGBoost commonly uses Mean Squared Error (MSE) as the loss function:  
\[l(y_{i},\hat{y}_{i})=(y_{i}-\hat{y}_{i})^{2} \tag{2}\]  
In XGBoost, each iteration trains a new tree to minimize the current residual. To boost performance, XGBoost uses gradient boosting. Each new tree aims to reduce the previous model's error. In each iteration, XGBoost calculates the loss function's gradient based on current predictions (residuals) and adjusts model parameters using this gradient info. During optimization, XGBoost computes the first-order derivative (gradient) and the  
Figure 1: Logging Curve Chart of Rock Mechanics Parameters. second-order derivative of the objective function to adjust the tree structure. The gradient of the objective function for each sample is calculated as follows:  
\[g_{i}=\frac{\partial l(y_{i},\hat{y}_{i})}{\partial\hat{y}_{i}} \tag{3}\]  
The second-order derivative is calculated as follows:  
\[h_{i}=\frac{\partial^{2}l(y_{i},\hat{y}_{i})}{\partial\hat{y}_{i}^{2}} \tag{4}\]  
These gradients and second derivatives reflect the model's error at each point, guiding subsequent learning. In each iteration, XGBoost updates the model using the following formula:  
\[\hat{y}_{i}^{(t)}=\hat{y}_{i}^{(t-1)}+\eta f_{t}(x_{i}) \tag{5}\]  
In the formula, \(f_{t}(x_{i})\) represents the tree learned in the \(t\)-th iteration, and \(\eta\) denotes the learning rate, which controls each tree's contribution to the final model. In XGBoost, each tree is built through a series of split nodes. The selection of split nodes is based on maximizing the reduction of the loss function after each split. Specifically, XGBoost uses a greedy algorithm to choose the optimal feature and split point at each node to divide the data. The gain from each split is calculated using the following formula:  
\[Gain(t)=\frac{1}{2}\left(\frac{\left(\sum_{L}g_{i}\right)^{2}}{\sum_{L}h_{i} +\lambda}+\frac{\left(\sum_{R}g_{i}\right)^{2}}{\sum_{R}h_{i}+\lambda}-\frac{ \left(\sum_{L}g_{i}\right)^{2}}{\sum_{L}h_{i}+\lambda}\right)-\gamma \tag{6}\]  
In the formula, \(\sum_{L}g_{i}\) and \(\sum_{R}g_{i}\) represent the sum of gradients and second-order derivatives of the left and right child nodes, respectively. \(\sum_{t}g_{i}\) represents the sum of gradients and second-order derivatives at the current node. \(\lambda\) and \(\gamma\) are regularization parameters that control the complexity of the tree. Using this gain formula, XGBoost selects the optimal split nodes to maximize the gain and constructs the tree structure. Regularization is a key component of XGBoost, helping prevent overfitting, especially with high-dimensional data.