## 4 Materials and Methods  
### XGBoost Algorithm  
[PERSON] and [PERSON] [32] introduced XGBoost, a robust machine learning framework that leverages tree boosting. Its scalability has found diverse applications across numerous engineering domains, delivering superior performance. This can be attributed to its capacity for efficient tree pruning, regularization, and the ability to process tasks in parallel. Gradient boosting, serving as the foundational model for XGBoost, iteratively amalgamates the estimations of several \"weak\" learners to create a singular \"strong\" learner. XGBoost employs a unique method to optimize the loss function by leveraging residuals for calibrating the previous prediction at each iteration [33; 34]. This calibration process is an integral part of enhancing the accuracy of the model. Moreover, to curtail the risk of overfitting--a common pitfall in machine learning models--XGBoost ingeniously integrates regularization directly into its objective function, which can be described by Equation (1). This inclusion of regularization serves as a penalty term, mitigating model complexity, thereby enhancing the generalizability and robustness of the model. \[J(\Theta)=L(\Theta)+\Omega(\Theta) \tag{1}\]  
In Equation (1), \(\Theta\) signifies the parameters that the model learns from the provided dataset. '\(\Omega\)' is the regularization term, a strategy that helps to control the model's complexity to prevent it from overfitting to the training data. '\(L\)' symbolizes the training loss functions, including square loss and logistic loss, which serve to assess the fit of the model to the training data. Equation (2) serves as the model's predictive function; according to decision tree theory, the model's output \(\hat{y}_{i}\) is derived from either a voting system or the average from a set '\(F\)' consisting of '\(K\)' trees. \[\hat{y}_{i}=\sum_{i=1}^{k}f_{k}(x_{i}),\ f_{k}\in F \tag{2}\]  
The objective function at the \(t\)-th iteration can be detailed using a more particular mathematical formulation, as expressed by:  
\[J^{(t)}=\sum_{i=1}^{n}L(y_{i},\hat{y}_{i})+\sum_{k=1}^{t}\Omega(f_{k}) \tag{3}\]  
In this case, \(n\) signifies the quantity of predictions, and \(\hat{y}^{(t)}{}_{i}\) can be characterized as:  
\[\hat{y}^{(t)}{}_{i}=\sum_{k=1}^{t}f_{k}(x_{i})=\hat{y}_{i}\ ^{(t-1)}+f_{t}(x_{i}) \tag{4}\]  
[PERSON] and [PERSON] [32] demonstrated that the regularization term \(\Omega(f_{k})\) for the decision tree is expressed as follows:  
\[\Omega(f_{k})=\gamma T+0.5\lambda\sum_{j=1}^{T}\omega_{j}^{2} \tag{5}\]  
'\(\lambda\)' modulates the penalty factor, while '\(T\)' denotes the number of leaf nodes in the Decision Tree (DT). The complexity of each of these leaf nodes is symbolized by '\(\gamma\)', and '\(\omega\)' serves as a vector housing the scores of each leaf node. The Loss Function (LOF) in XGBoost then implements a second-order Taylor expansion, diverging from the typical first-order approach used in general gradient boosting, as discussed by [PERSON] and [PERSON] [32] and [PERSON] et al. [35]. If we assume the MSE to be the LOF, the objective function can be derived through the equation provided below:  
\[J^{(t)}\approx\sum_{i=1}^{n}\left[g_{i}\omega_{q(x_{i})}+\frac{1}{2}\Big{(}h_ {i}\omega_{q(x_{i})}^{2}\Big{)}\right]+\gamma T+\frac{1}{2}\lambda\sum_{j=1}^ {T}\omega_{j}^{2} \tag{6}\]  
In Equation (6), \(g_{i}\) and \(h_{i}\) stand for the first and second derivatives of the MSE loss function, respectively. Meanwhile, the function \(q\) is responsible for mapping each data point to its corresponding leaf. Evidently, the LOF represented by Equation (7) is the aggregate of loss values for each individual data sample. Given that each data sample aligns with a single leaf node, the sum of the loss values across all leaf nodes can likewise express the LOF.