## 3 Machining Quality Prediction Method Based on XGBoost Hyperparameter Optimization
### XGBoost Algorithm  
Ensemble learning has become a prominent research direction in machine learning in recent years. Its core concept is to combine multiple weak learners to leverage the strengths of each. The XGBoost algorithm is a type of ensemble learning. It consists of multiple Classification and Regression Tree (CART) decision trees, where each tree predicts the difference between the current predicted value and the actual value. The predictionsfrom each decision tree are accumulated to obtain the final predicted value. The XGBoost modeling process is illustrated in Figure 1.  
\[D=\{(x_{1},y_{1}),(x_{2},y_{2}),L,(x_{n},y_{n})\} \tag{2}\]  
Suppose the training dataset is denoted as \(D\), defined by (2), where \(x_{i}\in\mathbb{R}^{m}\) and \(y_{i}\in\mathbb{R}\). Here, \(x_{i}\) is an M-dimensional vector representing the input features, and \(y_{i}\) is the sample's label. If the XGBoost model contains \(k\) regression trees, the model can be expressed as shown in the following Formula (3):  
\[\hat{y}_{i}^{(k)}=\hat{y}_{i}^{(k-1)}(x_{i})+f_{k}(x_{i}) \tag{3}\]  
where \(f_{k}(x)\) represents the \(k\)-th regression tree, which can be abbreviated as \(f_{k}\) in subsequent formulas. The regression tree filters the features of the samples, making each sample fall into the corresponding regression tree leaf nodes based on the feature filtering results. Each leaf node of the regression tree has a weight value \(\omega\), representing the leaf node's predicted value. Like other machine learning algorithms, XGBoost also has an objective function. By minimizing the objective function, the optimal parameters of the machine learning model can be obtained. The objective function of XGBoost is defined by the following Formula (4):  
\[F_{obj}^{(k)}=\sum\limits_{i=1}^{n}L(y_{i},\hat{y}_{i}^{(k)})+\sum\limits_{i= 1}^{k}\Omega(f_{i}) \tag{4}\]  
where \(\sum\limits_{i=1}^{n}L(y_{i},\hat{y}_{i}^{(k)})\) is the sum of the loss function (the cost function), used to evaluate the degree of conformity between the predicted value and the actual value, and \(\sum\limits_{i=1}^{k}\Omega(f_{i})\) is the sum of regularization terms, representing the model's complexity. The larger this value is, the more complex the model becomes, and the more prone it is to overfitting. The definition of regularization is shown in the following Equation (5), where the first term controls the complexity of the regression tree, and the second term controls the weight value \(\omega\) of the leaf nodes of the regression tree:  
\[\Omega(f)=\gamma T+\tfrac{1}{2}\lambda\parallel\omega\parallel^{2} \tag{5}\]  
where \(T\) represents the number of leaf nodes, and \(\gamma\) and \(\lambda\) are coefficients used to balance the proportion of the first and second terms. Figure 1: XGBoost modeling process. In solving the objective function with the minimum value as the criterion, the objective function shown in Formula (4) is challenging to solve in Euclidean space. Therefore, XGBoost uses the Taylor series expansion to approximate the objective function. By substituting Formula (3) into Formula (4), the following can be obtained:  
\[ \begin{array}{l}F_{obj}^{(k)}=\sum\limits_{i=1}^{n}\Bigl{[}L(y_{i},\hat{y}_{i} ^{(k-1)}+f_{k}(x_{i}))\Bigr{]}+\sum\limits_{i=1}^{k-1}\Omega(f_{i})+\Omega(f_{ k})\\ =\sum\limits_{i=1}^{n}\Bigl{[}L(y_{i},\hat{y}_{i}^{(k-1)}+f_{k}(x_{i}))\Bigr{]}+ \Omega(f_{k})+C_{1}\end{array}  \tag{6}\]  
where \(f_{k}\) is the predicted value of the output of the \(k\)-th round, \(\hat{y}_{i}^{(k-1)}\) is the prediction of the model for data sample \(i\) in the \(k\)-th round, \(L(y_{i},\hat{y}_{i}^{(k-1)}+f_{k}(x_{i}))\) is the error between the predicted value and the actual value of data sample \(i\) in the \(k\)-th round, and \(C_{1}=\sum_{i=1}^{k-1}\Omega(f_{i})\) are constants. To meet the accuracy requirements, the second-order Taylor series expansion is used to approximate the original objective function as follows:  
\[