## 4 Algorithm Analysis
### Feature Selection
The resulting OOB error after the replacement is then calculated as follows:  
\[F_{k}\ =\ \frac{1}{n}\sum_{j\ =\ 1}^{n}\left(y_{j}-\hat{y}_{j,k}\right)^{2} \tag{16}\]  
In the formula, \(F_{k}\) represents the OOB error after the replacement, and \(\hat{y}_{j,k}\) represents the OOB prediction value after the replacement. Compare the OOB errors before and after the feature disturbance to calculate the change in error \(D\). The greater the change, the higher the contribution to accurate prediction, and thus the greater the importance. The expression is:  
\[D\ =\ \frac{1}{n}\sum_{j\ =\ 1}^{n}\left(F_{j,k}-F_{j,\rho}\right) \tag{17}\]  
The model's generalization ability is evaluated through cross-validation, using the mean as a baseline to determine errors. By comparing the predicted values to this baseline, the coefficient of determination \(R^{2}\) is used to assess the model's accuracy. A value of \(R^{2}\) closer to 1 indicates a better fit, while a lower \(R^{2}\) suggests poorer prediction performance. \[R^{2}\ =\ 1-\frac{\sum_{j\ =\ 1}^{t}\left(z_{j}-\hat{z}_{j}\right)^{2}}{\sum_{j \ =\ 1}^{t}\left(z_{j}-z\right)^{2}} \tag{18}\]  
In the formula, \(t\) represents the number of samples used for training. \(z_{j}\) is the actual observed value, \(\overline{z}\) is the average of the observed values, and \(\hat{z}_{j}\) represents the predicted result from the training model. The initial number of decision trees for the RF model was set to 50, 100, 150, and 200. Multiple rounds of parameter tuning were conducted to ensure the model's accuracy and the consistency of the results across each training session. The feature importance rankings remained largely consistent across various decision trees [51]. When the number of decision trees was set to 150, there was a noticeable difference in the model's output. The R^2 score for the training set was 0.8815, and for the test set, it was 0.8582. These results indicate that the model demonstrates good fitting ability and strong generalization performance. To avoid the randomness of training results, the model was iteratively trained 100 times. The contribution values of input features were recorded when the coefficient of determination (R\({}^{2}\)) exceeded 0.8 for both the training and test sets. The average of these values was then used to represent the contribution of each input feature to the overall prediction. The feature importance ranking of the 21 input features is shown in Figure 4.  
Figure 3.0 Feature importance ranking. As shown in the figure above, there are a large number of features, with notable differences in their importance. Feature selection is performed to reduce the model's complexity. When the number of input features is too small, the XGBoost model is unable to capture the key information from the features or effectively explore the nonlinear relationships between the features and the output. This leads to poor model performance during training. The 21 features are arranged in descending order of importance, and the number of features is incrementally increased to train the XGBoost model, thereby validating the feasibility of the feature engineering method applied to RF [52]. As shown in Figure 5, in the initial stage, as the number of input features increases, the mean accuracy of the model gradually improves. When the number of input features reaches 14, the accuracy of each output feature essentially reaches its peak, with an R\({}^{2}\) mean of 0.912 for the 14 output features on the test set, indicating that these features play a crucial role in the model's prediction performance. After the number of input features exceeds 14, the improvement in model accuracy becomes relatively flat. This suggests that once the number of features reaches a certain threshold, the contribution of additional input features to the model diminishes and may even introduce redundant information.