## 6 Model Construction
### Determination of Model Architecture  
In this study, six representative machine learning algorithms were systematically selected as research carriers: XGBoost, CatBoost, and LightGBM, under the gradient boosting framework; RF, as an integrated learning method; SVM, as a representative of traditional machine learning; and MLP, as a typical model of deep learning.  
In the process of model construction, in order to ensure the rigor and scientificity of the performance comparison of different algorithms, this study formulated a standardized modeling process (shown in Figure 7). It is worth noting that all the comparison experiments followed the principle of the \"control variable method\". Firstly, MinMaxScaler was uniformly used for feature normalization, and each index was linearly mapped to the interval of [0, 1], which eliminated the influence of the magnitude and retained the distribution characteristics of the original data. Secondly, the hyperparameter optimization link was equipped with the cutting-edge auto-adjustment framework of Optuna, which can explore the parameter space efficiently with the sequence modeling technique based on Bayesian optimization. More importantly, the modeling procedure was standardized to ensure that the performance of different algorithms would be rigorous and scientific in the process of model construction (Figure 7). The sequence modeling technique based on Bayesian optimization efficiently explored the parameter space; more importantly, the optimization process strictly adopted the five-fold cross-validation method to randomly and uniformly divide the dataset into five similarly sized subsets (known as \"folds\"), denoted as Fold 1 to Fold 5. Then, the model underwent five rounds of iterations, with Fold 1 selected as the validation set in each round and the remaining folds selected as the training set. The model performance was calculated on the validation set in each round, and the average of the five rounds' performance was taken as the final performance of the model. Through such multiple data rotation and result aggregation, the robustness and data efficiency of model evaluation were significantly improved, both fully exploiting the value of data and effectively avoiding the risk of overfitting.  
Figure 6: Variance explained by each principal component.  
In the design of the evaluation system, this study adopted the average cross-validation accuracy as the core evaluation index. Specifically, a complete five-fold cross-validation process was executed for each Optuna trial, and the arithmetic average of the five rounds of validation accuracy was finally taken as the ultimate judgment of model performance. This design significantly improved the stability and credibility of the evaluation results. For the Optuna optimizer, this study set a search budget of 100 iterations, and the hyperparameter search space for each model is shown in Table 5.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Hyperparameter** & **Search Space** & **Parameter Type** \\ \hline \multirow{5}{*}{XGBoost} & n\_estimators & [200, 1000] & int \\  & learning\_rate & [0.01, 0.3] & float \\  & max\_depth & [3, 10] & int \\  & subsample & [0.6, 1.0] & float \\  & colsample\_bytree & [0.6, 1.0] & float \\  & gamma & [0, 0.5] & float \\  & reg\_alpha & [0, 1] & float \\  & reg\_lambda & [0, 1] & float \\ \hline \hline \end{tabular}
\end{table}
Table 5: Hyperparametric search space.  
Figure 7: Flowchart of model prediction.