# Accident Probability Prediction and Analysis of Bus Drivers Based on Occupational Characteristics
## 6 Results
### CatBoost Prediction Results  
Based on the optimal parameters by the Bayesian hyperparameters, the model was retrained on the training set. The final effect was evaluated on the test set, which can be seen in Table 6. The optimized CatBoost model more accurately distinguished the accident probability of bus drivers. It is obvious that all the model metrics improved, and especially, the false-positive and false-negative cases in the confusion matrix are significantly reduced. [PERSON]'s kappa (k) increased most outstandingly to 0.680 (0.61-0.80, substantially). Compared to the default parameter CatBoost, it improved by 0.025 with an increase of 3.82%, by 6.75% compared to LightGBM, by 18.06% compared to XGBoost, and by 24.54% compared to GBDT. The values of AUC and AUC-PR also obviously increased with 0.834 and 0.788, respectively. Particularly, the values for the AUC-PR improved, by 1.29% compared to the default parameter, by 1.68% compared to LightGBM, by 6.92% compared to XGBoost, and by 9.44% compared to GBDT. These all indicate that the optimized model minimized false alarms while accurately identifying the occurrence of accidents, making the model more accurate and robust. Figure 6: Bayesian optimization process. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Hyperparameter Optimization** & **Interpretation** & **Default Value** & **Range** & **Result** \\ \hline Iterations & Maximum number of trees that could be built & 500 & (50, 1000) & 739 \\ Depth & Depth of the tree & 6 & (1, 12) & 10 \\  & Rate at which the model weights were updated & & & \\ learning\_rate & after working through each batch of training & 0.009 & (0.01,1.0) & 0.271 \\  & examples & & & \\ l2\_leaf\_reg & Coefficient for the L2 regularization term of the & 3 & (1, 10) & 6 \\  & cost function & 1.0 & (0.1, 1.0) & 0.778 \\  & Random subspace method & 1.0 & (0.2, 1.0) & 0.92 \\ Subsample & Sample sampling ratio & 0.6 & (0.2, 1.0) & 0.92 \\  & Amount of randomness to use for scoring splits & & & \\ random\_strength & when the tree structure was selected, used to avoid & 1.0 & (0.0, 5.0) & 3.798 \\  & overfitting the model & & & \\ bagging\_temperature & Bayesian bootstrap used to assign random weights & 1 & (0.1, 10.0) & 2.26 \\  & to objects & & & \\ random\_seed & Random seed used for training & 1.0 & (0, 100) & 12 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Parameters and optimization ranges and results. Figures 7 and 8 demonstrate the statuses of the ROC and ROC-PR curves of the CatBoost model before and after Bayesian tuning of the different interpolated datasets. The conclusions that can be drawn are the following:  
1. The estimated datasets obtained using the mean imputation method or the MICEForest multiple imputation method have higher ROC and ROC-PR curves after Bayesian hyperparameter tuning. This points to the Bayesian optimization-based tuning process effectively having improved the model's robustness and predictive ability. This finding verifies the scientific validity and reasonableness of the Bayesian hyperparameter optimization method and provides an intuitive validation of the model's performance improvement. 2. The ROC and ROC-PR curves of the MICEForest multiple imputation method are remarkably higher than those of the mean imputation method, regardless of whether Bayesian hyperparameters adjusted the MICEForest multiple imputation method. This observation implies that MICEForest multiple imputation minimizes the loss of data information to a greater extent, resulting in more reliable imputed data. Meanwhile, this result implies that for model performance improvement, data integrity and quality are important, and the MICEForest multiple interpolation method has a better effect and reliability in data interpolation. In summary, the Bayesian hyperparameter optimization process enhanced the model's performance.