# Optimizing Faulting Prediction for Rigid Pavements Using a Hybrid SHAP-TPE-CatBoost Model
## 4 Model Construction  
In this section, the overall process of developing the model will be explained. As shown in Figure 3, 80% of the processed data were used as the training set to train the model, while the remaining 20% were used as the test set to evaluate the performance of the model. The Bayesian hyperparameter optimization algorithm was adopted to find the optimal hyperparameters.  
In this study, K-fold cross-validation (K-fold CV) and the TPE method were employed to optimize hyperparameters and enhance model performance while reducing computational expenses. K-fold CV helps evaluate the generalization ability of the model across different training sets [29], and the TPE algorithm automatically searches for the best combination of hyperparameters. This approach is a practical and effective technique for machine learning hyperparameter tuning, enabling significant improvements in prediction accuracy and efficiency.  
K-fold CV is a robust technique for addressing the biases caused by random sampling. The training set is divided into K subsets, and the process is repeated K times. During each iteration, one subset is designated as the validation set, while the remaining K-1 subsets are utilized for training and validation. The model's performance is assessed by  
Figure 3: The models building process.  
Figure 2: Correlation between variables.  
averaging the results of K validations. This method of cross-validation offers the advantage of mitigating biases and variances that may arise from a single random split. It also helps to prevent issues such as overfitting and underfitting of the model. Furthermore, as depicted in Figure 4, K-fold CV aids in determining the optimal range of hyperparameters, thereby preventing wastage of computational resources due to improper parameter range settings. By systematically evaluating the model's performance with different hyperparameter combinations, K-fold CV enables researchers to select the most suitable hyperparameters for achieving desirable results.  
Grid search is a widely used hyperparameter optimization method [30], but it suffers from slow speed and efficiency. Although random search is more efficient than grid search, it may overlook some crucial combinations, resulting in suboptimal hyperparameter settings. Bayesian optimization, on the other hand, is less likely to miss important parameter combinations in the search space and is less prone to getting stuck in local optima. In this study, the Hyperopt library was utilized, which utilizes TPE as the surrogate function to search for the optimal combination of hyperparameters and achieve improved model performance. Hyperopt involves four parameters: space, algo, max_evals, and fn. The space parameter defines the search space by specifying the range of hyperparameters. The algo parameter represents the surrogate function model, with TPE being used in this study. The fn parameter is the objective function, which aims to minimize the RMSE of  
Figure 4: Search for the optimal range of hyperparameters for the model through 5-fold cross-validation: (**a**) iterations, (**b**) learning rate, (**c**) depth, and (**d**) l2_leaf_reg.  
the validation set. Finally, the max_evals parameter determines the maximum number of iterations, which was set to 500 in this particular study. As shown in Table 4, the optimal hyperparameter combinations for the CatBoost model used in prediction are as follows: the optimal range for iterations is [40, 200], with an optimal value of 150; the optimal range for learning_rate is [0.01, 0.5], with an optimal value of 0.38; the optimal range for depth is [2, 10], with an optimal value of 3; and the optimal range for l2_leaf_reg is [0.01, 1], with an optimal value of 0.04.