# Ensemble-Learning-Based Prediction of Steel Bridge Deck Defect Condition
## 4 Results and Discussion  
### Hyperparameter Optimization  
The grid search method was used to optimize the hyperparameters of the developed ensemble learning model by first determining a range of values for each hyperparameter of the model and then generating multiple hyperparameter combinations. Next, these hyperparameter combinations were applied to the model and evaluated by 10-fold cross-validation on the training set. Finally, the mean value of the ten evaluations of each hyperparameter combination was compared, determining the optimal combination. In this study, accuracy was chosen as the main evaluation metric for hyperparameter optimization to select the hyperparameter combination with the best performance. The optimal hyperparameter combination in the RandomForest model is bootstrap = True; n_estimators = 100; criterion = 'gini'; min_samples_leaf = 1; min_samples_split = 2. The optimal hyperparameter combination corresponds to 10-fold crossover. The validation results are shown in Table 4.  
The optimal hyperparameter combinations in the Extra Tree model are bootstrap = False; n_estimators = 100; criterion = 'gini'; min_samples_leaf = 1; and min_samples_split = 2. The optimal hyperparameter combinations correspond to 10-fold cross-validation results shown in Table 5. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**True Value**} & \multicolumn{3}{c}{**Predicted Value**} \\ \cline{2-3}  & **Not Defect** & **Defect** \\ \hline Not defect & TP & FP \\ Defect & FN & TN \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Confusion matrix. The optimal hyperparameter combination in the AdaBoost model is algorithm = SAMME.R; Learning rate = 0.2; n_estimators = 200. The results of the 10-fold cross-validation corresponding to the optimal hyperparameter combination are shown in Table 6.  
The optimal hyperparameter combinations in the GBDT model are learning_rate = 0.15; max_depth = 10; n_estimators = 210; min_samples_leaf = 3; min_samples_split = 7. The  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{5}{c}{**Evaluation Metrics**} \\ \cline{2-6}  & **Accuracy** & **AUC** & **Recall** & **Precision** & **F1-Score** \\ \hline
1 & 0.8273 & 0.8693 & 0.8314 & 0.9838 & 0.9012 \\
2 & 0.8  
optimal hyperparameter combinations correspond to the 10-fold cross-validation results shown in Table 7.  
The optimal hyperparameter combination in the XGBoost model is base_score = 0.5; boosters = 'gbtree'; colsample_bylevel = 1; colsample_bynode = 1; colsample_bytree = 0.5; gamma = 0; learning_rate = 0.1; max_depth = 11; min_child_weight = 1; n_estimators = 260; objective = 'binary: logistic'; reg_alpha = 0.05; reg_lambda = 0.3. The optimal hyperparameter combinations correspond to the 10-fold cross-validation results shown in Table 8.  
The optimal combination of hyperparameters in the LightGBM model is bagging_fraction = 0.9; bagging_freq = 4; boosting_type = 'gbdt'; colsample_bytree = 1.0; feature_fraction = 0.4; importance_type ='split'; learning_rate = 0.5; min_child_samples = 71; min_child_weight = 0.001; min_split_gain = 0.4; n_estimators = 280; num_leaves = 6; reg_alpha = 1; reg_lambda = 0.7. The optimal hyperparameter combinations correspond to the 10-fold cross-validation results shown in Table 9.  
The mean values of the 10-fold cross-validation results corresponding to the optimal hyperparameter combinations of the six ensemble learning models on the training set are shown in Figure 3. From the figure, it can be seen that the XGBoost model has the best performance on the training set, and the AdaBoost model has the worst results.