## 5 Discussion and Future Work
### Optimal Risk Prediction Model  
#### 5.2.1 Performance of Risk Prediction Models  
In building predictive models based on full features and RFE-Top20 features, overall, the accuracy of all models in RFE-Top20 using the K-fold cross-validation technique (83.5% to 93.2%) was higher than that of RFE-Top20 with the Stratified K-fold cross-validation technique (87.4% to 90.7%). The precision (ranging from 54.4% to 84.2%) of all models of RFE-Top20 using the stratified K-fold cross-validation technology is almost similar to that of RFE-Top20 using the K-fold cross-validation technology (ranging from 68.5% to 84.2%). However, accuracy and precision do not guarantee that the performance results obtained are acceptable, and it may be biased towards the dominant class due to dataset imbalance. Due to the uneven distribution of classes, stratified K-fold cross-validation is used to overcome this problem [22], and five different performance evaluation indexes (accuracy, precision, recall rate, F1-score, AUC score) are used to compare the performance evaluation of classification models. The name of each model in Table 8 is represented by the cross-validation type, ML, and feature set combination. We count the frequency of the top five models for each performance metric. Table 9 lists the top three models in the six performance metrics. According to Table 9, using the stadium fire risk dataset for evaluation, Gradient Boosting + REF-Top20 + K-fold and Gradient Boosting + Full features + K-fold are both identified as one of the top five models for all six performance metrics. Meanwhile, the frequency ratio of the two models is 6:6. In terms of frequencies shown in Table 9, two models developed using Gradient Boosting have become the top models for this dataset, making Gradient Boosting the best performing ML algorithm in this study. The F1-score can be used as a metric to evaluate a classification model, which reflects the overall performance of the classification model. Through K-fold cross-validation, the F1-score of models developed using REF-Top20 features ranged from 61.3% to 81.9%. The F1-score of models developed with full features ranges from 48.0% to 81.9%. Based on F1-score, through the K-fold cross-validation technique, the model developed using Gradient Boosting of REF-Top20 features achieved the highest F1-score (81.9%), which determined it to be the best performing model. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Performance Metrics**} & \multicolumn{2}{c}{**Stadium Fire Risk Dataset**} \\ \cline{2-3}  & **Model** & **Value (\%)** \\ \hline \multirow{5}{*}{(a) Accuracy} & Gradient Boosting + REF-Top20 + K-fold & 93.2 \\  & Gradient Boosting + Full features + K-fold & 93.2 \\  & Bagging + Full features + K-fold & 92.4 \\  & RF + Full features + K-fold & 92.1 \\  & Bagging + REF-Top20 + K-fold & 92.1 \\ \hline \multirow{5}{*}{(b) Precision} & Gradient Boosting + REF-Top20 + K-fold & 84.2 \\  & Gradient Boosting + Full features + K-fold & 84.2 \\  & Bagging + Full features + K-fold & 82.5 \\  & Adaboost + REF-Top20 + K-fold & 81.9 \\  & Adaboost + Full features + K-fold & 81.9 \\ \hline \multirow{5}{*}{(c) Recall} & Gradient Boosting + REF-Top20 + K-fold & 84.3 \\  & Gradient Boosting + Full features + K-fold & 84.3 \\ \cline{1-1}  & Bagging + Full features + K-fold & 82.9 \\ \cline{1-1}  & Bagging + REF-Top20 + K-fold & 81.4 \\ \cline{1-1}  & RF + REF-Top20 + Stratified K-fold & 80.9 \\ \hline \multirow{5}{*}{(d) F1-score} & Gradient Boosting + REF-Top20 + K-fold & 81.9 \\  & Gradient Boosting + Full features + K-fold & 81.9 \\ \cline{1-1}  & Bagging + Full features + K-fold & 80.1 \\ \cline{1-1}  & RF + REF-Top20 + Stratified K-fold & 79.6 \\ \cline{1-1}  & Bagging + Full features + Stratified K-fold & 79.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Performance Analysis of Six Risk Prediction Classification Models.