## 5 Discussion and Future Work
### Optimal Risk Prediction Model
#### 5.2.1 Performance of Risk Prediction Models
We present the confusion matrix for the predictions of the best performing model on the test set in Figure 11. For a confusion matrix, all values from the upper left corner to the lower right corner of the diagonal are correctly classified data samples, where the sum of each row (from left to right) is the number of that class in the total sample. For example, in the third row, we have 41 samples belonging to \"Classes III\", of which our model correctly predicted 40 (97.5%). Similarly, \"Classes IV\" and \"Classes V\" have 96.5% and 90.9% accuracy of correct predictions, respectively. We observe \"Classes I\", the model performance is not ideal. With 14 samples correctly classified and nearly one-third of the data samples misclassified into different classes, it showed an accuracy of 60.8%. Obviously, the model is overfitting on \"Classes I\", which means that the sample data for \"Classes I\" in training was not enough. Overall, the classification effect of the gradient boosting model is good, and it can solve the problem of misdiagnosis to a certain extent. The AUC score can also be used as a metric to evaluate the performance of a classification model, as this metric is useful and informative for evaluating a model's ability to recognize different classes. Figure 12 shows the ROC curves of six machine learning algorithm classification models, most of which have AUC scores over 0.90, SVM and GBDT have the highest AUC score (0.94), and the corresponding AUC score of MLP classification model (0.88) below 0.90. Overall, the six prediction models are relatively stable and effective. However, the ROC curve more reflects the model's ability to correctly predict and rank positive and negative samples, but does not consider whether the distribution of positive and negative samples in the test data is balanced. Even though the distribution of the test data samples may change over time, the AUC value of the model will not be greatly changed (since the ROC curve is not sensitive), and tends to stabilize at a value. According to Figure 1, there are many instances of Class II and Class III samples in the stadium fire risk dataset, while there are few instances of Class V samples. Even if there is a category imbalance in the test samples, the model prediction results are good on the surface. Since the precision-recall curve is quite sensitive to the sample ratio, it can reflect the actual performance of the classifier as the sample ratio changes. Therefore, we also incorporate the precision-recall curve to further evaluate the prediction effect of the classification algorithm on the scenarios we apply. Figure 13 shows the precision-recall curves of the six machine learning algorithm classification models, AdaBoost obtains the highest AUC score (0.78), followed by SVM (0.77), and the model with the lowest AUC score is Gradient boosted decision tree (GBDT) (0.55). It can be seen from the results that the actual prediction effect of the GBDT model under the ROC curve is not ideal in the precision-recall curve, and the Adaboost and SVM models are more practical in this unbalanced dataset. With more balanced data, using AdaBoost or SVM may greatly improve model performance. Figure 12: ROC curves for six machine learning algorithms using REF-Top20 by k-fold cross-validation.