## 5 Results and Verification
### Model Verification
#### 5.3.2 Exploration of Prediction Power Using Receiver Operating Characteristic  
(ROC) Curve  
The ROC curve is a useful graphical method for exploring the prediction power in a classification model. In this study, the differentiated prediction power was evaluated by comparing the probability of predicting an actual fatal accident as a serious or a slight accident with the probability of predicting a serious accident as a fatal accident. The ROC curve can be drawn by calculating the sensitivity and specificity of the classification model. If the result of an actual positive case is predicted as positive by a \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Category**} & \multicolumn{4}{c}{**Estimated Probability**} & \multirow{2}{*}{**Predicted**} & \multirow{2}{*}{**Actual**} \\ \cline{2-2} \cline{5-6}  & **Slight Accident** & & & & **Serious Accident** & **Fatal Accident** \\ \hline
1 & 48.3\% & 40.0\% & 11.7\% & slight accident & slight accident \\
2 & 39.4\% & 56.7\% & 3.9\% & serious accident & serious accident \\
3 & 42.2\% & 30.3\% & 27.5\% & serious accident & slight accident \\
4 & 43.0\% & 55.8\% & 1.2\% & serious accident & serious accident \\
5 & 47.5\% & 36.4\% & 16.1\% & slight accident & slight accident \\
6 & 42.2\% & 30.3\% & 27.5\% & slight accident & slight accident \\
7 & 42.8\% & 47.4\% & 9.9\% & serious accident & serious accident \\ � & � & � & � & � & � \\
392 & 47.5\% & 36.4\% & 16.1\% & slight accident & slight accident \\
393 & 46.8\% & 46.4\% & 6.8\% & slight accident & slight accident \\
394 & 35.1\% & 50.7\% & 14.2\% & serious accident & serious accident \\
395 & 39.4\% & 56.7\% & 3.9\% & serious accident & serious accident \\
396 & 46.8\% & 46.4\% & 6.8\% & slight accident & slight accident \\
397 & 48.4\% & 48.4\% & 3.2\% & serious accident & serious accident \\
398 & 46.8\% & 46.4\% & 6.8\% & slight accident & slight accident \\
399 & 8.3\% & 84.5\% & 7.2\% & serious accident & serious accident \\
400 & 48.4\% & 45.5\% & 60.1\% & fatal accident & fatal accident \\
401 & 3.2\% & 27.8\% & 69.0\% & fatal accident & fatal accident \\
402 & 51.7\% & 47.1\% & 1.2\% & serious accident & slight accident \\
403 & 39.2\% & 46.5\% & 14.3\% & serious accident & serious accident \\ \hline \hline \end{tabular}
\end{table} 
Table 2: RORC severity prediction model construction. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Measurement Prediction**} & \multirow{2}{*}{**Slight**} & \multirow{2}{*}{**Serious**} & \multirow{2}{*}{**Fatal**} & \multirow{2}{*}{**Total**} \\ \hline slight & 13 & 0 & 1 & 14 \\ serious & 1 & 16 & 1 & 18 \\ fatal & 1 & 0 & 7 & 8 \\ Total & 15 & 16 & 9 & 40 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results of prediction power verification for the proposed RORC severity prediction model. certain model, this ratio (true positive rate) is referred to as sensitivity. On the other hand, when the result of an actual negative case is predicted as negative by a certain model, this rate is referred to as specificity [48]. The results are listed in Table 4.  
The higher the values of the sensitivity and specificity, the better the prediction performance. Thus, the ROC curve is expressed as a graph by defining the sensitivity on the \(y\)-axis and (1-specificity) on the \(x\)-axis. In the ROC curve, the area under the curve is called the area under the ROC curve (AUC), and this curve serves as a measure to evaluate the prediction power according to classification. When this value is close to 0.5, it indicates a low classification prediction power, and when this value is close to 1, it indicates a high classification prediction power [49]. Figure 6 shows the results of plotting the ROC curve with the Tree Augmented Naive Bayes (TAN) model. Because of the area under the ROC curve calculation, the AUC value was 0.763 for slight accidents, 0.797 for serious accidents, and 0.917 for fatal accidents. This shows that the classification prediction power for fatal accidents is the best.