# Weight Feedback-Based Harmonic MDG-Ensemble Model for Prediction of Traffic Accident Severity
## 4 Result and Performance Evaluation
### Performance Evaluation  
The data used for the performance evaluation is the record of fatalities in traffic accidents from the Korea Road Traffic Authority. This is open data composed of road risk index calculated through the fusion analysis of traffic accident information, information on frequent accident areas due to traffic weakness and accident characteristics, and data such as past traffic accidents, weather, and accidents. Through traffic accident information, it is possible to extract various types of information such as where an accident occurred, severity, road risk index, etc., that can reduce traffic accidents and improve the efficacy of social cost reduction measures. For performance evaluation, the traffic accident death data are preprocessed and divided into 70% training data, 10% validation data, and 20% testing data. The proposed weight feedback-based harmonic MDG-ensemble model is used to predict traffic accident severity based on the ensemble of multiple classification algorithms. Confusion matrix-based accuracy, recall, precision, and F1-Score are used for classification evaluation metrics. Performance evaluation is conducted on four aspects. The first is to compare the proposed ensemble model with the models based on random forest, KNN, and XgbTree as single algorithms. The second is to compare model accuracy according to the presence or absence of MDG-based variable selection. The third is to compare accuracy by applying arithmetic means, geometric means, and harmonic means to weights to evaluate the suitability for harmonic mean application in weight feedback. Finally, the performance evaluation of the proposed model is compared with a conventional accident prediction model. Table 3 shows the confusion matrix for the performance evaluation of traffic accident severity. This matrix is capable of presenting multi-categorical variables according to which models [38] are used. Traffic accident severity is classified as Slight, Serious, and very serious. _Pred_ represents a prediction value, and _Real_ is an actual value. The multi-categorical confusion matrix in Table 3, makes it difficult to measure the accuracy, recall, precision, and F1-Score. Therefore, it is converted into a confusion matrix of the binary type (True and False), which consists of true positive (TP), true negative (TN), false positive (FP), and false negative (FN). TP is the number correctly classified into a \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Real** & **Pred** & **Slight** & **Serious** & **Very Serious** \\ \hline Slight & a & b & c \\ Serious & d & e & f \\ very serious & g & h & i \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Confusion matrix for the performance evaluation of traffic accident severity. particular category; TN is the number classified into a particular category but is different from an actual value; FP represents the number correctly classified into categories other than a particular category; FN represents the number classified into categories other than a particular category. For instance, for the category Slight, the TP is a, TN is e + f + h + i, FP is i, and FN is b + c + d + g. As such, TP, TN, FP, and FN are drawn for each category, and thus accuracy, recall, precision, and F1-Score are defined [39]. Equation (2) shows the accuracy. This means the accurately predicted ratio for the total prediction. \[\mathbf{Accuracy}=\frac{\mathbf{TP}+\mathbf{TN}}{\mathbf{TP}+\mathbf{TN}+\mathbf{FP}+\mathbf{FN}} \tag{2}\]  
Equation (3) shows the recall. This means the accurate classification of an actual value in a particular category. \[\mathbf{Recall}=\frac{\mathbf{TP}}{\mathbf{TP}+\mathbf{FN}} \tag{3}\]  
Equation (4) shows the precision. This means that a number's classification in a particular category is consistent with an actual value. \[\mathbf{Precision}=\frac{\mathbf{TP}}{\mathbf{TP}+\mathbf{FP}} \tag{4}\]  
Equation (5) presents F1-score using precision and recall.