# Regional Traffic Event Detection Using Data Crowdsourcing
## 4 Performance Evaluation
The AUC results in Figure 6b show that all five models had performance levels of \(\geq\)0.9. SVC, linear SVC, and logistic regression had the best performance levels with values of \(\geq\)0.95. Figure 7 shows the precision, recall, and F-measures of binary classifiers. In terms of precision, random forest, and linear SVC exhibited the highest values at 0.9, followed by SVC and logistic regression models with values of 0.88 and 0.87, respectively. The Naive Bayes model exhibited the lowest precision among the examined models at 16%. In terms of recall, the random forest and linear SVC models also exhibited the highest values at 0.86, followed by SVC at 0.81 and logistic regression at 0.80. The naive Bayes model had the worst performance among the tested models with a recall of 0.53. In terms of the F-measure, which is the harmonic mean of precision and recall, random forest, and linear SVC, linear SVC had the highest value at 0.86, corresponding to a 43% performance increase over naive Bayes. Based on the results of ROC curves, AUC, precision, recall, and F-measures, we selected the linear SVC model as the binary classifier for traffic event detection. Figure 6: Binary classifier ROC curves and AUC. (**a**) ROC; (**b**) AUC. Figure 8 shows ROC curves and AUC for the six multi-class classifiers. Figure 8a shows the ROC curves, in which the curve for the naive Bayes model was the furthest from 1, whereas those of the linear SVC and SVC models were the closest to 1. Figure 8b shows the AUC values of all models, which exhibited values of 0.9 or greater. As such, their performance in this aspect was satisfactory. Of the evaluated models, the linear SVC model and SVC model performed best, with AUC values of 0.98. Figure 9 shows precision, recall, and F-measures for each class. For the Construction class, linear SVC and SVC performed best, with precision, recall, and F-measure values of 0.92, 0.93, and 0.92, respectively. For the Weather class, the naive Bayes model showed the best values, at 0.8, 0.82, and 0.86, respectively, whereas random forest showed the second-best values at 0.68, 0.93, and 0.78, respectively. For the Accident class, random forest, linear SVC, and SVC showed the highest precision, recall, and F-measure values at 0.98, 0.98, and 0.98, respectively. For the Congestion class, linear SVC and SVC had the highest performance, with values of 0.8, 0.98, and 0.88, respectively. For the Crowded Event class, linear SVC had the highest values, at 0.83, 0.68, and 0.  
second-highest values, at 0.7, 0.68, and 0.73, respectively. Finally, for the Others class, linear SVC and SVC had the highest values at 0.94, 0.6, and 0.73, respectively, and 0.94, 0.58, and 0.72, respectively. To compare the overall classification performance of the six models, Figure 10 shows the average values of their F-measures, which show that linear SVC had the best performance, with an average value of 0.83, corresponding to a performance difference of approximately 19% compared to that of the naive Bayes model. In our research, it was shown through performance evaluations the Linear Support Vector Machine (LSVM) achieved much better performance than Convolutional Neural Networks and naive Bayes. These results suggest that models that perform predictions focusing on the correlations between  
Figure 9: Multi-class classifier precision, recall, and F-measure by class: (**a**) Construction; (**b**) Weather; (**c**) Accident; (**d**) Traffic Jam (**e**) Crowded Event; (**f**) Others. specific words and labels rather than the inter-relationships between words within sentences are more consistent with our data characteristics. In other words, our crowdsourcing data are composed of specific words rather than relationships between words. Therefore, we show better performance when we apply our crowdsourcing data to SVM. As a result of the performance evaluations, linear SVC was chosen as the most suitable model for use as the binary classifier and the multi-class classifier for event classification.