# Regional Traffic Event Detection Using Data Crowdsourcing
## 4 Performance Evaluation
To evaluate the accuracy of the event occurrence region extraction, we extracted 100 random regions five times and calculated the accuracy based on the five repetitions. In this study, accuracy was evaluated based on the receiver operating characteristic (ROC) curve, area \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Region** & **ID** & **Date and Time** & **Content** \\ \hline \begin{tabular}{c} (Jeollabuk-do) \\ \end{tabular} & \begin{tabular}{c} ([PERSON] *) \\ \end{tabular} & 5 March 2021 17:22 & \begin{tabular}{c} (Accident between truck and \\ taxi in the left-hand lane of \\ the Jeonbuk Girls’ High \\ School entrance in Dongsan \\ Plaza, Baldan-ri Intersection, \\ Monguba-daero) \\ \end{tabular} \\ \hline \begin{tabular}{c} (Jeollabuk-do) \\ \end{tabular} & \begin{tabular}{c} ([PERSON] *) \\ (avid vie *) \\ \end{tabular} & 5 March 2021 21:26 & \begin{tabular}{c} (Please note the hat a collision \\ occurred in the second lane of \\ the Bongwon Bridge, \\ Gwangheungchang Station, \\ Seogang-ro, Seoul) \\ \end{tabular} \\ \hline \begin{tabular}{c} (Busan) \\ \end{tabular} & \begin{tabular}{c} (Jeollabuk *) \\ ([PERSON] *) \\ \end{tabular} & 5 March 2021 21:35 & \begin{tabular}{c} (Jungang-daero, Gwangmu \\ Bridge, Seoweon \\ Intersection congestion) \\ \end{tabular} \\ \hline \begin{tabular}{c} (Busan) \\ \end{tabular} & \begin{tabular}{c} ([PERSON] *) \\ \end{tabular} & 5 March 2021 21:36 & \begin{tabular}{c} (At the intersection of \\ Jungang-daero and Jocheon \\ Intersection, inside Jocheon \\ Intersection, lane 1 is under \\ construction) \\ \end{tabular} \\ \hline \begin{tabular}{c} (Busan) \\ \end{tabular} & \
\begin{tabular}{c} ([PERSON] *) \\ \end{tabular} & 5 March 2021 21:36 &
\begin{tabular}{c} (Guidance from the Incheon \\ Metropolitan Office of \\ Education: minimize \\ face-to-face interactions due \\ to the COVID-19 situation. A \\ drive-through Car TaGo Civil \\ Complaint Call service is \\ being provided to citizens) \\ \end{tabular} \\ \hline \hline \end{tabular}  
* is a mark to protect the informant’s personal information.  
\end{table} 
Table 4: Example of collected data. under the curve (AUC), precision, recall, and F-measure. In the proposed technique, the data were classified via binary classification into relevant data (those that influence traffic conditions) and irrelevant data (those that did not influence traffic conditions). The influential data were classified via multi-class classification according to six event types: construction, weather, accident, congestion, special events, and others. We determined which models were suitable for binary and multi-class classification using the classified data. The machine learning models used in the binary classifier are naive Bayes, random forest, SVC, linear SVC, and logistic regression. The parameters for all of the models are set to default. Machine learning models used in multiple classifiers are naive Bayes, random forest, SVC, linear SVC, BiLSTM, and TextCNN. The parameters for naive Bayes, random forest, and BiLSTM are set to default values. TextCNN constructs a Convolutional Neural Network (CNN) based on the model proposed in [14]. We applied dropout to the proposed technique to improve generalization and avoid model overfitting. We set the parameters of the linear SVC model to C = 1.0, penalty = \"L2\", multi_class = \"ovr\", and the parameters of the SVC model to C = 1.0, kernel = \"rbf\", decision_function_shape = \"ovr\". In the linear SVC and SVC models, we changed the value of C from 0.1 to 1000, but there was no significant change. Therefore, we performed a performance evaluation by changing the value of C to 1.0. Figure 6 shows the ROC curves and AUC of the binary classifiers. Figure 6a shows the ROC curves of the naive Bayes, random forest, SVC, linear SVC, and logistic regression binary classifiers. The curves indicate that the true positive rates (TPR) and false positive rates (FPR) of the five models used for the proposed technique were close to one.