# Risk Prediction for Winter Road Accidents on Expressways
## 4 Results  
Table 3 shows the mean and standard deviation of the validation prediction performance for the optimal hyper-parameters obtained by a 5-fold cross validation on the training data with resampling technique. Results of performance evaluation of data with resolved imbalance, the random forest model achieved an accuracy = 0.991, kappa = 0.982, and AUC= 0.986, which showed its superior prediction performance compared to the other models. When evaluated based on accuracy, the random forest model showed the highest accuracy, followed by XGBoost, neural network, and logistic regression. The optimal hyper-parameters selected for each model were as follows: random forest (mtry = 2), XGBoost (eta = 0.272, max_depth = 8, gamma = 1.351, colsample_bytes = 0.341, min_child_weight = 1, subsample = 0.824, nrounds = 455), and neural network = 0.975 (size = 20, decay = 0.057). As the random forest model and the XGBoost model use algorithms based on CART, variables that significantly reduce impurities can be identified [40]. Figure 2 shows the calculated average of the variable importance of the 10 training data generated by the SMOTE technique. Geographic location (lon, lat) and humidity (rh) were identified as important factors in both random forest and XGBoost. Thus, these are the most important factors in road accidents caused by weather. When high humidity combines with low temperature, fog and frost can occur, leading to the accumulation of ice on the roads, and this has a significant impact on weather-related accidents in the winter season [39]. Among the various types of weather information, temperature, wind speed, and air pressure were identified as important factors in road accidents caused by weather, and precipitation was relatively insignificant. Among the road geometry factors, roads with a short turning radius and a large angle of rotation were identified as the main causes of road accidents due to weather, and the presence or absence of tunnels and bridges was not significant. Among the road environment variables, altitude demonstrated the greatest effect on road accidents due to weather, and the difference in altitude from the surrounding area had an effect in the order of north, south, west, and east side, of the surrounding area. Table 4 shows the summary of the logistic regression model. 'Est.' denotes estimates of logistic regression model and Exp(Est.) denotes exponential estimates which means odds ratio. A 95% confidence interval (CI) shows the significance of the independent variables. The statistically significant variables of the logistic regression model are hour, latitude, and relative humidity. Hour and latitude have a negative relationship and relative humidity has a positive relationship with weather-related accidents. The results were similar to the order of important variables derived from machine learning. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline  & \multirow{2}{*}{**N**} & \multicolumn{2}{c}{**Accuracy**} & \multicolumn{2}{c}{**Kappa**} & \multicolumn{2}{c}{**Auc**} \\ \cline{3-8}  & & **Mean** & **SD** & **Mean** & **SD** & **Mean** & **SD** \\ \hline Logistic & 10 & 0.823 & 0.003 & 0.645 & 0.005 & 0.691 & 0.057 \\ Neural & 10 & 0.972 & 0.001 & 0.945 & 0.003 & 0.958 & 0.012 \\ Network XGBoost & 10 & 0.991 & 0.001 & 0.982 & 0.002 & 0.978 & 0.008 \\ Random Forest & 10 & 0.991 & 0.001 & 0.982 & 0.002 & 0.986 & 0.002 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The prediction performance of validation data of balance data. Table 5 presents the results of the prediction performance evaluation of the test data for different models using the optimal hyper-parameters. The random forest model demonstrated the highest prediction performance with an accuracy = 0.984, kappa = 0.658, and AUC = 0.907, F-measure = 0.992, G-mean = 0.904 which was far greater than that of the other models. The prediction performance, in increasing order of accuracy was, the random forest model, XGBoost, neural network, and logistic regression.