# Risk-Quantification Method for Car-Following Behavior Considering Driving-Style Propensity
## 5 Experimental Analysis
### LightGBM-Based Risk Prediction for Car Following
#### 5.3.2 Analysis of Risk Prediction Results  
Classification models are assessed using metrics such as accuracy, precision, recall, and F1-score. Accuracy, the proportion of correctly classified predictions out of the total predictions, is a widely utilized metric for evaluating the overall predictive performance on a global scale. However, these metrics encounter limitations when applied to unbalanced datasets, diminishing their relevance for assessing accuracy in scenarios characterized by imbalanced risk levels. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & **Characteristics** & **Characteristic Number** \\ \hline Characteristics of following vehicles & Vehicle speed & 1 \\ \hline  & Relative longitudinal distance & \\  & between the following vehicle & \\  & and the vehicle in front of and & 2 \\ Characteristics of interaction & behind it & \\  & with vehicles in front and & \\  & behind & Longitudinal speed of the \\  & following vehicle relative to the & 2 \\  & leading and following vehicles & \\ \cline{2-3}  & Driving style propensity of & \\  & leading and following vehicles & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Predictive characteristics of car-follow risk. The failure to accurately predict medium-high-risk conditions as such carries significant safety implications and should be minimized. From a safety perspective, erring on the side of caution by classifying a low-risk scenario as medium-high risk is preferable to misidentifying actual medium-high-risk conditions as lower risk. Consequently, the objective shifts toward maximizing the recall of medium-high risk categories, even if it means sacrificing some precision. The emphasis is thus on enhancing recall for medium- and high-risk states while maintaining high precision. An evaluation of Decision Tree, Random Forest, and LightGBM algorithms, as illustrated in Figure 10, compares the precision and recall rates across three heeling risk-prediction models. The Decision Tree algorithm exhibits lower performance metrics, whereas LightGBM outperforms with the highest overall precision and recall rates. In Figure 11, the recalls for different vehicle risk propensities are shown in different color depths, with darker colors indicating larger values. The Decision Tree algorithm achieves a recall of 0.695 for medium-risk classifications, indicating that 69.5% of medium-risk samples are correctly identified, with a high-risk classification recall of 0.575, resulting in an average medium and high-risk recall of 0.635. The Random Forest algorithm records a medium-risk recall of 0.686 and a high-risk recall of 0.806, averaging a medium-high risk recall of 0.746. LightGBM demonstrates superior recall rates of 0.916 for medium-risk and 0.814 for high-risk classifications, achieving an average medium-high risk recall of 0.865. This performance underscores LightGBM's superior effectiveness in identifying medium-high risk conditions accurately. Table 6 summarizes the metrics of LightGBM, and it can be seen that the performance is better and more suitable for car-following risk level prediction. Figure 11: Categorical recall. Figure 10: Comparison of forecast results. Figure 12 displays the significance of various features within the risk level prediction model, highlighting the pivotal role of interaction attributes in determining risk. The most critical feature is the relative speed to the lead vehicle, underscoring its impact on risk assessment. Equally significant are the relative distances to the vehicles ahead and behind, indicating their substantial influence on risk evaluation. The driving style propensity of the driver ranks as the second and third most influential factors, respectively, showcasing its considerable effect in modeling car-following behavior. The notable impact of driver style propensity on additional crash likelihood emphasizes its relevance in car-following scenarios.