# Urban Road Anomaly Monitoring Using Vision-Language Models for Enhanced Safety Management
## 3 Results
### Performance Evaluation of URA-VLMs
#### 3.3.1 Model Performance  
* Road anomalies classification  
After the model optimization, the overall accuracy of the URA-VLMs in road anomalies classification rose from 90.35% to 93.20%. Except for the identification and classification of road garbage (87.64%), the accuracy rates for classifying other road anomaly issues all reached 90% or higher (Table 9). Identifying road garbage poses greater challenges than detecting other anomalies, leading to lower accuracy rates. The low accuracy in road debris identification is attributed to its diversity, complex backgrounds, dataset limitations, and environmental factors [55]. These challenges are consistent across models, underscoring the need for further research and optimization in this area. Compared to state-of-the-art road anomaly AI models, URA-VLMs have achieved relatively high accuracy. A YOLOv3 was also employed for pothole detection, achieving a mean Average Precision of 65.65% [56]. A deep learning (DL) framework employing a convolutional neural network (CNN) classified four types of pavement cracks: longitudinal, transverse, alligator, and potholes, achieving an accuracy of 76%, which is considered reasonable compared to binary classification methods [57]. A proposed Mask R-CNN [58; 59] was utilized for pixel-wise segmentation of pavement regions to detect potholes and calculate their areas, achieving 90% accuracy in pothole area computation using a manually annotated dataset. The related research exhibits accuracies ranging from 65.65% to 90%, depending on the specific task and dataset. URA-VLMs have reached a notably high level of accuracy, showcasing better performance in comparison to alternative AI techniques. These large models utilize lightweight architectures and require minimal training data. * Road waterlogging depth estimation In assessing water depth within the validation dataset of 310 images, the URA-VLMs recorded a depth error of 33.57%, with a Mean Absolute Error (MAE) of 97.21 mm. The most \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**No.**} & \multirow{2}{*}{**Types**} & \multicolumn{2}{c}{**Number of Data Points**} & \multicolumn{2}{c}{**Model Accuracy (\%)**} \\ \cline{3-5}  & & **Points** & **InternVL-2.5 MPO (Preliminary Classification Prompt)** & **InternVL-2.5 MPO (Optimized Prompt)** \\ \hline
1 & Bad road & 214 & 90.19 & 95.33 \\
2 & Fallen tree & 296 & 96.62 & 96.62 \\
3 & Fire & 305 & 96.72 & 97.05 \\
4 & Flooding & 310 & 93.48 & 92.86 \\
5 & Garbage & 364 & 84.62 & 87.64 \\
6 & Normal & 675 & 92.00 & 92.00 \\
7 & Traffic accident & 433 & 92.84 & 93.53 \\
8 & Traffic jam & 437 & 79.18 & 93.59 \\  & Overall accuracy (\%) & 3034 & 90.35 & 93.20 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance evaluation of URA-VLM before and after optimization. significant discrepancy between the URA-VLMs and the annotated values was 470 mm (Table 10, Figure 6). Although the MAE indicates reasonable accuracy in depth estimation, it also reveals the potential for significant deviations. Notably, the maximum discrepancy between URA-VLMs and the annotated, highlighting the need for model refinement in certain extreme cases. In comparison, this finding aligns with previous research and enhances the MAE. The flood-GPT estimations yielded an MAE of 270 mm when compared to manual assessments [41]. Previous studies reported MAE values ranging from 70 mm to 310 mm [13; 26], which also employed deep learning methods to estimate floodwater depth from images sourced from online open dataset. The consistency in these results underscores the reliability of AI-driven approaches in flood depth estimation, with our model's performance demonstrating competitive accuracy within the established range of prior studies. * Road Safety level  
In the assessment of model safety levels, it is evident that the URA-VLM demonstrates strong performance, particularly the optimized version. The classification accuracy for the categories GREEN (92.42%), LIGHT GREEN (88.19%), and YELLOW (96.45%) approaches around 90% (Table 11).