# Innovative Approaches to Traffic Anomaly Detection and Classification Using AI
## 2 Algorithms and Methods
### Multimodal Large Language Model (MLLM)
MLLMs also face challenges in real-time inference, as their multimodal fusion and reasoning processes are computationally intensive, making them less suitable for deployment in live traffic environments compared to more streamlined CNN or GAN models. Additionally, many MLLM studies rely on curated datasets with well-lit, clean scenes, which limits their applicability to low-light or cluttered real-world settings. Recent research points to future directions such as more efficient training strategies, optimized architectures for low-latency use, and integration with complementary sensor data to enhance performance in practical deployments. CityLLaVA Efficient Fine-Tunning for VLMs in City Scenario  
In [63], the authors introduced CityLLaVA, a framework designed to optimize VLMs for road safety analysis in urban settings, focusing on insurance inspection and accident prevention. By enhancing the understanding and prediction of critical events in cities, CityLLaVA addresses key challenges in traffic safety. The framework utilizes the fine-tuning process shown in Figure 21, which incorporates four key strategies:  
* Bounding boxes optimize data preprocessing by selecting the best video views and focusing analysis on areas of interest, such as pedestrians and vehicles. * Question sequences and textual prompts guide the model to evaluate key elements like position, movement, and environment, adapting to various traffic scenarios. * Block expansion introduces additional decoder blocks, improving accuracy without significant overhead. * Increased prediction based on sequential questioning enhances accuracy by using previously obtained information in an ordered questioning approach. A major innovation in CityLLaVA is the use of visual and textual prompts. Visual prompts, guided by bounding boxes, help align information in each frame, while textual prompts, tailored to road safety, extract relevant details more effectively. The framework uses block expansion to improve fine-tuning efficiency, outperforming methods like low-rank adaptation (LoRA) [66]. Sequential questioning further enhances predictions by incorporating prior context. CityLLaVA achieved first place in the AI City Challenge 2024 [19] in the traffic safety description and analysis category, demonstrating its potential as a powerful tool for urban road safety analysis. VisionGPT [67] uses LLMs for real-time anomaly detection, enabling safe visual navigation for individuals with visual impairments. By combining the open-vocabulary object detection capabilities of Yolo-World with LLM intelligence, VisionGPT identifies obstacles and provides concise audio descriptions to inform users of potential hazards. The system analyzes camera frames, dividing each image into four regions--left, right, front, and ground--and classifies objects by position to assist in hazard identification. Detailed information on each object, including classification, size, and location, is recorded, with alerts triggered for significant obstacles, particularly in the \"ground\" or \"left/right\" regions. VisionGPT's integration of LLMs enhances adaptability to diverse contexts, with the study exploring the impact of prompt design for future improvements in accessibility. Experiments highlight the system's high accuracy in anomaly detection, particularly under low-sensitivity settings, and its compatibility across platforms. VisionGPT achieves an end-to-end latency of 60 ms on mobile devices, ensuring fast and reliable feedback for users. Finally, to conclude this section, the following Table 1 presents a structured comparative analysis of the most prominent AI-based approaches used in traffic anomaly detection. It outlines the key specifications, advantages, limitations, and--newly added--real-world applications for each method, offering a clearer understanding of their practical relevance. This synthesis aims to assist researchers and practitioners in identifying suitable techniques based on their operational requirements and deployment contexts.