# MT-CMVAD: A Multi-Modal Transformer Framework for Cross-Modal Video Anomaly Detection
## 1 Introduction
For instance, in some cases, motion features might be more relevant (e.g., suspicious movements in a crowd), whereas in other cases, visual features might be more indicative (e.g., an abandoned object in a public space). Static fusion strategies that assign fixed importance to different modalities can lead to suboptimal detection performance, particularly in diverse and dynamic environments. _Challenge II:_ Limited Long-Range Dependency Modeling Most existing VAD models, including those based on CNNs and RNNs, struggle to effectively capture long-term dependencies in video sequences [10]. While transformers offer an advantage in this regard, existing transformer-based VAD methods primarily focus on spatial attention within individual frames or short sequences, neglecting long-term temporal relationships across extended video durations [11; 12]. This limitation is particularly problematic for anomalies that evolve gradually over time, such as progressive equipment malfunctions or gradual behavioral shifts in a monitored environment. Without the ability to model long-range dependencies, these methods may fail to recognize contextually significant patterns that distinguish normal from anomalous behavior. Our Contribution: Multi-modal Transformer for Cross-Modal Video Anomaly Detection (MT-CMVAD) To address these challenges, we propose a novel Multi-modal Transformer for Cross-Modal Video Anomaly Detection (MT-CMVAD). As illustrated in Figure 1, our framework processes input videos through a multi-modal pipeline, dynamically evaluates cross-modal features, and outputs anomaly detection scores. Unlike existing methods, MT-CMVAD introduces a dynamic cross-modal attention mechanism, which adaptively integrates visual and motion features based on their relative importance in different scenarios. By leveraging the transformer's capability to capture long-range dependencies, our method enhances cross-modal interaction and enables more effective anomaly detection. Our framework consists of the following key components:  
* Transformer-based Multi-modal Architecture: We design a transformer-based model that effectively integrates visual and motion cues, enabling the model to capture rich contextual information across different modalities. * Cross-modal Attention Mechanism: We introduce a novel cross-modal attention module, which dynamically weighs the importance of visual and motion features to ensure that the most relevant modality is prioritized for each specific anomaly. * Long-Range Dependency Modeling: Our framework extends the temporal modeling capability of transformers, allowing the network to effectively detect gradually evolving anomalies by capturing long-term dependencies in video sequences. * Extensive Experimental Evaluation: We conduct comprehensive experiments on multiple benchmark datasets, demonstrating that our approach outperforms state-of-the-art methods in terms of both accuracy and robustness. By addressing the limitations of existing VAD methods, our proposed MT-CMVAD framework advances the field of video anomaly detection by offering a more adaptive and interpretable approach to multi-modal fusion. Through the integration of transformers, cross-modal attention mechanisms, and long-range temporal modeling, our method provides a powerful solution for real-world anomaly detection tasks in surveillance, security, and autonomous monitoring applications. The remainder of this paper is organized as follows: Section 2 systematically reviews related work in video anomaly detection, analyzing the evolution from traditional feature engineering to transformer-based cross-modal methods. Section 3 details our proposed MT-CMVAD framework, with Sections 3.1-3.4 elaborating on the hierarchical architecture, encoder-decoder design, and adaptive scoring mechanism. Section 4 validates our method through extensive experiments on three benchmark datasets (UCF-Crime, UBI-Fights, and UBnormal), including ablation studies on key components.