# MT-CMVAD: A Multi-Modal Transformer Framework for Cross-Modal Video Anomaly Detection
## 1 Introduction  
In recent years, video anomaly detection (VAD) has gained increasing attention in the field of computer vision due to its crucial role in security surveillance, intelligent monitoring, and autonomous system safety [1, 2]. The ability to automatically detect rare or unexpected events in video streams is essential for a wide range of applications, such as public security, industrial anomaly monitoring, and traffic surveillance [3]. Unlike traditional video classification tasks, where each sample is associated with a predefined category, anomaly detection presents unique challenges due to the inherently unpredictable nature of anomalous events and the lack of sufficient training samples. Furthermore, anomalies can exhibit significant variation, ranging from subtle irregular motions to drasticenvironmental changes, making it challenging to develop robust detection models that generalize well across different scenarios. Traditional approaches to VAD rely on handcrafted features, such as optical flow, histograms of oriented gradients (HOG), and local binary patterns (LBP), to capture the spatial and temporal characteristics of videos [4]. While these methods offer some interpretability, they often struggle in complex and dynamic environments due to their limited ability to model high-dimensional representations. The rise of deep learning has significantly advanced anomaly detection, with convolutional neural networks (CNNs) and recurrent neural networks (RNNs) becoming dominant techniques for feature extraction and sequential modeling [5]. More recently, unsupervised deep learning methods, such as autoencoders and generative adversarial networks (GANs), have been explored to learn normal event distributions and detect anomalies as deviations from these learned patterns [6]. However, despite their effectiveness, these deep learning-based approaches face challenges in capturing long-range dependencies and modeling global contextual relationships, which are crucial for understanding complex anomalies. Recently, transformer-based architectures have demonstrated remarkable performance in various vision-related tasks, including image recognition, object detection, and action recognition [6; 7]. Originally introduced for natural language processing (NLP) [8], transformers have been successfully adapted to visual tasks due to their ability to model long-range dependencies and capture global contextual information. This capability has made transformers particularly promising for video analysis tasks, leading to their adoption in video anomaly detection. Several recent studies have explored transformer-based VAD methods, leveraging self-attention mechanisms to enhance the representation of normal and abnormal events [7]. However, most of these approaches focus primarily on a single modality, such as RGB frames or optical flow, which limits their ability to fully understand anomalous events. In real-world scenarios, anomalies often exhibit complex multi-modal characteristics, where visual and motion cues provide complementary information. For instance, certain anomalies may appear visually subtle while exhibiting distinct motion patterns, whereas others may involve clear visual deviations with minimal motion changes. Thus, effectively integrating multi-modal information is crucial for building robust and generalizable anomaly detection models. Despite the advancements in multi-modal VAD, existing methods still face two major challenges:  
_Challenge I:_ Ineffective Cross-Modal Fusion Many existing multi-modal anomaly detection methods rely on simple feature concatenation or weighted summation to integrate different modalities [9]. While these approaches offer a straightforward way to combine information, they often fail to capture the intricate dependencies between different modalities. In real-world surveillance, the importance of visual and motion cues varies across different scenarios and anomaly types.