# Three-Stage Deep Learning Framework for Video Surveillance
## 1 Introduction  
The increasing prevalence of crime, ranging from minor offenses to violent acts, as highlighted in the \"2022 Crime Analysis\" report by the Supreme Prosecutors' Office of Korea, and shown in Figure 1, underscores the necessity of enhanced security measures. In this regard, the field of the video-based detection of abnormal situations, as referenced in numerous studies ([1; 2; 3; 4; 5; 6; 7; 8]), is experiencing a surge in interest. Research in this area, especially studies investigating surveillance video anomaly detection (SVAD) using deep learning [9], is progressing rapidly. This research domain has evolved to encompass a broader scope, extending beyond behavior-based detection to include advanced areas like human facial emotion recognition for anomaly detection [10], illustrating the expanding reach and depth of investigations in this field. These systems function by capturing real-time images across various indoor and outdoor settings, both public and private. However, the proliferation of security cameras leads to the generation of an immense volume of images, making manual monitoring susceptible to human error, as well as being labor-intensive and costly. Therefore, the automation of anomaly recognition in video footage is imperative. Video-based anomaly detection, in essence, aims to identify any unusual or non-standard activities and situations captured in video data, including incidents like violence, fires, health emergencies, unauthorized entry, and abduction. Unlike still images, which contain only spatial information, video data encompass temporal elements as well, necessitating the fusion of spatial and temporal data for accurate analysis due to the correlation between neighboring frames. Approaches to video-based anomaly detection, integrating both spatial and temporal aspects, encompass various learning paradigms: supervised, unsupervised, and semisupervised (one-class) learning. Supervised learning, which utilizes labeled data for both normal and abnormal instances, generally achieves higher accuracy. Nonetheless, it is limited by the need for diverse atypical samples. To overcome these challenges, our approach focuses on semi-supervised learning (or one-class learning), which primarily uses normal sample data for training. Within this domain, the one-class support vector machine (OCSVM) is widely used, and recent advancements in deep support vector data description (Deep SVDD) [11] have further extended this concept into the realm of deep learning. To address the limitations of semi-supervised learning methods, which require labeled normal samples, unsupervised learning techniques are being investigated. These techniques operate under the assumption that the majority of the data consist of normal samples and can be learned without labels. A prime example of this approach is the autoencoder [12; 13], featuring an encoder that compresses the input into a feature vector and a decoder that reconstructs the original input from this feature vector. While previous studies have primarily concentrated on harnessing three-dimensional information for learning, they have struggled with reliability in analyzing complex or dynamic video backgrounds and long video sequences. To overcome these issues, our research introduces a new three-tiered network model for video anomaly detection, adept at handling both the temporal and spatial aspects of video data. The key contributions of this study are outlined as follows. 1. Introduction of a Three-Stage Network: This paper introduces a groundbreaking three-stage deep learning architecture for anomaly detection in video streams, notably in CCTV systems. This approach effectively addresses both the spatial and temporal dimensions of video data, advancing beyond the capabilities of existing methods. 2. Utilization of Pre-Trained Networks for Feature Extraction: The first phase of our approach leverages pre-trained networks, such as Vision Transformers (ViT), to extract feature vectors from video frames.