# Three-Stage Deep Learning Framework for Video Surveillance
## 5 Conclusions  
In conclusion, this paper presents a three-stage deep learning architecture for anomaly detection in video surveillance, particularly focusing on CCTV footage. The architecture begins with the extraction of features from video frames, utilizing a range of pre-trained networks including Vision Transformer (ViT) and MobileNet. This approach allows for a comprehensive and effective analysis of the video data, leveraging the strengths of these varied networks. The second stage integrates BiLSTM with multi-head attention, blending the benefits of both techniques for effective temporal analysis. BiLSTM enhances the model's ability to understand temporal sequences by processing data in both the forward and backward directions, while multi-head attention allows the model to concurrently process and integrate multiple facets of the input sequence. This combination is particularly adept at handling complex, time-evolving data in video sequences. To address the challenges posed by long sequences, the third stage introduces a modified Transformer encoder layer. This layer, utilizing relative position embeddings, is adept at handling the dynamic nature of video data, capturing interactions across frames, and enhancing the understanding of inter-frame dynamics. The Transformer encoder, tailored to anomaly detection, further processes the output from the BiLSTM and multi-head attention layers, refining it for the final classification. The architecture concludes with a global maxpooling layer to distill key features, a dropout layer to prevent overfitting, and a sigmoid operation for final anomaly classification. The introduction of a fully connected layer further ensures precise classification based on the architecture's output. In this research, the proposed framework was validated using multiple open datasets containing various anomalous situations. The results demonstrate that it overcomes the limitations of existing video anomaly detection networks, achieving high detection accuracy and showing potential for application in real-time monitoring environments. This system enhances the monitoring accuracy and efficiency of security camera systems like CCTV and is applicable in diverse environments and situations, including scenarios without human presence in the footage. This versatility stems from the framework's ability to interpret features across the entire video and the relationships between frames, making it suitable for various scenarios and conditions. Recognizing the need for further research in detecting different types of anomalies, future studies will focus on enhancing the scalability of the proposed framework for broader applications in various settings. This research is expected to significantly improve the accuracy and scalability of video analysis in security camera systems like CCTV. \begin{table}
\begin{tabular}{c c} \hline \hline
**Number of Heads** & **AUC of CvlBiLT** \\ \hline
2 & 96.76 \\
4 & 98.24 \\
8 & 98.56 \\
12 & 97.78 \\
16 & 97.73 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of performance measurement according to the variation in the number of heads. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Position Embedding** & **UBI-Fights (AUC)** & **Crowd (ACC)** \\ \hline Absolute & 97.36 & 94.59 \\ Relative & 98.56 & 95.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance comparison of absolute and relative position embeddings in the CvlBiLT architecture. **Author Contributions:** Conceptualization, J.-W.L. ; Methodology, J.-W.L. ; Software, J.-W.L. ; Validation, [PERSON]-W.L. and H.-S.K. ; Formal analysis, J.-W.L. ; Investigation, H.-S.K. ; Resources, H.-S.K. ; Data curation, J.-W.L. ; Writing--original draft, J.-W.L. ; Writing--review & editing, [PERSON]-W.L. and H.-S.K. ; Visualization, J.-W.L. and H.-S.K. ; Supervision, H.-S.K. ; Project administration, H.-S.K. ; Funding acquisition, H.-S.K. All authors have read and agreed to the published version of the manuscript.