# Ensemble-Based Knowledge Distillation for Video Anomaly Detection
## 2 Related Works
### Weak Supervision for Video Anomaly Detection  
As a critical study for this domain, Ref. [17] represents a video anomaly detection task as a regression problem and utilizes deep multiple instance learning (MIL) based formulation. For providing ground-truth information in the training process, the study only uses video-level labels; the labels represent whether a video sample contains an abnormal scene or not. Another study, Ref. [14] assumes the weak supervision concept as a one-sided label noise problem and also deploys Graph Convolutional Network (GCN) structures by proposing feature similarity and temporal consistency concepts in videos. Ref. [20] proposes that normal scenes in videos contain predictable patterns, while anomaly scenes do not include such patterns. With this assumption, the study uses a margin learning approach with convolutional LSTM and encoder network structures to determine the boundaries between abnormal and normal patterns more precisely. Ref. [21] utilizes a unique inner bag loss function taking into account the highest and lowest anomaly score values in each bag within the MIL problem. By this formulation, the study's approach tries to make sure that a positive bag has a large difference between the lowest and highest score and a negative bag has a small difference. The approach also makes use of the temporal convolutional network (TCN) using pre-trained C3D features [22]. Ref. [23] deploys a temporal MIL approach by using an attention mechanism to determine which segment of the video is more crucial than another. The study also makes use of a temporal augmented network to leverage motion patterns for the classification process. Ref. [24] develops a Siamese neural network-based decision structure to fit a distance function between pairwise video sub-sequences by a data-driven concept. Consequently, if a video sub-sequence is not similar enough compared with another video sub-sequence by the decision network, then the video sub-scene is supposed to be abnormal. Ref. [25] relies on a multimodal approach that combines the visual and audio features of the video file. Then, these combined features are input to three-sectioned sub-neural networks working concurrently with each other and representing different aspects of relations between video sub-sequences. Ref. [26] advances the network structure by a distance metric-based clustering loss function to minimize the distance interval between clusters of normal videos and maximize the distance interval between clusters of abnormal videos. Also, the study uses the normalcy suppression approach to reduce the anomaly scores belonging to normal sections of the video file. Finally, it also deploys a batch-based training process depending on random selection to decrease inter-batch relations. Ref. [27] develops a multiple instance pseudo label generator structure, which utilizes a sampling approach to generate proper clip-level artificial annotations. The study also uses a self-attention-based encoder structure to focus on abnormal regions of videos and a self-training process. Ref. [28] thinks of the MIL concept as a Robust Temporal Feature Magnitude Learning (RTFM) concept, in which a feature magnitude learning function is used for focusing especially on rare abnormal patterns and preventing the model from having a large bias to normal events in videos. The RTFM approach also deploys dilated convolutions and self-attention mechanisms to handle variable-size temporal interval relations of patterns in videos. [PERSON]. [29] proposes a weak supervision MIL-based spatio-temporal classification mechanism to localize spatio-temporal regions in consecutive frames. Within this mechanism, the study develops a dual-branched structure model, in which each branched structure makes use of a relation module to handle possible object and behavior relationships for anomaly detection by a self-attention mechanism. The study also deploys a separate approach to pass learned representations from one branch structure to another. [PERSON].