# A Self-Paced Multiple Instance Learning Framework for Weakly Supervised Video Anomaly Detection
## 3 Proposed Method
### SP-MIL Framework
#### 3.2.2 Classifier Enhancement Using Self-Paced Learning  
The SP-MIL framework models the WS-VAD task as a segment-level supervised learning problem. The main idea is to initially learn easy segments and then gradually transfer the acquired knowledge to recognize harder ones. By gradually exposing the classifier to more segments and their pseudo-labels during training, the classifier can identify difficult anomalies more effectively than the baseline model, thereby achieving more unbiased prediction. Specifically, the SP-MIL framework consists of four primary steps: segment-level score prediction, segment-level pseudo-label generation, adaptive segment subset selection, and classifier updating. For illustrative purposes, let superscript \(t\) denote the \(t\)-th epoch of training. For each video \(\mathbf{X}_{k}=\{\mathbf{x}_{ki}\}_{i=1}^{N}\) with \(N\) segments, the segment features \(\{\mathbf{z}_{ki}\}_{i=1}^{N}\) are used to generate the anomaly scores \(\{s_{ki}^{t}\}_{i=1}^{N}\) using the classifier \(f_{t-1}\) from the \((t-1)\)-th epoch of training. These anomaly scores are then binarized to produce segment-level pseudo-labels \(\{y_{ki}^{t}\}_{i=1}^{N}\). Subsequently, the segment subset with index set \(I_{k}^{t}\) is adaptively updated based on the estimated age parameter \(\lambda_{k}^{t}\) and segment-level self-paced learning weights \(\{v_{ki}^{t}\}_{i=1}^{N}\). The age parameter vector for all \(K\) videos is represented as \(\mathbf{\lambda}_{t}=[\lambda_{1}^{t},\cdots,\lambda_{K}^{t}]\in\mathbb{R}^{K}\), where \(\lambda_{k}^{t}\) represents the age parameter shared for all segments in the \(k\)-th video during the \(t\)-th epoch of training. The self-paced learning weight matrix for all segments of all videos during the \(t\)-th epoch of training is represented as \(\mathbf{V}_{t}=[\mathbf{v}_{1}^{t},\cdots,\mathbf{v}_{K}^{t}]^{T}\in\mathbb{R}^{K\times N}\), where \(\mathbf{v}_{k}^{t}=[v_{k1}^{t},v_{k2}^{t},\cdots,v_{KN}^{t}]\) and \(v_{ki}^{t}\) represents the self-paced learning weight of \(i\)-th segment in the \(k\)-th video during the \(t\)-th epoch of training, which is a hard weight and takes the binary values 0 or 1, indicating whether the video segment is selected or not. Finally, these segments in the selected subset with their corresponding pseudo-labels are used for supervised training, refining the classifier \(f_{t-1}\) to obtain \(f_{t}\). More details for each step are introduced as follows. Additionally, the details of parameter updates at each step are illustrated in Figure 3.  
1. Segment-level score prediction. For each video \(\mathbf{X}_{k}\) during the \(t\)-th epoch of training, the classifier \(f_{t-1}\) takes video segments \(\{\mathbf{x}_{ki}\}_{i=1}^{N}\) as input and output segment-level anomaly scores \(\{s_{ki}^{t}\}_{i=1}^{N}\) by using the following formulation: \[s_{ki}^{t}=f(g(\mathbf{x}_{ki});\mathbf{\omega}_{t-1}),\quad\forall\;\mathbf{x}_{ki}\in \mathbf{X}_{k},\;\forall\;k\in\{1,2,\cdots,K\},\] (6) where \(\mathbf{\omega}_{t-1}\) are network parameters of classifier \(f_{t-1}\). 2. Segment-level pseudo-label generation. For each segment-level anomaly score \(s_{ki}^{t}\), its segment-level pseudo-label \(y_{ki}^{t}\) is generated using a pre-defined binarization threshold \(\eta\). The calculation formula is as follows: \[y_{ki}^{t}= \begin{cases}1\;(s_{ki}^{t}>\eta),\;\;\forall\;k\in I_{+}\\ 0,\;\forall\;k\in I_{-}\end{cases} \] (7) where 1 is an indicator function. Equation (7) demonstrates that during the \(t\)-th epoch of training, the segment-level pseudo-label is assigned to a value of 1 for segments in abnormal videos if their anomaly score exceeds the threshold \(\eta\); otherwise, it is set to 0. It is worth noting that for segments in normal videos, the segment-level pseudo-label is consistently set to 0. 3. Adaptive segment subset selection. The purpose of segment selection is to adaptively identify the most easily classified subset of segments from each video in every training epoch.