# A Self-Paced Multiple Instance Learning Framework for Weakly Supervised Video Anomaly Detection
## 3 Proposed Method
### SP-MIL Framework
#### 3.2.2 Classifier Enhancement Using Self-Paced Learning
The processes of segment subset selection and classifier updating are iterative. During the \(t\)-th epoch of training, this self-paced learning-based segment selection and classifier-updating strategy can be formulated as: \[\min_{\mathbf{\omega}_{t},\mathbf{V}_{t}}E(\mathbf{\omega}_{t},\mathbf{V}_{t};\mathbf{\lambda}_{t })=\sum_{k=1}^{K}\sum_{i\in I_{k}^{t}}v_{ki}^{t}l_{ki}^{t}+h(\mathbf{V}_{t};\mathbf{ \lambda}_{t}),\] (8) where \(\mathbf{\omega}_{t}\) represents the model parameter of \(f_{t}\), \(\mathbf{V}_{t}\) denotes the self-paced learning weight matrix, and \(v_{ki}^{t}\) represents the self-paced learning weight of \(i\)-th segment in the \(k\)-th video. \(\mathbf{\lambda}_{t}\) represents the age parameter vector, and \(\lambda_{k}^{t}\) denotes the age parameter shared for all segments in the \(k\)-th video, which control the difficulty of the video. \(l_{k}^{t}\) is the easiest subset with the index of segments selected in the \(k\)-th video. For each video segment and its generated pseudo-label during the \(t\)-th epoch of training, \(l_{ki}^{t}\) denotes the segment-level loss, which can be calculated using the following formula: \[l_{ki}^{t}=-y_{ki}^{t}\text{log}(f(g(\mathbf{x}_{ki});\mathbf{\omega}_{t-1}))-(1-y_{ki} ^{t})\text{log}(1-f(g(\mathbf{x}_{ki});\mathbf{\omega}_{t-1})).\] (9)\(h(\mathbf{V}_{t};\mathbf{\lambda}_{t})\) is the self-paced learning regularizer; according to [25], it can be defined as \(h(\mathbf{V}_{t};\mathbf{\lambda}_{t})=-\sum_{k=1}^{K}\sum_{i=1}^{N}\lambda_{k}^{t}v_{ki}^ {t}\). To solve the optimization problem (8), we firstly need to estimated the age parameter \(\mathbf{\lambda}_{t}\) according to the loss values of the video segments during the \(t\)-th epoch of training. In particular, for all segments \(\{\mathbf{x}_{ki}\}_{i=1}^{N}\) and their corresponding pseudo-labels \(\{\mathbf{y}_{ki}\}_{i=1}^{N}\) of video \(\mathbf{X}_{k},\lambda_{k}^{t}\) can be estimated as follows. We first compute their corresponding loss values \(\{l_{ki}^{t}\}_{i=1}^{N}\). It is assumed that the easier the segment, the lower the corresponding loss value. According to this assumption, we sort the segment loss values in ascending order. Let \(\mathbf{L}=[l_{k1}^{t},l_{k2}^{t},\,\cdots,l_{kN}^{t}]\) be a vector of the sorted loss values. Then, \(\lambda_{k}^{t}\) can be defined as the \(R\)-th element of \(\mathbf{L}\), that is,  
\[\lambda_{k}^{t}=\mathbf{L}(R), \tag{10}\]  
where \(R\) denotes the number of selected segments for the video \(\mathbf{X}_{k}\) and is defined as follows:  
\[R=\lfloor R_{0}+\frac{t}{T}\times(R_{T}-R_{0})\rfloor, \tag{11}\]  
where \(\lfloor\cdot\rfloor\) denotes the floor function, \(R_{0}\) and \(R_{T}\) represent the number of segments selected for the subset with index \(l_{k}^{t}\) during the initial and final training epochs, respectively, and \(T\) represents the maximum number of training epochs. It is worth noting that the age parameter describes the difficulty of each segment. According to the definition of \(\lambda_{k}^{t}\), it will increase gradually with the training epochs \(t\). Then, more difficult segments will be added to the training set for classifier updating. This is also the key of self-paced learning for the strategy of easy-to-hard sample selection. Once all \(\lambda_{k}^{t}\) has been well defined, the optimization problem (8) can be solved using the alternative convex search method. Specifically, when \(\mathbf{\omega}_{t-1}\) is fixed, the optimal \(\mathbf{V}_{t}\) can be calculated via Equation (12) [12], and when \(\mathbf{V}_{t}\) is fixed, existing off-the-shelf supervised learning methods can be used to determine the optimal \(\mathbf{\omega}_{t}\). \[v_{ki}^{t}=1\,\,(l_{ki}^{t}\leq\lambda_{k}^{t}),\,\,\,\,\,\,\forall\,\mathbf{x}_{ ki}\in\mathbf{X}_{k},\,\,\forall\,k\in\{1,2,\cdots,K\}, \tag{12}\]  
Figure 3: An example to illustrate the parameter updates at each step of the loop.