# Combining Keyframes and Image Classification for Violent Behavior Recognition
## 3 Proposed Method
### Our Training Method  
Current violence-detection systems mainly extract the spatiotemporal features of the video, leading to the accumulation of errors in consecutive frames. For example, the 3 DCNN-based method extracts 16 consecutive frames as input, and this method will bring many uncertainties. The movement of the camera and changes in illumination will generate more errors. In addition, these structures will increase the structural complexity of the model and increase the hardware requirements. The human pose in violent video frames is significantly different from that of normal video frames, and can thus be classified by the deep-learning model. This reduces the structural complexity of the model. We tested the recognition effect of ResNet18, ResNet34, ResNet101, DesNet121, VGG19, and GoogleNet on frames containing violence and proved that this is feasible. These models are trained by segmented-original image pairs. The segmented and original images guide each other, forcing the network to make consistent predictions for the original and segmented images. Specifically, if the model is robust to noise, the representation of an original image should also predict the target concerning its segmented version, and vice versa. This is equivalent to creating a feature space between the segmented image and the original image; theoretically, such a robust feature space can be found. We provide an interpretable theoretical proof below. **Proof**. \(A_{n\times m}\) is the perfect feature space of the segmented images. \(B_{n\times m}\) is the perfect feature space of the original images. \(n\) is the number of images, and \(m\) is the dimension of the image feature vector. \(a_{i}\) is the feature representation of \(segment\_image_{i}\), \(b_{i}\) is the feature representation of \(original\_image_{i}\), \(l_{i}^{\prime}\) is the model prediction feature of \(segment\_image_{i}\), and \(l_{i}^{\prime\prime}\) is the model prediction feature of \(original\_image_{i}\):  
\[A_{n\times m}=[a_{1},a_{2},a_{3}\cdots a_{n}]^{\mathsf{T}},a_{i}=(\alpha_{i1 },\alpha_{i2},\ldots\alpha_{im}),l_{i}^{\prime}=(x_{i1},x_{i2},\ldots x_{im})\]  
Figure 5: Our dataset. (**a**) Original image, (**b**) Segmented image. \[B_{n\times m}=[b_{1},b_{2},b_{3}\cdots b_{n}]^{T},b_{i}=(\beta_{i1},\beta_{i2} \cdots\beta_{im}),\ \ l_{i}^{\prime\prime}=(y_{i1},y_{i2,\cdots}y_{im})\]  
\[Loss=\sum_{i=1}^{n}\|l_{i}^{\prime}-a_{i}\|^{2}+\|l_{i}^{\prime\prime}-b_{i}\|^ {2}\]  
\[=\sum_{i=1}^{n}\sum_{j=1}^{m}\Bigl{[}\left(x_{ij}-\alpha_{ij}\right)^{2}+\left( y_{ij}-\beta_{ij}\right)^{2}\Bigr{]}\]  
When \(\Bigl{[}\left(x_{ij}-\alpha_{ij}\right)^{2}+\left(y_{ij}-\beta_{ij}\right)^{2} \Bigr{]}\) is at its smallest, \(Loss\) is also at its smallest. \(\Box\)  
\[f\left(x_{ij},y_{ij}\right)=\left(x_{ij}-\alpha_{ij}\right)^{2}+\left(y_{ij}- \beta_{ij}\right)^{2} \tag{1}\]  
We define the feature representation difference between the original image and its background-removed image as the distance. The segmented image is the foreground information of the original image, so the distance between them is small. Therefore, we add Function (2). Constraint (3) is obtained from Function (2). \[l_{i}^{\prime}-l_{i}^{\prime\prime}=\Delta l_{i},\Delta l_{i}=(\Delta_{i1}, \Delta_{i2},\Delta_{i3,\cdots}\Delta_{im}),\ \ x_{ij}-y_{ij}=\Delta_{ij} \tag{2}\]  
\[g\left(x_{ij},y_{ij}\right)=y_{ij}-x_{ij}+\Delta_{ij}=0 \tag{3}\]  
We aim to find the minimum value of \(f\left(x_{ij},y_{ij}\right)\) under Constraint (3). Then use the Lagrange multiplier to solve \(L\left(x_{ij},y_{ij},\ \lambda\right)=f\left(x_{ij},y_{ij}\right)+\lambda g \left(x_{ij},y_{ij}\right)\) to obtain the minimum point.