# Lightweight Violence Detection Model Based on 2D CNN with Bi-Directional Motion Attention
## 3 Proposed Violence Detection Model
### Improved EfficientNet-B0
The downscaled feature is given by  
\[\text{F}^{r}=\text{Conv}_{1}*\text{F},\ \text{F}^{r}\in\mathbb{R}^{\text{T} \times\text{C}/r\times\text{H}\times\text{W}}, \tag{2}\]  
where \(r=2\) is the reduction ratio, \(*\) indicates the convolution calculation. Based on the downscaled feature, motion information is then derived. Similar to ME [44], motion information is obtained by computing the feature-level difference between two adjacent frames. A \(3\times 3\) 2D convolution (Conv2) is used as the mapping function to deal with the problem of spatial misalignment. Specifically, given the features of two adjacent frames \(\text{F}^{r}(t)\) and \(\text{F}^{r}(t+1)\), we compute the bi-directional motion information as per the following:  
\[ \begin{array}{l}\text{M}_{f}(t)=\text{Conv}_{2}*\text{F}^{r}(t+1)-\text{F}^ {r}(t),\ t\in[1,\text{T}-1],\\ \text{M}_{b}(t)=\text{Conv}_{2}*\text{F}^{r}(t)-\text{F}^{r}(t+1),\ t\in[1, \text{T}-1],\end{array}  \tag{3}\]  
where \(\text{M}_{f}(t)\) is the forward difference and \(\text{M}_{b}(t)\) is the backward difference. For the last time step \(t=\text{T}\), we set the motion information as \(\text{M}_{f}(\text{T})=0\) and \(\text{M}_{b}(\text{T})=0\). Then, another \(1\times 1\) 2D convolution (Conv3) is employed together with a concatenation (concat) layer in each branch to restore the original shape of the input feature F. After that, forward and backward attention weights, \(\text{W}_{f}\) and \(\text{W}_{b}\), are generated separately by using sigmoid functions as per the following:  
\[ \begin{array}{l}\text{W}_{f}=\delta(\text{concat}(\text{Conv}_{3}(\text{M}_ {f}(t))))\\ \text{W}_{b}=\delta(concat(\text{Conv}_{3}(\text{M}_{b}(t))))\end{array} , \tag{4}\]  
where \(\delta(\bullet)\) is the sigmoid activation function, \(\text{concat}(\bullet)\) stands for concatenation function. The attention map A is computed by averaging the forward and backward attention weights as follows:  
\[\text{A}=0.5\text{W}_{f}+0.5\text{W}_{b}. \tag{5}\]  
As shown in Figure 5, a weighted feature can be obtained by performing Hadamard product operation \(\text{A}\odot\text{F}\). DropConnect is an effective method to improve the generalization ability of neural network models [49]. It randomly sets the weights to zero according to a certain probability. To improve the generalization ability of the model, we put a DropConnect structure after the \(\text{A}\odot\text{F}\) operation, where \(\mathcal{G}(\bullet)\) represents the DropConnect operation. Moreover, to preserve the original feature and, meanwhile, make use of the Bi-LTMA attention mechanism, we employ a shortcut connection to fuse the original feature with the weighted. Mathematically, the output feature of the Bi-LTMA module can be expressed as  
\[\text{F}^{r}=\text{F}+\mathcal{G}(\text{A}\odot\text{F}). \tag{6}\]  
Figure 5: The structure of the bi-directional long-term motion attention module. The TSM module [20] is adopted in the modified inverted residual block to enhance the temporal modeling of the improved EfficientNet-B0 further. As reported in [20], TSM module enables 2D CNNs joint spatial-temporal modeling via shifting part of the channels along the temporal dimension. Moreover, inserting the TSM module into 2D CNNs does not impose an additional computational cost. These merits of TSM are attractive to real-time video understanding. It is also worth noting that in-place TSM may lower the spatial feature learning capability of the backbone model to some extent [20]. Residual TSM is a promising alternative to tackle this problem. Noting that most stages of EfficientNet-B0 are inverted residual blocks, we insert bi-directional TSM modules into residual branches of stages of EfficientNet-B0 to realize temporal feature interaction and meanwhile keep good spatial feature learning capability of the backbone, as shown in Figure 4.  
Given a video clip, the principle of the bi-directional TSM module is demonstrated in Figure 6. As described above, the T three-channel images are obtained from a video clip after frame-grouping.