# Lightweight Violence Detection Model Based on 2D CNN with Bi-Directional Motion Attention
## 3 Proposed Violence Detection Model
### Auxiliary Loss  
We introduce an auxiliary loss to improve the classification capability of the proposed model. As shown in Figure 1, the auxiliary head is inserted in an intermediate stage of the improved EfficientNet-B0. Softmax classifier is employed in both the main branch and the auxiliary branch. In reality, auxiliary heads are used only in the phase of model training and discarded in the phase of inference [42]. Therefore, the auxiliary head does not impose any additional computational cost to the model during inference. In what follows, we introduce the auxiliary loss in detail.  
An important issue is to determine the intermediate layer to which the auxiliary head is attached. In principle, early layers of CNNs learn low-level features while later layers learn high-level features. It is the high-level features in the final layers that are important for classification [45]. As reported in [44], if a single auxiliary head is inserted at  
Figure 6: The principle of TSM module.  
a too-early layer of typical CNNs, the accuracy of the final classifier can be harmed badly. The underlying reason is that the auxiliary classifier attached to an early layer benefits short-term early features but may collapse information required to generate high-quality features in later layers. On the other hand, if the auxiliary head is inserted at a too-deep layer, the auxiliary loss will lose its advantage in combatting the gradient disappearance problem [19]. Based on these observations, we attach the auxiliary head right after stage 7 of the improved EfficientNet-B0, as shown in Figure 7.  
Another issue is to design the auxiliary branch reasonably so as to gain better final classification accuracy. In some past approaches, auxiliary classifiers were connected to hidden layers directly. Experiment results suggested that the final accuracies achieved were hardly improved [43]. As analyzed in [19], discrepancy in optimization directions of the main classifier and the auxiliary ones could degrade the model accuracy. In order to relieve the optimization inconsistency between the auxiliary and main classifiers, the auxiliary branch is designed to resemble the structure of stage 9 of the EfficientNet-B0 as shown in Figure 7. Intuitively, discriminative high-level features can be learned by the auxiliary branch in a way similar to the main branch.  
For a given video clip, the total loss is obtained as shown in Figure 1. The main classifier branch and the auxiliary classifier branch operate in parallel. For each of the T three-channel images \(\mathbf{I}=[\text{I}_{1},\,\text{I}_{2},\,\ldots,\,\text{I}_{\text{T}}]\), the two Softmax classifiers independently give instant scores, based on which the instant cross-entropy losses in both two branches can be calculated. A unique main loss \(\mathcal{L}_{main}\) and an auxiliary loss \(\mathcal{L}_{aux}\) are obtained by performing average pooling over the instant cross-entropy losses. The total loss is formulated as  
\[\mathcal{L}_{total}=\mathcal{L}_{main}+\lambda\mathcal{L}_{aux}, \tag{7}\]  
where \(\lambda\) is a tunable balancing coefficient.