# Abnormal Behavior Detection in Uncrowded Videos with Two-Stream 3D Convolutional Neural Networks
## 4 The Proposed Approach
### Model Training and Testing
#### 4.4.2 Finetuning on UFLV-DS Dataset  
For finetuning the model, the classification block comprising the last set of average pooling, dropout, 3D convolutional, reshape, and softmax layers was removed from i3D and the newly initialized classifier layer was retrained on UFLV-DS dataset. During this step, the weights for all previous convolutional layers were kept frozen during iterations 1-30. After the 30 th iteration, the stalled layers were unfrozen to enable the network to  
Figure 5: Implementation details of the model. Note that the architecture of each Conv3d layer shows the number of filters, size of the convolutional kernel, and stride. The architecture of each pooling layer includes the kernel size and stride. make weight adjustments to the trained layers for UFLV-DS. Next, the softmax classifier was trained for the video and optical flow streams separately and then the results from the two streams were fused. Specifically, for the video stream, the model was trained using frame sequences of 64 frames each obtained after the segmentation of videos of the dataset. The size of each clip of the frame sequence was \(224\times 224\). These clips were used as input of the i3D network with 3 channels to learn video features related to abnormal behaviors addressed in this study. At this stage, data augmentation techniques were also applied to the input of the RGB stream as discussed later in this section. The second stream used the optical graphs (based on L = 10 horizontal and vertical optical flow overlay frames in a stack of size 20, following [27]) from the UFLV-DS dataset as input in order to learn motion features specific to falling, loitering, and violence. So, within this stream, i3D with 2 channels takes optical flow frames of size \(224\times 224\) as input. The output layer contains a softmax with four possible outputs representing each of the classes i.e., fall, loitering, violence, and none. Specifically, the new classification block consists of the 3D average pooling layer (i.e., global_avg_pool of (2,7,7) with a stride (1,1,1)) followed by a dropout layer, a 3D convolutional layer (\(1\times 1\times 1\)), and reshape layers. Finally, a softmax layer provides the normalized class probabilities. As regards the fusion technique, there are two main approaches. In the first approach (referred to as early fusion or feature fusion), feature vectors obtained from different streams are merged and passed to one classifier to obtain the final score. In the second approach (referred to as late fusion or score fusion), output probabilities from separate classifiers of each stream are combined to get the final score. Further in this regard, different possibilities have been comprehensively investigated by [PERSON] et al. [53] in terms of \"where to fuse\" and \"how to fuse\". They found that late fusion simply by applying a summation of the features was an effective fusion technique. This work applied the same method and calculated the sum of the weighted score from each stream to get the final prediction. Moreover, while fine-tuning in the UFLV-DS, some data augmentation techniques were applied. First, during training, our system used random cropping both spatially and temporally similar to [31]. Precisely, for the spatial cropping, the system resized the smaller video to 256 pixels and then randomly cropped a \(224\times 224\) patch. In contrast, random temporal cropping was applied while picking the starting frame. On the other hand, during testing, the model took center crops of size \(224\times 224\) and applied the models convolutionally over the entire length of the video. Furthermore, this study tried to deal with the problem of an imbalanced dataset. The main challenge here was to handle a situation of conflicting requirements. On the one hand, since a judicious balance between the positive and negative examples of each class resulted in about 3 times more examples for the \"none\" class (resulting in high bias towards none), it was appropriate to reduce the number of negative examples.