# Abnormal Behavior Detection in Uncrowded Videos with Two-Stream 3D Convolutional Neural Networks
## 4 The Proposed Approach
### System Architecture and Flow  
When it comes to choosing the architecture for video representation, there is no clear front-runner. Some of the major current video architectures can be classified based on types of kernels (2D or 3D), input (RGB video or optical flow), and the information propagation across frames (LSTMs or feature aggregation over time). Different network architectures have been compared thoroughly in the literature [23; 31]. A brief comparison of a subset of major models based on these classification schemes is shown in Table 1.  
This work adopted the two-stream Inflated 3D architecture (i3D) [31] due to its outstanding performance in video datasets i.e., UCF-101, HMDB-51 [53], and Kinetics [54]. This architecture has outperformed the other models in training and testing on Kinetics with and without ImageNet pretraining [31]. The i3D architecture essentially inflates Inception v1 [35] to use 3D filters and pooling kernels. In this regard, the Inception architecture has two main strengths. First, the model architecture size is much smaller (despite increasing the depth) compared to others such as AlexNet and VGGNet due to the use of global average pooling instead of fully connected layers. The resulting massive savings in memory are important for small real-time applications such as abnormal behavior detection. Second, it uses a concept of a microarchitecture (network in the network) when constructing the macro architecture. More specifically, the Inception module of this architecture serves as a building block that fits into a CNN enabling it to learn convolutional layers with multiple filter sizes that are computed in parallel, and the resulting feature maps are concatenated along the channel dimension. So, the network can effectively learn the local features via smaller convolutions and more abstract features with larger convolutions. This makes the module a multi-level feature extractor. Furthermore, to enhance recognition performance, the model used a combination of video (RGB) and optical flow streams. Two-stream methods of behavior recognition traditionally use the RGB stream and optical stream to extract spatial and temporal data, respectively and then combine the information from both streams to predict behavior. Within each stream, i3D architecture was used that essentially converts the (2D) Inception module and the architecture into a 3D CNN. It inflates all filters and pooling kernels by adding a temporal dimension. So, the N \(\times\) N filters become N \(\times\) N \(\times\) N. The resulting inflated Inception module and the Inflated Inception v1 architecture are shown in Figure 4. In our implementation, the first i3D trains the video stream whereas the second is used to train the optical flow stream. Figure 3: (**a**) Sample frames sequence pertaining to falling from URFDS; (**b**) their corresponding optical flow horizontal displacement; (**c**) optical flow vertical displacement. The model was implemented by using the Keras framework (precisely tf.keras with TensorFlow 2.0). Figure 5 shows a detailed view of our implementation of the architecture. Note that in our implementation, the batch normalization [33] and activation functions follow each 3D convolutional layer except for the last convolutional layer (classification