# Abnormal Behavior Detection in Uncrowded Videos with Two-Stream 3D Convolutional Neural Networks
###### Abstract  
The increasing demand for surveillance systems has resulted in an unprecedented rise in the volume of video data being generated daily. The volume and frequency of the generation of video streams make it both impractical as well as inefficient to manually monitor them to keep track of abnormal events as they occur infrequently. To alleviate these difficulties through intelligent surveillance systems, several vision-based methods have appeared in the literature to detect abnormal events or behaviors. In this area, convolutional neural networks (CNNs) have also been frequently applied due to their prevalence in the related domain of general action recognition and classification. Although the existing approaches have achieved high detection rates for specific abnormal behaviors, more inclusive methods are expected. This paper presents a CNN-based approach that efficiently detects and classifies if a video involves the abnormal human behaviors of falling, loitering, and violence within uncrowded scenes. The approach implements a two-stream architecture using two separate 3D CNNs to accept a video and an optical flow stream as input to enhance the prediction performance. After applying transfer learning, the model was trained on a specialized dataset corresponding to each abnormal behavior. The experiments have shown that the proposed approach can detect falling, loitering, and violence with an accuracy of up to 99%, 97%, and 98%, respectively. The model achieved state-of-the-art results and outperformed the existing approaches. convolutional neural network (CNN); action recognition; video analysis; abnormal behavior detection; two-stream convolutional neural networks; 3D convolutional neural networks +
Footnote †: journal:  
the interest of many researchers (e.g., [8; 9; 10; 11; 12; 13]) that have proposed systems to ensure the safety of the elderly and people living alone. Another noteworthy topic in the context of uncrowded scenes is loitering detection. Loitering can be defined as the act of being in a specific public space for a given period with no clear objective. Some examples of the works in this category include [14; 15; 16]. In contrast, when a few people are involved instead of one person in the uncrowded scene, identifying violent behavior, such as fighting, kicking, and punching between two people, becomes intriguing. Numerous attempts have been made (e.g., [17; 18; 19; 20]) to detect violent behaviors in an uncrowded setting. Driven by the growth of deep learning over the past decade, convolutional neural networks have made remarkable achievements in action recognition in videos in general [21; 22; 23; 24] and abnormal behavior recognition [25] in particular. In this area, 2D convolutional neural networks (CNNs) combined with a two-stream network architecture are mostly utilized [26; 27]. The original two-stream architecture combines an RGB stream for spatial information and an optical stream for temporal information to improve the prediction accuracy. Furthermore, 3D spatiotemporal convolutions have also been found efficient for video action recognition [28; 29]. Even though 3D networks require a large number of parameters, their performance has shown significant improvement following the appearance of large-scale video datasets such as Kinetics and MiT [23; 24]. Previously, CNNs have been employed to jointly detect different abnormal behaviors. For example, [PERSON] et al. [30] proposed a method to detect \"illegal cutting\" and \"running\" behaviors in a specific context of electrical industry. Nevertheless, to the best of our knowledge, no previous study has addressed various kinds of abnormal behaviors involving falls, loitering, and violence, which are generally found in an uncrowded scene. Moreover, most of the studies that addressed abnormality detection have not exploited the full potential of deep learning due to their reliance on models trained on relatively smaller video datasets.