# Modelling a Spatial-Motion Deep Learning Framework to Classify Dynamic Patterns of Videos
## 1 Introduction  
Dynamic pattern classification of videos serves as an essential step in the process of analyzing video contents [1]. Automatic classification of videos is crucial for monitoring, indexing, and retrieval purposes. Recently, there is growing interest in developing deep learning architectures for dynamic pattern classification. However, to adopt deep learning perfectly is still considered as a unique challenge in computer vision due to the appearance and motion features of video data. A combination of appearance and motion features attempts should produce a better-performing model. Therefore, a proper deep learning model should be able to integrate both static appearances in single frames and temporal relation between consecutive frames in order to achieve excellent classification accuracy. Among the extensive video classification methods in the literature, the majority deal with the same pattern or physical action with different contexts or backgrounds. More specifically, video frames of a particular video consist of the same object, human activity, face, gesture, or scene. Beyond these general classifications, this study intends to classify the displays of artificial fireworks using existing fireworks videos to validate the proposed Dual-CNNSELSTM model in addition to the hand gestures and human actions videos. The proposed two-stream deep learning architecture interacts through thelong-term temporal feature extraction phase to input the information additionally learned from the consecutive time steps. Currently, both single and two-stream neural net architectures achieve remarkable success on video pattern classification and prediction applications. In practice, the convolutional neural networks (CNN) are very successful at image analysis tasks [2; 3]. Because of the high complexity of video data, most video classification based on deep learning models [4; 5; 6] demonstrated the same or similar performance to handcrafted features [7]. Apart from the static appearance features of images, video data additionally contain temporal motion features. Some recent studies [8; 9] used recurrent neural networks (RNN) to model long-term temporal information, and thus achieved acceptable performance. Motivated by such models, currently, researchers adopt the long short-term memory (LSTM) model to solve the problem of long and complicated sequences of dynamic gesture problems. Moreover, LSTM has become an essential part of deep learning models for image sequence modeling [10; 11]. Furthermore, the enhanced methods were proven to recognize human actions and gestures [12; 13; 14]. Practically, spatial and temporal features are needed to classify video data. For human action recognition, previous studies always considered similar ideas to those of image recognition. Unlike static images, dynamic patterns of videos consist of ever-changing motions with different target objects that have various appearances against different backgrounds. Thus, it is crucial to explore distinct spatiotemporal features to classify such dynamic patterns. Moreover, the work of [PERSON] et al. [15] proposed a combination of 3 DCNN and RNN, with the fully connected spatiotemporal features transferred into RNN. Inspired by this study, [16] proposed a combination of a 3 DCNN and LSTM model to classify types of hand gestures. However, prior studies did not address well the spatial relationship between consecutive time steps. Although some previous works based on video classifications presented good models [17; 18], we doubt whether they could satisfy the specific requirements, specifically for fireworks videos. In particular, even with the same firework type, the sample videos involve different object patterns in most time steps, as compared to previous models of other video classification tasks, such as human actions, scenes, objects, and faces.