# Modelling a Spatial-Motion Deep Learning Framework to Classify Dynamic Patterns of Videos
## 2 Related Works  
Video classification is a fundamental challenge in multimedia and computer vision communities. Successful state-of-the-art video classification systems rely heavily on multiple discriminative feature representations extracted from videos and, hence, most works have focused on designing robust features [9]. In order to achieve better performance, many existing video classification problems have been considered in the light of advances in the image domain. For instance, two studies [21; 22] extended 2D Harris corner detector and traditional SIFT features into 3D space to obtain the space-time interest points (STIP). The dense trajectory feature-based method reported that densely sample local patches at regular positions in time and space domains outperform the detected STIP points of video classification problems [23]. Furthermore, instead of working with more complex models, the work [24] focused on low-level features with their encoding, using the Fisher vector as an alternative to bag-of-words histograms to aggregate a small set of state-of-the-art, low-level descriptors, in combination with linear classifiers. In addition, several previous studies utilized different fusion techniques to combine the different information features. Most of the abovementioned hand-crafted features have achieved good performance using both appearance and motion information on video classifications; the ability to improve such features may be limited. With the promising results of DL models on image recognition tasks [2; 6; 25], several extensions to process videos have been recently proposed. Among them, one study has extended CNN models into the spatial-temporal domain by training on stacked video frames [4]. Also, one work [26] has compared multiple neural network architectures for action recognition, while [27] proposed a method to learn and compute generic spatial-temporal features more efficiently. Another study [6] proposed an interesting two-stream approach using two CNN networks, which were trained to capture the spatial and motion information using frames and stacked optical flows as input. Its final predictions were obtained by linearly averaging the prediction scores. Another two-stream approach based on traditional dense trajectories has recently reported reliable results [28]. Many kinds of research have been conducted to examine the temporal dynamics of videos. Among them, [29] introduced a hidden Markov model to capture video state changes, considering variable durations, while [30] combined feature patterns with parts in a maximum margin hidden conditional random field framework. Moreover, one study [31] proposed a method to train a linear ranking machine with video frames and then the output parameters used to obtain video-level representations. More recently, RNN has been used in many sequential modeling problems, such as image and video analysis [32; 33; 34], as well as speech recognition [35]. Naturally, videos can be decomposed into spatial and temporal components. Motivated by this factor, [9] proposed a two-stream approach, which breaks down the learning of video representation into separate feature learning of spatial and temporal clues. Apart from that, the study of [36] trained two-stream LSTM networks for action recognition. This study also tried to combine CNN models with LSTMs, but did not obtain a remarkable performance when training only with the LSTM model. However, the work discussed in [9] concatenates the outputs of standalone LSTM models and CNN models using spatiotemporal clues. This video classification model has observed that CNNs and LSTMs are complementary to state-of-the-art methods. In addition, the Long-term Recurrent Convolutional Networks (LRCN) model is capable of extracting spatial and temporal features. Moreover, 3 DCNN, which extracts spatiotemporal features, is superior in such tasks, since it uses the strong point of CNN to classify images and combine them with temporal features.