# On the Use of Deep Learning for Video Classification
## 3 Video Classification
### Developments in Video Classification over Time  
The existing approaches for video classification are categorized based on their working principle in Table 2. The trend observed for the classification of videos from the existing literature is that the recently developed state-of-art deep learning models are outperforming the earlier handcrafted classical approaches. This is mainly due to the availability of large-scale video data for learning deep architectures of neural networks. Besides an improvement in classification performance the recently developed models are mostly self-learned and does not require any manual feature engineering. This added advantage makes them more feasible for use in real applications. However, the better performing recently developed architectures are deeper as compared to the previously developed architectures which brings a compromise on the computational complexity of the deep architectures. \begin{table}
\begin{tabular}{c c c} \hline
**Categories** & **Working Principle** & **References** \\ \hline \multirow{6}{*}{Hand-crafted approaches} & These representations are handcrafted & Spatiotemporal Interest Points (STIPs) \\  & and employ various feature encoding & [26], iDT [27], SIFT-3D [28], HOG3D [29], \\ \cline{1-1}  & techniques, such as histograms & Motion Boundary Histogram [30], \\ \cline{1-1}  & and pyramids. & Cuboids [32], Action-Bank [31], 3D SURF [33], Dynamic-Poselets [34]. \\ \hline \end{tabular}
\end{table} 
Table 2: Different categories of approaches of video classification. Among the initially developed hand-crafted representations, improved Dense Trajectories (iDT) [27] is widely considered the state-of-the-art. Whereas, many recent competitive studies demonstrated that hand-crafted features [35; 36; 37; 38], high-level [39; 40], and mid-level [41; 42] video representations have contributed towards the task of video classification with deep neural networks. The hand-crafted models were among the very early developments of video classification problem. Later, 2D-CNNs were proposed for video classification, where image-based CNN models are used to extract frame level features and based on the frame level CNN features, some state-of-art classification models (for example SVM) are learned to classify videos. These 2D-CNN models do not require any manual feature extraction and these models performed better than the competing hand-crafted approaches. After successful development of 2D-CNN models where features are extracted from frame level, the same concept was extended to propose 3D-CNNs to extract features from videos. The proposed 3D-CNNs are computationally more expensive as compared to the 2D-CNN models. However, these models consider the time variations in feature extraction therefore these 3D-CNN models are believed to perform better as compared to 2D-CNN models for video classification [54; 58; 69]. The development of 3D-CNN models paved the way for fully automatic video classification models using different deep learning architectures. Among the developments using deep learning architectures, spatiotemporal convolutional networks are approaches based on integration of temporal and spatial information using convolutional networks to perform video classification. To collect temporal and spatial information, these methods primarily rely on convolution and pooling layers. Stack optical flow is used in two/multi-stream networks methods to identify movements in addition to context frame visuals. Recurrent