# On the Use of Deep Learning for Video Classification
## 5 Key Findings  
From the analysis of the existing literature, the following key findings are drawn for video classification task: (i) The visual description works better than the text and the audio description, and the combination of all modalities can contribute to better performance with an increase in computational cost. (ii) The architectures employing CNN/RNN for feature extraction have the ability to perform better than handcrafted features provided that enough data are available for training. (iii) Tensor-Train layer-based RNN such as LSTM and GRU perform better than the plain RNN architectures for video classification. (iv) It is sometimes necessary to use optical flow for datasets such as UCF-101. (v) It is \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Architecture Name** & **Parameters** & **Error Rate** & **Depth** & **Category** & **Year** \\ \hline ResNet & 25.6 M & ImageNet: 3.6 & 152 & & Depth + multi-path & 2016 \\  & 1.7 M & CIFAR-10: 6.43 & 110 & Depth + multi-path & 2016 \\ \hline DelugeNet & 20.2 M & CIFAR-10: 3.76 & & & \\  & & CIFAR-100: 19.02 & 146 & Multi-path & 2016 \\ \hline  & & CIFAR-10: 7.27 & & & \\  & & CIFAR-10 +: 4.60 & & & \\  & & CIFAR-10 ++: 4.59 & 20 & & \\  & & CIFAR-100: 28.20 & 40 & Multi-path & 2016 \\  & & CIFAR-100 +: 22.49 & & & \\  & & CIFAR100 ++: 21.49 & & & \\ \hline WideResNet & 36.5 M & CIFAR-10: 3.89 & 28 & & Width & 2016 \\  & & CIFAR-100: 18.85 & – & & \\ \hline Xception & 22.8 M & ImageNet: 0.055 & 126 & Width & 2017 \\ \hline Residual attention & 8.6 M & CIFAR-10: 3.90 & & & \\ neural network & & CIFAR-100: 20.4 & 452 & Attention & 2017 \\  & & ImageNet: 4.8 & & & \\ \hline ResNeXt & 68.1 M & CIFAR-10: 3.58 & 29 & & \\  & & CIFAR-100: 17.31 & - & Width & 2017 \\  & & ImageNet: 4.4 & 101 & & \\ \hline Squeeze and & & ImageNet: 2.3 & 152 & & Feature-map exploitation & 2017 \\ excitation networks & 27.5 M & ImageNet: 2.3 & 152 & & \\ \hline DenseNet & 25.6 M & CIFAR-10 +: 3.46 & 190 & & \\  & 25.6 M & CIFAR100 +: 17.18 & 190 & & \\  & 15.3 M & CIFAR-10: 5.19 & 250 & & \\  & 15.3 M & CIFAR-100: 19.64 & 250 & & \\ \hline PolyNet & 92 M & ImageNet: Single: 4.25 & – & Width & 2017 \\  & & Multi: 3.45 & – & & \\ \hline PyramidalNet & 116.4 M & ImageNet: 4.7 & 200 & & \\  & 27.0 M & CIFAR-10: 3.48 & 164 & Width & 2017 \\  & & CIFAR-100: 17.01 & 164 & & \\ \hline Convolutional block & & & & & \\ attention Module & (ResNeXt101 & 48.96 M & ImageNet: 5.59 & 101 & Attention & 2018 \\ (32 \(\times\) 4d) + CBAM) & & & & & \\ \hline Concurrent spatial and & & MALC: 0.12 & & & \\ channel excitation mechanism & – & Visceral: 0.09 & – & Attention & 2018 \\ \hline Channel boosted CNN & – & – & – & Channel boosting & 2018 \\ \hline Competitive squeeze and & & & CIFAR-10: 3.58 & 152 & Feature-map exploitation & 2018 \\ excitation network & 36.90 M & CIFAR-100: 18.47 & 152 & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: _Cont._not always helpful to use optical flow, especially for the case of videos taken from the wild, e.g., Sports-1 M. (vi) It is important to use a sophisticated sequence processing architecture such as LSTM to take advantage of optical flow. (vii) LSTMs, when applied on both the optical flow and the image frames, yield the highest performance measure for the Sports-1M benchmark dataset. (viii) Augmenting optical flow and RGB input helps in improving the performance. (ix) Optical flow modality provides complementary information. (x) The high computational requirement of optical flow limits its use in real-time systems. (xi) Multi-stream multi-class fusion can perform better than average fusion, weighted fusion, kernel average fusion, MKL fusion, and logistic regression fusion on datasets such as UCF-101 and CCV. (xii) In 3D group convolutional networks, the volume of channel interactions plays a vital role in achieving a high accuracy. (xiii) The factorization of 3D convolutions by separating spatiotemporal interactions and channel interactions can lead to an improvement in accuracy and a decrease in the computational cost.