# 3D-CNN-Based Fused Feature Maps with LSTM Applied to Action Recognition
## 4 Experiments
### Results and Comparison Analysis
#### 4.3.3 Comparison to the State-Of-The-Art Methods  
In this section, we further verify the effectiveness of our model, and compare our proposed approach to different existing state-of-the-art Human Action Recognition approaches on UCF101 and HDMB51 benchmark datasets. The comparison of results is reported in Table 4. We organize these baseline methods into different categories with respect to the type of features and network being used, including traditional, deep-learned features, very deep-learned features and hybrid features.  
Compared to traditional methods, our model performs the best by 4.5% on both datasets, Compared with RNN-based methods such as (LRCN) [25] and (LSTM) [27], our model outperforms these two methods by 4.3% and 10% on UCF101 datasets respectively. Different experiments indicated that our approach possesses higher discriminative power, even using fewer parameters. It can be also seen that some methods with both features such as TSN [17] and 3D conv--iDT [14] lead to a performance gain by a minimal margin on the UCF101 dataset. We can explain the decrease in prediction rate by fact that this dataset contains action classes with cluttered backgrounds and illumination changes, and TSN is pre-trained on the large-scale ImageNet dataset, which provides large scale size and diversity. Our approach is based on C3D, which is pre-trained on the UCF101 dataset. However, our introduced method outperformed the 3D conv--iDT by 0.9% and the TSN method by 0.7% on the HDMB51 dataset, and showed the highest recognition rate on small-scale datasets. A possible reason for this higher recognition accuracy is that our model is based on a hybrid deep learning model, and the introduction of LSTM temporally works well by capturing the long-term dependencies and boosting the recognition accuracy for complex action categories in the HDMB51 dataset. We can conclude that a combination of LSTM with a 3D convolutional network for the spatiotemporal stream achieves better results and obtains recognition rates of 92.9% and 70.1%  
\begin{table}
\begin{tabular}{c l c c c} \hline \hline \multicolumn{1}{c}{**Modality**} & \multicolumn{1}{c}{**Method**} & **Year** & **UCF101** & **HDMB51** \\ \hline \multirow{4}{*}{**Traditional**} & iDT+fisher vector [10] & 2013 & 84.7 & 57.2 \\  & Ordered Trajectory [36] & 2015 & 72.8 & 47.3 \\  & MPR [37] & 2015 & - & 65.5 \\  & MoFAP [38] & 2016 & 88.3 & 61.7 \\  & Trajectory Rejection [39] & 2016 & 85.7 & 58.9 \\ \hline \multirow{4}{*}{**Deep**} & Two-Stream [16] & 2013 & 88.9 & 59.4 \\  & FSTCN [40] & 2015 & 88.1 & 59.1 \\  & EMV-CNN [41] & 2016 & 86.4 & - \\  & DANN [42] & 2016 & 89.2 & 63.3 \\  & Dynamic Images [11] & 2016 & 89.1 & 65.2 \\  & LTC-CNN [43] & 2018 & 92.7 & 67.2 \\ \hline \multirow{4}{*}{**Very deep**} & C3D [13] & 2015 & 85.2 & - \\  & LSTM [27] & 2015 & 88.6 & - \\  & LRCN [25] & 2015 & 82.9 & - \\  & VideoLSTM [44] & 2016 & 89.2 & 56.4 \\  & 3D Convolution [14] & 2016 & 91.8 & 64.6 \\  & STPP-LSTM [45] & 2017 & 91.6 & 69.0 \\  & FCNs-16 [46] & 2017 & 90.5 & 63.4 \\  & Hidden-Two-stream [47] & 2017 & 90.3 & 58.9 \\  & Multi-LSTM [48] & 2018 & 90.8 & - \\ \hline \multirow{4}{*}{**Hybrid-Model**} & TDD-iDT [19] & 2015 & 91.5 & 65.9 \\  & C3D-iDT [13] & 2015 & 90.4 & - \\ \cline{1-1}  & TSN [17] & 2016 & 94.2 & 69.4 \\ \cline{1-1}  & 3D conv + iDT [14] & 2016 & 93.5 & 69.2 \\ \cline{1-1}  & SCLSTM [49] & 2017 & 84.0 & 55.1 \\ \cline{1-1}  & LTC-iDT [43] & 2018 & 92.7 & 67.2 \\ \hline \multicolumn{1}{c}{**Ours**} & LSTM-3D ConvNet & - & 92.9 & 70.1 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison to the state-of-the-art methods.  
on UCF101 and HDMB51 datasets respectively. This shows that there is a degree of complimentary between LSTM and convolutional neural network.