# Modelling a Spatial-Motion Deep Learning Framework to Classify Dynamic Patterns of Videos
## 5 Experimental Evaluation
### Performance of Dual-CNNSELSTM with Dual-3 DCNNLSTM Model  
In study, we focused on evaluating the performances of the baseline _Dual-3 DCNNLSTM_ model and the proposed Dual-CNNSELSTM model using the three experimental datasets discussed in Section 4. While training the models, we noticed the computational times for both models with three training datasets. Our proposed Dual-CNNSELSTM architecture took 4.8 h, 5.3 h, and 18.7 h to learn the parameters for fireworks, hand gestures, and HMDB51, respectively, while the Dual-3 DCNNLSTM learned similar datasets within 4.2 h, 4.6 h, and 16.5 h, respectively. Both models' average testing accuracies with the three datasets are illustrated in Table 1. The results demonstrated that the Dual-3 DCNNLSTM model had the best classification accuracy of 77.31% with the fireworks dataset. However, when we trained the Dual-CNNSELSTM model, the fireworks dataset had the second-highest performance at 79.87%, but that was higher than with Dual-3 DCNNLSTM. Apart from that, our Dual-CNNSELSTM model had the best classification results with all three datasets compared to the baseline model. Figure 5: Performance (%) comparison of single-stream network and two-stream network architectures with the proposed Dual-CNNSELSTM model. The hand gesture dataset with 11 dynamic classes achieved a significant testing accuracy of 81.62% with the proposed Dual-CNNSELSTM model. The accuracy and loss of the model during training is shown in Figure 6. The model reached training and validation accuracies of 83.81% and 74.09%, while the loss was down to 0.38%% and 0.59% for training and validation, respectively. For better understanding, the training accuracies of all three experimental datasets are presented in Figure 7. The time it takes to generalize the model also differs in different datasets with different classes and samples. Thus, the number of epochs may vary from dataset to dataset. Since the HMD51 dataset generalizes around the 100 th epoch to maintain the fairness of the experimental results, we limit the number of epochs in Figure 7 to 105. Moreover, we notice that the HMDB51 dataset obtained a considerably low accuracy with the two tested models. We believe one reason is the low quality of the videos compared with the other two datasets. In addition, both the fireworks and hand gesture datasets' videos have static backgrounds rather than the dynamic backgrounds of HMDB51's videos. Moreover, the dissimilar behavior in Figures 6 and 7 may be because of the different number of classes and different dataset sizes. Our fireworks dataset consists of eight classes with 1500 sample videos, while the hand gesture dataset contains 11 classes with 2100 videos. However, HMDB51 has 51 action classes with 6766 samples. When a dataset has a higher number of classes and samples, in some cases, it is hard to learn the model. Hence, it achieves a lower accuracy than the datasets with a small number of classes. Furthermore, the firework and hand gesture datasets have static backgrounds and no other moving objects except the target objects. Hence, in most of the cases, the objects are clearly visualized, and CNN can extract the features well. However, the videos of HMDB51 contain many variations such as human actions, dynamic and complicated backgrounds, as well as a lot of unnecessary data or noise. These drawbacks make it difficult for the model to learn and extract features and may cause the lower accuracy in the HMDB51 dataset. Moreover, in this stage, it is hard for either the fireworks or the hand gesture dataset to achieve higher performance than this due to some similar movements, objects, and shapes within several classes. Consequently, the results imply that the proposed model is most appropriate with static background dynamic patterns classifications. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Fireworks** & **Hand Gestures** & **HMDB51** \\ \hline Dual-3 DCNNLSTM & 77.31\% & 73.12\% & 67.84\% \\ Dual-CNNSELSTM & 79.87\% & 81.62\% & 69.86\% \\ \hline \hline \end{tabular}
\end{table}