# Action Recognition Algorithm of Spatio-Temporal Differential LSTM Based on Feature Enhancement
## 5 Conclusions and Prospect  
Human action recognition has many applications in today's society. Although existing networks can achieve good accuracy, many have limitations in application scenarios. In this paper, the internal structure of the LSTM unit is improved. A ST-D LSTM unit with high accuracy and high reliability is proposed and applied to action recognition. The ST-D LSTM unit updates and transmits action spatial feature change information: the differential operation of the spatial memory state is carried out in the process of transmission, and hence the ST-D LSTM has proportional, integral and differential operations. The ST-D LSTM can satisfy the requirements of rapidity, accuracy, and stability. In the verification experiments, the accuracy of the ST-D LSTM unit is better than that of the ST-LSTM unit in the UCF-101, HMDB-51, and Hollywood2 datasets, and its stability is no less than that of the ST-LSTM unit. However, due to the methods of data reading and transferring in deep learning, the differential calculation leads to a double increase in the amount of data. Therefore, the speed of the ST-D LSTM network cannot be guaranteed, and the amount of parameters needs to be further optimized. Compared with other action recognition algorithms based on deep learning, the ST-D LSTM unit shows good accuracy in the UCF-101 and HMDB-51 datasets. The ST-D LSTM unit is applied to the LRCN network in the experiments. Because the LRCN algorithm extracts features before processing them, the LRCN network applying in the ST-D LSTM unit does not achieve the end-to-end training. In the follow-up research, the ST-D LSTM unit can use convolutional calculations in the internal structure. The ST-D LSTM unit can be applied to other network frameworks to achieve the end-to-end training. Moreover, the ST-D LSTM unit can also be applied to other scenarios, such as attitude estimation, sequence prediction, and so on. Conceptualization, K.H. and F.Z. ; methodology, K.H. ; software, F.Z. ; validation, F.Z., Y.D. and J.J.; formal analysis, F.Z., J.J.; investigation, L.W. ; resources, K.H. ; data curation, K.H. ; writing--original draft preparation, F.Z. ; writing--review and editing, F.Z., L.W. and K.H; visualization, F.Z. ; supervision, K.H. ; project administration, K.H. ; funding acquisition, K.H. All authors have read and agreed to the published version of the manuscript  
The research in this paper is supported by the National Natural Science Foundation of China (42075130), Industry prospect and key core technology key projects of Jiangsu Province (BE2020006-2), the key special project of the National Key R&D Program (2018 YFC1405703), NUST Students' Platform for Innovation and Entrepreneurship Training Program (202010300050Z). I would like to express my heartfelt thanks to the reviewers who submitted valuable revisions to this article. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **UCF-101** & **HMDB-51** \\ \hline Two-stream Convolutional Network [28] & 73.00\% & 40.50\% \\ \hline \multirow{4}{*}{LRCN} & basic LSTM & 71.15\% & 39.99\% \\  & ST-LSTM & 72.73\% & 42.53\% \\ \cline{1-1}  & BiLSTM [22] & 70.00\% & 39.81\% \\ \cline{1-1}  & LSTM+attention & 72.40\% & 41.50\% \\ \cline{1-1}  & ST-D LSTM & 75.70\% & 44.11\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The accuracy comparison of various deep learning algorithms on UCF-101 and HMDB-51 datasets. **Institutional Review Board Statement:** Ethical review and approval were waived for this study, due to the data being provided publicly. **Informed Consent Statement:** Not applicable. **Data Availability Statement:** The code used to support the findings of this study are available from the corresponding author upon request.