# Action Recognition Network Based on Local Spatiotemporal Features and Global Temporal Excitation
## 5 Conclusions  
In the paper, we propose a local spatiotemporal extraction module (LSTE) and a channel time excitation module (CTE). The LSTE module first obtains difference features by computing the pixel-wise differences of neighboring video frames within each video segment, and then obtains local motion features by stressing the effect of the feature channels sensitive to difference information. The local motion features are fused with the spatial features to represent the local spatiotemporal information in each segment. The CTE module adaptively excites time-sensitive channels by modeling the interdependencies of channels in terms of time to enhance the global temporal information.  
Furthermore, we embed the LSTE module and the CTE module into the TSM network to build an action recognition network based on local spatiotemporal features and global temporal excitation (LSCT). On the Something-Something V1 and V2 datasets, we perform experiments and contrast the results with those results obtained by advanced action recognition methods. On the Something-Something V1, the accuracy of the LSCT network is 3.7% higher than the baseline method TSM, 2.3% higher than GST, and 1.9% higher than TEI. At the same time, we also conduct ablation studies, and the accuracy of the LSTE module and the CTE module increases by 2.5% and 0.9%, respectively, compared with the baseline method TSM. The results prove the effectiveness of the LSCT network.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Frames** & **Latency (Sec/Video)** & **Top-1 (\%)** \\ \hline TSM & \(8\times 1\times 1\) & 0.016 & 45.6 \\ TSM & \(16\times 1\times 1\) & 0.025 & 47.2 \\ LSCT & \(8\times 1\times 1\) & 0.033 & 49.3 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Analysis of real-time operation.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **FLOPs (G)** & **Top1** & **Top5** \\ \hline TSM [17] & 33 & 45.6 & 74.2 \\ TSM\({}_{40}\)[17] & 165 & 47.6 & 77.9 \\ TEI [19] & 33 & 47.4 & 77.2 \\ TEI\({}_{40}\)[19] & 165 & 49.0 & 79.0 \\ LSCT & 34 & 49.3 & 78.4 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results compared with other temporal modules.  
Although the LSCT network achieved good recognition accuracy, there is still room for improvement. First, the LSCT network utilizes the LSTE module to fully sample video frames to capture temporal information, which may sample redundant frames. Next, we will modify the network for how to extract key frames. Second, the parameters of the LSCT network are slightly higher than those of the TSM. The next step is to modify the network model to improve computing efficiency.  
Conceptualization, D.S., X.W. and S.L.; methodology, S.L. and X.W.; software, S.L.; validation, D.S., X.W. and S.L.; formal analysis, S.L. and X.W.; investigation, S.L.; resources, D.S. and X.W.; data curation, S.L.; writing--original draft preparation, S.L.; writing--review and editing, X.W. and P.Z.; visualization, S.L.; supervision, D.S., X.W. and P.Z.; project administration, D.S. and X.W.; funding acquisition, D.S. and X.W. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the Project of Shandong Provincial Major Scientific and Technological Innovation, grant No. 2019 JZZY010444, No. 2019 TSLLH0315; in part by the Project of 20 Policies of Facilitate Scientific Research in Jinan Colleges, grant No. 2019 GXRC063; and in part by the Natural Science Foundation of Shandong Province of China, grant No. ZR2020 MF138.  
Not applicable.  
The authors declare no conflict of interest.