# Variable Rate Independently Recurrent Neural Network (IndRNN) for Action Recognition
## 6 Conclusions  
This paper presented a variable rate IndRNN. It solves the problem of varying sample rates and performing speeds by generating inputs with different sampling rates. Accordingly, the IndRNN network processes inputs of different lengths through different time steps. By processing inputs of variable lengths and sampling rates, the network learns to be more adaptive and robust to samples captured with different setups and actions performed with different speeds. Moreover, with the IndRNN dynamically processing inputs of different lengths, the gradient behavior over different inputs is also changed, and the learning rate needs to be adjusted accordingly. By investigating the gradient backpropagation process under IndRNN, a new learning rate adjustment method was proposed, which adaptively accommodates the gradient differences for different layers. Compared with the existing methods, the proposed method can better handle the actions performed under variable sampling rates and variable speeds. Experiments on two large and widely used skeleton dataset were conducted, and better performance over the existing RNN-based methods wa demonstrated by the proposed method.  
The proposed method takes an approach of increasing training with sequences of different sampling rates and the corresponding learning strategy, but it is also worth investigating how to directly revise or formulate an IndRNN model dynamically with different sampling rates, instead of just training. Moreover, the proposed method only focuses on the temporal processing, while for the spatial processing, it takes the original joint coordinates as input. Therefore, to further improve the overall action recognition performance, better spatial processing methods are worth investigating, such as graph-convolution-based ones.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Cross Sample**} & \multicolumn{2}{c}{**Cross Subject**} \\ \cline{2-5}  & **Precision** & **Recall** & **Precision** & **Recall** \\ \hline HON4D [15] & 84.6\% & 84.1\% & 63.1\% & 59.3\% \\ Dynamic Skeletons [16] & 85.9\% & 85.6\% & 74.5\% & 73.7\% \\ AGNN [24] & 87.6\% & 88.1\% & 84.0\% & 82.0\% \\ P-LSTM [6] & 84.2\% & 84.9\% & 76.3\% & 74.6\% \\ Proposed & 89.20\% & 87.32\% & 80.47\% & 78.44\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results on large scale combined (LSC) dataset.  
**Author Contributions:** Conceptualization, Y.G., C.L., S.L., M.Y. and H.Y.; methodology, Y.G., C.L., S.L. and X.C.; software, Y.G. and C.L.; investigation, Y.G., C.L., S.L., X.C., M.Y. and H.Y.; writing, Y.G., C.L. and S.L. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research was partly funded by National Key R&D Program of China (2018 YFE0203900), the National Natural Science Foundation of China (No. 61901083, 62001092 and 62101512), Fundamental Research Program of Shanxi Province (20210302124031), the open project program of state key laboratory of virtual reality technology and systems, Beihang University (VRLAB2021A01) and SDU QILU Young Scholars program.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.  
**Conflicts of Interest:** The authors declare no conflict of interest.