# Variable Rate Independently Recurrent Neural Network (IndRNN) for Action Recognition  
[PERSON]  
1 School of Software, Shandong University, Jinan 250101, China; [EMAIL_ADDRESS] (Y.G. ); [EMAIL_ADDRESS] (X.C. )1 School of Software, Shandong University, Jinan 250101, China; [EMAIL_ADDRESS] (Y.G. ); [EMAIL_ADDRESS] (X.C. )1 School of Software, Shandong University, Jinan 250101, China; [EMAIL_ADDRESS] (Y.G. ); [EMAIL_ADDRESS] (X.C. )1 School of Software and Engineering, Shandong University, Jinan 250100, China; [EMAIL_ADDRESS] (S.L. ); [EMAIL_ADDRESS] (H.Y. )1 School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; [EMAIL_ADDRESS] 1  
[PERSON]  
3 State Key Laboratory of Dynamic Testing Technology and School of Information and Communication Engineering, North University of China, Taiyuan 030051, China334 School of Control Science and Engineering, Shandong University, Jinan 250100, China; [EMAIL_ADDRESS] (S.L. ); [EMAIL_ADDRESS] (H.Y. )1 School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; [EMAIL_ADDRESS] 1  
[PERSON]  
4 School of Control Science and Engineering, Shandong University, Jinan 250100, China; [EMAIL_ADDRESS] (S.L. ); [EMAIL_ADDRESS] (H.Y. )1 School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; [EMAIL_ADDRESS] 1  
[PERSON]  
1 School of Software, Shandong University, Jinan 250101, China; [EMAIL_ADDRESS] (Y.G. ); [EMAIL_ADDRESS]. action speeds of the whole dataset. By contrast, in real-world applications, the speeds of one action performed by different people are usually different. Consequently, the models trained with inputs of fixed speed may not generalize well. With samples obtained with different or non-uniform sampling rates and performing speeds between the training data and test data (in practice), the underlying semantic meaning of the observation at a time step is misaligned and, thus, the network cannot perform well. Another problem in training RNNs is the learning with sequences of varying lengths. When one sequence is extracted with different sampling rates, the lengths of the extracted samples are correspondingly different. Therefore, the RNN network needs to process sequences of varying lengths. With the recurrent connections of RNNs, RNNs can be naturally extended to process sequences of different lengths. However, the time steps gradient backpropagated is different when the lengths of the sequences are different. Accordingly, to make RNNs work with sequences of different lengths, the learning rate needs to be adjusted to adapt to the gradient changes. In spite of that, in most of the training strategies, the learning rate is set to be the same for different lengths. There are only few works [8; 9] on using RNNs with different lengths in the literature, which linearly adjusts the learning rate based on the length of the sequence (the longer, the larger). In contrast, in this paper, based on the recently proposed IndRNN [10; 11; 12], the learning rate adjustment can be obtained analytically and we demonstrate that the opposite works (the longer, the smaller), although it is not simply the inverse of a linear function. The contributions of this paper can be summarized as follows. * A variable rate IndRNN model is proposed to solve the problem of varying sampling rates or performing speeds that may exist between different sequences or between training and test (in practice). The model processes inputs of different lengths and extracted with different sampling rates. * An adaptive learning rate adjustment method is proposed based on the lengths of the sequences that the IndRNN network processes. Different adjustment factors are obtained for different layers with different gradient propagation properties. Experiments were conducted on the skeleton-based action recognition task.