second lstm layer first layer lstm global context memory softmax classifier initialize refine informativeness gate attention .
Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks