# Multi-Grained Temporal Clip Transformer for Skeleton-Based Human Activity Recognition
## 5 Conclusions  
To model inter-sub-action relationships and enable skeleton joint-level motion interaction, we proposed a multi-grained temporal clip transformer (MTC-Former) for skeleton-based activity recognition. The proposed MTCA can effectively learn multi-grained short-term sub-action motion information, where domain-specific priors on short-range motion patterns are embedded to enhance correlation modeling. The MSTI module can model long-term motion dependencies, correlations of short action clips, and skeletal motion interactions between human skeleton joints. Experimental results on NTU RGB+D, NTU RGB+D 120, and InHARD datasets demonstrate that the MTC-Former achieves state-of-the-art performance, achieving improved recognition performance compared to fixed transformer-based methods. Future work involves exploring the fusion of integrating RGB modality to compensate for the lack of fine-grained color and texture information for multimodal HAR in complex scene-understanding tasks. Conceptualization, C.L. and P.Z. ; Data curation, P.Z. and S.J. ; Formal analysis, P.Z. ; Funding acquisition, C.L. ; Investigation, P.Z., S.J. and Y.L. ; Methodology, P.Z. and C.L. ; Project administration, C.L. ; Resources, C.L. ; Software, P.Z. and Y.L. ; Supervision, C.L. ; Validation, C.L. and P.Z. ; Visualization, P.Z., S.J. and Y.L. ; Writing--original draft preparation, P.Z. ; \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Categories**} & \multirow{2}{*}{**Methods**} & \multirow{2}{*}{**Prams(M)**} & \multicolumn{2}{c}{**NTU RGB+D**} & \multicolumn{2}{c}{**NTU RGB+D 120**} \\ \cline{3-8}  & & & **X-Sub (\%)** & **X-View (\%)** & **X-Sub (\%)** & **X-Set (\%)** \\ \hline \multirow{2}{*}{CNN} & RotClips+MTCNN [43] & - & 81.1 & 87.4 & 62.2 & 61.8 \\  & Ta-CNN+ [44] & 1.06 & 90.7 & 95.1 & 85.7 & 87.3 \\ \hline \multirow{10}{*}{GCN} & ST-GCN [10] & 3.1 & 81.5 & 88.3 & 70.7 & 73.2 \\  & AS-GCN [47] & - & 86.8 & 94.2 & 78.3 & 79.8 \\  & 2s-AGCN [32] & 3.5 & 88.5 & 95.1 & 79.2 & 81.5 \\  & Shift-GCN [48] & & 90.7 & 96.5 & 85.9 & 87.6 \\  & MS-G3D [11] & 2.8 & 91.5 & 96.2 & 86.9 & 88.4 \\  & MSS-GCN [13] & - & 92.7 & 96.9 & 88.9 & 90.6 \\  & SPIA-Net [49] & 3.2 & 92.8 & 96.8 & 89.2 & 90.4 \\  & FR-Head [46] & 2.0 & 92.8 & 96.8 & 89.5 & 90.9 \\  & LG-SGNet [14] & 1.73 & 93.1 & 96.7 & 89.4 & 91.0 \\ \hline \multirow{10}{*}{Transformer} & ST-TR [16] & 12.1 & 89.9 & 96.1 & 82.7 & 84.7 \\  & STSA-Net [18] & 1.9 & 92.7 & 96.7 & 88.5 & 90.7 \\ \cline{1-1}  & TranSkeleton [19] & 2.2 & 92.8 & 97.0 & 89.4 & 90.5 \\ \cline{1-1}  & LG-STFormer [22] & 10.1 & 92.8 & 96.7 & 88.6 & 90.4 \\ \hline MLP & SiT-MLP [45] & 0.6 & 92.3 & 96.8 & 89.0 & 90.5 \\ \hline \multicolumn{1}{c}{} & MTC-Former (ours) & 2.5 & **93.2** & **97.1** & 89.7 & **91.4** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison results of experiment on the NTU RGB+D and NTU RGB+D 120 datasets. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **Methods** & **Modality** & **Params (M)** & **Top-1 (\%)** \\ \hline \multirow{3}{*}{CNN} & TSN [50] & RGB & 25.5 & 84.4 \\  & TSM [51] & RGB & 25.5 & 88.1 \\  & I3D [52] & RGB & 12.3 & 82.1 \\  & Non Local [53] & RGB & 30.25 & 86.3 \\ \hline \multirow{2}{*}{Transformer} & Timesformer [54] & RGB & 121 & 85.2 \\  & VideoSwin [55] & RGB & 28.2 & 87.6 \\ \hline \multirow{2}{*}{-} & MTC-Former (ours) & Skeleton & 2.5 & **88.3** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison results on the InHARD datasets. Writing--review and editing, C.L., S.J. and Y.L. ; All authors have read and agreed to the published version of the manuscript. This research was founded in part by the National Natural Science Foundation of China under Grant 62176086, in part by the Science and Technology Development of Henan Province under Grants 242102211055 and 242102210032, and by the Soft Science Research Program of Henan Province under Grant 252400410618. Not applicable. **Data Availability Statement**: Publicly available datasets were analyzed in this study.