## 5 Conclusions  
In this work, we propose RGBSformer, a two-stream pure Transformer-based approach for human action recognition, which can take RGB frames and skeleton heatmaps as input. We achieved the state-of-the-art performance on four widely used action recognition benchmarks. Compared to most skeleton-based action recognition algorithms which use absolute coordinates to represent the skeleton, we use skeleton heatmaps to extract the motion information contained in skeletons. The skeleton heatmaps can avoid the problem of reduced accuracy due to the bias caused by different ways of skeleton acquisition and can use lower computational consumption to solve the action recognition problem in multiplayer scenarios. From a practical application point of view, RGBSformer does not require motion capture systems or other sensors and can perform RGB and skeleton-based bimodal action recognition using only RGB cameras. It provides a framework for Transformer-based bimodal action recognition, which provides a reference for the application of Transformer on human motion recognition tasks. However, there are still two aspects of our framework that can be enhanced, and this is the direction of our research for subsequent work. Firstly, ablation studies show that RGBSformer still has room for improvement in 3D skeleton-based action recognition. In the future, we will explore more ways on Transformer-based frameworks to fuse RGB modality and other modalities such as 3D skeleton, depth, point cloud, and Wi-Fi for human action recognition. Secondly, RGBSformer, like all Transformer-based frameworks, requires a very large number of training samples and a large amount of computational overhead. How to obtain a better model using small samples and low overhead is still a problem. Figure 11: Examples of 2D skeleton consisting of 17 points and 3D skeletons reduced to 2D in dimensionality. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Skeleton Source** & **Top-1 Accuracy** & **Top-5 Accuracy** \\ \hline
2D skeleton & 90.0 & 99.4 \\
3D skeleton & 82.9 & 98.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The results of 2D and 3D skeletons from the skeleton stream only. The 3D skeletons from NTU RGB+D 60 are reduced to 2D points. Units are in %. **Author Contributions:** Conceptualization, J.S. and Y.Z. ; methodology, J.S., Y.Z. and B.X. ; software, J.S., Y.Z. and W.W.; validation, J.S., Y.Z. and W.W.; writing--original draft preparation, J.S. and Y.Z. ; writing--review and editing, J.S., Y.Z., W.W. and B.X. ; visualization, J.S. ; supervision, D.H. and L.C. All authors have read and agreed to the published version of the manuscript. **Funding:** This paper was funded in part by the National Natural Science Foundation of China under Grant 62072319, 62262074; the Sichuan Science and Technology Program under 2022 YFG0041 and 2022 YFG0159; the Luzhou Science and Technology Innovation R&D Program (No. 2022 CDLZ-6). **Institutional Review Board Statement:** Not applicable. **Informed Consent Statement:** Not applicable. **Data Availability Statement:** The FineGym99 dataset can be found in [[https://sdolivia.github.io/FineGym/](https://sdolivia.github.io/FineGym/)]([https://sdolivia.github.io/FineGym/](https://sdolivia.github.io/FineGym/)), accessed on 1 November 2022, the raw RGB videos can be downloaded from [[https://utexas.box.com/s/oq2](https://utexas.box.com/s/oq2) ce7i1 mdgoihdvy9o7 moolxxxsib42]([https://utexas.box.com/s/oq2](https://utexas.box.com/s/oq2) ce7i1 mdgoihdvy9o7 moolxxxsib42), accessed on 1 November 2022, (applied by the authors [[https://github.com/SDOlivia/FineGym/issues/11](https://github.com/SDOlivia/FineGym/issues/11)]([https://github.com/SDOlivia/FineGym/issues/11](https://github.com/SDOlivia/FineGym/issues/11)), accessed on 1 November 2022).