## 4 Results
### Ablation Studies  
**Different fusion ways. ** We compare two fusion methods of RGBSformer. Compared with the simple score fusion method, classification token fusion has greatly improved the results on the fast pathway. Compared with the skeleton stream, score fusion only improves the mean-class accuracy by 0.9%. The classification token fusion method has increased by 3%. Figure 9 shows the improvement and reduction of different fusion methods to the skeleton stream on the 99 action classes. The red bars indicate the improved accuracy of the skeleton flow after fusing the RGB modality. The green bar indicates the reduced accuracy of skeleton flow after fusing the RGB modality. Table 3 shows the result. The classification token fusion method improves the accuracy of more categories than simple fusion of scores rather than reducing the accuracy of the category. Unless otherwise specified, we use the classification token fusion in all experiments in this article. Figure 8: The normalized confusion matrix of the skeleton stream on NTU RGB+D 60 (**left**) and NTU RGB+D 120 (**right**). Figure 9: The improvement and reduction to the skeleton stream by different fusion methods. **Layers of skeleton stream. ** According to [10], the skeleton heatmaps are a mid-level feature for action recognition. To prove this, we change the number of attention layers of the skeleton stream. We first make the \(L^{\prime}=L\), which means the skeleton stream and RGB stream have the same amount of attention layers. Then, we reduce two layers at a time until the \(L^{\prime}=\frac{1}{2}L\). We report the mean-class accuracy and Top-1 accuracy on FineGym99. To simplify the experiment, we sample 16 frames from the heatmaps. Except for the number of layers, all parameters remain the same. Figure 10 shows the final results on the test set. The right subfigure shows the four loss curves of these experiments. It can be seen that more layers do not mean higher accuracy because the features may be \"overfitting\". Therefore, we use \(L^{\prime}=10\) in all other experiments in this paper. **Heatmaps for limbs and joints. ** Referring to [50], GCN-based methods for skeleton action recognition usually assemble results of joint stream and limb stream to obtain better recognition performance. We generate pseudo heatmaps of joints and joint-limbs on the FineGym99 dataset, then feed these two kinds of heatmaps to the skeleton stream only. The examples of the heatmaps are shown in Figure 3. The results are in Table 4. It can be seen that both joint-only heatmaps and joint-limb heatmaps can capture distinguishable features of human actions, and the joint-limb heatmaps can lead to an obvious performance boost. **3D poses vs. 2D poses. ** The NTU RGB+D dataset provides 3D skeletons consisting of 25 points, which are captured by Microsoft Kinect v2 sensors. In our experiments, the skeleton we extracted by adapting HRNet [27] pretrained on COCO-keypoint [44] consisted of 17 points. We generate heatmaps for both sorts of skeletons and conducted the experiments on NTU RGB+D 60 to explore whether different skeleton data sources affect the skeleton stream's performance. We reduced the dimensionality of 3D skeleton points to 2D skeleton points by dividing a 3D skeleton \((x,y,z)\) into three 2D points \((x,y)\), \((y,z)\), \((x,z)\) \begin{table}
\begin{tabular}{c c c} \hline \hline
**Heatmaps** & **Mean-Class Accuracy** & **Top-1 Accuracy** \\ \hline joint-only & 79.2 & 87.2 \\ joint–limb & 83.7 & 88.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The different heatmaps for skeleton-based human action recognition. We report the mean-class and Top-1 accuracy on the FineGym99 dataset. Units are in %. Figure 10: The performance of skeleton stream on FineGym99, while the number of layers is changing (**left**). The loss of the different layers during the training (**right**).