# Pyramid Spatial-Temporal Graph Transformer for Skeleton-Based Action Recognition
## 4 Experiments
### Comparisons with Other Approaches
#### 4.4.1 Experiments on the NTU-RGBD 60 Dataset  
The performance of our PGT was evaluated on both the CS and CV benchmarks as in [40]. As shown in Table 5, we compared our PGT with four kinds of methods: handcrafted features, RNN-based methods, CNN-based methods and GCN-based methods. GCN methods are generally the newest and most effective, and our work combined the architecture of graph convolution in GCNs with the self-attention in transformers to fuse local and global information. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Graph Embedding** & **Acc. (\%)** & **FLOPs (G)** & **Param. (M)** \\ \hline Basic & 89.1 & 4.32 & 13.12 \\ \hline Separate (PGT) & 90.9 & 4.01 & 12.36 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The ablation study on the NTU-RGBD 60 CS benchmark, denoting the effect of separate convolution in graph embedding. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Input Branch** & **Acc.(\%)** & **FLOPs(G)** & **Param.(M)** \\ \hline Joint & 87.3 & 1.00 & 3.09 \\ \hline Bone & 87.4 & 1.00 & 3.09 \\ \hline Joint Motion & 85.3 & 1.00 & 3.09 \\ \hline Bone Motion & 84.8 & 1.00 & 3.09 \\ \hline
4-s Fusion (PGT) & 90.9 & 4.01 & 12.36 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The ablation study on the NTU-RGBD 60 CS benchmark denoting the effect of the four-stream fusion strategy. When compared with the RNN-based and CNN-based methods, our PGT demonstrated superiority in graph convolution operations, which is more consistent with the data structures of skeletons. The accuracy increased by 6.7% or 6.2% when compared with the fuzzy-CNN [18], which extracted four complementary feature vectors by a fuzzy integral to capture different spatiotemporal dynamics. When compared with the RNN-based DS-LSTM [21], our model greatly improved the accuracy with a more rational structure instead of focusing on learning the temporal information with LSTM. As for most GCN-based methods, our PGT showed better performance with the proposed pyramid network consisting of graph embedding and self-attention modules. Additionally, based on the four-stream converged network architecture, the accuracy of our method was 0.9% and 1.7% higher than the MS-AAGCN [28] on both CS and CV, respectively. Compared with the outstanding shift-GCN [12], our method still performed better on the CS benchmark, which meant a more robust representation of various actions performed by different persons. Compared with the DC-GCN [30], our method showed a comparable performance. The DC-GCN adopted the decoupling aggregation mechanism in CNNs and a more effective dropout design (ADG) for graphs. It was noticed that the efficient-GCN [15] performed better than ours on the CS benchmark. However, our PGT still showed a comparable ability by combining graph convolution with the transformer attention mechanism. Most GCN-based methods follow the backbone of the ST-GCN [10] and make efforts for a more robust representation, a more efficient architecture, a more dynamic topology, etc. However, our PGT introduced a novel backbone which showed excellent potential by taking advantage of both GCNs and transformers. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Methods** & **CS (\%)** & **CV (\%)** & **Year** \\ \hline Lie Group [1] & 50.1 & 82.8 & 2014 \\ \hline STA-LSTM [7] & 73.4 & 81.2 & 2017 \\ \hline DS-LSTM [21] & 75.5 & 84.2 & 2020 \\ \hline Fuzzy-CNN [18] & 84.2 & 89.7 & 2021 \\ \hline SEMN [19] & 80.2 & 85.8 & 2021 \\ \hline ST-GCN [10] & 81.5 & 88.3 & 2018 \\ \hline AS-GCN [25] & 86.8 & 94.2 & 2019 \\ \hline
2 s AGCN [11] & 88.5 & 95.1 & 2019 \\ \hline TS-SAN [44] & 87.2 & 92.7 & 2020 \\ \hline MS-AAGCN [28] & 90.0 & 96.2 & 2020 \\ \hline
4 s Shift-GCN [12] & 90.7 & 96.5 & 2020 \\ \hline DC-GCN+ADG [30] & 90.8 & 96.6 & 2020 \\ \hline AMV-GCN [45] & 83.9 & 92.2 & 2021 \\ \hline
3 s RA-GCN [13] & 87.3 & 93.6 & 2021 \\ \hline ST-TR-AGCN [14] & 89.2 & 95.8 & 2021 \\ \hline Efficient-GCN [15] & 91.7 & 95.7 & 2021 \\ \hline PGT (Ours) & 90.9 & 95.9 & 2022 \\ \hline \hline \end{tabular}
\end{table}