# RotJoint-Based Action Analyzer: A Robust Pose Comparison Pipeline
## 5 Experiment
### TemporalRotNet-Based Action Recognition and Similarity Assessment  
We conducted training and testing of TemporalRotNet on the single-person subset of the NTU-RGB+D [51; 52] dataset. We performed two types of classification tasks, one was a closed-set classification task. In cross-subject evaluation, 40 performers were split into  
Figure 6: The video case study compares two 30 s fitness routines performed by different individuals from distinct perspectives and at varied speeds. The upper section shows a freeze frame of the first video, while the lower section displays the original-speed video from a rear view. After matching, a similarity score of 93.9 is achieved, enabling movement comparison and synchronization. two groups of 20 for training and testing. In cross-view evaluation, camera 1's data were used for testing, and cameras 2 and 3's data were used for training. As shown in Table 4, we observe that the training performance of TemporalRotNet based on RotJoint features is excellent in the classification task, surpassing that of the classical skeleton-based networks like SGN and 2s-AGCN. Furthermore, as shown in Table 5, we conducted a series of ablation experiments by utilizing both 2D and 3D skeleton-based features (provided as ground truth) as inputs. It can be observed that the performance of the skeleton-based networks is suboptimal. Additionally, the confusion matrix shown in Figure 7 reveals that these networks exhibit significant confusion, particularly when dealing with ambiguous actions. As shown in Figure 8, we compared the convergence of three baseline networks for TemporalNet. Among them, TemporalRotNet demonstrated the best convergence, reaching a loss value of approximately 2.600 within 300 epochs. The other task is open-set classification. After excluding multi-person samples, 66,360 motion videos from 70 action classes were used for training, and 22,752 samples from 24 action classes were used for testing. The action classes in the test set did not overlap with those in the training set. During testing, for the classification task, 24 target actions were randomly selected, and the test set, after being processed by TemporalRotNet, was fed into a similarity comparison classifier to achieve a training-free classification task. Other models took the tensor from the layer preceding the final activation layer as the feature. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Feature Base**} & \multicolumn{2}{c}{**NTU RGB+D 60**} & \multicolumn{2}{c}{**NTU RGB+D 120**} \\ \cline{3-6}  & & **Cross-Subject** & **Cross-View** & **Cross-Subject** & **Cross-View** \\ \hline SGN [53] & Skeleton & 89.0\% & 94.5\% & 79.2\% & 81.5\% \\
2s-AGCN [7] & Skeleton & 88.5\% & 95.1\% & 82.9\% & 84.9\% \\ PointNet++ [40] & Point & 80.1\% & 85.1\% & 72.1\% & 79.4\% \\
3 DV-Motion [41] & Voxel & 84.5\% & 95.4\% & 76.9\% & 92.5\% \\
3 DV-PointNet++ [41] & Voxel + Point & 88.8\% & 96.3\% & 82.4\% & 93.5\% \\ PSTNet [54] & Point & 90.5\% & 96.5\% & 87.0\% & 93.8\% \\ \hline TRNet (Ours) & RotJoints & **93.7\%** & **97.2\%** & **90.2\%** & **94.0\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of action recognition methods on NTU RGB+D 60 and NTU RGB+D 120 datasets’ test sets. The bolded values represent the maximum values across all settings. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{
\begin{tabular}{c} **Feature** \\ **Base** \\ \end{tabular} } & \multirow{2}{*}{**Accuracy**} & \multirow{2}{*}{**Precision**} & \multirow{2}{*}{**Recall**} & \multirow{2}{*}{**F1 Score**} & \multirow{2}{*}{**AUC**} \\ \hline Temporal2 DNet & 2D skeleton & 0.81 & 0.75 & 0.78 & 0.76 & 0.84 \\ Temporal3 DNet & 3D skeleton & 0.85 & 0.80 & 0.82 & 0.81 & 0.87 \\ TemporalRotNet & RotJoints & **0.93** & **0.91** & **0.93** & **0.92** & **0.95** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Ablation study of Temporal*Net with different feature bases.