# Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
## 4 Experiments and Discussion
### Result of Action Recognition
#### 4.2.2 Homemade Datasets  
We also evaluated the proposed algorithm with the others on the homemade datasets. Five-fold cross-validation was used in the experiments, which divided 80% of the data for training and 20% for testing each time. Tables 3 and 4 show the accuracy of the proposed algorithm on the Assembly Gesture Video Dataset and HoloLens ARAAT Dataset, respectively. Tables 5 and 6 present the recognition results and processing time on the homemade datasets compared with SSBoW [45], DSBoW [46], DTBoW [47; 48], and DFW [49]. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Action** & **Pointing (\%)** & **Moving (\%)** & **Grasping (\%)** & **Releasing (\%)** & **Scaling (\%)** & **Rotating (\%)** & **Null (\%)** \\ \hline \multirow{8}{*}{Result} & 93.4 & 96.2 & 91.6 & 91.2 & 91.7 & 90.6 & 95.2 \\  & 94.2 & 95.1 & 92.3 & 91.5 & 90.9 & 91.7 & 96.8 \\ \cline{1-1}  & 93.1 & 97.2 & 91.7 & 92.1 & 92.5 & 90.4 & 95.9 \\ \cline{1-1}  & 93.8 & 95.4 & 91.5 & 91.6 & 93.0 & 91.7 & 94.3 \\ \cline{1-1}  & 94.0 & 96.5 & 93.1 & 92.9 & 92.2 & 92.1 & 97.6 \\ \cline{1-1}  & 93.7 \(\pm\) 0.4 & 96.1 \(\pm\) 0.7 & 92.0 \(\pm\) 0.6 & 91.9 \(\pm\) 0.6 & 92.1 \(\pm\) 0.7 & 91.3 \(\pm\) 0.7 & 96.0 \(\pm\) 1.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The recognition accuracy on HoloLens ARAAT Dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Accuracy** \\ \hline DCCA & 65.5 \(\pm\) 5.5 \\ TCCA & 82.1 \(\pm\) 2.7 \\ Product Manifolds & 88.4 \(\pm\) 3.4 \\ Genetic Programming & 85.0 \(\pm\) 1.1 \\ Tangent Bundles & 91.7 \(\pm\) 0.9 \\ Cov3D & 93.3 \(\pm\) 1.2 \\ Proposed & 96.8 \(\pm\) 0.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The recognition accuracy on Sebastien Marcel Dynamic Hand Posture Dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Accuracy** \\ \hline DCCA & 65.5 \(\pm\) 5.5 \\ TCCA & 82.1 \(\pm\) 2.7 \\ Product Manifolds & 88.4 \(\pm\) 3.4 \\ Genetic Programming & 85.0 \(\pm\) 1.1 \\ Tangent Bundles & 91.7 \(\pm\) 0.9 \\ Cov3D & 93.3 \(\pm\) 1.2 \\ Proposed & 96.8 \(\pm\) 0.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The recognition accuracy on Assembly Gesture Video Dataset. The recognition rates of the proposed algorithm for each action were all over 90%. Outstanding performances of over 96% were achieved for several actions, such as \"Moving\" and \"Null\". The average accuracy of all actions was 93.1% and 93.3% for the Assembly Gesture Video Dataset and HoloLens ARAAT Dataset, respectively. The results show that the proposed algorithm can obtain the highest recognition accuracy across all algorithms from the table. In particular, for \"pointing\" and \"scaling\", the proposed algorithm outperforms all the others by a margin that showcases its advantage in dynamic gesture recognition. Among all the algorithms, SSBoW performed the worst in all action recognitions; this is because each frame from the input video could not be clearly classified to the correct type by SSBoW, which led to an overall low recognition rate. Frame-to-frame recognition results are presented in Figures 1  
To evaluate the performance of the proposed algorithm, we also conducted boundary segmentation experiments on the HoloLens ARAAT dataset. We labeled the ground truth boundary segmentation of the videos in advance. Because the action \"Null\" is the meaningless transition action, we excluded it from the ground truth. We developed a measure to evaluate the segmentation accuracy (SA), expressed as:  
\[SA=\frac{\left|F_{result}\cap F_{groundtruth}\right|}{F_{groundtruth}} \tag{21}\]  
where \(F_{result}\) denotes the frames of the segmentation result and \(F_{groundtruth}\) denotes the segmentation frames of the ground truth. Figure 16 demonstrates the accuracy of each action's segmentation on the HoloLens ARAAT dataset. DFW provided the best recognition results compared to SSBoW, DSBoW, and DTBoW. However, it is outperformed by the proposed algorithm.