# 3D Skeletal Joints-Based Hand Gesture Spotting and Classification
## 4 Experiments and Results
### Results and Analysis  
* **Hand gesture spotting:** To prove the performance of the proposed hand gesture spotting module, we evaluated the model on two datasets--the NVIDIA Dynamic Hand Gesture dataset and Chalearn LAP ConGD dataset. The frame-wise accuracy metric and edit distance score [37] are used to measure the gesture segmentation performance. The results and comparison with other methods are shown in Tables 2 and 3. From the results shown in these tables, our proposed approach achieved the best performance as compared to other methods in both datasets. Our approach gets higher frame-wise accuracy and edits distance score on NVIDIA Dynamic Hand Gesture dataset and Chalearn LAP ConGD dataset than existing works. The significantly improved experimental results proved the effectiveness of the proposed approach. * **Hand gesture classification:** The performance of our gesture classification module is evaluated by experiments conducted on the 20 BN_Jester dataset (without Depth modality) and NVIDIA Dynamic Hand Gesture dataset. The accuracy comparison with other approaches for isolated dynamic hand gesture classification is shown in Table 3.  
Table 3 shows that our gesture recognition module obtained a positive result. The recognition performance is improved by using 3D data information of key joints. Moreover, the recognition performance of our method is among the top performers of existing approaches, with an accuracy of 95.6% on the 20 BN-Jester Dataset and an accuracy of 82.4% on the NVIDIA Hand Gesture dataset. * **Continuous hand gesture spotting classification:** To entirely evaluate our approach on continuous dynamic hand gesture spotting recognition, we apply the Jaccard index [3] for measuring the performance. For a given gesture video, the Jaccard index estimates the average relative overlap between the ground truth and the predicted sequences of frames. A sequence S is given by \(i\)th class gesture label and binary vector  
Figure 5: Example of sequence frames segmentation by the Bi_LSTM network. The blue line is the given speed input, and the red line is gesture spotting output (a value of 1.0 indicates the gesture frames). ground truth \(G_{s,i}\), while the binary vector prediction for the \(i\)th class is denoted as \(P_{s,i}\). The binary vector \(G_{s,i}\) and \(P_{s,i}\) are vectors with 1-values indicating the corresponding frames in which the \(i\)th gesture class is being performed. So, the Jaccard Index for the given sequence S is computed by the following formula of  
\[J_{s,i}=\frac{G_{s,i}\ \bigcap\ P_{s,i}}{G_{s,i}\ \bigcup\ P_{s,i}} \tag{11}\]  
When \(G_{s,i}\) and \(P_{s,i}\) are empty vectors, the Jaccard Index \(J_{s,i}\) is set as 0. For a given sequence S containing L number of true class labels \(I_{s}\), the Jaccard Index is estimated by the function:  
\[J_{s}=\frac{1}{I_{s}}\sum_{i=1}^{l}J_{s,i} \tag{12}\]  
For all testing sequence of n gestures: \(s\) = {\(s_{1}\), \(s_{2}\),..., \(s_{n}\)} the mean Jaccard Index \(\overline{J_{s}}\)\(J\ \rightarrow_{s}J\ \rightarrow_{s}\) is applied to evaluation as follows:  
\[\overline{J_{s}}=\frac{1}{n}\sum_{j=1}^{n}J_{s,j} \tag{13}\]  
The spotting-recognition performance comparison of our proposed approach to the existing methods by evaluation experiment on the test set of Chalearn LAP ConGD dataset is shown in Table 4. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Mean Jaccard Index** \\ \hline
2S-RNN [4] & 0.5162 \\ Proposed in [3] (RGB + Depth) & 0.5950 \\ R-3 DCNN [5] & 0.5154 \\ Our proposed (3D\_CNN) & 0.5982 \\ Our proposed & **0.6159** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The spotting-recognition performance comparison of our proposed approach to existing methods on the test set of the Chalearn LAP ConGD dataset. Bold values are highest indices.