# Gesture Recognition Based on 3D Human Pose Estimation and Body Part Segmentation for RGB Data Input
## 4 Experimental Results
### Results  
The performance of the proposed approach for gesture recognition was evaluated by the experiments conducted on four presented datasets: UTD multimodal human action dataset, gaming 3D dataset, NTU RGB+D dataset, and MSRDailyActivity3D dataset. Comparisons of the results with previous approaches are reported in Table 1.  
Table 2 shows the comparison of the proposed network with existing networks on different datasets [7; 8; 19; 34; 35; 36; 37; 38; 39; 40]. The comparison shows that the proposed network outperformed the existing works. Note that the combination of the three subnetworks was better than each single data modality network as shown in Table 3. The color body part segmentation modality produced better performance than the RGB modality due to the removal of background noise. Classification of gestures by important change detection with the spatiotemporal interest point (STIP) [36] also exhibited the effectiveness of the proposed method. The proposed method was outperformed by only [37] in which additional sensor devices like an accelerometer or gyroscope were utilized in addition to the RGB input. These sensors were available only on special circumstances.  
For the NTU RGB+D dataset, to compare with the related model using only the RGB data input, our proposed method achieved outstanding results. While the ST-GCN network [19], which used a spatial-temporal graph convolutional network for a sequence of skeleton graphs, and the TSSI + GLAN + SSAN network [38], which utilized a two-branch attention architecture on skeleton images, obtained the highest accuracy in the literature, our method outperformed as shown in Table 1.  
In the MSRDailyActivity3D dataset, our method outperformed the quality networks for video classification with only RGB video data input such as C3D, T_C3D, or I3D. Comparing with two  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Datasets/Methods** & \begin{tabular}{c} **UTD Multimodal** \\ **Human Action** \\ \end{tabular} & \
\begin{tabular}{c} **Gaming 3D** \\ **RGB+D** \\ \end{tabular} &
\begin{tabular}{c} **NTU** \\ **Activity3D** \\ \end{tabular} \\ \hline C3D [7] & 85.3 & 89.1 & 83.3 & 87.5 \\ LRCN [8] & 83.0 & - & - & - \\ ST-GCN [19] & - & - & 88.3 & - \\ I3D [34] & 90.7 & 93.8 & 85.8 & 88.4 \\ T\_C3D [35] & 89.5 & 90.3 & 85.7 & 88.9 \\ STIP [36] & 70.3 & - & - & - \\ [37] (RGB, Accelerometer, Gyroscope) & 97.6 & - & - & - \\ TSSI + GLAN + SSAN [38] & - & - & 89.1 & - \\ Structure Preserving Projection (RGB+ Depth) [39] & - & - & - & 89.8 \\ ScTPM + CS-Miltp(RGB+ Depth) [40] & - & - & - & 90.6 \\ Proposed method & 96.7 & 95.3 & 90.4 & 90.3 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Gesture classification performance with different datasets (‘-’: not available).  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Accuracy (\%)** \\ \hline
3D\_ResNet (RGB) & 92.1 \\
3D\_ResNet (color body part segmentation) & 94.6 \\ LSTM (3D skeleton joints) & 95.4 \\ Fusion Result & 96.7 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Gesture classification performance in each modality and the fusion result on the UTD multimodal human action dataset.  
state-of-the-art models, the structure preserving projection method [39] and ScTPM + CS-Mltp [40], our method had the same or even a close performance.