# Visual Static Hand Gesture Recognition Using Convolutional Neural Network
## 3 Results
### Results of Using Skin Segmentation and Data Augmentation on the NUS II Dataset  
As a result of using data augmentation, each segmented image in the training dataset is augmented to 10 different images using rotation by an angle that ranges from 20 degrees to the left to 20 degrees to the right. Here in Figure 8, we can see an example of two original images and the 10 augmented images for each image. As we can see in Table 6, using data augmentation has improved the accuracy of our model and decreased the loss of training and validation. The accuracy has been enhanced from 93.5% without using skin segmentation to 96.5% by using data augmentation, which means that our model has improved by nearly 41.6%. Increasing the accuracy by 3% means that six more images have been classified correctly by using data augmentation. The reason behind that is the increase in the size of our training dataset to 10 \(\times\) 10 of its original size and decreasing the overfitting. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Confusion Matrix with an Accuracy of 96.5\% on NUS II Dataset**} \\ \hline  & **A** & **B** & **C** & **D** & **E** & **F** & **G** & **H** & **I** & **J** \\ \hline A & 19 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline B & 0 & 20 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline C & 0 & 0 & 20 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline D & 0 & 0 & 0 & 20 & 0 & 0 & 0 & 0 & 0 \\ \hline E & 0 & 0 & 0 & 0 & 19 & 1 & 0 & 0 & 0 \\ \hline F & 0 & 0 & 0 & 0 & 0 & 19 & 0 & 0 & 0 & 1 \\ \hline G & 0 & 0 & 0 & 0 & 0 & 0 & 20 & 0 & 0 \\ \hline H & 1 & 0 & 0 & 1 & 1 & 0 & 0 & 17 & 0 & 0 \\ \hline I & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 19 & 0 \\ \hline J & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: This table shows the confusion matrix of using skin segmentation and data augmentation with input size of 32 \(\times\) 32 on NUS II dataset. Figure 8: The figure shows an example of data augmentation implemented with two original images and their corresponding augmented images (10 augmented images for each original image). As shown in Table 7, we can see that the model has improved significantly; as shown, some classes (B, C, D, G, J) have improved in such a way that all their images are classified correctly using both data augmentation and skin segmentation. Other classes have only one image misclassified, and all the other 19 images are classified correctly, as we can see in classes (A, E, F, I). We can see here that the class with the least accuracy in all the experiments, which is class (H), has one more image classified correctly for a total of 17 correctly classified images instead of 16 images in the previous experiment. In Figure 9, we can see examples of three misclassified images. In the first image, it is misclassified as class (A). If we look at Figure 2, we can see some similarities between classes (H and A) in the shape of the hand. For example, in both of them, three fingers are folded, but the last two fingers make a difference. In the second misclassified image, the background has the same skin color, which makes it diffuse with the hand shape, making it more difficult for the CNN to classify correctly. In the third image, we can also see that the true and predicted classes (F and J) have a lot of similarities, as in both of them the hand is rotated by 90 degrees in the left direction, as can be seen in Figure 2.  
Results of Using Same Previous Architecture with Skin Segmentation and Data Augmentation on Marcel Dataset  
We have tested our proposed method on a different dataset, which is the Marcel dataset, to see how well it performs compared with state of the art methods. Using our proposed method, we obtained an accuracy of 96.57% with 18 misclassified images out of 554 images. As shown in Table 8, the accuracy obtained is higher than the state-of-the-art result of 76.10% reported on the Marcel dataset in Table 2 and the result of 85.59% reported on the Marcel dataset in [24]. We can see the confusion matrix in Table 9 and