# Visual Static Hand Gesture Recognition Using Convolutional Neural Network
## 3 Results
### Results of Using Skin Segmentation  
As a result of using skin segmentation, a lot of unwanted pixels have been removed from each image, making it easier for the \(CNN\) model to correctly classify images, as we will see later in the confusion matrix of this experiment in Table 5. In Figure 7, we can see an image example from our dataset before and after segmentation. We can see in the figure that the pixels that have the skin color are all present in the segmented image along, with some other pixels that have hue values close to the skin color, but we can see clearly that a lot of pixels in the upper part of the image have been removed. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multicolumn{8}{c}{**Confusion Matrix with an Accuracy of 90.5\% on NUS II Dataset**} \\ \hline  & **A** & **B** & **C** & **D** & **E** & **F** & **G** & **H** & **I** & **J** \\ \hline A & 17 & 0 & 1 & 0 & 0 & 0 & 1 & 1 & 0 & 0 \\ \hline B & 0 & 20 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline C & 0 & 1 & 19 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline D & 0 & 1 & 0 & 19 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline E & 0 & 0 & 0 & 0 & 20 & 0 & 0 & 0 & 0 & 0 \\ \hline F & 0 & 0 & 2 & 0 & 1 & 17 & 0 & 0 & 0 \\ \hline G & 0 & 0 & 0 & 0 & 0 & 0 & 20 & 0 & 0 & 0 \\ \hline H & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 14 & 2 & 0 \\ \hline I & 0 & 0 & 0 & 0 & 1 & 3 & 0 & 0 & 16 & 0 \\ \hline J & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 19 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: This table shows the confusion matrix of using dropout and not using skin segmentation or data augmentation on NUS II dataset. Figure 6: The figure shows the training and validation loss of the \(CNN\) model after adding dropout on NUS II dataset. On the x-axis is the epoch and on the y-axis is the output of the loss function. As we can see in Table 5, using skin segmentation has improved the accuracy of our model and decreased the loss of training and validation loss. The accuracy has improved from 90.5% without using skin segmentation to 93.5% by using skin segmentation. The misclassified images have decreased by 31.6%, from 19 misclassified images to 13 images. The reason behind that is that much unwanted data has been removed from every image, leaving only the pixels that have the skin color values. Due to the complex and challenging background, We can see that some pixels in the background have skin color too, which means that they are included in our image with hands. With a simpler dataset that has a clear background, the improvement in accuracy is much greater, as the image simply consists of only hands. However, we can also say that with these complex backgrounds, an improvement of 31.6% is pretty high and shows that skin segmentation has a very high effect. Now let us take a look at the confusion matrix and see which images have been classified correctly. As shown in Table 5, many classes have improved, and more images are classified correctly than in the first experiment. The image of class (F) has increased from 17 to 20, which means all images of class (F) are predicted correctly after doing skin segmentation. Furthermore, class (J) now has an accuracy of 100% as all its images are predicted correctly. Classes with the lowest accuracy from the first experiment (I, H) have increased their accuracy significantly, as we can see in class (H), which has increased from 14 to 16, which means that two more images are classified correctly. Looking at class (I), we can observe that it has increased from 16 to reach 19, which means that 3 more images are classified correctly.