## 4 Results
### Sign Recognition Accuracies: Neural Networks
#### 4.3.1 Hybrid ANN  
Table 11 lists the accuracies obtained when using the hybrid ANN structure as described in Section 3.2.4--\"_Hybrid ANN_\"--with a batch size of 5 for ISL digits and a batch size of 16 for the Cambridge dataset, the ASL alphabet and the ASL finger-spelling dataset. On the other hand, Figure 10 displays the corresponding accuracy graphs. and Figure 11 displays the corresponding loss graphs. All the accuracies are computed with the best weights over the training epochs.  
From Figures 10 and 11, it can be seen that for ISL digits and the Cambridge dataset, the diagnostic curves indicate a perfect fit with each of them attaining maximum possible accuracy as indicated in Table 11. This is because in the accuracy and loss graphs, the curves for the training and testing sets overlap over the training epochs, indicating a point  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**ISL Digits** & \
\begin{tabular}{c} **Cambridge Hand** \\ **Gesture Dataset** \\ \end{tabular} & **ASL Alphabet** &
\begin{tabular}{c} **ASL Finger-Spelling** \\ **Dataset** \\ \end{tabular} \\ \hline
100.000\% & 99.533\% & 95.368\% & 96.883\% \\ \hline \hline \end{tabular}
\end{table}
Table 11: Hybrid ANN accuracies.  
Figure 11: Hybrid ANN loss curves.  
Figure 10: Hybrid ANN accuracy curves.  
of stability for the deep-learning model. The curves for the ASL alphabet, however, show a little bit of overfitting judging from the small gap between the training and testing plots in both the accuracy and loss curves. Despite this, the small loss and a constant trend in the accuracy and loss curves are indicative of a representative feature set fed to the hybrid ANN as it is properly able to learn in the training stage. On the other hand, the curves for the ASL finger-spelling dataset show degradation after a certain point in its training, i.e., decrease in accuracy and increase in loss. This can be attributed to the particular mini-batch being trained. The ASL finger-spelling dataset contains variations among the same sign gestures owing to user-specific usage. There may be a possibility that for a given mini-batch sample, the current network weights are not able to generalize properly due to these variations. The large gap between the training and testing plots in the loss curve of the ASL finger-spelling dataset also indicate significant overfitting, i.e., a large generalization error on the testing set.  
If one takes a closer look at Table 11, the ISL digits have the highest accuracy since the sign images are taken in a simple backdrop that makes the hand segmentation process much easier and hence able to retain an accurate hand outline in the form of edges for the hybrid ANN. In contrast, the ASL alphabet has the lowest accuracy due to the presence of sign images taken in different lighting conditions which is why the skin mask is unable to retain the entire hand region in certain cases, causing information loss for the hybrid ANN.  
This alleviation is also the reason why the NUS II dataset acquires a greater accuracy of 90.8% in Table 12.  
Overall, for these two datasets, the CNN outperforms the ensemble methods when compared with convolutional-related features since all the convolutional features are being utilized to learn the complex patterns in the aforementioned datasets rather than applying PCA on said features which leads to information loss.