# SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition
## 3 Stacking of Distilled Vision Transformers (SDViT)
### Stacking
The incorporation of early stopping in the training phase is evident in these graphs, showcasing the model's adaptive behavior in halting the training process when the loss ceases to decrease. This strategic intervention signifies that the model ceases to enhance its generalization capability. This implementation of early stopping serves a dual purpose. Firstly, it optimizes computational resources by concluding the training process upon reaching an optimal point, eliminating the need for training to a fixed number of epochs. This results in a more efficient use of computational power and time, preventing unnecessary prolongation of the training phase. More critically, the early \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**No. of Trainable Parameters**} & \multicolumn{3}{c}{**Dataset**} \\ \cline{3-6}  & & **ASL** & **ASL with Digits** & **NUS Hand Gesture** & **Average** \\ \hline Stacked ViT & 278,843,904 & 100.00\% & 99.56\% & 100.00\% & 99.85\% \\ \hline SDViT & 269,345,280 & 100.00\% & 99.60\% & 100.00\% & 99.87\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results of stacked ViT compared against the proposed SDViT method on the three benchmark datasets. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Cross-Validation Set** & **Test Accuracy (\%)** & **F-Score** \\ \hline \multirow{8}{*}{ASL} & 1 & 100.00 & 1.00 \\  & 2 & 100.00 & 1.00 \\  & 3 & 99.99 & 1.00 \\  & 4 & 100.00 & 1.00 \\  & 5 & 99.99 & 1.00 \\ \hline \multirow{8}{*}{ASL with digits} & Average & 100.00 & 1.00 \\  & 1 & 99.20 & 1.00 \\  & 2 & 100.00 & 1.00 \\  & 3 & 100.00 & 1.00 \\  & 4 & 99.20 & 1.00 \\  & 5 & 99.60 & 1.00 \\ \hline \multirow{8}{*}{NUS hand gesture} & Average & 99.60 & 1.00 \\ \cline{1-1} \cline{2-4}  & 1 & 100.00 & 1.00 \\ \cline{1-1} \cline{2-4}  & 2 & 100.00 & 1.00 \\ \cline{1-1} \cline{2-4}  & 3 & 100.00 & 1.00 \\ \cline{1-1} \cline{2-4}  & 4 & 100.00 & 1.00 \\ \cline{1-1} \cline{2-4}  & 5 & 100.00 & 1.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results of the proposed SDViT method for each cross-validation set on the three benchmark datasets. stopping mechanism acts as a safeguard against overfitting. By recognizing when the loss on the validation set plateaus or starts to rise, the model prevents itself from becoming overly specialized to the training data. This proactive measure ensures that the model not only performs well on the training set but also exhibits robust generalization to new, unseen data. Figure 13 illustrates examples of misclassified samples from the benchmark datasets, mainly consisting of the four misclassified samples from the ASL with digits dataset. Two samples of the hand gesture for the letter O were misclassified as the number 0, one sample of the hand gesture for the letter V was misclassified as the number 2, and one sample of the

Figure 11: Cost function against epochs graph of the proposed SDViT approach on the ASL with digits dataset. Figure 12: Cost function against epochs graph of the proposed SDViT approach on the NUS hand gesture dataset. Figure 10: Cost function against epochs graph of the proposed SDViT approach on the ASL dataset. number 0 was misclassified as the letter O. The misclassification of the hand gestures for the letter O and the number 0 is due to their similarity, which can also result in misclassification by humans when viewed in isolation from contextual cues. The hand gesture for the letter V being misclassified as the number 2 is due to their close resemblance, with only subtle differences in the positioning of the thumb area, which can also result in misclassification by humans from certain angles. Figures 14, 15, and 16 display the confusion matrices for the ASL, ASL with digits, and NUS hand gesture datasets, respectively. For both the ASL and NUS hand gesture datasets, the confusion matrices reveal that all classes were predicted accurately. Figure 14: Confusion matrix of the proposed SDViT method on the ASL dataset.