# SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition
## 3 Stacking of Distilled Vision Transformers (SDViT)
### Stacking
Image samples representing each category in the ASL with digits dataset are depicted in Figure 7.

For the NUS hand gesture dataset [47], it comprises 10 classes and includes 2000 image samples. The 10 classes of hand gestures correspond to the letters A to J. To enhance the diversity in hand gestures, this dataset incorporates the largest number of signers among the three datasets, totaling 40 signers. Each signer performs each class five times. Image samples representing each category in the NUS hand gesture dataset are exhibited in Figure 8.

Figure 6: Illustrations of images representing each category within the ASL dataset. The label of each category is shown at the top of each subfigure. Figure 9 provides a visual representation of image samples from the three datasets, depicting the diverse challenges encountered in static hand gesture recognition. These challenges encompass five key categories: pose, lighting, background, occlusions, and similarity. In the pose challenges, images from the same class exhibit subtle variations in pose due to different signers. This introduces potential confusion for both human experts and machine learning models, as the nuanced differences in signing might lead to misinterpretations. Addressing lighting challenges, the displayed image samples capture hand gestures under suboptimal lighting conditions. Such conditions pose a difficulty in accurately discerning the hand gestures, thereby presenting a challenge for robust recognition. The background challenges showcase image samples against complex backgrounds. This complexity has the potential to mislead machine learning models, as elements of the background may be misconstrued as integral parts of the hand gesture. The occlusion challenge is exemplified by image samples where certain parts of the hand gestures, such as the thumb finger or a significant portion of the gesture, are obscured. This introduces ambiguity and difficulty in the interpretation of the hand gestures. Lastly, the similarity challenge is demonstrated through image samples where subtle differences exist between hand gestures of different classes. For instance, distinctions in the positioning of the thumb differentiate similar gestures, such as numbers and alphabets. These minute differences can be challenging for

Figure 8: Illustrations of images representing each category within the NUS hand gesture dataset. The label of each category is shown at the bottom of each subfigure. Figure 7: Illustrations of images representing each category within the ASL with digits dataset. The label of each category is shown at the bottom of each subfigure. both human experts and machine learning models to accurately discern. The presence of these challenges in static hand gesture recognition underscores the complexity of the task. The potential for confusion or misinterpretation not only poses difficulties for human experts but also demands sophisticated approaches in machine learning models to effectively navigate and overcome these challenges. ### Experimental Setup

To ensure robust experimentation, this paper employs five-fold cross-validation (CV), where the dataset is split into five separate subsets or folds. During each fold, four subsets will be used for training purposes, while the remaining subset will serve as the test set. This process will be repeated, with each subset being used as the test set in turn. By employing five-fold CV, every subset of the dataset will serve as the test set at least once, thereby

Figure 9: Illustrations of images representing each category of challenges found within the three datasets. reducing the risks of biased estimation and overfitting. Therefore, this method is commonly used by researchers in the field [3; 6; 17; 23; 41] as it is reliable for assessing model performance and mitigating the impact of data distribution variations among different subsets. Table 1 illustrates the dataset division into folds and their assignments for training and testing. The dataset is segregated into five folds, numbered from 1 to 5.