# Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition
## 5 Experimental Results
### Experimental Results from the Transfer Learning Approach
The InceptionResNetV2 model attained the highest accuracy of 0.9817 followed by MobileNet variants which attained an accuracy of 0.96. DenseNet169 achieved the lowest accuracy of 0.5587. Notably, the deep learning CNN-based architecture which was trained from scratch achieved an accuracy of 0.6323, which was low compared to using transfer learning with pretrained models. However, the CNN-3 experimentally chosen structure, the randomly initialed weights, and the long-time of training could change these results. Figure 5: Performance of the CNN-3’s best results using a different number of epochs. Hence, we believe that the effectiveness of using transfer learning is highly promising for ArSL recognition. \begin{table}
\begin{tabular}{p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt}} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Variants**} & \multicolumn{2}{c}{**One-Cycle**} & \multicolumn{2}{c}{**After Training/Fine-Tuning**} & & & \\ \cline{3-10}  & & & **Accuracy Loss** & **Recall** & **Precision AUC** & **Accuracy Loss** & **F1** \\ \hline Deep Learning & CNN-3 & - & 0.0653 & 4.1342 & 0.3819 & 0.8582 & 0.9734 & 0.6323 & 1.7599 & 0.4842 \\ \hline \multirow{7}{*}{Transfer Learning} & MobileNet & 0.9467 & 0.9608 & 0.9145 & 0.9912 & 0.9992 & 0.9641 & 0.8794 & 0.9500 \\ \cline{3-10}  & MobileNet & 0.9467 & 0.9624 & 0.9169 & 0.9898 & 0.9993 & 0.9665 & 0.8851 & 0.9510 \\ \cline{3-10}  & Xception & Xception & 0.0365 & 3.4626 & 0.8731 & 0.9778 & 0.9967 & 0.9408 & 0.8821 & 0.9205 \\ \cline{3-10}  & Inception & InceptionV3 & 0.8542 & 1.1860 & 0.7661 & 0.9530 & 0.9953 & 0.8851 & 1.0381 & 0.8420 \\ \cline{3-10}  & InceptionResNetV2 & 0.7913 & 7.9896 & 0.9700 & 0.9900 & 0.9995 & 0.9817 & 0.7415 & 0.9800 \\ \cline{3-10}  & DenseNet & DenseNet169 & 0.6743 & 1.6610 & 0.2889 & 0.8711 & 0.9276 & 0.5587 & 2.0299 & 0.3888 \\ \cline{3-10}  & BiT-m-r50x1 & 0.8690 & 0.4054 & 0.9128 & 0.6505 & 0.9582 & 0.9508 & 0.1901 & 0.7749 \\ \cline{3-10}  & BiT-m-r50x3 & 0.8784 & 0.4404 & 0.9510 & 0.4009 & 0.9570 & 0.9004 & 0.5046 & 0.6185 \\ \cline{3-10}  & ViT & ViT\_b16 & 0.8919 & 3.4436 & 0.8414 & 0.9519 & 0.9961 & 0.8919 & 0.9930 & 0.8925 \\ \cline{3-10}  & ViT & ViT\_132 & 0.0883 & 3.2602 & 0.8457 & 0.9483 & 0.9934 & 0.8868 & 1.0146 & 0.8928 \\ \cline{3-10}  & Swin & SwinV2 Tiny256 & 0.8769 & 1.0149 & 0.9713 & 0.9820 & 0.9993 & 0.9765 & 0.7284 & 0.9765 \\ \hline \multirow{7}{*}{Baselines} & VGG & VGG16 [39] & 0.8220 & 15.205 & 0.8621 & 0.8629 & 0.9413 & 0.8625 & 16.906 & 0.8641 \\ \cline{3-10}  & VGG19 & 0.8312 & 15.084 & 0.8558 & 0.8566 & 0.9380 & 0.8558 & 17.744 & 0.8558 \\ \cline{2-10}  & ResNet & ResNet50 & 0.9589 & 0.8674 & 0.9719 & 0.9904 & 0.9991 & 0.9824 & 0.7254 & 0.9808 \\ \cline{3-10 Figure 6: Performance of pretrained deep learning models used for ArSL classification. Additionally, our findings highlight the importance of selecting appropriate model architectures and hyperparameters to achieve the best possible performance for ArSL tasks. Figure 7 shows the performance of the BiT (Big Transfer) model used for ArSL classification. As can be seen, different variants of BiT may result in different performance values. The accuracy of BiT was above 90%. As another example, different variants of the vision transformers achieved different results as shown in Figure 8. ViT and Swin transformer models achieved great performance with stable learning over time. They achieved an 88-97% accuracy after fine-tuning for classification. As a comparison with other baseline works in ArSL in refs. [39; 40], we retrained the VGG16 VGG19, ResNet50, and ResNet152, which attained an 85-98% of accuracy when applied to our dataset. Our findings highlight that these baselines initiated the research direction of using pretrained models and transfer learning for ArSL classification and recognition, and our results from 11 variants of other state-of-the-art models confirmed it as a successful direction.