# Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition
## 5 Experimental Results
### Experimental Results from the Transfer Learning Approach  
Table 7 shows the experimental results from transfer learning for ArSL classification using 15 variants from the following models: VGG, ResNet, MobileNet, Xception, Inception, InceptionResNet, DenseNet, BiT, ViT, and Swin. In these transformers, the top layer of the model, which is responsible for classification was removed and our own classification layer was added on top. The proposed pretrained models, except for MobileNet, were initialized with pretrained weights from the ImageNet dataset, which contains millions of images and thousands of object categories. MobileNet models were initialized with random weights instead of pretrained weights. VGG19 and ResNet were used to provide powerful base \begin{table}
\begin{tabular}{l l c c c c c c c c} \hline \hline  & \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**One-Cycle**} & \multicolumn{3}{c}{**After Training/Fine-Tuning**} & & \\ \cline{3-10}  & & **Accuracy** & **Loss** & **Recall** & **Precision** & **AUC** & **Accuracy** & **Loss** & **F1** \\ \hline \multirow{3}{*}{CNN-1} & CNN-1V1 (10 epochs) & 0.7048 & 1.6866 & 0.1995 & 0.3775 & 0.7975 & 0.2644 & 3.6616 & 0.2612 \\ \cline{2-10}  & CNN-1V2 (10 epochs) & 0.7557 & 1.5515 & 0.1208 & 0.1277 & 0.6039 & 0.1244 & 9.0458 & 0.1158 \\ \hline \multirow{3}{*}{CNN-2} & CNN-2V1 (10 epochs) & 0.7618 & 1.4914 & 0.7578 & 0.9745 & 0.9936 & 0.8890 & 1.1705 & 0.8487 \\ \cline{2-10}  & CNN-2V2 (10 epochs) & 0.8447 & 1.2781 & 0.8929 & 0.9938 & 0.9982 & 0.9556 & 0.9266 & 0.9402 \\ \hline \multirow{3}{*}{CNN-3} & CNN-3V1 (10 epochs) & 0.1778 & 3.1198 & 0.2759 & 0.6394 & 0.9375 & 0.4574 & 2.2542 & 0.3398 \\ \cline{2-10}  & CNN-3V2 (10 epochs) & 0.0433 & 3.7693 & 0.1356 & 0.6222 & 0.9163 & 0.3708 & 2.4147 & 0.1962 \\ \cline{2-10}  & CNN-3V3 (10 epochs) & 0.0298 & 4.9315 & 0.1514 & 0.7491 & 0.9308 & 0.4099 & 2.2581 & 0.2104 \\ \cline{2-10}  & CNN-3V4 (10 epochs) & 0.0696 & 4.2459 & 0.2097 & 0.6267 & 0.9258 & 0.4016 & 2.3162 & 0.2682 \\ \cline{2-10}  & CNN-3 best (30 epochs) & 0.0744 & 3.3914 & 0.3932 & 0.8294 & 0.9748 & 0.6377 & 1.7238 & 0.5033 \\ \cline{2-10}  & CNN-3 best (100 epochs) & 0.0653 & 4.1342 & 0.3819 & 0.8582 & 0.9734 & 0.6323 & 1.7599 & 0.4842 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Ablation study results from deep learning CNN models trained for ArSL classification. Figure 4: Performance of various CNN model architectures used for ArSL classification. models for extracting rich features from input images that can be used to train a classifier. As can be seen in the left-half from the table below, using one-cycle of fine-tuning (i.e., one epoch) showed promising results with some models such as ResNet152, while it showed poor performance with large-sized models such as ViT132. The results after one cycle of fine-tuning the models indicated that the pretrained models and vision transformers required additional training to achieve competitive performance. To further validate the performance of our models, we also fine-tuned them for several epochs and reported their accuracy and loss results. The right-half of Table 7 shows the experimental results after implementing and fine-tuning the transformers for 10 epochs on 32 classes of signs, 48,645 images for training, and 5404 files for validation. Additionally, we used several figures to monitor the performance of the pretrained models and the vision transformers and that of the baseline models. Figure 6 shows the visualization of pretrained model performance values used for ArSL classification including MobileNet, MobileNetV2, Xception, InceptionV3, InceptionResNetV2, and DenseNet169. We noticed that the pretrained models MobileNet, MobileNetV2, and Xception showed robust learning, while InceptionV3, InceptionResNetV2, and DenseNet169 showed instable performance. Based on the results, after fine-tuning the transformers for ArSL classification, the pretrained models model achieved superior accuracy results in the range between 0.88 and 0.96.