# Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition
## 2 Related Works
### Overview of SL Classification Research Based on Methods  
For the recognition and classification of hand gestures, several machine learning methods have been employed such as principle component analysis (PCA) [4], support vector machine (SVM) [4; 28; 29; 30], K-nearest neighbor (KNN) [3; 28], and linear discriminant analysis (LDA) [4; 31]. Recently, deep learning methods have been employed widely in the literature and we focus on them in this paper. The following methods are commonly used for SL recognition: multilayer perceptron (MLP) [28], convolutional neural network (CNN) [23; 32; 33; 34; 35; 36], recurrent neural network (RNN) [37; 38], long short-term memory (LSTM) [32; 34], and transfer learning-based methods [14]. Table 1 shows the SL classification research studies using various feature extraction techniques coupled with different machine learning and deep learning methods. As can be seen in the table, different research works have employed CNN for SL classification [23; 30; 32; 34; 36]. In ref. [34], researchers developed a real-time sign language recognition system using CNN and a convex hull algorithm to segment the hand region from input images, employing YCbCr skin color segmentation. The CNN model they proposed consisted of an input layer, two convolution layers, pooling layers, a flattening layer, and two dense layers. When tested with real-time data, the system achieved an accuracy of 98.05%. A 3D CNN model was trained from British SL and achieved a 98% accuracy, outperforming state-of-the-art results on British SL [36]. A real-time system was created in ref. [32] to teach SL to beginners, which utilized a technique called skin-cooler modeling to separate the hand region from the background, and CNN was used to classify images. The system achieved a 72.3% accuracy on signed sentences and 89.5% on isolated sign words. In ref. [30], CNN was employed for hand gesture recognition using a modified structure from AlexNet and VGG16 models for feature extraction and SVM for classification of SL numbers and letters. Their two-level models achieved a 99.82% accuracy, better than the state-of-the-art models. Similarly, in ref. [23], CNN and VGG19 were applied for SL classification using single-handed isolated signs, which achieved 99% with VGG19 and 97% with CNN. RNN is a type of neural network that handles sequential data of different lengths. Unlike other neural networks, RNNs are not limited to fixed-size inputs, allowing for the processing of sequences with varying lengths or images of various sizes [28]. To address the issue of long-term dependencies, LSTM is a specific type of RNN. Regular RNNs struggle with predicting words stored in long-term memory and rely on recent information for predictions. However, LSTM networks have a built-in ability to retain information for extended periods. This is achieved through a memory cell as a container for holding information over time, making them highly suitable for tasks such as language translation, speech recognition, and time series forecasting [28]. Research works introduced RNN for SL classification [37; 38], to recognize static signs which attained a 95% accuracy in ref. [37], and to improve a real-time sign recognition system for American sign language, which obtained 99% in ref. [38]. In ref. [32], RNN and LSTM were used to classify SL based on breaking continuous signs into smaller parts and analyzing them with neural networks. This approach eliminated the need to train for different combinations of subunits. The models were tested on 942 signed sentences containing 35 different sign words. The average accuracy achieved was 72% for signed sentences and 89.5% for isolated words.