# Deep Forest-Based Monocular Visual Sign Language Recognition
## 2 Related Work  
Before the emergence of end-to-end learning, the main steps of visual sign language recognition could be divided into three parts: human skeleton detection, feature extraction, and semantic classification.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Index** & **Dataset** & **Country** & **Number of Signs** & **Sample Number** \\ \hline
1 & DGS Kinect 40 & Germany & 40 & 3000 \\
2 & SIGNUM & Germany & 25 & 33,210 \\
3 & Boston ASLLVD & USA & 3300 & 9800 \\
4 & ASL-LEX & USA & 1000 & 1000 \\
5 & LSA64 signs & Argentina & 64 & 3200 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Public sign language datasets.  
In terms of human skeleton detection, most traditional methods use the color-based skin segmentation model, which detects the human skeleton by the difference between the human's color and the background color [20]. [PERSON] et al. presented a dynamic skin detector based on face color tone and a skin-blob tracking technique for hand segmentation. It has a recognition rate of 97% in a signer-independent mode [21]. In recent years, along with the rise of depth cameras, e.g., Microsoft's Kinect, many researchers have tried to combine depth information with appearance information. It can provide RGB images, depth images, and skeleton data [22]. [PERSON] et al. adopted a 3D hand template with joint angle features with Microsoft's Kinect [23]. [PERSON] et al. used seven vision-based features from the RGB-D images and achieved accuracy results above 80% on average in Brazilian Sign Language [24].  
After obtaining the human skeleton data, it is necessary to extract features from them. There are two types of features: graphical features and interaction information between skeleton. Graphical features involve the Fourier, Zernike moments, the pseudo-Zernike moments, the Hu moments, the complex moments, the Gabor features, and others [25; 26]. [PERSON] and [PERSON] used the Hausdorff distance and Hu invariants to process hand movements in a universal sign language recognition system [27]. Since the depth camera is able to convert human body images into human joint information, the meanings of signs are embedded in the distribution of joints [28]. [PERSON] et al. proposed a characterization of sign language gestures articulated at different body parts as 3D motionlets, which describe the signs with a subset of joint motions [29].  
In the area of semantic classification, HMMs and DTW have been used to classify features since early times [30]. [PERSON] et al. performed the recognition process using an HMM. The results showed the efficiency of the proposed framework, with an accuracy of 83.77% on occluded gestures [31]. However, due to the high efficiency of SVM, many scholars have tended to use SVM and make some improvements on it. [PERSON] combined linear discriminant analysis (LDA) and SVM to form the basis of tenfold classification to recognize sign language symbols. His work ensures 97.3% accuracy on a random sign symbolic dataset of gestural communication [32].  
When neural networks, and especially deep learning methods, arose, the boundaries between feature extraction and semantic classification in SLR became blurred and even disappeared. A neural network can automatically learn and extract classification features from the input images [33]. [PERSON] et al. applied convolutional neural networks (CNNs) in the recognition of 3D motion-captured sign language. The 3D spatiotemporal information of each sign was interpreted using joint angular displacement maps (JADMs), which encoded the sign as a color texture image [16]. Although deep neural networks show strong performance, the performance of neural networks depends on the size and quality of the datasets.