# Deep Forest-Based Monocular Visual Sign Language Recognition
## 4 Experiments and Results
The PoF score is defined as  
\[\text{PoF}=\frac{N_{c}}{N_{A}-N_{t}}, \tag{15}\]  
where \(N_{C}\), \(N_{A}\), and \(N_{t}\) represent the number of correctly classified frames, the total number of frames, and the number of no-meaning _none_ frames. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Word/Sentence** & **Prediction1** & **Prediction2** & **Prediction3** & **Prediction4** & **Prediction5** & **Total** & **Classifier** & **PoF** & **RoS** \\ \hline Apple & Apple(40) & - & - & - & - & - & Deep Forest & 1.00 & 1/1 \\  & Apple(34) & Hello(6) & - & - & - & - & SVM & 0.85 & 1/1 \\ Banana & Banana(60) & None(25) & Again(16) & Come(14) & Walk(7) & 158 & Deep Forest & 0.45 & 1/1 \\  & Banana(63) & None(25) & Hamburger(19) & Friend(16) & Come(15) & & SVM & 0.47 & 1/1 \\ Drink(11) & Drink(11) & None(6) & Water(4) & Come(4) & Home(3) & & Deep Forest & 0.48 & 1/2 \\  & Drink(19) & Eat(8) & None(4) & Come(4) & Water(3) & 37 & SVM & 0.27 & 1/2 \\ Father-Walk & Walk(11) & None(8) & Father(4) & Again(4) & Egg(4) & 50 & Deep Forest & 0.36 & 1/2 \\  & Walk(10) & None(8) & Know(8) & Hello(6) & Father(5) & & SVM & 0.24 & 1/2 \\  & Toilet(6) & Home(28) & Hello(19) & Finish(15) & Drink(8) & 124 & Deep Forest & 0.44 & 2/3 \\  & Toilet(61) & Boy(18) & Man(10) & Hearing(7) & Hello(7) & & SVM & 0.49 & 1/3 \\ Walk-Dog-Yesterday & Walk(14) & Milk(9) & Dog(8) & Hello(6) & Other(4) & 53 & Deep Forest & 0.40 & 2/3 \\  & Walk(12) & Egg(8) & Milk(8) & Dog(7) & Hello(6) & & SVM & 0.23 & 1/3 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results for the testing data. Figure 7: Visualized features and extracted vectors from our private testing dataset. The RoS score is defined as  
\[\text{RoS}=\frac{S_{c}}{S_{A}}, \tag{16}\]  
where \(S_{C}\) represents the number of correctly recognized signs and \(S_{A}\) represents the number of signs that should be recognized. A high recognition rate for words with single symbol is shown in Table 4. For example, the word _apple_ has an excellent recognition rate of 100% because we clip the video by reserving only the key frames from the whole video. For the word _banana_, the sign _banana_ has 60 weighted frames, which is far more than that of the M3 sign _again_. After that, double-sign sentences can be recognized within a moderate recognition rate since some other signs are recognized by mistake. In the sentence _drink-water_, the sign _drink_ scores 11 weighted frames, but the sign _water_ only scores 4 weighted frames and the incorrect sign _come_ and _home_ score 4 and 3 weighted frames. A similar result appears in the sentence _father-walk_. In the triple sign sentences, since the transition frames between the key frames be a large proportion of the frames, the meaning of sentences might be confused. In the sentence _he/llo-where-toilet_, the core signs _toilet_ and _hello_ are correctly recognized, although some frames are wrongly recognized as _home_ and _finish_. In next step, we will focus on this issue, which can be handled by neural language process models such as long short-term memory (LSTM) [35; 36]. Overall, the correctly classified frames are dominant, and a promising performance is achieved in the isolated words. However, in terms of multi-word sentences, the desired signs sometimes might be missing or confused with other signs because the transition frames between two signs might be recognized incorrectly, and the ASLLVD dataset provides isolated signs for training. It is still not highly effective for multi-word sentences.