# Deep Forest-Based Monocular Visual Sign Language Recognition
## 4 Experiments and Results
The minimum precision, recall, and F1 scores fpr deep forest are 0.82, 0.79, and 0.86, respectively. This shows that the deep forest classifier has a better performance than the SVM classifier. In the private testing dataset, 11 signs chosen to make up 6 daily words/sentences are illustrated in Figure 5.1 We use a monocular camera to capture sign language videos of two people in two scenes: an office and a corridor with a black background. Six video clips are captured, containing 37 to 158 frames each. Figure 7 shows the visualized features and extracted vectors from the private testing dataset. All signs are picked from the 104 signs of the ASLIVD. The sign _none_, which has no meaning, is used to label the beginning and the end of a clip. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**SVM**} & \multicolumn{5}{c}{**Deep Forest**} \\ \hline
**Sign** & **Number** & **Precision** & **Recall** & **F1** & **Sign** & **Number** & **Precision** & **Recall** & **F1** \\ \hline None & 396 & 1.00 & 1.00 & 1.00 & None & 396 & 1.00 & 1.00 & 1.00 \\ Same & 124 & 0.91 & 0.94 & 0.93 & Same & 124 & 0.98 & 0.99 & 0.99 \\ Walk & 109 & 0.77 & 0.95 & 0.85 & Walk & 109 & 0.93 & 0.98 & 0.96 \\ Man & 108 & 0.54 & 0.72 & 0.62 & Man & 108 & 0.95 & 0.89 & 0.92 \\ Happy & 107 & 0.90 & 0.97 & 0.94 & Happy & 107 & 0.98 & 0.99 & 0.99 \\ Excuse & 102 & 1.00 & 1.00 & 1.00 & Excuse & 102 & 0.96 & 0.98 & 0.97 \\ Run & 97 & 0.97 & 0.99 & 0.98 & Run & 97 & 0.96 & 0.99 & 0.97 \\ Workout & 94 & 0.95 & 1.00 & 0.97 & Workout & 94 & 1.00 & 1.00 & 1.00 \\ Again & 89 & 0.92 & 0.93 & 0.93 & Again & 89 & 0.92 & 0.93 & 0.93 \\ Live & 84 & 0.93 & 0.92 & 0.92 & Live & 84 & 1.00 & 0.98 & 0.99 \\ Look & 80 & 0.66 & 0.97 & 0.78 & Look & 80 & 0.94 & 1.00 & 0.97 \\ Hamburger & 79 & 0.90 & 0.94 & 0.92 & Hamburger & 79 & 0.82 & 0.95 & 0.88 \\ Hurt & 75 & 0.93 & 0.99 & 0.95 & Hurt & 75 & 0.99 & 1.00 & 0.99 \\ Bird & 74 & 0.86 & 0.99 & 0.92 & Bird & 74 & 1.00 & 0.96 & 0.98 \\ Cat & 73 & 0.87 & 0.90 & 0.89 & Cat & 73 & 1.00 & 0.96 & 0.98 \\ Old & 66 & 0.79 & 0.58 & 0.67 & Old & 66 & 0.98 & 0.98 & 0.98 \\ Cold & 65 & 0.93 & 0.98 & 0.96 & Cold & 65 & 0.97 & 0.98 & 0.98 \\ Banana & 64 & 1.00 & 1.00 & 1.00 & Banana & 64 & 0.97 & 1.00 & 0.98 \\ Church & 63 & 1.00 & 0.98 & 0.99 & Church & 63 & 0.98 & 0.98 & 0.98 \\ Sleep & 63 & 0.73 & 0.92 & 0.82 & Sleep & 63 & 1.00 & 1.00 & 1.00 \\... &... &... &... &... &... &... &... &... &... \\ \hline Average & - & 0.88 & 0.87 & 0.86 & Average & - & 0.98 & 0.98 & 0.98 \\ Total & 4877 & - & - & - & Total & 4877 & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance of SVM and deep forest classifiers for the top 20 signs. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Index** & **Signs** & **Word/Sentence** \\ \hline
1 & Apple & Apple \\
2 & Banana & Banana \\
3 & Drink & Drink-Water \\
4 & Water & \\
5 & Father & Father-Walk \\
6 & Walk & \\
7 & Hello & \\
8 & Where & Hello-Where-Toilet \\
9 & Toilet & \\
10 & Dog & & \\
11 & Yesterday & Walk-Dog-Yesterday \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Private testing set. The deep forest and the SVM classifiers trained on the public ASLLVD with 104 signs are used in the test directly. A test clip is split into frames, and each frame is annotated by using these two classifiers. The results are handled based on the voting strategy. According to the expected number of signs, the corresponding top frequent signs are picked to combine the sentence. Table 4 shows the weighted results of each words. The top 5 weighted results are listed in the columns Prediction1, Prediction2, Prediction3, Prediction4, and Prediction5. The total column presents the number of frames for each sentence. The recognition numbers for each sign are weighted and listed in the brackets. Two widely used evaluation scores, precision of frames (PoF) and recall of signs (RoS), are employed to illustrate the results.