# Movement-in-a-Video Detection Scheme for Sign Language Gesture Recognition Using Neural Network
## 3 Result and Discussion
LSTM is used because of its capability of learning long-term dependencies. Two experiments are performed: (1) without hand localization and (2) with hand localization. Because hand localization is out of the scope of the study, we compared our results with those of the second experiment. Apart from the accuracy in predicting the gestures in the LSA64 dataset, Table 2 also lists the accuracies of predicting the gestures in DEVISIGN-D and SLR_Dataset. As illustrated in Table 2, the proposed method exhibits a higher accuracy (90.33%) of predicting short gestures (LSA64) compared with the combination of CNN and RNN (69%). Because the time consumptions listed in Table 1 are less using the proposed method, the results evidently reveal that the proposed method is effective in predicting short gestures. The method used by [PERSON] et al. [7] exhibits higher accuracy than the proposed method in predicting medium and long gestures (by 7.5% and 27.84%, respectively). However, the main objective of this study is to reduce both the complexity and cost of sign language recognition by extracting both the spatial and temporal information in a video and storing it in an image. This necessitates only a network capable of extracting spatial features and eliminating the need for other networks to extract the temporal features. Whereas [PERSON] et al. [7] used both CNN and RNN (specifically LSTM), this method used only CNN. Therefore, considering the reduction in the computing complexity and cost, a difference of 7.5% is reasonable. However, we consider the fact that accuracy in recognizing sign language is important. [PERSON] et al. [7] applied hand localization to increase the accuracy of gesture recognition in DEVISIGN-D and SLR_Dataset. Hand localization can be applied to improve the accuracy obtained using the proposed method as well. The results obtained in predicting long gestures (SLR_Dataset) is expected because more than five hand and arm poses and positions are required to complete a gesture. Because of the long duration needed to perform long gestures, it is inevitable that the gestures's make small movements. These small movements are also detected by the proposed algorithm. To avoid these issues, these small movements must be filtered out, with focus \begin{table}
\begin{tabular}{c c c c} \hline \hline  & & \multicolumn{3}{c}{**LSA64**} \\ \cline{3-4}
**Process** & **Data** & **CNN + RNN**[14] & **Proposed Method** \\ \hline preprocessing & Train & 3225 & 7221 \\ retrain CNN & Test & 783 & 1817 \\ retrain CNN & Train & 804 & 499 \\ predict spatial & Train & 202,502 & - \\ retrain RNN & Test & 10,022 & - \\ retrain RNN & Train & 170 & - \\  & TOTAL & 217,506 & 9537 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Time consumption (in seconds, s) by each training process. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **LSA64** & **DEVISIGN-D** & **SLR** \\ \hline CNN + RNN [14] & 0.6900 & - & - \\ BLSTM-3D [7] & - & 0.437 & 0.502 \\ proposed method & 0.9033 & 0.40 & 0.2236 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Summary of accuracies of classifying the gestures in LSA64, DEVISIGN-D, AND SLR_Dataset. only on the gestures that have a big difference between the initial and final arm poses and positions. In addition, Inception V3 is tuned only for the images containing spatial and temporal information of short gestures. As shown in Figure 9, the path of hands and arms are clear for short gestures and unclear for long gestures. Hence, to improve the model accuracy for long gestures, a feature extractor must be tuned for this type of images.