# Movement-in-a-Video Detection Scheme for Sign Language Gesture Recognition Using Neural Network
## 3 Result and Discussion
All movements, variations in position of objects, alterations in scenery, etc., are considered by the proposed method as a part of the gesture. All three images shown in Figure 12 contain information on the gesture expressing the word \"bright.\" However, the gesture involves only the movement shown under the yellow-marked area. The portions of the image under the blue-marked area correspond to the movement of the signer's hair during the implementation of the gesture. 2. Variations in the images are expected given that the presentation of each gesture depends on the signer, occupied area on the image, length of time a gesture is performed, etc. A few of the images trace the figure of the signer. Although the upper body is ideally stationary, small movements are unavoidable. It is important to note that in Figure 13, the areas occupied for this particular gesture are different although the same word \"bright\" is performed by 10 signers. 3. As shown in Figure 14, the images obtained by taking the sum of frame differences are similar for the words \"red\" and \"green\" and the words \"bright\" and \"skimmer\". Only the hand pose in performing the gesture is different. Finally, the images containing the sum of differences of consecutive frames for each gesture were used to train Inception V3. Other types of models with a CNN layer on its architecture can be used. CNN layer is a vital layer as it is mainly responsible for extracting the features on images. In this study, we will only use Inception V3. Figure 6:1 Variation in the image obtained by taking the sum of frame differences for the gesture “bright” performed three times by a signer. We first classified sign languages with short gestures (LSA64) and compared the result with those of other recent work [14]. LSA64 contains videos of 10 gestures performing 64 gestures five times each, i.e., 50 videos per gesture and a total of 3200 videos. To prevent overfitting, 70% of the data are used for training (35 videos per gesture), 10% for validation (5 videos per gesture), and the remainder for testing (10 videos per gesture). The processes and time consumption in training each method are illustrated in Figure 6:2 Two types of information are necessary for accurate sign language recognition: (1) spatial and (2) temporal information. Because CNN can capture only the spatial information, training a model with frame sequence requires additional network to preserve the temporal information in a sequence. The commonly used network to extract the temporal information is the RNN. However, because both the spatial and temporal information in the video can be extracted and stored in an image using the proposed method, there is no need for an additional network or RNN. As shown in Table 1, the proposed method reduced the training time consumption substantially (by 95.62%). The process that consumed more time is the prediction of the spatial feature. In that process, the class at which each frame in the sequence, is predicted. That is, the output of this process is a sequence of n-dimensional vectors containing the probability of each class. Herein, \(n\) is the number of classes. Because the proposed method  
Figure 6:3 Images obtained by taking the sum of frame differences for the gestures “red”, “green”, “bright”, and “skimmer” performed three times by a signer. Figure 6:4 Variations in the image obtained by taking the sum of frame differences for the gesture “bright” performed by 10 different signers. uses images that contain both spatial and temporal information of the videos, this process is eliminated. Thereby, the time consumption in training is reduced substantially. The accuracy of prediction of the gestures in the LSA64 dataset is presented in Figure 6:5  
Apart from the short gesture, we considered the medium and long gestures. We also compared the results of the proposed method with the combination of CNN and RNN architectures. [PERSON] et al. [7] uses long short-term memory (LSTM) rather than simple RNN.