# Movement-in-a-Video Detection Scheme for Sign Language Gesture Recognition Using Neural Network
## 2 Sign Language Recognition Method
### Sum of Frame Difference  
The difference between two consecutive frames is obtained using Algorithm 1. In this method, only three images are stored as the information is processed:  
1. Current frame, \(frame_{k}\)--contains the newly imported frame. 2. Preceding frame, \(frame_{k-1}\)--contains the preceding frame of \(frame_{k}\). 3. Sum-of-differences frame, \(frame_{s}\)--sum of the differences between \(frame_{k}\) and \(frame_{k-1}\). In Algorithm 1, Step 1 sets the \(frame_{count}\) to the total number of extracted frames. Step 2 initializes the value of \(frame_{s}\) to zero or to an empty frame. Steps 4-22 are repeated for each of the frames extracted in a video. Steps 4-22 are explained below:  
In Step 4, each frame is imported, and in Step 5, it is saved to the current frame. In Steps 6-10, the imported frame is examined to ascertain whether it is the first frame. If that is the case, it indicates that the preceding frame, \(frame_{k-1}\), is empty. This step is necessary because the current frame will always be subtracted from the preceding frame, as in Step 7. The difference between the current frame and its preceding frame is set as the difference frame, \(frame_{d}\). Given that the first frame has no preceding frame, it is directly set as the preceding frame for the next frame, as in Step 9. ```
1\(frame_{count}=\) total number of frames extracted;
2\(frame_{s}=0\);
3 foreach frame, \(frame_{i}\), do
4 import\(frame_{i}\);
5\(frame_{i}=frame_{i}\);
6 if\(i\
eq 1\)do
7\(frame_{d}=frame_{k-1}-frame_{k}\);
8
9\(frame_{k-1}=frame_{k}\);
10 endif
11 for each element in \(frame_{d}\), \(frame_{d,jk}\)do
12 if\(frame_{d,jk}>200\) or\(frame_{d,jk}<55\)do
13\(frame_{d,jk}=0\);
14 endif
15 endfor
16 for each element in \(frame_{d}\), \(frame_{d,jk}\)do
17 if\(frame_{d,jk}\
eq 0\)do
18\(frame_{d,jk}=\lfloor\frac{255}{frame_{count}-1}\rfloor\);
19 endif
20 endfor
21\(frame_{s}=frame_{s}+frame_{d}\);
22\(frame_{k-1}=frame_{k}\);
23 endfor
```  
**Algorithm 1** Difference of two consecutive frames  
Steps 12-14 are repeated for each pixel in the difference frame, \(frame_{d}\). These steps assign a pixel value of zero to the pixel above and below the upper and lower thresholds, respectively. An example of a pixel value and its corresponding color in grayscale mode is shown in Figure 3. Pixel values from 0 to 255 render a color gradient from black to white. A pixel value of 0 corresponds to black, whereas a pixel value of 255 corresponds to white. Pixel values between 0 and 255 render colors with different shades of gray. The result of subtracting the first frame from the second frame is shown in Figure 4. Only the specific part of the subject that moved was retained. In Figure 4, the pixels with values higher and lower than the upper and lower thresholds, respectively, are set to zero. The figure clearly shows the portion of the subject that moved and its final position in the whole frame. The values that yield the least noise in the output image are selected as the thresholds by trial and error. In Steps 12-14, although pixels with negative values are mapped automatically to black, they are set to zero for computational evidence. Steps 17-19 are repeated for each pixel in the difference frame, \(frame_{d}\). These steps assign a pixel value of  
\[frame_{d,\;jk}^{*}=\frac{255}{frame_{count}{-1}} \tag{2}\]  
Figure 4: Example of movement detection between two consecutive frames: (**a**) position of subject in the first frame; (**b**) position of subject in the second frame; (**c**) result of subtracting the pixels in the first frame from those in second frame; (**d**) result of setting pixel values higher and lower thresholds, respectively, to zero. Figure 3: Pixel values and their corresponding colors in grayscale mode. to a non-zero pixel. Here, * is used to differentiate the current value of \(frame_{d,\;jk}\) (used as a reference) to its new assigned value; \((j,k)\) are the coordinates of the non-zero pixel. In the example in Figure 4, the number of frame count is two.