# Recognizing Teachers' Hand Gestures for Effective Non-Verbal Interaction
## 3 Proposed Framework
### Shape Detection Based on Illumination Invariance  
The preprocessing steps of the gesture image are shown in Figure 2. The aim of preprocessing is to deal with the shape detection problem under different lighting conditions. Through this, we can uniformly process the data to align the shape of the gesture for efficient input into the DCNN model. First, we convert the RGB image to a grayscale image to remove the influence of color features; then, we convert the grayscale image to a binary image and select the foreground of the binary image as the extraction region of the gesture. The target data after the preprocessing step are imputed into the proposed CNN model, which is represented by the purple block in Figure 2.  
We use local binary pattern (LBP) to describe the texture features of gesture images. According to the literature [45], the LBP operator can compare the gray value of the center pixel and the adjacent pixel from a \(3\times 3\) window. The local area texture calculation based on LBP is shown in Formula (1). \[K_{\text{LBP}}(x_{c},y_{c})=\sum_{0}^{7}\,s(g_{p}-g_{c})2^{p} \tag{1}\]  
where \(g_{p}(p=0,1,\ldots,7)\) and \(g_{c}\) represent grayscale images of adjacent points and the center point, respectively, \((x_{c},y_{c})\) represents the coordinates of the center point, and \(s(g_{p}-g_{c})= \begin{cases}1,&g_{p}\geq g_{c}\\ 0,&g_{p}<g_{c}\end{cases} \). For image samples under different lighting conditions, the brightness at pixel \(a\) can be expressed by Formula (2). \[B(a)=g\,C(a)\,R(a)+p \tag{2}\]  
where \(B(a)\) is the brightness of pixel \(a\), \(C(a)\) and \(R(a)\) are the light intensity and surface reflectivity, respectively, and \(g\) and \(p\) are the camera parameters. It can be seen that the brightness of pixel \(a\) is determined by the light intensity of the light source and the reflectivity of the object. If the light intensity \(C(a)\) is kept at a certain value, the relationship between pixel brightness and object reflectivity can be computed by Formula (3). \[B^{\prime}(a)=k_{1}R(a)+p \tag{3}\]  
where \(k_{1}\) is the bias constant. Obviously, the brightness of pixel \(a\) is only related to the reflectivity of the object. Therefore, the influence of different light intensities in complex environments can be reduced by normalizing the pixels of the input image. In this study, MCT is used as the feature descriptor of image pixels. MCT is a local texture description operator without parameter adjustment. It can detect candidate regions of human gestures, capture the overall structural information of the image, and has strong robustness to illumination and detail transformation. For a \(3\times 3\) pixel window, MCT compares the gray value of the eight adjacent pixels of the central pixel with the regional average pixel. If the gray value of the pixel is greater than the average gray value, we set it to 1, otherwise we set it to 0; then, we convert any adjacent local block to binary, keeping its value consistent with the central block. Unlike traditional statistical transformation algorithms, MCT can compare each pixel value with the average pixel value of a local area, and is not significantly affected by lighting. At the  
Figure 2: Preprocessing of gesture images. same time, since only the difference and the average value of eight adjacent pixels in the local area are considered, the use of MCT can reduce the influence of noise at the central pixel \(a\) on image features, which has strong robustness to complex illumination changes. The MCT calculation process of pixel \(a\) is shown in Formula (4). \[\Gamma(a)=\otimes f\big{[}B_{\text{avg}}(a)<B(y)\big{]},(y=1,2,\dots,N) \tag{4}\]  
where \(\Gamma(a)\) is the MCT feature expression, \(N\) is the number of adjacent pixels of the central pixel \(a\), and \(B_{\text{avg}}(a)\) is the average pixel gray value of \(N\) pixels. When the operation result is true, the function \(f\) returns 1, otherwise it returns 0. \(\otimes\) is the arrangement of conversion results of the \(N\) pixels.