# Recognizing Teachers' Hand Gestures for Effective Non-Verbal Interaction
## 4 Experiments
### Algorithms and Evaluation Metrics  
To verify the effectiveness of the proposed framework in processing images with different backgrounds, we chose two object detection frameworks, YOLO and YOLOv3 [53], to compare with the proposed framework. The network structure of YOLO is based on the GoogleNet network, adding four convolutional layers and two fully connected layers. It divides the image into \(7\times 7\) grid cells. When the center of the object falls into a certain grid, the object is assigned two bounding boxes, and finally outputs a \(7\times 7\times 30\)-dimensional tensor. YOLO converts the target detection into a regression model, and the loss function uses the mean square error. YOLOv3 is an improved version of the YOLO algorithm. It uses a deeper network structure called Darknet53, which alternately uses \(3\times 3\) and \(1\times 1\) convolution and residual structures. YOLOv3 includes feature pyramid networks for object detection to achieve multi-scale detection. The settings of all parameters were consistent with the original documents. In addition, we selected the Bayesian attention model [38],  
Figure 6: Samples from the NUS-II dataset.  
the deformable part model (DPM) based on object detection [54], and the skin-color-based HSV [55] model as representatives for additional comparison.  
Some algorithms, e.g., [44], simply define gesture recognition as a supervised machine learning problem, which would generate a lot of background noise when building feature sets. Such methods cannot deal with gesture recognition images with poor shooting quality, such as the constant movement of the teacher's position and the light intensity variance in different weather conditions. To overcome this problem, we first detect the shape of the hand to remove the influence of backgrounds and backlights, and keep the extracted gesture features almost pure. Hand gesture detection is reformulated as a single-target detection task, and each pixel of the input image is divided into target and non-target classes; thus, hand gesture detection can be regarded as a binary classification problem. The target is identified if the key points of hand gestures fit for most of the geometric and statistical features, otherwise the pixel is placed in the non-target class. We use the receiver operating characteristic (ROC) curve [56] as the evaluation metric, because it is often used to evaluate the quality of a binary classifier. The \(x\) axis of the ROC curve represents the false positive rate, and the \(y\) axis of the ROC curve represents the true positive rate. The false positive rate is calculated by FP/(FP + TN), and the true positive rate is calculated by TP/(TP + FN), where FP is false positive, TP is true positive, TN is true negative, and FN is false negative. The closer the ROC curve of the algorithm is to the upper left, the more accurate the algorithm is. In addition, we evaluate the extent to which algorithms detect the shape of hand gestures at the first stage and then recognize their specific meaning using the detection accuracy and localization accuracy in the evaluation system of object detection. The detection accuracy means the extent to which algorithms detect the shapes and boundaries of the object, and the localization accuracy means how frequently the algorithms estimate labels of the object successfully. The higher the detection accuracy and localization accuracy score, the more accurate the algorithm is. Since YOLO is an object detection algorithm, its evaluation index is different from the classification problem. In order to compare with the proposed framework, we use the F1 value as the evaluation index, which is suitable for both object classification and object detection.