# Continuous Recognition of Teachers' Hand Signals for Students with Attention Deficits
## 3 Experimental Results  
We collected five in-class video sequences (video 1-video 5) which were manually labeled by a highly experienced teacher in inclusive education to evaluate the recognition performance of the proposed method. Each of the five video sequences depicts a teacher delivering a lecture using a blackboard, electronic whiteboard, or projection screen, with students seated in the classroom. The frame numbers of the five video sequences are 71,678, 73,285, 71,202, 46,404, and 32,992 with a frame rate of 30 frames/s. The resolution is 1920 \(\times\) 1080 for video 1-video 4 and 854 \(\times\) 480 for video 5. This study was approved by The Institutional Review Board / Ethics Committee of Mennonite Christian Hospital, Taiwan. Accuracy denotes the ratio of the number of correctly classified signals to the total number of signals. Equations (2)-(4) represent the \(\textit{Sensitivity}_{i}\), \(\textit{Specificity}_{i}\), and \(\textit{Precision}_{i}\) defined by \(\textit{True Positive}_{i}\) (\(\textit{TP}_{i}\)), \(\textit{True Negative}_{i}\) (\(\textit{TN}_{i}\)), \(\textit{False Positive}_{i}\) (\(\textit{FP}_{i}\)), and \(\textit{False Negative}_{i}\) (\(\textit{FN}_{i}\)) values for the class label \(i\in\{\textit{Pointing to left}\), \(\textit{Pointing to right}\), \(\textit{Non-pointing}\}\). Equation (5) denotes the \(\textit{F}_{1}\)\(\textit{score}_{i}\) defined by the \(\textit{Precision}_{i}\) and \(\textit{Recall}_{i}\) (\(\textit{Sensitivity}_{i}\)). The macro-average [38] is calculated by taking the average of each metric of the three classes. \[\textit{Sensitivity}_{i}(\textit{Recall}_{i})=\frac{\textit{TP}_{i}}{\textit{ TP}_{i}+\textit{FN}_{i}} \tag{2}\]  
\[\textit{Specificity}_{i}=\frac{\textit{TN}_{i}}{\textit{TN}_{i}+\textit{FP} _{i}} \tag{3}\]  
\[\textit{Precision}_{i}=\frac{\textit{TP}_{i}}{\textit{TP}_{i}+\textit{FP} _{i}} \tag{4}\]  
\[\textit{F}_{1}\textit{score}_{i}=\frac{2\times\textit{Precision}_{i}\times \textit{Recall}_{i}}{\textit{Precision}_{i}+\textit{Recall}_{i}} \tag{5}\]  
Figure 6 shows the confusion matrices for the five video frames. Standard assessments to measure the performance of the proposed method include accuracy, sensitivity, specificity, precision, and F\({}_{1}\) score in Table 2. Each metric of macro-average is also listed. The average accuracy, sensitivity, specificity, precision, and F\({}_{1}\) score achieve 88.31%, 91.03%, 93.99%, 86.32%, and 88.03%, respectively. Moreover, the average accuracy ranges from 83.66% to 91.30%, the average sensitivity ranges from 87.38% to 94.52%, the average specificity ranges from 91.59% to 95.64%, the average precision ranges from 80.88% to 89.97%, and the average F\({}_{1}\) score ranges from 83.40% to 91.09% for the five video sequences. Figure 6: Confusion matrices for video sequences. **(a)** Video 1. **(b)** Video 2. **(c)** Video 3. **(d)** Video 4. **(e)** Video 5. As shown in Table 2, we can see that the accuracy of video 3 is lower than those of the other videos. This is because the teacher in this video often uses a telescopic pointer to point at focus. Therefore, the extension of the teacher's hand is not as far as that of the teachers in other videos, and it makes the accuracy drop a little bit. Other videos in which the teacher is only pointing by hand have satisfactory results. The performance may be affected by empirical values of \(Th_{R}\) and \(Th_{t}\). If \(Th_{R}\) is chosen to be larger, it means that only when the teacher extends the arm fully can a pointing signal be detected. We choose \(Th_{R}\) as 0.5. Hence, the students can obtain the attention reminder for _Pointing to left_ or _Pointing to right_ in most cases. In Table 2, both averaged sensitivities of _Pointing to left_ and _Pointing to right_ are more than 95%. However, the average sensitivity of _Non-pointing_ is about 82.41%. If the minimum lasting time \(Th_{t}\) is selected to be shorter, it may easily lead to rapid signal switching due to unintended movements of the teacher's hands.