# Continuous Recognition of Teachers' Hand Signals for Students with Attention Deficits
## 2 Proposed Method  
Our study targets scenes captured from a video camera in an elementary school classroom. To provide a simple and practical attention cue to the students, the teacher's hand signals are classified into three kinds of signals: _Pointing to left_, _Pointing to right_, and _Non-pointing_ from the students' field of vision during class. The pointing focus may be at a blackboard, an electronic whiteboard, or a project screen. Identifying human activities is an important task that relies on video sequences or sensor data for prediction. This is a challenging computer vision task due to the diversity and complexity of input data, especially in applications like human motion recognition in videos. Typically, techniques such as extracting skeletal points are used to capture motion features, determining if actions are triggered or meet specific criteria, such as in yoga pose detection [26], [PERSON]'s disease diagnosis based on gait analysis [27], and predicting pedestrian intentions [28], rather than analyzing the entire video comprehensively. This approach helps to reduce the amount of data while enhancing efficiency and accuracy in activity recognition. Some studies [29; 30; 31; 32; 33] discuss the methodologies for detecting 2D or 3D keypoints in human poses, offering insights into the analysis of motion, gesture, posture, or action. Due to the intricacies and longer processing time associated with 3D detection, we choose a 2D approach. We exploit a computer vision-based machine learning library MediaPipe to find the teacher's skeleton. MediaPipe provides complete models for many different tasks, including pose [34; 35], hands [36], face [37], and so on. In this study, MediaPipe BlazePose [34; 35], detecting 33 landmarks for body pose tracking, is used to detect the teacher's skeleton in the video sequence. BlazePose, despite supporting only single-person detection, utilizes heatmaps and regression for keypoint coordinates, making it generally faster than OpenPose. Therefore, we adopt BlazePose, which balances accuracy with computational efficiency to better meet the needs of mobile and real-time applications.Figure 1 depicts the 33 landmarks detected by MediaPipe BlazePose and their numeric labels. Table 1 provides a detailed list of these landmarks, each with their corresponding BlazePose landmark names. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Number** & **Name** & **Number** & **Name** & **Number** & **Name** \\ \hline
0 & Nose & 11 & Left shoulder & 22 & Right thumb \\ \hline
1 & Left eye inner & 12 & Right shoulder & 23 & Left hip \\ \hline
2 & Left eye & 13 & Left elbow & 24 & Right hip \\ \hline
3 & Left eye outer & 14 & Right elbow & 25 & Left knee \\ \hline
4 & Right eye inner & 15 & Left wrist & 26 & Right knee \\ \hline
5 & Right eye & 16 & Right wrist & 27 & Left ankle \\ \hline
6 & Right eye outer & 17 & Left pinky & 28 & Right ankle \\ \hline
7 & Left ear & 18 & Right pinky & 29 & Left heel \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Landmark names of MediaPipe BlazePose model [35]. These 33 landmarks are from various parts of the human body. They were specifically chosen to provide a comprehensive understanding of the human pose, enabling the estimation of rotation, size, and position of the region of interest [34]. These landmarks represent specific points of interest of the pose measurement. Figure 2 shows some examples of the landmarks extracted by MediaPipe BlazePose and classified according to the three kinds of the teacher's hand signals. Our method aims to find simple rules to analyze the three recognized signals in the students' field of vision to facilitate fast implementation. Only the coordinates of the shoulder, wrist, and hip are required in our method. The initial status of the signal of a video sequence is _Non-pointing_ within the three kinds of signals.