# A Real-Time Dynamic Gesture Variability Recognition Method Based on Convolutional Neural Networks
## 3 Materials and Methods
### Preprocessing Module  
The MediaPipe Hand tool [29], an open-source software tool created by Google, is a comprehensive hand tracking and gesture recognition tool. It uses a set of 21 connections for each hand, including four for each finger, one at the center of the palm, and one at the wrist. By analyzing the movement and position of these connections, the tool can accurately track the rotation and movement of the hand in three-dimensional space. In addition to tracking the hand, the tool can also track the body skeleton, including the hand, pose, and head relative to the camera. This allows one to analyze dynamic gestures that involve movements of the entire body, not just the hands, using a 2D camera. In the preprocessing module, key gesture points are extracted using the MediaPipe Holistic [30] pipeline. It is designed to overcome the limitations of individual models for pose and hand components. The MediaPipe Holistic pipeline starts with an assessment of a person's posture using the Blazes pose detector and a subsequent reference model. Then, using the found pose landmarks, two regions of interest (ROI) for each hand are determined, and a reframing model is used to improve the ROI. Then, the full-resolution input frame is cropped to these areas of interest, and hand models specific to the gesture recognition task are used to evaluate the corresponding landmarks. In the final step, all the landmarks are combined with the pose model landmarks to obtain a total of 258 coordinates. These coordinates consist of 33 values for the pose (x, y, z) and 33 values for visibility, resulting in a total of 33 \(\times\) 4 = 132. Additionally, there are 22 values (x, y, z) for each hand, accounting for a total of 22 \(\times\) 2 \(\times\) 3 = 126 coordinates. This multi-stage approach allows the MediaPipe Holistic pipeline to achieve a high level of accuracy in detecting human pose and hand movements. Let us break it down further:  
Landmark Extraction: Initially, landmarks specific to the pose of a person, including body joints and facial features, are extracted using a pose estimation model. These landmarks provide information about the overall body position and orientation. Pose Landmarks: The pose landmarks consist of 33 coordinates (x, y, z) and 33 visibility values. The (x, y, z) coordinates represent the spatial position of each landmark, while the visibility values indicate whether the landmark is occluded or visible. Hand Landmarks: Hand landmarks are extracted separately for each hand using a hand tracking model. Each hand consists of 21 landmarks, resulting in a total of 22 values (including a wrist landmark). The (x, y, z) coordinates of these landmarks capture the hand's position, orientation, and finger movements. Figure 1: The proposed method. Combining Landmarks: Finally, the extracted pose landmarks and hand landmarks are combined to form a comprehensive representation of the person's pose. By merging these landmarks, the system obtains a total of 258 coordinates (132 from pose landmarks and 126 from hand landmarks) that collectively describe the person's body posture and hand gestures. This multi-step process enables the MediaPipe Holistic pipeline to accurately analyze and interpret human pose and hand movements, facilitating various applications such as gesture recognition, motion tracking, and augmented reality. After collecting the critical points of the dynamic gesture, when viewing the captured data, it is observed that not all the connections were displayed; based on this, the motions were grouped according to the demonstration form to embrace the maximum possible forms of gesture demonstration, taking into account the following properties:  
1. Palm orientation: * Palms facing the camera--gestures in which the palm faces the camera, and the fingers are visible to the viewer; for example, pointing or greeting. * Palms facing away from the camera--gestures in which the back of the palm faces the camera, and the fingers are not visible, for example, a demonstration of refusal or disapproval. 2.