# IPN HandS: Efficient Annotation Tool and Dataset for Skeleton-Based Hand Gesture Recognition
## 4 IPN HandS Dataset
### Analysis of Automatic Annotation Quality  
To better understand the necessity of manual refinement, we conducted a detailed comparison of annotation quality between our refined IPN HandS and the outputs from MediaPipe Hands [12] and MediaPipe H holistic [13]. We analyzed both the temporal consistency and spatial precision of the hand keypoints produced by these models, using IPN HandS as the reference. Specifically, we measured (i) the rate of missed detections, defined as frames where no valid hand landmarks were returned, and (ii) the keypoint error, computed as the normalized average Euclidean distance between corresponding landmarks across all detected frames. This evaluation was performed per gesture class to \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Gesture** & **Name** & **Instances** & **Total Frames** & **Avg. Duration (\(\sigma\))** \\ \hline No-Gesture & NOG & 2593 & 261,599 & 101 (118) \\ \hline Point-1F & P1F & 1222 & 221,811 & 182 (95) \\ Point-2F & P2F & 1191 & 224,976 & 189 (97) \\ Click-1F & C1F & 204 & 6021 & 29 (10) \\ Click-2F & C2F & 186 & 5665 & 30 (8) \\ Th-Up & TUP & 201 & 6525 & 31 (9) \\ Th-Down & TDN & 200 & 5529 & 27 (8) \\ Th-Left & TLT & 197 & 6255 & 30 (9) \\ Th-Right & TRT & 195 & 6230 & 30 (9) \\ Open-2 & OP2 & 194 & 10,690 & 52 (14) \\
2 Click-1F & 2C1F & 199 & 10,084 & 50 (15) \\
2 Click-2F & 2C2F & 215 & 10,681 & 49 (15) \\ Zoom-In & ZIN & 231 & 8901 & 36 (12) \\ Zoom-Out & ZOT & 208 & 7868 & 35 (12) \\ Grab & GRB & 534 & 8621 & 19 (6) \\ \hline
**Total (Gestures)** & & **5177** & **539,857** & **56 (23)** \\
**Total (All)** & & **7770** & **801,456** & **106 (115)** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Statistics of our IPN HandS dataset. Duration is measured in frames. Figure 6: Gesture classes of IPN HandS with hand skeleton annotations. Each column depicts sequential frames for a gesture instance from top to bottom, highlighting its temporal evolution. Gestures P1F, C1F, and 2C1F are not shown, as they are the one-finger variants of P2F, C2F, and 2C2F, respectively. reveal gesture-specific patterns in annotation errors and to assess the reliability of automatic methods across various motion dynamics and hand configurations. Table 5 presents the rate of missed detections and average keypoint errors across all gesture classes. Pointing gestures (Point-1F, Point-2F) show high rates of both missed detections and spatial errors, highlighting their sensitivity to temporal instability. In contrast, clicking gestures exhibit fewer missed detections but suffer from higher spatial errors and larger standard deviations, indicating inconsistent keypoint placement. While MediaPipe Holistic outperforms MediaPipe Hands overall, both models demonstrate substantial variability, reinforcing the need for manual refinement to ensure reliable and temporally coherent annotations. Figure 7 shows a temporal sequence of a pointing gesture annotated by MediaPipe Hands, MediaPipe Holistic, and our refined IPN HandS labels. The highlighted regions illustrate common failure cases in the automatic methods, including spatial drift and frame-level inconsistencies, which are effectively corrected in the refined annotations. In summary, the analysis confirms that while MediaPipe Holistic offers slight improvements over MediaPipe Hands, both automatic methods suffer from significant spatial and temporal inconsistencies, particularly in challenging gesture classes. These results highlight the limitations of fully automatic annotation pipelines and underscore the importance of manual refinement, as enabled by our IPN HandS. The resulting annotations provide a consistent and high-quality benchmark for skeleton-based HGR. In the following section, we demonstrate their utility by training recognition models using the refined annotations. Figure 7: Frame-by-frame comparison of a pointing gesture annotated by (**a**) MediaPipe Hands, (**b**) MediaPipe Holistic, and (**c**) the refined IPN HandS annotations.