# IPN HandS: Efficient Annotation Tool and Dataset for Skeleton-Based Hand Gesture Recognition
## 6 Discussion  
The results presented in the previous section demonstrate the substantial impact of annotation quality on skeleton-based HGR. By integrating spatially accurate and temporally consistent annotations, IPN HandS significantly improves recognition performance across gesture classes. However, despite these advantages, some gesture classes remain challenging. The 2C1F class, in particular, shows consistently lower recall and is frequently misclassified as P1F, even when trained on IPN HandS. This class exhibits high intra-class variability in finger articulation and orientation, and often visually overlaps with other single-finger gestures like P1F and C1F. Interestingly, this confusion appears asymmetric: while 2C1F is often mistaken for P1F, the reverse is less common. Moreover, a similar pattern does not occur between structurally analogous classes such as 2C2F and P2F, suggesting that the confusion is not solely due to general gesture ambiguity but may be rooted in class-specific motion patterns or labeling subjectivity. These findings indicate a potential trade-off between annotation precision and the model's ability to generalize under ambiguous class definitions. To address this, future  
Figure 8: Normalized confusion matrices for three evaluation settings. (**a**) Model trained and tested on IPN HandS. (**b**) Model trained and tested on MediaPipe Holistic annotations. (**c**) Model trained on IPN HandS and tested on MediaPipe Holistic annotations. work could explore data augmentation strategies that increase intra-class variability during training or leverage multi-hand and body-pose context to resolve ambiguities. In addition, supporting multi-hand annotations or integrating 3D hand pose estimation could offer richer spatial information to better disambiguate subtle gestures. These issues also highlight the necessity for more robust recognition models that can effectively utilize fine-grained annotations without overfitting to narrow gesture variants. We expect that more advanced architectures, such as Graph Convolutional Networks (GCNs) [28], Transformers [7], or hybrid models [10], could further benefit from the spatial and temporal consistency provided by the high-quality annotations of the IPN HandS dataset. Future work could explore these architectures to validate whether the performance gains observed with IPN HandS generalize across model families. However, in this study, we deliberately chose a simple LSTM baseline to isolate and evaluate the effect of annotation quality, independent of model complexity. On the other hand, the IPN HandS dataset offers more than a benchmark for HGR. Its precise landmark annotations across continuous videos create new opportunities for research in areas like temporal segmentation, gesture spotting, and motion pattern analysis. Additionally, the availability of consistent hand pose sequences under various conditions establishes a solid foundation for training and evaluating hand pose estimators, especially in specific applications such as sign language recognition, interactions in augmented reality (AR) and virtual reality (VR), and teleoperation. Recent developments in context-aware sim-to-real adaptation for human-robot shared control [32] emphasize the importance of robust, transferable recognition systems in surgical and robotics applications. The temporal coherence and annotation reliability of IPN HandS could facilitate such transfer learning approaches by offering cleaner motion sequences and reduced noise, thus enhancing generalization to real-world, safety-critical scenarios. In addition to our IPN HandS dataset, our contribution has broader implications for dataset development and HGR methodology. For instance, our annotation tool provides a practical solution for large-scale video-based landmark annotation. Its design effectively balances automation with manual control, supporting diverse correction strategies while minimizing annotator effort.