# IPN HandS: Efficient Annotation Tool and Dataset for Skeleton-Based Hand Gesture Recognition
## 5 Hand Gesture Recognition Evaluation
### Model Architecture  
We adopted a recurrent neural architecture based on Long Short-Term Memory (LSTM) [24] units to evaluate the impact of annotation quality on recognition performance, specifically for skeleton-based HGR using time-delimited sequences of annotated hand landmarks. LSTMs are a class of recurrent neural networks (RNNs) capable of modeling temporal dependencies in sequential data through gated memory cells that regulate the flow of information over time [25]. Their design enables them to effectively capture both short-term and long-term motion patterns, making them particularly suitable for recognizing hand gestures, which often unfold across multiple frames with varying temporal dynamics.  
In our setup, the input to the model is a sequence of 2D hand skeletons, where each frame is represented by a 42-dimensional feature vector corresponding to the flattened \((x,y)\) coordinates of 21 hand landmarks. We focus on the primary active hand, selecting a single hand per frame, as the IPN HandS dataset is constructed to support single-hand gesture recognition. Gesture instances are processed as fixed-length temporal sequences sampled from annotated gesture intervals.  
The LSTM model consists of three stacked unidirectional LSTM layers, each with 256 hidden units, followed by a fully connected layer that maps the final hidden state of the sequence to 14 gesture class logits. A dropout layer is applied between the LSTM layers to  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{2}{c}{**Missed Detections**} & \multicolumn{2}{c}{**Keypoint Error**} \\
**Gesture** & **MP Hands (\(\sigma\))** & **MP Holistic (\(\sigma\))** & **MP Hands (\(\sigma\))** & **MP Holistic (\(\sigma\))** \\ \hline Point-1F & 13.51 (15.8) & 12.89 (15.3) & 18.34 (35.6) & 11.51 (31.0) \\ Point-2F & 17.96 (16.9) & 14.74 (15.1) & 18.32 (32.3) & 12.95 (32.6) \\ Click-1F & 8.23 (11.3) & 7.88 (12.6) & 12.19 (26.8) & 7.92 (26.6) \\ Click-2F & 6.78 (8.0) & 4.80 (6.0) & 12.13 (29.8) & 7.85 (30.3) \\ Th-Up & 7.89 (92.2) & 6.56 (9.0) & 8.81 (15.3) & 5.64 (15.4) \\ Th-Down & 14.26 (15.5) & 20.46 (18.6) & 7.41 (14.7) & 9.59 (14.9) \\ Th-Left & 13.42 (13.7) & 11.11 (13.3) & 7.21 (14.0) & 6.07 (14.6) \\ Th-Right & 16.72 (16.6) & 11.67 (14.7) & 7.61 (14.8) & 7.16 (16.3) \\ Open-2 & 3.19 (4.6) & 1.94 (5.3) & 4.07 (12.3) & 2.37 (12.5) \\
2 Click-1F & 7.85 (12.1) & 6.57 (12.1) & 12.00 (25.1) & 7.97 (22.2) \\
2 Click-2F & 7.96 (11.3) & 6.19 (9.8) & 11.58 (25.5) & 7.03 (24.7) \\ Zoom-In & 6.17 (10.1) & 7.21 (13.1) & 6.32 (14.0) & 4.81 (18.0) \\ Zoom-Out & 4.88 (11.2) & 5.85 (11.3) & 4.73 (14.7) & 3.74 (13.4) \\ Grab & 4.25 (9.2) & 3.87 (9.1) & 4.67 (11.8) & 3.88 (13.5) \\ \hline
**Total** & **9.51 (11.8)** & **8.70 (11.8)** & **9.67 (20.5)** & **7.04 (19.0)** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Per-gesture comparison of missed detections and keypoint error between MediaPipe Hands and MediaPipe Holstic, using IPN HandS annotations as ground truth. Values are reported as percentages with standard deviation in parentheses (\(\sigma\)).  
improve generalization. The model is trained from scratch using a standard cross-entropy loss function and optimized with the Adam algorithm.