# A Normalization Strategy for Weakly Supervised 3D Hand Pose Estimation
###### Abstract  
The effectiveness of deep neural network models is intricately tied to the distribution of training data. However, in pose estimation, potential discrepancies in root joint positions and inherent variability in biomechanical features across datasets are often overlooked in current training strategies. To address these challenges, a novel Hand Pose Biomechanical Model (HPBM) is developed. In contrast to the traditional 3D coordinate-encoded pose, it provides a more intuitive depiction of the anatomical characteristics of the hand. Through this model, a data normalization approach is implemented to align the root joint and unify the biomechanical features of training samples. Furthermore, the HPBM facilitates a weakly supervised strategy for dataset expansion, significantly enhancing the data diversity. The proposed normalized method is evaluated on two widely used 3D hand pose estimation datasets, RHD and STB, demonstrating superior performance compared to the models trained without normalized datasets. Utilizing ground truth 2D keypoints as input, a reduction of 45.1% and 43.4% in error is achieved on the STB and RHD datasets, respectively. When leveraging 2D keypoints from MediaPipe, a reduction in error by 11.3% and 14.3% is observed on the STB and RHD datasets. deep learning; hand pose estimation; biomechanical variability; Hand Pose Biomechanical Model +
Footnote †: journal:  
Most recent approaches for 3D pose estimation rely on the 2D-to-3D lifting pipeline [6], which detects 2D keypoints firstly and then lifts them to 3D. Our work adopts this common practice strategy to detail the proposed normalization strategy. The lifting process is inherently depth ambiguous [6], meaning that one single 2D pose can correspond to multiple potential 3D poses. The mapping rules that determine which 3D pose corresponds to the given 2D pose are learned from the training data. However, current datasets are typically collected by multiple individuals, resulting in potential variability in biomechanical features both within and across datasets. Thus, training models on such inconsistent data can lead to learned mapping rules that generate 3D poses with inconsistent biomechanical features. Multi-dataset joint training is a widely used method [10; 11; 12; 13] for improving model generalization ability. However, disparities in annotating standards across datasets can negatively impact network performance, particularly due to differences in the location of the root joint on the hand. For instance, the Rendered Hand Pose Dataset (RHD) [13] marks the root joint in the wrist area, while the Stereo Hand Pose Tracking Benchmark (STB) [18] dataset marks it in the palmar center. Directly feeding the model with these data, which have significantly different palm structures, can result in the model receiving conflicting knowledge, leading to suboptimal pose estimation performance. In conclusion, current training methods for hand pose estimation face two main challenges: insufficient accommodation of biomechanical variability among samples and disparities in annotating standards across datasets. To address these issues, this study introduces a data normalization strategy aimed at enhancing model accuracy and generalization. This strategy comprises two main steps: first, aligning the root joint position in training data to a common reference point based on the target annotating standard, and second, normalizing biomechanical features using a Hand Pose Biomechanical Model (HPBM). This model is designed inspired by the biomechanical feature extraction proposed by [PERSON] et al. [12]. Specifically, the spatial angles of the finger bones are defined as dynamic parameters, while the palmar structure and bone lengths are regarded as static parameters, these designations are contingent upon the spatio-temporal consistency of specific individual parameters. After unifying the static parameters in training dataset with the HPBM, the normalized HPBM-encoded poses are acquired.