Toward Approaches to Scalability in 3D Human Pose Estimation
Authors: Jun-Hee Kim
Seong-Whan Lee
Venue: NeurIPS 2024 poster (2024)
Keywords: 3D Human Pose Estimation
Data generation
Pose Decompression
Primary area: machine_vision

Abstract: In the field of 3D Human Pose Estimation (HPE), scalability and generalization across diverse real-world scenarios remain significant challenges. This paper addresses two key bottlenecks to scalability: limited data diversity caused by 'popularity bias' and increased 'one-to-many' depth ambiguity arising from greater pose diversity. We introduce the Biomechanical Pose Generator (BPG), which leverages biomechanical principles, specifically the normal range of motion, to autonomously generate a wide array of plausible 3D poses without relying on a source dataset, thus overcoming the restrictions of popularity bias. To address depth ambiguity, we propose the Binary Depth Coordinates (BDC), which simplifies depth estimation into a binary classification of joint positions (front or back). This method decomposes a 3D pose into three core elements—2D pose, bone length, and binary depth decision—substantially reducing depth ambiguity and enhancing model robustness and accuracy, particularly in complex poses. Our results demonstrate that these approaches increase the diversity and volume of pose data while consistently achieving performance gains, even amid the complexities introduced by increased pose diversity.

Toward Approaches to Scalability in 3D Human Pose Estimation
Jun-Hee Kim
Seong-Whar Lee"
Dept. of Artificial Intelligence, Korea University, Scoul 02841, Republic of Korea kim, sw. ww.lee)0korea.aco
Abstract
In the field of 3D Human Pose Estimation (HPE), scalability and generalization across diverse real-world scenarios remain significant challenges. This paper addresses two key bottlenecks to scalability: limited data diversity caused by "popularity bias' and increased 'one-to-many' depth ambiguity arising from greater pose diversity. We introduce the Biomechanical Pose Generator (BPG), which leverages biomechanical principles, specifically the normal range of motion, to autonomously generate wide array of plausible 3D poses without relying on a source dataset, thus overcoming the restrictions of popularity bias. To address depth ambiguity, we propose the Binary Depth Coordinates (BDC), which simplifies depth estimation into binary classification of joint positions (front or back). pose into three core elements -2D pose, bone
This method decomposes decision-Suustanniannn -ubstantially reducing depth ambiguity and length, and binary depth decision enhancing model robustness and accuracy, particularly in complex poses. Our results demonstrate that these approaches increase the diversity and volume of pose data while consistently achieving performance gains, even amid the complexities introduced by increased pose diversity.
Introduction
3D Human Pose Estimation (HPE) is essential for determining the 3D locations of human joints in images or videos. plays a pivotal role applications such as person re-identification D, action human mesh recovery [5] 6). and virtual reality 7 g
recognition 2 3
As the importance of
this technology grows, there an increasing demand for more accurate and universally applicable models. Addressing the key challenges that limit the scalability of this field is therefore crucial.
In this paper, we identify two primary bottlenecks impeding the scalability of 3D HPE:
1. Difficulty in Collecting Diverse Datasets: Annotated 3D datasets for 3D HPE are typically collected in controlled indoor environments with limited motions performed by few individuals. This results in a 'popularity bias" [9]. where the data primarily reflects a small subset of human activities and demographics, thus restricting the diversity needed for effective model performance in various real-world scenarios. This lack of diversity means that models trained on such data may fail to generalize unseen poses and environments.
2. Exacerbation of the 'One-to-Many' Depth Ambiguity Problem: Depth ambiguity [0] [2] arises when a single 2D pose can correspond to multiple 3D interpretations. Increased pose diversity in 3D HPE intensifies this issue, complicating the depth estimation process. This ambiguity significantly impacts the accuracy of 3D pose estimation, as the model must choose from several plausible 3D configurations for given 2D input, often leading to incorrect depth predictions.
*Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

H36M
3DHP
3DPW
Figure 1: Spherical coordinate distribution for the vector connecting the right shoulder to the right elbow across datasets: H36M [20]. 3DHP [21]. 3DPW [22], and BPG, with valid regions highlighted in grey (taken from (23]
To address the first bottleneck, pose augmentation methods for 3D HPE L3 14 proposed. These methods leverage knowledge of labeled (source) data enhance the generalization capability of pose estimators on unseen data. However, relying solely on source data does not fully overcome the inherent limitations, the derived knowledge also carries the same 'popularity bias,' limiting its utility outside laboratory environments. Therefore, previous methods cannot guarantee the generation of out-of-source poses that are crucial for effective real-world applications.
15
16 have been
To overcome these limitations, we propose Biomechanical Pose Generator (BPG) that does not rely source data but utilizes biomechanical knowledge. The concept of 'Normal Range of Motion (NROM) [17] is commonly used in the medical field to describe the standard limits within which a joint can move without discomfort or injury. Our method applies the NROM constraint to the forward kinematic function 18), enabling the autonomous generation of a wide range of plausible 3D poses. This approach enhances the diversity and reliability of pose data.
Furthermore, recognizing that adult human proportions are generally consistent 19 we allow slight variations in proportions accommodate a broader range of body types. Consequently, BPG leverages biomechanical knowledge generate a broader spectrum of feasible 3D poses, thereby avoiding the popularity bias inherent in existing datasets, demonstrated in Figure This methodology provides the essential flexibility and adaptability required for accurately modeling and predicting the diverse range of human poses and activities in real-world environments.
To mitigate the one-to-many problem in 3D HPE, we de- veloped the Binary Depth Coordinates (BDC) that recon- ceptualizes depth estimation as a binary classification task. Instead of using traditional continuous depth coordinates, we limit each joint's depth to two options: front or back.
This simplification is based on geometric principles; once the length connected bones determined, two possible depth positions (front or back) can be calculated based the joint positions. This approach allows us to decompose each 3D pose into three main components: 2D pose, bone length, and binary depth decision. This decomposition effectively reduces depth ambiguity and significantly hances the model's accuracy and robustness for complex poses.
MAPPE mmml
30
Depth ambiguity ratio
As demonstrated in Figure poses with significant depth ambiguity have shown poorer performance in the baseline model, with increased data volumes exacerbating this is- suc. However, when employing the BDC, performance remains consistent regardless of ambiguity, and notably, poses with greater depth ambiguity exhibit improved per- formance as the amount data increases.
Figure 2: The MPJPE (in mm) across 3D HPE models against the depth am- biguity ratio is detailed Sec The baseline* employs BDC on the baseline model and H36M combines source data with additional data synthesized by BPG, expanding the total dataset to ten times the H36M training size.
This paper aims to advance the development of scalable solutions 3D HPE by addressing these issues, ultimately contributing to the creation of more accurate and adaptable technologies. By combining our data generation method with new approach to resolving depth ambiguity, we aim to establish foundation for future research and applications in this dynamic field.

---

2 Biomechanical Pose Generator
To address the limitations of current data collection methods, we propose the Biomechanical Pose Generator (BPG) that does not rely on source data but utilizes biomechanical knowledge. The concept of 'Normal Range of Motion (NROM)' commonly used in the medical field to describe the standard limits within which joint can move without discomfort or injury. Our method applies the NROM constraint to the forward kinematic function, enabling the autonomous generation of a wide range of plausible 3D poses. This approach enhances the diversity and reliability of pose data. Recognizing that adult human proportions are generally consistent, we allow slight variations in proportions to accommodate broader range of body types. Consequently, BPG leverages biomechanical knowledge to generate a broader speetrum of feasible 3D poses, thereby avoiding the popularity bias inherent in existing datasets. This methodology provides the essential flexibility and adaptability required for accurately modeling and predicting the diverse range of human poses and activities in real-world environments.
Satee-kinematic Model. The human body performs complex motions constrained by biomechanical principles. To capture these constraints, we utilize an siteo-kinematic model similar to [18 for generating realistic 3D human poses. This model is based on the biomechanical structure of the human body, including bones, joints, and their respective degrees of freedom (DOF). In the satem-iinematic model, bones can rotate at con- nected joints, and each joint is assigned specific number of DOF to represent natural human-like movements. For instance, elbows and knees each have one degree of rotational freedom (I-DOF), while shoulders, hips, neck, and pelvis each have three degrees of rotational freedom (3-DOF). Unlike typical osteo-kinematic models that use fixed bone lengths, our model allows for adjustments in human body proportions generate poses for a diverse range of subjects. The model includes 17 joints, 16 bone lengths, and 25 angle parameters. This detailed configuration ensures that the generated poses adhere to the nal- ural movement constraints of the human body while allowing for variability in body types. Figure Fillustrates the joints and degrees of freedom in the kinemaii model.
3-DOF
1-DOF
Figure 3: Osteo-kinematic Model with Joints and Degrees of Free- dom.
The esteo-kinematie model uses 41 parameters (lengths and angles) to create pose. Unconstrained random sampling of parameters may generate unnatural 3D poses, such as backward-bent arm or leg, which do not accurately represent natural human postures. To address this, we set boundaries using two predetermined human features: a NROM (17] and human limb bone length ratios 199.. The range of motion (ROM) refers the extent which bones can move through a complete spectrum of movements. Although each individual's 's ROM varies, each joint possesses a NROM representing healthy human movement. NROM encompasses three different axes of rotation, considering the movement direction of the joint. By aligning these axes with the DOF of the aateo-kinematic model, we define an interval [ gupper] for each 7th angle parameter. Hence, these intervals allow a self-created pose to enable natural human motion. Different people exhibit distinct bone lengths. However, the proportions of the human body remain relatively consistency for most individuals. This constancy permits us to set mean value for human limb length ratios I, for the jth bone. To obtain 3D human data with various ratios, we establish ratio range [(1 [(I-a)];, (1+))]] for each jth bone parameter, where E (0, 1) is a predetermined parameter. Using these intervals, j=1 and the angle parameter set = (0.)-11
Postural Noise.
By
we randomly obtain the bone parameter set {4, where B is the number of bones, P is the number of angle parameters, 1, ([(=aa)_ represents the bone parameter of the jth bone, and 0, ~ Uigtower, ppopery represents the 7th angle parameter. Finally, we define these two sets as the postural noise Ñ pose = (1. 0). which can be acquired randomly within boundaries representing the characteristics of ordinary individuals.
Forward Kinematic Function. Following
25
we use the forward kinematic function T to map
postural noise to joint positions:
T pose) -aa
(I)
3

---

where pab E R represents the synthesized 3D pose (see Appendix C.1
Each joint undergoes a local coordinate transformation based on its parameters. The global coordinates of each joint are calculated by multiplying a series of rigid transformation matrices obtained via the parameters from the root point to itself.
Pose Confidence. Although our method utilizes the NROM of individual joints to create realistic human poses, the relative positions of all bones must be considered ensure the plausibility of the pose. For instance, disregarding the bone position can result in a physical intersection, such as legs crossing or hands intersecting with the torso. For this reason, we calculate the bone distance dn,m between the th and m bones. We use the indicator function of the interval [dn. 00), where da.m the pre-calculated relative bone radius [19].
The pose confidence C' for PSD given by
B B-1
C=
II
II
1 D00.
(2)
m=n+1 n=1
If any bones get abnormally close, the confidence value becomes O; otherwise, it remains 1. This measure allows us to avoid poses that are anatomically possible but physically impossible. We eliminate poses with value of 0 using the pose confidence C' and only use poses with a value of 1.
Projection. We project the generated 3D poses into 2D poses using camera parameters, including a rotation matrix, translation vector, focal length, and principal point. Without predetermined camera settings, we generate these parameters within our framework. We sample rotation matrix R E R3x3 from uniform distribution on all 3D rotation matrices and the translation vector T e R3 from a uniform distribution on the interval [Tlower Tupper] that sets a camera location to avoid abnormally close or far positions. Although other methods use a limited rotation distribution, we use a comprehensive distribution to fairly represent all potential camera settings. For simplicity, we decide the focal length to be one. The principal point is fixed at zero, as the translation vector and the principal point serve similar functions. Using the camera parameters c = R,T. we define the projection function P that projects the self-created pose P3D to 2D pose P2D J following Drover et al. [26] as:
((D) = P2D pP2D
The projected 2D pose, along with the original 3D pose, constitutes the final output of our generator. Since this pose is generated solely using biomechanical constraints, this approach ensures the reliability and diversity of the pose and produces data free from popularity bias.
3 Binary Depth Coordinates
Our goal is to mitigate the depth ambiguity that arises when lifting 2D pose to 3D. Typical human pose estimators predict poses in the Camera Space Coordinates (CSC) system, which neglects perspective, often resulting in substantial projection errors. An alternative, the Normalized Device Coordinates (NDC), remaps the viewing frustum to a cube but still struggles with the inherent depth ambiguity in continuous space. The distinction between potential poses in continuous space not merely a matter of measurement but one of understanding the multidimensional nature of human motion, where multiple poses can be correct in different contexts. For instance, a hand extended forwards towards the camera or backwards away from it could represent equally plausible positions in given pose. However, the variance between such positions in continuous space is significant, introducing a level of complexity that can obfuscate the learning process, as the model must recognize and account for fundamentally different yet anatomically feasible poses that share similar 2D projections.
To address this, we introduce the Binary Depth Coordinates (BDC), novel framework that accom- modates the binary nature of depth in images and the continuity of real-world space. The BDC encodes joint's 's position using a tuple (x?? y .,,, s;, P.), where x?? and x?? specify the tth joint's 's location in the image plane. The additional elements 4, representing the bone length, and s,, a binary depth parameter indicating depth relative to the plane of the image, provide the necessary context for a mapping function to estimate the corresponding 3D coordinates (a:) .y. 2a²D) within the CSC. The parameter P: [P PM P?] stores the 3D coordinates of the preceding joint in the skeletal tree, helping to maintain the anatomical structure during transformations. In the BDC, the 3D depth 23p is computed by solving a quadratic equation derived from the geometric constraints of

---

Binary Coordinates (BDC)
Camera Space Coordinate (CSC)
lengths
Quadratic Formula
Image Plane
Lifting model
layer
depths
20
Refined 20 Trajectory
Features
3D pose
Binary depth paramater
Figure 4: Our estimation process using BDC begins by using a fully connected layer to generate the bone lengths 1, refined 2D joint, trajectoriy z0 and the binary depth parameters from the input 2D joint positions. These bone lengths and 2D trajectories are then processed through the quadratic formula to determine possible depth values (23D+ .23D).. The final 3D pose in the camera coordinate system obtained by combining the binary depth parameter with the possible depth values, resulting in the accurate reconstruction of the 3D skeleton.
the joint's location in space. The equation is formulated follows:
(x L - Pm) + (vl 3D PP") (s?
p:)
(4)
where x.? and u are the coordinates in the CSC calculated by back back-projecting a ray from the image plane using the camera parameters:
y22
Ce
3D
3D
2,
f
fy
Here, C2 and C2 denote the camera principal points, while fo and fy represent the focal lengths along the x and y axes, respectively. The solution to the quadratic equation provides two potential values for =@D as _3D+ and =; _3D- These values reflect the dual nature of depth perception in perspective projection. To resolve the ambiguity between these two depths, the binary depth parameter 8; is utilized. This parameter, derived from the BDC, indicates whether the joint is closer to or further from the camera relative its connected joint. Specifically: When s; 1, it is interpreted that the joint is closer the camera, thus selecting z 230- the depth value. Conversely, when 8; -1, the system chooses zi 23D- indicating that the joint is positioned further away from the camera. Our method precisely maps 2D poses to their accurate 3D configurations, leveraging observed and computed depth cues to resolve ambiguities and maintain anatomical consistency. The BDC integrates perspective projection through binary parameters and detailed bone length measurements, facilitating the accurate reconstruction of skeletal structure and overcoming depth ambiguity. By accurately calculating skeletal bone lengths and relationships, BDC enables the precise estimation of 3D poses, demonstrating significant advancement over traditional coordinate systems that fall short such complex scenarios.
3.1 Integration of BDC with Existing 3D HPE Models
The BDC is designed integrate seamlessly into existing 3D human pose estimation frameworks with minimal modifications. This compatibility is achieved by adapting BDC to the final layers of commonly used neural network architectures. By adjusting the dimensionality of the output layer to accommodate our parameters, BDC enables the prediction of 2D poses, bone lengths, and the binary depth parameter 8;, thereby facilitating its integration into existing models. Additionally, predicting independently for each joint may fail to capture the complex interdependencies between different body parts adequately. To address this, we predict 8; collectively for anatomically grouped segments-speciffcally specifically. the left arm, right arm, right leg, left leg, and torso. For example, when
5

---

predicting s; for the left arm, which typically involves three joints, there are 2 = 8 possible combinations of depth configurations. Thus, the model's task is to select one of these eight possible states. To achieve this, we utilize the Gumbel-Softmax, which allows differentiable sampling from discrete probability distributions. This approach enables the model efficiently manage the prediction complexity, enhancing both the accuracy and anatomical consistency of the 3D pose reconstructions. Figure iillustrates the integration of the BDC into the existing lifting model.
3.2 Depth Ambiguity Ratio
This section introduces Depth Ambiguity Ratio (DAR) quantitatively express the uncertainty in depth for each joint in a 3D pose. The DAR quantifies the uncertainty by normalizing the sum of the potential depth values, 2:- 2:: and 2; 230 obtained from the quadratic equation solving for depth. To normalize this sum by dividing it by twice the bone length,
ensure the ratio falls between 0 and denoted as 24,:
J
3D+
3D-
DAR
(6)
24:
This normalized metric serves as universal index to measure depth uncertainty, enhancing the analysis of 3D modeling accuracy and consistency across different implementations.
4 Related Work
3D human pose estimation. Initial approaches 27 28
30
31
32
33
35
36
in 3D human
pose estimation attempted extract 3D poses directly from monocular images without leveraging 2D pose estimations. However, with significant progress in 2D pose estimation techniques [37 38, (3], the method of lifting 2D poses to 3D has emerged as predominant strategy due to its effectiveness and scalability. Our work focuses on advancing these lifting-based frameworks [40 24
particularly in terms of scalability and generalization diverse real-world scenarios.
To enhance the lifting- framework, several complementary approaches have been explored. Leveraging temporal information apporches 442 24 43) across multiple frames is critical area in 3D HPE, as it can improve the accuracy and smoothness of 3D pose predictions. [24] utilized temporal convolutions over sequences of 2D keypoints to model temporal dependencies and reduce ambiguities inherent in single-frane frame predictions. Another effective strategy involves using rich image features. Approaches such [41) incorporate spatial relationships between keypoints directly from image features, enhancing the pose estimation process, Addressing the inherent depth ambiguity in generate multiple plausible 3D pose
3D pose estimation, mupti-hypothesis approaches [12 44 45 predictions and select the best fit. These methods use an evolutionary training data approach to generate multiple hypotheses for 3D poses from monocular images, thereby handling ambiguity and improving robustness.
Cross-domain learning for 3D HPE. Generalizing to challenge. Various strategies have been developed to address this by adapting training methods. For example, (28) introduced method for optimal network architecture selection for different body parts, while [6] enhanced robustness by processing local regions of the 2D pose independently. [47] proposed inference stage optimization to extract distributional knowledge specific target scenarios. Despite these advancements, they often struggle with the inherent limitations of training datasets, and self-supervised training frequently falls short in performance. To further enhance the generalization ability of 3D HPE, extensive research has focused on pose augmentation methods. Given the limitations of existing pose datasets and the challenges in collecting 3D pose data, generating additional 2D-3D pose pairs has proven beneficial. (28) suggested using body-part crossover and mutation operations
unseen poses presents significant
create new 3D poses, though constrained by dependence on the source dataset and fixed mutation ranges. [48] employed reinforcement learning for joint-angle rotation control without original datasets, but this self-supervised approach's 's performance is limited. (I5] and [13] used GANs for pose augmentation, showing improved cross-dataset performance, primarily due to variations in camera viewpoints and human positions rather than increased pose diversity. [4 generated 2D-3D pose pairs from 2D pose labels of the target domain, closely matching its distribution. [16] introduced network designs like interpolation sub-net and body-part grouping net. Despite these advances, generated datasets often lack diversity, remain heavily dependent on the

---

source dataset, and do not adequately consider the pose estimator's adaptability to the generated data, leading to potential training instability with complex poses.
Our work focuses on pose augmentation for 3D HPE. By synthesizing novel 3D poses and integrating them with source poses, it increases pose diversity and mitigates depth ambiguity issues.
5 Experiments
Our experiments were conducted on the Human3.6M (H36M) 20), MPI-INE-3DHP (3DHP) 22).
and 3DPW [22] benchmarks. H36M (20), a widely used benchmark for 3D human pose estimation, includes over 3.6 million indoor video frames captured from four angles with 7 subjects denoted as
S1. S5, S6, S7, S8, S9, and S11. 3DHP 21 consists of 1.3 million images and human pose data captured from 8 subjects using markerless motion capture system in green screen studio with 14 cameras, with six indoor and outdoor sequences evaluated following standard protocols. 3DPW [2 dataset is a challenging collection of in-the-wir wild 2D and 3D data, uniquely captured using a moving camera, unlike other datasets. We evaluate pose accuracy on H36M and 3DPW using mean per joint position error (MPJPE) and procrustes aligned mean per joint position error (PMPJPE), and on 3DHP with MPJPE, percentage of correct keypoints (PCK) within 150mm, and area under the curve (AUC), following established protocols.
5.1 Comparison on Pose Augmentation methods
In this section, Generator (BPG) in comparison to previous Pose Augmentation (PA) methods. Consistent with prior research and previous works 13 16 15), utilize the single-frame VPose (2 network to lift 21) poses to 3D. Our training sources include subjects S1, S5, S6, S7, and S8 from the H36M dataset, and we employ a 16-keypoint human model for evaluation. The volume of synthesized poses matches that of the source data.
the generalizability of the data generated by our Biomechanical Pose
Source-dataset evaluation results. We con- ducted a series of experiments to assess the impact of integrating poses synthesized by the BPG with source data on the performance within the source dataset. For the evaluation, we utilized all available training data from
Table 1: Results on the H36M dataset, using the full
set training data, S9 and S1 1 are the targets.
Method
PA
MPJPE
P-MPJPEJ
VPose 22
52.7
40.6
EvoSkeleton 50
50.9
38.0
PoseAug [3
50.2
39.1
the source dataset and employed HRnet [37 to generate predicted 2D keypoints as input, ensuring consistency with prior approaches. As depicted in Table II our method achieves improvements in MPJPE by 3.3mm and in P-MPJPE by 2.5mm. Furthermore, it is evi- dent that using only the BPG still results in performance improvements over traditional methods. These results suggest that our syn- thesized data can produce a more diverse set of poses compared to traditional pose aug- mentation techniques. Furthermore, we ex- amined the feasibility of achieving high accu- racy with reduced amount of training data. For this evaluation, we utilized subsets the source dataset, specifically S1 S1 + S5.
DH-AUG [15
49.8
38.3
CEE-Net [16]
47.3
36.8
BPG
46,9
36.5
BPG + BDC
44.0
34.3
Table
Results on the H36M dataset, using a subset of the training data. We report MPJPE for evaluation.
Method
PA
S1
SI
S5
VPose 24
65.2
57.9
EvoSkeleton 55
61.5
54.6
PoseAug [13
56.7
51.3
DH-AUG [15]
v
52.2
47.0
CEE-Net 16
51.9
46.7
BPG
51.5
46.1
BPG + BDC
46.8
41.2
and used ground truth 2D keypoints as input, maintaining consistency with previous methods. As illustrated in Table our experiments demonstrate that leveraging medical knowledge generate poses allows our method to significantly enhance performance over existing methods, even with less training data. Notably, the relative performance gains observed with reduced training data were substantially greater than those achieved with the full training dataset. Furthermore, it is evident that using only the BPG still results in performance improvements over traditional methods. This finding underscores the robustness of our approach, which maintains high accuracy and significantly outperforms conventional methods even when the training data is limited.
7

---

GraphMDN GraphMMNe GFosse (Fposee
GraphMEDe
40
Goose
poise
38
90
Niye (tonn
Noyp (nom)
70-
28
50
training
training
Figure Results of added training data on MPJPE GraphMDN 45 and GFpose 44 *; Indicates that the model uses the BDC. Left: source dataset (H36M). Right: cross-domain (3DHP). The x-axis shows the ratio of added training data relative the original dataset size (0 to 8).
Cross-dataset evaluation results. To eval- uate whether the poses synthesized by the BPG contribute to improved generalization performance, we conducted experiments in cross-dataset environment using the 3DPW and MPI-INF-3DHP datasets. For the eval- uation, we utilized all available training data from the source dataset and employed ground truth 2D keypoints as input, consis- tent with previous methods. Our approach demonstrated performance improvements across both datasets. As shown in Table 3 our method improved PCK by 2%, AUC by 7%, and MPJPE by 7. 7.9mm. Notably, signif- icant performance gains were observed in the 3DPW dataset, which features outdoor scenes with moving cameras. As indicated Table 4. our approach achieved improve- ments of T5. 7mm in P-MPJPE. Furthermore, is evident that using only the BPG still re- sults in performance improvements over tra- ditional methods. These results underscore the effectiveness of our synthesized data in enhancing generalization performance, par- ticularly in challenging real- world environments.
Table 3: Cross-dataset evaluation results on 3DHP
dataset.
Method
PA
PCK
AUC1
MPJPFJ
VPose [24]
80.9
42.5
102.3
EvoSkeleton [0
81.2
46.1
99.7
RepNet [51
81.8
54.8
92.5
PoseAug
88.6
577.3
73.0
DH-AUG [5
89.3
57.9
71.2
PoseGU [14
86.3
57.2
75.0
CEE-Net [16
89.9
58.2
69.7
BPG
91.2
58.3
69.1
BPG BDC
91.8
62.3
61.8
Table 4: Cross-dataset evaluation results on 3DPW
dataset.
Method
PA
P-MPJPEJ
MPJPEJ
VPose 24
94.6
125.7
VIBE 27
81.6
122.5
PoseAug
13
V
81.6
119.0
DH-AUG
15
V
79.3
112.8
PoseGU 14
V
92.3
CEE-Net [66]
V
76.8
BPG
V
66.0
96.5
BPG BDC
61.1
90.3
5.2 Analysis on BPG and BDC.
We evaluate the performance of our BDC by comparing poses across varying levels of depth ambiguity. using the proposed depth ambiguity ratio to quantify and analyze the results. In this experiment, utilize multi-hypothesis model [4] for 3D HPE, which are designed to address depth ambiguity. As illustrated in Figure[2] baseline model exhibit decreased performance on poses with high depth ambiguity. This trend is exacerbated when additional training data is introduced. However, when applying the BDC to existing models, performance remains consistent regardless depth ambiguity. Furthermore, adding more data improves performance, even for poses with increased depth ambiguity. This observation highlights the robustness of the BDC mitigating depth ambiguity problem and enhancing overall model accuracy.

---

Table 5: Ablation study with different BPG strategies on testing H36M, where the model was trained exclusively on generated data.
Method
NROM
w/o RR
Ratio
PC
MPJPE
P-MPJPE
Kinematic model
122.3
106.2
Variant A
65.7
56,6
Variant
64.2
54.3
Variant C
V
62.6
53.9
Variant D
J
V
61.9
52.3
BPG
V
59.8
50.6
Scalability of integrating BDC and BPG. In this experiment, we analyze performance trends as the size of the training dataset increases using BPG. We utilize multi-hypothesis model [45] for 3D HPE, designed to address depth ambiguity. By leveraging BPG, we expand the dataset size based on fixed source dataset. As shown in Figure S increasing the dataset size using BPG enhances cross-domain performance. However, for baseline models, performance on the source dataset declines as the dataset size increases, indicating potential overfitting and increased pose complexity. Conversely, integrating BDC with baseline models results in improved cross cross-domain performance and consistent performance gains on the source dataset as the dataset size increases. This highlights the effectiveness BDC in enhancing model robustness and scalability, mitigating the challenges of depth ambiguity, and improving overall performance both source and cross-domain evaluations.
BDC effects on 3D HPE models. We in- vestigated whether utilizing the BDC in ofte-of-thheart (SOTA) 3D HPE models across various tasks could lead to higher accuracy. For the training phase, we utilized all available training data from the source dataset and adhered to each model's s specific training protocols to en- sure consistency and fairness in evalua- tion.
Table 6: Results on H36M dataset. $9,S11 1 are the target.
Task
Method
MPJPE
P-MPJPE
GraphMDN 45
46.2
36.3
GraphMDN
41.7
30.9
Multi-hypo
GFpose 44
36.6
30.5
GFpose*
35.1
29.3
Conpose 41
41.6
33.9
Image features
Conpose*
38.5
33.8
MixSTE [42
40.9
32.6
Multi-frame
MixSTE*
38.3
32.2
As shown in Table 6 our approach consistently improved performance across all tasks in source dataset evaluations. The integration of BDC resulted in measurable improvements in key metrics, demonstrating the robustness and effectiveness of our method in enhancing model accuracy.
Ablation study BPG. In ablation study, evaluated various strategies within framework, including normal range of motion (NROM), rotational restriction (RR), body ratio variation, and Pose Confidence (PC). Only data synthesized with the BPG used training data. The results presented in Table Sable5Shige highlight a consistent trend in performance enhancement as each strategy is successively incorporated. Specifically, NROM, as indicated by Variant A, has the most significant influence on performance. However, both Variants A and C indicate that introducing RR can somewhat diminish performance. The findings from Variants C and emphasize the benefits of incorporating diverse pose proportions for better generalization. Finally, the outcome of Variant D and BPG warn of the pitfalls of including physically improbable postures without PC, suggesting that such postures can hinder effective learning. From these observations, it is clear that the confluence of all strategies is essential for achieving peak performance.
Ablation study on BDC. In this ablation study, compare the performance of various coordi- nate systems on a multi-hypothesis model 45 for 3D HPE, designed address depth ambigu- ity. Specifically, we evaluate the Camera Space Coordinate (CSC), Normalized Device Coordi- nate (NDC), and BDC systems. The results, presented in Table highlight the effectiveness of the BDC system in improving model performance. The CSC and NDC systems serve as baselines for comparison. Additionally, we examine the impact of applying BDC without and with grouped segments (GS). As indicated, the BDC system, particularly when integrated with GS, demonstrates superior performance reducing both MPJPE and P-MPJPE. This underscores the potential of BDC with GS in addressing depth ambiguity and enhancing the overall accuracy of 3D HPE models.
Table 7: Results on H36M dataset. $9,51 are the target.
Method
MPJPE
P-MPIPE
CSC
46.2
36.3
NDC
44.3
32.4
BDC w/o GS
42.6
31.5
BDC
41.7
30.9

---

Conclusion
In this study, we introduced the BDC and the BPG to address key challenges in 3D HPE, particularly the issues of dataset diversity and depth ambiguity. Our approach leverages biomechanical principles and a binary classification framework to generate a wide array of plausible 3D poses and simplify depth estimation, respectively.
We demonstrated that integrating BDC and BPG into state-of-the-arr 3D HPE models significantly improves accuracy and generalization across various datasets and tasks. Experiments the H36M, 3DHP, and 3DPW datasets showed consistent performance enhancements, indicating that our synthe- sized data offers substantial benefits over traditional pose augmentation techniques, leading to more reliable 3D pose estimations in diverse scenarios.
Limitations. Despite the promising results, our approach has certain limitations. The BPG generates poses based on single-frame data only, missing temporal dynamics crucial for motion analysis. Extending BPG to handle multi-frame data could generate more realistic poses. Additionally, while integrating BDC and BP enhances model performance, generating additional data beyond a certain point does not lead to further improvements, suggesting a saturation point. Future work should optimize the amount and type synthetic data to maximize performance without unnecessary computational overhead.
Acknowledgements
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea govermmen((MSIT) (No. RS-2019-11190079, Artificial Intelligence Graduate School Program (Korea University), No. 2021-0-02068, Artificial Intelligence Innovation Hub, No. 2S-2024-00457882 Al Research Hub Project, and No. RS- RS-2024-00336673, AI Technology for Interactive Communication of Language Impaired Individuals).
References
[1] Su, J. S. Zhang, J. Xing, W. Gao, and Q. Tian, "Pose-driven deep convolutional model for person re-identification." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 3960-3969.
S..
[2] Y. Hua, W. Wu, C. Zheng, A. Lu, M. Liu, C. Chen, and S. Wu, "Part aware contrastive learning for self-supervised action recognition." in Proceedings of International Joint Conference on Artificial Intelligence, 2023, pp. 855-863.
[3] Z. Lu, H. Wang. Z. Chang, G. Yang, and H. Shum, "Hard no-box adversarial attack on skeleton- based human action recogmition with seleton--otionninfforee gradient," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4597-4608
[4] H. Yan, Y. Liu, Y. Wei, Z. Li, G. Li, and L. Lin, "Skeletonmae: graph- based masked autoencoder for skeleton sequence traininggg"" in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 5606-5614
[5] C. Zheng, X. Liu. G.-J. Qi. and C. Chen, "Potter: Pooling attention transformer for efficient human mesh recovery." in Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2023, pp. 1611-1620.
[6] C. Zheng, M. Mendieta, T. Yang, G.-J. Qi, and C. Chen, "Feater: An efficient network for human reconstruction via feature map-base transformer." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 13 9394-13954,
[7] V. Guzov, A. Mir, Sattler, and G. Pons-Moll, "Human poseitioning system (hps): 3d human pose estimation and self- ceff-localization in large scenes from body-mounted sensors." in Proceed- ings of the IEEF/VE Conference on Computer Vision and Pattrr Recognition (CVPR), 2021, pp. 2318-4329,
10

---

[8] H. Yi, C.-H. P. Huang, S. Tripathi, L. Hering, J. Thies, and M. J. Black, "Mime: Human Human-aware 3d scene generation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 12965-12976.
[9] M. Yamada, L. Sigal, and M. Raptis, "No bias left behind: Covariate shift adaptation for discriminative 3d pose estimation," " in Proceedings of the European conference on computer vision (ECCV). Springer, 2012, pp. 674-687.
[10] W. Li, H. Liu, H. Tang, P. Wang, and L. Van Gool, "Mhformer: Multi-hypothesis transformer for 3d human pose estimation," in Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13 147-13 156.
[11] X. Ma, J. Su, C. Wang, H. Ci, and Y. Wang, "Context modeling in 3d human pose estimation: A unified perspective." in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2021, pp. 6238-6244 6244.
[12] T. Wehrbein, M. Rudolph, B. Rosenhahn, and B. Wandt, "Probabilistic monocular 3d hu- man pose estimation with normalizing flows," in Proceedingg of the IEEE/CVE International Conference on Computer Vision (ICCV), 2021, pp. 11 199-11 208.
[13] K. Gong, J. Zhang, and J. Feng. "Poscaug: A differentiable pose augmentation framework for 3d human pose estimation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 8575-8584 -5884.
[14] S. Guan, H. Lu, I.. Zhu, and G. Fang, "Posegu: 3d human pose estimation with novel human pose generator and unbiased learning." " Computer Vision and Image Understanding, vol. 233, p. 103715, 2023.
[15] L. Huang, J. Liang, and W. Deng, "Dh-aug: Dh forward kinematics model driven augmentation for 3d human pose estimation," in Proceedings of the European conference on computer vision (ECCV). Springer, 2022, pp. 436-453.
[16] H. Li and C.-M. Pun, "Cee-net: complementary end-to-end network for 3d human pose genera- tion and estimation," in Proc. AAAI Conf. Artif. Intell., vol. 37, no. 1, 2023, pp. 1305-1313.
[17] Soucie, C. Wang. A. Forsyth, S. Funk, M. Denny, K. Roach. D. Boone, and H. T. C. Network. "Range of motion mcasurements: reference values and database for comparison studies," Haemophilia, vol. 17, no. 3, pp. 500-507, 2011.
[18] T. Koritnik, T. Bajd, and M. Munih, "A simple kinematic model of human body for virtual environments," in Advances in Robot Kinematics: Motion in Man and Machine, 2010, pp. 401-408.
[19] A. Pietak, S. Ma. C. W. Beck, and M. D. Stringer, "Fundamental ratios and logarithmic periodicity in human limb bones," Journal of Anatomy, vol. 222, no. 5, pp. 526-537, 2013.
[20] C. lonescu, D. Papava, V. Olaru, and C. Sminchisescu, "Human3.6m" Large scale datasets and predictive methods for 3d human sensing in natural environments," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 7, pp. 3325-1339, 2013.
[21] D. Mehta, H. Rhodin, D. Casas, P. Fua, O. Sotnychenko, W. Xu, and C. Theobalt, "Monocular 3d human pose estimation in the wild using improved enn supervision," in Proc. Int. Conf. 3D Vis., 2017, pp. 506-516.
[22] T. Von Marcard, R. Henschel, M. J. Black, B. Rosenhahn, and G. Pons-Moll, "Recovering accurate 3d human pose in the wild using imus and a moving camera," in Proceedings of the European conference on computer vision (ECCV), 2018 pp. 601-617.
[23] I. Akhter and J. Black, "Pose-conditioned joint angle limits for 3d human pose reconstruction." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1446-1455.
[24] D. Pavllo, C. Feichtenhofer, D. Grangier, and M. Auli, "3d human pose estimation in video with temporal convolutions and semi-supervised training," Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7753-7762.
11

---

[25] X. Zhou, X. Sun, W. Zhang, S. Liang, and Y. Wei, "Deep kinematic pose regression," in Proceedings of the European conference on computer vision (ECCV) w, 2016, pp. 186-201.
[26] D. Drover, R. MV, C.-H. Chen, A. Agrawal, A. Tyagi, and C. Phuoc Huynh, "Can 3d pose be learned from 2d projections alone?" Proceedings of the European conference on computer vision (ECCV) 2018, pp. 0-0.
[27] M. Kocabas, Athanasiou, and M. J. Black, "Vibe: Video inference for human body pose and shape estimation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 5253-5263.
[28] S. Li, I.. Ke, K. Pratama, Y.W.. Tai, C.-K. Tang, and K.-T. Cheng, "Cascaded deep monocular 3d human pose estimation with evolutionary training data," " in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2020, pp. 6173-6183,
[29] D. Mehta, S. Sridhar, O. Sotnychenko, H. Rhodin, M. Shafici, H.-P. Seidel, W. Xu, D. Casas, and C. Theobalt, "Vnect: Real-time 3d human pose estimation with a single rgb camera," ACM Transactions on Graphics, vol. 36, no. 4, pp. 1-14, 2017.
[30] M. R. I. Hossain and J. J. Little, "Exploiting temporal information for 3d human pose estimation," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 68-84.
[31] X. Sun, B. Xiao, F. Wei, S. Liang, and Y. Wei, "Integral human pose regression," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 529-545.
[32] D. Tome, C. Russell, and L. Agapito, "Lifting from the deep: Convolutional 3d pose estimation from a single image," in Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2500-2509.
[33] J. Xu, Z. Yu, B. Ni, J. Yang. X. Yang, and W. Zhang. "Deep kinematics analysis for monocular 3d human pose estimation." in Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2020, pp. 899-908.
[34] K. Zhou, X. Han, N. Jiang, K. Jia, and J. Lu, "Hemlets pose: Learning part-centric heatmap triplets for accurate 3d human pose estimation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2344-2353.
[35] G.-J. Yang, J.-H. Kim, and S.-W. Lee, "Geometry-driven self-supervision for 3d human pose estimation." " Neural Networks, vol. 174, p. 106237, 2024.
[36] G.-J. Yang, J. Kim, H.-W. Kim, G.-H. Lee, and S.-W. Lee, "Egpose: Explicit and geometric self-supervision for 3d human pose estimation," Procedia Computer Science, vol. 222, pp. 387-396, 2023.
[37] K. Sun, B. Xiao, D. Liu, and Wang. "Deep high-resolution representation learning for human pose estimation." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5693-5703.
[38] Y. Yuan, R. Fu, L. Huang, W. Lin, C. Zhanng X. Chen, and J. Wang, "Hrformer: High-resolution vision transformer for dense predict," Advances in neural information processing systems, vol. 34, pp. 7281-7293, 2021.
[39] Y. Xu, J. Zhang, Q. Zhang, and Tao, "Vitpose: Simple vision transformer baselines for human pose estimation," Advances in Neural Information Processing Systems, vol. 35, pp. 38 571-38 584, 2022.
[40] J. Martinez, R. Hossain, J. Romero, and Little, "A simple yet effective baseline for 3d human pose estimation," " in Proceedings of IEEF/CVF International Conference on Computer Vision (ICCV), 2017, pp. 2640-2649.
[41] Q. Zhao, C. Zheng, M. Liu, and C. Chen, "A single 2d pose with context is worth hundreds for 3d human pose estimation," Advances in Neural Information Processing Systems, vol. 36, 2024.
12

---

[42] J. Zhang, Z. Tu, J. Yang, Y. Chen, and J. Yuan, "Mixste: Seq2seq mixed spatio- tempool encoder for 3d human pose estimation in video." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13 232-13322...
[43] C. Zheng, S. Zhu, M. Mendieta, T. Yang, C. Chen, and Z. Ding, "3d human pose estimation with spatial and temporal transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 11 656-11 665.
[44] H. Ci, M. Wu, W. Zhu, X. Ma, H. Dong, F. Zhong, and Y. Wang. "Gfpose: Learning 3d human pose prior with gradient fields," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 4800-4810.
[45] T. Oikarinen, D. Hannah, and S. Kazerounian, "Graphmdn: Leveraging graph structure and deep learning to solve inverse problems," in International Joint Conference on Neural Networks (IJCNN). IEEE, 2021, pp. 1-9.
[46] A. Zeng, X. Sun, Huang, M. Liu, Q. Xu, and S. Lin, "Smnet: Improving generalization in 3d human pose estimation with conference on computer vision (ECCV), 2020, pp. 507-523.
approach," in Proceedings of the European
[47] J. Zhang. X. Nic, and J. Feng, "Inference stage optimization for cross-scenario 3d human pose estimation," " Advances in Neural Information Processing Systems, vol. 33, pp. 2408-24024 2020.
[48] K. Gong, B. Li, J. Zhang. T. Wang, J. Huang, M. B. Mi, J. Feng, and X. Wang, "Posetriplet: Co-evolving 3d human pose estimation, imitation, and hallucination under self-supervision" " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 11 11011-1100 027.
[49] M. Gholami, B. Wandt, H. Rhodin, R. Ward, and Z. J. Wang, "Adaptpose: Cross-dataset adaptation for 3d human pose estimation by learnable motion generation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 035-13 085.
[50] S. Li, L. Ke, K. Pratama, Y.-W. Tai, C.-K. Tang, and K.-T. -T. Cheng, "Cascaded deep monocular 3d human pose estimation with evolutionary training data," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 6173-6183.
[51] B. Wandt and B. Rosenhahn, "Repnet: Weakly supervised training of an adversarial reprojection network for 3d human pose estimation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7782-7791.
13

---

Appendix
A Implementation Details
For the comparison of pose augmentation methods, without BDC, we utilize the single-frame version of VPose [24] the pose estimator. When applying BDC, we introduce four ID convolution layers at the end of VPose 244 to estimate parameters such root depth, 2D pose, bone length, and binary depth. In the case of the multi-hypothesis model [4], we create four fully connected layers to estimate the same parameters: root depth, 2D pose, bone length, and binary depth. The training loss remains consistent with the losses used the original models. Specifically, VPose [24] employs
Mean Squared Error Loss (MSELoss), while the multi-hypothesis model 45 uses Mixture Density Network (MDN) loss.
We train the models on the H36M dataset for 50 epochs with a batch size of 1024. All components optimized using the Adam optimizer with an initial learning rate of 0.001, which linearly decays over time. For the BPG, training is conducted on a single Nvidia GTX 3090 Ti GPU with a batch size of 1024 for 100 epochs, taking approximately two days complete the experiments. The Adam optimizer is utilized with an initial learning rate of 3e-4, which decreases linearly during the training process. To stabilize the training process and enhance performance, we pre-train the models on synthetic data before end-to-end training. This pre-training phase ensures that the models are better initialized and capable of more robust learning during comprehensive training phase.
B Visualization
We conducted extensive experiments to evaluate the impact our synthesized data using BPG on HPE models. Figure that our synthesized poses occupy significantly larger area compared to the existing dataset. However, this did not immediately translate into performance improvement; rather, it resulted in performance decline. To identify the cause of this decline, examined the per- frame performance the baseline model's poses F The results indicated that performance degradation occurred in most scenarios. Nonetheless, there was promising aspect: in environments where the baseline model failed, our approach showed performance improvement. Furthermore, when integrating the BDC into the baseline model, observed performance enhancement across almost all regions. In Figure 5 BB the visualization of the areas where performance declined after pose augmentation revealed that the results of complex poses appeared with flattened depth. This indicates that the increased pose diversity led to heightened depth ambiguity, causing predictions to average towards the center. However, by utilizing our BDC, observed robust predictions even with increased pose diversity, indirectly mitigating depth ambiguity.
BPG
BPQ
(4) Right to right wrist
(b)
(c) Right right
hip knee
Figure 6: Spherical coordinate distribution for the vector across datasets: H36M 20 and BPG, with valid regions highlighted in grey (taken from 23 )
14

---

A
Baseline + H36M
140
Baseline + H36M*
Baselinet H36M
Sample point vertical line
120
100
40
20
400
600
800
1000
1200
1400
B
Figure A Frame-wise comparison H36M. Bascline* employs BDC on the Baseline model [45] and #36M* combines source data with additional data synthesized by BPG, expanding the total dataset times the H36M training size. B: The result of visualizing the poses the vertical line in Figure
C Formula details
C.1 Forward Kinematics Function.
T(N pose {t.0)) takes joint variable vector 0 [01, .a.0.... 0,] and link length vector 1 1= [1, .....,..... []] as input, and returns transformation matrix representing the position and attitude of the end effector. The mathematical representation of this is
(R,01)) 0
d(0,1)
T 1)
where R(0,1) I) is 3 3 rotation matrix, and d(0, 1) is 3 position vector. This matrix represents the position and attitude of the end effector in the robot's reference frame.
The forward kinematics function is defined as follows
T(O, = (0,,,, .. (,, 4.)
where (O,, the individual transformation matrix for each joint This transformation matrix represents the rotation and displacement of each joint, and typically has the form
cos) sin(O.) cos(as) sin(0,) sin(as)
sin(0,) cost (,) cos(as)
O
l de sin(a;) di cos(as)
sin (a;) (os(
A; (0,
cos 0 sin(a.)
1
15

---

S9A
Figure Qualitative results of ground truth poses (blue) and predicted poses (red) on various datasets, H36M (rows 1, 2), 3DHP (row 3), and 3DPW (rows 4, 5).
Figure 9: Visualization of synthesized poses with our BPG.
16

---

where 4, is the length of link i, and d; and a; are the Denavit-Hartenberg (DH) parameters of the robot.
So, the full formula for the forward kinematics function is
a T 1) II (0,44)
This formula represents sequentially multiplying the individual transformation matrices for each joint compute the overall transformation matrix. This allows us get the position and pose of the end effector.
The forward kinematics function T,0, 1) takes a joint variable vector 10,, 02. .. On] and as input, and returns transformation matrix representing the
link length vector 1 (1,,,,...,4 In] position and attitude of the end effector. The mathematical representation of this is
1 0
d(0,1)
T(0.1) 1)
where R(0, 1) is 3 x 3 rotation matrix, and d 1) is 3 x 1 position vector. This matrix represents the position and attitude of the end effector in the robot's reference frame.
The forward kinematics function is defined as follows
T(e, 1) (1,
A, (n)
where (0,O;,4) ) is the individual transformation matrix for each joint i. This transformation matrix represents the rotation and displacement each joint, and typically has the form
cos( sin(O:) cos(a:) sin(O,) sin(a;)
sin(0,) cos(O,) coss(as)) cos (.) sin(a.)
O sas) cos(a;)
4 - de sin(as) d cos(as)
A,(0,44
0
where 4 is the length of link i, and d, and ag the Denavit-Hartenherg (DH) parameters of the robot.
So, the full formula for the forward kinematics function is
T 1) = II A, (0, = PaD
1=1
This formula represents sequentially multiplying the individual transformation matrices for each joint to compute the overall transformation matrix. This allows us to get the position and pose of the end effector.
In summary, the forward kinematics function T maps the initial joint positions and link lengths (which constitute the kinematic model) to the 3D position and orientation of the end effector:
C.2 The Indicator Function
Let X be set, and let A be subset of X. The indicator function x of subset of set X is defined as:
[ ifreA A
1 (x)x
i
17

---

C.3 Camera Projection Function.
The camera projection function P maps a 3D point X [X, Z, in homogeneous coordinates 2D point x [u, v, 1] on the image plane. The mathematical representation of this function is
x
where P the camera projection matrix, defined
P K[R[T)
Here, K is the camera intrinsic matrix, R is the rotation matrix representing the camera orientation, and I' is the translation vector representing the camera position. The intrinsic matrix K is given by
fx
0
4
K =
fy
Cy
0
1
where fa and fy are the focal lengths in the x and y directions, respectively, and ca and cy are the coordinates of the principal point.
The full camera projection function is thus represented as
x K[
This formula maps a 3D point X the world coordinate system to a 2D point x on the image plane, taking into account the camera's intrinsic parameters and its pose in the world.
C.4 Solving the quadratic equation for 2.D.
Given the equation:
(x)
2,
P.m)
(23
p.)2
= I2
and the transformations:
x
Cx
3D
y
3D
y
Cy 3D
74
fz
fg
we substitute these into the original equation:
2D
Cx
24
P.
2D
Cy.
30
pM
+
+22
p.)2 =1
fy
Expanding and rearranging terms, we get quadratic equation in 2??: 3D
A(23D)2 + B2
=C=
where:
(x)
Cp )2
2D c2) )2
+1
f2
P2
(n 2D
PF
(u
P. P.
B -2
fz
fg
(PT) + (P!)2 + (P.)2 12
The solutions for 3D are given by the quadratic formula:
B BB² 4AC
3D
2A
18

---

Therefore, the two roots are:
B + VB² 4AC
3D+
2A
-B VB² 4AC
3D
2A
D
Pseudo-Code
Algorithm 1 Biomechanical Pose Generator (BPG)
1: Input: Normal Range Motion (NROM) intervals fower and 0 upper. mean limb length ratios 1,, proportion range parameter a, Camera intervals Tlower and 7upper
2: Output: Generated 3D pose P'D. projected 2D pose P2D
3: Initialize pose parameters Mpse f (
4: for each bone j do
5: Sample I ~ Lu((1 (1++
6: N -ppse-appenn(())
7: end for
8: for each joint do
9: Sample 0g ~ U ((Boower, Oupper,
10: Npose-append(0),
11: end for
P3D TTe)N))
13: for each bone pair (n, m)) do
Calculate dm.m
15: if dn < dn.m then
16:
Reject pose P'D and resample
end if
18: end for
Sample rotation matrix R ~ ((S((3))
20: Sample translation vector T E R ~ U(Tlowrr, Trppper)
Set focal length f =
22: Set principal point Pp = (0
23: Define camera parameters e = { RR, TT, (sPp)
24: Project P3D to 2D pose PzD using the projection function p(c, PaD)
return 2p. P3D
19

---

Algorithm 2 Binary Depth Coordinates (BDC)
1: Input: 2D joint positions Pan, bone lengths I, binary depth parameter camera parameters
(Cus far
fwr fy). root depth 23D, Skeleton Tree
2: Output: 3D joint positions P3D
3: Initialize P3D + {
4: Compute the 3D coordinates of the root joint:
5: 3D
2D -c ,3D
20
6: y0
Mo
fg
c
20
7: Append (ar
1/
3D 3D P30
20
8: for each joint i in tree order 7 do
9:
Compute the 3D coordinates a
M using:
r Cz
x
(7)
Sx
y1
Cy.
(8)
fy
10: Solve the quadratic equation for 2iD:
(o? P2. pa)em))))))) )2 + (v? Ps y.ppreat(e)) )2 + (z,3 -P (=..prremss)) ) 17
(9)
11:
This yields two potential values: 3D+ and 3D-
Use the binary depth parameter s; select the correct depth:
3D+
3D
8; =
(10)
if s; = -1
Compute x?D,,??
.9?" 30 with the selected
23D:
x
3D
Cz 3D
(11)
fx
yi
Cy 3D
2
(12)
fg
14:
Append (x:3), yD 2,2) to Pap
15: end for
16: return PaD
20

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the
relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims in the abstract and introduction clearly state the contributions of introducing the Biomechanical Pose Generator (BPG) and Binary Depth Coordinates (BDC) to address scalability and depth ambiguity in 3D Human Pose Estimation (HPE)
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
21

---

Justification:
The paper discusses the limitations, such the BPG generating poses based on single-frame frame data only and the saturation point of generating additional data beyond certain point.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results requiring proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results. the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides detailed experimental results, including data on various datasets like H36M, 3DHP, and 3DPW, and discusses performance metrics and methods.
22

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel urchitecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
The code can be found in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
hhipss.c//ublic/uuddds htwp:////eooeSeeesooeoooooo
www.obiiiiiiioooooooiiooo for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
23

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies training and testing details, including data splits and evaluation metrics across different datasets and tasks.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: The paper includes tables and figures that report performance metrics with appropriate statistical measures, such MPJPE and P-MPJPE, though specific error bars are not mentioned.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computer resources used are provided in Appendix A

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: There is no indication that the research violates the NeurIPS Code of Ethics. The methodologies used align with ethical guidelines for research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper mentions the broader impacts, focusing on the improvements in
3D pose estimation which can benefit various applications such as virtual reality and action recognition.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
25

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not pose risks related to the misuse of data or models, and no specific safeguards are mentioned.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes)
Justification: The paper properly cites sources and references for datasets and methods used throughout the research.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper documents new methodologies like the BPG and BDC, providing sufficient details on their development and application.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
26

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjeets
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with human subjects that would require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
27


Reviews:
[{"review_id": "1xmTXkfYvC", "reviewer": "Reviewer_1aXZ", "summary": "This paper address the task of 3D Human Pose Estimation from monocular RGB. The authors make two main contributions: The Biomechanical Pose Generator (BPG) and the Binary Depth Coordinates (BDC). BPG is a 3D human pose generator that leverages the \"Normal Range of Motion\" (NROM) that is used in the medical field to describe standard biomechanical limitations. With it, BPG is capable of generating biomechanically sound 3D human poses by randomly sampling joint angles and bones that lie within a certain ratio to each other.\nBDC is a coordinate system that decompose a 3D pose into constituents. Specifically, it decomposes it into the 2D coordinate, bone length, a binary depth parameter indicating the closeness to the image plane as well as the 3D coordinates of the parent joint. This decomposition, so the authors claim, allows models to better deal with depth ambiguity. \nExperimental results demonstrate that the proposed approach achieves better performance over the compared related work on a variety of datasets (cf. Tbl 1-4). Ablative studies demonstrate that BDC helps keep performance steady even in the face of larger depth ambiguity (Tbl. 5) and that related work can benefit as well from switching to the proposed coordinates (Tbl 6.)", "questions": "- How would BPG compare to randomly sampling SMPL poses?", "limitations": "The authors address limitations of their methods, such as not taking temporal dynamics into account.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors properly motivate and evaluate their approach. Depth ambiguity in monocular RGB is a challenging problem to address. I particularly liked Tbl. 5 that demonstrated that BDC is capable of handling even larger depth ambiguities.\n- The paper was easy to digest and understand.\n- One of the main strength of this paper is that BDC can be combined with other related work, yielding improvements (Tbl. 6)", "weaknesses": "- My biggest concern about the paper is that BDC is very similar conceptually to \"Hand Pose Estimation via Latent 2.5D Heatmap\nRegression\", Iqbal et al., ECCV'18. Yet there is no mention of the paper, let alone any comparisons. The mentioned paper also addresses with depth ambiguity by decomposing the 3D pose into 2D pose and a root-relative depth vector. Addressing the differences, performing comparisons with this approach would better contextualize as well as strengthen the contribution of the paper.\n- BPG shows to improve performance by improving the 2D to 3D lifting component. Yet, it's contribution is rather sparse, as it essentially amount to performing forward kinematics on bounded joint angle and bone lengths. It does not take into consideration statistics on poses. Certain poses are more common, due to them corresponding to actual human movement patterns (such as walking) that are affected by gravity. Randomly sampling poses without taking such statistics into consideration may generate a range of synthetic poses that are unrealistic, leading to non-optimal improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed and insightful review. Your feedback on the similarities to existing research and suggestions for further comparisons are greatly appreciated and will help strengthen our work.\n\n## How does BDC differ from \"Hand Pose Estimation via Latent 2.5D Heatmap Regression\" by Iqbal et al. (ECCV'18)?\n\nWe appreciate the concerns regarding the conceptual similarity of BDC. However, we would like to emphasize that our approach is differentiated from existing methods by introducing several novel components specifically designed to address depth ambiguity.\nWe discussed this thoroughly in the general response under \"More Need for Comparison to Similar Works on BDC.”\n\n## BPG's contribution seems sparse, lacking pose statistics consideration\n\n### Response to Concerns on BPG's Consideration of Pose Statistics\n\nThank you for raising concerns about BPG's consideration of pose statistics. While we acknowledge the potential issues, we believe the contribution of BPG extends beyond merely improving the 2D to 3D lifting component and addresses fundamental biases present in existing training datasets.\n\n#### Addressing Sparse Contribution and Forward Kinematics\n- **Beyond Forward Kinematics**: BPG is not limited to performing forward kinematics on bounded joint angles and bone lengths. Instead, it incorporates Normal Range of Motion (NROM) to ensure biomechanically plausible poses, which existing methods fail to generate. This approach enhances the realism and applicability of synthesized poses, moving beyond the constraints of traditional forward kinematics.\n\n#### Consideration of Pose Statistics\n- **Intentional Exclusion of Pose Statistics**: We deliberately chose not to incorporate pose statistics from existing datasets to avoid reinforcing inherent biases. These datasets often include everyday poses that do not adequately challenge the model during training and testing. By randomly sampling poses, BPG mitigates these biases, leading to a more robust training environment.\n  \n  - **Example**: Existing datasets dominated by walking and standing poses could lead to a model that performs well in such common scenarios but poorly in less frequent, more complex poses. Our approach aims to prevent this by ensuring diverse and unbiased pose sampling.\n\n#### Mitigating Unrealistic Pose Generation\n- **Ensuring Realism with NROM and Pose Confidence**: While random sampling might raise concerns about unrealistic poses, we have introduced NROM and Pose Confidence metrics to ensure physical and biomechanical validity. Figures 1 and supplementary Figure 5 illustrate how BPG effectively removes biases and avoids generating implausible synthetic poses.\n\n#### Evidence of Performance Improvement\n- **Empirical Results**: The performance improvements of BPG are well-documented in Table 1 and Table 2, where even without the inclusion of pose statistics, BPG outperforms existing methodologies. This indicates that our approach not only enhances 2D to 3D lifting but also contributes to generating more varied and realistic poses.\n  \n  - **Table 4 Insights**: Specifically, BPG demonstrates significant performance gains in challenging environments such as 3DPW, underscoring its robustness and generalization capabilities across different scenarios.\n\nIn summary, while BPG does not incorporate traditional pose statistics, its innovative use of NROM and random sampling addresses inherent biases and enhances the realism of generated poses. This approach not only improves performance but also ensures the model's applicability to a wider range of real-world scenarios, ultimately contributing to more robust and generalized training outcomes.\n\n\n## How would BPG compare to randomly sampling SMPL poses?\n\nWe appreciate your insightful question regarding the comparison between BPG and randomly sampling SMPL poses. When generating poses using random sampling with SMPL, we observed a performance metric of 120.4, which aligns closely with the performance of the Kinematic model without the Normal Range of Motion (NROM) constraints, as shown in Table 8. This result underscores the critical role of NROM in our method.\n\nWithout the biomechanical constraints provided by NROM, both a simple Kinematic model and a sophisticated model like SMPL yield similar performance outcomes. This similarity suggests that the primary advantage of BPG lies in its incorporation of NROM, which ensures that the generated poses are biomechanically plausible and reflect realistic human motion patterns.\n\nFurthermore, even when NROM is incorporated into SMPL, the performance remains comparable to existing methodologies. However, this approach significantly increases computational resources and processing time. Integrating NROM directly into SMPL results in a much higher computational cost and longer processing times, making it less efficient for practical applications.\n\nThank you for bringing this question to our attention, as it highlights the fundamental strengths of our approach.\n\n## **Concluding Remarks**\n\nWe will carefully address the concerns raised by reviewer 1aXZ and incorporate their valuable suggestions in our rebuttal to improve our manuscript. We hope that these responses will adequately resolve the concerns and lead to a favorable reevaluation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4ivtAGZGqM", "reviewer": "Reviewer_9ZcB", "summary": "The authors propose a 3D human pose estimation framework that incorporates data augmentation and depth ordering information. The main contributions are two-fold: First, the proposed Biomechanical Pose Generator (BPG) generates plausible body poses based on kinematic constraints, which is used for data augmentation. Second, the Binary Depth Coordinates (BDC) disambiguate the projective depth of each joint by classifying whether the joints are positioned towards or away from the camera. The proposed framework achieved state-of-the-art performance in single-frame 3D human pose estimation settings.", "questions": "Method\n- In line 155, the focal length of the camera matrix is set to 1 for BPG, is it also the case for the datasets used? Or the camera matrix provided in the datasets are used?\n- In line 179, what is the meaning of \"depth relative to the plane of the image\". I guess $s_i$ is the depth relative to the preceding joint not the image plane.\n\nExperiments\n- Why did the authors use different baseline architectures in Sec. 5.1 and 5.2?\n- How much portion of augmented data from BPG used for experiments in Sec. 5.1?\n- Given that using only BPG increases the error in Fig. 4 left, how could it be possible to achieve better performance in Table 1 and 2 when only BPG is used?\n- Why didn't the authors use BPG in Table 6?\n- What is the difference between Variant E and BPG in Table 8?", "limitations": "Suggestions\n- In Fig. 4, it should be clearly stated what * means. It would be better to use GFpose+BDC instead of GFpose*.\n- More detailed explanation about how $T$ in Eq (1) and ${d}_{m,n}$ in Eq (2) are formulated would clarify the methods.\n\nTypos\n- Line 185, by the projection from -> by back-projecting a ray from\n- Line 212, duplicated sentences.\n- Line 115, to peed -> of", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The proposed method achieves state-of-the-art results in various 3D HPE datasets.\n- The effect of data augmentation is validated in cross-domain learning settings.", "weaknesses": "My major concern lies on the novelty of the contribution.\n\n- There are numerous research papers that regularize 3D human pose based on kinematic constraints. The authors did not clarify the distinctiveness of BPG from these conventional works, except for stating that BPG achieved better performance. An analysis showing how the proposed BPG generates more plausible poses compared to previous augmentation methods is required, either by displaying the generated poses or by showing qualitative estimation results.\n- The concept of BDC is similar to [1] which learns ordinal depth information. The authors should cite the paper and discuss the difference.\n\nThe paper also contains\tambiguously explained parts or lacks details about their methods. Please refer to Questions section.\n\n[1] G. Pavlakos et al., \"Ordinal depth supervision for 3d human pose estimation\", CVPR 2018", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough and thoughtful review. Your comments on the novelty and clarity of our contributions, as well as your specific questions, are extremely valuable and will guide us in refining our manuscript.\n\n## **How does BPG differ from existing kinematic constraint-based methods?**\n\nThank you for your insightful question. Our Biomechanical Pose Generator (BPG) differentiates itself from existing kinematic constraint-based methods in several key aspects.\n\nFirstly, BPG employs a new biomechanical approach that adheres to realistic human movement constraints, unlike existing methods that often focus solely on statistical kinematic constraints of the given dataset. While existing methods also use kinematic constraints, they depend on source datasets, which can introduce biases due to the limited and specific conditions under which the data was collected. This dependency can lead to biased pose generation that is difficult to eliminate. In contrast, BPG utilizes the biomechanical principle of NROM (Normalized Residual Orthogonality Method) independently of datasets, ensuring diverse and unbiased pose sampling.\n\nSecondly, the concept of a biomechanical pose generator that does not rely on source datasets is novel. While many methods require pre-existing datasets to generate poses, BPG can automatically generate diverse and reliable human poses without being constrained by dataset variability and bias.\n\nThe performance improvement of BPG is due to the removal of biases in existing data, achieved by integrating NROM without relying on source datasets.\n\nTo further illustrate BPG's advantages, qualitative estimation results are shown in Figure 7 of the appendix, and the generated poses are depicted in Figure 8. Additionally, the general response PDF provides a comparison of pose distributions with existing augmentation methods. Existing methods follow the limited distribution of their datasets, whereas our method ensures robustness by eliminating bias. We believe these details clearly demonstrate the diversity and realism of poses generated by BPG compared to existing methods.\n\n\n## **BDC concept is similar to [1]. The authors should cite and discuss the difference.**\n\nWe appreciate the concerns regarding the conceptual similarity of BDC. However, we would like to emphasize that our approach is differentiated from existing methods by introducing several novel components specifically designed to address depth ambiguity.\nWe discussed this thoroughly in the general response under \"More Need for Comparison to Similar Works on BDC.”\n\n## **Questions**\n\n> In line 155, the focal length of the camera matrix is set to 1 for BPG, is it also the case for the datasets used? Or the camera matrix provided in the datasets are used?\n> \n\nWe appreciate the reviewer's attention to detail. We used the camera matrix provided in the datasets, not the simplified focal length of 1 as set for BPG.\n\n> In line 179, what is the meaning of \"depth relative to the plane of the image\". I guess is the depth relative to the preceding joint not the image plane.\n> \n\nThank you for pointing out this ambiguity. You are correct; it should indeed be the depth relative to the preceding joint. We appreciate your careful reading and will correct this in the manuscript.\n\n> Why did the authors use different baseline architectures in Sec. 5.1 and 5.2?\n> \n\nIn Section 5.1, we employed a common architecture typically used in augmentation work to facilitate a fair comparison with other augmentation methods. In contrast, Section 5.2 utilizes multi-hypothesis models, which are specifically designed to address depth ambiguity. This distinction was made to highlight how our methodology better handles depth ambiguity compared to traditional approaches.\n\n> How much portion of augmented data from BPG used for experiments in Sec. 5.1?\n> \n\nAs indicated in line 282, the augmentation ratio was set to 1. This means the amount of augmented data was equal to the original data.\n\n> Given that using only BPG increases the error in Fig. 4 left, how could it be possible to achieve better performance in Table 1 and 2 when only BPG is used?\n> \n\nThis is a valid concern. We addressed this question in detail in the general response under \"Confusion on Experimental Results.\"\n\n> Why didn't the authors use BPG in Table 6?\n> \n\nThe results for multi-hypothesis models are presented in Figure 4. The integration of image features with BPG was not feasible due to the inability to generate realistic images. Additionally, as described in the Limitations section of the paper, BPG cannot generate temporal poses, which limits its application in scenarios requiring sequential data.\n\n> What is the difference between Variant E and BPG in Table 8?\n> \n\nThere is a typographical error in the manuscript. Variant E involves using only NROM and PC. Additionally, the results in Table 8 were trained using only BPG-generated data and were evaluated using the H36M test set.\n\n## **Suggestions**\n\n> In Fig. 4, it should be clearly stated what * means. It would be better to use GFpose+BDC instead of GFpose*.\n> \n\nThank you for your suggestion. We will take that into account and ensure that the figure is updated to use \"GFpose+BDC\" instead of \"GFpose*\" for better clarity.\n\n> More detailed explanation about how $T$ in Eq (1) and 𝑑𝑚,𝑛 in Eq (2) are formulated would clarify the methods.\n> \n\nWe appreciate your request for more details. A more detailed description of #T# in Ep (1) and 𝑑𝑚,𝑛 in Eq (2) are formulated would clarify the methods.\n\n## Typos\n\nThank you for pointing out these errors. We will correct them to improve the clarity and accuracy of our manuscript.\n\n## **Concluding Remarks**\n\nWe will diligently address the feedback from reviewer 9ZcB and integrate their valuable suggestions to enhance our manuscript. We trust that our detailed responses will resolve the highlighted concerns and lead to a positive reevaluation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aegUTJu6me", "reviewer": "Reviewer_696C", "summary": "This paper introduces two components aimed at addressing challenges in 3D human pose estimation, specifically in terms of scalability and generalization. The authors propose a Biomechanical Pose Generator (BPG), which incorporates biomechanical principles to generate plausible new 3D poses. They also introduce Binary Depth Coordinates (BDC), a component designed to mitigate the depth ambiguity encountered when lifting a 2D pose to 3D. The paper includes ablation studies to demonstrate the impact of each component, and compares these new approaches to existing pose augmentation methods.", "questions": "As I mentioned in the weaknesses section, I would like to learn more about the effect of adding more synthesized poses to Human 3.6M and evaluating on the same source of data as currently the results in the Table 1 and Figure 4 are a bit confusing to me.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper’s focus on addressing the challenge of limited datasets and enhancing the generalizability of the method is interesting and to the best of my knowledge the idea of biomechanical pose generator which does not rely on a source dataset is novel. Also, the authors’ attention to the depth ambiguity in 3D pose estimation from a single image adds a value to the field. The authors have conducted comprehensive experiments and ablation studies, which provide valuable insights into the effectiveness of the proposed components. The inclusion of cross-dataset evaluation is crucial, as it allows for a robust assessment of the Biomechanical Pose Generator (BPG) component’s effectiveness.", "weaknesses": "1- The paper is generally well-written, but some parts could be clearer. Including a figure to illustrate the entire system could significantly help reader comprehension. For example, a diagram showing the VPose (or any baseline) architecture and the integration of the BDC component might be more effective than a text-only description. Additionally, including some implementation details about the BDC component in the main paper could improve the flow of information.\n\n2- There are some ambiguities in the experiment section that need clarification. When referring to the “source-dataset”, it would be helpful to specify whether this refers to the Human 3.6M dataset or the newly synthesized poses. Similarly, when discussing evaluations on 3DHP and 3DPW, it would be beneficial to mention the specific subset used, such as the test set.\n\n3- There appears to be some confusion between Table 1 and the results in Figure 4 (left). While Table 1 shows improvements in the Human 3.6M results when adding new poses generated from BPG, Figure 4 (left) indicates that adding more data increases the MPJPE error (without integrating BDC). This seems contradictory and could benefit from further explanation.\n\n4- Typos: There are a couple of typographical errors that need correction. On Line 177, (xi) is repeated twice instead of yi. On Line 287, BDC should be corrected to BPG.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comprehensive and constructive review. Your insights on the scalability and generalization aspects, along with suggestions for improved clarity, are invaluable and will be instrumental in enhancing our paper.\n\n## Suggestions for Improving Clarity\n\nWe appreciate the reviewer's positive feedback on the overall quality of our paper and their constructive suggestions for improving clarity. We agree that including a figure to illustrate the entire system could significantly aid reader comprehension. The diagram showing the baseline architecture and the integration of the BDC component has been added to the PDF linked in the General Response. Additionally, we have provided more detailed implementation specifics of the BDC component in the main paper. To aid understanding of the BDC, pseudo-code is included and can be found in Appendix D, Algorithm 2. These enhancements will make our manuscript clearer and more informative.\n\n## Ambiguities in the Experiment Section\n\nWe appreciate the reviewer's observation regarding ambiguities in the experiment section. We will carefully address these ambiguities and provide the necessary clarifications to improve the clarity and comprehensibility of our manuscript. Bringing this to our attention helps ensure that our revisions will enhance the overall quality of our work.\n\n## Confusion between Table 1 and Figure 4 (left)?\n\nThis is a valid point. We clarified this issue in the general response under \"Confusion on Experimental Results.”\n\n## Typos\n\nWe acknowledge the reviewer's attention to detail regarding the typographical errors in our manuscript. We will correct the repeated (xi) on Line 177 to yi and change BDC to BPG on Line 287. Highlighting these issues helps us improve the accuracy and readability of our manuscript.\n\n## **Concluding Remarks**\n\nWe are grateful for the constructive feedback from reviewer 696C and will incorporate their valuable suggestions to refine our manuscript. We believe that our comprehensive responses will satisfactorily address the concerns and result in a favorable reassessment.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sHXkfoc40i", "reviewer": "Reviewer_Gv4b", "summary": "Existing data in 3D human pose estimation are typically collected indoors with human actors. To address this scalability issue, the authors propose to synthesize 3D human pose data via an Osteo-kinematic model and introduce biochemical constraints for better physical plausibility. Additionally, to deal with the inherent ambiguity in single-view depth estimation, the authors introduce Binary Depth Coordinates to explicitly model the relative spatial relation between adjacent joints. Extensive experiments verify the effectiveness of the proposed approach.", "questions": "1. In Fig. 2 and Fig.4, adding synthesized data consistently decreased performance for some baseline methods, e.g., GFpose. This seems counterintuitive to me. As the authors mentioned, overfitting might be a reason; do the authors have any other insights regarding this? Does this phenomenon indicate there is still a gap between the real data and synthesized data?", "limitations": "The authors have included the limitations in Sec.6. However, the requirement for camera intrinsics may also be considered a limitation and discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Leveraging biomechanical prior knowledge to synthesize physically plausible human data is $\\textbf{intuitive}$ and $\\textbf{interesting}$.\n2. Comprehensive experiments verify the effectiveness of the proposed data augmentation approach (BPG) and Binary Depth Coordinates (BDC). Specifically, BDC can be applied to different methods, e.g., image-based and lifting-based, showing superior generalization ability.", "weaknesses": "1. $\\textbf{Repeated text}$: The first paragraph of Sec.2 appears to be a copy-paste from the abstract, which is highly discouraged.\n2. $\\textbf{Requirement of camera intrinsics}$: While BDC shows notable performance gains to baselines, solving depth requires camera intrinsics (principal point and focal length), typically not required by current 3D HPE methods. This requirement may introduce additional constraints for in-the-wild inference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable and detailed review. Your feedback on the use of biomechanical knowledge and the practical limitations of our approach provides important guidance for further improving our work.\n\n## Repetition in the First Paragraph of Section 2\n\nWe sincerely thank the reviewer for pointing out the repetition in the first paragraph of Section 2. We appreciate your careful review and constructive feedback. We will ensure that this repetition is addressed and revised appropriately to enhance the clarity and originality of our manuscript. Thank you once again for your valuable input.\n\n## Requirement of camera intrinsics\n\nWe appreciate the reviewer's insightful comment regarding the requirement of camera intrinsics for solving depth with BDC. We understand that this may seem like an additional constraint compared to some current 3D HPE methods.\n\nHowever, we would like to clarify that the use of camera intrinsics in our equations primarily serves to illustrate the theoretical framework. In practice, similar to other research [1,2,3], we employ approximations for both inference and training, which effectively mitigate the need for precise camera intrinsic parameters. This approach allows our method to be applied more flexibly in in-the-wild scenarios without introducing significant constraints.\n\n[1] J. Sosa et al., \"Self-supervised 3D Human Pose Estimation from a Single Image\", CVPR 2023\n\n[2] B. Wandt et al., \"ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Poses\", CVPR 2022\n\n[3] Z. Yu et al., \"Towards alleviating the modeling ambiguity of unsupervised monocular 3d human pose estimation\", ICCV 2021\n\n## Insights on Decreased Performance with Synthesized Data\n\nWe appreciate the reviewer's question regarding the decreased performance when adding synthesized data for some baseline methods, such as Gfpose, as shown in Figures 2 and 4.\n\nBy examining Figure 6 (Appendix), we can provide additional insights into this phenomenon. It appears that the inclusion of excessive synthesized data can exacerbate the depth ambiguity problem for some baseline methods. As a result, these models tend to predict the middle ground rather than accurately differentiating between the front and back, leading to a degradation in performance.\n\nHowever, our Binary Depth Coordinates (BDC) approach mitigates this issue effectively. BDC's novel decomposition strategy and its handling of depth ambiguity ensure that the model maintains high performance even with the addition of synthesized data. This demonstrates the robustness of our method in addressing depth ambiguity and improving overall accuracy, unlike the observed degradation in baseline methods.\n\n## **Concluding Remarks**\n\nWe deeply appreciate the insightful comments provided by reviewer Gv4b and will thoughtfully incorporate their valuable suggestions in our rebuttal. We are confident that these responses will address the concerns raised and contribute to an improved evaluation of our manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their insightful feedback and thought-provoking questions regarding our work. We greatly appreciate the recognition of the clarity, relevance, and novelty of our contributions.

We were pleased to receive positive comments from many reviewers.
**Reviewer 696C** acknowledged the novelty of BPG, stating, "The idea of a biomechanical pose generator which does not rely on a source dataset is novel," and appreciated our attention to depth ambiguity in 3D pose estimation.
**Reviewer Gv4b** found BPG intuitive and effective, mentioning, "Leveraging biomechanical prior knowledge to synthesize physically plausible human data is intuitive and interesting," and noted its superior generalization ability across different methods.
**Reviewer 1aXZ** highlighted our handling of depth ambiguity, stating, "Tbl. 5 demonstrates BDC's capability in managing larger depth ambiguities," and praised its combinability with other work for yielding improvements.
**Reviewer 9ZcB** recognized our method's strong performance, stating, "The proposed method achieves state-of-the-art results in various 3D HPE datasets," and validated its data augmentation effects in cross-domain learning settings.

Some concerns were raised by more than one reviewer, so we decided to address them here in this general response. The other concerns are addressed in individual responses.

## Comparison to Similar Works on BDC (1aXZ, 9ZcB)

**Handling Depth Ambiguity**: By estimating poses in discrete space rather than the continuous space used in traditional methods ([1], [2]), we can better handle depth ambiguities. This is achieved through a binary depth parameter that indicates the relative depth to the preceding joint, allowing us to manage the uncertainty of relative depth. Unlike methods that must consider a continuous range of possible depths, our approach only needs to handle two discrete possibilities for depth, significantly simplifying the prediction task and improving robustness. As a result, our approach demonstrates robust performance even for poses with significant depth ambiguity, as shown in Figure 2.

**Decomposition Strategy**: Unlike [1], which primarily decomposes the problem into 2D pose and depth, our method further decomposes it into 2D coordinates, bone length, a binary depth parameter indicating the relative depth to the preceding joint, and the 3D coordinates of the parent joint. This comprehensive decomposition allows for a more granular handling of depth ambiguity, making it easier to isolate and address specific sources of error in the pose estimation process. **The results of the NDC representing the same method as in [1] are shown in Table 7, highlighting the improved granularity and performance.**

**Direct 3D Pose Estimation**: While the depth-ordered learning method [2] employs additional models to reconstruct 3D structures and predict poses in continuous space, our methodology eliminates this step. By leveraging geometric principles, our BDC framework enables a direct and efficient transformation into 3D poses without the need for additional models. This approach simplifies the overall process and improves robustness. The simplicity of not requiring multiple models or additional reconstruction steps means our approach can be more efficient and less prone to compounding errors. As demonstrated in our comparative analysis in Table 6, our method can be seamlessly integrated into various models, showcasing its versatility and effectiveness across different architectures.


[1] U.Iqbal et al., "Hand Pose Estimation via Latent 2.5D Heatmap Regression", ECCV 2018

[2] G.Pavlakos et al., "Ordinal depth supervision for 3d human pose estimation", CVPR 2018

## Confusion on Experimental Results (9ZcB, 696C)

We sincerely appreciate the reviewers' observations regarding potential confusion between the results presented in Tables 1, 2, and Figure 4. We would like to clarify that these results are derived from different types of tests, each serving to evaluate specific aspects of our method's performance.

Firstly, it is important to note the distinct types of tests from which the results in Tables 1 and 2 and those in Figure 4 are derived:

1. Single Hypothesis Test (Tables 1 and 2): These results reflect the prediction of a single 3D pose. This evaluation provides a straightforward measure of our model's performance under a standard testing scenario where one pose is predicted for each instance.
2. Multi-Hypotheses Test (Figure 4): The results depicted in Figure 4 arise from a multi-hypotheses test, where multiple possible 3D poses are predicted. This approach is particularly useful for assessing the model's capability to generate diverse pose predictions and handle greater ambiguity.

To ensure clarity and avoid any potential confusion, we provide the performance metrics for the single 3D pose prediction scenario using the same evaluation criteria as those employed in Figure 4. We hope this allows for a more direct comparison and helps contextualize the different performance measures. Below is a table that shows these performance metrics to facilitate a direct comparison:
|Model\Amount|0|0.1|0.2|0.4|0.6|0.8|1|2|4|6|8|
|-|-|-|-|-|-|-|-|-|-|-|-|
|Vpose|52.7|51.9|51.1|49.0|47.5|47.0|46.9|48.5|50.2|52.5|53.9|

These results indicate that the Single Hypothesis Test initially shows performance improvement differently from the Multi-Hypotheses Test. This demonstrates that using BPG alone is more effective than other existing augmentation methods. Additionally, the value for the amount of 1 matches the results presented in Table 1.

We acknowledge that presenting results from different models and tasks can create complexity and potential confusion. To enhance the comprehensibility of our findings, we are open to switching or adding performance plots for the same model. This adjustment will streamline our presentation and make it easier for readers to consistently interpret the results.

Meta-review: All reviewers agreed to accept the paper. The paper originally got BA, BR, BR, and WA, and, after the rebuttal period, two reviewers raised the ratings, leading to final ratings of BA, BA, BA, and WA. 

Reviewers recognized the major contribution of this paper in leveraging biomechanical prior for synthesizing physically plausible human samples, in the framework for  3D human pose estimation. While reviewers were concerned about the similarity of the proposed one with existing approaches, after the authors' rebuttal and discussion period, most reviewers’  concerns have been resolved, and all reviewers agreed that the paper's shape is above the bar for NeurIPS. 

The AC also supports accepting the paper and strongly suggests the authors to follow the reviewers’ feedback in the camera-ready version. In particular, the AC suggests the authors significantly improve the clarity and readability of the paper, trying to include better and higher quality figures to better support the proposed method.
Decision: Accept (poster)