PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space
Authors: Jinghong Zheng
Changlong Jiang
Yang Xiao
Jiaqi Li
Haohong Kuang
Hang Xu
Ran Wang
Zhiguo Cao
Min Du
Joey Tianyi Zhou
Venue: NeurIPS 2025 poster ()
Keywords: Human pose estimation
anchor to joint prediction
depth estimation
Primary area: applications

Abstract: 3D human pose lifting from a single RGB image is a challenging task in 3D vision. Existing methods typically establish a direct joint-to-joint mapping from 2D to 3D poses based on 2D features. This formulation suffers from two fundamental limitations: inevitable error propagation from input predicted 2D pose to 3D predictions and inherent difficulties in handling self-occlusion cases. 
In this paper, we propose PandaPose, a 3D human pose lifting approach via propagating 2D pose prior to 3D anchor space as the unified intermediate representation. Specifically, our 3D anchor space comprises: 
(1) Joint-wise 3D anchors in the canonical coordinate system, providing accurate and robust priors to mitigate 2D pose estimation inaccuracies. 
(2) Depth-aware joint-wise feature lifting that hierarchically integrates depth information to resolve self-occlusion ambiguities. 
(3) The anchor-feature interaction decoder that incorporates 3D anchors with lifted features to generate unified anchor queries encapsulating joint-wise 3D anchor set, visual cues and geometric depth information.
The anchor queries are further employed to facilitate anchor-to-joint ensemble prediction.
Experiments on three well-established benchmarks (i.e., Human3.6M, MPI-INF-3DHP and 3DPW) demonstrate the superiority of our proposition.
The substantial reduction in error by 14.7% compared to SOTA methods
on the challenging conditions of Human3.6M and qualitative comparisons further showcase the effectiveness and robustness of our approach.

PandaPose:
3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space
Jinghong Zheng' Changlong Jiang'
Yang Xi Jiaqi Li
Haohong Kuang' Hang Xu
Ran Wang Zhiguo Cao² Min Du Joey Tianyi Zhou 6,7
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
Sccooo of Artificial Intelligence and Automation, Huazhong University of Science and Technology School of Journalism and Information Communication, Huazhong University of Science and Technology
Schooo of Future Technology, Huazhong University of Science and Technology
ByteDaace Inc.
Centre for Frontier Research, Agency for Science, Technology and Research, Singapore 7 Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore
*Corresponding author
(deepzheng, changlongj. Yang_ Xiao, [ijiaqi_mail]_eeeeeeeepplppjjpe eecauusttuuu... {haohong_ kuang,uua.... baxg_xx,rex..mmmm zgcao @hust. edu.cn bingwen. ai@bytedance.coom seey__hhoua-SSe saaieaastarreduu...
Abstract
3D human pose lifting from a single RGB image is a challenging task in 3D vision. Existing methods typically establish direct joint-to-joint mapping from 2D to 3D poses based on 2D features. This formulation suffers from two fundamental limita- tions: inevitable error propagation from input predicted 2D pose to 3D predictions and inherent difficulties in handling self-occlusion cases. In this paper, we propose PandaPose, 3D human pose lifting approach via propagating 2D pose prior to 3D anchor space as the unified intermediate representation. Specifically, our 3D anchor space comprises: (1) Joint-wise 3D anchors in the canonical coordinate system, providing accurate and robust priors mitigate 2D pose estimation inac- curacies. (2) Depth-aware joint-wist wise feature lifting that hierarchically integrates depth information to resolve self-oo oecfuoccluioon ambiguities. (3) The anchor-feature interaction decoder that incorporates 3D anchors with lifted features to generate unified anchor queries encapsulating joint-wise 3D anchor set, visual cues and geometric depth information. The anchor queries are further employed to facilitate anchor-to-joint ensemble prediction. Experiments three well-established bench- marks (i.e., Human3.66 6.M, MPI-INF-3DHP and 3DPW) demonstrate the superiority of our proposition. The substantial reduction in error by 14.7% compared to SOTA methods on the challenging conditions of Human3.6k 66 and qualitative comparisons further showcase the effectiveness and robustness of our approach.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Image
20 poen
lifing network
CA
Ours
GT
(4)
2D feature
2D 3D mapping
Lifting within 2D plane
5D pose
Indoor environment
3D anchor space
Freditsd
CA.PF
PoseFrrer
Ours
2D-to-3D feature lifting
anchor-to-joint paradigm
3D
Lifting within 3D anchor space (Ours)
20 joint gositon
Outdoor environment
an yoir pooitio
Jcent whse
Figure 1: Comparison between different 2D-to-3D human pose lifting manners. Previous methods (a) generally concern 2D in in-plane feature and directly predict 3D pose. Our method (b) mitigates depth ambiguity by lifting in-plane feature to 3D space and interacting with joint-wise 3D anchors. Then 3D pose will be estimated via machor-to-joint ensemble prediction within 3D anchor space. Our method is robust to both indoor and outdoor occlusion scenarios.
Introduction
Monocular 3D human pose estimation from a single RGB image has wide applications including
action recognition [44, 60, 5J, virtual reality [11] and human-computer interaction [16, 34]. Compared to sequence based methods [27, 57, 20, 50], image based counterparts [48, 54, 55] generally have higher potential for real-time applications and insensitive to the length of input sequence. Recently, benefiting from the advancement in 2D human pose estimation [4, 33), the research of 3D pose lifting with input predicted 2D pose has drawn researchers' attention. However, many challenges remain in this field, including estimated 2D pose error propagation, high-frequency self-occlusion, etc. [53, 15].
Currently, the state-of-the-ar image-based approaches [54, 55, 59] attempt to perform pose lifting by introducing image features to supplement spatial context as shown in Fig. (a)). Although these methods demonstrate promising performance, they exhibit inherent limitations. First, they attempt to establish the one-to-one mapping from 2D to 3D pose, increasing the dependency on the 2D pose prediction quality. Consequently, minor noise in the input poses could lead to significant deviations of predicted 3D pose, resulting in limited robustness. Second, self-occlusion of human body on images quite common due the monocular nature. Existing methods [55, 50, 28] primarily rely on image descriptive clues for 3D pose characterization, failing to explicitly model depth dimension and consequently struggling with depth ambiguity and self-occlusion challenges. To overcome these limitations, we propose to facilitate 3D pose listing via propagating 2D pose prior to 3D anchor space. Therefore, joint-wise 3D anchor setting and depth-aware feature lifting are proposed for providing robust anchor initialization suppress noise and integrating hierarchical depth to climinate ambiguities in self-occlusion, forming the 3D anchor space, as shown in Fig. 1(b).
For joint-wise 3D anchors, we aim to propagate the input 2D pose priors into 3D joint priors that possess both robust error tolerance and high accuracy. This mitigates error propagation issue in previous pose lifting methods, which overly rely on the accuracy of input poses. Specifically, we utilize 3D anchor as a coarse initialization relevant to input poses and predict 3D anchor-to-joint offsets for each joint in ensemble manner, rather than directly estimating 3D joint positions. The preliminary exploration prior works [42, 14] are limited to simplistic global fixed anchor setting, which often results in excessively long anchor-to-joine regression offsets, leading to degradation in accuracy and robustness with insufficient exploitation of the input pose. contrast, we propose joint-wise local anchors that fully exploit the guidance of the global 2D pose context. By adaptively setting a cluster of anchors near each joint based on its 2D position in a learnable manner, aiming to achieve trade-off' between robustness to errors in the input and accuracy of the initial 3D priors.
Self-occlusion remains significant challenge for current pose lifting methods, given the monocular nature of inputs images and poses. Solely rely on in-plane features can be severely semantically corrupted in occluded regions. Incorporating spatial structural information serves as a potential solution recover predictions for self-occluded regions in other domains [17, 6,51]. However, simply introducing depth maps proves inadequate for millimeter-level human pose estimation, particularly failing to resolve depth ambiguity in self-occluded joints. Consequently, propose joint-wise

---

feature lifting module that aligns image features to the domain of 3D anchor features under the guidance of intermediate joint-level depth supervision. For the supervision, in the absence of dense ground truth depth maps, we extract the depth value of ground truth joints instead of the complete depth as supervision. This approach not only inherently furnishes crucial prerequisites for the reconstruction of self-occluded joints through depth stratification but also facilitates more practical approach to fitting the actual joint depth distributions. At the feature level, we leverage 2D pose priors for feature sampling of visual features to resist background noise, simultancously reducing computational memory cost. The sampled features interact with depth information before being projected into 3D anchor space, enabling robust joint localization against occlusions.
The initial anchor queries, derived from joint-wise 3D anchor setting, undergo multiple attention- based interactions with depth features and lifted image features within the 3D anchor space. The resulting output queries integrate cross-modal information from visual, depth, and geometric anchor data, which are then transformed into anchor offsets and weights towards each joint. This anchor- to-joint prediction mechanism produces robust 3D pose predictions that maintain both resilience to input 2D pose inaccuracies and effectiveness against self-occlusion.
The superiority of our proposed PandaPose is verified on three well-established datasets (i.e., Hu- man3.6M [13], MPI-INT-3DHP [26], 3DPW [39]). The experiments demonstrate our approach essentially outperforms all the state-of-the-art image based counterparts. Especially under challeng- ing scenarios (e.g. occlusion) in Human3. 6MM our method achieves a significant improvement of 11.3% MPJPE and 14.7% in PA-MPJPE. Overall, the main contributions of this paper include:
We propose PandaPose to address image based 3D human pose lifting via propagating 2D pose prior to 3D anchor space as intermediate representation, and achieve effective and occlusion-resistant 3D pose estimation through machor-to-ooin ensemble prediction;
We design joint-wise 3D anchor setting to provide an accurate and robust mapping 31) joint, thereby mitigating the impact of input 2D pose inaccuracies;
A novel 2D-to-3D feature lifting method is proposed to resist self-occlusion and depth ambiguity issues, via estimating joint-wise depth distribution with sparse depth supervision.
2 Related Works
2D-to-3D human pose lifting. With advancements in 2D human pose estimation [4, 33], lifting 2D poses to 3D has become critical research area, with methods generally categorized into sequence-
based and image- based approaches. Sequence-base based methods [27, 22, 40, 47, 57, 19, 20, 50, 31, 28] use long temporal sequences using GCN [52] or Transformer [38] to model spatial-temporal correlations in 2D poses. Despite significant performance gains with longer sequences [50, 20], these methods also face increased computational complexity and memory demands. With single frame imput, some works [43, 48, 56, 21] attempt to model the spatial relationships within the human skeleton to capture the spatial correlation between 2D and 3D poses. Other approaches propose to leverage visual features [54, 55, 59] from 2D pose estimators to compensate for the weak descriptive capability of 2D pose positions. However, due to the limitations of in-plane features, they commonly lack explicit modeling of depth, leading to challenges in handling depth ambiguity and self-occlusion In our work, we explicitly incorporate joint-wise depth distribution to enhance feature with depth awareness and alleviate prediction difficulty using an adaptive macho-to--ointt regression manner.
Anchor based pose estimation. In hand pose estimation, some works [42, 12, 8] designed anchor-to-joint paradigm by treating 2D anchors as local regressors to estimate hand poses. A2J- Transformer [14] further proposed combining the 3D anchor setting with Transformer [23] to enhance the resistance towards occlusion. However, the anchor settings in these methods are fixed and cannot adapt to the content of the image, leading to suboptimal performance in certain scenarios. We are the first to explore the use of anchors in 3D human pose lifting and leverage 2D pose prior to enhance anchor design from static to learnable adaptive.
Visual feature with 3D enhancement. For pose estimation, some works expanded visual features into 3D voxels [37, 45, 32, 25] and applied 3D CNNs to regress 3D human poses. Although effective, the accuracy of 3D human pose estimation may be constrained by the voxel resolution due to limitations in memory usage and computational cost. To efficiently construct 3D feature representations, some works in 3D object recognition [29, 17, 51, 6, 46] have made positive explorations. These methods
3

---

3D anchor space setting
3D anchor space with lifted feature
Anchor-to-joint
ensemble prediction
Predicted 3D pose
local anchor
global anchor
ID-to-3D feature lifting
Image features
pose estimator
joint position
Interaction
anchor position 3D joint position Anchor-to to-joont offset
Anchor-feature
Joint wise distribution
Pose prior based feature sampling
interaction
Input image
Figure Overview of PandaPose pipeline. Given input single-frame 2D pose and intermediate image features, we adaptively sample anchors in 3D space, By estimating joint-wise depth distributions and employing 2D pose prior based sampling strategy. lift features from 2D to 3D domain. After 3D anchor-feature interaction, we obtain predicted 3D pose through anchor-to- jo-jonnt ensemble prediction.
typically extend in-plane features into 3D space using given or predicted depth maps. However, single foreground depth map is insufficient for accurate 3D pose estimation, especially under self-occlusion with significant depth ambiguity. Therefore, we propose to predict joint-wise depth distributions and integrate them with in-plane features to lift them into depth-aware 3D features.
3
Method
3.1 Overview of main pipeline
The main technical pipeline is shown in Fig. 2. Following the two-stage pipeline of SOTA pose lifting methods [55], given the input RGB image of size x W 3, an off-the-shelf 2D pose estimator [33] generates the corresponding 2D human pose PiD E RN22 along with intermediate pyramid feature maps, where N, is the number joints. Given the input 2D pose, we propose form 3D anchor set via adaptive joint-wise local anchor and fixed global anchor setting (Sec. 3.2). To lift the input visual 2D in- in-plane features into depth-aware 3D features, we predict the joint-wied wise depth distribution maps Distp and depth embedding FD (Sec. 3.3). We also design a 2D pose prior based feature sampling strategy to extract joint-related visual features F, (Sec. 3.4), aiming to minimize computational costs while filtering out background noise. The 3D anchors
encoded as learnable anchor queries Qanchor and interact with lifted 3D features within 3D anchor space, thus making anchor queries as an unified representation encapsulating 3D anchors, visual cues and depth information (Sec. 3.5). Finally, the predicted 3D pose p3D R,x3 obtained through -ocoortto-joint ensemble prediction (Sec. 3.6).
3.2 Joint-wise adaptive 3D anchor setting
In contrast to current pose lifting methods that typically establish a direct joint-to-joint mapping from 2D to 3D poses, we construct 3D anchors as coarse initialization relevant to the input 2D poses and then predict 3D anchor-to-join offsets for each joint in ensemble manner. Previous anchor-based methods [42, 14] typically employ fixed anchor settings sparsely located in 3D space, lacking adaptability to specific pose patterns and thus leading large offsets for distant anchors from human regions (Fig. 3). Given that predicting large offsets involves a greater margin for error, it potentially degrades overall performance in joint localization, especially under occlusions 2D pose inaccuracies. To address this, we propose evolving the anchor setting from static to dynamic adaptive by leveraging 2D pose priors. We conduct comparison with different anchor settings the Human3. 3.6M [13] (Fig. 3). By selecting top-50 anchor-to-joint weights, we identify informative anchors contributing most to joint prediction and calculate their proportion relative to the total. Our adaptive setting significantly reduces the offset from 154.6mm to 69.7mm.
The adaptive 3D anchor generation procedure is shown in Fig. 4. The input 2D pose p2D is first normalized to [-1, 1]. Then 3D local anchors are adaptively generated, producing set K sampled 3D offsets 8 IR K33 for each joint J::
8, Linear(P?,)
8, N
(I)

---

Fixed anchor
Fixed anchor Adaptive anchor
offiet
Adaptive
anchor
Anchor type
Mean offset
Informative
Global context guldance
anchor ratio
anchor
154.6mm
Local anchor setting
anchor setting
Global
Adaptive
69.7mm
Prodicted 2D pose
Joint-wiis 3D local anchor
3D anchor space
Informative anchor
GT 20/30 joint
Comparison between different anchor settings. Adaptive anchor is more effective at joints distant from human regions.
Figure 3:
Figure 4: Illustration of adaptive 3D anchor set- ting including joint-wise local anchor and global fixed anchor.
Due to the consideration of global 2D pose context, it has a good adaptability to locally inaccurate 2D joints, as shown in Fig. 10. Simultancously, the normalized 2D joint position (Jz- Jy) is initialized as a 3D position at depth 0, formed (jr- Jy. 0). The 3D local anchor set Atocal is then generated by adding the sampling offsets to the corresponding joint 3D position (Jx. Jy.0) 0):
Atocal (a P. = (i. j.. j.u. E J.k K).
(2)
To enhance the model's generalizability and training stability, we integrate a subset of global fixed anchors to complement local adaptive anchors with a stable global context. Specifically, we preset 3D global anchors Alooha 2256x3 which uniformly distributed on the plane of root joint in 3D space with in-plane stride S = H/16 and Sy W/16. Finally, the 3D anchor set A is the combination of global anchors and local anchors:
A = Aglobal U Alocal-
The 3D anchor set A are then used to facilitate anchor-feature interaction in the form of anchor query Qunchor- Their 3D position p&D is the initial position for anchor-to-joint prediction.
3.3 Joint-wise depth distribution estimation
Downsmmped
frature
In 3D human pose lifting, one of the key chal- lenges lies in resolving self-occlusion. Current methods are generally limited by relying strained 2D features, which leads to difficulties in handling depth ambiguity. However, simply estimating a single depth map as a supplement to image features is insufficient for achieving pre- cise pose estimation requirements. First, pose estimation focuses more on the relative depth relationships between joints, and a single depth map cannot provide effective depth predictions for occluded or 2D-prxximity joints. Second, without ground truth depth maps, how to en- able models to perform more accurate depth estimations remains a challenge. As illustrated in Fig. 5, joints far apart in 3D space to appear close in 2D planes due to projection distortion, often overlapping in the downsampled feature map. This leads to ambiguous training, where point maps to multiple GT depths, and inac- curate inference, where multiple joints share the same incorrect depth. Thus, the inherent depth ambiguities cannot be truly well resolved.
depth tabel depth Share vere Seuth Inference
contustore
Cloccnn plance
Figure 5: Existing methods typically predict single depth map. Due to projection distortion, joints far apart in 3D space may appear close in 2D planes, often overlapping in the downsampled feature map, causing significant confusion during training and inference.
Crain
dreax
Figure Illustration of joint-wise depth distri- bution. Each joint has a corresponding predicted depth map, allowing occluded or nearby joints on the plane to have distinct depth distributions.
Therefore, we innovatively design a josn approach to predict individual depth distributions for each joint as shown in Fig. 6. The process of joint-wic wise depth distribution estimation is shown Fig. 7(a). We utilize a light- depth net to estimate joint-wise depth (architecture is provided in Appendix A.2). We use multi-scale features from pretrained 2D pose estimator to estimate depth distribution maps at corresponding scales. To balance efficiency with feature richness, we choose single H/8 x W/8 resolution feature map as input and estimate depth distributions for each
5

---

joint independently. To simplify the continuous depth value prediction, we segment the depth range [-dmin: Amar] into Knin bins, treating each bin as a distinct class, thereby formulating depth estimation as classification task [30, I.. The resulting depth distribution maps N, x H/8 x W/8 x Knin are then upscaled via interpolation match different image resolutions, forming Distp- Additionally, we use single Transformer encoder layer [7] on the features output by depth net to generate depth-aware embeddings FD. encoding geometric depth cues for 3D anchors.
To address the challenge of generating accurate depth distribution consistent with 3D pose in the absence of the ground-truft -ruth dense depth map, we introduce leveraging 3D pose annotations as sparse supervision. Consequently, we first map the depth value of GT 3D pose to the corresponding bin as the label, and then compute binary cross entropy loss [6] separately for each joint depth map. To mitigate learning complexity and filter out irrelevant noise, the loss calculation is limited to the r x r region surrounding the 21) joint.
Ldepth N
logPt (n.0)
log (n.1)
where N N, xr indicates the total number of joint-wise depth supervision pixels.
3.4 2D pose prior based feature sampling
For 3D human pose estimation, attention-base models typically process each pixel, which can introduce background noise and increase computational load without improving accuracy. To address these, we propose a 2D pose prior based feature sampling strategy improve the feature extraction of traditional attention modules. Our method leverages 2D poses as prior, selectively focusing on features within a r r region around joints instead of the whole image (see Fig. 7(b)). This approach reduces irrelevant information interference while enhancing computational efficiency. Regarding multi-scale features, our strategy is applied to the first two high-resolution layers, with the sampling radius proportional to the resolution (r = H/ 16), preserving global semantic information in lower- resolution layers. And we normalize image feature channels to the same Cy. The similar sampling strategy is employed for depth distributions, ensuring that the N₂ tokens fed into the anchor feature interaction decoder are pixel-aligned, where N₂ is the number of pixel tokens that are summed after flattening the multi-layer sampled feature map. The sampled image features F₁ BNxxC, and depth distributions Distp E &MNxXKi are then processed with the anchor-feature interaction decoder for further interaction within 3D anchor space, enabling robust joint localization against occlusions.
3.5 3D anchor feature interaction
(a) Joint-wise depth estimation
Depth
Sparse joint-wise supervision
embedding
Within 3D anchor space, we propose to leverage learnable adaptive 3D anchors that interact with both depth and visual features to enhance spa- tial context understanding, addressing the chal- lenges of accurately capturing spatial relation- ships and depth information in pose estimation. Structure of anchor feature interaction decoder shown in Fig. 8. Based on learnable adaptive 3D anchors, encode them as learnable chor queries Qanctor to predict 3D pose through anchor-feature interaction decoder. Each de- coder layer comprises a depth cross-attention layer, an inter-anchor self-attention layer, and 3D deformable cross-attention layer. The depth cross-attention layer captures latent depth fea- tures, enabling anchors to adaptively understand spatial contexts from sopth-guided regions, en- hancing the perception of inter-joint depth rela- tionships. In this process, we linearly transform the anchor query and depth embedding into the query Qp. key Kp. and value Vp:
DepthNet
3D
Distp
depth
(b) Joint prior based feature sampling
Self attention
Image
Joint
20 visual Fj
features
feature
Figure 7: Processes of joint-wise depth distribu- tion estimation and 2D pose prior based feature sampling.
Anchor guery
Output anchor query
Depth-aware cross attention
Inter-anchor self attention
3D deformable cross attention
Disto Depth
Depth
embadding
feature distribution
Figure 8: Structure of 3D anchor feature interac-
tion decoder.
Qp Linear Qanchor).
KD. VD Linear(Fp).
(5)

---

Parameters (M) for
Method
Lifting Module
MPJPE:
PA-MPIPE
PoseFormer
ICCV'2]
9.5
44.3
34.0
MHFormer [20)
CVPR'22
351
24.8
43.0
34.4
Sequence
MixSTE [50]
CVPR'22
243
33.6
40.9
32.6
based
P-STMO [31]
BCCV'22
STCFomer [35]
CVPR'23
243
18.9
41.0
32.0
KTPFormer (28)
CVPR'24
243
35.2
40.1
31,9
Full lest
GraphSH [43]
CVPR'21
3.7
51.9
HCSF [48]
ICCV'21
47.9
39.0
GraFormer [56]
CVPR'22
51.8
Diffpose [10]
CVPR
1.9
49.7
Image based
Zhou et [5|]
HiPART [58]
CVPR'25
2,4
42.0
CA-PF [55]
NeurlP5'22
14.1
41.4
33.5
PandaPose
(1.61)
(0.81)
Challenging suther
CA-PF [55]
NeurIPS'25
14.1
82.0
PandaPose (Ours)
15.2
73.1 (9.3))
69.9 (12.1
Table 1: Comparison with state-of saae--of-thh-artt methods on Human3. 6M. MPJPE and PA-MPJPE are reported in millimeters. The best results are shown in bold. Our method not only achieves leading performance on the full test set, but also shows significant improvement on challenging subset.
Then, the depth-aware 3D anchor queries are fed into the inter-anchor self-attention layer to promote articulated clues between anchors. Finally, lift the flatten in-plane features F, into 3D space using the joint-wise depth distribution Distp via the outer product FID = Distp @ Fy, and apply 3D deformable eross- cross-attention (DCA) layer to enable 3D anchor queries effectively aggregate visual characteristics of the 3D scene. Then, using the 3D anchors as reference points, for a specific anchor located at the position Pa. we perform feature interaction through 3D deformable cross-attention:
DCA(a)
Σ MEN
Wab(Fann Pa AS
Following [61], each anchor is associated with adaptive sampling points near PA. whose offsets AS, are predicted from the input anchor query Qanchor through linear layer. This allows the Transformer to dynamically attend to sparse yet semantically rich local regions in data-driven manner, enhancing both efficiency and accuracy. The term &DF Pa + AS, n) denotes the trilinear interpolation to sample features from the expanded 3D feature map Fan- The output anchor query forms a unified representation in the 3D anchor space, integrating multimodal information and helping solve the depth ambiguity issues.
3.6 Anchor-to-joint prediction
With anchor queries Qanchor output from decoder, we use MLP layers to extract the offsets and weights W of the anchors with respect all joints. The joint positions positioned in the form of weighted sum of anchor-to-joint offsets:
P&D Σ Wa, Oa,).
where Pj and Pa indicate the 3D position of the certain joint j and anchor a. Oa,i denotes the offset from a towards j. Waj is the softmax-derived weight of a towards j.
3.7
Loss Function
We train the joint-wise depth distribution map in sparsely supervised manner (Sec. 3.3), denoted as Ldepth, and use MPJPE [27, 57] to supervise the training of 3D pose, denoted as Lpose- The overall loss function is formulated
L= d Lpose + +1 Ld
(8)
Here we set 4 = and 42 = 1 for scale balance.
Experiments
4.1 Datasets and evaluation metrics

---

Method
PCK
AUCT T
1PIPE
Method
MPPPE
PA-MPIPE
test
STRGCN [2]
112.9
69.6
GraForuer [56]
79.0
13.8
VideoPose [27]
101.8
63.0
HCSP [48]
Li of [18]
81.2
82.1
46.2
46.1
99.7
PoseFormer [57]
118.2
73.1
Zbou [l[59]
88.2
59.3
Learning [49]
91.1
54.3
CA-PF [55]
32.7
PCT [9]
83.1
53.9
PandaPosse (Ours)
98.6 (0.61)
75.8 (0.41)
31.8 (0.91)
Di
82.7
53.8
Challenging subset
Dise [41]
79.3
50.1
CA-PF [55]
$4.5
53.2
66.6
HiPART [58]
77.2
48.8
PandaPose (Ours)
94.3 (9.81)
(9,31)
(14.8))
PandaPose (Ours)
74.9(2.)))
(6.9(1.9))
Table 2: MPI-INF-DDHP comparisons with image- Table 3:
Cross-dataset comparison of our
based methods.
method with SOTA methods on 3DPW.
Global fixed
Adaptive local anchor
MPJPEL (Full)
MPJPE (Challenging)
Anchor feature
Depth distribution
MPJPEJ
MPIPE
anchor
(Full)
(Challenging)
PundaPose w/o anchor
42.1
81.9
2D
40.9
80.8
(1.30)
(5.00)
3D
Single
40.3 (0.6.)
(1.9
40.1 (2.13)
74.0 (221)
3D
Joint-wisc
39.8 (1.I)
73.1 (7.7L)
39.8 (2.3g)
73.1 (8.11)
Table 5: Ablation study of joint- joint-wise 3D feature lifting.
Table 4: Anchor setting strategy comparison.
We train our model separately the two commonly used 3D human pose datasets (i.e., Human3.6M [13] and MPI-INF-3DER tiveness of PandaPose. To better verify the general- ization ability, train our model Human3. ..6 and conduct a cross-dataset evaluation on 3DPW [39]
162
MIXSTE (T-243)
(T-243)
[26]) demonstrate the effec-
142
CAPPITT-1]
Ours (T+1)
120
115.6
NR.PE
800
in-the-wild dataset. To better illustrate the robust
occlusion and 2D pose inaccuracy, we select samples from Human3. 6M and MPI-INF-3DHP with an error 5 between the predicted 2D pose and GT 2D pose as the challenging subset (= 5% in dataset). Addi- tional general information about each dataset and the evaluation metries are provided in Appendix A.I.
40.
40.2
32.6
10
Randon noise soale
Figure 9: We add Gaussian noise with vary- ing scales to the input 2D poses of different methods to test the robustness to noisy inputs.
4.2 Implementation details
Our model is implemented with PyTorch. Follow- ing [55], we use pre-trained frozen HRNet-w32 [33] image backbone and only extract the pyramid fea- ture maps for model input. For fair and in line with
20000-per 20pple
Figure 10: Adaptive 3D anchor setting com- parison under inaccurate/GT 2D pose. We calculate the centroids of two anchor sets and compute the distance between them (i.e., cen- troid distance [36]). The adaptive 3D anchors from inaccurate 2D pose closely match the GT 2D pose distribution, showing a minor centroid distance.
previous works, we use CPN-detected [4] 2D pose as input in Human 6M and 3DPW, GT 2D pose in MPI-INF-3DHP. The experiments are conducted on 2 NVIDIA RTX 3090 GPUs, using Adam W opti- mizer [24] with a learning rate of 4 and weight decay of 0.98 in total 30 epochs. All ablation studies are conducted on Human3.66 66 [13].
4.3 Comparison with state-of-the methods
Human3.6M dataset. We evaluate our model with SOTA methods on Human3.66 in Table 1, covering both image based and sequence based methods. When compared to SOTA image based methods, our model demonstrates a notable improvement MPJPE, reducing the MPJPE from 41. 41.4mm to 39.8m 39.8mm (1. 1.6mm decrease), alongside 0.8mm reduction in PA-MPJPE. Notably, our fingle-rrmme model matches the performance of SOTA sequence based methods, achieving an MPJPE from mmm to 39.399 8mm without requiring temporal context. To showcase our method's 's superiority, we conducted a comparison with CA-PF [55] in challenging subset. Our method achieves a significant improvement by 9.3mm in MPJPE and 12.1mm in PA-MPJPE. Visual comparison under challenging subset is provided in Fig. 11. We additionally made a performance comparison for the non-challenging case in Appendix A.3.
8

---

Method
MPIPE (Pult))
MPIPE (Chulllnging))
Method
GPU Memory (M) MPJPE
Regression
41.6
77.2
w/o feature sampling
21670
40.0
Classilscation with 16 bims
10.7
74.8
Rundom sampling
13788
45.6
Clasification with
2D pose peior feature sampling
13784
39.8
Classification with 128 bins
40.2
74.7
Table 6: Comparison of depth discretization strat- egy.
Table 7: Feature sampling strategy comparison.
MPI-INF-3DDE F3HHP dataset. We evaluate the performance on MPI-INF-3DHP dataset in Table 2. Our PandaPose achieves the best result, outperforming the existing SOTA image based models by 0.6% in PCK, 0.4% in AUC and 0.9mm in MPJPE. Under challenging subset (e.g.row 3 in Fig. 11), our method achieved a significant advantage by 9.8% in PCK, 9.3% in AUC and 14.8mm in MPJPE.
3DWW dataset. We evaluate our model pre- trained on Human3.6M to 3DPW dataset, as shown in Table 3. Under the same cross-dataset setting, our model achieves the SOTA perfor- mance with a notable improvement of 2.3mm in MPJPE and 1.9mm in PA-MPIPE, showcasing the strong generalization ability of our method.
Input
Ours
Ground truth
H3.6M
4.4 Ablation study
3D anchor setting. To validate the effectiveness of the anchor-to-joint regression and our adap- tive 3D anchor setting, we conducted ablation studies as shown in Table 4. The baseline model, which directly regresses 3D poses using MLP from the output feature of decoder, shows a sig- nificant performance drop (MPJPE decreases by 2.3mm in full test set and 8. Imm in challenging subset compared the to-jont regression manner). Further, both global and adaptive local anchors improve performance, with adaptive lo- cal anchors providing better accuracy due to their closer alignment with joint positions. In challeng- ing subset, adaptive anchor improved by 2.2mm compared to the global anchor. When combining global and local anchors, integrating both global and local context yields the optimal performance for the model. These findings prove that an ef- fective regression manner and carefully designed 3D anchors can enhance the performance of 3D pose estimation.
Figure 11: Visual comparison on challenging cases (e.g. significant occlusion or 2D pose inac- curacy). The circles highlight locations where our method has better predictions.
Input
Reconstruction
Input
Reconstruction
Figure 12: Visualization samples out of dataset from Internet including various scenarios and vir- tual avatars.
Joint-wise 3D feature lifting. To assess the effectiveness of lifting 2D features to depth-aware 3D features, we conduct ablation studies on key propositions in Table 5. We first remove the entire depth branch and use only the 3D anchor and in-plane 2D feature for interaction as the baseline. Next, we predict a single depth map and add the 3D deformable cross-attention and depth cross-attention to facilitate feature lifting. The performance improved, highlighting the importance of spatial context for 3D pose estimation. Subsequently, we predict joint- wise depth distributions instead of only one single depth map. It is observed that the performance improved, especially under the challenging subset, MPJPE shows a significant improvement by 7.7mm, indicating that fine-graine inn-grrined depth information at the joint level contributes to the accuracy of 3D pose estimation, particularly under selffocclusions.
Depth Discretization Strategy We replace the classification depth head with a regression head for comparison, as shown in Table 6. The classification approach achieves better accuracy, with the optimal bin number being 64. The improved performance can be attributed to the better alignment between the task complexity and the lightweight classification head, which helps alleviate fitting difficulty. Especially under challenging cases with significant occlusion, the predicted MPJPE improved by 4. Lmm (77.2- - 73.1). We also observe that using too many bins can degrade performance.

---

Robustness to noisy 2D pose. We add random Gaussian noise to the imput 2D pose to compare the performance under varying noise levels in Fig. 9. As 2D pose estimate quality decreases, our method retains higher accuracy than others, demonstrating lower sensitivity to 21D pose estimator stability and greater practical flexibility. As shown in Fig. 10, our adaptive anchors demonstrate superior generalization under challenging scenarios, maintaining distribution close to the ground truth even with inaccurate 2D poses.
2D pose prior based feature sampling. We verify the effectiveness of 2D pose prior based feature sampling as shown in Table 7. At the batch size 176, our strategy achieves stable performance while using nearly half the GPU memory compared to utilizing the full multi-scale feature map. In contrast, random sampling of an equal number of features leads significant performance drop, indicating that our approach efficiently captures essential features while reducing environmental noise.
4.5 Qualitative analysis
To intuitively demonstrate the superiority and generalization of our method, we evaluate our method on samples out of datasets (i.e., Human3.6M and MPI-INF-3DHP) as in Fig. 12. To ensure diversity, select different events, viewpoints, and clothing, and also test model virtual avatars. Our method exhibited promising results, revealing potential for real-world applications. Additional visualizations, analysis of efficiency and failure cases are provided in Appendix A.
5
Conclusions
In this paper, propose PandaPose, novel approach for 3D human pose lifting by propagating 2D pose prior to 3D anchor space as a unified intermediate representation. 3D anchor space comprises: woint-wise 3D anchors in the canonical coordinate system that offer accurate and robust priors to mitigate errors from 2D pose estimation; depth-aware joint-wise feature lifting that hicrarchically integrates depth information to resolve ambiguities caused by self-occlusion; and mnchor-feturn interaction decoder that combines 3D anchors with lifted features to generate unified anchor queries encapsulating joint-wise 3D anchor sets, visual cues, and geometric depth information. The anchor queries are further employed to facilitate anchor-to-joint ensemble prediction. Experiments on Human3.6M, MPI-INF-3DHP and 3DPW datasets demonstrate that PandaPose only addresses the aforementioned challenges but also achieves state-of-the-art performance, especially under challenging scenarios. Limitations and broader impacts are discussed in Appendix A.7.
Acknowledgment.
This work is supported by the National Natural Science Foundation of China (Grant No. 62271221) and Taihu Lake Innovation Fund for Future Technology, Huazhong University of Science and Technology (HUST), under Grant 2023-B-8. This research is also supported by Joey Tianyi Zhou's A*STAR SERC Central Research Fund (Use-inspired Basic Research).
References
[1] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In IEEE Conf. Comput. Vis. Pattern Recog., pages 6009-4018, 2021. 6
[2] Yujun Cai, Liuhao Ge. Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, and Nadia Magnenat Thalmann. Exploiting ppatial-tempora relationships for 3d pose estimation via graph convolutional networks, In Proceedings of the IEEE/CVF international conference computer vision, pages 1272-2281 2019, 8
[3] Btang-Chiehi Chen, George Papandreou, lasonas Kokkinos. Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell., 40(4):834 848, 2017. 14
[4] Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cascaded pyramid network for 2018. 2, 3, 8, 1.5
pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7103-7112,
[5] Seunggeun Chi, Hyung gun Chi, Qixing Huang, and Karthik Ramani. Infogen++: Learning representation by predicting the future for online skeleton based action recognition. IEEE Trans. Pattern Anal. Mach. Intell., 2024.
10

---

[6] Xiaomeng Chu, Jiajun Deng. Yuan Zhao, Jianmin Ji. Yu Zhang, Houqiang Li, and Yanyong Zhang. Oa-bev: Bringing object awareness bird's s-yee-view representation for multi-camera 3d object detection. arXiv preprint arXiv:230100571, 2023. 2, 3.3,
[7] Alexey Dosovitskiy. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv
preprint arXiv:2010.1192) 2020. 6
[8] Linpu Fang, Xingyan Liu, Li Liu, Hang Xu, and Wenxiong Kang. Jgr-p2o: Joint graph reasoning based pixel-to-offset prediction network for 3d hand pose estimation from a single depth image. In Eur. Conf. Comput. Vis., pages 120-137, 2020. 3
[9] Zigang Geng, Chunyu Wang. Yixuan Wei, Ze Liu, Houqiang Li, and Han Hu. Human pose compositional tokens. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 660-671, 2023.
[10] Jia Gong, Lin Geng Foo, Zhipeng Fan, Qiuhong Ke, Hossein Rahmani, and Jun Liu. Diffpose: Toward more refiable 3d pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 13041-13051, 2023. 7,
[1I] Nate Hagbi. Oriel Bergig. Jihad El-Sana, and Mark Billinghurst. Shape recognition and pose estimation for mobile augmented reality. IEEE Trans. Vis. Comput. Graph., page 1369-1379, Oct 2011. 2
[12] Weiting Huang, Pengfei Ren, Jingyu Wang, Qi Qi, and Haifeng Sun. Awr: Adaptive weighting regression for 3d hand pose estimation. In AAAL, volume 34, pages 11061-11068, 2020. 3
[13] Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans. Pattern Anal. Mach. Intell., 36(7): 1325-1333,, 2013. 3, 4, 8, 14, 15
[14] Changlong Jiang. Yang Xiao, Cunlin Wu, Mingyang Zhang, Jinghong Zheng, Zhiguo Cao, and Jocy Tianyi Zhou. Arjetrensf Anchor-to-jon transformer network for 3d interacting hand pose estimation from single rgb image. In IEEE Conf. Comput. Vis. Pattern Recog,, pages 8846-8855, 2023, 2, 3,
[15] Woo-Shik Kim, Antonio Ortega, PoLin Lai, Dong Tian, and Cristina Gomila, Depth map distortion analysis for view rendering and depth coding. In IEEE Int. Conf. Image Process., pages 721-724, 2009. 2
[16] Branislav Kisačanin. Vladimir Pavlovic, and ThomasS. Huang, Real-Time Vision for Human- Human-Compuen Computer Interaction. Jan 2005. 2
[17] Hongyang Li, Hao Zhang, Zhaoyang Zeng, Shilong Liu, Feng Li, Tianhe Ren, and Lei Zhang. Dfa3d: 3d deformable attention for 2d-to-3d feature lifting. In Int. Conf. Comput. Vis., pages 6684-6693, 2023. 2,
[18] Shichao Li, Lei Ke, Kevin Pratama, Yu- fu-Wing Tai, Chi-Keung Tang, and Kwang- Ting Cheng. Cascaded deep monocular 3d human pose estimation with evolutionary training data, IEEE Conf. Comput. Vis. Pattern Recog,, pages 6173-6183, 2020.
[19] Wenhao Li, Hong Liu, Runwei Ding, Mengyuan Liu, Pichao Wang, and Wenming Yang. Exploiting temporal contexts with strided transformer for 3d human pose estimation. IEEE Trans. Multimedia, 25:1222-1293, 1293, 2022. 3
[20] Wenhao Li. Hong Liu. Hao Tang, Pichao Wang, and Luc Van Gool. Mhformer: Multi- hypothesis transformer for 3d human pose estimation. IEEE Conf. Compur. Vis. Pattern Recog, pages 13147- 13156, 2022. 2, 3, 7, 15, 16
[21] Wenhao Li. Mengyuan Liu, Hong Liu, Tianyu Guo, Ti Wang, Hao Tang, and Nicu Sebe. Graphmlp: A graph like architecture for 3d human pose estimation. Pattern Recognition, 158:1 2025.
[22] Ruixu Liu, Ju Shen, He Wang, Chen Chen, Sen-ching Cheung, and Vijayan Asari. Attention mechanism exploits temporal contexts: Real-time 3d human pose reconstruction. In IEEE Conf. Comput. Vis. Pattern Recog., pages 5064 0044-50733200 2020.
[23] Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint 2rXiv:2201.12329 2022. 3
[24] Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv: 171.:i:::111.05 1.05101, 2017. 8
[25] Jameel Malik, Ibrahim Abdelaziz, Ahmed Elhayek. Soshi Shimada, Sk Aziz Ali, Vladislav Golyanik, Christian Theobalt, and Didier Stricker. Handvoxnet: Deep voxel- network for 3d hand shape and pose estimation from a single depth map. In IEEE Conf. Comput. Vis. Pattern Recog.. pages 7113-7122 2020. 3
11

---

[26] Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko. Weipeng Xu, and Christian Theobalt. Monocular 3d human pose estimation the wild using improved enn supervision. In 2017 International Conference on 3D Vision, pages 506-516. IEEE, 2017. 3, 8,
[27] Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. human pose estimation in video with temporal convolutions and semi-supervised training. In JEEE Conf. Comput. Vis. Pattern Recog., pages 7753-7762, 2019, 2, 7.8
[28] Jihua Peng, Yanghong Zhou, and PY Mok. Ktpformer: Kinematics and trajectory prior knowledge- enhanced transformer for 3d human pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1123-1132, 2024. 2, 3, 15
[29] Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Eur. Conf. Comput. Vis., pages 194-210. Springer, 2020. 3
[30] Cody Reading, Ali Harakeh, Julia Chae, and Steven L Waslander. Categorical depth distribution network for monocular 3d object detection. In IEEE Conf. Comput. Vis. Pattern Recog.. pages 8555-8564, 2021. 6
[31] Wenkang Shan, Zhenhua Liu, Xinfeng Zhang, Shanshe Wang. Siwei Ma, and Wen Gao. P-stmo: Pre- trained spatial temporal many-to-one model for 3d human pose estimation. In Eur. Conf. Comput. Vis., pages 461 478.. Springer, 2022. 3,7
[32] Vinkle Srivastav, Keqi Chen, and Nicolas Padoy, Selfpose3d: Self-supervised multi-person multi-view 3d pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog,, pages 2502-2512, 2024. 3
[33] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-rosolution representation learning for human pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pages (693-5703, 2019. 3, 4, 8,
[34] Svenstrup. S. Tranberg. H.J. Andersen, and T. Bak. Pose estimation and adaptive robot behaviour for human-robot interaction. In 2009 IEEE International Conference on Robotics and Automation, May 2009.
[35] Zhenhua Tang, Zhaofan Qiu, Yanbin Hao, Richang Hong, and Ting Yao. 3d human pose estimation with spatio-temporal criss-cross atlention. In IEEE Conf. Comput. Vis. Puttern Recog., pages 4790-4799, 2023.
[36] H. Teng, D. Chatziparaschis, X. Kan, A. Ray-Chowdhury, and K. Karydis. Centroid distance keypoint detector for colored point clouds. 2023. 8
[37] Hanyue Tu, Chunyu Wang, and Wenjun Zeng. Voxelpose: Towards multi-camera 3d human pose estimation
in wild environment. In Eur. Conf. Comput. Vis., pages 197-212, 2020. 3
[38] A Vaswani. Attention is all you need. In Adv. Neural Inform. Process. Syst., 2017. 3
[39] Timo Von Marcard, Roberto Henschel, Michael J Black, Bodo Rosenhahn, and Gerard Pons-Moll. Re- covering accurate 3d human pose in the wild using imus and a moving camera. In Proceedings of the European conference on computer vision (ECCV), pages 601-617, 2018. 3, 8, 14
[40] Jingbo Wang, Sijie Yan, Yuanjun Xiong, and Dahua Lin. Motion guided 3d pose estimation from videos. In Eur. Conf. Comput. Vis., pages 764-780. Springer, 2020. 3
[41] Weiquan Wang. Jun Xiao, Chumping Wang. Wei Liu, Zhao Wang, and Long Chen. Dipose: Discrete diffusion model for occluded 3d human pose estimation. CORR, 2024. 8
[42] Xiong, Boshen Zhang, Yang Xiao, Zhiguo Cao. Taidong Yu, Joey Tianyi Zhou, and Junsong Yuan. A2j: Anchor-to-joint regression network for 3d articulated pose estimation from a single depth image. In Int. Conf. Comput. Vis., pages 793 802, 2019, 2, 3.4
[43] Tianhan Xu and Wataru Takano. Graph stacked hourglass networks for 3d human pose estimation. In IEEE Conf. Comput. Vis. Pattern Recog. pages 66105-16114, 14, 2021. 7,7
[44] Tingbing Yan, Wenzheng Zeng, Yang Xiao, Xingyu Tong, Bo Tan, Zhiwen Fang, Zhiguo Cao, and Jocy Tianyi Zhou, Crossglg: Llm guides one-shot skeleton-basee 3d action recognition in a cross-level manner. arXiv preprint 2403 10082, 2024. 2
[45] Hang Ye, Wentao Zhu, Chunyu Wang. Rujie Wu, and Yizhou Wang. Faster voxelpose: Real-time 3d human pose estimation by orthographic projection. In Eur. Conf. Comput. Vis., pages 159, 2022. 3
12

---

[46] Zhu Yu, Runmin Zhang, Jiacheng Ying, Junchen Yu, Xiaohai Hu, Lun Luo, Si- Si-Yuan Cao, and Hui-Liang Shen. Context and geometry aware voxel transformer for semantic completion. arXiv preprint urXiv:4405136675 2024.
[47] Ailing Zeng. Xiao Sun, Fuyang Huang, Minhao Liu, Qiang Xu, and Stephen Lin. Smet: Improving generalization in human pose estimation with a aplit-and-recombbe approach. Eur. Conf. Comput. Vis., pages 507-523. Springer, 2020. 3
[48] Ailing Zeng, Xiao Sun, Lei Yang. Nanxuan Zhao, Minhao Liu, and Qiang Xu. Learning skeletal graph neural networks for hard 3d pose estimation. In Int. Conf. Comput. Vis., pages 11436- 11445, 2021. 2, 3, 7,
[49] Jianfeng Zhang. Kehong Gong, Xinchao Wang, and Jiashi Feng. Learning to augment poses for 3d human pose estimation in images and videos. IEEE transactions on pattern analysis and machine intelligence, 1508):10012-10 10026, 2023. 8
[50] Jinlu Zhang, Zhigang Tu, Jianyu Yang, Yujin Chen, and Junsong Yuan. Mixste: Seq2seq mixed spatio- temporal encoder for 3d human pose estimation in video. In IEEE Conf. Comput. Vis. Pattern Recog., pages 13232-13242, 2022. 2, 3, 7, 15, 16
[51] Renrui Zhang. Han Qiu, Tai Wang. Ziyu Guo, Ziteng Cui, Yu Qiao, Hongsheng Li, and Peng Gao. Monodetr: Depth-guide transformer for monocular 3d object detection. In Int. Conf. Comput. Vis., pages 9155-9166, 2023. 3
[52] Si Zhang, Hanghang Tong, Jiejun Xu, and Ross Maciejewski. Graph convolutional networks: a compre- hensive review. Computational Social Nenworks, 6(1):1-23 2019. 3
[53] Siqi Zhang, Chaofang Wang. Wenlong Dong, and Bin Fan. A survey on depth ambiguity of 3d human pose estimation. Applied Sciences, 12(20):10591, 2022.
[54] Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, and Dimitris N Metaxas. Semantic graph convolutional networks for 3d human pose regression. In IEEE Conf. Comput. Vis. Pattern Recog., pages 3425 3435, 2019. 2,
[55] Qitao Zhao, Ce Zheng, Mengyuan Liu, and Chen Chen. single 2d pose with context is worth hundreds for 3d human pose estimation. In Adv. Neural Inform. Process. Syst., volume 36, 2024. 2, 3, 4, 7, 8, 14, 15, 16
[56] Weixi Zhao, Weiqiang Wang, and Yunjic Tian. Graformer: Graph-oriented transformer for 3d pose estimation. In IEFE Conf. Comput. Vis. Pottern Recog,, pages 20438-204417 2002-2204,2222.... 7,8
[57] Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang, Chen Chen, and Zhengming Ding. 3d human pose estimation with spatial and temporal transformers. In Int. Conf. Comput. Vis., pages 11656-16 11665, 2021. 2, 3, 7,8
[58] Hongwei Zheng, Han Li, Wenrui Dai, Ziyang Zheng, Chenglin Li, Junni Zou, and Hongkai Xiong. Hipart: Hicrarchical pose autoregressive transformer for occluded 3d human pose estimation. arXiv preprint arXiv:2503.2333), 2025. 7,8
[59] Feng Zhou, Jianqin Yin, and Peiyang Li. Lifting by inage-leveraging image cues for accurate 3d human pose estimation. In AAAI, volume 38, pages 7632-7640, 2024. 2, 3, 7, 8
[60] Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, and Yizhou Wang. Motionbert: A unified perspective on learning human motion representations. In Int. Conf. Comput. Vis., pages 15085-15099, 2023. 2
[61] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for and-to-end object detection. arXiv preprint 2001:22010.04159, 2020. 7
13

---

Jaint-wise depth distribution maps
Conv2d 3x3
Dilate Conv
ASPP Conv
Conv2d 1x1
Proj
Conv2d
Image feature map
Transformer encoder layer
Depth embedding
Figure Illustration of weight depth network.
MPJPE (Challenging subset) 27377 samples
MPJPE (Non-challenging subset) 515967 samples
MPJPE (Fall set) 543344 samples
CA-PHI55] PandaPose (ours)
82.4
39,9
41.4 39.8 (1.66)
73.1 (9.3.)
(8.8).
Table Performance comparison of different subsets on Human3. 6. set. The non-challenging
subset refers to the remaining samples after excluding the challenge subset from the full test set.
A Technical Appendices and Supplementary Material
A.1 Datsets and evaluation metrics
Human3.6N [13] is widely used benchmark for 3D human pose estimation. Following previous protocols, we train our model on subjects (S1, S5, S6, S7, S8) and evaluate on 2 subjects (S9, S11). We report MPJPE and PA-MPJPE Human3.6M. We select samples with error between the predicted 2D pose and GT 2D pose as the challenging subset (27.4k samples, about 5% in test set). These samples with 2D pose inaccuracy generally have strong occlusions confusing pose patterns, as shown in Fig. 11.
PPI-INF-3DHD [26] is also widely used benchmark for 3D human pose estimation, collected in both indoor and challenging outdoor environments. We report PCK (Percentage Correct Keypoint) with the 150 mm range, AUC (Area Under Curve) and MPJPE as evaluation metrics. Following the same principles as described above, we selected 11I samples (about 4%) as the challenging subset.
3DPW [39] is challenging in the-wild dataset. We train our model on Human3.66 3.6 and test it on 3DPW evaluate the generalization ability. We report MPJPE and PA-MPJPE on 3DPW.
A.2 Depth network structure
The lightweight depth network we employ is illustrated in Fig. 13. This network has approximately 5M parameters and composed multiple convolutional layers. takes a single-layer image feature map as input, which is then processed through a feature projection layer. Subsequently, the convolutional layers extract deep features, which are further refined by an ASPP Fig.[3] (Atrous Spatial Pyramid Pooling) module to capture multi-level information. From this process, two outputs are derived:
The depth features are passed through Ix1 convolution to generate depth distribution maps for each joint.
A single-layer Transformer encoder, comprised of the self-attention mechanism, processes the depth features to produce depth embedding.
A.3 Performance analysis on different subsets
In Table 1, we compare the performance of PandaPose with other SOTA methods on the full test set as well as on the challenging subset to demonstrate superiority, particularly terms of robustness to occlusions and 2D pose inaccuracies. Furthermore, it would raise concern that improvement on difficult samples could come the expense of weaker performance on easier cases. To address this concern, conducted additional experiments in Table 8 by excluding the challenging subset from
14

---

Imput
CA-PF
Ours
Ground truth
Input
CA-PF
Ours
Ground truth
Figure 14: Visual comparison in challenging cases. The circles highlight locations where our method has better predictions.
CA-PF
Ours
30
Figure 15: Failure case visualization under motion blur self-occlusion.
the full test set, defining it the non-challenging subset, and compared it with CA-PF [55]. Our approach exhibits notably improvements even on relatively normal cases.
A.4 Performance analysis on different 2D pose estimator
We select three widely used 2D pose estimators (i.e., CPN [4], HRNet-w32 [33], HRNet-w4l w48 [33]) for ablation experiments to examine how predicted 3D pose quality varies with input 2D accuracy. The results are listed in Table 10. The accuracy of PandaPose remains relatively stable with respect to the errors of the 2D pose estimator. Compared with CPN and HR-Net-w48, the 2D accuracy (mAP) decreases by 5.5%, but the error of PandaPose only increases by 1.5%. Meanwhile, in Figure 9 of the main text, we gradually add Gaussian noise to the 2D pose to study the method tolerance to noise levels and
method maintains the best anti-noise ability. When the pixel-level noise scale ranges from 0 to 5, the eтror of PandaPose only increases by 16% (from 39.8 to 48.2), while the image-based SOTA CA-PF [55]increases by 56% (from 41.4 to 64.9), and the sequence- basendd SOTA KTPFormer [28] increases by 112% (from 40.1 to 86.9). The noise resistance can be attributed to different 3D anchor setting and anchor-to-joint ensemble prediction mechanism.
A.5 Running efficiency comparison
Table 9 presents an efficiency comparison among our proposed method (PandaPose), the state- of-the-art image based method (CA-PF [55]) and sequence based methods (MixSTE [50] and MHFormer [20]) on Human3.6M [13]. All metries are evaluated NVIDIA RTX3090 GPU.
15

---

Lifting Module Parameters(M)
MPJPE (Full)
MPJPE L (Challenging)
Method
Sequence hased
MHFormen(TT (=2)) [20]
24.8
45.9
MHFormer(T=351] [20]
24.8
43.0
MixSTE(T=243) [50]
33.6
40.9
Image based
CA-PF [55]
19
14.1
41.4
82.4
PandaPose w/ decoder
15.2
39.8
73.1
PandaPose w/ decoder
12.7
40.3
75.6
Table 9: Running efficiency comparison.
2D pose estimator
mAP COCO f MPJPE on Human 3.6M
CPN HR-Net-w33
68.6
40.1
74.4
39.8
HR-Nettw44
76.3
39.5
Table 10: Comparison of 2D pose estimators on COCO and Human 3.6M datasets.
Due to the differences in experimental setup, we omit the FPS metric for the requence-based methods. Compared the sequence-based methods, PandaPose takes fewer parameters while maintaining comparable performance. To demonstrate the superiority of PandaPose, we reduce the number of layers in the decoder from to This adjustment result in parameter reduction of 2.5M, with only a performance drop of 0.5mm on the full test set and 2.5mm on the challenging subset. Compared CA-PF [55], PandaPose reduces parameters by 1.4M while maintaining comparable running efficiency. Notably, on the challenging subset, PandaPose outperforms CA-PF by 6. 6.8mm MPJPE. This also indicates that PandaPose holds essential potential to further efficiency enhancement, but still ensuring promising performance.
A.6 More visualization results
We present more visualization cases in Fig. 14 and compare them with state-of-the-an image based method (CA-PF [55]) and ground truth. Our method demonstrates superior handling of the relative depth relationships between joints when dealing with severe self-occlusion noisy 2D pose inputs, resulting in more accurate 3D pose predictions.
representative failure cases shown in Fig. 15, In scenarios with severe motion blur or self-occlusion, the human subject may become confused with the background, leading inaccurate predictions of 2D poses. Thus, the quality of 3D pose predictions will be affected. Our method, due to its explicit modeling of the relative depth relationships between joints and the ensemble prediction, is capable of predicting relatively more reasonable
Additionally,
integration of anchor-to-jon 3D human poses in such scenarios.
A.7 Limitations and boarder impacts
Despite the achievements of our method in image based 3D human pose estimation, several limitations remain that are worth further improvement. As a finglefrram method, PandaPose lacks temporal smoothness compared sequence- basedd based methods, which use information from adjacent frames. This can lead to jitter pose estimation during continuous actions, especially with rapid or complex movements. Additionally, PandaPose introduces the processing of image features and complex operations including feature lifting and ensemble prediction the 3D anchor space. While these steps improve accuracy, they inevitably come with computational resource cost. Future research will aim to develop more lightweight approaches with similar performance but reduced computational and memory costs, exploring efficient feature lifting and optimization methods. Our approach exclusively utilizes publicly available datasets during the training process, thereby having no broad societal impact, not involving AI ethics, and involving any privacy-sensitive data.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The paper accurately conveys the contributions and scope of this work in the abstract and introduction sections, and provides bullet-point summary at the end.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations of the method in Appendix A.7.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch provide intuition.
17

---

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All experiments presented in this paper are reproducible. We will release the code and model following the acceptance of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We will release the code and model after the acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc/
for more details.
18

---

While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: gwwgggdi//oceeuuiiiee
/ //niss
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide a detailed description of our experimental setup and results in Sec. 4 of the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Our experiments are stable across multiple runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the eтror bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
19

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Sec. and Appendix A.5, we provide a detailed account of our computa- tional overhead and model efficiency.
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htth:://ttts://eeeeee http://sccccccbiiiitcttccccciaaa
Answer: [Yes]
Justification:
The paper adheres to the NeurIPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: In Appendix A.7, elaborate the lack of societal impact of our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
20

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks. mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper poses no such risks as elaborated Appendix A.7.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper employs publicly available datasets and code for training and comparative evaluation, adhering all protocol restrictions that accompanied their release, and cites the relevant literature.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi twooee com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Upon acceptance of the paper, we will release our model and code under the CC BY-SA 4.0 license.
Guidelines:
21

---

The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasct/cod//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The core method development in this research does involve LLMS any important, original, or non-standard components, we only use LIM during paper writing for grammar check and polish.
Guidelines:
22

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy (https: /ttprri////upp.../ httpippssc///oofeeencn//////// 22055/2222222/22 for what should or should not be described.
23


Reviews:
[{"review_id": "eIvCdq8WnD", "reviewer": "Reviewer_2274", "summary": "The paper proposes a method to improve the accuracy and robustness of 3D pose estimation.\n\nThe approach builds on top of 2D pose estimation and aims at \n(i) overcoming limitations of 2D pose estimation accuracy\n(ii) handling the issue of overlapping joints (self-occlusion) and related problems of per-joint depth estimation\n\nFor (i) the authors introduce an \"adaptive anchor sampling\" method, whereby they sample multiple positions around the estimated 2D joint, and use these as additional hypotheses beyond the bottom-up 2D pose estimate. Each of those anchors points provides its own 3D joint estimate, which are then aggregated into a single estimate through a convex combination - the combination weights are set by a softmax unit. \n\nFor (ii) the authors estimate a separate depth map per joint, rather than using a single depth channel. This allows every joint to receive its own supervision at training time, and at test time can provide distinct depth estimates for overlapping joints, thereby handling self occlusion. \n\nSeveral ablation studies confirm the merit of the main contributions, while the results outperform the state-of-the-art on the Human3.6 dataset.", "questions": "Could you please provide timings of this method compared to competing baselines? This would be very relevant to interactive applications like the ones described in the introduction.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \n- Some novel components are fairly validated - e.g. clear gains thanks to anchor strategy, demonstrated robustness to pose errors (also shown in Table 9).\n\n- Interesting improvements on H3.6M compared to similar methods that require a sequence as input. \n\n\nWeaknesses:\n- Poor presentation. Even though I am quite knowledgeable in the field, it took me a long time to read the paper. \n* The introduction section is too long. The authors attempt to cram the whole method in the intro with an intricate technical description that tries to be intuitive but is uninitelligible. For instance l. 43-45, 48-51, 57-59 are filled with technical jargon that is not explained, and the reader is left to guess what the authors mean. The introduction should be a high-level overview of the method, not a detailed description of the components.\nBeyond that the text is frequently confusing, with convoluted language that does not help the reader understand the method. For instance, l. 59-61:  \"For the supervision, in the absence of dense ground truth depth maps, we extract the depth value of ground truth joints instead of the complete depth as supervision.\" \nThis is a very verbose way of saying your depth loss is defined only the respective joints - and is out of place for an introduction.\n\nBeyond confusing the reader, parts of the introduction read like propaganda, where the authors make claims that are not substantiated by the text immediately preceding or following them - for instance the text in lines 61 to 65 is impossible to validate before someone has read the full paper even then it’s unclear if this is true (maybe this is true for the second part)\n\n* having spent two pages on an intro, then the authors spend another page on related work. They then try to cram every useful illustration or discussion in tiny figures of very low resolution, and with captions that do not explain what the reader should take away from them. For instance the  captions of Fig 11,12 do not explain what there is notice, and what the conclusion should be. Also the resolution is so low that one can hardly see what is in the image (eg MPI on 11)\nSimilarily, for Figure 10 an experiment is described ony in caption (not in the text), and the images are of very low resolution. It is impossible to see anything. After careful inspection it seems that for the occlusion the centroid has moved quite a bit. It is not clear if this is expected.\n\n- Complexity: partly because of the poor presentation, but more importantly because the method is quite complicated, I find it hard to see what to take away from the paper. \nThere are many bells-and whistles which are introduced like systems, with little justification: e.g. in lines 224-227: \"Each decoder layer comprises a depth cross-attention layer, an inter-anchor self-attention layer, and a 3D deformable cross-attention layer\". This design is never validated compared to a simpler baseline, and it is impossible to see if it is really necessary and how to reproduce it. \n\nSome other things that are fairly standard are introduced with an unwarranted level of complexity - e.g. the idea of using a separate depth map per joint. A whole section is dedicated to this (3.3), the authors mention (l. 179): \"we innovatively design a joint-wise approach to predict individual depth distributions for each joint as shown in Fig 3\" \n\nwhile it is common practice in multi-person pose estimation  to predict per-joint depth maps or depth offsets: \nSMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation, ECCV 2020 \n3D Human Pose Estimation via Explicit Compositional Depth Maps, AAAI 2020\nHEMlets Pose: Learning Part‑Centric Heatmap Triplets for Accurate 3D Human Pose Estimation” (Zhou et al., ICCV‑19)\n\nResults:\nHard to say if the present results are practically siginificant. The main evaluation is on Human3.6M which is plagued by a very low number of subjects and background variation. The results on 3DPW are more interesting, but the authors do not compare with other methods that perform better. In particular over the past five years, several works on 3D human mesh recovery (arguably a harder problem than 3D pose estimation, that is also studied by a bigger part of the community) have reported substantially better results on 3DPW, e.g.: \n\nHMR 2.0:  Humans in 4D: Reconstructing and Tracking Humans with Transformers\nCameraHMR: Aligning People with Perspective: \nTokenHMR: Advancing human mesh recovery with a tokenized pose representation\n\nUnderstandably, some of them use different datasets for training. But I still find it more illuminating to use datasets and benchmarks similar to those used in the main 3D pose estimation works, rather than only look at Human3.6M results. \n\n\nMinor issues: \n\nMissing references: please see above on joint-level depth maps. Also other works that have similar ideas - e.g. \novercoming the spatial resolution of 3D voxels: Integral Human Pose Regression, ECCV 2018\nPer-joint depth estimation: I2L-MeshNet, ECCV 2020 \n\nPose prior: misnomer. There is no prior idea in the sense of prior knowledge, or prior distribution (it is conditional on the 2D joints found in the input image)\n\nCanonical coordinate system: misnomer. For humans a canonical coordinate system would be pose invariant in 3D e.g. an A-pose skeleton. I do not see anything like that in the paper, rather the authors use a coordinate system that is defined by the 2D pose estimate, which is not canonical in any sense.", "was_revised": true, "final_justification": "I remain on the fence regarding this paper because of the presentation and narrow focus of the work, but I understand that the authors are serious about their evaluation and have done their best to address the feedback of all reviewers. \nI stick to my borderline reject recommendation since I still think that there is lots of space for improvement - but I would not object to the paper getting accepted if the authors do the extra work required for the camera-ready version.", "rebuttal": "We appreciate your careful reading of our manuscript and your comments on the writing. Actually, compared with previous methods, PandaPose has many differences in both the overall framework and specific design implementation. Due to limited space, we spend considerable effort in the introduction to help readers quickly establish the logical flow (problem - motivation - method - results), and provide further details in later sections. In the future, we will strengthen the connection between the concepts in the introduction and Figure 1, optimize the layout, and add explanatory content to eliminate misunderstandings (if accepted, there will be an additional page). Meanwhile, our writing has been generally well-received by the other four reviewers, with an average score of clarity above 3. Next, we will address all your concerns below.\n## **Introduction writing**\nFor Line 43-45, we aim to clarify that the overall architecture is built on a more advantageous initial distribution of 3D anchors and the anchor-to-joint mapping approach. This is the key innovation and main advantage of our method. Line 48-51 then detail the design of the 3D anchors. Line 57-59 and 61-65 summarize the introduction, analyze the design concepts and motivations, and explain the method's superiority in handling challenging samples, as well as its efficiency and accuracy. We expect readers to grasp important concepts such as 3D anchor space, anchor-to-joint mapping, and joint-wise depth early in the introduction.\n\n## **LayerOut and Figure**\nFollowing the comparison protocol as previous works (CA-PF, MixSTE etc.), **Figure 11**,  as a qualitative comparison, highlights the parts our method has better prediction in circles as in the caption. **Figure 12** serves as an OOD experiment to tests the generlization. **Figure 10** is dedicated to demonstrating the robustness of the proposed joint-wise anchor in the case of 2D pose prediction errors as in Line 323 and the caption. All figures in paper are vector graphics and carefully designed. It is recomended to view the paper through a standard PDF reader and zoom in on the illustrations for clearer observation of details. We will enlarge some images into full-width figures in subsequent versions for better viewing.\n## **Complexity of decoder**\nRegarding the decoder design, our approach is an improvement upon the standard deformable Transformer decoder\\[1\\] (comprising self-attention and deformable cross-attention). Specifically, we introduce a depth-domain cross-attention module and extend the original 2D deformable cross-attention to 3D deformable cross-attention. It is worth emphasizing that the deformable attention mechanism has been widely validated for its effectiveness across numerous computer vision tasks. Its successful application is also evidenced by pose estimation related works such as DeforHMR\\[2\\] (3DV 2025) and DeFormer \\[3\\](CVPR 2023), providing a solid foundation for our design choice.\nWe further supplement the ablation experiment on the improvement of prediction by the improved components as table below. Both modules are designed to model the depth domain, and experiment proves that they can significantly enhance 3D pose prediction quality, as reflected by the decreasing MPJPE.\n\n| Depth cross-attention | Deformable attention type | MPJPE on Human3.6M |\n| --- | --- | --- |\n| no | 2D | 41.2 |\n| yes | 2D | 40.9 |\n| no | 3D | 40.3 |\n| yes | 3D | 39.8 |\n\n> [1] Deformable DETR: Deformable Transformers for End-to-End Object Detection\n>\n> [2] DeforHMR: Vision Transformer with Deformable Cross-Attention for 3D Human Mesh Recovery\n>\n> [3] Deformable Mesh Transformer for 3D Human Mesh Recovery\n## **Joint-wise Depth Clarification**\nOverall, our approach emphasizes the necessity of accurate joint-wise absolute depth for 3D pose estimation, differing fundamentally from the methods you mentioned. SMAP[1] models root joint depths in multi-person scenarios without estimating specific joint absolute depth values, leading to potential error accumulation. The limb depth map method[2] interpolates regressed joint depths based on limb rigidity, limiting accuracy due to interpolation errors and failing in non-rigid motions. HEMlets[3] encodes only relative depth ordering without quantitative depth values, which is insufficient for precise 3D coordinate calculation.\nIn contrast, our method directly estimates and utilizes joint-specific metric depths, avoiding error accumulation and capturing local depth variations caused by joint movements. Unlike SMAP's reliance on a long estimation chain prone to noise and limb depth maps' dependence on rigid limb assumptions, our approach ensures robust and high-precision 3D positioning through direct quantitative depth estimation.\n\nMoreover, the aforementioned methods lack theoretical analysis on joint-wise depth estimation. In Section 3.3, we provide a detailed theoretical and technical justification of our approach—setting us apart from previous works. Through qualitative and quantitative analysis (Figure 5, Figure 6, Table 5), we clearly articulate the motivation behind our design. Technically, our method introduces a lightweight depth encoder that predicts a joint-wise depth map with real physical values in a classification manner—distinct from existing approaches. This provides a more accurate and robust depth prior, enabling more effective feature interaction in the subsequent decoder.\n> [1] SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation\n>\n> [2] 3D Human Pose Estimation via Explicit Compositional Depth Maps\n>\n> [3] HEMlets Pose: Learning Part‑Centric Heatmap Triplets for Accurate 3D Human Pose Estimation\n## **Result on 3DPW**\nOur experimental setup is specifically designed to evaluate cross-dataset generalization—training on Human3.6M and testing on 3DPW. This protocol is widely adopted to assess model robustness under domain shifts, such as differences in camera setups, subjects, and environments. In Table 3, all compared methods are SOTA approaches evaluated under the same cross-dataset setting, focusing on single-person 3D human pose estimation.\nWe follow mainstream practices in our comparisons, and the performance metrics for all cited methods are taken directly from their publicly available results—including latest HiPART (CVPR 2025), ensuring both fairness and up-to-date benchmarking.\nWe emphasize that our goal is not to establish new absolute performance records on 3DPW spanning different settings and methodological paradigms, but rather to evaluate how well models generalize from controlled, lab-based settings (Human3.6M) to more challenging, in-the-wild scenarios (3DPW). This evaluation paradigm is highly relevant in real-world applications, especially when labeled in-the-wild data is limited or expensive to acquire.\n\nIt is worth noting that many mesh-based methods achieving strong performance on 3DPW are trained on large, mixed datasets that include in-the-wild or synthetic data. A direct comparison under such conditions would be unfair. To enable a more equitable comparison with mesh-based approaches, we retrain TokenHMR (CVPR 2024) under our same experimental setup. Under this setting, TokenHMR achieves an error metric MPJPE of 78.5 on 3DPW, which is higher than our result of 74.9, further demonstrating the effectiveness of our method in this generalization scenario.\n## **Clarification of Human3.6M**\nIn response to the reviewer's doubts about whether the results on Human3.6M are practically significant, we would like to clarify that Human3.6M remains a widely recognized standardized benchmark in the field of 3D pose estimation, with advantages in high-quality and controllable annotations. Recently published papers still use this dataset as a core benchmark ( HiPART[CVPR25], UPose3D[ECCV24] etc.).\nWe also observe that the metrics of current 3D human mesh methods on Human3.6M are often inferior to those of 3D human pose estimation (as shown in the table below).\n\nMore importantly, although Human3.6M is a relatively \"clean\" dataset, we observe that existing pose methods still perform poorly in occlusion scenarios within this dataset. For instance, we select complex scenarios in the dataset by measuring 2D pose error > 5(line 262-263 in main text), which often represent self-occlusion and motion blur. Existing methods often lose information about key body parts, leading to a decline in overall pose estimation accuracy. This observation is precisely the motivation and core contribution of our work. Our method demonstrates significant performance improvements in these challenging subsets. For instance, on challenging subset (~5% in the full test set), the MPJPE of ours is 9.3 mm lower than that of the current SOTA method CA-PF (73.1 vs 82.4) as shown in Table 1 of main text.\n\n| Method | MPJPE on Human3.6M |\n| --- | --- |\n| ProHMR (Mesh-based)| 65.1 |\n| HMR2.0  (Mesh-based)| 52.8 |\n| ScoreHMR  (Mesh-based)| 47.9 |\n| Ours | 39.8 |\n## **Minor issues**\n1. Pose prior：In our paper, we adopt the term \"Pose prior\" following the convention of previous related works (e.g., C3P[1] [ECCV2022], DiffPose[2] [CVPR2023]), which also used this expression in similar contexts. In these works, \"prior\" does not strictly refer to prior knowledge or distribution in the Bayesian sense, but more broadly refers to the initial pose hypothesis derived from the 2D joint positions in the input image. This usage aims to emphasize the process by which the model uses the known 2D information as a basis to predict 3D poses. We will also strive to supplement further background in the revised version to eliminate ambiguity.\n\t\n2. Canonical coordinate system：The canonical coordinate system in the text refers to the 3D world coordinate system with actual physical values, rather than the 2D pose coordinate system. The 3D anchors are located in the same coordinate system with 3D human pose. We will also further strengthen this concept in the paper."}, {"review_id": "SyaQh6Nysj", "reviewer": "Reviewer_MoUK", "summary": "This work introduces PandaPose, a novel monocular 3D human pose estimation framework. In its first stage, PandaPose employs a pre-trained 2D pose detector to extract 2D joint coordinates from a single RGB image. The methodological innovation resides entirely in the second stage, which focuses on precisely and robustly estimating 3D poses by leveraging both the 2D coordinates and their associated intermediate visual features. Departing from conventional approaches that directly lift 2D poses to 3D representations, PandaPose designs a sophisticated lifting network that: (a) incorporates intermediate features from the 2D network alongside joint coordinates, and (b) introduces a 3D anchor space to jointly resolve depth ambiguity.", "questions": "As noted above, could the following be addressed in the revision:\n1. Provide detailed comparative data on computational and memory cost.\n2. Investigate the framework’s tolerance to 2D pose estimation inaccuracies—specifically, conduct ablation studies to analyze how different 2D detectors impact final 3D accuracy? This analysis should verify whether the system maintains consistent 3D pose outputs when input 2D precision varies marginally.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. ​​Innovative Methodology​​: The framework integrates (a) an adaptive 3D anchor space and (b) a multimodal feature interaction decoder, demonstrating notable methodological novelty.\n​​2. Superior Performance​​: Achieves state-of-the-art (SOTA) results on standard benchmarks, with efficacy further validated through a constructed \"challenging subset\".\n\nWeakness:\n​​1. Efficiency Limitations​​: While acknowledging extra computational and memory overhead in the appendix, parameter count alone fails to adequately characterize efficiency. We recommend supplementing with concrete quantitative metrics (e.g., GFLOPs) visualized via scatter plots to explicitly illuminate the accuracy-efficiency trade-offs.", "was_revised": true, "final_justification": "I've carefully reviewed the comments of the other reviewers. I believe the weaknesses raised by reviewer 2274 are not that critical. The rebuttal solved most of my concerns, and I'll keep my rating.", "rebuttal": "We appreciate Reviewer MoUK recognizing the novelty and superior performance in our method. Below, we address your concerns from two perspectives:\n\n## **Efficiency analysis**\n\n| method | FPS | Param | Inference GPU memory cost（M） | GFLOPs | MPJPE on Human3.6M (full) | MPJPE on Human3.6M (challenging) |\n| --- | --- | --- | --- | --- | --- | --- |\n| MixSTE (T=81) | \\- | 33.6 | 192 | 46.3 | 42.4 | \\- |\n| MixSTE (T=243) | \\- | 33.6 | 295 | 139.1 | 40.9 | \\- |\n| CA-PF (T=1) | 19 | 14.1 | 310 | 8.2 | 41.4 | 82.4 |\n| PandaPose w/3 decoder (T=1) | 16 | 15.2 | 323 | 16.6 | 39.8 | 73.1 |\n| PandaPose w/2 decoder (T=1) | 18 | 12.7 | 307 | 15.4 | 40.3 | 75.6 |\n\nWe supplement the GFLOPS metric and inference memory cost based on Table 7 in the appendix. Our method achieves superior performance with significantly fewer FLOPs compared to sequence-based MixSTE. Compared to the previous image-based SOTA CA-PF, although our method has slightly higher GFLOPs (still at an absolutely low level compared with MixSTE), it benefits from the highly parallel design of the attention implementation, resulting in comparable FPS and GPU memory cost. Moreover, our method demonstrates advantages in overall accuracy, especially in challenging cases, indicating its high efficiency and effectiveness.\nDue to the limitations of the rebuttal policy, we can only provide tables instead of scatter plots, and we will add scatter plots corresponding to the above data in the revised version. Overall, PandaPose achieves the best accuracy-efficiency balance.\n\n\n## **Tolerance to 2D pose estimation inaccuracies**\n\nWe also thank the reviewer for the advice on evaluation of the tolerance to 2D pose estimation inaccuracies.\nWe select three widely used 2D pose estimators (i.e., CPN, HRNet-w32, HRNet-w48) for ablation experiments to examine how predicted 3D pose quality varies with input 2D accuracy. The results are listed below\n\n| **2D pose estimator** | **mAP on COCO(**$\\\\uparrow$**)** | Method | **MPJPE on Human 3.6M(full)** | **MPJPE on Human 3.6M(challenging)** |\n| --- | --- | --- | --- | --- |\n| CPN | 68.6 | CA-PF | 41.6 | 83.7 |\n| CPN | 68.6 |  Ours | **40.1** | **73.7** |\n| HR-Net-w32 | 74.4 | CA-PF | 41.4 | 82.4 |\n| HR-Net-w32 | 74.4 | Ours | **39.8** | **73.1** |\n| HR-Net-w48 | 76.3 | CA-PF | 39.8 | 81.1 |\n| HR-Net-w48 | 76.3 | Ours | **39.5** | **72.3** |\n\n- The accuracy of PandaPose remains relatively stable with respect to the errors of the 2D pose estimator. Compared with CPN and HR-Net-w48, the 2D accuracy (mAP) decrease by 7.7, but the error of PandaPose only increase by **1.5%**, while CA-PF increase by **4.5%**. \n\t\n- Meanwhile, as mentioned in Lines 262-263, our criterion for selecting challenging cases is based on 2D pose prediction error, i.e., these cases are the worst-performing part of 2D pose prediction in the dataset. We outperform CA-PF by a significant margin on the MPJPE in this subset (73.1 v.s. 82.4, 12.1↓). This also demonstrates our better tolerance to 2D noise.\n\t\n- Moreover, in Figure 9 of the main text, we gradually add Gaussian noise to the 2D pose to study the method tolerance to noise levels and our method maintains the best anti-noise ability. When the pixel-level noise scale ranges from 0 to 5, the error of PandaPose only increases by **16%** (from 39.8 to 48.2), while the image-based CA-PF increases by **56%** (from 41.4 to 64.9), and the sequence-based SOTA KTPFormer increases by **112%** (from 40.1 to 86.9). The noise resistance can be attributed to the design of anchor ensemble mechanism based on local and global anchors with ensemble joint prediction.\n\t\n\nIn conclusion, PandaPose achieves state-of-the-art robustness to input 2D poses, which stems from our effective 3D anchor setup and anchor-to-joint mechanism."}, {"review_id": "ZjVIAn8jcI", "reviewer": "Reviewer_G3sQ", "summary": "PandaPose presents a new approach for 3D human pose estimation from a single RGB image, tackling issues like errors from 2D pose predictions and self-occlusion. Key innovations include setting individual 3D anchors for each joint, depth-aware feature lifting to fix depth issues, and a decoder that combines different features for better joint prediction. It also uses joint-specific depth distributions for more accurate depth modeling and a smart feature sampling method based on 2D pose priors. Tests show it outperforms existing methods, especially in complex scenarios like occlusion.", "questions": "As shown in the Weaknesses section, A reasonable explanation will make me raise the score for this work.", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\nA series of  modules based on anchor points, which work in coordination to achieve excellent performance in 3D human pose estimation:\n- Joint-wise 3D Anchors: Combine the adaptive anchors generated by the input 2D poses through a network with the fixed anchors.\n- Joint-wise Depth Estimation: Instead of predicting a single depth value, PandaPose estimates depth distributions for each joint, addressing the common issue of depth ambiguity in closely placed joints, which makes it more accurate in challenging scenarios.\n- Efficient Feature Sampling: The 2D pose prior-based feature sampling reduces computation by focusing on relevant areas around joints, which not only makes the process more efficient but also helps improve the overall accuracy by filtering out unnecessary background noise.\n- Anchor-Feature Interaction Decoder: The decoder uses a multi-modal attention mechanism, combining visual (from Efficient Feature Sampling), depth (from Joint-wise Depth Estimation), and geometric (from Joint-wise 3D Anchors) features. This helps PandaPose enhance joint localization, particularly when parts of the body are occluded.\nThis anchor-based system fully exploits the 2D pyramid features and achieves better performance.\n\n# Weakness\n## Major\n- This anchor-based system appears logically sound. However, it remains unclear what advantages anchors offer compared to directly predicting the offset of the 3D root node, and why such advantages exist. The author should provide a comprehensive analysis or reasoning for this design choice around line 310. This is my primary concern.\n## Minor\n- What's the initial Qanchor in line 218? Is it the embedding from the anchor offset? That's not clear.", "was_revised": true, "final_justification": "My concerns have been resolved.", "rebuttal": "We would like to thank Reviewer G3sQ for the recognition of innovations, writing clarity and excellent performance. We also sincerely appreciate your attention to the anchor-based design in our method.\nWe will comprehensively address your concerns from multiple perspectives: paradigm comparison, principle analysis spanning distribution, training stability, and ensemble mechanism, as well as relevant quantitative results.\n\n1. **Inclusiveness of the paradigm**: `directly predicting the offset of the 3D root node` can be seen as a special case of our method. Anchor-based PandaPose initially sets global fixed anchor and the joint-wise local anchor, then combines the two types of anchors to regress the offset for obtaining 3D joint predictions. When the joint-wise anchor is discarded and the global fixed anchor is set to the root node with its number decrease to 1, PandaPose degenerates into the method you describe. We quantitatively evaluated this strategy, yielding an MPJPE of 42.7. Our final experimental results showed a 6.8% reduction in error compared to this baseline.\n\t\n\n| **Anchor strategy** | **MPJPE on Human3.6M** |\n| --- | --- |\n| Directly predicting the offset of the 3D root node | 42.7 |\n| Global anchor | 40.8 |\n| Ours (Global anchor + Local anchor) | 39.8 |\n\nIn the revised version, we will supplement this comparison to Table 4 in the main text to further improve the completeness of the experiment.\n\n2. **Analysis of the essential principle**:\n\t* The global fixed anchor and joint-wise anchor we propose efficiently deliver a robust initialization for 3D pose lifting that is closer to the target joint distribution compared to the root node. Subsequently, the final prediction can be obtained by simply predicting an offset with a smaller numerical range ( 91.3mm v.s. 456.2mm in world coordinate system, based on average offset statistics). This substantially lowers the difficulty of directly predicting from the 3D root node to the target 3D joint.\n\t\t\n\t* On the other hand, a smaller predicted offset also makes network training more stable and the convergence speed faster. Specifically, under the exact same dataset (Human3.6M) and computing environment, aligning the training settings, when the training MPJPE loss converges to 60mm, our method requires about 280 iterations, the root node strategy requires about 510 iterations, and CA-PF requires about 1350 iterations.\t\n\t\n\t* Additionally, our anchor mechanism essentially serves as an ensemble prediction for the joint, generating the coordinates of the target joint by integrating the offset/weight of multiple anchors.  This characteristic of \"ensemble\" makes the overall prediction more fault-tolerant to local noise, which has also been widely proven effective for performance improvement [1] and robustness against outliers [2].\n\t\t\n| Method | Ours | Root node strategy | CA-PF |\n| --- | --- | --- | --- |\n| Iterations when training MPJPE converges to 60mm | 280 | 510 | 1350 |\n\nWe will also supplement the paper with the detailed discussion above to more comprehensively explain the advantages of our motivation and anchor design.\n\n3. **Justification of** $Q\\_{anchor}$**.** The initial $Q\\_{anchor}$ is derived from the 3D anchor set. We map it to a high-dimensional space through a linear layer, thus encoding it as a **learnable anchor query,**  which is used to interact with RGB & depth features in the decoder. This is also mentioned in Line 221-224 of the main text.\n\t\n\n> \\[1\\] ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers\n>\n> \\[2\\] Uncertainty-based offline reinforcement learning with diversified q-ensemble"}, {"review_id": "t6NGnpVBnA", "reviewer": "Reviewer_HbVK", "summary": "This paper introduces PandaPose, a novel approach for 3D human pose estimation from a single RGB image. The key innovation lies in propagating 2D pose priors into a 3D anchor space, which consists of joint-wise 3D anchors, depth-aware feature lifting, and an anchor-feature interaction decoder. The method addresses two major challenges in existing approaches: error propagation from noisy 2D inputs and depth ambiguity in self-occluded cases. Experiments on Human3.6M, MPI-INF-3DHP, and 3DPW demonstrate state-of-the-art performance, with significant improvements in challenging scenarios.", "questions": "1. How was the number of 3D anchors determined? Do the learned anchors exhibit specific semantic properties, such as spatial discrimination or differentiation between certain types of 3D points?\n\n2. Possible typo: Should \"Figure 13\" be \"Table 6\"? While the 2D feature sampling strategy reduces GPU memory and computational costs, it does not lead to significant performance improvement. This raises a question: If the 2D pose estimation contains large errors, could the sampled features deviate from the desired content? If so, why does the performance not degrade noticeably?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**  \n1. The paper is clearly written, with a logical structure and well-organized presentation.\n2. Well-Motivated Design. The use of joint-wise 3D anchors and depth-aware lifting addresses known weaknesses in previous pose lifting works that rely on fixed mappings or 2D-plane representations.\n3. Strong Empirical Results. The method achieves SOTA performance across multiple benchmarks, with notable improvements. The ablation studies convincingly validate the contributions of each component.\n\n**Major Weaknesses:**  \n1. Depth Discretization Strategy. Although using classification for depth estimation is pragmatic, it might introduce quantization errors. Could a regression-based approach be adopted for this task? If feasible, it would be valuable to include a performance comparison between the two approaches.\n2. Computational Cost. The paper introduces additional intermediate representations and extra computational steps for image-keypoint depth feature interactions. What is the inference speed of the proposed algorithm? Does it meet real-time requirements?", "was_revised": true, "final_justification": "Thank you for the author's response. I will keep my score.", "rebuttal": "We appreciate the recognition by Reviewer HbVK of our writing, well-motivated design, and empirical results. We will address all concerns regarding weaknesses and questions in the following.\n\n## **Depth Discretization Strategy**\n\nThanks for the insightful suggestion. We simply replace the classification depth head with a regression head for comparison as shown in table below. The classification approach achieves better accuracy, with the optimal bin number being 64. The improved performance can be attributed to the better alignment between the task complexity and the lightweight classification head, which helps alleviate fitting difficulty — a benefit particularly evident in challenging cases. We also observe that using too many bins can degrade performance, which aligns with findings in previous lightweight depth estimation methods \\[1,2,3\\]. We will include these results and discussions in the revised version to further clarify the advantages of our method.\nWe kindly note that our insight regarding the depth head does not lie in whether it is for classification or regression, but rather in the joint-wise approach. Ordinary predicting single depth map in principle cannot reflect joint depth in cases of self-occlusion (as shown in Figure 5 of the main text). The proposed joint-wise module addresses this issue by independently predicting the depth distribution for each joint, improving robustness as shown in Table 5 row2 - row3 of main text. Especially under challenging cases with significant occlusion, the predicted MPJPE decreased by 2.8mm (75.9 - 73.1).\n\n|  | MPJPE on Human3.6M| Challenge case |\n| --- | --- | --- |\n| Regression | 41.6 | 77.2 |\n| Classification with 16 bins | 40.7 | 74.8 |\n| Classification with 64 bins | 39.8 | 73.1 |\n| Classification with 128 bins | 40.2 | 74.7 |\n\n\n\n> \\[1\\] Deep ordinal regression network for monocular depth estimation, CVPR2018\n>\n> \\[2\\] Adabins: Depth estimation using adaptive bins, CVPR2021\n>\n> \\[3\\] idisc: Internal discretization for monocular depth estimation， CVPR2023\n\n\n## **Computational cost**\n\nIn Table 7 of the appendix, we compare the parameters and FPS with the SOTA image-based method CA-PF, as shown in the following table (assuming real-time application requires the single-frame input). Our method demonstrates significantly superior accuracy, especially in challenging cases, while FPS and inference consumption remain at a similar level to CA-PF. Following their claim of efficiency, PandaPose also meets the real-time requirement.\n\n| method | FPS | Param | Inference GPU memory | MPJPE on Human3.6M (full) | MPJPE on Human3.6M (challenging)|\n| --- | --- | --- | --- | --- | --- |\n| CA-PF (T=1) | 19 | 14.1 | 310 | 41.4 | 82.4 |\n| PandaPose w/3 decoder (T=1) | 16 | 15.2 | 323 | 39.8 | 73.1 |\n| PandaPose w/2 decoder (T=1) | 18 | 12.7 | 307 | 40.3 | 75.6 |\n\n\n## **Number setting and semantic characteristics of 3D anchor**\n\n\nRegarding the **the number setting of 3D anchors**, we determine it based on the trade-off between performance and computational overhead. We conduct ablation experiments under different anchor numbers (see the table below) and find that increasing the anchor number from 149 to 596 continuously improve performance; however, when it is further increased beyond 596, the performance improvement become limited, while the computational overhead increase significantly. Therefore, we finally set the anchor number to 596 as the balance point between performance and efficiency.\n\n| Anchor numbers | MPJPE on Human3.6M | GFlops |\n| --- | --- | --- |\n| 149 | 40.5 | 14.2 |\n| 596 | 39.8 | 16.6 |\n| 1022 | 39.7 | 18.9 |\n\nRegarding the **semantic characteristics of anchor**, although we are unable to present the visualization results of anchor in the paper due to the rebuttal policy, we have verified through quantitative and qualitative analysis whether its spatial distribution has semantic significance.\nSpecifically, we qualitatively and quantitatively observe the distribution and contribution of different types of anchors (adaptive local anchor / fixed global anchor) to the final joint prediction (as shown in Figure 3 of the main text). By selecting top-50 anchor-to-joint weights, we identify informative anchors contributing most to joint prediction. (Line 139-141) The distribution of the final informative anchors shows obvious clustering for the target joint (mean offset 73.4mm), with offsets to other joints all significantly larger than this value (mean offset 161.9 mm), demonstrating the specificity of the distribution.\nWe also analyze the changes in the offset of each anchor to the corresponding joint during the training process. This offset shows a clear downward trend during training as shown in the table below, indicating that the model gradually learns the accurate correspondence between anchors and 3D joints. This further demonstrates that anchors are not randomly distributed but gradually converge to positions consistent with the semantics of the human body structure.\n\n| Training step | Mean 3D anchor-to-joint offset |\n| --- | --- |\n| 10 | 259.12 mm |\n| 100 | 101.9 mm |\n| 1000 | 72.8mm |\n\nIn summary, our analysis shows that the learned 3D anchors are not only effective in terms of performance but also possess certain semantic significance. We will further supplement these analyses in the revised manuscript to more clearly demonstrate the design motivation and characteristics of the anchors.\n\n\n## **2D feature sampling stratery**\n\nThanks for the insightful comment. We have considered this issue and implemented targeted optimizations in the design of **feature sampling strategy** and **anchor mechanism**:\n\n1. For **feature sampling strategy**, based on the four-level multi-scale feature maps after downsampling, we perform feature sampling only on the lower two levels while preserving global semantic feature from two lower resolution levels (Line 205-207). Since 2D pose coordinates are inherently less sensitive to prediction errors after downsampling, we also enhance robustness to localization inaccuracies by sampling features within an r×r neighborhood around each 2D joint (Line 202-203).\n\t\n\t  To further demonstrate the optimizations above, we conducte an investigation into the similarity of visual features and the impact on final performance under noisy 2D pose inputs as table below. The resolution of the image is 256\\*192. When the noise scale is controlled within a certain range (0-5 pixels), the sampled image feature similarity can maintain relatively high stability (100% - 95.21%). When noise overflows to a large extent (10 pixels), the image features will deviate to a certain degree (85.33%) but still at a relatively high similarity.\n\t\n2. Apart from the optimization of the feature sampling strategy, we additionally adopt the **anchor mechanism** based on local and global anchors to ensure that our model can provide reliable 3D predictions even when there are significant errors in 2D pose estimation.\n\t*  For Local Anchors, we sample a set of 3D local anchors for each 2D joint using learnable sampling offsets. As shown in Figure 10 of the main text, even when 2D pose estimations exhibit substantial deviations due to motion blur or severe self-occlusion, these local anchors still effectively capture the positions of 3D joints.\n\t\t\n\t* For Global Anchors, we established a series of fixed global anchors that cover the potential human pose space (Line 149-152). This ensures that, despite significant shifts in local feature details caused by pose estimation errors, the global anchors can serve as stable reference points to provide global context, thereby maintaining the quality of the final 3D pose estimation.\n\t\t\n\t* Additionally, our method does not solely rely on a single offset prediction, but instead adopts an ensemble approach for offset prediction. This means that for each anchor, we generate multiple offset predictions and determine the final position through weighted averaging (Eq.7). This method not only improves the robustness of the prediction but also effectively reduces the overall performance degradation caused by inaccurate individual predictions.\n\t\t\n\nTo demonstrate the effectiveness and superiority of the above optimizations. We compare with other SOTA methods to test the robustness to noisy inputs (see table below and Figure 9 in main text). When the noise scale is 10 pixels, our MPJPE will decrease by 63.8%, but the magnitude is still not as severe as that of CA-PF by 151.9% and MixSTE by 284.8%.\n\n| Noise scale (pixel-level）| Sampled feature cosine similarity | MPJPE（Ours） | MPJPE (CA-PF) | MPJPE (MixSTE)|\n| --- | --- | --- | --- | --- |\n| 0 | 100% | 39.8 | 41.4 | 40.9 |\n| 2 | 98.61% | 41.4 | 43.5 | 48.3 |\n| 5 | 95.21% | 48.2 | 64.9 | 86.9 |\n| 7 | 91.37% | 51.2 | 78.3 | 115.6 |\n| 10 | 85.33% | 64.9 | 104.3 | 157.4 |\n\nFinally, we would like to thank you for the typo in Fig13, and we will update this to Table 6 in the revised version."}, {"review_id": "iKBG4fT0uD", "reviewer": "Reviewer_Cg4S", "summary": "The authors propose a method to lift 2D pose estimations to 3D. This method attempts to mitigate the problem of self-occlusion and depth ambiguity by careful model design. The experimental verification demonstrates the effectiveness of the proposed methods.", "questions": "Please address the questions in Weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The authors design a new method for 2D-to-3D pose lifting. They propose several interesting components:\n- an extended set of joint anchors\n- separate joint depth distribution modeling \n- a feature fusion strategy that leverage 2D joint and depth estimation.\n\nThrough the experiments on public dataset, they demonstrate the effectiveness of this method with the lowest reconstruction errors. The ablation study is thorough and show the necessity of each design. The results on 3DPW dataset also demonstrate the robustness and the generalization ability of the proposed method. \n\nGenerally, the paper is detailed and well structured.\n\nFor the Weaknesses, there are several points needed to be addressed:\n- In Eq. 1, what is `Linear`? Is it a neural network? If so, is it optimized in the training of the whole pipeline?\n- For the plane of root joint in Line 151, is it fixed to `z=0`? \n- In the training, are the GT 2D points or the estimated 2D points used?\n- In Eq. 6, what are these sampled points? How are they generated? And how to estimate the offset?\n- What is the noise level in Table 9? The unit of noise scale is not clear.\n- Providing some qualitative results on 3DPW would benefit the evaluation, as it has GT and is in less constrained outdoor setting.\n- For other methods shown in Table 3, are they trained on 3DPW?\n- Since the experiments are conducted with the 2D pose estimator [33], I'm wondering whether the results will change a lot if using different pose estimators?", "was_revised": true, "final_justification": "The authors' rebuttal addresses my concerns, and considering the contribution and strength of this paper, I keep my rating.", "rebuttal": "We sincerely appreciate your positive feedback regarding the novelty and effectiveness of our method and your recognition of the detailed and well-structured writing. In the following, we address your questions point by point.\n\n> - In Eq. 1, what is `Linear`? Is it a neural network? If so, is it optimized in the training of the whole pipeline?\n> \t\n\n`Linear` means a linear projection layer responsible for estimating 2D joint to 3D anchor offsets $\\\\delta_J$from the input 2D pose $P\\_J^{2D}$, and is optimized end-to-end during the training process.\n\n> - For the plane of root joint in Line 151, is it fixed to `z=0`?\n> \t\n\nYes, follow the common practice of 3D human pose estimation methods (CA-PF\\[NeruIPS'23\\], HiPART\\[CVPR'25\\], KTPFormer\\[CVPR'24\\] etc.), we set the root joint depth $z$ to 0.\nWe will add the explanation regarding this in Line 151.\n\n\n> - In the training, are the GT 2D points or the estimated 2D points used?\n> \t\n\nWe strictly follow the same input as the previous methods (CA-PF\\[NeruIPS'23\\], HiPART\\[CVPR'25\\], KTPFormer\\[CVPR'24\\] etc.) for fair comparison, i.e., using the same estimated 2D pose for training and testing (Line 269-272).\n\n\n> - In Eq. 6, what are these sampled points? How are they generated? And how to estimate the offset?\n> \t\n\n$$DCA(a) = \\\\sum\\_{n \\\\in N} W\\_n \\\\phi(F\\_{3D}, P\\_a + \\\\Delta S\\_n)$$\nFor an input anchor query $Q\\_{anchor}$ and corresponding anchor position $P\\_a$, we treat $P\\_a$ as reference point to execute deformable attention\\[1\\]. Aligned with \\[1\\], we equip each anchor with $N$ sampling points nearby to dynamically interact with $F\\_{3D}$. These sampled points are generated by adding *sampling offsets* $\\\\Delta S\\_n$to the reference points $P\\_a$ (i.e. 3D anchor positions). The sampling offsets $\\\\Delta S\\_n$ are predicted from the input queries (also referred as anchor queries $Q\\_{anchor}$ in our paper) through a linear projection layer. This design enables the Transformer to dynamically focus on sparse but semantically meaningful local regions in an adaptive manner, improving both efficiency and accuracy.\nDue to space limitations, we will add detailed explanations for Eq.6 in the revised version, and add a flowchart for the signal to enhance clarity.\n\n\n> \\[1\\] Deformable DETR: Deformable Transformers for End-to-End Object Detection\n\n\n> - What is the noise level in Table 9? The unit of noise scale is not clear.\n> \t\n\nThe noise level in Table 9 is measured in pixel units, based on images of resolution 256×192.\n\n\n> - Providing some qualitative results on 3DPW would benefit the evaluation, as it has GT and is in less constrained outdoor setting.\n> \t\n\nWe agree that qualitative results on 3DPW would be valuable for demonstrating the effectiveness and generalization of our method, especially in unconstrained outdoor settings. Unfortunately, due to the constraints of the rebuttal policy, we are unable to include additional visualizations at this stage. We will include more qualitative results on 3DPW in the revised version to further support the evaluation.\nIn the meantime, to demonstrate the generalization ability of outdoor scenes, we present the comparison results of MPI-INF-3DHP with SOTA and GT (Figure 11) as well as the qualitative results of in-the-wild internet images (Figure 12).\n\n> - For other methods shown in Table 3, are they trained on 3DPW?\n> \t\n\nThe methods listed in Table 3 are all 3D human pose estimation approaches that follow the cross-dataset setting (training on Human3.6M and testing on 3DPW). The performance metrics of these methods are taken from their latest publicly available reports. We compare against them under the same evaluation protocol to ensure a fair comparison. Compared to the latest SOTA method HiPART (CVPR 2025), we achieve significant improvements in performance (74.9 vs 77.2, 2.3 $\\\\downarrow$), demonstrating the strong generalization capability of our approach.\n\n> - Since the experiments are conducted with the 2D pose estimator \\[33\\], I'm wondering whether the results will change a lot if using different pose estimators？\n> \t\n\nWe select three widely used 2D pose estimators (i.e., CPN, HRNet-w32, HRNet-w48) for ablation experiments to examine how predicted 3D pose quality varies with input 2D accuracy. The results are listed below\n\n| **2D pose estimator** | **mAP on COCO $\\uparrow$** | **MPJPE on Human 3.6M $\\uparrow$** |\n| --- | --- | --- |\n| CPN | 68.6 | 40.1 |\n| HR-Net-w32 | 74.4 | 39.8 |\n| HR-Net-w48 | 76.3 | 39.5 |\n\nThe accuracy of PandaPose remains relatively stable with respect to the errors of the 2D pose estimator. Compared with CPN and HR-Net-w48, the 2D accuracy (mAP) decrease by 5.5%, but the error of PandaPose only increase by **1.5%.** \nMeanwhile, in Figure 9 of the main text, we gradually add Gaussian noise to the 2D pose to study the method tolerance to noise levels and our method maintains the best anti-noise ability. When the pixel-level noise scale ranges from 0 to 5, the error of PandaPose only increases by **16%** (from 39.8 to 48.2), while the image-based CA-PF increases by **56%** (from 41.4 to 64.9), and the sequence-based SOTA KTPFormer increases by **112%** (from 40.1 to 86.9).\nThe noise resistance can be attribute to different 3d anchor setting and anchor-to-joint ensemble prediction mechanism."}]

Author rebuttal:

Meta-review: Four of the five reviewers recommend acceptance. The fifth point out that the paper lacks clarity. I urge the authors to take his comments to heart and to improve the writing and organization when preparing the final version.
Decision: Accept (poster)