Towards Generalizable 3D Human Pose Estimation via Ensembles on Flat Loss Landscapes
Authors: Jumin Han
Jun-Hee Kim
Seong-Whan Lee
Venue: NeurIPS 2025 poster ()
Keywords: Lifting-based 3D Human Pose Estimation
Primary area: applications
TL;DR: We propose an adaptive scaling mechanism and an ensemble approach that combines flat-region solutions to enhance 3D HPE generalization.

Abstract: 3D Human Pose Estimation (HPE) is a fundamental task in the computer vision. Generalization in 3D HPE task is crucial due to the need for robustness across diverse environments and datasets. Existing methods often focus on learning relationships between joints to enhance the generalization capability, but the role of the loss landscape, which is closely tied to generalization, remains underexplored. In this paper, we empirically visualize the loss landscape of the 3D HPE task, revealing its complexity and the challenges it poses for optimization. To address this, we first introduce a simple adaptive scaling mechanism that smooths the loss landscape. We further observe that different solutions on this smoothed loss landscape exhibit varying generalization behaviors. Based on this insight, we propose an efficient ensemble approach that combines diverse solutions on the smooth loss landscape induced by our adaptive scaling mechanism. Extensive experimental results demonstrate that our approach improves the generalization capability of 3D HPE models, and can be easily applied, regardless of model architecture, with consistent performance gains.

Towards Generalizable 3D Human Pose Estimation via Ensembles on Flat Loss Landscapes
Jumin Han Department of Artificial Intelligence Korea University. Seoul, South Korea juminhan@korea.ac.
Jun-Hee Kim
Department of Artificial Intelligence
Korea University, Scoul, South Korea
jh kimkkorea.aa ac. kr
Seong-Wine Whan Lee Department of Artificial Intelligence Korea University, Seoul, South Korea sw www.eeeeeeea.... kc
Abstract
3D Human Pose Estimation (HPE) a fundamental task the computer vision. Generalization in 3D HPE task is crucial due the need for robustness across diverse environments and datasets. Existing methods often focus on learning relationships between joints enhance the generalization capability, but the role of the loss landscape, which is closely tied to generalization, remains underexplored. In this paper, we empirically visualize the loss landscape of the 3D HPE task, revealing its complexity and the challenges it poses for optimization. To address this, we first introduce simple adaptive scaling mechanism that smooths the loss landscape. We further observe that different solutions on this smoothed loss landscape exhibit varying generalization behaviors. Based on this insight, we propose an efficient ensemble approach that combines diverse solutions on the smooth loss landscape induced by our adaptive scaling mechanism. Extensive experimental results demonstrate that our approach improves the generalization capability 3D HPE models, and can be easily applied, regardless of model architecture, with consistent performance gains.
Introduction
3D Human Pose Estimation (HPE) is one of the essential computer vision tasks. In recent years, has received increased attention due its successful application in the areas of autonomous driving [33] 33), robotics 12, and industrial safety [31] along with the advancement of Deep Neural Networks (DNNs). Especially, the domain of industrial safety, it is crucial to ensure the generalization capability of the model as its accuracy can be directly tied to safety concerns.
Unfortunately. it is challenge get robust 3D HPE model due to the inherent complexity and variability human poses across different datasets and domains. Previous researches have addressed these challenges by focusing multi-hypothesis concept [8] 4
22 19
learning better posture
information [16 T0 29]. designing elastic model architectures 36 27 and data augmentation 28 Although previous methods have achieved notable performance improvements, they often overlook the structure of the loss landscape, which plays critical role in understanding model generalization and stability [].
In our analysis, we found that the loss landscape of 3D HPE task can be highly complicated and has multiple disconnected local minima. Such disconnected modes pose significant challenges for
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

3
Cc
Roce
2014
0
0
+000
3001
4800
3001
122
(a)
(b)
(c)
(d)
Figure 1: The local and global loss landscape when K is 3. (a) Local loss landscape of C1- (b)
Local loss landscape of C2. (c) Local loss landscape of C3- (d) Global loss landscape of D. The local loss landscapes have various shape depending the degree of depth ambiguity. Especially, the global loss landscape and some local loss landscapes have multiple local minima. oc2 lc2, and Oc, model parameter around local minimum of each local loss landscape. Note that the local loss landscape of C₁ is a result when the model trained with only Cj.
optimization and may adversely affect the generalization performance of the model (i.e. converging to the sharp minimum). Despite its critical impact on generalization, there has been lack of attempt to design training or inference strategies that explicitly leverage this insight in the 3D HPE field. We argue that this structural insight is critical to enhance the generalization ability of the model because simply optimizing a single model over such loss landscape may result in a biased convergence toward sharp minimum, which is not desirable.
To mitigate this, we propose a simple adaptive scaling mechanism to smooth the loss landscape, effectively reducing the likelihood of existence of high-loss barriers between local minima and forming loss landscape geometry where diverse solutions reside on low-loss region. Furthermore, we found that the diverse solutions on the smoothed loss landscape by our adaptive scaling mechanism exhibits various generalization behavior. Based on this observation, we also introduce an efficient ensemble strategies with the adaptive scaling mechanism that combine multiple solutions on the smooth loss landscape without destructive interference to improve robustness and generalization ability of the model. It is worth noting that our method requires a similar amount of time as training single model and can be applied regardless of the model architecture.
Our contributions are as follows:
We propose a novel and simple ensemble method that can integrated with simple mod- ifications, regardless of the model structure. The proposed method is designed to induce flatness of loss landscape while concurrently enable parameter-efficient ensemble method, and can be implemented casily.
We shed light on the difficult learning process of 3D HPE task through a analysis from a loss landscape perspective. To the best of our knowledge, we are the first take this direction of analysis in a 3D HPE task.
Our method enhances performances of the model for the representative model architectures (MLP, CNN, GCN, and Transformer) of 3D HPE in benchmark datasets such Human3, 6M 14,, MPI-INF-DDHP 24]. 3DPW 30 and BEDLAM (2).
2 The Loss Landscape of 3D Human Pose Estimation
In this section, we explore the loss landscape of 3D HPE task. During our experiment, we leverage the dataset, Human3.61 3.6 [14), which is popularly utilized benchmark dataset in the 3D HPE task. For the network, we utilized the BaselineNet from foundation work (33]. For the results other network architectures, the Appendix. We start our analysis from elaborating the experimental setting and ending with summary our analysis and key insight.
2.1 Experimental Setting
Considering the loss value for whole dataset D at parameter 0 is linear combination of loss values for individual examples in D at 0, analyzing loss landscapes for subsets of an effective approach to understand the loss landscape. To efficiently explore the high-dimensional loss landscape of 3D

---

Train Lose (log seale)
Clos C Cles
Gwe
Text MAVE
Cless
Train Loun
Train MPUPE
Trein Lods
3draw 3902
Epoch
DAR (c)
Epoch
(a)
(b)
(d)
Figure 2: The different loss landscape characteristics depending on each class. (a) Class wise training loss per epoch when K is 5. (b) Top-1 eigenvalue of each local loss landscape when K is 21. (c) Class-wise original train MPJPE (blue line) and the gap between MPJPE after small input perturbation and original MPJPE (red line). Note that the (b) and (c) are results when the model is trained with the whole dataset D when K is 21. (d) The training trajectory of the model when the model is trained with D) C2o when K is 20 (blue line is train loss and green line is test accuracy).
trajectory
HPE, we first divided the dataset according to the degree of ambiguity of each example, which can be different depending on inherent data patterns, rather than random partitioning. The inherent depth ambiguity of 3D HPE can be understood as a one-to-many mapping between the input and output space, which means multiple valid loss values can exist for a single input. Especially, high degree of depth ambiguity implies a high variance of the valid loss values for a single input. Since this property directly affects the geometry of the loss landscape, partitioning based on depth ambiguity is an appropriate choice. To quantify of the depth ambiguity per example, we borrow a definition, Depth Ambiguity Ratio (DAR), from a recent work [T6.. With this DAR, we can add the quantified depth ambiguity to each data pair. The train dataset can be re-expressed as D {(z;, Vr. u₂ ) The æ, y are the coordinates of 2D pose and 3D pose, respectively. The u is the DAR for a data pair (z, v) and N is the size of dataset. To partition the dataset based on the values of u, we sorted {ur) 1 in ascending order and defined the bin edges { by selecting values from the sorted values such that bo < bi < ..... < bx- Each example (24, Di, 2s2 ) is assigned to the k-th bin if it satisfies bg-1 S u4 < by for = 2. K 1, and bx- S u; S bb for k K. See Appendix for the detailed binning procedure. From now on, we call the subset assigned to the k-tth bin as class Cn. So, the degree of the depth ambiguity for each class can be ordered as C₁ < C2 < ... < Ck. For convenience, we call the loss (landscape) of each class as local loss (landscape) and the loss (landscape) of whole dataset as global loss (landscape) from now on.
2.2 Local Loss Landscape of Each Class
Considering that the global loss is linear combination of the local losses, analyzing each local loss landscape significantly aids in understanding the global loss landscape. To this end, we begin by analyzing the individual local loss landscapes. At first, we present simple results that the training loss roughly varies depending on each class in Figure aaa). There can be various reasons for this, the main factor of this would be the different local loss landscape for each class. In this reason, we explore the top-1 eigenvalue Amaz of the Hessian matrix for each class to see the local loss landscape curvature around the trained model parameter with D in Figure b As expected, the curvature of local loss landscapes are different depending on each class. While the Amaz of local loss landscapes for the both extremely low and middle degrees of depth ambiguities are similar, their geometries are different to each other. To show this, we inspect the loss value after input perturbation for each class in Figure c The data for extremely low degree of depth ambiguity exhibited a tendency to be memorized by the model because the model cannot appropriately address small input perturbation. Considering the extremely low degree of depth ambiguity as one-to-one mapping relationship between input space and output space, this is natural. On the other hand, the data for high degree of depth ambiguity exhibited steep loss landscape, which implies that they are not being learned enough and the model is located on the unstable loss landscape. These all results indicate that the local loss landscape for each class shaped differently, suggesting that the global loss landscape can be highly complicated depending on the individual local loss landscape. To show this empirically, we train the network with C2 when K is 20. We can observe that the training trajectory highly complicated when we see the training loss and test accuracy through epochs in Figure dd). This
'e estimate the 1 eigenvalue Amar of each local loss landscape following 34
"Similar to 20). we perturb the input 2D pose in scale-invariant manner with Gaussian noise. The perturbed [d] ||||| - where the d ~ N 1).
input with perturbation scale expressed Zpert + d
3

---

indicates that a small change of arbitrary local loss landscape may lead to highly complicated global loss landscape.
2.3 Visualizing Loss Landscapes
In this section, we visualized the local and global loss landscape of 3D HPE task to inspect the local loss landscape in detai As expected, the local minimum of each local Ioss landscape are located in the different parameter space as shown in Figure ! As we mentioned above, the global loss is linear combination of local losses, thus leading to the highly complicated global loss landscape that potentially has disconnected multiple local minima, which cannot be distinguished from the perspective of model. This is because the model cannot distinguish which one is better because the gradient at those local minima are all same as zero. In fact, even when the K is 3, the global loss landscape has two local minima shown in Figure[]d) which implies that the model can converge to different local minimum depending on their initial parameter or some arbitrary dominant local loss landscape, or even cannot be converged without delicated training strategy. For the results of various K values, see Appendix. These disconnected local minima can exhibit different generalization behavior depending on their own sharpness and the loss value at the minimum (5 1 32
2.4 Summary and Key Insight
As shown in Section 2.3 the local minimum of each local loss landscape can be differently located in the parameter space and they are all potential local minima in the global loss landscape. If the local minima of global loss landscape are disconnected at their local minimum, the model tends to be overfitted to an arbitrary dominant class C₃ so that generalization capacity can be deteriorated or exhibits highly unstable training process. Hence, important to reduce the likelihood that disconnected local minima may exist in the global loss landscape. This consideration leads us to our key insight to mitigate this as follows:
each other and their loss level are different
Flatness of local minimum of each local loss landscape.
If the local minimum of each local loss landscape is sufficiently flat, there will be no significant high loss barriers between them, which reduces the likelihood of multiple disconnected local minima the global loss landscape. We empirically validated this intuition in Section 4 This property facilitates stable training of 3D HPE model and may improve the generalization capability of the model. Based on this, we introduce a simple method motivated by this insight in the following section.
3 Proposed Method
Motivated by our key insight in Section 2 we aim to mitigate the problem of highly complicated loss landscape of 3D HPE task. To this end, we first propose a simple yet effective adaptive scaling mechanism that smooths the loss landscape of 3D HPE model. We elaborate our detailed method in following sections.
3.1 Adaptive Scale Adjustment of Prediction
In general, the behavior and output of 3D HPE network can be expressed simply by the following equation, where the network predicts the 3D pose ÿ.
i = =f(oo())),
where is input 2D pose, 96 is the network before last layer, and fo is last layer. Our core idea is to encourage flatness in the local minimum of the local loss landscape. Since the meaningful maximum value of K is the dataset size, in the case where the number of class K [D] this is equivalent to promoting flatness around the local minimum local loss landscape for every individual training example. One simple method for this is apply adaptive scaling mechanism to Eq. T follows.
fo(go()) (x))
( (os (x))) +1
*e visualized the loss landscape following (6). For batch normalization 13 pass for one epoch to calculate the running mean and standard deviation of activations of each layer with interpolated network weights.
we run one additional forward

---

where hy : I + R that predicts scalar value and a is ReLU function. The d is output dimension of Iz. This simple mechanism induces the flatness of entire loss landscape.
For example, the i in Eq. 1 can be expressed aÿ a where is scalar value. This is
o(he(go + 1. Then for a overall similar form of Eq. 2 where aÿ = fo(g)(x)) and a =
network parameter set of Eq. 2 S {8, o, }. this means that there exists multiple solutions such that {5 o (ho(gg(()) ((h))(9g(()))++1
(/) that predict the same ÿ in Eq. 2 depending on the value of 1. This redundancy creates a smooth region in the loss landscape, which is known to correlate with better generalization. The empirical validation of this is provided in Section 4 Also, this simple mechanism enhances expressive power of the model because the scale parameter a (hy (x)( +1 is input-dependent, thus the model capable of representing more diverse functions that could not be expressed before.
(x))
The adaptive scaling mechanism allows for various parameter solutions by intentionally introducing re- dundancy into the loss landscape. This naturally raises important question: how do these distinct solutions differ in their generalization behavior? To explore this, conducted simple yet illustrative experiment to examine the generalization behaviors of models trained under the adaptive scaling scheme with different random seeds. Interestingly, we ob- served that the resulting models from different ran- dom seeds exhibited diverse generalization behaviors, as shown Figure 3] We calculate the deviation of model predictions for joints that are difficult to es- timate. For example, the deviation of j-th joint for i-th model can be calculated as p)- Hy where Pj is R >>" pa and p is a j-th joint of prediction from model trained with random seed i. This observation led our second key insight: ensembling these solutions further improve generalization capability.
Joint Deviation
Figure 3: Deviation of predictions from each
solution on the smooth loss landscape for diffi- cult joints of 3DHP (24). Note that the models are trained with H36M 14].
effectively leverage this diversity to
3.2 Finding Multiple Solutions on Smooth Loss Landscape
The second key insight is corresponding to find multiple solutions on the same global loss landscape that are reside on the same low-loss level region. This can be done by leveraging multiple regression heads (fe and hi in Eq. 2
fol Axg
Our proposed framework for efficient ensembling is shown In Figure. The representation 90 (x) from the encoder is fed
foz
hpp
k(,,,)
P2
- g0
into each regression head to get fo. (z)(x)) and ha, (go(2) (z)) for i-th regression head. With these representations from i-th regression head, we can get ū: in Eq. for each head. Then, the loss function T( y) applied to each fi- While each function, parameterized by {Φ. 02, 1/r), serves same purpose (estimating target y), they are represented differently, thus leading to enhanced expressive power. With this formation, we can get diverse parameter solution set {S.}} where S;
fem hrp
RCTe Aga)
Fix
Figure 4: Overall proposed frame- work with m regression heads with adaptive scaling mechanism. The op- eration k( b) represents abb)+1
{ O,, } on the smooth loss landscape induced by our adaptive scaling mechanism. Consequently. the proposed framework facilitates an efficient ensemble strategy to boost generalization ability of the model. Because each solution S; lies on smooth and flat region of the loss landscape, which is known to be associated with improved stability and generalization ability of the model. So when the number of regression heads is M, the formulation of this framework and the loss function during training can be expressed as:
fo. (go(x))
1
M
T. = kKJo (z)))) hw (o)())))
Lis
a (ho,, (9o (z:))
M
5

---

where k(a, b) represents obi+1 Y/ as follows:
and L, L(v:. v) for 1 S S M. During inference, we averaged
Fina!
(ū) + U2 + + ūM)
(4)
M
Additionally, we intentionally initialized the parameters of regression heads differently to get diverse solution set. This is to prevent each head from converging to similar minimum in the global loss landscape. Also, it is worth noting that this ensemble strategy is valid when the adaptive scaling mechanism is combined. Since the loss landscape is highly complicated without adaptive scaling mechanism and potentially has multiple local minima with different loss level that cannot be distinguish from the perspective of the model, the differently initialized each regression head may converge to different local minima with different loss level, which could reduce the benefit of ensembling. We empirically validated this in Section 4.6
4 Experiments
We experiment our method on benchmark datasets, Hluman3.6M (H36M) 14] MPI-INF-3DHP
(3DHP) (24), 3D Poses in the Wild (3DPW) [30), and BEDLAM [2]. H36M [14 is broadly utilized for training and evaluation in 3D HPE task. They are consisted of over 3.6 million 3D pose and 2D pose pairs. We utilize the data from subjects 1, 5, 7, and training set, while the data from subjects 9 and 11 are utilized as test set following the literature of 3D HPE. 3DHP [24] presents rather complex situations compared to H36M [14, including diverse indoor and outdoor scenarios. 3DPW [30] is widely-used benchmark for 3D human pose estimation in unconstrained outdoor environments, featuring over 51,000 frames of video with accurate ground-truth 3D pose and shape. [ is large-scale synthetic video dataset for 3D human pose and shape estimation, featuring realistic clothing and high diversity body shapes and motions. This dataset is specifically designed to train models that generalize real-world scenario. We report metrics, Mean Per Joint Position Error (MPJPE) and Procrustes Aligned MPJPE (PA-MPJPE), on H36M [14] following convention. For 3DPW (24), report Percentage of Correct Keypoints (PCK) within 150mm, Area Under the Curve (AUC), and MPJPE to evaluate the generalization capability of the model with our approach. For network architecture, utilize representative architecture (i.e. MLP, CNN, GCN, and Transformer) of 3D HPE task to evaluate our proposed method.
BEDLAM
4.1 Comparison on H36M
In this section, we apply our proposed method to a variety of baseline models 23 order comprehensively evaluate the effectiveness and generalization capbility of our approach. Specifically, we integrate our method into multiple representative 3D HPE architectures and compare their performances before and after the application of our method in Table To gain deeper insights into the individual contributions of each component within our framework, we conduct ablation studies under the following settings: (1) standard training without any modifications, (2) training with the adaptive scaling mechanism only, and (3) training with both the ensembling method and the adaptive scaling mechanism. The experimental results show that our proposed method consistently leads to performance improvements across a wide range of network architectures, highlighting its robustness and versatility. This consistent gain suggests that the integration of the ensembling strategy with the adaptive scaling mechanism generates a synergistic effect, effectively enhancing the generalization ability of the model. Moreover, these results indicate that our approach contributes to reshaping the loss landscape in a way that facilitates better optimization and improved final
27 38 37 in
Table 1: Before and after applying our method to various models. We report MPJPE and PA PAMME
in single frame setting for evaluation on H36M. Similar other 3D HPE works 27 36 21 we use cascaded pyramid network 3 as 2D pose detector.
MPJPE +Saale
PA-MPJPE
Architecture
Method
Basic
+Both
Basic
+Scale
+Both
MLP CNN
BaselineNet 23
55.4
54,2
53.6
44.2
43,1
42.7
VPose 27
55.7
54.9
54.4
44.0
43.4
43.3
GCN Transformer
SemGCN
37
64.8
61.6
59.9
51.2
48.9
48.3
PoseFormer
18
56.2
55.5
54.5
44.3
44.4
43.9

---

Standard Training
SAM Training Adaptive
Line
High koo
Train Loss
Test MPJPE
Amax
Une
Line
02
DAR
Pc.
6cc
(a)
(b)
Figure 5: (a) Top-1 eigenvalue of each local loss landscape when K is 21 after applying SAM [5 and adaptive scaling mechanism. (b) Global loss landscape of after applying adaptive scaling mechanism alone. (c) Training loss and test accuracy along the line between local minimum (fc,, Ac,. and (c,) of local loss landscape. The dashed lines refer the test accuracy. The blue line for standard model and green line is for model with adaptive scaling mechanism. Note that these results are from BaselineNet 23
accuracy. Overall, the results validate the practical utility of our method as a simple yet meaningful enhancement existing 3D HPE frameworks.
4.2 Cross-Dataset Evaluation on 3DPW
To further assess how effectively our proposed method enhances the generalization ability of 3D HPE models, we conducted a cross-dataset evalu- ation. This type of evaluation is crucial for under- standing whether a model trained on one dataset can perform well on data from a different domain, which is often more reflective real-world deploy- ment scenarios. We trained each model with a combined ground-truth training set of the H36M, 3DHP, and BEDLAM and then, evaluated its per- formance on the 3DPW dataset. During testing, we used ground-truth 2D poses from 3DPW input and predicted the corresponding 3D poses using the trained models. We report three standard metrics
Table 2: Cross-dataset evaluation 3DPW.
Method
PCK
AUC
MPJPE
BasclineNet 23
85.8
47.6
87.2
+Ours
87.1
50.8
81.9
VPose 27
85.3
47.4
88.8
+Ours
86.5
47.9
86.0
SemGCN 37
78.2
41.4
104.3
+Ours
84.4
48.7
88.7
PoseFormer 18
84.3
47.2
90.9
+Ours
86.0
50.3
84.1
to comprehensively assess performance: PCK, AUC, and MPJPE. As presented in Table 2 our method consistently leads significant improvements in all three metrics across various baseline architectures. These results strongly suggest that our approach enhances the generalization ability of the model beyond the training data distribution, making more robust unseen domains and varying data characteristics. Such improvements in generalization capability are particularly valuable for practical applications where the test-time data may differ from the training distribution.
4.3 Flatness of Local minima
To investigate whether our adaptive scaling mechanism effectively smooths the loss landscape, we estimated the top-1 eigenvalue of the Hessian matrix when only the adaptive scaling mechanism was applied. As shown in S). the curvature of each local loss landscape became noticeably flatter compared that of baseline. Additionally, we visualized the global loss landscape under the adaptive scaling mechanism alone in case of BaselineNet (23) to compare the global loss landscape without our adaptive scaling mechanism Figure d). As illustrated exhibited smoother topology with a single local minimum. This provides a significant advantage in network convergence and, as demonstrated Section 4.2] the flatness of the loss landscape correlates with improved generalization performance. Furthermore, in (igureBjc). we investigate the training loss and test MPJPE along the line between each local minimum. Our adaptive scaling mechanism exhibits better training loss and test MPJPE through the all points on the line between each local minimum. Also there was no high loss barrier between each local minimum. This validates our intuition in Section?3 2.4
dd)
FigureB]bb the landscape

---

Joint Indox
Perturbation Level
Figure 6: (a) Diverse solution set on the train loss landscape under our adaptive scaling mechanism. Omy 0m and Omg are solutions when the number of heads M is 3. (b) Diverse solution set on the test loss landscape under our adaptive scaling mechanism. (c) Joint-wise deviation for predictions of PoseFormer [38]. (d) Robustness against input Gaussian noise. AMPJPE represents the gap between MPJPE for perturbed input and original MPJPE.
4.4 Example of Diverse Solutions Loss Landscape
DNNs typically consist of millions to even billions of parameters, making it challenging to analyze their loss landscapes directly. Nevertheless, several effective techniques have been proposed to visualize the loss landscape in a interpretable manner (20 6 Similar to the [6. we visualize the loss landscape of BaselineNet [23] with 3 heads in Figure 6 Because visualizing the loss landscape with all solutions becomes challenge when the number of solutions exceeds 3. The visualization results show that all solutions reside on low-loss and flat regions the training loss landscape. Similarly, observe that the corresponding test loss landscapes also place the solution set in flat regions with low-loss level. Figure c).. The predictions exhibited diversity across different heads. These results suggest that ensembling solutions can lead to a highly stable model, benefiting from low bias and reduced variance. Such stable models crucial in real- worldworld applications, where consistent and reliable predictions are essential under varying and potentially noisy conditions.
further investigate the joint-wise prediction diversity for PoseFormer [38] in
4.5 Comparison with Other Loss Landscape Flattening Method
There various approaches to promoting flatness the loss landscape. A representative example Sharpness- Aware Minimization (SAM (5]), which explicitly encourages flatness by applying pertur- bations and penalizing sharp regions in the loss landscape. However, such methods operate without considering the characteristics of individual local loss landscapes, instead applying a global flatten- ing effect. As result, they may disproportionately flatten regions around specific local minimum, po- tentially overlooking others. In fact, we investigate the top-1 eigenvalue of Hessian matrix when applying SAM in Figure 5 (a. As we expected, the
Table 3: Comparison with SAM on H36M.
Method
MPJPE
PA-MPJPE
BaselineNet 23
54.8
43.8
+SAM 5
56.1
44.4
+Scale only
54.2
43.1
VPose (27
55.7
44.0
+SAM
57.3
45.7
+Scale only
54.9
43.4
SAM tends to flatten the loss landscape in a biased manner, favoring certain local regions over others. We also experiment the performance of 3D HPE + SAM in Table The results show that applying SAM to a 3D HPE model yielded no performance gain, which confirmed our hypothesis. Note that the models are trained for longer duration because of the slow convergence of SAM and the perturbation radius set as 0.05 for SAM training.
4.6 Ensembling without Adaptive Scaling Mechanism
We further validate that the proposed ensembling method performs effectively when combined with the adaptive scaling mechanism in Table As we expected, the ensembling approach alone does not achieve satisfactory results in the absence of adaptive scaling, primarily due to the highly complex and irregular nature the global loss landscape. This complexity can cause each regression head to converge to different local minima, leading to severe gradient conflicts between heads lack coherence among the predictions and diminishing the overall benefit of ensembling. To further clarify this, evaluate the combination of another loss landscape flattening method, SAM,

---

with the ensemble strategy. The results indicate that 3D HPE SAM Ensemble achieves better performance than ensemble alone, which supports that the loss landscape of 3D HPE is highly com- plicated. All these findings underscore the crucial role of adaptive scaling mechanism in aligning the optimization trajectories of individual heads and en- hancing ensemble performance. The adaptive scal- ing mechanism harmonizes the learning dynamies across heads, facilitating more coordinated updates and yielding a more effective ensemble representa- tion.
Table 4: Results esembling without adaptive scaling mechanism H36M.
Method
MPJPE
PA-MPJPE
BaselineNet [23]
54.8
43.8
+SAM [5] and Ens.
54.5
43.2
+Ens. only
73.3
58.1
VPose (2
55.7
44.0
+SAM [5] and Ens.
55.3
44.0
+Ens. only
69.8
55.2
4.7 Robustness Ensembling Strategy
We further experiment the robustness of our model to the input noise. Lifting-based 3D HPE, a prevalent approach in the literature, typically relies on the output of 2D pose detector as input. As result, the input inevitably contains noise. Therefore, evaluating the robustness of the 3D HPE network input noise crucial. To simulate noisy inputs, we perturb the 2D pose inputs with Gaussian noise of varying magnitudes in input scale-invariant manner. In this reason, we experiment our method evaluate the robustness to input noise on the dataset 3DHP 24 for the BaselienNet [23] in Figure d).. Note that the model is trained with H36M [14]. The model with our method shows better generalization capability. Especially, our model is robust when the noise level is high, which suggests that our method can generalize better for unseen poses. Stability under high noise levels is crucial, as it enables fail-safe mechanisms, in which the robot system can still recognize a human shape in danger zone even when the predicted pose is inaccurate.
4.8 Converging Speed
As mentioned above, the adaptive scaling mechanism serves to smooth the loss landscape. Consequently, it is natural think that the optimization of models with adap- tive scaling mechanism becomes more casier than before. To illustrate this, we compare the training loss trajectories of the model with and without adaptive scaling mechanism during 20 epochs on H36M [4 in Figure 7 As we ex- pected, the model with adaptive scaling mechanism shows a faster convergence. The training loss with adaptive scal- ing mechanism shows lower training loss throughout the epochs. Especially, the test accuracy is significantly lower at the beginning of the training. With this property, while our adaptive scaling mechanism facilitates faster conver- gence, the model with adaptive scaling mechanism may need to be early-stopped to avoid undesirable overfitting.
220
Trai MriF
Toii Lope
Trinn Lese Ahag
Wiew fies VSVi
Epoch
Figure 7: Loss trajectories with adaptive scaling mechanism. ASM refers the adaptive scaling mechanism.
Related Work
5.1
3D Human Pose Estimation
wide range of methods have been proposed for 3D HPE. The prevalent research direction in this area is to lift 2D keypoints in image coordinate system to 3D pose coordinates in camera coordinate system
23 27
37
38 There are various representative network architectures starting from transformer-
based model [2 38). GCN-based model (37]. CNN-based model [27]. and MLP-based model
23 B3). These researches proposed elastic network design to learn better posture information to enhance the generalization capability of the 3D HPE model. On the other hand, augmentation-basse
approaches 77 16. 21 have been proposed to address the limited availability of training data in 3D HPE task. These methods primarily aim to improve model generalization by enhancing data diversity through various augmentation techniques. While these methods have demonstrated notable performance improvements, they overlooked the perspective of the loss landscape, which is closely

---

related to generalization. In this work, we aim to provide a novel perspective on 3D HPE by analyzing its loss landscape and propose a simple yet effective approach based on this analysis.
5.2 Loss Landscape and Generalization
Improving generalization through loss landscape smoothing has become prominent research di- rection deep learning, as flatter minima are often associated with improved robustness and better out-of-distribution performance. This phenomenon was revealed by pioneer work [9]. After this, number influential techniques have been proposed to address this, most notably Sharpness- Minimization (SAM) [51, which explicitly penalizes sharp minima by optimizing for the worst-case loss within a neighborhood around each parameter update. With SAM, various SAM-variant method was proposed in recent years [17 32 [5]. However, these methods incur high computational costs due to the use of double backpropagation during training. Nevertheless, we argue that the insight they provide is valuable, and applying this work, we aim to analyze the loss landscape of 3D HPE to leverage this insight and propose a simple yet effective method based the analysis.
the 3D HPE task is worthwhile despite the overhead. In
5.3 Deep Ensemble Learning
Deep ensemble learning has emerged as a widely used technique to improve the robustness and generalization of neural networks, particularly in settings where model uncertainty and out-of- distribution (OOD) performance are critical. [18] introduced a simple yet effective deep ensemble method by independently training multiple neural networks with random initialization, showing improved predictive uncertainty estimation and better calibration compared to single models. Based on this. [ introduced simple yet effective deep ensemble method by leveraging multiple checkpoints during training process. 66] proposed fast geometric ensembling method by utilizing connected modes property to enhance the generalization capability. [15] proposed weight ensembling method to find more flatter region on the loss landscape. Despite their effectiveness, these approaches tend to be computationally expensive or rely on two-stage training pipelines, which limit their practicality. In this work, aim to propose parameter-effieient ensembling method to boost generalization capability of the 3D HPE model.
6 Conclusion
In this work, we analyzed the loss landscape of the 3D HPE task. Our empirical investigation showed that the loss landscape of the 3D HPE can be highly complicated and potentially have multiple local minima, which poses significant challenge during training process. This observations highlight the importance of considering loss landscape geometric properties during optimization. To mitigate these issues, we introduced a novel and simple adaptive scaling mechanism to flatten the loss landscapes. With this adaptive scaling approach, we also provide a efficient ensembling method to boost the generalization ability of the model. The proposed method effectively addresses the challenges identified in our analysis, and our experimental results validate the soundness and practicality of our approach. Furthermore, our adaptive scaling mechanism could be beneficial for tasks with highly complicated loss landscapes, such as 3D HPE, although must also account for task-specific characteristics, including the location where adaptive scaling mechanisms are applied.
Acknowledgements
This work was supported by Institute of Information & communications Technology Planning Evaluation (IIP)) grant funded by the Korea (MV)) (No. RS RS-2011-1111009 Artificial Intelligence Graduate School Program (Korea University), No. IITP-2025-RS, 00105-RR-2002-0200455 Artificial Intelligence Star Fellowship support program to nurture the best talents, No. ITTP-2025 0024-00436857, Information Technology Research Center, and No. RS 28-502, Al Research Hub Project).
10

---

References
[1] Maksym Andriushchenko and Nicolas Flammarion. Towards understanding sharpness-aware minimization. In International Conference on Machine Learning, pages 639-668. PMLR, 2022.
[2] Michael Black, Priyanka Patel, Joachim Tesch, and Jinlong Yang. Bedlam: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8726-8737, 2023.
[3] Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cascaded pyramid network for multi-person pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2018.
[4] Hai Ci, Mingdong Wu, Wentao Zhu, Xiaoxuan Ma. Hao Dong, Fangwei Zhong, and Yizhou Wang. Gfpose: Learning 3d human pose prior with gradient fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4800-4810 4800,480, 2023.
[5] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness aware minimization for ef- ficiently improving generalization. In International Conference on Learning Representations. International Conference Leamnng Representations, 2020.
[6] Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in Neural Information Processing Systems, 31, 2018.
[7] Kehong Gong, Jianfeng Zhang, and Jiashi Feng. Poseaug: A differentiable pose augmentation framework for 3d human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8575-8584, 2021.
[8] Jumin Han, Jun-Hee Kim, and Seong- Whan Lee. Propose: Probabilistic 3d human pose estimation with instance-level distribution and normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3338-3346, 2025.
[9] Sepp Hochreiter and Jürgen Schmidhuber. Simplifying neural nets by discovering flat minima. Advances in Neural Information Processing Systems, 7, 1994.
[10] Mir Rayat Imtiaz Hossain and James J Little. Exploiting temporal information for 3d human pose estimation. In Proceedings of the European Conference Computer Vision, pages 68 68-84, 2018.
[11] Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E Hopcroft, and Kilian Q Weinberger. Snapshot ensembles: Train 1, get m for free. arXiv preprint arXiv: 1704.00109 2017.
[12] Isabella Huang and Ruzena Bajesy, High resolution soft tactile interface for physical human-robot interaction. In IEEE International Conference Robotics and Automation, pages 1705-1711. IEEE, 2020.
[13] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448 456. PMLR, 2015.
[14] Catalin lonescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions Pattern Analysis and Machine Intelligence, 36(7):3325 12305,133,2220 1339,2013. 2013.
[15] Pavel Izmailov, Dmitrii Podoprikhin. Timur Garipov. Dmiity Vetrov, and Andrew Gordon Wilson. Averag- ing weights leads to wider optima and better genoralization. arXiv preprint arXiv:1803.054: 2018.
[16] Jun-Hee Kim and Seong. Whan Lee. Toward approaches to scalability in 3d human pose estimation. Advances in Neural Information Processing Systems, 37: 37:105476-105 1055002, 2024.
[17] Jungmin Kwon, Jeongseop Kim, Hyunseo Park, and In Kwon Choi. Asam: Adaptive sharpness-aware minimization for scale-invarian learning of deep neural networks. In International Conference on Machine Learning, pages 1905-5914 PMLR, 2021.
[18] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems, 30, 2017.
[19] Chen Li and Gim Hee Lee. Generating multiple hypotheses for 3d human pose estimation with mixture density network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 9887-9895, 2019.
11

---

[20] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in Neural Information Processing Systems, 31, 2018.
[21] Shichao Li, Lei Ke, Kevin Pratama, Yu-Was YunWing Tai, Chi-Keung Tang, and Kwang-Ting Cheng. Cascaded deep monocular 3d human pose estimation with evolutionary training data. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 6173-6183, 2020.
[22] Wenhao Li, Hong Liu, Hao Tang, Pichao Wang, and Luc Van Gool. Mhformer: Multi hapttisypoohesis transformer for 3d human pose estimation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 13147-13156, 2022.
[23] Julieta Martinez, Rayat Hossain, Javier Romero, and James J Little. A simple yet effective baseline for 3d human pose estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pages 2640-2649, 2017.
[24] Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, and Christian Theobalt. Monocular 3d human pose estimation in the wild using improved enn supervision. In International Conference on 3D Vision, pages 506-516. IEEE, 2017.
[25] Peng Mi, Li Shen, Tianhe Ren, Yiyi Zhou, Xiaoshuai Sun, Rongrong and Dacheng Tao. Make sharpness- aware minimization stronger: A sparsified perturbation approach. Advances in Neural Information Processing Systems, 35:30950-30962, 2022.
[26] Chang-Beon -eom Park and Seong- Whan Lee. Real-time 3d pointing gesture recognition for mobile robots with cascade hmm and particle filter. Image and Vision Computing, 29:51-63, 2011.
[27] Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 3d human pose estimation in video with temporal convolutions and semi-supervised training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7753-7762, 2019.
[28] Qucheng Peng, Ce Zheng, and Chen Chen, A dual-aagmentor framework for domain generalization in 3d human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2240-2249, 2024.
[29] Wenkang Shan, Haopeng Lu, Shanshe Wang. Xinfeng Zhang, and Wen Gao. Improving robustness and accuracy via relative information encoding in 3d human pose estimation. In Proceedings of the 29th ACM International Conference on Multimedia, pages 3446-3454, 2021.
[30] Timo Von Marcard, Roberto Henschel, Michael J Black, Bodo Rosenhahn, and Gerard Pons-Moll, Re- covering accurate 3d human pose in the wild using imus and a moving camera. In Proceedings of the European Conference on Computer Vision, pages 601 617. Springer, 2018
[31] Arso M Vukicevic, Ivan Macuzic, Nikola Mijailovic, Aleksandar Peulic, and Milos Radovic. Assessment of the handcart pushing and pulling safety by using deep learning 3d pose estimation and iot force sensors. Expert Systems with Applications. 183:115371, 2021.
[32] Pengfei Wang, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Sharpness-aware gradient matching for domain generalization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 3769-3778, 2023.
[33] Feiyi Xu, Feng Xu, Jiucheng Xic, Chi-Man Pun, Huimin Lu, and Hao Gao. Action recognition framework in traffic scene for autonomous driving system. IEEE Transactions on Intelligent Transportation Systems, 23(11):22301 2223111111,,,,,,,, 2021.
[34] Zhewei Yao. Amir Gholami, Kurt Keutzer, and Michael W Mahoney. Pyhessian: Neural networks through the lens of the hessian. In IEEE International Conference on Big Data, pages 581-590. IEEE, 2020.
[35] Ailing Zeng, Xiao Sun, Fuyang Huang, Minhao Liu, Qiang Xu, and Stephen Lin. Smet: Improving generalization in 3d human pose estimation with split-and-reoombine approach. In Proceedings of the European Conference on Computer Vision, pages 507-523. Springer, 2020.
[36] Jinlu Zhang, Zhigang Tu, Jianyu Yang, Yujin Chen, and Junsong Yuan. Mixste: Seq2seq mixed spatio- temporal encoder for 3d human pose estimation in video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13232-13242, 2022.
[37] Long Zhao, Xi Peng. Yu Tian, Mubbasir Kapadia, and Dimitris N Metaxas. Semantic graph convolutional networks for 3d human pose regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3425-3435. 2019.
12

---

[38] Ce Zheng, Sijie Zhu, Matias Mendieta, Taojiannan Yang. Chen Chen, and Zhengming Ding. 3d human pose estimation with spatial and temporal transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11666-11663 1663, 2021.
[39] Jingxiao Zheng. Xinwei Shi, Alexander Gorban. Junhua Mao, Yang Song. Charles R Qi. Ting Liu, Visesh Chari, Andre Cormman, Yin Zhou, et al. Multi-modal 3d human pose estimation with weak supervision in autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4478-4487, 2022.
[40] Christian Zimmermann, Tim Welschehold, Christian Domhege, Wolfram Burgard, and Thomas Brox. 3d human pose estimation in rgbd images for robotic task learning. In IEEE International Conference on Robotics and Automation, pages 1986- 1992. IEEE, 2018.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We made contributions in the abstract and introduction clearly by introduc- ing our adaptive scaling mechanism and efficient ensembling strategy.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We create section for limitations in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper doesn't 't include theoretical results that requires rigorous proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: In the paper, we provide detailed experimental results on benchmark dataset such as H36M and 3DHP and reported the metric (MPJPE, PCK, and AUC).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[Yes]
Justification:
The code is included in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/dde/eeee Cooeeeoodeesssseeeeee //nips.com htte://alllllgiiiee///oeeeuaaass ssiniiiiccypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We provided a detailed analysis setting in the analysis section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: We discuss about the tables and figures with standard evaluation metric in the 3D HPE area.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We explain the GPU we used in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The method we proposed fitted to the ethical guidelines for research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide potential positive societal results for our method in the introduction
section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper has no risks about the data or models that have a high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly cites sources and resources such dataset properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduced adaptive scaling mechanism and efficient ensembling strategy appropriately.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does not involved with any crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper doesn't have potential risks.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: We don't 't use LLMs for important core methods in the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy htttap///aaarrrpppcccc/oootrreep for what should or should not be described.
20


Reviews:
[{"review_id": "bH6vmGb1nI", "reviewer": "Reviewer_8o6Y", "summary": "This work presents a training and ensemble strategy for 3D human pose estimation (3D HPE) based on loss landscape analysis. The motivation is that the 3D HPE’s loss surface contains multiple disconnected sharp minima that hinder generalization. To address this problem, an adaptive scaling mechanism is adopted to smooth the loss landscape and encourage the discovery of diverse, generalizable solutions. Building on this, they propose a simple and architecture-agnostic ensemble method that effectively leverages solution diversity to enhance robustness, all with minimal training overhead. Experiments are conducted on Human3.6M and MPI-INF-3DHP, to validate the effectiveness of the proposed designs on 4 typical architectures, MLP, CNN, GCN, and Transformer. The proposed designs achieve promising improvement (over 1.6% at most time) in both MPJPE and PA-MPJPE.", "questions": "1. Please add the details of experiment settings. \n2. Experiments that train with more datasets would be very helpful to identify the significance of the proposed method.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths.\n1. New perspective.  \nThis paper introduces a insightful approach by analyzing the 3D HPE at the perspective of loss landscape, which is rarely explored in this field. \n2. Generalizable design.  \nThe proposed adaptive scaling mechanism seems to be easy to added to different model and architecture-agnostic. This great property makes it widely applicable. Experiments on Human3.6M and MPI-INF-3DHP validate it. \n3. Insightful experiments.  \nThis paper provides insightful experiments to show that the proposed method leads to flatter local minima by estimating the top-1 eigenvalue of the Hessian matrix and visualizing both local and global loss landscapes. This analysis shows that adaptive scaling reduces curvature and eliminates high-loss barriers between solutions, which supports the claim of improved generalization.  \nAlso, the comparison with Sharpness-Aware Minimization (SAM), an existing method with global flattening effect, shows that the proposed   method can avoid biasing the landscape around specific local minimal. \n\n\nWeakness. \n1. Lacking sufficient details of experiment settings.   \nTo further convince the readers, we really need to add more details of the experiments on H36M and MPI-INF-3DHP, including the training datasets used in Tab. 3 and 4, how to combine the SAM, etc. The SAM comparison would benefit from stronger experimental support, such as better tuning or combining with ensembling.  \nAlso, the Hessian estimation method (e.g., power iteration, approximation details) is not clearly described and should be explicitly stated for reproducibility.\n2. The models are only trained on Human3.6M. Whether the proposed can help the model trained on more datasets remains unclear.   \nOne potential limitation is that all experiments are conducted with models trained exclusively on the Human3.6M dataset. While the cross-dataset evaluation on MPI-INF-3DHP is valuable, it remains unclear whether the proposed adaptive scaling mechanism can provide similar benefits in more realistic settings where models are trained on large-scale and diverse datasets (e.g., combined H36M + MPI-INF-3DHP + BEDLAM, and validate on 3DPW). In such settings, the optimization landscape may already exhibit flatter or more entangled structures, potentially reducing the marginal gain from the proposed smoothing strategy. The generality of the method in large-data regimes thus remains an open question and could be further explored.", "was_revised": true, "final_justification": "Via training with an additional in-the-wild dataset, SkiPose, and test on 3DPW, the proposed method still achieve promising improvement than the baseline model. Such an experiment may indicate that the proposed method could be beneficial to models trained on large-scale and diverse datasets. Different from BEDLAM, SkiPose is relatively limited in diversity of human motion. Please consider to show the results using BEDLAM in the final version. Because we still can't be sure of how much improvement the proposed method can bring to the models trained with large-scale and diverse datasets, the significance of the proposed method is hard to determine in its current state, therefore, I keep the original score.", "rebuttal": "We sincerely thank the reviewer for recognizing the strengths of our work, especially our loss landscape perspective, generalizable design, and insightful analysis. We appreciate the constructive feedback and provide the following clarifications:\n\n> # W1. More details of experimental settings.\n\n- **Dataset & Training**: Tab. 4 and 5 use Human3.6M (Train: S1,5,6,7,8 / Test: S9,11) without flip augmentation. All models in Tab. 4 and 5 are trained for 60 epochs due to the slow convergence, especially under SAM and ensemble settings.\n- **SAM Settings**: We used perturbation radius = 0.05 in Tab. 3. Below are results for various radiuses, with and without our ensembling strategy (3 heads):\n\n\n|Model|0.01|0.02|0.03|0.05|0.07|\t\n|-|-|-|-|-|-|\n|BaselineNet + SAM|55.0|55.5|55.8|56.1|56.5|\t\n|BaselineNet + SAM + Ens.|**54.2**|**54.4**|**54.5**|**54.5**|**54.6**|\t\n|VPose + SAM|56.3|56.7|57.1|57.3|57.8|\t\n|VPose + SAM + Ens.|**55.0**|**55.1**|**55.3**|**55.3**|**55.5**|\n\nAs we expected, the performances are not satisfactory with various perturbation radii of SAM.\nHowever, SAM + Ensemble shows some performance gains over SAM alone, as compared to Table 4 in the main paper. This supports our claim that the loss landscape of 3D HPE is highly complex, with diverse local minima that can be better exploited via ensembling.\n\nAlso, comparing to our method (BaselineNet+Scale: **54.2**, BaselineNet+Both: **53.6**, VPose+Scale: **54.9**, and VPose+Both:**54.4**), the results show that our method outperforms SAM. \n\n- **Hessian estimation**: We use the power iteration method with maximum 100 steps and a tolerance of 0.001 for top-1 eigenvalue computation.\n\n> # W2. The effect on more diverse training dataset.\n\nWe train our method on a combined dataset (H36M+3DHP+SkiPose) and evaluate the model on 3DPW. Note that we try to download BEDLAM, however, downloading BEDLAM takes a couple of days. In this reason, we leverage SkiPose instead of BEDLAM. \nSkiPose provides dynamic motion diversity, which reflects diverse training data distribution.\nFor details, we use GT 2D pose input for all model and set the number of heads as 3.\nThe results show our method can be more effective on diverse dataset, which supports our motivation. \n\n|Model|MPJPE &darr;|PA-MPJPE &darr;|PCK &uarr;|\n|-|-|-|-|\n|BaselineNet|92.2|57.5|84.1|\n|BaselineNet+Ours|**85.9**|**55.8**|**85.9**|\n|VPose|93.6|59.6|84.3|\n|VPose+Ours|**87.1**|**56.4**|**85.8**|\n\nWe explain this results as below. \n\n3D HPE is an ill-posed problem — a single 2D input can correspond to multiple plausible 3D poses. This implies that, at a given parameter point $\\theta$, a single input can induce multiple possible loss values due to depth ambiguity. We can quantify this ambiguity using the Depth Ambiguity Ratio (DAR). The higher DAR leads to greater variance in possible loss values, which in turn suggests sharper or more irregular loss landscapes. This is empirically supported by Fig. 2(b) in the paper. Therefore, when training on more diverse datasets (which often have a higher range of DAR), the resulting loss landscape becomes more complex and fluctuating.\n\nIn fact, by examining the top-1 eigenvalue of the Hessian matrix for each $C_k$ in the combined dataset, we can see that the local loss landscape geometry becomes significantly sharper compared to the model trained only on Human3.6M.\nBelow are results when we set K as 21. For hessian estimation, we set power iteration as 100 and tolerance as 0.001.\n\n|Class|$C_1$|$C_2$|$C_3$|$C_4$|$C_5$|$C_6$|$C_7$|$C_8$|$C_9$|$C_{10}$|$C_{11}$|$C_{12}$|$C_{13}$|$C_{14}$|$C_{15}$|$C_{16}$|$C_{17}$|$C_{18}$|$C_{19}$|$C_{20}$|$C_{21}$|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|$\\lambda_{max}$(H36M)|2.0|1.4|1.3|1.2|1.1|1.1|1.1|1.1|1.1|1.1|1.2|1.3|1.5|1.8|2.2|3.0|3.6|4.0|4.0|4.4|8.3|\n|$\\lambda_{max}$(combined dt.)|**3.8**|**3.3**|**2.9**|**2.6**|**2.5**|**2.4**|**2.4**|**2.3**|**2.3**|**2.4**|**2.6**|**3.0**|**3.7**|**4.7**|**6.4**|**8.1**|**9.3**|**16.2**|**67.8**|**60.1**|**62.8**|"}, {"review_id": "PvfGLionBV", "reviewer": "Reviewer_MPr4", "summary": "This paper addresses the challenge of generalization in 3D Human Pose Estimation (HPE). The authors argue that the loss landscape of 3D HPE is highly complex and has multiple disconnected local minima, which can negatively impact model generalization. To address this, they propose a simple adaptive scaling mechanism that smooths the loss landscape, reducing the likelihood of high-loss barriers between local minima. Furthermore, they introduce an efficient ensemble approach that combines diverse solutions on the smoothed loss landscape. The method is designed to be model-agnostic and easy to implement. Experimental results on benchmark datasets such as Human3.6M and MPI-INF-3DHP demonstrate improvements in generalization performance across various network architectures.", "questions": "1. How do you ensure the diversity of solutions when using different random seeds? Is there a risk of the models converging to similar local minima despite different initialization?\n\n2. Can you provide more insights into how the adaptive scaling mechanism affects the model's learning process? Are there any potential drawbacks or limitations to this mechanism?", "limitations": "Yes, but it is in the supplementary material.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n1. The paper provides a novel analysis of the 3D HPE task from a loss landscape perspective, which is underexplored in previous research. This analysis offers valuable insights into the challenges of optimizing 3D HPE models and their generalization performance.\n\n2. The proposed adaptive scaling mechanism and ensemble approach are simple yet effective. They are designed to induce flatness in the loss landscape, which is associated with better generalization. The method shows consistent performance improvements across different model architectures and datasets.\n\n3. The experiments are comprehensive and well-designed. The authors demonstrate the effectiveness of their method on multiple benchmark datasets and compare it with various state-of-the-art architectures. The results show significant improvements in generalization performance, validating the practical utility of the proposed approach.\n\n4. The method can be easily integrated into existing 3D HPE frameworks regardless of the model architecture. This makes it a versatile enhancement for the community.\n\n\nWeaknesses: \n\n1. While the paper provides empirical evidence of the effectiveness of the proposed method, the theoretical understanding of why and how the adaptive scaling mechanism and ensemble approach work is somewhat limited. More in-depth theoretical analysis could strengthen the paper. \n\n2. The paper primarily focuses on the generalization performance within the context of benchmark datasets. It would be beneficial to discuss the applicability of the method in real-world scenarios where data distributions may vary more significantly.", "was_revised": true, "final_justification": "The rebuttal addressed most of my concerns. I would keep my original rating.", "rebuttal": "We sincerely thank the reviewer for the positive comments and for recognizing the value of our contributions. We truly appreciate the careful reading and constructive feedback. Below, we provide detailed responses to each of the reviewer’s points.\n\n> # W1. A simple theoretical understanding of our method.\n\nWe provide a simple theoretical understanding of our method below. \n\n- **For Adaptive Scaling Mechanism**, \n\n\nThe formulation of Eq. 3 in the main paper is $\\frac{f_{\\theta_i}(X)}{\\sigma(h_{\\psi_i}(X))+1}$, where $\\sigma$ is ReLU function, $g_\\phi(x) = X$, and $x$ is input.\n\nAnd let’s say that $\\frac{f_{\\theta_i}(X)}{\\sigma(h_{\\psi_i}(X))+1} = \\frac{f_{\\theta_j}(X)}{\\sigma(h_{\\psi_j}(X))+1} = \\tilde{y}$. So each head has same output and same loss value.\n\nWhen $\\sigma(h_{\\psi_i}(X))+1 = a$ and  $\\sigma(h_{\\psi_j}(X))+1 = a+\\Delta a$, then $f_{\\theta_i}(X) = a\\tilde{y}$ and $f_{\\theta_j}(X) = a\\tilde{y} + \\Delta a\\tilde{y}$. ($a$ is positive)\n\nBy substitution, $\\sigma(h_{\\psi_j}(X)) - \\sigma(h_{\\psi_i}(X)) = \\Delta a$ and $f_{\\theta_j}(X) - f_{\\theta_i}(X) = \\Delta a\\tilde{y}$.\n\nBy the Taylor expansion, $f_{\\theta_j}(X) \\approx  f_{\\theta_i}(X) + \\nabla_\\theta f_{\\theta_i}(X)^T (\\theta_j - \\theta_i)$.\nSimilary, $\\sigma(h_{\\psi_j}(X)) \\approx \\sigma(h_{\\psi_i}(X)) + \\nabla_\\psi \\sigma(h_{\\psi_i}(X))^T (\\psi_j - \\psi_i)$.\n\nFor simplicity, we address the case of $\\theta$ here.\n\nBy the substitution, $\\nabla_\\theta f_{\\theta_i}(X)^T (\\theta_j - \\theta_i) \\approx \\Delta a\\tilde{y}$.\n\nHere, the gradient $\\nabla_\\theta f_{\\theta_i}(X)$ has large enough rank and is non-zero in practical.\n\nSo when $\\Delta a \\rightarrow 0$, the $(\\theta_j - \\theta_i) \\rightarrow 0$. This implies that there exists connected region in $\\theta$ space where shares same loss value depending on the value of $\\Delta a$.\n\n\n\n- **For Ensemble strategy**,\n\n\nLet’s say that $S_j = S_i + \\Delta S_{ij}$, where $S_i = (\\phi, \\theta_i, \\psi_i)$. \n\nBy Taylor expansion of  $F_{S_j}(x) = y,   F_{S_i+\\Delta S_{ij}} \\approx F_{S_i}(x) + \\nabla_S F_{S_i}(x)^T \\Delta S_{ij}$.\n\nIf $\\nabla_S F_{S_i}(x)^T$ and $\\Delta S_{ij}$ are non-zero, then $F_{S_i}(x) \\neq F_{S_j}(x)$, which means they are different function.\nIn practical, the gradient is not zero vector.\nAnd to make $\\Delta S_{ij} \\neq 0$, we intentionally initialized each head differently as mentioned in the paper.\n\nAs a result, each $F_{S_i}(x)$ is different functions so this can justify our ensemble strategy.\n\n\n> # W2. Real-world scenario (more diverse dataset).\n\nwe trained on a combined dataset (H36M + 3DHP + SkiPose), where SkiPose adds dynamic, realistic postures, better reflecting real-world scenarios.\nFor details, we use GT 2D pose input for all model and set the number of heads as 3.\nOur experiments show consistent improvements across metrics on 3DPW, confirming that our method benefits from and effectively handles increased data diversity.\n\n|Model|MPJPE &darr;|PA-MPJPE &darr;|PCK &uarr;|\n|-|-|-|-|\n|BaselineNet|92.2|57.5|84.1|\n|BaselineNet+Ours|**85.9**|**55.8**|**85.9**|\n|VPose|93.6|59.6|84.3|\n|VPose+Ours|**87.1**|**56.4**|**85.8**|\n\nWe can explain this as following. Since 3D HPE is inherently a one-to-many problem due to depth ambiguity, a single input can be matched to multiple solutions (the model can have multiple valid loss values for a single input). As a result, the resulting loss landscape becomes highly complex as data diversity increases. So our approach that smooths such complex landscapes can work well for diverse dataset.\n\n> # Q1. Ensuring the diversity of solutions from each head.\n\nWe address this question through a simple theoretical understanding.\n\nLet’s say $F_i(x) = \\frac{f_{\\theta_i}(g_\\phi(x))}{\\sigma(h_{\\psi_i}(g_\\phi(x)))+1}$ and $L=\\frac{1}{M}\\sum_{i=1}^{M}L_i(F_i(x),y)$, where $M$ is the number of heads and $L_i$ is loss value of $i^{th}$ head.\nThen, $\\frac{\\delta L}{\\delta \\theta_i} = \\frac{1}{M}\\frac{\\delta L_i}{\\delta F_i}\\frac{\\delta F_i}{\\delta f_i}\\frac{\\delta f_i}{\\delta \\theta_i} + \\alpha$.\nSimilary, $\\frac{\\delta L}{\\delta \\psi_i} = \\frac{1}{M}\\frac{\\delta L_i}{\\delta F_i}\\frac{\\delta F_i}{\\delta h_i}\\frac{\\delta h_i}{\\delta \\psi_i} + \\beta$.\n\nAt the start of training, we intentionally initialized each head differently in weight scale. This means that $F_i(x) \\neq F_j(x)$ and $L_i \\neq L_j$ at the start of training.\nThus, $\\frac{\\delta L}{\\delta \\theta_i} \\neq \\frac{\\delta L}{\\delta \\theta_j}$ at the start of training, which means each head get different loss gradient from the start of training. This also holds in the case of $\\psi$.\n\nHowever, while the possibility is low, there is no rigorous guarantee that each head will always converge to different local minima. For this reason, as mentioned in the paper, we intentionally initialized each head with different parameter values to encourage diversity in their convergence behavior.\n\n> # Q2. Insight for the impact of adaptive scaling mechansim on the learning process of the model.\n\nAs we mentioned in the paper, the adaptive scaling mechanism smooth the loss landscape, which means that the optimization becomes easier than before. \nThis means that the adaptive scaling mechanism accelerates convergence. So, the early stopping may be needed when applying the adpative scaling mechansim to avoid overfitting.\nIn fact, when we investigate the loss trajectories w/wo adaptive scaling mechanism only through the epochs, we can see the adaptive scaling mechanism boost the converging speed as shown in below table.\nNote that E1, E2, … refer to the epoch.\n\n|Model|E1|E2|E3|E4|E5|E6|E7|E8|E9|E10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|BaselineNet (Loss)|0.1137|0.0084|0.0045|0.0032|0.0025|0.0021|0.0018|0.00165|0.0014|0.0013|\n|BaselineNet+Scale only (Loss)|**0.0172**|**0.0031**|**0.0023**|**0.0019**|**0.0016**|**0.0014**|**0.0013**|**0.0012**|**0.0011**|**0.0010**|\n|BaselineNet (MPJPE)|115.8|92.1|91.3|74.6|66.1|61.2|60.6|58.5|60.2|59.5|\n|BaselineNet+Scale only (MPJPE)|**71.9**|**63.1**|**59.4**|**57.3**|**56.4**|**56.5**|**55.3**|**55.4**|**55.3**|**54.8**|"}, {"review_id": "Q0xv90DBmh", "reviewer": "Reviewer_s6cb", "summary": "This paper poses the critical issue of generalization in 3D Human Pose Estimation (HPE) by shifting focus from network architecture design to the loss landscape geometry. The authors first analyze the complex and fragmented loss landscapes present in HPE, particularly those arising from depth ambiguities. They introduce an adaptive scaling mechanism that encourages flatness in the local minima of the loss surface and propose an efficient ensembling framework based on multiple regression heads. This approach allows the model to explore and average diverse but high-performing solutions within a smoothed loss region. Experiments on Human3.6M and MPI-INF-3DHP demonstrate consistent improvements across multiple architectures (MLP, CNN, GCN, Transformer), both in-domain and cross-dataset.", "questions": "The paper focuses on 3D HPE. Could the proposed method be applied to other tasks with similarly complex loss landscapes, such as 2D pose estimation or object detection?", "limitations": "Yes.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1. Novel perspective on generalization in 3D HPE: The paper introduces an underexplored but highly relevant perspective by analyzing the loss landscape of 3D HPE. The findings are well-motivated and supported by visualizations and eigenvalue analysis.\n2. Simple yet Effective Adaptive Scaling: The adaptive scaling mechanism is elegant, architecture-agnostic, and incurs minimal additional computational cost. \n3. Thorough Experimental Validation: The paper conducts extensive experiments with multiple architectures and benchmarks, including cross-dataset evaluations, noise robustness, and comparison with SAM. Ablation studies clearly isolate the contribution of each component.", "was_revised": true, "final_justification": "Thank you for your response. I will keep the original score.", "rebuttal": "We thank the reviewer for raising this insightful question regarding the applicability of our method beyond 3D human pose estimation.\n\n> # Q1. The possibility of applying our method to other tasks.\n\nWhile our approach is tailored to the 3D HPE specific challenge, we believe the underlying principle, “encouraging local flatness in the loss landscape”, could potentially be beneficial for other tasks that suffer from similar optimization difficulties due to complex or multimodal loss surfaces. However, successful adaptation would likely require careful consideration of task-specific factors. For instance, the location or strategy of applying adaptive scaling might need to be modified depending on the architecture and characteristics of the target task. So, further investigation is needed, and we leave this as an exciting direction for future work."}, {"review_id": "9FCgP6Dnyr", "reviewer": "Reviewer_X5JK", "summary": "This paper explores the loss landscape in 3D human pose estimation task, and proposes an adaptive scaling mechanism to smooth the loss landscape. The authors visualize the loss landscape and show its impact on performance improvement. Based on this, they introduce a simple, model-agnostic ensemble method that leverages the smoothed landscape to combine diverse solutions. The approach improves performance across various HPE methods on benchmarks like Human3.6M and MPI-INF-3DHP.", "questions": "Please see the weaknesses listed above. I would be inclined to increase my rating if these concerns are addressed.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper is well-structured and easy to follow.\n2. The adaptive scaling mechanism and ensemble method are simple enough to reproduce.\n3. The experiments clearly demonstrate the smoothed loss landscape and the performance improvement.\n\nWeaknesses\n1. As mentioned in the limitations, the paper lacks theoretical analysis of the proposed method and its results, making it difficult to understand the underlying principles and justify its effectiveness.\n2. Section 3.2 does not explain how the different regression heads are obtained. Are they trained on different datasets, or are they simply initialized with different random seeds?\n3. In Figure 6(d), the performance of both methods is similar under small perturbations. At a perturbation level of 0.2, ΔMPJPE exceeds 100, which is significant. Although some robustness is shown under large perturbations, such a large error renders the results practically unusable.\n4. It is unclear how the value of K is selected and how it impacts the results. The paper lacks experiments or analysis to support this design choice.", "was_revised": true, "final_justification": "The authors’ response addressed my concerns, and I have decided to raise my ranking from 3 to 4.", "rebuttal": "We appreciate the reviewer for taking the time to carefully review our paper. We deeply appreciate the constructive comments. We are pleased to hear that our contributions were well received. In this response, we aim to carefully address the feedback provided by the reviewer.\n\n> # W1. A simple theoretical understanding of our method.\n\nWe provide a simple theoretical understanding of our method below. \n\n- **For Adaptive Scaling Mechanism**, \n\n\nThe formulation of Eq. 3 in the main paper is $\\frac{f_{\\theta_i}(X)}{\\sigma(h_{\\psi_i}(X))+1}$, where $\\sigma$ is ReLU function, $g_\\phi(x) = X$, and $x$ is input.\n\nAnd let’s say that $\\frac{f_{\\theta_i}(X)}{\\sigma(h_{\\psi_i}(X))+1} = \\frac{f_{\\theta_j}(X)}{\\sigma(h_{\\psi_j}(X))+1} = \\tilde{y}$. So each head has same output and same loss value.\n\nWhen $\\sigma(h_{\\psi_i}(X))+1 = a$ and  $\\sigma(h_{\\psi_j}(X))+1 = a+\\Delta a$, then $f_{\\theta_i}(X) = a\\tilde{y}$ and $f_{\\theta_j}(X) = a\\tilde{y} + \\Delta a\\tilde{y}$. ($a$ is positive)\n\nBy substitution, $\\sigma(h_{\\psi_j}(X)) - \\sigma(h_{\\psi_i}(X)) = \\Delta a$ and $f_{\\theta_j}(X) - f_{\\theta_i}(X) = \\Delta a\\tilde{y}$.\n\nBy the Taylor expansion, $f_{\\theta_j}(X) \\approx  f_{\\theta_i}(X) + \\nabla_\\theta f_{\\theta_i}(X)^T (\\theta_j - \\theta_i)$.\nSimilary, $\\sigma(h_{\\psi_j}(X)) \\approx \\sigma(h_{\\psi_i}(X)) + \\nabla_\\psi \\sigma(h_{\\psi_i}(X))^T (\\psi_j - \\psi_i)$.\n\nFor simplicity, we address the case of $\\theta$ here.\n\nBy the substitution, $\\nabla_\\theta f_{\\theta_i}(X)^T (\\theta_j - \\theta_i) \\approx \\Delta a\\tilde{y}$.\n\nHere, the gradient $\\nabla_\\theta f_{\\theta_i}(X)$ has large enough rank and is non-zero in practical.\n\nSo when $\\Delta a \\rightarrow 0$, the $(\\theta_j - \\theta_i) \\rightarrow 0$. This implies that there exists connected region in $\\theta$ space where shares same loss value depending on the value of $\\Delta a$.\n\n\n\n- **For Ensemble strategy**,\n\n\nLet’s say that $S_j = S_i + \\Delta S_{ij}$, where $S_i = (\\phi, \\theta_i, \\psi_i)$. \n\nBy Taylor expansion of  $F_{S_j}(x) = y,   F_{S_i+\\Delta S_{ij}} \\approx F_{S_i}(x) + \\nabla_S F_{S_i}(x)^T \\Delta S_{ij}$.\n\nIf $\\nabla_S F_{S_i}(x)^T$ and $\\Delta S_{ij}$ are non-zero, then $F_{S_i}(x) \\neq F_{S_j}(x)$, which means they are different function.\nIn practical, the gradient is not zero vector.\nAnd to make $\\Delta S_{ij} \\neq 0$, we intentionally initialized each head differently as mentioned in the paper.\n\nAs a result, each $F_{S_i}(x)$ is different functions so this can justify our ensemble strategy.\n\n> # W2. How to obtain the different regression heads?\n\nAs mentioned in the main paper and Supplementary Sec. D, we obtain different regression heads by initializing each head differently. Specifically, we use orthogonal initialization [1] for each head to promote diversity, and further add scaled noise sampled from $N(0,I)$ to encourage variation in weight magnitudes.\n\nAll heads are trained on the same dataset so they observe the same data and labels. We also experimented with training each head on a different data split, but we observed no significant improvement in diversity or downstream performance. Therefore, we chose the simpler setup of using the same dataset with diversified initialization.\n\nWe will clarify this point more explicitly in the revised version.\n\n[1] A. Saxe et al., “Exact solutions to the nonlinear dynamics of learning in deep linear neural networks,” ICLR, 2014.\n\n> # W3. Noise robustness of our method.\n\nWe present detailed results at small perturbation level for BaselineNet in MPJPE.\n\n|Model|0|0.01|0.03|0.05|0.07|0.09|0.1|\n|-|-|-|-|-|-|-|-|\n|BaselineNet|89.8|90.2|93.6|100.4|108.6|118.3|123.7|\n|BaselineNet+Ours|**85.6**|**85.7**|**89.9**|**97.9**|**106.4**|**116.2**|**121.5**|\n\nIt is true that while our method consistently outperforms the baseline across all noise levels in terms of MPJPE, the relative robustness is not superior at low perturbation levels.\nHowever, we would like to clarify that our goal was to demonstrate the overall stability of our method under noise, particularly in scenarios that simulate substantial input variation. In fact, we interpret robustness under large perturbations as a proxy for the model's ability to generalize to unseen or out-of-distribution poses as also described in the paper (line319-320), which are common in real-world applications such as outdoor or sports environments. This was the motivation behind the analysis.\n\nWe will clarify this in the future version.\n\n> # W4. The impact of K.\n\nIn Section 2 of the main paper, we use K = 3 for visualization purposes only—this allows us to clearly and easily depict the geometry of local loss landscapes and their corresponding local minima within a shared coordinate system. To address how different values of K impact the global loss landscape, we conducted further experiments with K = 4, 5, 6, 7, 8. We present loss values on the global loss landscape at linearly interpolated parameter between local minima of each local loss landscape. The results are presented in the table below. Note that $L_{max}(\\theta_{C_i}, \\theta_{C_j})$ refers to $max_{\\alpha \\in [0,1]}\\left(L((1-\\alpha)\\theta_{C_i}+\\alpha\\theta_{C_j})\\right)$. As we are not allowed to present figure, please understand the way we present results.\n\nWhen K=4.\n|Model|$\\theta_{C_1}$|$L_{max}(\\theta_{C_1}, \\theta_{C_2})$|$\\theta_{C_2}$|$L_{max}(\\theta_{C_2}, \\theta_{C_3})$|$\\theta_{C_3}$|$L_{max}(\\theta_{C_3}, \\theta_{C_4})$|$\\theta_{C_4}$|$L_{max}(\\theta_{C_4}, \\theta_{C_1})$|$\\theta_{C_1}$|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|BaselineNet (Loss)|0.015|**0.056**|0.007|**0.060**|0.005|**0.038**|0.010|**0.086**|0.015|\n|VPose (Loss)|0.012|**0.012**|0.003|**0.007**|0.002|**0.008**|0.005|**0.012**|0.012|\n\nWhen K=5.\n|Model|$\\theta_{C_1}$|$L_{max}(\\theta_{C_1}, \\theta_{C_2})$|$\\theta_{C_2}$|$L_{max}(\\theta_{C_2}, \\theta_{C_3})$|$\\theta_{C_3}$|$L_{max}(\\theta_{C_3}, \\theta_{C_4})$|$\\theta_{C_4}$|$L_{max}(\\theta_{C_4}, \\theta_{C_5})$|$\\theta_{C_5}$|$L_{max}(\\theta_{C_5}, \\theta_{C_1})$|$\\theta_{C_1}$|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|BaselineNet (Loss)|0.018|**0.019**|0.010|**0.041**|0.007|**0.041**|0.006|**0.033**|0.013|**0.114**|0.018|\n|VPose (Loss)|0.012|**0.012**|0.005|**0.007**|0.002|**0.030**|0.002|**0.012**|0.006|**0.013**|0.012|\n\nWhen K=6.\n|Model|$\\theta_{C_1}$|$L_{max}(\\theta_{C_1}, \\theta_{C_2})$|$\\theta_{C_2}$|$L_{max}(\\theta_{C_2}, \\theta_{C_3})$|$\\theta_{C_3}$|$L_{max}(\\theta_{C_3}, \\theta_{C_4})$|$\\theta_{C_4}$|$L_{max}(\\theta_{C_4}, \\theta_{C_5})$|$\\theta_{C_5}$|$L_{max}(\\theta_{C_5}, \\theta_{C_6})$|$\\theta_{C_6}$|$L_{max}(\\theta_{C_6}, \\theta_{C_1})$|$\\theta_{C_1}$|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|BaselineNet (Loss)|0.019|**0.066**|0.127|**0.519**|0.008|**0.036**|0.007|**0.030**|0.007|**0.036**|0.019|**0.127**|0.019|\n|VPose (Loss)|0.014|**0.015**|0.006|**0.006**|0.003|**0.005**|0.002|**0.004**|0.003|**0.008**|0.008|**0.014**|0.014|\n\nWhen K=7.\n|Model|$\\theta_{C_1}$|$L_{max}(\\theta_{C_1}, \\theta_{C_2})$|$\\theta_{C_2}$|$L_{max}(\\theta_{C_2}, \\theta_{C_3})$|$\\theta_{C_3}$|$L_{max}(\\theta_{C_3}, \\theta_{C_4})$|$\\theta_{C_4}$|$L_{max}(\\theta_{C_4}, \\theta_{C_5})$|$\\theta_{C_5}$|$L_{max}(\\theta_{C_5}, \\theta_{C_6})$|$\\theta_{C_6}$|$L_{max}(\\theta_{C_6}, \\theta_{C_7})$|$\\theta_{C_7}$|$L_{max}(\\theta_{C_7}, \\theta_{C_1})$|$\\theta_{C_1}$|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|BaselineNet (Loss)|0.022|**0.062**|0.016|**0.018**|0.010|**0.047**|0.008|**0.053**|0.007|**0.030**|0.009|**0.036**|0.019|**0.145**|0.022|\n|VPose (Loss)|0.014|**0.014**|0.007|**0.009**|0.003|**0.004**|0.002|**0.008**|0.002|**0.005**|0.003|**0.008**|0.008|**0.014**|0.014|\n\nWhen K=8.\n|Model|$\\theta_{C_1}$|$L_{max}(\\theta_{C_1}, \\theta_{C_2})$|$\\theta_{C_2}$|$L_{max}(\\theta_{C_2}, \\theta_{C_3})$|$\\theta_{C_3}$|$L_{max}(\\theta_{C_3}, \\theta_{C_4})$|$\\theta_{C_4}$|$L_{max}(\\theta_{C_4}, \\theta_{C_5})$|$\\theta_{C_5}$|$L_{max}(\\theta_{C_5}, \\theta_{C_6})$|$\\theta_{C_6}$|$L_{max}(\\theta_{C_6}, \\theta_{C_7})$|$\\theta_{C_7}$|$L_{max}(\\theta_{C_7}, \\theta_{C_8})$|$\\theta_{C_8}$|$L_{max}(\\theta_{C_8}, \\theta_{C_1})$|$\\theta_{C_1}$|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|BaselineNet (Loss)|0.023|**0.060**|0.016|**0.054**|0.011|**0.087**|0.008|**0.042**|0.007|**0.036**|0.009|**0.040**|0.009|**0.053**|0.023|**0.125**|0.023|\n|VPose (Loss)|0.015|**0.017**|0.008|**0.011**|0.005|**0.006**|0.003|**0.005**|0.002|**0.007**|0.002|**0.004**|0.003|**0.010**|0.010|**0.015**|0.015|\n\nThese values consistently demonstrate that the local minima of each local loss landscape remain disconnected on the global loss landscape, confirming our first key insight, the \"Flatness around each local minimum of local loss landscape\", holds irrespective of K. Furthermore, considering the inherent redundancy and imblance in 3D HPE datasets, this behavior is likely to generalize across different values of K. Since the maximum meaningful value of K is the dataset size, we design our method under the assumption that K equals to the dataset size.\n\nWe will clarify this in the future version."}, {"review_id": "F9V5zqsj3P", "reviewer": "Reviewer_sYww", "summary": "Overall, this submission focuses on improving the generalization performance of 3D human pose estimation models. It achieves this with the help of flat loss landscape and ensemble.", "questions": "Overall, the reviewer currently votes for borderline rejection for this submission due to the following concerns.\n\n1. Firstly, the reviewer is confused about the relationship between the proposed method and investigation and the 3D HPE task. Specifically, while it can be admitted that generalization is important in 3D HPE, it is also important for almost any other computer vision tasks. Thus, why the authors focus on improving the generalization performance of models while specifically focuses on 3D HPE is confusing, especially considering that the reviewer seems to also fail to get the relationshipf between the proposed method and the 3D HPE task. Instead, it seems that both parts of the methods are quite general methods that are not specifically relevant to the 3D HPE task.\n\n2. Moreover, the reviewer also appreciate the authors to better elaborate and explain on the significance behind its key insights. Specifically, the first key insight that the authors seem to aim to elaborate in Sec 2.4 seems to be existing and relatively general. The reviewer thus fails to understand why this insight need to be re-mentioned particularly in the 3D HPE context. Similarly, the second key insight that ensembling can improve generalization capability looks to be also very very general and common sense from the reviewer's perspective.\n\n3. Meanwhile, the authors claim that \"our method requires nearly the same amount of time as training a single model\". Yet, this claim seems to be very inadquately supported. Specifically, the reviewer seems to only see from Table 2 in Appendix that, across different baselines, the proposed method all obviously increase the inference time. Yet, the reviewer seems to see no clear comparison w.r.t. the training time. The reviewer appreciates explanation over the improper support of this claim.\n\n4. Lastly, in the introduction line 23-28, the authors seem to want to show that many 3D HPE methods have focused on enhancing robustness. While this is the case, in the experimental section, the authors seem to compare to none of these robustness enhancing 3D HPE works. Instead, it only uses several methods as baselines and apply the proposed method on these baselines. This leads the experiments to be significantly non-comprehensive.\n\nGiven the reviewer's above concerns, the reviewer currently tends to reject this submission.", "limitations": "Yes.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n\nOverall, this submission is easy to follow while it emphrically also demonstrates good performance.\n\nWeaknesses:\n\n(See questions section for more details).", "was_revised": true, "final_justification": "After reading the authors rebuttal, the reviewer now gets the relationship between their method and this particular 3D HPE task. The reviewer has raised point from 3 to 4. Yet, the reviewer strongly suggests the authors to carefully edit their camera ready based on the answers provided in the rebuttal period.", "rebuttal": "We appreciate the reviewer for taking the time to carefully review our paper. We deeply appreciate the constructive comments. We are pleased to hear that our paper is easy to follow.\n\n> # W1. The relationship between our method and the rationales of our analysis and 3D HPE task.\n\nA key distinction between 3D HPE and other computer vision tasks lies in its inherent ambiguity, which means that a single 2D input can correspond to multiple valid 3D poses (the model can have multiple valid loss values for a single input). This one-to-many mapping sets 3D HPE apart from problems with more deterministic input-output relationships. We quantify this ambiguity using the Depth Ambiguity Ratio (DAR), which measures the degree of depth uncertainty in the data. The higher DAR, the higher variance of possible loss values for a single input. This justify our analysis based on the DAR value, which directly affects the geometry of loss landscape. Our analysis based on the DAR shows that loss landscape of 3D HPE is highly complex, which is underexplored previously. Such loss landscape poses significant challenges for optimization in 3D HPE task, as standard training methods struggle to find flat and generalizable minima in the presence of this complexity. To address this, our method adapts adaptive scaling mechanism to smooth loss landscape, improving generalization ability specifically in 3D HPE task.\n\n\n> # W2. The connection between our key insights and 3D HPE task.\n\nOur first key insight focuses on “Flatness around local minimum of each **local** loss landscape”, which is different from “Flatness around local minimum of **global** loss landscape” in the standard deep learning literature.\nThe key difference between these two lies on the fact that our first key insight considers the different local loss landscape geometries, which is not considered in the other. Especially, this consideration has significant impact on 3D HPE task (unlike other computer vision tasks) considering the impact of DAR on the global loss landscape of 3D HPE task as we mentioned above. \nAs a result, previous loss landscape flattening method (e.g. SAM) smoothed the loss landscape in a biased manner as shown in Fig. 5(a) in the paper, whereas our adaptive scaling mechanism preserves balanced flatness across local loss landscape. This is desirable property for one-to-many problem such as 3D HPE.\n\nFor our second key insight, “Finding multiple solutions”, it is not about just “doing ensemble”. \nAs mentioned in the paper, the second key insight is motivated from the adaptive scaling mechasim and **plays a key role** to connect between adaptive scaling mechanism and ensembling on the smoothed loss landscape.\nIn fact, our ensembling strategy **without** adaptive scaling mechanism doesn’t work as shown in Tab. 4 in the paper. \nThis means that our ensembling strategy from second key insight is mutually complementary to adaptive scaling mechanism and facilitates an effective ensemble to further boost generalization ability of the 3D HPE model with adaptive scaling mechanism.\n\nAs a result, our key insights are derived from the 3D HPE-relevant issues and motivations.\n\nwe will clarify this in the future version.\n\n> # W3. Training time.\n\nWe present training time comparison for one epoch with various model architectures.\nWe use a single NVIDIA A6000 GPU for this comprarison.\nFor FLOPs, please refer to the supplementary material Sec. E.\n\n|Model|Time (sec.)|Model|Time (sec.)|Model|Time (sec.)|Model|Time (sec.)|\n|-|:-:|-|:-:|-|:-:|-|:-:|\n|Baseline|18|VPose|22|SemGCN|39|PoseFormer|40|\n|Baseline+Ours|19|VPose+Ours|24|SemGCN+Ours|42|PoseFormer+Ours|43|\n\n\n> # W4. Comparison with other works.\n\nThe goal of our work is fundamentally different from prior methods aimed at improving robustness in 3D HPE. While other methods focus on enhancing robustness through data augmentation (e.g. PoseAug[2]), proposing their own model architecture (e.g. RIE[1]), and multi-hypothesis concept (e.g. ProPose[3]), we approach the problem from a loss landscape perspective—by addressing the optimization difficulty caused by loss landscape complexity.\nImportantly, our method is orthogonal to these prior approaches and can be combined with them. We present this by applying our method on top of RIE, PoseAug, and ProPose, resulting in consistent improvements across benchmarks.\nThe models are trained on training dataset of H36M and evaluated on test sets of H36M or 3DHP. For Ours, we set the number of heads as 3.\n\n|Model|MPJPE(H36M)&darr;|PA-MPJPE(H36M)&darr;|MPJPE(3DHP)&darr;|PA-MPJPE(3DHP)&darr;|\n|-|:-:|:-:|:-:|:-:|\n|RIE[1]|53.1|42.7|89.4|68.3|\n|RIE[1]+Ours|**52.9**|**41.8**|**86.8**|**67.3**|\n|BaselineNet+PoseAug[2]|54.1|42.8|82.8|63.1|\n|BaselineNet+PoseAug[2]+Ours|**53.9**|**42.6**|**81.1**|**61.8**|\n|VPose+PoseAug[2]|54.7|43.1|83.8|63.8|\n|VPose+PoseAug[2]+Ours|**54.2**|**42.8**|**81.7**|**60.7**|\n\nFor multi-hypothesis concept, we report results in single-hypothesis setting below.\n\n|Model|PCK(3DHP)&uarr;|AUC(3DHP)&uarr;|\n|-|:-:|:-:|\n|ProPose[3]|84.4|52.1|\n|ProPose[3]+Ours|**84.6**|**52.2**|\n\n[1] W. Shan et al., “Improving robustness and accuracy via relative information encoding in 3D human pose estimation,” ACM, 2021.\n\n[2] K. Gong et al., “Poseaug: A differentiable pose augmentation framework for 3D human pose estimation,” CVPR, 2021.\n\n[3] J. Han et al., “Propose: Probabilistic 3D human pose estimation with instance-level distribution and normalizing flow,” AAAI, 2025."}]

Author rebuttal:

Meta-review: This paper addresses the generalization challenge in 3D Human Pose Estimation by analysing the geometry of its loss landscape. The paper proposes an adaptive scaling mechanism to smooth the loss surface, enabling models to find more diverse and generalisable solutions. Building on this, the paper proposes a simple ensemble strategy that effectively leverage this diversity. Experiments on Human3.6M and MPI-INF-3DHP show consistent performance gains across multiple architectures.

Reviewers initially raised several concerns including the unclear relationship between the proposed method and 3D HPE, the lack of experiments supporting generalisation to the diverse datasets, and limited theoretical justification of the adaptive scaling mechanism and ensemble approach. During the rebuttal, the authors addressed these concerns satisfactorily, and as a result all reviewers become generally positive about this paper. However, it is important that the requested revisions are incorporated into the final version, in particular showing the results using BEDLAM as noted by reviewer 8o6Y.

Based on the overall positive feedback, the AC recommends acceptance.
Decision: Accept (poster)