Title: 3D Human Pose Estimation with Muscles
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D human pose estimation

physics-based computer vision
Abstract: We introduce MusclePose as an end-to-end learnable physics-infused 3D human pose estimator that incorporates muscle-dynamics modeling to infer human dynamics from monocular video. Current physics pose estimators aim to predict physically plausible poses by enforcing the underlying dynamics equations that govern motion. Since this is an underconstrained problem without force-annotated data, methods often estimate kinetics with external physics optimizers that may not be compatible with existing learning frameworks, or are too slow for real-time inference. While more recent methods use a regression-based approach to overcome these issues, the estimated kinetics can be seen as auxiliary predictions, and may not be physically plausible. To this end, we build on existing regression-based approaches, and aim to improve the biofidelity of kinetic inference with a multihypothesis approach --- by inferring joint torques via Lagrange’s equations and via muscle dynamics modeling with muscle torque generators. Furthermore, MusclePose predicts detailed human anthropometrics based on values from biomechanics studies, in contrast to existing physics pose estimators that construct their human models with shape primitives. We show that MusclePose is competitive with existing 3D pose estimators in positional accuracy, while also able to infer plausible human kinetics and muscle signals consistent with values from biomechanics studies, without requiring an external physics engine.


Peer reviews:
comments: SUMMARY:
The paper introduces MusclePose, an end-to-end learnable physics-infused 3D human pose estimator that integrates muscle dynamics modeling to infer human dynamics from monocular videos. Unlike existing physics-based pose estimators that rely on external physics optimizers or produce implausible kinetics, MusclePose employs a multihypothesis approach to compute joint torques via Lagrange’s equations and muscle torque generators (MTGs). It also outputs detailed human anthropometrics based on biomechanics studies, enhancing biofidelity.

STRENGTHS AND WEAKNESSES:
## Strengths

1. Compared with PhysPT, the proposed ground reaction forces and moments (GRFM) have better mathematical modeling.
2. The introduced MTGs (Muscle Torque Generators) can generate more reasonable joint torques while providing the muscle activation levels.

## Weaknesses
### Major
1. PhysPT said that "The discrete contact status also introduces a non-differentiable process in estimating the forces." How does GRFM deal with this issue?

### Minor
1. Does the MTG model all joints uniformly? Is it currently less effective for joints with 3 DOF (e.g., shoulders)? How does the paper ensure that fitting errors in complex joints do not propagate to the global dynamics estimation?
2. Could you clarify the specific location of MTG in Reference [46] (cited in lines 63 and 207)? A more detailed explanation of the MTG implementation would be helpful.
3. Why are Coriolis and centrifugal forces included in the Lagrangian formulation? Is the algorithm's reference frame the global/world coordinate system?

QUESTIONS:
As shown in the Weaknesses section, A reasonable explanation will make me raise the score for this work.

LIMITATIONS:
Yes, limitations are discussed in the supplementary material.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-17

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MusclePose, a 3D human pose estimator that integrates muscle-dynamics modeling and detailed anthropometrics for inferring human dynamics from monocular video. The authors claim that MusclePose achieves competitive positional accuracy while providing more physically plausible kinetic inferences compared to existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n(1) Novelty in Physics-Infused HPE: The core strength of MusclePose lies in its novel integration of muscle-dynamics modeling and detailed anthropometrics directly into an end-to-end learnable physics-infused 3D human pose estimator. This goes beyond typical physics-based pose estimation methods that often rely on external physics optimizers or treat kinetics as auxiliary predictions.\n\n(2) Improved Biofidelity of Kinetic Inference: The paper demonstrates that MusclePose can infer plausible human kinetics and muscle signals consistent with biomechanics studies, without requiring an external physics engine. This is an improvement over prior regression-based approaches, where kinetic predictions may not be physically plausible. The use of a multihypothesis approach, inferring joint torques via Lagrange's equations and muscle dynamics modeling, contributes to this biofidelity.\n\n(3) Detailed Anthropometric Prediction: MusclePose's ability to predict detailed human anthropometrics based on biomechanics studies is an advancement, contrasting with existing methods that use simpler shape primitives. The ablations confirm that this custom anthropometric prediction leads to better kinetics.\n\nWeaknesses:\n\n(1) Limited Scope of External Force Modeling and the Evaluation Setting: The paper assumes only foot-ground contact for simplicity. While this simplifies the problem and the formulation, it limits the applicability of MusclePose to scenarios involving more complex interactions with the environment or multiple external forces, which are acknowledged as underconstrained problems. Moreover, the paper states that it \"trained and evaluated on sequences with feet-ground contact only\" and explicitly removed non-feet-ground contact sequences from both H36M and 3DPWoc datasets for evaluation. This narrower evaluation scope contrasts directly with PhysPT, which claims to \"adhere to the standard protocol and evaluate our method on all actions,\" implying evaluation on sequences beyond just ground contact. This discrepancy in evaluation protocols makes direct comparisons of generalizability and robustness between MusclePose and methods like PhysPT unfair, as MusclePose's performance on more diverse actions (e.g., sitting, climbing, freestyle movements) remains unassessed.\n\n(2) Qualitative Evaluation of Muscle Signals: While the paper states that MusclePose can infer plausible muscle signals, the evaluation of these signals is primarily qualitative, comparing them to trends from biomechanics studies and EMG data. A more rigorous quantitative assessment, if possible, would strengthen the claim.\n\n(3) Limited Evaluation Metrics: While the paper focuses on MJE/PJE for positional accuracy, it's true that PhysPT, a competing method, utilizes additional physical plausibility metrics such as ACCL (acceleration), VEL (velocity), FS (foot sliding), and GP (ground penetration). The omission of these quantitative physical plausibility metrics in the main evaluation for MusclePose is a notable limitation. This limits a direct and comprehensive comparison of the physical plausibility aspects between MusclePose and PhysPT, especially since MusclePose's core contribution is enhanced biofidelity in kinetics. While the paper includes quantitative metrics like residual forces and out-of-range joint torques (Table 2) , and qualitative comparisons of joint torques and muscle signals against biomechanics studies (Figure 3 and 4), the inclusion of standard physical plausibility metrics like ACCL, VEL, FS, and GP would provide a more complete picture of its physical realism from a kinematic perspective.\n\n(4) Under-discussed Computational Efficiency: While the paper claims inference speed on par with purely kinematic pose estimators, a more detailed analysis of the computational cost and real-time performance of MusclePose compared to its baselines would be beneficial.\n\nQUESTIONS:\n(1) The \"kinematics-kinetics trade-off\" is mentioned as a possible reason for slightly worse positional accuracy on 3DPWoc. Could you elaborate on this trade-off in the context of MusclePose? How does the model balance enforcing physical plausibility (kinetics) with maintaining high positional accuracy (kinematics)? Is there a tuning mechanism or a loss weighting that allows for prioritizing one over the other, and if so, how was it determined?\n\n(2) The paper lacks qualitative results showcasing MusclePose's performance on real-world images or videos. Could the authors provide visual examples of predicted 3D poses and body shapes overlaid on input video frames to demonstrate the model's output in a real-world context?\n\n(3) PhysPT, a competing method, employs several physical plausibility metrics (ACCL, VEL, FS, GP) and evaluates on \"all actions\". In contrast, MusclePose's evaluation is limited to sequences with \"feet-ground contact only\". Could the authors justify the omission of a broader evaluation set and these standard physical plausibility metrics, and explain how the presented results still provide a fair comparison with state-of-the-art methods like PhysPT?\n\n\n(4) The \"with \\alpha \" case in the ablations initializes muscle signals based on angular velocity relative to its limit, with an additional offset term regressed. Could you provide more details on why this specific initialization was chosen and what impact it has on the learning process and the resulting muscle signals, especially considering the observed discrepancies in Figure 4 (e.g., for ankle plantarflexion)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper proposed a learning framework for physics-based human pose estimation (PHPE) from monocular videos, taking into consideration of the kinetic constraints. The estimator accounted for individualized anthropometrics, showed improved accuracy compared to purely kinematics-based models, and demonstrated some degree of bio-fidelity.

STRENGTHS AND WEAKNESSES:
The proposed framework incorporated kinetics when estimating joint positions and individualized anthropometrics aiming to improve the pose estimation accuracy.  The estimator didn't need a physics engine nor musculoskeletal simulation and could be trained end-to-end, which makes it a potentially useful tool in computer vision. The model design rationale, architecture and implementations are clearly presented (with more details in supplementary). Evaluations were conducted on several standard 3D human pose estimation benchmarks, showing improved or comparable estimation accuracy, and some degree of bio-fidelity.

The improvements in pose estimation were not that significant. A MTG module was used for muscle activation-net joint torque estimation, which overlooked the detailed muscle dynamics and the redundancy problem was somehow avoided. Mismatches exist for the estimated human body and biomechanical studies. To better validate the bio-fidelity, simultaneous human behavioral and EMG recordings could be used. The motion included are mainly gross movements, further improvement may demonstrate the performance on fine-motor movements.

QUESTIONS:
How accurate or realistic are the estimations of ground reaction forces and external forces, since the contact is simplified?
What's the inference efficiency?

LIMITATIONS:
The authors discussed some limitations in the supplementary.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method to refine the output of existing forward-kinematics pose regressors (SMPL prediction models) by imposing constraints inspired by muscle models such as torque, limb lengths. (more below)\n\nSTRENGTHS AND WEAKNESSES:\n+ I think the proposed novel components are well motivated. It could potentially be used to augment existing inference or evaluation methods.\n\n+ Figure 4 seems to show muscle activations are meaningful.\n\nSome of the muscle-related claims can be misleading, e.g. application to \"whole-body musculoskeletal dynamics of athletes\" (L55, L58) and biofidelity. SMPL joints labels are not consistent with anatomy (e.g. hips; https://files.is.tue.mpg.de/black/talks/SMPL-made-simple-FAQs.pdf). Some of this paper's core designs (e.g. passive and active torque losses) were inspired by and are analogous to muscles, but the interdisciplinary claims do not seem compelling enough. Also, I feel that simpler alternatives could have been explored and compared against.\n\nConsidering the main selling point is biofidelity and physical plausibility, I think more analysis on temporal consistency (e.g. smoothness, jitter, or torque continuity metric) would have been helpful.\n\nCLIFF might be a little dated now. More recent, motion-based approaches such as CoMotion (ICLR 2025) could have been insightful.\n\nComparison with existing energy-minimization approaches in vision (limb length symmetry constraints, foot skating loss) would have been helpful, e.g. hypothetically, results showing those existing techniques should be used with the proposed ones can be directly helpful to other researchers.\n\nQUESTIONS:\nL240 says an example consists of 16 frames, but how much real-world time does this correspond to? AMASS has a very high FPS (60-120+ Hz) depending on the subset. I think it is important to know how much motion is captured.\n\nL269 says, the worse performance on 3DPWoc is within reasonable margin compared to SOTA (the best one in the table is CLIFF). But according to L259, CLIFF was used as input to the proposed model. This input-output relationship could be made clear. It's not necessarily a limitation considering other metrics may have improved (Table 3 in supp).\n\nCould you explain the \"with alpha-hat\" ablation in Table 2 more? I found it hard to understand.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-03"}