Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors propose a scene-aware pose estimation framework based on the concept of embodiment. Different from previous work with multistage optimization, non-causal inference, and complex contact modeling for the scene-aware pose, this method is simple(only one stage) and recovers robust scene-aware pose in the simulated environment. Besides, to simulate good poses, they disentangle the camera pose and use a multi-step projection gradient defined in the global coordinate frame as the movement cue for our embodied agent.
This method only needs 2D observation and can be trained on synthetic datasets for real-world pose estimation. They achieve the state-of-the-art on PROX dataset without using any training data in this dataset.
strengths_and_weaknesses: Strengths:
+ The main contribution--simulating scene-aware humanoid to mimic visual observation and recover 3D poses in the real-world environment is significant. I believe it will be a good baseline for the scene-aware pose estimation.
+ This method achieves promising results on the challenging poses of PROX dataset.
+ The MGP is reasonable on 3D pose recovery and achieves significant improvements shown in ablation studies.
+ This paper is well written and easy to follow.
Weakness:
-The simplified scene representation causes some failure cases in the poses of sitting or laying.
- The contact modeling between motion and scene is a little bit unclear.
questions: 1) It would be better to discuss how the contact information models in this paper. I think it is one of the most important differences between this work and other simulation-based mocap methods. 
2) Does this method need to train new agents for different motion sequences?
limitations: Please follow the weakness.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a method for estimating 3D human pose given a monocular RGB video observing a human moving through a scene tha has been previously reconstructed (i.e. a scene for which a 3D mesh reconstruction is available).  The method is based on physical simulation of the human moving through a geometrically simplified version of the scene.  The simulation is driven by a multi-step projection gradient connecting 2D pose keypoints to the controller that drives the humanoid pose in simulation.

The experiments evaluate the proposed method as well as ablations against a breadth of methods from prior work on two datasets: PROX and H36M.  Performance is evaluated in terms of pose accuracy (mainly joint error metrics where ground truth pose is available) and pose plausibility (mainly distance to scene geometry, interpentretation frequency and distance etc.). The results show that the approach is competitive with prior work that uses image features on the H36M dataset, and mostly outperforms prior work on the PROX dataset.
strengths_and_weaknesses: Strengths
+ The physics-based formulation is novel and offers a complemetary approach to the 3D human pose estimation problem statement relative to most prior work.  The fact that it can provide strong performance is quite impressive given the limited amount of input information that is given compared to other methods
+ The paper is well-written and presents a breadth of ablations that analyze the impact of different components of the method on performance

Weaknesses
- The approach relies on a "semi-automatic" simplification of the 3D scene geometry to address failure modes due to reconstruction noise.  I would have liked to see a more detailed discussion of this issue and also of the actual process for creating these simplified scenes.  This is an important detail as it constitutes a significant modification to the input available to the presented method relative to prior work.  Ideally, there would be an "ablation" that would report results based on the original (un-simplified) scene geometry to concretely measure the impact and value of this semi-automatic step
questions: I would like the authors to respond to the point regarding scene simplification.
limitations: Limitations are described in the last section of the paper, mainly focusing on the simplification of human bodies and scene geometry to rigid bodies.  As far as I can tell, there is no discussion of potential negative societal impacts (despite a statement pointing to the supplement in the checklist).
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper presents a 3d human pose framework based on both simulators and 2d/3d human estimators. It recovers both global 3D human poses and local human poses. The methods can be applied to daily activities such as PROX dataset.
strengths_and_weaknesses: Strengths:
It sounds novel to me as I am not a simulator guy but a learning and 3d person guy. Using these simulators naturally will help generate lots of physics priors and encode the physics into the learning process.
I may not provide enough evidence for the novelty of the simulation part.

Weaknesses:
I did not see the comparison with "Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture", if so, what is the difference, can you work on the dataset they are working on?

I think the performance is somewhat unsatisfying, we can see several metrics in Table 2 is not SOTA.

Several typos: 
line 287:  scene penetration frequency Freq. and distance Pen..
line 204: keytpoints
line 202: rotationrR 
questions: As I understand, the geometry in PROX is soft, is that reasonable to fully treat the sofas as hard/always flat? should we consider the deformation during the interaction of the geometries?
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper aims to perform single-camera pose estimation. It uses 2D key points and then uses a 3d simulated scene that the person is within, to enable an improvement in the pose estimation. The approach is tested in H36M and PROX datasets
strengths_and_weaknesses: Strengths
The work doesn't require complex contact modelling or multi-stages of optimization processes, it is a single inference step.
The work proposes a temporal gradient projection to smooth the estimation over time.
The work is able to perform well on the POX dataset



Weaknesses
The H36M dataset is ill-suited for scene modelling as there are few objects within the foreground and seems to perform poorly and is distracting for the paper
The MPG is heavily based on the work of [3]
There is very limited experimental discussion around the work of the PROX dataset, making it hard for a reader to fully appreciate the performance of the approach
There is a disconnect between the 2D keypoint detection and the 3D/depth "perfect" generation of the scene, 
questions: For the  MPG whatabout if temporal prediction neural networks such as LSTMs or transformers were used instead?
Could a further dataset demonstrate the performance better than H36M
limitations: none stated
review_title: null

Meta-review: metareview: The submission initially received mixed reviews.  After rebuttal, all reviewers felt their concerns reasonably addressed and recommended acceptance (though one didn't update the score).  The AC agrees. The authors are encouraged to revise the paper accordingly. 
confidence: Certain
Decision: Accept