Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a general approach to leveraging differentiable simulator for the downstream tasks of system identification and visuomotor control WITHOUT requiring access to the true underlying states. They do so by predicting a "rendering-invariant state" that results in a much stabler loss landscape and reduces domain gap / mismatch. Experiments over 4 environments indicate the merits of this approach over current state-of-the-art.
main_review: ### Strengths

* **S1** This paper tackles the challenging (inverse) problem of directly recovering object properties (system identification) or control parameters (visuomotor control) from image/video observations. The current best approach to this ($\nabla$Sim) proposes to compose differentiable physics simulation and differentiable rendering to result in a computation graph where ground-truth physical parameters are no longer required. However, this approach has a crucial shortcoming: it works well only when the reference and target trajectories are drawn from the same distribution (i.e., identical physics and rendering engines used across both). This work proposes RISP to bridge that gap (bridging this gap is crucial to enable several downstream, including real-world, applications).

* **S2** The paper is very well-presented. The core ideas are easy to follow, and the rationale for incorporating the rendering gradients into a regularization term is well laid-out. Of the three contributions, I believe this to be the more significant one (as also corroborated by Fig. in section 4.6)

* **S3** This approach (RISP) demonstrates strong performance over current art -- this is over multiple environment settings (rigid, articulated, deformable object identification; control).

In general, this is well-executed work and I would as such recommend acceptance. However, there are a few issues I hope to see discusses/addressed over the rebuttal phase.

### Weaknesses

* **W1** Clarifying the impact of differentiable physics simulation vs rendering: Are the train sets generated using a different differentiable physics engine? (the manuscript mentions they're generated using a different rendering engine -- but, arguably, generating them using a different physics engine as well would create a wider domain gap) This also raises several interesting questions such as: will RISP generalize to scenarios where the underlying dynamics change too? If not, are there other domain randomization / regularization methods that can assist?

* **W2** Baseline choices: Do all baselines use all frames in the reference trajectory to compute the loss. Just as dense pixelwise loss baselines are ablated upon by bringing in other baselines that treat the image as a whole in loss computation; a similar analogy may be drawn at the sequence level (i.e., computing the full pixelwise loss across all frames in a sequence, vs. using pixelwise loss across select frames in a sequence)

* **W3** Disentanglement/Compositionality of the learned RISP: The current training strategies do not seem to explicity focus on disentangling state representations or improving compositionality of the learned representation. (This seems clearly out of scope for the current work, but I'd like to bring this discusison point here to perhaps prompt addition of clarifying statements in the manuscript). Does this have an imact on e.g., where the object may lie within an image, and perhaps impact the extension of RISP to simulations with multiple objects?


### Minor comments

These comments are nitpicks/typos and are easily addressed in a minor revision. The authors needn’t respond to these

* “Articulate-body” -> articulated body
* “Simulates” -> simulate
summary_of_the_review: This is a well-written paper describing an idea of substantial interest to the physical reasoning and the visual reasoning communitties. I only have clarifying concerns with this work and feel the paper will be well-rounded if the limitations of the work are discussed upfront.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper focuses on the problem of estimating dynamic parameters of a physical system from videos under unknown rendering conditions.
It presents a novel idea of using a rendering-invariant state-prediction (RISP) network that predicts the state from a rendered image and can be integrated into a framework with a differentiable simulation and rendering engine for parameter estimation. Training this network is done using domain randomization technique with synthetic data. Additionally, the paper introduces a novel gradient loss that further pushes the network to be invariant to rendering parameters.
main_review: Strengths:

- The paper is very well written and contributions are clear.
- The idea of using a rendering-invariant state-prediction as a pre-step before the gradsim-like framework is novel and sensible.
- The idea of the using the gradient with respect to the rendering parameters as a regularization is novel and interesting. It could additionally be very used in many other fields where training on synthetic data is used with random rendering parameters.
- The Combination of weak/strong/oracle baselines chosen in the experiments are very appropriate for comparison.
- The paper performs a large range of experiments using several environment and tasks to study the effectiveness of the proposed method compared to baselines.
- The ablation study shows the impact of the gradient loss on the state prediction model performance.

Weaknesses:

- It is not clear if the training and testing rendering environments are the same. This means that the parameter distribution in both training (for the RISP) and testing cases come from the same distribution. In reality, often the test parameter distribution comes from a different distribution. It would be interesting to see results when the train and test rendering parameter distributions are different.
- It would be interesting to see how the various methods would perform on real videos.
summary_of_the_review: The paper presents mainly two new ideas, the RISP network for state estimation and the gradient loss for regularization. Both are novel and can be interesting and impactful for the community and push forward the state-of-the-art in using synthetic data for parameter estimation. The quantitative and qualitative experimental results on synthetic data show clear improvements compared to the previous methods.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a new method of predicting physics simulation parameters and rendering configurations from an RGB video. Unlike previous methods that calculate the loss function in image space, this work proposes to calculate in the simulation state space to avoid the issues (e.g. being stuck in a local minimum) when the reference video is very different from the generated video. Specifically, a rendering-invariant state-prediction (RISP) network is pretrained with the generated data from a differentiable renderer under various rendering conditions.  The RIST network is then appended to the output of a differentiable renderer to make the pipeline from simulation and rendering parameters to states predicted from images fully differentiable. Besides a state prediction loss term for training this pipeline, a novel regularization term is used to enforce the RISP network rendering invariant. In addition, a new training strategy to efficiently calculate the gradient for the loss function is proposed. The experiments have shown that the proposed method significantly outperforms the state-of-the-arts and the proposed components contribute to the final results with big improvements. 
main_review: This work addressed a crutial problem. The proposed method is novel and neat. The paper is well-written. I like the main idea and technically novel components proposed, such as the rendering invariant regularization term and efficiently calculating the second-order gradient. The results have demonstrated the importance of these components and the improvements over previous methods. 
For the weakness of this paper, it is not clear how this method works on real datasets. I suggest that the proposed method be evaluated on more datasets, especially real datasets. 
summary_of_the_review: This paper is a solid submission, which solves an important problem and proposes some novel ideas. I expect the authors to evaluate the method on more datasets, especially real datasets.
review_title: null

Meta-review: This paper proposes a method to solve the inverse problem of identifying parameters of a dynamic physical system from image observations. The main idea is to train a rendering-invariant state-prediction (RISP), which estimates the inverse mapping from the pixel to the state domain. The authors introduce a new loss to this end, and an efficient gradient computation of the loss.

The paper received three clear accept recommendations. The reviewers discussed the potential improvement of RISP when combined to disentanglement methods, and also raise several concerns regarding experiments, e.g. rendering conditions during training and testing, or evaluation on real data. The rebuttal did a good job in answering reviewers' concerns, and the reviewers especially appreciated the new results on real videos. Eventually, all reviewers recommended a clear acceptance of the paper.

The AC's own readings confirmed the reviewers' recommendations. The paper is introduces very solid contributions for solving the complex task of physical parameter identification in the unobservable setting. The paper is also clear and well written, and validated with convincing experimental results. Therefore, the AC recommends acceptance.
Decision: Accept (Oral)