RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation
Authors: Pingchuan Ma
Tao Du
Joshua B. Tenenbaum
Wojciech Matusik
Chuang Gan
Venue: ICLR 2022 Oral (2022)
Keywords: differentiable rendering
differentiable simulation
system identification

Abstract: This work considers identifying parameters characterizing a physical system's dynamic motion directly from a video whose rendering configurations are inaccessible. Existing solutions require massive training data or lack generalizability to unknown rendering configurations. We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem. Our core idea is to train a rendering-invariant state-prediction (RISP) network that transforms image differences into state differences independent of rendering configurations, e.g., lighting, shadows, or material reflectance. To train this predictor, we formulate a new loss on rendering variances using gradients from differentiable rendering. Moreover, we present an efficient, second-order method to compute the gradients of this loss, allowing it to be integrated seamlessly into modern deep learning frameworks. We evaluate our method in rigid-body and deformable-body simulation environments using four tasks: state estimation, system identification, imitation learning, and visuomotor control. We further demonstrate the efficacy of our approach on a real-world example: inferring the state and action sequences of a quadrotor from a video of its motion sequences. Compared with existing methods, our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.

Published as a conference paper at ICLR 2022
RISP: RENDERING-INVARIAN STATE PREDICTOR WITH DIFFERENTIABLE SIMULATION AND RENDERING FOR PROSS-DOMAN PARAMETER ESTIMATION
Pingehuan Ma*
Tao Du
Joshua B. Tenenbaum
MIT CSAIL
MIT CSAIL
MIT BCS, CBMM, CSAIL
pcma@osail misiiimiie mit. edu
taodu@csail
jbt @mti..t... edu
Wojciech Matusik
Chuang Gan
MIT CSAIL wojciech&csail edu edu
MIT-IBM Watson Lab
ganchuang@csail .mit. edu
ABSTRACT
This work considers identifying parameters characterizing physical system's dynamic motion directly from a video whose rendering configurations are inacces- sible. Existing solutions require massive training data lack generalizability unknown rendering configurations. We propose a novel approach that marries do- main randomization and differentiable rendering gradients to address this problem. Our core idea is to train a rendering-invariant state- prediction (RISP) network that transforms image differences into state differences independent of rendering con- figurations, c.g., lighting, shadows, or material reflectance. To train this predictor, we formulate a new loss on rendering variances using gradients from differentiable rendering. Moreover, we present an efficient, second-ordes method to compute the gradients of this loss, allowing learning frameworks. We evaluate our method in rigid-body and deformable-boke simulation environments using four tasks: state estimation, system identification, imitation learning, and visuomotor control. We further demonstrate the efficacy our approach on a real-world example: inferring the state and action sequences of quadrotor from video of its motion sequences. Compared with existing methods, our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations
be integrated seamlessly into modern deep
INTRODUCTION
Reconstructing dynamic information about physical system directly from video has received considerable attention the robotics, machine learning, computer vision, and graphics communities. This problem is fundamentally challenging because of its deep coupling among physics, geometry, and perception of a system. Traditional solutions like motion capture systems (Vicon; OptiTrack;; Qualisys) can provide high-quality results but require prohibitively expensive external hardware platforms. More recent development in differentiable simulation and rendering provides an inexpensive and attractive alternative to the motion capture systems and has shown promising proof-of-concep: results (Murthy et al., 2020). However, existing methods in this direction typically assume the videos come from a known renderer. Such an assumption limits their usefulness inferring dynamic information from unknown rendering domain, which is common in real-world applications due to discrepancy between rendering and real- real-world videos. Existing techniques for aligning different rendering domains, e.g., CycleGAN (Zhu al., 2017), may help alleviate this issue, but they typically require access to the target domain with massive data, which is not always available. To our best knowledge, inferring dynamic parameters of physical system directly from videos under unknown rendering conditions remains far from being solved, and our work aims to fill this gap.
"Equal contribution
Videos, code, and data available the project webpage:
..mi .mit.edu.

---

Published as a conference paper at ICLR 2022
Quadrotor
Cube
Hand
Rod
Figure 1: gallery of our four environments (left to right) across three rendering domains (top to bottom). For each environment, train RISP with images under varying lighting, background, and materials generated from a differentiable render (top). Each environment then aims to find proper system and control parameters to simulate and render the physical system (middle) that it matches the dynamic motion of a reference video (bottom) with unknown rendering configurations. We deliberately let three use renderers with vastly different rendering configurations.
We propose a novel approach combining three ideas to address this challenge: domain randomiza- tion, state estimation, and rendering gradients. Domain randomization is classic technique for transferring knowledge between domains by generating massive samples whose variances can cover the discrepancy between domains. We upgrade it with two key innovations: First, notice that image differences are sensitive to changes in rendering configurations, which shadows the rendering- invariant, dynamics-related parameters that genuinely aim to infer. This observation motivates us to propose sonderrnn-invariin state predictor (RISP) that extracts state information of a physical system from videos. Our second innovation renderer. Essentially, requiring the output of RISP to be agnostic to rendering configurations equals enforcing its gradients for rendering parameters be zero. We propose new loss function using rendering gradients and show an efficient method for integrating it into deep learning frameworks.
leverage rendering gradients from differentiable
Putting all these ideas together, we develop a powerful pipeline that effectively infers parameters of physical system directly from video input under random rendering configurations. We demonstrate the efficacy of our approach on variety of challenging tasks evaluated in four environments (Sec. and Fig. 1) as well as in real-world application (Fig. 4). The experimental results show that our approach outperforms the state-of-the-art techniques by large margin in most of these tasks due to the inclusion of rendering gradients in the training process.
In summary, our work makes the following contributions:
We investigate and identify the bottleneck in inferring state, system, and control parameters of physical systems from videos under various rendering configurations (Sec. 3.1);
We propose novel solution combining domain randomization, state estimation, and rendering gradients to achieve generalizability across rendering domains (Sec. 3.2);
We demonstrate the efficacy of our approach several challenging tasks in both simulation and real-world environments (Sec. 4).
RELATED WORK
Differentiable simulation Differentiable simulation equips traditional simulation with gradient information. Such additional gradient information connects simulation tasks with numerical opti- mization techniques. Previous works have demonstrated the power of gradients from differentiable simulators in rigid-body dynamics (Geilinger et al., 2020; Degrave al., 2019; de Avila Belbute-Peres et al., 2018; Xu et al., 2021; Hong et al., 2021; Qiao et al., 2021a), deformable-bodd dynamics (Du al., 2021 b; Huang al., 2020; Du et al., 2021a; Hu et al., 2019h Gan et al., 2021; Hahn et al.,
2

---

Published as a conference paper at ICLR 2022
Source domain (training)
Random rendering parameters
Differentiable renderer (training)
Training data
Random states
Source domain
System parameters
Rendering parameters
Differentiable simulation
Differentiable renderer
Predicted states
Actions
States
Images
RISP
Loss
System
Rendering parameters
parameters
Predicted
Simulation
Renderer
states
Actions
States
Images
Target domain
We first train RISP using images rendered with random states and rendering parameters (top). We then append RISP to the output of differentiable renderer, leading to fully differentiable pipeline from system and control parameters states predicted from images (middle). Given reference images generated from unknown parameters (dashed gray boxes) in the target domain (bottom), we feed them to RISP and minimize the discrepancies between predicted states (rightmost green-gray box) to reconstruct the underlying system parameters, states, actions.
Figure An overview of our method (Sec. 3).
2019; Ma et al., 2021; Qiao al., 2021b), fluids (Du et al., 2020; McNamara al., 2004; Hu et al., 2019a), 9a), and co-dimensional objects (Qiao et al., 2020; Liang et al., 2019). We make heavy use of differentiable simulators in this work but our contribution is orthogonal them: we treat differentiable simulation black box, and our proposed approach is agnostic to the choice of simulators.
Differentiable rendering Differentiable rendering offers gradients for rendering inputs, e.g., light- ing, materials, or shapes (Ramamoorthi et al., 2007; Li et al., 2015; Jarosz et al., 2012). The state-of-the-arr differentiable renderers (Li et al., 2018; Nimier-David al., 2019) powerful in handling gradients even with global illumination or occlusion. Our work leverages these renderers but with different focus on using their gradients as a physics prior in a learning pipeline.
Domain randomization The intuition behind domain randomization (Tobin et al., 2017; Peng al., 2018; Andrychowicz al., 2020; Sadeghi & Levine, 2017; Tan al., 2018) is that model can hopefully cross the domain discrepancy by seeing a large amount of random data in the source domain. This often leads to robust but conservative performance in the target domain. The generalizability of domain randomization comes from a more robus! model that attempts to absorb domain discrepancies by behaving conservatively, while the generalizability of our method comes from a more accurate model that aims to match first-order gradient information.
METHOD
Given video showing the dynamic motion of physical system, our goal is to infer the unknown state, system, or control parameters directly from the video with partial knowledge about the physics model and rendering conditions. Specifically, physical system (c.g., Newton's 's law for rigid-body systems) and the camera position, but the exact system, control, or rendering parameters not exposed.
assume know the governing equations of the
To solve this problem, we propose a pipeline (Fig. 2) that consists of two components: 1) a differen- tiable simulation and rendering engine; 2) the RISP network. First, render the state of physical system and outputs images under varying rendering configuration. Next, the RISP network learns reconstruct the state information from these generated images. Putting
engine simulate and

---

Published as a conference paper at ICLR 2022
these two components together, we have a pipeline that can faithfully recover dynamic information of a physical system from a new video with unseen rendering configurations.
3.1 DIFFERENTIABLE SIMULATION AND RENDERING ENGINE
Given a physical system with known dynamic model M, we first use differentiable simulator to simulate its states based on action inputs at each time step after time discretization:
5(+1 -Ma (s,, a.), Vi 0, .N 1
(1)
where N is the number of time steps in a rollout of physies simulation, and s,, 8:41 and represent the state and action vectors the corresponding time steps, respectively. The & vector encodes the system parameters in the model, e.g., mass, inertia, and elasticity. Next, we apply a differentiable renderer to generate an image I; for each state s;:
I (s.). Vi = 0, 0. N.
Here, vb is a vector encoding rendering parameters whose gradients are available in the renderer R. Examples of af include light intensity, material reflectence, background color. By abuse of notation, we re- re-write the workflow of our simulation and rendering engine to a compact form:
{I,} = Ry RM (a.))
{s.}
In other words, given initial state So and sequence of actions {a.). we generate sequence of states {s:} from simulation and renders the corresponding image sequence {I:]. The task of recovering unknown information from reference video (Iter) can be formulated as follows:
min ao.
(I.) R. (a,1)],
(5)
where C is a loss function penalizing the difference between the generated images and their references. Assuming that the simulator M and the renderer R. are differentiable with respect their inputs, we can run gradient-bart based optimization algorithms to solve Eqn. (4). This is essentially the idea proposed in Vsim, the of-te-of-thh-ar method for identifying parameters directly from video inputs (Murthy et al., 2020). Specifically, V im defines L as norm on pixelwise differences.
One major limitation Eqn. (4) that it expects reasonably similar initial images (I. and references (Ięef) to successfully solve the optimization problem. Indeed, since the optimization problem is highly nonlinear due to its coupling between simulation and rendering. local optimization techniques like gradient-descent can be trapped into local minima easily if {L; and (IęTF) are not close enough, While VSim has reported promising results when {I,} and {Iref} are rendered with moderately different bb we found in our experiments that directly optimizing defined on the image space rarely works when the two rendering domains are vastly different (Fig. 1). Therefore, we believe it requires a fundamentally different solution, motivating us to propose RISP in our method.
3.2 THE RISP NETWORK
The difficulty of generalizing Eqn. (4) across different rendering domains is partially explained by the fact that the loss L is defined on the differences in the image space, which is sensitive to changes in rendering configurations. To address this issue, we notice from many differentiable simulation papers that a loss function in the state space is fairly robust to random initialization (Du et al., 2020; state-like space. More coneretely, we introduce the
Liang et al., 2019), inspiring us to redefine RISP network N' that takes as input an image I and outputs a state prediction $ = (I). We then redefine the optimization problem in Eqn. (4) as follows (Fig. 2):
min (a.
L(No H),
(6)
s.t. (I.) RRIMa (a.
Note that the network No. parametrized by 0. is pre-trained and fixed in this optimization problem. Essentially, Eqn. (6) maps the two image sequences to the predicted state space, after which the
4

---

Published as a conference paper at ICLR 2022
standard creceent-descent optimization follows. A invariant state vector regardless of the choice of rendering parameters v,, allowing Eqn. (6) to match the information behind two image sequences {Ii) and (I even when they are generated from different renderers Ryp. Below, we present two ideas to train the network N:
network /V' can be interpreted as an
"inverse renderer" R that recovers the rendering-ivvariant
The first idea: domain randomization Our first idea is to massively sample state state-rend pairs (sj: 4by) and render the corresponding image I; Ry, (s;), giving us training set D {(8), 1/ }- We then train N to minimize the prediction error:
Ceroor (O,D)=
Σ (8) (
sS (L)
(8)
Cene
The intuition is straightforward: No learns to generalize over rendering configurations because it sees images generated with various rendering parameters sb. This is exactly the domain randomization idea (Tobin et al., 2017), which we borrow to solve our problem across different rendering domains.
The second idea: rendering gradients One major bottleneck in domain randomization its needs for massive training data that spans the whole distribution of rendering parameters 1p. Noting that perfectly rendering-invarian A must satisfy the following condition:
NNo (s) O,
Vs, 1,
Eup
we consider adding a regularizer to the training loss:
aNo (Rey, (s,))
triin (0, D)
Cenor
Σ L)E
(10)
Orby
where || ||e indicates the Frobenius norm and y a regularization weight. The intuition is that by suppressing this Jacobian to zero, we encourage the network N to flatten out its landscape along the dimension of rendering parameters vb, and invariance across rendering configurations follows. To implement this loss, we apply the chain rule:
@N (R (s,)) (No @No(I,) aI, aI, T5.y
(11)
Orp;
where the first term (L,) 01, term Jey be obtained from the state-of-the-art differentiable renderer (Nimier-David al., 2019). We can now see more clearly the intuition behind RISP: requires the network's sensitivity about input images to be orthogonal to the direction that rendering parameters can influence the image, leading to a rendering-invarian
is available in any modern deep learning frameworks and the second
prediction.
We stress that the design of this new loss in Eqn. (10) is non-trivial. In fact, both Cenor and L have their unique purposes and must be combined: Camer encourages ' to fit its output to individually different states, and ( attempts to smooth out its output along the &p dimension. Specifically, d cannot be optimized as a standalone loss because it leads to a trivial solution of o always predicting constant states. Putting Cemor and L together forces them to strike a balance between predicting accurate states and ignoring noises from rendering conditions, leading to a network. / that truly learns the "inverse renderer" R-
remains to show how to compute the gradient of the regularizer _ with respect to the network parameters 0, which is required by gradient-basee optimizers minimize this new loss. As the loss definition now includes first-order derivatives, computing its gradients involves second-order partial derivatives, which can be time-consuming if implemented carelessly with multiple loops. Our last contribution is to provide an efficient method for computing this gradient, which can be fully implemented with existing frameworks (PyTorch and mi mitsuba-2 in our experiments):
Theorem Assuming forward mode differentiation is available in the renderer R and reverse mode differentiation is available in the network N. we can compute a stochastic gradient time per image using pre-computed data occupying OEE, | space. 80
ac in O(s)] 01)
5

---

Published as a conference paper at ICLR 2022
In partieular, stress that computing the gradients of _ does not require second-order gradients in the renderer R, which would exceed the capability of all existing differentiable renderers we are aware of. We leave the proof of this theorem in supplemental material.
Further speedup Theorem states that it takes time linear to the network size and state dimension compute the gradients of _ The O |s 01) time cost is affordable for very small rigid-body systems (e.g., |s] < 10) but not scalable for larger systems. Therefore, we use a slightly different regularizer our implementation:
2 Onp;
Ctraii (0, D) L
Σ (
11.
(12)
In other words, we instead encourage the state prediction error to be rendering-invariant. It can be seen from the proof in Theorem that this new regularizer requires only ((0) time to compute its gradients, and we have found empirically that the performance of this new regularizer comparable to Eqn. (10) but is much faster. We leave a theoretical analysis of the two regularizers to future work.
4 EXPERIMENTS
In this section, we conduct various experiments to study the following questions:
Q1: Is pixelwise loss on videos across rendering domains sufficient for parameter prediction?
Q2: If pixelwise loss is not good enough, are there other competitive alternatives to the state- prediction loss our approach?
Q3: Is the regularizer
rendering gradients in our loss necessary? Q4: How does our approach compare with directly optimizing state discrepancies?
Q5: Is the proposed method applicable to real-world scenarios?
We address the first four questions using the simulation environment described Sec. 4. and answer the last question using real-world application at the end of this section. More details about the experiments, including ablation study, can be found in Appendix.
4.1 EXPERIMENTAL SETUP
Environments We implement four environments (Fig. I): rigid-body environment without contact (quadrotor), a rigid-body environment with contact (cube), an articulated body (hand), and deformable-body environment (rod). Each environment contains a differentiable simulator (Xu et al., 2021; Du et al., 2b) and a differentiable renderer (Li et al., 2018). We deliberately generated the training set in Sec. using a different renderer (Nimier-David et al., 2019) and used different distributions when sampling rendering configurations in the training set and the environments.
Tasks We consider four types of tasks defined on the physical systems all environments: state estimation, system identification, imitation learning, and visuomotor control. The state estimation task (Sec. 4.2) require a model to predict the state of the physical system from a given image and serves as a prerequisite for the other downstream tasks. The system identification (Sec. 4.3) and imitation learning (Sec. 4.4) tasks aim to recover the system parameters and control signals of a physical system from the video, respectively. Finally, in the visuomotor control task (Appendix), replace the video with a target image showing the desired state of the physical system and aim to discover the proper control signals that steer the system the desired state. In all tasks, we use photorealistic renderer (Pharr et al., 2016) to generate the target video or image.
Baselines We consider two strong baselines: The pixelwise-loss baseline used by Vsim (Murthy et al., 2020), which is the state-of-thh-ar method for identifying system parameters directly from video inputs. We implement Vsim by removing RISP from our method and backpropagating the pixelwise loss images through differentiable rendering and simulation. We run this baseline to analyze the limit of pixelwise loss downstream tasks (Q1). The second baseline is preceptual- loss (Johnson et al., 2016), which replaces the pixelwise loss in VSim with loss functions based on high-level features extracted by a pre-trained CNN. By comparing this baseline with our method, we can justify why we choose let RISP predict states instead of other perceputal features (Q2).

---

Published as a conference paper at ICLR 2022
We also include two weak baselines used by VSim: The average baseline is a deterministic method that always returns the average quantity observed from the training set, and the random baseline returns a guess randomly drawn from the data distribution used to generate the training set. We use these two weak baselines to avoid designing environments and tasks that are too trivial to solve.
Our methods We consider two versions of our methods Sec. 3: ours-no-grad implements the domain randomization idea without using the proposed regularizers, and ours is the full approach that includes the regularizer using rendering gradients. By comparing between them, we aim to better understand the value of the rendering gradients in our proposed method (Q3).
Oracle Throughout our experiments, we also consider an oracle method that directly minimizes the state differences obtained from simulation outputs without further rendering. In particular, this oracle sees the ground-truth states for each image in the target video or image, which is inaccessible to all baselines and our methods. We consider this approach to be an oracle because a subset of our approach that involve differentiable physics only, but it needs perfect state- catee-rediction network. This oracle can give us an upper bound for the performance of our method (Q4).
Training We build our RISP network and baselines upon a modified version of ResNet-18 (He et al., 2016) and train them with the Adam optimizer (Kingma & Ba, 2014). We report more details of our network architecture, training strategies, and hyperparameters in Appendix.
4.2 STATE ESTIMATION RESULTS
quadrotor
cube
hand
rod
average
0.5994 0.0000
0.5920 + 0.0000
0.2605 + 0.0000
0.9792 + 0.0000
random
0.9661 0.7548
0.9655 + 0.8119
0.8323 + 0.5705
1.2730 + 0.7197
ours-no-grad
0.3114 + 0.3191
0.2805 + 0.3199
0.1155 + 0.0505
0.0201 + 0.0087
ours
0.1505 + 0.1163
0.1642 + 0.1887
0.0974 + 0.0255
0.0194 = 0.0048
Table 1: State estimation results (Sec. 4.2). Each entry in the table reports the mean and standard deviation of the state estimation error computed from 800 images under 4 rendering configurations.
In this task, we predict the states of the physical system from randomly generated images and report the mean and standard deviation of the state prediction errors in Table 1. Note that we exclude the perceptual-loss and pixelwise-loss baselines as they do not require a state prediction step. Overall, we find that the state estimation results from our methods are significantly better than all baselines across the board. The two weak baselines perform poorly, confirming that this state-estimation task cannot be solved trivially. We highlight that our method with the andering-gradiens loss predicts the most stable and accurate state of the physical system across the board, which strongly demonstrates that RISP learns make predictions independent of various rendering configurations.
4.3 SYSTEM IDENTIFICATION RESULTS
Our system identification task aims predict the system parameters of a physical system, e.g., mass, density, stiffness, or elasticity, by watching a reference video with known action sequences. For each environment, we manually design sequence of actions and render a reference video of its dynamic motion. Next, we randomly pick an initial guess of the system parameters and run gradient-based optimization using all baselines, our methods, and the oracle. We repeat this experiment times with randomly generated rendering conditions and initial guesses and report the mean and standard deviation of each system parameter in Table 2. The near-perfect performance of the oracle suggests that these system identification tasks are feasible to solve long as reliable state estimation is available. Both of our methods outperform almost all baselines by a large margin, sometimes even by orders of magnitude. This is as expected, since the previous task already suggests that our methods can predict states from a target video much more accurate than baselines, which is a crucial prerequisite for solving system identification. The only exception is in the cube environment, where the pixelwise loss performs surprisingly well. We hypothesize it may be due to its relatively simple geometry and high contrast from the background (Fig. 1).
7

---

Published as a conference paper at ICLR 2022
quadrotor mass
cube stiffness
hand joint stiffness
Young's modulus
random
9.22e-2-2..3e-2
2.31e-119.57e-2
5.70e-1
1.30e6.1
pixelwise-loss
7.22e=2::522e-22
2.24e-311.74c--
1.4e-111.62e-2
8.50e5
perceptual-loss
6.45e-2-:5.23e-2
1.16e-115.83e-2
1.10--111.88e-
8.32e5.11.455
ours-no- grad ours
6.07c-2=4.34e-2
1.16e-1:6.20-2
4.85e-2+2.166-22
8.78c4+1..225
1.18e-2-1.93e-2
6.76e-347.23--3
3.96e-2:22.3ee2
9.31e1::.22el
oracle
2.36e-5-244c-5
1.15c-348.60e-4
3.92e-3114.40-44
4.36c0::3.57c0
Table 2: System identification results (Sec. 4.3). Each entry reports the mean and standard deviation of the parameter estimation computed from 4 random initial guesses and rendering conditions.
Random
Ours
Reference
Figure 3: Imitation learning in the hand environment (Sec. 4.4). Given a reference video (bottom row, shown as five intermediate frames), the goal is to reconstruct a sequence of actions that resembles its motion. We show the motions generated using randomly chosen initial guess of the actions (top row) and optimized actions using our method with rendering gradients (middle row).
4.4 IMITATION LEARNING RESULTS
Our imitation learning tasks consider the problem of emulating the dynamic motion of reference video. The experiment setup is similar to the system identification task except that we swap the known and unknown variables the environment: The system parameters infer the unknown sequence of actions from the reference video. Note that the cube enviromment is excluded because it has control signals. As before, we repeat the experiment in all environments times with randomly generated rendering configurations and initial guesses of the actions. We report the results in Table and find that our method with rendering gradients (ours in the table) achieves much lower errors, indicating that resemble the motions in the video much more accurately. The errors from pixelwise-loss have smaller variations across rendering domains are larger than ours, indicating that it struggles to solve this task under all four rendering configurations. In addition, the oracle finds a sequence of actions leading to more similar motions than our method, but it requires full and accurate knowledge of the state information which is rarely accessible from video. We visualize our results in the hand environment in Fig. 3.
now known, and the goal is to
4.5 A REAL-WORLD APPLICATION
Finally, we evaluate the efficacy of our approach on real-worle application: Given video of flying quadrotor, we aim to build its digital twin that replicates the video motion by inferring reasonable sequence of actions. This real-to-sim transfer problem challenging due to few reasons: First, the real-world video contains complex textures, materials, and lighting conditions unknown to us and unseen by our differentiable renderer. Second, the quadrotor's real-world dynamics is polluted by environmental noises and intricate aerodynamic effects, which ignored in our differenniable simulation environment. Despite these challenges, our method achieved qualitatively good result with standard differentiable rigid-body simulator and renderer, showing its generalizability across
8

---

Published as a conference paper at ICLR 2022
quadrotor
hand
average
28.40 0.00
7.62 + 0.00
30.20
+ 0.00
random
1120 113
10.27 0.91
29.89 0.61
pixelwise-loss
12.65 1 0.13
6.71 1 0.07
29.57 1 0.85
perceptual-loss
N/A
5.10 1.80
14.61 1 5.13
ours-no res-no-gun grad
25.07 = 5.76
7.12 0.71
0.83 + 0.35
ours
2.63 = 1.86
1.52 0.18
1.05 0.27
oracle
0.79 + 0.44
0.02 + 0.02
0.28 0.16
Table 3: Imitation learning results (Sec. 4.4). Each entry reports the mean and standard deviation of the state discrepancy computed from 4 randomly generated initial guesses and rendering conditions. N/A indicates failure of convergence after optimization.
moderate discrepancies in dynamic models and rendering configurations (Fig. 4). Our approach outperforms all baselines by large margin in this task, which we detail in Appendix.
0.0s
5.0s
10.0s
14.4s
Input Video
Ours
Figure 4: Imitation learning in the real-world experiment (Sec. 4.5). Given reference video (top row), the goal is to reconstruct a sequence of actions sent to a virtual quadrotor that resembles its motion. We illustrate the motions reconstructed using method (bottom row).
5 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK
We proposed a framework that integrates rendering-invariant state-prediction into differentiable simulation and rendering for cross-domain parameter estimation. The experiments in simulation and in real world have shown that our method is more robust than pixel-wise or perceptual losses on unseen rendering configurations. The additional ablated study further confirms the efficiency and efficacy comes from using the rendering gradients in our RISP network.
Despite its promising results, RISP still has few limitations. Firstly, require a differentiable simulator that can capture the dynamic model of the object, which may not be available for real- world scenes with intricate dynamics, e.g.. fluids. second limitation is that our method requires knowledge of camera parameters, which is not always accessible in real-world scenario. This limitation can be resolved by incorporating in RISP the gradients for camera parameters available in modern differentiable renderers. Lastly, we assume moderately accurate object geometry is available, which can potentially be relaxed by combining NeRF (Mildenhall et al., 2020) with our approach to infer the geometry.
ACKNOWLEDGMENTS
We thank Sai Praveen Bangaru for discussions differentiable rendering. This work was supported by MIT-IBM Watson AI Lab and its member company Nexplore, ONR MURI, DARPA Machine Common Sense program, ONR
and Mitsubishi Electric.

---

Published as a conference paper at ICLR 2022
REFERENCES
OpenAl:
Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. Learning dexterous in- in-hand manipulation. The International Journal of Robotics Research, 39(1):3-20. 2020. 3
Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and J Zico Kolter. End- to-end differentiable physics for learning and control. Advances in Neural Information Processing Systems, 31, 2018. 2
Jonas Degrave, Michiel Hermans, Joni Dambre, et al. A differentiable physics engine for deep learning in robotics. Frontiers in neurorobotics, pp. 6, 2019. 2
Tao Du, Kui Wu, Andrew Spielberg, Wojeiech Matusik, Bo Zhu, and Eftychios Sifakis. Functional optimization of fluidic devices with differentiable stokes flow. ACM Trans. Graph., 39(6), nov 2020. ISSN 0730-0301.
doi: (0.11445//1146534477799 URL https://doi.. org/000.
1145/3414 385 3417795. 3,4
Tao Du, Josie Hughes, Sebastien Wah, Wojeiech Matusik, and Daniela Rus. Underwater soft robot modeling and control with differentiable simulation. IEEE Robolics and Aulomation Letters, 6(3); 4994-5001, 2021a.
doi: 10. R..1109/RA..221..00 33070305.
Tao Du, Kui Wu, Pingchuan Ma. Sebastien Wah, Andrew Spielberg. Daniela Rus, and Wojeiech Matusik. DiffPD: Differentiable projective dynamics. ACM Trans. Graph., 41(2), nov 2021b. ISSN 0730-0301. doi: 10.1145/3490168 URL https: http::://do.... http:///dd.o.o/o...oooo// 11.1100.1555 145/34 901 68. 2, 6, 14
Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, et ThreeDWorld: platform for interactive multi-modal physical simulation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski. and Stelian Coros. ADD; Analytically differentiable dynamics for multi-body systems with frictional contact. doi: 00.1145/3414685.447770 URL https: wrr.orggg//oo 10. 114 1111113411140 33/334446668 685 3417766. 2
ACM Trans. Graph., 39(6), nov 2020. ISSN 0730-0301.
David Hahn, Pol Banzet, James M. Bern, and Stelian Coros. Real2Sim: Visco-elastic parameter
estimation from dynamic motion. ACM Trans. Graph. 38(6), nov 2019. ISSN 0730-0301. doi: 10.145//35089.3356644 URL https: doi. org/ /00./11.000 1 1145/ 33550335558999 .00.088..3355555 3356548. 2
Kaiming Hc. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016. 7, 14
Yining Hong, Li Yi, Joshua B Tenenbaum, Antonio Torralba, and Chuang Gan. PTR: A benchmark for part-based conceptual, relational, and physical reasoning. In Advances In Neural Information Processing Systems, 2021.
Yuanming Hu, Luke Anderson. Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Fredo Durand. DiffTaichi: Differentiable programming for physical simulation. In International Conference on Learning Representations, 2019a.
Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu, Daniela Rus, and Wojeiech Matusik. ChainQueen: A real-time differentiable physical simulator for soft robotics. In 2019 International Conference Robotics and Automation (ICRA), 2
pp. 6265-6271, 2019b. doi: 10. (O9/CCAA.0099.779999
Zhiao Huang, Yuanming Hu, Tao Du. Siyuan Zhou, Hao Su, Joshua B Tenenbaum, and Chuang Gan. PlasticineLab: A soft-body manipulation benchmark with differentiable physics. In International Conference Learning Representations, 2020. 2
10

---

Published as a conference paper at ICLR 2022
Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, and Konstantinos Bousmalis. Sim-to-real via sim-to-sim: Data- efficient robotic grasping via wellomizze--o-cannnii adaptation networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12619-12629, 2019. doi: 10.1 10.100//VVPR.201.01199 14
Wojeiech Jarosz, Volker Schönefeld, Leif Kobbelt, and Henrik Wann Jensen. Theory, analysis and applications of 2d global illumination. 31(5), sep 2012. ISSN 0730-0301. doi: 10.1145/2231816. 2231823. URL https: : /woi. org/ 0 1145/22311111 81 6, 2231 823. 3
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694-711. Springer, 2016. 6
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint
arXiv:1412.69880 69800 2014. 7, 14
Tzu-Mao Li, Jaakko Lehtinen, Ravi Ramamoorthi, Wenzel Jakob, and Frédo Durand. Anisotropic gaussian mutations for metropolis light transport through hessian-hamiltonian dynamics. ACM doi: 10.1145/2816995588888888 URL https: //doi. wwt.///////// 10. 1ee.11111/11990 (11.8881179..11888888 2818084.
Trans. Graph., 34(6), oct 2015. ISSN 0730-0301,
Tzu-Mao Li, Mïika Aittala, Frédo Durand, and Jaakko Lehtinen. Differentiable monte carlo ray
tracing through edge sampling. ACM Trans. Graph., 37(6). dec 2018. ISSN 0730-0301. doi: 10.1145/3222127.3275111 URL https: ottp.o........./////32222 21111///2222..2277222222
3, 6
Junbang Liang, Ming Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. Advances in Neural Information Processing Systems, 32, 2019. 3, 4
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts, arXiv preprint arXiv:1608. 160.0038000 2016. 14
Pingchuan Ma. Tao Du, John Z. Zhang. Kui Wu, Andrew Spielberg, Robert K. Katzschmann, and Wojeiech Matusik. DiffAqua: A differentiable computational design pipeline for soft underwater swimmers with shape interpolation. ACM Trans. Graph., 40(4), jul 2021. ISSN 0730-0301. doi: 00.11445/345006634559333 URL https: (w..do..oogroo........ 10. 1 3450626...6622 3459832.
Antoine McNamara, Adrien Treuille, Zoran Popović, and Jos Stam. Fluid control using the adjoint method. In ACM SIGGRAPH 2004 Papers, SIGGRAPH '04, pp. 449-456, New York, NY, USA, 2004. Association for Computing Machinery. ISBN 9781450378239 doi: 10. 186562.
1015744. URL httptttts://dd org/10 10. 1114/11111 865 62 1015744.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. European conference on computer vision, pp. 405-421 Springer, 2020. 9
J Krishna Murthy, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Párent-Lévesque, Kevin Xie, Kenny Erleben, et al. gradSim: Differ- entiable simulation for system identification and visuomotor control. In International Conference on Learning Representations, 2020. 1, 4, 17
Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. Mitsuba 2: A retargetable forward and inverse renderer. ACM Trans. Graph., 38(6), nov 2019. ISSN 0730-0301. doi: 10. 00,1145/335589.33554999 URL https: /tt.://///oll..og111111515////// 5,6
3356498. 3,
OptiTrack. OptiTrack motion capture systems. https:: 10-05.
Accessed: 2021-
Xue Bin Peng, Marcin Andrychowicz, Wojeiech Zaremba, and Picter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In 20/8 IEEE International Conference on Roboties and Automation (ICRA), pp. 3803-3810, 2018. doi: 10. 2009//CAA.201..846660
Matt Pharr, Wenzel Jakob, and Greg Humphreys. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, 2016. 6
11

---

Published as a conference paper at ICLR 2022
Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Scalable differentiable physics for learning and control. In International Conference on Machine Learning, pp. 7847-7856. PMLR, 2020. 3
Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming C Lin. Efficient differentiable simulation of articulated bodies. In International Conference on Machine Learning, pp. 8661-8671. PMLR, 2021a.
Yiling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Differentiable simulation of soft multi-body systems, Advances in Neural Information Processing Systems, 34, 2021b. 3
Qualisys. Qualisys motion capture systems. https::/www.qualt com.aalssysccoo.........mm. Accessed: 2021- 10-05.
Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint
arXiv:1710.05941, 2017. 14
Ravi Ramamoorthi, Dhruv Mahajan, and Peter Belhumeur. A first-order analysis of lighting, shading, and shadows. 26(1):2-es, jan 2007. ISSN 0730-0301, doi: 0111145/118762.111
URL
https: /doii orroor/g/1/ /10.//1..00
Fereshteh Sadeghi and Sergey Levine. CAD2RL: Real single-image flight without a single real image. In Robotics: Science and Systems, 2017.
Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: Learning agile locomotion for quadruped robots. In Robotics: Science and Systems, 2018.
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 JEEE/RSJ International Conference on Intelligett Robots and Systems (IROS), pp. 23-30, 2017. 3,5
doi: 2..109///OS.201..20000
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv: 2016. 14
Vicon. VICON: Awwrd-winning motion capture systems. https: /www. viw.n. http:s//wwww.iccoccoo//// cessed: 2021-10-05.
Ac-
Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, and Pulkit Agrawal. An end-to-end differentiable framework for contact-aware robot design. In Robotics: Science and Systems, 2021. 2, 6, 14
Jun-Yan Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242-2251, 2017. doi: 10. 10.1109//CCV.2017.2.444
12

---

Published as a conference paper at ICLR 2022
A PROOF OF THE THEOREM
For brevity, we remove the summation in _ and drop the index focus deriving the gradients of the Frobenius term with respect to the network parameters 0. Let G be the Jacobian matrix inside the Frobenius norm, and let and j be its row and column indices. We now derive the gradient of fee with respect to the k-th parameter in f as follows:
7
2G aG.
Σ
(13)
0
0
a aNo(T) a
Σ 2G ij 00 k
(14)
0I
Jupy"
(
OI
Σ 2G
(15)
0100x
0rby
8j
(N)(
01
Σ
2G ij O
(16)
0000
N
(N)(
a
Σ
2
(17)
0100:
OI
Orp 0eb
The derivation suggests that we can loop over the state dimension (indexed by i) and run backpropa- gation obtain 0L only and remains constant throughout the whole optimization process, so we can pre-compute them in the training set, occupying an extra space of O IT) Typically, |tby is small number (less than 10 in most of our experiments). data set. This is also why we prefer a differentiable renderer that offers forward mode differentiation.
00 assuming that all ØI Sup,
@I depends on the training data are available. In fact, Bib,
affordable to store it with the images generated in the
Assuming all aI are ready to use, we can implement Eqn. (17) by two backpropagations for Bhyy each state dimension i. First, backpropagating through N to obtain @Ng(T), Ø1 Next, we use J4, AL to
assemble the adjoint vector (the second summation in Eqn. (17)). Finally, we use the adjoint vector to backpropagate again through the second-order partial derivatives 2000 need for looping over k in the second backpropagation. The total time cost therefore O[|s (0))
No Note that there is no
B IMPLEMENTATION DETAILS
B.1 ENVIRONMENTS
B.1.1 QUADCOPTOR
State space We define the physical system of quadcoptor as a rigid body system without contact. We explicitly mark a task as "failed" when the center of the quadcoptor has an Euclidean distance to the origin (0, 0, 0) of over 1000 at anytime. We define the state of the quadcoptor as its world position (x, y, 2) and rotation angles (yaw, pitch, roll). To alleviate the negative impact from the discontinuity of rotation angles, we design the output of the network as:
(x, y, 2, sin(yaw), sin(pitch), sin(roll), cos cos(roll)),
(18)
and later restore the rotation angles by atan2.
Action space The control signals are 4-d representing the torques applied on propellers. The magnitude of control signals is clamped to |-500,5000
Parameter space In system identification experiment, we uniformly sample the mass of the quadcoptor within (2.8, 3.2). The ground-truth value of the mass is 3.
13

---

Published as a conference paper at ICLR 2022
B.1.2 CUBE
State space We define the physical system of cube as a rigid body system with contact. Similar to quadcoptor environment, we design the output of the network 9-d vector following 18.
Parameter space In cube environment, adopt soft contact model paramterized by the spring stiffness k In system identification experiment, we train A = log10 kn for better numerical stability and performance. We uniformly sample Å within [5.5, 6.5). The ground-truth value of A is 6.
B.1.3 HAND
State space We define the physical system of hand as an articulated body system powered by (42). We fix the palm the world frame and model the articulations revolve joints. There 13 joints in this environment, and each joints is clamped within [- -. J. We define the state space of hand as 13-d vector representing the angles of each joint 131...13.
Action space The hand is controlled by applying torque on each joint. We paramterize the control signal of each finger at time by:
4 = m, sin 2T min 1.0, max (0.0. (t (()-6,)) 5))))
where 0 S my S and 0 S 6, S 40 denote the action magnitude and action bias of finger j. and T = 40 is the period.
For each finger, we let the joint stiffiness of all joints on the finger available for training, making the parameter space a 5-d space. We uniformly sample the joint stiffness within [0.0, 1.0). The ground-trum values of the joint stiffness are all 1.0.
Parameter space
B.1.4 ROD
State space We define the physical system of rod soft-body dynamics system powered by (6). We fix both ends of the rod and apply a gravity force on it. We define 10-d state space by evenly selecting sensors along the central spine and track their positions along the vertical direction.
Action space We apply an external force on the center of the rob vertically the action signal. We clamp the action within [-500, 500).
Parameter space. We parameter the rod environment by Young's modulus We train Ag = log10 E for better numerical stability and performance. We uniformly sample XE within [4, 7) where the ground-truth value of Ag is
B.2 THE RISP NETWORK
Network setting We build the network upon modified version of ResNet-18 (10). All layers followed by instance normalization (40) without affine parameters, following James et al. (15). Since ReLU degenerates second derivatives to constant values, replace it with Swish non-linearities (35). We change the last layer of ResNet- 18 backbone to a linear projection layer with the same number of output with the state dimension.
Training We use Adam optimizer (18) with learning rate of le-2 and weight decay of le-6 for training. The learning rate has a cosine decay scheduler (22). We train the network for 100 epochs. We set the batch size to 16 by default. To avoid model collapse in the early stage of training, linearly increase the magnitude coefficient of the rendering gradients from 0 to environment-speciff value: we it to for quadrotor, 20 for cube, 100 for hand, and 30 for rod.
14

---

Published as a conference paper at ICLR 2022
quadrotor
hand
rod
average
1.38 + 0.00
0.23 0.00
0.89 0.00
random
63.93 1 9.44
0.32 0.06
0.87
pixelwise-loss
3.02 + 1.35
0.25 0.05
0.05 1 0.01
perceptual-loss
1.59 1 0.26
0.28 1: 0.05
0.11 + 0.06
ours-no no-grrdd
1.30 = 0.15
0.23 0.06
0.04
= 0.0002
ours
0.54 0.25
0.11 0.02
0.04 = 0.0006
oracle
0.15 + 0.09
0.002 + 0.00
0.01 + 0.02
Table 4: Visuomotor control results. Each entry reports the mean and standard deviation of the state discrepancy computed from 4 randomly generated initial guesses and rendering conditions.
B.3 TRAINING DETAILS
We share the experiment settings across imitation learning, and visuomotor control experiments experiments. We use RMSProp optimizer with a learning rate of 3e-3 and momentum of 0.5. We optimize the system parameters the action parameters by 100 iterations.
C MORE EXPERIMENTS
C.I VISUOMOTOR CONTROL
We consider visuomotor control task defined as follows: given a target image displaying the desired state of the physical system, optimize a sequence of actions that steer the physical system the target state from a randomly generated initial state. We set the target states by selecting them from the ground truth in the imitation learning tasks. We report in Table the state error computed from experiments repeated with various rendering configurations and initial guesses. The state error is defined as LI distances between the desired state and the final state from the simulator. The smaller state error and standard deviation from our methods in Table 4 shows the advantages of our approach over other baselines. before, the performance from all methods capped by the oracle, which requires much more knowledge about the ground-truth state.
C.2 REAL- WERLDWORLD EXPERIMENT
C.2.1 PROBLEM SETUP
We consider an imitation learning task for the real-world quadrotor. The imitation learning takes as input the intrinsic and extrinsic parameters of the camera, a real-world quadrotor video clip recorded by the camera, empirical values of the quadrotor's system parameters, and the geometry of the quadrotor represented mesh file. We aim recover an action sequence that the simulated quadrotor resembles the motion in the input video as closely as possible.
C.2.2 EXPERIMENTAL SETUP
Video recording To obtain real-world video clip, we set up calibrated camera with known intrinsic and extrinsic parameters. During the whole recording procedure, the camera remains fixed position and pose. After we obtain the original video clip, we trim out the leading and tailing frames with little motion and generate the pre-processed video clip with total length of 14.4s and frequency of 10Hz.
Quadrotor To replicate a potential deployment, we choose commercial quadrotor that publicly available to users. We manually control the quadrotor that it follows smooth trajectory.
Motion capture (MoCap) system To evaluate the performance of various methods, also build a MoCap system to log the position and rotation of the quadrotor. We attach six reflective markers to the quadrotor that the MoCap system tracks the local coordinate system spanned by these markers.
15

---

Published as a conference paper at ICLR 2022
Note that only use MoCap data for test use, is not necessarily step in our pipeline, and none of our methods ever used the MoCap data as a dependency.
Network and dataset details To be consistent with other experiments, architecture for RISP as the quadrotor environment. We uniformly sample the position and pose ensuring that the quadrotor is observable the screen. Every sampled state comes with different rendering configuration. We generate in total 10k data points and divide them by 8:2 for training and testing.
the network
C.2.3 IMITATION LEARNING
Here we perform the imitation learning task where we reconstruct the action sequence from the real- world video clip. This task is much more challenging than the simulated ones. First, the world physics has unpredictable noises that is impossible to be modeled perfectly by the differentiable simulation. Additionally, the real-world video has unseen appearances including but not limited to the lighting source and the floor texture. It is also noteworthy that the real-world video is exccedingly long which increases the dimension of action sequence dramatically.
In order to solve these challenges, we design a slightly different pipeline of imitation learning for the real-world experiments using RISP. We first run RISP the input video to generate target trajectory of the quadrotor. Next, minimize loss function defined as the difference between the target trajectory and simulated trajectory from a differentiable quadrotor simulator, with the action at each video frame the decision variables. Because this optimization involves many degrees of freedom and a long time horizon, we find a good initial guess crucial. We initialize the action sequence using the output of a handcrafted controller that attempts to follow the target trajectory. Such an initial guess ensures the quadrotor stays in the camera's ' view and is used by our method and the baselines.
0.0s
5.0s
10.0s
14,
RISP
Oratlin
EntSin-nttrred
Figure Imitation learning the real-world experiment. Given a reference video (top), the goal is to reconstruct sequence of actions that resembles its motion. We illustrate the motions reconstructed using our method (upper middle), the pixelwise loss (lower middle), and enhanced variant (bottom).
C.2.4 RESULTS
We show the results the world experiment in Fig. 5 by visualizing a representative subset of frames.
16

---

Published as a conference paper at ICLR 2022
out 1.
curens grad
Ground-ruth
cum random
surent-grad (random
surent-grad
6,7
and frames
Loss (pbrt vs redner)
Every frame
0.6
Every 5th
First and middle frames
0.5
100
0.2
0.1
4.5
1.0
1.80
2.5
4.64
6.34
1.07
1.80
2.55
4.50
6.3
10a
Repoch
Young's Moduluss
(b)
Figure 6: (a) The number of epoch versus loss curves. The solid and dashed curves represent the training curves of ours and ours-no-grad respectively. The color of the lines indicates the number of rendering configurations in training set where green and orange are and 10, and red curve samples different rendering configuration for every training data. (b) The loss versus Young's 's modulus curves. The orange, blue, green, and gray curves represent the loss landscapes with respect to the Young's modulus given different temporal sampling rates. The red vertical line shows where the ground-truth Young's ss modulus locates.
validate the robustness of our method, we aggressively randomized the rendering configu- ration so that it differs significantly from the input video. Our method reconstructs similar dynamic motion without access to the rendering configuration or the exact dynamic model in the input video.
RISP
GradSim We compare the pixelwise loss from VSim (26) with our method by sharing the identical rendering configuration and initial guesses. We observe that the pixelwise loss does not provide valid guidance to the optimization and steers the quadrotor out of the screen immediately. We presume the major reason behind the degeneration of pixelwise loss the assumption of a known rendering configuration in the target domain. However, such information is generally non-trivial to obtain from real-world video.
GradSim-Fnhanced According to our presumption above, we propose two modifications for improvement: First, we manually tuned the rendering configuration until it appears as close as possible the real-world video, and second, we provide it with a good initial guess of the action sequence computed based the ground-truth trajectory recorded from motion capture system. Note that the good initial guess from motion capture system is generally impossible to access in real- world applications and is not used in RISP GradSim. Even with these strong favors, GradSim- GradSim-Enhanced Enhanced only recovers the very beginning of the action sequence and ends up with an uncontrollable drifting out of screen.
C.3 ABLATION STUDY
C.3.1 RENDERING GRADIENTS
We study the impact of rendering gradients on the data efficiency by the comparison between ours and ours-no-grad. We reuse the state estimation data sets (Table on quadrotor but vary the number of rendering configurations between and 10. We then train both of our methods for 100 epochs and report their performances on a test set consisting of 200 randomly states, each of which augmented by 10 unseen rendering configurations. We show the result on Fig. 6a. The right inset summarizes the performances ours (solid lines) and ours-no-grad (dashed lines) under varying number of rendering configurations, with the green, orange, and red colors corresponding to results trained on 1, 10, and randomly sampled rendering configurations. It is obvious to see that solid lines reach lower state estimation loss than their dashed counterparts, indicating that rendering gradient digs more information out the same amount of rendering configurations. is worth noting that with only 10 rendering configurations (orange solid line), method with rendering gradients
17

---

Published as a conference paper at ICLR 2022
achieves lower loss than the one without but using randomly sampled rendering configurations (red dashed line), which reflects the better data efficiency.
By comparing ours and ours-no-grad from Table 1, 2, 3, and 4, we that having the rendering gradients our approach crucial to its substantially better performance. We stress that having a coneriig-invariint state estimation is the source of generalizability in approach and the key to success in many downstream tasks.
C.3.2 TEMPORAL SAMPLING RATE
We study the impact of temporal sampling rate on the performance of RISP using four different sampling strategies: sampling densely every frame, sampling sparsely every 5th frame, sampling only on the first and last frames, and sampling only on the first and middle frames. We reuse the system identification task in rod environment and plot the loss landscapes of them around the ground- truth Young's s Modulus in Fig. 6b. We observe that RISP robust against different sampling rates due to the same global optima of all curves. Additional, except for the extreme case with substantial information loss (first and last frames), most of the curves are unimodal with one local minimum indicating easier problem in optimization. Thus, believe it is safe to expect our method with standard gradient gradient-based optimizer to succeed under various temporal sampling rates.
18


Reviews:
[{"review_id": "jhND8oLvNOy", "reviewer": "Reviewer_UwVk", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- The paper is very well written and contributions are clear.\n- The idea of using a rendering-invariant state-prediction as a pre-step before the gradsim-like framework is novel and sensible.\n- The idea of the using the gradient with respect to the rendering parameters as a regularization is novel and interesting. It could additionally be very used in many other fields where training on synthetic data is used with random rendering parameters.\n- The Combination of weak/strong/oracle baselines chosen in the experiments are very appropriate for comparison.\n- The paper performs a large range of experiments using several environment and tasks to study the effectiveness of the proposed method compared to baselines.\n- The ablation study shows the impact of the gradient loss on the state prediction model performance.\n\nWeaknesses:\n\n- It is not clear if the training and testing rendering environments are the same. This means that the parameter distribution in both training (for the RISP) and testing cases come from the same distribution. In reality, often the test parameter distribution comes from a different distribution. It would be interesting to see results when the train and test rendering parameter distributions are different.\n- It would be interesting to see how the various methods would perform on real videos.", "summary_of_the_paper": "The paper focuses on the problem of estimating dynamic parameters of a physical system from videos under unknown rendering conditions.\nIt presents a novel idea of using a rendering-invariant state-prediction (RISP) network that predicts the state from a rendered image and can be integrated into a framework with a differentiable simulation and rendering engine for parameter estimation. Training this network is done using domain randomization technique with synthetic data. Additionally, the paper introduces a novel gradient loss that further pushes the network to be invariant to rendering parameters.", "summary_of_the_review": "The paper presents mainly two new ideas, the RISP network for state estimation and the gradient loss for regularization. Both are novel and can be interesting and impactful for the community and push forward the state-of-the-art in using synthetic data for parameter estimation. The quantitative and qualitative experimental results on synthetic data show clear improvements compared to the previous methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "T_jLJ0tiA_J", "reviewer": "Reviewer_qnnY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work addressed a crutial problem. The proposed method is novel and neat. The paper is well-written. I like the main idea and technically novel components proposed, such as the rendering invariant regularization term and efficiently calculating the second-order gradient. The results have demonstrated the importance of these components and the improvements over previous methods. \nFor the weakness of this paper, it is not clear how this method works on real datasets. I suggest that the proposed method be evaluated on more datasets, especially real datasets. ", "summary_of_the_paper": "This paper presents a new method of predicting physics simulation parameters and rendering configurations from an RGB video. Unlike previous methods that calculate the loss function in image space, this work proposes to calculate in the simulation state space to avoid the issues (e.g. being stuck in a local minimum) when the reference video is very different from the generated video. Specifically, a rendering-invariant state-prediction (RISP) network is pretrained with the generated data from a differentiable renderer under various rendering conditions.  The RIST network is then appended to the output of a differentiable renderer to make the pipeline from simulation and rendering parameters to states predicted from images fully differentiable. Besides a state prediction loss term for training this pipeline, a novel regularization term is used to enforce the RISP network rendering invariant. In addition, a new training strategy to efficiently calculate the gradient for the loss function is proposed. The experiments have shown that the proposed method significantly outperforms the state-of-the-arts and the proposed components contribute to the final results with big improvements. ", "summary_of_the_review": "This paper is a solid submission, which solves an important problem and proposes some novel ideas. I expect the authors to evaluate the method on more datasets, especially real datasets. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "3P-jw5LNaiX", "reviewer": "Reviewer_paRJ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\n* **S1** This paper tackles the challenging (inverse) problem of directly recovering object properties (system identification) or control parameters (visuomotor control) from image/video observations. The current best approach to this ($\\nabla$Sim) proposes to compose differentiable physics simulation and differentiable rendering to result in a computation graph where ground-truth physical parameters are no longer required. However, this approach has a crucial shortcoming: it works well only when the reference and target trajectories are drawn from the same distribution (i.e., identical physics and rendering engines used across both). This work proposes RISP to bridge that gap (bridging this gap is crucial to enable several downstream, including real-world, applications).\n\n* **S2** The paper is very well-presented. The core ideas are easy to follow, and the rationale for incorporating the rendering gradients into a regularization term is well laid-out. Of the three contributions, I believe this to be the more significant one (as also corroborated by Fig. in section 4.6)\n\n* **S3** This approach (RISP) demonstrates strong performance over current art -- this is over multiple environment settings (rigid, articulated, deformable object identification; control).\n\nIn general, this is well-executed work and I would as such recommend acceptance. However, there are a few issues I hope to see discusses/addressed over the rebuttal phase.\n\n### Weaknesses\n\n* **W1** Clarifying the impact of differentiable physics simulation vs rendering: Are the train sets generated using a different differentiable physics engine? (the manuscript mentions they're generated using a different rendering engine -- but, arguably, generating them using a different physics engine as well would create a wider domain gap) This also raises several interesting questions such as: will RISP generalize to scenarios where the underlying dynamics change too? If not, are there other domain randomization / regularization methods that can assist?\n\n* **W2** Baseline choices: Do all baselines use all frames in the reference trajectory to compute the loss. Just as dense pixelwise loss baselines are ablated upon by bringing in other baselines that treat the image as a whole in loss computation; a similar analogy may be drawn at the sequence level (i.e., computing the full pixelwise loss across all frames in a sequence, vs. using pixelwise loss across select frames in a sequence)\n\n* **W3** Disentanglement/Compositionality of the learned RISP: The current training strategies do not seem to explicity focus on disentangling state representations or improving compositionality of the learned representation. (This seems clearly out of scope for the current work, but I'd like to bring this discusison point here to perhaps prompt addition of clarifying statements in the manuscript). Does this have an imact on e.g., where the object may lie within an image, and perhaps impact the extension of RISP to simulations with multiple objects?\n\n\n### Minor comments\n\nThese comments are nitpicks/typos and are easily addressed in a minor revision. The authors needn’t respond to these\n\n* “Articulate-body” -> articulated body\n* “Simulates” -> simulate", "summary_of_the_paper": "This paper proposes a general approach to leveraging differentiable simulator for the downstream tasks of system identification and visuomotor control WITHOUT requiring access to the true underlying states. They do so by predicting a \"rendering-invariant state\" that results in a much stabler loss landscape and reduces domain gap / mismatch. Experiments over 4 environments indicate the merits of this approach over current state-of-the-art.", "summary_of_the_review": "This is a well-written paper describing an idea of substantial interest to the physical reasoning and the visual reasoning communitties. I only have clarifying concerns with this work and feel the paper will be well-rounded if the limitations of the work are discussed upfront.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method to solve the inverse problem of identifying parameters of a dynamic physical system from image observations. The main idea is to train a rendering-invariant state-prediction (RISP), which estimates the inverse mapping from the pixel to the state domain. The authors introduce a new loss to this end, and an efficient gradient computation of the loss.

The paper received three clear accept recommendations. The reviewers discussed the potential improvement of RISP when combined to disentanglement methods, and also raise several concerns regarding experiments, e.g. rendering conditions during training and testing, or evaluation on real data. The rebuttal did a good job in answering reviewers' concerns, and the reviewers especially appreciated the new results on real videos. Eventually, all reviewers recommended a clear acceptance of the paper.

The AC's own readings confirmed the reviewers' recommendations. The paper is introduces very solid contributions for solving the complex task of physical parameter identification in the unobservable setting. The paper is also clear and well written, and validated with convincing experimental results. Therefore, the AC recommends acceptance.
Decision: Accept (Oral)