Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: *Summary*
This paper describes DiffSim, a differentiable programming system for learning with physical simulation. The system (built on the Taichi system) allows users to specify a forward simulation in a Python-like syntax, after which the program is compiled and iteratively run in both forward-mode and gradients computed for system parameters and controllers, as desired. A variety of simple simulations are included, demonstrating that the automatically generated CUDA code runs as fast as hand-written CUDA code (and noticeably faster than TensorFlow or PyTorch implementations), while requiring far fewer lines of code. The final section details two issues--time of  impact errors due to discrete time intervals and gradient explosions with long time horizons--and some potential solutions.

*Rating*
The paper is interesting and easy to read. While some part of the underlying functionality of DiffSim is directly derived from previous work (Taichi), the paper does describe a non-trivial contribution.

I lack the background to comment constructively about expectations for these simulations or the fidelity of the methods in this paper. What evidence can you offer regarding the physical fidelity achievable and how that relates to issues of scalability, gradient behavior, size of time steps, code complexity, etc.? For a sense of context, what might be needed to simulate a 7 DoF robotic arm and learn a controller that would reasonably transfer to a real robot?

Overall, I'm optimistic about this paper, and would tend to vote for acceptance.

*Notes*
pg3: define k (spring stiffness?)
pg4: what is the value of 'mass' for this simulation?
Fig 8: what is the x-axis in the two right plots? initial height?
Fig 10: right plot title should probably be "Gradient Explosion with Damping"
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper presents a programming language for building differentiable physics simulators. This is a very interesting goal, as differentiable systems are a crucial building block for many deep learning methods and similar optimization techniques. 

The system presented by the authors is certainly impressive. Unfortunately, the paper itself covers a wide range of topics, and consists of an overview of the language with a programming tutorial, a collection of ten results, and a brief discussion of problems when computing gradients. 

The core of the proposed work, the programming language seems to be quite powerful. However, it seems to be built on an existing system, which was published as a programming language for simulation in this years siggraph asia conference (Taichi: A language for high-performance computation on spatially sparse data structures. In SIGGRAPH Asia 2019 Technical Papers, pp. 201. ACM, 2019a). This ICLR submission seems to extend this system to build and provide gradient information automatically along with the simulation itself. There seem to be few technical challenges here, and many aspect discussed in section 2 are shared with the original simulation language.

The examples cover a nice range of cases, from simple mass spring systems and a rendering case to complex 3d simulations. Here, I was a bit surprised that the paper only compares to autograd, which has been succeeded by jax. The latter also provides a compiler backend to produce GPU code with gradients, and as such seems very closely related to the proposed language. From the submission, it's hard to say which version has advantages. The examples seem to be a sequence of demos of the language, rather than illustrating different technical challenges or improvements for a scientific conference. Or at least a discussion of these differences is currently missing in the text.

Section four also mostly gives the impression of a loose discussion. The gradients for rigid body impacts are interesting, but seem relevant only for a subset of 2D examples shown in the paper. The discussion of gradient explosions is quite ad-hoc, and would be stronger with a more detailed analysis.

The submission as a whole aims for a very interesting direction, but I think the paper would benefit from focusing on a certain range of problems, such as the rigid body control cases, in conjunction with topics such as the improved gradients. Instead, the current version tries to combine this topic with a systems overview, a tutorial and loosely related discussions. Combined with the length of 10 pages, I think the work could use a revision rather than being accepted in its current form.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper introduces DiffSim, a programming language for high-performance differentiable physics simulations. The paper demonstrates 10 different simulations with controller optimization. It shows that the proposed language is easier to use and faster than the other alternatives, such as CUDA and TensorFlow. At the end, the paper provides insightful discussions why the gradient of the simulation could be wrong.

Differentiable physics simulation is an important research area, especially for optimal control and reinforcement learning. While I am impressed by the large variety of examples demonstrated in the paper, I am leaning towards rejecting the paper because of its poor presentation. The paper only gives a simple and high-level example of the language (optimizing the rest length of springs that form a triangle), very brief descriptions of 10 examples and some discussions about the difficulty of computing useful gradients, but without any in-depth discussion how everything is implemented. This is not enough for an ICLR paper. For example, the paper does not answer some of the fundamental problems of differentiable physics. For example, collision and contact are inherently non-differentiable. How does the paper handle it in the examples of locomotion and billiards (Figure 4)? In addition, how does the paper back-propagate the gradient through the incompressibility conditions (Poisson solve) of fluid simulation? 

Here is my suggestions how to improve the writing. There are several ways to write the paper, with different emphasis. If this paper is more about introducing a new programming language, Appendix B Compiler Design and Implementation would be important and should be moved to main text. If the paper want to emphasize how to handle the non-differentiable cases of the simulation, then detailed derivations of contact, collision, and linear/nonlinear solving (due to incompressibility conditions or implicit integrators) should be presented. If the paper would like to demonstrate how differentiable physics simulation can help with controller optimization, then two to three examples, such as the locomotion control for soft bodies or rigid bodies, should be analyzed in far more details, and compared with traditional method without differentiable simulation. It is good to focus on one of the above points, based on the venue that this paper is submitted to. Currently, the paper is trying to touch all three. But due to the page limit, it is not thorough, or detailed in any one of them.

-------------------------Update after rebuttal------------------------------
Thank you for the revision of the paper and the additional comparisons with Jax. The revised version reads much better. The response and the revision addressed most of my concerns. Thus, I raised my rating to weak accept.
review_title: Official Blind Review #1

Meta-review: The paper provides a language for optimizing through physical simulations. The reviewers had a number of concerns related to paper organization and insufficient comparisons to related work (jax). During the discussion phase, the authors significantly updated their paper and ran additional experiments, leading to a much stronger paper.
Decision: Accept (Poster)