Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents FluidLab, a simulation environment with a diverse set of manipulation tasks involving fluid. The simulation environment is supported by FluidEngine, which is built with Taichi. In addition, the paper also provides some domain-specific optimization techniques to best utilize differentiable physics in liquid manipulation. 

strength_and_weaknesses: Strength:

1. This work covers a variety of interesting fluid manipulation tasks, such as ice cream and latte art, while being fully differentiable.

2. This work also supports complex fluidic material behavior.

3. It supports customization of the environment.


Weaknesses:

1. The sim-to-real gap can be a significant concern. Since there is no real robot execution, the reader does not know how much of a gap there is between the proposed simulation and the real world.

2. The proposed benchmarking effort does not benchmark SOTA methods for fluid manipulation. The benchmark itself should measure the performance of recent SOTA methods to show real progress in this area of research and to provide a promising direction. Methods of fluid manipulation can be considered, for example, SPNets [1].
Methods of differentiable RL can be considered: SHAC [2], PODS [3].


3. Table 1 compares existing simulators with rigid and liquid object types, with some important works left out of the literature review. These works are also differentiable and support various object types: DiffSim [4], Disect [5].


4. Key insights from the benchmark results are missing. Considering these results, it would be more valuable to provide a clear direction or insight to guide the researcher.

[1] Connor Schenck, Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural networks. CoRL, 2018.

[2] Jie Xu, Miles Macklin, Viktor Makoviychuk, Yashraj Narang, Animesh Garg, Fabio Ramos, and Wojciech Matusik. Accelerated policy learning with parallel differentiable simulation. ICLR, 2022. 

[3] Miguel Zamora, Momchil Peychev, Sehoon Ha, Martin Vechev, Stelian Coros. PODS: Policy Optimization via Differentiable Simulation. ICML, 2021. 

[4] Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming C. Lin. Scalable differentiable physics for learning and control. 2020.

[5] Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos. Di-sect: A differentiable simulation engine for autonomous robotic cutting. RSS, 2021. 

Questions:

1. How helpful is gradient sharing? Are there any ablation study results to help with the understanding?

2. The task of Latte Art seems to be highly nonconvex, but DP and DP-H accomplish this task almost perfectly. We all know that differential-based trajectory optimization eventually reaches a local minimum, how can DP and DP-H tackle the local minimum problem and reach the global optimum? In detail, what are the initial actions of DP and DP-H?
clarity,_quality,_novelty_and_reproducibility: Clarity: the paper is generally clear and easy to read.
Quality: in general, the paper is of decent quality.
Reproducibility: considering the system is complex, without the code, it is hard for one to judge the reproducibility.
summary_of_the_review: I believe validating the simulator with the real robot is a must when the task is designed for robot manipulation. In addition, as a benchmark paper, demonstrating recent SOTA methods to show real progress in this area of research and providing a promising direction is very important.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper describes a differentiable simulator for a series of tasks tailored towards robotic interactions. The system is MPM-based, and presents a range of different tasks that focus on interactions with fluids, all the way from two-dimensional transport to 3D manipulations. The paper then reiterates on the argumentation from previous work that using directional information in the form of gradients outperforms model-free RL approaches. On the side, the paper also briefly discusses modifications of the simulation based gradient calculations.
strength_and_weaknesses: The paper presents an interesting range of test cases, and fluid interactions are definitely and interesting and challenging task for learning methods. The test cases also exhibit a nice complexity, and the paper contains a good range of classic RL methods that are compared to the differentiable physics variant.

A big question that I had while reading is also how this engine is positioned in the existing environment of differentiable simulators. For the MPM cases, the engine seems to build on Taichi, but doesnt make clear how it extends it. Taichi already provides gradients since DiffTaichi - so is it merely a selection of scenes for Taichi, or are there changes to the simulator? The paper is currently written to advertise the "FluidEngine" simulator. However, it does not argue why it wold make sense to choose this simulator over existing ones, i.e. Taichi for MPM, or Phiflow for single phase cases (a discussion of the latter seems to be missing). In order to claim this system as a contribution, I think it would be important to run comparisons to show what advantages are to be gained over alternatives such as Taichi, Phiflow etc.

The selection of test cases also seems somewhat arbitrary to me. Why is there such a large focus on latte art and ice cream? These cases make for nice visuals, but if the goal of the paper is to establish these scenes as a more widely used benchmark, this seems strange to me. The remaining 6 cases establish a wider range of different behavior, and look more meaningful to me. In general, I would recommend to add structure to the selection and ordering of the cases. Also it would be good to argue why these particular scenes are worth considering as test cases. I definitely found it strange to start with "latte-art" instead of basic settings than build up to more complex ones.

A smaller note, but I would also caution against statements that the DP variants yield fundamentally better learned states than the RL variants. In the limit of infinite compute a well-tuned RL algorithm will eventually discover these as well. DP might get there much faster, but there's no reason RL should not discover these states at some point.
clarity,_quality,_novelty_and_reproducibility: As mentioned above, it is an interesting direction, but the advantages over existing simulators are left unclear in the current submission. A better motivation (and potential adjustment) of the test cases would also be a good idea.

summary_of_the_review: My main open question is that of advantages over existing simulators. Table 1 is not sufficient here: other differentiable fluid simulators like (Diff)Taichi, Phiflow and JAX-Fluids are already out there, so I don't understand why they're left out, and how the work distinguishes itself here. The RL comparisons are interesting, and the tweaks for proper convergence likewise. Here, I think the paper undersells a bit: there's likewise no effort to put this into context of existing works at ICLR, and as such they don't play a very prominent role. Thus, taken together from the current state of the submission, this looks like a paper below the acceptance threshold to me.

---

Post rebuttal: the authors have provided a nice update, together with the other assessments, I'd be happy to support accepting this paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper introduces FluidLab, a platform for simulating interactions of complex fluids and solid objects. The platform includes a fully-differentiable and GPU-accelerated physics engine (FluidEngine), as well as an OpenGL renderer. FluidEngine is used to evaluate a few existing algorithms in the context of the FluidLab task set -- a set of 10 complex fluid manipulation tasks modeled on real-world scenarios that the authors define and introduce in the paper. The experimental part of the paper compares model-free RL to trajectory optimization based on sampling and differentiable physics. Access to gradient information of the simulation is shown to significantly benefit trajectory optimization.

strength_and_weaknesses: The paper is built around FluidEngine, which is also one of its main strengths. It represents a significant aggregation of capabilities in a single simulator, with alternatives falling short in terms of supported types of matter, differentiability, and source code access. An additional advantage in this context is the usage of the OpenAI Gym API. 

A weak side of this work is the lack of information about the validation of the physics engine. The authors are using existing methods, but implement them from scratch. A physics engine is a complex piece of software, which can suffer from subtle bugs which are hard to notice outside of a rigorous evaluation in setups with known solutions.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written, well-organized, and should be understandable by people without a background in physical simulations. It refers to prior works as necessary. The authors promise to release the code for the engine and the sample simulations, which should allow users to reproduce the results presented in the paper. The paper itself of course does not offer sufficient space to fully document a complex engineering artifact like FluidEngine. 

Additional novelties introduced in the paper are the set of complex tasks forming FluidLab, and a couple of 
techniques for making better use of gradients and stabilizing the optimization problem (soft contact model, starting with a restricted temporal region, cross-particle gradient sharing). 

Questions/suggestions:
* What are the limitations of the solver in terms of density ratios, Reynolds numbers, etc? Are there any concerns about numerical stability?
* How is mass taken into account in the SDF representation?
* The text mentions that the mesh provides boundary conditions for the fluid; is momentum transfer from the fluid to the mesh allowed as well, i.e. is the coupling bidirectional?
* I found it surprising that gases are simulated as an *incompressible* fluid, when one of the basic properties of gases is their compressibility. Could you please clarify the methodological choices here?

summary_of_the_review: Overall, this appears to be a significant piece of work, introducing a very useful piece of software, and proposing a set of standard tasks which require more complex physical interactions than those typically used in similar setups. The authors correctly note that a lot of prior work in the field of fluid manipulation involves relatively simple scenarios. The paper represents a significant advance towards more complex settings.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: This paper presents FluidEngine and FluidLab. FluidEngine is a differentiable physics engine that can simulate rigid objects, deformable objects, liquids, and gasses by combining different methodologies together. FluidLab is a suite of MDP environments for evaluating the efficacy of different algorithms on physical manipulation tasks. The tasks in this paper are diverse (from latte pouring to HVAC control) and manipulate many different kinds of objects. The results show that algorithms that can take advantage of the gradients provided by the engine perform significantly better on the tasks than algorithms that can’t. 
strength_and_weaknesses: This is a strong experimental paper. It presents a differentiable physics engine alongside a series of tasks to demonstrate its value. The engine combines many different techniques to allow it to simulate a variety of material types such as rigid objects, plastics, liquids, and gasses. It also can compute the gradients of the state of the simulation, making it valuable for gradient-based techniques. Furthermore, the paper adds to this a set of experimental tasks designed to show the value of a differentiable physics engine. The tasks span a wide variety, from latte art to ice cream to HVAC control. The results in the paper clearly show that the gradient-based techniques are better able to solve the tasks, and in some cases the gradient-free techniques can’t solve them at all.

The biggest weakness of this paper is that it doesn’t present any novel methodology. It’s essentially an announcement of a new library (FluidEngine) and an empirical argument around why it’s useful (FluidLab). I actually don’t see this as much of a weakness, if at all. With the fast pace of research these days, there is a strong incentive to focus on novel methods and sacrifice all else. This means that things like infrastructure and datasets (everyone makes their own, the days of Imagenet are over) and repeatable experiments often get overlooked. The fact that this paper stops to focus on that is actually beneficial to the field, and can provide a good testbed for future algorithms developed in this area. It’s worth a publication to bring this to the attention of a wider audience.

clarity,_quality,_novelty_and_reproducibility: The paper is clear and well written. While it is not novel in the algorithmic sense, it does add a new tool for researchers to use as well as a set of standardized experiments to compare against. Furthermore, because this paper both makes the engine and the experiments public (or will do so upon acceptance according to the paper), reproducibility is high. It should be easy for future researchers to download the code and rerun the experiments on their own.

There are a few minor things missing however. The paper doesn’t mention what the size of each timestep is (e.g., 0.1s). There also isn’t any mention of how fast the engine is for each of the tasks. Does it run in real time on a modern computer? Slower? Faster? Finally, several RL policies are trained on the tasks. It’s never stated what the input is, although it is explicitly stated the input is not rendered images. If the state of the world is represented mostly as particles, it is unclear how those particles are input into the RL policies. This is non-trivial, and should be clarified.

summary_of_the_review: Overall, this is a good paper. It presents a new differentiable physics engine and a set of standardized tasks to go with it. The results in the paper clearly show that the use of gradients vastly improves the performance of the policies. While this paper doesn’t present a novel algorithm, its contribution as a platform and experimental paper make it a high quality publication.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a new simulation environment for different manipulation tasks involving fluids. It had received 4 expert reviews with a relatively high rating average of 7, but an initially high spread, with two yet unconvinced reviewers. Their concerns were on missing benchmarking of existing methods and on positioning.

The authors provided answers and could address most of the comments and issues raised by the reviewers.

A consensus quickly emerged on the quality of the work: the introduction of a useful new software and benchmark, requiring more interactions than existing work. The AC concurs and recommends acceptance.
justification_for_why_not_higher_score: While this is a strong experimental paper, no new methodology has been introduced.
Could be an oral, but I do not immediately see a reason for it.
justification_for_why_not_lower_score: Could be also a poster
Decision: Accept: notable-top-25%