Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes a novel technique to perform fluid simulations. Specifically, they promote the idea of using spatial convolutions to model how particles interact with nearby particles. Compared to graph-based models, this approach has several advantages and yields a model that can be conveniently trained end-to-end. The authors also develop a specific type of continuous convolution that yield better and faster inference than the benchmark algorithms. 

To main contribution in this paper is the idea of using spatial convolutions to model particle interactions. Even though the obtained results contain significant errors compared to ground truth, the paper indicates a promising strategy others may leverage on to develop even more accurate deep learning based simulators. Considering that they have also plausibly argued that their specific algorithm is already state of the art, I view this as a significant contribution. Having said this, the contribution is really to use a well-known technique (spatial convolutions) on a new problem (fluid simulations). My understanding is that ICLR primarily wants to promote general learning techniques and I am not convinced that this paper contains any significant contributions in this field. 

The authors also develop a specific network architecture that they compare with other deep learning architectures for continuous convolutions. Unfortunately, the design contains a number of questionable choices and I suspect that the main reason that existing architectures for deep learning using continuous convolutions perform worse is that their hyper-parameters have been fine-tuned for a different task. As an example of a questionable choice, why do you “exclude the particle at which we evaluate the convolution” in your convolutions? 

Minor remarks: 
* You include a constant 1 in the input feature vectors. Assuming that the neurons in your network have weights and biases, this constant is completely redundant. Of course, this also means that you can include it without ruining your performance, but why would you?
* I found the explanation of Lambda in Figure 1 too short to be understandable. 
* In (7), you seem to be using convolutions between functions that have not been pre-mirrored, and it would be better to then express (5) and (6) on the same form. 
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: [Summary]

This paper proposes to learn fluid dynamics by combining the position-based fluids (PBF) framework and continuous convolution. They use dynamics particles to represent the fluids, and static particles to describe the scene boundaries, and employ continuous convolution to learn the interactions between the particles of different kinds. They have demonstrated the effectiveness of the proposed method by comparing it with several state-of-the-art learning-based and physics-based fluid simulators. Their method outperforms the baselines in terms of both accuracy and efficiency. They have also shown that the model can extrapolate to terrains that are more complex than those used in training, and are useful in estimating physical properties like the viscosity of the fluids.


[Major comments]

For now, I slightly lean towards acceptance, as I like the idea of combining PBF and continuous convolution for fluid simulation, and the method seems to have a much better performance than the baselines. The experiments have also convincingly demonstrated the method's generalization ability to terrains of various geometry and fluids of different viscosity. However, I would still like the authors to address my following questions.

My primary concern about the proposed method is the scope of its applicability. One of the benefits of using learning-based physics engines is that they directly learn from observations while making very few assumptions towards the underlying dynamics, which gives them the potential to handle complex real-world scenarios. The model in this paper, however, heavily relies on the PBF framework that may limit its ability to simulate objects like rigid bodies and other deformable materials. I would be curious to know the authors' views on how to extend their model to environments with not just fluids, but also other objects of various material properties.


[More detailed questions]

Will the method run faster than DFSPH, given that the timestep is much larger than the timestep used by DFSPH, 0.02 ms vs. 0.001 ms? Will the learning-based physics engine have the potential to outperform the physics-based physics engine in terms of efficiency?

For estimating the viscosity of the fluids, how well does the gradient descent on the learned model perform comparing with black-box optimization, e.g., Bayesian Optimization using the ground truth simulator?

In the SPNet paper, they have also tried to solve the inverse problem of estimating the viscosity of the fluids. It would be great to include a comparison to see if the proposed method can outperform SPNet in terms of efficiency and accuracy.

Equation 8 smooth out the effect between particles of different distances. How sensitive is the final performance of the model to the specific smoothing formulation? Is it possible to learn a reweighting function instead of hardcoding?

In figure 3, the model's rollout is a bit slower than the ground truth. The authors explained the phenomenon using the "differences in the integration of positions and the much larger timestep." I do not quite get the point. Could you elaborate more on this? Also, it might be better to include labels for the two columns in figure 3 to make it more clear.

In the experiment section, the authors claimed that SPNets take "more than 29 days" to train. Correct me if I am wrong, but from my understanding, SPNets directly write Position-Based Fluids (PBF) in a differentiable way, where they can extract gradients. Except for the tunable parameters like viscosity, cohesion, etc., I'm not sure if there are any learnable parameters in their model. Could the authors elaborate on what they mean by "the training time" of SPNets?

From the videos, DPI-Nets does not seem to have a good enough performance in the selected environments. I can see why their model performs not as good since they did not use as much of a structure in the model. But from the videos of DPI-Nets, it seems that they perform reasonably well in scenes like dam break or shake a box of fluids. Would you please provide more details on why they are not as good in the scenes in this paper?

The data was generated using viscosity varying between 0.01 and 0.3. How well can the model do extrapolate generalization? It would be great to show some error plots indicating its extrapolate performance.

Why there are no average error numbers for SPNets?

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper applies 3D convolutions to the problem of Lagrangian fluid simulation. The primary difficulty in this is that, unlike for Eulerian fluid simulation, which represents the fluid as a grid and adapts nicely to 3D convolutions, in Lagrangian simulations the fluid is represented as an unordered set of particles. It is not straight-forward to apply 3D convolutions on such a data structure, however this paper proposes a method to apply the same regular-grid kernels used in grid-based convolutions to the particle structure. To do this, several points around the kernel are evaluated by first using trilinear interpolation between the particles to get feature values at those points and then convolving those values with the kernel weights. This results in a new particle in the next layer up with those features. In the paper, this method is used to train the weights of the network to reproduce fluid dynamics generated by a simulator. The results show that the proposed method was able to model fluid dynamics over 2 timesteps more accurately than other methods and can do so quickly. 

While I have some reservations about this paper (detailed below), on the whole I think it is a quality contribution and should be accepted. This paper contributes a novel method for performing 3D convolutions on unordered particle sets, and it shows that the learned fluid dynamics generalize to novel situations. One major hurdle to applying modern convolutional learning techniques to Lagrangian methods is the mismatch between the layout of the data (unordered particles) and the layout of the kernels (regular grid). This paper presents a novel way of bridging that divide, and it shows that the proposed method actually works by applying it to the problem of fluid dynamics and successfully learning it. However, one major concern I had was that it seems all of the training data was generated in box-like environments, which could easily lead to overfitting. This was alleviated by the results showing that although the network was trained only in boxes, it generalized to environments with channels and waterfalls (as seen in the video). This is a powerful result and shows that this method really did learn fluid dynamics and not just a shortcut that only works in boxes.

I do think this paper can be improved in a few aspects however. The biggest issue is that the quantitative analysis of the core functionality (reproducing fluid physics) is lacking. The paper only reports results for error after at most 2 timesteps, which is not nearly long enough to determine if the output is accurate. Furthermore, the results are only reported for the box scenes, not the generalization scenes mentioned above. Qualitatively, from the videos, it is clear that the output does at least somewhat model fluid dynamics, but it would be much better to have hard numbers to back that up. I suspect the authors discovered that Lagrangian systems are sufficiently chaotic that after only a few timesteps the particle positions have diverged significantly. This is not a bug but a feature of such systems. In Lagrangian fluids, the particles are but an approximation of the fluid, and unlike Eulerian systems*, multiple different sets of particles can approximate the same fluid. This makes particle position only useful as a measure of error if the trained model can perfectly reproduce the fluid dynamics. But of course it can't (trained networks aren't ever perfect in practice), and so small errors quickly compound into large particle position disparities. So even though the trained network models the fluid well overall, the particles end up in completely different locations. Instead a better error metric would be something like measuring the difference between the surface of the fluids, or the velocities or densities at various locations. These are agnostic to the particular particle positions, but still measure how well two different sets of particles represent the same fluid. Using a metric like this, it would be nice to see error graphs over time for both the box and generalization scenes.

A couple other smaller points. The chaotic divergence behavior of DPI-Nets seems inconsistent with that paper. Is this possibly a bug in the way it was implemented here? Additionally, the paper states that the convolutions of SPNets were "specifically designed to implement the position-based fluids algorithm" but that it was used in the paper with a much larger number of channels. If it was designed only to work for that one algorithm, how were the number of channels increased? That is unclear. Also, the average error for SPNets is not shown in Table 1 and it is not stated why.

*Assuming same grid shape, size, and position.
review_title: Official Blind Review #4

Meta-review: The paper proposes an approach for N-D continuous convolution on unordered particle set and applies it to Lagrangian fluid simulation. All reviewers found the paper to be a novel and useful contribution towards the problem of N-D continuous convolution on unordered particles. I recommend acceptance.  
Decision: Accept (Poster)