Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: - Summary

This paper presents a graph-network-based architecture for learning to perform mesh-based simulations, which can be run more efficiently than the full, "ground-truth" simulations. The experiments demonstrate that the proposed method is able to learn to simulate a wide range of different physical scenarios. Moreover, the presented results also demonstrate an ability to generalize to configurations different from the one seen in training.
 

- Pros

The paper is clearly written and strikes a good balance of presenting the relevant information in its main body and including the necessary details in the supplementary material. The additional videos provided are helpful in demonstrating the model's capabilities. 

The experiments performed are comprehensive, with comparisons to a wide variety of significant baselines (e.g., other graph neural networks, grid based methods, etc.) and evaluations of the impact of diverse hyperparameters.

Most importantly, the results presented are strong, containing both impressive qualitative demonstrations and good quantitative performance when compared to baselines. 
 
The ability to operate directly on meshes, in contrast to previous work that operated on structured grids, makes this method more relevant to real-world applications.


- Cons

The proposed method is, conceptually, a straightforward evolution of previously proposed graph-network-based methods for physics simulations, with most notable similarities to Sanchez-Gonzalez et al. (2020) ([33] in-paper reference).

The ability to generalize to higher-dimensional meshes is only briefly (quantitatively) evaluated (more on this in the comments below).


- Reasons for score

Overall, given the "pros" described above, notably the strength of the results presented on a wide variety of tasks, I recomend this paper for acceptance. 
As mentioned above, the method proposed is a modification of a graph-network based method previous employed to simulate particle dynamics. In order to work on meshes, some modifications are proposed. such as adding mesh- and world-space connections to the graph, learning to predict the S matrix to remesh, etc. As such, this works stand on the strength of the practical results it achieves. The results demonstrate an impressive ability of the proposed method to learn to capture both Eulerian and Lagrangian mesh simulations. The wide diversity of tasks for which the model works well is unprecedented. The experimentation is extensive, with comparisons to significant baselines and evaluations of the relevance of different hyperparameter choices.

Moreover, methods that learn to simulate physical processes more efficiently, and applications of graph neural networks are two research directions that have garnered a lot of interest in recent work. As an intersection of these two areas this paper should be of interest to a wide audience in the conference.
 

- Additional comments

One important drawback of these types of physical simulation methods, which require training on ground truth data, is that training itself requires a large amount of data from the very process we want to simulate. If the learned model only presents a limited ability to generalize, this can severely limit the applicability of the method. This does not seem to be the case here, as the experiments seem to demonstrate an good ability to generalize to unseed scenarios, provided in part by the ability of the graph neural network to learn local/scale-invariant interactions.
In this direction, an important capability of the model, suggested in the paper in the "tassel sub-experiment" of the FlagDynamic experiment, is the ability to generalize to much larger meshes than the ones seen in training. As mentioned in the "cons" section above, however, this ability is only briefly (qualitatively) evaluated. It would be interesting to see a more thorough evaluation of this ability, due to its potential large importance to practical applicability, by allowing the model to be trained on smaller examples and then scaled up at inference time.
Moreover, a thorough analysis of the error trade-offs when moving from smaller to larger meshes would also be interesting.

In the section on "Key hyperparameters", within the experimental results, it is claimed that "the model performs best given the shortest possible history ..., with any extra history leading to overfitting. This differs from GNS [33], which used $h \in 2 \dots 5$ for best performance." Do the authors have any hypothesis as to why this might be the case here, versus the observation in [33]?
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:

The paper presents a scheme for applying graph networks to surface and volume meshes for simulation of fluid flow and structural analysis.
To this end, the method adds problem specific positional features to the mesh nodes and uses a message passing approach to compute the desired outputs.
For Lagrangian systems the discretization is augmented by adding additional edges between nodes that fall below a distance threshold.
The experiments demonstrate better performance than grid-based or other graph-based approaches on the selected problems.


Score:

I am on the fence here. I think the paper has some good results but only scratches some important questions with respect to the mesh.
Further, claims like computational efficiency are only weakly supported.
I hope my questions and concerns in can be addressed.


Pros:

* The videos are good and show the stability of the method and the generalization extent.

* The network architecture is straightforward and simple. 
  Learning the sizing field is a good addition and a good future direction for adapting the mesh.

* The experiments show well the advantages of adaptive grids versus regular grids and meshless methods.


Cons:

* The claim that this approach can be used to speed up simulations is not well supported.
  Table 1 gives the inference time per frame but does not give any information about the hardware used in the experiments.
  (The appendix mentions a V100 GPU but I could not find information about the CPU which is used for the GT simulation.)
  The comparison would also be more interesting if GPU solvers would be added to the comparison.
  As an alternative the graph network can be run on the CPU.

* The appendix contains some information about the remesher for the cloth simulation but apart from that meshing is not studied in the paper despite being crucial for this method.
  The meshes in the datasets use triangles and tetrahedra but another possible choice can be quads and hexahedra.
  I would be valuable to know if the proposed method can generalize to this, needs to be trained for this specifically or does not work at all in this case.
  A similar direction would be to experiment with badly shaped elements that are large but have small area/volume to check if the learning approach is more robust.

* The related work should be extended with more works that share similar goals (e.g. improving runtime).
  - Capuano and Rimoli, "Smart finite elements: A novel machine learning application" Computer Methods in Applied Mechanics and Engineering 2019.
  - Luo et al., "NNWarp: Neural Network-Based Nonlinear Deformation" IEEE Transactions on Visualization and Computer Graphics 2018.


Questions:

The description of the world-space edges is not clear to me. Since r_w is an absolute threshold, can this lead to crossing edges in extreme situations?

The "Computational efficiency" section mentions that the network can do much larger time steps. 
What is the time step used during inference for the network? What is the time step for the classical simulator?

Can you give a bit more information about the elements used in the DeformingPlate dataset? Are these linear elements?

The flag videos show generalization with respect to rotation about the gravity axis.
Is the network solution invariant with respect to all 6 DOF?

What is the memory footprint of the method? Can problems with millions of nodes be simulated on a recent GPU?


review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Contributions

The authors present a graph convolution architecture suitable for learning physical simulation on meshes.
This method supports dynamically re-meshing to locally adapt to the complexity of the simulated physics, and is stable to relatively long rollouts of physical simulations.

Strong points

The architecture presented is generic and seems suitable for a wide variety of physical tasks. This point is supported by multiple experiments on diverse tasks.
Getting a second architecture to learn the sizing field is very effective. The authors demonstrate a way to supervise this second network even when the sizing filed is not exposed by the ground truth physical simulator. This way is experimentally and quantitatively demonstrated to be robust. As a consequence, the whole architecture remains generic enough to be applicable to emulation tasks even when the simulator is not providing an explicit sizing tensor.

The proposed architecture is surprisingly stable in long rollouts, and does not diverge nor accumulates errors from previous steps. As noted in the Appendix, robustness is enforced via a noisy training scheme from previous literature. Thanks to its convolutional architecture and positional encoding, it can be trained on small systems that are inexpensive to simulate and then later be applied to larger, more complex and fine grained problems. This has the potential to enable fast physical emulation without huge training costs.
Adding one-hot encoding allows to model complex interactions between physical systems, such as an actuator using on an elastic plate (DeformingPlate experiment).

Finally, comparison against existing methods (based on CNN, graph convolutions or particle-based representation) seems favorable to the presented technique.

The paper is well written. Linking to anonymised videos showing experiment results is well done and provides an enjoyable reading experience.

Weak points

Authors do not provide ablations studies on the main novelties of their graph convolution architecture. A fully grounded paper would show the quantitative benefits of:
Adding world edges in addition to existing mesh edges
Adding labels on edges that are initialised with distances
Dynamically remeshing at each step: what happens if you just do not re-mesh? Fig. 3b provides an intuition for it, but I cannot find any quantitative study of it. More than that: in table 1, FlagSimple (fixed meshing) metrics are comparable or better than FlagDynamic. This seems to suggest re-meshing is not useful in this case.
The above 3 points are really what sets this paper apart from the rest of the literature on graph convolution nets. Seeing how they impact results seems mandatory to assess how MeshGraphNet is more than just an existing mesh convolutional network architecture plugged into a physical problem.

About benchmarks: it is not clear why the concurrent method GCN was not taken in full. Its super-resolution part was removed for the purpose of this submission, but it seems to be inherently part of it. Removing this component is a-priori an alteration of its architecture, and the reasons for discarding it are not justified.
Moreover, in comparing to GCN the authors conclude that the key reason for MeshGraphNet to outperform GCN is the positional encoding added to edges, but this claim is not supported experimentally. This could either be done by removing it from MeshGraphNet, or somehow add something similar to GCN.

Since GCN seems good at predicting static values but very quickly diverge for longer rollouts, the two tasks seem quite different. As noted in the Appendix, training noise was added to make the presented method robust to longer rollouts. A fair comparison with GCN would also include this.

Generalization is mostly demonstrated qualitatively : the difference between the training/testing scenarios are not stressed enough - apart for the impressive extrapolation case from Airfoil. For example, the generalisation on FishFlag remains qualitative, but is not quantitatively assessed, and similarly for the Windsock. “Visual similarity” is not 100% satisfying.

Recommandation

Weak accept: I recommend to accept the paper for the promising results it displays, but would like to see the missing quantitative results I mention above addressed - ablation study and experiments.

Questions

In grossly in decreasing order of relevance (first ones = useful to assess the paper / last ones = personal curiosity):
1 - Why was the super-resolution module of GCN removed?
2 - Varying wind speeds/directions on the flag experiment: since each time step is computed separately, to what extent does it really demonstrate generalisation to vary wind direction at each tilmestep, knowing that they were each independently seen during training?
3 - On the Windsock experiment demonstrating generalisation: how much is due to the architecture (convolutions that can by nature be applied to bigger graphs) and how much the network has “learned” the physics?
4 - In Table 1, what is causing t_full to be strictly larger than t_model for the FlagSimple experiment, where no graph re-meshing or re-computation is happening?
5 - What is the chosen time step for 1 prediction? Since it is fixed, how was it chosen?
6 - How are mesh space coordinates u_i’s attributed? Does it matter ?
7 - Does jointly predicting related physical values help? This is something usually observed in the literature, but did you observe similar results? For example in the CylinderFlow experiment you have one network predicting the change in momentum and the pressure field. Does it degrade RMSE to get 2 networks, one for each of these quantities?

Additional recommandations

Clarity could be improved. I make the following remarks, that did not influence on my rating of the paper:
Section 3.1 “Encoder” : I would split the encoding part into two, to clearly mark the difference between initial values provided to the network : (A) one is defined by physical quantities that you provide as input to the mesh vertices (you denote it q_i). The other (B) is more of an internal, method-specific pre-processing step and consists in creating world edges, and labelling world and mesh edges with their length. Presenting (A) and (B) in the same paragraph can be confusing, since they really are not on the same level of processing. (A) is the real, physical input, (B) is already a pre-processing step.
Section 3.1 “Processor” : why denote GraphNet blocks as P_i if it not mentioned anywhere else? Moreover, using _i as an index is confusing since it is elsewhere reserved to graph node indices.
History size h is mentioned in the introduction of 3.1, in 4 (FlagDynamic and SphereDynamic) and in Fig. 5d, but does not fit the formalism presented in 3.1. It should  be presented in a way that accommodates for the possibility of feeding more than just the previous step’s information.
Section 4: could be improved with regards to what is provided as input to the network. For example, u_i is always provided as input, so why repeat it? As far as I understand it, what matters is what you input as physical properties q_i, and what/how you label n_i. In this section q_i are well defined, but for some experiments I have trouble seeing what you encode into n_i - it is only explicated for DeformingPlate.
Section 4: all experiments and datasets are described except for FlagSimple. Is this an oversight?
Globally, clarifying the formalisation (3.1) and values (4) of the network’s inputs would help.

Typos or minor improvements:
3.2, top of page 5: “iff” > “if and only if”
End of A.1 page 13: “… is appended TOO the node” > “to the node”
review_title: Promising results but lack some ablations and quantitative evaluations

final_score: rating: 10: Top 5% of accepted papers, seminal paper
review_content: The paper proposes a generic strategy to accelerate time integration of physical systems, including elasticity, fluid simulation, and cloth simulation. The results are impressive not only for  the generality, but because this is the first paper that I read that shows a fair speedup combined with stability for such complex physical systems. I wish the authors attached the source code so that I could analyze it more closely to make sure that no shortcuts were taken in the evaluation.

Assuming that the evaluation is correct, this is a seminal paper that will allow to use data-driven methods in scientific computing applications.

Comments:
- the entire discussion about adaptive refinement seems out of place. Would the method also work without and is this an extra feature? Or is it necessary to use this feature to make it work? My understanding is that this is optional, and only helps, in particular for cloth. Can you confirm that this is the case?
- I do not understand the sentence: “Each output feature is then processed by a forward-Euler integrator with ∆t = 1”. It only becomes clear after reading section 4. Please clarify earlier
- Why is the model stable for hundreds of steps? Is this due to training with noise? Please do experiment to try to identify which feature makes it stable. Will it eventually break if you let the system run for thousands of frames
- while the learning baseline are clearly described, I do not understand how the timings are measured for the GT in table 1. Is every row computed with a different software, as listed in A.1? If so, are all of these using a single core or multiple? This should be clarified, especially since the machine learning results are run on a gpu, making the comparison not fair. Could you report also the timing for doing the prediction of the NN using a single CPU core, or alternative run the iterative solvers of the classical methods on the same GPU?
- Is there a case where you could notice that the integration is not stable? Maybe by reducing the noise in the training or by reducing the training data size or by simulating scenes far from the training data? It would be interesting to know what the limitations of the method are and where there is room for improvement.
- Could you add a simple nearest neighbor baseline?
- It would be interesting to add a comparison of running time *for a given accuracy* instead of simply showing what is the error in the table. This could be achieved running many simulations with analytic methods until they match the error of the data-driven model wrt a finely discretized ground truth.

Overall, my impression is that the results look amazingly good: it is the first paper I read that shows that data-driven method can beat standard scicomp approaches on complex scenes. The reason for the performance of  the methods likely lies in the combination of using mesh and space variables, plus making the integration step explicit and enriching the training with controlled noise. I wish the authors attached the source code of  their method and of baselines (and I encourage them to do it after publication), I look forward to experimenting with this method and test it on my applications. 

Update:
The revision further improved the paper and addressed most of my comments. I am still positive and voting for accepting this paper.
review_title: Possibly a game-changing paper for ML for scientific computing

Meta-review: The paper develops a methodology for using graph neural networks for mesh-based physics simulation. This extends recent work that focused on grids or particles to mesh-based domains, which are challenging due to irregular (and possibly changing) connectivity. The reviewers had some concerns but recognized that this is an important work that will be of broad interest and may have significant impact.
Decision: Accept (Spotlight)