Venue: ICLR 2022 Track Blog

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The blog post compared to the paper adds more information on related work and compares them in more depth.

However, to me the blog post didn't really add any insights that weren't already in the paper. It doesn't provide a much better intuition than the original paper as to why the technical innovation works and is significant. For instance, in the MeshGraphNet section it spends the majority of the space simply telling the read what MeshGraphNets do, rather than focusing on some intuitive mental model, which gets a brief and too vague mention in terms of the superposition principle. 

Possibly having a consistent specific example problem on which to explain the difference, e.g. showing that some interactions that weren't captured before are now captured would have been useful.

Overall, I simply felt I was reading a paper written in a blog post style rather than a blog post that gave additional insights or intuitions.
review_title: Not enough value added compared to the paper

final_score: rating: 4: Ok but not good enough - rejection
review_content: Note: the main reason for confidence score 4 rather than 5, is that while I am very familiar with the central models being discussed, I am not as familiar with the non-ML related literature, although I still believe that I am familiar enough to provide a fair assessment.

This blog attempts to do three things:



*   Explain the MeshGraphNets method.
*   Point out similarities and differences between the learned GNN-based simulators: GNS (particle-based) and MeshGraphNets (mesh-based) 
*   Make connections from these learned methods to more traditional approaches such as Molecular Dynamics (MD) and SPH

Although the methods discussed indeed share many similarities, and often try to solve the same problems; the blog often states strong claims without providing much scientific support (see example claims and arguments in the detailed review). While the rigor of a blogpost shouldn't be the same as that of a paper, as a reviewer I found myself disagreeing with many claims throughout the reading. Although I could see the motivation behind many of such claims, a few rounds of reviews might be needed in order to have information that is mostly correct or at least not too controversial. That is why I am leaning towards rejection. 

Also some of the technical details about the MeshGraphNet model are incorrect, and the explanation for GNNs is also a bit confusing (see detailed review).

So all in all, I think this is a good effort to make connections between these different approaches but the significance of the blog's added value gets blurred with the model inaccuracies, and with strong and possibly controversial claims, lacking evidence or more convincing arguments.  


### Claims that are too strong and need either rephrasing or some references.



1. (Numerical Methods…) "which are computationally expensive and require complete knowledge of the governing PDE and boundary conditions" → this is not true, equations can be approximations as well, certain parameters can be estimated etc. there is some general truth here, but sentence needs to be soften with words often, generally, etc
2. "However, achieving the same high accuracy of classical methods has proven to be challenging. A core reason for this gap in accuracy is the emphasis classical methods, such as Finite Elements, place on their mesh discretization." → missing references for "proven" to be challenging, and the core reason besides it (could be many other things, such as knowledge/use of the equations,  adaptive time stepping, convergence properties, etc). 
3. "We claim that molecular dynamics and smoothed particle hydrodynamics are the ancestors of all graph-based, learned particle simulators and show how graph-based approaches naturally extend to meshes." → later in section 'Fluid-particle model' the authors provide an example of how one relates the methods by given a set of points, constructing a Delaunay triangulation for the points, and considering that to be the mesh. Note that method wouldn't be suitable for a challenging problem (even in 2D), since representing the domain geometry and placement of mesh elements is key (also note that in 3D the Delaunay triangulation might result in skewed elements, what would interfere with the accuracy of the numerical numerical chosen. So this is a very strong claim, that doesn't hold in general, and needs to be written (it's okay to make the connections and for simple problems that's an ok reduction, but the claim is broader than that).  Also, in general, I find the similarities with MD a little bit far fetched? Even if it is true that MD was one of the first applications of GNNs, I believe most of the inductive biases / motivations behind the MeshGraphNets model can be written and explained in terms of 17th century physics (except perhaps for the fixed radius of connectivity, but it seems overkill to reference MD just to say that under some conditions close interactions are more important than far interactions)
4. "While the MeshGraphNets framework is very general, there are scenarios, such as when the future state of only a few chosen observables is desired, where other approaches would lead to better results." → needs stronger evidence/arguments
5. "The authors combine the conventional SU2 solver on a coarse grid with a graph network to achieve a significant reduction in computational costs. However, this model’s performance lacks behind MeshGraphNets on more complex benchmarks." → the work cited doesn't include direct comparison to MeshGraphNets on any benchmark – so it's unclear which method would be better on what
6. "What sets MeshGraphNets apart from these two paradigms is its independence from a latent space which has to fulfill a set of properties, as is the case for Hamiltonian Generative Networks, and Koopman Embeddings. Especially for Koopman embeddings, the question of finding the right coordinate system remains an open and unsolved challenge, making MeshGraphNets the more general framework for the time being." → This conclusion is very strong, since the methods are aiming for different goals and looking at different problems. In fact, providing similar powerful models that carry a latent space is possibly one of the most interesting directions, and does not make MeshGraphNets more general, but if anything, more restricted on its applicability to non-Markovian states.  
7. “Molecular dynamics” “most of their compute time on the computation of accelerations,”. There is a crucial difference here though, which is that MD computes instantaneous accelerations, while GNS computes the finite difference of the approximation of the accelerations which is optimal for the integrator used (in this case Euler). This is what makes GNS so successful: it can dramatically increase the time step size, because the model is already learning to compensate for any integration errors introduced by the low order integrator, which is something that MD does not do, and why MD benefits more from using higher order integrators and GNS/MeshGraphNets.
8. “Time extrapolation and training noise”. This comparison is very speculative. No evidence has been shown that the reason training noise is used to account for the fact that the state is sampled, and the explanation given by the authors is different (used to teach the model to stay within the training distribution). I would argue noise should not be needed for the reason claimed in the blog post because the model is not actually learning from real world data where the state may be sampled and incomplete but trained to approximate deterministic simulators for which the state is fully specified.


### Incorrect/Inaccurate claims about the MeshGraphNet/GNS model



1. "It learns an input-output map over the initial mesh, which can then be remeshed adaptively during rollout, to increase the mesh resolution in areas with large residuals"  → The output map will not be necessarily on the initial mesh. Something like "by using a GNN, the model learns to make predictions on the simulation mesh, and the framework allows adaptive mesh representation during rollouts, to increase…"
2. "In addition to introducing meshes, another important contribution of the MeshGraphNets paper is adaptive remeshing. In a nutshell, some of the output features of the model are devoted to learning the curvature, avoiding complicated conventional estimates of that quantity." → the model is not necessarily trying to learn the curvature but rather a sizing field that is indicative of the mesh resolution for each region. The sizing field controls the edge length of elements. For the cloth example curvature is part of the computation of the sizing field, but there are other estimates involved. E.g., the same technique could be applied to a flat domain in fluid simulation and use other quantities (such as gradients or distance to obstacle) to determine the desired target resolution. 
3. "Scaling MeshGraphNets (so far applied to 5k nodes max) to industrial applications with millions of nodes would require parallelizing, which is yet to be explored." → Although scaling is indeed a concern for GNN based applications for a large number of graph nodes, this paper did have examples with 20K nodes (generalization experiment), so this sentence needs correction
4. In the Notation section, there seems to be a confusion around graph nodes/edges and it's embedding representations. It is unclear what "contains MLP encoding" means when defining a variable.
5. "Processor: It’s a multilayer Graph Network. The exact number of layers is a hyperparameter; 5-6 worked well for this paper. " → the main models use 10 (GNS) and 15 (MeshGraphNets) message passing steps
6. "it seems like SPH is the least useful method because it requires physical knowledge and cannot include meshes." → least useful is a very strong claim, since SPH will be the preferred method in many cases (for example, if you want guaranteed convergence properties, or you have a case where a GNN wouldn't scale). Also not sure if the 4 methods are really comparable at that level, as they solve different problems (particle states vs mesh states, or pure forward dynamics vs potentials of conservative systems).
7. "However, the authors of the MeshGraphNets paper found out that superimposing the solutions computed on both edge sets EM and EW separately is more powerful than training the model to implicitly distinguish the origin of the interaction." → I don’t think this was explicitly claimed on the MeshGraphNets paper.
8. “Spatial derivative, e.g. velocity, acceleration.” Those are temporal derivatives of positions, not spatial derivatives, or what do you mean?
9. “We want to briefly discuss the GNs algorithm as used in MeshGraphNets”, I believe, both the algorithm and code, is incomplete. For MeshGraphNets, there are separate message aggregations for world and mesh edges which are then concatenated, and input together a single node edge update. And this is repeated at each step of message passing (https://github.com/deepmind/deepmind-research/blob/1642ae3499c8d1135ec6fe620a68911091dd25ef/meshgraphnets/core\_model.py#L60)
10. “Processor: Same as GNS”. because of the previous reason this statement is not true.
11. “but they do not let both edge sets interact during one rollout step” I believe this is incorrect. As indicated in the previous points the world and mesh edges interact at every message passing step via the node update, and each rollout step contains up to 15 message passing steps, so they interact 15 times.
12. “Looking at the main part of the table, i.e. Encoder-Processor-Decoder, we clearly see that the job of the Graph Network is to approximate the gradient of the potential P in a latent space described by the encoder and decoder. “ Not sure I agree. First: the gradients of the potential are the instantaneous accelerations, which is not what the model predicts. Second: you can interpret Newton second law without the need for involving potentials just forces. Third: It is not always possible to write a potential easily (e.g. non-conservative systems). I think it is ok to draw an analogy, but “clearly see” is too strong of a claim, as what the GraphNetwork does is strictly more general than approximating the gradients of a potential.
13. “The GNS paper faces problems with long rollouts - the longest rollout increase is demonstrated on the Ramps-Large example with ~8.5x extrapolation” The fact that GNS does not show longer rollouts does not mean it could not rollout for longer on particle systems. What makes the biggest difference is that, when modeling a mesh, MeshGraphNets has access to rest distances at all steps, so it can correct back to that, while this information was not provided for boxes in GNS (E.g. in GNS the box could have treated as a mesh, rather than as a set of particles, also given rest distances for box-box edges, but that was not the case). But for pure particle systems, GNS should be as stable as MeshGraphNets, as it uses the same noise techniques.


### Suggestions and minor comments



1. "Deep Learning is especially well suited for problems, in which the physics (PDEs and boundary conditions)..." → remove first , 
2. "Until DeepMind proposed the MeshGrapNets framework"  → Until Pfaff et al. proposed
3. 'Basic Graph Networks' section needs a bit of refining, extra references and making the definitions more accurate, e.g, "It defines a class of functions that take a graph as input, perform computations such as convolution, and again output a graph." → doesn't mention that GNN's are neural networks. 
4. Including a bit more background on the basics of Molecular Dynamics would be helpful for understanding the bigger picture
5. Depending on the target audience perhaps more visuals in the intro, with examples of meshes.
6. “Strengths” One very important missing strength is the potential for faster inference time than non-learned simulators.
7. “Weaknesses” “Long training times and many model parameters”, this does not mean much without comparing it to something.
8. I feel the “Physical biases” section belongs a bit more to the “MeshGraphNets and its Predecessor” section since those are the pieces of work that made an emphasis into how GNNs can be used to capture those very well.
9. “Some followup works to MeshGraphNets extending …” Consider including https://arxiv.org/abs/2202.00728
10.  “Now, as you more or less know how MD and SPH work, let’s compare them to GNS!” Actually both “Molecular Dynamics” and “SPH” sections, which are presented before this sentence, already have full paragraphs comparing them to GNS.
11. I would recommend adding more explicit references to “Basics: Graph Networks”, for example, Interactions Networks, MPNNs, GCNs..
12. “Update section”, perhaps mention that the target acceleration used in the loss is computed as the inverse of the Euler update equation (unlike on sph, and MD, where the acceleration is the instantaneous acceleration). This subtlety is critical for the model not to be limited by the accuracy given by a low order integrator.
13. “The geometric relations, excluding the neighbors lists, are included for MD in the potential.“  Could you define what you consider “geometric relations” . Not sure I follow why they are included in the MD potential, but not included in GNS, but included in the MeshGraphNets mesh.
14. Consider “deep learning” instead of “Deep Learning”.
15. Consider mentioning the predecessor work GNS at the top this, since a large fraction of the blogpost is dedicated to that.
16. Notation: separate u\_ij, x\_ij, into different rows, defined after u\_i, x\_i.
17. Notation: I think separating the mesh from the graph notation is fine, but then in the Mesh notation “Set of graph nodes” should be changed to “Set of mesh vertices”, and in the Graph notation you should probably also reference an adjacency matrix.
18. Notation: “MLPs used to… in each GN layer.” consider adding a layer index to the notation.
19. “The very same line of reasoning is employed in the mesh refinement of the MeshGraphNets algorithm” Missing ending full stop at the end.
review_title: Good effort to relate MeshGraphNets to previous literature, but lacks precision

Decision: Reject