Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper considers dynamic modifications of a GNN, where nodes are inserted, node features are augmented and edges are modified, near the boundary of a triangulated surface mesh, using heuristics. The main intended application area is that of simulating granular flows, which are a challenge to model analytically. Experimental results are shown, comparing the boundary graph GNN outputs with simulations obtained using a compute heavy discrete element method, for two cases of interest in industrial machine design: hoppers and rotating drums. The experimental results include both qualitative illustrations and animations and quantitative comparisons (mixing entropies and flow positions).
main_review: Strengths:
- The paper is well written and fairly easy to follow and their is sufficient detail in the supplementary material to suggest that reproducibility is not an issue.
- The simulation results and their presentation (both the discrete element method used as ground truth and the simulation of the boundary graph GNN) are well done.
- With regard to the simulation community this method could have practical value since it doesn't require analytical modeling. How easily it could be adapted to other types of machinery though is not entirely clear. The claim that the heuristic method matches ground truth leaves me feeling a bit uneasy though, given the lack of a theoretical analysis or model. Perhaps this is just intractable for granular flows.

Weaknesses:
- A significant weakness in my opinion is the somewhat incremental methodological contribution. Whereas I can accept the practical value of boundary modifications for the application in mind, the idea of adding virtual nodes to a simulation based on a heuristic ball radius, and the consideration of additional node features (as detailed in Section 3) is modest as a contribution to either a "representation" or the "learning" of it, which is the general requirement of ICLR.
- The message passing to update node embeddings seems to be directly based on past work. The specifics of what these messages contain (the new node features, i.e., beyond triangle inclination) are difficult to tease out. Computational considerations, such as how additional coordinate locations are chosen to minimize distance between boundary points and "real" nodes are not addressed. There's an attempt to illustrate the idea in Fig. 2 but it falls short of an algorithm and lacks a complexity analysis. 
- Much of Section 4, which discusses further details of how the graph is made "dynamic" appears to be a coverage of implementation issues. Here the additional features features are listed: 1) real or virtual particle, 2) normal vectors but how these features are related to the physics of granular flow is not discussed at any length.
- I found the title to be a bit misleading. While the paper agues for a dynamic modification of a GNN at its boundary for 3D simulations, the experiments and the particular instantiation appear to be almost entirely for granular flow.
summary_of_the_review: Overall, whereas the ideas are easy to follow and the illustrations are well done and the experimental results demonstrate a match between boundary graph GNN outputs and compute heavy simulations (Figs 4 - 6) the contribution of this paper in terms of new methods or representations or representation learning are modest with regard to where the bar is, in my view, for ICLR. I do not doubt that the method has practical value, and clearly a lot of work went into organizing the experiments, conducting them, and putting together the results. Perhaps this paper would be a better fit to a simulation conference or one with a more engineering oriented focus. The idea to dynamically alter graph structure - and as far as I could tell there are not too many other methodological innovations - is a bit thin with regard to novel content. I'm also puzzled by the choice of title - the choice "3D simulations" appears to be quite general, and it is not adequately supported by the paper's content (the focus on granular flows).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents BGNN to dynamically model geometric boundaries for high quality 3D simulation, with the particular focus on the granular flow simulation. For a geometry represented as a 3D surface mesh, the BGNN updates the edges and insert virtual nodes to allow an accurate modeling of particle interactions with triangularised boundaries. 
main_review: Strengths:

1. The paper proposes a solution to a less studied but important issue. To this end, it is potentially of broad interest in the community. 

2. The BGNN seems efficient in that every time it just insert one virtual particle due to its specific mechanism while some other SOTA methods for GNN-based 3D simulation have to calculate/update the positions of several particles. 

3. Experimental results demonstrate that BGNN works well on the challenging 3D granular flow processes of hoppers and rotating drums.

Weaknesses:

1. The construction of the BGNN largely follows the framework proposed by Sanchez-Gonzalez et al. (ICML’20), which makes the technical novelty of the work a bit limited.

2. The proposed method is merely tested on two the hoppers and the rotating drums. More diverse geometries should have been considered for experiments to demonstrate the generalisation capability of the BGNN. And there is no comparison with previous methods such as Li et al., 2018, Ummenhofer et al., 2019, Sanchez-Gonzalez et al., 2020 and Pfaff et al.,2020.

3. For Equation 9, the detail of the aggregation is not provided. At least some analysis and ideally an ablation study for a mean or max operation should be presented for this vital step.

In addition, there also exist some minor issues:

1. The title of the paper is inappropriate as the term '3D simulation' is exremely general. Perhaps replace it with '3D granular flow simulation'.

2. Figure 2, as the illustration of one of the key ideas of the paper, is poorly designed and barely informative. It is expected to illustrate how the proposed BGNN dynamically modifies and enhances the graph structure to include boundaries, rather than just exhibits the static local configuration at one time step.

summary_of_the_review: Based on the strengths and weaknesses of the paper listed above, I tend to take a slightly negative stance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes BGNNs, which are an extension to existing GNNs for particle simulation (e.g. Li et al, Ummenhofer et al, Sanchez-Gonzalez et al), aimed at improving the efficiency of the model when particles are near boundaries with complex triangular geometries.

Specifically, the authors propose to parametrize triangular boundaries such that, for any "real" particle/node close to any triangle of the boundary surface, a new "virtual" particle/node is added at the intersection between the boundary surface and the normal between the real particle and the surface. Furthermore this "virtual" particle/node has additional features such as the normal vector to the surface. The authors then implement the model in two geometries (drum and hopper) for two types of simulation (cohesive and non cohesive particles).

The main claims given the model are:

C1. This approach to describe boundaries is more efficient than those approaches used in prior work.

C2. The model makes predictions for the chosen domains that match quantities of interest to the engineering problem, for each simulation type, and can generalize to larger systems at test time.

C3. The model can be faster than state of the art hardcoded simulation methods.
main_review: Strengths:

S1: The proposed approach sounds like a very plausible way to encode triangularized boundary surfaces as nodes and edges in a graph. 

S2: The idea is very simple, and can be combined in a very straightforward way with existing models, without any detrimental effects to them (e.g. models can still learn a variety of dynamics, generalize to large systems, etc.).

Weaknesses:

W1: The technical contribution of this paper is sort of a "featurization" contribution, rather than a "modelling" contribution, e.g. it focuses on the problem of how to build an input graph that you can give to a GNN for a particle simulation with triangular boundaries. While this is the first paper (that I know off) that focuses on that specifically, some of the previous papers were already using sort of similar strategies. For example, Li et al. were already using a single particle to represent each wall in the container (rather than multiple), also adding edges according to the normal/closest distance between the surface and the particles (which does not present the scalability problems indicated by the authors). Ummenhofer et al. also used normals as boundary node features. Sanchez-Gonzalez et. al, included clipped normal distances to walls as additional node features to account for the cubic container to avoid the need to use large numbers of particles for the container walls. The approach proposed in this paper unifies all of those approaches in a sensible and general way, however, the technical novelty feels perhaps a bit too incremental. For comparison, in those 3 previous baselines, the choice of how to build the input graph is sort of a minor implementation detail compared to other contributions, while in this case, the paper focuses solely on that encoding detail.

W2: The paper does not compare against any of those previous baselines using alternatives ways to encode boundaries (e.g. approaches that model surfaces as dense layers of particles). This is probably fine since the authors are not trying to claim that the model accuracy is better at modeling the dynamics, just that it is more efficient as it can do the same with less edges and nodes for the boundaries (and you don't need to actually run a baseline to believe that less nodes and edges will yield a faster model). However, it does take a bit away from the experimental results from the paper, since it probably indicates many of the previous baselines can also model those domains to similar accuracy, and achieve similar flow, mixing entropy and generalization performance.

W3: Authors claim that the model can be faster than state of the art simulation methods. I believe this is probably true, but I feel it is not very well supported as it is a very small part of the paper. E.g. I am not sure if it is fair to say that BGNNs are non-optimized for GPUs, since neural networks are sort of optimized for GPUs.

Other comments:

O1: The description of the contribution could be a bit more clear. If I understood correctly, the idea is quite simple: For every pair of real particle and boundary triangle that are close, add an extra virtual particle at the closest point of the boundary triangle, including the normal of the boundary as an extra feature. Once you have added those particles, just apply *almost* the same model as in the baselines on teh resulting graph. Currently the description of this takes two sections and almost 3 pages (middle of page 3 to near-top of page 6), and there is quite a bit of redundant info between section 3 and 4. I think the paper could benefit from a simpler and shorter description. Or perhaps I am missing something about the methods, in which case I apologize.

O2: Some of the math seems missing some descriptions and parts. For example I believe m_ij in equation 4 and 9 are not defined anywhere. Also, I believe the paper does not currently indicate which of the node features p_i and x_i are used to build the initial set of node embedded features h_i. I assumed absolute positions x_i are *not* used to compute the embedding and just used to build the relative positional features via subtraction at the edges (so the model is translation equivariant), but I am not sure whether absolute positions are used as features or not is clarified anywhere.

O3: Table 2, it is unclear what the time steps column indicates. At first glance it seems to read that BGNN takes 158s to run 100 timesteps, while LIGGGHTS takes 356s to run 1 timestep. Consider renaming the columns to "timesteps/per step", "step size" or something like that, and "full trajectory wall clock time".
summary_of_the_review: The paper proposal for how to create an input graph for a particle simulation interacting with complex geometries is plausible. However, considering how large overlap with prior work there is, except for the "graph encoder" part of the model, I am not sure if this contribution on its own is large enough to meet the ICLR bar. Specifically, I am not sure the paper will bring many new insights to the average ICLR attendee, beyond seeing it as an implementation detail for existing models. In this sense I am borderline between weak accept and weak reject.

I would be much more inclined towards a stronger accept if the focus was a bit more applied:
1. Here's an actual application, with realistic data, that engineers really care about, and also here's a specific set of generalization settings that engineers really care about.
2. Here's how you adapt an existing method (e.g. one of Li et al, Ummenhofer et al, Sanchez-Gonzalez et al, Pfaff et al.) to this problem including changes to the encoder to avoid wasting compute on large regions consisting only of boundaries, which ideally really matters if you want to get good running times in that application.
3. Here's a detailed study of why engineers in those fields should prefer the learned model over other state of the art non-learned model. Including plausibility of generating a training dataset of the required size, limits of generalization, running time, trade offs, etc.

Of course, it may be that a different venue could be more appropriate for a paper like that. Although as an ML conference attendee, I would personally be interested on the specifics of any evidence of the promise of end-to-end learned models replacing traditional simulators in any domain.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a novel contribution to the use of Graph Neural Networks (GNN) in the simulation of 3D physical phenomena. This contribution consists of the optimization in the calculation of the interactions of the particles with the contour surfaces that allows to optimize the need to add nodes in these limits. The results show an acceleration of the simulation calculation without affecting the accuracy of the result. 
main_review: The paper is very interesting and well written. Despite its difficulty, it is easy to understand. The background and related work are quite complete and enough to understand the paper. However, there are some parts of the paper that should be explained better to improve its understandability and reproducibility:

1.- The input to the GNN are node coordinates and features but, which kind of features are we talking about?

2.- As commented in point 2, the GNN is composed of 1 encoder, the core message passing layers and the read-out layers. Authors do not specify the configuration of the 5 core-message passing layers nor encoder and read-out layers.

3.- mij appears in (4) and (9) but authors do not explain its meaning. It could be the message passing of Gilmer et al 2017. However, in this case, the mij is a linear combination of the neighbors embedings that have already introduced in both formulas. It would follow the Gilmer proposal if it was eij instead of mij. Please, this part must be explained better.

4.- In page 7, the authors state that "We use 5 message passing layers with 128 and 512 nodes for intermediate node and edge representation". But, what structure does it has?. This is the first time that authors talk about using edges in the GNN model. Which information do the edges have? How are their embedings built?

5.- In the results, the authors comment that they introduce a 25% of variability in the number of particles but do not justify this decision.

6.- A comparison with Sanchez-Gonzalez 2020 would be interesting to show the limits of the improvement introduced. 
summary_of_the_review: Pros: 
1.- A very interesting contribution to GNN applied to 3D simulation.
2.- It could be a first step towards using GNNs in complex mechanical and biomechanical simulation.
3.- The increase in speed without affecting too much the precision is a very interesting contribution.

Cons:
1.- The model must be explained better for easier understanding (check the main review).
2.- It is not clear the differences between the base GNN model (without the dynamic boundary nodes/edges addition) and the one proposed by Sanchez-Gonzalez 2020. (Maybe because cons 1).
3.- Lack the comparison with other similar approximations.
review_title: null

Meta-review: This paper describes means of incorporating boundary conditions into graph neural networks used for simulation.  Assorted techniques dynamically adjust computations near a boundary.  Reviewers agreed this work is well written but disagreed regarding whether the scope of the contribution (and application, which focuses mainly on granular flow despite the title of the paper) merits publication at a top conference like ICLR.

The authors rebut the claims of limited scope strongly, but the experiments in the paper somewhat belie the claims of broad scope.  Some comparisons are also missing to Sanchez-Gonzalez et al. 2020, which appears to be closely related.

The AC agrees that the scope of the work and contribution here have not met the bar for publication.  Reviewer QRXr has some thoughtful suggestions for ways to improve this work in future submissions, or sharing with an audience that can better appreciate the application-oriented contributions might be a reasonable direction to take.
Decision: Reject