Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a neural PDE solver based on adn encode-process-deode architecture that respects boundary conditions thanks to a novel GNN-based gradient operator. Other than the proposed version of an E(n)-equivariant GNN nonlinear solver they also propose a different encoding process for boundary condition treatments in the encoded space. Experiments comprise prediction of the gradient field from a given scalar field to verify the expressiveness of the proposed version of the GNN-based gradient operator; and the task of learning incompressible flow. Other than respecting boundary conditions by construction, results show important improvement with respect to the state of the art neural PDE solvers.
strengths_and_weaknesses: The idea seems to be very simple and based on existing methods but is effective. The connection to global pooling in computing the step size $\alpha$ of the Barzilai–Borwein method is also interesting and perhaps deserves some more discussions. The paper is sometimes hard to read. And not very clear.
e.g.,
3.2 Where it is explained that the weight should be kept small to 'respect' information in the neighborhood.
3.2 Doesn't explain how the model can be generalized to vectors or higher rank tensors. (edited)

Moreover, in the paper it doesn't really explain how experiments are performed so I assume the reader is often very familiar with this kind of experiments to test neural pde solvers otherwise they need toexplain it better. Especially how GNN gradient operators predict gradients. There are some more explanations in the Appendix but overall the description remains unclear.
questions: Can you elaborate more on the Dirichlet encoder-decoder? I do not understand whether it is as simple as it looks or perhaps there is something more. My understanding is that the encoder distinguish betweenboundary nodes and nodes that are not on the boundary and just apply a the pseudoinverse transformation on boundary nodes for decoding. In this case I don’t understand its role in performing better predictions other than just trivially enforcing boundary conditions at the end of the processing.
limitations: Authors address potential societal impact of their work and mention the fact that the properties of the model limit its applicability domain. They mention that the proposed method is not suitable for solving inverse problems but don't elaborate much on that. The improvement with respect to the state of the art (here Brandstetter et al.) is significant. Perhaps a a few simple experiments similar to those in Brandstetter et al would give more information on whether improvement in predictions is really due to the constraints enforced by the proposed architecture.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work proposed to use E(n)-Equivariant Graph Neural Network to solve PDEs. Main differences compared to existing works include 1) using a boundary node encoder and pseudoinverse decoder to enforce boundary conditions and achieve better long-term accuracy. 2) embed the PDE inside the model to achieve better long-term accuracies, via the proposed NeumannIsoGCN
strengths_and_weaknesses: ***novelty*** The formulation is new and the global encoding and embedded PDE inside the GNN is novel, to the best of my knowledge. 

 ***quality and clarity*** The work is well written and well-motivated with good background introductions and well-organized experimental results. The accuracy speed trade-off plot is interesting.

***significant*** The work demonstrates their work on solving an important task.

questions: - in eq) 17, does $i$ refer to time steps, or gradient descent steps? If it refers to # of time steps, does propose approach generalize dynamic simulation of arbitrary timesteps?  
limitations: My primary concern is also one point that I don't quite understand if the $i$ in eq. 17 refer to time steps. If so, how could it generalize to potentially longer time scales?
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors’ primary contribution is the development of neural network layers that enable the treatment of Dirichlet- and Neumann-type boundary conditions in encoded space (the “boundary encoder”, “Dirichlet layer”, “NeumannIsoGCN layer” and the “pseudoinverse decoder”). For Dirichlet-type BCs, they accomplish this by constructing a decoder design such that the entire model will learn to approximate the identity function at the Dirichlet portion of the boundary (the “Dirichlet layer” and “pseudoinverse decoder”). The authors deal with Neumann-type BCs by extending the IsoGCN layer from the prior art to include terms that convert the Neumann BC into a penalty/constraint embedded into the architecture of the layer (“NeumannIsoGCN”).

A secondary contribution of the paper is the augmentation of a neural nonlinear solver where, by computing the “optimal step size” for a gradient descent step, they achieve a filter analogous to global pooling. This aids the neural model in learning highly nonlinear dynamics, such as exhibited by the incompressible Navier-Stokes equations.

These constructions are supported both by experiments and comparisons with the prior art as well as an ablation study to show the merit of each proposed component.

strengths_and_weaknesses: Strengths

* The paper is generally sound and the proposed model indeed outperforms the prior art model under comparison.

* The model achieves zero error on the Dirichlet boundary, as claimed.

* The ablation study can serve to show that each proposed component indeed contributes to the performance of the PENN model.

* Also of note is Figure 1 which presents an overview of the main contributions in a clean, legible and aesthetically pleasing way.

Weaknesses

* The title of the work appears to be slightly misleading in the sense that E(n)-equivariance stems from the work of Horie et al (2021). and not from the current work. I would suggest choosing a more descriptive title, such as “Physics-Embedded Neural Networks for Nonlinear Dynamics with Mixed Boundary Conditions”.

* I cannot find any information on the actual network architecture that is used. How is the domain encoded into the network? How flexible can the domain be? How about supporting heterogeneous coefficients?  It seems like the authors refer the reader to previous works, but I would expect this info to be present at the current paper or at least in the supplementary material. Also, from a brief look I couldn't find the architecture in the authors' code (but maybe I'm wrong). In any case - I do not see how one can replicate the results in this work.

* I would expect more comparisons to other NNs and classical solvers, particularly FEM/FVM and AMG-based solvers. The authors claim that they have tested their model against classical solvers, but I found no evidence of this in the main paper or supplementary PDF.

* I find it difficult to trust comparisons that do not detail metrics such as a number of trainable parameters or FLOPs. As it currently stands, I am unable to judge the tradeoff I would be making by using PENN as opposed to MP-PDE, for example. Consider adding these metrics as their absence makes true comparison very difficult. Figure 4 only partially addresses this concern but it does not present any speed/accuracy tradeoff for PENN (like it does for OpenFOAM and MP-PDE) and any speed/accuracy/parameter-count tradeoff is not considered at all.

questions: 1) The discussion of PINNs in section 2.2.1 strikes me as unrelated to the current work, or at least the connection isn’t very clear to me. I would advise the authors to clarify the limitation of PINNs they are discussing and make the connection or contrast with the current work explicit.

2) The same, but to a lesser degree, for GNNs in section 2.2.2. Please clarify the difference to the current work.

3) It seems to me that the main paper omits rather important details that are given in the first paragraph of section C.3 (lines 461-469 in the supplementary PDF). I realize that the page limit makes it difficult to include all the information, but consider finding a place for some or all of this information in the main paper.

4) How does the proposed method perform on domains that were not seen in the training set?

limitations: No limitations.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors proposed a novel design, building on top of IcoGCN to add two types of boundary conditions to the solution of the solver, Dirichlet conditions constraining the values of the functions at the boundaries, and Neumann conditions, constraining the gradients of the values at the boundaries. By leveraging the E(n) equivariant properties of the IcoGCN backbone design, the authors demonstrated improved performance compared to SOTA baselines such as MP-PDE, in particular with respect to translation and rotation of the domain. Compared to the original IcoGCN, the authors demonstrated that the proposed method is able to strictly satisfy Dirichlet and Neumann conditions, leading to improved accuracies of the solutions.
strengths_and_weaknesses: Strengths:
* Embedding the physical constraints into the design of the model itself by combining the PDE solution process with the model is an elegant and more generalizable approach to enforcing physical constraints, compared to simply using a loss penalty as an auxiliary loss only during training time.
* By incorporating the IcoGCN backbone, the authors are able to demonstrate translational and rotational equivariance of the final model, which is a very strong and desirable property missing in many works in the physics informed machine learning literature.
* The experimental evaluations are compelling. Not only did the authors compare with two reasonable baselines to demonstrate improved enforcement of boundary conditions and rotation/translation invariance, they also performed a runtime analysis between speed and accuracy tradeoffs, even considering the OpenFoam as a baseline. This is not too commonly evaluated among physics informed ML literature and I am happy to see it included in this work.
* The paper is clear and well written. In particular, I enjoyed the writeup for the backgrounds section, which doesn't assume too much prior knowledge on the subject matter and is easy to follow.

Weaknesses:
* Though the authors branded the novelty of this work on the E(n)-Equivariant properties of the model, it is not a unique contribution from this work since it is based off of the IcoGCN work which this work used as a backbone. Stressing this property (even in the title) feels like an oversell to me.
* Though the overall accuracy / runtime tradeoff for the proposed model is compelling (Fig. 4), I would like to see some thoughtful redesign of the model to allow flexibly adjusting accuracy / runtime also for this learned model. One suggestion is to use the results from this model to initialize a coupled PDE solver, so that with a good guess leveraging data prior, the model can achieve the same guaranteed accuracy at a faster speed.

questions: One main thing I am trying to understand more deeply is how the proposed algorithm combines the first-principle physics with data priors. One way is to predict the results only using data prior, perhaps adding physics equations as an auxiliary loss (e.g., PINNs), which requires lots of training data, but leads to poor generalization. Another extreme is to only solve from the first principles, given as PDEs, using a PDE solver (as is the case in OpenFOAM), which requires no training data, but has very good generalization. In this work, the model does seem to require training on a given dataset (albeit a small dataset of only 203 examples), yet solves for a PDE using the Neural nonlinear solver in Sec 3.3. Please help me understand how the learned data prior is incorporated in this process? 

Nit: Eqn (19) in the appendix has an extra parenthesis.
limitations: For the experimental evaluations, it would be interesting to see how the model generalizes to a different range of physical parameters, such as Reynolds number. Such evaluations seems to be lacking in the experiments.
review_title: null

Meta-review: metareview: The paper proposes a E(n)-equivariant neural PDE solvers that can satisfy boundary conditions provably. The reviewers acknowledged the importance of the studied problem setting and generally appreciated the results. The paper is nicely written and provides both strong experimental results and theory. Indeed, a range of interesting experiments demonstrate the effectiveness of the proposed method. I want to thank the authors for their detailed responses that helped in answering some of the reviewers' questions. (The reviewers have provided detailed feedback in their reviews, and we strongly encourage the authors to incorporate this feedback when preparing a revised version of the paper.) In summary, this paper is a clear accept. Well done!
confidence: Certain
Decision: Accept