Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents a neural network model for solving steady-state PDEs on regular grids. The proposed approach is rooted in the principles of Domain Decomposition, which consists of two major steps: solving PDEs on subdomains in parallel and synchronizing the solutions across the subdomain boundaries, i.e., geometric interfaces partitioning the whole domain. The proposed method makes heavy use of encoders and decoders to solve PDEs in each subdomain. On top of that, the method uses fixed-point iterations to stitch these solutions together.

Roughly speaking, the proposed method can be seen as a deep-learning version of Domain Decomposition solved in a learned latent space. 

strengths_and_weaknesses: Strengths:
1. This paper shows a respectful attempt to combine a classic numerical method with modern deep-learning techniques.
2. The paper presents lots of experiments to evaluate the proposed approach.

Weaknesses:
1. The description of the technical method is vague without rigorous mathematical details. Part of the approach is still mysterious to me.
2. The method and experiments did not reveal much insight from drawing the analogy between Domain Decomposition and neural networks.
I will elaborate on them in the next section.

Some small comments about the figures:
1. It is probably better to place Fig. 2 on page 4.
2. I have found Fig. 19 in the supplemental material helpful. I suggest putting a simplified version of it in the main text if space permits.

questions: I actually like the general idea in this paper, but I think many critical technical contents are described quite vaguely and are difficult to evaluate.

**Problem definition**
1. The proposed approach is tied to a specific spatial discretization scheme (Cartesian grids). While using grids is definitely OK, it should be clearly stated earlier in the paper, e.g., acknowledging it as an assumption in your problem definition.
2. The related work section should also briefly mention papers like GNNs or finite-element methods using triangle or hex meshes so that readers are aware of other spatial discretization schemes.
3. Eqn. (1) is an incomplete definition of PDEs without stating their boundary conditions.
4. The discretization scheme lacks details and misses some subtle points, e.g., how is the smooth geometric surface in the car example handled using your regular grid?
5. Similarly, the concept of “PDE conditions” is also vague and refers to various things in different problems (Sec. A.1.1: “source” in Poisson equation, “geometry” in RANS, physical parameters in other problems, etc.). It is probably better to rename “PDE conditions” so that readers like me don’t mistakenly think it refers to boundary conditions.

**Technical method**
1. Lines 112-114: “Iterative solution algorithms…at convergence.” This statement is specifically for Domain Decomposition, but the context seems to imply all PDE solvers fall into this pattern. Perhaps better to clarify it?
2. Some background knowledge about flux conservation is probably necessary.
3. Overall, I feel Sec. 2.2 and 2.3 lack enough details for people to reproduce the method. The Laplacian example (Eqn. 5) does not help to clarify the method (Eqn. 5 isn’t more informative than Fig 2 or Alg 1). A concrete, step-by-step description of running your approach on a sample problem (Poisson or Laplacian) will be much appreciated.

**Experiments**
While I appreciate the tremendous efforts this paper puts into experiments, I feel many of them are repetitive. Experiments that can tell us what makes combining Domain Decomposition and neural networks a good idea are rare in the paper. One experiment that I really like is Fig. 3C because
- Domain Decomposition is designed for solving large PDE problems due to its superiority in time/spatial complexity;
- Therefore, we have a good reason to hypothesize the proposed method should scale up to resolutions unseen by previous neural network methods;
- Fig. 3C confirms this point with convincing results.

However, other experiments are not very informative to me, although it definitely does not hurt to have them.
- Just like I don’t expect Domain Decomposition to solve PDEs more *accurately* than other traditional methods (CG/multigrid without splitting the domain), I don’t see a strong reason to believe the proposed approach could solve PDEs more accurately than other network models, yet Fig. 3A-3C shows your method achieves smaller MAE than all other baselines across the board. What component in your method makes this happen? Are baselines carefully tuned? I would prefer to see an experiment analyzing it more deeply, e.g., by looking into the differences between your network architecture and that in the baselines.
- Another experiment that I would like to see is to study the influence of latent vector dimensions. As a thought experiment, if we set the latent vector dim = input dim and enforce encoder = identity, does it reduce your method to the standard Domain Decomposition method? By varying the latent vector dim, I would like to understand how it pushes your approach away from the standard Domain Decomposition approach. 

limitations: **Limitations**: Sort of. I think the paper should elaborate more on its limitation (See my comments above), but I can understand it may be difficult due to the page limits and won’t hold it against the paper.

I really appreciate that the authors put “steady-state” in the title to reveal the scope of the PDEs they consider in the first place.

**Potential negative societal impact**: N/A.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes an algorithm to solve PDEs via an iterative procedure that uses a learned solver for a subdomain. Iterations on multiple of these inferred solutions are performed to obtain a global solution. As such the method represents a hybrid approach between more traditional iterations and learned solvers, with the main goal of achieving solvers for high resolutions without having to re-train models each time. The paper focuses on steady state problems, which limits the scope but at the same time represents a meaningful starting point.

strengths_and_weaknesses: The method is presented in terms of domain subdivisions, with a flux-based coupling for the interfaces between the rectangular subdomains. The iterative solve is performed with an encoded latent space representation for the local solutions. The resulting algorithm is quite straight forward, but the "convergence" condition could be explained a bit more, I believe. This simply seems to be a delta of the latent space vectors, i.e., not a convergence to a true or reference solution, but only one in terms of satisfying the states between neighbors (there's no "residual" or other metric for the overall quality of the solution here).

The first example shown is, unfortunately, directly one of the less convincing ones. The wake behind the "car" and the boundary layer show some clear differences, and the learned solutions seems to have stronger axis aligned artifacts which I guess stem from the piece-wise solutions of the NN. It's good to see a 3D case, though.

The other ablations are interesting, and nicely illustrate the properties of the algorithm. I didn't find the de-noising too convincing, though - this is a classic task for autoencoders, but not a typical issue for numerical solvers.

One aspect that I didnt understand was highlighting that the training is "unsupervised". As explained in the appendix, the training inherently needs many precomputed solutions. This is misleading: unsupervised implies that solutions are not precomputed. Please clarify in the main paper, that the autoencoders are trained with collections of pre-computed sample solutions.

Overall, I think the paper presents an interesting combination of learned and traditional solvers, and combinations of the form proposed here have not been studied a lot. It's good to see how well these piece-wise autoencoders work, and I'm sure the authors can iron out the smaller issues above in a final revision.

questions: One aspect that I didnt understand was highlighting that the training is "unsupervised". As explained in the appendix, the training inherently needs many precomputed solutions. This seems misleading to me, unsupervised implies that solutions are not precomputed, or did I overlook something here?

limitations: I think these are covered quite well via the ablations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a method to efficiently perform neural simulation of a high-resolution PDE, by dividing the simulation domain into patches, encoding per-patch latents, and learning the simulation on a stencil of neighboring latents.
strengths_and_weaknesses: The paper differentiates itself from the common way of learning PDEs by (1) patch-based encoding and (2) iterative application of a local kernel to solve a steady-state problem.
I think the paper's biggest strength is using (1) to solve systems which are otherwise too big to ingest into a NN model; while there are lots of related methods (e.g. GNN-based physics models like MeshGraphNets also operate on local elements to improve generalization), I haven't really seen demonstrations on really big systems like 256^3.
There's quite a few methods which use iterative updates to solve PDEs (e.g. the class of methods surrounding Bai et al.'s Deep Equilibrium Models, or Rubanova et al. Constraint-based graph network simulator), although in a slight different setting. This aspect of the paper however feels very understudied to me (see questions below). 
More broadly, while the paper shows plenty of experiments and ablations, I found it very hard to distill concrete findings out of it. It's e.g. unclear what part of the model's performance are due to (1) or (2), and due to vague writing it's unclear how exactly experiments and comparisons were performed. The same is true for the method; it introduces lots of similar-looking symbols which are not properly defined (e.g. how are boundaries and source terms encoded and processed), and the training procedure was very unclear (see questions).
Finally, I'm a bit suspicious of the comparison results. UNets are generally a solid baseline for this kind of problem, but UNet results looked unreasonably bad; this may be a case of a misimplemented or undertuned baseline.
questions: I think with a bit of work on clarifying method and results this could be a much stronger paper. If those are addressed sufficiently I will raise my score.

Method:
- The paper talks a lot about using insights from classical PDE solvers, but I couldn't find how those factor into the method. Can you make this clearer? E.g. is there any reason to believe the flux-conservation autoencoder actually preserves flux better than a standard method trained on overall MSE? 
- How exactly is the flux-conservation module trained? The simulation output is already at the fix point, so if trained on this data wouldn't it just learn to pass through eta_c and ignore the neighbors?
- Similarly, isn't the distribution shift between train (consistent patches) vs. the first iteration on test (random values with possible discontinuities) a problem?

Experiments/Results:
- How many iterations are needed? How accurate is the model when just run for 1 step?
- Are the baselines also iterated or run one-step?
- How is resolution scaled? CoMLSim could use more patches at test time, but e.g. UNet resolution can't easily be changed without re-training
- Why are the baselines so bad? I certainly have seen much better prediction with UNet/ResNet-like architectures on very similar datasets.
- In general, I can see how this method is more memory efficient and can tackle bigger domains compared to basic UNet, and perhaps better at generalization. But it's very surprising to me why those would have worse accuracy in the in-domain setting; if anything, I would have suspected the reverse, due to UNet's scale hierarchy which is generally very helpful for steady-state PDEs. If it isn't just an undertuned baseline, the reason for this should be properly investigated.

limitations: There isn't really any discussion on limitations beyond that OOD generalization is hard.
review_title: null

Meta-review: metareview: This papers proposed a new method to predict accurate and generalizable PDE solutions on high-resolution grids. All reviewers found the paper is interesting and are positive about the paper. Please address the reaming concerns in the next version.
confidence: Less certain
Decision: Accept