Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper presents a set of sufficient criteria that guarantee the stability of numerical schemes used to solve scalar hyperbolic PDEs with periodic BCs and presents a predictor-corrector algorithm to implement such schemes. The prediction step is performed by a Machine Learned function (can also be other functions? such as Method of Lines) and finds the values of fluxes at desired locations to propagate the flow. The corrector step modifies the predicted flux values to ensure "sufficient criteria for stability" are satisfied.
The approach was applied to two examples, and the results show that this approach achieves "stabilization of unstable numerical schemes" and "reduction of numerical damping" while globally preserving conserved properties.
strengths_and_weaknesses: Strengths:

a) The algorithm (and the predictor-corrector approach) presented is relatively simple to implement in various numerical schemes

b) the results seem to indicate improvement in stability and numerical damping effects with almost no computational cost increase

Weakness:

a) Hard to generalize to other systems

b) Conserves properties (only?) on a global scale, and accuracy deteriorates in the presence of sharp gradients in the solution (for example, shocks in the flow).



questions: A) Is there a way to alter C in equation (5) to be a function of index j and replace it with C_j in the corrector step to allow for local conservation of properties while preserving discrete L2-norm?

B) Equations 3b cannot be exact since flux at cell boundaries can still vary in a direction parallel to the cell boundary. What does ``exact'' mean in this context?

C) Why does the discrete energy of the system increase in the second numerical example in the globally stabilized scheme with no damping? What is the source of this energy coming into the system?

D) How do the 'beta'-parameters in the corrector step influence the performance/accuracy of the numerical scheme?
limitations: The application of results seems limited - there does not seem to be an explanation of how "sufficient conditions for stability" (see section 2) are derived, nor is there a guideline to obtain such criteria for other PDEs and boundary conditions. While the paper states this method could be extended for hyperbolic PDEs with a source term, higher spatial dimensions, and other boundary conditions but does not explain how. In that sense, the application is limited and specific to a particular PDE. Further, extension to vector-valued hyperbolic PDEs does not seem trivial as sufficient conditions are likely to change.


Soundness:
There are no apparent errors in the math; however, the main results of the paper are not compartmentalized into "theorems and proofs." Hence the exact limitations of the method are not explicitly stated anywhere. Furthermore, all the summation terms are missing index limits. Therefore actual treatment of the boundary terms and correction applied to fluxes at the boundary is not clear (the correction to fluxes is probably the same for interior and boundary  since periodic boundary conditions on an \R-interval are the same as no boundary conditions on a spherical manifold; however, such details are missing in the paper).

Presentation:
The paper's content is presented concisely and to the point; however, repeated use of technical terms without ever defining them leads to a lack of clarity. Many statements made are inadequately explained or supported. For example, the last sentence of the introduction, missing summation index limits in all the equations, last paragraph in section 3, second paragraph in section 5, etc.

Contribution:
Lack of novelty (a standard predictor-corrector approach), lack of mathematical rigor in the presentation, and limited application (only hyperbolic PDEs with periodic BCs, Method of lines, scalar PDE) make the overall contribution to be marginal.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper is concerned with designing stable solvers for scalar conservation laws on rectangular domains with periodic boundary conditions.
The central idea is to translate a stability condition into the requirement that in the continuous-time limit, the discrete $L^2$-norm of the solution must be non-decreasing.
This is equivalent to requiring $\sum_j u_j(f_{j+1/2} - f_{j-1/2}) \geq 0$ ($u_j$ and $f_{j\pm1/2}$ are discretisations of the variables in the PDE).
Now the main idea of the paper is to rewrite this inequality condition with summation-by-parts and to identify what needs to be added to each $f_{j+1/2}$ for the condition to always hold.


strengths_and_weaknesses: I appreciate the idea presented in this work, and I like the honest list of the limitations of the proposed method.
But I am unsure about the motivation/premise behind the outlined algorithms, I struggle with the lack of mathematical rigour, especially in the introductory Sections 2 and 3, and I have too many questions about the proposed scheme to be able to recommend acceptance.
I think this paper should be rejected.
More details on what I consider strengths and weaknesses are below.

### Strengths 

* Machine-learned PDE solvers are a subject that seems to be of interest to the Neurips community
* Designing a stable finite volume solver is an important task for PDE simulation
* The general idea is simple and could be implemented by anyone that wishes to adopt the proposed method
* Related work and limitations are assessed thoroughly


### Weaknesses

* The paper claims to be concerned with _machine-learned solvers_, but the content of the paper is independent of _learning_ solvers: the results apply to any algorithm that predicts the flux $f_{j + 1/2}$. 
The two experiments do not learn PDE solvers but consider a-priori-chosen schemes: centred flux, and the _already-stable_ MUSCL scheme.
I am not sure whether this lack of a connection to machine learning makes the results less or more interesting. But given the venue and the paper's title and abstract, the disparity between mentioning "machine-learned PDE solvers" frequently on the one hand, and not connecting to learning PDE solvers, on the other hand, makes the message of the paper slightly confusing.
* The motivation for this paper is stated in line 88f.: "While machine-learned FV solvers are capable of ensuring stability via classical techniques [27, 25], we anticipate that such choices of flux will be too restrictive and too diffusive to outperform state-of-the-art numerical methods." 
It seems like the submitted paper provides a solution to an _anticipated_ problem. There is no evidence, experimental or referenced, that shows how "such choices of flux will be too restrictive and too diffusive" is an actual concern.
In my view, this limits the expected impact of this work significantly.
* The readability of the paper is inhibited by a lack of rigour in technical components, especially in Sections 2 and 3. I understand that the authors attempt to keep the mathematical load on the reader as light as possible, but it makes the introduction difficult to follow, sometimes even difficult to verify/falsify. Some issues are listed in the questions below.
questions: 
* Line 25: "and as a result, often do not maintain stable solutions". I would appreciate a reference that reveals/discusses this issue.
* Line 31: "the answer to this question" Which question? 
* Lines 24/25 and 27-34: I would appreciate a more formal definition of stability early in the paper. I find the exposition in 27-34 too high-level to be able to falsify/verify its statements.
* Lines 40f: "We note that the method can also be used when the right hand side (RHS) of eq. (1) is nonzero by treating the right hand side terms using classical techniques and applying global stabilization only to the divergence terms": please elaborate. If necessary, in an appendix. The statement, taken as is, is difficult to falsify/verify.
* Line 52: what is $S$? Please define all terms in the equation.
* Line 54f: "we can see that the discrete L2 norm is non-increasing": Please define these terms more rigorously; the paper only states $d/dt \int u^2/2 dx \leq 0$.
* Lines 54f: "we now list four conditions sufficient for ensuring that a solver of eq. (1) is stable:" Please embed the four conditions that you list into the literature. Does everyone mean these four conditions when discussing stability? Some references would help the understanding.
* Line 57: "the integral of the solution (...) is conserved": this statement would be easier to follow if you wrote down the integral, because it is not obvious whether the integral is over time, space, or both.
* Line 65: "Integrating eq. (1) over a grid cell" Please elaborate/formalise. For someone that does not already know finite volume schemes, I expect that it would be difficult to follow.
* Line 67: Please define $f_{j+1/2}$ prior to Equation (3).
* Lines 79/80: Please elaborate on the terms "spurious oscillations" and "numerical diffusion". 
* Line 96f: "Thus, eg. (4) will be satisfied if the following transformation is made" Could you please write down the proof of this statement somewhere? This might be the most central contribution of the paper, it should not be up to the reader to verify it.
* Eq. (5): what does the $\equiv$ symbol mean? Please define it.
* Line 108: "which violates the physical property..." The paper mentions this in Section 8, and I also find it potentially problematic if each $f_{j + 1/2}$ depends on the solution over the entire domain because it seems to differ quite drastically from conventional method-of-lines- or finite-volumes-solvers. 
I would appreciate a more thorough discussion of this possible drawback.
* Figure 1: I am not able to deduce from the figure how "global stabilization ensures that the discrete L2 norm is conserved."
* Figure 2(b): the two different shades of dark blue are incredibly difficult to distinguish.
* Lines 168f: "that adjusts the (...) output if that output moves the solution towards instability": what is the price in accuracy? I find it difficult to believe that adjusting for stability does not cost accuracy. This aligns with the statements made in Lines 255f, but is there any way of understanding or at least quantifying this trade-off?
* Eq. (10): please define the notation $\langle N \mid 1 \rangle$. I suppose it refers to an inner product?

limitations: In my view, the biggest limitation of the method seems to be understanding how much accuracy, respectively general reliability of the solver is traded for improved stability.  This is mentioned in Section 8, but not discussed very thoroughly (but other, more minor limitations are discussed thoroughly).
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the scalar hyperbolic PDE problems on a uniform grid with periodical boundary conditions. The paper proposes a numerical finite-volume solver that guarantees to generate stable solutions to the PDEs by construction. The key observation is to notice that the stability of the solution can be enforced by properly reconstructing the cell boundary flux so that it ensures a non-increasing L2 norm of the solution. Therefore, it proposes a correction scheme for the cell boundary flux called “global stabilization”. While the main result is theoretical, one can properly combine it with machine learning solvers to generate a stable solver.
strengths_and_weaknesses: Strengths:
- The paper studies a very critical question that, in my opinion, is a bottleneck of many existing machine learning solvers for PDEs.
- The mathematical insight behind the proposed solution is convincing.
- I also appreciate that the proposed numerical scheme stabilizes the solution and removes numerical diffusion at the same time.

Weaknesses:
- The paper requires advanced background knowledge of PDEs and is not very friendly to readers unfamiliar with numerical methods for PDEs. I think having more illustrations will be nice, e.g., an inset around Eqn 3 that displays a uniform grid with locations of each variable u and f.

questions: 1. Line 71: Is the last equation = 0 because of the periodic boundary conditions (so that f_0 and f_last are known to be the same)?

2. Although I can see that Eqn. 5 guarantees the L2 stability, I cannot see how it is derived based on the explanation in lines 95-96. I’d appreciate it if more intuition can be provided when deriving Eqn. 5.

3. On a related note, lines 109-110 mentioned Eqn. 5 can be used for both stabilizing a solver and reducing numerical damping and presented two numerical experiments to support them. Is there any theoretical analysis on the numerical diffusion introduced by Eqn. 5?

4. Because of 2 and 3, I have found Sec. 5 a bit difficult to follow. Maybe a better way is to first state the goal is to come up with a correction scheme on f so that it stabilizes the solver and reduces numerical damping, then explains how this goal leads to a derivation of Eqn. 5.

limitations: I really like the fact that the authors are upfront about the limitations of the proposed approach, especially by pointing out many assumptions that they made about the PDE system and numerical schemes. To me, these issues (assuming periodical boundary conditions, regular grids, limited PDE types, etc.) are not deal-breakers but rather show the potential of inspiring follow-up papers.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: I thank the authors for an interesting read.

Rollout stability of PDE solvers for time-dependent hyperbolic PDEs is a property baked into classical solvers, yet it is unaddressed in the machine learning for PDEs community. As such it is a common problem for learned solvers that they are sometimes unstable. When and under what conditions this instability arises is poorly understood and often swept under the table.

This submission seeks to resolve that with a technique that the authors call `global stabilization’. They introduce an L2-stability criterion, which states that the L2 norm of the solution is non-increasing over time. This is a well-known property of hyperbolic PDEs in conservation form. The beauty of this is that enforcing this property directly ensures that stability is met, since stability is defined as the solution blowing up and this cannot happen if solutions are norm non-increasing in time.

The technique is general and works as an ad hoc projection step to be used on the output of any model, but it is also handcrafted and there is no guarantee that it is the most general such stabilization method.

strengths_and_weaknesses: The paper tackles a very important and much overlooked issue in the ML for PDE solving literature. I welcome this contribution and see it as a positive aspect of the submission.

The paper boasts a nice related work section, which situates in the landscape of contemporary ideas adequately.

The limitations section of the paper is very open and honest. Furthermore it opens a lot of interesting questions, such as, how do we make accurate ML solvers? This makes it particularly transparent for a machine learning paper. I thank the authors for this addition.

The technique is general and works as an ad hoc projection step to be used on the output of any model, but it is also handcrafted and there is no guarantee that it is the most general such stabilization method. As such, as the authors admit themselves, while the method achieves the desired stability, it does not guarantee accuracy and looking at their results in Figure 3, it is certainly the case that standard measures of performance such as vorticity correlation take a large hit. This is the greatest weakness of the paper.

From what I can make out, the paper has no explicit experimentation section. I think this makes it hard to verify its efficacy.

questions: I think a short discussion on different kinds of stability would be helpful. The kind focussed on in this submission is “eigenvalue stability” (aka weak stability, absolute stability, time-stability, practical stability, or P-stability) where the timestep is kept finite and time is allowed to tend to infinity as opposed to simply “stability” (aka zero-stability, D-stability, Lax stability, or Lax-Richtmyer stability) where time is kept fixed and the timestep tends to zero.

Conditions 2 and 3 on page 2 state that mass conservation and discrete L2-norm non-increase require the continuous time limit. Is this strictly true? One could discretize time again and these (in)equalities still hold.

While the four conditions on page 2 are sufficient, they do not guarantee accuracy (and therefore convergence). (After reading through the end I note here that this point is acknowledged in the limitations section). What is missing to guarantee accuracy?

Equation 5 is just one way to satisfy the L2-stability condition, but it does not look to be the most general expression that could be used here. Why did you choose this expression? Am I perhaps wrong and is this the most general expression that could be used?

Figure 1: it is not clear from the figure that at t=0.5 the red solution is less stable than the blue solution.

Lines 158 - 160: I see that there is a choice to be made here as to whether the ML solver should model the properties of the continuous or the discrete equations. To me it does not necessarily have to be one or the other. Is there a natural reason why you assert it must be that the ML solver must inherit the properties of the discrete equations only?

Is it such a limitation that global stabilization breaks causality? In many numerical systems it often happens that discretization breaks notions of locality. For instance, computing high order derivatives at a point requires that we use very large stencils, despite the fact that derivatives are defined entirely locally.

limitations: I believe that the authors have addressed most of the limitations of this work. The one remaining one is that I do not believe the proposed technique to be the most general possible solution and as such it may be that ML solvers will lose functional expressivity on some level.
review_title: null

Meta-review: metareview: Thank you for your submission to NeurIPS. All four reviewers authors are enthusiastic of this work, though three of the four reviewers had major concerns with the actual submission. In response, the authors carried out a major revision within a very short turn-around, completely rewriting most of the paper. All four reviewers are highly appreciative of the authors' responsiveness in replying to the reviews, in submitting new revisions taking into account reviewer comments. 

Unfortunately, some concerns remain even after the author-reviewer discussion period closed. Broadly, the three reviewers all felt that there had been too many edits of the paper to be able to verify/falsify the newest content in all detail. The new revision is sufficiently different as to necessitate a fresh set of reviews, in a way that the conference format of NeurIPS is not able to facilitate. 

Overall, the core idea is interesting and potentially very useful. The paper studies a critical problem typically overlooked by the current trend of combining ML with PDE solvers, and that it is transparent and upfront about its limitations. The paper asks a good question that can inspire important follow-up works. I highly encourage the authors to take reviewer comments into account, to expand into a complete, detailed experimental section and compare against other ML-based approaches, and to resubmit to an upcoming ML conference.

confidence: Certain
Decision: Reject