Kati8/ICLR_2023_2025 huggingface dataset:
authors: Nithin Chalapathi
authors: Yiheng Du
authors: Aditi S. Krishnapriyan
reviews: 
Comments: The paper presents a novel Physics Informed (PI) framework to scale the imposition of "Hard Constraints" (HC) with the use of a Mixture of Experts (MoE) Network. The hard constraints are imposed on $m$ sampled points $x_1, ..., x_m \in \Omega$ by solving a non-linear least squares problem (NLSS), and the optimization is made possible by using a differentiable solver. The paper proposes a framework that decomposes the domain $\Omega$ into subdomains $\Omega_k$ before solving the NLSS problem on each subdomain.

They tested the framework as a neural PDE solver on 1D diffusion-sorption and turbulent 2D Navier-Stokes equations in a data-constrained regime. The method outperforms existing soft and hard-constrained methods in terms of accuracy, and scales well at inference w.r.t the number of sampled points.
Strengths: The paper is well written and easy to follow. The method takes inspiration from the domain decomposition which is standard in the the literature for solving Partial Differential Equations.
I appreciated the details in the Methods section, particularly the explanations on the forward and backward pass of the architecture.
The method clearly outperforms the existing  soft- and hard-constrained baselines. The claim of the paper to the scaling is well supported with a solid inference time analysis.
Weaknesses: Unless I am mistaken, there is no clear formulation of the output function $u(x, t)$ at inference except in Figure 1, and if I understand the figure correctly, then $u(x, t) = \sum_k b(x,t)^T w_k =b(x,t)^T( \sum_k w_k)$. In this case, the sum of $w_k$ is the weights used to query the function over the domain $\Omega$, and  as $w \neq w_k$ a priori, we do not know if the constraints are hardly imposed on any sampled points. Therefore, at inference I do not think that the PDE can be constrained in a hard setting with this architecture.

I also assume that solving an equation on a domain $\Omega$ with boundaries $\delta \Omega$ is not always equivalent to solving the equation separately on different subdomains and that precautions must be taken. There is no mathematical derivation of the domain decomposition problem, and therefore we do not know if the solution found with the subdomains is close enough to that of the solution on the full problem. 

I also understood from Figure 1 that the different domains were not overlapping and represented "chunks" of the domain, but I am puzzled by the grid artifacts of the method in Figure 3. The solution seems to showcase some strange aliasing which would suggest overlapping domains.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 2 fair
Clarity: 3 good
reviews: 
Comments: Methods
- This paper utilizes the Fourier Neural Operator (FNO) as the NN architecture to learn a set of N scalar-valued functions as the basis functions; enforces the hard constraints by using differentiable optimization to find a linear combination of basis functions to satisfy the constraints.
- To deal with larger systems which contain a large number of sampled points and basis functions, mixture-of-experts is used to decompose the spatialtemporal domain.
- In backward pass, to compute the gradients of the differentiable optimization, the implicit function theorem is used. For mixture-of-experts, the Jacobian is reconstructed from the individual Jacobians from all experts. 

Experiments:
- Two cases: 1D diffusion-sorption and 2D turbulent Navier-Stokes 
- Compared to training via a physics-informed soft constraint (PI-SC) and physics-informed hard constraint (PI-HC). 
    - Achieve higher accuracy and lower time cost.
Strengths: - The paper proposes a new way to decompose the complex dynamical systems with a large number of points in the spatiotemporal domain to smaller solvable systems, by utilizing the mixture-of-experts method. It makes the system scalable and performant (with parallel computing).
- In the 2 test cases provided in the paper, the paper’s method has higher accuracy and lower time cost compared with other two methods.
Weaknesses: The experiments are relatively limited - the paper only tests on 2 cases, one for 1D and another for 2D. In each experiment, only one set of environment parameters (e.g.,, only Reynolds number = 1e4 is used for the Navier-Stokes case) are tested.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: This paper proposes a scalable way to enforce hard constraints expressed as partial differential equations into neural networks in order to faithfully model physical phenomena. The problem they try to tackle is given by the fact that backpropagating through constraints over large meshes is a highly non-linear problem that grows in dimensionality with respect to the mesh and neural network size. The authors propose to impose the constraints over smaller decomposed domains, each of which is solved by an expert.
Strengths: **Novelty:**

I am not in a position to judge the novelty of the paper as there is very little overlap with my work. From the related work mentioned in the paper, the paper seems novel. 


**Significance:**

The paper is of high interest to a subset of the ML community.
Weaknesses: **Clarity:** 
 
The paper is mostly quite clear. Personally, I would have benefitted from an ongoing example, where it is made clear what the outputs of the neural networks are, what the constraints are, and how the problem is divided in that case. Even better, it would have been nice to use the same example to move from one constraint to multiple ones.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good