Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods
Authors: Lise Le Boudec
Emmanuel de Bezenac
Louis Serrano
Ramon Daniel Regueiro-Espino
Yuan Yin
Patrick Gallinari
Venue: ICLR 2025 Poster (2025)
Keywords: Physics-informed Deep Learning
PDE Solver
Parametric PDE
PINNs
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We present a data-driven method to accelerate the optimization of physics-informed models.

Abstract: Physics-informed deep learning often faces optimization challenges due to the complexity of solving partial differential equations (PDEs), which involve exploring large solution spaces, require numerous iterations, and can lead to unstable training. These challenges arise particularly from the ill-conditioning of the optimization problem, caused by the differential terms in the loss function. To address these issues, we propose learning a solver, i.e., solving PDEs using a physics-informed iterative algorithm trained on data. Our method learns to condition a gradient descent algorithm that automatically adapts to each PDE instance, significantly accelerating and stabilizing the optimization process and enabling faster convergence of physics-aware models. Furthermore, while traditional physics-informed methods solve for a single PDE instance, our approach addresses parametric PDEs. Specifically, our method integrates the physical loss gradient with the PDE parameters to solve over a distribution of PDE parameters, including coefficients, initial conditions, or boundary conditions. We demonstrate the effectiveness of our method through empirical experiments on multiple datasets, comparing training and test-time optimization performance.

Published as a conference paper at ICLR 2025
LEARNING A NEURAL SOLVER FOR PARAMETRIC PDEs TO ENHANCE PHYSICS-INFORMED METHODS
Lise Le Boudec'
Emmanuel de Bezenac"
Louis Serrano
Ramon Daniel Regueiro-Espino
Yuan Yin 3
Patrick Gallinari 1,4
Sorbonne Université, CNRS, ISIR, F-75005 Paris, France
2 INRIA Paris, France
3 Valeo.ai, Paris, France
Criteo Al Lab, Paris, France
ABSTRACT
Physics-informet deep learning often faces optimization challenges due to the com- plexity of solving partial differential equations (PDEs), which involve exploring large solution spaces, require numerous iterations, and can lead unstable training. These challenges arise particularly from the ill-conditioning of the optimization problem caused by the differential terms in the loss function. To address these issues, we propose learning a solver, i.e., solving PDEs using a physics-informed iterative algorithm trained on data. Our method learns condition a gradient descent algorithm that automatically adapts to each PDE instance, significantly accelerating and stabilizing the optimization process and enabling faster conver- gence of physics-aware models. Furthermore, while traditional physics-informe methods solve for a single PDE instance, our approach extends to parametric PDEs. Specifically, we integrate the physical loss gradient with PDE parameters, allowing our method to solve over distribution of PDE parameters, including coefficients, initial conditions, and boundary conditions. We demonstrate the effectiveness our approach through empirical experiments on multiple datasets, comparing both training and test-time optimization performance. The code is available Intpps://github.ccmm/// www.://./cm//aeae///eceaaa http://aaaaaeeeeeeeaaaaaaaaaaaaa wwwpllaaaaataatiaoooeeeee
INTRODUCTION
Partial Differential Equations (PDEs) are ubiquitous as mathematical models of dynamical phenomena in science and engineering. Solving PDEs is of crucial interest researchers and engineers, leading to huge literature on this subject (Evans) 2010: Salsa 2015). Traditional approaches to solving PDEs such finite difference, finite element analysis, or spectral methods (Zienkiewicz al. 2005 Veque 2007) often come with stability and convergence guarantees but suffer from a high computational cost. Improving numerical PDE solvers through faster and more accurate algorithms remains an active research topic (Ziekieeiczz et al.) 2005).
PDE solvers usually rely on discretization and/or linearization the problem through various techniques to simplify the computations. Iterative methods such as Jacobi, Gauss-Seidel, Conjugate Gradient, and Krylov subspace methods can then used solve the resulting systems. Unfortunately, many PDEs have ill-conditioned nature, and these iterative processes computational resources. Preconditioning techniques are often essential to mitigate this, though they require precise customization the specific PDE problem, making the development of effective solvers a significant research endeavor itself. Yet, the computational demands, time, and expertise required develop these algorithms sometimes make them impractical sub-optimal for specific
demand extensive
Conresponding author: lise. leboudec@isii www.uddeeiiir.uum....... fr. Work done during post-doc Sorbonne University.

---

Published as a conference paper at ICLR 2025
classes of problems. Instead of relying on hand-designed algorithms, researchers have investigated. as an alternative, the use of machine learning to train iterative PDE solvers Hsich et al. (2019); Li
et al. (2023a); Rudikov et al. (2024) Kopaničáková et al. (2023). >. These approaches usually parallel the classical numerical methods by solving a linear system resulting from the discretization of a PDE, for example, using finite differences finite elements. A preconditioner is learned from data by optimizing a residual loss computed w.r.t. a ground truth solution obtained with a PDE solver. This preconditioner is used on top of a baseline iterative solver and aims at accelerating its convergence. Examples of baseline solvers are the conjugate gradient (Li et al. 2023a Rudikov et al. 2024) or the Jacobi method Hsiih et al. (2019).
Another recent research direction investigates the use of neural networks for building surrogate models in order to accelerate the computations traditionally handled by numerical techniques. These methods fall into two main categories: supervised and unsupervised. The supervised methodology consists of first solving the PDE using numerical methods to generate input and target data and then regressing the solution using neural networks in the hope that this surrogate could solve new instances of the PDE. Many models, such as Neural Operators, lie within this class (Li et al. 2020; Raonic et al. 2023 Bartolucci et al. 2023) and focus learning the solution operator directly through a single neural network pass. Unsupervised approaches, involve considering a neural network as solution of the PDE. The neural network parameters found by minimizing the PDE residual with gradient descent. Methods such as Physics-Informed Neural Networks (PINNs) (Raissi et al. 2019). or DeepRitz (E & Yu 2018) fall under this category. This family of methods is attractive as it does not rely on any form of data, but only on information from the PDE residual. However, they exhibit severe difficulties during training (Krishnapriyan et al., 2021 Ryck et al. (0223) often requiring many optimization steps and sophisticated training schemes (Krishnapriyan et al., 2021 Rathore et al.] 2024). The ill-conditioned nature of PDE residual loss appears again in this context, making standard optimizers such as Adam inappropriate (see appendix [E6] and appendix E.7 for visualization of the ill conditioning of this loss landscape). A detailed review of the existing literature is described in appendix A
In this work, we consider having access to the PDE as in unsupervised approaches and also to some data for training our neural solver. Our objective is to solve the optimization issues mentioned above by learning an iterative algorithm that solves the PDE from its residual, defined as in the PINNs framework (see fig. D. This neural solver is trained from data, either simulations observations. Different from the classical ML training problem which aims at learning the parameters of a statistical model from examples, the problem we handle is learning to learn, (Adrychowicz al. (2016), i.e. learning an iterative algorithm that will allow us to solve a learning problem. When vanilla PINNs handle a single PDE instance, requiring retraining for each new instance, we consider the more complex setting of solving parametric PDEs, the parameters may include boundary or initial conditions, forcing terms, and PDE coefficients. Each specific instance of the PDE, sampled from the PDE parameter distribution, will then be considered as a training example. The objective is then to learn solver from a sample of the parametric PDE distribution in order to accelerate inference on new instances of the equation. With respect to unsupervised approaches, our model implements an improved optimization algorithm, tailored to the parametric PDE problem at hand instead of using a hand-defined method such as stochastic gradient descent (SGD) or Adam. As demonstrated in the experimental section, this approach proves highly effective for the ill pol-posed problem of optimizing the PINNs objective, enabling convergence in just few steps. This is further illustrated through gradient trajectory visualizations in appendix [..6] In the proposed methodology. the neural solver will make use of the gradient information computed by a baseline gradient method to accelerate its convergence. In our instantiation, we will use SGD as our baseline algorithm, but the method could be easily extended to other baselines. Our model deviates from the traditional preconditioning methods by directly optimizing the non-linear PDE residual loss (Raissi et al., 2019) without going through the discretization steps. Our contribution includes
Setting an optimization framework for learning to iteratively solve parametric PDEs from chysics-informed loss functions. We develop an instantiation of this idea using an SGD baseline formulation. We detail the different components of the framework as well as training and inference procedures.
Evaluating this method challenging PDEs for physics-infomma method, including failure cases of classical PINNS and showing that solves the associated optimization issues and accelerates the convergence.
2

---

Published as a conference paper at ICLR 2025
Extending the comparison to several parametric PDEs with varying parameters from 1d static to 2d+time problems. We perform a comparison with baselines demonstrating significant acceleration of the convergence w.r.. baselines.
Inference
Me (+)
fo St 3i
02 Ma(a)
Data loss
Optimiser: Fr on
Physical "ha Jr.
Computes
XL steps
Optimises
Optimizer (Adam, ...) on
Training
Figure Optimization scheme a physics-informed method with our framework.
MOTIVATION
Let us first motivate our objective with a simple example. Physics-informed neural networks (PINNs) promising tool for optimizing neural networks an unsupervised way solve partial differential equations (PDEs). However, these methods are notoriously difficult to train (Krishnapriyan et al. 2021; Ryek et al., 2023). As an illustrative example of this challenge, let solve the zero-boundary Poisson equation 1d on Ω [-TT, nT. Note that this section is intentionally informal, we provide rigorous statements and proofs in Appendix
Poisson equation, 1d. The solution given by u(x) = sin(kx).
u" u"(x) - 2 sin (kx), u( ((-TT) = 0, u(TT) = 0.
(1)
Physics-inforned machine learning relies on an ansatz space of parametric functions, e.g. neural networks ue Ω R. minimizing the following loss in order satisfy the constraints in equation I
L = -/11b(0) - f(x)) dr,
LYDE Lun. + ALBC,
1 [uol u 1 2
LBc
As a simple example, consider the parametrization given by considering a linear combination of Fourier features widely used (Tancik et al. 2020 ue( = -h- k OLdx(I), with (n(x) </2x*
(-+(x)= / cos(k:r) and dx(x)) J sim (kar for S < K.
This simple but informative example yields a tractable gradient descent algorithm, as the associated updates are linear in the parameters, governed by a matrix A and constant b:
- θ 1/ V/ (O))
(I 1 +
(2)
Note that though the ansatz is linear in is not linear in

---

Published as a conference paper at ICLR 2025
with A whose condition number is x(A) = Amax (A)/ /aa//mmnn (A) 2 K
0
0
T
0
14
0
(2(
A
+ A
[0)(T) )2( dx(T)]-
: K4
(T)
This implies that the condition number of A increases extremely rapidly in the ratio between the highest and lowest frequencies of the network. Given that the rate of convergence to the optimum O = A can be bounded as
||0 - l* = (1 (1- (1-c//(4)))(1)) - 1r,
the number of steps (e) required to obtain an error of size at most e, ie., | O0 S increases linearly in the condition number, i.e. the fourth power the maximal frequency K:
(e) = O (m(A) O (K* 1n
(5)
We believe that this simple example clearly illustrates and highlights the fact that PINNs-even when considering linear basis, and when the PDE linear-suffer heavily from ill-conditioning: if 500 steps are required in order to achieve given error when K 5, roughly speaking, 312 500 steps are required for only K 25. This result extends to more general linear systems of equations and linear ansatz, as explained in appendix [
Our objective the following will be accelerate the convergence of such systems in this context-as well as extend them to the non-linear setting. To do so, in the following section, we will learn how to transform the optimization problem in such way that the number of gradient descent iterations small. The resulting method can be seen standalone, iterative solver is not only applicable to different PDEs but can handle a wide range initial/boundary conditions and parameters.
APPROACH
In order optimize PDE-based losses, we propose learn a physics-based optimizer that will fulfill two objectives: (i) allowing a fast test-time optimization given a new PDE and (ii) solving without retraining parametric PDEs, with varying PDE coefficients > P forcing terms f. and initial/boundary conditions g using the same model. We present the general framework below and propose an instantiation that leverages linear combination of basis functions as the ansatz.
3.1 PROBLEM STATEMENT
Let us consider the following family of boundary value problems parameterized by with domain Q representing both space and time, with A potentially nonlinear differential operator, B the boundary operator, the initial/boundary conditions, and source term f:
N(u f in Ω ((u) = on an
(6)
Note that different PDEs can be represented in this form, amounting to changing the parameters The goal here is develop a generic algorithm that is able solve the above problem, yielding an approximate solution given the PDE and different sets of inputs (Y, f.
For training, we assume access
dataset of M problem instances, represented by the PDE parameters (Ys, fr. M and to associated target solution ( given on m point grid (2,))
The solutions M M will be used to train the neural solver. At inference, for PDE instance, only the PDE parameters provided and we do not have access solution points M
Note that PDE coefficients be functions, an example the Darcy PDE in section 4.1
4

---

Published as a conference paper at ICLR 2025
3.2 METHODOLOGY
Physics-informeds neural networks consider an ansatz we parametrized by some finiteddimensional
O. The parameters O iteratively updated by minimizing a criterion LEDE (e.g. the PDE residual), which assesses how well the ansatz 'ie meets the conditions specified in equations [nd 77] As introduced for example for PINNs (Raissi et al., 2019) or Deep Galerkin method (Sirignano & Spiliopoulos, 2018), we consider LPDE to be given by the strong formulation of the residual LRes. plus boundary discrepancy term LBC: LPDE = L... + ALBC, ; O.
Σ [N(ue:>)a)) f(x))) LB Σ Byen
( 2,E00
(8)
As illustrated in section performing gradient descent, or alternatives such as Adam and L-BFGS on such a highly ill-conditioned loss LPDE leads severe training difficulties (Krishnapriyan et al. 202I). The key idea in our work is to improve a baseline gradient descent algorithms with the neural solver. More precisely starting from a baseline gradient algorithm, SGD in our instantiation, instead of considering the classical update, we first transform the gradient using a neural network Fg with parameters O, depending on the values of the PDE parameters as well as on other inputs such as the residual gradient provided by SGD: V CPDE The objective is to transform, through the neural solver Fo. the ill-conditioned problem into a new, simpler problem that requires fewer steps in order to achieve given error.
Once the neural solver Fg has been learned, inference can be performed on any new PDE as follows (see inference algorithm|| [. Starting from initial ansatz parameter Og it is iteratively updated by this solver. At iteration 1, the steepest direction the loss LPdE is first computed with autograd. Then, the gradient is transformed, in PDE parameter dependant way, with F.:
1 = O - (Vee(Ve F.g)
The objective is to iteratively refine the ansatz to closely approximate the true solution after a series of L iterations, ideally small for efficiency.
This approach can be seen as learning the iterates in PDE solver to achieve a low loss, similar to the residual minimization methods in PDEs (Elman et al. 2014). By design, this solver intended be applicable to different PDEs, as well as various sources, boundary conditions, and initial conditions. This flexibility allows for a broad range of applications, making it versatile tool in solving complex PDEs with varying characteristics.
Designed as a parametric PDE solver, Fo is trained with input target data from different sets of PDE parameters, outlined in section 3.1 Once trained, it will be used without retraining on new instances of the PDE, i.c. with new values of the PDE parameters. The underlying hypothesis is that even though the solutions may be different for different inputs and parameters, the solution methodology remains relatively consistent. This consistency is expected to enhance the algorithm's ability to generalize across novel scenarios effectively.
3.3 TRAINING OF -NFORMEDD SOLVER
Choice of Ansatz ue. A very common choice (Shen et al. 2011) is to consider a family of basis functions (x) {vi(=))[ and consider the ansatz be given by its linear span ue ue(x) (x) formulation is generic in the sense that it can also accommodate nonlinear variants.
= - (z). In the following, we consider this linear reconstruction, although our
As indicated in eq. (0), the solver F, will be trained from samples of the PDE parameter distribution f.9) and from the associated samples of the solution u. We first describe below the inference step aiming iteratively updating the parameters θ of the solution function uh while F, is held fixed. We then describe how the @ parameters of the solver are trained. Please refer to [ thht illustrates the interaction between the steps.
Note that other formulations of the loss may also be considered in straightforward manner.
'n the following we use "solver" with F, as a short-hand to refer to our proposed method. "Although we have found this may further complicate training.
5

---

Published as a conference paper at ICLR 2025
Inference The inference step is performed at fixed values of the solver parameters Fe. It con- sists, for given instance of the PDE with pa- rameters (7%, fi-gh), in finding the best + with few steps of the solver using eq. (9) It is il- lustrated in fig. grey box and formalized in algorithm T starting from initial parame- ters Oo. we compute the Physical loss LyDe using the ansatz uêg- The PDE derivatives in LPDE. can be computed by hand or automatic differentiation depending on the application 6
Algorithm 1: Inference using the neural PDE solver.
Data: Og E IR. PDE (7,
Result: 0,, E R"
for = 0.. do
Or+1 Or F (VLoDe(Gt),
end return O
Algorithm 2: Training algorithm for learning to optimize physics-infin informed losses.
Then, eq. (9) is used to update the parameters for a given number of steps L. The final solution is reconstructed using the linear combi- nation, u (x) Ei= (1)), introduced in section 3.2 with the computed coefficients ④. Note that inference does not make use of the sampled target solutions (u=) M the grid points(x;,)?= --
Data: On IR", PDE (7, sample values u(I) Result: Fe
for 1... epochs do
for (PDE, x, u) in dataset do
Initialize Op
)= computed on -These targets are used exclusively for training the neural solver.
Estimate 0 from Oo (%: using
algorithm |
Reconstruct uee (2)
Training the neural solver Training amounts
Update with gradient descent from
to learning the parameters of the solver F. and performed with a training set of PDE parameters and simulation data considered as end return Fo
the data loss in eq. (T))
end
ground truth (7t, fe, 9s, ) corresponding to PDE instances (i.e. with different parameters
y and/or forcing terms f and/or initial/boundary conditions g). See fig. [ white box and algorithm 2 The objective is learn a solver F₂ able, inference, to converge a target solution in small (2 to 5 in our experiments) number of steps. For that, an optimizer (Adam in our experiments) is used to update the Fo parameters. The training algorithm makes use of the data associated with the different PDE instances by sampling PDEs in batches and running algorithm on several PDE instances. For each PDE instance, one starts from an initial parameter value O, and then performs two optimization steps (see algorithm 2 (i) one consists in solving in the ansatz parameters @ using the neural solver using algorithm [ leading to O2; (ii) the second one the optimization of the solver parameters @- We train the outputs directly to match the associated ground truth (u, (2, (z5)))z= using the data loss:
CDNIA E.,f.g Illue, "u.ff9lll
(10)
The expectation is computed on the distribution of the PDE parameters (y. f)). The solution is entirely determined by these parameters as indicated by the notation ul.f.g- llue, - w|l denotes a
distance between the target (u(2y)) )) and the forecast (uez (()))²= with m the trajectory size In practice, one samples a set of PDE instances (7is gi) and for each instance a corresponding sample ut.
Thcoretical analysis and relation to preconditioning Analyzing the behavior of the inference algorithm is challenging due to the non linear nature of the solver. We however could get some intuition using simplifying assumptions. We build on the ideas introduced section for the simple case of the Poisson equation, for which an explicit analytical solution could be derived. We provide in appendix B a proof for a more general case and give below in theorem Tour main result. This shows that the number of steps induced by F for the proposed algorithm is significantly less than the number of steps required by the baseline PINNs algorithm. This results is obtained under two main assumptions: (i) F behaves like its linearization and (ii) the descent operator JF used in our algorithm, allows us to reach the optimum of LDATA
'In our experiments, we computed the derivative by hand when possible since it fastens computations. TT simplify the notation, used fixed grid size m. However, this framework can be used with different grid sizes, as well as irregular grids. See Ablation in appendix table

---

Published as a conference paper at ICLR 2025
Theorem 1. (Convergence rate in the linear case). Given linear ansatz, ue ue(x) - 1 (x))(x),
assume the conditioner FF behaves like its linearization P Jacobian( [)) meaning that F can be replaced by P at any point. Let A be the matrix derived from the PDE loss eq. ( for the Poisson equation or eq. (15) in the more general case. Denote by k( A) the condition number of the matrix A.
The number of steps (e) required achieve error ||0: 0r 1| S
satisfies:
- (x(P))) (1)
(11)
Moreover, minimizes LDATA this necessarily implies x(PA) = 1 x(A). Consequently the number of steps is effectively reduced, i.e., (e) < (()) with N(e) the number of steps of the vanilla PINNs.
Proof. We sketch the main insights here and refer appendix B for the proof and detailed analysis.
Using linearization of neural solver, can be shown that the solver performs as pre-conditioner on the linear system.
Assuming that solution uz provided by the solver reaches the optimum ut, and that the training set is such that the learned parameter θ vectors span the whole parameter space of the model, then the convergence of the solver is guaranteed
optimal rate.
In practice, and shown in the experiments (section 4 the convergence rate is significantly improved w.r.t. the reference baseline gradient algorithm.
EXPERIMENTS
We present the datasets used in the experiments in section comparison with selected baselines in section 4.2 and a test-time comparison with different optimizers demonstrating the remarkable effectiveness the proposed method in section 4.3 Finally, we make comparison the training and inference time in section 4.4 Experimental details and additional experiments can be found in the appendices: ablations are in appendix and additional results and visualization are in appendix E.6
appendix E.7 and appendix
4.1 DATASETS
We consider several representative parametric equations for our evalua- tion. More details about the data gen- eration are presented in appendix[[] Our objective is to learn neural solver able to solve quickly and ac- curately a new instance of PDE, given its parametric form, and the values of the parameters y, forcing terms f and initial/boundary condi- tions 9. i.e. (y, f. u. Solving is performed with few iterations of the neural solver (algorithm [ For that, one trains the neural solver on a sample of the PDE parame- ter instances, see table I for the pa- rameter distributions used for each parametric PDE. Evaluation is per- formed on unseen sets of parame- ters within the same PDE family. Helmholtz: We generate a dataset following the 1d static Helmholtz equation æ" (z: + w2 u(x)) = with
Table 1: Parameters changed between each trajectory in the considered datasets.
Dataset
Parameters
Distribution
U/0.5, 50]
Helmholtz
Ug
NO 1)
.N
A
41[100, 100]
Poisson
(O 1)
N 1)
U(1 5] 41-5,
NLRD
V4,((0, (-A + 91) -2)
Darcy
a(x)
with s/ 12 1R, + 3 lR
u12 x 10 3, 10 10-21 2,,
Jmar
2/0.5, -0.5]
Heat
Kr. Ky
2, 4100 2m
boundary conditions u0)) uo and (0) Up- We generate 1, (024 trajectories with varying w, Mo.
7

---

Published as a conference paper at ICLR 2025
and by with spatial resolution of 256. Poisson: We generate a dataset following the 1d static Poisson
equation with forcing term: (x) = f(x) with u(O) = "i and u0) = z/p. The forcing term f is periodic function, f(z) = K K 0:12 (Tz( with K- = 16 and r r- -0.5. We generate 1. 000 trajectories with varying Uy Up and f (through changing a,) with spatial resolution of 64. In Krishnapriyan et al. (02II) Toloubidokhti et al. (2024). the authors propose (NLRD). This PDE has been shown to be a failure case for PINNS (Krishnapriyan et al. 2021). We generate 1, 000 trajectories by varying the parameters of the PDE: y and p (see table D). Spatial resolution is 256 and temporal resolution is 100. The PDE is solved on [0, 112. Darey Flow: The 2d Darcy Flow dataset is taken from (Li et al. 2020) and is commonly
a non-linear reaction-diffuseee
used in the operator learning literature (Li et al. 2023b Goswami et al.) (022). For this dataset, the forcing term is kept constant f = and a(x) is piece- wiie constant diffusion coefficient taken from (Li et al. 2020). We kept 1, 000 trajectories (on the 5, 000 available) with spatial resolution is 64 x 64. Heat: The 2d + é Heat equation is simulated as proposed in (Zhou & Farimani] 2024). For this dataset, the parameter V/ is sampled from 2 x 10 10-3,2 2 x 10 10-2] and initial conditions are a combination of sine functions with a varying number of terms, amplitude, and phase. A summary of the datasets and the varying parameters for each PDE are presented in table and more details on the dataset are provided in appendix C Experiments have been conducted on NVIDIA TITAN V (12 Go) for 1d datasets to NVIDIA RTX A6000 GPU with 49Go for 1d + time or 2d datasets. For all datasets, 800 PDEs are considered during training and 200 for testing. All metrics reported are evaluated on test samples (i.e. PDEs not seen during training. Coefficients as well as initial and/or boundary conditions can vary from training).
4.2 COMPARISON WITH BASELINES
We performed comparisons with several baselines including fully data-driven supervised approaches trained from data-loss only, unsupervised methods relying only on PDE loss, and hybrid techniques trained from PDE+ + DATA losses. Network size and training details are described in appendix [ In this experiment, we considered training the models using the training sets (physical losses or MSE when possible) unless stated otherwise.
Fully supervised We train a standard MLP learn the mapping (7, f.g) + O using as loss function
LDATA = g. Il] - Un, (. with uh (z) = Em= (z), the (.) being fixed B-Spline basis functions (see appendix DJ. We denote this baseline as MLP+basis. Unsupervised We compare our approach with unsupervised physics-informed models (Raissi et al. 2019). While the initial version of PINNs solves only one PDE instance at time and requires retraining for each new instance, we developed here a parametric version of PINNs (PPINNs) where the PDE parameters are fed to the network (similarly to (Zhang al. 2023). ). Finally, we used (Cho et al. 2024)'s 's (P2INNs) method as physics-infon informeed baseline specifically designed for parametric PDEs. In addition to meNNssmethods, DeepONet (PO-DeepONer for Physics- Only DeepONet) (Wang et al. 2021b), which is designed to learn an operator for function-to-funcii mappings from physical losses and handles parametric PDEs. The mapping learned for the two unsupervised baselines is (z., (,.) + (.y.f,g())) In order to provide a fair comparison with our optimization method, we fine-tuned the unsupervised basclines for each specific PDE instance for few steps (10 or 20). Comparison to preconditioning We compare our approach with vanilla PINNs (Raissi et al. 2019). i.e. by fitting one PINN per PDE in the test set and averaging the final errors. We optimize the PDE losses using L-BFGS (Liu & Nocedal 1989) and refer to this baseline as PINNs+L-BFGS. As discussed in (Rathore et al., 2024). L-BFGS can be considered as a nonlinear preconditioning method for Physics-Informen training strategy proposed by (Rathore et al., 2024) i.e. trained PINNs using successive optimizer (Adam+ + L-BFGS). This baseline is denoted as PiNNs-multi-ppt For these baselines, one model is trained and evaluated for each PDE in the test set. We report the reader to appendix for more details on the training procedure. Hybrid Finally, we compare our proposed method with neural operators, i.e., models trained to learn mappings (x, Y, f. + Un.f.g (x) using combination of physical and data loss: LDATA + LPDE. We use as baselines Physics- Informed Neural Operator (PINO) (Li et al., 2023b) and Physics-Informed DeepONet (PI-DeepONet) (Goswami et al.) 2022). As already indicated, for a fair comparison, the Unsupervised and Hybrid baselines are tine-tuned on each specific PDE instance for a few steps (10 on all datasets except for Heat for which 20 steps made). Ours We represent the solution tie with a linear combination of B-Spline functions for y (Piegl & Tiller 1996). This was motivated by the nature of B-Splines which allows to capture local
we also compare our solver to the Physic-informet
methods and fastens convergence. Finally, we use the
8

---

Published as a conference paper at ICLR 2025
phenomena. However, other bases could be used such Fourier, Wavelet Chebychev Polynomials. The neural solver Fe is composed of Fourier Layers (FNO) (Li et al. 2020) that allow us to capture the range of frequencies present in the phenomenon. We reter the reader appendix D for more details about the construction the B-Spline basis and the training hyper-parameters.
Table 2: Results of trained models metrics in Relative MSE on the test set. Best performances highlighted in bold, and second best are underlined.
1d
Id+time
2d
2d+time
Baseline
Helmholtz
Poisson
NLRD
Darcy-Floo
Heat
Supervised
MLP basis
4.66e-2
1.50e-1
2.85e-4
3.56e-2
6.00e-1 1
PINNs+L-BFGS
9.86e-1
8.83e-1
6.13e-1
9.99e-1
9.56e-1
PINNS-multi-ppl
8.47e-1
1.18e-1
7.57e-1
8.38e-1
6.10e-1
Unsupervised
PPINNs
9.89e-1
4.30e-2
3.94e-1
8.47c-1
1.27e-1
P2INNs
9,90c-1
1.50e-1
5.69e-1
8,38c-1
1.78c-1
PO-DeepONer
9.83e-1
1.43e-1
4.10e-1
8.33e-1
4.43e-1
PI-DeepONet
9.79e-1
1.20e-1
7.90c-2
2.76e-1
9.18c-1
Hybrid
PINO
9.99e-1
2.80e-3
4.21e-4
1.01e-1
9.09e-3
Neural Solver
Ours
2.41e-2
5,56e-5
2.91e-4
1.87e-2
2.31e-3
Table 2 presents the comparison with the baselines. We recall that the evaluation set is composed of several PDE instances sampled from unseen PDE parameters ff. f.).. The proposed method ranked first or second on all the evaluations. The most comparable baselines are the unsupervised methods, since at inference they leverage only the PDE residual loss, as method does. Therefore our method should be primarily compared to these baselines. Supervised and hybrid methods both incorporate data loss and make different assumptions while solving a different optimization problem.
Results:
Table illustrates that unsupervised Physics-informed baselines all suffer from ill-conditioning and do not capture the dynamics. Compared to these baselines, the proposed method improves at least by one order of magniude in all cases. PINNs baseline performs poorly on these datasets because of the ill-conditioning nature of the PDE, requiring numerous optimization steps to achieve accurate solutions (appendix D). This is observed on PINNs models for parametric PDEs (PPINNs and P2INNs) as well as on PINNs fitted on one equation only (PINNs+L-BFGS and PINNs-multi-opt), observe that our neural solver has better convergence properties than other Physics-Informed methods. As will be seen later it also converges much faster.
The supervised baseline performs well on all the PDEs except Poisson and Heat. The data loss used which is well-behaved and does not suffer from for training this model is the mean square We note that our method reaches similar or better optimization problems as the PDE loss does. performances every datasets, while relying only on physical information inference (algorithm 1 and solving a more complex optimization problem.
The hybrid approaches, do not perform well despite taking benefits from the PDE+DATA loss and from adaptation steps at test time. Again, the proposed method is often one order of magnitude better than the hybrids except NLRD, where has comparable performances. This shows that the combination of physics and data losses is also hard to optimize, and suffers from ill-conditioning.
4.3 OPTIMIZATION FOR SOLVING NEW EQUATIONS
The main motivation for our learned PDE solver is to accelerate the convergence solution, w.r.t. predefined solvers, for a new equation. In order to assess this property, we compare the convergence speed at test time inference with classical solvers, PINNs, and pre-trained PINO as detailed below. Results are presented in fig. for the Poisson equation with performance averaged on 20 new instances of the Poisson equation. This experiment is also performed on the other datasets in appendix [F]
Baseline optimizers As for the classical optimizer baselines, we used SGD, Adam (Kiggma & Ba 2015), and L-BFGS (Krishnapriyan al. 2021). These optimizers used to learn the coefficient of

---

Published as a conference paper at ICLR 2025
the B- B-Spline basis expansion in the model (=(x) = Ei= (z), This provides direct comparison to our iterative neural solver. PINNs We also compare to the standard PINNs (Raissi et al. 2019). i.e. by fitting one Neural Network (NN) per equation. Note that this requires full training from scratch for each new equation instance and this is considerably more computationally demanding than solving directly the parametrie setting. This baseline is similar the Adam optimizer mentioned above, except that the ansatz for this experience is multilayer perceptron instead of a linear combination of B-Spline basis. Hybrid pre-training strategies Finally we compare against the hybrid PINO pre- trained on set of several parametric PDE instances and then fine- tune on new PDE instance using only the PDE loss associated to this instance. Ours We train our model as explained algorithm and show here the optimization process at test time. In order to perform its optimization, our model leverages the gradient of the physical loss and the PDE parameters (coefficients, initial/boundary conditions). In this experiment, we use L = 5 steps for a better visualization (whereas, we used = 2 in table 2)
fig. compares the number of opti- mization steps required for the different meth- ods. Our neural solver converges very fast in only a few steps (5 here) to a good approxima- tion of the solution, while all the other meth- ods require thousands of iterations- we stopped here at 10, 000 steps. The classical optimizers (SGD, Adam, L-BFGS) do not converge for new equation. The baseline PINNs trained here from scratch on each new equation show an er- ratic convergence behavior. Pretrained PINO be- haves better than the other baselines but still did not converged after 10, 000 steps. This clearly demonstrates the potential of our learned solver to deal with physical losses w.r.t. alternative pre-defined solvers.
Result:
Teehcin sptriiation exlution oT
141
347
LEYES
Optmitation stes
Figure 2: Test-time optimization based on the phys- ical residual loss LIDE for new PDE Poisson.
4.4 COMPUTATIONAL TIME
An important aspect of solving PDEs is the computational time required for each solution. Methods should find a trade-off between achieving high performance and maintaining reasonable computational costs. In appendix E.3 provide the training (table [6) and inference (Ta times for our method compared to various baselines. Our results show that while our method has comparable training and inference times other approaches, it demonstrates substantially better precision (table 2
CONCLUSION
We have presented a PDE solver learned from data that allows fast test-time optimization of physical losses. Our method succeeds to considerably accelerate the optimization process for the complex problem of minimizing physical losses and several orders of magnitude faster than classical hand- defined optimization methods such as Adam L-BFGS. Limitations and Future Work While efficient, the proposed method can be further improved. First, training our iterative algorithms requires more memory than standard machine learning models due to the complexity of backpropagation through iterations, which becomes challenging in higher-dimensional bases. Second, we have focused on solution approximations expressed as a linear expansion in predefined bases. More expressive representations, such as neural networks, could be explored: however, our preliminary experiments indicate increased ill-conditioning due to the compositional nature of neural networks. More sophisticated training schemes could enhance the optimization process. Future work will investigate these directions to improve scalability and broaden the applicability of the proposed method. Reproducibility Statement Hyper-paramett
baselines configurations and training details are detailed in appendix D tables 4 and 5] and algorithms | and The creation of the datasets is explained in appendix C and tables and 3 Finally, provide theoretical analysis of the
model, under an ideal scenario in appendix
B Code is available at https:/ / /gi thub. com/ 2ailesB/
neur al par womeericcsolveee
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
Solving PDE is of crucial interest in many applications of science and engineering. While do directly target such real-world applications in this paper, one should acknowledge that solvers can be used in various ranges of scenarios including weather, climate, medical, aerodynamics, industry, and military applications.
ACKNOWLEDGMENTS
We acknowledge the financial support provided by DLACLIM 21-CE23-0017-02), PHLUSIM (ANR-23-CE23-0025--)) and PEPR Sharp (ANR-23-PEIA- 0008", "ANR", "FRANCE 2030").
DEEPNUM

---

Published as a conference paper at ICLR 2025
REFERENCES
Marcin Andrychowicz. Misha Denil, Sergio Gomez, Matthew W. Hoffmann, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning learn by gradient descent by gradient descent, 2016.
Jason Ansel, Edward Yang. Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary Vetito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, C. K. Luk, Bert Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Shunting Zhang, Michael Suo, Phil Tillet. Xu Zhao, Eikan Wang, Keren Zhou, Richard Zou, Xiaodong Wang. Ajit Mathews, William Wen, Gregory Chanan, Peng Wu, and Soumith Chintala. Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. In Proceedings of the 29th ACM International Conference Architectural Support for Programming Languages and Operating Systems, Volume 2, ASPLOS '24, pp. 929-947, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400703850. doi: 10.1145/362066.... 36446666...4 URL https:://ooioii 000/10. 0145/3620660 3640366
Francesca Bartolucei, Emmanuel de Bézenac, Bogdan Raonić, Roberto Molinaro, Siddhartha Mishra, and Rima Alaifari. Are neural operators really neural operators? frame theory meets operator learning, 2023.
Andrés Beltrán-Pulicee Ilias Bilionis, and Dionysios Aliprantis. Physics-informed neural networks for solving parametric magnetostatic problems. IEEE Transactions Energy Conversion, 37(:): 2678-2689, 2022.
Alex Bihlo. Improving physics-informed neural networks with meta-leamed meteieaaamedoiiimiiatoi.. Journal of Machine Learning Research, 25(14):1-26, 2024. UKL/tttp.//mmlrrroo////eeer///// html
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. Learning to optimize: A primer and a benchmark, 2021.
Woojin Cho, Minju Jo, Haksoo Lim, Kookjin Lee, Dongeun Lee, Sanghyun Hong, and Noseong Park. Parameterized physics-informed neural networks for parameterized pdes, 2024. URL fttps:://arxiv.orr/aaa 20..
Filipe de Avila Belbute- Yi fan Chen, and Fei Sha. HyperPINN: Learning parameterized
differential equations with physics-informed hypemetworks.. In The Symbiosis of Deep Learning and Differential Equations, 2021. URL http::
Weinan E and Bing Yu. The Deep Ritz Method: A Deep Learning-Bat
-aaeed Numerical Algorithm for Solving Variational Problems. Communications in Mathematics and Statistics, 6(1):1-12, March 2018. ISSN 2194-671X. doi: 10.10077:403004-11--0002
Howard C. Elman, David J. Silvester, and Andrew J. Wathen. Finite Elements and Fast Iterative Solvers: with Applications Incompressible Fluid Dynamics. Oxford University Press, 2nd edition, 2014. ISBN 9780199678792.
Lawrence C. Evans. Partial differential equations. American Mathematical Society, Providence, R.I., 2010. ISBN 9780821 849743 0821849743.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (eds.), Advances Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL htt.:ttpss.//proceedngggggs neurips cc/
paper_ paperr_flle//ppprr/00/////////// ottp://ew.....aaaooooeeeeeeeeeea Paper. pdff
Somdatta Goswami, Aniruddha Bora, Yue Yu, and George Em Karniadakis. Physics-informet informed deep
neural operator networks, 2022.
12

---

Published as a conference paper at ICLR 2025
Dan Hendrycks and Kevin Gimpel. Gaussian Error Linear Units (GELUs), 2016.
Jun-Ting Hsich, Shengjia Zhao, Stephan Eismann, Lucia Mirabella, and Stefano Ermon. Learning neural pde solvers with convergence guarantees, 2019.
Xiang Huang, Zhanhong Yc, Hongsheng Liu, Beiji Shi, Zidong Wang, Kang Yang, Yang Li, Bingya Weng, Min Wang. Haotian Chu, Fan Yu, Bei Hua, Lei Chen, and Bin Dong. Meta-autoodecoder for solving parametrie partial differential equations, 2022.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
Alena Kopaničáková,, Hardik Kothari, George Em Karniadakis, and Rolf Krause. Enhancing training of physics- chysics-informed informed neural networks using domain-decompositioo based preconditioning strategies. arXiv preprint arXiv:2306. 17448, 2023.
Nikola Kovachki, Zongyi Li. Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces, 2023.
Aditi S. Krishnapriyan, Amir Gholami, Shandian Zhe, Robert M. Kirby, and Michael W. Ma- honey. Characterizing possible failure modes in physics-ifformed neural networks. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, vir- tual, pp. 26548-26560, 2021. URL dF438e5286f 2277
nwwripssse.eeee wwtppppee//2//ee/cc /hash/
Randall J. Le Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time- Depeennnddd Problems. SIAM, Philadelphia, PA, 2007.
Kc Li and Jitendra Malik. Learning to optimize, 2016.
Yichen Li, Peter Yichen Chen, Tao Du, and Wojciech Matusik. Learning preconditioners for conjugate gradient PDE solvers. In Andreas Krause, Emma Brunskill. Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 19425-19439. PMLR, 23-29 Jul 2023a. URL httlhttts///pooeeee htt.:e/eeiin.........rss////////
mlr wwtpppss/////112eeee..
html
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations, 2020.
Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations, 2023b.
Dong C. Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical Programming, 45:503-528, 1989. URL https://appp wwwanticscholar..oot.
CorpusID: 5681609
G.R. Liu. Meshfree Methods: Moving Beyond the Finite Element Method, Second Edition. CRC Press, 2009. ISBN 9781420082104. URL WRL/httss://book http://bbookk.ogggglo... fwrpbookksidd--ww------
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence. 3(3):218-223 March 2021. ISSN 2522-5839. doi: 10310
$422566001-000302-5
org.iorr/1000 /0. 1 021 003002-5
Les Piegl and Wayne Tiller. The NURBS Book. Springer-Verie Verlag, New York, NY, USA, second edition, 1996.
Tian Qin, Alex Beatson, Deniz Oktay, Nick McGreivy, and Ryan P. Adams. Meta-pde: Learning to solve pdes quickly without mesh, 2022.
13

---

Published as a conference paper at ICLR 2025
M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential
equations. Journal of Computational Physics, 378:686-707, 2019. ISSN 0021-9991. doi: https///doi.rrg/11.0111......pp arricleepp//902219911333111
10.045. URL htw..: URR. wttp://iieeeeereee..tooosoiiiiie
Bogdan Raonić, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de Bézenac. Convolutional neural operators for robust and accurate learning of pdes, 2023.
Pratik Rathore, Weimu Lei, Zachary Frangella, Lu Lu, and Madeleine Udell. Challenges in training pinns: A loss landscape perspective. In Forty-first International Conference on Ma- chine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openrevi ew. net
Alexander Rudikov, Vladimir Fanaskov, Ekaterina Muravleva, Yuri M. Laevsky, and Ivan Oseledets. Neural operators meet conjugate gradients: The feg-no method for efficient pde solving, 2024.
Tim De Ryck, Florent Bonnet, Siddhartha Mishra, and Emmanuel de Bézenac. An operator precondi- tioning perspective on training in physics-informe machine learning, 2023.
L. S. H. Numerical Analysis of Partial Differential Equations. Pure and Applied Mathematics: A Wiley Series of Texts, Monographs and Tracts. Wiley, 2012. ISBN 9781118111116. URL
https://books.goott google. www.froooosssiicciiiaaaoo
Sandro Salsa. Partial Differential Equations in Action: From Modelling to Theory UNITEX).
Springer Publishing Company, Incorporated, 2nd edition, 2015. ISBN 3319150928.
Jie Shen, Tao Tang, and Li-Lian Wang. Spectral Methods: Algorithms, Analysis and Applications. Springer Publishing Company, Incorporated, 1st edition, 2011. ISBN 154071040x
Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics, 375:1339-113311 1364, 2018.
Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüggrr, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning, 2023.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich- Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33:7537-7547, 2020.
Maryam Toloubidokhti, Yubo Ye, Ryan Missel, Xiajun Jiang, Nilesh Kumar, Ruby Shrestha, and Linwei Wang. DATS: Difficulty-awe task sampler for meta-learning physics- informed neural networks. In The Twelfth International Conference Learning Representations, 2024. URL
https://openreview mett/forumiddEEvyyFss.
Kiwon Um, Robert Brand, Yun, Fei, Philipp Holl, and Nils Thuerey. Solver-in-the-lon Learning from differentiable physics to interact with iterative pde-solvers, 2021.
Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-iinfeieeee -inforee neural networks. SIAM Journal on Scientific Computing, 43(5):A3055 A3001, 2021a.
Sifan Wang, Hanwen Wang. and Paris Perdikaris. Learning the solution operator of parametric partial differential equations with physics inffrmeedd deeponets, 2021b.
Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Jourmal of Computational Physics, 407618 2022. ISSN 0021-9991. doi: http:://oooooy//10001//..oooo2p1 #tttccee/pi//00022199111111000
URL http://ss/www..iieeeeeeeeett..t/
14

---

Published as a conference paper at ICLR 2025
Yuan Yin, Vincent Le Guen, Jérémie Dona, Emmanuel de Bézenac, Ibrahim Ayed, Nicolas Thome, and Patrick Gallinari. Augmenting physical models with deep networks for com- plex dynamics forecasting*. Journal of Statistical Mechanics: Theory and Experiment, 2021 (12):124012, December 2021. ISSN 1742-5468.
doi: 10. 12.1088/14225546//ccaaa02
URL
1088/174444 5468/ ac3ae5
Runlin Zhang, Nuo Xu, Kai Zhang, Lei Wang, and Gui Lu. parametric physics-informed deep learning method for probabilistic design of thermal protection systems. Emergies, 16(9), 2023, ISSN 1996- 1073. doi: 10.3390/en 16093820. URL, http::tts:///// hwwp:://wwwwwww.. wwp.
Anthony Zhou and Amir Barati Farimani. https://arxiv.or attp://////...1112///2 17722
Masked autoencoders are pde learners, 2024. URL
O. C. Zienkiewicz, R. L. Taylor, and J. Z. Zhu. The Finite Element Method: Its Basis and Fundamen- tals. Butterworth-Heiinemamm Oxford; New York, 6th edition, 2005.
15

---

Published as a conference paper at ICLR 2025
A DETAILED RELATED WORK
PDE solvers: Many tools for numerically solving PDEs have been developed for years. The standard methods for PDE include Finite Differences (FD), Finite Volume (FV), Finite Element Method (FEM), spectral and multigrid methods, and many others (S. H 2012 Liu 2009). While these methods are widely used, they often suffer from high computational cost for complex problems or high-precision simulations. To address these challenges, integrating deep learning (DL. into solvers has emerged as a promising approach. Current solutions include incorporating correction terms into mathematical solvers to reduce numerical errors (Um et al. 202I). Some work such Hsich et al. (2019); Li et al. (203a);; Rudikov et al. (2024); Kopaničaková et al. (2023) build a method to directly enhance the convergence of numerical solvers through preconditioner learning. As an example, Rudikov et al. (2024);[ Li et al. (2023a) use neural operator to approximate conditioner for the flexible conjugate gradient method or Hsieh et al. (2019) for the Jacobi method. Another example of preconditioner learning lies in Li et al. (2023a), where the author uses GNN to assess symmetry
and positive definiteness.
Unsupervised training: Physics-Infornen Neural Networks (PINNs) Raissi et al., 2019) have been a pioneering work in the development of DL method for physics. In these models, the solution is neural network that is optimized using the residual loss of the PDE being solved. However, this method suffers from several drawbacks. First, as formulated in Raissi et al. (2019), PINNS can solve one instance of an equation at a time. Any small change in the parameters of the PDE involves a full retraining of the network. Efforts such as BultráánPulido et al.] (2022); Zhang et al.)
(2023); Cho et al. (2024) have attempted to address this limitation by introducing parametric versions PINNs capable of handling parametric equations, while Huang et al. (2022); Qin et al. (2022)
explores meta- leta-learning approaches. Other approaches to improve PINNs generalization include using neural operators (Wang et al. 2021b), or hyper-network (de Avila selbute-Peres et al.) (021).
Moreover, PINNs have shown convergence difficulties: Krishnapriyan et al. (2021) show that PINNs losses have complex optimization landscapes, complicating training despite adequate neural network expressiveness. Approaches like those detailed in WWng et al. (2022) adopt Neural Tangent Kernel (NTK) perspective to identify reasons for failure and suggest using adaptive weights during training to enhance performance. Additionally, studies such as Ryck et al. (023) demonstrate that PINNs suffer from ill-conditioned losses, resulting in slow convergence of gradient descent algorithms. Recently, (Rathore et al., (2024) shows how training strategies can improve the convergence of Physics-Informed Neural Networks and show that specific optimizers such as L-BFGS act as conditioners on the physical losses.
Supervised training: In contrast to the unsupervised training of Physics-informed Neural Net- works, purely data-driven models have demonstrated remarkable capabilities for PDE simulation and forecasting. In most of the existing literature, the entire solver is replaced by DL architecture and focuses on directly computing the solution from a given input data. A widely studied setting is operator learning which learns mappings between function spaces (Li et al. 2020: Kovachki et al. 2023 Lu et al., 202T). This method is very efficient, with the downside of relying on quite large quantities of data for training in order to ensure adequate generalization. A Additionally, the neural network does not have access to the PDE in itself, only indirectly through the data. To ensure physical constraints in purely data-driven training. hybrid models have been proposed. The latter relies both the available physical knowledge and some data. Some examples include the Aphinity model (Yin) et al. 2021) (where the authors assume partial knowledge of the physics and learned the remaining dynamics from data), Physics-informed Deep Operator Networks (PIDON) (Wang et al. 2021b Neural Operator (PINO) (Li et al. 2023b) (DeepONet
(Goswami et al., 2022), Physics-informet
architecture (Lu et al. 2021) or Neural Operator models (Kovachki et al. 2023 Li ct al. 2020
respectively with a combination of data and physical losses).
Improving the learning scheme and optimizers through data-driven training has been studied since Li & Malik (2016) and Andrychowicz et al. (2016). These works propose to learn the optimizer of neural networks, which are classically optimized through gradient- gradient-based algorithms such Adam. They focus improving training strategies for neural networks, which do not suffer from the optimization issues and ill-conditionin properties of physics-informet losses. We refer the reader to the survey of Chen et al. (2021) for a complete overview. The closer work to ours is the very
Learning to solve:
16

---

Published as a conference paper at ICLR 2025
recent work of Bihlo (2024) in which the author assesses the capabilities of learned optimizers for physics-inis boysics-informed neural networks. The main difference with our work relies on the problem setting. Bihlo (2024) considers learning optimizer single equation, and for different neural networks initialization, while we focus on efficiently solving several instances of parametric PDE with varying PDE parameters 7: J.
B
THEORETICAL ANALYSIS OF OUR METHOD AND PINNS
Setting. We consider the following linear PDE:
Du(x) = f(x), x E Ω u(x) = g(z), aΩ,
(12)
where Ω C R is an open bounded domain, D is linear differential operator, f(x) a given function in Q and g(z) is given function on the boundary An.
B. THEORETICAL ANALYSIS OF PINNS
Our aim to find an approximate solution uh (z), parameterized by θ E RN,, θ = [0:] that minimizes the loss function:
LPDE (0) LRes( + (O)
(13)
where:
CReS (0) 2 (Due(x) (zx)) dx, LBc( 1 / (ue(x) g(x))² dr, 2
(0)))dda
and À > is regularization parameter balancing the PDE residual and boundary conditions.
perform gradient descent updates with step size n,. step 0, updates write as:
0k+1 = 9 - 1/ (p))
We establish the following theorem regarding the convergence rate gradient descent.
Theorem 2 (Convergence rate of PINNs). Given a linear ansatz us(x) ΣN (,0;(x), the number of steps (ss) required to achieve error ||0 O*1l2 < s satisfies:
N(e) O (n(A) (!)),
(14)
where x(A) the condition number of the matrix jaax defined by:
Aij / (Dos(x)) (Do, (z), dr / (1)))
(15)
Proof. Since ue(x) = E (x), we have:
(u)(#) dr(x),
a(Duo(1)) (x).
00,
00,
The gradient of the residual loss is:
VoLRe((H) = (Due - f(x)) Dō(a)) dr.
17

---

Published as a conference paper at ICLR 2025
where Do(x) is the vector with components (z). Similarly, the gradient of the boundary loss is:
VeLBc(() = / (ea(a) ((x)) ((x) dz.
where ((x) is the vector of basis functions evaluated at æ. Therefore, the total gradient is:
VeCrDE(O) AO b,
where the (positive, semi-definite) matrix and vector are defined as:
/ (Doi(z))(oo( (Do, dx / 8n Jn
A1,
(x(x),
(16)
b, | (())Do + X A/f0 ((z)
Thus, the gradient descent update becomes:
*+1 = - - b).
Subtracting @r (the optimal parameter vector satisfying AQ* = b) from both sides:
K+I 0 = =0 -' -' - 6*).
Simplifying:
Ok+1 0" -id-nAip ( 0*).
By recursively applying the update rule, we obtain:
+ - = (Id - -r).
Since A is symmetric positive definite, it has eigenvalues Àι < A2 S ... S An with A > 0. To ensure convergence, we require 1j
Amaax (4) Choosing !j Amaz(A] with e < 2, we have:
A
1 m/ 1
Amax (A)
The spectral radius of Id 7A is:
max
{11
Xmnaxe (A) .11 a max {1-
Amin (A)
x((A) 11 c
where x<AA Amax(d) Amnia (A) is the condition number of A. By choosing c ensure - cl < Ra(a < 1. Thus, the convergence factor is:
and since x(A) 2 1, we have
P 1 k(A)
Therefore:
c (1-=(()) x(A)
)* | -0
| - - <
To achieve ||0, O²I12 & e, the number of iterations N(e) satisfies:
18

---

Published as a conference paper at ICLR 2025
N(e) In (c/l(00 - 12)
x(A)
Using the inequality In(1 - 2) v -r for 0 < x < 1, we get:
021 N(e) x(A) In (100 00
&
Thus:
((e) O (x(A) In (4))
We have shown that for linear ansatz (² = Ein (z), the convergence rate of gradient descent depends linearly on the condition number ki () of the system matrix A. large condition number impedes convergence, requiring more iterations to achieve desired accuracy
B.2 THEORETICAL ANALYSIS OF OUR METHOD
In practice, often work with multiple data points. For each data point, there is an associated parameter vector O E R. We are interested in the iterative update where the gradient is transformed by a neural network FF:
0 = - (V
(17)
where , represents the parameter vector iteration 1. Recall that F is trained minimize the loss after T. iteration steps for M data points:
M
DDATA
uk
(18)
Theorem 3. (Convergence rate of our method). Given linear ansatz ue(x) Em= (,),(x), assume F behaves like its linearization P
8F
The number of steps N' required to achieve v=
dv
an error 0|0 O*Il2 S
satisfies:
(3 (((P))) (!))
(19)
Moreover, if F minimizes CDATA this necessarily implies s(PA) = 1 S x(A). Consequently, the number of steps is effectively reduced, i.e., (s) < (().).
Proof. Since F behaves like its linearization P, the gradient descent update becomes (refer proof of Theorem 2 for steps):
0 - bb)
Let @" the optimal parameter vector minimizing LYDE- Then, the difference between the parameter vector iteration and the optimal parameter vector is:
0 - -"
0*) =(Id ()PA)()) 0*).
By recursively applying this update until the final step I., obtain:
19

---

Published as a conference paper at ICLR 2025
Ø - (Id - ())) -
Since we have multiple data points, each with its own parameter vector, consider the concatenation when necessary. Let's introduce E as the matrix whose columns are the parameter vectors:
E 1
Similarly, E" contains the optimal parameter vectors for each data point. The update for all data points be written collectively:
E.. E (Id- (Pd- =*). Since F minimizes LDATA, we have E,. E", implying:
(Id - -
Given that the values of Eo are iid and sampled randomly from continuous distribution, because the of singular matrices has measure zero, the square matrix (Eo [3o - is full rank (i.e., invertible), with probability 1. Thus, the only way for the above equality to hold is if:
(Id - nPA)* = 0.
This means Id - -yPA is nilpotent of index L. Consequently, all eigenvalues of Id rpA are zero, implying that all eigenvalues of PAA are equal to 17 which is the optimal condition number. Referring to the convergence analysis in Theorem 2
1 leading to (P) Amax (PA)/ Amia (PA) 1,
have:
(²) < x(PA) In
Eo - E 112
Which directly implies
0 (1)
(20)
With x(PA) 1, this leads to the desired result:
(' o (In (t)) (k(A)) (2)) =: N().,
Thus, the number of iterations required is significantly reduced compared to the case without the neural network preconditioner.
Discussion The convergence proofs for our method fundamentally rely the assumption of linearity the underlying problems. important to note that the theoretical analysis does not extend to non-linear cases. Consequently, for non-linear scenarios, the theory should be viewed primarily as a tool for building intuition or providing motivation, rather than a definitive proof. This due to the lack of established methods for rigorously studying the non- con-irncar regime, no known results currently address such cases.
20

---

Published as a conference paper at ICLR 2025
Under these conditions:
This optimal condition number implies that convergence is not only guaranteed but also optimal, requiring fewer iterations.
Guaranteed Convergence: The method reliably achieves convergence the optimal solution due to the reduced condition number.
Optimal Convergence Speed:
With x(PA) = 1, the neural network provides an enhanced convergence rate, resulting in fewer required iterations compared to the original system without the neural network.
21

---

Published as a conference paper at ICLR 2025
C DATASET DETAILS
For all datasets, we kept 800 samples for training and 200 as testing examples (except otherwise stated in the experiments).
C.1 HELMHOLTZ.
We generate a dataset following the 1d static Helmholtz equation eq. 21 For x E [0, 1[.
u(x)" + (u((x)
0,
((
(21)
w'(
60
The solution can be analytically derived: u(x) = a cos(wr + 3), with 3 arctan( we0
upp
cos(
and directly computed from the PDE data. We generate 1, 024 trajectories for training and 256 for testing with uo, Fig ~ N(00 1), and ~ (0..5, 50) and compute the solution on [0, 1] with spatial resolution of 256. For training, we keep 800 samples and use the complete dataset for the additional
experiments presented in figs. 12a and 2b
Moreover, we sub-sample the spatial resolution by 4 and
keep 64 points for training.
Trajoctory sample for Helmholtz w-34.482
Trajectory sample for Helmholtz 6/-.722
1.-
0.0
-1.0
0.1
2.6
(a)
(b)
Figure Samples from the Helmholtz Dataset.
C.2 POISSON
We generate dataset following the 1d static Poisson equation eq. 22 with forcing term. For E [0.
f(x),
u(O)
Uo
(22)
(0(
Un-
We chose f be non-linear forcing term: f(x) & E a sin(wx), with a; 100,00, 100), we used K = 16, 0.5, and solve the equation using backward finite difference scheme. We generate 000 trajectories with ug. to ~( 1) and compute the solution on [0, 1] with spatial resolution of 64.
Deaction-Difffssion We use a non-linear reaction-diffusion used in (Krishnapriyan et al. 2021 Toloubidokhti al.) 2024). This PDE has been shown be failure case for PINNs (Krishnapriyan et al. 2021). The PDE states as follows:
du(t,x)
8 pu((t
pult, x) u(t, x)
(23)
01
Ox
u(0,x) c- -2(
(24)
22

---

Published as a conference paper at ICLR 2025
Trajectory sample
forcing
Trjjctooyy sample
forcing
6.7
30
65-
64-
-07
02
-00
-00
-0.1
0.00
0123
0.50
0.7
1.00
0.00
0.25
0.3
2.75
100
0.0
025
0.50
0.5
1.00
0.00
0.2
0.00
5.75
1.00
(a)
Figure 4: Samples from the Poisson Dataset.
We generate 800 trajectories by varying v in [1,5] and p in [-5,5]. Spatial resolution is 256 and temporal resolution 100, which we sub-sample by 4 for training, leading to a spatial resolution of 64 25. The PDE is solved on [0, 1J2 in (Toloubidokhti et al. 2024
Trojectory sample Reaction Diffusion 2-245 p-0.50
Trojectory somple Reaction Difflssionn
Initial condition
y- 1-1.754 4.35.
Initial condition g
1.0
1.0
3.5
3.0
0.03
(a)
(b)
Figure 5: Samples from the Diaction-Diffusionn Dataset.
Reaction-Diffusion with initial conditions: To complexify the setting, we also change the initial condition of the problem (NLRDIC in the following). The initial condition is expressed follows:
6/4
u(r 0)
(25)
1=1
Where a, are randomly chosen in [0, 1] and h = 1 is the spatial resolution.
23

---

Published as a conference paper at ICLR 2025
Trajectory sample Beuction-Sifluee y -3822 p=2.972
Trajectory sample Eilliaiin 2-2239 p-0.449
Initia condition
Initial condition
140
2.-
0.7
26-
25
0.5
0.4
0.1
02-
0.1
00
20-
0.0
Figure 6: Samples from the Reaction-Diffusion Dataset with initial conditions changed.
C.3
DARCY FLOW
The 2d Darcy Flow dataset taken from (Li et al. 2020) and is commonly used in the operator learning literature (Li et al. 2023b Goswami et al. 2022).
- -V.(a(x)Vu(r)) = (ax x (0,
(26)
u u(x) 0
1)
(27)
For this dataset, the forcing term f is kept constant f and a(x) is piece-wise constant diffusion coefficient taken from (Li et al. 2020). We kept 1, 000 trajectories (on the 5, 000 available) with spatial resolution of 64 x 64.
Tra ita,octory semple for Darcy
Associatedd parametor
Trajectory samole for Darcy
Associ stod perameter
30
40
50
50
(a)
(b)
Figure 7: Samples from the Darey Dataset.
C.4 HEAT
As proof that our method can handle 2d + time, we consider the dataset proposed by (Zouu & Farimani 2024
tu( x. v.t)
wVVu(z, O,
(28)
u(I, v)
Σ
A₁ sin j=1
2Tlaaa
2Tlyiy + db))-
(29)
L
Where L 2, 1/ is randomly chosen between 10 3 x 10 21, A, in integers in (1, 3) and á is in [0, 2m). As difference with ZZhou & Farimani
0.5, 0.5), Lzj. Lay are
2024), randomly
24

---

Published as a conference paper at ICLR 2025
chose J between 1 and Jmax 5 to have more diversity in the represented frequencies in the data. The PDEs are sampled with spatial resolution 64 in x and and temporal resolution 100. However, during training the spatial resolution and 1. As for other PDEs, we use 800 trajectories for training and 200 for testing.
subsampled by 4 and the coordinates are scaled between 0
(a)
(b)
Figure Samples from the Heat dataset.
C.5
ADDITIONAL DATASET: ADVECTION
The dataset is taken from
Takamoto et al. 2023).
0u(t,
au((,z) B
dr O, 1), E (0,2),
(30)
Øt
u w(0, vo()), x (0, 1).
(31)
Where 3 is constant advection speed, and the initial condition is wo(x) Ek.=k... A, sin(ka
Φi), with k; 2zm, and n, are randomly selected [1, 8). The author used N for this PDE. Moreover, A₁ and de are randomly selected in [0, 1] and (o, 2T) respectively. Finally, Ly is the size of the domain (Takamoto et al. 2023).
Le
The PDEBench's Advection dataset composed several configurations of the parameter (40.1,0.2, 0.4, 0.7, 1, 4, 7)), each of them is composed of 10, 000 trajectories with vary- ing initial conditions. From these datasets, we sampled a total of 1, 000 trajectories for 3 E {0.2, 0.4, 0.7, 1, 4} (which gives about 130 trajectories for each (). This gives dataset with different initial conditions and parameters. Moreover, during training, we sub-sampled the trajectories by leading to a grid of resolution 25 for the t-coordinate and 256 for the x-coordinate.
Trajectory sample for Adtoection
Trajectory sumple Advection
5-2.0
Initial condition
8-0.4
Initial Initial condition.
2.7
3.75
230
30
3.2
0.5
1.00
0.75
0.3
0.50
0.55
01
060
0.0
Figure 9; Samples from the Advections Dataset.
25

---

Published as a conference paper at ICLR 2025
C.6 SUMMARY OF PROBLEM SETTINGS CONSIDERED
A summary of the datasets and parameters changing between 2 trajectories presented in table
Table 3: Parameters changed between each trajectory in the considered datasets in the main part of the paper as well additional datasets (Advections and NLRDIC).
Dataset
Changing PDE data
Range / Generation
W
[0.5, 50]
Helmholtz
4g
N(0, 1)
NO 1)
0
A
[-100, 100]
Poisson
40
No 1)
%
(O 1)
[1, 5]
Reaction-diffusion
P
|-5, 5]
V4, (-A + +1)
Darcy
a(z)
with v 12 1lg +3 1R.
[2 10-3,2 20-21
Jmax
1,
Heat
A
-0.5
Kz K
(1,2,3) 3)
[0. 2m
3
{0.2, 0.4, 0.7, 1, 2,
A;
[0, 1]
Advection
6,
[0, 2T
(2kTT 18
V
[1, 5)
NLRDIC
P
[-5. 5, 5
00, 1]
26

---

Published as a conference paper at ICLR 2025
D IMPLEMENTATION DETAILS
We add here more details about the implementation and experiments presented in section
We implemented all experiments with PyTorch (Ansel et al. 2024). We estimate the computation time needed for development and the different experiments to be approximately 300 days.
D. B-SPLINE BASIS
We chose a B-Spline basis construct the solution. We manually build the spline and compute its derivatives thanks to the formulation and algorithms proposed in (Piegl & Tiller 1996). We used Splines of degree
and constructed the Splines with different configurations:
Take N + equispaced nodes of multiplicity 1 from & local basis with no discontinuities (see fig. [10a] represented by a shifted spline along the x-axis (denoted as shi fted in the following).
+ d N This gives smooth
nodes multiplicity 1 and nodes of multiplicity (typically on the boundary nodes: 0 and 1). This means that nodes and 1 are not differentiable (see fig. Tob). We call this set-up equisedced.
Use N +
Bosts Functions
WIth opiictty Basis Funchion
muppiiiityy
0.8
10
0.-
(a)
(b)
Figure 10: B-spline basis with N = 10 terms with shifted spline (Left) and higher multiplicity nodes
(Right). Dashed lines represent nodes' position with color the darker, the higher the multiplicity.
Higer-dimension basis For 1d + time, 2d dataset, and 2d + time we build 2 (or 3) dimensional B-spline basis, i.e., we treat the time coordinates as spatial ones. To build such bases, we compute the Cartesian product between the 2 (or 3) Id-bases, 1 per dimension. This means that for 2d dataset, for which we chose to use bases of size N1 and N2 for each coordinate, the resulting basis will have N1 + N2 N1 N2 terms. This method makes the training more costly and several techniques to improve its scalability could be used. For 3d datasets, the number of terms in the basis cubic.
D.2 TRAINING DETAILS
In our experiments, neural networks are trained using the Adam optimizer. For network optimization, we employ a smooth /1 lloss for our solver while for other baselines, we use MSE loss and/or physical losses. All models are trained for at least 1, 500 epochs on datasets composed of some sampling of and/or ff and/or g. not stated otherwise, we train our proposed method for 750 epochs and baselines for 1, 500 epochs. We use the Adam optimizer and an initial learning rate of 0.001. We use an exponential learning rate scheduler that lowers the learning rate by 0.995 every epoch. Experiments conducted on NVIDIA TITAN V (12 Go) for 1d datasets and NVIDIA RTX A6000 GPU with 49Go for 1d + time, 2d datasets or 2d time. We recall in algorithm 3 the pseudo-code for training proposed method.
27

---

Published as a conference paper at ICLR 2025
Algorithm 3: Training algorithm for learning to optimize physics losses.
Data: Og E IR", PDE (Y, f: g), sample values u(x)
Result: Fe
for e = 1... epochs do for (PDE, x, u) in dataset do
Initialize O
Estimate Ø2 from Oo (7 f. using Algorithm ! Reconstruct ue, (α)
Update the parameters & of JF with gradient descent from the data loss in Equation 10.
end end
return F
We make use of two nested components: the solver for providing the approximate solution the PDE and the optimizer that conditions the training of the solver. Both are using gradient descent but with different inputs and objectives. The former optimizes the PDE loss (inner loop), while the latter optimizes the gradient steps of the solver through conditioning (outer loop). Optimization of the two components proceeds with an alternate optimization scheme. In particular, this implies that the map Fe is kept fixed during the inner optimization process.
D.3 MODELS
We present here the training details for our model and the baseline. The results can be found in table 2 In all our experiments, details of the model architecture on each dataset are presented in table
GeLU (Hendryeks & Gimpel (016) activation function. The
Hyperparameters for Helmholtz: For the baselines, we empirically searched hyperparameters to allow the network to handle the high frequencies involved in the Helmholtz dataset. Unfortunately, other network architectures did not improve the results. We proceed similarly for other datasets.
For the Helmholtz dataset, we trained our model for 2000 epochs with plateau learning rate scheduler with patience of 400 epochs. For the Poisson dataset, we trained parametrie PINN model for 5000 epochs with a cosine annealing scheduler with maximum number of iterations of 1000. For the Reaction-diffusion dataset, we consider an initial learning rate of 0.0001 instead of 0.001. Finally, P2INNs is trained for 5, 000 epochs using the Adam optimizer and an exponential scheduler with patience 50.
Training of PPINNs:
The conditioning of the problem highly depends on the parameters of the PDE and initial/boundary conditions. This can lead to unstable training when optimizing PINNs, requiring specific parameter configurations for each PDE. To avoid extensive research of the best training strategy, found configuration that allowed good fitting of most of the testing dataset. This means that in the values reported in table 2 we removed trainings for which the losses exploded (only few hard PDEs were removed, typically between none to 20). This prevents us from extensive hyper-pan parameter tuning on each PDE. Please note that this lowers the reported relative MSE, thus advantaging the baseline. We detail in tables an hyper-parameters for each dataset.
PINNs baselines (PINNs + L-BFGS and PINNs-multi-opt):
Basis configuration: For all datasets, we use Splines of degree built with shi fted nodes. We change the number of terms in the basis depending on the problem considered. For the Id-problem, we use 32 terms. For the 2d-problem, 40 elements are the basis each dimension, except for Reaction-Diffussion where the variable has 20 terms. Moreover, during training, we use the projection of the initial conditions and/or parameters and/or forcing terms function in the basis as input to the networks. Finally, for experiments using the Hear dataset (i.e. 3d basis), we used 15 terms for the and y spatial coordinates and 10 terms for the t-coordinate.
28

---

Published as a conference paper at ICLR 2025
Table 4: Architecture details of our model and baselines
Dataset
Model
Architecture
1d
1d time
2d
2d + time
Helmholtz
Poisson
NLRD
Darcy
Heat
MLP depth
3
3
3
3
3
PINNs
MLP width
256
256
256
256
256
MLP depth
5
3
3
PPINNS
MLP width
64
256
256
256
256
Branch Net depth
2
5
5
5
Branch Net width
256
256
256
256
256
PO-DeepONet
Trunk Net depth
3
5
5
Trunk Net width
256
256
256
256
256
Enc params depth
4
4
4
4
4
Enc params width
256
256
256
256
256
Emb params
128
128
128
128
128
Enc coord depth
3
3
3
3
3
P2INNs
Enc coord width
256
256
256
256
256
Emb coord
128
128
128
128
128
Dec depth
6
6
6
Dec width
256
256
256
256
256
Activations
GeLUU
GeLU
GeLU
GeLU
GeL.U
MIPP depth
5
5
5
5
5
MLP + basis
MLP width
256
256
1,024
1,024
1,024
Branch Net depth
3
2
5
5
5
Branch Net width
256
256
256
256
5
Pl-DeepONet
Trunk Net depth
3
2
5
5
5
Trunk Net width
256
256
256
256
256
FNO depth
3
3
4
3
3
FNO width
64
64
64
64
64
FNO modes
16
10
20
7
PINO
FNO modes 2
5
20
7
FNO modes 3
5
FNO fe dim
64
64
64
64
64
FNO depth
3
3
3
3
3
FNO width
64
64
64
64
64
FNO modes
16
16
10
20
7
Ours
FNO modes 2
5
20
7
FNO modes
5
FNO fc dim
64
64
64
64
64
Table 5: Hyperparameters for PINNs+I-BFGS baseline.
Dataset
Hyper-parameter
ld
1d time
2d
2d time
Helmholtz
Poisson
NLRD
Darcy
Heat
epochs learning rate
100
000
1,500 1c-4
1,500
1,000
lc-4
le-5
le-3
lc-3
29

---

Published as a conference paper at ICLR 2025
Table 6: Hyperparameters for PINNs-multi-ope baseline.
Dataset
Hyper-parameter
1d
1d + time
2d
+ time
Helmholtz
Poisson
NLRD
Darcy
Heat
epochs Adams
800
800
800
1, 200
1, 1,200
epochs L-BFGS
200
200
200
300
300
epochs total
1,000
1,000
1,000
1,500
1,500
learning rate Adam
le-4
le-5
le-4
le-3
le-3
learning rate L-BFGS
1
le-3
1
1
30

---

Published as a conference paper at ICLR 2025
E ADDITIONAL EXPERIMENTS
E. ABLATION
We experimentally show some properties of our iterative method. These evaluations are made on a test composed of several instances of PDE with varying configurations (7, f.9) 9) that unseen during training. These experiments are performed on the Helmholtz equation which appeared as one of the most complex to optimize in our evaluation. We used L 5 in our method for better visualization, unless stated otherwise.
Error w.r.f the number of steps L On fig. 11 for a better generalization. However, we observed in our experiments that the generalization error stabilizes or even increases after the proposed 5 steps. This limitation of the solver should be investigated in future work in order to allow the model make more iterations. Note that for this experiment, we lowered batch size (and adapted the learning rate accordingly) when the number of steps increased (fig. 11.
we show that having more optimizer steps allows
Error with number of stezs.
2x13"
18"
14
12x10
20t4
2.0
numbor sta5s.
Figure 11: Error on the test set (Helmholtz equation) w.rl the number of iterations
Error w.r.t the number of training samples On figs. 12a and 12b we show that compared to other physics-informed baselines, our solver requires less data to learn to solve PDE. Note that in this simple example, the MLP+basis baseline also performed well. However, as shown in table [] this is not the case for all the datasets. The contribution of the iterative procedure clearly appears since with 2x less data, our model performs better than this baseline.
Data requiirement comparison.
Date requirement comparison,
1014
10
10-2
Outs
bosls
PiDessONNe
PiDessONer
1:? Number of training samp
107
30
Number training samp
(a)
(b)
Figure 12: Relative MSE on the test set w.n the number of training samples on Helmholtz (left) and Darcy (right) datasets.
31

---

Published as a conference paper at ICLR 2025
Optimization with LYDE Table shows that optimizing our network with physical loss greatly complicates training. Indeed, adding an ill-conditioned loss to the standard MSE makes training ill-conditioned.
Table 7: How the physical loss complexifies training and lowers performances. Metries in Relative MSE on the test set of Helmholiz equation.
Input
Relative MSE
LPDE + LDATA
1.11 2.19e-2
LDATA
Iterative update & SGD-based update In table g we compare two optimizer configurations. "Direct" means that the network directly predicts the parameters for the step (i.e. Ox+1 F CPDDE (Ox); 7/ f. g)), while "GD", corresponds to the update rule described before and using SGD as the base algorithm O++1 θ FF CPDe 7, f. g). The latter clearly outperforms the direct approach and shows that learning increments is more efficient than learning direct mapping between two updates. As shown table increasing the number of steps improves the performance (shown here for and 5 update steps). However, the performance does not improve anymore after few steps (not shown here).
Table 8: Comparison of different optimizer configurations for solving the Helmholtz equation. Metries in Relative MSE on the test set.
Relative MSE
N-steps
Direct
GD
1-step 5-step
1.08e-1
8.5e-2
9.07e-2
2.19e-2
Optimization with different inner learning rates table we study the performance of our proposed method with different inner learning 7). As expected, higher learning rate leads to better performances since the optimization is taking bigger steps.
Table Ablation on the inner learning rate. Metrics in Relative MSE on the test set of Helmholtz equation.
learning rate
Relative MSE
0.01
7.32e-2
0.1
4.93e-2
2.19e-2
Quantifying the importance of input feature for the learned solver As indicated in eq. (9p the inputs of our learned solver are (Y, f. 9, V (fD)) We performed experiments by removing either y or (PDE from the input (For the Helmholtz equation, there is no forcing term f). The BC are kept since they are part of the PDE specification and are required to ensure the uniqueness of the solution. This experiment (table 10 illustrates that conditioning on the PDE parameters y is indeed required to solve the parametric setting. Without 7, the solver has hint which instance should be used. The addition of the gradient information, V. which at the core of our method, is also crucial for improving the convergence and validates setting.
32

---

Relative MSE Published as a conference paper at ICLR 2025
Table 10: Effect of using the gradient as input wer the PDE parameters. Metries Relative MSE on the test set of Helmholtz equation.
Input
Relative MSE
y
3.75e-
g
1.07e-1
V
2.19c-2
We showed by a simple experiment that our model can handle nonlinear cases (see table TT). We propose model the solution using non-linear combination of the basis terms 0i. The relation between the dy modeled using a simple NN with one hidden layer and a tanh activation function. This experiment is performed on the Poisson PDE.
Table II:
Nonlinear combination of the basis. Relative MSE on the test set for our proposed method and comparison with other non linear models and optimizers.
baselines
PINNs+L-BFGS PPINNs
8.83e-1
4.30e-2
Ours
3.44e-3
Network architecture We show in table 12 an ablation on the layer type used for Fe in our experiments: MLP, Residual Network (ResNet), FNO and modified version of a MLP (ModMLP) taken from (Wang et al. 2021a) inspired from attention mechanism. We conducted this experiment on the Helmholtz dataset.
Table 12: Ablation different layer types.
Metric the test set of the Helmholtz equation.
Layer type
Relative MSE
MLP 8.25e-1 6,87c-1
ResNet
ModMLP
5.55e-2
FNO
2.41e-2
Irregular grids We show in table 13 an ablation on different types of grids: regular in table and irregular. The latter were created by sampling uniformly 75% of the points in the original grid. The sampled grids are different between each trajectory both during the training and testing phases. We conducted this experiment on the Helmholtz dataset. Finally, we show some reconstruction examples figs. 13a and 13b
Table 13: Comparison of the performances when training our solver using regular irregular and different grids for each PDE. Metrics on the test set.
Grid
Relative MSE
regular
2.41e-2
irregular
3.38e-1
This experiment shows that our method is capable of handling irregular grids. We observe a decrease in the performances (table [3) and in the reconstruction quality figs. aal 13b However, also note that our method is still capable of reconstructing the dynamic of the PDE, where most of the considered baselines failed to capture the oscillation in the Helmholtz PDE solution.
33

---

Published as a conference paper at ICLR 2025
Aucunsrrccion Groundtruth
0.-
54.0on
Nuconzzucion Ground truth
-0.5
-1.3
(a)
(b)
Figure 13: Comparison of the reconstruction of the solution when models are trained on regular (left)
and irregular grid (right).
Error as function of the PDE parameters values We illustrate in fig. 14 the behavior of the reconstruction of the MSE varying the PDE parameter values. We conducted this experiment on the Helmholtz PDE and varied w from -5 to 55 i.e. extrapolation of 10% beyond the parameter distribution (and kept fixed boundary conditions).
Evolution the MSE with PDE Heimholtz equation
107
Docp
10R
10-1
10-4
10-1
Figure 14: MSE comparison with PDE parameters w of the Helmholtz dataset and extrapolation outside of the training distribution.
We observe in fig. 14 that, as expected, solving Helmholtz PDE outside the training distribution fails for our model well for all baselines. The proposed method performs well inside the training distribution (as already observed in table 2 and behaves similarly to fully supervised methods on Oft-tO-Distriuttion (OOD) examples. Finally, other baselines, involving physical only hybrid training, only predict a mean solution.
E.2 STATISTICAL ANALYSIS
We provide statistical analysis some datasets and baselines. These experiments conducted using = 5 (instead of L in table 2). For computational cost reasons, we did not make this experiment all datasets but kept datasets in 1d (table (4)) dataset in 1d+time, and dataset in 2d (table 15) so that several configurations and data sizes are represented.
This analysis shows the robustness of proposed method w.r.f. initial seed.
34

---

Published as a conference paper at ICLR 2025
Table 14: Results of trained models with error bars (std errors) on 1d datasets metries Relative MSE on test set. Best performances are highlighted in bold, and second best are underlined.
1d
Baseline
Helmholtz
Poisson
Supervised
MLP basis
5.26e-2 +: 7.56e-3
1.58e-1 + 7.98e-3
PPINNS
8.33e-115.61e-3
3.59e-2:t2.1e--2
Unsupervised
PO-DeepONet
9.84e-146.93-44
1.79e-1#3.09e-2
Pl-DeepONer
9.8le-1
1.25e-1
Hybrid
PINO
3.27e-3 + 1.33e-3 4.07e-5 = 2.65e-5
Ours
2.17e-2 1.12e-3
Table 15: Results trained models with error bars (std errors) on 1d + time, and 2d datasets metrics in Relative MSE test set. Best performances are highlighted in bold, and second best are underlined.
1d time
2d
Baseline
Idnlrd
Darcy-Flow
Supervised
MLP basis
2.83e-5 + 6.83e-7
3.78e-2+2.09e-3
PPINNs
4.64e- + 1.92e-2
9.99e-142.63e-2
Unsupervised
PO-DeepONet
4.18e-1 + 1.04e-2
8.32c-142.51o-4
Pl-DeepONet
7.88e-2 + 1.96e-4
2.72c-1 + 4.44e-3
Hybrid
PINO
8.00e-5 + 1.00e-5
1.17e-1 + 1.42e-2
Ours
2.61e-5 + 2.53e-6
1.62e-2 + 3.06e-4
E.3 COMPUTATIONAL COST
Finally, we detail here the training and inference times of our method as well baselines.
As we can see in table 16 our model takes longer to train due to the iterative process occurring at each epoch. However, note that this is training time; inference time is similar to other methods (see table 17
We detail justification for this additional training time compared to PINNs variants below:
Comparison with vanilla PINNs: Consider the following experiment. Suppose we train a classical PINN on single instance of the Darcy PDE. Based on the training times shown in TableT6 using this method, we performed 1, 500 steps, which took 420 minutes for training (please note that the performances were less accurate than our model's performance). If we wanted to train a PINNs on each PDE of our entire test dataset for 15000 epochs (sometimes even more steps are required), this would take 4200 minutes or stated otherwise approximatly 3 days. Suppose now, that one wants to solve an additional equation. This will require an average of 0.226 seconds (see table 17) with our method, while PINNs would require an entirely new training session of approximately 20 minutes for 15, 000 steps. This makes method 5, 000 times faster than traditional PINNs for solving any new equation.
Comparison with PINNS parametric variants: Now let us consider two parametric variants of PINNs designed to handle multiple PDEs (PPINNs for parametric PINNs, P2INNs for the model proposed by (Cho et al., 2024)). In table 2 models was trained for 5, 000 epochs only. Let us consider training further as suggested for vanilla PINNs, until 15, 000 epochs. First, this training would require approximately 19h30m. Then, the optimization problem is still ill-conditioned, training further would probably not significantly improve the performance. We can extend this reasoning to the P2/NNs baseline, for which we observed similar performance and behaviors.
35

---

Published as a conference paper at ICLR 2025
Table 16: Training time of the experiments shown in Table of the paper on single NVIDIA
TITAN RTX (25 Go) GPU. d stands for days, h for hours, m for minutes.
Dataset
Helmholtz
Poisson
Idnlrd
Darcy
Heat
MLP+ basis
30m
20m
1h10m
2h
4h45
PPINNS
15m
20m
4h15m
6h30m
1d2h
P2INNs
2h
3h
11h
1d7h
1d8h
PODON
10m
10m
3h30m
1d9h
22h
PIDON
10m
10m
3h30m
1d10h
22h
PINO
15m
10m
1h10m
45m
2h40
Ours
30m
30m
4h30
10h15
1d 13h
Table 17: Inference time, averaged (in seconds) on the test set. All experiments are conducted single NVIDIA RTX A6000 (48Go). We report the mean time computed the test set to evaluate the baselines as performed in in the paper (i.e. with test-time optimization steps when applicable and 20 steps on the Heat dataset). We consider as inference the solving of PDE given its parameters and/or initial/bouddary conditions.
Dataset
Helmholtz
Poisson
Idnlrd
Darey
Heat
MLP basis
1.12e-2
1.18e-2
1.25e-2
1.19e-2
1.66e-2
PINNs+L-BFGS
274
136
369
126
234
PINNs-multi-op
15.5
25.5
16.5
105
90
PPINNs
3.09e-1
2.03e-1
2.91e-1
3,22e-1
5,45e-1
P2INNs
2.84e-1
3.09e-1
6.76e-1
1.29
1.23
PODON
3.27e-1
2.71e-1
4.38e-1
6.32e-1
8.85e-
PIDON
3.32e-1
2.96e-1
4.43e-1
6.35e-1
8.80e-1
PINO
3.14e-1
1.24e-1
5.19e-1
2.21e-1
8.08e-1
Ours
2.58e-1
2.16e-1
2.84e-1
2.26e-1
2.90e-
E.4 ADDITIONAL DATASETS
We provide additional experiments 2 new datasets: Non-linear complex setting and Advections. We refer the reader to appendix [ for the details about the PDE setting. These datasets were not included in the main part of the paper due to lack of space in the results table.
in a more
Table 18: Results of trained models additional datasets metrics in Relative MSE on the test set. Best performances are highlighted in bold, and second best are underlined.
Baseline
NLRDIC
Advections
Supervised
MLP + basis
9.88c-4
6.90e-2
PPINNs 3.7le-1
4.50e-1
Unsupervised
PO-DeepONel
4.36e-1
5.65e-
PI-DeepONet
5.39e-2
4.26e-1
Hybrid
PINO
3.79e-3
6,51e-4
Ours
1.41e-3
5,39c-3
These additional datasets show cases where baselines are performing very well on the considered PDE. For Advections, PINO reached very good performances. We believe that the Fourier layers used the model fit well the phenomenon. Indeed, the solution is represented using a combination of sine moving with time. This simple dynamics is easily captured using Fourier transformations. Our B B-Splines basis can be sub-optimal for this dataset. For the complexified version of NLRD, NLRDIC, is the supervised baseline that performs best. This dataset does not present high frequencies, and the MLP looks sufficiently expressive to find the coefficient in the basis. However, this baseline had difficulties in reconstructing the higher frequencies in Advections. Even if our model does not
36

---

Published as a conference paper at ICLR 2025
perform best these datasets, it is ranked second. We believe that these additional results show the robustness of our method across different physical phenomena.
E.5 TRAINING BEHAVIOR
We show in this section the evolution of the MSE as the training progresses (see fig. 131
Evelution duiing training on the train and sest of the Holmihotz couotion.
20
216
<20
d6
760
opach
Figure 15: MSE during training on the training and testing sets. Example shown the Helmholtz equation for results as presented in table 2
Reconstruction during training.
Reconstruction during training.
50
100
1.0
200
150
250
0.5
-0.5
1uth
-15
-1.5
0.0
0.2
0.4
2.6
1.0
0.0
0.2
0.6
9.8
1.0
(a)
(b)
Figure 16: Reconstruction of the solution PDEs during training. Example drawn from the test set.
In fig. 15 we show the MSE evolution both the train and test (evaluation every 50 epochs) and on figs. 16a and T6b some reconstruction (from the test set), with respect to the training epochs. The training set (800 PDE trajectories) corresponds PDEs used update the network parameters. while the test set (200 trajectories) are unseen PDEs for the network. This means that the model has not been trained or optimized on these PDFs. The test is composed of PDEs with varying PDE parameter values (w) and boundary conditions (ug. up). This illustrates that the generalization performance on new PDEs within the training distribution is rapidly achieved by the network.
37

---

Published as a conference paper at ICLR 2025
E.6 LOss LANDSCAPES
In this section, we propose visual representation of the optimization paths in the loss landscape. Figures 17 and I8 illustrates the behavior of the vanilla PINNs algorithm and of our "learning to optimize" " method. The plots represent respectively a 2D visualization of the physical loss landscape LPDE (fig. 17) and of the data loss landscape LDAIA (fig- (fi around the approximate solution (in our basis) of a given Helmholtz equation. The function basis used in this experiment has a size of 32. We used the technique in (Garipov et al.) 2018) to visualize the solution coordinates in the basis θ in the loss landscape (more details in appendix [.7.1). Superimposed on the loss background, we plot three trajectories obtained by starting from initial random vector Op.
Figure 17 visualizes the sharp and conditioned landscape of the physical loss LEDE, while Figure 18 shows the better conditioned landscape of the MSE data loss LDATA The figures provide intuition on how the proposed algorithm operates and improves the convergence. The optimizer J modifies the direction and magnitude of the stochastic gradient descent (SGD) updates in the physical loss landscape (Figure 17 left plot). It achieves this by utilizing solution values at various sample points to adjust the gradient, steering toward the corresponding minimum the mean squared error (MSE) landscape (Figure 18 central plot). While the descent remains within the residual physics error space, the learned optimizer provides an improved gradient direction, enhancing convergence efficiency (Figure 17 right plot). We describe below the different figures.
Left Columns in Figures and and 18 these figures show the gradient path (100 steps) obtained by directly optimizing the physical loss LIDE, similar to the vanilla PINNs algorithm. This trajectory highlights the ill-conditioning of the optimization problem associated to LPDE.
Center Column in Figures 17 and 18 these figures plot the trajectory of gradient- based optimization algorithm trained with a mean squared error CDATA data loss (100 steps), under the assumption that the solution values known at collocation points. While these quantities are not available our case, this visualization is included to illustrate the differences in convergence behavior between physics-based based and MSE-based loss functions when both are accessible.
Right Column in Figures 17 and 18] these plots illustrate the behavior of algorithm (the solver is trained with 2 gradient steps for this example). It demonstrates the effect of our learned optimizer and the significant improvements achieved compared to standard gradient descent algorithm on the physical loss.
We describe in appendix E.7 the construction of the figures figs. and 18
Optimization path in the L'DE loss landscape. Anars
Care
Ours
-25
452
34
38
34
O
Trajectory
O2
Figure 17: Loss landscapes and optimization trajectories on an instance of the Helmholtz PDE. The figure represents the PDE loss LYDE landscape and superimposed are examples of optimization paths computed using SGD on the physical loss (left), the data loss (center), and our method (right).
38

---

Published as a conference paper at ICLR 2025
Optimization path in the DATA loss landscape.
SG3 on Cav
5GD on Aoots
Ours
45
45
45
Trajectory
Figure 18: Loss landscapes and optimization trajectories on an instance of the Helmholtz PDE. The figures represents the DATA loss LATA and superimposed examples of optimization paths computed using SGD on the physical loss (left), the data loss (center), and our method (right).
E.7 PROJECTION THE BASIS FOR VISUALIZATION
Following the technique described in Garipov et al. 2018). we plot 2D slice of the loss function around the minimum solution. We explain below the method used for creating figs. 17 and
E.7.1 CREATE A BASIS FOR PROJECTION
We want visualize the loss landscapes around parameter solution O* of a given PDE. Due the high dimensionality of the parameter space, we use the method proposed in (Garipov et al. 2018). that involves creating 2d slice of the loss landscape for visualization. This methods projects the multidimensional landscape onto a 2D basis (u, v) so that any solution vector @ could be written as θ au + Se, with, a, 8 R and u, v E RN, 2 orthonormal vectors. Let Qu Vect (u, v} denote this projection space. For the visualization, the basis must include the solution vectors found by our algorithm. For that:
1. We start by rumning our optimization algorithm from random vector parameters On to find θ, RRN, our approximate PDE solution. We set wi O and set a second vector W2 Op. chosen here as the initial vector of our optimization process.
2. We generate a third vector E R randomly.
3. We then set: u = W22 W1 and (wa w1)
<un W1 W01 (w2 w1)
4. The resulting normalized vectors à Tu] and 6 [e form orthogonal basis containing
W1, We and Wy.
5. Using the solution parameters O2 as the origin for our newly created basis, we can compute slice the loss landscape where each solution O could represented in the (u, v) basis with its projection coordinates (a,, (9) as Oprojection + oá + Bö.
E.7.2 PLOT TRAJECTORIES
Finally, we run a baseline optimization procedures and get trajectories {Ot) with I. being the number of optimization steps, c.g. gradient descent, on the physical loss, the MSE loss our proposed method. We project the O, 's in the created basis (see appendix [E7.I] and plot this projection of the optimization trajectory on the loss landscape Qu,e as visualized on figs. and
E.7.3 ILLUSSRAATIN OF THE I.L-CONDITIONING OF THE PDE LOSS
As illustration of the ill-conditioning of the PDE loss, replicate fig. 17 using basis that further emphasizes this aspect. To build this basis, we use the procedure described in appendix by using 2 eigenvectors of the hessian of the PDE loss. First, compute Hess(Lpor) and its
39

---

Published as a conference paper at ICLR 2025
eigenvectors decomposition. We select the vectors associated to the highest and lowest eigenvalues, and respectively set them to W2 and W/g The resulting landscape visualization is shown fig.
Optimization path in the LPDE loss landscape.
SGD on
SGD
Ours
-45
44
44
9:
Irajectory
G
Figure 19: Loss landscapes and optimization trajectories on instance of the Helmholtz PDE. These optimization trajectories are computed using SGD on the physical loss (left), the data loss (center), and our method (right). The background represents the PDE loss LPDE.
Figure 19 clearly illustrates the characteristic shape of the ill-conditioned function LPDE- The two directions, extracted from the highest and lowest eigenvalues, are clearly visible on this PDE loss landscape. This highlights the difficulty of this optimization problem for standard descent methods.
40

---

Published as a conference paper at ICLR 2025
F QUALITATIVE RESULTS
This section is dedicated to visualization of the results of our model, baselines and optimizers, presented in section For each PDE considered, chose samples the test sets and compute the solutions with our model and the different baselines. We provide visualization samples with = ie. results proposed in tables and t3 detail more precisely the evolution of the solution at several steps of optimization. 3 datasets are shown with L 2: Heat for computational reasons (training with additional datasets trained only with - 2 (Advections and NLRDIC). Then, we show the evolution of the reconstruction of the solution with our method i.e. we plot the solution each step of the
much more expensive when the dimension of the problem increases) and the
optimization (figs. 20al 20b 23a 23b 26a 26b 29a 29b
32a 32b
35a
35b
38a
and 38b) and
compare the final prediction with baselines (figs, 2
21b
24a
276
30a
30b
33a 33b
36a 36b 39a and 39b). Finally, chose 20 (6 for Hear) PDES and reproduce fgg for every
dataset. More precisely, we optimize one PINN per PDE using Adam, we fit our basis using several optimizers (GD, ADAM, L-BFGS and our learned optimization process) and we fine-tune the learned PINO for 10, 000 steps and visualize the evolution of the MSE (averaged at each step on the selected PDE). These figures show the relevance of learning the optimizer when using physics-informed losses
(figs. 22 25 28
37 and 40
F.1 HELMHOLTZ
Buokution the recoretraction solut with optimization stege
Buckution the reconstrction of the solution optimizateo stepe.
3.0
2.0
2.5
2.01
0.8
0.6
(a)
(b)
Figure 20: Reconstruction of the solution using our optimizer on the Helmholtz dataset.
Solution for Helmholtz
Solution for Helmholtz
1.0
1.0
0.5
Solution
Miie no Deepotee P-hoool
Solution
0.0
0.0
PPBNS 10 20g
*
vi ipeas PACuspONe:
-1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.5
0.8
1.0
(a)
(b)
Figure 21: Visual comparison of the solutions for the Helmholtz equation.
41

---

Published as a conference paper at ICLR 2025
Test-time optimization evolution of MSE
10a
10a
ADAM
L-BFGS
MSE
10-1
PINN
PIND
Ours
Objertive
10-2
0
10"
10a
10a
10a
10f
Optimization steps
Figure 22: Test-time optimization based on the physical residual loss LPDE on Helmholtz. Note that, even though hardly visible this figure, the optimization is running very slowly and the PINN MSE (orange) decreases for the last steps. This dataset will probably need even more steps before convergence.
F.2 POISSON
Buolution the recenstrucion the solut on with optiiizzton stegs
Broluton of reconcrrccionn ofe solutin optinizadoo stegs.
Umeak
5.0
Soudton
2.5
-.5
(a)
(b)
Figure 23: Reconstruction of the solution using our optimizer on the Poisson dataset.
Solution for Poisson
Solution for Poisson
2
Grund put
PRNAS
Salution
1-
M.Peboe
Solution
X
Gronne PRAS
10-Dwezhhtt
0-
PRO
X
Dux
22
0.0
0.2
0.6
0.8
1.0
0.0
0.2
0.6
0.8
1.0
X
(a)
(b)
Figure 24: Visual comparison of the solutions for the Poisson equation.
42

---

Published as a conference paper at ICLR 2025
Test-time optimization evolution of MSE
10
10a
10
ADAM
L-BFGS
MSE
PINN
10
PIND
Ours
Objertive
10
10
10"
10a
10a
10a
10
Optimization steps
Figure 25: Test-time optimization based on the physical residual loss LYDE on Poisson.
F.3
DEACTION-DIFFUSICN
Evolution the reconstruction of the solution with optimization steps.
Ground truth
Initialization
Step
Step
Step
Step
Step Output
Evolution the reconstruction of the solution with optimization steps.
Ground truth
Initialization
Stop
Step
Stop
Step 4
Step 5: Output
x
(b)
Figure 26: Reconstruction of the solution using optimizer on the Reaction Diffusion dataset. Reaction-Diffusion
43

---

Published as a conference paper at ICLR 2025
Solution for
Ground truth
PPINNS
PO-DeepONet
MiL+baasi
Pi-DeepONet
PINO
Ours
Salution for Reaction-diffusion
Ground truth
PPINNS
PO-DeepONet
MLP+bass sis
Pi-DeepONet
PINO
Ours
Figure 27: Visual comparison of the solutions for the Reaction Diffuion- equation.
Test-time optimization evolution of MSE
10
10-1
ADAM
MSE
10-2
L-FGSS
PIND
Ours
10a3
10
1
10"
101
10a
10a
10*
Optimization steps
Figure 28: Test-time optimization based on the physical residual loss CYdE on NLRD.
44

---

Published as a conference paper at ICLR 2025
F.4 DARCY
Evolution of the reconstruction of the solution with optimization steps.
Ground truth
Initialization
Step 1
Step 2
Step
Step
Step S: Output
(a)
Evolution the reconstruction of the solution with optimization steps.
Ground truth
Initialization
Step
Step
Step
Step 4
Step S: Output
y
y
Figure 29: Reconstruction of the solution using our optimizer on the Darey dataset.
Solution for Darcy flow
PO-DeepONet
PI-DeepONet
(a)
Solution for Darcy flow
Ground truth
PRINNS
PO- De-DeopONet
MLP+basis
RI-DeepONet
PIND
Ours
(b)
Figure 30: Visual comparison of the solutions for the Darey equation.
45

---

Published as a conference paper at ICLR 2025
Test-time optimization evolution of MSE
10a
ADAM
L-BFGS
MSE
PINO
Ours
Objective
10
10a
101
10a
10a
10*
Optimization steps
Figure 31: Test-time optimization based on the physical residual loss LPDE on Darcy.
46

---

Published as a conference paper at ICLR 2025
F.5 HEAT
Evolution of the reconstruction of the solution with optimization steps. Ground truth
Initialization
Step
Step
tw0.00
te024
twc.00
te0 12
96
Evolution of the reconstruction of the solution with optimization steps. Ground truth
Initialization
Step
Step
1s0.00
180.24
te2.8
tm0.72
(b)
Figure 32: Reconstruction of the solution using our optimizer on the Heat dataset.
47

---

Published as a conference paper at ICLR 2025
Ground truth
PPINNS
PO-DeepONet
MLP-basis
DeepoNet
(a)
PO-DeepONet
Pe-DeppONet
Figure 33: Visual comparison of the solutions for the Heat equation.
Test-time optimization evolution of MSE
10a
10-1
L-BFGS
MSE
PINN
Ours
Objective
10
10-4
0
10a Optimization steps
10a
10a
Figure 34: Test-time optimization based the physical residual loss LPDR Heat. For computational reasons, this experiment has been conducted on 1, 000 steps only.
48

---

Published as a conference paper at ICLR 2025
F.6 ADVECTION
Evolution of the reconstruction of the solution with optimization steps.
Ground truth
Initialization
Stop
Stop
(a)
Evolution of the reconstruction of the solution with optimization steps.
Initialization
Step
X
(b)
Figure 35: Reconstruction of the solution using our optimizer on the Advection dataset.
49

---

Published as a conference paper at ICLR 2025
Solution for Advection
Ground truth
PPINNS
PO-DeepONet
MLP+basis
PtDeepONet
PINO
Ours
(a)
Solution for Advection
Ground truth
PPINNS
PO-DeepONet
MLP+basis
Pt-eeppNNet
PINO
Ours
Figure 36: Visual comparison of the solutions for the Advection equation.
Test-time optimization evolution of MSE
10
ADAM
L-FGSS
MSE
PIND
1
Ours
109
10a
10a
10a
10*
Optimization steps
Figure 37: Test-time optimization based on the physical residual loss LyDE on Advection.
50

---

Published as a conference paper at ICLR 2025
F.7 NON-LINEAR REACTION- DEACTION-DIFFUSION WITH INITIAL CONDITIONS
Evolution of the reconstruction of the solution with optimization steps.
Ground Initialization
X
(a)
Evolution of the reconstruction of the solution with optimization steps.
Initialization
X
(b)
Figure 38: Reconstruction of the solution using our optimizer on the NLRDIC dataset.
51

---

Published as a conference paper at ICLR 2025
Solution for Reaction-Diffusinn Diffusion
Ground truth
PPINNS
PO-DeepONet
MLP+basis
PI-DeepONet
PINO
Ours
Solution for Reaction-Diffusion
Ground truth
PPINNS
PO-DeepONet
MLP+basis
PI-DeepONet
PINO
Ours
Figure 39: Visual comparison of the solutions for the NLRD with varying IC equation.
Test-time optimization evolution of MSE
10
ADAM
10
L-FGSS
MSE
PIND
Ours
10
10-4
109
10a
10a
10a
10*
Optimization steps
Figure 40: Test-time optimization based on the physical residual loss CYDE on NLRD with varying IC.
52


Reviews:
[{"review_id": "sbbl3vsMJe", "reviewer": "Reviewer_dXw8", "summary": "This paper proposes a method to overcome the challenging optimization of PDE-loss in physics-informed methods by learning a modification on the gradient (w.r.t. model parameters), which intuitively serves as a preconditioner. The experiment displays a noticeable acceleration in convergence.", "questions": "-\tDo you have any intuition on what the model $\\mathcal{F}_{\\rho}$ is learning? The paper interprets it as a preconditioner, but this answer is very vague and lack of information.\n\nThink this way: given infinite data and a perfect training, what is the resulting mapping of $\\mathcal{F}_{\\rho}$?\n\nThen consider: what is the effect of $\\mathcal{F}_{\\rho}$? Does it necessarily (or, ideally) help reduce the condition number?\n\n-\tAre the baseline hybrid methods trained with exactly the same dataset (and same amount of data) as the proposed method?\n\n-\tL308, isn’t $m$ the number of grid points instead of trajectories?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "-\tThe method is full of imaginations!\n\n-\tThe empirical result in the test cases investigated are surprising. The proposed method help significantly accelerate the convergence of pde-loss optimization.", "weaknesses": "-\tThe theoretical analysis (main text and appendix B) is superficial and should not be considered as ‘proof’. In L814, the only important part of convergence analysis is circumvented by saying ‘the introduction of P as a preconditioner often results in κ(PA)<κ(A)’ instead of showing it. Thus the author actually didn’t prove anything. In fact, the effeteness of the method is highly related to whether it can help mitigate the condition number, which might not hold true, not supported by either theoretical or empirical evidence.\nAlso, L790-791 simplify the problem so much that the derivation could no longer provide insight to understanding the proposed method, unless the authors demonstrate the approximation to $Pv$ is reasonable in some sense.\n\n-\tTo train the model $\\mathcal{F}_{\\rho}$, one has to first fix a set of grid point. This is neither flexible nor suitable to leverage the advantageous of PINN. In many PINN papers, the pde residual are estimated in a Monte-Carlo manner by drawing random points and computing its pointwise residual, so as to eliminate the heave computational and memory cost. Moreover, since the paper are target at solving a family of PDEs, it is always the case that one has to use finer grids / higher resolution to adequately capture small-scale effects for some harder instances in the large PDE class.\n\n-\tRegarding the method itself, in my opinion, the performance significantly depends on how well $\\mathcal{F}_{\\rho}$ \n\ngeneralizes to unseen or even out-of-distribution data. Intuitively it seems like it would be harder for $\\mathcal{F}_{\\rho}$ \n\nto generalize than the surrogate model (as in PINO or physics-DeepONet) itself, since the mapping $\\mathcal{F}_{\\rho}$ try \n\nto approximate is more complicate. A systematic study on how well $\\mathcal{F}_{\\rho}$ needs to be learnt so as to ensure reasonable improvement on pde-loss convergence is necessary to make this method practical.\n\n-\tAs mentioned in the limitations, the method will possibly suffer from a memory issue and is slow to train since it needs to back-propagate through several optimization iterations. This  hinders the practicability of the method.\n\n-\tThe PDE considered are either 1D or linear, which are all quite toyish. The paper could benefit from studying their methods in some equations will optimization indeed brings severe challenges for physics-informed methods, for instance Navier-Stokes with relatively high Reynolds numbers. Otherwise, the experiments are not convincing.\n\n-\tMany details important for evaluating this paper are only mentioned in the appendix, e.g. the training and inference time cost. I am aware of the page limit, but the content should be briefly summarized in the main text, otherwise few readers notice they are there.\n\nTo conclude, although the paper is rich in novelty and has impressive performance in the experiments shown in the paper, and has the potential to provide new insight to the community, it is still not fully convincing- more understanding, discussion, and evaluation are necessary. Therefore, though I appreciate the author’s effort, I vote for rejection.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ylE1PsQ158", "reviewer": "Reviewer_N1S1", "summary": "Physics-informed Neural Networks (PINNs) are popular neural network-based approaches for solving partial differential equations and provide a compelling alternative to traditional PDE solvers. Unfortunately, the PINN objective yields a difficult optimization problem to train the network. The difficulty stems from the presence of an ill-conditioned differential operator in the loss, which leads to an ill-conditioned objective. The present paper proposes a framework for solving parametric PDEs based on learning an optimizer tailored to that class of problems. The authors provide a heuristic argument showing that the learned optimizer can be viewed as preconditioning the original loss. Preliminary experiments show the proposed approach yields better performance relative to existing methods.", "questions": "1) Why did you not include wall-clock times?\n\n2) Why did you not include a comparison(s) with a more sophisticated variant of PINN(s)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- This is the first time I've seen the use of learned optimizers in SciML. I think this approach has the potential to be very useful.\n- The potential connection between learning the optimizer is interesting, though still somewhat tenuous; see weaknesses below.\n- The proposed framework outperforms vanilla baselines, which is good.", "weaknesses": "**Related work**\n\n While I have not seen the idea of learning an optimizer applied in the SciML literature, it certainly is not new in the overall ML literature. The authors should explain how their approach fits into this existing body of work in the related work section.\nThe authors should mention Cho et al.'s recent work, which proposed a very effective method for solving parametric PDEs using PINNs.\n\n**Theory**\n\nThe theoretical support for the benefit of learning the optimizer is weak. \nThe argument provided in Appendix B is heuristic and relies heavily on the assumption that both networks are well-approximated by their first-order Taylor expansions.\nThis regime only holds under certain strong assumptions, which the paper neglects to mention.\nFinally, it is not apparent why it should be the case that $\\kappa(PA) \\ll \\kappa(A)$. \nSo, the potential connection between the learning optimizer is interesting but very tenuous.     \n \n**Experiments**\n\nThe experiments in the paper are very weak. \nComparing to the vanilla PINN alone using L-BFGS for training is inadequate. \nAs is well known, better performance is achieved by using the combination of Adam+L-BFGS. \nPlus, more sophisticated optimizers, like Muller et al.'s natural gradient approach or Rathore et al.'s NNCG, can greatly enhance PINN training.\nIt would be much more compelling to compare to one of these more sophisticated approaches to training PINNs.\n\nThe paper mentions the ability to cover parametric PDEs as a contribution, but none of the experiments demonstrate this advantage. Moreover, the method introduced in this paper should be compared to a dedicated PINN method for parametric PDES like that of Cho et al. (2024).\n\nThe authors' approach clearly has a higher time cost than the vanilla approach, but they do not provide runtimes for their method relative to the vanilla method. This makes it challenging to develop a concrete idea of how much more expensive their approach is.\n\n**Presentation**\n\nThe text has many grammatical errors, which can sometimes make the presentation difficult to follow. I recommend the authors do a careful read-through and make appropriate edits. \nThis will greatly improve the readability of the paper.\n\n**Overall** \n\nI believe the paper contains some interesting ideas that have the potential to enhance the applicability of PINNs. \nUnfortunately, the paper is not ready for publication at ICLR. \nThe experiments are lacking, so it is difficult to determine if the approach here would improve over more sophisticated variants of PINNs,  which are used in practice.\nIn addition, the theoretical contribution is too heuristic, and the presentation needs polishing.\n\n**References**\n\nCho, W., Jo, M., Lim, H., Lee, K., Lee, D., Hong, S. and Park, N., 2024. Parameterized physics-informed neural networks for parameterized PDEs. arXiv preprint arXiv:2408.09446.\n\nMüller, J. and Zeinhofer, M., 2023, July. Achieving high accuracy with PINNs via energy natural gradient descent. In International Conference on Machine Learning (pp. 25471-25485). PMLR.\n\nRathore, P., Lei, W., Frangella, Z., Lu, L. and Udell, M., 2024. Challenges in training PINNs: A loss landscape perspective. arXiv preprint arXiv:2402.01868.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TVUhTar6C8", "reviewer": "Reviewer_g6hN", "summary": "This paper proposes an optimization process that can learn a set of parametric PDEs quickly and precisely while addressing the ill-conditioning issues that traditional PINNs face. The authors first train a physics-informed neural network on a dataset containing different PDE coefficients, initial conditions, and boundary conditions, enabling the network to predict PDE solutions in the initial step. Then, a solution approximation expressed as a linear expansion in a pre-defined basis is optimized to adapt to each PDE instance. This method enables fast learning of different PDE instances and effectively overcomes ill-conditioning issues.", "questions": "My understanding is that the objective function for training the neural solver is data loss + PDE loss, which represents a soft-constrained optimization problem. Have you considered using a hard-constrained method, as in Lu et al. [1]?\n\n[1] Lu, Lu, et al. \"Physics-informed neural networks with hard constraints for inverse design.\" SIAM Journal on Scientific Computing 43.6 (2021): B1105-B1132.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "A very good paper demonstrating a novel approach to addressing PINN’s limitations, with extensive experimental results to support the claims. \n\nThe paper presents strong motivation and a thorough review of related work. \n\nThe methods and the overall paper are easy to follow.", "weaknesses": "I have some questions about the training of the neural solver.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XYqbjirvaS", "reviewer": "Reviewer_2vqB", "summary": "This paper addresses the problem of optimising Physics informed deep neural networks for solving PDE. Specifically, the paper addresses the issue of ill-conditioned optimization problems. \n\nUsing a Fourier representation, the authors clearly lay out the problem with ill conditioning of general PINNs and give a numerical example to illustrate this. \n\nThe paper describes how the problem can be transformed using a neural solver into one that can be optimised using gradient descent more quickly than the original problem.", "questions": "My suggestions are included in the \"Weaknesses\" section and I would be very pleased to hear the authors' response to these suggestions.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall the paper does a good job of explaining the problem and describing the proposed solution. However there are a number of issues that need to be expanded upon in order to improve the thoroughness and reliability of the findings. \n\nIn general I think the paper represents a useful contribution to the literature and I am leaning towards an acceptance. However, I do think that it could be significantly improved if some of the findings detailed in the appendices could be brought into the main body of the paper as described in this review. Although the paper is fully 10 pages long, some of the descriptive text, especially in the early part of the paper could be made more concise to make space for these valuable aspects of the findings to be included in the main body of the paper.", "weaknesses": "The parameters $\\Theta$ are updated using gradient descent. This is nested within a additional optimisation process for tuning the parameters $\\varrho$. Some results should be included in the main body of the paper which show the CPU time, including for the optimisation of $\\mathcal{F}_\\varrho$. There are some results presented in Appendix F but these should be discussed in the main paper as they are important for evaluating the usefulness of the method presented. \n\nThe paper describes the problem in terms of the condition number $\\kappa(A)$. A sketch proof is included in Appendix B to show that convergence is guaranteed and its speed is enhanced using this structure. However the wording around this sketch proof is quite confusing. Section B.2 there states \"The introduction of $P$ as a pre-conditioner \\textit{often} results in $\\kappa(PA)<\\kappa(A)$.\" and \"the condition number is improved under \\textit{some} optimality conditions. However in Sections B.3 and B.4 convergence and optimality is described as \\textit{optimal} and \\textit{guaranteed}. This inconsistency should be addressed due to the importance of this in supporting the empirical findings within the main part of the paper. \n\nThe results in the paper show very good improvements in the optimisation time and the MSE for the example PDEs used. However a range of parameters for each PDE is quoted in table 1. In table 2 the MSE results for the different equations are shown, however there is no indication of how these errors vary within the parameter ranges. There are some results presented  for this in Appendix E, however these should be brought into the main part of the paper as it is important to show the robustness of the method. \n\nIn general the paper is well written and formatted but  In line 436 and 437, the sentence reads ``Therefore our method should be primarily compared toe these baselines.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4kFhlcCqJ5", "reviewer": "Reviewer_TPuR", "summary": "The paper propose a method that seems to be a PINO method to tackle learning of PDEs with the presents supervision from both data and equations.", "questions": "mentioned above.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "The authors tackle and important problem in AI and science.", "weaknesses": "The writing and position of the paper is very limiting. \n\nA few basic comments before we get to the main ones.\n-1 In the intro:\nPlease note that when a PDE equation is given, that is complete supervision. Please avoid calling it unsupervised. PDE equation is absolute supervision, sufficient to solve the problem completely. \n-2 In the intro:\nPlease pay more attention; there is a difference between neural networks and neural operators. One is for learning on the finite-dimensional setting; one is for functional data, that is your case. I only found out on page 8 that the authors talk about Fourier layers, and then I had to go through the referenced paper to realize that the authors use a neural operator for their solver. Even there, please explain you are doing operator learning and using neural operators. Please also define your neural operator architecture. FNO is not even defined or abbreviated in your work. Please bring the abbreviation and explain in your paper that your work is neural operator-based. \n\n-3 In 3.3 \nThe authors need to define varrho in the equations in this section. I guessed what that was after reading page 8. There is no varrho in the equations in 3.3.\n\n-4 In 3.3\nWhy choose linear span? We know such a thing is fine for easy problems and not sufficient for complex problems. That's why even the first paper proposing the PINO paradigm also deals with complex problems and doesn't use linear span simplification. \n\n-5 In 3.3\n What does Solve even mean in alg2? I only guessed its meaning.\n\n-6 In 3.3. Please be mindful of term solve and term prediction for F. The F predicts something and doesn't solve anything in the sense that solvers in the field of applied math solve. \n\n-7 In 3.3 \nReading line 304, it feels that the authors first do PINO with only physics and then PINO with data. However, the intro implied that it is the other way. Which is done here?\n\n-8 In 3.3 what is trajectory in line 308?\n\n\nThe main comment is the presentation. Reading the paper, it seems the paper is aiming to advance PINO. However, reading abstract, intro, and other sections up to page 8, the paper does not even talk about PINO. Everything the paper promotes and sugest they are doing is already propsed and establisehd in PINO paper. Please read the PINO paper carefully and possition your paper according to that. Also, in both abstract and intro, mantion what is new in your work. I could not find anuthing different than prior methods. And I am sure there are many things new and novel in this paper, but I just couldn't parse it when abstract and intro are all talking about basics of PINO.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)