PIED: Physics-Informed Experimental Design for Inverse Problems
Authors: Apivich Hemachandra
Gregory Kang Ruey Lau
See-Kiong Ng
Bryan Kian Hsiang Low
Venue: ICLR 2025 Poster (2025)
Keywords: Physics-Informed Neural Network
PINNs
Experimental Design
AI For Science
Active Learning
Data Selection
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: An experimental design framework for PDE-based inverse problems that uses PINNs and its training dynamics, in a fully differentiable architecture to perform continuous optimization of design parameters.

Abstract: In many science and engineering settings, system dynamics are characterized by governing partial differential equations (PDEs), and a major challenge is to solve inverse problems (IPs) where unknown PDE parameters are inferred based on observational data gathered under limited budget. 
Due to the high costs of setting up and running experiments, experimental design (ED) is often done with the help of PDE simulations to optimize for the most informative design parameters (e.g., sensor placements) to solve such IPs, prior to actual data collection. This process of optimizing design parameters is especially critical when the budget and other practical constraints make it infeasible to adjust the design parameters between trials during the experiments.
However, existing experimental design (ED) methods tend to require sequential and frequent design parameter adjustments between trials. Furthermore, they also have significant computational bottlenecks due to the need for complex numerical simulations for PDEs, and do not exploit the advantages provided by physics informed neural networks (PINNs) in solving IPs for PDE-governed systems, such as its meshless solutions, differentiability, and amortized training. 
This work presents Physics-Informed Experimental Design (PIED), the first ED framework that makes use of PINNs in a fully differentiable architecture to perform continuous optimization of design parameters for IPs for one-shot deployments. 
PIED overcomes existing methods' computational bottlenecks through parallelized computation and meta-learning of PINN parameter initialization, and proposes novel methods to effectively take into account PINN training dynamics in optimizing the ED parameters. 
Through experiments based on noisy simulated data and even real world experimental data, we empirically show that given limited observation budget, PIED significantly outperforms existing ED methods in solving IPs, including for challenging settings where the inverse parameters are unknown functions rather than just finite-dimensional.

Published as a conference paper at ICLR 2025
PIED: PHYSICS-INFORMED EXPERIMENTAL DESIGN FOR INVERSE PROBLEMS
Apivich Hemachandra" Gregory Kang Ruey Lau"it,
'Department of Computer Science, National University of Singapore, Singapore 117417 *CNRS@CREATE, Create Way, #08-01 Create Tower, Singapore 138602
(apivich, gregl au B @comp.
edu.sg sg
See-Kiong Ng
Bryan Kian Hsiang Low
Department of Computer Science, National University of Singapore, Singapore 117417 seek feekiongeuus. nus. edu..edduu......... sg, lowkh&comp nus edu.sg
ABSTRACT
In many science and engineering settings, system dynamics are characterized by governing partial differential equations (PDEs), and major challenge is to solve inverse problems (IPs) where unknown PDE parameters are inferred based on observational data gathered under limited budget. Due to the high costs of setting up and running experiments, experimental design (ED) is often done with the help of PDE simulations to optimize for the most informative design parameters (e.g., sensor placements) to solve such IPs, prior to actual data collection. This process of optimizing design parameters is especially critical when the budget and other practical constraints make it infeasible to adjust the design parameters between trials during the experiments. However, existing ED methods tend require sequential and frequent design parameter adjustments between trials. Fur- thermore, they also have significant computational bottlenecks due the need for complex numerical simulations for PDEs, and do not exploit the advantages provided by physies informed neural networks (PINNs) solving IPs for PDE- governed systems, such as its meshless solutions, differentiability. and amortized training. This work presents Physics-Informe Informed Experimental Design (PIED), the first ED framework that makes use of PINNs in fully differentiable architecture to perform continuous optimization design parameters for IPs for one-shot de- ployments. PIED overcomes existing methods' computational bottlenecks through parallelized computation and meta-lcarning of PINN parameter initialization, and proposes novel methods effectively take into account PINN training dynamics in optimizing the ED parameters. Through experiments based on noisy simulated data and even real world experimental data, we empirically show that given limited observation budget, PIED significantly outperforms existing ED methods in solving IPs, including for challenging settings where the PDE parameters are unknown functions rather than just finite-dimen dineesidiansi......
INTRODUCTION
The dynamics of many systems studied in science and engineering can be described via partial differential equations (PDEs). Given the PDE governing the system and the properties of the system (which we refer to PDE parameters), we can perform forward simulations system behaves. However, in practice, the true PDE parameters are often unknown, and we are instead interested in recovering the unknown PDE parameters based on observations the system's behaviors. This problem of recovering the unknown PDE parameters is type of inverse problem (IP) (Vogel 2002 Ghattas & Willcox 2021), and have been studied in classical mechanics (Tanaka Buil 1993 Gazzola et al. 2018), quantum mechanics (Chadan al. 1989) geophysics (Smith et al.
predict how the
2021
Waheed et al. 2021), and more. It is challenging to directly solve IPs given observational data
Equual contribution.

---

Published as a conference paper at ICLR 2025
since PDE parameters often affect the behavior of the PDE solution in complex ways. This is further complicated in practice where data acquisition (c.g., making measurements from experiments or field trials) is often costly and only limited number of observations can be made, making the choice of which observations to make given a limited budger also critical to solving IPs.
Experimental design (ED) methods aim to tackle the data scareity problem by optimizing design parameters, such as sensor placement locations, to yield the most informative measurements for estimating the unknown inverse parameters (Razavy 2020 Alexanderian 2021). These methods typically make use of forward simulations based on guesses of the true inverse parameter to find the best design parameter for the measurements. However, these methods are less practical in many PDE-informed IPs due to significant computational bottlenecks in forward simulations and solving IPs, especially for systems with complex forward models and PDEs. In particular, simulators using conventional PDE solvers often are costly to run, and return discretized (mesh-basede approximate solutions which are often incompatible with efficient continuous gradient- based optimization methods.
Physics-Informed Neural Networks (PINNs) are neural networks that incorporate PDEs and their initial/boundary conditions (IC/BCs) into the NN loss function (Raissi et al. 2019). and have been successfully applied to various science problems (Chen et al.) 2020 Cai et al.) 2021; Jagtap et al., 2022). PINNS are especially well-suited to tackle ED for IPs, as they (1) allow easy incorporation of observational data into the inverse problem solver through the training loss function, (2) can be used for both running forward simulations and directly solving IPs with the same model architecture, (3) are continuous and differentiable w.r.t. the function inputs, and (4) have training whose costs can be amortized, e.g., through transfer learning. However, to our knowledge, there has not been an ED framework for IPs that fully utilizes these advantages from PINNS for ED problems.
In this paper, we present Physics-Informeee Experimental Design (PIED), the first ED framework that makes use of PINNS in a fully differentiable architecture to perform continuous optimization of design parameters for IPs for one-shot deploymen"' Our contributions are summarized as follows:
We propose novel ED framework that makes of PINNs both forward simulators and inverse solvers in a fully differentiable architecture to perform continuous optimization of design parameters for IPs for one-shot deployments (Sec. 3
We introduce the use of a learned initial NN parameter which are used for all PINNs in PIED (Sec. [.]). based on first-order meta-learning methods, allowing for more efficient PINN training over multiple PDE parameters.
We present various effective ED criteria based on novel techniques for quantifying the training dynamics of PINNS (Sec. 4.2) These proposed criteria are differentiable w.r.t. the design parameters, and therefore can be optimized efficiently via gradient- based methods.
We empirically demonstrate that PIED is able outperform other ED methods in- verse problems (Sec. 5). both in the case where the PDE parameters of interest are finite- dimensional and when they are unknown functions.
2 BACKGROUND
In this section, we provide formalism of the experimental design (ED) problem for PDE-based inverse problems (IPs), and physics-informer neural networks (PINNs) which will be used in our proposed framework. We also discuss existing works on ED and PINNs applied to solving IPs.
2.1 PROBLEM SETUP
Inverse problems. Consider system described by PDE of the form
(x) f(x) Vx E X and (xx') g(x') Va ox
(1)
where u : X' + Rdd describes the observable function (solution of the PDE) over a coordinate variable X' C pdd. (where time could be subcomponent), and / E Rd are PDE parameters
'The code for the project can be found at / 2 Examples of PDFs, specifically those in our experiments, can be found App. D For simplicity we assume 33 is finite-dimensional. In our experiments, we demonstrate how method can also be extended cases where / function of J..
com hoom/apivii ctppvi/iiieeeed
h/pied
2

---

Published as a conference paper at ICLR 2025
D is PDE operator and B is operator for the initial/boundary conditions (IC/BCs) at boundary 0X C X. Different PDE parameters 3 results in different observable function u that satisfies ①.. which for convenience will be denoted by u.g.
For inverse problems (IPs), the operators D and and functions f and g are known, and the task is to estimate the unknown PDE parameter of interest, So. that cannot be observed directly. Instead, we can only make noisy measurements of the corresponding observable function Up at M observation inputs X = { C X to get observation values Y = {us, (z,) + E = where we assume Gaussian noise Ej ~ 2²). In general, X could possibly be constrained to a set of feasible configurations S S x To solve the IP, we could use an inverse solver that finds the PDE parameter 3 that fits best with the observed data () where
8(X, Y) = arg min [lu((X) -
Experimental design. Unfortunately, the observations Y are typically expensive to obtain due to costly sensors operations. In many settings, the observation inputs also have to be chosen in a one-shot rather than in a sequential, adaptive manner due to the costs of reinstalling sensors and inability to readjust the design parameters on-the-fly. Hence, the observation input X should be carefully chosen before actual measurements are made. As we do not know the true PDE parameter 80. a good observation input should maximize the average performance of the inverse solver over its distribution p()). In the experimental design (ED) problem, the goal is therefore to find the observation input X which minimizes
L(X) ( (8) [| Y) -11"
or the expected error of the estimated PDE parameter w.r.t. the possible true PDE parameters.
ED methods can be deployed in the adaptive setting, where multiple rounds of observations are allowed, and the design parameter be adjusted between each rounds based on the observed data. However, there are many practical scenarios where non-adaptive ED are desirable. For instance, field scientists who incur significant cost in planning, deployment, and collection of results for each iteration may strongly prefer to perform a single round of sensor placement and measurements as opposed sequentially making few measurements per round and frequently adjusting sensor placement locations, Another example is when the phenomena being observed occurs in a single time period, and all observations have to be decided beforehand. In these cases, is more practical to optimize for single design parameter upfront and collect all observational data for the IP in one-shot. Unlike past works, our work aims to optimize performance for one-shot deployment.
Physics-informen neural networks. To reduce the computational costs during the ED and IP solving processes, we will use physics-informed neural networks (PINNs) (Raissi et al., 2019) to simulate PDE solutions and solve IPs. PINNs are neural networks (NNs) ûg with NN paramaters 0 that approximates the solution u to (I) by minimizing the composite loss
[X) 211
Diuo (Xp) s(Xp) 2 +
B No (Xbb) g(Xx))l
C(0, ((0,3;X,Y)
X|X
2 2
2 2|X,
(4)
where (X, Y) are observational data, and X, C X and Xs C 0x are collocation points to enforce PDE and IC/BC constraints respectively. PINNs can be used both as forward simulators when / is known but observational data is available (Cobs (0: X, Y) 0), as inverse solvers when observational data is available but 33 is unknown and is learned jointly with 0 during PINN training.
2.2 RELATED WORKS
Existing ED methods in the literature include those that adopts Bayesian framework (Chaloner &
Verdinelli 1995
Long et al. 2013 Belghazi et al.) 2018 Foster et al. 2019) and those that aims to
'ee ahuse notations by allowing the set of X to an argument for functions which takes in x E X as well. Foor simplicity consider one IC/BC, however the loss can be generalized to include multiple IC/BCs by adding similar loss terms for each constraint.
3

---

Published as a conference paper at ICLR 2025
construct policy for choosing the optimal experimental design (vanova et al. 2021 Lim al. 2022).
Many of these ED methods utilize forward simulations that can be easily and efficiently queried from, which is often unavailable in PDE-based IPs due to the need to numerically solve complex PDEs. ED and IP solving methods that rely on numerical simulators (Ghattas & Willcox 2021; Alexanderian 202T: Alexanderian al. (024) are computationally expensive due to repeated forward simulations required during optimization, and are usually restricted to some PDEs. Furthermore, numerical solvers often require input discretization and cannot be easily differentiated, which restricts the applicable optimization techniques.
Meanwhile, PINNS have been extensively used model systems and solve IPs across many scientific domains (Chen et al. 2020 Cai et al. 202T Waheed et al.) [02T]] Bandai & Ghezzehei 2022] Jagtap et al., 2022; Shadab et al. 2023), due to the simplicity in incorporating observational data with existing PDE from the respective domains, and its ability to recover PDE parameters with one round of PINN training. However, these works assume that the observational data to be used for IPs have already been provided, and do not consider the process of observational data selection and how it may affect the IP performance. Further material on related works are in App. C
EXPERIMENTAL DESIGN LOOP
In this section, first discuss how IPs can be solved with PINNs given set of observational data (X, Y), and how the choice of observation inputs X matters especially when there is limited observation budget. Thereafter, we present our proposed ED framework, PIED, which optimizes for the observation input while fully utilizing the advantages provided by PINNs.
3.1 SOLVING INVERSE PROBLEMS WITH PINNS
The process of solving the IP with PINNs is summarized in Fig, la As presented in Sec. 2 the goal is to find the true PDE parameter 3 of the system based on (2 by conducting experiments at limited number of observation input X to obtain observational data (X, Y), where Y are noisy measurements of the observable function ug- This can be directly solved using PINN-based inverse solver that is trained to minimize the composite loss in (4) to find the inferred 3 the system.
However, given limited observation budget, the choice of observation input X becomes important in achieving good estimate 3- some points are more informative than others, and allows the inverse solver to achieve better estimates with lower variance. Intuitively, we could interpret the limited observation as information bottleneck, where the information on a system's 's 3 would be compressed from the entire observable function into a relatively low-dimensional, noisy representation. The better the choice of X, the better could extract the value of 3 via the inverse solver.
3.2 COMPONENTS OF THE ED DESIGN LOOP
To optimize for X, ED methods rely several forward simulations of the system for different reference 3 values before collecting any observational data from actual, costly experiments. Multiple possible X would then need to be tested on all these sets of simulations, assess the inverse solver's performance. However, existing methods that rely on conventional numerical integration methods do not enable efficient parallel computation and differentiability for gradient-based optimization. Instead, we propose to fully utilize the advantages provided by PINNs to tackle the ED problem.
To achieve this, we propose our ED framework PIED, which is visualized in Fig. lb parallel threads, each representing different reference 3, values. Similar to existing ED methods, the range of values could be informed by domain knowledge. Within each thread i, we have three components: (1) forward simulator that returns an observable function uss which approximates the PDE solution U,3,, (2) a observation selector that generates observational data based on uz (X) at observation inputs X (consistent across all threads), and (3) an inverse solver that takes in the observational data to produce estimates of the PDE parameter 3.. Finally, require (4) criterion that captures how good the estimates are across all threads on aggregate, and an input optimization method to select the single best observation input according the criterion.
We consider N
4

---

Published as a conference paper at ICLR 2025
Figure I: Comparison between observation selection and solving IPs in real life (Fig.
versus the
proess as modelled in the PIED framework (Fig. 1b
(a) Inverse problem in real life
Roal-iffe system
Octaining obsorvational data
Inferance inverse salver
TIil Experieent
(x,Y)
Inverse Salver
Real System PDE
Estimated PDE
Observational Data
Observation
Obeervation imput X
(b) PIED framework
Forward Simulator (Eqn. 5)
Observation Selector (Eqn.
Inverse Solver (Egn. B)
Thread
8g
/ G
Ox
(X, 2.)
A
Thread N
F
kv)
8 Learned
Reference
Optimal X*
Optimal point selection based on criterion xx.
PINN-based forward simulator (F). For the first component, PIED uses PINNs simulate what the observable function Ma; given reference B, would be in each thread. Specifically, forward PINN is trained with loss LPDE f, (,) from with fixed 3; to generate Ug over the input space X,
forward simulator
Ug úo Up.
Note that unlike classical simulators based on numerical integrators that require fixed discretization of the input space X. the trained PINNs ú.3 represent learned functions that are meshless, and therefore be queried at and even differentiated w.r.t. any input x X. This is major advantage for ED, as it allows us to flexibly test a continuous range X without having to re-run the forward simulators many times for different discretization, and also enables efficient gradient-based optimization.
Our use of PINNs as forward simulators also offer significant computational advantages. First, modern software and hardware allows for very efficient training of multiple NNs in parallel (for example, using increasingly accessible and powerful GPUs and via the vmap function on JAX), unlike classical simulators. Second, the various PINNs different thread can all be initialized from common pre-trained base model (which we discuss in Sec. 4.1. speeding convergence of the PINN during training. This will be even more advantageous if researchers start sharing open- sourced pre-trained models, similar to what is currently being done for large language models. Third, we can reuse the trained forward PINNs to approximate the dynamics of the inverse PINNs and consequently boost the effectiveness of the downstream ED process, as we will describe in Sec. 4
Observation selector (0x). Given the trained forward PINNS {ūp, J the second component
applies a "sieve" that queries all forward PINNs with the same M observation input = (-, M to produce the respective sets of noisy predicted observations, i.e., for each i 1,...,N:
observation selector Ox (X, Y.) = (1 1= ( (2z) + +
(6)
NNoe that actual experimental data required for the training of forward PINNs, explained in Sec.
5

---

Published as a conference paper at ICLR 2025
where Ej ~ (0, o²) are i..d. noise added to model the observational data generation process. The observation input X applied here is the candidate design paramete problem different choices of X will yield different sets of observational data (X, Y;) that impacts the quality of the PDE parameter estimate, as per 3
E to be optimized for in the ED
PINN-based inverse solver (I). For each 8r, the predicted observational data (x, X.) from 6 will
then be used to train an inverse solver PINN (inverse PINN) with loss function L(0, 30, X, YY) from (4 to return an estimated PDE parameter B;, i.e., for each i = 1,...,N.
(x,Y)
solver I
Note that in our ED framework, both the forward simulators and inverse solvers are PINNs and have the same architecture. This enables to develop effective approximation techniques based on predicting the dynamics of the inverse PINNS using its corresponding forward PINN, significantly reducing computational cost for the ED process, and also the eventual IP process in Sec. 3.2 that also uses the inverse PINN. We elaborate further on these techniques in Sec.
Criterion and optimal observation input selection. Finally, we could compute the ED criterion
in 3) for given X, which evaluates how well the estimates 3r from the inverse solvers matches the corresponding reference parameter values 3, across all parallel threads i on average,
N Σ a: (x)
N 18
1
a(x)
(8)
We can then perform optimization using the observation selector and inverse solver components, where we search for the observation input X that maximizes aXX). Note that the same forward PINNS {ūg. Ji=1 could be re-used for all iterations and hence only need to be computed once. As our PINN inverse solver is differentiable, we can back-propagate through directly to compute Vxa(X) (). This allows a to be optimized using gradient-base optimization methods, instead of methods such as Bayesian optimization which do not typically perform well on high-dimensional problems, or require combinatorial optimization over discretized observation inputs.
4 EFFICIENT IMPLEMENTATION OF PIED
In this section, we introduce training and approximation methods which further boosts the efficiency of PIED. We first propose meta-learning approach to learn PINN initialization for efficient fine-tuning of all our forward and inverse PINNs across different threads (Sec. 4.1). before proposing approximations of our criterion in (8) to effectively optimize for the observation inputs (Sec. 4.2
4.1 SHARRD META-LEARNED INITIALIZATION FOR ALL PINN-BASED COMPONENTS
In PIED, we could significantly reduce computational time and benefit from amortized training by pre-training and using shared initialization for all PINN components in our framework. We can efficiently achieve this with REPTILE (Nichol et al. 2018 Liu et al. (021). first-order meta- learning algorithm. To see this, note that all PINNS (forward and inverse) in PIED are modelling the same system and may benefit from joint training, but have different PDE parameters 3r leading to different observable functions and hence could be interpreted as different tasks to be meta-learned.
Specifically, when implementing PIED, we first use REPTILE to learn a shared PINN initialization 8si for the set of reference B3, if we do not already have a pre-trained model for the system interes Given this, can then (1) efficiently fine-tune Asi across different reference 3 with fewer training steps to produce the forward PINNs [ū,s, N (2) re-use Ost for the approximation of inverse PINNS performance optimize our ED criterion (elaborated in Sec. 4.2 and (3) re-use OSi for the final
inverse PINN applied actual experimental data to find the true 5" (Sec. 3.1 Once learnt, Osi can
Thhe ohservation input can either he freely chosen or constrained to certain configurations (see App. F 'Note that no actual experimental data is needed for this pre-training phase, just like in the training of forward PINNs. Rather, training is done hased LyDe (A. in (4), enforced at collocation points.

---

Published as a conference paper at ICLR 2025
(a) Random Init.
(b) Shared Init.
(c) Train Loss
(d) Test MSE Error
10-2
0.5
10-a
10
20 10"
10"
107
10a
Training Steps
Training Steess
Figure 2: Results for meta-learning a shared NN initialization for PINNs trained on ID damped oscillator case. The shared initialization (blue line) in Fig. 2b exhibits similar structure to PDE solutions us for different values of /3 (faint green lines), unlike the random initialization (blue line) in Fig. 2a In Figs. 2c and 2d] we show that this translates better average train and test loss performance of PINNs with shared initialization compared to random initialization w.r.t different 3.
(a) Random Init.
(b) Shared Init.
(c) Sample Eikonal equation solution
1.05
4-
0.75
6
5
5
-0.60
0.30
0.0
0.00
Figure 3: Results for learning a NN initialization for PINNs trained on Eikonal equation case. Fig. 3a represents randomly initialized PINN, while Fig. 3b represents the shared initialzation for the PINN. Fig. 3d shows sample PDE solutions Ug for different random PDE parameters B.
also be re-used many times on different IP instances with the same PDE setting, further reducing computational costs in practical applications.
To demonstrate the benefits learning a shared NN initialization for PINNs, we visualize various PDE solutions (green lines) for the ID damped oscillator setting, along with the output of the PINN with random initialization and shared initialization (blue lines), in Fig. 2a and Fig. respectivel Note how the PINN with the meta-learned, shared initialization share qualitatively similar structure to the PDE solutions Ug for different values B, c.g., replicating the damping effect observed in the various solutions. In contrast, the random initialization in Fig. 2a dissimilar to any of the PDE solutions. This advantage can also be observed quantitatively in Figs. 2c and 2d where we plot the train and test loss for forward PINNs when initialized with 0sı versus when initialized randomly. We can see that PINNs initialized with the shared 6si initialization have faster training convergence with lower train and test loss compared those that are random initialized.
We also demonstrate this advantage for the 2D Eikonal equation setting, where the PDE parameters functions of the input space X. In Fig. 3 visualize the shared parameters learned for the Eikonal equation, which again shows that the qualitative structures and even the scaling of the PDE solutions can be meta-learned beforehand in order speed up the PINN training process. We provide further empirical results in App. [.] which demonstrates that this also applies for inverse PINNs and forward PINNs trained for other PDEs.
4.2 APPROXIMATE CRITERIA FOR PERFORMANCE OF THE INVERSE SOLVER
We now present two methods effectively optimize for the observation inputs by approximating the inverse solver performance and criterion in 8
Few-step Inverse Solver Training (FIST) Criterion. Our first method stems from the insight that we do not need to fully solve for 3, in PIED's 's inverse solvers to find the observation input X that maximizes (8). Drawing inspiration from Lau al. (2024a) with results showing that informative training points that lead to faster training convergence also result in lower PINN generalization error bounds and better empirical performance, propose only partially train the inverse PINN for
7

---

Published as a conference paper at ICLR 2025
few training steps to get intermediate estimate of 8i, and use this perform gradient- based optimization for X with (8) by back paok-propggating through the PINN inverse solver. To get more informative gradient signals, this is done with inverse PINNs that are closer convergence rather than at early stages of training when most choices of X would result in good performance gains.
Hence, our method FIST first initializes the inverse PINNS for each thread with perturbed NN parameters from the corresponding converged forward PINN for 3; before partially training them for training steps to obtain the estimate 3/ computing the criterion in (8). and performing gradient descent optimization for X. We present the pseudocode for FIST in Alg. of the Appendix.
Model Training Estimate (MoTE) Criterion. Our second method involves directly approximating
the inverse solver output 3, at convergence while minimizing training. We do so by performing kernel regression with the empirical Neural Tangent Kernel (eNTK) of the PINN (Jacot et al.) 2018 Wang
et al. 2022; Lau et al., 2024a), given a set of observation input X. Specifically, under assumptions that the PINN is in the linearized regime and trained via gradient descent with ( the predicted PDE
parameter B at convergence can be estimated by (IC/BC terms omitted for notational simplicity)
J Jos.
Jo p.u
[ agcon (X)- -Y Di : (Xp) - s(xp)
B(X,Y) g(0)
JT.
J J. p.a J...Jp. p.Bl
where (8(0), )(0)) are the initialization parameters, Jabs, = Voúgao (X), Jp.0 VoD (gro0 (Xpp).
and Jp VgDie ugro: : (X,) (note that V guggc (X) = 0). This estimate for 3 in 9 is adapted from Lee et al. (218)), and is verified in App. [F2. which includes further details on the assumptions and derivation. Note that while the use of NTK in PINNs is not new (Wang et al., 2022 Lau et al.
2024a), previous works have not utilized NTK to directly quantify the performance of PINNs in
solving inverse problems as we have done.
In practice, as noted by Lau et al. (2024a). finite-witth PINNs have eNTKs that evolve over training before they can better reconstruct the true PDE solution. Hence, for the MoTE criterion, we either first do r steps of training on the Osi initialized inverse PINN before computing the eNTK and performing kernel regression, or use perturbed version of the forward PINN parameter to perform kernel regression. We present the pseudocode for MoTE in Alg. in the Appendix.
In App. we provide further discussion about the proposed criteria for approximating the in- verse solver performances. In App. G we summarize the full implementation process for PIED, incorporating both approximation methods and the full ED loop from Sec.
RESULTS
In this section, we present empirical results to demonstrate the performance of PIED on ED problems for a range of scenarios based on different PDE systems, both from noisy simulations from real physical experiments (details of the specific PDEs are in App. For scenarios using simulation data, we injected noise and lowered data fidelity for the evaluation dataset to represent noisy limited sensor capabilities. We compare PIED against various benchmarks, such as approximations of the expected mutual information (Beighazi al. 2018 Foster al., 2019) and criterion from optimal sensor placement literature (Krause et al. 2008), in addition to random and grid-hased methods, with details in App. H.44| In each scenario, the ED methods compute their optimal observation input, which are then evaluated on multiple IP instances (i.e., different ground truth PDE parameters (3). We then report the mean error across the different PDE parameters in accordance to the loss term in (3) Additional details on the experimental setup are listed in App. H We present subset of experimental results in the main paper, and defer the remaining results to App. [[] The code for the project can be found at httpt::///// /////ttttttt wwigtttthh..... cww www./oo...pp..ico. vi ch-h/p
Finete-dimensional PDE parameters. We first present two ED problems on IPs where the PDE parameters corresponds to multiple scalar terms representing certain physical properties of the system. The first scenario is the ID time-dependent wave equation with inhomogeneous we incorporated realistic elements such as restrictions in sensor placements. The second is the 2D Navier-Stokes equation, which is challenging setting relevant many important scientific and
speeds, where
8

---

Published as a conference paper at ICLR 2025
Fintee-dimensioal
Punction-valuee
Real dataset
Dataset
Wave (x10
Navier-Stokes (x10
Eikonal 10')
Groundwater (x10')
Cell Growth (x")
Kandom
(1.26)
(3.53)
182 (0.09)
(1.77)
(0.26)
Grid
8.90 (0.73)
4.51 (0.60)
1.56 (0.22)
2.27 (2.26)
3.19 (0.23)
MI
4.46 (0.92)
6.08 (2.10)
2,02 (0.01)
2.10 (0.32)
3.14 (0.59)
VBOED
4.63 (1.64)
4.33 (0,98)
1.82 (0.50)
2.29 (1.09)
2.82 (1.77)
FIST (ours)
3.87 (0.76)
2.10 (1.45)
0.74 (0.02)
1,93 (0.08)
2.62 (0.11)
MoTE (ours)
3.81 (2.34)
1 (0.11)
0.76 (0.02)
2.00 (0.60)
2.83 (0.04)
Table I: Results of the ED methods for the various experimental scenarios. Each result reports the median of the expected loss (i.e., in (3)) across trials, and the figure in bracket represents the semi-in interuutequu range. The results of the best performing ED methods in each dataset are in bold.
industrial problems. The results presented in Table [] Compared benchmarks, both FIST and MoTE perform better the scenarios, producing optimal choices that have lower expected loss when evaluated across datasets of multiple B values. The performance gap between and benchmarks more significant for the 2D Navier-Stokes scenario, which is more challenging, making the optimization of X more important in obtaining good estimates of the inverse problem.
methods
PDE parameters that are functions of input space. We further demonstrate that PIED can be applied complex scenarios where the PDE parameter of interest input space X. To estimate 3 in these cases, we parameterize it using small NN, and learn it together with the PINNs in our framework. The scenario we considered is the 2D Eikonal equation, which is used in wide range of applications such as seismology. robotics image processing. In this problem, the sensors can be placed freely in X, yielding high-dimensional design parameter. The results in Table demonstrate that in this complex scenario, our PIED methods still able to outperform the benchmarks and recover the correct iinctionvvaluee PDE parameter. In addition, methods also produce consistently good results, can be seen from the small comi-interquartile range (SIQR) of our expected loss.
function defined over the
We further amalyze the performance of PIED by visualizing the observation inputs X chosen by FIST and the loss of the PDE parameter (Fig. 4 Note that FIST automat- ically adjusts the choice of X based on un- derlying structure of the problem, allowing it to choose X more evenly minimize the error of 3. This provides it with advantages over more beuristics-based approaches like the space-fillingg Grid method.
True Solution
Random
Grid
FIST
2-
0.3
Inverse problem on real-life dataset. Simulation datasets may not be able to fully represent the challenges of realistic scenar- ios, due to the complexity of the real-life noisy experimental data. Hence, to fur- ther validate the advantages of PIED and analyze its performance on more realistic scenarios, applied observation lection problems on real data that are col- lected from physical experiments in differ- ent domains of natural science. Specifi- cally, we consider the ED problem applied to the groundwater flow dataset collated by Shadab et al. (2023), and the scratch assay cell population growth data collected by Jin et al. (2016). where the design parameters indicate the location to make the observations at constrained. fixed time intervals. As can be seen in Table our PIED methods outperform benchmarks in both scenarios, despite the challenges posed by the datasets. For example, we can see in Fig. 5a that the groundwater flow dataset no longer fits typical i.i.d. noise assumptions that existing ED method work under, but FIST still performs well (selected observations
VIX v
0.12
0.08
0.06
0.02
5.00.0
Figure 4: Example of ED process involving the Bikonal equation using different methods of observation selec- tion. Top row: the true observation T'z, y) and its approximation via PINNs. Middle row: the true un- known function v(z, y) and the recovered estimations based on observation inputs. Bottom row: the error of the reconstructed v(z, y).

---

Published as a conference paper at ICLR 2025
(a) Groundwater flow
(b) Cell population growth
10.3
lol
1.85
h00
1.0
0.90
0.75
0.60
0.45
3
0.30
0.15
0.00
0.5
Figure 5: Visualization of real-life experimental data used in our tests, along with demonstration of observations selected by FIST. Fig. example of groundwater flow data from Shadab et al. (2023). Gray points represent the collected data, while the blue points are the observations chosen by FIST. The black line represents the prediction from the corresponding inverse PINN. Fig. [b)] example of cell population growth data from Jin et al. (2016). The left figure shows the cell population data which are collected at 12 hour intervals, while the right figure shows the population prediction from the inverse PINN. In both figures, the blue points represent the observations chosen by FIST.
blue, and prediction plotted as a dotted line). In Fig. [b] we can see that the observations chosen by FIST spatially close to each other in order better interpolate the effects from spatially-related PDE parameters, resulting in its better performance compared to benchmarks.
Advantages of PINNs over numerical solvers in ED. In the results above, we implemented the benchmark ED meth- ods applied to PINNs as forward simula- tors. This allowed the methods to ben- efit from the advantages of PINNs. In fact, existing benchmarks using classical numerical solvers would produce signifi- cantly worse results than those reflected in Table 1 To see this, consider the 2D Eikonal scenario, and use the Eikonal equation solver implemented by White et al.] (2020) which uses the fast march- ing method (Sethian 1996) the numer- ical simulator. Due to the lack of gradi- information, optimization is done using Bayesian optimization. Fig. 6a shows that using PINNs allow the correct function be recovered much more accurately regard- less of the ED method used. Furthermore, we see in Fig. 6b that using PINNs allow the overall ED and IP to done more ef-
(a) TP Results
(b) ED and IP timing
Exoected line Error
ED
xx10
Inference (50x)
Bunning time
10000
2x1
75000
101
2500
MOTE loll
Random-+PPM
VBGED FPRM
gin
SespAA Ni Sim
FIST
Bandom PIN
VBDED PPN
V0c1D-3i0 Sim
Ni-sim
FIST
MoTE
Rancom-
nancom
Figure 6: Results for various benchmarks using numeri- cal simulators and PINNs. Fig. 6a| error of recovered PDE parameters. The thick blue line represents the per- formance of the Random method, while the dashed line represents the best performance. Fig. 6b) time required run the ED algorithm and to run 50 rounds of infer- ence to solve the inverse problem.
ficiently as well.
6 CONCLUSION
We have introduced PIED, the first ED framework that utilizes PINNs as both forward simulators and inverse solvers in a fully differentiable architecture to perform continuous optimization of design parameters for IPs. PIED selects optimal design parameters for one-shot deployment, and allows exploitation of parallel computation unlike existing methods. We have also designed effective criteria for the framework which are end-to-end differentiable and hence can be optimized through gradient- based methods. Future work could include applying PIED to other differentiable physics-infool informed architectures, such operator learning methods.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This research/project is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISGG--PDD/2223-011 DesCartes which is supported by the National Research Foundation, Prime Minister's 's Office, Singa- pore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme. The computational work for this article was partially performed on resources of the National Super- computing Centre, Singapore (https//wwww.scccsg))
and part of the programme
REFERENCES
Alen Alexanderian. Optimal experimental design for infinite-dimensional Bayesian inverse problems governed by PDEs: a review. Inverse Problems, 37(4)004300, March 2021. ISSN 0266-5611. Publisher: IOP Publishing.
Alen Alexanderian, Ruanui Nicholson, and Noemi Petra. Optimal design of large-scale nonlinear bayesian inverse problems under model uncertainty. Inverse Problems, 40(9):095001, 2024.
Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, An- drew Gordon Wilson, and Eytan Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. Proc. NeurIPS, 2020.
Toshiyuki Bandai and Teamrat A. Ghezzehei. Forward and inverse modeling of water flow unsatu- rated soils with discontinuous hydraulic conductivities using physics-informed neural networks with domain decomposition. Hydrology and Earth System Sciences, 26 36(16):449944495 August 2022. ISSN 1027-5606. Publisher: Copernicus GmbH.
Joakim Beck, Ben Mansour Dia, Luis FR Espath, Quan Long, and Raul Tempone. Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain. Computer Methods in Applied Mechanics and Engineering, 334:523-555 -53,,,,,,,,,, June 2018. ISSN 00457825.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual Information Neural Estimation. Proc. ICML, pp. 531-540, July 2018
Lorenz T. Biegler, Omar Ghattas, Matthias Heinkenschloss, and Bart Bloemen Waanders. Large-Scale PDE-Constraiie Optimization: An Introduction. In Large-Scale DDE-Constrained Optimization, pp. 3-13, Berlin, Heidelberg, 2003. Springer. ISBN -78-3-442-55508-4
Sacha Binder. Wave equation simulations dd/2d (équation de d' d'alemmbet). https://githu: com/ honn//sachabbddee//veeeeeo http://avvveeeaatiinnnnaaaa simuuattiiiiii......... ions] 2021.
Mathieu Blondel, Quentin Berthet, Marco Cuturi, Roy Frostig, Stephan Hoyer, Felipe Llinares-López.
Fabian Pedregosa, and Jean-Philippe Vert. Efficient and modular implicit differentiation. arXiv,
2021.
Joseph Boussinesq. Recherches théoriques l'écoulement des nappes d' d' infiltrées dans le sol sur le débit des sources. Journal de Mathématiques Pures el Appliquées, 10:5-78, 1904.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: Composable transformations of Python+NumPy programs, 2018.
Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis, Physics- Informed Neural Networks for Heat Transfer Problems. Journal of Heat Transfer. 143(6):06080,, June 2021, ISSN 0022-1481, 1528-8943.
K. Chadan, P. C. Sabatier, and R. G. Newton. Inverse Problems in Quantum Scattering Theory. Springer Berlin Heidelberg, Berlin, Heidelberg, 1989. ISBN 978-3-642 -331199-339999---- 978-3-642-833177 5.
11

---

Published as a conference paper at ICLR 2025
Kathryn Chaloner and Isabella Verdinelli. Bayesian Experimental Design: A Review. Staristical Science, 10(3), August 1995. ISSN 0883-4237.
Yuyao Chen, Lu Lu, George Em Karniadakis, and Luca Dal Negro. Physics-informed neural networks for inverse problems in nano-optics and metamaterials. Optics Express, 28(8): 1618, April 2020. ISSN 1094-4087.
Zhao Chen, Yang Liu, and Hao Sun. Physics-informed learning of governing equations from scarce data. Nature Communications, 12(1):6136, December 2021. ISSN 2041-1723.
Zhiliang Chen, Gregory Kang Ruey Lau, Chuan-Sheng Foo, and Bryan Kian Hsiang Low. Duet: Optimizing training data mixtures via feedback from unseen evaluation tasks, arXiv preprint arXiv:2502.00270, 2025.
Zhongxiang Dai, Gregory Kang Ruey Lau, Arun Verma, Yao Shu, Bryan Kian Hsiang Low, and Patrick Jaillet. Quantum Bayesian Optimization. In Proc. NeurIPS, 2023a.
Zhongxiang Dai, Quoe Phong Nguyen, Sebastian Shenghong Tay, Daisuke Urano, Richalynn Leong, Bryan Kian Hsiang Low, and Patrick Jaillet. Batch Bayesian Optimization for Replicable Experi- mental Design. In Proc. NeurIPS, November 2023b.
Adam Foster, Martin Jankowiak, Elias Bingham, Paul Horsfall, Yee Whye Teh, Thomas Rainforth, and Noah Goodman. Variational Bayesian Optimal Experimental Design. In Proc. NeurlPS, volume 32, 2019.
Peter I. Frazier. A Tutorial on Bayesian Optimization. arXiv, 2018.
Mattia Gazzola, Levi H. Dudte, A. G. McCormick, and Lakshminarayanan Mahadevan. Forward and inverse problems in the mechanics of soft filaments. Royal Society Open Science, 5(6), June 2018.
Omar Ghattas and Karen Willcox. Learning physics-based models from data: perspectives from inverse problems and model reduction. Acta Numerica, 30:445-554, May 2021. ISSN 0962-4929, 1474-0508.
Apivich Hemachandra, Zhongxiang Dai, Jasraj Singh, Kee-Kiong Ng, and Bryan Kian Hsiang Low. Training-fres free neural active learning with intializationnrooouttees guarantees. In Proc. ICML, pp. 11971, July 2023.
Desi R. Ivanova, Adam Foster, Steven Kleinegesse, Michael U. Gutmann, and Thomas Rainforth. Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods. In Proc. NeurIPS, volume 34, pp. 25785-25798, 2021.
Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural Tangent Kernel: Convergence and Generalization in Neural Networks. In Proc. NeurlPS, volume 31, 2018.
Ameya D. Jagtap, Zhiping Mao, Nikolaus Adams, and George Em Karniadakis. Physics-informed neural networks for inverse problems in supersonic flows. Journal of Computational Physics, 466: 111402, October 2022. ISSN 00219991.
Wang Jin, Esha T. Shah, Catherine J. Penington, Scott W. McCue, Lisa K. Chopin, and Matthew J. Simpson. Reproducibility of scratch assays is affected by the initial degree of confluence: Experi- ments, modelling and model selection. Journal of Theoretical Biology, 390: February 2016. ISSN 0022-5193.
Pang Wei Koh and Percy Liang. Understanding Black-box Predictions via Influence Functions. In Proc. ICML, 1885-1894, July 2017.
Andreas Krause, Ajit Singh, and Carlos Guestrin. Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies. The Journal of Machine Learning Research, 9:235-284 June 2008. ISSN 1532-4435.
John H. Lagergren, John T. Nardini, Ruth E. Baker, Matthew J. Simpson, and Kevin B. Flores. Biologically informed neural networks guide mechanistic modeling from sparse experimental data. LLoS Computational Biology, 16(12): 1008462, December 2020. ISSN 1553-734X.
12

---

Published as a conference paper at ICLR 2025
Gregory Kang Ruey Lau, Apivich Hemachandra, See- Ng. and Bryan Kian Hsiang Low. PINNACLE: PINN Adaptive ColLocation and Experimental points selection. In Proc. ICLR, April 2024a.
Gregory Kang Ruey Lau, Wenyang Hu, Liu Diwen, Chen Jizhuo, See-Kiong Ng. and Bryan Kian Hsiang Low. Dipper: Diversity in prompts for producing large language model ensem- bles in reasoning tasks. In NeurlPS 2024 Workshop on Foundation Model Interventions (MINT), 2024b.
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep Neural Networks as Gaussian Processes. arXiv, March 2018.
Jachoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl- Dickstein, and Jeffrey Pennington. Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):
124002, December 2019. ISSN 1742-5468.
Vincent Lim, Ellen Novoseller, Jeffrey Ichnowski, Huang Huang, and Ken Goldberg. Policy- Based Bayesian Experimental Design for Non-Differe Differentiable Implicit Models. arXiv, March 2022.
Xu Liu, Xiaoya Zhang, Wei Peng, Weien Zhou, and Wen Yao. A novel meta-learning initialization method for physics-informed neural networks. Neural Computing and Applications, 34:14511 14534, 2021.
Quan Long, Marco Scavino, Raúl Tempone, and Suojin Wang. Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations. Computer Methods in Applied Mechanics and Engineering, 259:24-333 39, June 2013. ISSN 0045-7825.
Quan Long, Mohammad Motamed, and Raál Tempone. Fast Bayesian optimal experimental design for seismic source inversion. Computer Methods in Applied Mechanics and Engineering, 291: 145, July 2015. ISSN 0045-7825.
Jay I. Myung, Daniel R. Cavagnaro, and Mark A. Pitt, A tutorial on adaptive design optimization. Journal of Mathematical Psychology, 57(3):53-67, June 2013. ISSN 0022-2496.
Quoe Phong Nguyen, Bryan Kian Hsiang Low, and Patrick Jaillet. An cefformation-thherreed frame- work for unifying active learning problems. In Proc. AAAI, pp. 9126-9134, 2021.
Alex Nichol, Joshua Achiam, and John Schulman. On First-Order Meta-Leammin Algorithms. arXiv, October 2018. arXiv:1803.000 ..0999 [es].
Tom Rainforth, Adam Foster, Desi R. Ivanova, and Freddie Bickford Smith. Modern Bayesian Experimental Design. arXiv, February 2023.
Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, February 2019. ISSN 0021-9991.
Mohsen Razavy. An Introduction to Inverse Problems in Physics. World Scientific, July 2020. ISBN 978981 1221675.
James Sethian. A fast marching level set method for monotonically advancing fronts. Proceedings of the National Academy of Sciences, 93(4):1591-159) February 1996. ISSN 0027-8424, 1091- 6490.
Mohammad Afzal Shadab, Dingcheng Luo, Eric Hiatt, Yiran Shen, and Marc Andre Hesse. Investi- gating steady unconfined groundwater flow using Physics Informed Neural Networks. Advances in Water Resources, 177:104445, July 2023. ISSN 0309-1708.
Jonathan D. Smith, Kamyar Azizzadenesheli, and Zachary E. Ross. EikoNet: Solving the Eikonal equation with Deep Neural Networks. IEEE Transactions on Geoscience and Remote Sensing, 59 116:51668810666 December 2021. ISSN 0196-2892, 1558-0644.
13

---

Published as a conference paper at ICLR 2025
Masataka Tanaka and Huy Duong Bui (eds.). Inverse Problems in Engineering Mechanics: IUTAM Symposium Tokyo, 1992. Springer Berlin Heidelberg, Berlin, Heidelberg, 1993. ISBN 978-3-642- 52441-7 1144---977---642-22449444
John Taylor. Classical Mechanics, Information and Interdisciplinary Subjects Series. University Science Books, 2005, ISBN 9781891389222
Aad van der Vaart. Asymptotic Statistics. Asymptotic Statistics. Cambridge University Press, 2000. ISBN 9780521784504.
Curtis R. Vogel. Computational Methods for Inverse Problems, Society for Industrial and Applied Mathematics, January 2002. ISBN 978-0-89871-550-77
Umair bin Waheed, Ehsan Haghighat, Tariq Alkhalifah, Chao Song, and Qi Hao. PINNeik: Eikonal solution using physics-informa neural networks. Computers & Geosciences, 155:104833, October 2021. ISSN 0098-3004.
Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why PINNs fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, January 2022. ISSN 0021- 9991.
Malcolm C. A. White, Hongjian Fang, Nori Nakata, and Yehuda Ben-Zion PyKonal: A Python Package for Solving the Eikonal Equation in Spherical and Cartesian Coordinates Using the Fast Marching Method. Seismological Research Letters, 91(4)22378-238,, June 2020. ISSN 0895-0695.
Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A comprehensive study of non- adaptive and residual- adaptive sampling for physics- infores- neural networks. Computer Methods Applied Mechanics and Engineering, 403:1 15671, 2023. ISSN 0045-7825.
Zhaoxuan Wu, Yao Shu, and Bryan Kian Hsiang Low. DAVINZ: Data Valuation using Deep Neural Networks at Initialization. In Proc. ICML, pp. 144150224177 June 2022.
Xinyi Xu, Zhaoxuan Wu, Arun Verma, Chuan Sheng Foo, and Bryan Kian Hsiang Low. FAIR: Fair collaborative active learning with individual rationality for scientific discovery. In Proc. AISTATS, 2023.
14

---

Published as a conference paper at ICLR 2025
A REPRODUCIBILITY STATEMENT
The codes for our implementation, scripts for running experiments and the required dataset are attached in the supplementary materials of the paper submission.
B
NOTATIONS
Table 2: List notations used throughout the paper
Symbol
Meaning
Example
D
PDE operator
Boundary condition operator
X
Input domain
0x
Boundary of input domain
Set of feasible observation inputs
B
PDE parameter
Ug
Solution for with PDE parameter 3
Estimate of PDE parameter from inverse solver
NN parameter
ùg
NN with parameters 0 PINN training loss Observation loss for PINN Collocation points loss for PINN
L Lobs LPDE F Us Ox Y
Forward simulator PINN, with NN parameter 0,, which estimates u,s Observation selector with input X
Mock observation output from ug (X) Inverse solver
ED criterion without inverse ensemble approximation ED criterion computed based on outputs thread i of framework Parameterization for observation input Observation input corresponding to parameter y Set of valid observation input parameterization Derivative or Jacobian w.r.t. æ
a
X,
S V
A
Hessian w.r.t.
a.
Closed interval between and b
C RELATED WORKS
Inverse problems. Inverse problem (IP) (Vogel 2002; Ghattas Willcox 2021 is an commonly
studied class of problem in many science and engineering disciplines such as classical mechanics (Tankka Bui 1993 Gazzola et al. 2018), quantum mechanics (Chadan et al. 1989) or geophysics (mmith et al. [222]] Waheed et al. 2021). Many methods of solving IPs have been proposed, often involving minimizing the objective as stated in (2), possibly with addition of some regularization
terms. One such method is by using the Newton-conjugat gradient method (Biegler et al. 2003
Ghattas & Willcox 2021) to optimize the objective as stated in 2 However, the method relies on finding the optimal /3 through gradient update steps, which requires computing the gradient and Hessian of the objective function with respect to 3. The computation
the gradient and Hessian is often done by reformulating the IP (which can be viewed as a constrained optimization problem) to instead be based on the Lagrangian, then using adjoint methods to compute the corresponding gradient or Hessian (Ghattas & Willcox 202T). This typically results in gradient and Hessian computations requiring only some finite rounds of forward simulations instead. The computed Hessian can often also be used in Laplace's approximation in order to obtain posterior distribution p( X, Y) for the inverse parameter (Long et al.) 2013 Beck et al.) 2018 Ghattas & Willcox 2021). However, this method is still restrictive since may involve careful analysis of the PDE that involved in gradient
the IP in order to form the correct Lagrangian and compute the adjoint. Furthermore, computation would require one forward simulation the system, which is prohibitive if the forward simulation step itself is expensive.
15

---

Published as a conference paper at ICLR 2025
Physics-informed neural networks. In recent years, physics-informen neural networks (PINNs) have been proposed as another method used to both perform forward simulations of PDE-base problems (Raissi et al. 2019) and for solving IPs (Raissi et al., 2019). PINNs solve PDEs by parameterizing the PDE solution using a neural network (NN), then finding the NN parameter such that the resulting NN obeys the specified PDE and the IC/BCs. This is done via collocation points, which are pscudo-training points for enforcing the PDE and IC/BC soft constraints. PINNs are difficult to train in many PDE instances, such as when the solution is known have higher frequencies (Wang et al. 2022). As result, many works have been proposed in improving the trianing of PINNs by rescaling the loss functions of PDEs (Wang et al. 2022). or through more careful selection of collocation points (Wu et al. 2023 Lau et al. 2024a).
Experimental design. Typically, in solving IPs, the observational data will not be available right away, but instead has to be measured from some physical system. Due the costs in making measurements of data, it is important that the observations made are carefully chosen to maximize the amount of information that can be obtained from the observations. Experimental design (ED) a problem which attempts to find out what the best data to observe in order to gather the most information about the unknown quantity of interest (Rainforth al. [023). ED closely related to data selection, active learning (Nguyen et al.) 2021 Hemachandra et al. 20233 Lau et al.) 2024b Xu al., 2023) and Bayesian optimization (Dai et al., 2023a Chen et al. [2025), , where the aim is to select input data so that we are able to recover the unknown function or the optimum of the unknown function.
A certain variant often considered in ED is Bayesian experimental design (BED). In the BED framework, we assume prior p(3) on the inverse parameter to compute. For given design parameter d, the system observes some output y.
3 d) p(3)
p(3)d,y) BB-r(() p(y)
(10)
(gyl,,ck d)
Given the inference we can compute the expected information gain (EIG), which is sometimes also known as the Bayesian D-optimal criterion. EIG criterion is defined as the expected Kullback-Leible divergence between the prior p(3) and the posterior p( (3dd, y), averaged over the possible observations y. More formally, this can be written as
(d) = Ey-p(ria) DKL. (PKI(((8 vldd,)))ll(())))) = Hp) Bprr)( HHpp()d, =
(11)
where the posterior is defined in (IO). A naive approximation technique is to perform a nested Monte Carlo (NMC) approximation (Myung al. 2013).
p(yel Br,0, d) M p(y: (B,i,d)
(IG
log
(12)
MI
where Bi,0, Bi, Bigj v(3) and y p(ylB,,, d). The estimate approaches the true EIG as N, .M + co. In practice, however, using the NMC estimator results in a biased estimtor for finite N and M, and results in slow convergence with N and M. To improve on the NMC estimator, various schemes have been proposed mainly to remove the need to perform two nested MC rounds, including variational methods (Foster et al. 2019) and Laplace approximation methods (Long al., 2015) Beck et al. 2018).
D EXAMPLES OF PDES CONSIDERED IN THIS PAPER
In this section, we provide an extensive list PDEs which are considered in our experimental setup, and the ED setup and dataset used in experiments. We divide them into PDEs where our experiments are based on simulation data (generated either from some closed-form solution or some numerical simulators), and PDEs which are based on real data (collected from physical experiments). The latter provides an interesting use case for PIED since is able demonstrate its performance on realistic scenarios with real noisy data.
16

---

Published as a conference paper at ICLR 2025
D.1 PDES WITH SIMULATION DATA
Damped oscillator. The damped oscillator is of the introductory second-order ordinary differ- ential equation (ODE) in classical mechanics [Taylor 2005). We consider the example due to the existence of closed-form solution and its nice interpretation under our ED framework.
Imagine a masi-g system which is laid horizontally. The spring has spring constant & and experiences a resistive force which proportional its current speed, where the constant of proportionality is We let the attached mass have a mass of M. We also assume the case where there are no external driving forces on the system. By applying the relevant forces into Newton's 'ss law of motion, the displacement of the mass
xt)) can be expressed by the differential equation
dx dr M
1
kæ O.
(13)
dt2
dt
dæ
Given the IC of z(0) Fo and
(0) vo. we can write the solution as Taylor 2005
dt
Ae rt cos( VWj +
if y wo. if y = Wp.
x(t)=
(Bt + C)e De -et (//
(14)
Fe
1-- -- (1- Fe-
if y > Wps
where y = ///2,,w. = v /k/,, and A, B, C, D, are constants which depends on zo and Uo-
know the system follows the PDE in (I3), with some known value of vn E [0, 1] and op E [-1, I]. We set M = 1, and would like to compute the values for 4] and & [0, 4]. In our IP, we allowed to make three noisy observations three timesteps 1,,42,4 13 E [0, 20). which can be chosen arbitrarily. We note that while it is unrealistic for these measurements to be made arbitrarily, we do so in order to be able construct a simple toy example which can be experimented with. The resulting true observation were computed using the closed form solution in 14). and has added noise with variance 10-3
In our experiments,
assume
1 E [0,4]
Wave equation.
For simplicity, we consider the ID wave equation, which is given by
8u 0t
222 82u
(15)
Ox.
where v represents the speed of wave propagation, which may be scalar or function of In the inevrse problem setup, one may be required to recover the wave velocity given measurements of u(z,t), t).
In our experiments, assume we have system which follows the wave equation given by (15 over the domain x E [0, 6] and E [0, 6]. We fix the IC u(x, 0), and assume the wave velocity in the form
0
if x = 0,
V1
if < x < 4
(xx)
(16)
1/2 S x < 6,
0
if = 6.
In this case, 01 122 [0.5, 2] are the PDE parameters to be recovered in the IP. In the ED problem, we restrict the points to be placed only at regular time intervals, i.c., following (25) where we restrict n, 72,77 f' [0, 6] and let t (0, 0.2, 0.4, 6). The true observations using code from Binder (2021). with outputs interpolated on continuous domain and truncated to
numerically generated
the nearest 6 decimal places to simulate cases where measurements can only be made up finite precision. Examples of these possible solutions can be found in Fig.
Navier-Stokes equation. Navier-Stoll equation a well-studied PDE which describes the dy- namies of fluid. In our experiment, we consider the stream function of an incompressible 2D fluid,
17

---

Published as a conference paper at ICLR 2025
(a) 1.34, U2 1.43
01
1.81,
1.12
01
0.61, 1.45
0.96
1.00
0.72
0.75
0.90
0.48
0.50
0.72
0.24
4-
0.25
0.54 4
0.00
0.36
0.00
0.25
0.18
0.24
0.50
2
0.00
0.48
0,75
-0.18
0.72
1.00
-0.36
0.96
1.25
Figure 7: Examples of solutions for wave equation 115 with wavespeed in the form in (16). The blue points are example of possible set of observations which are made at fixed timesteps.
which can be written as
du
Ou
du
Op
Ouu
pu
+
+ H
+
(17)
0
0x
Oy
dx
Ox²
Oy²
Öu
Ou
du
Op
00
+ p
u
+ 1
+
(18)
Ot
0x
Oy
Dy
0x
0n²
du
0v
0y = O.
(19)
dr
In our IP, we consider the steady state flow inside a pipe, where we let 0,u = 0, = Then, the velocities u(x, y), v(z, y) and the pressure p(z, y) only dependent the 2D spatial coordinates. We assume the viscosity y is given and the goal is to recover the density p. In the ED problem, we allowed freely choose the spatial location to make measurements of u, u.p. The ground truth data is simulated using ANSYS Fluent.
Eikonal equation. Consider the Eikonal problem setup, which often use to reconstruct material composition of some region based on how waves propagated through the medium reacts. Its equation relates the wave speed v(x) at a point and the wave propagation time (xx) at point with PDE given
by (Smith et al. 2021
T(x) (Vo(x)) with T(*o)=
=
(20)
where To where the wave propagates from. The goal of the IP to recover the true function v. However, this involves conducting seismic activities at different set values of zo, and obtaining the corresponding reading for T(x) at specified values of x.
In our experiments, assume that we have system which follows the equation specified in (20). and the goal is recover the values u(x) for the entire domain. We fix 2o, then for each TP instance, we draw random ground truth v(z) using NN with random initialization. For the ED problem, the aim is to find 30 random observations from the 2D input domain [0,5] x [0, 5] to observe values of (zx). The observation inputs are only required to be within the input domain. The true observations are generated using PYKONAL package (White et al.) 2020), with outputs interpolated continuous domain and truncated the nearest decimal places simulate cases where measurements can only be made up to finite precision
D.2 PDES WITH REAL DATA
We now describe some of the PDEs we have conducted experiments with where real-life data available for. Note that real-life experimental data is often searce, and often the true PDE parameters not readily available. To obtain use the reported values from the corresponding data source when we can. the case where this is possible, resort numerically computing the PDE parameters using the whole training set.
ground-truth values for the inverse problem,
attempt to
18

---

Published as a conference paper at ICLR 2025
Groundwater flow. In scenario, consider the steady-state Dupuit-Boussinge equation (Boussinesq 1904). which given by
dh dx
Kh
= 0
(21)
dx
where K is the hydraulic conductivity.
In our experiments, we use the data provided in (Shadab et al. 2023), which reports the flow profile of some liquid at various flow rates across cell filled with 2mm beads. In this dataset, we treat the hydraulic conductivity K as an unknown quantity we would like to recover in our inverse problem.
In the ED problem, the algorithms have to choose values of make the observations h(r).
Cell growth. Scratch assay experiments can often be modelled via a reaction-diffusion PDE Jin ct al. 2016), which can be written as
0p ppp dt dr
+ Flpl
(22)
where F[p] is some function of p and e1 is an unknown constant.
In our experiments, we use the scratch assay data collected by Jin et al. 2016). whose dynamics have also been studied in other subsequent papers Lagergren al. (2020); Chen et al.) (2021). Based on Chen et al. (2021). we will let Flp)= = Cap + cap" in our case, where and C3 are unknown. The data collected at five timesteps every 12 hours for total 48 hours. We use the values obtained by Chen et al. (2021) the ground truth values for C12 C2 C3- In our ED problem, the IP with the initial population values p(z, t = 0) are given, and the algorithms choose the values of æ to query the cell population at, where the corresponding values will be provided each of the other four timesteps.
PARAMETERIZATION OF INPUT POINTS
Due to operational constraints, the observation input X often cannot be set arbitrarily, but instead restricted to some feasible set S C X For example, the observations may be only be possible at specific time intervals, or must be placed in certain spatial configurations like a regular grid. Hence, unlike existing ED works,
allow for both freely-chosen observation inputs and constrained configurations which can be parameterized by some design parameter E S, C i2 where d S M din- Specifically, we consider of the form
= (X, where X, ( j=1
(23)
In this form, finding the X which optimizes criterion is then the same performing optimization on instead, which can be seen as a continuous optimization problem.
To better illustrate the input points parameterization method, provide examples of possible methods to constrain the input points and how they be expressed in the appropriate forms. Fig. B graphically demonstrate what some of these observation input constraints may look like.
(a) Free point placement
(b) Regular time intervals
(c) Regular grid (ID)
2
V1
V2
Y2
V3
Figure 8: Examples of observation input placements.
19

---

Published as a conference paper at ICLR 2025
Points placed freely in input space. In this case, the points be placed anywhere in X without restriction. To parameterize this, we define d = Md and define
x, = {(1 12 -an) ). (Tau+t Ndn+2 Y2da ... (Tidd dh 7Mda-da++ Ydd
(24)
Points placed at regular time intervals. In this case, assumee the points are placed at chosen
spatial locations, and makes mcasurements at fixed time intervals f1 t2,.... ty. This is realistic in the case where the PDE solution evolves over time, and it makes sense fix the location of the sensor but allow it to make readings throughout the evolution the system over time. In this case, only needs encode the spatial location where the sensors should be placed. Specifically, if there is one spatial dimension, then the parameterization for the sensors can be chosen as
X, (z,t) 72. ....... 7al and t {ti, 12..,,...,,..1. .....ts)).
{71,
(25)
Points placed in regular grid. In this case, the points placed in regular grid at regular intervals. This provides one way to add extra constraints for sensor configurations to reduce the dimension of the problem. For demonstration,
ID problems, we want to allow placement of sensors in total, we can let d = 2 and parameterize the sensor placements as
X,
21 71
72
71
71
+
72 71
12
(26)
S
F FURTHER DETAILS ABOUT THE EXPERIMENTAL DESIGN CRITERION
We describe the criteria used further. Note that we adjust the notations to incorporate the parameteri- zation of input as discussed in App.
F.1 FEW-STEP INVERSE SOLVER TRAINING CRITERION
F.1.1 PSEUDOCODE FOR CRITERION COMPUTATION
Algorithm Criterion estimation by Few-step Inverse Solver Training
1:
function (X,,
2:
// Perturbation of NN and estimated PDE parameters
3:
a+ + Eo where Eg (0, σ2)
4:
3 t where Eg ~ (0,
5:
Y,, < (X.))
6: // Partial training stage
7:
Initialize (O(U), 3(0))
8:
for j 1,..., do
9:
The training may be replaced with other gradient-based methods as well in practice
(63 < 10--1) v Vo (000:--))),---- X,, Y,
10:
11:
(00 305-13 1/ ((0 30-1); Xzy Yn)
12: return -1(3(r) - 111122
Furthermore, a large value of r can also make the criterion not differentiable in practice due to the need perform back-propagation over the gradient descent update steps. Fortunately, we find that in our experiments, using I 200 is usually sufficient given the correct perturbation noise level is set.
F.2 MODEL TRAINING ESTIMATE CRITERION
F.2.1 ASSUMPTIONS AND PROOF 9
In this section, we describe the approximation of training dynamics used in Model Training Estimate, which relies approximation of NN training using NTKs (Jacot al., 2018) and has been used
20

---

Published as a conference paper at ICLR 2025
extensively in various active learning and data valuation methods (Wu et al. 2022 Hemachandral
iet al. 2023 Lau et al. 2024a).
We consider PINNs in the lincarized regime to demonstrate the validity of the approximation given in ( The results will be an extension from that given in (Lee et al. 2019). and is an assumption that
has been used in past PINN works (Lau et al. 2024a). For convenience, will drop the subscript and write the learnable PDE parameter 3 and the NN parameters 0.
We first recall the assumptions for the linearized regime of NNs. Following past works on the NTK for NNs (Jacot al. 2018 Lee et al. 2019) and PINNs (Lau et al. 2024a), we assume that
(x(x) Ugcon (x) J..000 (0- ((0)
(27)
and
3 3(0) 0 g(o)
Dlūo, (z)) Ps Diugo, (z)) +
[(()] p, p.0
(28)
We briefly discuss the consequences of this approximation. The linearized regime only holds when the learned parameters are similar the initial parameters, which based on past works will hold when the NN is wide enough or when the NN is near convergence. This is unlikely to hold in the real training of PINNs. except in the case where the initialized parameters are already close to the true converged parameters anyway. Nonetheless, in our work, we do not use the assumptions to make actual predictions on the PDE parameters in the IP, however only use it to predict the direction of descent for y, which would only use the local values anyway. Furthermore, we can also perform some pre-training in order to get closer to the converged parameters first as well to make the assumptions more valid.
Let ((0) and g(L) be the learned PDE parameter and NN parameter respectively at step t of the GD training. We will write ((t) Voūgc (X,), J(C) VoDiūgro, (Xxp), and f(()
p.3
DaDiao, (Xp) (note that V Lage (X,)) = 0). We can then write the GD training step as
0 3007
VaL(Ol) 300; 100,, Xyy Y) VoL(OC)) ,(00 X.,
((c)
17
(29)
{(()
daces (X,) - Yn Diūgios (xx)) - s(xp)
p.
(30)
J(
7.8
J(
Under the linearized regime, we can see that, j(() Fo 7.0- pB n. these approximations to obtain
J(1] so JLo and J1y ~ 510 We can use
0 [(t) _3(07 a [3()
(31)
0t (0() 000)] Ot ((())
[B(0] 3:00
agcon (X,) - Y, Di [(0))((p)))))))))))))))))))))[ - s(Xp)
R3 I J! T
TT
(32)
@())) g(o)
which can be solved to give
[ - {800] ((o) - )(0) = -((00 T (1,00) T (T - no 55
Apcor (X,) - 8,
D (iggoos (xx) - f(Xp).
(33)
At convergence, i.e., when L - oc, we can reduce the results for gl00) - 3(00 as
igr (Xx)) - Y (I())((()) 7-1 D Diugos ](0) f(x))
B(x0) - 3(00= ~
((0)
(34)
P.
as claimed in g
21

---

Published as a conference paper at ICLR 2025
F.2.2 PSEUDOCODE FOR CRITERION COMPUTATION
Algorithm Criterion estimation by Model Training Estimate
1: function (
2: Y, <- u (X,)
3:
if reuse forward PINN parameters then
4:
((ا) perturbed version of forward PINN parameters
5: (6) perturbed version of Bg
6: else
7:
Initialize (010), for j = 1,..., do // The training may be replaced with other gradient- based methods as well in practice
Can set ((0) to Osi as well
8:
9:
((3) -- ( n - y;
11:
360 + -(- vvV -V
To prevent backpropagation over the GD training
Set V.,(()) = 0 and V v²,3())=
14:
Perform estimation
B(00) = =3(r)
_(() P.B
(I J( (
)())))
1-1
2ocn (X,) Y, Dluger (x,) 1(x))
(35)
15:
return -113(o0) - -3:112
Note that in Line 13 of Alg. we set the gradients V V,g(rr and V,,8(r) to zero. This done since when implementing the function, it will be possible to write g and ((()) as explicit functions in terms of 6, meaning that when performing propppropggttion over the Model Training Estimate criteria, it will also consider these derivatives as well. This can cause memory issues due to the need of back-propagating over many GD steps. Therefore, by explicitly stating that the gradient zero, it avoids problems during the back- poop-propagttion phase. In practice, this can be done via the loop_gradient function on JAX, for example.
A speedup that can be applied on MoTE is to instead of performing initial pretraining to obtain the eNTK, we could reuse the NN parameters from the forward PINN in order to compute the eNTK instead. We find that this trick is useful since it gives performance almost good performing initial training in each criterion computation, while being more efficient since no additional training needs to be done.
F3 TOLERABLE INVERSE PARAMETER CRITERION
F.3.1 MOTIVATION
To demonstrate the flexibility of PIED, we present another possible method which aims to find the optimal X in (8) without approximating 3. by taking the opposite approach of choosing X to query noisy observations that would have the least harmful impact when training an inverse PINN already initialized with the right 3.. Intuitively, given an inverse PINN already trained to the correct PDE parameter B, = 8,, a bad choice of X would possibly cause the PDE parameter to drift other incorrect 3' with further training, while a good choice would retain the correct 3y-
Specifically, during inverse PINN training, the observational data (X, Y;) influences the loss in explicitly through Lobs- though also implictly through LYDE- see this, note that (X, Y,) changes Lots during training, which adjusts NN parameters 8, via gradient descent optimization, including potentially drifting 3; to nearby value B². We could approximate how this shift from Br to 3 changes the NN parameters from 0, 0g as (33),, is minimized
0.(3") 0,
(O. VoLrDn(O,, Bi)),
(36)
22

---

Published as a conference paper at ICLR 2025
giving approximation of the overall impact of (X, Yi) (4) via Lobs (8,());; Y,). given by
(x,((3') & Cots (8,))) = 1170 (X) 1111?
(37)
Given choice of X and its observation values Y;, we could characterize how likely the inverse solver would remain 33 drift to neighbouring 8¹ after training by considering the Hessian of (x,+
a "larger" Hessian means a lower chance that 3, will change as the inverse solver undergoes training to minimize loss, since it is harder for the gradient- based training to "leave" the narrow range of B. Hence, to optimize for we could maximize the criterion
ÔTIP, (X) = log det Viy (3.)
(38)
F.3.2 ASSUMPTIONS AND ROUGH PROOF OF 36
We demonstrate the validity of (36). which is adapted from the proof in van der Vaart (2000). For convenience, we will drop the subscript and consider the NN parameters and PDE parameters
Suppose we fix a B, and let 0 = arg ming (eDe("',B) Since (0, 3) is minima of LPDE, we can see that (O, (3 O.
Let 0(3") - arg ming CPDE( ,1), Our goal is to approximate 0(3)) when 3 As 3. 0(33) = 0(3) = 0(33) - 0. We can
Let PCPDE ,0',''')))))) ) = LPDE(O', 38)) CFDE 3²) and -0(3") =
use this to write
(oCr)((((3))) Ba)
VoLLDE ( =0(0",,)))
(39)
R V V (0, + (0,, (0, (00))
(40)
VoLrDE(O, + ALD (,, 8¹) + (V)CrDE(O, (') (0(5')
(41)
AC (V)CrDE (6, 88) 23()))
(42)
where (40) arises from performing Taylor expansion on VoLrDE (8()), around 0.
Since (0(8"), 8*) is a minima of LPDE, we know that VoCPDE (0(3) 3') 0, and therefore we can solve for A0(3")) to obtain
A0 (V)LrDe(O, ()) A LPDE(0,
(43)
which can be rewritten to match the form (36).
Some readers may question whether (36) is valid for NNs where the learned NN parameter may not be a global minima. note that despite this, we are only interested in the curvature around minima anyway, and Furthermore, this technique has been used for NNs in other applications as well, one notable instance being the influence function (Koh & Liang 2017) which aims study how the test performance of supervised learning tasks changes as certain training examples are upweighed. In the paper, they able design scoring function based the same mathematical tool and successfully interpret performances of NNs. In our work, we find that despite the assumptions on the global minima is not met, we still able achieve good empirical results well.
still inspect the change of that minima the loss function changes regardless.
F.3.3 CHOICE OF Loss FUNCTION USED IN 36
Note that in (36. compute the Hessian w.r.t. the forward PINN loss. Some readers may wonder why the overall PINN loss from is not used instead.
This choice is due to two main reasons. First, it is more computationally efficient. Notice that the change in NN parameter in (36) depends on LPDE, and therefore are independent the observations and hence independent of the design parameters y. This leads the optimizing of the final criterion to not require differentiating (36) with respect to y, reducing the computational load during optimization. We find that doing does not cause significant effect in the obtained design parameter 7"
Second, this matches more closely to the inverse problem setup as described in Sec. In the IP as described, the objective ( is usually to find the / whose output matches that of Y In TIP, the
23

---

Published as a conference paper at ICLR 2025
tolerable parameters is the Hessian based on the objective (37 which also be interpreted in similar way as (2). Furthermore, through this interpretation, TIP also exhibits a stronger connection to Bayesian methods, as discussed in App. F33.4
F.3.4 BAYESIAN INTERPRETATION OF TIP
Interestingly, TIP can be seen as the application of the Laplace approximation on the posterior distribution of 3 after observing data (X,, Y), assuming uniform prior and Gaussian observation noise, and view the criterion score ÔTT,, as the information gain of 3; given observational data (X,, a.) Assume that the observed data is generated from the true underlying function with added Gaussian noise. Then, the likelihood function can be written as
M p(Y (Y = NNX 1 = II N ((uglup(an,)).o))
(44)
If we assume uniform prior over B (i.e., assume p(5) = for some constant c), then it is simple to show that p[3| Y) 33 X,) (x)pp(())XX()) (Xy), where p(Y]x)) can be treated as constant. In this case, we can write the log posterior as
N
logp(3) Y) =
Σ
log ((ss)luseen
(45)
j=1
N
(u)(1)g) y;)²
-oog((((X.))
(46)
20
j=1
|| (X))
21
constant.
(47)
20
To make 447) tractable, we can apply Laplace's approximation on the posterior. To do so, we perform a Taylor expansion on log p(3) Y) around the MAP of the distribution. In this case, we would
expect the MAP to be at the true PDE parameter 3o. where this would give
log logp(3]X,,, Y) = O. Once expanded,
03
log logp(81X,,,)) logr(33)X,, Y) + (3 - 30)
V3 logg((B/X y) (B (3-)
(48)
7 11u.0 (X,) - -Y1127
100g((0010 X,, Y)-
202 (33 Ba)
(B Boo-
(49)
202
Note that this can also be written as
V2 11 (X,,) - Y112 7
X Y) Re X(3, Y) exp
- (3 -3))
(B - 30)
(50)
20
which confirms that the Taylor expansion approximates the posterior distribution as Gaussian
Ilu., (X,)) Y Since
distribution with mean MLppllaee = So and covariance matrix Lapllaee
V3
20
the posterior is approximated as multivariate Gaussian distribution, is simple approximate the entropy of / as distributed by p(3) X,, Y' using the entropy of multivariate Gaussian distribution as
M
H|3|X Y]
log det Laplace + log 2ne
(51)
2
2
log det V (X.) - 2 + constant.
(52)
2
Finally, from we can approximate the EIG as
EIG(7) = - (X [|
Y']]
(53)
[H[ E
[I] X
(54)
H|[ E (B80
log det Villus, (X,) - r'1//
+ constant
(55)
24

---

Published as a conference paper at ICLR 2025
where 55) uses the approximation of entropy in 52 Note that HH]3] is constant independent of y and therefore can be ignored.
Note that in the derivation so far, we have assumed that are able to compute the PDE solution ug and be able to compute how the solution output changes w.r.t. G. This, fortunately, is made possible using PINNs. Specifically, in the likelihood distribution p(Y|3, Xx,) from (44) and sampled from in the expectation in (55), we can replace the us with NN 20(3) with parameter ((3) as defined in (36). Similarly, inside the expectation term of (55), we can replace " ( (X,) with (a( and y
with noisy reading of (.())))) Ultimately, ignoring additive constants, this gives
EIG(?) ~ - log det v [1ū) -aaca)) (Xx) =1l?
(56)
B=Bb
For simplicity, we can ignore the additive Gaussian noise in the approximation of Y, i.e., ignore the term, and the term inside the expectation is the same as that in (38)
This analysis links TIP some existing ED methods for IPs based on Laplace's approximation (Beck et al., 2018) Alexanderian et al. 2024). Despite these links, our method remains novel in that through the use of PINNs, we can consider the Hessian of the PDE solution directly. allowing the resulting criterion to be differentiable w.r.t. y. This is unlike past works which often require more careful analysis of the specific PDE involved, and relies on discretized simulations (and therefore are not differentiable w.r.t. the input points). We note that while previous works have proposed the use of the Hessian of the learned inverse posterior distribution (Beck et al. 2018 Alexanderian et al. 2024), our method is novel in that it considers the Hessian (and hence the sensitivity) of the PDE solution directly, rather than of a posterior distribution. This is due to the usage of PINNs and its differentiability in the ED process directly, which has not been done in past works.
We comment about the assumptions required to arrive at the approximation in (56).
We assume that the data is generated with random Gaussian noise. This is standard assumption as done in other IP and ED methods in the literature.
We assume that the posterior distribution unimodal. Note that this assumption does not always hold. One example where this assumption does not hold would be in the case where multiple values of B3 may represent the same PDE parameter (e.g.. 33 represents NN parameterization of an inverse function). In our experiments, we find that the performance of the method remains good regardless. Furthermore, we believe that the issue can be mitigated by considering the problem under some embedding space ((3) where two embeddings similar when their parameterizations represent similar functions. This is likely possible by adjusting our criterion to incorporate & through Lagrange inversion theorem, however will defer this point to a future work.
Another case where this assumption may not hold is when there are some degeneracy in the inverse problem solution. In this case, the problem cannot be alleviated anyway unless more observations data are acquired (a simple way to think about this is when there is only one observation reading is allowed, and therefore a good PDE parameter solution will not be obtainable regardless of the ED method).
We assume that the unimodal distribution is maximal at Sp. Given that the distribution is unimodal, this point would likely hold since the pseudo-observation from the NN is already obtained from using PDE parameter of Bu.
F.3.5 APPROXIMATION OF HESSIAN IN 36
Instead of computing the Hessian of CPDE directly, we can also employ a trick which avoids computing the Hessian directly, but instead approximates the Hessian based on the first-order derivatives.
Suppose we define
X 1/2 11/ to - R(0,8)
(57)
| -1/2 (B/ (00 () 2(xb))
(B
We can see that
1
(eDE(0.3)) R(B,3) (T,,3)R(O, R(0,3).
(58)
2
25

---

Published as a conference paper at ICLR 2025
We then write
VoLPDE(O,3) VgR(O,, R(0,8)
VoR(O,3)_R((B))
(59)
and
CODE B) VeR(0, ( VVR((, B) (R. R),.3).
(60)
In the that (0, 3) are obtained after PINN training has converged, would have R(0,3) = which means we can write
(O ())(0,3) ( VoR(O,
(61)
We therefore can approximate the Hessian as used in (36) using first-order derivatives instead.
F.3.6 COMPARISON WITH OTHER BENCHMARKS
Table 3 shows shows
the results of TIP compared to our other methods. We see that while in some cases we are able to get comparable results to FIST or MoTE, often will perform worse than these other methods. This is likely due to the stronger assumptions that are required for the method.
Table 3; Results for the inverse problems using TIP criterion compared other proposed criteria.
The table is interpreted similarly to Table [
Finite-dimen dimeeiiimeniinn
Function- valued
Real dataset
Wave (x10 1)
Navier-Stoker Stokes (x10 +)
Eskonal (x 10")
Groundwater (x 10*) Cell Growth (x10")
FIST
3.87 (0.76)
(1.45
0.74 (0.02)
1.93 (0.08)
2.62 (0.11)
(2.34)
1.18 (0.11)
(0.02)
(0.60)
(0.04)
TIP
5.11 (0.01)
9.04 (2.04)
0.75 (0.01)
2.25 (0.26)
2.94 (0.23)
G COMPLETE ALGORITHM FOR PIED
We summarize PIED via pseudocode presented in Alg. The ED procedure consists of three main phases learning of the shared PINN parameter initialization, the criterion generation phase which consists of performing forward simulations and consequently defining the criteria, and the criterion optimization phase which proceeds to perform constrained continuous optimization on the criterion.
Note that in framework, the forward simulation only has to be be ran in parallel using packages which allows for parallelization such as vmap on JAX. We also find that the forward simulation be used without explicitly injecting artificial noise, while still giving observation inputs which work well for IPs involving noisy data.
per ED loop, and can all
In the criterion optimization phase, use projected gradient descent to ensure the resulting design parameter is in the bounded space. However, other constrained optimization algorithms could be used well, e.g., L-BFGS-B. We repeat the optimization loop over many runs due the potential non-convexity of the criteria, to obtain better estimate of the optima.
H DETAILS ABOUT THE EXPERIMENTAL. SETUP
H.1 GENERAL ED LOOP AND IP SETUP
Our experiment consists of two phases. In the first phase, we perform the ED loop using PIED or with the other benchmarks. Here, we allow fixed number of forward simulations using PINNs, which the ED methods can query from as many times wants. Each ED methods have time restrictions, where they allowed to run either for a certain duration, or until they have completed some fixed number iterations.
After the ED methods have selected the optimal design parameters, the same design parameters used to test on multiple instances of the IP (we run at least 10 of such instances depending on how much computation resources the specific problem requires). In each instance of the IP, draw random ground-truth PDE parameter, and generate the observations according to the model and the random ground-truth PDE parameter. The IP is solved using inverse PINNs to obtain guess of the
26

---

Published as a conference paper at ICLR 2025
Algorithm 3 PIED
// Learning shared NN parameters Randomly initialize 8si
2: for rounds do
3: Randomly sample .1.-....31. .
4: for I.....k
k do
5:
0 -
NN parameter after training Osi with training loss LPDE (0,
6: Osi 1
Used initialization for all proceeding forward and inverse PINNs
Criterion generation phase
7: for i = 1,.., .M do
8: Randomize PDE parameter |
9: ūgs <- F(3,)
Forward simulation
10: if use FIST criterion then
11: Define É; to FIST criterion from Alg. 1
12:
else use MoTE criterion then
13:
Define å; to MoTE criterion from Alg.
Define aggregated criterion a(x,) - āx,,)
I/ Criterion optimization phase
Initialize "best E S, randomly repeat
16:
Initialize 7' E S,,, randomly for training steps do
18:
Gradient- based optimization Gradient ascent since the criterion should be maximized
19:
7/ + 7 (,,
20:
? projs, (9)
Perform projection s.t. 7 S
21:
if (xx))) al X mee) then
22:
Pbest 6 ?
23:
until computational limit hit
return Thest
Table 4: Architectures of used NNs
Problem
Depth
Width
Activation
Output transformation
Damped oscillator
6
tanh
None
1D wave
3
16
sin
None
2D Wavier-Stokes
6
16
sin
None
2D Eikonal (modelling (g))
6
8
tanh
(x, v) 1/12 xo
2D Eikonal (modelling ())
1
16
sin
(x. v) + |y| + 0.2
Groundwater flow
2
8
tanh
None
Cell population
2
8
tanh
None
PDE parameter. For each instance, we can obtain an error score, which measures how different the PDE parameter estimate from the ground truth value.
For cach problem, we repeat the ED and IP loop five times, where in each time we obtain multiple values for the error. our results, we report the distribution of all the error scores obtained through the percentile values of the error (i.e., percent of all IP instances using certain ED methods have errors at most x), removing some the extreme values (in the main paper, we remove the top and bottom ten percent, while in the Appendix we show the distribution via boxplot removing the outliers). This done since some PDE parameters result IPs which are easier than others, and to demonstrate the performance of each ED methods across all possible PDE parameters.
H.2 PINN AND PINN TRAINING HYPERPARAMETERS
The architectures of the PINNs and other NNs used are listed in Table 4 Note that only multi-layer perceptrons in our experiments. The training process hyperparameters for the forward and inverse PINNs are listed in Table
27

---

Published as a conference paper at ICLR 2025
Table 5: Training hyperparameters
Problem
Training steps
# PDE Col. Pts.
IC/BC Col. Pts.
Optimizer
Damped oscillator
30k
300
I
Adam (Ir = 0.01)
ID wave
200k
15k
2k
L BFGS
2D Wavier-Stoker
100k
2k
300
Adam (Ir = 0.001)
2D Eikonal
50k
10k
1
Adam (Ir 0.001)
Groundwater flow
50k
500
L-BFGS
Cell population
50k
1k
100
Adam (Ir 0.001)
H.3 SCORING METRIC
To judge how well our ED methods perform, we will use the error L(,, ()) of the PDE parameter B. When 3 has finite dimensions represented scalar value vector value), then the loss is simply the MSE loss, i.e.,
I(3,3) 113-8112
(62)
For the case where 3 is function, select some number of test points {0r,) Mer and compute the MSE loss of the estimated function on those test points, i.e.
Nos
L(8 = Σ [|((TT)) (l.
(63)
H.4 BENCHMARKS FOR THE SCORING CRITERION
In this section we describe a few scoring criteria we use as a benchmark. We first describe the benchmarks which based on methods of estimating the expected information gain (EIG).
Mutual Information Neural Estimator (MINE) (Belghazi al.) 2018). The estimator utilizes Donsker-Varadhan representation of the KL Divergence to show that we can provide a lower bound the EIG as
EIG(X,)) 2 (X,)
(64)
A 11g.3 (T log Eer 1e (o0
(65)
where Tę is a parametrized family of functions.
Note that while the estimator CDv(X,) relies on sampling p(Y 1, Xy) and p(Y|X,)) di- rectly, this would require rumning many forward simulations for different samples. Instead, to sample from these two distributions, we draw M random samples of 3 and approximate p(3) with a mixture of Dirac-delta distributions, i.e.,
M Σ 111
~ M
5(3 - 3:) where ,1,..., ~ p(3).
(66)
In this case, the forward simulation only needs to be ran for M samples of 3 and the distributions (3, and p(Y] can be efficiently approximated.
Variational Bayesian Optimal ED (VBOED) estimator (Foster et al., 2019). The original paper desicribes multiple estimators for the EIG, however we will use the variational marginal estimator. The estimator utilizes the fact that we can provide upper bound the EIG as
BIG(X,) < Umarg (X,) & E(y.n)-((8))1118xxx log p(Y13,X,)) (X
(67)
where 94 is variational family parametrized by d. To compute the EIG, find the à which minimizes Umary (X,). Instead of computing the upper bound exactly, we use an empirical estimation based samples of (Y, (8) generated from the PINNs with added noise. Similar
28

---

Published as a conference paper at ICLR 2025
Table 6: Hyperparameters used for different criteria in PIED. Note that for MoTE, the case when = oc refers to when we use the forward PINN for the NTK regression step.
FIST
MoTE
Dataset
Pp
Damped oscillator
0.5
50
30
ID wave
0.5
200
2D Navier-Stokes
0.5
100
00
2D Eikonal
0.01
200
0
Groundwater flow
0.01
100
00
Cell population
0.1 100
1000
MINE, we approximate p(3) with mixture of Dirac-delta distributions (66 such that only a limited number PINN forward simulations are required.
Note that while Foster al. (211)) does propose variational NMC (VNMC) estimator as well, this requires computing p( p(Y, 31x,) = p(3) (Y X,, for a randomly sampled B. is not feasible to compute p(Y|,, X,)) on the fly since this would require running costly forward simulation for the randomly sampled 3, and therefore the method is not included for this benchmark.
We also use other benchmarks which are not based on the EIG, listed as follows.
Random. The design parameters are chosen randomly.
Grid. The design parameters are chosen such that sensor readings are placed regularly in some fashion. For the 1D examples, the sensors are placed such that they all regularly spaced out. For the 2D examples, the sensors are placed such that they are shaped regular 2D grid with each sides having equal number of sensors possible. Note that optimization is done, but instead the observation input configuration fixed per problem.
Mutual information (MI) (Krause et al.) 2008). The criterion considers the outputs Yx., of the chosen observation input X, and the outputs Yx, of some test set X,, and defines the score to the mutual information between Yx,, and Yx,, i.e.
Crsn = MI(YxAX,;
H Yx,] H[XXAX,
(68)
In our experiments, we approximate the observation outputs via a Gaussian pro- cess (GP) whose kernel is the covariance of the PDE solutions, i.e., K(x, Cov Cov (ove)) [us(2)) us(x')), where we approximate the covariance using the forward simulations. By approximating the output using a GP, the entropies H[[x,xx,] and [H[[xxx, Yx,] can be written directly in terms of the approximate kernel function. Also, since we do not perform discretization and treat the problem as combinatorial optimization one due to the additional point constraints, we chose to let X, \ X,, = X2 for simplicity.
We also run the two criteria proposed as the benchmark, listed below.
Few-step Inverse Solver Training (Alg. I.. For each of the trial, we note the value of parameter perturbation of and training steps used.
Model Training Estimate (Alg. 2). For each trial, we note how many initial training steps are used, or if we just re-use the NN parameters from the forward PINN for the eNTK.
In Table 6 we show the hyperparameters for the criteria we use for main results.
In the results, we add "+ST" suffix to indicate the benchmark where the shared NN initialization is used for of the forward and inverse PINNs involved during ED and IP phases. MINE, VBOED, NMC and MI are optimized using Bayesian optimization (Frazier 2018).
H.5 IMPLEMENTATION AND HARDWARE
All of the code were implemented based on the JAX library (Bradbury et al. 20188, which allows for NN training and auto-differentiation of many mathematical modules within. Criteria which are
29

---

Published as a conference paper at ICLR 2025
optimized by Bayesian optimization done using BoTORCH (Balandat al. 2020). while criteria optimized using gradient- based methods are done so using JAXOPT Blondel et al. 2021) I).
The damped oscillator, wave equation, Eikonal equation and groundwater experiments on a machine with AMD EPYC 7713 64-Core Processor and NVIDIA A 1100-XXM4400GB GPU, while the remaining experiments were done AMD EPYC 7763 64-Core Processor CPU and NVIDIA L40 GPU.
conducted
I ADDITIONAL EXPERIMENTAL RESULTS
1.1 ADDITIONAL RESULTS FROM EXPERIMENTS ON LEARNED NN INITIALIZATION
In Fig. 9 we present examples of the learned NN initialization and also its performance for forward PINNs for the 2D Eikonal equation example. We see that this shows similar trends to the examples from before as shown in Fig. 2
In Fig. 10, we show the test error for an individual forward PINN when performing forward simulation for value of B, when using and not using learned NN initialization. We see that when using learned NN initialization, the test loss typically is already lower than that from random initialization the start, and also tends towards convergence much faster. Even when the test loss higher the start, it able to catch up to the performance of the randomly initialized PINN under much fewer training steps.
(a) Random Init.
(b) Learned Init.
(c) Train Loss
(d) Test MSE Error
1.20
1.05
10
4-
0.90
Test Error
0.75
Trin
2-
0.45
Rand. Learted
Rand. Leaitedd
0.15
10a
10"
10
10
10'
10
Training Steps
Training Sleps
Figure 9: Results for learning a NN initialization for PINNs trained on 2D Eikonal equation case. The interpretation is the same for that in Fig.
(a) Damped Oscillator
(b) 2D Eikonal Equation
Thaniing
Staring
Figure 10: Examples of test error forward PINNs for each problems for different values of 3. Each plot represents the PINN training for one random random value of 3. The dotted blue lines represent when the PINN is initialized with a random NN parameters, while the solid yellow lines represent when the PINN is initialized form the learned initialization.
1.2 TEST DISTRIBUTION MISMATCH
In Fig. II we present the results for the damped oscillator experiments for when the prior distribution during the ED process and for the tested IPs are different. In the distribution mismatch case, we use PDE parameter range μ, k E [0, 2] during the ED process, but use the values μl, k E [0, 4] for the true ground truth value in the IP process. From the results, able to retain good performances over the benchmarks even when the prior distribution the PDE parameters misspecified.
that our benchmarks are still
30

---

Published as a conference paper at ICLR 2025
(a) Distribution match
(b) Distribution mismatch
Expected Inv. Param. Error
Expected Inv. Param. Error
10
10
10-1
10-1
Random
Grid
MI
VBOED
FIST (ours)
MoTE (ours)
Random
Grid
MI
VBOED
FIST (ours)
MoTE (ours)
ED Method
ED Method
Figure 11: Results for the ID damped oscillator example for distribution match and mismatch, In both cases, the thick blue line represents the performance of Random baseline, while the dashed line represents the best performing benchmark.
J DISCUSSIONS ON GENERAL LIMITATIONS AND SOCIETAL IMPACTS
In our experiments, we have mostly conducted experiments based on vanilla PINN architectures. We have not done verification whether PIED works well for other more complex architectures such as physics-informn deep operators. Future work on this area would be interesting.
The work also relies on the use of PINNs as the forward simulators and IP solvers, and are not applicable to other forward simulators or IP solvers. While PINNs are well-suited for both tasks, they also still pose practical problems such as difficulty training for certain problems. The problem can be mitigated through more careful selection of collocation points (Wu et al., 2023 Lau al. 2024a). which will be interesting to consider in future works to further boost the performance of PIED.
We believe the work has minimal negative and significant positive societal impacts, since they can be used many science and engineering applications where costs of data collection from experiments be prohibitive. This means that the cost barrier in performing effective scientific experiments be lowered allowing for further scientific discoveries. We note that our work could potentially be applied for range of scientific research, which may include unethical harmful research done by malicious actors. However, this risk applies to all tools that accelerate scientific progress, and we believe that existing policies and measures guarding against these risks sufficient.
31


Reviews:
[{"review_id": "Ol4duy3HAl", "reviewer": "Reviewer_XCZE", "summary": "The paper proposes a novel algorithm for solving the inverse problem in PDEs - that is, estimating the parameters of a PDE governing the dynamic characteristics of a system.  In particular, the paper assumes that (a) obtaining (x,y) observations of the system is expensive, so we must select our observation points carefully; and (b) observations require an initial setup that we cannot (practically) repeat, so we must specify our test points up-front and not dynamically as in e.g. Bayesian Optimization.\n\nTo tackle this problem the paper suggests physics-informed experimental design (PIED) that uses two sets of PINNs to select appropriate test points for a given system of PDEs (with a-priori unknown parameters).  The general approach uses a set of PINNs operating as forward simulators to generate functions satisfying the PDEs, sampling these for a set of points X, then using PINNs as inverse solvers to estimate the PDE parameters from the observations.  The efficacy of the points X is measured as the difference between the \"real\" PDE parameters (used in the forward simulators) and the corresponding estimated estimated PDE parameters.", "questions": "See previous.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The algorithm is certainly interesting.  The PIED framework certainly looks practical.  The motivation behind and justification of each step is presented, and the experimental results look good.", "weaknesses": "One point that needs to be address in the paper is that of computational cost.  After all, one of the motivations of using PINNs in parallel is computational efficiency, so it would be good to have a comparison in terms of same.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LwTAzbhN92", "reviewer": "Reviewer_QnMu", "summary": "This paper suggests PIED, a method for optimal experimental design for PDE inverse problems via Physics-informed neural networks (PINN). The PIED framework consists of three steps: 1. A PINN to learn a forward simulator that maps parameters $\\beta$ to solutions of the PDE $u_\\beta$. 2. An observation selector that returns noisy predicted observations $\\tilde Y= u_\\beta(X) + \\epsilon$ at a fixed number of sensor placements $X$. 3. A PINN inverse solver that predicts the ground truth parameter $\\hat \\beta(X, \\tilde Y)$ from $X$ and $\\tilde Y$. 4. The optimal sensor placements $X$ are found by minimising the MSE between $\\beta$ and $\\hat \\beta(X, \\tilde Y)$ which requires back-propagation through the inverse problem solver. The paper claims the following contributions and innovations: \n- The weight initialisations of all PINN-based components are shared and from a pretrained model which stabilises and accelerates training.\n- The forward simulator for various choices of $\\beta$ are learned in parallel\n- The mean square error between $\\hat\\beta(X, \\tilde Y)$ and $\\beta$ is approximated by learning the inverse solver with a smaller number of training steps (FIST criterion for ED)\n- The mean square error between $\\hat\\beta(X, \\tilde Y)$ and $\\beta$ is approximated through a linearisation of the PINN training dynamics (MoTE criterion for ED)\n\nThe framework is tested on optimal experimental design for sensor placement on Eikonal, Wave, and Navier-Stokes equations as well as groundwater and cell growth dynamics, and benchmarked against Bayesian experimental design, random sensor placements, and sensor placements on a grid.", "questions": "- I would suggest to describe the PIED framework as described in algorithm 3 more clearly in the main text of the paper\n- After the experimental design is complete and sensor placements $X$ are obtained: Are the obtained sensor placements supposed to be universally informative and optimal for various downstream tasks of the practitioner, also those that involve classical simulators, or is the intention to always perform inference of the parameter $\\beta$ with the trained inverse PINN (which may be inaccurate compared to a slower classical solver). \n- Related to the previous question: Is it possible that the PINN memorises information about the PDE system at arbitrary evaluation points through the shared initialisation $\\theta_{SI}$, such that the sensor locations $X$ are not the most informative to make predictions about the system, but only the most informative when we additionally have access to a PINN with parameters $\\theta_{SI}$. In other words, could it be that the optimal experimental design is different when one wants to use classical simulators + MCMC to solve the inverse problem?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Overall, the paper is well written and seems methodologically sound.\n- The paper addresses a challenging experimental design problem with little existing approaches.\n- There is sufficient experimental results to support to proposed approach.\n- The paper combines an interesting set of ideas: \n\t- Meta learning and shared weights for more efficient PINN training. \n\t- Approximate training dynamics through of the inverse solver to make back-propagation through the inverse solver feasible.", "weaknesses": "- The approach is mostly motivated by the comparison with classical simulators, but other types of neural surrogate models for the forward simulator are not mentioned. This is probably because most such approaches like Fouier Neural Operators don't investigate the ED downstream task. Nevertheless, this made me wonder how much this approach actually relies on Physics-informed neural networks. For example, the processing in parallel threads would be typical for all approaches that target ED with a neural network.\n- The description of the algorithm/framework is a little inconsistent. The PIED framework in Figure 1 suggests that the PIED framework is trained end-to-end, while Algorithm 3 in the appendix clarifies that training proceeds in three phases that are not interleaved: Optimising for initial weights,  training to define ED criteria, optimisation of ED criterion. I wish that this was clearer from the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6lIHlWpEty", "reviewer": "Reviewer_X3tF", "summary": "The authors developed a physics-informed experimental design approach for the inverse problems to find the optimal and most informative design parameters. The paper is well-organized.", "questions": "1. In Fig. (c) and Fig. (d), the training loss and test error at final stage are similar for both methods. How can this be used to highlight the advantages of the proposed method.\n2. How to define the value of μ in Eqs. (17) and (18). It is a very important parameter to determine the difficulty for solving the NS equations.\n3. How to select the noise variance for different experiments. The noise variance may have a great influence for the prediction results. Can the method still be available when the noise variance is very large.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The theoretical analysis of PINN is thorough.", "weaknesses": "The effect of some experiments is not significant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EOprz7KcM6", "reviewer": "Reviewer_CY8h", "summary": "This paper solves the one-shot experiment design using PINNs in both forward and inverse problems. It overcomes computational bottlenecks by parallelism and meta-learning of initialization. The experiments on both synthetic and real-life datasets show the performance improvements.", "questions": "1. What's the relationship between the 'PINN for ED' and the 'adaptive-sampling PINN for inverse problem'?\n2. Can a trained PINN generalize to a new ED problem with new $\\beta$? Can the model (including the meta-learning part) generalize to a new distribution of $\\beta$？", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. this paper joins many advanced techniques together to solve the problem.\n2. most of the representation is clear, with extensive experiments and details.", "weaknesses": "1. The overall algorithm flow is unclear in the main text. Figure 1b is too general and missing almost all of the details of the technique.\n2. Does not compare with existing NN-based experiment design methods.\n3. Almost all techniques are adopted from existing literature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)