Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning
Authors: Mara Daniels
Liam Hodgkinson
Michael W. Mahoney
Venue: ICLR 2026 Poster (2026)
Keywords: physics informed
gaussian process
model selection
uncertainty quantification
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We derive a single objective criterion for physics-informed machine learning that can be optimised to ensure both a strong fit to data and adherence to a differential equation.

Abstract: Physics-informed machine learning (PIML) integrates prior physical information, often in the form of differential equation constraints, into the process of fitting ML models to physical data. Popular PIML approaches, including neural operators, physics-informed neural networks, and neural ordinary differential equations, are typically fit to objectives that simultaneously include both data and physical constraints. However, the multi-objective nature of this approach creates ambiguity in the measurement of model quality. This is related to a poor understanding of epistemic uncertainty, and it can lead to surprising failure modes, even when existing metrics suggest strong fits. Working within a Gaussian process regression framework, we introduce the Physics-Informed Log Evidence (PILE) score. Bypassing the ambiguities of test losses, the PILE score is a single, uncertainty-aware metric that provides a selection principle for hyperparameters of a physics-informed model. We show that PILE minimization yields excellent choices for a wide variety of model parameters, including kernel bandwidth, least squares regularization weights, and even kernel function selection. We also show that, prior to data acquisition, a special data-free case of the PILE score identifies a-priori kernel choices that are "well adapted" to a given PDE. Beyond the kernel setting, we anticipate that the PILE score can be extended to PIML at large, and we outline approaches to do so.

Published as a conference paper at ICLR 2026
UNCERTAINTY-AWAR DIAGNOSTICS FOR PHYSICS-INFORME MACHINE LEARNING
Mara Daniels
Liam Hodgkinson
Department of Mathematics
School of Mathematics and Statistics
Massachusetts Institute of Technology ma raraan@miit... edu
University of Melhourne
lhodgkinsore me lb. edu au
Michael W. Mahoney
ICSI, LBNL, Department of Statistics University of California at Berkeley nmahoney&stat. berke bekeley.... edu
ABSTRACT
Physics-informec machine learning (PIML) integrates prior physical information, often in the form of differential equation constraints, into the process of fitting ML models to physical data. Popular PIML approaches, including neural operators, physics-informed neural networks, and neural ordinary differential equations, are typically fit to objectives that simultaneously include both data and physical con- straints. However, the multi-objective nature of this approach creates ambiguity in the measurement of model quality. This is related poor understanding of epis- temic uncertainty, and it can lead to surprising failure modes, even when existing metrics suggest strong fits. Working within Gaussian process regression frame- work, we introduce the Mmysics-Informel Log Evidence (PILE) score. Bypassing the ambiguities of test losses, the PIL.E score a single, uncertainty-awar aware metric that provides selection principle for hyperparameters of physics-informed model. We show that PILE minimization yields excellent choices for a wide variety of model parameters, including kernel bandwidth, least squares regularization weights, and even kernel function selection. We also show that, prior to data acquisition, a special "data-free" case of the PILE score identifies a a-priori kernel choices that are "well adapted' to a given PDE. Beyond the kernel setting, we anticipate that the PILE score be extended to PIML large, and we outline approaches do
INTRODUCTION
A great challenge in machine learning (ML) in general and Scientific ML (SciML) in particular involves the development of models that combine in principled ways data-driven information (as common in ML) and domain-driven information (as is common in physical and other sciences). Strategies that attempt achieve this grouped under the umbrella of physics-informee machine learning (PIML). These approaches consist of general purpose tools for scientific computation that also enjoy the scalability and flexibility of high-dimensional ML. PIML methods include Physics- Informed Neural Networks (PINNs) (Raissi et al. 2019) (which led to large body of empirical
(Krishnapriyan al.
2021
Karniadakis al. 2021
Sirignano Spiliopoulos 2018 Sahli Costabal
et al. [000 et al. 2021
Geneva & Zabaras
2020
Xu & Darve|[0220) and theoretical (Minakowski
Richter 2023 Lu et al.
2021: Doumècce et al.) 2024b) analyses), Neural Ordinary Differential
Chen et 2018] Krishnapriyan et (003), Neural Operators (Kovachki ct al. 2023), and Neural Discrete Equilibrium (NeurDE) (Benitez et al. 2025). Unfortunately, PINNs and related methods are notoriously difficult to train (Krishnapriyan
Equations (Neural ODEs)
al. 202I); they lack robust a posteriori error estimates that are typically available for classical numerical partial differential equation (PDE) solvers; and they lack strong grounding in statistical theory. These issues exacerbated by the multi-objective nature of PINNs, as well as other PIML methods that incorporate domain knowledge a soft regularization.

---

Published as a conference paper at ICLR 2026
It is (easy and thus) popular to view physical constraints in terms of regularizers, in manner analogous to ridge regression (Karniadakis et al. (02I). However, practical implementations involve delicate trade-off between errors noisy observations and adherence the imposed (or assumed) physical equations. Indeed, one interpretation of the "failure modes" results of Krishnapriyann et al. (202T) is that while reducing physical error is of interest, this strategy becomes nuanced (and error- prone) when the model is unable to satisfy the physical constraints perfectly, or when the constraints are misspecified. In these cases, without sufficient validation data (which is common in scientific settings), it becomes challenging to determine whether model is suitable fit. These challenges are not limited to neural network models they are common many other methods that aim combine data-driven ML models with domain-driven physical models. It is critical to understand how to quantify the quality of PIML models, in manner analogous to how one quantifies model quality in statistical learning theory.
In this paper, we provide a first step toward solving this PIML model selection problem, addressing the problem under the Physics-Informed Kernel Learning (PIKL) framework (Pförtner et al. 2024). PIKL considers a Gaussian process (GP) model for solving linear PDEs under known conditions, and it offers powerful, uncertainty- aware approach to PIML. One advantage of the GP framework is that it offers a structured, probabilistic approach that allows for rigorous uncertainty quantification (UQ) (which is often missing in other physics-informed model<") Another advantage of GPs lies in their ability to seamlessly incorporate multiple forms of data acquisition, including noisy pointwise observations of the solution and derivative data derived from the governing PDE. This flexibility enables principled integration of prior knowledge about the system, while maintaining a Bayesian framework for uncertainty estimation.
Our main contributions are as follows:
(I) We introduce a model selection criterion called the Physics-Informee Log Evidence (PILE) (Section 3). The PILE criterion provides theoretically-groume of different kernel choices for PIML tasks; and it can be used, e.g., to optimize hyperparameters of the GP model, including the kernel function, its bandwidth, and regularization parameters.
way assess the suitability
(II) We provide an empirical evaluation demonstrating that the PILE criterion is reliable indicator of model performance (Section 5). Models optimized using PILE exhibit strong predictive accuracy and adherence physical constraints, By studying the challenging wave- equation setting introduced in Krishhapriyan et al. (202T), we show that the PILE score can not only diagnose model misspecification, but it can be used to identify the 'best' kernel function for the problem at hand, leading to vastly improved performance.
Overall, we claim that free energy metrics provide the solution to the multi-objective bottleneck in PIML. establishing a single number that can be optimised to ensure both a strong existing data and adherence to a governing differential equation. Diagnostics using free energy can be conducted both a priori using our Fredholm determinant metric, at the stage of architecture selection before fitting to data, and a posteriori using our PILE score, after the model has been fitted to data. Our case studies demonstrate how the use of these tools can bypass well-known pitfalls PIMI., highlighting scenarios where model choice will lead to an undesirable fit.
2
BACKGROUND AND RELATED WORK
2.1 LINEAR PARTIAL DIFFERENTIAL EQUATIONS
An s-th order linear differential operator D: C*)()) - (2) for integer 2 1 has the form
ga:
gaa
Df(x)= = Σ (x)
ox aa f(x),
dz
||
where Q is an open, bounded subset of Rd with C² boundary, where 2 is multi-index, and where {ca : ||a|]] S $} are (92) coefficient functions. Given g E Co (S2) and e (O(2), we say
Sooe exceptions exist, including Neural Processes Garnelo al. 2018 Kim et al. 2019 but they will prove to be disadvantageous in our framework due to intractable marginal like ihood..
2

---

Published as a conference paper at ICLR 2026
that f solves the Dirichlet boundary value problem (BVP) if Df(x) = g(x) for x E Q,
f(x) = h(z) for z E On.
(1)
Under assumptions, solves the Dirichlet BVP if and only if it minimizes the energy functional
E() = |D] - 00)22 PP//99l?>00)
+ |l. h|] (aa))
(2)
This variational problem a key ingredient in the formulation of equation T as a ML problem. Note, however, that we can extend our formulation far beyond the Dirichlet setting encompass wide range of mixed boundary conditions. To simplify matters, observe that the formulation is no less general if g 0, we can instead take the differential operator gld. Let be as before, but
now let B; C* ($2) + (T))),, i 1...p P be a family of operators, where each I; C a.. We can now consider the general mixed boundary condition
(f for Ω
(z) 0 for x Iú, i= 1...p
This formulation can encode the following boundary conditions:
Dirichlet: p = I 80, and f (f h)) an is the restriction of to M.
Neumann: 1, 11 i2, and f f (v- V h)laa, where is the unit normal to A..
Robin: -= 11 = 82, and B,f f (af + bv V h)lan for a, b E IR
Cauchy: = 2, 1 I2 = 02, B = (f - hr)lr, and B2f = (v Vf h2)lr..
Now any solution f is minimizer of the energy functional
E(f) = 1D]= (2 + Σ |Baflii(c
(3)
Letting Af(z) = (Df(x))a, (zf ) (xf(x))r,)) is Lebesgue on J2 and Hausdorff on each r so that E(f) = ||A ||AA This notation will become convenient later. While integral constraints also be incorporated naturally within this setup (Hansen et al. 2023), but here we restrict attention to differential operator constraints to avoid complicating our analysis.
RP+1, we can define a measure μ that
2.2 GAUSSIAN PROCESS REGRESSION
A Gaussian process (GP) f on Q C Rd, denoted f ~ GP k), is stochastic process where for x Q + [0, x)), any projection onto
some mean function m Ω + IR and a kernel function functiio
finitely many points X = {xi)]= C Ω is multivariate Gaussian:
f(z))) ~ (((m(z)))) (,,,,,
As covariance matrices necessarily symmetrie positive semi-definite, require that the Gram matrix (k(#,, x; )) ") is positive semi-definite matrix, for any {a: (Any function & with this property is said to be positive semi-definite.) We further require be continuous on Ω Q and to have Jn k(x, x dr < co.
GP regression is framework which provides Bayesian perspective kernel regression, along with a probabilistic interpretation of the commonly-used kernel ridge regularization. Given independent and identically distributed inputs I, [R and outputs y; for 1... n, a Gaussian likelihood
(f. 24) ~ (zx)), (n), p(u ff, & exp( (y: fzx)))²). i = 1=1,.. l,...,
is imposed, where 7 > 0 is hyperparameter representing the assumed noise level of the observations. can take y - 0+. For notational convenience, let X (2t,) (2.))
In the noise-free setting, Raxd and Y = (w)) E R To apply Bayes' theorem, the practitioner chooses a GP prior f~ GP kk),, for À > 0 regularization hyperparameter. In the absence of prior information. is common to choose m = 0. To perform inference, the prediction and uncertainty for the output of input x' is measured using the posterior predictive distribution (Rasmussen & Williams, 2006 Equation 2.)
[(x') (x1, (1,, .... (In, Bn) NNF('')
3

---

Published as a conference paper at ICLR 2026
(Kx + X 'gg for kg = (k(2), and Kx (k((a,, (a(m;,,)))) xy)) In addition to the point estimate f(x)), the posterior variance kig can be taken as a calibrated measure of predictive uncertainty.
where f(x) (Kx + Ay and a(z')
Kernel ridge regression (KRR) is prediction method (with limited UQ) that estimates the output f(x) for a given input x by minimizing regularized loss over a reproducing kernel Hilbert space (RKHS). Recall that every positive-definite kernel k induces RKHS, H, generated by the span of || (see Appendix A for details). KRR considers estimators of the form:
{k( +xeen with norm
" (f(())) - vi) + Y
f= arg min JEH
From Kanagawa et al. (2018 Theorem 3.4), turns out that f= f. the mean predictor from the GP formulation, and so GP regression extends KRR to include an estimate of the uncertainty. Simultaneously, any optimization problem of the form 4 has natural interpretation in terms of an underlying GP, providing uncertainty estimates for solutions to (4) We will make significant use of this relationship to construct our uncertainty-awe diagnostics.
2.3 UNCERTAINTY AND DIAGNOSTICS IN GP
One of the advantages with treating prediction task through the lens of statistical models is the capacity to analyse and estimate uncertainty. Bayesian methods, including GPs, naturally account for prediction uncertainty in the posterior predictive distribution, offering credible intervals for any quantile of uncertainty (Gelman et al.) 1995). For GPs, the uncertainty about the prediction contained in the posterior covariance kernel x + E(x,r). Estimates of uncertainty are only as effective as the underlying model. Fortunately, the treatment of uncertainty often unlocks a wide array of diagnostic techniques, providing valuable feedback to the practitioner. In Bayesian statistics, the fundamental indicator of the quality of particular prior is the marginal likelihood, also known as the evidence, given by
Zn E- [P(Y X))-
(5)
This quantity can be thought of the likelihood assigned by the prior to the observed data. It is typically convenient to work instead with the quantity
2 (Kx Fn = log 2, 2 TT(Kx +
+ +T)
2 log det det (K2 + +T) log
1
n2
(6)
2m
which is called the (negative) log-marginal likelihood, alternatively the Bayes free energy. In practice, it is common to perform model selection by maximizing the marginal likelihood (equivalently, minimizing F,,) with respect to hyperparameters of the prior. This is called an empirical Bayes procedure (Krivoruchko Gribov 2019), and it the main inspiration of the PILE score which we introduce in Section 3 Aside from model selection, the free energy is also effective for model tuning. This process is referred to as empirical Bayes (Efron) 2024); and, provided that not too many parameters tuned this way, it is often effective (Lotfi et al. 2002).. For GPs, bandwidth tuning in the kernel, and the selection of the noise level 7, are both often conducted by minimizing Bayes free energy (Rasmussen & Williams 2006 5.4.1);
also Gribov & Krivoruchko (2020). However, information criteria criteria--especialy especially Bayesian ones are not equivalent to test or cross- cross-validation error, which leads the question: when do they behave similarly? Fortunately, for GPs, it is known that the Bayes free energy behaves similarly to test error at least when the number of training and test points are large; see, for example. Hodgkinson et al. (202a)) Luxburg & Bousquet (2004); Jin et al. (022).
PHYSICS-INFORMER KERNEL LEARNING
The GP formulation of PIKL is based on finite sample approximation of equation 2 using (possibly
noisy) observations of the graph (ʃ, Af) at points x E Ω. Following Doumèche et al. (0024a), we consider minimizers of the physics-informet empirical risk over ff in RKHS H:
Σ (f(x.) - ? + 1 |AAAll 1=1
1 I
Ln(() = 7
(7)
data loss
physics loss
regularization
4

---

Published as a conference paper at ICLR 2026
The first term is the data loss, prescribing adherence of f to collected observations in the form of input-output pairs (2,, 14), with I; E f2 and yi E R. These can be used in addition to, or in place of boundary conditions. The second term is the physics loss, which enforces that f obey the prescribed equation Aʃ = 0. The third term biases the estimator towards a more regular solution, avoiding spikes and other singular behavior. For example, the RKHS H, associated with the Matern kernel of smoothness parameter
0 is equivalent to the Sobolev space W (S2) (Wendland 2004 Corollary 10.13), and so we have [ x ( the sum of the L norm of the first derivatives of f. The temperatures 7, P. > 0 are arbitrary and control the relative importance of these three terms. Since scaling L does not change its minimizer, is typical to fix one parameter and vary the other two. For quantifying the uncertainty in model predictions pointwise in space, however, selecting the correct scale of = is required to have accurate and calibrated estimates of the pointwise posterior variance, and thus all three parameters are needed.
(Q)
One approach to solving using KRR, taken in Doumèche et al. (2024a b). is to identify a new RKHS H' with norm I H I + (00,000000 Alternatively, our approach relies on the observation that for a GP ff ~ (P k) over Q supported on a Banach space 9 f Q + R, the pushforward of the process by bounded linear operator A B B' is itself GP, supported on B', with parameters
Aʃ ~ GP(Am, (A @ A)t).
This observation has been applied numerous times in the literature to enforce PDE or other linear constraints GP via conditioning on the value of Af (Härkönen et al. 2023] Lemma 2.1), (Pförtner ct al. 2024 Corollary 2). Macedo & Castro (2010), [Solin et al. (2018). While optimal in theory, the corresponding kernel of this space requires deep knowledge of the operator A and its eigenspectrum. Furthermore, samples drawn from a GP can be drastically less regular than the functions contained in the RKHS associated with its covariance kernel ?
To avoid this technical issue and to weaken our required assumptions on k, one can approximate equation by innsiie-informed version of KRR by estimating the L norm using quadrature rule {(w;, 2;) = I....., m) for on Ω. One option is a Monte Carlo rule that selects 2; uniformly at random over St and each I, with equal weighting w; -1, For improved precision, we opt for Gaussian quadrature rules. Our estimated loss function becomes
1 (((O)) yn
v:)² + (Af(z))) 1 " 111
I (f)
(8)
The advantage of this approach is that, provided we can formulate Aff can be solved using the representer theorem. Let be a fixed RKHS with reproducing kernel k. For multi-index a E Zq.
H as RKHS, 8
denote by kf x²) and 05 k(xx, x') the iterated partial derivative of the first argument and second arguments, respectively: 0g = 0m: ... deu ge: d k(x x'), #f
where 0 denotes the a,-th partial derivative in .. To proceed, we require assumptions on k and A.
Assumption 3.1 (Kernel Differentiabilityy. Assume that k has continuous s-th partial derivative, that is, for any multi-index a E 2 ||a]]] S 0g k(z, x²) ) C x ?)
Assumption 3.2 (Bounded Coefficients). Assume that the coefficients {ca : a Z } for D and each B, are all uniformly bounded max||al] Ss [| (| < C < co for some C' > 0.
By Proposition A.I ji Appendix under these assumptions, the RKHS associated with i is contained
in the image of A. Indeed, as in Proposition A.3 the graph Aff : f E H} itself an RKHS.
Using this fact, in Theorem [A.4 we obtain a representer theorem which can used to solve (
Theorem i. the basis for our variant of the PIKL. procedure. Given input-output pairs (1;; v))
and quadrature rule ((w), 2,)) }m a solution to (8), and therefore an approximate solution to
can be obtained by finding the coefficients (a, that minimize (13). This procedure general and
particularly effective when and are not too large (Doumèche et al. 2024a).
"This can be seen by the fact that if the Cameron-Martee space associated with GP k) is infinite dimensional, then it has zero Gaussian measure (Bogachev) 1998] Theorem 3.5.1). In other words, with probability one, GP(m, k) has I| oo whenever H not finite dimensional.
5

---

Published as a conference paper at ICLR 2026
4 PHYSICS-INFORMER LOG EVIDENCE (PILE)
Now that we have demonstrated how PIML can be posed in terms of a KRR problem, we consider the interpretation of this finding in terms of GP regression, which encodes a natural notion of uncertainty, As discussed in Section 2.2 the estimator f of Theorem A.4 coincides with the posterior predictive mean of GP model. It can be verified that this model is prescribed by
(9) where each r, is interpreted as an observation of a boundary condition, which can be taken to be zero to enforce the constraint Aʃ 0. Our formulation targets situations where practitioner may also have potentially noisy access to the boundary data and forcing function equation [ this case, one might take ry to be nonzero. Our goal is to remain as flexible possible with respect to modes of data acquisition (i.c., possibly corrupted boundary data, interior observations of f(x) or Df(x) at arbitrary x E ??) well with respect to the model used represent f. This is key benefit of the PIML approach.
(f.g) GP(O, nka),
Y/ f(,,) ~ (f)(.))) 12). ry (g2)) ~ N(g(z)), p
The uncertainty in the prediction fis encoded in the covariance of this GP: for x', E Ω
Cor(f(z',, = (ka 2'), (z',21) S Em.n
where
Kuz tm H
Haz Gez + npW
k(2', X) (Id @ A)k(X, 2T
(Id @ A)k(z', Z)] (A A)k(z', (0(2' .2]
Em,
Se'
One of our main contributions of this work is to propose the negative log- marginal likelihood of this GP as an intrinsic uncertainty-awaat measurement of model quality. We refer to this as the Physics-Informed Informed Log Evidence (PILE). Inspired by standard protocol for tuning GPs, the PILE score doubles as a model selection criterion for optimizing hyperparameters, including p. n, and any other hyperparameters for the kernel including bandwidth.
Definition 4.1 (Physics-Informedd Log Evidence (PILE)). The Physics-Informed Log Evidence (PILE) criterion is 2 F Fm, where Fm.n is the Bayes free energy of the GP equation 9
Pm, m + n
E Y
log Em,n. log(227).
m +
where y = (y)...... Mn T1 Tmm
The PILE criterion is to be interpreted as lower is better.
Unlike the empirical risk (8) which can be computed in quadratic time by the equation (13. computing the PILE score generally requires cubic time. Fortunately, for large m. n, several algorithms exist to compute the marginal likelihood quickly and under memory constraints Gardner et al. (0188) Ameli
al. (2255).
Connection to the Fredholm Determinant. Let now consider the scenario where data (zy, y4) is prescribed and the PIKL framework is applied to find a solution f E H to Af = 0. In this case, F3 0 can also be chosen and the PILE score simplifies to its last two terms. By taking m - DC, minimizers of (8) should become minimizers of our original problem by the following assumption.
This is imposed
Assumption 4.2. For any f (②)), as m oc, the quadrature rule converges, that is, En (z,) m= Sa (z))d((2)
Our next objective is to show that as m + co, the PILE score in this scenario converges in a suitable sense to a Fredholm determinant, which provides surprising quantifier of the effectiveness of particular choice of kernel k to solve given problem. The Fredholm determinant is fascinating object with a complex history; and it typically appears only in the study of random matrices and determinantal point processes (Derezinski & Mahoney] 2021). We refer to Appendix [B] for its definition (and for the proof of the following result) and to [Brnemaan (2010) for details of its computation in practice.

---

Published as a conference paper at ICLR 2026
Bandwidth (p-
Data (/ 0.01)
Physics mhhh"
Physics
(PPL2-0)
0.06
3.50
8
PILE ninimizer
9.05
3.25
0.05
3.25
0.06
PILE
9.04
3.00 &
0.04
3.00
0.04
0.43
2.75
0.03
2.75
0.42
2.50
0.02
2.50
0.11
RBF Bandwidth
Physiis reg.larizatinn
Data regularcation
Figure 1: Automatic hyperparameter selection with PILE. PILE score and relative PPL2-G error sources for varying bandwidth, physics regularization, and data regularization parameters. Error bars show 2: coverage, where the empirical standard deviation of PPL2-G (blue bars) and PILE (red bars). (Left) Bandwidth selection via minimizing the PILE score provides an accurate fit, balancing the data and physics generalization errors. (Middle, Right) After selecting the optimal bandwidth h", we sequentially minimize PILE first with respect to the physics regularization parameter then with respect the data regularization parameter y. For small values of p and y, PILE diverges the regression model overfits the noisy observations.
Theorem 4.3. Let 9 (2,, RP-1) - RPt1) be the integral operator
(z)) - / (A& 2)
1
ao AXT
Letting Cm = mmp - Em log w. + mmmmlolg22m (2go)(2r)), as m + oc. the sequence of normalized PILE scores converge to the Fredholm determinant: m C. m=230 Do = log det + g).
Note that one can also choose p may En log( mi) so that the normalized PILE score becomes equal to the PILE score. This choice of p provides an avenue to calibrate uncertainty. Theorem 4.3 provides a new interpretation of the Fredholm determinant of integro- differential operators of the form of G in terms of the base model difficulty of solving differential equations within the corresponding RKHS induced by k. In turn, model selection for a given problem prescribed by the operator A be achieved in data-indes data-independent fashion by comparing values of Pu- In Appendix [.22 provide a case study, in which we use the Fredholm determinant to select the best kernel from family of anisotropic RBF kernels. When used to solve ID convection PDE, we observe drastic improvement the capacity of the PIML. model to fit the target solution.
5
CASE STUDIES
In this section demonstrate how the PILE score can be used to select hyperparameters such as kernel bandwidth, loss regularization, and can even be used select the kernel function from parametrized family.
5.1 HYPERPARAMETER SELECTION USING THE PILE SCORE
Our first case study, shown in Figure examines the baseline efficacy of the PILE score, and the Bayes free energy at large, for solving the multi-objective ambiguity in the PIML problem. The point of the study is to demonstrate the practicality of our method by using it to automatically select all the relevant hyperparameters for the problem. As a simple test case, we focus on solving a Poisson equation with Dirichlet boundary conditions, the domain Ω
1)2
a - .(x) for
(wx 0 for I E 0n
(10)
and with forcing function ( 10 10sin(2mr) sin (2wy). A 2D (type-1) Chebyshev quadrature
scheme is used to determine the gridpoints 2(1,)) and corresponding weights w(iij) for j,j = -1.....7 at which to evaluate the derivative loss: letting 8; cos( 2mmg
2x+1,) we set 2(1.)) = (81, 8,) E R²

---

Published as a conference paper at ICLR 2026
True solution
Undersmoothed 0.15)
20
0
20
10
10
Optimal smoothing 0.35)
Oversmoothed 0.58)
20
20
10
10
Figure 2: Optimizing PILE prevents under- and over-smoothing. Qualitative plot of the negative effects of oversmoothing and undersmoothing in PIKL. Each panel shows f. 9 on the left and right, respectively. When is too small, the derivative estimate is undersmoothed and irregular. When is large, oversmoothing effects prevent the model from fitting the derivative.
and ((ij) = 4/m². The type-1 Chebyshev grid points are known to have excellent properties for numerical approximation integrals on 1): for example, f. E C (S2), then the integration error llg - D D (ي)))) converges as e (m)) (Trefethen 2019 Chapters 7, 8). This choice of quadrature scheme enables us to closely approximate the true L (222 norm of the PDE residual.
In the physics informed setting, there are two sources of error: data enror, measuring the fit of f to f; and physics error, measuring the fit of g = Df. Following Hodgkinson et al. (2023a), we define the (unnormalized) data PPL.2-G error as
Rdata ( Jn Fes))(((s)) f(=)) dz Enje w;E((((E(.)) f((u))))),
over f(z) ~ lui ((m marginalized over g. Analogously,
Ryhys Jn Eg(=) [(((2) D)())) da Exj=l w. w;B[([ (=((,)))
the (unnormalized) physics PPL2-G error, marginalized over The quadrature points =(1.)) depend > m) to ensure accurate L²((2) approximations. For appropriate comparison, we normalize errors: Raaaaa) Tf
on meval, chosen to be large (meval
Ran(f) and R R
TDJIl
Both should be small, but since they cannot generally vanish simultaneously, the optimal point on the Pareto front of these losses (the best solution to the PDE) is unclear.
typical challenge in RKHS and GP regression selecting the bandwidth of shift invariant kernel. A shift invariant kernel with bandwidth h > 0 has the form (,, , 3) = =(==" These kernels enjoy special analytical properties and fast randomized approximations (Rahimi & Recht 2007). making them popular in practice. It is important to tune the kernel bandwidth for optimal performance: if too large, the kernel becomes oversmoothed and the resulting regression estimator has high bias; whereas if it is too small, the resulting regression estimator may overfit observations or take near-zero values on unseen data. In this experiment, we train the regression model using grid size n = m 13, so that there are total of 132 noisy observations B(i.)) f(((),)) + E(i.)) 2(E.) Df(=(.)) + ((.))) for independent ((.))) (1.)) (O, 1), with i,j 1... Despite the high noise level and the relatively small number of samples, the regularized regression estimator matches the target function when fit with the optimal bandwidth according to PILE score; the PILE score has successfully overcome the multi- objectvv problem. When used to select p, 7 > 0, divergence of the PILE score is accurate indicator of model overfitting.
5.2 DIAGNOSING AND AVOIDING MODEL FAILURE WITH THE DATA-FREE PILE SCORE
In this case study, we analyze the well-known wave equation baseline introduced by Krishnapriyan (021), which was shown to break vanilla multilayer perceptron methods. We observe that for an
8

---

Published as a conference paper at ICLR 2026
isotropic RBF kernel, there is bandwidth that simultaneously achieves good physics and data fits, and PILE diagnoses this by selecting an oversmoothed 'all zero' solution. Perhaps more surprisingly, when consider broader class of "anisotropic RBF" kernels, find that optimizing the data-free tata-free PILE score (Figure(4) with respect to kernel function yields a model with excellent physics and data fit. The PIL.E-optima kernel can be identified automatically, prior data acquisition, and with domain knowledge. Following (Krishnapriyan et al. 2021] Section 3.1), consider the convection PDE of the form
{
#1 (t,xx) + ( 0, t [0,1], [0, 00
8e [((0,)) sin(
(11)
We set = 1000, m = 20 and assume access to observations of the form yi =
fx; + Gi+
i i= 1. n1 for æ; Unif (0, 1] x [0. 2T]). We observe empirically that fitting an RBF kernel as in Section leads to pathological behavior, summarized in Figure 3 As shown in Figure (left), the
data loss is only small at bandwidths h R3 0.1, while physics loss blows up at bandwidths h =s 0.15.
Physics
Data
1.0
1.0
1.0
0.0
2.5
5.0
50
-1.
155
-20
95-
-14
-14
800
Bondwiith
Figure 3: PILE diagnoses model failure. Fitting the convection PDE equation 11 with an RBF kernel. There no appropriate bandwidth for this problem and the PILE score diagnosis this by selecting the 'all zeros' solution.
Instead, we could consider an anisotropic family of kernels defined by hyperparameters [-x,, T]. > 0, and given by
(x, - { (a- (e. Σg.+
cos(()) (in( cos(
sin s s
cos sin(O)
sin sin
-
cos(O)
To find an appropriate kernel among this family, we select 0, to minimize the data free PILE score, whose loss landscape is shown in Figure After kernel selection, the loss basins of the physics and data loss become drastically better conditioned, leading to an excellent model shown in Figure
R² 0288
0 = 1.41, S = 0.50
Data test
0.84
lons 10
E100
mommteer
0.86
0.0
Foa
0.3
0.88
2os
204
1.5
0.90
08:5
2.5
E 1.5]
0.92
Bundweoth
Figure Hyperparameter selection after kernel adjustment. (Left, Middle) Fitting the convection PDE, equation [T] with an anisotropic RBF kernel. By choosing the kernel with minimum Fredholm determinant, we can automatically identify a "good" kernel for the continuity PDE. (Right) Fredholm determinants of ko, plotted for 0 E [-иr, æ] and s E [0.5, 1.5). This quantity is empirically minimized 0* 1.41, 8* ss 0.5 (shown in red).

---

Published as a conference paper at ICLR 2026
CONCLUSIONS
We have introduced the PILE score, a model selection metric and diagnostic tool for physics-informed kernel learning to avoid failure modes. Our method amounts hyperparameter selection via Empirical Bayes, thereby removing ambiguities involved in parameter selection under two competing loss terms. While already practical, it useful recognize how PILE can extend other contexts.
Nonlinear Operators. We studied linear differential operators with kernel-based models. but PIKL also extends nonlinear PDEs (Chen et al. 2021). following the framework of Dashti et al. (2013). The nonlinear operator A is linearised via its Fréchet derivative DA at the solution f: and the PIKL setup and PILE score is applied using D.A in place of A. The justification for this PILE score follows the arguments of [Wacker 2017).
Neural Networks. The PIKL framework is distinct from both PINNs and Neural ODEs (since neural network regression differs from KRR). Approximations of the free energy are typically derived using Laplace approximations, such although extensions exist when these approximations fail (Drton & Plummer 2017). Of particular relevance is the Interpolating Information Criterion (IIC) (Hodgkinson al. 2023b), designed for models that interpolate data, akin to GP-based PIML. Inspired by IIC, one could evaluate the PILE score for trained PINNs using the empirical neural tangent kernel (Novak al., 2022 Jacot et al.) 2018) in place of k. At present, this is expensive, although approximations such as those seen in 2025 suggests this might yet become tractable.
the derivation of the BIC (Schwarz) 1978).
Ameli et
ACKNOWLEDGEMENNT
MWM would like acknowledge the DOD DARPA AIQ program, the DOE LBNL LDRD program, and the NSF for partial support of this work. This material is based upon work supported by the U.S. Department Energy, Office of Science, Office of Advanced Scientific Computing Research, Department of Energy Computational Science Graduate Fellowship under Award Number(s) DE- SC0023112.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Siavash Ameli, Chris van der Heide, Liam Hodgkinson, Fred Roosta, and Michael W Mahoney. Determinant estimation under memory constraints and neural scaling laws. arXiv preprint S.X1::2503.04424,, 2025.
J. A. L. Benitez. J. Guo, K. Hegazy, I. Dokmanic, M. W. Mahoney, and M. V. de Hoop. Neural equilibria for long-term prediction of nonlinear conservation laws. Technical Report Preprint: arXiv:2501.0699 .06933, 2025.
V.I. Bogachev. Gaussian Measures. Mathematical surveys and monographs. American Mathematical Society, 1998. ISBN 9780821810545. URL http:httts//// htttpps:/bbokss wowoooooooooooool.oooooooooooooe
google le. www/books
6tmA
Folkmar Bornemann. On the numerical evaluation of Fredholm determinants. Mathematics of Computation, 79(270):871-915, 2010.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Yifan Chen, Bamdad Hosseini, Houman Owhadi, and Andrew M Stuart. Solving and learning nonlinear PDEs with Gaussian processes. Journal of Computational Physics, 447:1 10668, 2021.
Masoumeh Dashti, Kody JH Law, Andrew M Stuart, and Jochen Voss. MAP estimators and their consistency in Bayesian nonparametric inverse problems. Inverse Problems, 29(9):095017, 2013.
M. Derezinski and M. W. Mahoney. Determinantal point processes in randomized numerical linear algebra. Notices of the AMS, 68(1):34 45, 2021.
Nathan Doumèche, Francis Bach, Gérard Biau, and Claire Boyer. Physics-informed kernel learning. arXiv preprint arXiv:2409. 13766, 2024a.
Nathan Doumèche, Francis Bach, Gérard Biau, and Claire Boyer. Physics-informed machine learning as kernel method. In Proceedings of Thirty Seventh Conference on Learning Theory, volume 247
of Proceedings of Machine Learning Research, pp. 1399-1450. PMLR, 2024b.
Mathias Drton and Martyn Plummer. A Bayesian information criterion for singular models. Journal of the Royal Statistical Society Series B: Statistical Methodology. 79(2):323-388 2017.
Bradley Efron. Empirical Bayes: Concepts and methods. In Handbook of Bayesian, Fiducial, and Frequentist Inference, pp. 8-34, Chapman and Hal//CRC, 2024.
Jacob Gardner, Geoff Pleiss, Kilian Q Weinberger, David Bindel, and Andrew G Wilson. GPyTorch: Blackbox matrix-matro Gaussian process inference with GPU acceleration. Advances in neural information processing systems, 31, 2018.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint 107.v:180..01622. 2018.
Andrew Gelman, John Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis. Chapman and Hal//CRC, 1995.
Nicholas Geneva and Nicholas Zabaras. Modeling the dynamics of PDE systems with physics- constrained deep auto-regressive networks. Journal of Computational Physics, 103:109056, 2020.
Alexander Gribov and Konstantin Krivoruchko. Empirical Bayesian kriging implementation and usage. Science of the Total Environment, 722:137290, 2020.
Derek Hansen, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, and Michael W Mahoney. Learning physical models that can respect conservation laws. In International Conference on Machine Learning, pp. 12469-12510 12510. PMLR, 2023.
Mare Härkönen, Markus Hangge--egrmmmn,, and Bogdan Raită. Gaussian process priors for systems of linear partial differential equations with constant coefficients. In Proceedings of the 40th International Conference Machine Learning, ICML'23. JMI..org, 2023.

---

Published as a conference paper at ICLR 2026
Liam Hodgkinson, Chris Van Der Heide, Fred Roosta, and Michael W. Mahoney. Monotonicity and double descent in uncertainty estimation with Gaussian processes. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 13085-13117. PMLR, 2023a.
Liam Hodgkinson, Chris van der Heide, Robert Salomone, Fred Roosta, and Michael W Mahoney. The interpolating information criterion for overparameterized models. arXiv preprint arXiv:2307.07785 2023b.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
Hui Jin, Pradeep Kr Banerjee, and Guido Montufar. Learning Curves for Gaussian Process Regression with Power-Law Priors and Targets. In International Conference Learning Representations, 2022.
Xiaowei Jin, Shengze Cai, Hui Li, and George Em Karniadakis. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. Journal of Computational Physics, 426: 109951, 2021.
Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, and Bharath K Sriperumbudur. Gaus- sian processes and kernel methods: review on connections and equivalences. arXiv preprint arXivv 2018.
George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6): 440, 2021.
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garelo,, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. Attentive neural processes. arXiv preprint
2019.
Konrad Knopp. Theorie und Anwendung der unendlichen Reihen, volume 2. Springer-Verlag, 2013.
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Arirzadenesheli, Kaushik Bhattacharya, Andrew Stwart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89):1-97, 2023.
Aditi S. Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in -nnfores-nformeed neural networks. Advances in Neural Information Processing Systems, 34, 2021.
Aditi S Krishnapriyan, Alejandro F Queiruga, N Benjamin Erichson, and Michael W Mahoney. Learning continuous models for continuous physics. Communications Physics, 6(1):319, 2023.
Konstantin Krivoruchko and Alexander Gribov. Evaluation of empirical Bayesian kriging. Spatial Statistics, 32:100368, 2019. ISSN 2211-6753
Sanae Lotfi, Pavel Izmailov, Gregory Benton, Micah Goldblum, and Andrew Gordon Wilson. Bayesian model selection, the marginal likelihood, and generalization. International Con- ference Machine Learning, pp. 14223-1422 14247. PMLR, 2022.
Yulong Lu, Jianfeng Lu, and Min Wang. A priori generalization analysis of the deep ritz method for solving high dimensional elliptic partial differential equations. In Proceedings of Thirty Fourth Conference on Learning Theory, volume 134 of Proceedings of Machine Learning Research, pp. 3196-3241, PMLR, 2021.
Ulrike von Luxburg and Olivier Bousquet. Distance-based classification with Lipschitz functions. Journal of Machine Learning Research, 5(Ju)):69-66) 665, 2004.
I. Macêdo and R. Castro. Learning Divergence-free and Curl-free Vector Fields with Matrix-valued Kernels. IMPA, 2010. URL //tttpss://ook..... google com/ http//ooks/// XxxxxXXEAAAXXX
P. Minakowski and T. Richter. A priori and posteriori error estimates for the Deep Ritz method applied to the Laplace and Stokes problem. Journal of Computational and Applied Mathematics, 421:114845, 2023. ISSN 0377-0427.
12

---

Published as a conference paper at ICLR 2026
Roman Novak, Jascha Sohl-Dickstein, and Samuel S Schoenholz. Fast finite width neural tangent kernel. In Interaational Conference on Machine Learning, pp. 17018-17044, PMLR, 2022.
Marvin Pförtner, Ingo Steinwart, Philipp Hennig, and Jonathan Wenger. Physics-informed Gaussian process regression generalizes linear PDE solvers, 2024. URL httppppt///ax....oog////////
2212. 12474
Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems, volume 20, 2007.
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informet neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.
Francisco Sahli Costabal, Yiho Yang, Paris Perdikaris, Daniel E. Hurtado, and Ellen Kuhl. Physics- informed neural networks for cardiac activation mapping. Frontiers in Physics, 8:42, 2020.
Gideon Schwarz. Estimating the dimension of model. The annals of statistics, pp. 461-4644 1978.
Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339-1366 2018.
Arno Solin, Manon Kok, Niklas Wahlstrom, Thomas B. Schon, and Simo Sarkka. Modeling and interpolation of the ambient magnetic field by Gaussian processes. Trans. Rob, 14(4):1112-1127, August 2018. ISSN 1552-3098. doi: 10. 109/TTO..018..883000
Steinwart and A. Christmann. Support Vector Machines. Information Science and Statistics. Springer New York, 2008. ISBN 9780387772424, URL https: http://oookk.oogge...//.. com/
books?id=Hunme 2ookk:7.d+UUnggrpXY4110
Lloyd N. Trefethen. Approximation Theory and Approximation Practice, Extended Edition. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2019.
Philipp Wacker. Laplace's method in Bayesian inverse problems. arXiv preprint arXiv: (oKiv:7701.000 1701.07989, 2017.
Holger Wendland. Scattered Data Approximation. Cambridge Monographs Applied and Compu- tational Mathematics. Cambridge University Press, 2004.
Kailai Xu and Eric Darve. Physics constrained learning for data-driven inverse modeling from sparse observations. arXiv preprint urXiv:200..10511, 2020.
A REPRODUCING KERNEL HILBERT SPACES
We provide brief summary of reproducing kernel Hilbert spaces; for more details, refer the interested reader to Steinwart & Christmann (2008). Let H be a Hilbert space of functions f X' - IR. with inner product (:: ). и- H is called reproducing kernel Hilbert space if for every f E H. the evaluation functional
(2), is bounded, i.c., [=(() S M, If for some M, +oo and all H. By the Riesz representation theorem, this implies that for any E X, there exists k2 H such that f(x) = (f. kz)H= These elements k₂ are called the feature maps. The reproducing kernel of the Hilbert space is given by
: H + IR defined for E X' by laf
k(x, v) (krz ky)
The Moore-Aronszaip theorem (Steinwart & Christmann 2008 Theorem 4.21) implies that every positive-defie definite kernel induces a unique reproducing kernel Hilbert space, so the kernel completely defines H. One also deduce the class of functions contained in the RKHS from the kernel. For example, if the kernel differentiable, this implies that elements of are also differentiable, as in the following proposition.
13

---

Published as a conference paper at ICLR 2026
Proposition A.1 (Corollary 4.36 of Steinwart & Christmann 2008 ). Let f H. For I [s] the derivative 8 f(x) E (12) exists and admits the bound:
10"h f(x)) S Il
Hence Df E (S)) each f E H.
Furthermore, the image of H under a linear operator is typically also an RKHS. The following proposition shows that the image of under a linear differential operator is an RKHS with new kernel defined in terms of the operator.
Proposition A.2. The space
{g E :B: E with g(x) D](x)} endowed with the norm
l = inf llf f:Df=a
the unique RKHS associated with the kernel
g( v) = (D @ D)k(Z,z)).
Moreover, g(x,z') = (gx, I where the feature map IN H is given by gx(x') = (D x (d)
Proof. It is sufficient to consider the case Df(x) = ca (x)0* f(x) for some a E E. as the general result follows from summing over terms of f.. For any a, it holds y + 02 v) E by repeated
application of Lemma 4.34 of Steinwarr & Christmann (2008). Hence,
(I ld)(z, )le (c)( [1 ld)k| <c,
and
= (0 @ @ld)k(2,)))
= (z)2)) k2) DD((x),
by the chain rule.
Proposition A.3. The graph ((SAf) H) RKHS HA with norm
l[ffAf))la llfll
and a multi-valued reproducing kernel li given by
k(I,x') ka((1, 2). (x', = (ld A)k(2,
(Id 8 A)k(2',z) (A & A)k(2,
Proposition A.2 is sufficient to prove Proposition A.3 To see this, note that the space HA
{(f.Af) + H fe H} can be equipped with the inner product acting only on the first coordinate: ((f.Af). Ag)) = (f. g) H. The evaluation functionals are guaranteed to be bounded by Proposition A.2 and the form of the kernel follows.
Here, we let W diag(w;) that 1/2 can be interpreted as the elementwise square root. Theorem A.4 (Representer Theorem). The minimizer f arg ing Lm (f) satisfies
fE span [k((, xx)) U {(ld @ A)
(12)
for f = E ask(,,x,) + ΣØ= (1 & A)k(, 24), the coefficients (a, B) E gmt minimize
I Krz a Has 31 + 1 1 (HJ a Gsz
[a]
Kg
H
(13)
2n
B
H
G.s
which is now equal to umm where [Kezlis = k(0,, Munlis (Id @ D)k(a,, 2j), and
[Gzzl = (D @ D)k(zzzzz)
14

---

Published as a conference paper at ICLR 2026
Proof of Theorem A.4 Property equation 12 follows from the fact that the || projection of any E H onto the span does not affect the values of f(x;) nor Df(zz)). By the representer property for H,
(Kx, 92, H (kx, (-), (ld D)k(- (Id 2),
and by Proposition A.2
92,, 9g,, = (D @ (2 24). Plugging these identities into [| yields
K
Has
a
3
H
Gz=
Eqn equation 13 Eqn. equation
12 rewritten terms of a.
B FREDHOLM DETERMINANTS
Let H be a separable Hilbert space with an inner product (5-) and let KCHH) denote the space of compact linear operators
+ H. The spectrum
spec(A) = {} ker(Al - C) + {0}},
of any A E K(Hf) is countable and can accumulate only at zero. Hence, there is a (possibly infinite) sequence of eigenvalues (A))) For any A KC(H). let
oi (A) 2 (A) ... ... 0,
denote the ordered singular values of A, defined as the square root of the eigenvalues of A Definition B.I. A compact operator K(H)) is trace class if Σn. on (A) < +00.
By Holder's inequality, for any trace class operator A, the Schatten norm satisfying
|| I Σ (A)",
P21
is finite. Using the definition to verify whether operator is trace class is a near-impossible task in general. Fortunately, the following theorem provides conditions that guarantce in our kernel setting that all operators are trace class.
Theorem B.2. Let k be a symmetric cestiivedeeiniie kernel on Ω Ω such that x + k(x, x) e L2((2) for Q Rd. The integral operator K (22 (22 defined by
(z) - kl2, (y)
trace class.
We ready define the Fredholm determinant of operator. Definition B.3. The Fredholm determinant of trace class operator A is given for p > 0 by det(T + pA)
II + (A)).
It is straightforward to verify that the Fredholm determinant can only be defined for a trace class operator, since (Knopp 2013 p. 232)
+ pl] S det(( + pA) S exp(p)|A[]i).
Note that for H = Rmm the Fredholm determinant reduces to the standard determinant, as a bounded
linear operator R and
+ R can be represented as matrix Am over the basis elements (es (m
det(T + pA) = II + ps.. (A)) = det( + pAm)
==1
In the case of integral operators, the Fredholm determinant is expressed in terms of the infinite series (Bornemann) 2010 eqn. (3.7))
p"
det(1 + pK) = 1
det(k(zi, 2;) da dæn.
n!!
12
n=
15

---

Published as a conference paper at ICLR 2026
Proof of Theorem 4.3 Starting from the PILE score,
m m.n log det(G + ) + m log()
logdel( W/ - -nol) + m log(2 Σ
log wi
(,, + C..
and consequently,
mm Cm log det(( + ((A @ A)k(a, w/1
The result now follows from (Bornemann 2010 Theorem 6.1).
16


Reviews:
[{"review_id": "f11LOMJt8g", "reviewer": "Reviewer_Jkyr", "summary": "This paper proposes PILE (Physics-Informed Log Evidence), a principled and uncertainty-aware model selection metric for Physics-Informed Machine Learning. Working within a Gaussian Process-based Physics-Informed Kernel Learning framework, the authors show that PILE resolves the core multi-objective tension between data-fit and physics-fit by providing a single score that quantifies model quality, enabling reliable tuning of kernel hyperparameters, regularization weights, and noise levels. They further show that in the absence of data, PILE converges to a Fredholm determinant, which can be used to select kernels a priori that are well-adapted to a given PDE. Through case studies, including a challenging convection PDE known to cause PIML failures, the paper demonstrates that PILE not only identifies when a model or kernel is misspecified, but also guides the choice of kernels and parameters that yield high-quality solutions, thereby offering a robust diagnostic and model-selection tool for PIML.", "questions": "1. In the data-free PILE formulation, the theory requires $m\\rightarrow\\infty$ for convergence to the Fredholm determinant. In practice, how do the authors recommend choosing $m$ for a given PDE and kernel? Are there heuristics for determining when $m$ is \"large enough\" for the PILE score to be reliable?\n\n2. Have the authors observed how sensitive the PILE score is to the number and distribution of collocation/quad points? For example, do the kernel rankings stabilize beyond a certain $m$? Any such empirical guidance would be useful for practitioners.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper offers a novel contribution to physics-informed machine learning by introducing PILE, a principled and uncertainty-aware model selection criterion based on Bayesian marginal likelihood. Unlike existing PIML approaches that rely on heuristic loss weighting or manual hyperparameter tuning, this work reframes the problem through Bayesian evidence maximization, providing a single score that jointly captures data-fit, physics-fit, model complexity, and uncertainty calibration. A particularly original aspect is the theoretical result showing that, in the absence of data, the PILE score converges to a Fredholm determinant of a PDE-informed integral operator. This reveals a connection between PIML, Gaussian processes, and spectral operator theory, opening a new line of investigation.\n\nThe paper is technically strong, with a rigorous derivation of PILE from the Bayes free energy of the physics-informed GP model. The theory is well supported by clear assumptions and proofs, and the empirical results, though modest in scale, effectively illustrate the practical value of the method. In particular, the convection PDE case study demonstrates that PILE can identify kernel choices that significantly improve solution quality, underscoring the method’s practical utility.\n\nThe presentation is generally clear and well organized. The progression from standard GP regression to physics-informed kernel learning and finally to the PILE formulation is logical and accessible for readers with a background in GPs or kernel methods. While some of the operator-theoretic aspects may remain abstract for a broader audience, the core ideas are well communicated and supported by helpful examples.\n\n\nThis work is significant because it directly tackles a key limitation of current PIML methods: the lack of a principled mechanism for model and hyperparameter selection. By enabling both data-informed and data-free kernel selection, the proposed framework has the potential to meaningfully reduce reliance on trial-and-error and improve robustness in scientific ML workflows. The core ideas are sufficiently general that they could influence future developments beyond GP-based methods, including uncertainty-aware neural operators and physics-based deep learning. Overall, the paper makes a meaningful, well-founded, and potentially impactful contribution to the field.", "weaknesses": "While the paper is strong overall, a few areas could be improved to enhance its practical impact. The work is primarily method-driven and introduces a novel and well-motivated framework, and I did not identify major weaknesses in the core methodology or theoretical development. My comments are therefore more about opportunities to further strengthen the empirical validation. The experiments are limited to relatively low-dimensional PDEs with simple settings e.g., the main results focus on a 1D Poisson equation and a 1D convection problem and demonstrating the method on more challenging benchmarks (e.g., 2D/3D heat equation, 2D convection-diffusion equation etc.) would better showcase its broader applicability. In addition, the connection to Fredholm determinants is theoretically elegant but only demonstrated through a single kernel-selection case study. Further analysis of stability and generality across PDE types and kernels would help strengthen confidence in its practical utility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IHSP7UeR9S", "reviewer": "Reviewer_hf97", "summary": "The paper addresses the problem of measuring the model quality in the case of physics-informed machine learning (PIML). Since PIML companies data and physical constraints (like differential equations) to train models, this leads to multi-objective approach which is hard to evaluate. Current methods can produce models that look good by standard metrics but still fail in surprising ways due to poor understanding of epistemic uncertainty. The authors work within a Gaussian process regression framework and introduce the Physics-Informed Log Evidence (PILE) score. The PILE score provides a single, uncertainty-aware metric that bypasses the ambiguities of traditional test losses. The paper addresses a critical gap in PIML methodology.", "questions": "- How would PILE score behave when a model completely violates a hard constrain and when it has an ok residual error in a soft constraint? Is is possible to distiguish between hard and soft constraints?  \n\n- How does the computational cost of evaluating the PILE score scale as constraint complexity increases? What are computational limitations? Or in other words, up to what dimensionality is it reasonable to use it (given it's higher computational complexity and cubic complexity of GPs)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The multi-objective evaluation issue is an important problem in PIML that practitioners struggle with. Having a principled diagnostic is valuable.\n- A single metric for hyperparameter selection is very usable (as opposed to juggling multiple competing objectives such as data loss vs. physics loss vs. test error).", "weaknesses": "- The current set-up is limited to Gaussian processes (not major, but it would obviously great to have something model agnostic). \n- This also limits the problems in which PILE would be useful to rather smaller scale problems, as GPs don't scale well. The PILE score itself introduces additional complexity. Hence, it is not fully clear what is practical limitations in terms of the computational cost and up to what degree it would be feasible to use it in practice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OL91MvA0JV", "reviewer": "Reviewer_Pt1Q", "summary": "This paper presents a metric, named PILE, to evaluation principle for hyperparameters for physics informed models. The metric is for kernel learning only.", "questions": "1. I have concerns over the phrase \"multi-objective nature of the approach\" of balancing both the data loss and physical constraints. Although it is true that the two components may draw the overall loss to different ends of the loss function spectrum, the ultimate goal is to train a model most generalizable. In my view, the multi-objectiveness is only the superficial aspect of the optimization. \n2. It appears that the metric is only applicable to physical systems modeled by PDEs, hence the background information in section 2.1. Would the method applicable to other systems, such as differential-algebraic equations? \n3. Issue with Fig 1, although the general trend of three component is very clear, the magnitude as shown in the y-axis is very small. Would the numerical noise affect the solution?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper addresses an important topic for incorporating the knowledge into data-driven learning. \n2. Thorough theoretical treatment of the problem.", "weaknesses": "1. Applicable to kernel learning only. It's still an open problem whether it can be extended to other ML techniques especially neural networks. Although kernel learning is highly capable, there are still a large selection of NN-based ML methods which would greatly benefit from physics knowledge.\n2. The organization and exposition of the paper can be further improved. It is good to be mathematically rigorous, the exposition can be improved to give the readers more intuition, instead of piles of math equations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CmtCX36sYC", "reviewer": "Reviewer_hS2h", "summary": "This paper introduces the Physics-Informed Log Evidence (PILE), a novel, uncertainty-aware model selection criterion for Physics-Informed Machine Learning (PIML). The work is situated within the Physics-Informed Kernel Learning (PIKL) framework, which uses Gaussian Processes (GPs) to solve linear PDEs. The key insight is to use the Bayesian marginal likelihood (or Bayes free energy) to navigate the multi-objective trade-off between data fidelity and physics constraints—a common pain point in PIML. The authors demonstrate that minimizing the PILE score effectively selects hyperparameters (e.g., kernel bandwidth, regularization weights) and even the kernel function itself. A particularly innovative contribution is the \"data-free\" PILE, which connects to Fredholm determinants and allows for a priori kernel selection suited to a given PDE, before any data is observed. Empirical results on Poisson and convection equations show that PILE reliably diagnoses and prevents common PIML failure modes.", "questions": "How does the computational cost of PILE scale with both the dimension of the PDE domain 𝑑 and the number of data/quadrature points What are the practical limits for its application in higher-dimensional or large-scale problems?\n\nThe extension to neural networks via the NTK is mentioned. Could you elaborate on the practical challenges of computing or approximating the PILE score in this setting, given the known limitations of the NTK approximation in finite-width networks?\n\nHow sensitive is the PILE score to the choice and quality of the quadrature scheme used to approximate the physics loss? Would different quadrature methods or levels of accuracy substantially change the diagnostic outcome?\n\nIn the data-free kernel selection, the Fredholm determinant is used. Are there intuitive interpretations of its value that could help a practitioner understand why one kernel is “better adapted” to a PDE than another?\n\nIs code available to reproduce the experiments and facilitate the adoption of the PILE score by the community?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This paper makes a clear and original contribution by introducing the Physics-Informed Log Evidence (PILE) score as an uncertainty-aware diagnostic and model selection principle for physics-informed machine learning. The work is significant in that it addresses the long-standing ambiguity in balancing data loss and physics loss in PIML, by providing a single, principled metric rooted in Gaussian process regression. A notable strength is the dual perspective: the authors present both a priori diagnostics (via a Fredholm determinant in the data-free case) and a posteriori diagnostics (via PILE after training).\n\nThe quality of the work is high. The theoretical development is rigorous, linking PIML to marginal likelihood in GPs, and the empirical case studies (Poisson, wave, and convection PDEs) clearly show how PILE can guide hyperparameter selection, prevent over/under-smoothing, and diagnose failure modes. The exposition is well-structured and accessible, with illustrative figures that support the main claims. Overall, the paper advances originality by integrating uncertainty quantification into model evaluation, and its significance lies in offering a practical tool for robust diagnostics in PIML.", "weaknesses": "The primary limitation is that the methodology is restricted to kernel-based Gaussian process models. While the authors outline possible extensions to neural network–based PIML methods (e.g., PINNs, neural operators), these are not demonstrated. Empirical validation on nonlinear PDEs or higher-dimensional benchmarks would substantially strengthen the generality claims.\n\nA second limitation is computational scalability. The PILE score requires cubic time in the number of data and quadrature points, which the authors acknowledge. They cite approximate methods for marginal likelihood computation, but no experiments demonstrate that these approximations keep PILE practical at scale. More discussion or demonstrations here would increase the impact.\n\nFinally, while the connection to the Fredholm determinant is mathematically elegant and provides a novel perspective on data-free kernel selection, its practical utility is demonstrated only in a single anisotropic kernel case study. Further evidence that this insight generalizes would help substantiate its significance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)