Understanding Generalization in Physics Informed Models through Affine Variety Dimensions
Authors: Takeshi Koshizuka
Issei Sato
Venue: NeurIPS 2025 poster ()
Keywords: Physics-Informed Models
Learning Theory
Generalization bound
Affine Variety
Primary area: theory

Abstract: Physics-informed machine learning is gaining significant traction for enhancing statistical performance and sample efficiency through the integration of physical knowledge. However, current theoretical analyses often presume complete prior knowledge in non-hybrid settings, overlooking the crucial integration of observational data, and are frequently limited to linear systems, unlike the prevalent nonlinear nature of many real-world applications.
To address these limitations, we introduce a discrete weak form that unifies collocation and variational methods, enabling the incorporation of incomplete and complex physical constraints in hybrid learning settings.
Within this formulation, we establish that the generalization performance of physics-informed regression in such hybrid settings is governed by the dimension of the affine variety associated with the physical constraint, rather than by the number of parameters. This enables a unified analysis that is applicable to both linear and nonlinear equations. We also present a method to approximate this dimension and provide experimental validation of our theoretical findings.

Understanding Generalization in Physics Informed Models through Affine Variety Dimensions
Takeshi Koshizuka' and Issei Sato
Department of Computer Science, The University of Tokyo booshizuka--akkeshi1884444 sato) @gg.cc.u-tokyo.ac.j.
Abstract
Physics-informed machine learning is gaining significant traction for enhancing statistical performance and sample efficiency through the integration of physical knowledge. However, current theoretical analyses often presume complete prior knowledge in non-hybrid settings, overlooking the crucial integration of obser- vational data, and frequently limited linear systems, unlike the prevalent nonlinear nature of many real-world applications. To address these limitations, we introduce unified residual form that unifies collocation and variational methods, enabling the incorporation of incomplete and complex physical constraints in hy- brid learning settings. Within this formulation, we establish that the generalization performance of physics-informed regression in such hybrid settings is governed by the dimension of the affine variety associated with the physical constraint, rather than by the number of parameters. This enables unified analysis that is applicable both linear and nonlinear equations. We also present a method approximate this dimension and provide experimental validation of our theoretical findings.
1 Introduction
In recent years, physics-informed machine learning (PIML) has garnered significant attention [35 25
[9]. PIML represents hybrid approach that integrates physical knowledge into machine learning models for tasks involving physical phenomena. These hybrid models can leverage inherent physical structures, such as differential equations (36), conservation laws (23 and symmetries (3), inductive biases. This integration has the potential to enhance both sample efficiency and generalization capabilities. These models have been empirically applied wide range of phenomena. with successful applications observed in areas including thrombus material properties [41], fluid dynamics
8 24
turbulence and heat transfer problems
Despite these empirical successes, the theoretical analysis of PIML remains underdeveloped, which potentially undermines the reliability of these methods. Notably, in practical scenarios, prior knowl- edge of the governing differential equations, particularly their source terms or boundary conditions, often incomplete. Consequently, learning frequently involves hybrid approach of fitting actual observational data alongside incorporating physical constraints. However, much of the existing theoretical research focuses on settings where complete prior knowledge of the differential equations is assumed. Furthermore, the scope of existing analyses is often limited to linear diffferential equations or systems exhibiting strong regularity, creating gap between theory and application.
To bridge this gap, we propose versatile analytical framework for -onssics-nformee regression using linear hypothesis classes over nonlinear features in hybrid settings. Our key idea is to formulate the differential equation constraints by introducing a Unified Residual Form. This form, defined on a finite set of trial functions and measure, provides practical approximation of physical constraints. This formulation unifies the collocation-basel constraints, used in Physics-Infon
Neural Networks
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(PINNs) [36], and the standard unified residual form constraints, used in the variational and finite element methods. Within this framework, the learning weights of physics-informed linear regressor are shown to be defined on an affine variety associated with the differential equation. Crucially, our analysis aims to elucidate the impact of incorporating physical prior knowledge on the generalization capacity of these models. We then establish that the generalization capacity of these models is determined by the dimension of this affine variety, rather than by the number of parameters. This novel perspective enables a unified analysis applicable to various equations, including nonlinear ones. To support our theoretical findings, we introduce a method for approximately calculating the dimension of the affine variety and provide extensive experimental validation. Our results illustrate that even in scenarios with a large number of parameters relative to the amount of data, the incorporation of physical structure reduces the intrinsic dimension of the hypothesis space, thereby mitigating overfitting and corroborating our theoretical claims.
2 Related Work
Since the seminal work by Raissi et al. 36 on PINNs, PIML has rapidly emerged as significant field of study. This area has been comprehensively surveyed in the literature by [3
12 19 Leveraging the high function approximation capabilities of neural networks 20 26 14 these models have been
25
employed as versatile surrogates for solving various equations. In contrast, linear models are also used
because of their interpretability. consistency with classical numerical solvers [8] and the close
relationship between Partial Differential Equations (PDEs) and kernel methods [37 27 16]
Recently, methods that exploit underlying conservation laws [23 21 and symmetries 3 13 in
addition to the equations themselves, have also been developed.
Recent studies have made advances in the theoretical understanding of PINNs. Shin 38 rigorously showed that the minimizer of the PINN loss converges to the strong solution the data size approaches infinity for linear elliptic and parabolic PDEs under certain conditions. These findings were extended by Shin al. [39] into general framework applicable to broader linear problems, with the loss function formulated in both strong and variational forms. Mishra and Molinaro [30 31] use the stability properties of the underlying PDEs derive upper bounds on the generalization error of PINNs. Subsequent research has applied this analytical framework to various specific equations 6 29 However, studies explicitly addressing the impact of physical structure on generalization capabilities are still limited. Arnone et al. S proved that for second-order elliptic PDEs, the physics- informed linear estimator using a finite element basis converges at a rate surpassing the Sobolcy minimax rate. Doumèche et al. [15] quantified the generalization capacity of the physics-informed estimator for general linear PDEs using the concept of effective dimension [10], a well-known metric in the analysis of the kernel method. The effects of incorporating the structures of nonlinear complex equations, as well as conservation laws and symmetries, into models on generalization, have yet to be thoroughly analyzed.
Minimax risk Analysis
In this section, we explain how introducing physical structures can improve the generalization capacity of linear models. In Section 3.1, we outline the problem setup. In Section 3.2, we perform a minimax risk analysis, showing that the generalization capacity mainly determined by the dimension of the affine variety. In Section 3.3 we show that our theory aligns with existing theories on linear operators. Notations are summarized Appendix A
3.1 Problem Setup
We consider the regression problem, which aims learn the unknown function f": m + IR that satisfies the differential equation. We have a dataset consisting of n observations, denoted as {< y4 ) where T; Ω U 8Ω represents the input within the domain S C 1R the boundary in and y/ R represents the corresponding output. Observations are sampled independently from a probability distribution P on the domain 52 U aQ x R. The relationship between the observations and the true function can be expressed as:
= fz + Ei & O00,

---

where E, represents normally distributed noise with mean zero and variance o². The target function f
the solution of the differential equation, i.e., | for given operator 9: -22 - L2
where T,2 (S2) denotes the space of square-integrable functions on domain f2 C IR... For a more detailed background on the problem setting, please refer to Appendix
To formulate prior knowledge of the governing differential equations. we first introduce a unified residual form, which captures physical constraints an integrated or averaged sense. Such formulations naturally arise in variational and finite element methods, and are particularly well-suited to hybrid settings where only partial physical supervision available. Formally, let T = {(th. pa } be finite collection of trial functions and measure pairs, where each th I2m - R is a smooth trial function and μ4 M + IR is measure the o-algebra 2 over the domain f2. Then, the unified residual form of the knwon differential equation O is defined by
Unified Residual Form:
(ff1. = J. ((f((z) (x(x) (x) = O, k k= K.
We then impose the differential equation constraint through the unified residual form defined above. The resulting ihysics-informed regression problem reads:
fn = arg min TEF(3TT) 1m1
1
Σ
wu f(x:)) + Am]1511
(I)
F(2,7)= = [s: ( vk) = 0, v(vk; m4)) E ET)
where A is a regularization parameter and || || denotes the standard 1.2 norm. This formulation relaxes the classical smoothness requirements while still leveraging mhysics-informed constraints via a unified measure-based approach: choosing Borel measures leads to an approximation of standard weak solutions, whereas choosing Dirac measures leads to an approximation of the strong- form residuals used in the PINN framework.
Physics-Informed Linear Regression (PILR) Setup: Let B = {0r R. + RJ be a fixed basis. Define the basis vector (xx) (é (æ), $2(x), d=(=)] E Rd, the design matrix + (Ø(*1), ((2)), ... ( Raxd and the target vector y Unl R
The physics- physics-informed informed feasible set is
V(9,377 = {w Rd: (9 w Φ) thx)ms 0, ((1E,11) E TT, E EB.
(2)
The problem Eq. ① reduces to the physics-informet linear regression given by
i = arg min 1yy tw P
WEVn
where Vn = V(2,
the affine variety constrained by the fz-ball B2 (R) with radius R > 0 and || ll is the f2-norm.
The set of coefficients constitutes an affine variety as represents the set of solutions to the K polynomial equations in the d variables with real coefficients. For example, when m 1 and 2lf f d. f. the affine variety V is defined by the solution set of the polynomial equations (w) Ei (( ((d (5) dyy vhk) wy 0 for i 1....,K K. We perform minimax risk analysis based on the dimension dy of this affine variety because the affine variety V is crucial to determine the size of the intrinsic hypothesis space.
Minimax risk: The goal of our analysis is obtain the upper bound of the minimax risk for PILR in Eq. 3.. which is defined by
min max || - w wI EVA
Here, w" E VR represents the optimal weight vector. The corresponding optimal hypothesis fw- w" & within our hypothesis space H (w & : E VRJ defined as the best approximation of the true function f": f.. w= & arg min f. sH II5 - 11.
We strongly recommend referring the example Section 5.1 to intuitively understand our problem setting.
3

---

3.2 Main Theorem
In this section, we present an upper bound on the minimax risk for PILR. The bound interpretable and sufficiently sharp, revealing how physical constraints reduce hypothesis complexity and enhance generalization. We begin by stating the definition and assumptions underpinning our analysis.
Definition 3.1 ((3, d set). An affine variety C Rd is called a (3, dry--reullr set if the following conditions hold: (1) For almost all affine subspaces L S _R of dimension dg satisfying d - dz. S dy. the intersection V n L has at most 3 path-connected components. (2) For almost all affine subspaces L C IRd of dimension dp with dp > dy, the intersection V nn L is empty. See Appendix B.2 for illustrative explanations.
Assumption 3.2 (Boundedness of basis functions). For the basis function Φb = [oi..... Qa]T, where 6 E B, assume that there exists positive constant M such that (|(((x))) S M for all f
Assumption 3.3. Assume there exists a constant > 0 such that @ 1 2 / for all
w< E (2R).
Assumption 3.4 (Stability of estimator). Assume there exists a constant r > 1 such that || w22l22 < (T - 1)]] w 22, for the estimators w) and UD of the optimal weights wi and wi. respectively.
Next, we present the upper bound on the minimax risk. The complete proof is provided in Appendix D
Theorem 3.5 (Minimax Risk Bound). Let ((2
B, T) be the (3, dyy -regular affine variety defined in Eq. Q Suppose Assumptions 3.2 hold. Then, there exists posilive constant C, independent of n. dy. d, and B, such that for any E (0, 1), with probability at least 8, the minimax risk for
PILR defined by Eq. (4) is bounded by
(V dy log(ddd)
log 23 n
log (2/()
min max 1100 w0112 Cn TOMRR 1 w EVE
2
(5)
11
n1
Proof Sketch. The proof proceeds in two steps. In the first step, we upper bound the minimax risk by the supremum of sub-Gaussian random process defined over the metric space (VR. || .12).. The second step utilizes Dudley's s integral theorem, which bounds the supremum of the process by an integral involving its covering number, specifically: f VV(RR, E. l122) de. To apply Dudley's theorem effectively, we employ Lemma B.2 to obtain an explicit upper bound for the covering number. Substituting this bound into Dudley's integral and performing the integration yields the desired high-probabillty minimax risk bound.
Theorem 3.5 demonstrates that the minimax risk is primarily governed by the intrinsic dimension dy of the affine variety V, rather than the ambient input dimension d, particularly when the topological complexity parameter is small. For comparison, standard least-squares estimation over an b2lball (R) C Id yields minimax risk rate of order ((Vd/n), which is optimal for unconstrained linear regression in d-dimensional space. In contrast, our result shows that when dy < d, incorporating physical structure into the hypothesis space through differential constraints significantly sharpens the risk rate, yielding improved generalization.
On the Role of 3. The parameter 3 captures the topological complexity of the affine variety and appears as regularity constant in the generalization bound. Its upper bound can be estimated via the Petrovskii Petrovski--Oleinik- Oleinik- Milnorr inequality 33 32 which provides a bound on the sum of Betti numbers of a semialgebraic set. Specifically, if the variety V v (2 C R is defined by polynomial constraints {p& ) of maximal degree p. then is (p(2p- - 1) d dy -rrgglarr. This implies that as the degree p of the defining polynomials increases, the variety can exhibit more intricate topological features, such as additional holes and disconnected components.
How dy and |3 Arise from the Covering Argument. The minimax risk is bounded via Dudley's entropy integral, which requires control over the covering number (V(R,, e. l12).. Following the geometric approach of Zhang and Kileel 42), the affine variety V c Rd is sliced using family of

---

linear subspaces {L. LEN, and each intersection V Lg is covered by Euclidean balls of radius E. The total covering is then given by
V VcU U (22
VEVOL,
In this construction, the intrinsic dimension dy controls the number of subspaces required to suf- ficiently cover V. while the parameter B. corresponding to the sum of Betti numbers, governs the covering number of each individual section V n Ly Topologically, 3 can be interpreted as quan- tifying the number of topological features (e.g.. holes) in V, and thus reflects the local geometric complexity encountered within each subspace. For reference, the standard covering number of the Euclidean ball satisfies (By R. || (12) I (1 + 2R/e)", highlighting the advantage replacing ambient-dimension dependence with complexity parameters intrinsic the constraint set.
Key Insights. central contribution of our analysis its interpretability through the lens of intrinsic complexity measures. The dimension dy plays role analogous to the VC dimension in classification [I] or the pseudo-dimension in regression [34], serving as a proxy for the effective capacity of the hypothesis space. This dimensional viewpoint clarifies how the incorporation of phys- ical constraints-vi differential equation structure-can substantially reduce hypothesis complexity, even in high-dimenn digensiieniiona ambient spaces. While this may come the cost of slightly looser constants compared to hinimax-optimee bounds, the resulting rate is still sharp enough to meaningfully capture the generalization benefit of innfoce- inductive bias. Empirical evidence supporting this theoretical advantage is presented in Section and an alternative analysis via Rademacher complexity provided in Appendix FF
Effect of the Trial Function Set T. The set of trial functions T encodes the imposed physical constraints, typically derived from governing differential operator 2. The cardinality K = [7] quantifies the amount physical knowledge embedded in the learning problem. Increasing the number of trial functions leads to more restrictive constraint set, which geometrically corresponds to lower-dimensional affine variety. Specifically, if T1 C Tz. then it follows that
( V,2,B,T)
dy(9)88Ta) dy(9),BTT.)
which highlights how adding more physical constraints systematically reduces hypothesis complexity and improves generalization behavior.
3.3 Analysis on Linear Operator
We discuss the special case where 2 linear operator. The second term in Eq. 5 vanishes because the
inequality indicates 3 = 1. Thus, the minimax risk is O dd log(dyd)/n). Furthermore, the affine variety Vss the solution set of homogeneous system of linear equations. That is, the affine variety can be written as ((2, B) = {w Dw = 0} using the matrix D I2 Kxd defined by Dr., (I[o,, vx)pp. The affine variety is linear subspace of dimension dy dim ker D. From the rank-nullity theorem, dy d rank D, indicating that the higher the rank of the matrix D, the better the minimax risk of regression.
We show that our theory is consistent with existing theories. The effect of incorporating physical structure, represented by linear differential equations, on generalization has been analyzed within the framework of kernel methods by Doumèche et al. 15 16). They argued that the physical structure smooths the kernel and reduces the effective dimension, leading to an improvement in the &g predictive error. We first present the definition of the physics-informed
informed (PI) kernel.
Definition 3.6 (PI kernel [15] 16]). Given basis B = { trial functions (with single measure) T (Vv,, n)) k and a linear operator 2, the PI kernel associated with the affine variety "12,B77 T) = w E Rdd Dw = 0) is defined as:
RM (M v) ( (M- 1/2 112 M 1/20bb()))
(6)
with M(C,v)=(I+*))
TE = (W2, we) - [ bk)
Here, is the identity matrix, and the matrix T positive semi-definite, The parameters E, control the balance between the L'Ereuularization and the constraints derived from the operator G.
5

---

Doumèche et al. [16] showed the effective dimension deu (5, of the PI kernel is evaluated above by a computable quantity as follows:
(5)
Σ
(7)
(Eo(BM-IB) I+a-11
where a-)) denotes the spectrum (set of eigenvalues) of the given matrix. B Rdxd is the Gram matrix of the basis function, i.e., Bj,y' = (Ø), 6 Next, we provide an explicit upper bound on the effective dimension of the PI kernel defined using the affine variety:
Proposition 3.7. The effective dimension of the PI kernel associated with the affine variety V( = {w Dw = 0} with dimension dy is upper bounded by
dy den - 1 1+
1 2 vaj 1 S
d E
d
1++++ 1
j=1
j=dv
where {az) denote the positive eigenvalues of the matrix D TTD.
Proposition 3.7 indicates that as the dimension of the affine variety dy = d rank decreases, the upper bound the effective dimension the PI kernel decreases accordingly. Since the matrix D TD is positive semi-definite, all eigenvalues satisfy aj 2 0. The intrinsic dimension dy corresponds precisely the number of zero eigenvalues (a: 0). The terms in the second sum G > dy.) involve strictly positive eigenvalues aj O. Given that > O, have I+t+va; T+E Thus, when the intrinsic dimension dy decreases, the number of terms the first sum (with the larger value 1/(1- {)) decreases, while the number of terms in the second sum (with smaller values + vox)) increases. This shift towards smaller-valued terms leads to an overall reduction in the complexity bound.
The PI kernel framework from the previous literature quantifies the complexity of the hypothesis space through the entire spectrum of the matrix D combined with the base kernel (Φ(x)) (by)),, restricting the analysis primarily to linear target operators S. In contrast, our approach allows for analysis of linear and non-linear operators by focusing solely on the intrinsic dimension dy (the count of zero eigenvalues), rather than analyzing the entire eigenvalue spectrum.
Consequently, our theoretical results align with the existing PI kernel theory 15 16
4 On the Dimension of an Affine Variety
In general, the dimension the affine variety = (w R² (w) 0, Ak= = K} defined by polynomials {p*] has many equivalent definitions. In particular, the following statements are all equivalent.
Definition 4.1. The maximal length of the chains Vo C .... Vdy of non-empty subvaricties of V.
Definition 4.2. The degree of the denominator of the Hilbert series of the affine variety V.
Definition 4.3. The maximal dimension of the tangent vector spaces the non-singular points U USV C Rd of the variety, i.e., dy = max rank [Vp((w) (w)
(px(w)]
Although Definition 4.1 clearly indicates that the dimension represents the complexity of the set V. is difficult to calculate the dimension according to this definition. Definition 4.2 shows that the dimension represents the algebraic complexity of the polynomial ring. Definition 4.3 characterizes the dimension based on the local structure of the affine variety, making it suitable for numerical calculation as discussed in Section 4.2 It generalizes the rank-nullity theorem dy d rank D in the linear case, as mentioned Section 3.3 The details of the concepts associated with these
definitions are given in Appendix
4.1 Lower Bound
We demonstrate that the dimension dy of the affine variety can be characterized by the linear part of the operator Z.

---

Proposition 4,4. Suppose the operator 2 can be decomposed as Φ = L + g where L is nonzero linear differential operator and F is nonlinear operator. Then, we have dy(z) I dy(
Combining the result of Proposition 4.4 with Theorem 3.5 suggests that the nonlinear part & of the operator increases the affine variety dimension, which has negative effect on generalization. Furthermore, the dimension of the affine variety associated with the linear part I can be easily calculated by the rank of the matrix. Therefore, the lower bound of the dimension of the affine variety associated with the nonlinear operator Z can be easily determined, allowing us to estimate the minimum required amount of data
4.2 Numerical Calculation Method
the dimension of an affine variety is typically obtained by calculating the degree of the denominator of the Hilbert series, by using Gröbner bases. However, the worst-case time complexity of Buchberger's algorithm 77,, the standard method for computing Gröbner bases, is double exponential in the number of variables d. Therefore,
According to Definition 4.2
the basis of 4.3 we approximate dy w from the affine variety with a suitable distribution and then computing
by sampling wi.
px(=")] When the operator Z nonlinear, we perform simulations with various boundary conditions and project the obtained solutions onto the basis B to sample w" E V. For linear operators. the dimension does not depend on the particular weight w, and the rank of the matrix discussed in Section 3.3 precisely determines dy. Assuming the use of standard rank computation algorithms, the computational complexity of this numerical approach is O(N min K, d) Kd) for the nonlinear case, and O (mmin(H) K, d) Kd) for the linear case. This complexity is practical and feasible for most scenarios considered in our setting.
max Ef w..,w,
rank (V [pi((w)
5 Experiments
To evaluate the generalization performance of thyyics--nformee linear regression (PILR) compared to ridge regression (RR) using basis functions B, we conducted experiments on representative differential equations. We varied the data size n and parameter count and report test MSE (mean + standard deviation) across 10 random initial or boundary conditions. Experimental details are provided in Appendix C
When the operator 2 is linear, PILR approximates the solution to Eq. as 16
+ + + nM) Φ y.
where M depends on hyperparameters & and (see Eq. (6); setting = 0 yields RR.
For nonlinear equations, we train models by minimizing soft-constrained loss using the Adam optimizer. Hyperparameters & and y are tuned via validation MSE.
5.1 Learning Strong Solutions
In this section, we investigate the strong solutions of the classical harmonic oscillator and the diffusion equation with periodic boundary conditions, by employing the Dirac measure, which corresponds to the collocation method used in PINNs. The solutions to these equations can be obtained analytically. Through these straightforward examples, we demonstrate both analytically and numerically that the generalization performance is determined by the dimension of the affine variety.
Harmonic Oscillator
The initial value problem of harmonic oscillator [ = with a spring constant k2 and mass m. in the domain Ω [0, is given by:
d
h y, y(0) = yo. (0) to.
df y
(8)
m,
where yo and up are the initial position and velocity, respectively. The solution to the initial value problem is analytically given by y(t) Yo cos(wi) + sin(wt), w /hs/m,.. The settings for
the basis and the trial functions with the measure Φj E B, (w', m4) E of indices 1 S S de and 1 < k < Ke are as follows:
&1 (x) = 1, (x)) = cos (w)zz) Φ2j+1 (x) sin (wjr). (x) = 1, p4 8207
(9)
7

---

(1 200
PILR
(n-R((-40)))
(R
(2)(--40)
RR (n 23)
RR 40)
KR (n 60)
MSB on the test data
100
MSE datn
10-4
10-1
900
10"
10
Train
60
80
number of parameters
number of purumelens
(a) Harmonic Oscillator
(b) Diffusion Eq.
(c) Predictions Harmonic Oscillator
Figure 1: Experimental results for the strong solutions. (a, b) Test MSE (log scale) vs. number of
parameters for the harmonic oscillator (a) and diffusion equation (b). The plots compare RR and PILR for three different data sizes n, showing the mean and standard deviation across 10 initializations. (c) Predictions of harmonic oscillator using a 33- 33-parameter model trained on 20 samples: RR and PILR, with training data points indicated.
where W/ = jit is the j-th frequency and 6z, is the Dirac measure centered at the point $2 T which is uniformly sampled from data.
Then, the dimension of the affine variety is dy = 2, representing the essential degrees of freedom of the solution. Figure Ta supports our theory experimentally. For RR, the generalization performance degrades as the number of parameters d = 2d, + increases due to overfitting, as shown in Fig. Tc In contrast, for PILR, the performance remains stable regardless of the number of parameters d owing to the lower dimension of the affine variety dy = 2.
Diffusion Equation The initial value problem for the one-dimensional diffusion equation Ulu] with diffusion coefficient e and periodic boundary conditions is given by:
0
0²u = 0, c Oz
(x, t) E |-E,E x [0.T] E |_
u(x 0) = un ux()) ((-E,t) t), du (-E,t) oz (E,t) E [0, T]
(10)
We define the basis functions ei E B and the test functions with measures (v. 11) E T as follows:
025.5" cos(waate -w3it $2j+1.5" = sin(w,r)e -wwft uh. = 1, Pk )( (1I)
where the frequency is w jr/E. The indices in the ranges 0 1 S. k < Kr, and < k Kz.
d 0 < j' dr.
The analytical solution is expressed
linear combination of the above basis functions. The number of bases is d = 2d + 1, while the dimension of an affine variety is given by dy = 2 min(ds, dy di Figure lb shows the results when set 1.0, Jmax 1, de 2, and vary dy The results indicate that the generalization performance of PILR does not deteriorate as dz increases, in contrast to RR.
5.2 Learning Weak Solutions
In this section, we investigate weak solutions for the harmonic oscillator and the diffusion equation, employing variational framework with Borel measures. The governing equations and basis functions are identical to those in Section 5.1
Harmonic Oscillator
We define the trial functions t for 1 I h: < K,
1b (x)
( cos(wwz), t/2 (x) sin (wa.)))
(12)
where W/ := As is the frequency. The associated measure A4 is the Lebesgue measure on = [0, T]. 7
The dimension of the affine variety remains dy = 2, consistent with the strong solutions. Experimen- tal results in Fig. caacorim this. The performance of PILR stable and independent of the number of basis functions, unlike RR, which shows performance degradation model complexity increases.

---

(2-30)
(4-28)
X00
15
50
00
300
(a) Harmonic Oscillator
(b) Diffusion Eq.
Figure 2: Experimental results for the weak solutions. (a. b) Test MSE (log scale) number of parameters for the harmonic oscillator (a) and diffusion equation (b). The plots compare RR and PILR for three different data sizes showing the mean and standard deviation across 10 initializations.
Diffusion Equation The trial functions thx, combine piecewise constant basis in time and Fourier basis in space. For indices 1 S k S Kg and S < Kg, they are:
dkk.2k' 1 (z, t) [fte (t) (asg 1/x,2k (z, Its. (1)))) sin(war:x),
(13)
where I (1) is the indicator function for the time interval, defined
{
if E tx+1) otherwise
1 Le (t)
t
(14)
and wyy = Kx The associated measure the Lebesgue measure on [- x [0,
The dimension of the affine variety, dy 2 (dn, de) 1, is identical to the strong solution case. The results Fig. 2b show that PILR's s generalization performance remains robust as the number of spatial basis functions d, increases, demonstrating its advantage over RR.
5.3 Learning Numerical Solutions
In this section, learn approximate solutions using numerical methods that finite difference for four equations. In this setting, we consider the affine variety of the difference equation 2n and the base functions Bn and the trial functions with the measure Th corresponding the numerical method with step size h. We first validate our theory using linear and nonlinear Bernoulli equations discretized by the explicit Euler method.
Discrete Bernoulli Equation uniform step size h:
We discretize the Bernoulii equation on the interval Ω = T] with
Mx M-+1
Pyr - Q = O, T T 0......
n1
T
T
where y, = yt,,). We consider two parameter regimes (P.Q. set to (1.0, 0.0) for the lincar case and to (1.0, 2) for the non-linear case. The initial value yo is sampled from (/00, 1). and the reference solution is calculated explicitly by Euler. Further details on the choice of n4, basis/trial functions, measure (1,,,,,). 1,), and implementation are given in Appendix G.2
Discrete Diffusion Equation We discretize the one -inednmnnsional diffusion equation over [-E x [0,T] with step sizes h = (hz, h1) and diffusion coefficient c(u):
2n [1] 4
un
3/+1 2 2uj + u
cuu))
0, j= j. Rez -....., ne:
h,
h
where 2J ((xx,,t)) We consider two cases: c(u) 1.0 for the linear case and c(u) 0.1/ (1+²)) for the nonlinear case. Periodic boundary conditions are imposed in æ. More details on the grid, the basis / trial functions, and the numerical setup given in Appendix G.2
Tables D and 2 show that PILR achieves higher performance than RR for large values of d. While the dimension dy is independent of the time discretization step size in the Fuler method, it depends on the spatial discretization step size in the FDM. We include supplementary experiments Appendix where we fix the ambient dimension d and vary the size of the trial-function set T.

---

Table I: Experimental results for the discrete linear and nonlinear Bernoulli equations approximated by the explicit Euler method. The settings include various step sizes h. The number of parameters (basis) d, and the calculated dimension of the affine variety dy.
Dn
Linear Bernoulli eq.
Nonlinear Bernoulli eq.
Settings
h
1 1/100
1/200
1/100
1/200
d 100
200
100
200
Dimensions
dy
RR
0.48 = 0.32
0.63
0.43
0.60 - 0.41
0.72 - 0.49 0.013 = 0.0018
Test MSE
PILR
0.012 = 0.0025 0.011 + 0.0013
0.013 + 0.0024
Table 2:
Experimental results for the discrete linear and nonlinear diffusion equations approximated by the FDM. The settings include various step sizes h (he,hz)) The number of parameters (basis) d, and the calculated dimension of the affine variety dy.
DR (hr,hz)
Linear diffusion eq.
Nonlinear diffusion eq.
Settings
(1/400, 4010
2/10)
(1/400, 2/20)
(1/200, 2/10)
(1/200, 2/20)
8020
2010
4020
Dimensions
dy
10
20
10
20
RR
2.21 = 0.56
2.14 + 0.57 0.79 = 0.16
1.12 0.40
1.11 0.40
Test MSE
PILR
1.13 = 0.30
0.26 0.11
0.22 = 0.10
5.4
Impact of Basis Misspecification on Generalization
This section considers a practical scenario where the basis functions misspecified, situation that can occur during manual design or through random selection, as in an Extreme Learning Machine (ELM) 22 Any such misspecification can degrade performance by increasing the approximation error. As detailed in Appendix C.4 the total error is composed of this approximation error and an estimation error. While theory demonstrates that physical constraints reduce the estimation error, the overall model performance limited by the magnitude of the approximation error.
To demonstrate this effect, we conducted an experiment on the Harmonic Oscillator, intentionally omitting the known analytical frequency from the basis functions. Other experimental settings were identical to those in Section 5.1. With 10 data points, the performance was exceptionally poor. For basis size of d = 17, the test MSE was approximately 1.435 6466, of which the approximation error constituted nearly the entire amount at 1. 1.430. Increasing the basis size to d = 33 had negligible effect; the test MSE remained high at 1.434 as the approximation error was unchanged.
This result clearly shows the total error being dominated by the approximation error. It underscores prerequisite for our theory: the improvement in generalization from thysics-informed constraints is achieved only when the model possesses sufficient expressive capacity to represent the true solution.
6 Conclusion
This study introduces a framework for analyzing physics-informed models through the lens of affine varieties induced by the governing differential equations. We establish that generalization performance is governed by the dimension of this variety, rather than the number of model parameters, a finding that unifies existing thcories for linear equations. We further provide method for calculating this dimension and present experimental validation confirming that this intrinsic dimension effectively mitigates overfitting in highly parameterized settings. Although our analysis centers on linear regression models, the proposed geometric framework is broadly applicable to both linear and nonlinear differential equations, as our experiments demonstrate. This work offers a foundational, geometric interpretation of generalization that establishes a promising, though challenging, direction for future theory-guide
model selection, such as the optimal choice of basis and trial functions. Future work includes the extension and validation of our framework for other architectures, such as NN and EL.M The framework can also be extended to differential equations with unknown parameters by analyzing augmented parameter space.
10

---

References
[1] Yaser S Abu-Mostafa. The vapnik-chervonenkis dimension: Information versus complexity in learning. Neural Computation, 133):112-317, 1989.
[2] Radoslaw Adamczak. A tail incquality for suprema of unbounded empirical processes with applications to markov chains. 2008
[3] Tara Akhound-Sadegh, Laurence Perreault- Johannes Brandstetter, Max Welling, and Siamak Ravanbakhsh. Lie point symmetry and physics-informed networks. Advances in Neural Information Processing Systems, 36, 2024.
[4] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623-2631 2019.
[5] Eleonora Arnone, Alois Kneip, Fabio Nobile, and Laura M Sangalli. Some first results on the consistency of spatial regression with partial differential equation regularization. Statistica Sinica, 32(1):209-238, 2022.
[6] Genming Bai, Ujjwal Koley, Siddhartha Mishra, and Roberto Molinaro. Physies informed neural networks (pinns) for approximating nonlinear dispersive pdes. arXiv preprint mexix:2004.05560 2021.
[7] Bruno Buchberger. A theoretical basis for the reduction of polynomials to canonical forms. ACM SIGSAM Bulletin, 10(3):19-29, 1976.
[8] Shengze Cai, Zhicheng Wang, Frederík Fuest, Young Jin Jeon, Callum Gray, and George Em Karniadakis. Flow over an espresso cup: inferring 3-d velocity and pressure fields from tomographic background oriented schlieren via physics-informed neural networks. Journal of Fluid Mechamics, 102 2021.
[9] Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis. Physics-informee neural networks for heat transfer problems. Journal of Heat Transfer, 143(6): 060801, 2021.
[10] Andrea Caponnetto and Ernesto De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7:331-368, 2007.
[11] Yifan Chen, Bamdad Hosseini, Houman Owhadi, and Andrew M Stuart. Solving and learning nonlinear pdes with gaussian processes. Journal Computational Physics, 447:1 10668, 2021.
[12] Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through thysics-informee neural networks: Where we are and what's next. Journal of Scientific Computing, 92(3):88, 2022.
[13] David Dalton, Dirk Husmeier, and Hao Gao. Physics and lie symmetry informed gaussian processes. In Forty-first International Conference on Machine Learning, 2024.
[14] Tim De Ryck and Siddhartha Mishra. Error analysis for thysics-informel neural networks (pinns) approximating kolmogorov pdes. Advances in Computational Mathematics, 48(6):79, 2022.
[15] Nathan Doumèche, Francis Bach, Gérard Biau, and Claire Boyer. Physics-informed machine learning as a kernel method. In The Thirty Seventh Annual Conference on Learning Theory, pages 1399-1450. PMLR, 2024.
[16] Nathan Doumèche, Francis Bach, Gérard Biau, and Claire Boyer Physics-informet kernel learning, arXiv preprint 4rXhv:2409.13786 2024.
[17] Richard M Dudley. The sizes compact subsets of hilbert space and continuity of gaussian processes. Journal of Functional Analysis, 1(3):290-33, 1967.

---

[18] Federico Ferraccioli, Laura M Sangalli, and Livio Finos. Some first inferential tools for spatial regression with differential regularization. Journal of Multivariate Analysis, 189: 104866, 2022.
[19] Zhongkai Hao, Songming Liu, Yichi Zhang, Chengyang Ying, Yao Feng, Hang Su, and Jun Zhu. Physics-informed machine learning: A survey on problems, methods and applications. arXiv preprint
2022.
[20] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359-366, 1989.
[21] Zheyuan Hu. Ameya D Jagtap, George Em Karniadakis, and Kenji Kawaguchi. When do extended physics-informed neural networks (xpinns) improve generalization? SIAM Journal on Scientific Computing, 44(5)
2022.
[22] Guang- Bin Huang, Qin Yu Zhu, and TheenK Siew. Extreme learning machine: theory and applications. Neurocomputing. 70(1 (0(1-)):89-500) 2006.
[23] Ameya D Jagtap, Ehsan Kharazmi, and George Em Karniadakis. Conservative physics-infone infforeed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:1 13028, 2020.
[24] Xiaowei Jin, Shengze Cai, Hui Li, and George Em Karniadakis. Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations. Journal of Computational Physics, 426:109951, 2021.
[25] George Em Karniadakis, loannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422-44) 2422-440,4 2021.
[26] Gitta Kutyniok, Philipp Petersen, Mones Raslan, and Reinhold Schneider. A theoretical analysis of deep neural networks and parametric pdes. Constructive Approximation, 55(1):73-125, 2022.
[27] Da Long, Zheng Wang, Aditi Krishnapriyan, Robert Kirby, Shandian Zhe, and Michael Mahoney. Autoip: A united framework to integrate physics into gaussian processes. In International Conference on Machine Learning, pages 14210-14222. PMLR, 2022.
[28] John Milnor. On the betti numbers of real varieties. Proceedings of the American Mathematical Society, 15(2):275-280 1964.
[29] Siddhartha Mishra and Roberto Molinaro. Physics informed neural networks for simulating radiative transfer. Journal of Quantitative Spectroscopy and Radiative Transfer, 170:107705, 2021.
[30] Siddhartha Mishra and Roberto Molinaro. Estimates on the generalization error of physics- informed neural networks for approximating class of inverse problems for pdes. IMA Journal of Numerical Analysis, 42(2):981-1022, 2022.
[31] Siddhartha Mishra and Roberto Molinaro. Estimates on the generalization error of physics- informed neural networks for approximating pdes. IMA Journal of Numerical Analysis, 43(1): 1-43, 2023.
[32] Olga Arsen' evna Oleinik. Estimates of the betti numbers of real algebraic hypersurfaces. Matematicheski Sboriik, 70(3):635-640, 1951.
[33] Ivan Georgievich Petrovskii and Olga Arsen' evna surfaces. lavestiya Rossüskoi Akademü Nauk. Seriya Mutematicheskaya, 13(5):389 1949.
"esen'evnaaa Oleinik. On the topology of real algebraic
[34] David Pollard. Empirical processes: theory and applications. Ims, 1990.
[35] Rahul Rai and Chandan K Sahu. Driven by data or derived through physics? a review of hybrid physics guided machine learning techniques with cyher-physical system (cps) focus. IEEe Access, 8:71050-71073, 2020.
[36] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physies-informed neural networks: deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
12

---

[37] Robert Schaback and Holger Wendland. Kernel techniques: from machine learning to meshless methods. Acta numerica, 15:543- 639, 2006.
[38] Yeonjong Shin. On the convergence physics informed neural networks for linear second-order elliptic and parabolic type pdes. Communications in Computational Physics, 28(5):2042-2074, 2020.
[39] Yeonjong Shin, Zhongqiang Zhang, and George Em Karniadakis. Error estimates of residual minimization using neural networks for linear pdes. Journal of Machine Learning for Modeling and Computing, 4(4), 2023.
[40] Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics- informed deep learning for turbulent flow prediction. Proceedings the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1457-1466, 2020.
[41] Minglang Yin, Xiaoning Zheng, Jay D Humphrey, and George Em Karniadakis. Non-invasive inference of thrombus material properties with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 375:113603, 2021.
[42] Yifan Zhang and Joe Kileel. Covering number of real algebraic varieties and beyond: Improved bounds and applications. arXiv e-prints, pages arXiv-2311, 2023.
13

---

A
Notation
Symbol
Description
Data
f
True function to be learned
S R
Input domain
m
Input dimension
Number of observations
(c.,9s)
i-th observation (z₃; input, y;: output)
Target vector [V/.....
o²
Noise variance
Normally distributed noise following (V, o²)
Affine Variety and Variables
Φ
Differential operator
B = [6)}
Basis functions
j-th basis function from Basis vector at i
Φ(x)
E Rd
4 E Raxd
Design matrix (i-th row is (
T {(tx,
Finite collection of trial function and measure pairs.
Number of basis functions [B] (ambient dimension)
K
Number of trial functions [T]
T)
Affine variety defined by 2, T (set of weight vectors)
dy
Dimension of the affine variety V
w, tú, w"
Weight vectors (learnable, estimated, optimal)
BR(
& ball of radius R
VR
Affine variety constrained by the b2-ball (V n (2())) 2 regularization parameter
An
11 |1
Vector & norm, function L norm w.r.t. Borel measure
(2, (::)9
Vector Euclidean inner product, function inner product w.r.t. measure
Geometric/ / Complexity Measures
V VÇR" Rd (3, )-reuurr set
General affine variety Regularity condition Dimension of V Codimension d - dz
dy
codim(L)
Upper bound connected components of intersections e-covering number w.r.t. & norm 1|2-
&VV,&,
Analysis Constants
M
Upper bound constant for the basis functions, such that [|ø(z)]l2 S M Upper bound constant for the lower eigenvalue of design matrix +. Stability constant of the estimator Probability parameter
r
8
Linear Operators / PI Kernel
D.D
Constraint matrix when the operator D linear; Da.j is its entry Linear, nonlinear parts of 7
L
dv(z). dy(2)
Dimensions under L, - PI kernel defined with regularization matrix M
(M,
E.V
Hyperparameters of the PI kernel (controlling the balance) Basis Gram matrix, its entry Trial Gram matrix, its entry Effective dimension of the PI kernel Spectrum of matrix and its entry (eigenvalues)
B. By
T. TET
(o o()), 0,0;
Dimension Calculation
Pk N
Defining polynomial
Number of samples (w7, wx))
14

---

6.75
B. (1)
10
0.00
5.25
350
4 0.0
0.5
1.0
0.75
151.5 -1.0
-0.5
00.11
11.5 1.00
1.5
0.00
Figure 3: Illustration of the construction of the E-covering of the affine variety V C R² and the associated loss landscape. The black curve represents (3, dy) regular affine variety with dimension dy 1. The color gradients depict the loss landscape L(w) = Σx= IP (w))) of the equations defining V = {w P% (w) = 0, 4k = 1, .... K} The blue dotted line represents &g ball of radius R. The affine variety constrained with the l ball is covered by g-balls centered at the intersections of V with four given subspaces [ shown as red points. The upper bound on the number of intersections of every subspace with the variety is B3 while the actual maximum number is formed by the subspace L. (the yellow dotted line). The loss landscape of the equations is zero on V and locally convex around the points in V.
B Mathematical Background on Affine Varieties
In this section, we provide a formal definition of several concepts related to affine varieties and review the definition of the dimension of an affine variety, as briefly described Section
An affine variety is a fundamental concept in algebraic geometry. It is a subset of an affine space, defined the solution set to system of polynomial equations. Let K[w) denote the set of poly- nomials in the variables w (Wi,-.-, wa) Kd over field IK. (often IR C). An affine variety .(p....,PP)) C Kd defined by the polynomials Pl, PKK E [[[w] is given by:
V(p..... PR) = {w E K p& (ur) 0, Vk =
K}.
The geometry of an affine variety is determined by the set of all polynomials that "vanish" V. i.c., those that become zero for every point V. This set is called the ideal of the affine variety, denoted T(V), and is defined as follows:
I(V) = {p K[w] p(w) O, Vw E V}.
The generating polynomial set |K km of the affine variety V is subset of the ideal I(V).
The coordinate ring over V, denoted KVV, is introduced to identify polynomials that yield the same values on the variety V. Specifically, K[V] is defined as the quotient of the polynomial ring KJ by the ideal I(V), i.e., K[w)/[(V). In the coordinate ring K[V] K [[W), the difference between p and q vanishes on V, i.e., p(w) = 7(w) for all w E V. or equivalently p q E T(V). Thus, P and q are considered the same element. From another viewpoint, the coordinate ring K(V) can be considered set of polynomials not included the ideal T(V).
Based on the above definitions, we review the definition of the dimension dy of the affine variety in Appendix B.I and the regularity in Appendix B.2
B.1
Dimension of Affine Varieties
B.1.1 Geometric View
Considering the affine variety V as affine space, we can naturally define subvariety as an "subset" of the variety that also satisfies polynomial equations. Let qı,... Is be polynomials in a ring. Define (q...... qs} the smallest ideal generated by Vi.....95. qs; that is, (gI.....gs) qs) consists of all finite
15

---

1= 3. A subvariety sums of the form Ei=1 Tiq: where each is in the ring: (01.... 9s) = {E
U of is defined as the zero set of a subset ideal (q)..... ................................ Qs) K[w)] / [[V) given by:
U = w E K (.(()) = 0, Vg. E (gi,...,,s)) }
By using the concept of subvarieties, the dimension of an affine variety is defined as follows: Definition 4.1. The maximal length of the chains Vo C. V1 .... Váv of non-empty subvaricties of V.
This definition intuitively represents the size of V by the maximal length an increasing sequence of subspaces, If the generating polynomials {pic k== are all linear, the dimension of V is defined as the maximal length of increasing sequence of linear subspaces within V, which corresponds the dimension of V as linear space.
When we focus the local structure, the following equivalent definition is obtained:
Definition 4.3. The maximal dimension of the tangent vector spaces the non-singular points US V C Rd of the variety, ie., dy = max rank [Vpi (w) Vpx(wr)
From this definition. we can that the dimension dy is global quantity that summarizes the local linearized structure of the affine variety V at a point.
For example, let K = R and V C R be the plane: V = {(x, 2) x = 0}. A chain of subvarieties within V Vi C V C V2, where Vo = {00 O, 0)} (a point, O-dimensional), Vi {(4, 0, () E R) (a line, I-dimensional), and V/2 V itself (the plane, 2-dimensional). The maximal length of the nested subvarieties two, i.e., dy 2, which means that plane has two degrees of freedom.
B.1.2 Algebraic View
The structure of an affine variety is determined by the ideal I(V). Intuitively, the larger T(V) is, the polynomial constraints there are, which means that V becomes smaller, and consequently, the coordinate ring K[V] also becomes smaller. From this perspective, it is natural to expect a deep connection between the dimension of the coordinate ring [V[] (and similarly the ideal I(V)) and the dimension of the affine variety V.
To explore this connection, we first discuss the dimension of the coordinate ring K[V] using Krull dimension. The ideal p C R in a polynomial ring R is prime if Va, E R, ab E = a E p or b p. The definition of the dimension of the affine variety through the Krull dimension is shown below.
Definition B.I. The Krull dimension of the coordinate ring IK V|: The maximum length d of the chain of prime ideals Po C P1 C C Pa in the coordinate ring [[]..
This definition signifies that the dimension of affine variety is characterized in the world of polynomial sets by the maximal length of an increasing chain "subsets" within the coordinate ring, corresponding to Definition 4.1 from geometric perspective.
In contrast, the size of the coordinate ring K[V] can also measured using Hilbert series. First, by homogenizing the defining equations by adding one variable K, we embed the affine variety V C Kd into the projective variety P kd- The projective variety P(h C. Kd+1 defined by the homogeneous polynomials h1, h...... hg E K((w,,)) 7)], is given by:
P( hg) = {ww,)) E kd+1, h4 hgww, 7) = 0, Vk
K].
The dimension of the variety is also increased by one, i.e., dp = dy 1. The coordinate ring K[P] = (w, (P) of the projective variety P can be decomposed into subgroups (called the graded coordinate ring) as follows:
[
Sp.
So K.
PEN
where Sp is the set homogeneous polynomials of degree p modulo the ideal T(P). As metric for the size of the coordinate ring IK[P), the Hilbert function H(p) and filbert-Poincarl series HS((t are
16

---

defined as follows:
Il Tk==1 (1 thx)
H(p) = dim Sp. HS(()
Σ PEN
H(p)e
(1 1)o+1
where dim denotes the Krull dimension and P1:...: .....,PK are the degrees of the homogeneous polyno- mials .i....,,hg ...
The Hilbert function represents the dimension of "subspace" of the decomposed coordinate ring. and the Hilbert series is the generating function of the sequence of the Hilbert function, which is also a rational function with a pole at homogeneous components of the algebra with respect to the degree. According to the dimension theorem, the Krull dimension of the projective variety P matches the order of the Hilbert series at the pole 1, which is one of the most important results in commutative algebra.
These measures indicate the growth of the dimension of the
Therefore, the dimension of the affine variety defined using the Hilbert series, follows: Definition 4.2. The degree of the denominator of the Hilbert series of the affine variety V.
Given the Gröbner basis of the ideal I (P)), the Hilbert series can be easily computed, leading to an efficient estimation of the dimension of the affine variety dy.
B.2 Regularity of Affine Varieties
We informally define the concept of regular set for real affine varieties, which is used in Section 3.2 (for a formal definition, see Definition 2.1 in [42).
A affine variety V C d is (3, dv )-regular set if:
1. For almost all affine planes L with codim(L) S dy in Rd, V n L has at most 3 path- connected components.
2. For almost all affine planes L with codim(L) dy in Rd, V n L is empty.
The notion codim represents the codimension. For an affine subspace L C Rd, its codimension is defined by codim(L) (L) d dy. Simply put, codimension how many dimensions you are "missing" when comparing smaller space inside bigger space.
A regular set restricts the complexity of variety V. Intuitively, the complexity of V can be measured by the number of connected components in its cross sections. For instance, complex shape may have cross sections that split into multiple connected components. The larger the number of connected components 3, the more complex the topology of V. Moreover, the dimension at which we slice the variety is also important. If the slice (affine plane) is large enough in dimension, i.e., the codimension is small (< dy). then any intersection of the slice with V is limited to at most | connected pieces. Otherwise, the slice typically does not intersect V at all. For example, consider the circle V = {(x, y) E I2 x² + - 11 0} A line (codim(L) L) = 1) intersects the circle in at most two points. For single point (codim (L) 2), almost points do not lie in the circle; that is, intersections with higher codimension affine subspaces are almost empty. This implies that the circle a (2, 1) regular set.
B.3 Covering number of Affine Varieties
Lemma B.2 (Zhang and Kileel 42)). Let V C Rdd be a (8, d-v--ruellaa set in the ball B2(R) with the radius R. Then for all E (V))
log log,N(V,,, l12) < dy log 2Rddd + log 23.
(15)
This upper bound is obtained by slicing the affine variety V with subspaces (Ly)sEN within Rd and covering V with balls centered the intersections of T., and V, i.e., V -. U. Bz(w; e).
The covering for the two-dimensional case is illustrated in Fig. 3 The first term, (2Rdvd/e) represents the number of subspaces T., needed to cover the entire space. by the intrinsic dimension dy of the affine variety, although it is still influenced by the ambient
mainly determined
17

---

dimension d. The quantity & in the second term denotes the number of intersections between single subspace L and the variety V, and represents the covering number V L. Topologically. corresponds to the Betti numbers of the affine variety, which informally represent the number of holes given, for example, by the Serrovkkii-leini-MMiiiiii inequality [33 32 28). Specifically, an affine variety V n TB2 (R) defined by polynomials (P& [ of maximum degree and the 22-ball is (p(2p 1) dy >-reuular. This intuitively suggests that as the maximum degree of polynomials increases, the topology of the affine variety becomes more complex.
in V. The upper bound on the quantity
C Detailed Background on Problem Formulation
This appendix expands the formulation introduced in the main text, highlighting why a hybrid physics-data approach required.
C.1 Governing System
Let Ω C Rd be a bounded domain with boundary ØR. For a differential operator 9: 2: - L2()). the true state f*: IN satisfies the boundary-value problem
2 =
in 82
(16)
=
on 00,
(17)
where and g smooth but may be only partially observed inferred indirectly.
C.2 Available Information
practice one seldom knows and y exactly; instead one has:
Noisy pointwise observations. dataset ((x,,,)) with ys (x;) + Ess Es ~ (0, o²), where I; 12 aR
Weak-form physics information. Linear functionals (u)= = (u, dh) = 1,..., Ny, with trial functions 4/4 C2 (S2) and measures μık. together with the corresponding targets (x) = 1 (f)'))
C.3 Hybrid Surrogate Model
representative hybrid approach is the Physics-Informed Neural Network (PINN) neural surrogate fm= Ω + R, its parameters w are obtained by minimising
36 Given
No
((w)=
Σ (1. vil
+>
-
(18)
Nu
N k=1
Eidelity
physics residual form)
with hyper-parameter X > 0 balancing empirical fit and physical consistency.
Connection to standard collocation method In the standard collocation method, the unified residual forms reduce to pointwise strong-form residuals: specifically, for each k; we set
(xr 1, A
8
where 6z, is the Dirac measure at collocation point Tg. In this case, the linear functional 4 becomes
(u, (h) p = /
4 (u)
u(x) do.. u(xx).
and thus the physies term penalizes the pointwise physies residuals:
Nr
[[fw((xx) vxxx))
18

---

Limiting regimes.
Pure data fitting: X = 0 reduces Eq. 18 to standard supervised learning on D..
Fully physics-informeet If {4x] dense and N, oo, the residual term enforees Eq. (16) everywhere.
Truly hybrid: Finite N, with incomplete {4x -atpical in engineering- partial physies, while the data term compensates for the missing information.
C.4 Error decomposition and analysis
To directly measure how physics-based inductive bias improves the generalization ability of the machine learning models, fix the physies information known immutable prior. Let Hose denote the unrestricted hypothesis class (e.g. linear models or neural networks). We consider two ways of incorporating the physics prior into the base hypothesis class Hhuse:
Hard setting: Enforce the differential equation residual exactly by:
N. foadt E arg min Σ (((91)))((())) = Haard f k=1
(19)
Soft setting: Allow a relaxed residual tolerance:
fE)(E) E { House
N
(1e(5))4(0)) $e)= = Hoon-
(20)
N=1
For H. (Haadd (=)) let ff E H be the learned solution and fi = arg min Ilf - the best attainable approximation. By the triangle inequality, have:
|| - s fill + I1 Íll-
(21)
In this decomposition:
The term || - füll represents the approximation error: the best achievable error within the hypothesis class H.
The term Ilf represents the estimation error: the deviation due to finite data.
Our primary focus is to derive bounds the estimation error |[ fll thereby quantifying how well the learned solution converges to the best physics-constrant approximation.
In particular, our analysis centers on the hard constraint setting, where H. = Hhard on the linear base hypothesis
Hbase {fw T TTo R. o, B}.
with B chosen basis. Under the hard constraint, the admissible hypothesis class amounts restricting the parameter vector w to lie on an affine variety induced by the PDE residuals. More explicitly,
Hhant { E V(2,8,T), dy E
where V( 13, T) denotes the set of coefficient vectors satisfying the algebraic constraints generated by the operator acting on the basis and tested against the functionals T {(x
1N.
C.5
Extension: Incomplete Operators with Learnable Parameters
The above framework can be generalized to settings where the governing differential operator itself is only partially known and contains learnable parameters. Formally, suppose that instead of fixed operator Z LZ2 + (22), we consider a parametric operator
D. (2) + (2))),
c R
19

---

where с denotes a vector of unknown coefficients to be simultaneously estimated from data. In this case, the admissible hypothesis space naturally
Haug c(f f Hoase (x ((v), k= :..
defined as augmented constraint set over the joint variable (f.
The error decomposition then applies this extended space: for c²) denoting the best attainable pair in H Haug. the learned solution (f' è) satisfies
|| All
1 f1l
approsimation
estimation
with both terms now understood relative the augmented parameter space.
Typical cases.
Unknown diffusion coefficient. Consider the diffusion equation d.u - cÂu = O, where the diffusion constant 0 is unknown. Here (c) is scalar parameter. Under the hard constraint, the admissible hypothesis class can be written as
Hang { (Sw:c) (w. c) E Vo.f (, 23
E)E EB]
where V-)) denotes the algebraic variety of coefficienttparameterr pairs (w, c) that satisfy the residual constraints induced by T = {4x [4 Thus both approximation and estimation errors are quantified in this augmented parameter space.
N.
Unknown diffusion term (learned surrogate). In cases where the diffusion operator itself is not specified, one may introduce a surrogate to represent its action. The PDE constraint becomes
0efu 0o = 0,
lcading to the hypothesis class
Haug = { vo) (w, 0) E
v0, 10. oj B}.
Here Ug serves as a learnable proxy for the unknown diffusion term. Our error decomposition applies verbatim this augmented parameter space, with approximation error defined relative to the best attainable pair (w" 8*) and estimation error measuring the deviation of the learned (w, 0) from this target.
In summary, by enlarging the hypothesis space to include both explicit unknown coefficients and implicit unknown operator surrogates, the proposed error decomposition continues hold, thereby providing principled means of quantifying generalization in uperator-learning settings with incom- plete physics.
D Proof for Theorem 3,5
Theorem 3.5 (Minimax Risk Bound). Let V(9, 1,9,B77 B,T) be the (3, dreguuuular affine variety defined
Eq. Z Suppose Assumptions 3.4 hold. Then, there exists positive constant C, independent of n, dhz, d, and 3, such that for any E (0, 1), with probability at least - 8, the minimax risk for
PILR defined by Eq. (4) is bounded by
(v dy loglddy)) (dylogllddd) n
log 2,3 n
1og(2/0)
min max 1| u00 SCn Co-CCooo 'MMRR 20 w W'EVR
(5)
n
Proof. Step I:
We first upper bound the prediction error by term that represents the supremum of empirical process in the metric space of the affine variety. Using Lemma D.I I we get:
ق)) 2 < 2e (w w).
20

---

We denote Xw = (w ut) as the random process in the metric space (VR. 1l22. Note that the estimator random variable depending on the parameter w and the noise E. Then, the minimax risk is bounded as follows.
||
( w²)12 - 2 7 WEVR
min max T w011 de WVEVR
2 S min max do EVER Y
sup Xw
(22)
The first inequality holds by Assumption 3.3
Step 2: Next, we calculate the supremum of the empirical process X using the covering number. For all w1, w2 E VR, it is shown that the variable Xwy Xwy has sub-Gaussian increments with respect to the metric || 1|::
Xwi
Xw
ca((ur (d1) (w22 (i2)) (x:))
cil((w1 W2) - -221 - (<x
(23)
$
Σ E [[] - w2ll2 + ||20 202ll2) M 1=1 M w) w2/12Me,
where is the zero-mean Gaussian random variable with variance no? The second inequality holds by the The last inequality holds by Assumption 3.4
inequality and the third holds by the triangle inequality and Assumption 3.2
From Eq. (23) the random process Xw: Xwr has Gaussaann increments as follows.
||xur
Xwng 1l S MT) k1 ww - w2l12.
where is the standard Gaussian random variable and centered random process Zap Xw E[xw]] || Zuz ll ||x Xwy llve holds because [|x,, Xan, ll is sub-Gaussian.
|| 1l is the sub-Gaussian norm. For the
Using Lemma D.2 we obtain the following bound with some constant Co:
E sup w S CoynooMR (Vdy log dyd Vlo 2/3)
(24)
WEVR
Next, using Dudley's s integral tail bound, we have:
Pr sup
ww < E sup zw + CovinMT2R//og(00 2 21-0 -
WEVR
wEVa
By incorporating the non-centered process Xw, obtain:
Pr
sup Nw
sup
[E E sup zw Covna Cr2moo22R///g00/ 1 6. (25)
WEVR WEVR
wEVR
To bound [[[x]], we note that:
E [[w]]
E
Φ (w [w)]
= E E T6
E I
E
1
(26)
<
(2:))
1= 1=1
RRMR
21

---

Here, the third inequality follows from the Sauchy-Schward inequality, and the fourth inequality is derived from the fact that |=" $ ,112/(o ))ll2)² follows a chi-squared distribution with degrees of freedom and ti E VR.
By combining Eq. 24 Eq. 25
and Eq. (26 obtain the following bound with constant C::
(V
dy log dyd
log 2
log(6/
Pr sup Xw S COMTR
+ 2
z 8.
Ti WEVR
Y2
This completes the proof.
Lemma D.1. Let in be a minimizer the following optimization problem:
1 11y &w
i arg argargmn min
(27)
wEVR 71
is the affine variety constrained with the bz-ball, y 2w" E the observed vector, is the design matrix, w" E VR is the true parameter vector, and = [,,,...,;E is the noise vector with each E; independently following zero-mean Gaussian distribution. Then, under these conditions, we have:
where VR = -(S, n B2
[ -6)1 < 2e (ww" -0))
(28)
Proof. Since v minimizer of Eq. (27 we have:
yy fw ? I l/ Dw 1| = ||=12
The left-hand side can be expanded as:
1 Pú 2 |1y &w" + &w" D
||e 4ww" u
Thus, have:
||e 1 (w
20) 11 < |E
Expanding the left-hand side, we get:
| (ww" u ub
|| 2e "(w"" 20) + B r8)112
Subtracting || from both sides, we obtain:
"(w"" w)) [ ub)1 2
This completes the proof.
Lemma D.2. Let zw be the zero-mean random process the metric space (Vn, || Il22), which have the following lub-Gaussian increments. For all w W2 E Vv.
[izw,
Zwz ll
A||w] W2 1|2.
where I1,,, is the mub-Gaussian norm. is a positive constant. Then, the expectation of the supremum of the process can be bounded as follows.
E sup Zn CAR do dyd Vlog wEVR
where C. is positive constant.
Proof. Using Dudley's integral inequality [17 the zero-mean random process:
/. Vlog.V(Vn. 1|2)de.
E. sup 70 S CoA CoAf. WEVa
(29)
Since the set VR is (8, dyy) regular set from Lemma 2.13 by Zhang and Kileel 42 Lemma
B.2
shows the upper bound of covering number for any E (0, 2/t as follows.
log.N(Vn. l12) < dy log 2ddyd + log 2,3.
22

---

We substitute the above inequality to Eq.
29
2Rdvd
E sup 7w0 S CoA wEVR
vdv
dv
log
)de + 2 log 2/3
The integral in the first term can be calculated using substitution and integration by parts. Let
2ddyd
2Rdyd
1
log
de
log
de.
We substitute = 2Rdyd, u:= log(x/E) into the integral:
log dud u 1/2 (-xe
To solve the above integral, we use the formula for integration by parts:
1= 1-u uglog -
+
2000
2
1
log plogdud
11/2 " "du
2R Vlg + Rdyd
112 edu.
log Jlgdyd dyd
The integral in the second term can be upper bounded as follows.
"du S
"du = [-e
"Joo llgguu (dvd)
dud
We obtain the following bound with some constant C.
E sup zw S CAR (Vdylog dooogdyy++ + log 23
wEVR
E Proof for Proposition 3.7 and Proposition 4.4
Proposition 3.7. The effective dimension of the PI kernel associated with the affine variety V(9.B.T) ( Dw 0} with dimension dy upper bounded by
d
(e v)
Σ j=1
I+E Σ I+E++ vaj I+
j=dy
where {az)) denote the positive eigenvalues of the matrix DTD.
From Theorem 4.2 in 15 and Equation 15 in [6], the effective dimension bounded as follows:
Proof.
det v)
Σ
Σ
(30)
1 1+a
1+a-1
aC BB)
oEa(M 1
where M = + TD e Rdxd and B E Rdxdd is the Gram matrix of the basis functions, i.c.. By = (0,, ) for all 65 oy B.
Since the matrix DDD is positive semi-definite, the eigenvalues of the matrix M in ascending order (-) are given by
0,(M) - { & (1 (1......dv)
E vaj
<))
Therefore, the matrix M is positive definite, and the eigenvalues of M- are a for all E (M)) Combining this with Fq. (30). we obtain the first inequality. The second inequality is obtained when == 0.
D
23

---

Proposition 4,4. Suppose the operator 2 can be decomposed as 2 = L + I, where L is nonzero linear differential operator and & is nonlinear operator. Then, we have dv(z) I dy(
Proof. The point w 0 lies on V( and if L + 0. not singular. The Jacobian rank of polynomials pw(w) 2 w Φ), 1x) nz in 0 is equal dv(2). By Definition 4.3 we have dy(2)) S dy(Ø).
F Minimax Risk Analysis for Physics-Informed Models with General Architectures via Rademacher Complexity
In this section, we extend our analysis to general model architectures parameterized by polynomial functions of the weights. Our primary objective is to establish minimax risk framework grounded Rademacher complexity. This allows us to handle richer hypothesis spaces while incorporating structural constraints imposed by physical laws.
F.1
Notation and Definitions
To set the stage, we introduce several fundamental notions that will be used throughout the complexity analysis. We hegin with norms for vector- and matrix-valued objects, which help measure the size and regularity of functions and parameters. These norms provide the foundation for bounding Rademacher complexity.
Definition E1 (Mixed Norm for Vector- Valued Functions). Let f : X RRdduut be a vector-valued function. Its (oo, -nomm is defined by
dout Σ
1/p [f(x))
= == sup [=)
where Ilf(z))lp
(31)
2EX
The above norm enables to uniformly control the p-norm magnitude of the function outputs across the entire input domain.
Definition F.2 (Matrix p-Norm). For a matrix W e Rd x xdout, we regard W as vector vec(W) e Rduadou,, Its norm defined using the standard vector p-norm:
1/p
m
|W =
WoP
(1Sp<x)).
This definition allows us to consistently measure parameter magnitudes, regardless of whether they appear as vectors matrices.
Definition F.3 (Rademacher Complexity). Let F be function class and S = (æ1,.... Fn } an i.i.d. sample. The empirical Rademacher complexity defined as
E,
sup fEJ
To f(x))
where Th...., Tn are independent Rademacher variables taking values in (+1}. The Rademacher complexity is obtained by further taking the expectation of (F) over the random sample S.
Rademacher complexity serves central tool for quantifying the richness of hypothesis spaces and will be essential in deriving minimax risk bounds.
F.2 Definition of the Hypothesis Space
We now define the hypothesis space of interest. To ensure well-posedness of our analysis, the class required to satisfy boundedness and Lipschitz continuity conditions.

---

Definition F.4 (Lipschitz Polynomial Hypothesis Space), Let H. denote hypothesis space consisting of functions fw IR parameterized by R", where each fw polynomial in w.
The parameter domain is restricted to
E VR V( V(2,7)n B2(R),
(32)
where the affine variety V(2,T) is given by
V(2; T) = { (w R (2[
Dk) = 0, ((Ik, E ET)
(33)
Here, (R) = ww R 1| RJ denotes the Euclidean ball of radius R. Furthermore, there exists constant 4 > 0 such that fw fw s éx/lw w'll2, Yw, w' VR. ensuring Lipschitz continuity of the cerameter-to-funettee mapping.
(34)
The above construction provides a general framework. Next, we highlight an important special case relevant to mhysics-informer neural networks (PINNs).
Special Case: Polynomial PINN
Definition F.5 (Polynomial PINN Hypothesis Space). Let Hr, denote the hypothesis space repre- sented by fully-connected neural network of depth L with polynomial activation Φi:
fw (x) WLO(WL-1 L-1 (..6(W))......
with parameter vector vec(W1, W2) E Rd. The parameter domain is restricted to
w VR V(9. T) B2)),,
where V(2, 7) is the affine variety in equation 33
To ensure the polynomial PINN setting remains mathematically well well-posed, we introduce additional assumptions on boundedness and Lipschitz continuity.
Assumption F.6 (Uniformly Bounded Target and Hypothesis Class). The true regression function f 1T + IR is uniformly bounded as Il I Fiaax. Moreover, every hypothesis fu E H satisfies the same bound: Il F Fmax-
Assumption E.7 (Lipschitz Continuity and Boundedness of Polynomial Activation). The polynomial activation function á is uniformly bounded, || loc 2 S MM for some constant Ms > 0. Moreover, Lipschitz continuous with constant Lg, i.e., for any 21, 22 E R:
[[ - of (222)l12 S L - 22112.
Finally, we show that under the above assumptions, polynomial PINNs inherit Lipschitz property at the function level.
Lemma F.8 (Lipschitz Property of Polynomial PINNs). Suppose Assumption E.7 holds. For two polynomial PINNs fw: E HL with parameters w' E Vn the mapping from parameters to functions is Lipschitz continuous, iee.,
where the Lipschitz constant Eu,, depends on the network architecture as f fw ol S LHe w|w lawlllwwor - w'll2
(35)
ex, = M (RT 1-11
R(RLg)
(36)
RL
Consequently, the polynomial PINN hypothesis space Hr satisfies the Lipschitz condition in Defini- tion F.4 and therefore
Hr S H.
Proof. Let h2 and M/ be the outputs of layer for parameters w and wn respectively. We can establish recursive inequality: || W
[[ao(h,-1) Wio(h,-
[We(0(he-1) 6(h1 1)) + II( (2)
[Well2Lollle- 1/- 11.0011022222 I|We- - (Vllaal@(h/_1)l1
R 11/ +EM.
Solving this recurrence relation for |l, - 11 = I15 - sw llo yields the constant lz. The
relationship between covering numbers follows directly.
D
25

---

F.3 Generalization Bound
Based on the assumptions, we can now control the Rademacher complexity of the Lipschitz polyno- mial hypothesis class H. through Dudley integral bound.
Lemma F,9 (Dudley Integral Bound for Physics-Informet Models), Let H be the hypothesis space de- fined in Definition F.4 where the underlying affine variety V is (8, -reguar, Then the Rademacher complexity H. is bounded as
(v
(vax
In dy
( (2Rdv l7 Fiaxx
In ln()
R(HL) S C Fmax Cu
(37)
71
for some constant C O.
Proof. First, for any wi, W2 E VR and corresponding fwr Swz E H. Lipsitz continuity reads
ww w2ll
S E = Ilf swoll S Elu
Hence
N WH, || 111) s N (VR.E,11 II)
By Dudley's 's integral bound and Lemma B.2 on by covers.
12
Fmax
R(H) -
Vn (H, |l de
Vn
12
Fmux Pr
V W e, l1p)
VT
dy In Fmareu
(2Rdyd d
In(2
12
de + Fmax lx
n
Define
Fmaxff
dR d E
de.
Similar calculation in Lemma D.2 shows
2R dv d Ex Fmax
S Fax Hy In
Combining these estimates yields the stated bound.
Lemma F.10 (Maximum of sub-exponential random variables). Let Xi,..., Xn be independent, identically distributed sub-exponential random variables satisfying ||Xilly> S v for every i E (1....,n), n.. Then there exists an absolute constant > such that
max Sii
[X|]
I C Colog.. n.
Proof. From the assumption || l/ S we have the standard sub-exponential tail bound:
Pr
max X; 2 t < exx Isi5n
(-2 min
1
12 2
4 E
1
Moreover, for all 2 O, we can uniformly bound the maximum via a union bound:
Pr max X, 2 1 Pr U(x,20) 20 IsiSn
a
a Σ
Pr(X, 2
t2 1
11
S enp
min
2
2v
(38)
Now consider two cases based on the relation between and the exponent.
26

---

Case 1: Suppose
8²
1)
n S exp
2
min
2v²2
2
Then, by inequality equation 38 we have
12
2v 1) )
Pr max X; 2 1 2xx
min
2
2
1
t² t
< 2 exp
min
log
2222*
2v
which already provides stronger tail decay than ultimately require.
Case 2:
Suppose
t
t 2
exp
min
2
2v
Then, the probability on the right-hand side of Equation
38
satisfies
12
1))
Pr max ISiSn
X; 2 #
exx Blog n
min
> >222-220 22 >
2x2
2v
which is vacuously bounded above by 1. Thus, it does not affect the validity of our bound. In either case, obtain the uniform upper bound
1 min
t 11)
)
Pr max Isisn
< 2 exp
3 3log n
2v 2v
2(v ))
= 2 2xxp
min
2(U (luvlognj'
This tail bound implies that
max X, 13i5h
< logog n.
for some universal constant C > 0.
Lemma F.11 Let Iw (z, v) == (fw( - y)² Ex,, (X) for fw H. Under Assumption F.6 assume the noise Y (X) is sub- Gaussian with proxy variance o?. Then there exists constant C >> 0 (independent of w) such that
lowllo. S 244Fmax + +)), sup (X, & C(4Fmax + 0)*
Proof. By the triangle inequality for the nub-Gaussian norm,
[X) Y 11or fl fxXill + |ell
< 4Fmax +o
Using the identity || = =112132 for any Z, we get
(1( Y) 5 (4 + +0²
By the triangle inequality in the -norm
I1 (X, KIl)
< (X) - 1) + (X) -
< 2(4Fmus max + +0)2
Finally, to bound the second moment,
E (X, = Var (sw (X, C(4Fmax + 0)
for suitable constant C > 0. This completes the proof.
27

---

Lemma F.12. Let E(u, y) = (u - y)?. The Rademacher complexity of the composite class H.
{(fu(z) y)² fw H} satisfies
R(E o H. S (4Fmax + 20 2/ R(H),
where Fmax is uniform bound on | (2)] and o² is the noise variance.
Proof. Define for each example (Zi, Vi) the function
Φ.(u) (u-10 (u-))
For any u, v E R,
(0:(w) -0((v] = u² v²2 + 2ys (v u) (u v) (u + 2y:)
S ([u) + |u| + 2/y:l) | v/ Since [=w( F Faa for all x and l I max + |ey|. it follows that (d)(u) de(1)) I (4Fmux + 2|eyl) Iu -/ Applying the Rademacher contraction lemma to the empirical Rademacher complexity Rs.
R(e H.) = Ex Rs(E {t)
I 1x (4 2/5/) Rs (Pt)
= (4Fimxx + + Ex (7s
Since B.[|=|] < 2 for Gaussian noise,
A(E o H0) S (4Faux + 20 2/T2 R(H).
This completes the proof.
Finally, combining the Rademacher complexity bound with Adamczak's 's concentration inequality yields the following generalization bound for general physics-informed architectures.
Theorem F.13 (Generalization Bound for Physics-Informee Models). Let fw H be the Lipschitz polynomial hypothesis defined by Definition [F.4] and assume that Assumption holds. Then there exist constants Co, C1, C2 > such that, with probability at least 6,
sup Ln ()) [(w))
(39)
wEVn
2 Fiax eu
(Vex
dy In
2R dy d Ex Fmax
(2
Co 4Fmax + vov
71
cilll max o²
C2(4Fman max + o=
log
12
maxx
log
log
(40)
n
n
Proof. Apply Adamezak's concentration inequality 2 to the supremum:
1
Pr
sup
In i=1
(X,, Yz)
< CEsup
Ban (X Yi)
-1
w
w
i=1
(41)
Pnn č.V
In
exp
exp
where constant, V and are quantities defined by
V = sup Elg., (X, ))
|| max sup Iw (X,, Y.)le,, Isisn
28

---

From Lemmas F.10 and F.11
we have
sup Elow ( (X, X, C1/C + to)
max sup on (X,, Yo) S C2(4Fmax + o)² logn. igiSn w
where C1, C2 are constants.
By substituting the above quantities into inequality Equation 411 and converting it into high- probability bound, we obtain, with probability at least - 6, the following result:
sup w
Jel x Yi)
< CE sup
yuo (ii Vi)
(42)
Ci C1(4F max + to)a
C2(4Fmax + o)² 10)2
12
+ max
log 8'
log 8
We bound the expectation via symmetrization and Rademacher complexity:
Ssup
Ir (X,, Yi)
E sup
((((x)) YY) -((()(()))))))))))))))())) - -2)
i=1
Sup
Ti ((f(x,) -))=-(((x)) 1999)
2E sup
Tx(( YY)²
2 L H),
(43)
where tÜ are Rademacher variables and R({ H) denotes the Rademacher complexity of the squared loss class.
Combining Lemmas F9 and F.12 with Equations
42 and 43 these bounds complete the proof.
O
Remark F,14 (Limitations of Theorem F.3 While Theorem provides a theoretical generaliza-
tion bound for polynomial PINNs with neural network surrogates, two caveats are worth emphasizing. First, the affine variety induced by the hard constraints is defined by system of high-degree poly- nomial equations. The resulting algebraic structure is computationally intractable characterize explicitly, which limits both theoretical validation and practical implementation. Second, recent find- ings suggest that the generalization behavior over-parameterized models such as neural networks not governed solely by the geometry of the hypothesis space, but is strongly affected by the implicit bias of the optimization algorithm. Hence, bounds of the form in Theorem F.13 may substantially deviate from the empirically observed performance.
G Experimental Detail
G.1 Experiments Strong Solution
In the experiments in Section 5.1 strong solutions to the equations are obtained analytically. The analytical solution with added Gaussian noise was used as data, the variance of the Gaussian noise was set to 0.01. The hyperparameters L regularization weights and differential equation constraint weights and / were searched in the range [1e-9, le-2] using the Optuna library
(4) The configuration
with the smallest MSE on the validation data among 100 candidates was selected. All experiments were conducted on a MacBook Air equipped with an Apple M3 chip and 64 GB of unified memory. No external GPU or cluster computing resources were used.
29

---

Harmonic Oscillator: The initial value problem of a harmonic oscillator Øy] = with spring constant Kg and mass my on the domain f
[0. is given by:
d²
he
1 de
s(0) - yo. dt - on-
m,
We set the parameters m K2 1.0, T = 2x. The initial position and velocity (vo. vol are generated from the normal distribution (1,, 7), where an all-ones vector and / is the identity matrix. The solution to the initial value problem is analytically given by:
vt) y/ cos(wt) + Do
sin(wt), = v Via/ma.
The settings for the basis functions and the trial functions with the measure 6y B, (1h Ma) ET are as follows:
2 )) (2)+1(x)= sin 2 (i= .... dr),
(1(2) -, 22,(2) cos
tb (x) 1, 14 82, (A: 1, ..... K),
where de (2,4,8, 16) is the set of the number of basis functions, and z O is uniformly sampled from data with 100.
Diffusion Equation: The initial value problem for the one-dimensional diffusion equation [uu] 0 with diffusion coefficient and periodic boundary conditions is given by:
0 u Öt
82
2 m)
(x, E [-E, x [0, T), u(x 0) = =0(x)) Or "
x E [-
du
du
u (-E,t)) = (E,t))
Oz (-E, t) Ōr (3,
We set the parameters == 1.0, == T 2x. The initial value ug is given by:
ju E
uo (2)
Σ f=0
Aj cos (wyx) + By sin (wyyx) wj
(44)
where [Aj, B are generated from the normal distribution M11, 7) for all 0, ................................ , and
Jmax set to 1. The solution to the initial value problem is analytically given by:
Jmac
u(I, ()
Σ
[Ag cos (wyzx) + B, sin (wyax) ouft, ew0f,
.==0
The settings for the basis functions and the trial functions with the measure Φj E B, (t/, (K) E ET follows:
(1 = (2, 1) = cos(w,rre
oost1o (x,t) = sin(w, -u3ee ou3it, -ow)t
(j 1, dz, jr =1, ..., de), (u(a,, 1) 1, 1 = ((()))) 11)) (k (k=1. ..... K):
where d, 2, d, (10, 15, 20, 25) the sets of the number of basis functions, and (Ix, 4x) E Ω is uniformly sampled from data with K 50 x 500.
G.2 Experiments on Numerical Solution
we numerically simulate the Bernoulli equation using the explicit Euler method and the diffusion equation using the finite difference method (FDM). The data used are the numerical solutions with added Gaussian noise of variance 0.01. The method for hyperparameter search is the same as described in Appendix G.T For the nonlinear equations, we use the Adam optimizer with a learning rate x 10-2, along with an exponential learning rate scheduler. The training is performed for maximum of 2000 epochs, utilizing early stopping technique.
In the experiments in Section 5.3
30

---

Discrete Bernoulli Equation: The discrete Bernoulli equation N = 0 with the step size h on the domain Q [0, I"] given by:
D/ Ur Vr + Pyr QU?
h
where y/- = y(4r) and y++1 7(4 + h) are evaluations on the grid {4- [4=);=1 with n4 = T. We set the constant parameters (P, Q. p) to (1.0, 0.0, 0.0) for the linear case and to 01.0, 2.0) for the non-linear case. We use varying ni E (100, 200} with T 1.0 for both cases. The initial state yo generated from the standard normal distribution (0, 1) for both cases. The ground-truth solution to the initial value problem numerically solved by the explicit Euler method with step size h. The settings for the basis functions and the trial functions with measure o, E Br. (Vr, M7) Th are as follows:
0.5,
[1 ift (1)) (T= 1...,,.,, nt);
(1(1) 00 otherwise
4- = (t(,,, 14 de, (T= ne).
where n, = T is the same as the number of basis and trial functions, corresponding to the ground- -tuut solutions.
Discrete Diffusion Equation: The one-dimensional discrete diffusion equation Pr. [u] = with the
step size h [h,, hz]] and the diffusion coefficient c(u) on the domain Q = (- 3 x [0, T is
given by:
"
17+11 2j c(ug)
2J-1 2uJ + +u3-1
Pn(w) [u]
he
h
where uz == u(ry (r), u1t1 = u( ht), and 11 u(æ; + har, (-) are evaluations
the n2 x ny size grid { x {t,]7=1. where n2 = 23 and n, = T. The periodic boundary condition is adopted in the spatial domain, i.e., u
ho
h.
u3 for any E [d]. The diffusion coefficient
c(u) 1.0 is used for the linear case and c(u) = 0.1/(1 + u²) for the nonlinear case. We use varying (mq n22) E {(400, 10), (400, 20), (400, 30)) with E 1.0 and 7 1.0 for both cases. The initial value is generated with the same setting shown in Eq. (44). The ground-trute touud solution the initial value problem is numerically solved by the FDM with step sizes h; for the time domain and he for the spatial domain. The settings for the basis functions and the trial functions with measure 6j. E BR, (Wi.7, (f..) E Th are as follows:
1 if tz E [Ty, #5+1] x [tr, (j=1,-. 1f 1,..., nz); (x, {
otherwise
vsrr (z, t) Øg. (z, Hs.r = 6(3y)t))
(j I...., Ozy T ==1..... n)),
where n, = - and n, = # are the same as the number of basis and trial functions, corresponding to the ground- -ruth solutions.
H Additional Experimental Results
For each benchmark (discrete linear/nonlinear Bernoulli and Heat equations), we fix the number of basis functions d and vary the size of the trial-function set T. Reducing [7] relaxes the algebraic constraints on the learned solution, which turn increases the dimension the associated affine variety dy. As reported in Tables Figs. 4 and 5 when [7] decreases (and thus dy increases), the Test MSE steadily increases. This consistent rising trend of Test MSE with larger dy demonstrates that models endowed with fewer trial functions (i.e. weaker constraints) generalize more poorly.
31

---

(a) Linear Berouuli eq.
Settings
h
1/100
d dy
100
Dimensions
20
40
Test MSE (PILR)
0.012 = 0.0023
0.13 +: 0.082
0.33 + 0.22
(b) Nonlinear Bernoulli eq.
Settings h
1/100
d
100
Dimensions
dy
10
20
40
Test MSE (PILR)
0.17 +: 0.11
0.21
0.14
0.33 + 0.23
Figure 4: Experimental results for PILR on the discrete Berouulii equations.
(a) Linear Heat eq.
Settings
(h2, h2)
(1/400, 2/10)
d dy
4010
Dimensions
110
210
410
Test MSE (PILR)
1.6 + 0.35
1.9 + 0.44
2.0 + 0.49
(b) Nonlinear Heat eq.
Settings
(h4, hx)
(1/200, 2/10)
d
2010
Dimensions
dy
110
210
410
Test MSE (PILR)
0.37 + 0.11
0.43 + 0.14
0.56 = 0.19
Figure 5: Experimental results for PILR the discrete Heat equations.
NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No] ", is perfectly acceptable to answer "[No] provided
32

---

proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions, including the unification of collocation and variational methods via unified residual form, the establishment that generalization is determined by the affine variety dimension, and the method to approximate this dimension, all of which are addressed in the paper (Sections 3, 4, and 5).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Section 6 (Conclusion).
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
33

---

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Assumptions for theoretical results like Theorem 3.5 and Proposition 3.7 are stated. Proofs are provided in the appendices (e.g., Appendix for Theorem 3.5, Appendix E for Proposition 3.7), with proof sketches or main ideas often presented in the main text.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 5 describes the experimental setup, and Appendix G provides further
details on the experimental setup, including the equations, domain, boundary conditions, and network architectures used, which should allow for reproduction of the main findings.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
34

---

reproducible, they While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We plan to release the full codebase as part the supplementary material. The repository will include scripts and instructions to reproduce all main experiments. Since all data used in the experiments is synthetically generated, the released code also includes utilities to generate this data, ensuring full reproducibility without reliance on external datasets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //nips.co httpss.cc/ppe wtppppppluuddeeeeeeeppppppppppee
SroodeSubiissionPono
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section particularly Appendix GreenddxxGeeerriiie describe experimental settings, includ- ing PDEs, network architectures, number of data points, and training points. Details like specific optimizer parameters (e.g.. learning rate) are present in Appendix G, which aims to provide details for reproducibility.
35

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Experimental results are reported with means and standard deviations computed over multiple runs (10 random seeds per setting), as described in Section [ and detailed in Appendix G Error bars shown in the plots represent standard deviations. The randomness arises from the sampling of initial/bouddary conditions and optimization initialization, which are fixed across methods for fair comparison.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Details on computational resources provided in Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
36

---

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
The research presented is theoretical and methodological, focusing on mathe- matical understanding of PIML, and does not appear to raise concerns conflicting with the NeurIPS Code of Ethics.
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
The paper focuses foundational thcoretical aspects of physics-informed machine learning and does not explicitly discuss potential positive or negative societal impacts of this specific theoretical advancement. A broader impacts statement is not included.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The work is theoretical, focusing understanding generalization in PIML. It does not introduce new models or datasets that pose a high risk for misuse requiring specific safeguards.
Guidelines:
37

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
The paper primarily builds upon established mathematical concepts (c.g.. differential equations, affine varieties) and uses synthetically generated data for experiments (Appendix G), not relying on external datasets or codebases that would require explicit licensing details.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, papprswi thcode. com dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The codebase implementing our proposed method (PILR) will be released as part of the supplementary material. It includes configuration files, training scripts, and utility functions for solving representative PDEs. A README file documents how to run each experiment, and the code is released encourage reuse and extension by the community.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
38

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research does involve crowdsourcing or experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
The research does not involve human subjects, so IRB approval is not applica-
ble.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core methodology of this research focuses on physics-informed machine learning theory and does not involve the use of LLMSs as an important, original, non- standard component.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (uttp:///eeurrreccc////ofrrrrrrp for what should or should be described.
39


Reviews:
[{"review_id": "bu7EQKW5Ko", "reviewer": "Reviewer_sUwm", "summary": "This paper proposes a theoretical analysis to study the generalisation of physics-informed linear regression problems where the physical information is introduced in the form of weak constraints on the hypothesis space. The authors prove a probabilistic minimax risk bound on the estimation error in terms of the intrinsic dimension dV of the affine variety associated with the physical constraints. A practical estimator for dV is proposed and experiments on linear and nonlinear systems are usedd to demonstrate the bound.", "questions": "- Could the author provide an example where something else than a Borel or Dirac measure for μk would be a useful choice? This is just out of personal curiosity\n- Certainly due to my lack of familiarity with algebraic geometry I'm still struggling to understand what are path-connected components in a (β, dv)-regular set. Could the author provide some intuition on this?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\n- I found the theoretical analysis proposed by this paper very compelling. The analysis of the role of constraints as the dimensionality of their induced variety is elegant and provides theoretical tools to understand the generalization of PINN that matches practical experience from working with them. This is a very welcome piece of theoretical understanding which will hopefully help inform future design choices for physics-informed learning models.\n- It is great to see alignment of the proposed theory with previous work on PI-kernels under linear operators, this is precisely a question I had in mind while reading section 3 and it was refreshing to see the authors address it immediately\n- The author provide a practical rule of thumb to get a lower bound on dV so the theory can guide model design\n- Generally, the paper has a clear exposition which doesn't overwhelm the reader with unnecessary technicalities in the main text, but refers to comprehensive theoretical background in the supplementary material (I haven't checked the proofs)\n- The experiments are clear and convincing, addressing the kind of ablation questions one would typically expect. This feels like polished work.\n\n\n\nWeaknesses\n- The assumptions made in Section 3 are reasonable for theoretical analysis, but real physical systems may in practice violate them - that could be worth a mention for readers coming with a physics-background\n- One thing that somewhat got me confused is that the paper claims that in contrast with prior work, they don't assume complete knowledge of the differential equation. Yet, it seems that the variety in (2) is defined in terms of the exact differential equation operator D. I suppose that \"incomplete knowledge\" referred to the weak form of constraints, but I thought for a second we would be assuming some sort of approximate differential operator D.", "was_revised": true, "final_justification": "I appreciated the paper, which I believe will make a significant contribution to the field, and will be happy to see it published.", "rebuttal": "Thank you very much for taking the time to review our manuscript. Your feedback is greatly appreciated.\n\n> Q1: The paper's assumptions are reasonable for theory but may not hold in practice.\n\nWe agree that acknowledging the potential gap between our theoretical assumptions and real-world application is crucial. We will add the following statement to Section 3 to clarify this for readers:\n\n\"While we assume Gaussian noise for analytical tractability, noise in observational data from physical phenomena can be far more complex. Bridging this gap is an important direction for future work.\"\n\n> Q2: The claim of 'incomplete knowledge' is confusing, as the operator $\\mathscr{D}$ seems to be fully known.\n\nWe thank the reviewer for highlighting this ambiguity. Our use of \"incomplete knowledge\" refers specifically to the source terms or boundary conditions, not the differential operator $\\mathscr{D}$ itself, which we assume to be known, as detailed in Appendix C.\n\nTo resolve this confusion, we will revise the manuscript as follows:\n\n* Line 26: prior knowledge of the governing differential equations, **particularly their source terms or boundary conditions**, is often incomplete.\n* Line 34, 90: formulate the ~~incomplete~~ differential equation constraints\n* Line 96: the discrete weak form of the **known** differential equation $\\mathscr{D}$ is defined by\n\nFurthermore, we will add a note in the Appendix that our framework can be extended to cases where $\\mathscr{D}$ is partially unknown (e.g., an unknown diffusion coefficient $c$). This would involve analyzing an affine variety $\\mathcal{V}$ defined by polynomial constraints on an augmented parameter space that includes both the model weights $w$ and the unknown physical parameter $c$. For the linear diffusion equation, this would correspond to the variety $\\mathcal{V} = \\\\{(w, c) : \\langle \\mathscr{D}_c[w^\\top \\phi], \\psi_k \\rangle = 0\\\\}$. We will add a note on this possible extension in the Appendix.\n\n> Q3: Could the author provide an example where something else than a Borel or Dirac measure for $\\mu_k$ would be a useful choice? \n\nYes. A key example arises when extending the problem to differential equations on a curved manifold, such as the unit sphere $\\mathbb{S}^2$. In such settings, it is natural to define the integral using the Riemannian volume measure $\\mathrm{d}\\mathrm{Vol}_{\\mathcal{M}}$. For instance, Zhou and Shi (2025) recently adopted this approach on manifolds.\n\nZhou, Hanfei, and Lei Shi. \"Weak Physics Informed Neural Networks for Geometry Compatible Hyperbolic Conservation Laws on Manifolds.\" arXiv preprint arXiv:2505.19036 (2025).\n\n> Q4: I'm still struggling to understand what are path-connected components in a $( \\beta, d\\_V )$-regular set. Could the author provide some intuition on this?\n\nWe appreciate the opportunity to provide more intuition. We have a detailed explanation in Appendix B.2, using the minimal example of a circle in a 2D plane, which we summarize here. \n\nConsider a circle defined by $V = \\\\{ (x, y) \\in \\mathbb{R}^2 \\mid x^2 + y^2 - 1 = 0 \\\\}$. The ambient space dimension is $d = 2$, and the circle's intrinsic dimension is $d_V = 1$. In this 2D plane, we can consider affine subspaces $L$, such as lines (1-dimensional) and points (0-dimensional). The definition of a $(\\beta, d_V)$-regular set depends on the codimension of these subspaces, which is $\\text{codim}(L) = d - \\dim(L)$.\n\n**Case 1: Intersection with Lines**\n\nFor any line $L$ in the plane, its dimension is $\\dim(L) = 1$. The codimension is therefore $\\text{codim}(L) = 2 - 1 = 1$. Since $\\text{codim}(L) \\leq d_V$ (as $1 \\leq 1$), this falls under the first case of the definition. Here, $\\beta$ is defined as the maximum number of path-connected components of the intersection $V \\cap L$.\n\nLet's see what happens when we intersect our circle $V$ with a line $L$:\n* **No intersection**: If the line misses the circle, $V \\cap L$ is empty (0 components).\n* **Tangent line**: If the line touches the circle at a single point, $V \\cap L$ is a single point (1 path-connected component).\n* **Secant line**: If the line cuts through the circle, $V \\cap L$ consists of two distinct points. Since these points are separated, this set has two path-connected components.\n\nThe maximum number of path-connected components we can get is **2**. Therefore, for the circle, $\\beta = 2$.\nIntuitively, $\\beta$ is related to the topology of the variety $V$. A line can intersect the single \"loop\" of a circle at most twice.\n\n**Case 2: Intersection with Points**\n\nNow, consider a point $L$ in the plane. Its dimension is $\\dim(L) = 0$. The codimension is therefore $\\text{codim}(L) = 2 - 0 = 2$.\n\nSince $\\text{codim}(L) > d_V$ (as $2 > 1$), this falls under the second condition, which requires  $V \\cap L$ to be empty for \"most\" such subspaces. This condition essentially requires that the intersection $V \\cap L$ is empty for \"most\" such subspaces. \nThis holds for the circle, as most points in the 2D plane do not lie on the circle itself.\n\nThe parameter $\\beta$ is crucial because it bounds how many disconnected pieces of the variety we must cover when we intersect it with a subspace L. As our covering construction in the paper (following [41]) places the centers of ϵ-balls on these intersections $V \\cap L$, $\\beta$ directly influences the final covering number bounds by controlling the complexity of these intersections.\n\n[41] Yifan Zhang and Joe Kileel. Covering number of real algebraic varieties and beyond: Improved453bounds and applications. arXiv e-prints, pages arXiv–2311, 2023.454."}, {"review_id": "g01OwPqx3c", "reviewer": "Reviewer_gn41", "summary": "The paper proposes a theoretical framework for physics-informed linear regression. They theoratically show that generalization cap is governed by the dimension of an affine variety induced by physical constraints.\n\nThey also naturally derive from this assertion to the linear operator case (the rank-nullity theorem), which is commonly used in traditional linear ROM and galerkin methods.\n\nThe unified weak-form formulation and minimax risk analysis on the affine variety are novel to the nonlinear operator cases, and experiments on 3 benchmark PDEs demonstrate the effectiveness of their theory.\n\nMy overall take: paper is well writen with minor improvements for clarity. I recommend boarderline accept and look forward to revision.", "questions": "1. How are \\((\\psi_k, \\mu_k)\\) chosen for later 2 cases? For the 1st case only Dirac measures are used, does the theory still hold for general \\(\\mu_k\\)? (maybe an ablation study?) \n2. Can the framework handle *unknown terms in \\(\\mathcal{D}\\)* (not just \\(v/g\\))? If no, any thoughts?\n3. I am also curious how does \\(d_{\\mathcal{V}}\\) (and the final performance) change if the basis \\(\\{\\phi_j\\}\\) is misspecified (e.g., missing key frequencies)?  You mentioned mis-used basis can downgrade performance, maybe show an ablation study?", "limitations": "No; the authors mention many good applications/extensions, however miss this one.\n\nAlthough they \"claimed\" this method can handle partial known physics, by their Appendix C, this actually refers to unknown source term or boundary value. What if some term(s) in $\\mathcal{D}$ is also unknown? How would they like to approach this?", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### **Strengths**  \n- The minimax bound and connection to kernel methods are solid advances for linear physics-informed models.  \n- The discrete weak form unifies collocation and variational constraints.  \n- Experiments confirm that smaller \\(d_{\\mathcal{V}}\\) correlates with better generalization (Section 5).  \n\n### **Weaknesses & Clarifications Needed**  \n#### **1. Inconsistent Experimental Setup**  \n- The later 2 experiments do not specify the trial functions \\((\\psi_k)\\) or measures \\((\\mu_k)\\), despite their theoretical importance.\n- Also, for the 1st experiment, the authors choose collocation (\\(\\mu_k = \\delta_{x_k}\\)), which is actually the strong form and contradicts the general weak-form claims.  \n- **Request**: Clarify the choice of \\((\\psi_k, \\mu_k)\\) for all experiments. If Dirac measures are used, justify why this is a good choice (eg, by ablation study).  \n\n#### **2. Misleading \"Partial Knowledge\" Description**  \n- The paper implies handling incomplete physics (e.g., unknown terms in PDEs) in its abstract and introduction, but Appendix C reveals the \"unknown partial knowledge\" is merely *source terms \\(v\\)* or *boundary conditions \\(g\\)* (not \\(\\mathcal{D}\\) itself). This reads misleading.\n- **Request**: Revise the abstract/introduction to clearly clarify that \\(\\mathcal{D}\\) is assumed fully known. If the method cannot handle missing terms in \\(\\mathcal{D}\\), state this limitation explicitly in future works.  \n\n#### **3. Fixed Basis Misrepresentation**  \n- Through reading and many terms like \"data-driven\", it's not clear if the basis \\(\\{\\phi_j\\}\\) is fixed or learned. This only becomes confirmed after reading the studied PDEs and the chosen basis for them. Maybe mention it earlier in methodology section.", "was_revised": true, "final_justification": "No score adjustment; authors agreed to incorperate my suggestions; the reply is satisfactory; the current shape of manuscript is worth boarderline accept. Hence I kept my score.", "rebuttal": "Thank you very much for your careful review of our submission. We highly value your constructive feedback and suggestions.\n\n> Q1. How are $(\\psi\\_k, \\mu\\_k)$ chosen for later 2 cases?\n\nThe explicit definitions for the trial functions and measures used in these experiments are provided in Appendix G.2, as mentioned on line 330. These are written right after line 804 and right after line 818.\n\n> Q2. For the 1st case only Dirac measures are used, does the theory still hold for general (\\mu_k)? If Dirac measures are used, justify why this is a good choice?\n\nWe thank the reviewer for this insightful question. The reviewer is correct that our original experiments focused heavily on the Dirac measure. To demonstrate that our theory holds for general measures, we have conducted an additional experiment for the Harmonic Oscillator using sine and cosine test functions (i.e., $\\mathcal{T}$ includes basis functions from $\\mathcal{B}$) with the Borel measure. \n\nIn this case, we obtain $d\\_V=2$, just as with the Dirac measure. Consequently, the results are nearly identical to those originally shown in Figure 1 (a). The new results are as follows:\n| d | 5 | 9 | 17 | 33 |\n| --- | --- | --- | --- | --- |\n| PILR(n=10) | 0.012 ± 0.002 | 0.012 ± 0.002 | 0.012 ± 0.002 | 0.012 ± 0.002 |\n| RR (n=10) | 0.014 ± 0.002 | 0.031 ± 0.019 | 1.015 ± 0.433 | 1.187 ± 0.525 |\n| PILR(n=20) | 0.011 ± 0.002 | 0.011 ± 0.002 | 0.011 ± 0.002 | 0.011 ± 0.002 |\n| RR (n=20) | 0.012 ± 0.002 | 0.014 ± 0.003 | 0.071 ± 0.055 | 1.002 ± 0.479 |\n| PILR(n=40) | 0.010 ± 0.001 | 0.011 ± 0.001 | 0.011 ± 0.001 | 0.011 ± 0.001 |\n| RR (n=40) | 0.011 ± 0.002 | 0.012 ± 0.002 | 0.016 ± 0.005 | 0.118 ± 0.123 |\n\nThese results confirm that our core findings are robust to the choice of measure. We will add this result as a new figure in the Appendix.\n\nIt is important to note that the goal of our paper is not to prescribe the optimal choice of trial functions or measures, but rather to demonstrate that our theoretical framework holds across a variety of settings for $\\mathscr{D}$, $\\mathcal{B}$, and $\\mathcal{T}$.\n\n> Q3. Revise the abstract/introduction to clearly clarify that D is assumed fully known. If the method cannot handle missing terms in D, state this limitation explicitly in future works.\n\nWe thank the reviewer for bringing this point to our attention. Our use of \"incomplete knowledge\" refers specifically to the source terms or boundary conditions, not the differential operator $\\mathscr{D}$ itself, which we assume to be known, as detailed in Appendix C.\n\nTo resolve this confusion, we will revise the manuscript as follows:\n\n* Line 26: prior knowledge of the governing differential equations, **particularly their source terms or boundary conditions**, is often incomplete.\n* Line 34, 90: formulate the ~~incomplete~~ differential equation constraints\n* Line 96: the discrete weak form of the **known** differential equation $\\mathscr{D}$ is defined by\n\n> Q4: Can the framework handle unknown terms in D? If no, any thoughts?\n\nYes, our framework will be readily extended to a setting where the operator $\\mathscr{D}$ has unknown parameters (e.g., an unknown diffusion coefficient $c$) that are learned simultaneously from the data. For instance, in the diffusion equation example, if $c$ were also to be learned, one could simply analyze the affine variety $\\mathcal{V} = \\{(w, c) : \\langle \\mathscr{D}_c[w^\\top \\phi], \\psi_k \\rangle = 0\\}$ defined by polynomials over an augmented parameter space including both the solution weights w and the operator parameters c. We will add this point to the Appendix as a direction for future work.\n\nHowever, to improve the main paper's clarity and focus, we will remove the ambiguous \"incomplete\" description regarding $\\mathscr{D}$ and explicitly state that the operator is assumed to be known, as detailed in our response to Q3.\n\n> Q5: Through reading and many terms like \"data-driven\", it's not clear if the basis is fixed or learned.\n\nThe basis functions are fixed, not learned, as part of the modeling process. To clarify this, we will revise line 104 as follows:\n\nLine 104: \"Let B be a fixed basis (i.e., not learned like a neural network).\"\n\n> Q6: I am also curious how does $d_V$ (and the final performance) change if the basis $\\phi_j$ is misspecified (e.g., missing key frequencies)? You mentioned mis-used basis can downgrade performance, maybe show an ablation study?\n\nThis is an excellent question. We have performed the requested ablation study for the Harmonic Oscillator. In this experiment, the basis $\\\\{\\phi\\_j\\\\}$ is intentionally misspecified by omitting the known frequency of the analytical solution. We used the Dirac measure and a constant trial function, with $K := |\\mathcal{T}|$ set sufficiently large. The results are below:\n\n| total error / approx_error | d=17 | d=33 |\n| --- | --- | --- |\n| PILR (n=10) | 1.435 ± 0.646/1.430 ± 0.645 | 1.434 ± 0.646/1.430 ± 0.645 |\n| PILR (n=40) | 1.435 ± 0.646/1.430 ± 0.645 | 1.435 ± 0.646/1.430 ± 0.645|\n\nThe results show that performance is poor and does not improve with more data $n$ and less dimension $d$. This is because a misspecified basis introduces a large approximation error (the inability of the hypothesis space to represent the true solution), which becomes the dominant factor over the estimation error that our theory bounds.\n\nThe decomposition of the total error is defined in Appendix C.4, Eq. (14):\n\n$$\n\\\\|f^* - \\hat{f} \\\\| \\leq \\\\underbrace{\\\\|f^* - f^*\\_{\\\\mathcal{H}} \\\\|}\\_{\\\\text{approximation error}} + \\\\underbrace{\\\\|f\\_{\\\\mathcal{H}} - \\\\hat{f} \\\\|}\\_{\\\\text{estimation error}},\n$$\n\nwhere $ f^* $    is the true function and $f^*\\_{\\\\mathcal{H}}$ is the best possible approximation within the hypothesis $\\\\mathcal{H} = \\\\{ w^{\\\\top} \\\\phi : w \\\\in \\\\mathcal{V}\\_R \\\\}$.\n\nIn this misspecified case, the approximation error term is large and irreducible. The physically constrained solution space may not contain any non-trivial solution, leading to $d_V=0$. We will add this experiment and discussion to the Appendix."}, {"review_id": "92v4hJh5h2", "reviewer": "Reviewer_6adr", "summary": "This paper analyzes the generalization of physics-informed learning through affine variety learning. Although the number of model parameters can be large, leading to poor generalization, the physical (differential equation) constraint largely restricts the hypothesis space -- for example, the 1D harmonic oscillator example only has $d_v=2$, avoiding a generalization error dependent on the number of model parameters. They propose a numerical method to estimate $d_v$, and apply the method to analyze two linear systems in order to verify the correctness of their theory.", "questions": "* Line 156, what is $d$ (\"ambient input dimension\")? For example, when we use a 1000-parameter PINN to solve a 2D PDE, is $d=2$ or $d=1000$? If $d=2$, then the situation $d_v\\ll d$ becomes impossible. \n* Line 307, the diffusion coefficient is 0?\n* I'm slightly confused by the goal of Physics-Informed Linear Regression (PILR). Is it a theoretical framework or a training method? Section 3 seems to frame it as a theory, while Section 5 seems to suggest that it is a training method.\n* Can you give a simple nonlinear example in action?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* This paper is well-formatted and looks professional (I did not see typos, but I also disclaim that I do not understand all theorems. See questions)\n* Understanding the generalization behavior of physics-informed learning (especially the benefit of physical constraints) is an important topic \n\nWeaknesses:\n* I found this paper mathematically dense. I think that some parts can be dropped without compromising the main idea. For example, I am confused by $\\beta$. My best attempt to understand it is that it is related to topology due to the nonlinear differential operators, but otherwise it remains mysterious. In fact, both examples in the experiment section are linear, making the topological analysis unnecessary.", "was_revised": true, "final_justification": "Thanks to the authors for addressing my questions. I would love to raise the score to 4.", "rebuttal": "We would like to express our sincere gratitude for your time and effort in reviewing our manuscript. Your comments have been very helpful.\n\n> Q1: Line 156, what is d (\"ambient input dimension\")? For example, when we use a 1000-parameter PINN to solve a 2D PDE, is d=2 or d=1000?\n\n$d$ is the number of parameters in the model, not the dimension of the PDE's domain. So in your example, $d=1000$.\n\n> Q2: Line 307, the diffusion coefficient is 0?\n\nWe apologize for this typo. The correct equation is:\n\n$$\n\\frac{\\partial u}{\\partial t} - c \\frac{\\partial^2 u}{\\partial x^2} = 0.\n$$\n\nWe will correct this in the revised manuscript.\n\n> Q3:  I'm slightly confused by the goal of PILR. Is it a theoretical framework or a training method? Section 3 seems to frame it as a theory, while Section 5 seems to suggest that it is a training method.\n\nThe goal of our paper is to provide a theoretical framework for analyzing the learning problem we refer to as PILR. Section 3 details this theoretical analysis, while the purpose of Section 5 is to empirically validate our theory, not to propose a new training method. In fact, the algorithms used to solve the PILR problem in Section 5 are existing, standard methods: the analytical solution from [16] for linear cases and Adam optimizer for nonlinear cases (see lines 283-287).\n\nTo prevent this misunderstanding in the manuscript, we will revise the text to clarify this distinction. The revised sentence at the beginning of the experimental section will read:\n\nLine278: To validate our theoretical analysis, our experiments compare the generalization performance of two fundamental approaches to learning the target function $f^*$: standard Ridge Regression (RR) with the fixed basis $\\\\mathcal{B}$, which uses only observational data, and Physics-Informed Linear Regression (PILR), which additionally incorporates constraints from physical laws. \n\n> Q4: Can you give a simple nonlinear example in action?\n\nYes. We conducted experiments with two nonlinear equations: the nonlinear Bernoulli equation and the nonlinear diffusion equation in Section 5. \n* For the Bernoulli equation, the nonlinear term is $Qy^{\\\\rho}_{\\tau}$, where we set $\\rho = 2$.\n* For the diffusion equation, the diffusion coefficient depends on the solution: $c(u) = 0.1/(1 + u^2)$\n\nOur experiments confirm that even in these nonlinear cases, the improvement in generalization error is well-explained by the dimension of the resulting affine variety, $d_V$.\n\n> Q5. I think that some parts can be dropped without compromising the main idea. For example, I am confused by $\\beta$. My best attempt to understand it is that it is related to topology due to the nonlinear differential operators, but otherwise it remains mysterious.\n\nWe understand that $\\beta$ is an abstract concept, but it is a crucial component of our work.\n\nA key strength of our theory is its applicability to nonlinear differential equations, which makes the topological analysis involving $\\beta$ indispensable. To clarify its role, we provide several explanations in the manuscript:\n\n* A concise definition in Lines 159-165.\n* A discussion of how $\\beta$ appears in the covering number upper bound in Lines 166-177.\n* A detailed, intuitive explanation with examples in Appendix B.2.\n\nHere is a brief summary of the intuitive explanation from Appendix B.2:\n\nConsider a circle defined by $V = \\\\{ (x, y) \\in \\mathbb{R}^2 \\mid x^2 + y^2 - 1 = 0 \\\\}$. The ambient space dimension is $d = 2$, and the circle's intrinsic dimension is $d_V = 1$. In this 2D plane, we can consider affine subspaces $L$, such as lines (1-dimensional) and points (0-dimensional). The definition of a $(\\beta, d_V)$-regular set depends on the codimension of these subspaces, which is $\\text{codim}(L) = d - \\dim(L)$.\n\n**Case 1: Intersection with Lines**\n\nFor any line $L$ in the plane, its dimension is $\\dim(L) = 1$. The codimension is therefore $\\text{codim}(L) = 2 - 1 = 1$. Since $\\text{codim}(L) \\leq d_V$ (as $1 \\leq 1$), this falls under the first case of the definition. Here, $\\beta$ is defined as the maximum number of path-connected components of the intersection $V \\cap L$.\n\nLet's see what happens when we intersect our circle $V$ with a line $L$:\n* **No intersection**: If the line misses the circle, $V \\cap L$ is empty (0 components).\n* **Tangent line**: If the line touches the circle at a single point, $V \\cap L$ is a single point (1 path-connected component).\n* **Secant line**: If the line cuts through the circle, $V \\cap L$ consists of two distinct points. Since these points are separated, this set has two path-connected components.\n\nThe maximum number of path-connected components we can get is **2**. Therefore, for the circle, $\\beta = 2$.\nIntuitively, $\\beta$ is related to the topology of the variety $V$. A line can intersect the single \"loop\" of a circle at most twice.\n\n**Case 2: Intersection with Points**\n\nNow, consider a point $L$ in the plane. Its dimension is $\\dim(L) = 0$. The codimension is therefore $\\text{codim}(L) = 2 - 0 = 2$.\n\nSince $\\text{codim}(L) > d_V$ (as $2 > 1$), this falls under the second condition, which requires  $V \\cap L$ to be empty for \"most\" such subspaces. This condition essentially requires that the intersection $V \\cap L$ is empty for \"most\" such subspaces. \nThis holds for the circle, as most points in the 2D plane do not lie on the circle itself.\n\nThe parameter $\\beta$ is crucial because it bounds how many disconnected pieces of the variety we must cover when we intersect it with a subspace L. As our covering construction in the paper (following [41]) places the centers of ϵ-balls on these intersections $V \\cap L$, $\\beta$ directly influences the final covering number bounds by controlling the complexity of these intersections.\n\n[41] Yifan Zhang and Joe Kileel. Covering number of real algebraic varieties and beyond: Improved453bounds and applications. arXiv e-prints, pages arXiv–2311, 2023.454."}, {"review_id": "h5kkK86QpY", "reviewer": "Reviewer_qf6r", "summary": "This paper presents a novel theoretical framework for analyzing the generalization performance of physics-informed regression models based on the affine variety induced by the PDE constraints. The main theorem of the paper identifies the dimension of the affine variety as the dominant factor governing generalization in physics informed models, rather than the number of model parameters. The discrete weak framework adopted also holds for both collocation and variational methods, allowing hybrid learning settings which are common in the physics-informed field. The authors provide a rigorous minimax risk analysis, propose a tractable approximation method for computing variety dimension, and empirically validate the theory using synthetic ODE/PDE benchmarks.", "questions": "I am open to raise my score in case the authors address the following questions.\n\n* Can the authors provide further intuition on how their study applies to more common architectures as neural networks? In principle the approach of solving a least squares problem to find the estimator is similar to the idea behind extreme learning machines (which also exists in the physics-informed version). Can the theory in your article be applied to such a case?\n* What is the practical outcome of Theorem 3.5? Can the authors clarify, based on the aforementioned theorem, what should be good practices for building/training physics-informed models?\n* For the case of PINNs, the training is typically performed on a dense set of collocation points, in order to cover properly the whole training domain. How relevant is ensuring good generalization in these scenarios?", "limitations": "The authors addressed the limitations of the paper.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "I consider the following as key strengths of the paper:\n* The central insight of the study is quite interesting and can be used to explain several performance issues of physics-informed models on specific PDEs (as Navier-Stokes).\n* The paper connects concepts from statistical learning theory and algebraic geometry in very intuitive and elegant way.\n* The paper is quite clear despite the intrinsic complexity of the theoretical framework\n\nHowever, I feel the paper has two major flaws:\n* The majority of the discussions in the paper are focusing on the PILR which is linear. This hinder the applicability of the study to architecture as PINNs, or other more common architectures in the physics-informed machine learning field.\n* While the theoretical outcome is interesting per-se, I feel that the paper lacks some practical insights based on the theory.", "was_revised": true, "final_justification": "The discussion with the authors shed a positive light on their already good submission. I recommend acceptance for the paper.", "rebuttal": "We greatly appreciate the time and expertise you invested in reviewing our work. Your comments provided important guidance for our revisions.\n\n> Q1: Can the authors provide further intuition on how their study applies to more common architectures as neural networks? In principle the approach of solving a least squares problem to find the estimator is similar to the idea behind extreme learning machines (which also exists in the physics-informed version). Can the theory in your article be applied to such a case?\n\nWe thank the reviewer for this insightful question.\n\nAs the reviewer correctly notes, our theory applies directly to Extreme Learning Machines (ELM) or random feature models. In these cases, the weights of the final layer correspond to the vector $w$ in our analysis.\n\nExtending the theory to standard multi-layer Neural Networks (NNs) is more challenging. In principle, for NNs with polynomial activations (which can approximate smooth activation functions like tanh), one could derive an inequality similar to our Theorem 3.5, as outlined in the proof strategy at the end of this response. However, we identify two significant challenges at this stage:\n\n* The system of equations defining the affine variety $V$ would consist of highly complex, high-degree polynomials. This makes the numerical computation of $d\\_V$ computationally prohibitive, thereby hindering both theoretical validation and practical application.\n\n* Recent research indicates that the generalization of over-parameterized models like NNs is influenced not only by the geometry of the hypothesis space but also significantly by the implicit bias of the optimization algorithm. Consequently, a bound like our Theorem 3.5, which solely considers the geometry of the hypothesis space, may exhibit a significant gap from empirically observed performance.\n\n> Q2: What is the practical outcome of Theorem 3.5? Can the authors clarify, based on the aforementioned theorem, what should be good practices for building/training physics-informed models?\n\nThe primary practical outcome of Theorem 3.5 is the use of the affine variety dimension, $d_V$, as a guiding principle for the efficient selection of basis/trial functions.\n\nFor instance, in large spatio-temporal domains where using a dense set of collocation points (a common practice in PINNs) is computationally infeasible, $d_V$ can serve as a metric to guide an effective and tractable placement of these points prior to training.\n\nAs a promising future direction, our framework suggests an alternative to methods like weak PINNs where trial functions are learned simultaneously with the main solution. Instead, one could first optimize the set of trial functions by explicitly minimizing $d_V$ before the main training process begins.\n\n> Q3: For the case of PINNs, the training is typically performed on a dense set of collocation points, in order to cover properly the whole training domain. How relevant is ensuring good generalization in these scenarios?\n\nOur theory is applicable regardless of whether the set of collocation points is dense or sparse. It provides a formal upper bound on the generalization error and helps determine the amount of training data, $n$, required to guarantee a desired performance level.\n\nAs we state on lines 188-194, when the collocation points are sufficiently dense, we expect the information from the differential operator $\\mathscr{D}$ to be maximally utilized. In our framework, this corresponds to a configuration that minimizes the intrinsic dimension of the solution space, i.e., $d\\_{\\\\mathcal{V}} = \\\\min\\_{\\\\mathcal{T}} \\\\dim(\\\\mathcal{V}(\\\\mathscr{D}, \\\\mathcal{B},\\\\mathcal{T}))$. \n\n---\n\n## Outline of the Proof Strategy for NNs \n\nThe goal is to derive a generalization bound similar to our Theorem 3.5 for the hypothesis space $\\mathcal{H}\\_L$ of an NN by controlling its Rademacher complexity. While the overall framework is similar to that for linear models, we introduce a key extension to handle the non-linear, hierarchical structure of NNs.\n\n* **Step 1: Reduction from Loss Class to Hypothesis Class**\nFollowing standard learning theory, the generalization error is bounded by the Rademacher complexity of the loss class, $\\mathfrak{R}(\\ell \\circ \\mathcal{H}\\_L)$. We then apply the Contraction Principle, leveraging the Lipschitz property of the loss function, reducing the problem to bounding the Rademacher complexity of the hypothesis class itself, $\\mathfrak{R}(\\mathcal{H}\\_L)$. This step is common to both linear models and NNs.\n\n* **Step 2: Bounding Hypothesis Complexity via Dudley's Integral**\nNext, we bound  $\\mathfrak{R}(\\mathcal{H}\\_L)$ using Dudley's integral theorem. This theorem bounds the complexity via an integral over the covering numbers of the hypothesis space, $\\mathcal{N}(\\mathcal{H}\\_L, \\varepsilon, \\\\|\\cdot\\\\|\\_{\\infty})$. This step represents the key departure from the analysis of linear models.\n\n* **Step 3: Bounding the NN's Covering Number**\nDirectly evaluating the covering number for a complex function space like that of an NN is often intractable. Our approach is to bound the covering number of the function space by that of the more tractable parameter space.\n   * **Stability Analysis:** Through a recursive, layer-by-layer stability analysis, we relate distances in the parameter space (weights) to distances in the function space. Specifically, we show that if two parameter vectors $w$ and $w'$ are $\\varepsilon$-close, their corresponding functions $f\\_w$ and $f\\_{w'}$ are bounded by $\\\\|f\\_w - f\\_{w'}\\\\|\\_{\\\\infty} \\\\le \\\\ell\\_L \\\\varepsilon$. The stability constant $\\\\ell\\_L$ depends on the network's architecture (e.g., depth $L$ and weight norm $R$).\n    * **Covering Number Relation:** This stability analysis yields the critical inequality $\\\\mathcal{N}(\\\\mathcal{H}\\_L, \\\\ell\\_L \\\\varepsilon, \\\\|\\\\cdot\\\\|\\_{\\\\infty}) \\\\le \\\\mathcal{N}(V\\_R, \\\\varepsilon, \\\\|\\\\cdot\\\\|\\_2)$, which bounds the function space's covering number by the parameter space's.\n\n* **Step 4: Leveraging the Geometry of the Parameter Space**\nThis reduces the problem to bounding the covering number of the parameter set $\\mathcal{V}\\_R$ by applying the Lemma B.2. \n\n* **Step 5: Final Assembly**\nFinally, we substitute the covering number bound from Step 4 into the Dudley integral from Step 2 (accounting for the scaling by the stability constant $\\ell_L$). Solving this integral yields a concrete upper bound on $\\mathfrak{R}(\\mathcal{H}\\_L)$ in terms of $d\\_V$ and $\\ell\\_L$. This result is then used in the generalization inequality from Step 1 to complete the proof of the final theorem for PINN (similar results of Theorem 3.5).\n\n## Additional Key Assumptions\nThis extension requires two key assumptions:\n* Lipschitz continuity and boundedness of the activation functions.\n* A norm constraint on each layer's weight matrix."}]

Author rebuttal:

Meta-review: After discussion, all reviewers centered on a positive assessment. Two main weaknesses were identified:

1. The paper made a misleading claim about not requiring knowledge of the PDE. The AC’s opinion is that this is, in general, a common criticism of PINN-type methods and not unique to this paper. However, the authors should directly address this concern in the revised manuscript and make sure to clarify what aspects of the problem need to be known for the proposed method. 
2. The paper is limited to only some types of physics informed models. This is also not disqualifying.

All authors ultimately concurred that the paper would be a valuable contribution to the field. Thus, the AC finds no reason to overturn consensus and recommends accept.
Decision: Accept (poster)