Refined generalization analysis of the Deep Ritz Method and Physics-Informed Neural Networks
Authors: Xianliang Xu
Ye Li
Zhongyi Huang
Venue: ICML 2025 poster (2025)
Keywords: Deep Ritz Method
Physics-Informed Neural Networks
Fast Rates
Generalization Analysis
Primary area: theory->learning_theory
TL;DR: In this paper, we present refined generalization bounds for the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).

Abstract: In this paper, we derive refined generalization bounds for the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs). For the DRM, we focus on two prototype elliptic partial differential equations (PDEs): Poisson equation and static Schrödinger equation on the 
$d$-dimensional unit hypercube with the Neumann boundary condition. Furthermore, sharper generalization bounds are derived based on the localization techniques under the assumptions that the exact solutions of the PDEs lie in the Barron spaces or the general Sobolev spaces. For the PINNs, we investigate the general linear second order elliptic PDEs with Dirichlet boundary condition using the local Rademacher complexity in the multi-task learning setting. Finally, we discuss the generalization error in the setting of over-parameterization when solutions of PDEs belong to Barron space.

Refined Generalization Analysis of the Deep Ritz Method and Physics-Informet
Neural Networks
Xianliang Xu Ye Li234
Zhongyi Huang!
Abstract
cessing and reinforcement learning have sparked interest in applying machine learning techniques to solve various types of PDEs. In fact, the idea of using machine learning to solve PDEs dates back to the last century (Lagaris et al., 1998), but it has recently gained renewed attention due to the significant advancements in hardware technology and the algorithm development.
In this paper, we derive refined generalization bounds for the Deep Ritz Method (DRM) and Physics-Infommed Neural Networks (PINNs). For the DRM, we focus on two prototype elliptic par- tial differential equations (PDEs): Poisson equa- tion and static Schrödinger equation on the d- dimensional unit hypercube with the Neumann boundary condition. Furthermore, sharper gen- eralization bounds are derived based on the lo- calization techniques under the assumptions that the exact solutions of the PDEs lie in the Bar- ron spaces or the general Sobolev spaces. For the PINNs, we investigate the general linear sec- ond order elliptic PDEs with Dirichlet boundary condition using the local Rademacher complex- ity in the multi-task learning setting. Finally, we discuss the generalization error in the setting of vee-parramttrrzzatoon when solutions of PDEs belong to Barron space.
There are numerous methods proposed to solve PDEs using neural networks. One popular method, known as PINNs (Raissi et al., 2019), utilizes neural network to represent the solution and enforces the neural network to satisfy the PDE constraints, initial conditions and boundary conditions by encoding these conditions into the loss function. The flexibility and scalability of the PINNs make it a widely used framework for addressing PDE-related problems. The Deep Ritz method (Yu et al., 2018), on the other hand, incor- porates the variational formulation into training the neural networks due to the widespread use of the variational formu- lation in traditional methods. In comparison to PINNs, the form DRM has a lower derivative order, but the fact that not all PDEs have variational forms limits its applications. Both methods hinge on the approximation ability of the deep neural networks.
1. Introduction
Partial Differential Equations (PDEs) play a pivotal role in modeling phenomena across physics, biology and engi- neering. However, solving PDEs numerically has been a longstanding challenge in scientific computing. Classical nu- merical methods like finite difference, finite element, finite volume and spectral methods may suffer from the curse of dimensionality when dealing with high- dimenssonn PDEs. Recent years, the remarkable successes of deep learning in diverse fields like computer vision, natural language pro-
The approximation power of feed-forward neural networks (FNNss with diverse activation functions has been stud- ied for different types of functions, including smooth func- tions (Lu et al., 2021a), continuous functions (Shen et al., 2022), Sobolev functions (Belomestny et al., 2023; Yang et al., 2023b;a; Yarotsky, 2017), Barron functions (Barron, 1993). It was proven in the last century that sufficiently large neural network can approximate target function in a certain function class with any given tolerance. Specif- ically, it has been shown in Hornik (1991) that the two- layer neural network with ReLU activation function is a universal approximator for continuous functions. More re- cently, specific approximate rate of neural networks has been shown for different function classes in terms of depth and width. Lu et al. (2021a) showed that a ReLU FNN with width ONN log N) and depth O log L) can achieve ap- proximation rate (N 1,2//d) for the function class C*([o, 1]d) in the _ norm, which is nearly optimal. In the context of applying neural networks to solve PDEs, the focus shifts to the approximation rates in the Sobolev norms.
DDepartment of Mathematical Sciences, Tsinghua University, Beijing, China "College of Computer Science and Technology, Nanjing University of Acronautics and Astronautics, Nanjing, China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China "State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China. Cor- respondence to: Ye Li <yeli20@ www.eeuaaaccccoo edu.en. Zhongyi Huang <zhongyih@@ue Srnpiih@ttsinn tsiighua.cdu..co edu.en>.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Refined Generalization Analysis of the DRM and PINNS
Belomestny et al. (2023) utilized multivariate spline de- rive the required depth, width, and sparsity of a ReL.2² deep neural network to approximate any Hölder smooth function in Hölder norms with the given approximation error. And the weights of the neural network are also con- trolled, which is essential to derive generalization error. Yang et al. (2023b) derived the nearly optimal approxima- tion results of deep neural networks in Sobolev spaces with Sobolev norms. Specifically, deep ReLU neural networks with width (N log N) and depth (o L) can achieve approximation rate tions in w ((0, 1)d) with W.1,00 norm. For higher or- der approximation in Sobolev spaces, Yang et al. (2023a) introduced deep super ReLU networks for approximating functions in Sobolev spaces under Sobolev norms Wmp for m E N with m 2 2. The optimality was also estab- lished by estimating the VC-dimension of the function class consisting of higher-order derivatives of deep super ReI.U networks.
2022), Deep Finite Volume Method (Cen & Zou, 2024), and forth.
Fast rates in machine learning: In statistical learning, the excess risk is expressed as the form ( (CoMP,,(F) ( where is the sample size, COMP,, n. (F) measures the complexity of the function class F and E [ 1] represents the learning rate. The slow learning rate 1 (a = 5) can be easily derived by invoking Rademacher complexity (Bartlett & Mendelson, 2002), but achieving the fast rate 1. (a 1) is much more challenging. Based on localization tech- niques, the local Rademacher complexity (Bartlett et al., 2005; Koltchinskii, 2006; 2011) was introduced to statisti- cal learning and has become a popular tool to derive fast rates. It has been successfully applied across a variety of tasks, like clustering (Li & Liu, 2021), learning kernels (Cortes et al., 2013), multi-task learning (Yousef et al., 2018), empirical variance minimization (Belomestny et al., 2017), among others. Variants of Rademacher complexity, such as shifted Rademacher complexity (Zhivotovskiy & Hanneke, 2018) and offset Rademacher complexity (Liang et al., 2015), also offer potential direction for achieving the fast rates (Duan et al., 2023; Kanade et al., 2022; Yang et al., 2019).
n
Vn
for func-
In this work, we focus on the DRM and PINNs, aiming to derive sharper generalization bounds. Compared to Jiao et al. (2021); Duan et al. (2021), the localized analysis utilized in this paper leads to improved generalization bounds. We believe that this study provides unified framework for deriving generalization bounds for methods that solve PDEs involving machine learning.
Generalization bounds for machine learning based PDE solvers: Based on the probabilistic space filling arguments (Calder, 2019), Shin et al. (2020) demonstrated the con- sistency of PINNS for the linear second order elliptic and parabolic type PDEs. An abstract framework was introduced in Mishra & Molinaro (2022) and stability properties of the underlying PDEs were leveraged to derive upper bounds on the generalization error of PINNs. Following similar meth- ods widely used in machine learning for deriving generaliza- tion bounds, the convergence rate of PINNs was derived in Jiao et al. (2021) by decomposing the error and estimating related Rademacher complexity. For the DRM, when the solutions are in the spectral Barron space, Lu et al. (2021c) demonstrated the generalization error bounds of two-layer neural networks for solving the Poisson equation and static Schrödinger equation, but in expectation and with the slow rates. When solutions of the PDEs fall in general Sobolev spaces, Duan et al. (2021) established non-asymptotic con- vergence rate for DRM using a method similar to Jiao et al. (2021). The most relevant work to ours is Lu et al. (2021b), which used peeling methods to derive sharper generalization bounds of the DRM and PINNs for the Schrödinger equa- tion on a hypercube with zero Dirichlet boundary condition. However, Lu et al. (2021b) assumed that the function class of neural networks is a subset of Hj. which challenging to achieve. For the DRM, the peeling method in Lu et al. (2021b) cannot be applied to derive the generalization error of the Poisson equation, as in this scenario, the population loss isn't 't the expectation of the empirical loss. For PINNs, Lu et al. (2021b) required strong convexity and only con-
1.1. Related Works
Deep learning based PDE solvers: Solving high- dimensional PDEs has been a long-standing challenge in scientific computing due to the curse of dimensionality. In- spired by the ability and flexibility of neural networks for representing high dimensional functions, numerous studies have focused on developing efficient deep learning-based PDE solvers. In recent years, the PINNs have emerged as flexible framework for addressing problems related to PDEs and have achieved impressive results in numerous tasks. Despite their success, there are areas where further improvements can be made, such as developing better op- timization targets (Chiu et al., 2022) and neural network architectures (Ren al., 2022; Zhang et al., 2020). Inspired by the use of weak formulation in traditional solvers, Zang et al. (2020) proposed to solve the weak formulation of PDEs via an adversarial network and the DRM (Yu et al., 2018) trains a neural network to minimize the variational formulations of PDEs. By reformulating the parabolic PDEs as backward stochastic differential equations, (Han et al., 2018) introduced deep soariing-basedd approach that can handle general high dimensional parabolic PDEs and similar method has been used for high dimensional eigenvalue prob- lems (Han et al., 2020). In addition, there are other methods that combine traditional techniques with deep learning, such as Deep Least- Squares Methods (Cai et al., 2020; Lyu et al.,
2

---

Refined Generalization Analysis of the DRM and PINNS
sidered the static Schrödinger equation with zero Dirichlet boundary conditions, which results in the loss function of PINNs comprising solely the interior term, In contrast, our approach does not require strong convexity and is applicable to general linear second-order elliptic PDEs.
share similar forms with PINNs.
Moreover, we investigate the complexity of over- parameterized two-layer neural networks when approx- imating functions in Barron space, and demonstrate meaningful generalization errors in the setting of over- parameterization. Additionally, in the Discussion Sec- tion, we explore other boundary conditions for the Deep Ritz. Method
1.2. Contributions
For the aspect of approximation via neural networks, we show that the functions in (92) can be well ap- proximated in the H norm by two-layer ReLU neural networks with controlled weights, and similar results are also presented for functions in (² in the H² norm. Compared to the results in Lu et al. (2021c), our approximation rate is faster and the Barron space in our setting is larger than the spectral Barron space in Lu et al. (2021c). Compared with other approxima- tion results for Barron functions (Siegel & Xu, 2022a; Siegel, 2023), the constant in our result is independent of the dimension.
1.3. Notation
For E Rd, |æ|p denotes its p-norm and we use |x| as shorthand for |z|2. We denote the inner product of vectors 2, y E IR" by x y. For the d-dimensional ball with radius r in the p-norm and the boundary of this ball, we denote them by (r) and (r) respectively. For a set F that a sub- set of metric space with metric d, use MVF d, e) to de- note its covering number with given radius and the metric d. For given probability measure P and sequence of random variables {X.)== distributed according to P. we denote the empirical measure of P by Pn,. i.c. Pa = E 5x,-
For the DRM, we derive sharper generalization bounds for the Poisson equation and Schrödinger equation with Neumann boundary condition, regardless of whether the solutions fall in Barron spaces or Sobolev spaces. Our methods rely on the strongly convex property of the variational form and a new localized analysis. For the Poisson equation, the expectation of empirical loss is not equal to the variational formulation, which complicates the analysis. Additionally, for the static Schrödinger equation, the strongly convex property cannot be simply regarded the Bernstein condition in Bartlett et al. (2005), as the solutions of the PDEs often do not belong to the function class of neural networks in our setting. After applying a novel error decomposition technique, we are able to utilize the lo- cal Rademacher complexity to derive sharper bounds.
For the activation functions, we write o (x) for the ReLU activation function, i.e., (x) := (max(0, x))² And we use a for a for simplicity. Given a domain Ω C TRdd, we denote |St] and [aS2] the measure of ft and its boundary 0N respectively.
2. Deep Ritz Method
2.1. Set Up
Let Ω = (0, 1)d be the unit hypercube on Rd and in be the boundary of Ω. We consider the Poisson equation and static Schrödinger equation on with Neumann boundary condition.
Poisson equation:
For the PINNs, we regard this framework as a see- nario within multi-task learning (MTL). At this time, there are two key points: one is that the loss functions are non-negative and the other one is that non-exact oracle inequality suffices. To achieve our goal, we extend the entropy method to derive a Talagrand-type concentration inequality for MTL, which offers better constants than those provided by Theorem in Yousefi et al. (2018). Consequently, similar results to those in single-task setting can be established, yielding a non- exact oracle inequality tailored for PINNs. Unlike Lu et al. (2021b), which required the strong convexity, our approach does not impose this requirement. While we have only presented results for the linear second order elliptic equations with Dirichlet boundary conditions, our method can serve as a framework for PINNs for a wide range of PDEs, as well as other methods that
Ou
Au = f in S2 dv
on Mn.
(I)
Static Schrödinger equation:
Ou
- -Au + Vu = f in S
dv = 0 on M.
In this section, we follow the framework established in Lu et al. (2021c), which characterizes the solutions through variational formulations. For completeness, the detailed results are presented as follows.
Proposition 2.1 (Proposition 1 in Lu et al. (2021c)) ).
(1) Assume that f E L (22 with Jg fdr = 0. Then there ex- ists a unique weak solution upp E H = {u E H (9)) Ja udr = 0} to the Poisson equation. Moreover, we have
3

---

Refined Generalization Analysis of the DRM and PINNS
that
and
up- arg min Ep(
E En,5(u)
(Q)
S/ + (Sa udx) -2 2 L sude
= arg min (EH
(8)
(3)
where we write En and &, for the empirical losses of the Poisson equation and static Schrödinger equation respec- tively. Note that the expectation of P,, is not equal to Ep(u), which restricts the applicability of common meth- ods, such as local Rademacher complexity, in deriving a fast generalization rate for the Poisson equation.
and that for any u E.EEE.. (22),
Eplu) - -Ep(up)) s I|- (iplir Cp(Ep(u)- - Ep())))) (4)
where Cp = m2x + 1,2] and op is the Poincaré constant on the domain Ω
(2) Assume that f. V.V. (St) and that < Vmii (xx) S Vimax < 0o for all x E Qt and some constants V Vmnn and Vmaa- Then there exists unique weak solution Mis E (22) to the static Schrödinger equation. Moreover, we have that
2.2. Main results
The aim of this section is to establish a framework for de- riving improved generalization bounds for the DRM. In the setting where the solutions lie in the Barron space B (2)), we demonstrate that the generalization error between the empirical solutions from minimizing the empirical losses and the exact solutions grows polynomially with the under- lying dimension, enabling the DRM to overcome the curse of dimensionality in this context when the optimization error is omitted. Furthermore, when the solutions fall in the gen- eral Sobolev spaces, we provide tight generalization bounds through the localization analysis.
ug-- arg min (u) (C)((())
(5)
(AA
V//u// -2 fuday IVuPP+Vluudd
arg min
52
and that for any u E 11 (9)).
Es(u) - Es(ug) Es(u)-Es(ug) I|u ugg (0) S min(1, Vmni)
max(1, Vmna)
(6)
We begin by presenting the definition of the Barron space, as introduced in Barron (1993).
Throughout the paper, we assume that f E L and V E (I) with 0 < Vmin I V(x) Vinax 00. The boundedness is essential in our method for deriving fast rates and it also leads to the strongly convex property in Proposition 2.1 (2). There are also some methods for deriv- ing generalization error beyond boundedness, as discussed in Mendelson (2015; 2018); Lecué & Mendelson (2013). However, these approaches often require additional assump- tions, such as specific properties of the data distributions or function classes, which can be difficult to verify in practice.
(9)) = ( :0 - C:
1 == inf / (1 (1+ [++WW))))( < oo), fo|0==f J
(9)
where the infimum is over extensions Se E L (Rd) and fe is the Fourier transform of fe- Note that we choose 1-norm for in the definition just for simplicity.
There are also several different definitions of Barron space (Ma et al., 2022) and the relationships between them have been studied in Siegel & Xu (2023). The most important property of functions in the Barron space is that those func- tions be efficiently approximated by two-layer neural networks without the curse of dimensionality. It has been shown in Barron (1993) that two-layer neural networks with sigmoidal activation functions can achieve approxi- mation rate mm) under the L norm, where m is the number of neurons. And the results have been extended to the Sobolev norms (Siegel & Xu, 2022a;b). However, some constants in these extensions implicitly depend on the dimension and there is possibility that the weights may be unbounded. To address these concerns, we demon- strate the approximation results for functions in the Barron space under the H' norm. Additionally, for completeness, the approximation result in W (12) with W norm is also presented, which was originally derived in Yang et al.
The core concept of DRM involves substituting the func- tion class of neural networks for Sobolev spaces and then training the neural networks to minimize the variational formulations. Subsequently, we can employ Monte-Carlo method to compute the high-dimensional integrals, as tradi- tional quadrature methods are constrained by the curse of dimensionality in this context.
Let (X be an i.i.d. sequence of random variables distributed uniformly in Ω. As in our setting, the volume of Q is 1, thus the empirical losses can be written directly as ((U)
1 Σ ([Vu(X)))= 2((x))(X.)) +
1
Σ
n
up (X,)
2
4

---

Refined Generalization Analysis of the DRM and PINNS
(2023b).
with probability as least e
Proposition 2.2 (Approximation results in the H' norm).
Ey(un,r) - Er(u7.)
(1) Barron space: For any f E (9)), there exists a two- such that
md mollgg
1+ E
(12)
layer neural network fm E
I CM2 log M
+
n
If 11-mm Smll (1)) S cllfllss(o)mmm (+++)) (3+1))
where C is a universal constant and M is the upper bound (10)
for [|f lu]. (l
By taking m - (ā) 200ai1) we have
m
where 1m, = EΣ 1=1
no x +tt)) |0n/11 1,t, E
Ep(un,p) - Ep(utp)
11,,),, Σ hul S BB) positive constant B and e is universal constant.
s SCM² log. M ((
2034417
t
(13)
log T2 +
(2) Sobolev space: For any f E ( with 2
2 and [l S 1, any N, L E N4, there exists a
(2) For up D(N, L, B Iup (l)) then with probability at
Wh,, taking F
ReLU neural network o with the width (34+d)2* (N +
1) log.(88) and depth 16 + 1) 1og2 (4L) such that
least 1 - c-
[I(()) ( -(=)))/))))) S C(k, 2(1, (11)
Ep(un,r) - Ep(up)
c (NI) log L)3 + (NL) (NL)-4(A-1)) 1(kk-))/d + t
where C(k, d) is the constant independent with N, L.
n
(14)
Remark 2.3. When approximation functions in (S)), our derived bound exhibits a faster rate than the bound of m- presented in Xu (2020). Although our bound slower than the bound m (++ ) shown in Siegel & Xu (2022a), it is important to note that the constant within the approxi- mation rate of Siegel & Xu (2022a) may depend exponen- tially on the dimension and the weights of two-layer neural contrast, the
where n 2 C(NL) N log L)s and C is a constant independent of N, L,n.
By taking N = L = n 11444((1111, we have
(l t
- -Ep(p) S SC -42
(15)
network could potentially be unbounded. constant in our approximation is dimension- iinriiinennneeeeeentt and the weights also applicable to the differently defined Barron spaces in Ma et al. (2022), yielding approximation result similar to that in Proposition 2.2 (I).
The generalization error for the static Schrödinger equa- tion shares a similar form with that in Theorem 2.4, but the proof methodology differs. Although the method for the Poisson equation is also applicable to the static Schrödinger equation, it is quite complicated. Unlike (7), it can be seen from (8) that the expectation of the empirical loss for the static Schrödinger equation is the energy functional (5) in Proposition 2.1 (2). This allows the use of local Rademacher complexity after employing new error decomposition ap- proach.
controlled. Moreover, our method is
For the convenience of expression, we write ((N, L, B) for the function class of ReLU neural networks in Proposition 2.2 (2) with width (34+ + kd (N + +1) log,(8 depth 22 + 1) log, (47.) and 11 norm bounded by B such that the approximation result in Proposition 2.2 (2) holds for any f W)),x (Ω) with [lfllw).>>)) S 1.
Theorem 2.5. Let u's solve the static Schrödinger and Wn.S be the minimizer of the empirical loss En,s in the function class F.
With the approximation results above, we can derive the generalization error for the Poisson equation and the static Schrödinger equation through the localized analysis.
(1) For uis E B taking F = Fm, (5llu3 (l)))))) then with probability as least 1 - -t
Theorem 2.4 (Generalization error for the Poisson equation). Let upp E H!!()) solve the Poisson equation and ua be the minimizer of the empirical loss En,P in the function class F.
Es(un,s) Es(ug) S CM2 md log
2
(16)
where C is a universal constant and M is the upper bound for I|ffllz>, | ( [|
(o)), (I) For up E (S),, taking F Fm, (5]lu² (6)lbb//)))))))))))) then
5

---

Refined Generalization Analysis of the DRM and PINNS
By taking m = (7) 2(58+3), we have
where S indicates a constant depending only on the upper bound M defined in Theorem 2.4 (1) is omitted.
Es(un,s)- - -Es( S CM2 ((1)
Bd12
Remark 2.8. Due to the equivalence between H -eroor and the energy excess as shown in Proposition 2.1, we are able to deduce the generalization error for both the Poisson equation and the static Schrödinger equation under the H norm. For example, one can derive that for the Poisson equation, if up E (92), then
log nt +
(17)
(2) For Ug E Wh.,()) (I)), taking FF
(N,, L, Bllug Blluggllwt. (1)) then with probability at least 1 - e
3++2
Es(un,s) - {s(ss)
|| up (1)) S SCM M
(#)
1 12 log n
+
(lo)) N log L)3 + (NL) 4(k- 11/d
1)
(22)
SC
n
n
(18)
3. Physics-Informed Neural Networks
where 2 C(NL)) (log N log L) and C' is a constant independent of N,L. n.
3.1. Set Up
In this section, we will consider the following linear second order elliptic equation with Dirichlet boundary condition.
By taking N- L n m we have
Es(um,s) - Es(gg) S C
n 2422 (log n) 6
1
d
Ois + 1-=1
b,ú,u - +cu = f, in Ω
(19)
(23)
Remark 2.6. By utilizing the strong convexity of the en- ergy functional and localized analysis, we improve the con-
u g. on an, where aij E C(Q)), bi, E (P)) 9 E 1 (00) and Ω C (0, 1)d is an open bounded domain with properly smooth boundary.
vergence rate n 2 as shown in Duan et al. (2021) to n - Furthermore, when the solution belongs to 3d42 (92), our convergence rate (5) 2وअرتيy is faster than n &
In the framework of PINNs, we train the neural network u with the following loss function.
in Lu et al. (2021c) and explicitly demonstrates its depen- dency on the dimension.
/ [ag(t)0)u(()) + Σ (c()))(((x) C(u) = Ja iJ=1
In the setting of over-parameterizzaionn (i.e. is large enough), the generalization bounds in (14) and (16) become meaningless. Fortunately, the function class of two-layer neural networks in Proposition 2.2 (1) forms a convex hull of a function class with a covering number similar to that of VC-classes. Consequently, we can extend the convex hull entropy theorem (Theorem 2.6.9 in (Vaart & Wellner, 2023)) to the H norm, enabling us to derive meaningful generalization bounds in this setting.
+ +c(x)u(x)= (c)) + 1 (u(v) - g(u))²
(24)
By employing the Monte Carlo method, the empirical ver- sion of L can be written as
CN
Proposition 2.7. Under the same settings in Theorem 2.4 (1) and Theorem 2.5 (1), we have that
Ω N
ai (xx)0x0,, +
Σ ((xx)0Ou((N)
(I) with probability at least 1 - et
f( (2xx)) + Jan
N2 (u(Yi) - =
+ c((xx))(XX (Xx) -
S (d J+
(
3d4T
1
1
2(3411)
N₂
(25)
71
(20)
where N (N), { N1 and {= are i..d ran- dom variables distributed according to the uniform distribu- tion (()) on Ω and ((a2) on O. respectively.
N₂
1+ 3/
m
n
Given the empirical loss LN. the empirical minimization algorithm aims to seek un which minimizes Lx that is:
(2) with probability least 1 - -t
Es(un,s) S a
1
+
1
1+ $M
t
uN E arg mig (x,,),
NEF
m
(21)
where F is parameterized hypothesis function class.
6

---

Refined Generalization Analysis of the DRM and PINNS
3.2. Main Results
to indicate the class of functions consisting of vector-valued functions.
We begin by presenting the approximation results in the !!² norm.
To derive sharper generalization bounds for the PINNs, we require results from the field of MTL, with a core component being the Talagrand-type concentration inequality. Yousefi et al. (2018) has established a Talagrand-type inequality for MTL. Of independent interest, we provide a new proof based the entropy method. Moreover, the concentration inequality derived from this method yields better constants compared to those offered by Theorem in Yousef et al. (2018).
Proposition 3.1 (Approximation results in the H² norm).
(I) Barron space: For any f E (9), there exists a two- layer neural network fm E fm.. such that
fm-llllera0 s scllflbaa +)+
(26)
where 2 = ( noz(w: + 1.):
|4s|1 1,4 E
Theorem 3.2. Let F = (f == (J1,".., .T)) be a class of vector-valued functions satisfying max sup lft(z)) S ISTST ZER,
[1,, Σ hil S for positive constant B and e is universal constant.
b. Also assume that X == (X) (T.N.) (1..1)))11)) is a vector of E'= Nr. independent random variables. Let (otha
(2) Sobolev space: For any f E WWk,00 with any integer K3 2 2, there exists some sparse ReLU neu- ral network Φ E (LL, W B; H) with L (1), W O(K4) O(K)) B 1, = O(1), such that
and
be a sequence of independent Rademacher variables. If
T
+ sup Σ Var(fe(Xt)) S then for every & > 0, with 1EF
probability at least 1 c=
C
11((x) - (()))(())))))) S Kk-2
(27)
sup(Pf- - (Pff-NN) PNff) FEF
sup
where C is a constant independent of K, ((L,W,S, H) denote the function class of R neural networks with depth L, width W and at most S non-zero weights taking their values in [-B, B). Moreover, the W2200 norms of functions in (L, W,S, B; H) have the upper bound H.
S inf a>0
((1+
ET nT
+
1 (1
bx
nT
(29)
where n = minisIsT No and the multi-task Rademacher complexity of function class F is defined as
The framework of PINNs can be regarded as form of multi- task learning (MTL), as a single neural network is designed to simultaneously learn multiple related tasks, involving the enforcement of physical laws and constraints within the learning process. In contrast to traditional single-task learning, MTL encompasses T supervised learning tasks sampled from the input-output space X1 x x111..... ... .y x Y/r respectively. Each task is represented by an independent random vector (Xe, Yi) distributed according a probabil- ity distribution 14.
1 T
R(F) = Ex,
sup f
(30)
Moreover, the same bound also holds for ye (PN Pf).
Remark 3.3. In comparison with the concentration inequal- ity provided in Yousefi et al. (2018), which is stated
SET nf
12bx
sup - -N < (R(F) feF
(31)
nf
Before presenting our results, we first introduce some no- N: tations. Let (X; y be sequence of i.i.d. random samples drawn from the distribution 14, for f = 1,...,77 For any vector-valued function f = (fi,-.. fr), we de- note its expectation and its empirical part as
)==
our result exhibits improved constants by taking 1.
Note that the loss functions of the PINNs are all non- negative, which facilitates the derivation of analogous re- sults to those obtained in the single-task context. With the results in MTL, the generalization error for the PINNs can be established.
T PS: = T E Ph. PN = Σ Pr.
T
(28)
Theorem 3.4 (Generalization error for PINN loss of the linear second order elliptic equation).
where N. = (Ni,... Nr), Pft = [X.) and L EN f((X)). We denote the component- Pi = N. wise exponentiation of f as f = (ff.... ..... .f) for any a E R. In the following, we use bold lowercase letters to represent vector-valued functions and bold uppercase letters
Let u² be the solution of the linear second order elliptic equation and n = min(N1, (NN,NNg Nz).
(1) 1f u² E B3)), taking F = Fm, (1 then
7

---

Refined Generalization Analysis of the DRM and PINNS
with probability at least 1
e
Therefore, under the settings of Theorem 3.4, we can obtain the generalization error for the linear second order elliptic equation in the H 2 norm, Note that here we require the second-order elliptic equation (23) to satisfy the strong el- lipticity condition and the boundary to possess a certain degree of smoothness. For error estimates similar to (36) for a broader class of PDEs, see reference Zeinhofer et al. (2024).
[(uN) S SCC() m log
1 m
3/
t
+
n
(32)
where C. is a universal constant and (22 max{dF M2, C(Tr, S), 22 +00||M²)) ((Tr,)) is the constant in the Trace theorem for Ω
=
By taking m = n 3, we have
3d
For the PINNs, we only focus on the L loss, considered in the original study (Raissi et al., 2019). Actually, the design of the loss function should incorporate some priori estimation, which serves as a form of stability property (Wang et al., 2022). Specifically, the design of the loss function should follow the principle that if the loss of PINNs [(u) is small for some function u, then u should be close to the true solution under some appropriate norm. For instance, Theorem 1.2.19 in Garroni & Menaldi (2002) shows that, under some suitable conditions for domain Si and related functions dij, bi, e, f g. the solution u² of the linear second order elliptic equation satisfies that
[(uN) S M)) ((
2(30+17
t log
-
(33)
11
(2) If v² (h) for k. 3. taking F SL W, B; with L O W = O(K)), =
O(K"), B 1, H- = O(1), then with probubility at least 1-e
Koloog K + logn)
2k
+ 1
((UN)
SC C
+
n
K
71
(34)
|| ([))())) SC (11fIl200) + ll9ll 1r (00)))
(37)
where C is a constant independent of K, N.
By taking K= = n 2 we have
Thus, if we apply the loss
#)
[(u) = Luu Ml + 1|u gll
(38)
[(uN) SC n 1 logn n + n2
(35)
H a (an))
we may obtain the generalization error in the H² norm. However, this term |lsl| H (ag) is challenging to compute because it also requires ensuring Lipschitz continuity with respect to the parameters, which is essential for estimating the covering number. We leave this as a direction for fu- ture work. On the other hand, some variants of PINNs do not fit the standard MTL framework. For instance, within the extended mmysics-informet neural networks (XPINNs) framework, to ensure continuity, samples from adjacent regions have cross-correlations. The detailed theoretical framework for XPINNs remains an area for future research. And the detailed future directions and limitations of this work are deferred to the Discussion section of the appendix.
26-4 Remark 3.5. The convergence rate n 2-26-4 is faster than n ur4x-8 presented in Jiao et al. (2021) and is same as that in Lu et al. (2021b) for the static Schrödinger equation with zero Dirichlet boundary condition. However, our result does not require the strong convexity of the objective function. More importantly, the objective function in Lu et al. (2021b) only involves one task. Moreover, our method can be ex- tended to a broader range of PDEs, as our approach does not impose stringent requirements on the form of the PDEs.
Note that in certain cases, for instance, when Ω = (0, 1)d, the constant C(Tr, Q2) is at most d, at this time, [(ux) in Theorem 3.4 (1) only depends polynomially with the underlying dimension. Moreover, when solutions belong to Barron space, similar Proposition 2.7, can also derive generalization bounds in the prramtarameerrize setting for PINNs. These bounds are omitted here for simplicity.
4. Conclusion
In this paper. we have refined the generalization bounds for the DRM and PINNs through the localization tech- niques. For the DRM, our attention was centered on the Poisson equation and the static Schrödinger equation on the d-dimensional unit hypercube with Neumann boundary condition. As for the PINNs, our focus shifted to the gen- eral linear second elliptic PDEs with Dirichlet boundary condition. Additionally, our method is adaptable to a wider variety of PDEs, such as dependet ones, since our approach is not constrained by the form of the PDEs. In both neural networks based approaches for solving PDEs,
Although Theorem 3.4 provides a generalization error for the loss function of PINNs, it is often necessary to measure the generalization error between the empirical solution and the true solution under a certain norm. Fortunately, from Lemma C.11, we can deduce that
||ul - 12 H ( (3))
S CaN - -f 01122(0) + ||uN - =0002(00;))= = Cur((
(36)
8

---

Refined Generalization Analysis of the DRM and PINNS
considered two scenarios: when the solutions of the PDEs belong to the Barron spaces and when they belong to the Sobolev spaces. Furthermore, we believe that the methodologies established in this paper can be extended to a variety of other methods involving machine learning for solving PDEs.
Belomestny, D., Iosipoi, L.., Paris, Q. and Zhivotovskiy, N. Empirical variance minimization with applications in variance reduction and optimal control. arXiv preprint arXiv:1712.04660 2017.
Belomestny, D., Naumov, A.. Puchkin, N., and Samsonov, S. Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise- polynomial activations. Neural Networks, 161:242-25 2023.
Acknowledgments
We would like to acknowledge helpful comments from the anonymous reviewers and area chairs, which have im- proved this submission. This work was partially supported by the National Natural Science Foundation of China (No. 12025104, No. 62106103, ), and the basic research project (I2F240021A24).
Belomestny, D., Goldman, A., Naumov, A., and Samsonov, S. Theoretical guarantees for neural control variates in meme. Mathematics and Computers Simulation, 2024.
Cai, Z., Chen, J., Liu, M., and Liu, X. Deep least-squares methods: An unsupervised learning based numerical method for solving elliptic pdes. Journal of Compula- tional Physics, 420:109707, 2020.
Impact Statement
This paper presents work whose goal is to advance the field of scientific machine learning. There are many potential socictal consequences of our work, none which we feel must be specifically highlighted here.
Calder, Consistency of lipschitz learning with infinite unlabeled data and finite labeled data. SIAM Journal on Mathematics of Data Science, (4):780-812, 2019.
References
Cen, J. and Zou, Q. Deep finite volume method for partial differential equations. Journal of Computational Physics, 517:113307, 2024.
Agmon, S., Douglis, A., and Nirenberg, L. Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions. i. Com- munications pure and applied mathematics, 12(4): 623-727, 1959.
Chen, M., Jiang, H., Liao, W., and Zhao, T. Nonparametric regression on low-dimensional manifolds using deep relu networks: Function approximation and statistical recov- ery. Information and Inference: A Journal of the IMA, 11 (4):1203-1253, 2022.
Anthony, M., Bartlett, P. L.. Bartlett, P. L., et Neural network learning: Theoretical foundations, volume 9. cambridge university press Cambridge, 1999.
Chen, Z., Lu, J., Lu, Y., and Zhou, A regularity theory for static schrödinger equations on in spectral barron spaces. SIAM Journal on Mathematical Analysis, 55(1): 557-570, 2023.
Barron, A. R. Universal approximation bounds for super- positions sigmoidal function. IEEE Transactions on Information theory, 39(3):930 945, 1993.
Chiu, P.H., Wong, C., Ooi, C., Dao, M. H., and Ong, Y.-S. Can-pinn: A fast physics-informed neural network based coupled-automatt caatooaati-numerccall method. Computer Methods in Applied Mechanics and Engineering, 395: 14909, 2022.
Bartlett, P. L. and Mendelson, Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463-482. 2002.
numerical differentiation
Bartlett, L., Bousquet, O., and Mendelson, S. Local rademacher complexities. The Annals of Statistics, 33(4): 1497-1537, 2005.
Cortes, C., Kloft, M., and Mohri, M. Learning kernels using local rademacher complexity. Advances in neural information processing systems, 26, 2013.
Bartlett, I.., Harvey, N., Liaw, C., and Mehrabian, A. Nearly-tight ve-dimension and pseudodimension bounds for piecewise linear neural networks. The Journal of Machine Learning Research, 20(1):22552301) 2019.
Du, S. S., Zhai, X., Poczos, B., and Singh, A. Gradient descent provably optimizes over-parameterized neural networks. arXiv preprint arXiv:1810.02054 2018.
Bauer, B. and Kohler, M. On deep learning as remedy for the curse dimensionality in nonparametrie regression. The Annals of Statistics, 2019.
Duan, C., Jiao, Y., Lai, Y., Lu, X., and Yang. Z. Conver- gence rate analysis for deep ritz method. arXiv preprint arXiv:2103 13330, 2021.

---

Refined Generalization Analysis of the DRM and PINNS
Duan, C., Jiao, Y., Kang, L.,, Lu, X., and Yang, J. Z. Fast excess risk rates via offset rademacher complexity. In International Conference Machine Learning, pp. 8697- 8716. PMLR, 2023.
Kohler, M. and Langer, S. On the rate of convergence of fully connected deep neural network regression estimates. The Annals of Statistics, (9(4):2231-2244 2021.
Koltchinskii, V. Local rademacher complexities and oracle
Ern, A. and Guermond, J.-L. Theory and practice of finite elements, volume 159. Springer, 2004.
inequalities in risk minimization. The Annals of Statistics, pp. 2393-2656, 2006.
Farrell, M. H., Liang, T., and Misra, S. Deep neural net- works for estimation and inference. Econometrica, 89(1); 181-213, 2021.
Koltchinskii, V. Oracle inequalities in empirical risk mini- mization and sparse recovery problems, 2011.
Kosorok, M. R. Introduction to empirical processes and semiparametric inference, volume 61. Springer, 2008.
Gao, Y., Gu, Y., and Ng. M. Gradient descent finds the global optima of two-layer physics-informed neural net- works. In International Conference on Machine Learning, PMLR, 2023.
Lagaris, I. E., Likas, A., and Fotiadis, D. I. Artificial neu- ral networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 9(5); 987-1000, 1998.
pp.
Garroni, M. G. and Menaldi, J. L., Second order elliptic integro-differentia problems. CRC Press, 2002.
Lecué, G. and Mendelson, S. Learning subgaussian classes: Upper and minimax bounds. arXiv preprint arXiv:1305.4222 2013.
Girault, V. and Raviart, P.-A. Finite element methods for Navier-Stokes equations: theory and algorithms,, vol- ume 5. Springer Science & Business Media, 2012.
Lei, G., Lei, Z., Shi, L., Zeng, C., and Zhou, D.-X. Solv- ing pdes on spheres with physics-infomme convolutional neural networks. Applied and Computational Harmonic Analysis, 74:101714, 2025.
Han, J., Jentzen, A., and E. W. Solving high-dimensional partial differential equations using deep learning. Pro- ceedings of the National Academy of Sciences, 1 115(34): 8505-8510, 2018
Lei, Y., Ding, L., and Bi, Y. Local rademacher complexity bounds based on covering numbers. Neurocomputing, 218:320-330, 2016.
Han, J., Lu, J., and Zhou, M. Solving high- nigh-dimenssonal eigenvalue problems using deep neural networks: diffu- sion monte carlo like approach. Journal of Computational Physics, 423: 109790,, 2020.
Li, S. and Liu, Y. Sharper generalization bounds for cluster- ing. In International Conference on Machine Learning, pp.
He, J. Li, L., Xu, J., and Zheng, C. Relu deep neu- ral networks and linear finite elements. arXiv preprint arXiv: 1807,03973, 2018
PMLR, 2021.
Liang, T., Rakhlin, A., and Sridharan, K. Learning with square loss: Localization through offset rademacher com- plexity. In Conference on Learning Theory, pp. 1260- 1285. PMLR, 2015.
Hornik, K. Approximation capabilities of multilayer feed- forward networks. Neural networks, 4(2):251-257, 1991.
Jiao, Y., Lai, Y., Li, D., Lu, X., Wang, F., Wang, Y., and Yang, J. Z. A rate of convergence of physics informed neural networks for the linear second order elliptic pdes. arXiv preprint arXiv:2109. 007001780, 2021.
Liu, F., Dadi, L., and Cevher, V. Learning with norm con- strained, over-parameterizee, two-layer neural networks. Journal of Machine Learning Research, 25(138): 2024.
Jiao, Y., Shen, G., Lin, Y., and Huang. J. Deep nonparamet- ric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics, 51(2):91-716, 2023.
Lu, J., Shen, Z., Yang, H., and Zhang, S. Deep network approximation for smooth functions. SIAM Journal on Mathematicll Analysis, 53(5):5465-5566 2021a.
Kanade, V., Rebeschini, P., and Vaskevicius, T. Exponential tail local rademacher complexity risk bounds without the bernstein condition. arXiv preprint arXiv:2202.1 11461, 2022.
Lu, Y., Chen, H., Lu, J., Ying, L.,, and Blanchet, J. Machine learning for elliptic pdes: Fast rate generalization bound, neural scaling law and minimax optimality. arXiv preprint arXiv: 10.02110.06897, 2021b.
Klusowski, J. M. and Barron, A. R. Approximation by combinations of relu and squared relu ridge functions with 1I and 10 controls. IEEE Transactions on Information Theory, 64(12):7649-7656 2018
Lu, Y., Lu, J., and Wang. M. A priori generalization analysis of the deep ritz method for solving high dimensional elliptic partial differential equations. In Conference on learning theory, pp. 3196-3241. PMLR, 2021c.
10

---

Refined Generalization Analysis of the DRM and PINNS
Luo, T. and Yang, H. Two-layer neural networks for partial differential equations: Optimization and generalization theory, arXiv preprint arXiv:2006. 15733, 2020.
Shen, Z., Yang, H., and Zhang, S. Optimal approximation of relu networks in terms of width and depth. Journal de Mathématiques Pures et Appliquées, 157:101-135, 2022.
Lyu, L., Zhang, Z., Chen, M., and Chen, J. Mim: A deep mixed residual method for solving high-order partial dif- ferential equations. Journal of Computational Physics, 452:110930, 2022.
Shin, Y., Darbon, J., and Karniadakis, G. E. On the con- vergence of physics informed neural networks for lin- ear second-order elliptic and parabolic type pdes. arXiv preprint arXiv:2004.018: 01806, 2020.
Ma. C., Wu, L.,, et al. The barron space and the flow-induced function spaces for neural network models. Constructive Approximation, 55(1):369-406, 2022.
Siegel, J. W. Optimal approximation of zonoids and uni- form approximation by shallow neural networks, arXiv preprint arXiv:2307.15288 2023.
Makovoz, Y. Random approximants and neural networks. Journal of Approximation Theory, 85(1):98-109, 1996.
Siegel, J. W. and Xu, J. High-order approximation rates for shallow neural networks with cosine and reluk acti- vation functions. Applied and Computational Harmonic Analysis, 18:1-26,4 2022a.
Maurer, A. Entropy and concentration. Harmonic and Applied Analysis: From Radon Transforms to Muchine Learning, pp. 55-100, 2021.
Siegel, J. W. and Xu, Sharp bounds on the approximation rates, metric entropy, and n-widths of shallow neural networks. Foundations of Computational Mathematics, pp. 1-57, 2022b.
Mendelson, S. Improving the sample complexity using global data. IEEE transactions on Information Theory, 48(7):1977-1991, 2002.
Siegel, J. W. and Xu, J. Characterization of the variation spaces corresponding to shallow neural networks. Con- structive Approximation, pp. 1-24, 2023.
Mendelson, S. Learning without concentration. Journal of the ACM (JACM), 62(3):1-25, 2015.
Mendelson, S. Learning without concentration for general loss functions. Probability Theory and Related Fields, 171(1-2):49-502, 2018.
Sreekumar, and Goldfeld, Neural estimation of statis- tical divergences. Journal of machine learning research, 23(126):1-75, 2022.
Mishra, S. and Molinaro, R. Estimates the generalization eтror of physics-informed neural networks for approxi- mating a class of inverse problems for pdes. IMA Journal of Numerical Analysis, 1222), 2022.
Vaart, A. d. and Wellner, J. A. Empirical processes. In Weak Convergence and Empirical Processes: With Applications to Statistics, pp. 127-384. Springer, 2023.
Wainwright, M. High-dimensional statistics: non- asymptotic viewpoint, volume 48, Cambridge university press, 2019.
Mohri, M., Rostamizadeh, A., and Talwalkar, A. Founda- tions of machine learning. MIT press, 2018.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Wang. C., Li, S., He, D., and Wang, L. Is 12 physics in- formed loss always suitable for training physics informed neural network? Advances in Neural Information Pro- cessing Systems, 35:8278-8290, 2022.
Wang. Y. and Dong, An extreme learning machine- based method for computational pdes in higher dimen- sions. Computer Methods in Applied Mechanics and Engineering, 418: 16578, 2024.
Ren, P., Rao, C., Liu, Y., Wang. J.-X., and Sun, H. Phycr- net: Physics-infooi informed convolutional recurreet network for solving spatiotemporal pdes. Computer Methods in Applied Mechanics and Engineering. 389:11 2022.
Wu, S., Zhu, A., Lu, B., al. Convergence of physics- informed neural networks applied to linear second-order elliptic interface problems. Communications in Compu- tational Physics, 33(2):596-63 627, 2023.
Schmidt- Schmidt-Hieber, J. Nonparametric regression using deep neural networks with relu activation function. The Annals of Statistics, 2020.
Schumaker, L. Spline functions: basic theory. Cambridge university press, 2007.
Xu, The finite neuron method and convergence analysis. arXiv preprint arXiv:2010.01458 2020.

---

Refined Generalization Analysis of the DRM and PINNS
Yang, J. Sun, S., and Roy, D. M. Fast-rate pac-bayes generalization bounds via shifted rademacher processes. Advances in Neural Information Processing Systems, 32, 2019.
Yang, Y.. Wu, Y., Yang, H., and Xiang, Y. Nearly opti- mal approximation rates for deep super relu networks sobolev spaces. arXiv preprint arXiv:2310.10766, 2023a.
Yang, Y., Yang, H., and Xiang, Y. Nearly optimal ve- dimension and pseudo-dimension bounds for deep neural network derivatives. Advances in Neural Information Processing Systems, 36:21721-21766, 2023b.
Yarotsky, D. Error bounds for approximations with deep relu networks. Neural Networks, 94:103-114, 2017
Yousefi, N., Lei, Y., Kloft, M., Mollaghasemi, M., and Anagnostopoulos, G. C. Local rademacher complexity- based learning guarantees for multi-task learning. The Journal of Machine Learning Research, 1913,113851441 2018.
Yu, B. et al. The deep ritz method: a deep learning-based nu- merical algorithm for solving variational problems. Com- munications in Mathematics and Statistics, 6(1):1-12,
2018.
Zang, Y., Bao, G., Ye, X., and Zhou, H. Weak adversarial networks for high-dimensional partial diffcrential equa- tions. Journal of Computational Physics, 41 411:109409, 2020.
Zeinhofer, M., Masri, R., and Mardal, K.-A. A unified framework for the error analysis neural networks. IMA Journal of Numerical Analysis, pp. drae081, 2024.
Zhang, M., Li, Q., and Liu, J. On stability and regularization for data-driven solution of parabolic inverse source prob- lems. Journal of Computational Physics, 474:1 2023.
Zhang, R., Liu, Y., and Sun, H. Physics-informee multi-lstm networks for metamodeling of nonlinear structures. Com- puter Methods in Applied Mechanics and Engineering, 369:113226, 2020.
Zhivotovskiy, N. and Hanneke, S. Localization of ve classes: Beyond local rademacher complexities. Theoretical Com- puter Science, 742:27-49, 2018
12

---

Refined Generalization Analysis of the DRM and PINNS
Appendix
The Appendix is organized into four parts: Proof Section 2, Proof of Section 3, Auxiliary Lemmas, and Discussion.
A. Proof of Section 2
A.1. Proof of Proposition 2.2
The proof follows similar procedure to that in Barron (1993), but the method in Barron (1993) can only yield slow rate of approximation. We start with a sketch of the proof. For any function in the Barron space, we first prove that it belongs to the N' (92 closure of the convex hull of some set. Then estimating the metric entropy of the set and applying Theorem in Makovoz (1996) (see Lemma C.4) leads to the fast rate of approximation.
For the function f E (92), according to the definition of Barron space, we can assume that the infimum can be attained at the function fe. To simplify the notation, we write fe as f. since fela = From the formula of Fourier inverse transform and the fact that f is real-valued,
f(x) Re L Ju.z (W))d
[ cu (
Re
Bd
cos(w> x + ((w))l{(w))l
(39)
B cos(ww- x B(w)) A(dw)
glx,w)
where B = S + ) )²] A A(dw)
(1+lwl)) ((W)) is probability measure. e'0(w) is the phase of f(w) and
B
B cos(ww x ((w)
((x,w)
(40)
(1 [w|))
From the integral representation of f and the form of g, i.. (39) and (40), deduce that f is in the H (S2) closure of the convex hull of the function class
Geos (B) = { [Beos(w x x+
B cos(w
w t R
(41)
(1+ [w11)2
It could be easily verified via the probabilistic method. Assume that {wi] is sequence of i.i.d. random variables distributed according to A, then
E
[1((x)
1 n
Σ 1=1
g( w,, wl)
-Sa E
f((x)
1
1
Σ Vgla wi)l dx
Ja
1 / Var(g(z, + 1 1 Tr(Cor([V(r,
n.
B[llg( w) I (1))
n
2B²
where the first equality follows from Fubini's theorem and the last incquality holds due to the facts that [g(2z w)) S B and [g((,, w) & B for any N,w.
13

---

Refined Generalization Analysis of the DRM and PINNS
Then, for any given tolerance 0, by Markov's inequality,
P
p
[f(x))
1 Σ
gl,, ((r,w))ll > - 2 [f(x)
Σ
gat wi)
(0)
S
2B ne
1=1
By choosing large enough n such that
2B ne"
1, we have
(1(())))
1 Σ ((r, S 0,
P
which implies that there exist realizations the random variables {wi)i-. such that [lf(x)) - " Therefore, the conclusion holds.
Σ m
((), S
Next, we going to show that those functions in Geoo(() are in the H H (?t closure of the convex hull of the function class FF U (0), where
(b) = (bo - x +t): W// = 1, E [-1,
(42)
for any constant b IR.
Note that although (B) consists of high-dimensional functions, those functions depend only the projection of multivariate variable æ. Specifically, each function g(z,w) B cos(w-=+() Geos (B)) the composition of one- (1+ w|1)²
> dimensional function g(z) = cos( wh+++) and a linear function = [w x with value in [-1, 1]. Therefore, it suffices to (1+|=11)²
prove that the conclusion holds for g(z) on [-1, 1], i.e., to prove that for each w, is in the ([-1,, closure of convex
hull of F! (5B) U F (-5B) U (0), where
(b) == (bo + (): : or 1,4 (
(43)
for any constant IR. Then applying the variable substitution leads to the conclusion for g(x,w).
In fact, it is easier to handle that in one-dimension due to the relationship between the ReLU functions and the basis function in the finite element method (FEM) (He et al., 2018), specifically the basis functions in the FEM can be represented by ReLU functions. To make it more precise, let us consider the uniform mesh of interval [-1, 1] by taking + 1 points
= zo < 21 < < mm = 1,
and set h 2 m 1- -1 h, Fm h. For 0 S i S m, introduce the function (p)(2), which is defined follows:
(2 1)1)); if 2 E [.- 2/).
h
(x(2)
(44)
(2+41 - 2): if E [2,, 2+1,
0, otherwise.
Clearly, the set (yo. pml basis of P/., which vector space of continuous, piece-wise linear functions (IP Lagrange finite element, see Chapter of Emn & Guermond (2004) for more details). And yss can be written as
a(= - z(-1) - 20 z - 2.) + o(= 21+1)
(=(=)
(45)
h
Now, ready to present the definition of interpolation operator and the estimation of interpolation error (Em & Guermond, 2004) (Proposition 1.5 in Em & Guermond (2004)).
Consider the so-called interpolation operator
Iz U.vU 1(--1,1) + Σ
u(2))pp P).
(46)
0
14

---

Refined Generalization Analysis of the DRM and PINNS
Then for all h and E /F][-1,1],, the interpolation error can be bounded as
| - -- & 1/lle" [[ and || - (1X'0)"[[(((1.1))) S hlle" [[
(47)
By invoking the interpolation operator and the connection between the ReLU functions and the basis functions, we can establish the following conclusion for me-dimensional functions.
Lemma A.1. Let g E (²((1,, with || B for = 1, Then there exists two-layer ReLU network 9m of the form
Iml (2)
Σ a, +t),,
(48)
with la:|
2B
lasl
5B, 141 < 1, 6 (-1,1) S s 6m such that
m
4v2B
l1g - 9m 1mllm ((-.I)
(49)
Therefore, g is in the ([-1,, 1)) closure of the convex hull of F. (FB(
Proof. Note that from (45) and (46), the interpolant of g can be written combination of ReLU functions as follows
m
(1) Σ (())))p((2)
((i) (02 21-1) 20(2 - 2i) + a 2++1)
(50)
(<z0)(((== 2-1) - -20(2 20)) + g(21)0(== -20)
m g( 2g(2z) g(((+1)
+
0 - 2:)
= 9((=)) + g(22) g(20)
m- (((-1)) -2g(2;) + g(2++1)
0(2 - 20) +
o(2 - -24).
h
By the mean value theorem, there exist Eo E [2 21] and & E [=-11, 2441 for S í S m such that g(21) - g(20)
g ( and g(2+1)) 2g(2;) + g(z++1) = g (E for 1 < m m-1. Therefore, I2 (g) can be rewritten
(0) g(20) + + (Eu)o(( (2- 20) +
Σ g (E))((2 24)..
(51)
On the other hand, the constant can also be represented as combination of ReLU functions on [-1, By the observation that (2) + o-=2) |2]. we have that for any 2 E [-1,1]
+ +|+
o( + 1) + a 1) a(=
(52)
2
2
Plugging (52) into (51) yields that
glz0)
+ 0 1))
g (Eo)o(z - zo)
[+ = Σ
2m
1 m
(53)
2g (E:)o(z - 2;)
+
15

---

Refined Generalization Analysis of the DRM and PINNS
Combining the expression of (g) and the estimation for interpolation error, i.e. (53) and (47), leads to that there exists two-layer neural network Im of the form
6m
9m (2) [x)
0,o + tz),
with |a:|
2B m
Σ ag
5B, S LLG 6m 1 such that
4v2B
lgg 9m (1
m
Although the interpolation operator can be view piece-wise linear interpolation of g, which is similar Lemma 18 in Lu al. (2021c), our result does not require (0) and the value of g at the certain point is also expressed combination of ReLU functions. Specifically, the 9m in Lemma of Lu et al. (2021c) has the form Jm (2) = c Ee a, 44), where e g(0) and they partition [-1, 1] by 2m points with 20 1, Zm 0, 22m
And our result can also be
extended in W
([-1, ( norm like Lemma 18 of Lu al. (2021c). Note that on 21
(=)))))) ((==1) 24 2
21-1
+g(z)
h
h
which is the piece-wise linear interpolation of g. Then by bounding the remainder in Lagrange interpolation formula, we have (g) [x--n]] h²
| [ and
- g((2-1) - g (=) - -1 (I) (z) - g=)()
h s (E:) - 9 (z))
(54)
< hhlg 1 1.2:)
where the first inequality follows from the mean value theorem.
Therefore, (g) 91lw1=*(-1111) 2B
Lemma A. implies that for any w, the one-dimension function g(=) cos( wh+++) in the H+((-1,1) closure of (1+1w11)
convex hull of (5B) (-5B) U {0). Then applying the variable substitution yields that those functions in Geos in the (12) closure of the convex hull of the function class (5B) U. (0). Specifically, for any function
h IR + I and w E Rdd with W|| 1, without loss of generality, we can assume that wi 0. Then for the integral
/
[k(w =)) = / IM(w . dr),
let y1 w. x2 T2, ydd Tdd we have
Wa 1o:00t--aaa Wa
W1 {'
J Ih(ww 2
(y) ... dya < (Mlr))dyr
Wi Jo
Therefore, the conclusion holds for Geos Recall that f is in the H (*)) closure of the convex hull of Geos thus we have the following conclusion.
Proposition A.2. For any given function in B²()), f is in the H²) closure of the convex hull of Fo (511 f()) U. {0], i.e., for any >> there exist m and Wis tt, a,, S i m such that
ll(())
aolw: x +t))ll Sc,
(55)
where [wil1
1,4 E [-1,1], sis m and Σ lasl < 5|]
16

---

Refined Generalization Analysis of the DRM and PINNS
Proposition A.2 implies that functions in (2)) can be approximated by linear combination of functions in Fo))..
Recall that (1) {ar + |w/1 1, f [-1, 1]]. For simplicity, we write F, for Fo (1).
Then to invoke Theorem I in Makovoz (1996) (see Lemma C.4), it remains to estimate the metric entropy of the function class Fo, which is defined as
en (F)) == :=n({{ : Fa can be covered by at most n sets of diameter S e under the H' norm).
(56)
By Lemma C.6, we just need to estimate the covering number of Fo, which is easier to handle. Proposition A.3 (Estimation of the metric entropy). For any E N,
En (Fo) S en 34,
where c is a universal constant.
Proof. For (Wi, 11), (2) E (1) x [-1, 1], we have
|orww . x t1) (w + 1z) (1)
lo(w x 41) ow + 12) 12) / Volwu x x1) Volw2 + 12)1
< / I(1 w2) x + (t1 - 12) + |w [1ll - (22 Pdx
2((w1 wall 1/1 12 + / 1(w1 W2) w2) (w) (w1 w2 (w// (wn
(57)
s 2(w wel? + 181 - tzl²) + 2/w1 - wall + / [1w1241200 - 442 dx
S 4(ww1 - wel? 1 tzl?) + / | [[(nr04t120) [uxa+t220
where the first inequality is due to that is I-Lipschitz continuous, the second and the third inequalities follow the from the mean inequality and the fact that the 2-norm is dominated by the 1- norm.
is challenging to handle the first and second terms simultaneously due to the discontinuity of indicator functions, thus we turn to handle two terms separately. Note that the first term is related to the covering of ( [-1, 1] and the second term is related the covering of VC-class of functions (see Chapter 2.6 of Vaart Wellner (2023) or Chapter 9 of Kosorok (2008)). Therefore, we consider a new space Gi defined
91 [(ww, t), I (wwm+120)) :
Obviously, it is subset of the metric space
G2 = ((W)
(1g ww E (1) t1, E [-1,
with the metric d that for bj
(w1) (t), T (ww) (1ug=++1200) b2 = (10 (10
113
d(b,, (2) = /2(w will + |t| -RP) + || (1 aw2
The key point that G2 can be seen as a product space of (1) x [-1,1] 1] and the function class Fi = (1w22+200) (w, E (1) x [-1, 1]} is a VC-class. Therefore, we can handle the two terms separately.
By defining the metric d, in (1) x 1]
di ((w), (wf.tt) 2((w) w [t] 12
17

---

Refined Generalization Analysis of the DRM and PINNS
and the metric d2 in F1
d2 (fuu)=+td200 (u33=+02200) || au lug (1820220))l1
the covering number of O2 can be bounded as
N( d, ( (.
- 2)
2) NNF,,,
As Fi is a subset of the collection of all indicator functions of sets in a class with finite VC-dimension, then Theorem 2.6.4 in Vaart & Wellner (2023) implies
S K(d+ (2)
2d
with universal constant K, since the collection of all half-spaces in Fd is VC-class of dimension d + (see Lemma 9.1 9.12(i) in Kosorok (2008)).
By the inequality Vlal + [9] S Vlal + V1bl. we have
2/ - will + 101 - 192 v2(Ww) will + 10] - -t1))
therefore
V2 ) 11 (-1,1))) ((OB
v2
2 c).
Combining all results above and Lemma C.5, we can compute an upper bound for the covering number of G1.
s( d. ;
s <1(0B((1) x [-1, 9) NN(F1, d2, $
V2 () )
V2 - d2 9
<N(O
3d
< k(d+ 1)
where is a universal constant.
Therefore, applying Lemma C.6 yields the desired conclusion.
Note that in Proposition 2.2, we require 1; 1- instead of 6; E [- due to the measurability (see Remark A.4). At this time, the approximation result does not change. In fact, for any w E Rd,, taking sequence (tn heE that is monotonically increasing and tends to 1, we can deduce that (lo(w + (m)l + [la(w x 1))) It suffices to prove that
I [( 110
/ Muwrtt 11or4120) Pdr 0.
Since the function t r T(u<t) is left-continuous for any E TR, that [(wx+4,20)) [1o+1200 for all x E Q. Then, applying the dominated convergence theorem leads to the conclusion.
A.2. Proof of Theorem 2.4
The proof is based on enror decomposition and the peeling method. The key point is the fact that Sn ur (x)dz 0. thus for any H (S)),
( u(a))ae) = -[((uto) - n² (z))dr) -
(u(x)) u =²(x))2do < 1|u - = -1'12
(58)
18

---

Refined Generalization Analysis of the DRM and PINNS
which implies that u is close enough to u" the H norm, then (Sn u(x)dd)) 2 is also proportionately small. Furthermore,
if is bounded, can also prove that the empirical part of (Jn u(z)dx) ? i.e.
T5 Σ u(xi)
is also small in high
probability via the Hoeffding inequality.
In the proof, we omit the notation for the Poisson equation, i.e., we write & and En for the population loss Ep and empirical loss& E respectively. Additionally, we assume that there is a constant M such that |u',, I. - M.
Assume that un the minimal solution obtained by minimizing the empirical loss & in the function class F, here we just take F parameterized hypothesis function class. When considering the specific setting, we can choose F to be the function class of two-layer neural networks deep neural networks. Additionally, we assume that those functions in F and their gradients are bounded by M in absolute value and 2-norm.
Recall that the population loss and its empirical part are
E(u) = L. /u((o) - J. 2)()))(())de + (% ulo)aa))
(59)
and
2 E(x.)((x)) + n Σ Vu(x,))
a
( n
(n(
(60)
n
By taking ur E arg min ||u - u² H (9)) we have the following error decomposition:
- E(u,) E(u") = E(un) Au, + ((,) En(u))) + (Ex( E(u')
S E {un) (un) + (u5) - -E(u")
E(u,) - AE (un) + A(E, - -En(u") + (,,,(u)) - E(r')
(61)
= (E(un) - -E(u))) - (un) - En(())) + +(E((UFF= - (n()))
S (up((E(u) - E(u")) - A(E,, - Ex,) + ((F) - En (u)),,
wef
where the first inequality follows from the definition of u, and À is a constant to be determined. In the following, we estimate the two terms separately.
Rearranging the term En En (u") yields
) Eu)
2
1
Vuz (2,)1 +
7 H (xi)
1 n
2 " f(X,)uF(X)
Vu (x
u² (x,))²
2
Σ f(X.)u
n
(62)
(1Vu)(X)) - 2f((x))x(XX) - (uu (X,)) - 2f(Xx))
72 (1E )
ト ux(xx)
0 + 02
where in the last equality, we denote the right two terms the second equality as Φi and d2 respectively.
Define
Mxx (Vuu)())--2(()))())))))))))
2f(x))" (x)),
19

---

Refined Generalization Analysis of the DRM and PINNS
then by the boundedness of uys, Vuzl, u", |Vul and S, we can deduce that
Var(h) _ P(h2) s S.22 uu u (1 (()
2 app and 1h [[|h] S ssup hb] s 12M2,
(63)
where Capp denotes the approximation enror in the H' (92) norm, i.e., Capp = l|uy u (8))
Therefore, from Bernstein inequality (see Lemma C.1) and (63), we have that with probability at least 1 - e
1 Vuy (x,)² (f(X;) (IV -10r))))) (x
I B[b(X)] + 24M2
4M
capp +
n
(64)
=E(uF) -E(u)) - (1/ uzdx
24M
4.
+
app
<0 Gapp + M
n
where the last inequality follows by the basic inequality av s + b for any a, b > and Proposition 2.1.
For o the Hoeffding inequality (see Lemma C.2) implies
( - Σ
P (X.)) - J uz(())d 2 2M
2f
S 2e
Therefore with probability at least 1 - -e
2
2
ux(X;)
- > u²
ex,) (X:)
71
us (Xi)
1=1
(65)
12
/ (z(2)dz
+ +1/8 1- (z(2))
2
Σ
uy
uz(z))
2
< 2
Combining the upper bounds for di, and oz, i.c. (64) and (65), we can deduce that with probability as least 1 3e 31
(u(ur) En(u.) s cc
Camp +
M2t
(66)
n
Plugging this into the error decomposition (61) yields that with probability as least 36-1
M26
E(u,) E(u*) sup [(E(u) E(u") E. XC WEF
Cámp +
(67)
n
For the first term in the right of (67), we employ the peeling technique to establish upper bound for it.
Let Po be a positive constant to be determined and 2 for t: 2 1.
Consider the sets Fs := {u E F Pk-1 < u|u- - u (1 S p for 2 and Fo = {u E F; ||u - u² (1 S SPo for
k=0
20

---

Refined Generalization Analysis of the DRM and PINNS
The boundedness of the functions in F, u" and their respective gradients implies that
M
K == max k - C'log
Po
since PK = 2 Po and suPeer ||u u²ll (() I 4M
Then for the fixed constant E (0, 1), set 6₁
8 k+1 for 0 S k S K.
From Lemma C.8, we know that with probability at least - 6%
sup (E(u) - E(u")) - (En(u) - En(u))) uEFk
C aM2 log(23/n)
M ppo log (23
M² log 8
(68)
n
n
71
M log 1 &
aM2
4b M
log
where a, 3, a, b are constants depending on the complexity F (see the definitions in Lemma C.8).
Note that
Pk S max 2p
S max 21/u - u² 11 (1)
(69)
max (Cp(E(u) E(u"))}
S Po + (Cp(E(u) - E(("))
holds for any E F& and
1 = log
K +1
M
log
S log + C'log log
(70)
5%
5
Pu
Therefore, setting Po 1/n, then with (69) for pr. for the right terms in (68), we can deduce that the following inequality holds for all u E Fx.
2 Pka log (2
CV
n
M2(po + 2Cp(E(() - {(ur)))alog(233))
sov
M2 Pap Pna log (2
2M²Cp(E(u) - (og(2
V
CV
n
n
(71)
M po (23(2
C( E(u) EE(u))
CV
n E(u) E(ur)
4C
n
M aoa log(288/
MºCpaloog288
+ O(V
n
n
((n) - E(u")
4 + CogCpalog(23//
n
where the third inequality follows from the basic inequality a S a + for any a, 0. Similarly, with the upper bound for log ,., i.e. (70), we can deduce that
M pu log
E(u) E(u*)
CCpM2 & + log log(n
(72)
n
4
M² log
M2 à + log log(n.
(73)
21

---

Refined Generalization Analysis of the DRM and PINNS
and
CV pa log M
4b E(u) E(u)-E(m E((")
CM2Cpalog M
(74)
4
n
Combining (71), (72), (73), (74) and (68) yields that with probability at least 1 - 64 for all u E Fk.
(E(u) E(u") (() - Ex("))
s M Cr CCpM + log log +
4b M M C
(75)
Note that E 82 8, therefore the above inequality (75) holds with probability at least 1 uniformly for all F, i.e.,
(E()) E(u*)) (u) En (u*))
ueF
M 1og
CpM log log(nM2)) M²Cra log
(76)
Sc
n
By taking À and = in (76), together with the error decomposition (61), have that with probability least
1 - -
E(un) - E(u")
(0( CpM loglog( +
2
(77)
c
Capp +
From Lemma C.9, we know that (1) when F Fm, (5]lu (.))))
b cM, cmd,8 3 cM2 a cmd,
where e is a universal constant.
(2) when F = (N,,, L, Bllllllw
b Cn, CN N log L) Cn,a CN N log L)
where > CL N log L)3 and C' is a constant independent of N, L.
Finally, recall the tensorization of variance:
Varls(X X,)) SE
Var, Vxr
whenever XI, Xn independent, where
Var.f(æ1, 2n) Varlf(z1, T1
.n)]..
Combining this fact and the observation of the product structure of [0, 1]d yields that the Poincaré constant is a universal constant.
Hence, the conclusion follows.
Remark A.4. In the proof of Theorem 2.4, have made implicit assumption that the empirical processes measurable. Typically, when considering some empirical process, corresponding functions are Lipschitz continuous with respect to the parameters and the parameter space is separable, thus the measurability holds directly. However, in our setting where ReLU neural networks are used in the DRM, the functions fail to satisfy the Lipschitz continuity with respect to the parameters. Thus, it's necessary to discuss the measurability of the empirical processes. For simplicity, we only consider the two-layer neural networks.
22

---

Refined Generalization Analysis of the DRM and PINNS
Here, require the concept of pointwise measurability. Recall that function class F of measurable functions in X' is pointwise measurable if there exists countable subset C F such that for every F, there exists sequence {9m} g with 9m (z) + f (a) for every X' (see Chapter 2.3 in Vaart & Wellner (2023) or Chapter 8.2 in Kosorok (2008)).
Note that when applying two-layer neural networks in the DRM, the term (2x++22)) is not Lipschitz continuous with respect to w and t. Fortunately, we can adapt the proof of Lemma 8.12 1212 in Kosorok (2008) to show that the function class pointwise measurable. Specifically, consider the function class
g = -1---04- w
where Q is the set consisting of all rationals.
Fix w and 1, we can construct [Wmm 4m)) as follows: pick wm E (1) n Qd such that Iwmm - w/1 S 11((2m) and pick tm E (++ 1/2mm), + 1/m]. Now, for any x E [0, 1]d, we have that
1 4m-m Stm? 4 (-w-sttm (wo (wm-
Since (w-m - w)) xi| S 14im w|1 - 1/(2m), we have that I' == Im + (Wm w) . 1 - 0 for all and T'm - 0 as m - DO. Note that the function + [ (wgx)) is cigtt-oontiuuuus for any u E R, so that [- Stmm + L (-wrret) for all E [0, 11 Thus, the pointwise measurability is established.
Therefore, for the function class of two-layer neural networks 1B).
m
(B) (B)
Σ nol t.): wwl11 1, (-1,1), Σ hl S B
we can pick 7/ wy, tú to be rationals. To prove the measurability for the empirical processes of the form suPuer(E(u) AE (u)), where F is related to ReLU functions and their gradients, it remains to focus on the term pf.
Note that for u, ú E (B) with the forms
u(x)= = Σ no x +.), ((x) = Σ ho(ws (w)s x x f.),
we have that
IP(VV 2 - 2fu) P([V 2f
< CC - Vúl + Plu (
C
Σ
1 To + يسا 00sl2 + lt: - Egl + PI( - (1 110011201)
The dominated convergence theorem implies that
[1 1m ttt 0.
Therefore, with little abuse of notation, we have supaer (w)) sup (E(u) (u)), which implies that the empirical processes in the proof of Theorem 2.4 are measurable, as the parameters in F can be replaced by rationals.
A.3. Proof of Theorem 2.5
Proof. For the static Schrödinger equation, we can also use the method in the proof of Theorem 2.4 or other methods in Lu et al. (2021b), Lei et al. (2025) and Farrell al. (2021), due the similarity between the problem and the generalization error of 2 regression with bounded noise. However, the methods mentioned above are quite complex. Here, we provide a simple proof through different error decomposition and LRC, which can be easily adapted for other problems with similar strongly convex structures.
As before, in the proof, we write and En, for the population loss Eg and empirical loss En respectively. Additionally, we assume that [u*), [V [V & M for some positive constant M.
23

---

Refined Generalization Analysis of the DRM and PINNS
Recall that
w" = arg min {u) :- / Vul² + Vluldo -2 (9)
2 J fudx
(78)
n
and Un is the minimal solution to the empirical loss E, in the function class F. We also assume that sup lu]. suPue /vl s M.
Through an error decomposition, the same as that for the Poisson equation (61), we have
E(um) E(u))
{uu)) (un) + X(En(un) En(uF)) (Ex(u=) - E(u")
S E {un) (un) + (EF( -E(u")
{(u)) (un()) + ((E((F)) - -En(")) + (En(()) - E(u')
(79)
(E(u,) - E(u")) (un) - En(u")) + ((E((u)) - En(n))
S supl(E(u)= - E(u")) - (1) - Ex((")) ((E((F)) En (u))),
aEF
where the first inequality follows from the definition of un and X is a constant to be determined. Let Copp = l|u== - u (=( be the approximation error.
From the Bernstein inequality, we can deduce that with probability least 1 c-
(En(u)) - En((")) - (E(ux) - -(( 1 (tVar(g)
E||g||=>
(80)
n
3n
where
g(z) = (|uuz|² + 2
V(z)|u' / 2 2f(x)u* (x)).
-
From the boundedness of uF,u", Vuz, Vu,, f and V. we can deduce that [g| S 8M2 and Var(g) - Pg & cM u² 1 = 2M kpp
(81)
Therefore, plugging (81) into (80) yields that with probability at least - c-
2tcM2 aap
81M
En (u(F En (n(")) S ce
+
3n
(82)
t.²
S
Gupp
n
where the first inequality follows from Proposition 2.1 and the second inequality follows from the mean inequality. Plugging (82) into the error decomposition (79) yields that
E(un) - E(u") < sup[(E(u) - ((((((())(())))))))))(((((((((() - ((),(u)) - En("")) + Ac ( (2
Capp + M
(83)
WEF
holds with probability at least -t
Note that (E(u) E(n")) X(En(u)= En("") can be rewritten
(E(u)= - E(u")) - ((E,(u)) Ex = Ph - APah,
(84)
where := ((o))))))+(V)))))))))2222o)m (VV)))()))))((())))))+(()()))))) (z) -x))= 2f(xx(²)))) And the form (84)
motivates the use of LRC.
To invoke the LRC, we begin by defining the function class
H = [(Vv(x)) (z))) ((())=())))))))))))))))))))))))) - (VV ( (())))))))+)))))))))())((()()))) -ffx))(())) u uEF)
and functional on H as T(h) = 2h2 It is easy to check that
Var(h) S T(h) & 2 Ph,
(85)
24

---

Refined Generalization Analysis of the DRM and PINNS
Ph2 S cM2llu (a) (M)(((()) E(u")) cM It implies that the functional 7 satisfies the condition of Theorem 3.3 in Bartlett et al. (2005).
Following the procedure of Theorem 3.3 in Bartlett et al. (2005), we are going to seek sub-root function and compute its fixed point.
Define the sub-root function
M log
(r) = 800 star(H, Ph3 <) + 704
(86)
where star(H, {ah a [0, h H} and invoking the star-hull of H around 0 is make to be sub-root function.
Next, our goal is to bound the fixed point of sb.
r 2 (r)), then Corollary 2.2 in Bartlett et al. (2005) implies that with probability at least 1
(h E star(H,0): Ph3 < C (h E star (Har(H, : Pah < 2r);
and thus
8M2
ER, E star(H, Ph2 r) & (h E star(H. 0) Pah22 s 2r) +
(87)
Assume that r* is the fixed point of 4, then
((r)) S cM e star(H, P,h² s 2r*) + M logn
(88)
n
where we universal constant represent the upper bound for the constants in the definition of (r(),, i.e. (86).
To estimate the first term in (88), we need the assumption about the empirical covering number of H. Assumption A.S. For any e > 0, assume that
(2) a.s
N(H, {(
for some constant 3 > SuPлcи |/|- Then by Dudley's 's theorem,
ER, E star(H, 0) P.h²2 s 2r*)
22
$
c
Vlog log,N(e, star (ttartt (), (P
Ju
V2r=
log 11 H. I2(P)) (2+1)au
<
E
V
J0
B
<
log
de
cB
log
dc
B
r+ log
V
vTB
r+ log
M2
where the fourth inequality follows from Lemma C.7 and the last inequality follows by the fact that =/===(r)))))) 2 c 15 logn
25

---

Refined Generalization Analysis of the DRM and PINNS
Therefore,
a
Vng M2
M logn
r" S
cM2
r log
+
n
n
which implies
S cMr
a
log
vnB M
log n
The final step is to estimate the empirical covering numbers of the function classes of two-layer neural networks and deep neural networks, i.c., determine and 3 for F Fm (5]lug) (s)) and F ( L, B Blluug (w)))(())
(1) When F = m (5) u (o))) estimation of the covering number of H is almost same as the estimation of g for the two-layer neural networks in Lemma C.9 (1). is not difficult to deduce that or = cmd, B = cM2. For simplicity, we omit the proof.
(2) When method that in Lemma C.9 (2).
we can also deduce that CN N log L)², B C'n by similar Blluglll
As result, given the upper bound for applying Theorem 3.3 in Bartlett et al. (2005) with À = 2 allows us to reach the conclusion.
A.4. Proof of Proposition 2.7
Proof. (I) We first consider the setting of Poisson equation. From Lemma 2.5 in (Mendelson, 2002), obtain that for bounded function class F, if
12
then
Va 2 2-p p
v
ER. (F: max
T
212
(89)
2
where a 2 2,0 2 and C only depends the upper bound for the functions in F.
Compared to the original Lemma 2.5 in (Mendelson, 2002), here in (89), we provide the result with explicit dependence a,p.
6d At this point, what we need to modify is the proof of Lemma C.8 Note that Theorem D.2 implies that a = Cd. p = 3d42
and the localization process in the proof of Theorem 2.4.
For Lemma C.8, from (150), it suffices to estimate (6) and wh (2) (d), since vh (aa())) remains unchanged.
(0)
For (1()) we only need estimate the global Rademacher complexity. Specifically, from Dudley's theorem, we have
(F) x
VT
0
cP
Va
(90)
Vn - h
Va
2 vn"
Thus,
Va
VH
* (6) S V8
(91)
2- vn
For (12 we can replace (155) with (89). Thus we can obtain Lemma C.8 under the new covering number assumption.
It remains only to modify the localization process in the proof of Theorem 2.4, i.e., the procedure following equation (67).
We take Pa n पीं so that n- + - ,
26

---

Refined Generalization Analysis of the DRM and PINNS
Then when k O, (68) becomes
sup [(E(w) - E(u")) - (E)(u) En (u))) MEJo
Va
247
va v Pb
10o
p+2
+
+
+
2 p
2 - p Vn
(92)
(u) E(²)
Va
Va
Po
1Po
t
p+2
+
4
2--P)
p
vn
72
(()) E(u")
Va
2+p
tpo
t
4
2
--11
n 11
where the second inequality is due to that (u) {w²) 2 and the last inequality follows from that n S n 227 = po.
For 0, we now only need to focus on the first term the right-hand side of equation (89), applying Young's 's inequality with 2+p²0 2-p* we have
Cva 2-p n PPk
n
2-m Pk
COV C2-p
2
n
b
CVa
24p
1 -p p
$ C 2 -p
11
FIZ
+
C 4
(93)
1 (C/a)
12 - p
$ C 2 2-pp
11 P
+
2 Pk-1
C
1 CVā
1
$ C p-p/
p
+
[E(u)) E(u")]
where C' can be chosen such that the last inequality holds for all Fe-
Finally, by following the remaining steps in the proof of Theorem 2.4, we can conclude.
(2) For the static Schrödinger equation, since the proof utilizes the local Rademacher complexity, we only need to provide an upper bound for the fixed point r". This can be achieved through the Dudley's 's theorem, similar to (90).
B. Proof of Section 3
B.1. Proof of Proposition 3.1
Proof. (1) The proof mainly follows the procedure in the proof the Proposition 2.2, but the tools from the FEM may not work for ReLU2 functions. Therefore, we turn to use Taylor's theorem with integral remainder, which enables us to establish a connection between the one-dimensional C functions and the ReLU² functions. And the method has been also used in Klusowski & Barron (2018); Xu (2020).
Recall that Taylor's 's theorem with integral remainder states that for f E. IR that has 1 continuous derivatives in some neighborhood U of æ o, then for E U
+ 1 (a)(2 a)+ + ...... + (() f(x) = f
hd (x a)k + I ((++1) (x t)*
di.
kd
Similar as the proof of Proposition 2.2 (1), for any f E (22), we have
f(x) / Re
g(x, ((dww),
27

---

Refined Generalization Analysis of the DRM and PINNS
where B fza(1 (1 [11 (w) A(dw) + [=11)) and
B cos((s + ((w)) (1 + 1011)
g(a, w)
Therefore, f is in the (9t) closure of the convex hull of the function class
B cos(w x x+tt
Geos(B) =
(1+ (11)) : WERRER
REER
Note that any function g(z, w) (cos(w-r+()) a composition of one-dimensional function g(2) B coa( w/ ا+++ and (I+ w|1)
(1+|=11)
linear function | with value in [-1,1]. 1]. Therefore, in order to prove that is in the H²) closure of the convex
hull the function class Fo, (cB) FFa (-cB) ( U {0), suffices to prove that is in the ( 1]) closure of the convex
hull of the function class Fl (cB) Fl (-cB) (0), where
02
(b) == (b02(w x x t): W/ = 1, e |-1, L}} and Fo,, (b) = 1
:e=
for any constant à E IR.
For
B coss( cos([w]12 +t) B[cos([w]) 2) cost sin([w]) sin
g(2) =
(1 + k|133
(1 + w0113
with [-1, applying Taylor's theorem with integral remainder for cos([w> ([o|[[11 12) and sin([w[i=) the point 0, we have
Wol 22 1// |Wlt sin((w[1s)) (2 s)²
cos([w]12) =
2 ds
and
sin(lw) 12) |w|1 1// [wl cos([w]1s) (= - 8)2 ds.
Note that 2,,2,1 can be represented by combinations of ReL.U² functions, specifically
2² = 02(2) + o2(-2), (2+ 1)2 - (z - 1)² 1=+ + (z 1)2 22
4
2
Therefore, we only need prove that the integral remainders are in the ([-,,,, closure of the convex hull of the function class Fl. (cB) UFL FF, (-cB) {0}. In the following, the constant may change line by line, but it is still universal constant, so we still denote it by c.
Due to the form of the integral remainder, we consider the general form M(=) = So p())(2 =) with E C((-1,1)) By the fact that (z - s)2 = (a- s)} + (-2+ + 5) we have
L. ((s((= s)²ds = S' ((s)(2= - s) + L' ( A + A2
(000==s)dds
In the following, we aim to prove that
A A2 [ ((s)(2 - (o)(2=s)2dde S ((-s))-2- == B1 + B2,
which enables the method used in the proof of Proposition 2.2 (1) to be feasible.
(1) When z 2 0. it is easy to obtain that
A '' p(s)(z (s((a-s))dds / ((s)(== ()))(==)){dss) B,, and A₂ 0, B2 = 0.
(94)
28

---

Refined Generalization Analysis of the DRM and PINNS
Therefore, A1 A2 B1 B2-
(2) When < 0, it is easy to check that A1 B1 = 0. Therefore, it remains only to check that A2 B2. For A2, we can deduce that
A2 l ( 2
(0s)( --2 + ds
I. p(s) (=2 s) + { p(s)(-2 $)
(95)
(())( -= + t)
S. x0 ((-v)(-== v) dy B2,
where the third equality follows by that 12 (())(=()=))))))))))))))))) -= + ds= and the fifth equality is due to the variable substitution
y.
Combining (94) and (95), we can deduce that
h(=)= L: (0s)(z - d) L p(s)(z ((s)(2-s){ds L. (--s)(=2--s))) - s)
(96)
The next step is to prove that h is the (1-111 closure of convex hull of Fl (cB) U Fl (-cB) (-cB)
Let h1(=) = So (())((= d), h2(2) = So ((-s)(-2 -=-)) ds, then h(2) = h1 h2(2).
Note that (=) lo' 2x((s( ds and (=) So'
ds a.e., since (z s)+ is differentiable for a.e.
Let {8: be an i.i.d. sequence of random variables distributed according the uniform distribution of the interval [0, 1]. then by Fubini's theorem
2 n (z -;)3
E (2)
n
[1((1-11])
((si)(z -s)
2p((s)(2= si)+ + +1hh" Σ
20((;) 2 dz
E
(2) Σ
12 [2) Σ
1m1
Var(p()) (2) (ar(2p())(2 )+) + Var (2r(((2())))) (2p()) de
C 14
n
where the last inequality follows from the boundedness y. And the same conclusion also holds for (2(=) and l(2)).
Therefore, we can deduce that is in the 2 1]) closure of convex hull of the function class F (cB))FF, (-cB)u(O).
Then applying the variable substitution yields that for any f (S2) and such that
0, there exists two-layer d2 neural network
11((x) Σ a,oz(wi . x + ()) Se
(97)
where [w/l
1, |tal
I 1
Σ
Σ layl S cllff (1)(()))))))))))))))))))) and e is universal constant.
Just the proof of Proposition 2.2 (1), remains only to estimate the metric entropy of the function class
F² = faz(w x + t) |w/1 1,4 (-1,1)
29

---

Refined Generalization Analysis of the DRM and PINNS
under the H² norm.
For (wi, (1), (w2, t2) E (1) x [-1, have
||02 x x +1) 02 (w2 t2) (
||02(1 x + t1) - 02(w2 x + t2))((a())) + 1112wr0(wr x+ 2wzor(w2 x + t2)lll
[wiwwn (w1 2wziway (w2
= ii) + (i + (iii,,
where we denote the i-th element of the vector wi by wis for 1, 11.211ii5 d.
For (i), since d2 4-ppschit in [_,
(i) = ||o2(|] x + (1) 02(w2 x 12) (0))
I [6l w2) x (41 - 12) 12)
(98)
S 32( w2 lf
t2 -2
For (ti),
(ii)
[1 (w1 x + (1) 20z0(w2 + t2) (1
4|][(w] - (w) r + (1) + (2(0(() I + t1) olw2 x f2))11l1??200)
(99)
[w] w2) x 4 1 I (0) + 81l + +11 own x + b2) ())
- 32/w1 - woll + 16( W1 w + |t1 t)
where the first inequality follows from the mean inequality and the boundedness of For (dit),
d
(itt)
2wgntay (22 l|
1=1
[Wiw1) (owWyy + W2nl2y ((un ((u2=tt22)))
(100)
< 8
WhwWy - 22 + (W2)w2y)) (1 (o) -100m0t00000 (2a00)
1=1 ==1
2 2/1211 - w2r/ ///1 2 + 2/wir w2y 12 ( 2 + (W2rw22)) (1 - 2[02 1122(0)
1=1 |=1;=
32 w2 i 8|1 (w1 (ww2 200 12
where the last inequality follows from the fact that |wi| < |wl|1 1, /w21 S [12/ =
Combining the upper bounds for (i), (ii), (idt), we obtain that
||α2(w] or (1) 02 (wz + 12) P I 112([wn +|1 1212) 8|| (woo+t,20) 22 (101)
Therefore, based the same method used in the proof of Proposition A.3, deduce that
en (F2) S en 3a.
Finally, applying Theorem in Makovoz. (1996) (see Lemma C.4) yields the conclusion for f in B3).
30

---

Refined Generalization Analysis of the DRM and PINNS
(2) Recall that based the spline theory, Belomestny et al. (2023) has demonstrated the approximation rates for Hölder continuous functions with sparse ReLU neural networks. Then Belomestny al. (2024) extended these results for sparse ReLU neural networks. In fact, the approximation results also hold for Sobolev functions, we only need to replace the Theorem 3 in Belomestny et al. (2023) with the results from Schumaker (2007) on approximating Sobolev functions with multivariate splines. For simplicity, we omit the proof.
B.2. Proof of Theorem 3.2
Before the proof, we first provide some preliminaries about the entropy method, which is a common method to derive concentration inequalities. For Ω = IIk=1 S4. = I]k=1 Mig, where μ. is a probability measure, let (S2, Σ) be measurable space and A(22) denote the algebra of bounded, measurable real valued function on For f E A. A,8 E I, define the expectation functional as
Elgef1] Elear]
Eag [g]
Elge for A,
where Z = =Ele25 is the normalizing quantity. Then, we can define the entropy as
Eint 3E [ʃ] log Zzz-
The connection between the entropy and the exponential moment makes the entropy method popular for deriving concentra- tion inequalities, i.e.,
log E|c s Sd Enty(7) dr
(102)
holds for any f E and B O. For any real-valued function F on Ω and y 24 for (1,...,)) define the substitution operator St F as
(F) .. ,2,) == F(x 3k-1, M, 2K+11 ,,,,, i.e, the k-th argument is simply replaced by y. And define the operator V2 A + by
(103)
ViF(z)= Σ Bynee -
(104)
Proof. Assume that sup sup
[f(())] < b and 1/2 sup Σ Var(f((X))) Sr. JEF(
15t5T sEt,
Let
T Σ t=1
N:
1
Z = sup fcF
(rn
P)f = sup
No Σ f((x)) Ef(())
(105)
T
fEF
T
t=
1=1
and
F(x) =
1 sup
n
f((1)) - Ef(()),
(106)
2b fEF
N
t=1
where n = minjgisT N2 and = (a),
1
No
Define
T
N.
n1
W(x) = sup
(1((+)) - EA(())) - (107)
fEF
Similar to Theorem 38 in Maurer (2021) for the single task, fix (Sw)) and assume that the maximum in the definition of achieved f (f.....ffy F.
31

---

Refined Generalization Analysis of the DRM and PINNS
Then for any y, (F(x)
Sy Fx))+
n 26N,
(f(()ee))) - fe(w))+, therefore
ΣΣ
By Byrpert
[(LF -5,
T
((f((=t)ff((u))) -f(w)))))))
46
No
1=1
T
No
n
(f((z))- -((o))
48
N?
(108)
T
n
Bugrete (fflct) - Ef(())) (f((v) - Bf(())))) ]
412
N?
1=1
T
2
N.
Σ
Σ
(fe(x)) - ( - Ef((x;))²
4b
N
t=1
s W
where X; follows the distribution jdt, i.e., pu = Pc-
Therefore, V S W. Then equation (26) in Maurer (2021) yields that for 0 < <>SB < 2,
7 Entr(r) S 2 y log Be V2F
I 7 Y log Ee
(109)
Next, going the prove that W is self- coff-bounding. that Lemma 32 (i) in Maurer (2021) can be applied to bound We" Assume that the maximum in the definition of W is achieved at f = (fi,
fr) E F, then for any y.
n² (W W/, S N?
n²
((feaz)) - -E(())) (f((v) - BF(X)))
46²N? (fe(xt) Ex(())))
therefore
T
N.
(xW(x)= = Σ
E
Eyrup -W
- (t))
1=1
1=1
N
1
E Bynne [((f((=)) Bf(( -(F()) E((()))
1664
N
(110)
Σ (f((1)) - BA(X
1664
T
π²
(f(()) Ef(())))²
4
2
W.
Combining 10) with Lemma 32(i) in Maurer (2021), we have
[[ + +W >F[W]
log ] < 2
(111)
1-7/2'
Plugging (111) into (109) yields that
Ent z 7
log W
1
, [[[
722
B[W]
(112)
2-7
2 2-7 1-7/2 7/2
(1 -//2222
2
32

---

Refined Generalization Analysis of the DRM and PINNS
Combining (102) and (112), conclude that
f Ende(2)
log (FF-FF) - 3
dy
72
[
(113)
(1 - 2/222
[[
1 3/2
2
In fact, the above inequality implies that F sub-gamma random variable. Thus with the following lemma, we can derive the concentration inequality for F.
Lemma B.1. Let Z be random variable, A, be some constants, for any (0,1/B) holds
log Ble < AX2
2(1 BA)
then all æ 2 O,
P(Z EZ EZ
v2Ar Bx) se
Applying Lemma B.1 with A = WWW, B = 1/2 for F, we can deduce that with probability at least 1 - e
PSBF + /2aEW + æ
(114)
2
From the definitions of F and Z, i.e. (106) and 105), we have Z
26F of
then with probability at least e
25 nT
bx n.
Z <
EZ
2rEW
+
(115)
Note that EZ < (FF and
T
N
EW
E sup FEF
71
(((()) - -x(())) -XX(()) Ef(()))²
402
N
t=1
N
1
E sup
11
[[()(2)) - EF(())) B(A(X)) - Eff( + EF(()))
412
FEF
t=1
2
N.
2 n
2E sup
d(A(())) Ef(()))² + sup 2
E(f((x?) BA(X)))
412
FEF
2
FEF
No
(116)
T
N
$
(SbE sup
π
(f((())) - Ef((x))) - -2nTr)
402
N
fEF
1=1
(16bnTR(FF + 2nTr)
462
4nTR(F)
nTr
b
26
where the first inequality follows from the standard symmetrization technique and the second inequality follows from the contraction property of the Rademacher complexity.
33

---

Refined Generalization Analysis of the DRM and PINNS
Plugging (116) into the concentration inequality for Z, (115), we have
2b
br
Z 2sEZ
EZ
V
22
n.
2b
(FF
ni'rr
ba
< 22)(F) +
2x0
+
nT
262
n2
=2((F) -2 8laR(FF)
DT
br
(117)
nT
nT
nT
(boR((F
bx
2R((F) + 22
2 n.
n/
nT
or
1 + 4
bx
S 21 + (RR(FF) + +2
+
nT
a
nT
where the last inequality follows from the inequality 2vab I oa +
b for any 0, 0.
B.3. Proof of Theorem 3.4
In the following, assume that for any f (fi... " 5.5 Define
UN(F) = sup(Pf - Pxf).
fEF
Lemma B.2. For normalized function class F., F
F Pf2v f: |EF]
(118)
and assume that for some fixed constants K > 1 and r > 0.
UN(F,) <
r
(119)
bk
Then for any f E F the following inequality holds:
K
r bk
Pf
Pn +
(120)
K - 1
Proof. Let us consider two cases:
P: rr.
Pf
For the first case,
p f F,, therefore
r K
Pf < UNS+UN(F
+
PNf +
K
K-1
bK
For the second case, pf f p f F thus
P r
pp f PN pfz ft S PN ppp f +
r
r
r
bK
Basic algebraic transformation yields that
Pf
PS K
PNf
< Puf
bK
which implies
K
K
r bK
Pf <
PNf < K-
PN
K-1
34

---

Refined Generalization Analysis of the DRM and PINNS
Lemma B.3. Let us consider a sub-root function (r) with fixed point r" and suppose that Vr >> r".
(r))= 2 bR(F,)
(121)
Then for any K 1, we have that, with probability least - e " Vf E JF
K
+ 32Kr* (10b + 8
Pf K -
PNf
(122)
b
nT
Proof. The aim is to find some such that UN
then applying Lemma B.2 yields the conclusion.
bK
Note that the variance of functions in F, is at most r. For any f Fr, we consider two cases:
Pf rr. 2: Pf >r
For the first case, p E Fr, thus Var (ाpttT)) = Var(f) S Pf2 ST.
For the second case,
Var T P = Var (PFad) S P
,2
p
= Pf2
<r.
Then applying Theorem 3.2 for Un(F,F with a = 1, we have that with probability at least 1 - e
UN(F,) S 4R(F,) +
ET
5bx
nT
nT
(r)
2r
5bx
+ 2
nTT
nf
VT
5bx
+ +2
n.T
nT
AVF + B,
where the third inequality follows from the property of the sub-root function, i.c., (0)//F ( = vr for any
> 1" and A 1yF +
2 nf.
B
Sbg xT
Solving the equation
AVF B bK
yields that
bKA + 1KK2 A + 4bKB
VF
2
Thus
A2
r 2 2
and
T 62 + 26KB.
Therefore by Lemma B.2, we have
K - Px + r
Pf
bK
K
K Px + 4-b + 2B
(123)
K
16r
4x
10br
Pxf
2bK
82
nT
nf
K
32
(106 + 8bK) a)
PNf+
K
nT
35

---

Refined Generalization Analysis of the DRM and PINNS
There remain some problems regarding the selection of the sub-root function & and the computation of its fixed point. Just as in the single-task scenario, we can take j as the local Rademacher averages the star-hull of IF around O.
Specifically, let
((r) = 1 (f: f star(F,0), sr} 145² (ng(
(124)
n/
where star(F.0) faf: f E F E [].
Note that the normalized function class F, defined in the Lemma B.2 subset of the function class {f stor 0), f r). thus ()(r) 2 (RF,.
For the first term in the definition of (r), i.. (124), with the following lemma, can translate the ball in (P) into the ball in 1,2 (PN), so that Dudley's s theorem can be applied.
Lemma B.4. Let G be class of vector-valued functioo that map X into [-b, b)"
with b O. For every x > 0 and satisfy
14b²x
2 16bER :::ge:e9e g 9 Pg sr} +
(125)
nT
then with probubility at least 1 - e=
(g 9: Pg r) c (g 9: PNg2 2r).
(126)
Proof.
Define 9, = := e g rr.
Note that ||g2 < 62, Var(g") < Pg - 12 Pg v b². Then applying the Theorem 3.2 for 9, with a = 1 yields that with probability at least for any g E 9 such that -2 g,.
ber
5b2r
PNg² < Pg + AERN g g Pg <r) +2
2
+
ni
niT
7
s rr+ 80ER 9. Pg < +
r
2
nf
2r,
where the second inequality follows from the contraction property of the Rademacher complexity and the mean inequality.
Remark B.S. Although the contraction property used in the proof Lemma B.4 slightly different from the standard form (see Lemma 5.7 in Mohri et (2018)), it just an adaptation of the standard one.
Specifically, let T be 1,-Lipschitz functions from R to IR for 1, m and 01. on be Rademacher random variables. Then for any set A C R., the following inequality holds.
EG sup aE A
Σ
< Eg sup Σ
0,444
aeA i=1
For completeness, we give a brief proof. By the Fubini's theorem, we have
E, sup Σ (as) Ery,o A im1
Eor, [sup Um (a)) + om Pm (mm)), aEA
1-
where Um (()) = Σ (a:))
1=1
36

---

Refined Generalization Analysis of the DRM and PINNS
From the proof of Lemma 5.7 in Mohri et al. (2018), we know
Eg m (sup (m) () + om & Pm (am)) S Eg (sup 1m-)(aa + omtmam] aEA
acA
Proceeding in the same way for all other 0, & m) leads to the conclusion. In fact, we have used the conclusion with x² in the proof of Lemma B.4.
(1,) Ni
With Lemma B.4, we can bound r* as follows.
Lemma B.6.
16bERN (s: f star(F, Px 22r) + 16b² + 14b² log(nT)
(127)
nT
Proof. From Lemma B.4 and the fact that
(+r()r) = 16bERN (f: f E star(F,0), Pf <r²}+ + 1412 log(ni)
nT
we can deduce that with probability least 1 1
nI'
{f :f star Pf.) s rr²} C (f f star PN PN 2r"}.
Therefore,
r" 16b BRN(F: f star (F.0), 2 2r b
14b² log(nT)
n'''
n'l'
16h2 + 14b2 log(nT)
16bERN (f f star (F,0), PN s 2r*)+
nf
Now, we are ready use the Dudley's theorem bound the first term in the right.
Specifically, define Far = {f: : f star(F,0) 0), P2 < 2r), with the samples (Xi) (x)) fixed, define a random process (EF),EF.,- as
X
ff(( for f (fr,... (r) E F.r
(128)
From the fact that of is sub-gaussian, we can deduce that for any and f
F.r
EEe A(X Ee
S
S f
where
1 and Vnl
T N
a(f.f) =
Σ
(129)
N
It implies that |X,, 1lv. CKd(f. f') with universal constant C.
Then using Dudley's theorem yields that
diam(F...)
d. (ogNNFF.r,,,da s CK far (f.sN,.r.ddd.o
E sup Xy < CK JEF,r
V log sr,
(130)
where diam (Fs,r) = supj.f' .F. d(f.f).
37

---

Refined Generalization Analysis of the DRM and PINNS
Proof of Theorem 3.4: In the following, we assume that F is a parameterized hypothesis function class be determined. When considering the framework of PINNs for the linear second order elliptic equation as MTI. the function class in MTL. associated with F is defined as
= {u ([Q((L((() - (00(())))))((( g(w)))) u
(131)
Note that here we use notation represent function in F and to denote the corresponding vector-valued function associated with u.
Then the empirical loss can be written as
2
(u(
[
N1
aiyl (Xxx)),)u(X) + Σ (xx(x))))((x)) +xXx))((x))) ((x())))))())))))
s(xa)
N1
1=1
f
N2
Jan
+
(u(Yx) 9(Yn))²
N2
uP,
where = (Ni, N2) and n = min(N1, N2).
The aim is to seek uv e FF which minimizes Lx. equivalent seek uv E IF which minimizes Pau i.c.
UN E arg min Pru.
(132)
WEF
Assume that u" is the solution of the linear second order elliptic PDE and there is constant M such that Jusyl [bu], le), lsl, |u"1, |0:u 1,, | S M and (u], [0.ul, [0,yu| S M for any F, 1 S i,j < d.
Then sup u max( Therefore, with probability at least -t
100,10[(u g) - c(1 M + ( = b with universal constant c.
tV (Var(uy)
2bt
PNuN S PruF & Puz + +
2n
2n
ttPuu
bt.
(133)
< Puf + +21
+
2n
3 Pur
2bt
+
2
where uy (Lu, - (1) - -)
ur E arg min || - u. 11?rr and the second inequality follows from vector version of the Bernstein inequality.
Theorem 3.2 by taking F {ur} and 4, T 2, which be Here, we define the approximation error capp ||u u" ||)))
Then applying Lemma B.2 with K 2 yields that with probability at least 1 - -
64r
13bt
Pun < 2P 2 +
+
b
(134)
64
17ht
< 3Pu +
+
b
which implies that
128,
34bt
[(ux) = 22 uN S 3C(uF) +
+
(135)
b
38

---

Refined Generalization Analysis of the DRM and PINNS
Note that [(ux) can be bounded by the approximation error, since for any a E (2)
| (Lu - ff)dx +
/ 100
[(u)=
9u
bi0,u + -ou cu--8 :))
2
1
dr + I (u-g)ddy g)
Σ
aij +
Σ
h==11
on
1
Σ
d
aiydiy u²)+ +
Σ
d
(cf (0)) b,0i(u u²) + tcuu ceu--ii
dx + / (u 2
(136)
12
11=1
==1
08
d
$3
Σ
aij0is(u- u²) +
Σ (u + (dlu- (c]udr]))dr+
/ (u- u² Pdy J0n
(,==
s M2 u² 1 (0) + CTr,, 0)2 Iu u" (()
- M3 + (TT,, ()), u u² =1
where in the last inequality, we use the boundedness of aej by, and the Sobolev trace theorem with the constant C(Tr, 9t) that depends only on the domain f
Thus,
C(uF) & (3d + ((Tr,
(137)
and with probability at least 1 - -2e- -t-
128r* 34bt
[(uN) 2PNuN < 3(3d M² + C(Tr, )
kap
(138)
b
It remains only to bound the fixed point r With Lemma B.6, it suffices to bound the covering number of F under d, which is done in the Lemma C.10. Thus, we have the following results.
(1) For the two-layer neural networks, we know
log. d, S emd log (
(139)
where c is a universal constant.
Therefore
md
cba log n
r S cb
log
de +
n
J
md
dc ())
1 (1)
cb² log
cb²
V log
(140)
mdr
26
cb? log n
S cbl
log
+
Vr*
n
mdr
et? log n
$ <bb/
/lo n
where second inequality follows from Lemma C.7.
implies that
mogdlogn
(141)
n
(2) For the deep neural networks, we know
K log.N(F, S CKd log
)
(142)
39

---

Refined Generalization Analysis of the DRM and PINNS
where C' is a constant independent of K.
Similar to that in (I), we have
Clog K + log n)
(143)
with constant C independent of K, N,
C. Auxiliary Lemmas
Lemma C.1 (Bernstein inequality). Let X;, Si S n be i.i.d. centered random variables a.s. bounded by b 0G in
" absolute value. Set o² EX2 and Sn Σ X,. Then, for all 0,
2
bt
P Sn 2
S see
n
3n
Lemma C.2 (Hoeffding inequality). Let X;, 1 S i S n be i.d.d centered random variables a.s. bounded by in
absolute value. Set S, = Σ Xú then for all > 0,
==1
21
P
|p by S 2e 2
Lemma C.3 (Bounded difference inequality). Let X,.... Xm E xm be set of m 2 1 independent random variables and assume that there exists C1, Cm such that f xm + R satisfies the following conditions:
If " -s.-.. ; [mm
f(x1," ... -y. ;. Tmml S Ci.
for all i E [m] and any points #1. Im, Iq X. Let f(S) denote f(X1,'.. Xm), then, for all 0, the following
inequalities hold:
-26
P(((S) - B(J(S)) 26) exp
Σ1 c
26
P(f(S) - E(f(S)) S --0 exp
Σ ch
Lemma C.4 (Theorem in Makovoz (1996)). Let + {b),, ... be arhitrary bounded sequence of elements of the Hilbert space H. For every f E H of the form
Σ coo. Σ ccal <
and for every natural number n, there is a = Σ; a,d; with at most n non-zero coefficients a, and with Σ, |a; S E1lo. for which
|| - -gl < 2e,, (D)n 1/2 Σ lcal-
The definition of metric entropy en is given in Proposition A.3. Lemma C.5 (Covering number of (1) in the 7,1 norm), For any >
12 d-1
(1
Proof. By the symmetry (1)( it suffices to consider the set
((11, xa) E
(144)
40

---

Refined Generalization Analysis of the DRM and PINNS
(V)B l1,e) S (S, l).)).
Note that for (#1,--. aa) aB((1) za is determined by 21,-. Id--1 Thus the problem of estimating the covering number of (1) can be reduced to estimating the covering number of
S [(111-. x8-1): + 3a-1 S 1,,,,,22000 2.1siiid 1}.
(145)
which is a subset of (1).
Thus, there exists 2- cover of B Bd-1(1 (1) with cardinality ($)d-1 which we denote by C. Although C' is also a 5ー cover of S₁, the elements in C' may not belong to St. To fix this issue, we can transform C' to a subset of S₁ and the transformation doesn't change the property that C' is 1- cover of St. Specifically, for (V1,-.. Ma-1)) E C, we do the transformation as follows
By Lemma 5.7 in Wainwright (2019), we know
which
(y1, . - (11 (012)), ... vdd- I 120)).
Note that
y1 (m120) +... ya- 16-1 S 1vil + ................................ + 1ya-1l S
(146)
and for any (x1, ". ,.d-1) E Si
ya- (va-1 10pa-1203 | 1 20: + ... |Ma-1 1..0-11-11------
203 S |2] yil +... |2a-1 Ma-1l:
(147)
which imply that after transformation, it is subset of S₁ and also 1- cover Sj. For simplicity, we still denote by C.
Now ready give e-cover of S via extending C' to subset of (1). Define C. {(y,.". Ma)
(y1,-.. (8--1) C, ya (y1 ..... vd-1)}
Thus for any Ia)) E S, since (I1, xa-1) E 81 and C' is a h-cover of Sr, there exists a element of C, we denote it by (* , ( such that
I1 21 + | zd-1 S c 2
(148)
Note that for za = 1 (21 + ... zd-1), (21, 2a) Ce and
| 211 ... + |z-1 zd + 1x - zal
| 21|
|T8-1 - zd- 1| + |2| - 21 + xd-1 z2-11
2([21 21 +.... + [dd- zd-1l)
S 6,
which implies that C. a e-cover of S.
Recall that |Cel = |C| (-)-_1 then (OBB (1,,)) < 2d (9)==11 = =2 212)
Note that in this lemma, our goal is not to investigate the optimal upper bound, but to give an upper bound with explicit dependence on the dimension.
Lemma C.6 (Equivalence between metric entropy and covering number). Let (T, be a metric space and there is continuous and strictly increasing functionngfunction f IR+ + R2, such that for any 0.
e) f(e)),
Then for any > 0,
(x < f
where -1 represents the inverse of f.
Proof. It's obvious, since N(T, < (()-((m)) = n.

---

Refined Generalization Analysis of the DRM and PINNS
Lemma C.7. For any 1, we have
1 log de I 2 log
4
Proof. For 0 < x S 1, let f(x) = v log 1. g(x) = vo, h(x) = x log 1. then f(x) = g(h(x)). Note that g is increasing,
concave and h is concave, thus
f(Ax + (1 = g(h(Ar + (1 A)u))
2 (A(((()) - (1 A)h(n))
2 Ag(h(xz) + (1- N)g(h(())
= =)((2) + ())))((),
which means f is concave in [0, 1].
Let e = y², then
f V log de =
(2)
y log
dy
2
2 x
log
2
2
3 -
log
3
4
< 2x
log
where the first inequality follows from Jensen's 's inequality.
Lemma C.8 (The remaining part of the proof of Theorem 2.4). For the function class F and
g =
- (2())))))) FF
assume that for any 0.
( a.s. and N(9. (I(e()))))) (2)) a.s.
(
N(F. 1l (Ia(())))))
for some positive constants b, a. 3 with b > sup [f], 8 > sup lg). Then we have that with probability at least 1 - -t
sup (E(u) - E(u")( - (En(u) - En())) wEF;
2 log(28 (2)
M28a log(28
M2
C
(149)
n
n
M2t
aM
46
log
n
n
M
where
FB == {u E F; ||u u ( 2τ}
and C a universal constant.
42

---

Refined Generalization Analysis of the DRM and PINNs
Proof. As before, rearranging (8 - ((a,)))) Ex"))) yields that
sup (E(u)- - E(u")( - E wEFs
(( (z()))
I [(Vu(2)) 2f(x)u(z)) - 2f(x)=((( (x))) dr
sup wEF(d)
[(Vn(())) 2((X))((x))) ((V''())) 2/(Xi) (X.)]
((x))d (1/ 1
2
- n Σ u(x))
1
1
(xi) )
)
(150)
Σ
n'
n
1 Σ (((s) 2 n
1
S sup (P- - Pn)g + sup sEG(d)
+
T67(6)
n
1=1
v1 + 1 (8) + (1) (6),
where
( = ((Vv(()) - 2/(x))(x)) (
2/(z)) (x)) u E ||u u ((2 5.}.
Applying the Hoeffding inequality for 2 (3) (8), we can obtain that with probability at least 1 - e
2 S n
n
2M
1 (0)
u (X:)
(151)
n
For (6), we can deduce that
(1s u(z)dde) -
)
-(
(8) sup
n
wEF())
1=1
[Pu)) (Pru)
sup
MET(6)
[Pu)) -((Pnu PP)) + Pu)
(152)
sup wEF(1)
= sup wEF(()
[P(Pu)((P - Pr.ju) - (Pau - PP
< 2v8 sup (IP-Pa)ul. #EF(()
where the last inequality follows from the fact that for any u E F()).
PPu| = =15 udæ = = =15 u"'"dd ((((u-u))d
(f((u-
S
Vo.
Note that here, we only need positive upper bound for (6). In (152), for fixed F(6), (P (P-P) -P))) s 0. then it is obvious that
2(P - (Pru - Pu)² < 0 < (P
(P - PP))) > O, then
2(Pu)( - Pa)uu) (Pru - Pu)2 < 2(Pu) - Pa)u)) = P|Pul| P,)ul..
Therefore, in any case, (152) holds true.
43

---

Refined Generalization Analysis of the DRM and PINNS
Then, to bound yn 2 (6), suffices to bound the empirical process inequality and the symmetrization technique, we can deduce that with probability least 1 e-
sup wEF(6)
1(P - P.)ul. By applying the bounded difference
2t (P - Pajul M M
sup (E)((
(PP (PP-Pa)al E sup wEF(()
n
S 2E sup
Σ {x((),)
+ M V
(153)
wEF(
t=1
1
2f
< 2E sup WEF
+ +M
The first term is the expectation of the empirical process and it can be easily bounded by using Dudley's theorem. Specifically,
sup WEF π
Σ {x((
-xxL,
sup MEAUl- n
1 Σ Gu(X;)
12 M
$ Ex
VIggN(FU(-F). |l (lxa(P))) u)du
Vn Jo
12 M
S Ex
Vlog2N(F,11 11 (Iea(P))) uddu
Vn Jn
12
M
1lgg2 2 log ddu
(154)
Vn
2M + vab [ x2
12
Vlog du
Vn
12
46
log 2M + 2 aM
2 v log
Vn
M
a.M²
4b
CC
log
M
where the fifth inequality follows by the fact that b M and Lemma C.7.
Now, it remains only to bound w/t (6).
Recall that
(6)) - (VV (F -2f()))))))))))" (z)) (x))
u uE Fl ---luuuu1 201121100) S 57.
Therefore, we can deduce that [g| < 6M2 and Var(g) I P(g²) < 4M2S for any g E G()). Then, from Talagrand's inequality for empirical processes (Theorem 2.1 in Bartlett al. (2005) with a 1), we obtain that with probability least
1 e-
8M
16M
sup (P (P- Px)g < AER, (G(8)) + 9E9(6)
(155)
Note that Pg < 2M for any g (( therefore
ER. S (g.. Pg² < 4M25).
The right term frequently appears in the articles related the LRC and can be more easily handled than the term on the left.
44

---

Refined Generalization Analysis of the DRM and PINNS
By applying Corollary 2.1 in Lei al. (2016) under the assumption for the empirical covering number of G, we know
a. log(8/))
log
Eg E 9: Pg² 4M28) s
(156)
n
where C' is a universal constant.
Combining the upper bounds for v/n (0), v (5) and oh (6), i.e. (151), (152), (154) and (149), the conclusion holds.
Lemma C.9. For the empirical covering number of F and G defined in the Lemma C.8, we can deduce that (1) when F Fm.,1 (B), we have
cB
mld+1)
emax(M B²)
crad
N(F, (PPa),)
and (N &
(157)
where M a upper bound for lfl and c is universal constant.
(2) when (N, I., B), we have
CN N log Lya
CN2 22 N log L.
Cn
Cn
N(F, ((P,))(
and ((()),
(158)
where C' is a constant independent of N, L and n CN (log N log L)
Proof. (1) For the function class of two-layer neural networks, recall that
Fm. (B)
(w x + +4)
Σ hal S B, |0i1 1,4, E 1-1,11)
Due to the Lipschitz continuity of a, we can just consider the covering number in the L norm.
Without loss of generality, we can assume that 1. Then
un = Σ (wt x +tt (l), 1,2,
we have
[x) - u2(xx)] =
Σ x (w x + (2) 720 + 1?)
m
h (w) x + ()) - (f x +1)
1=1
m
16d (W) x +
(a x t olw x +))
7 - h will + |t t ) 2/7
t=1
where the last inequality follows from that a is bounded by 2 in absolute value and is I-Lipschitz continuous. Therefore, when
Σ hi - -il < 4 and w - -w - 1.1 +lS sisss m
we have that supres (x) uz(x)) S, which implies
45

---

Refined Generalization Analysis of the DRM and PINNS
) m mld-1) 1) (-)m m
(
m(d+1)
(PP,))) S
where is a universal constant.
Therefore, (B) cB (=B) m(d+1) where assume that B 2 1.
Recall that
g - (=(((u(t)222/((uuub) (Vv 2f())) (1)) u FF.
Since u" is fixed, the estimation for the term f(x)u(x) can be conducted in the same manner as for F. Therefore, we only need to estimate the first term
For
Uk Σ M Mal or Fm) 112
have
Vu, 2- Vuzg (
2|| Vui V u2 (PP)))
2||
hew (w - 7/,6 (uf (20
m
< 22)
Σ
w/ (ut - n7w? (nt 120
1=1
m
|1( - 78 )w (w) + (/ (w] [u]=2+1/00 - wf 4uf o0
2
Σ
I/ 7/2 + 2
w (1:-0+1/20) w/
1=1
i=1
2
7/
+ 2
wfll + (w] (=]:a++)20) - [122= = h
1=1
where the first inequality follows from that Vuz S Vuuh s 1 for k = 1,2 and the second, third, fourth and the last inequalities follow from the triangle inequality.
Thus
Σn hit 7/1 4
and |w will l1 (10 4,2
S iiiss m,
we can deduce that ||| // Uuz (l) Cc
Based on same method the proof of Proposition A.3, the (P.) covering number of the function class (/uu/2 u E F} can be bounded
()) m ())
(d- (d-1+2d)m
( 3md
Combining the result for F. we obtain that
cmax(MB B²)
and
NG (P(.),,
<
where M is a upper bound for |f] and is universal constant.
(2) Note that the empirical covering number I(FFL?((P),) c) can be bounded by the uniform covering number ((F n, c).
which is defined as
N(F, n, c) = sup NVFFl,,, -1oo))
2,
46

---

Refined Generalization Analysis of the DRM and PINNS
where Zn (21,--, =n) and F 2., {(f(=)), :(z,)) FE.
As for the uniform covering number, it can be estimated using the pseudo-dimension (F)m(F). Specifically, let J be class of function from X' to [-B,B) BJ. Then for any > O, we have
2em
Pdimm(]]
N(F, n,
(Pdim((F)
for vt 2. (J) (See Theorem 12.2 in Anthony et al. (1999)).
From Bartlett et al. (2019) and Yang et al. (2023b), we know that
Pdimn (V) 2 C22 log L log N and Pdim(DD)) S CN2 L2 log Llog N
with constant C independent with N, L, where v is the function class ReLU neural networks with width N and depth
L.
Therefore, deduce that for F = (N,, L, B), we have
(n C. N log 1.)a
(
NFF ((Pa)))
and
CN² 1/ N log 1.)a
Cn
NCG <
with constant C independent of N, L and 2 C.2 Nlog L,), as the width and depth of Φ(N, L, B) ON log N) and OLL log L) respectively.
Lemma C.10 (Estimation of the covering numbers for PINNs).
(1) For F = Fm) with B = -MM,, we have
log. OogN(F,,gg S cmd log
with universal constant
(2) For F = (L,,, W. B; H) with I, I. 0(1), - O(Kd), B H = 011), we have
logNN(F,d,)) s CCdl0g log (K)
where C' a constant independent of K.
Proof. Recall that
(Lu- -)² (-
(-Σ
Σ
azy (r)())), + Σ ((())aa())+c())((aa))) f(x)
w==
and
F= = {u = (L))(L - f(x)) (10[((())))))) - -())))) u NEF}.
(1) For the two functions u ([1)(LL - f)2, (0 9)²), a f)2, (ū g)²) E F, where u, a belong to (B) and are of the form
u(x)== =
m022aux x +t), ū(z)) = Σ
Rkoz( x x +x+)
km
47

---

Refined Generalization Analysis of the DRM and PINNS
respectively. We write as (w1, (2) and (i1, (ie) for simplicity. As for the samples from s and 02, denote their empirical measure as
Ni ox, and PN₂ == N2 t=1
N2
PN.
Σ
8y.
Ni
respectively.
Now, we are ready to estimate du, ú), recall that
d(u, ū) V 1 / Ilu ūz uallEA(PN)) 1|22 u2 (PNN))
(1lu) (ille(rs,) 1lu2 U2 -02211112P00 (p(x))))
which allows to estimate these two terms separately. From the boundedness of related functions, we have
||u1 - u1 uIll, = [[] f)² -)- [2](Lū f)
< cdf M& a))l
and
I22 (2lle
|| (u- g)3 - 110 (1((N)))
S <cMoallllu ūl (((N)))
Therefore, it can be turned to bound [[(u u) ())l,>(pxx)) and 1 ()) For [[ -nn--r))eze ())l))(PM))) applying the triangle inequality yields
[ ()le((())) -
aij ū) + Σ b,0 8) c(u -))l((())))
< Σ
0ig (u- ()lL + Σ (nlu--))llaaa,. + ||c(u ())l)a(PM))
i,j=
= A + A2 + Ag.
Note that 0,u, are Lipschitz continnuus with respect to the parameters, thus for A2, have
d A
Σ
6,0,(u ())l=())))(PP))))
==1
Σ
by (()
m
Σ
2h:
Σ
nw a (we x + (x) - faa x + [s)) (11=(00
1=1
Σ
26:
(7k TW) x + tx) + mn x + tx) - T x x Ex) )
1=1
m
4M
7M
7M
+2
Σ
o x x mwaio To)lle-(0)
48

---

Refined Generalization Analysis of the DRM and PINNS
where the last inequality follows from the facts that [by] M, S i d and we = -wis ww)), Ei=1 | = 1. And we denote the second term by A22, then
A22 = 2M
o (wx x (x) f x + 6x)l
==1
1=
2M
Σ
Σ (w (w)) (s) Fh (o 1x) o( + (
1=1
d
$ 4M
Ki 1ut - 0111 + 2M
>
Fallot 0xl1 + 1tx txll)
i=1
i=1
170
m
4M
FI F
-k/1
2M
Σ
[iee(1wx - wxh1 + lte Ege).
where the inequality follows from the triangle inequality and the facts that a is 1-Lipschitz continuous and ||o (x $. Combining the results for A2, we have
A2 S 4M
lonk
Jel + 4M
Fallwa Wx/1 + 2M
hul((wx wk/l + 114 [x.)
k=1
k=1
Similarly, we have
Aa llc(u a) ())l2A(PM))
S 4M
Myk Txll + 4M
Fiu 1(wa 2xel1 + 12x [xl)
1=1
and
Iu x|| (P(x))
< 4
7
+
Fill( -xk11 1 Its Fx
As A₁ involves the second derivative of 02, the method described above cannot be applied. However, we can borrow the idea from the proof of Proposition A.3.
A (u- ()1()P((N))
m
= 2||
Wk T "Awi Mwikllue [uee - Awn 2
2||
(Trwwk T "Awk -F (on)) Two Awn (wn
< 2 k=1
hawe T Aww Sooe Awa +
Fra Awx (17 100041220)11110pp) Σ
km
For the first term, we have
m
m
TRwe T Awi F1aa Awa 5
Σ
17t =x) T Awall 17x T Aui w/ Awx)]
Mh* -ull + Fiollwk TA(wx wit) + Fo Alwx wx)l
k= 1
m
< M
Σ k=1
Irk 7x + 2/5m|lax (nll))
49

---

Refined Generalization Analysis of the DRM and PINNS
where the inequalities follow from the triangle inequality and the fact that for any a (B(()), I& and matrix A R with |A(i,)) & M(1 i,j S d), have |a. Ayl (A x vl | xxxx x lyl Mly
Thus we obtain the final upper bound for A,.
m
m73
A1 - 2M
2 (hx Fxl + 2/7 (kl1) 2M
Fhl (wertt220) 20000
Combining all results above, we can deduce that
d(u, ú) S old M(ae) Σ (mT - 7xl 15 wx|1)
Fi (ov
1 1t 20 201
Similar to bounding the empirical covering number of g for the two-layer neural networks in Lemma C.9 (I), the covering number of F under d is
c c(d2M + M M
end
c c(d2M411 + M (22))01)
emd
cb
cmd
where is a universal constant.
(2) Note that d(u, ú) s Cl| ( then Proposition 1 Belomestny et al. (2024) implies that
log.(FF11 -llca c) & CK
where C' a constant independent of K.
Therefore, the conclusion holds.
Lemma C.11 ((Agmon al., 1959)). For (+) (200))
d
d
||
sc
- Σ
arjuuuu u + Σ
b cu
+ +
(2)
iJ=1
1=1
H (()
(159)
|| (11
S Ca
agO u >
be + cul (2220000) + ||u| Racaor
i,j=1
where Cn is a constant that depends only on Ω.
D. Discussion
D.I. Over-paramettrized setting
In the context over--arameent vver-parameterization, the generalization bounds for two-layer neural networks may become less meaningful due to the term m/n. However, fortunately, the function class two-layer neural networks in Proposition 2.2 and Proposition 3.1 forms convex hull of function class with covering number similar that of VC-classes. Consequently, extend the convex hull entropy theorem (Theorem 2.6.9 in Vaart & Wellner (2023)) to the H norm, allowing us to derive generalization bounds that are independent of the network's 's width. Theorem D.2 is modification of Theorem 2.6.9 in Vaart & Wellner (2023) to obtain explicit dependence on the dimension.
This section is inspired by two works Sreekumar & Goldfeld (2022) and Liu et al. (2024). Sreekumar Goldfeld (2022) utilizes two-layer neural networks to estimate statistical divergences and establishes a non-asymptotic absolute error bound
50

---

Refined Generalization Analysis of the DRM and PINNS
using techniques from empirical processes. key component is the estimation of the metric entropy of the closed convex hull (see Theorem 58 and equation A.33 in Sreekumar & Goldfeld (2022)). Compared to Theorem 2.6.9 in Vaart & Wellner (2023), Sreekumar & Goldfeld (2022) provides explicit constants with respect to the dimension d but does not offer proof and is limited to the L norm. Liu et al. (2024) considers different type of Barron space (Ma et al., 2022) and derives generalization bounds under path norm constraints the over- weer-parameterized regime. Moreover, Liu et al. (2024) provides a slightly more detailed proof for the estimation the metric entropy of the closed convex hull, but this is still within the L norm, with ((d)) dependence on dimension We extend the results from the L norm to the H norm and establish the explicit dimensional dependence, specifically obtaining an (dd) bound, which is the same as that of (Sreekumar Goldfeld, 2022).
Lemma D.1. Let F be arbitrary set consisting of measurable function f - B of finite H (Q)-dimmeee diam(F). Then for every >> we have
enE
N(ediam(F) con,((F) (Q))) < e
Proof. Assume that JF {fi,'.. , fn).
For given in the n-dimensional simplex. Let Y1, Yg be i..dd random elements such that P(Y -fz) >; for j 1,..,, k and is is natural number be determined. Then we have
EY, Σ A and VEY, EVY, Σ X
Let Yv = - Y, then the independence implies
1 E B| EY, ( = F2 Σ B|y, - EXI S _ (diam(F))²
Therefore, Markov inequality implies that there is at least realization of Yx that have (i)md) to the convex combination Σ)=1 Azfī. Note that every realization has the form k Σ für- where some functions fy in the set F may be used multiple times. As such forms are at most Clk n# we can deduce that
most
N(k (in)
s C Ch+k-1 S 5 M))
where the last inequality follows from Stirling's inequality.
For 1, we can take & [ then the monotonicity of the function et(1 + )** and the fact < 1/2 + 1 s
imply that
k (1
12 k x(e+ TeT ) S
enE
6
(160)
2
For > the right term in (160) is larger than thus the conclusion holds directly.
Theorem D.2. Let Q be probability Q and let F be class of measurable functions with || == sup || (
feF
and
F, s cc ( 0ceci
for some V 2 1. Then we have
log (ellFl(222.cc.mm(( < kV +2) 2r (1) 2
where is universal constant.
51

---

Refined Generalization Analysis of the DRM and PINNS
Proof. Note that every element in the convex hull of F has distance e to the convex hull of e-net over F. Accordingly, given a fixed e, it suffices to consider scenarios where the set F is finite.
Set W 1 + v and L = CVVV Fl.l..2 Then the assumption implies that F can be covered by n balls of radius at most In-1/V for every natural number n. Form sets F1 F2 C... C such that for each n, the set Fr is maximal, Ln-l/V 1/V separated net over F. Thus Fn has most elements. We will show by induction that there exist constant C₁ and Dj depending only on C' and V such that sup, C V DR xo and for 2 3V.
log.N(CxII" .conv(Fnte)) H (Q)) S DAn, n 1.
The proof consists of nested induction argument. The outer layer is induction & and the inner layer is induction on n.
First, we apply induction for n, i.e., for k- 1, we will prove the conclusion for each For fixed no 10. it suffices to choose Cj Lno W L10-W 2 || so that the statement is trivially ture for n S no = 10, i.c., C₁ 2 10 C C-1/V, For 10 < I 100, set m = Li thus S m S 10. By the definition of Fm, each f E Fn Fm has distance at most Lm-1/V of some element / of Fm. Thus each element of conv(F) can be written as
Σ My = Σ Eutf IEF
Σ Ay(f - Ammf-mmmf) Fmf), JEF
SEF
where uy 2 and Enz = M, = 1. Taking as the set of function f - Tmf with ranging over Fn - Fm. thus conv(Fr) C. conv(F conv for a set In consisting of most elements, each of norm smaller than Lm-1/VV then diam(Gn) S 21m Applying Lemma 17 for Gn with defined by m m-1/V 1 Cn W i.e., ediam(G,) S 112Ln we can find 322Ln WW -net over conv(G,) consisting of at most
12-20 8
(e + enc² 2212 -(e
m
of
32
32
elements, where the inequality follows from the facts that (e enx) for n 10. Applying the induction hypothesis to Fm to find C₁ [mm -net over conv(Fm) consisting of at most et elements, where we choose D, = 1. This defines partition of conv(Fm) into m-dimensional sets of radius at most For any fixed element h in the .. for X = (A1, Am) E Rm. And we denote the ball C. - -net over (Fn) assume that h = A1fi, +... Sin
! is increasing with respect 0 and | 2 a
2 10
W, Without loss of generality. we can assume that Fm = {fa, fiz fiz. fin.
centered at h with H (Q) radius C, by
H = {ā (21,... ... ()m) E C.:.. h Anff, +... + Amnfen 1h All S C. W) where A is a subset of RT.
Note that
|/- h = [|Arft, +... +...-mmfin - Aifi, -...- Am (1
$ [\1 - Aulllf (1===((()) + ... Amm Am (
< (lA1 A11 -1l+...4 12m 11.1l/1Fll...
Thus if ||- Āl[1 S C.C m -w then |/h hllmr( I C1Lm- Therefore, A c {A E RM J| A112 5 C.CVV m mW7. By Lemma 5.7 in Wainwright (2019), we can find a C.C. -net of under the distance || 111 consisting of at most
6C,C4/V m [.CCCVV-W
- W m
n2 "1 (12 m
elements. Moreover, it yields 1 W-nt of И under (Q). Select function from each of the given sets. Then, construct all possible combinations of the sums f + by preceding procedure, where f is associated with conv(F...) and g is associated with conv(Gn). These form Ln W -net over conv(F,) of cardinality bounded by
eC?
en/ 10 (12(20
ee
32
20
52

---

Refined Generalization Analysis of the DRM and PINNS
This is bounded by ex for some suitable choice of C1. Specifically, note that for V 2 the term attains the maximum V thus it is bounded by
32 400m
e (12(2()))) (c+ eCf
of
32-400
We can just take C1 = 1000. This concludes the proof for k = 1 and 10 < n S 100. Proceeding in the same way yields that the conclusion holds for every nt.
We continue by induction on k. By similar construction as before, conv(Fnxx) C conv(Fa(t-1jo) + conv (9n,)) for a set (m. containing at most nk9 elements, each of norm smaller than L(n(k 1)9)-1/V, so that conv(gm.)) S 2/--/VV/g -/V22/V Applying Lemma D.I to conv(9n,k) with c = 2 19/V 2- 22 -/V n 11/2, we can find an Lk 2n W.
2Ln - I/VK
net over conv(G..k) consisting of most
n2 20 +3
enkac² (e +
ck 3 -4
2
2 2 +3
elements. Apply the induction hypothesis to obtain In W nnet the set Conv(Fn(k-1))) with respect H consisting at most e elements. Combine the nets as before to obtain Ck- Ln- -net over conv(Fnno) consisting of at most e Pe" elements, for
Ln
1
C Ck-1 + K2a
D3 = DD-1 + + log(1 (1 + 2- 733 Φ-1)
1 8 -2)
For 2($ 22 2 2, the resulting sequences Cj and D2 are bounded. By setting g = 3V, i.e., 2( - -2) = 2, we have
D₁ Dx-1 22 1 + log(1 1 + 2 2--
122
Therefore, for any k; we deduce that C C1 2 and D₁ D1 + KV, where K is universal constant. Recall that C1 max(10WC 1-1/V, 1000), thus sup, C₁ < max C-V/V, 1000) +
Finally,
2V
C&C+
(
log (on) (Q)) < sup Dr
S SKVCC +2)
where is universal constant.
For the function class two-layer neural networks considered in the DRM, i.e.,
F = (a(w x + 1), aww x + 1),0: 0: W/| = 1 (-1,1))
thus for any probability measure Q on [0, 1]d we have || S 3 and
FF s C(d + 1)(4))d-1 CC) 3d
where C' is a universal constant.
Then, applying Theorem D.2 yields that
1
log (eg s Kd
where is universal constant.
53

---

Refined Generalization Analysis of the DRM and PINNS
As result, in Theorem 2.5 for deriving the generalization error for the static Schrödinger equation, can deduce that the fixed point r" satisfies
d
which yields meaningful generalization bound in the setting of over-parameetrrizaton..
D.2. Other boundary conditions for Deep Ritz Method
Let Q C [0, 1]dd be convex bounded open set and 0Ω be the boundary of Q. Consider the elliptic equation on with Neumann boundary condition:
Du
-u + wu = h on Ω
On = g on on,
(161)
where
h E (1),
H
E (9))
(162)
From the variation method, the Ritz functional can be defined by
E(u)= / (
H Vull 2
hu dx - / (gTu)ds, 1 wllu?
(163)
on
where 7 is the trace operator.
Then we can deduce that then unique weak solution u' E (22) of (1614) is the unique minimizer of & over H (S). Moreover, the Ritz functional possesses similar strongly convex property as described in Proposition 1. Specifically, for any E H (22).
||u - un' (1 S E(u) - E(u") S ---""pp ul
(164)
At this point, derive the fast rate for equation (161), we can employ the LRC from the multi-task learning setting. This is due to the strongly convex property of the Ritz functional (161), which is similar to the approach used to derive faster generalization bounds for the static Schrödinger equation. Specifically, Theorem B.3 in Yousefi et al. (2018) can be seen as a generalization of Theorem 3.3 in Bartlett et al. (2005) to the multi-task setting, thus combining it with the error decomposition in (79) can lead to the conclusion for the Ritz functional (163). For the sake of brevity, we omit the proof here.
For the Robin boundary condition:
au On
+ B
=
on Q 3 R, 0,
(165)
the corresponding Ritz functional retains strong convexity properties analogous to (163), provided the bilinear form remains coercive. To approximate homogeneous Dirichlet conditions (u 0, on AR), under certain conditions, we can prove that ||u wo) = ((3), where us is the solution to the Robin boundary condition (165) and uo is the solution under the homogeneous Dirichlet boundary condition.
set g and let , - 0+. Since
D.3. Lower bounds
The derivation of lower bounds plays critical role in nonparametric statistics. From the upper and lower bounds, we can establish whether the estimator achieves minimax optimality. What Lu et al. (2021b) has achieved better than our work is that they also derived lower bounds for both DRM and PINNs. Their results show that the bound for DRM is not minimax optimal, whereas that for PINNs is minimax optimal. However, the metric used in Lu et al. (2021b) to evaluate PINNs is the 2 norm, which requires strong convexity assumption on PDEs and neural network functions to belong to H.- Such assumptions appear too stringent.
Recent studies (Farrell et al., 2021; Schmidt-Hieber 2020; Jiao et al., 2023; Bauer & Kohler, 2019; Kohler & Langer, 2021; Chen et al., 2022) have shown that neural network-based estimators can achieve minimax optimal rates for regression problems under certain conditions. These bounds are estimated under the L2 norm. However, for PINNs, different PDEs require distinct norms to measure the discrepancy between the empirical and true solutions, which differs significantly from the regression framework. Consequently, lower bounds may only be discussed within the semi-norm structure of PINNs loss functions. Thus, the minimax optimality of the derived bounds for both PINNs and DRM remains an open
54

---

Refined Generalization Analysis of the DRM and PINNS
question warranting rigorous investigation. Future work will examine the applicability of Le Cam's and Fano's ss methods in this context.
D.4. Limitations and future Work
In the paper, we have made the assumption that all related functions are bounded, as required for the localization analysis. However, these assumptions can sometimes be strict. Therefore, it is crucial to investigate settings where the boundedness is not imposed.
Utilizing ReL.U neural networks in the DRM presents optimization challenges due to the non-differentiabiiity of the ReLU function's derivative. One potential approach is to employ randomized methods to tackle the objective functions, like using random neural networks (Wang & Dong, 2024). Specifically, for a two-layer neural network, we sample the weights of the hidden layer from certain probability distribution, which allows us to focus solely on optimizing the parameters the output layer, thereby simplifying the problem to an easy optimization task. The methods for deriving improved generalization error remain valid under stronger assumptions. For instance, when the solutions belong to 8 (12,, employing ReLU² neural networks allows us leverage gradient descent or stochastic gradient descent methods.
For the PINNs, the loss functions play a crucial role for solving PDEs. It is worth paying more attention to the design of loss functions for different PDEs. As discussed in Section 3, using the L loss function can only yield results in the H To obtain results in the H' or H2 the boundary residual term must employ the H//2 or Hr3/2-norm. However, using the H/1/2 H/3/2-norm not only complicates computations but also introduces difficulties in deriving generalization bounds, because computing fractional Sobolev norms may render the empirical loss function non-Lipschitz continuous with respect to the parameters (see, e.g., Definition 1.2 in Girault & Raviart (2012) for the definition of fractional norms), In practice, we can relax the H 1 and H norms.. For instance, for the H -nomm, we may use the loss function:
((u) = |Lu - fli + ||u - gl (00)-
Similar ideas have also been applied in elliptic surface problems (Wu et al., 2023). Moreover, the design of loss functions is equally crucial for inverse problems (Zhang et al., 2023).
The optimization error is beyond the scope of this paper. Gao et al. (2023); Luo & Yang (2020) have considered the optimization error of the two-layer neural networks for the PINNs inspired by the work Du et al. (2018). However, all these studies focus on the over- aver-parameterized regime, relying on the lazy training property of neural tangent kernel (NTK).
The requirements of the function class of deep neural networks may be impractical. Achieving these requirements in practice might be accomplished by restricting the weights of the networks, but doing so can make optimization more difficult. Thus, it worth exploring whether there are more efficient methods.
The solution theory of PDEs in the Barron spaces remains unclear. Lu et al. (2021c) has addressed the problem for the Poisson and static Schrödinger equations in the Spectral Barron spaces, yielding a priori estimates similar to the standard Sobolev regularity estimate. As for the Barron spaces, Chen et al. (2023) has studied the regularity of solutions to the space static Schrödinger equation B (Rd). However, the results of Lu et al. (2021c) and Chen et al. (2023) do not work for (92). Despite this, at least, there exists solutions in the B²( as H + c ( for any e > 0.
55


Reviews:
[{"review_id": "lDFbkfUZLq", "reviewer": "Reviewer_sXtr", "summary": "The paper presents refined generalization error bounds for two Machine Learning (ML) based methods used to solve partial differential equations (PDEs) - the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs). The main technical contribution made in this paper is to provide a sharper generalization bound for DRMs and PINNs based on localized Rademacher complexity techniques. For the DRMs, the analysis is mainly based on the Poisson Equation and the static Schrodinger Equation on the high-dimensional unit hypercube with Neumann boundary condition. For the PINNs, the paper focuses on the general linear second elliptic PDEs with Dirichlet boundary condition. Compared to previous studies, tighter bounds get obtained and unrealistic assumptions also get removed.\n\n## Update After Rebuttal\nThe reviewer remains positive about the theoretical results presented in the paper, so the reviewer would like to remain the score. Nevertheless, the authors should probably discuss their answers to Q3 and Q4 mentioned in the review in detail when revising the manuscript and include some numerical experiments if possible.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time in reviewing our manuscript and your valuable insights. Below we provide a point-by-point response to your concerns.\n\n**Q1**: Experiments.\n\n**A1**: We agree that experiments would further strengthen our work. In future work, we plan to account for optimization error and conduct experiments to validate our results. For experimental validation of DRM in the setting of Barron space, the experiments in [4] show that even in 100-dimensional cases, the DRM achieves a relative error of less than $20$%. Moreover, when comparing experimental results with theoretical bounds, [4] found that the convergence rates in their generalization error may not be sharp. Therefore, their experiments can, to some extent, justify our findings, as our theoretical setting aligns with theirs—but we provide better generalization bounds.\n\n**Q2**: References.\n\n**A2**: We sincerely appreciate the reviewer for pointing out these important references. In the revised version, we will follow the format of [1] and incorporate all suggested references [2-8] with proper discussion. \n\n**Q3**: Lower bounds.\n\n**A3**: What [1] has achieved better than our work is that they also derived lower bounds for both DRM and PINNs. Their results show that the bound for DRM is not minimax optimal, whereas that for PINNs is minimax optimal. However, the metric used in [1] to evaluate PINNs is the $H^2$ norm, which requires strong convexity assumption on PDEs and neural network functions to belong to $H_0^1$. Such assumptions appear too stringent.\n\nRecent studies have shown that neural network-based estimators can achieve minimax optimal rates for regression problems. These bounds are estimated under the $L^2$ norm. However, for PINNs, different PDEs  require distinct norms to measure the discrepancy between the empirical and true solutions, which differs significantly from the regression framework. Therefore, whether the bounds we derived for PINNs and DRM are truly optimal still requires further investigation in future research.\n\n**Q4**: Comparison of proof strategies with [1].\n\n**A4**: For the DRM, we also analyze the Barron space setting and derive novel generalization bounds that remain meaningful even in overparameterized regimes. This advancement surpasses the capabilities of [1]'s approach (see Proposition 2.7). For the Poisson equation, the variational form $\\mathcal{E} _{P}(u)$ is not equal to the expectation of its corresponding empirical part $\\mathcal{E} _{n,P}(u)$. This makes the local Rademacher complexity (LRC, [3]) and the method in [1] infeasible. Morover, for example, the core lemma (Lemma B.4) in [1] requires the condition (from [3]) that \n$$R _n({f\\in \\mathcal{F}: \\mathbb{E}[f]\\leq r}) \\leq \\phi(r),$$ \nwhere $R _n$ is the empirical Rademacher complexity. Then in [1], an appropriate functions class $\\mathcal{F}$ can be chosen such that $\\mathbb{E}[f] = \\mathcal{E} _{P}(u)-\\mathcal{E} _{P}(u^{* })$. By doing so, the strong convexity can be used. However, for the Poisson equation, there does not exist a function class $\\mathcal{F}$ such that $\\mathbb{E}[f]=\\mathcal{E} _{P}(u)-\\mathcal{E} _{P}(u^{* })$. In contrast, this work develops a novel peeling technique to establish fast rates. The static Schrödinger equation shares similar strong convexity properties with certain classical problems like bounded-noise regression [2]. Although approaches from [1], [2] and our Poisson equation method remain applicable, they are much complicated. To address this, we instead develop a novel error decomposition method that allows direct application of the results in [3] (LRC), yielding better generalization bounds through a more concise proof framework.\n\nFor PINNs, [1] only considered the case of the static Schrödinger equation and assumed that the neural network function class is a subset of $H_0^1$. This assumption makes PINNs contain only interior terms, thereby giving them a strong convexity property similar to that of the DRM. Thus, the approach for PINNs in [1] is identical to that for the DRM. In contrast, we consider more general PDEs and treat PINNs as a multi-task learning (MTL) problem. The key difference from the DRM is that for PINNs, we only require a non-exact form of oracle inequality, eliminating the need for the strong convexity. Then, by using LRC under MTL (where we also derive an improved Talagrand-type concentration inequality for MTL), we obtain sharper generalization bounds. Moreover, this approach can also be extended to other types of PDEs and neural-network-based PDE-solving methods similar to PINNs.\n\n**References**:\n\n[1] Machine learning for elliptic pdes: Fast rate generalization bound, neural scaling law and minimax optimality. ICLR,2022.\n\n[2] Deep neural networks for estimation and inference. Econometrica,2021.\n\n[3] Local Rademacher complexities. AoS,2005.\n\n[4] A Priori Generalization Analysis of the Deep Ritz Method for Solving\nHigh Dimensional Elliptic Partial Differential Equations. COLT,2021.", "claims_and_evidence": "The main claim made in this paper is that the convergence rates associated with the DRM and PINN can be further improved based on techniques like the peeling method and localized Rademacher complexity. Detailed proofs are provided to support the main claim here.", "methods_and_evaluation_criteria": "N/A (This is a theoretical paper). However, the reviewer does think that the paper's impact might be increased if the authors can add a few numerical experiments to justify their findings.", "theoretical_claims": "For DRMs the main theoretical claims are in section2, which are results for the Poisson Equation and the static Schrodinger Equation on the high-dimensional unit hypercube with Neumann boundary condition. For PINNs the main theoretical claims are in section 3, which are results for general linear second elliptic PDEs with Dirichlet boundary condition. The reviewer didn't find any significant issue with the proofs of the main claims.", "experimental_designs_or_analyses": "N/A (This is a theoretical paper). However, the reviewer does think that the paper's impact might be increased if the authors can add a few numerical experiments to justify their findings.", "relation_to_broader_scientific_literature": "This paper, which studies how to improve the convergence rate of the DRMs and PINNs, should be mainly situated as the application of nonparametric statistics and learning theory (high-dimensional statistics) in scientific machine learning (SciML/AI4Science), i.e., the theoretical analysis of algorithms in SciML/AI4Science.", "essential_references_not_discussed": "Given that solving PDEs via ML-based methods is a popular field recently, it might be beneficial for the authors to do a review of related methodology proposed in the current literature. See for instance the literature review section in [1]. Some important references that the authors didn't cite here include [2,3,4,5,6,7,8].\n\nReferences:\n\n[1] Lu, Y., Blanchet, J. and Ying, L., 2022. Sobolev acceleration and statistical optimality for learning elliptic equations via gradient descent. Advances in Neural Information Processing Systems, 35, pp.33233-33247.\n\n[2] Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. and Anandkumar, A., 2020. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895.\n\n[3] Lu, L., Jin, P., Pang, G., Zhang, Z. and Karniadakis, G.E., 2021. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3), pp.218-229.\n\n[4] Chen, Y., Hosseini, B., Owhadi, H. and Stuart, A.M., 2021. Solving and learning nonlinear PDEs with Gaussian processes. Journal of Computational Physics, 447, p.110668.\n\n[5] Han, J., Jentzen, A. and E, W., 2018. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34), pp.8505-8510.\n\n[6] Khoo, Y., Lu, J. and Ying, L., 2021. Solving parametric PDE problems with artificial neural networks. European Journal of Applied Mathematics, 32(3), pp.421-435.\n\n[7] Sirignano, J. and Spiliopoulos, K., 2018. DGM: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375, pp.1339-1364.\n\n[8] Zang, Y., Bao, G., Ye, X. and Zhou, H., 2020. Weak adversarial networks for high-dimensional partial differential equations. Journal of Computational Physics, 411, p.109409.", "other_strengths_and_weaknesses": "The main strength of this article is the techniques used in its proofs, which does address issues like the infeasible assumption (some solution represented by a neural network in the $H^{1}_0$ space). However, for the weakness of this article, the reviewer's main concern is that the authors might have to compare their work with the previous work [1] in a more thorough way. Firstly, given that [1] establishes not only upper bounds but also information theoretical lower bounds on the expected estimation error, would it be possible for the authors to provide some intuition on whether it would be possible to derive information theoretic lower bounds for the cases considered in this paper? (i.e, are the bounds established here minimax optimal or not?) To the best of the reviewer's knowledge, this seems to be one important criteria for articles focusing on learning theory and nonparametric statistics. Secondly, it seems that [1] also used techniques like peeling and localized Rademacher complexity, which is similar to the proof strategy deployed in this paper - would it be possible for the authors to further comment on the proof novelty in this paper?\n\nReferences:\n\n[1] Lu, Y., Chen, H., Lu, J., Ying, L. and Blanchet, J., 2021. Machine learning for elliptic pdes: Fast rate generalization bound, neural scaling law and minimax optimality. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=mhYUBYNoGz.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "bBw269fgwl", "reviewer": "Reviewer_iXkp", "summary": "This paper presents a refined generalization analysis of two popular deep learning-based methods for solving partial differential equations (PDEs): the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs). The authors derive sharper generalization bounds for these methods under different assumptions about the solutions of the PDEs, particularly when the solutions lie in Barron spaces or Sobolev spaces.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our article and for your insightful comments. Let us address your concerns point by point.\n\n**Q1**: Missing references.\n\n**A1**: We appreciate the reviewer for pointing out this important literature. It has established rigorous analysis of PICNN on the sphere, bridging the gap in understanding PINNs on manifolds. In the revised version, we will cite it with proper discussion.\n\n**Q2**: Experiments.\n\n**A2**: Regarding why we did not include experiments, like most theoretical papers, we have chosen to omit optimization error. In future work, we plan to account for optimization error and conduct experiments to validate our theory. For experimental validation of the Deep Ritz Method (DRM) for high-dimensional PDEs in the setting of Barron spaces, the experiments in [1] demonstrate that even in 100-dimensional cases, the DRM achieves a relative error of less than $20$% for Poisson equation and static Schrödinger equation. Moreover, when comparing experimental results with theoretical bounds, [1] observed that the convergence rates in their generalization error may not be sharp. Therefore, their experiments can, to some extent, support our findings, as our theoretical setting aligns with theirs—while we provide better generalization bounds.\n\n**Q3**: Assumptions for the solutions.\n\n**A3**: As the article primarily focuses on theoretical aspects, we–like most existing works–initially consider simpler cases. Investigating the setting where solutions belong to more complex spaces remains an important direction for future research.\n\n**Q4**: Extension to other PDEs.\n\n**A4**: In our analysis for PINNs, we require that the expected loss function of PINNs can be controlled by certain Sobolev norms between the exact and approximate solutions. For time-dependent PDEs, such as \n$$\\partial_t u=\\Delta u+f(u),$$\nwhere $u^{* }$ is the true solution and $f$ is the nonlinear term assumed to be Lipschitz. For simplicity we only consider the interior term, then the loss function is \n$$L(u)=||\\partial _t u-\\Delta u-f(u)|| _{L^2(0,T;L^2(\\Omega))}^2.$$\nThen we have\n\\begin{equation*}\n\\begin{aligned}\nL(u)&=||\\partial _t u-\\Delta u-f(u)-\\partial _t u^{* }+\\Delta u^{* }+f(u^{* })|| _{L^2(0,T;L^2(\\Omega))}^2\\\\\\\\\n&\\lesssim ||\\partial _t u-\\partial _t u^{* }||  _{L^2(0,T;L^2(\\Omega))}^2+||\\Delta u-\\Delta u^{* }|| _{L^2(0,T;L^2(\\Omega))}^2+||u-u^{* } || _{L^2(0,T;L^2(\\Omega))}^2.\n\\end{aligned}\n\\end{equation*}\nThen combining our method with neural network approximation results for spatiotemporal Sobolev spaces can yield similar generalization bounds. \n\n**Q5**: Comparison with traditional methods.\n\n**A5**: The error bounds of finite element method depend on the maximum mesh diameter $h$ and the computational complexity scales as $O(h^{-d})$ in high dimensions. Moreover, the error constants grow significantly with increasing dimension $d$, leading to the curse of dimensionality.\n\nIn this work, under the assumption that the solutions belong to Barron spaces, the relevant constants demonstrate at most polynomial dependence on the dimension, making the generalization bounds meaningful even in high-dimensional settings.\n\n**Q6**: The use of $\\text{ReLU}^k$.\n\n**A6**: The choice of activation functions is just to demonstrate that our method can achieve better results. Our method also works for other activations. To illustrate this, we provide a concrete example using tanh for the static Schrödinger equation.\n\nFor the DRM, the generalization bound takes the form\n$$\\frac{\\alpha}{n}+\\epsilon _{app}^2,$$\nwhere $n$ is the sample size, $\\alpha$ represents the neural network complexity and $\\epsilon _{app}$ is the approximation error. Theorem 5.1 of [2] shows that for the solution $u _{S}^{* }\\in W^{k,\\infty}$\nof the static Schrödinger equation, there exists a two-hidden-layer tanh network with width $O(N^d)$ achieving an approximation error $O(N^{-(k-1)})$. In this case, we have  $\\alpha=O(N^d)$, then taking a proper $N$ yields\n$$\\frac{\\alpha}{n}+\\epsilon _{app}^2=\\frac{N^d}{n}+\\frac{1}{N^{2(k-1)}}=O(n^{-\\frac{2k-2}{d+2k-2}}).$$\nThis result exactly coincides with the derived rate in Theorem 2.5(2) of our work.\n\n**Q7**: Optimality.\n\n**A7**: Some studies have shown that neural network-based estimators can achieve the minimax optimal rates for regression problems. These upper and lower bounds are estimated under the $L^2$ norm. However, for PINNs, different PDEs may require distinct norms  to measure the discrepancy between approximate and exact solutions, which significantly differs from the framework of regression problems. Therefore, whether the improved generalization bounds for PINNs derived in this work are indeed optimal still requires further validation in future studies. \n\n**References**:\n\n[1]: A Priori Generalization Analysis of the Deep Ritz Method for Solving\nHigh Dimensional Elliptic Partial Differential Equations. COLT 2021.\n\n[2]: On the approximation of functions by tanh neural networks. Neural Networks 2021.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "No", "experimental_designs_or_analyses": "No", "relation_to_broader_scientific_literature": "The contributions of the paper are related to the community of deep learning theory and scientific machine learning.", "essential_references_not_discussed": "The below paper is about the theory of PINNs by CNN.\n\nLei, G., Lei, Z., Shi, L., Zeng, C., & Zhou, D. X. (2025). Solving PDEs on spheres with physics-informed convolutional neural networks. Applied and Computational Harmonic Analysis, 74, 101714.", "other_strengths_and_weaknesses": "Strengths:\n1. The authors achieve sharper generalization bounds compared to previous works, particularly for DRM and PINNs, which is a solid contribution to the field.\n\n2. The proposed methods provide a unified framework for deriving generalization bounds for machine learning-based PDE solvers, which can be extended to other PDEs and methods.\n\nWeaknesses:\n1. While the theoretical contributions are solid, the paper lacks empirical validation. It would be beneficial to see how the derived bounds hold in practice, especially for high-dimensional PDEs.\n\n2. The analysis relies on the assumption that the solutions lie in Barron spaces or Sobolev spaces. While these are reasonable assumptions, they may not hold for all PDEs, and the paper does not discuss the implications when these assumptions are violated.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. The paper claims that the framework can be extended to other PDEs, including time-dependent ones. Could the authors provide some insights on how this extension would work?\n\n2. How do the generalization bounds of DRM and PINNs compare with those of traditional numerical methods like finite element or finite difference methods, especially in high dimensions?\n\n3. There may be some concerns on the use of $ReLU^k$ in deep neural networks. Are they commonly used in sovling PDE?\n\n4. Do you have any insights regarding the optimality of the derived rate?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "35iL41OFL0", "reviewer": "Reviewer_FWfm", "summary": "The manuscript presents a detailed error analysis for physics-informed neural networks (PINNs) and the deep Ritz method (DRM) for linear elliptic equations. Both, the case of the true solution belonging to a Barron space and a Sobolev space are discussed. Additionally, certain approximation theoretic results are provided, noteably approximation rates in Barron spaces are proven with explicit control over the neural networks weights, which is of independent interest. The manuscript is carefully written, well understandable and technically mature.\n\n## update after rebuttal\nI maintain my positive assessment of the manuscript.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your thorough review and thoughtful suggestions. Below we provide detailed responses to each of your comments.\n\n**Q1**: Missing experiments.\n\n**A1**: We agree that numerical validation would enhance the work.  About experimental validation, the experiments in [1] have demonstrated that even in 100-dimensional cases, the Deep Ritz Method (DRM) attains a relative error of less than 20% for the Poisson equation and static Schrödinger equation in the setting of Barron space. Furthermore, when comparing experimental results with theoretical bounds, [1] observed that the convergence rates in their generalization bounds may not be sharp. Consequently, the experimental findings in [1] provide partial support for our results, as our theoretical framework aligns with theirs—while offering improved generalization bounds.\n\nRegarding why we did not include experiments: Like most theoretical papers, we have chosen to omit optimization error. In future work, we plan to account for optimization error and conduct experiments to validate our findings.\n\n**Q2**: Consider $H^2$ conforming networks for Deep Ritz Method (DRM) .\n\n**A2**: We totally agree with you. In fact, if we assume that the solutions belong to Barron space $\\mathcal{B}^s(\\Omega)$ with $s\\geq 3$,  then $\\text{ReLU}^2$ activation functions (i.e. $H^2$ conforming networks) could be employed for DRM, and our theoretical framework Theorem 2.4&2.5 still remains valid.\n\nIn this work, both DRM and PINNs are analyzed under the weakest solution regularity assumption -- specifically, we assume the solutions reside in relatively low-order Barron spaces (e.g. $\\mathcal{B}^3(\\Omega)$ in Theorem 3.4). This fundamental assumption governs our choice of activation functions throughout the paper.  For example, for solutions in $\\mathcal{B}^s(\\Omega)$ with $s\\geq 4$, $\\text{ReLU}^3$ activations would become feasible for PINNs, and our theoretical framework Theorem 3.4 remains valid.\n\n**Q3**: Regarding the positioning of this work.\n\n**A3**: We can understand the reviewer’s concern about whether our purely theoretical error analysis fits ICML’s scope, especially given its focus on theoretical contributions without numerical simulations. However, **we believe this work aligns with ICML’s mission to advance  machine learning theory, as it addresses certain challenges arising in physics-informed learning where commonly used ML tools fail or are insufficient**. Whereas papers in mathematical journals often rely on numerical integration with well-established error estimates, our work tackles the more complex setting of Monte Carlo methods for computing high-dimensional integrals arising in neural network loss functions---a setting that requires different theoretical tools. Specifically, we contribute by (1) developing a new theoretical framework for physics-informed learning and (2) deriving meaningful error bounds for Barron spaces in the overparameterized regime, advancing the theoretical foundations of machine learning for scientific computing domain.\n\n**Q4**: Missing references.\n\n**A4**: We sincerely appreciate the reviewer for pointing out this important reference. It indeed provides valuable insights, particularly for the second-order elliptic equations where Lemma 1 parallels our Lemma C.11. Additionally, this reference  provides a thorough analysis for other types of PDEs, which broadens the scope. In the revised manuscript, we will cite this reference and incorporate this discussion on page 8 to offer readers a more comprehensive perspective.\n\n**Q5**: Extension to more general activation functions.\n\n**A5**: While our current analysis primarily focuses on $\\text{ReLU}^k$\nactivation functions, the framework can indeed be extended to more general activations. Here we outline a concrete example using tanh activations for the static Schrödinger equation (Theorem 2.5(2)):\n\nFor the Deep Ritz method, the generalization bound under this setting takes the form:\n$$\\frac{\\alpha}{n}+\\epsilon _{app}^2,$$\nwhere $n$ is the sample size, $\\alpha$ represents the neural network complexity and $\\epsilon _{app}$ is the approximation error in the $H^1$ norm. When employing tanh activations, Theorem 5.1 of [2] demonstrates that for the solution $u _{S}^{* }\\in W^{k,\\infty}$\nof the static Schrödinger equation, there exists a two-hidden-layer tanh network with width at most $\\mathcal{O}(N^d)$ achieving an approximation error $\\epsilon _{app}=\\mathcal{O}(\\frac{1}{N^{k-1}})$. In this setting, we have $\\alpha=\\mathcal{O}(N^d)$. Then taking a proper $N$ yields:\n$$\\frac{\\alpha}{n}+\\epsilon _{app}^2=\\frac{N^d}{n}+\\frac{1}{N^{2(k-1)}}=\\mathcal{O}(n^{-\\frac{2k-2}{d+2k-2}}).$$\nThis matches the convergence rate derived in Theorem 2.5(2) of our work.\n\n**References**:\n\n[1]: A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Partial Differential Equations. COLT 2021.\n\n[2]: On the approximation of functions by tanh neural networks. Neural Networks 2021.", "claims_and_evidence": "All results are substantiated with complete proofs.", "methods_and_evaluation_criteria": "Not applicable.", "theoretical_claims": "I checked the proof strategy but not the technical details.", "experimental_designs_or_analyses": "Not applicable.", "relation_to_broader_scientific_literature": "The work is well contextualized.", "essential_references_not_discussed": "None that I am aware of.", "other_strengths_and_weaknesses": "Strengths: \n- The analysis is -- up to the optimization error -- complete. The manuscript is the most comprehensive error analysis of the DRM and PINNs that I am aware of. \n- It treats both the setting of Sobolev spaces and Barron spaces.\n\nWeakness:\n- The manuscript would benefit from simulations illustrating some of the theoretical findings. I think this is essential and will add value to the paper. From a practitioners point view, it is important to know to which extent theory and practice meet.\n- From a practical/optimization standpoint, it is more natural for the deep Ritz case to consider $H^2$ conforming networks. While $H^1$ clearly suffices for the variational energies to be well defined, one typically needs another derivative to perform gradient-based optimization schemes. For PINNs, the authors seem to already consider $\\operatorname{ReLU}^3$ networks (i.e. the extra derivative), at least in the case of the results for Sobolev functions.", "other_comments_or_suggestions": "The main concern I have with this paper is whether it fits the scope of ICML. It is a pure error analysis paper with no simulations. In my opinion, the manuscript would benefit from the review process in a mathematical journal focused on numerical analysis more than the review process at ICML. This concern does not influence my rating of the paper.\n\nOn page 8 of the manuscript, the authors discuss the relation of the PINN loss to the error measured in certain norms. There are estimates available in the literature for precisely this question, see https://academic.oup.com/imajna/advance-article-abstract/doi/10.1093/imanum/drae081/7904789", "questions_for_authors": "- Can the results be extended to more general activation functions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bUZcjvmTNr", "reviewer": "Reviewer_sHz9", "summary": "The paper proposed refined generalization bounds for the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs), including the Poisson equation and static Schrödinger equation on the $d$-dimensional unit hypercube with the Neumann boundary condition.\n\n## update after rebuttal \nMost of my concerns have been solved, so I maintain my positive rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time in reviewing our manuscript and your valuable insights. Below, we address your questions point by point.\n\n**Q1**: Extension of theories to more complex high-dimensional problems.\t\n\t\n**A1**: In this work, we analyze two scenarios: (1) when the solutions of PDEs reside in Barron spaces, and (2) when they belong to Sobolev spaces. For Sobolev spaces, certain constants in our results demonstrate exponential dependence on dimensionality, which limits their applicability to high-dimensional problems. This limitation is inevitable because the complexity of Sobolev spaces grows dramatically with increasing dimensions, leading to exponential scaling of certain constants in neural network approximation errors. In contrast, for Barron spaces, the constants in the generalization error exhibit at most polynomial dependence on dimensionality, making our results meaningful for high-dimensional settings. \n\nTo establish fundamental theoretical insights, this work primarily analyzes the settings where solutions in Barron or Sobolev spaces. The extension to more complex spaces is an important direction for future research. We conjecture that our methodology can be generalized to these settings with appropriate modifications.\n\n\n**Q2**: Extension to other types of boundary conditions and nonlinear PDEs.\n\n**A2**: Regarding the extension to other boundary conditions, such as the following mixed boundary conditions:\n$$ -\\Delta u+Vu =f \\ in  \\ \\Omega, \\ u+\\beta\\frac{\\partial u}{\\partial n}=g \\ on   \\ \\partial \\Omega,$$\nwhere $\\beta\\neq 0$. A direct analysis shows that the solution $u^{* }$ of this equation satisfies that \n$$u^{* }=\\mathop{\\arg\\min} _{u \\in H^1(\\Omega) } \\mathcal{E}(u):=\\mathop{\\arg\\min} _{u \\in H^1(\\Omega) } \\int _{\\Omega} \\left(||\\nabla u||^2+Vu-2fu \\right)dx+ \\frac{1}{\\beta} \\int _{\\partial \\Omega} \\left(u^2-2gu \\right)ds. $$\t\nMoreover, for any $u\\in H^1(\\Omega)$, we have\n$$ \\mathcal{E}(u)-\\mathcal{E}(u^{* })\\lesssim ||u-u^{* }|| _{H^1(\\Omega)}^2\\lesssim \\mathcal{E}(u)-\\mathcal{E}(u^{* }),$$\nwhich is similar to the strong convexity property required in our analysis for Neumann boundary conditions (see equation (4) and (6)), suggesting that our method can be naturally extended to mixed boundary conditions. For further details, please refer to Section D.2 (line 2851), where we discuss other boundary conditions.\n\nRegarding the extension to nonlinear PDEs within the PINNs framework, our method can also  generalize to a broad class of nonlinear PDEs. For instance, let us consider the following equation:\n$$\\mathcal{D}(u)=f(u) \\ in \\ \\Omega, \\ u=g \\ on   \\ \\partial \\Omega,$$\nwhere $\\mathcal{D}$ is a linear differential operator and $f$ is the Lipschitz nonlinear term. Here, for simplicity, we denote the loss function as\n$$L(u)=||\\mathcal{D}(u)-f(u)|| _{L^2(\\Omega)}^2+||u-g|| _{L^2(\\partial \\Omega)}^2.$$\nAssume that $u^{* }$ is the true solution, then we can deduce that\n\\begin{align*}\nL(u)&=||\\mathcal{D}(u)-f(u)-\\mathcal{D}(u^{* })-f(u^{* })|| _{L^2(\\Omega)}^2+||u-u^{* }|| _{L^2(\\partial \\Omega)}^2 \\\\\\\\\n&\\lesssim ||\\mathcal{D}(u)-\\mathcal{D}(u^{* })|| _{L^2(\\Omega)}^2 +||f(u)-f(u^{* })|| _{L^2(\\Omega)}^2+ ||u-u^{* }|| _{H^1(\\Omega)}^2\\\\\\\\\n&\\lesssim ||\\mathcal{D}(u)-\\mathcal{D}(u^{* })|| _{L^2(\\Omega)}^2 +||u-u^{* }|| _{L^2(\\Omega)}^2+ ||u-u^{* }|| _{H^1(\\Omega)}^2.\n\\end{align*}\nTherefore, this bound is similar to that in the linear case, and our method remains applicable.\n\nMoreover, we agree with the reviewer's suggestion. Extending this framework to broader physical problems (e.g., inverse problems) is indeed an important direction.\n\n**Q3**: Practical applications.\n\n**A3**: In this work, we primarily focus on generalization analysis and approximation rates, while optimization aspects are beyond the scope of this study. As you rightly pointed out, the computational resource and time requirements of neural network-based PDE solvers indeed limit their applicability to complex physical systems and large-scale problems. Developing computationally efficient optimization methods for these solvers with guaranteed accuracy remains a key focus of our ongoing research.\n\n**Q4**: Issues in viewing PINNs as a multi-task learning  problem.\n\n**A4**: From a theoretical perspective, formulating PINNs as a multi-task learning problem allows us to derive tighter generalization bounds through the use of local Rademacher complexity in the multi-task setting. In the experimental perspective, as you mentioned, interactions between different tasks may indeed lead to certain issues, since the loss functions contain multiple different additive terms that\ncan disagree and yield conflicting update directions. Some recent studies, such as [1] and [2], have developed efficient algorithms to mitigate such conflicts.\n\n**References**:\n\n[1]: Config: Towards conflict-free training of physics informed neural\nnetworks. ICLR 2025.\n\n[2]: Dual cone gradient descent for training physics-informed\nneural networks. NeurIPS 2024.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Not available.", "theoretical_claims": "The paper presents a logically structured set of theoretical claims supported by established mathematical techniques.", "experimental_designs_or_analyses": "Not applicable. There are no experiments to validate the proposed theoretical claims.", "relation_to_broader_scientific_literature": "This paper proposed  refined generalization analysis for Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "1. Although the paper demonstrates good generalization performance in low-dimensional cases, how to extend these theories to more complex high-dimensional problems, especially when the solution of the PDE belongs to complex spaces, still requires further research.\n\n2. The paper primarily focuses on Neumann boundary conditions and linear PDEs, with limited in-depth discussion of other types of boundary conditions (such as mixed boundary conditions) or nonlinear PDEs. Extending this work to more diverse physical problems is a potential research direction.\n\n3. While the paper proposes improved generalization analysis and approximation rates, the consumption of computational resources and time remains an issue in practical applications, particularly for complex physical scenarios and large-scale problems. Further exploration is needed on how to reduce computational costs while maintaining accuracy.\n\n4. Viewing PINNs as a multi-task learning problem is an effective framework, but it may lead to interference between tasks, especially when the nature of the tasks varies significantly. Further optimization of task balancing and interaction in multi-task learning could enhance the model’s performance.", "other_comments_or_suggestions": "No.", "questions_for_authors": "Please refer to the Other Strengths And Weaknesses section.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The manuscript considers generalization error analysis for two approaches of using neural networks to solve PDEs. The reviewers are unanimously positive and the authors have addressed most concerns during the discussion phase.
Decision: Accept (poster)