Original: where the function class ˆ S r (Ω) is deﬁned as:
Revised: where the function class ˆ S r (Ω) is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where S (Ω) =
Revised: where S (Ω) =

Original: (cid:110) h :=
Revised: (cid:110) h :=

Original: | Ω | · (cid:104) 12 (cid:16) (cid:107)∇ u (cid:107) 2 − (cid:107)∇ u ∗ (cid:107) 2 (cid:17) + 12 V ( | u | 2 − | u ∗ | 2 ) − f ( u − u ∗ ) (cid:105) (cid:111) .
Revised: | Ω | · (cid:104) 12 (cid:16) (cid:107)∇ u (cid:107) 2 − (cid:107)∇ u ∗ (cid:107) 2 (cid:17) + 12 V ( | u | 2 − | u ∗ | 2 ) − f ( u − u ∗ ) (cid:105) (cid:111) .

Original: Then applying the Peeling Lemma B.4 to any function h ∈ S (Ω) helps us upper bound the local Rademacher complexity R n ( ˆ S r (Ω)) with the function φ deﬁned in equation B.57:
Revised: Then applying the Peeling Lemma B.4 to any function h ∈ S (Ω) helps us upper bound the local Rademacher complexity R n ( ˆ S r (Ω)) with the function φ deﬁned in equation B.57:

Original: [Equation]
Revised: [Equation]

Original: Combining all inequalities derived above yields:
Revised: Combining all inequalities derived above yields:

Original: [Equation]
Revised: [Equation]

Original: Secondly we’ll apply the Talagrand concentration inequality, which requires us to verify the condition needed.
Revised: Secondly we’ll apply the Talagrand concentration inequality, which requires us to verify the condition needed.

Original: We will ﬁrst check that the expectation value E [ h ] is always non-negative for any h ∈ S (Ω) :
Revised: We will ﬁrst check that the expectation value E [ h ] is always non-negative for any h ∈ S (Ω) :

Original: [Equation]
Revised: [Equation]

Original: We will proceed to verify that any ˜ h
Revised: We will proceed to verify that any ˜ h

Original: = E [ h ] − h E [ h ]+ r ∈ ˜ S
Revised: = E [ h ] − h E [ h ]+ r ∈ ˜ S

Original: r (Ω) is of bounded inf-norm.
Revised: r (Ω) is of bounded inf-norm.

Original: We need to prove that any h ∈ S (Ω) is of bounded inf-norm beforehand.
Revised: We need to prove that any h ∈ S (Ω) is of bounded inf-norm beforehand.

Original: Using boundedness condition listed in equation B.56 implies:
Revised: Using boundedness condition listed in equation B.56 implies:

Original: [Equation]
Revised: [Equation]

Original: | Ω
Revised: Ω

Original: [Equation]
Revised: [Equation]

Original: By taking M := | Ω | ( V max + 3) C
Revised: By taking M := | Ω | ( V max + 3) C

Original: [Equation]
Revised: [Equation]

Original: we then have (cid:107) h (cid:107) ∞ ≤ M for all h ∈ S (Ω) .
Revised: we then have (cid:107) h (cid:107) ∞ ≤ M for all h ∈ S (Ω) .

Original: Note that the denominator can be lower bounded by | E
Revised: Note that the denominator can be lower bounded by | E

Original: [ h ] + r | ≥ r > 0 .
Revised: [ h ] + r | ≥ r > 0 .

Original: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜ h (cid:107) ∞
Revised: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜ h (cid:107) ∞

Original: = sup x ∈ Ω | ˜ h
Revised: = sup x ∈ Ω | ˜ h

Original: ( x ) | as follows:
Revised: ( x ) | as follows:

Original: [Equation]
Revised: [Equation]

Original: We will then check the normalized functions
Revised: We will then check the normalized functions

Original: E [ h ] − h ( x ) E
Revised: E [ h ] − h ( x ) E

Original: [ h ]+ r in
Revised: [ h ]+ r in

Original: ˜ S
Revised: ˜ S

Original: r (Ω) have bounded second moment, which is satisﬁed because of the regularity results of the PDE.
Revised: r (Ω) have bounded second moment, which is satisﬁed because of the regularity results of the PDE.

Original: We aim to show that there exist some constants α, α (cid:48) > 0 , such that for any h ∈ S (Ω) , the following inequality holds:
Revised: We aim to show that there exist some constants α, α (cid:48) > 0 , such that for any h ∈ S (Ω) , the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: The RHS of the inequality follows from strong convexity of the DRM objective function proved in
Revised: The RHS of the inequality follows from strong convexity of the DRM objective function proved in

Original: Theorem B.1:
Revised: Theorem B.1:

Original: [Equation]
Revised: [Equation]

Original: The LHS of the inequality follows from boundedness condition listed in equation B.56 and the
Revised: The LHS of the inequality follows from boundedness condition listed in equation B.56 and the

Original: QM-AM inequality:
Revised: QM-AM inequality:

Original: [Equation]
Revised: [Equation]

Original: By picking α (cid:48)
Revised: By picking α (cid:48)

Original: = 4 min { 1 ,V min } and α = 1 3 C 2 (1+ V 2 max ) , we have ﬁnished proving inequality B.60.
Revised: = 4 min { 1 ,V min } and α = 1 3 C 2 (1+ V 2 max ) , we have ﬁnished proving inequality B.60.

Original: Then we can can upper bound the expectation E
Revised: Then we can can upper bound the expectation E

Original: [˜ h 2 ] as:
Revised: [˜ h 2 ] as:

Original: [Equation]
Revised: [Equation]

Original: Using the fact that E [ h ] ≥ 0 and inequality B.60, we can lower bound the denominator | E [ h ] + r | 2 as follows:
Revised: Using the fact that E [ h ] ≥ 0 and inequality B.60, we can lower bound the denominator | E [ h ] + r | 2 as follows:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we can deduce that:
Revised: Therefore, we can deduce that:

Original: [Equation]
Revised: [Equation]

Original: Hence, any function in the localized class
Revised: Hence, any function in the localized class

Original: ˜ S
Revised: ˜ S

Original: r (Ω) is of bounded second moment.
Revised: r (Ω) is of bounded second moment.

Original: It is easy to check that for any ˜ h ∈ ˜ S r (Ω) , we have
Revised: It is easy to check that for any ˜ h ∈ ˜ S r (Ω) , we have

Original: [Equation]
Revised: [Equation]

Original: i.e. any function in the localized class
Revised: i.e. any function in the localized class

Original: ˜ S
Revised: ˜ S

Original: r (Ω) is of zero mean.
Revised: r (Ω) is of zero mean.

Original: Now we have veriﬁed that any function ˜ h ∈
Revised: Now we have veriﬁed that any function ˜ h ∈

Original: ˜ S
Revised: ˜ S

Original: r (Ω) satisﬁes all the required conditions.
Revised: r (Ω) satisﬁes all the required conditions.

Original: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s Concentration inequality given in Lemma B.3, we have:
Revised: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s Concentration inequality given in Lemma B.3, we have:

Original: [Equation]
Revised: [Equation]

Original: By using the upper bound deduced above and plugging in the expressions of β and σ
Revised: By using the upper bound deduced above and plugging in the expressions of β and σ

Original: [Equation]
Revised: [Equation]

Original: we can rewrite
Revised: we can rewrite

Original: Talagrand’s Concentration Inequality in the following way.
Revised: Talagrand’s Concentration Inequality in the following way.

Original: With probability at least 1 − e − t , the inequality below holds:
Revised: With probability at least 1 − e − t , the inequality below holds:

Original: [Equation]
Revised: [Equation]

Original: Let’s pick the critical radius r 0 to be:
Revised: Let’s pick the critical radius r 0 to be:

Original: [Equation]
Revised: [Equation]

Original: Note that concavity of the function φ implies that φ ( r ) ≤ r for any r ≥ r ∗ .
Revised: Note that concavity of the function φ implies that φ ( r ) ≤ r for any r ≥ r ∗ .

Original: Combining this with the ﬁrst inequality listed in B.57 yields:
Revised: Combining this with the ﬁrst inequality listed in B.57 yields:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, applying equation B.61 yields:
Revised: On the other hand, applying equation B.61 yields:

Original: [Equation]
Revised: [Equation]

Original: Summing the three inequalities above implies:
Revised: Summing the three inequalities above implies:

Original: [Equation]
Revised: [Equation]

Original: By picking r = r
Revised: By picking r = r

Original: [Equation]
Revised: [Equation]

Original: we can further deduce that for any function u ∈ F (Ω)
Revised: we can further deduce that for any function u ∈ F (Ω)

Original: [Equation]
Revised: [Equation]

Original: the following inequality holds with probability 1 − e − t :
Revised: the following inequality holds with probability 1 − e − t :

Original: [Equation]
Revised: [Equation]

Original: Multiplying the denominator on both sides indicates:
Revised: Multiplying the denominator on both sides indicates:

Original: [Equation]
Revised: [Equation]

Original: Substituting the upper bound above into the decomposition ∆ E ( n ) ≤
Revised: Substituting the upper bound above into the decomposition ∆ E ( n ) ≤

Original: [Equation]
Revised: [Equation]

Original: gen + 32 ∆ E app + t 2 n yields that with probability 1 − 2 e − min { C q , 1 } t , we have:
Revised: E [Equation] + 32 ∆ E app + t 2 n yields that with probability 1 − 2 e − min { C q , 1 } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Simplifying the inequality above yields that with probability 1 − 2 e − min { C q , 1 } t , we have:
Revised: Simplifying the inequality above yields that with probability 1 − 2 e − min { C q , 1 } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Moreover, using strong convexity of the DRM objective function proved in Theorem B.1 implies:
Revised: Moreover, using strong convexity of the DRM objective function proved in Theorem B.1 implies:

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above yields that with probability 1 − 2 e − min { C q , 1 } t , we have:
Revised: Combining the two bounds above yields that with probability 1 − 2 e − min { C q , 1 } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Deep Neural Network Estimator.
Revised: Deep Neural Network Estimator.

Original: For any N ∈ Z + , there exists some Deep Neural Network in Φ( L, W, S, B ) with L
Revised: For any N ∈ Z + , there exists some Deep Neural Network in Φ( L, W, S, B ) with L

Original: = O (1) , W = O ( N ) , S =
Revised: = O (1) , W = O ( N ) , S =

Original: O ( N ) , B = O ( N ) , such that the approximation error ∆ E app
Revised: O ( N ) , B = O ( N ) , such that the approximation error ∆ E app

Original: =
Revised: =

Original: O ( N − 2( s − 1) d ) and generalization error ∆ E gen = O ( N log N n ) .
Revised: O ( N − 2( s − 1) d ) and generalization error ∆ E gen = O ( N log N n ) .

Original: With optimal selection
Revised: With optimal selection

Original: N = n d d +2 s − 2 to balance the bias and variance, we can achieve n − 2 s − 2 d +2 s − 2 log n convergence rate for the DNN estimator.
Revised: N = n d d +2 s − 2 to balance the bias and variance, we can achieve n − 2 s − 2 d +2 s − 2 log n convergence rate for the DNN estimator.

Original: Theorem B.10.
Revised: Theorem B.10.

Original: (Final Upper Bound of DRM with Deep Neural Network Estimator) Under the assumptions in Theorem B.9, we consider the Deep Ritz objective with the sparse Deep
Revised: (Final Upper Bound of DRM with Deep Neural Network Estimator) Under the assumptions in Theorem B.9, we consider the Deep Ritz objective with the sparse Deep

Original: Neural Network function space Φ( L, W, S, B ) , where the parameters L
Revised: Neural Network function space Φ( L, W, S, B ) , where the parameters L

Original: = O (1) , W =
Revised: = O (1) , W =

Original: O ( n d d +2 s − 2 ) , S = O ( n d [Equation] +2 s 
Revised: O ( n d d +2 s − 2 ) , S = O ( n d d +2 s − 2 ) , B = O ( n d d +2 s − 2 ) .

Original: [Equation] 2 ) , B = O ( n d d +2 s − 2 ) . Then we have that the DNN estimator ˆ u DNNDRM = min u ∈ Φ( L,W,S,B ) E DRM n ( u ) satisﬁes the following upper bound with high probability:
Revised: Then we have that the DNN estimator ˆ u DNNDRM = min u ∈ Φ( L,W,S,B ) E DRM n ( u ) satisﬁes the following upper bound with high probability:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: On the one hand, by taking s = 1 and p = 2 in Theorem B.8 proved above, we have that there exists some Deep Neural Network u DNN ∈
Revised: On the one hand, by taking s = 1 and p = 2 in Theorem B.8 proved above, we have that there exists some Deep Neural Network u DNN ∈

Original: Φ( L, W, S, B ) with L = O (1) , W = O ( N ) , S =
Revised: Φ( L, W, S, B ) with L = O (1) , W = O ( N ) , S =

Original: O ( N ) , B = O ( N ) , such that.
Revised: O ( N ) , B = O ( N ) , such that.

Original: [Equation]
Revised: [Equation]

Original: Applying strong convexity of the DRM objective function proved in Theorem B.1 yields the following upper bound on the approximation error of DRM:
Revised: Applying strong convexity of the DRM objective function proved in Theorem B.1 yields the following upper bound on the approximation error of DRM:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, from Lemma B.17 proved above, we know that the function φ ( ρ ) that upper bounds the local Rademacher complexity of the Deep Neural Network space is of the same magnitude as (cid:113) S 3 L ρ n log ( BWn ) .
Revised: On the other hand, from Lemma B.17 proved above, we know that the function φ ( ρ ) that upper bounds the local Rademacher complexity of the Deep Neural Network space is of the same magnitude as (cid:113) S 3 L ρ n log ( BWn ) .

Original: By plugging in the magnitudes of L, W, S, B , we can determine the critical radius r ∗ : (cid:114)
Revised: By plugging in the magnitudes of L, W, S, B , we can determine the critical radius r ∗ : (cid:114)

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above with Theorem B.9 yields that with high probability
Revised: Combining the two bounds above with Theorem B.9 yields that with high probability

Original: [Equation]
Revised: [Equation]

Original: we have:
Revised: we have:

Original: [Equation]
Revised: [Equation]

Original: By equating the two terms above
Revised: By equating the two terms above

Original: [Equation]
Revised: [Equation]

Original: we can solve for the optimal N that yields the desired bound:
Revised: we can solve for the optimal N that yields the desired bound:

Original: [Equation]
Revised: [Equation]

Original: Plugging in the optimal N gives us the magnitudes of the four parameters L = O (1) , W =
Revised: Plugging in the optimal N gives us the magnitudes of the four parameters L = O (1) , W =

Original: O ( n d d +2 s − 2 ) , S = O ( n
Revised: O ( n d d +2 s − 2 ) , S = O ( n

Original: [Equation]
Revised: [Equation]

Revised: s

Original: [Equation]
Revised: [Equation]

Original:  = O ( 
Revised: 2 ) , B = O ( n

Original: [Equation]
Revised: [Equation]

Original: 2 ) , as well as the ﬁnal rate:
Revised: 2 ) , as well as the ﬁnal rate:

Original: [Equation]
Revised: [Equation]

Original: Truncated Fourier Series Estimator.
Revised: Truncated Fourier Series Estimator.

Original: For any ξ ∈ Z + , there exists some Truncated Fourier Series in F ξ (Ω) with approximation error ∆ E app
Revised: For any ξ ∈ Z + , there exists some Truncated Fourier Series in F ξ (Ω) with approximation error ∆ E app

Original: =
Revised: =

Original: O ( ξ − 2( s − 1) ) and generalization error ∆ E gen = O ( ξ d n ) .
Revised: O ( ξ − 2( s − 1) ) and generalization error ∆ E gen = O ( ξ d n ) .

Original: With optimal selection ξ = n 1 d +2 s − 2 to balance the bias and variance, we can achieve n − 2 s − 2 d
Revised: With optimal selection ξ = n 1 d +2 s − 2 to balance the bias and variance, we can achieve n − 2 s − 2 d

Original: +2 s −convergence rate for the Fourier estimator.
Revised: +2 s −convergence rate for the Fourier estimator.

Original: Theorem B.11.
Revised: Theorem B.11.

Original: (Final Upper Bound of DRM with Truncated Fourier Series Estimator) Under the assumptions in Theorem B.9, we consider the Deep Ritz objective with the Truncated Fourier
Revised: (Final Upper Bound of DRM with Truncated Fourier Series Estimator) Under the assumptions in Theorem B.9, we consider the Deep Ritz objective with the Truncated Fourier

Original: Series function space F ξ (Ω) , where the parameter ξ =
Revised: Series function space F ξ (Ω) , where the parameter ξ =

Original: Θ( n 1 d +2 s − 2 ) .
Revised: Θ( n 1 d +2 s − 2 ) .

Original: Then we have that the Fourier estimator ˆ u FourierDRM =
Revised: Then we have that the Fourier estimator ˆ u [Equation]

Original:  min u ∈ F ξ (Ω) E DRM n ( u ) satisﬁes the following upper bound with high probability:
Revised: = min [Equation] E DRM n ( u ) satisﬁes the following upper bound with high probability:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let’s ﬁrstly derive the function φ ( ρ ) that upper bounds the local Rademacher complexity ofS
Revised: Let’s ﬁrstly derive the function φ ( ρ ) that upper bounds the local Rademacher complexity of

Original: [Equation]
Revised: [Equation]

Original: , where F ρ,ξ (Ω) = (cid:110) v ∈ F ξ (Ω) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 1 (Ω) ≤ ρ (cid:111) denotes the localized Truncated Fourier Series space.
Revised: , where F ρ,ξ (Ω) = (cid:110) v ∈ F ξ (Ω) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 1 (Ω) ≤ ρ (cid:111) denotes the localized Truncated Fourier Series space.

Original: From Talagrand Contraction Lemma B.2, Lemma B.6 and Lemma B.7 proved above, we haveR n
Revised: From Talagrand Contraction Lemma B.2, Lemma B.6 and Lemma B.7 proved above, we have

Original: [Equation]
Revised: [Equation]

Original: S

Original: [Equation] Ω)) [Equation] n
Revised: where Π ξ u := (cid:80) (cid:107) z (cid:107) ∞

Original: [Equation]

Original: [Equation]

Original: n

Original: [Equation]

Original: n

Original: [Equation] i [Equation] (cid:105)(cid:35)
Revised: ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is smaller than ξ and Π >ξ u := (cid:80) (cid:107) z

Original: [Equation]
Revised: [Equation]

Original: where Π ξ u := (cid:80) (cid:107) z (cid:107) ∞

Original: ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is smaller than ξ and Π >ξ u := (cid:80) (cid:107) z (cid:107) ∞

Original:  ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is larger than ξ .
Revised: ∞ ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is larger than ξ .

Original: Then, the critical radius r ∗ can be determined as follows:
Revised: Then, the critical radius r ∗ can be determined as follows:

Original: [Equation]
Revised: [Equation]

Original: Moreover, by taking α = s and β = 1 in Lemma B.19 and applying strong convexity of the DRM objective function proved in Theorem B.1, we can upper bound the approximation error ∆ E app as below:
Revised: Moreover, by taking α = s and β = 1 in Lemma B.19 and applying strong convexity of the DRM objective function proved in Theorem B.1, we can upper bound the approximation error ∆ E app as below:

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above with Theorem B.9 yields that with high probability, we have:
Revised: Combining the two bounds above with Theorem B.9 yields that with high probability, we have:

Original: [Equation]
Revised: [Equation]

Original: By equating two of the three terms above, we can solve for ξ that yields the desired bound:
Revised: By equating two of the three terms above, we can solve for ξ that yields the desired bound:

Original: [Equation]
Revised: [Equation]

Original: Plugging in the optimal ξ gives us the ﬁnal rate:
Revised: Plugging in the optimal ξ gives us the ﬁnal rate:

Original: [Equation]
Revised: [Equation]

Original: B.4. P HYSICS I NFORMED N EURAL N ETWORK
Revised: B.4. P HYSICS I NFORMED N EURAL N ETWORK

Original: Theorem B.12 (Meta-theorem for Upper Bounds of Physics Informed Neural Network) .
Revised: Theorem B.12 (Meta-theorem for Upper Bounds of Physics Informed Neural Network) .

Original: Let u ∗ ∈
Revised: Let u ∗ ∈

Original: H s (Ω) denote the true solution to the PDE model with Dirichlet boundary condition:
Revised: H s (Ω) denote the true solution to the PDE model with Dirichlet boundary condition:

Original: [Equation]
Revised: [Equation]

Original: where f ∈ L 2 (Ω) and V ∈ L ∞ (Ω) with V − 12 ∆ V > C min , 0 < C min < V ( x ) ≤ V max and − ∆ V ( x ) ≤ V max .
Revised: where f ∈ L 2 (Ω) and V ∈ L ∞ (Ω) with V − 12 ∆ V > C min , 0 < C min < V ( x ) ≤ V max and − ∆ V ( x ) ≤ V max .

Original: In Theorem B.2, it has been proved that u ∗ can be obtained by minimizing the loss E ( u ) :
Revised: In Theorem B.2, it has been proved that u ∗ can be obtained by minimizing the loss E ( u ) :

Original: [Equation]
Revised: [Equation]

Original: For a ﬁxed function space F (Ω) , consider the empirical loss induced by the Physics Informed Neural
Revised: For a ﬁxed function space F (Ω) , consider the empirical loss induced by the Physics Informed Neural

Original: Network:
Revised: Network:

Original: [Equation]
Revised: [Equation]

Original: where { X j } nj =1 are datapoints uniformly sampled from the domain Ω .
Revised: where { X j } nj =1 are datapoints uniformly sampled from the domain Ω .

Original: Then the Physics Informed
Revised: Then the Physics Informed

Original: Neural Network estimator associated with function space F (Ω) is deﬁned as the minimizer of E n ( u ) over the function space F (Ω) :
Revised: Neural Network estimator associated with function space F (Ω) is deﬁned as the minimizer of E n ( u ) over the function space F (Ω) :

Original: [Equation]
Revised: [Equation]

Original: Moreover, we assume that there exists some constant C > 0 such that all function u in the function space F (Ω) , the real solution u ∗ and f, V satisfy the following two conditions.
Revised: Moreover, we assume that there exists some constant C > 0 such that all function u in the function space F (Ω) , the real solution u ∗ and f, V satisfy the following two conditions.

Original: • The gradients and function value are uniformly bounded
Revised: • The gradients and function value are uniformly bounded

Original: [Equation]
Revised: [Equation]

Original: • All the functions in the function space F (Ω) satisﬁes the boundary condition
Revised: • All the functions in the function space F (Ω) satisﬁes the boundary condition

Original: [Equation]
Revised: [Equation]

Original: At the the same time, for any ρ > 0 , we assume the Rademacher complexity of a localized function space T ρ (Ω) := [Equation] h :=
Revised: At the the same time, for any ρ > 0 , we assume the Rademacher complexity of a localized function space T ρ (Ω) :

Original: | Ω | · (cid:2) [Equation] ∆ u [Equation]
Revised: = (cid:110) h :=

Original: u

Original: [Equation] f ) 2 − (∆ u ∗ − V u ∗ + f ) [Equation] (cid:3) (cid:12)(cid:12)(cid:12) (cid:107) u − u ∗ (cid:107) 2 H 2 ≤ ρ (cid:111) can be upper bounded by a sub-root function φ = φ ( ρ ) :
Revised: | Ω | · (cid:2) (∆ u − V u + f ) 2 − (∆ u ∗ − V u ∗ + f ) 2 (cid:3) (cid:12)(cid:12)(cid:12) (cid:107) u − u ∗ (cid:107) 2 H 2 ≤ ρ (cid:111) can be upper bounded by a sub-root function φ = φ ( ρ ) :

Original: [0 , ∞ )
Revised: [0 , ∞ )

Original: → [0 , ∞ ) , i.e. φ (4 ρ ) ≤ 2 φ ( ρ ) and R n ( T ρ (Ω)) ≤ φ ( ρ ) ( ∀ ρ > 0) .
Revised: → [0 , ∞ ) , i.e. φ (4 ρ ) ≤ 2 φ ( ρ ) and R n ( T ρ (Ω)) ≤ φ ( ρ ) ( ∀ ρ > 0) .

Original: [Equation]
Revised: [Equation]

Original: For all constant t > 0 .
Revised: For all constant t > 0 .

Original: We denote r ∗ to be the solution of the ﬁx point equation of local Rademacher complexity r = φ ( r ) .
Revised: We denote r ∗ to be the solution of the ﬁx point equation of local Rademacher complexity r = φ ( r ) .

Original: There exists two constants C p , C q such that with probability 1 − C p exp( − C q t ) , we have the following upper bound for the Physics Informed Neural Network Estimator
Revised: There exists two constants C p , C q such that with probability 1 − C p exp( − C q t ) , we have the following upper bound for the Physics Informed Neural Network Estimator

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: To upper bound the excess risk ∆ E ( n ) , following(Xu, 2020; Lu et al., 2021b; Duan et al., 2021), we decompose the excess risk into approximation error and generalization error with probability 1 − e − C q t , where C q > 0 is some constant:
Revised: To upper bound the excess risk ∆ E ( n ) , following(Xu, 2020; Lu et al., 2021b; Duan et al., 2021), we decompose the excess risk into approximation error and generalization error with probability 1 − e − C q t , where C q > 0 is some constant:

Original: [Equation]
Revised: [Equation]

Original: where the expectation is on all sampled data.
Revised: where the expectation is on all sampled data.

Original: The inequality of the third line is because ˆ u PINN is the minimizer of the empirical loss E n in the solution
Revised: The inequality of the third line is because ˆ u PINN is the minimizer of the empirical loss E n in the solution

Original: set F (Ω) , so we have E n (ˆ u PINN ) ≤
Revised: set F (Ω) , so we have E n (ˆ u PINN ) ≤

Original: E n ( u F ) .
Revised: E n ( u F ) .

Original: The last inequality is based on the Bernstein inequality.
Revised: The last inequality is based on the Bernstein inequality.

Original: The variance of h = | Ω | · (cid:2) 
Revised: The variance of h = | Ω | · (cid:2) (∆ u − [Equation] u + [Equation]

Revised: ) 2 −

Original: [Equation]
Revised: [Equation]

Revised: ∆ u

Original: [Equation]
Revised: [Equation]

Original:  ) 2 (cid:3) can be bounded byα
Revised: + f ) 2 (cid:3) can be bounded by

Original: [Equation]
Revised: [Equation]

Original: (cid:2) E ( u F ) − E ( u (cid:63) ) (cid:3) due to the strong convexity of the variation objective (B.69).
Revised: E ( u F ) − E ( u (cid:63) ) (cid:3) due to the strong convexity of the variation objective (B.69).

Original: According to the Bernstein inequality, there exists some constant C q > 0 , such that with probability 1 − e − C q t we have:
Revised: According to the Bernstein inequality, there exists some constant C q > 0 , such that with probability 1 − e − C q t we have:

Original: [Equation]
Revised: [Equation]

Original: Note that C.5 holds for all function lies in the function space F .
Revised: Note that C.5 holds for all function lies in the function space F .

Original: Thus, we can take u F := arg min u 0 ∈ F (Ω) (cid:16) E ( u 0 ) − E ( u (cid:63) ) (cid:17) and ﬁnally get
Revised: Thus, we can take u F := arg min u 0 ∈ F (Ω) (cid:16) E ( u 0 ) − E ( u (cid:63) ) (cid:17) and ﬁnally get

Original: [Equation]
Revised: [Equation]

Original: This inequality decompose the excess risk to the generalization error ∆ E gen := E (ˆ u PINN ) − E ( u ∗ )
Revised: This inequality decompose the excess risk to the generalization error ∆ E gen := E (ˆ u PINN ) − E ( u ∗ )

Original: +
Revised: +

Original: E n ( u ∗ ) − E n (ˆ u PINN ) and the approximation error ∆ E app = inf u F ∈ F (Ω) (cid:16) E ( u F ) − E ( u (cid:63) ) (cid:17) .
Revised: E n ( u ∗ ) − E n (ˆ u PINN ) and the approximation error ∆ E app = inf u F ∈ F (Ω) (cid:16) E ( u F ) − E ( u (cid:63) ) (cid:17) .

Original: From the lemmata proved in Section B.3, we already have an estimation of the approximation error’s convergence rate.
Revised: From the lemmata proved in Section B.3, we already have an estimation of the approximation error’s convergence rate.

Original: So now we’ll focus on providing fast rate upper bounds of the generalization error for the two estimators using the localization techinque(Bartlett et al., 2005; Xu, 2020).
Revised: So now we’ll focus on providing fast rate upper bounds of the generalization error for the two estimators using the localization techinque(Bartlett et al., 2005; Xu, 2020).

Original: To achieve the fast generalization bound, we focus on the following normalized empirical process
Revised: To achieve the fast generalization bound, we focus on the following normalized empirical process

Original: [Equation]
Revised: [Equation]

Original: First, we try to bound the expectation of the normalized empirical process.
Revised: First, we try to bound the expectation of the normalized empirical process.

Original: Applying the Symmetrization Lemma B.1, we can ﬁrst bound the expectation as
Revised: Applying the Symmetrization Lemma B.1, we can ﬁrst bound the expectation as

Original: [Equation]
Revised: [Equation]

Original: where the function class ˆ S r (Ω) is deﬁned as:
Revised: where the function class ˆ S r (Ω) is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where T (Ω) =
Revised: where T (Ω) =

Original: (cid:110) h :=
Revised: (cid:110) h :=

Original: | Ω | · (cid:2) (∆ u − V u + f ) 2 − (∆ u ∗ − V u ∗ + f ) 2 (cid:3) (cid:111) .
Revised: | Ω | · (cid:2) (∆ u − V u + f ) 2 − (∆ u ∗ − V u ∗ + f ) 2 (cid:3) (cid:111) .

Original: Then applying the Peeling
Revised: Then applying the Peeling

Original: Lemma B.4 to any function h ∈ T (Ω) helps us upper bound the local Rademacher complexity
Revised: Lemma B.4 to any function h ∈ T (Ω) helps us upper bound the local Rademacher complexity

Original: R n ( ˆ T r (Ω)) with the function φ deﬁned in equation B.66: 
Revised: R n ( ˆ T r (Ω)) with the function φ deﬁned in equation B.66: [Equation] h [Equation]

Revised: Ω

Original: [Equation]
Revised: [Equation]

Revised: 1 n

Revised: [Equation]

Revised: [Equation]

Revised: r

Revised: [Equation]

Revised: r .

Original: Combining all inequalities derived above yields:
Revised: Combining all inequalities derived above yields:

Original: [Equation]
Revised: [Equation]

Original: Secondly we’ll apply the Talagrand concentration inequality, which requires us to verify the condition needed.
Revised: Secondly we’ll apply the Talagrand concentration inequality, which requires us to verify the condition needed.

Original: We will ﬁrst check that the expectation value E [ h ] is always non-negative for any h ∈ S (Ω) :
Revised: We will ﬁrst check that the expectation value E [ h ] is always non-negative for any h ∈ S (Ω) :

Original: [Equation]
Revised: [Equation]

Original: We will proceed to verify that any ˜ h
Revised: We will proceed to verify that any ˜ h

Original: = E [ h ] − h E [ h ]+ r ∈ ˜ T
Revised: = E [ h ] − h E [ h ]+ r ∈ ˜ T

Original: r (Ω) is of bounded inf-norm.
Revised: r (Ω) is of bounded inf-norm.

Original: We need to prove that any h ∈ T (Ω) is of bounded inf-norm beforehand.
Revised: We need to prove that any h ∈ T (Ω) is of bounded inf-norm beforehand.

Original: Using boundedness condition listed in equation B.65 implies:
Revised: Using boundedness condition listed in equation B.65 implies:

Original: [Equation]
Revised: [Equation]

Original: By taking M := | Ω | ( V max + 2) 2 C 2 , we then have (cid:107) h (cid:107) ∞ ≤ M for all h ∈ T (Ω) .
Revised: By taking M := | Ω | ( V max + 2) 2 C [Equation] we then have (cid:107) h (cid:107) ∞ ≤ M for all h ∈ T (Ω) .

Original: Note that the denominator can be lower bounded by | E
Revised: Note that the denominator can be lower bounded by | E

Original: [ h ] + r | ≥ r > 0 .
Revised: [ h ] + r | ≥ r > 0 .

Original: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜ h (cid:107) ∞
Revised: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜ h (cid:107) ∞

Original: = sup x ∈ Ω | ˜ h
Revised: = sup x ∈ Ω | ˜ h

Original: ( x ) | as follows:
Revised: ( x ) | as follows:

Original: [Equation]
Revised: [Equation]

Original: We will then check the normalized functions
Revised: We will then check the normalized functions

Original: E [ h ] − h ( x ) E
Revised: E [ h ] − h ( x ) E

Original: [ h ]+ r in ˜ T
Revised: [ h ]+ r in ˜ T

Original: r (Ω) have bounded second moment, which is satisﬁed because of the regularity results of the PDE.
Revised: r (Ω) have bounded second moment, which is satisﬁed because of the regularity results of the PDE.

Original: We aim to show that there exist some constants α, α (cid:48) > 0 , such that for any h ∈ T (Ω) , the following inequality holds:
Revised: We aim to show that there exist some constants α, α (cid:48) > 0 , such that for any h ∈ T (Ω) , the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: The RHS of the inequality follows from strong convexity of the PINN objective function proved in
Revised: The RHS of the inequality follows from strong convexity of the PINN objective function proved in

Original: Theorem B.2:
Revised: Theorem B.2:

Original: [Equation]
Revised: [Equation]

Original: The LHS of the inequality follows from boundedness condition listed in equation B.65 and the
Revised: The LHS of the inequality follows from boundedness condition listed in equation B.65 and the

Original: QM-AM inequality:
Revised: QM-AM inequality:

Original: [Equation]
Revised: [Equation]

Original: By picking α (cid:48)
Revised: By picking α (cid:48)

Original: = 1 min { 1 ,C min } and α = 1 2 M 2 max { 1 ,V 2 max } , we have ﬁnished proving inequality B.69.
Revised: = 1 min { 1 ,C min } and α = 1 2 M 2 max { 1 ,V 2 max } , we have ﬁnished proving inequality B.69.

Original: Then we can can upper bound the expectation E
Revised: Then we can can upper bound the expectation E

Original: [˜ h 2 ] as:
Revised: [˜ h 2 ] as:

Original: [Equation]
Revised: [Equation]

Original: Using the fact that E [ h ] ≥ 0 and inequality B.69, we can lower bound the denominator | E [ h ] + r | 2 as follows:
Revised: Using the fact that E [ h ] ≥ 0 and inequality B.69, we can lower bound the denominator | E [ h ] + r | 2 as follows:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we can deduce that:
Revised: Therefore, we can deduce that:

Original: [Equation]
Revised: [Equation]

Original: Hence, any function in the localized class
Revised: Hence, any function in the localized class

Original: ˜ T
Revised: ˜ T

Original: r (Ω) is of bounded second moment.
Revised: r (Ω) is of bounded second moment.

Original: It is easy to check that for any ˜ h ∈ ˜ T r (Ω) , we have
Revised: It is easy to check that for any ˜ h ∈ ˜ T r (Ω) , we have

Original: [Equation]
Revised: [Equation]

Original: i.e. any function in the localized class
Revised: i.e. any function in the localized class

Original: ˜ S
Revised: ˜ S

Original: r (Ω) is of zero mean.
Revised: r (Ω) is of zero mean.

Original: Now we have veriﬁed that any function ˜ h ∈
Revised: Now we have veriﬁed that any function ˜ h ∈

Original: ˜ S
Revised: ˜ S

Original: r (Ω) satisﬁes all the required conditions.
Revised: r (Ω) satisﬁes all the required conditions.

Original: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s Concentration inequality given in Lemma B.3, we have:
Revised: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s Concentration inequality given in Lemma B.3, we have:

Original: [Equation]
Revised: [Equation]

Original: By using the upper bound deduced above and plugging in the expressions of β and σ , we can rewrite
Revised: By using the upper bound deduced above and plugging in the expressions of β and σ [Equation] we can rewrite

Original: Talagrand’s Concentration Inequality in the following way.
Revised: Talagrand’s Concentration Inequality in the following way.

Original: With probability at least 1 − e − t , the inequality below holds:
Revised: With probability at least 1 − e − t , the inequality below holds:

Original: [Equation]
Revised: [Equation]

Original: Let’s pick the critical radius r 0 to be:
Revised: Let’s pick the critical radius r 0 to be:

Original: [Equation]
Revised: [Equation]

Original: Note that concavity of the function φ implies that φ ( r ) ≤ r for any r ≥ r ∗ .
Revised: Note that concavity of the function φ implies that φ ( r ) ≤ r for any r ≥ r ∗ .

Original: Combining this with the ﬁrst inequality listed in B.66 yields:
Revised: Combining this with the ﬁrst inequality listed in B.66 yields:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, applying equation B.70 yields:
Revised: On the other hand, applying equation B.70 yields:

Original: [Equation]
Revised: [Equation]

Original: Summing the three inequalities above implies:
Revised: Summing the three inequalities above implies:

Original: [Equation]
Revised: [Equation]

Original: By picking r = r
Revised: By picking r = r

Original: [Equation]
Revised: [Equation]

Original: we can further deduce that for any function u ∈ F (Ω) , the following inequality holds with probability 1 − e − t :
Revised: we can further deduce that for any function u ∈ F (Ω) , the following inequality holds with probability 1 − e − t :

Original: [Equation]
Revised: [Equation]

Original: Multiplying the denominator

Original: [Equation]
Revised: [Equation]

Original: Substituting the upper bound above into the decomposition ∆ E ( n ) ≤ ∆ E gen + 32 ∆ E app + t 2 n yields that with probability 1 − 2 e − min { 1 ,C q } t , we have:
Revised: Substituting the upper bound above into the decomposition ∆ E ( n ) ≤ ∆ E gen + 32 ∆ E app + t 2 n yields that with probability 1 − 2 e − min { 1 ,C q } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Simplifying the inequality above yields that with probability 1 − 2 e − min { 1 ,C q } t , we have:
Revised: Simplifying the inequality above yields that with probability 1 − 2 e − min { 1 ,C q } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Moreover, using strong convexity of the PINN objective function proved in Theorem B.2 implies:
Revised: Moreover, using strong convexity of the PINN objective function proved in Theorem B.2 implies:

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above yields that with probability 1 − 2 e − min { 1 ,C q } t , we have:
Revised: Combining the two bounds above yields that with probability 1 − 2 e − min { 1 ,C q } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Deep Neural Network Estimator.
Revised: Deep Neural Network Estimator.

Original: For any N ∈ Z + , there exists some Deep Neural Network in Φ( L, W, S, B ) with L
Revised: For any N ∈ Z + , there exists some Deep Neural Network in Φ( L, W, S, B ) with L

Original: = O (1) , W = O ( N ) , S =
Revised: = O (1) , W = O ( N ) , S =

Original: O ( N ) , B = O ( N ) , such that the approximation error ∆ E app
Revised: O ( N ) , B = O ( N ) , such that the approximation error ∆ E app

Original: =
Revised: =

Original: O ( N − 2( s − 2) d ) and generalization error ∆
Revised: O ( N − 2( s − 2) d ) and generalization error ∆

Original: E gen = O ( N log N n ) .
Revised: E gen = O ( N log N n ) .

Original: With optimal selection
Revised: With optimal selection

Original: N = n d d +2 s − 4 to balance the bias and variance, we can achieve n − 2 s − 4 d +2 s − 4 log n convergence rate for
Revised: N = n d d +2 s − 4 to balance the bias and variance, we can achieve n − 2 s − 4 d +2 s − 4 log n convergence rate for

Original: PINN estimator.
Revised: PINN estimator.

Original: Theorem B.13.
Revised: Theorem B.13.

Original: (Final Upper Bound of PINN with Deep Neural Network Estimator) Under the assumptions in Theorem B.12, we consider the PINN objective with the sparse Deep
Revised: (Final Upper Bound of PINN with Deep Neural Network Estimator) Under the assumptions in Theorem B.12, we consider the PINN objective with the sparse Deep

Original: Neural Network function space Φ( L, W, S, B ) , where the parameters L
Revised: Neural Network function space Φ( L, W, S, B ) , where the parameters L

Original: = O (1) , W =
Revised: = O (1) , W =

Original: O ( n d d +2 s − 4 ) , S =
Revised: O ( n d d +2 s − 4 ) , S =

Original: O ( n
Revised: O ( n

Original: [Equation]
Revised: [Equation]

Original: 4 ) and B = O ( n
Revised: 4 ) and B = O ( n

Original: [Equation]
Revised: [Equation]

Original: Then we have that the DNN estimator 
Revised: Then we have that the DNN estimator ˆ

Original: [Equation]
Revised: [Equation]

Original: ( u ) satisﬁes the following upper bound with high probability:
Revised: u ) satisﬁes the following upper bound with high probability:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: On the one hand, by taking s = 2 and p = 2 in Theorem B.8 proved above, we have that there exists some Deep Neural Network u DNN ∈
Revised: On the one hand, by taking s = 2 and p = 2 in Theorem B.8 proved above, we have that there exists some Deep Neural Network u DNN ∈

Original: Φ( L, W, S, B ) with L = O (1) , W = O ( N ) , S =
Revised: Φ( L, W, S, B ) with L = O (1) , W = O ( N ) , S =

Original: O ( N ) , B = O ( N ) , such that.
Revised: O ( N ) , B = O ( N ) , such that.

Original: [Equation]
Revised: [Equation]

Original: Applying strong convexity of the PINN objective function proved in Theorem B.2 yields the following upper bound on the approximation error of PINN:
Revised: Applying strong convexity of the PINN objective function proved in Theorem B.2 yields the following upper bound on the approximation error of PINN:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, from lemma B.18 proved above, we know that the function φ ( ρ ) that upper bounds the local Rademacher complexity of the Deep Neural Networks u DNN is of the same magnitude as (cid:113)
Revised: On the other hand, from lemma B.18 proved above, we know that the function φ ( ρ ) that upper bounds the local Rademacher complexity of the Deep Neural Networks u DNN is of the same magnitude as (cid:113)

Original: S 3 L ρ n log ( BWn ) .
Revised: S 3 L ρ n log ( BWn ) .

Original: By plugging in the magnitudes of L, W, S, B , we can determine the critical radius r ∗ :
Revised: By plugging in the magnitudes of L, W, S, B , we can determine the critical radius r ∗ :

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above with Theorem B.12 yields that with high probability, we have:
Revised: Combining the two bounds above with Theorem B.12 yields that with high probability, we have:

Original: [Equation]
Revised: [Equation]

Original: By equating the two terms above, we can solve for the optimal N that yields the desired bound:
Revised: By equating the two terms above, we can solve for the optimal N that yields the desired bound:

Original: [Equation]
Revised: [Equation]

Original: Plugging in the optimal N gives us the magnitudes of the four parameters L = O (1) , W =
Revised: Plugging in the optimal N gives us the magnitudes of the four parameters L = O (1) , W =

Original: O ( n
Revised: O ( n

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) , as well as the ﬁnal rate:
Revised: , as well as the ﬁnal rate:

Original: [Equation]
Revised: [Equation]

Original: Truncated Fourier Series Estimator.
Revised: Truncated Fourier Series Estimator.

Original: For any ξ ∈ Z + , there exists some Truncated Fourier Series in F ξ (Ω) with approximation error ∆ E app
Revised: For any ξ ∈ Z + , there exists some Truncated Fourier Series in F ξ (Ω) with approximation error ∆ E app

Original: =
Revised: =

Original: O ( ξ − 2( s − 2) ) and generalization error ∆ E gen = O ( ξ d n ) .
Revised: O ( ξ − 2( s − 2) ) and generalization error ∆ E gen = O ( ξ d n ) .

Original: With optimal selection ξ = n 1 d +2 s − 4 to balance the bias and variance, we can achieve n − 2 s − 4 d
Revised: With optimal selection ξ = n 1 d +2 s − 4 to balance the bias and variance, we can achieve n − 2 s − 4 d

Original: +2 s −convergence rate for the Fourier estimator.
Revised: +2 s −convergence rate for the Fourier estimator.

Original: Theorem B.14.
Revised: Theorem B.14.

Original: (Final Upper Bound of PINN with Truncated Fourier Series Estimator) Under the assumptions in Theorem B.12, we consider the PINN objective with the Truncated Fourier Series function space F ξ (Ω) , where the parameter ξ =
Revised: (Final Upper Bound of PINN with Truncated Fourier Series Estimator) Under the assumptions in Theorem B.12, we consider the PINN objective with the Truncated Fourier Series function space F ξ (Ω) , where the parameter ξ =

Original: Θ( n 1 d +2 s − 4 ) .
Revised: Θ( n 1 d +2 s − 4 ) .

Original: Then we have that the Fourier estimator ˆ u FourierPINN = min u ∈ F ξ (Ω) E PINN n ( u ) satisﬁes the following upper bound with high probability:
Revised: Then we have that the Fourier estimator ˆ u FourierPINN = min u ∈ F ξ (Ω) E PINN n ( u ) satisﬁes the following upper bound with high probability:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let’s ﬁrstly derive the function φ ( ρ ) that upper bounds the local Rademacher complexity of T ρ (Ω) :
Revised: Let’s ﬁrstly derive the function φ ( ρ ) that upper bounds the local Rademacher complexity of T ρ (Ω) :

Original: = (cid:110) h :=
Revised: = (cid:110) h :=

Original: | Ω | · (cid:2) (∆ u − V u + f )

Original: [Equation]

Original:  − (∆ u [Equation] − V u ∗ + f ) 2 (cid:3) (cid:12)(cid:12)(cid:12) u ∈ J ρ,ξ (Ω) (cid:111) , where
Revised: | Ω | · (cid:2) (∆ u − V u + f ) 2 − (∆ u ∗ − V u ∗ + f ) 2 (cid:3) (cid:12)(cid:12)(cid:12) u ∈ J ρ,ξ (Ω) (cid:111) , where

Original: J ρ,ξ (Ω) =
Revised: J ρ,ξ (Ω) =

Original: (cid:110) v ∈ F ξ (Ω) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 2 (Ω) ≤ ρ (cid:111) denotes the localized Truncated Fourier Series space.
Revised: (cid:110) v ∈ F ξ (Ω) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 2 (Ω) ≤ ρ (cid:111) denotes the localized Truncated Fourier Series space.

Original: From Talagrand Contraction Lemma B.2, Lemma B.6 and Lemma B.8 proved above, we have
Revised: From Talagrand Contraction Lemma B.2, Lemma B.6 and Lemma B.8 proved above, we have

Original: Rn
Revised: R

Original: [Equation]
Revised: [Equation]

Original:  u − Π ξ u ∗ (cid:107) 2 H 1 (Ω) ≤ ρ (cid:105)(cid:35) + E X (cid:34)
Revised: T [Equation] u − Π ξ u ∗ (cid:107) 2 H 1 (Ω) ≤ ρ (cid:105)(cid:35) + E X (cid:34)

Original: E σ (cid:104)n n (cid:88) i =
Revised: E σ (cid:104)n n (cid:88) i =

Original: σ i ∆Π >ξ u ∗ ( X i ) (cid:105) (cid:35) + E X (cid:34)
Revised: σ i ∆Π >ξ u ∗ ( X i ) (cid:105) (cid:35) + E X (cid:34)

Original: E σ (cid:104)n n (cid:88) i =
Revised: E σ (cid:104)n n (cid:88) i =

Original: σ i Π >ξ u ∗ ( X i ) (cid:105) (cid:35) (cid:46) E X (cid:34)
Revised: σ i Π >ξ u ∗ ( X i ) (cid:105) (cid:35) (cid:46) E X (cid:34)

Original: E σ (cid:104) sup v ∈ F ρ,ξ (Ω) n n (cid:88) i =
Revised: E σ (cid:104) sup v ∈ F ρ,ξ (Ω) n n (cid:88) i =

Original: σ i v ( X i ) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 2 (Ω) ≤ ρ (cid:105)(cid:35) + E X (cid:34)
Revised: σ i v ( X i ) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 2 (Ω) ≤ ρ (cid:105)(cid:35) + E X (cid:34)

Original: E σ (cid:104) sup v ∈ F ρ,ξ (Ω) n n (cid:88) i =
Revised: E σ (cid:104) sup v ∈ F ρ,ξ (Ω) n n (cid:88) i =

Original: σ i ∆ v ( X i ) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 2 (Ω) ≤ ρ (cid:105)(cid:35) + E X (cid:34)
Revised: σ i ∆ v ( X i ) (cid:12)(cid:12)(cid:12) (cid:107) v (cid:107) 2 H 2 (Ω) ≤ ρ (cid:105)(cid:35) + E X (cid:34)

Original: E σ (cid:104)n n (cid:88) i =
Revised: E σ (cid:104)n n (cid:88) i =

Original: σ i ∆Π >ξ u ∗ ( X i ) (cid:105)(cid:35) + E X (cid:34)
Revised: σ i ∆Π >ξ u ∗ ( X i ) (cid:105)(cid:35) + E X (cid:34)

Original: E σ (cid:104)n n (cid:88) i =
Revised: E σ (cid:104)n n (cid:88) i =

Original: σ i Π >ξ u ∗ ( X i ) (cid:105)(cid:35) (cid:46) (cid:114) ρ nξ d 2 + (cid:114) (cid:107) Π >ξ u ∗ (cid:107) 

Revised: σ i Π >ξ u ∗ ( X i ) (cid:105)(cid:35) (cid:46) (cid:114) ρ nξ d 2 + (cid:114) (cid:107) Π >ξ u ∗ (cid:107)

Original: Hn (cid:46) (cid:114) 
Revised: Hn (cid:46) (cid:114) ρ

Original: [Equation]
Revised: [Equation]

Original: where Π ξ u := (cid:80) (cid:107) z (cid:107) ∞
Revised: where Π ξ u := (cid:80) (cid:107) z (cid:107) ∞

Original: ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is smaller than ξ and Π >ξ u := (cid:80) (cid:107) z (cid:107) ∞
Revised: ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is smaller than ξ and Π >ξ u := (cid:80) (cid:107) z (cid:107) ∞

Original: ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is larger than ξ .
Revised: ≤ ξ u z φ z ( x ) is the projection to the Fourier basis whose frequency is larger than ξ .

Original: Then, the critical radius r ∗ can be determined as follows:
Revised: Then, the critical radius r ∗ can be determined as follows:

Original: [Equation]
Revised: [Equation]

Original: Moreover, by taking α = s and β = 1 in Lemma B.19 and applying strong convexity of the DRM objective function proved in Theorem B.2, we can upper bound the approximation error ∆ E app as below:
Revised: Moreover, by taking α = s and β = 1 in Lemma B.19 and applying strong convexity of the DRM objective function proved in Theorem B.2, we can upper bound the approximation error ∆ E app as below:

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above with Theorem B.12 yields that with high probability, we have:
Revised: Combining the two bounds above with Theorem B.12 yields that with high probability, we have:

Original: [Equation]
Revised: [Equation]

Original: By equating two of the three terms above, we can solve for ξ that yields the desired bound:
Revised: By equating two of the three terms above, we can solve for ξ that yields the desired bound:

Original: [Equation]
Revised: [Equation]

Original: Plugging in the optimal ξ gives us the ﬁnal rate:
Revised: Plugging in the optimal ξ gives us the ﬁnal rate:

Original: [Equation]
Revised: [Equation]

Original: C P ROOF OF M ODIFIED DRM
Revised: C P ROOF OF M ODIFIED DRM

Original: In this section, we provide the proof of the modiﬁed deep Ritz method here.
Revised: In this section, we provide the proof of the modiﬁed deep Ritz method here.

Original: We ﬁrst provide a similar meta-theorem as we did for DRM.
Revised: We ﬁrst provide a similar meta-theorem as we did for DRM.

Original: Theorem C.1 (Meta-theorem for Upper Bounds of Modiﬁed Deep Ritz Method) .
Revised: Theorem C.1 (Meta-theorem for Upper Bounds of Modiﬁed Deep Ritz Method) .

Original: Let u ∗ ∈ H s (Ω) denote the true solution to the PDE model with Dirichlet boundary condition:
Revised: Let u ∗ ∈ H s (Ω) denote the true solution to the PDE model with Dirichlet boundary condition:

Original: [Equation]
Revised: [Equation]

Original: where f ∈ L 2 (Ω) and V ∈ L ∞ (Ω) with 0 < V min ≤ V ( x ) ≤ V max > 0 .
Revised: where f ∈ L 2 (Ω) and V ∈ L ∞ (Ω) with 0 < V min ≤ V ( x ) ≤ V max > 0 .

Original: In Theorem B.1, it has been proved that u ∗ can be obtained by minimizing the loss E ( u ) :
Revised: In Theorem B.1, it has been proved that u ∗ can be obtained by minimizing the loss E ( u ) :

Original: [Equation]
Revised: [Equation]

Original: For a ﬁxed function space F (Ω) , consider the empirical loss induced by the Modiﬁed Deep Ritz
Revised: For a ﬁxed function space F (Ω) , consider the empirical loss induced by the Modiﬁed Deep Ritz

Original: Method ( N ≥ n ) :
Revised: Method ( N ≥ n ) :

Original: [Equation]
Revised: [Equation]

Original: where { X (cid:48) i } Ni =1 and { X j } nj =1 are datapoints uniformly and independently sampled from the domain Ω .
Revised: where { X (cid:48) i } Ni =1 and { X j } nj =1 are datapoints uniformly and independently sampled from the domain Ω .

Original: Then the Modiﬁed Deep Ritz estimator associated with function space F (Ω) is deﬁned as the minimizer of E N,n ( u ) over the function space F (Ω) :
Revised: Then the Modiﬁed Deep Ritz estimator associated with function space F (Ω) is deﬁned as the minimizer of E N,n ( u ) over the function space F (Ω) :

Original: [Equation]
Revised: [Equation]

Original: Moreover, we assume that there exists some constant C > 0 such that all function u in the function space F (Ω) , the real solution u ∗ and f, V satisfy the following two conditions.
Revised: Moreover, we assume that there exists some constant C > 0 such that all function u in the function space F (Ω) , the real solution u ∗ and f, V satisfy the following two conditions.

Original: • The gradients and function value are uniformly bounded
Revised: • The gradients and function value are uniformly bounded

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: C

Original: [Table]
Revised: [Table]

Original: (C.3)
Revised: (C.3)

Original: • All the functions in the function space F (Ω) satisfy the boundary condition
Revised: • All the functions in the function space F (Ω) satisfy the boundary condition

Original: [Equation]
Revised: [Equation]

Original:  At the the same time, for any ρ > 0 , we assume the Rademacher complexity of the following vector-valued function space
Revised: on [Equation] At the the same time, for any ρ > 0 , we assume the Rademacher complexity of the following vector-valued function space

Original: [Equation]
Revised: [Equation]

Original: can be upper bounded by a sub-root function φ = φ ( ρ ) : [0 , ∞ )
Revised: can be upper bounded by a sub-root function φ = φ ( ρ ) : [0 , ∞ )

Original: → [0 , ∞ ) , i.e. φ (4 ρ ) ≤ 2 φ ( ρ ) and R N,n ( S ρ (Ω)) ≤ φ ( ρ ) ( ∀ ρ > 0) , (C.4) where R N,n ( S ) :
Revised: → [0 , ∞ ) , i.e. φ (4 ρ ) ≤ 2 φ ( ρ ) and R N,n ( S ρ (Ω)) ≤ φ ( ρ ) ( ∀ ρ > 0) , (C.4) where R N,n ( S ) :

Original: = R N ( { h 1 | ( h 1 , h 2 ) ∈ S } ) + R n ( { h 2 | ( h 1 , h 2 ) ∈ S } ) .
Revised: = R N ( { h 1 | ( h 1 , h 2 ) ∈ S } ) + R n ( { h 2 | ( h 1 , h 2 ) ∈ S } ) .

Original: For all constant t > 0 .
Revised: For all constant t > 0 .

Original: We denote r ∗ to be the solution of the ﬁx point equation of local Rademacher complexity r = φ ( r ) .
Revised: We denote r ∗ to be the solution of the ﬁx point equation of local Rademacher complexity r = φ ( r ) .

Original: There exists two constants C p , C q such that with probability 1 − C p exp( − C q t ) , we have the following upper bound for the Modiﬁed Deep Ritz Estimator
Revised: There exists two constants C p , C q such that with probability 1 − C p exp( − C q t ) , we have the following upper bound for the Modiﬁed Deep Ritz Estimator

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: To upper bound the excess risk ∆ E ( N,n ) :
Revised: To upper bound the excess risk ∆ E ( N,n ) :

Original: =
Revised: =

Original: E (ˆ u MDRM ) − E ( u ∗ ) , following(Xu, 2020;
Revised: E (ˆ u MDRM ) − E ( u ∗ ) , following(Xu, 2020;

Original: Lu et al., 2021b; Duan et al., 2021), we decompose the excess risk into approximation error and generalization error with probability 1 − e − t :
Revised: Lu et al., 2021b; Duan et al., 2021), we decompose the excess risk into approximation error and generalization error with probability 1 − e − t :

Original: [Equation]
Revised: [Equation]

Original: where the expectation is on all sampled data.
Revised: where the expectation is on all sampled data.

Original: The inequality of the third line is because ˆ u MDRM is the minimizer of the empirical loss E n in the solution
Revised: The inequality of the third line is because ˆ u MDRM is the minimizer of the empirical loss E n in the solution

Original: set F (Ω) , so we have E N,n (ˆ u MDRM ) ≤ E N,n ( u F ) .
Revised: set F (Ω) , so we have E N,n (ˆ u MDRM ) ≤ E N,n ( u F ) .

Original: The last inequality is based on the Bernstein inequality.
Revised: The last inequality is based on the Bernstein inequality.

Original: For any u F ∈ F (Ω) , we use h F , 1 , h F , 2 to denote the following two functions:
Revised: For any u F ∈ F (Ω) , we use h F , 1 , h F , 2 to denote the following two functions:

Original: [Equation]
Revised: [Equation]

Original: Applying Bernstein’s inequality twice to h F , 1 and h F , 2 implies that there exists some constant C q , such that with probability 1 − 2 e − C q t , the following two inequalities hold simultaneously:
Revised: Applying Bernstein’s inequality twice to h F , 1 and h F , 2 implies that there exists some constant C q , such that with probability 1 − 2 e − C q t , the following two inequalities hold simultaneously:

Original: [Equation]
Revised: [Equation]

Original: Note that the variance sum E [ h 2 F , 1 ] + E [ h 2 F , 2 ] can be upper bounded by α (cid:48)
Revised: Note that the variance sum E [ h 2 F , 1 ] + E [ h 2 F , 2 ] can be upper bounded by α (cid:48)

Original: α (cid:2) E ( u F ) − E ( u (cid:63) ) (cid:3) due to the strong convexity of the variation objective (C.9).
Revised: α (cid:2) E ( u F ) − E ( u (cid:63) ) (cid:3) due to the strong convexity of the variation objective (C.9).

Original: Adding the two inequalities above implies with probability 1 − 2 e − C q t we have:
Revised: Adding the two inequalities above implies with probability 1 − 2 e − C q t we have:

Original: [Equation]
Revised: [Equation]

Original: Note that C.5 holds for all function lies in the function space F .
Revised: Note that C.5 holds for all function lies in the function space F .

Original: Thus, we can take u F :=
Revised: Thus, we can take u F :=

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: and ﬁnally get:
Revised: and ﬁnally get:

Original: [Equation]
Revised: [Equation]

Original: This inequality decomposes the excess risk to the generalization error ∆ E gen := E (ˆ u MDRM ) −
Revised: This inequality decomposes the excess risk to the generalization error ∆ E gen := E (ˆ u MDRM ) −

Original: E ( u ∗ ) + E N,n ( u ∗ ) − E N,n (ˆ u MDRM ) and the approximation error ∆ E app = inf u F ∈ F (Ω) (cid:16) E ( u F ) −
Revised: E ( u ∗ ) + E N,n ( u ∗ ) − E N,n (ˆ u MDRM ) and the approximation error ∆ E app = inf u F ∈ F (Ω) (cid:16) E ( u F ) −

Original: E ( u (cid:63) ) (cid:17) .
Revised: E ( u (cid:63) ) (cid:17) .

Original: From the lemmata proved in Section B.3, we already have an estimation of the approximation error’s convergence rate.
Revised: From the lemmata proved in Section B.3, we already have an estimation of the approximation error’s convergence rate.

Original: So now we’ll focus on providing fast rate upper bounds of the generalization error for the two estimators using the localization techinque(Bartlett et al., 2005; Xu, 2020).
Revised: So now we’ll focus on providing fast rate upper bounds of the generalization error for the two estimators using the localization techinque(Bartlett et al., 2005; Xu, 2020).

Original: To achieve the fast generalization bound, we focus on the following two normalized empirical processes:
Revised: To achieve the fast generalization bound, we focus on the following two normalized empirical processes:

Original: [Equation]
Revised: [Equation]

Original: where the space S (Ω) is deﬁned as:
Revised: where the space S (Ω) is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: First, we try to bound the expectation of the two normalized empirical processes.
Revised: First, we try to bound the expectation of the two normalized empirical processes.

Original: Applying the
Revised: Applying the

Original: Symmetrization Lemma B.1, we can ﬁrst bound the two expectations as:
Revised: Symmetrization Lemma B.1, we can ﬁrst bound the two expectations as:

Original: [Equation]
Revised: [Equation]

Original: where the function classes ˆ S r [Equation] k (Ω) (1 ≤ k ≤ 2) are deﬁned as:
Revised: where the function classes ˆ S r,k (Ω) (1 ≤ k ≤ 2) are deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: Applying the modiﬁed Peeling Lemma B.5 to any function h
Revised: Applying the modiﬁed Peeling Lemma B.5 to any function h

Original: = ( h 1 , h 2 ) ∈ S (Ω) helps us upper bound the sum of the two local Rademacher complexities R N ( ˆ S r, 1 (Ω))
Revised: = ( h 1 , h 2 ) ∈ S (Ω) helps us upper bound the sum of the two local Rademacher complexities R N ( ˆ S r, 1 (Ω))

Original: + R n ( ˆ S r, 2 (Ω)) with the function φ deﬁned in equation C.4:
Revised: + R n ( ˆ S r, 2 (Ω)) with the function φ deﬁned in equation C.4:

Original: [Equation]
Revised: [Equation]

Original: Combining all inequalities derived above yields:
Revised: Combining all inequalities derived above yields:

Original: [Equation]
Revised: [Equation]

Original: Secondly we’ll apply the Talagrand concentration inequality to the two function classes ˜ S r, 1 (Ω) and ˜ S r, 2 (Ω) , which requires us to verify the conditions needed.
Revised: Secondly we’ll apply the Talagrand concentration inequality to the two function classes ˜ S r, 1 (Ω) and ˜ S r, 2 (Ω) , which requires us to verify the conditions needed.

Original: We will ﬁrst check that the expectation sum E [ h 1 ] + E
Revised: We will ﬁrst check that the expectation sum E [ h 1 ] + E

Original: [ h 2 ] is always non-negative for any ( h 1 , h 2 ) ∈ S (Ω) :
Revised: [ h 2 ] is always non-negative for any ( h 1 , h 2 ) ∈ S (Ω) :

Original: [Equation]
Revised: [Equation]

Original: Next, We will verify that ˜ S r, 1 (Ω) satisﬁes all three requirements.
Revised: Next, We will verify that ˜ S r, 1 (Ω) satisﬁes all three requirements.

Original: At ﬁrst, we will show that any ˜ h 1 =
Revised: At ﬁrst, we will show that any ˜ h 1 =

Original: E [ h 1 ] − h 1 E [ h 1 ]+ E [ h 2 ]+ r ∈ ˜ S r, 1 (Ω) is of bounded inf-norm.
Revised: E [ h 1 ] − h 1 E [ h 1 ]+ E [ h 2 ]+ r ∈ ˜ S r, 1 (Ω) is of bounded inf-norm.

Original: We need to prove that any h 1 ∈ S 1 (Ω) is of bounded inf-norm beforehand.
Revised: We need to prove that any h 1 ∈ S 1 (Ω) is of bounded inf-norm beforehand.

Original: Using boundedness condition listed in equation C.3 implies:
Revised: Using boundedness condition listed in equation C.3 implies:

Original: [Equation]
Revised: [Equation]

Original: By taking M 1 :
Revised: By taking M 1 :

Original: = C 2 , we then have (cid:107) h 1 (cid:107) ∞
Revised: = C 2 , we then have (cid:107) h 1 (cid:107) ∞

Original: ≤ M 1 for all h 1 ∈ S 1 (Ω) .
Revised: ≤ M 1 for all h 1 ∈ S 1 (Ω) .

Original: Note that the denominator of ˜ h 1 can be lower bounded by | E [ h 1 ] + E [ h 2 ] + r | ≥ r > 0 .
Revised: Note that the denominator of ˜ h 1 can be lower bounded by | E [ h 1 ] + E [ h 2 ] + r | ≥ r > 0 .

Original: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜
Revised: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜

Original: h 1 (cid:107) ∞ =
Revised: h 1 (cid:107) ∞ =

Original: sup x ∈ Ω | ˜
Revised: sup x ∈ Ω | ˜

Original: h 1 ( x ) | as follows:
Revised: h 1 ( x ) | as follows:

Original: [Equation]
Revised: [Equation]

Original: Also, it is easy to check that for any ˜
Revised: Also, it is easy to check that for any ˜

Original: h 1 ∈ ˜ S r, 1 (Ω) , we have
Revised: h 1 ∈ ˜ S r, 1 (Ω) , we have

Original: [Equation]
Revised: [Equation]

Original: i.e. any function in the localized class ˜ S r, 1 (Ω) is of zero mean.
Revised: i.e. any function in the localized class ˜ S r, 1 (Ω) is of zero mean.

Original: Moreover, we take σ 21 = sup ˜ h
Revised: Moreover, we take σ 21 = sup ˜ h

Original: 1 ∈ ˜ S
Revised: 1 ∈ ˜ S

Original: r, 1 (Ω) E
Revised: r, 1 (Ω) E

Original: [˜ h 21 ] to be the upper bound on the second moment of functions in ˜ S r, 1 (Ω) .
Revised: [˜ h 21 ] to be the upper bound on the second moment of functions in ˜ S r, 1 (Ω) .

Original: Now we have veriﬁed that any function ˜ h
Revised: Now we have veriﬁed that any function ˜ h

Original: 1 ∈ ˜ S
Revised: 1 ∈ ˜ S

Original: r, 1 (Ω) satisﬁes all the required conditions.
Revised: r, 1 (Ω) satisﬁes all the required conditions.

Original: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s
Revised: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s

Original: Concentration inequality given in Lemma B.3, we have:
Revised: Concentration inequality given in Lemma B.3, we have:

Original: [Equation]
Revised: [Equation]

Original: Moreover, We will verify that ˜ S r, 2 (Ω) also satisﬁes all three requirements.
Revised: Moreover, We will verify that ˜ S r, 2 (Ω) also satisﬁes all three requirements.

Original: At ﬁrst, we will show that any ˜ h 2 =
Revised: At ﬁrst, we will show that any ˜ h 2 =

Original: E [ h 2 ] − h 2 E
Revised: E [ h 2 ] − h 2 E

Original: [ h 1 ]+ E [ h 2 ]+ r ∈ ˜ S r, 2 (Ω) is of bounded inf-norm.
Revised: [ h 1 ]+ E [ h 2 ]+ r ∈ ˜ S r, 2 (Ω) is of bounded inf-norm.

Original: We need to prove that any h 2 ∈ S 2 (Ω) is of bounded inf-norm beforehand.
Revised: We need to prove that any h 2 ∈ S 2 (Ω) is of bounded inf-norm beforehand.

Original: Using boundedness condition listed in equation C.3 implies:
Revised: Using boundedness condition listed in equation C.3 implies:

Original: [Equation]
Revised: [Equation]

Original: By taking M 2 :
Revised: By taking M 2 :

Original: = ( V max + 2) C 2 , we then have (cid:107) h 2 (cid:107) ∞ ≤ M 2 for all h 2 ∈ S 2 (Ω) .
Revised: = ( V max + 2) C 2 , we then have (cid:107) h 2 (cid:107) ∞ ≤ M 2 for all h 2 ∈ S 2 (Ω) .

Original: Note that the denominator of ˜ h 2 can be lower bounded by | E [ h 1 ] + E [ h 2 ] + r | ≥ r > 0 .
Revised: Note that the denominator of ˜ h 2 can be lower bounded by | E [ h 1 ] + E [ h 2 ] + r | ≥ r > 0 .

Original: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜
Revised: Combining these two inequalities help us upper bound the inf-norm (cid:107) ˜

Original: h 2 (cid:107) ∞ =
Revised: h 2 (cid:107) ∞ =

Original: sup x ∈ Ω | ˜ h 2 ( x ) | as follows:
Revised: sup x ∈ Ω | ˜ h 2 ( x ) | as follows:

Original: [Equation]
Revised: [Equation]

Original: Also, it is easy to check that for any ˜
Revised: Also, it is easy to check that for any ˜

Original: h 2 ∈ ˜ S
Revised: h 2 ∈ ˜ S

Original: r, 2 (Ω) , we have
Revised: r, 2 (Ω) , we have

Original: [Equation]
Revised: [Equation]

Original: i.e. any function in the localized class ˜ S r, 2 (Ω) is of zero mean.
Revised: i.e. any function in the localized class ˜ S r, 2 (Ω) is of zero mean.

Original: Moreover, we take σ 22 = sup ˜ h 2 ∈ ˜ S
Revised: Moreover, we take σ 22 = sup ˜ h 2 ∈ ˜ S

Original: r, 2 (Ω) E
Revised: r, 2 (Ω) E

Original: [˜ h 22 ] to be the upper bound on the second moment of functions in ˜ S r, 2 (Ω) .
Revised: [˜ h 22 ] to be the upper bound on the second moment of functions in ˜ S r, 2 (Ω) .

Original: Now we have veriﬁed that any function ˜ h
Revised: Now we have veriﬁed that any function ˜ h

Original: 2 ∈ ˜ S
Revised: 2 ∈ ˜ S

Original: r, 2 (Ω) satisﬁes all the required conditions.
Revised: r, 2 (Ω) satisﬁes all the required conditions.

Original: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s
Revised: By taking µ to be the uniform distribution on the domain Ω and applying Talagrand’s

Original: Concentration inequality given in Lemma B.3, we have:
Revised: Concentration inequality given in Lemma B.3, we have:

Original: [Equation]
Revised: [Equation]

Original: By applying a union bound to the two inequalities derived in C.7 and C.8, we can derive that with probability at least 1 − 2 e − t , the inequality below holds:
Revised: By applying a union bound to the two inequalities derived in C.7 and C.8, we can derive that with probability at least 1 − 2 e − t , the inequality below holds:

Original: [Equation]
Revised: [Equation]

Original: By the deﬁnition of β 1 and β 2 , we have that the term 2 t ( β 1 + β 2 ) n can be upper bounded by:
Revised: By the deﬁnition of β 1 and β [Equation] we have that the term 2 t ( β 1 + β 2 ) n can be upper bounded by:

Original: [Equation]
Revised: [Equation]

Original: Now we will derive some upper bound on the sum σ 1 + σ 2 .
Revised: Now we will derive some upper bound on the sum σ 1 + σ 2 .

Original: By deﬁnition we have that: ( σ 1 +σ 2 ) 2 ≤
Revised: By deﬁnition we have that: ( σ 1 +

Original: [Equation]
Revised: [Equation]

Original: σ 21 + σ

Original: [Equation]
Revised: [Equation]

Original: =

Original: [Equation]
Revised: [Equation]

Original: E [ h 1 ] + E [ h 2 ] + r | 2 .
Revised: [ h 1 ] + E [ h 2 ] + r | 2 .

Original: Now it sufﬁces to derive an upper bound of
Revised: Now it sufﬁces to derive an upper bound of

Original: [Equation] [Equation] [Equation] 22 ]|
Revised: E [ h 21 ]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original:  1 ]+ E [ [Equation] 2 ]+ r | 2 for any h ∈ S (Ω) .
Revised: 22 ] | E [ h 1 ]+ E [ h 2 ]+ r | 2 for any h ∈ S (Ω) .

Original: The existence of such an upper bound is guaranteed because of the regularity results of the PDE.
Revised: The existence of such an upper bound is guaranteed because of the regularity results of the PDE.

Original: We aim to show that there exist some constants α, α (cid:48) > 0 , such that for any h ∈ S (Ω) , the following inequality holds:
Revised: We aim to show that there exist some constants α, α (cid:48) > 0 , such that for any h ∈ S (Ω) , the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: The RHS of the inequality follows from strong convexity of the DRM objective function proved in
Revised: The RHS of the inequality follows from strong convexity of the DRM objective function proved in

Original: Theorem B.1:
Revised: Theorem B.1:

Original: [Equation]
Revised: [Equation]

Original: The LHS of the inequality follows from boundedness condition listed in equation C.3 and the QM-AM inequality:
Revised: The LHS of the inequality follows from boundedness condition listed in equation C.3 and the QM-AM inequality:

Original: [Equation]
Revised: [Equation]

Original: By picking α (cid:48)
Revised: By picking α (cid:48)

Original: = 4 min { 1 ,V min } and α = 1 2 C 2 (1+ V 2 max ) , we have ﬁnished proving inequality C.9.
Revised: = 4 min { 1 ,V min } and α = 1 2 C 2 (1+ V 2 max ) [Equation] we have ﬁnished proving inequality C [Equation]

Revised: [Equation]

Original: Then we can can upper bound the term
Revised: Then we can can upper bound the term

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [ h 22 ] | E [ h

Original: [Equation]
Revised: [Equation]

Original: as:
Revised: as:

Original: [Equation]
Revised: [Equation]

Original: Combining the bounds derived above helps us upper bound the term (cid:113) 2 tn ( σ 1 + σ 2 ) as below:
Revised: Combining the bounds derived above helps us upper bound the term (cid:113) 2 tn ( σ 1 + σ 2 ) as below:

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: using the two upper bounds on (cid:113) 2 tn ( σ 1 + σ 2 ) and 2 t ( β 1 + β 2 ) n , we have
Revised: using the two upper bounds on (cid:113) 2 tn ( σ 1 + σ 2 ) and 2 t ( β 1 + β 2 ) n , we have

Original: [Equation]
Revised: [Equation]

Original: Let’s pick the critical radius r 0 to be:
Revised: Let’s pick the critical radius r 0 to be:

Original: [Equation]
Revised: [Equation]

Original: Note that concavity of the function φ implies that φ ( r ) ≤ r for any r ≥ r ∗ .
Revised: Note that concavity of the function φ implies that φ ( r ) ≤ r for any r ≥ r ∗ .

Original: Combining this with the ﬁrst inequality listed in C.4 yields:
Revised: Combining this with the ﬁrst inequality listed in C.4 yields:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, applying equation C.10 yields:
Revised: On the other hand, applying equation C.10 yields:

Original: [Equation]
Revised: [Equation]

Original: Summing the three inequalities above implies:
Revised: Summing the three inequalities above implies:

Original: [Equation]
Revised: [Equation]

Original: By picking r = r
Revised: By picking r = r

Original: [Equation]
Revised: [Equation]

Original: we can further deduce that for any function u ∈ F (Ω)
Revised: we can further deduce that for any function u ∈ F (Ω)

Original: [Equation]
Revised: [Equation]

Original: the following inequality holds with probability 1 − 2 e − t :
Revised: the following inequality holds with probability 1 − 2 e − t :

Original: [Equation]
Revised: [Equation]

Original: Multiplying the denominator on both sides indicates:
Revised: Multiplying the denominator on both sides indicates:

Original: [Equation]
Revised: [Equation]

Original: Substituting the upper bound above into the decomposition ∆ E ( n ) ≤ ∆ E gen + 2∆ E app + 4 tn yields that with probability 1 − 4 e − min { 1 ,C q } t , we have:
Revised: Substituting the upper bound above into the decomposition ∆ E ( n ) ≤ ∆ E gen + 2∆ E app + 4 tn yields that with probability 1 − 4 e − min { 1 ,C q } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Simplifying the inequality above yields that with probability 1 − 4 e − min { 1 ,C q } t , we have:
Revised: Simplifying the inequality above yields that with probability 1 − 4 e − min { 1 ,C q } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Moreover, using strong convexity of the DRM objective function proved in Theorem B.1 implies:
Revised: Moreover, using strong convexity of the DRM objective function proved in Theorem B.1 implies:

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above yields that with probability 1 − 4 e − min { 1 ,C q } t , we have:
Revised: Combining the two bounds above yields that with probability 1 − 4 e − min { 1 ,C q } t , we have:

Original: [Equation]
Revised: [Equation]

Original: Truncated Fourier Series Estimator.
Revised: Truncated Fourier Series Estimator.

Original: Next we aim to show that the truncated Fourier series estimator can achieve the min-max optimal rate using the MDRM objective function.
Revised: Next we aim to show that the truncated Fourier series estimator can achieve the min-max optimal rate using the MDRM objective function.

Original: For any ξ ∈ Z + satisfying ξ 2 <
Revised: For any ξ ∈ Z + satisfying ξ 2 <

Original: Nn , there exists some Truncated Fourier Series in F ξ (Ω) with approximation error ∆ E app
Revised: Nn , there exists some Truncated Fourier Series in F ξ (Ω) with approximation error ∆ E app

Original: =
Revised: =

Original: O ( ξ − 2( s − 1) ) and generalization error ∆ E gen = O ( ξ d − 2 n ) .
Revised: O ( ξ − 2( s − 1) ) and generalization error ∆ E gen = O ( ξ d − 2 n ) .

Original: With optimal selection ξ = n 1 d +2 s − 4 to balance the bias and variance, we can achieve n − 2 s − 2 d
Revised: With optimal selection ξ = n 1 d +2 s − 4 to balance the bias and variance, we can achieve n − 2 s − 2 d

Original: +2 s − 4 convergence rate for the
Revised: +2 s − 4 convergence rate for the

Original: Fourier estimator.
Revised: Fourier estimator.

Original: Theorem C.2.
Revised: Theorem C.2.

Original: (Final Upper Bound of MDRM with Truncated Fourier Series Estimator) Under the assumptions in Theorem C.1, we consider the Modiﬁed Deep Ritz objective with the Truncated
Revised: (Final Upper Bound of MDRM with Truncated Fourier Series Estimator) Under the assumptions in Theorem C.1, we consider the Modiﬁed Deep Ritz objective with the Truncated

Original: Fourier Series function space F ξ (Ω) , where the parameter ξ =
Revised: Fourier Series function space F ξ (Ω) , where the parameter ξ =

Original: Θ( n 1 d +2 s − 2 ) .
Revised: Θ( n 1 d +2 s − 2 ) .

Original: By assuming ξ d N < ξ d − 2 n , we have that the Fourier estimator ˆ u FourierMDRM = min u ∈ F ξ (Ω) E MDRM N,n ( u ) satisﬁes the following upper bound with high probability:
Revised: By assuming ξ d N < ξ d − 2 n , we have that the Fourier estimator ˆ u FourierMDRM = min u ∈ F ξ (Ω) E MDRM N,n ( u ) satisﬁes the following upper bound with high probability:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Following the same proof as shown for the DRM upper bound in Theorem B.11, we ﬁrstly need to determine the critical radius r ∗ :
Revised: Following the same proof as shown for the DRM upper bound in Theorem B.11, we ﬁrstly need to determine the critical radius r ∗ :

Original: [Equation]
Revised: [Equation]

Original: For we have assumed ξ d N < ξ d [Equation] 2 n , the solution of the ﬁxed point equation is r ∗ (cid:39) ξ d − 2 n + 1 n + ξ − 2( s − 1) .
Revised: For we have assumed ξ d N < ξ d − 2 n , the solution of the ﬁxed point equation is r ∗ (cid:39) ξ d − 2 n + 1 n + ξ − 2( s − 1) .

Original: On the other hand, by taking α = s and β = 1 in Lemma B.19 and applying strong convexity of the
Revised: On the other hand, by taking α = s and β = 1 in Lemma B.19 and applying strong convexity of the

Original: DRM objective function proved in Theorem B.1, we can upper bound the approximation error ∆ E app as below:
Revised: DRM objective function proved in Theorem B.1, we can upper bound the approximation error ∆ E app as below:

Original: [Equation]
Revised: [Equation]

Original: Combining the two bounds above with Theorem C.1 yields that with high probability, we have:
Revised: Combining the two bounds above with Theorem C.1 yields that with high probability, we have:

Original: [Equation]
Revised: [Equation]

Original: By equating the two of the three terms above, we can solve for ξ that yields the desired bound:
Revised: By equating the two of the three terms above, we can solve for ξ that yields the desired bound:

Original: [Equation]
Revised: [Equation]

Original: Plugging in the expression of ξ gives the ﬁnal upper bound:
Revised: Plugging in the expression of ξ gives the ﬁnal upper bound:

Original: [Equation]
Revised: [Equation]

Original: D P ROOF OF THE L OWER B OUNDS D. P RELIMINARIES ON T OOLS FOR L OWER B OUNDS
Revised: D P ROOF OF THE L OWER B OUNDS D. P RELIMINARIES ON T OOLS FOR L OWER B OUNDS

Original: In this section, we list the standard tools we use to establish the lower bound.
Revised: In this section, we list the standard tools we use to establish the lower bound.

Original: The main tool we use is the Fano’s inequailty and the Varshamov-Gilber Lemma.
Revised: The main tool we use is the Fano’s inequailty and the Varshamov-Gilber Lemma.

Original: Lemma D.1 (Fano’s methods) .
Revised: Lemma D.1 (Fano’s methods) .

Original: Assume that V is a unifrom random variable over set V , then for any markov chain V → X → ˆ V , we always have:
Revised: Assume that V is a unifrom random variable over set V , then for any markov chain V → X → ˆ V , we always have:

Original: [Equation]
Revised: [Equation]

Original: Lemma D.2 (Varshamov-Gillbert Lemma,(Tsybakov, 2008) Theorem 2.9) .
Revised: Lemma D.2 (Varshamov-Gillbert Lemma,(Tsybakov, 2008) Theorem 2.9) .

Original: Let D ≥ 8 .
Revised: Let D ≥ 8 .

Original: There exists a subset V = { τ (0) , · · · , τ (2 D/ 8 ) } of D − dimensional hypercube H D
Revised: There exists a subset V = { τ (0) , · · · , τ (2 D/ 8 ) } of D − dimensional hypercube H D

Original: = { 0 , 1 } D such that τ (0) =
Revised: = { 0 , 1 } D such that τ (0) =

Original: (0 , 0 , · · · , 0) and the (cid:96) 1 distance between every two elements is larger than D 8 :
Revised: (0 , 0 , · · · , 0) and the (cid:96) 1 distance between every two elements is larger than D 8 :

Original: [Equation]
Revised: [Equation]

Original: , for all
Revised: , for all

Original: [Equation]
Revised: [Equation]

Original: D. P ROOF O F L OWER B OUND
Revised: D. P ROOF O F L OWER B OUND

Original: In this section, we provide the proof of the lower bound for learning a PDE.
Revised: In this section, we provide the proof of the lower bound for learning a PDE.

Original: Our proof uses standard
Revised: Our proof uses standard

Original: Fano method to establish minimax lower bound but ﬁnally leads to a non-standard convergence rate.
Revised: Fano method to establish minimax lower bound but ﬁnally leads to a non-standard convergence rate.

Original: We state standard results for Fano methods in Appendix D.1.
Revised: We state standard results for Fano methods in Appendix D.1.

Original: Following is the proof our main lower bound.
Revised: Following is the proof our main lower bound.

Original: Theorem D.1 (Lower bound) .
Revised: Theorem D.1 (Lower bound) .

Original: We denote u ∗ ( f ) to be the solution of the PDE 2.1 and we can access randomly sampled data { X i , Y i } i =1 , ··· ,n as described in Section 2.2.
Revised: We denote u ∗ ( f ) to be the solution of the PDE 2.1 and we can access randomly sampled data { X i , Y i } i =1 , ··· ,n as described in Section 2.2.

Original: DRM Lower Bound.
Revised: DRM Lower Bound.

Original: For all estimators ψ : (cid:0) R d (cid:1) ⊗ n × R ⊗ n → H s (Ω) , we have
Revised: For all estimators ψ : (cid:0) R d (cid:1) ⊗ n × R ⊗ n → H s (Ω) , we have

Original: [Equation]
Revised: [Equation]

Original: (D.1)
Revised: (D.1)

Original: PINN Lower Bound.
Revised: PINN Lower Bound.

Original: For all estimators ψ : (cid:0) R d (cid:1) ⊗ n × R ⊗ n → H s (Ω) , we have
Revised: For all estimators ψ : (cid:0) R d (cid:1) ⊗ n × R ⊗ n → H s (Ω) , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We construct the following bump function to construct the multiple hypothesis test used for proving the lower bound.
Revised: We construct the following bump function to construct the multiple hypothesis test used for proving the lower bound.

Original: Consider a simple C ∞ bump function supported on [0 , 1] d
Revised: Consider a simple C ∞ bump function supported on [0 , 1] d

Original: [Equation]
Revised: [Equation]

Original: where ξ : R → R is a non-zero funtion in C ∞ ( R ) with support contained in [0 , 1] and satisﬁes ξ ( x ) (cid:54) = 0 , ddx ξ ( x ) (cid:54) = 0 .
Revised: where ξ : R → R is a non-zero funtion in C ∞ ( R ) with support contained in [0 , 1] and satisﬁes ξ ( x ) (cid:54) = 0 , ddx ξ ( x ) (cid:54) = 0 .

Original: Then ∇ g ( x ) (cid:54) = 

Revised: Then ∇ g ( x ) (cid:54) =

Original: and the support of function g is [0 , 1] d .
Revised: and the support of function g is [0 , 1] d .

Original: Next, we take m =
Revised: Next, we take m =

Original: [ n 1 2 s − 4+ d ] and consider a regular gird x ( j ) , j ∈ [ m ] d .
Revised: [ n 1 2 s − 4+ d ] and consider a regular gird x ( j ) , j ∈ [ m ] d .

Original: According to the VarshamovGilbert lemma, there exist 2 m d / 8 (0 , 1) -sequences τ (1) , · · · , τ (2 md/ 8 ) ∈ { 0 , 1 } m d such that (cid:107) τ ( k ) − τ ( k (cid:48) )
Revised: According to the VarshamovGilbert lemma, there exist 2 m d / 8 (0 , 1) -sequences τ (1) , · · · , τ (2 md/ 8 ) ∈ { 0 , 1 } m d such that (cid:107) τ ( k ) − τ ( k (cid:48) )

Original: (cid:107) 2 ≥ m d 8 for all 0 < k (cid:54) =
Revised: (cid:107) 2 ≥ m d 8 for all 0 < k (cid:54) =

Original: k (cid:48) ≤ 2 m d / 8 .
Revised: k (cid:48) ≤ 2 m d / 8 .

Original: Then we construct the multiple hypothesis as
Revised: Then we construct the multiple hypothesis as

Original: [Equation]
Revised: [Equation]

Original: where ω is a constant to be determined later.
Revised: where ω is a constant to be determined later.

Original: It is easy to ﬁnd out that u k ∈ C s .
Revised: It is easy to ﬁnd out that u k ∈ C s .

Original: Then we reduce solving the PDE to a multiple hypothesis testing problem, which considers all mappings from n sampled data to the constructed hypothesis Ψ : (cid:0) R d (cid:1) ⊗ n × R ⊗ n → V :=
Revised: Then we reduce solving the PDE to a multiple hypothesis testing problem, which considers all mappings from n sampled data to the constructed hypothesis Ψ : (cid:0) R d (cid:1) ⊗ n × R ⊗ n → V :=

Original: { u i | i = 1 , 2 , · · · , 2 m d / 8 } .
Revised: { u i | i = 1 , 2 , · · · , 2 m d / 8 } .

Original: Then we apply the local Fano method and check that we can obtain a constant lower bound of P ( ˆ V (cid:54)
Revised: Then we apply the local Fano method and check that we can obtain a constant lower bound of P ( ˆ V (cid:54)

Original: =
Revised: =

Original: V ) for any estimator
Revised: V ) for any estimator

Original: ˆ V .
Revised: ˆ V .

Original: From the local Fano method, we know that
Revised: From the local Fano method, we know that

Original: [Equation]
Revised: [Equation]

Original: where P k denotes the joint distribution of the sampled data ( x, y ) .
Revised: where P k denotes the joint distribution of the sampled data ( x, y ) .

Original: In speciﬁc, x follows a uniform distribution on [0 , 1] d and y =
Revised: In speciﬁc, x follows a uniform distribution on [0 , 1] d and y =

Original: f ( X )+ (cid:15) , where (cid:15) is independently sampled from a standard Gaussian distribution N (0 , 1) .
Revised: f ( X )+ (cid:15) , where (cid:15) is independently sampled from a standard Gaussian distribution N (0 , 1) .

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: Using Fano inequality, if we select m ∝ [ n [Equation] s
Revised: Using Fano inequality, if we select m ∝ [ n 1 2 s

Original: [Equation]
Revised: [Equation]

Original: 4+ d ] then we have the following lower bound when ω is taken to be sufﬁciently large:
Revised: 4+ d ] then we have the following lower bound when ω is taken to be sufﬁciently large:

Original: [Equation]
Revised: [Equation]

Original: At the same time, we can estimate the separation of the hypotheses in two different norms:
Revised: At the same time, we can estimate the separation of the hypotheses in two different norms:

Original: • Deep Ritz Method:
Revised: • Deep Ritz Method:

Original: [Equation]
Revised: [Equation]

Original: • Physic Informed Neural Network:
Revised: • Physic Informed Neural Network:

Original: [Equation]
Revised: [Equation]

Original: Plugging in m ∝ [ n 1 2 s
Revised: Plugging in m ∝ [ n [Equation] s

Original: [Equation]
Revised: [Equation]

Original: 4+ d ] , we know that with constant probability we have
Revised: 4+ d ] , we know that with constant probability we have

Original: [Equation]
Revised: [Equation]

Original: E I NTUITION B EHIND THE S UB - OPTIMALITY OF THE U NMODIFIED D EEP R ITZ M ETHODS
Revised: E I NTUITION B EHIND THE S UB - OPTIMALITY OF THE U NMODIFIED D EEP R ITZ M ETHODS

Original: In this section, we aim to discuss the intuition behind the sub-optimality of the unmodiﬁed DRM via using the truncation Fourier basis.
Revised: In this section, we aim to discuss the intuition behind the sub-optimality of the unmodiﬁed DRM via using the truncation Fourier basis.

Original: To simplify the notation, in this section we consider the following simplest Poisson equation ∆ u
Revised: To simplify the notation, in this section we consider the following simplest Poisson equation ∆ u

Original: = f on the hypercube with zero Dirichlet boundary condition.
Revised: = f on the hypercube with zero Dirichlet boundary condition.

Original: To illustrate the necessity of the modiﬁcation we made, we consider the difference between the following two estimators
Revised: To illustrate the necessity of the modiﬁcation we made, we consider the difference between the following two estimators

Original: • Estimator 1.
Revised: • Estimator 1.

Original: We use the truncated Fourier basis estimator to learn the right hand side function f
Revised: We use the truncated Fourier basis estimator to learn the right hand side function f

Original: and then we invert the PDE exactly to get the estimated u . • Estimator 2.
Revised: and then we invert the PDE exactly to get the estimated u . • Estimator 2.

Original: We plug in a parametrization of the truncated fourier basis into the empirical
Revised: We plug in a parametrization of the truncated fourier basis into the empirical

Original: DRM objective
Revised: DRM objective

Original: We would like to point out that estimator 1 isn’t build for computational consideration .
Revised: We would like to point out that estimator 1 isn’t build for computational consideration .

Original: Instead, we use it to consider the statistical limit of our sampled data.
Revised: Instead, we use it to consider the statistical limit of our sampled data.

Original: We ﬁrst show that the estimator 1 can achieve the minimax optimal estimation error.
Revised: We ﬁrst show that the estimator 1 can achieve the minimax optimal estimation error.

Original: Error Of Estimator 1 Firstly, we show that if one wants to learn the function u in H 10 norm, one need to learn the right hand side function f in H − 1 0 norm.
Revised: Error Of Estimator 1 Firstly, we show that if one wants to learn the function u in H 10 norm, one need to learn the right hand side function f in H − 1 0 norm.

Original: The H − 1 0 norm is deﬁned as the dual norm of the H 1 norm, i.e. (cid:107) u(cid:107)
Revised: The H − 1 0 norm is deﬁned as the dual norm of the H 1 norm, i.e. (cid:107) u

Original: [Equation]
Revised: [Equation]

Original: =max [Equation] [Equation]
Revised: =

Original: [Equation]
Revised: [Equation]

Original: u, v (cid:105) .
Revised: u, v (cid:105) .

Original: Once we assume we have an estimate ˆ f of f in
Revised: Once we assume we have an estimate ˆ f of f in

Original: H − 1 0 , we can have an estimate of u via ˆ u := (∆) − 1 ˆ f , whose distance to u in the H 1 norm satisﬁes:
Revised: H − 1 0 , we can have an estimate of u via ˆ u := (∆) − 1 ˆ f , whose distance to u in the H 1 norm satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: Estimator 1 using the truncated Fourier estimator to estimate the right hand side function f .
Revised: Estimator 1 using the truncated Fourier estimator to estimate the right hand side function f .

Original: Suppose we can access a random sample of observed data as { x i , f ( x i ) } ni =1 , then the Fourier coefﬁcient f
Revised: Suppose we can access a random sample of observed data as { x i , f ( x i ) } ni =1 , then the Fourier coefﬁcient f

Original: z := (cid:104) u, φ z (cid:105) can be estimated as ˆ f z := 1 n (cid:80) ni =1 f ( x i )
Revised: z := (cid:104) u, φ z (cid:105) can be estimated as ˆ f z := 1 n (cid:80) ni =1 f ( x i )

Original: φ z ( x i ) .
Revised: φ z ( x i ) .

Original: To bound the estimation error of ˆ f := (cid:80) (cid:107) z (cid:107) ∞ ≤
Revised: To bound the estimation error of ˆ f := (cid:80) (cid:107) z (cid:107) ∞ ≤

Original: Z ˆ f z φ z in H − 1 0 , we ﬁrst apply the bias-variance decomposition:
Revised: Z ˆ f z φ z in H − 1 0 , we ﬁrst apply the bias-variance decomposition:

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst bound the bias term (cid:107) E ˆ f − f (cid:107) 2 H − 1 .
Revised: We ﬁrst bound the bias term (cid:107) E ˆ f − f (cid:107) 2 H − 1 .

Original: Given E ˆ f = (cid:80) (cid:107) z (cid:107) ∞ ≤ Z f z φ z , we have that for a truncation set Z of the from Z := { z ∈ N d |(cid:107) z (cid:107) ∞ ≤ Z } , the bias term can be controlled by:
Revised: Given E ˆ f = (cid:80) (cid:107) z (cid:107) ∞ ≤ Z f z φ z , we have that for a truncation set Z of the from Z := { z ∈ N d |(cid:107) z (cid:107) ∞ ≤ Z } , the bias term can be controlled by:

Original: [Equation]
Revised: [Equation]

Original: Next we estimate the variance of the estimator by decomposing the variance into the following sum:
Revised: Next we estimate the variance of the estimator by decomposing the variance into the following sum:

Original: [Equation]
Revised: [Equation]

Original: Finally we achieve a Z − 2( s − 1) + Z d − 2 n upper bound for estimator 1.
Revised: Finally we achieve a Z − 2( s − 1) + Z d − 2 n upper bound for estimator 1.

Original: With optimal selection of Z , we can achieve the min-max optimal convergence rate n − 2 s − 2 d
Revised: With optimal selection of Z , we can achieve the min-max optimal convergence rate n − 2 s − 2 d

Original: +2 s − 4 .
Revised: +2 s − 4 .

Original: Difference Between Estimator 1 and Estimator 2 Next we aim to understand the Deep Ritz
Revised: Difference Between Estimator 1 and Estimator 2 Next we aim to understand the Deep Ritz

Original: Method objective function via plugging in a truncated Fourier series estimator.
Revised: Method objective function via plugging in a truncated Fourier series estimator.

Original: We consider an estimator of the form u = (cid:80) ˆ u z φ z ( x ) , which lies in the space of truncated Fourier series.
Revised: We consider an estimator of the form u = (cid:80) ˆ u z φ z ( x ) , which lies in the space of truncated Fourier series.

Original: Then the empirical DRM objective function can be expressed as
Revised: Then the empirical DRM objective function can be expressed as

Original: [Equation]
Revised: [Equation]

Original: We observe that (E.1) is a quadratic formula with respect to the Fourier coefﬁcients u := ( u z ) (cid:107) z
Revised: We observe that (E.1) is a quadratic formula with respect to the Fourier coefﬁcients u := ( u z ) (cid:107) z

Original: [Equation]
Revised: [Equation]

Original: ∞ ≤ Z .
Revised: ∞ ≤ Z .

Original: Thus, we can rewrite it as the following matrix form
Revised: Thus, we can rewrite it as the following matrix form

Original: [Equation]
Revised: [Equation]

Original: , where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Based on the matrix formulation E.2, we can compare the solution given by the two estimators
Revised: Based on the matrix formulation E.2, we can compare the solution given by the two estimators

Original: • Estimator 1: The Fourier coefﬁcients of the solution of Estimator 1 are
Revised: • Estimator 1: The Fourier coefﬁcients of the solution of Estimator 1 are

Original: [Equation]
Revised: [Equation]

Original: • Estimator 2: The Fourier coefﬁcients of the solution of Estimator 2 are
Revised: • Estimator 2: The Fourier coefﬁcients of the solution of Estimator 2 are

Original: [Equation]
Revised: [Equation]

Original: Note that E ˆ A = (cid:0) (cid:107) z (cid:107) 2 (cid:1) (cid:107) z (cid:107) ∞ ≤ Z .
Revised: Note that E ˆ A = (cid:0) (cid:107) z (cid:107) 2 (cid:1) (cid:107) z (cid:107) ∞ ≤ Z .

Original: Thus, we can further introduce another variance from the sampling of A .
Revised: Thus, we can further introduce another variance from the sampling of A .

Original: By directly estimating ˆ u 1 − ˆ u 2 , we will show that this term will be larger than the ﬁnal convergence rate.
Revised: By directly estimating ˆ u 1 − ˆ u 2 , we will show that this term will be larger than the ﬁnal convergence rate.

Original: Notice that
Revised: Notice that

Original: [Equation]
Revised: [Equation]

Original: Next we aim to bound (cid:16) ( E ˆ A ) − 1 − ˆ A − 1 (cid:17) .
Revised: Next we aim to bound (cid:16) ( E ˆ A ) − 1 − ˆ A − 1 (cid:17) .

Original: We ﬁrst use the Matrix Bernstein Inequality(Tropp, 2015) to bound the H 1 distance between ˆ u 1 and ˆ u 2 .
Revised: We ﬁrst use the Matrix Bernstein Inequality(Tropp, 2015) to bound the H 1 distance between ˆ u 1 and ˆ u 2 .

Original: According to the Matrix Bernstein Inequality, we have that with probability 1 − e − t , the following inequality holds
Revised: According to the Matrix Bernstein Inequality, we have that with probability 1 − e − t , the following inequality holds

Original: [Equation]
Revised: [Equation]

Original: + t n, (E.6) where (cid:107) · (cid:107) H is the matrix operator norm respect to the vector (cid:107) · (cid:107) H deﬁned as (cid:107) z (cid:107) 2 H =
Revised: t n, (E.6) where (cid:107) · (cid:107) H is the matrix operator norm respect to the vector (cid:107) · (cid:107) H deﬁned as (cid:107) z (cid:107) 2 H =

Original: z (cid:62) diag (cid:0) (cid:107) z (cid:107) 2 (cid:1) − 1 (cid:107) z (cid:107) ∞ ≤ Z z .
Revised: z (cid:62) diag (cid:0) (cid:107) z (cid:107) 2 (cid:1) − 1 (cid:107) z (cid:107) ∞ ≤ Z z .

Original: Note that
Revised: Note that

Original: [Equation]
Revised: [Equation]

Original: When n is large enough, we know that 12 I (cid:54) I
Revised: When n is large enough, we know that 12 I (cid:54) I

Original: + ( E ˆ A ) − 1 (cid:16) ˆ A − ( E ˆ A ) (cid:17) (cid:54)
Revised: + ( E ˆ A ) − 1 (cid:16) ˆ A − ( E ˆ A ) (cid:17) (cid:54)

Original: I with high probability.
Revised: I with high probability.

Original: Thus the term (cid:107) ˆ u 1 − ˆ u 2 (cid:107) 2 H 1 is at the scale of (cid:13)(cid:13)(cid:13)(cid:16)
Revised: Thus the term (cid:107) ˆ u 1 − ˆ u 2 (cid:107) 2 H 1 is at the scale of (cid:13)(cid:13)(cid:13)(cid:16)

Original: ( E ˆ A ) − ˆ A (cid:17)(cid:13)(cid:13)(cid:13) 2 H ≈ Z d n , which is of the same magnitude as what we get from the empirical process approach in our main proof.
Revised: ( E ˆ A ) − ˆ A (cid:17)(cid:13)(cid:13)(cid:13) 2 H ≈ Z d n , which is of the same magnitude as what we get from the empirical process approach in our main proof.

Original: It is also larger than Z d − 2 n , which is the magnitude of the variance term for ˆ u 1 .
Revised: It is also larger than Z d − 2 n , which is the magnitude of the variance term for ˆ u 1 .

Original: Therefore, here we conjecture that the our bound for DRM itself is tight and leads to the sub-optimal convergence rate.
Revised: Therefore, here we conjecture that the our bound for DRM itself is tight and leads to the sub-optimal convergence rate.