Breaking the Curse of Dimensionality for Parametric Elliptic PDEs
Authors: Marius Zeinhofer
Alex Kaltenbach
Venue: Submitted to ICLR 2023 (2023)

Abstract: Motivated by recent empirical success, we examine how neural network-based ansatz classes can break the curse of dimensionality for high-dimensional, non-linear elliptic partial differential equations (PDEs) with variational structure. The high-dimensionality of the PDEs can either be induced through a high-dimensional physical domain or a high-dimensional parameter space. The latter include parametric right-hand sides, parametric domains, and material constants. Our main result shows that any scheme, that computes neural network based $W^{1,p}$-approximations, leverages the extraordinary approximation capabilities of neural networks and, thus, is able to beat the curse of dimensionality if the ground truth solution is smooth or possesses Barron regularity. Popular examples of $W^{1,p}$-convergent schemes include, e.g., the Deep Ritz Method and physics-informed neural networks. We present numerical experiments supporting our theoretical findings.

Under review as a conference paper at ICLR 2023
BREAKING THE CURSE OF DIMENSIONALITY FOR PARAMETRIC ELLIPTIC PDEs
Anonymous authors Paper under double-blind review
ABSTRACT
Motivated by recent empirical success, we examine how neural network-basse ansatz classes can break the curse of dimensionality for high-ding nigh-dimensional, non- linear elliptic partial differential equations (PDEs) with variational structure. 'hee high-dimensionality of the PDEs can either be induced through a high-dimensional physical domain high high-dimensional parameter space. The latter include paramet- ric right- right-hand sides, parametric domains, and material constants, Our main result shows that any scheme that computes neural network based W aaprpoxpixxiiiiinnns leverages the extraordinary approximation capabilities of neural networks and, thus, is able to beat the curse of dimensionality if the ground truth solution is smooth or possesses Barron regularity. Popular examples of include, e-g., the Deep Ritz Method and physics-informed neural networks. We present numerical experiments supporting our theoretical findings.
schemes
INTRODUCTION
High-dimensional partial differential equations (PDEs) arise naturally in applications with either high-dimensiona. domain, high-dimensional parameter space, possibly with both. The former includes the Schrödinger equation in quantum physics, the Black-Scholes equation in finance, and the Mamiiton-lacoii-elllne
equation in control theory, we refer to WWinan al. (202T): Bellman (1954). On the other hand, examples of problems with high- -igh-dimensional parameter space are ubiquitous in engineering applications, for instance, in varying material properties, right-hand sides or even in the form of varying computational domains, as discussed in Hennigh et al. (202)
Ohlberger Rave (2016).
For problems with high-dimensions physical domain, classical mesh- based approximation schemes face the curse of dimensionalityy meaning that the computational cost increases exponentially with the dimension of the problem. In the case of parametric problems, one is typically interested in querying the PDE solution for many different parameter instances, possibly with low inference time. To this end, classical methods need repeatedly solve the equations for every required parameter instance, a potentially prohibitively expensive or slow computational task, see Biegler et al. (2007). Even assuming additional, favorable structure of the solution of high-dimensional PID may it be a latent dimensionnllty of the solution or a high degree of smoothness it remains a challenge for classical methods to approximate the solution with an acceptable accuracy, especially in situations of non-linear solution manifolds as discussed in Ohlberger & Rave (0166; Lee & Carlberg 2020)).
Artificial neural networks have shown great potential in the approximation of high-dimensional functions, among those computer vision, classification and natural language processing tasks and are known to possess extraordinary approximation capabilities with the possibility to achieve dimension- independent approximation rates for certain function classes, see Ma et al. (2022); Barron (1993); Yarotsky (017): : Gühring Raslan (202T): Gühring et al. (2020). Therefore, investigating artificial neural networks as ansatz classes for the solution of PDES or PDE solution operators has recently
gained increased interest for high-dimensional and parametric problems. We refer to Kutyniok et al.)
(2022); Weinan & Wojtowytsch (0022) Jentzen et al. (2021) ); Chen et al. (2021) for theoretical studies. Successful empirical results of neural betwork-based applications to PDEs posed in high bigh-diimnssooal
include|Hermann spaces include Hermann et al. (2020); ; Yu & El (2018) ; Han et al. (2018): Sirignano & Spiliopoulos
(2018). For the parametric setting, we direct the reader to Li et al. 2021 Khoo et al. (2021): Lee
Carlberg 2020); Geist al. (202T).

---

Under review as a conference paper at ICLR 2023
Of the aforementioned contributions, the theoretical works either focus approximation theo- retic results or consider linear problems without parametric dependencies. We discuss the relation to contribution in detail in Section [.1 The approximation theoretic results guarantee the existence of a neural network with desirable approximation rates but provide no practical way to compute the neural network. For non-linear, parametric PDEs with p-structure, our results deliver the necessary PDE analysis in a setting suitable for neural network ansatz functions - to alleviate this problem. Instead of explicitly constructing neural network, we show that suffices to find neural network approximation that is close "in energy" to the ground truth solution. This makes a significant dif- ference: Energy approximations can found by using the variational energy loss function or more generally by any W -covvergeen approximation scheme a natural property of a reasonable approximation algorithm.
To summarize, our main contributions are the following:
We show that every energy convergent approximation scheme for the p-Dirichlet energy utilizing neural network ansatz functions can leverage the extraordinary approximation capabilities of neural networks. Further, explain which assumptions on the ground truth solution allow neural networks to beat the curse of dimensionality. We extend these results to parametric problems, where the neural network approximates simultaneously in the physical and the parameter space. Our contributions are the first quantitative error estimates for parametric and non-linear elliptic PDEs for neural network approximation schemes.
From a mathematical point of view, the analysis of non-linear ansatz classes is novel in the case of the p-Laplacian. Existing literature exclusively exploits strategies based on optimality conditions (Galerkin orthogonality) only available for linear ansatz classes, hence, excludes neural networks. Further, to the best of the author's knowledge, error estimates for parametric problems have not been considered in the existing literature.
1.1 MAIN RESULT AND RELATED WORK
For clarity, we present our main result for the case of parametric right-hand side and with homogeneous Neumann boundary conditions. However, different boundary conditions and parametric dependencies are covered by our analysis. We explain this in the Appendix and refer to Section
Consider physical domain Ω C Rdu, dg N, and parameter space P S RRde dp N. Further, let E (1, oo) be fixed and denote by (f(T,-)) TTE a parametric family of right-hand sides. We study the non-linear p-Laplace problem as a prototypical, non-linear elliptic PDE. More precisely, we want to find u² P x Ω + E satisfying
- div (V u (T. | P u² (T, x)) = f(r,æ) for a.e. (T,x) EPX
(1)
subjected for simplicity homogeneous Neumann boundary conditions. The case p = 2 retrieves the classical Poisson equation. In this example, the parametric dependencies are induced through the right-hand side and both 12 and P may be high-dimensional. Then, we seek a neural network us with input (T, TT E Rd x Rd that approximates the solution u" P x Ω E simultaneously in the physical domain Ω and the parameter space P. Essential for the statement of our result is the reformulation of equation equation I as minimization problem. We find u" E TPP(P WL.P(D)) as a minimizer of E (P, W WL.P(2)) + TR, for every v E LP,(P, W..P(()) defined by
E(v) = 1 - / V ())
d f(T, u(T f(T,z))((Tx)) do dr.
(2)
Then, our main result is the following.
Theorem 1. Let Ω C Bdd, dg N, be bounded Lipschitz domain and P C Rd dp E N, an open Moreover, let f TP (P Q), p (1), be such that Ja f(T dr 0 for P. Denote by u² E [P WLP(( weak solution of the parametric p-Laplace problem with homogeneous Neumann boundary conditions, i.e.,
- -div((V 2
v=)) = f
in P x Ω P a..
(3)
O,
2

---

Under review as a conference paper at ICLR 2023
Let MM C WP.PP(P x (2) be any subset that contains the zero fumction and let E M be arbitrary.
Setting
M = {uE E M I / S 2 =u/
it holds
8(6) BE V V V V (Pxa)
ifp E (2,
VV - Vnu 1 2
6(v) + inf V v (P)))
if E (1,2)
where := E(v) se (()) is the optimization error and the implicit constants depend on p. and [ only.
We stress again that the choice of Neumann boundary conditions is for simplicity of presentation. similar result holds for Dirichlet boundary conditions employing an appropriate penalty scheme. mean-value-free serves to guarantee the well-posedness in the Neumann boundary value case. The main reason to pass from M to M is solely of technical nature, rooted in Lemma 7 and not of relevance for the interpretation of the result.
The fact that -T) for a.c. T
Leveraging the Power of peproxxinationnThheore approximation theorems allow to estimate the infimum in Theorem I deduce error decay rates. Note that there is no further requirement on the approximating function E M than that it is "good" quasi-optimizer of M -> R. i.e., that 5(v) is sufficiently small. Thus, any algorithm that produces approximate solutions that converge in energy is able to fully leverage the approximation capabilities of neural network ansatz classes - up to the exponent 2/p or p/2, which is due to the non-linearity. Furthermore, energy convergence is equivalent to convergence in the W--P-semi-non drastic difference of our contribution to mere approximation theoretical results that only guarantee the existence of well-approximmatig network, yet don't 't unveil how such an approximation should be found. In this sense, our contribution is orthogonal to approximation theoretical results as it can be combined with these to extend them.
Results. As we discuss below, suitable
We stress the
Finally note that, analyzing the effect of a solution scheme on the achievable value of 5(v) is a difficult problem, typically connected to non-convex optimization task, that do not study in this article.
Using Smoothness to Beat the Curse of Dimensionality. We can utilize quantitative universal approximation results to estimate the infimum in Theorem In some situations, this allows to beat the curse of dimensionality. Assume that the solution u to equation 3 is a member of W x 2 for some k E N, & 1. Then, for every n E N. we may use Theorem 4.9 Gühhrin & Raslan (2021 l) to guarantee the existence of a fully connected neural network architecture with ReLU -cctivatto with parameter space O, of dimension (n)) such that, setting M Fe.. where Fe,. denotes the realization set of the ansatz class, it holds
Sa+ap
inf 2 - V u [|) 2 | (P
for the case 2 2 and with 2 replaced by 2 in the case 2. Hence, for arbitrary E M, 0 E On. we get
p
a V.u V 3 6n 1/1 ("("))"**+|["/)>>>>>>""""""""""" (2) d'
dotap
if E (2)
(uo) 1/2 |Iu'l (1) 8 क
if E (1,2)
where 5, (up)) E(ug) inf wee., E(u)). This shows given sufficient smoothness of u² E M that the eтror of the neural network approximation does not decay exponentially slow in the dimension E N of the parameter space θ,, More precisely, the result requires dimension-dependor. smoothness with the smoothness parameter E N, k > 1, scaling like le ~ da + dp. However, the assumption of smoothness is very natural in the context of (linear) elliptic PDEs and holds also in the parametric case, see Lemma B which gives an easily verifiable criterion when the smoothness assumption holds.
'The subsets we have in mind consist of neural network functions of a given architecture and, thus, M neural network. But any choice of M is admissible.
"To see this, we refer to equation 7 in Proposition 2
TThis result holds also for other activation functions, we refer the original work.
3

---

Under review as a conference paper at ICLR 2023
Employing Barron Regularity to Beat the Curse of Dimensionality. A different situation where the curse of dimensionality can be circunvented is when u* E M is a member of the Barron space B, or can be "well--ppe apprrximaox by Barron functions. We refer to Barron (993); Ma al. (2022) for a definition of the Barron space. In essence, members of B can be approximated with respect to the -orm by shallow neural networks with cuimension-independee the width of the shallow network. Hence, setting = 2 and assuming u* E B, we can estimate for an arbitrary shallow neural network ug E M
rate of n n-1/2, where E N is
- 71
U. V T. [IP(Pxa 83
|/2e
||1s
The assumption u" E B is too restrictive in general, cf. the discussion in Weinan & Wojtowytschi (2022). However, assuming that the data f E (P 22) (and possibly coefficients) are of Barron regularity, it was recently established that the solution u² E M can be approximated by Barron functions with Barron norm growing only polynomially in the dimension, yielding the rate 1/2 for shallow networks of width (dn) C log(n), where C is a constant, we refer to Chen et al. (021). Note that the result in Chen al. (2021) far does only hold for linear elliptic PDEs and special activation functions, and does not include parametric dependencies. Our result, then, shows that this error decay rate is preserved for any energy-convergent approximation scheme and is, in fact, not mere approximation result.
Related Work For PDEs that admit stochastic representation, several situation are known in which the curse of dimensionality can be circumvented, Jentzen et al. (202T); Hann al. (2018); Weinan et al. (202T). These results are of approximation theoretic nature and do not provide way to construct the approximating network. Further, this approach crucially relies on the stochastic representation of the PDE's solution and, thus, is not generally applicable.
The works Xu 2020); Jiao et al.) (202T): | Duan et al. (202T): Müller & Zeinhofer (2022) are similar to contribution since they consider elliptic equations and provide Céa type Lemma and, consequently, are not only approximation theoretic results. However, they neither analyze non-linear
nor parametric equations.
The contributions in Chen et al. (2021): Weinan & Wojtowytsch (2022) mark the beginning of a regularity theory for elliptic equations with respect to Barron spaces. These results are complementary to our analysis in the sense that they can be combined with our contribution. For instance, the main result Chen al. (202T) states that solution linear elliptic PDE with Barron data is "almost" of Barron regularity and can be approximated with polynomial rate with respect to the dimension. Our analysis, then, guarantees that every neural network approximation that is close in energy the ground truth solution realizes this rate.
Notation For a Banach space X with norm || ||x X + X-R2o R20. we denote by X", its (topological) dual space equipped with the norm || 1lxx X + Rzo. defined by ||='||x- == supj|ali (x" x)x for all x= E X". Here, (,,)xx X x X - IR denotes the duality pairing, defined by (x", x)x = x (x) for all xx E X* and x E X.
Forp E oo). we denote by IP())), the space of (Lebesge-)men mmeasurabm functions u: Ω + R that are integrable in p-th power, i.e., Sa |u|p dr < oo if E [1, oo) and ess sup [x)] < 00 if = co. Endowed with the norm [|u]|=_(0]= == (So |u|p dz)r r if p [1, oo) and ||u]] (0])z=(()) == ess sup [z))
if p= the space (2() forms Banach space, which is separable if E [1, oo) and reflexive if p E (1, oo), cf. (Adams & Fournier 2003 Chapter 2).
For l E N and p E [1, oo), we denote by W*)((S), the space of functions in IP((S) with distributional derivatives up to ki-th order in (2)) Endowed with the norm |и|| ((llwwra))) == En D'ullLLra)) the
space &. (S2) forms Banach space, which is separable if E [L. oo) and reflexive if E (1, oo). cf. (Adams & Fournier 2003 Chapter 3). For A E N and p E [1, co], we denote by W (S2), the
closure of all compactly supported, smooth functions C (1) in W..P((2).
We always denote parameter-dependde functions by boldface letters, e.g., u. U. W.... and parameter- independent functions by bol-boldface letters, e.g., u, u, w. ... In the same spirit, denote by : LP; (..)( - IR, the parametric p-Dirichlet energy equation 2 and by E WI-P(S2) + R, the non-parametric p-Dirichlet energy (c.f. equation b).
...
4

---

Under review as a conference paper at ICLR 2023
2 PROOF OF THE MAIN RESULT
In this section, we provide the proof of Theorem For clarity, we first consider the non-para- metric case and extend the results afterwards to include parametric dependencies.
2.1 PROOF OF THE NON-PAR PARAMETRIC SETTING
The main step in the proof the non non-parametric version of Theorem Lis to show that conver- gence in energy, i.e., (l Fl(")) (n. oo) (cf. equation 6) for neural network approximation My E .M of the ground truth solution u² E M,, is equivalent to the convergence t4, - u" (n - oo) in the Sobolev topology. To establish this in a quantitative fashion, we need an optimal measure the convexity of the p-Dirichlet energy (cf. equation 6). This is given through the bi-variate, symmetric mapping p Wl))( x W1-P(()) + R. defined by
(r w) = [F(())) - F(Vw) 1222 for all o, w WL-P(2),
(4)
where F; F. d is defined by
F(a) = |a| 2 for a E Rd.
The map pi: (L. (L) I is the optimal distance measure for the p-Dirichlet problem. This is embodied in the two-sided estimate proved in the next proposition, see equation 7 that relates convergence energy to convergence terms of P W1 W (S2)
In the literature. 03 (L)P(()) W ( IR is usually referred to as the Natural Distance, cf. Diening & Rūzīčka (2007): Diening et al. (007): Diening Ettwein (2088 Kaltenbach Rúzička (2022).
Next, we establish Céa type Lemma in terms of p W (S2) x ww.. (92) + R. see Lemma This decomposes the distance of u, u" into an optimization error and an approximation theoretic contribution. Finally, we study the relation of p W- x W (S?) - I to the standard Sobolev topology in Lemma 7
From technical perspective, the central estimate in equation is proved via a Taylor expansion of the p-Dirichlet energy around its minimizer u' E W/--P(S)). However, care needs to be taken since E W.. + R is not twice continuously differentiable and subtle regularization procedure needs to be employed to rigorously carry out the expansion.
Proposition 2. Let Ω C Rd d N, be a bounded domain, f WLP((S))', (1, (L), and let U C (L)) be a closed subspace such that Poincaré's 's inequality is valid, ie., there exists a constant Cp O such that for every U U, it holds
[|]| S Cc [oll
Moreover, define E: + for every U by
E(u) = J /ul dx (f. u)
(6)
Then, the following statements apply:
(i) There exists unique minimizer ut for E U + R.
(ii) There exists constant c(p) > 0, depending only on E o0) and not depending on d E N, such that for every v U, holds
c(p) -1 (V u²) S E(v) E(w") E(u") < c(p) (F, Moreover, we can choose c(p) > O such that -p e(p)) E (1,
(7)
Proof. The proof is provided in the Appendix, see A
Remark 3 (The Case p 2). In the case p 2, we retrieve the well-known Dirichlet energy. Further, cquality holds in equation with constant c(p) = 1/1 More precisely, for every v E U, we have that
1 2
((e) - Blu')
Vt - Vu 122 = p (v."").
This can shown by a straight-forward Taylor expansion E U -+ around u² E U, cf. Müller & Zeinhofer (2022)
5

---

Under review as a conference paper at ICLR 2023
Remark 4 (The Role of the Space U). The space U encodes boundary conditions, for example, = =W) is an admissible choice. However, when choosing U (l' and requiring that the right-hand side f E WS2""SS)" vanishes constant functions, Proposition 2 stays valid with the exemption of the uniqueness of the minimizer u² E U. In this case, u" с U is only determined up additive constants. This be seen by considering the energy E. on the quotient space W.I.((2) modulo the constant functions. On this space, a Poincaré inequality is available.
An immediate consequence of Theorem is is the following Céa type lemma.
Lemma (Céa Lemma). Let the assumptions of Proposition 2 be satisfied. Moreover, let M S U be an arbitrary subset. Then, there exists constant c(p) > 0, depending only on p E (1, oo) and not depending on N, such that for every v E M, holds
( u²) c(p) (ō(v) inf (0(
8EM
where =(v)= E(v) - infrem E(ū) is the optimization error. Moreover, we can choose c(p) > 0 such that (p+> c(p)) (1,
Proof of Lemmna
Let v M be arbitrary. Then, by referring to Theorem we find that
c(p) u²) - E(v) inf E(7) + inf E - E(w") S 5(v) + c(p) inf o (ū, u). REM
REM
6EM
Remark Note that we do not need impose any structure on the M, in particular, it does not need to possess linear structure. This, in contrast to classical formulations of Céa's Lemma, allows us choose M as an ansatz class consisting of neural networks.
In order arrive error decay rates in Sobolev topology, need the relation of p to the Wli (((())ssmmiimn) semii-orm.. norm.
Lemma 7 (Relation Between Natural Distance and (e-P-Sem--Nomm) Let Ω S Rd, E N, be bounded domain and p (1, oo). Then, there exists constant c(p) > 0, depending only p E (1, 20) and not depending on E N, such that the following relations apply:
(i) p [2, 20), then for every u, v E Wl-r(S), holds
dpp) IVu Vv Feli S (u,
S <((p) (IV)) + [[Vollerm)))))** Vell
(ii) If (1, then for every v, w E W WLPP((), holds
c(p) (u, v) & Vu- - Vulli
$ c(p) ( (| + [[Vlllz(()))) m2--))
Moreover, we can choose clp) > 0 such that (p c(p)) E (1,,
Proof. The proof is provided in the Appendix, see A
We are now able to prove the main result setting excluding parametric dependencies. Theorem 8. Let f E WLP(O)', p oo), be such that (J, w) for all R. Moreover let u E wl) be a weak solution of the p-Laplace problem with homogeneous Neumann boundary conditions, i.e., u' E W.-P(s)) minimizes E W-)P((Q) + R. for every Wl)P(S) defined by
[Vol" dx () E(v) -
(8)
Jq
Let M C WlP((() be any subset that contains the zero function and let u E M be an arbitrary. Setting
M {u E M [|Vu|l 22[VV""l

---

Under review as a conference paper at ICLR 2023
it holds
8(u) U/ inf Vē
Vu Vu"
ip E (2,
IV Vu" (o))j 2
8(v)1/2 inf V
ify (1,2)
Le
where 8(0) == E(v) inf BEM (() is the optimization error and the implicit constant depends on p.
Q and |l only.
Proof. ad p E o0). If p [2. oo), then we estimate, using the relation of the natural distances to Sobolev semi-normss as described in Lemma Céa's Lemma [] and the coercivity estimate in Lemma 13 obtain
VV - Vu Vu"ll Pepg
S c(p) (v,
S c() ((v) ((v)+inf inf (i.
S c(p)
6(v)+
inf BENT
+
1V- Vu' [2pp())))
S c(p)
p-2 Lr
V Vu- V (())( inf BE
< c(p) + -3 cpp Ω2)
inf V Vu" Vu'll V
BEM
ad (1, p (1, 2]. then, again, using the relation of the natural distance to Sobolev semi- norms (cf. Lemma and Céa's Lemma we obtain
| - Vu [ I c(p) Villll eocaje + Vu l1 ())))))
+
(6(b)²+ inf VV- E REM
Thus, it remains to estimate the first factor in the equation above. We use the coercivity estimate of Lemma 13 to obtain
Velle>(o) S. c(p, 22) (E(v) + IIfl (()))))))))))))))))))))))))))( c(p. (L)
where we used that 0 E MM to be able to estimate E(v) S 8(v). As result, again applying Lemma 13 follows that
(iVelleroa) + Vu Vu ((2(0))))
2 c(p,
5(4) 7
W1P(1)
ス ||/
c(p,
((0)
Assuming 5(v) S 1, holds d(v) +(0} < 25( ł, which concludes the proof.
2.2 PROOF OF THE PARAMETRIC SETTING
As detailed in the introduction, the energy formulation we use for a p-Laplace problem with a parametric right-hand side f E T.P' (P 12) and parameter space P C Rdo, dp N, for every (PPP,W))(())) is defined by
E(v) E(b)= = ff 1
[
V dx dx - -f f(T, O(T,)) dx dr. [p [
Before proving the error decay rates of Theorem [ need to identify the correct function space 4 for the definition of E. the case of a parametric right-hand side, this is straight-forward and the space 14 is standard Bochner space, see Proposition For varying domains as parametric dependency, the corresponding function spaces are intricate, we refer to Appendix
varying exponent
Next, need to guarantee that the minimizer of & indeed solves the parametric problem. This carried out in Proposition 9 and encoded the fact that u (T, -) E WP))(Q) for a.e. T E P minimizes Fr, in the notation of this Proposition.
7

---

Under review as a conference paper at ICLR 2023
Proceeding derive enror estimates, want mimic the strategy of the non-parametric case.
This crucially relies on the fact that the constants in equation 7 do not depend on the right-hand side. As consequence. can prove two-sided estimate as in Proposition 9 with analogue of Pr (l))( x (L))()) - given by P (P,,, WL))( x (P,P, W (92) + IR, for every
v, (P,P, (1)) defined by
(U, w) f (T((TT) ()
Finally, we can proceed in the non-parametric case and derive Céa Lemma.
Proposition 9 (Variable Right-Hand Sides). Let Ω C Rdd, dn E N, and P S Rdv, dp E N, be bounded domains and p E (1, 3.). Assume U WW) a closed subset that satisfies Poincaré inequality, as in Proposition 2 Moreover, we define the Bochner-Lebesgue space
u = (P),U))
For fixed f E L' (P x 12), we define the variable right-hand side p-Dirichlet energy & :4 - IR for every E u by
E(w) 8(O)= 1 P ( dr [ v(T, v(T,-))d)) dr;
VzV(T,)) Pr
where the gradient V for a.e. E P is be understood with respect to the variable Ω only. Then, the following statements apply:
(i) There exists a unique (parametric) minimizer u' E of E +R
(ii) For a.e. T E P. the function u (T,-) E U is the unique minimizer of E + R for every U defined by
Ex(1):= 1 Vol ((T,-)udr.
p n
(iii) Furthermore, for every v E U, it holds
c1p)- u²) < - S c(p) P3 u")
(9)
where c(p) > 0 is the constant from Proposition 7
Proof. We prove this for more general parametric dependencies the Appendix, see
Remark 10. Requiring f(T,) to be mean-value-free for a.e. T E P, we may, again, drop the assumption of Poincaré inequality on the space as we explained in Remark 4 this case, we cannot expect the minimizer to be unique.
The estimate is the key to establish a Céa type Lemma for the energy & : 4 + R. In the situation of Proposition we can accomplish this as in Lemma More precisely, for any fixed E M U4 it holds
(v) & c(p) ((()) + inf ((((
(10)
BEM
where 6(v) = Evv) - inf E()) the parametric optimization error. With all the previous work, the Main Theorem is can now be proved similar way as the case without parameters. We postpone the proof to the Appendix, see A.2
NUMERICAL EXAMPLES
We give two examples, with high dimensional physical domain and one with high- dimensional parametric rinht-hand side. Our goal is to investigate the error in dependence on the dimension. To find good neural network approximations, we employ Deep Ritz Method for training.
8

---

Under review as a conference paper at ICLR 2023
Example 1
As PDE posed high-dimensional physical domain, consider
-Au + = in 2
with homogeneous Neumann boundary conditions, and = (0. 10,1)da, da E N. manufactured solution, we use u E (L)) for every = (Z Fdg)
E 52 defined by
du
c u u(x)
cos(T::))
where e >> 0 is chosen such that has uniform (92) norm. Then, the right-hand side f e ( for every x is given via
f(x) = 1
Example 2 As problem with high-dimensione parametric right- agnt- side, we consider in P Ω
-u" +
with homogeneous Neumann boundary conditions, parameter space P = [-1, 1] dp E N. and physical domain 5 (0, As manufactured solution and right-hand side, we use L'(P, ((2) and f f L2 x Q), for every (T, < x) E P x S2 defined by
dr
dp-1
u(T,x)=
Tk
cos(kT=) and f(r,x)
Σ Th cos(k
1.
Neural Network Architecture and Training We employ fully-connected ReLU"-netword with four hidden layers and varying width as well as Deep Ritz energy formulation as loss function. To resolve the minimization problem, The appearing integrals are discretized using Monte-Carlo approximations, where new random points drawn for every update in the gradient descent. The optimization is until no further improvement is seen in approximating the ground truth, which our examples happens typically in around 10, 000 to 20, 000 iterations.
employ the Adam optimizer with learning rate set to 0.001.
0.25
0.05
Wisth umor
036
Witth 32, H² entr
Width 26
Wi-2h 32 Width
0.00
25
30
10
Paramoticc Dimena
Figure 1: The plot on the left shows the relative errors obtained from Example and the plot on the right reports the errors for Example 2. Here, dashed lines represent relative H' errors and solid lines stand for relative I errors.
Discussion In Figure I we report the relative errors obtained for the two examples in dependence on the total dimension d dg + dp N of the parametric cylinder examples, exponential increase of the error is observable at least for moderately high dimensions. Although impossible for us quantify how well the empirically found solutions resolve the minimization, i.e., how large the quantity d(un) in Theorem is. the experiments still confirm the promising behavior of neural networks for solving high-dimensional and parametric problems.
Q C Rd x Rd In both
[
AUTHOR CONTRIBUTIONS
All authors contributed equally all parts of the manuscript.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Robert Adams and John Fournier. Sobolev Spaces. Elsevier Science, 2003.
Andrew R Barron. Universal Approximation Bounds for Superpositions of Sigmoidal Function. IEEE Transactions on Information theory. 39(3):930-94 1993.
Richard Bellman. The Theory of Dynamic Programming. Bulletin of the American Mathematical Society, 6006)500-5515, 1954.
Lorenz T Biegler, Omar Ghattas, Matthias Heinkenschloss, David Keyes, and Bart van Bloemen Waan- ders. Real-time PDE-constrained Optimization. SIAM, 2007.
Ziang C Jianfeng I Lu, and Yulong Lu. On the representation of solutions to elliptic pdes in barron spaces. In Advances in Neural Information Processing Systems, volume 34, pp. 6465-6465. Curran Associates, Inc., 2021.
Bernard Dacorogna. Direct methods in the calculus of variations, 2nd ed, volume 78. Springer, New York, 01 2007. ISBN 978-3-642-51442-55 doi: 1..1007/978-3-642-54444
Michel C. Delfour and Jean-Paul Zolésio. Shapes and geometries, volume 22. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2011.
Lars Diening and Frank Ettwein. Fractional estimates for non-differentiable elliptic systems with general growth. Forum Mathematicum, 20(3):523-555 2008.
Lars Diening and Christian Kreuzer. Linear Convergence of an Adaptive Finite Element Method for the p-Laplacian Equation. SIAM Journal on Numerical Analysis, 46(2):614-638, 2008.
Lars Diening and Michael Rüžička. Interpolation operators in Orlicz-Sobolev spaces. Numer. Math., 107(1):107-129, 2007.
Lars Diening, Carsten Ebmeyer, and Michael Rúzička. Optimal convergence for the implicit space- time discretization of parabolic systems with p-structure. SIAM Journal on Numerical Analysis, 45 (2):457-472, 2007.
Lars Diening, Petteri Harjulehto, Peter Hästö, and Michael Rüžička. Lebesgue and Sobolev spaces with variable exponents. Springer, Heidelberg, 2011.
Chenguang Duan, Yuling Jiao, Yanming Lai, Xiliang Lu, Qimeng Quan, and Jerry Zhijian Yang. Analysis of deep ritz methods for laplace equations with dirichlet boundary conditions. arXiv preprint arXiv:2111.02000 2021.
Luca Esposito, Carlo Nitsch, and Cristina Trombetti. Best constants in poincar e inequalities for convex domains. Journal of Convex Analysis, 20, 2013.
Moritz Geist, Philipp Petersen, Mones Raslan, Reinhold Schneider, and Gitta Kutyniok. Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networks. Journal of Scientific Computing, 88(1):1-37, 2021.
Pierre Grisvard. Elliptic Problems in Nonsmooth Domains. SIAM, 2011.
Ingo Gühring and Mones Raslan. Approximation rates for neural networks with encodable weights in smoothness spaces. Neural Networks, 134:107-130, 2021.
Ingo Gühring, Gitta Kutyniok, and Philipp Petersen. Error bounds for approximations with deep ReLU neural networks in W norms. Analysis and Applications, 18(05):803-855 2020.
Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 1 115(34)8505-85MM -510, 2018
Oliver Hennigh, Susheela Narasimhan, Mohammad Amin Nabian, Akshay Subramaniam, Kaus- tubh Tangsali, Zhiwei Fang, Max Rietmann, Wonmin Byeon, and Sanjay Choudhry. Nvidia TimnetTM; An ai- accelleled multi-physics simulation framework. In International Conference on Computational Science, pp. 447-461. Springer, 2021.
10

---

Under review as a conference paper at ICLR 2023
Jan Hermann, Zeno Schätzle, and Frank Noé. heep-neural-network solution of the electronic Schrödinger equation. Nature Chemistry, 12(10):891-89) 897, 2020.
Arnulf Jentzen, Diyora Salimova, and Timo Welti. A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients. Communications in 167-12057 2021.
Mathematical Sciences,
Yuling Jiao, Yanming Lai, Yisu Luo, Yang Wang. and Yunfei Yang. Error Analysis of Deep Ritz Methods for Elliptic Equations. arXiv:2107 14478, 2021.
Alex Kaltenbach. Theory of Pseudo-Monotone Operators for Unsteady Problems in Variable Exponent Spaces. dissertation, Institute of Applied Mathematics, University of Freiburg, 2021.
Alex Kaltenbach and Michael Rúžička. Variable exponent Bochner-Lebesgue spaces with symmetric gradient structure. Journal of Mathematical Analysis and Applications, 503(2), 2021.
Alex Kaltenbach and Michael Rúzička. Convergence Analysis of Local Discontinuous Galerkin Approximation for Nonlinear Systems with Orlicz-Struccurre. arXiv:2204.0990 2022.
Yuchaw Khoo, Jianfeng Lu, and Lexing Ying. Solving parametric PDE problems with artificial neural networks. European Journal of Applied Mathematics, 32(3):421-435, 2021.
Thomas Koerber. Sharp Estimates for the Principal Eigenvalue of the p-Operator. Calculus of Variations and Partial Differential Equations, 57, 2018
Gitta Kutyniok, Philipp Petersen, Mones Raslan, and Reinhold Schneider. A Theoretical Analysis of Deep Neural Networks and Parametric PDEs. Constructive Approximation, 55(1):73-125, 2022.
Kookjin Lee and Kevin Carlberg. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics, 404, 2020.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. International Conference on Learning Representations, 2021.
Chao Ma, Lei Wu, al. The barron space and the flow-induced function spaces for neural network models. Constructive Approximation, 55(1):369-406, 2022.
Johannes Müller and Marius Zeinhofer. Error Estimates for the Deep Ritz Method with Boundary Penalty. 3rd Conference on Mathematical and Scientific Machine Learning (MSMI.2022). 2022.
Philipp Nägele. Monotone operator theory for unsteady problems on non-cylindrical domains. Ph.d. thesis, Institute of Applied Mathematics, University of Freiburg, 2015.
Philipp Nägele and Michael Rúzička. Generalized Newtonian fluids in moving domains. J. Differential Equations, 264(():835-88)) 866, 2018.
Philipp Nägele, Michael Rázička, and Daniel Lengeler. Functional setting for unsteady problems in moving domains and applications. Complex Var. Elliptic Equ., 6261):66-6588 97, 2017.
Svetoslav Nakov and Ioannis Toulopoulos. Convergence estimates of finite elements for a class of quasilinear elliptic problems. Computers & Mathematics with Applications, 104:87-112, 2021.
Mario Ohlberger and Stephan Rave. Reduced Basis Methods: Success, Limitations and Future Challenges. Proceedings of the Conference Algorimyy, pp. 1-12, 2016.
Michael Rúžička. Nichtineare Funktionalanalysis. Eine Einführung. Berlin: Springer. xii, 2004.
Justin Sirignano and Konstantinos Spiliopoulos. DGM:
deep learning algorithm for solving partial differential equations. Journal of computational physics, 15511339-1344 2018.
E Weinan and Stephan Wojtowytsch. Some observations on high digensimensional partial differential equations with Barron data. In Mathematical and Scientific Machine Learning, pp. 253-269. PMLR, 2022.
11

---

Under review as a conference paper at ICLR 2023
E Weinan, Jiequn Han, and Amulf Jentzen. Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning. Nonlinearity, 35(1):278, 2021.
Jinchao Xu. The finite Neuron Method and Convergence Analysis. Communications in Computational Physics, 28, 2020.
Dmitry Yarotsky, Error bounds for approximations with deep ReLU networks. Neural Networks, 94: 104-114,220 2017.
Bing Yu and Weinan E. The Deep Ritz Method: A Deep Learning- Basedd Numerical Algorithm for Solving Variational Problems. Communications Mathematics and Statistics, 6(1):1-12, 2018.
A PROOFS
Here, collect the proof's that were deferred to the Appendix.
A.1 PROOF OF PROPOSITION 2
In order to prove Proposition 2 we need some preparation. The first two lemmas analyze the point-wise properties of the function F' Fd + Rd. for every that induces the natural distance measure p WL-P((D)) x W IR defined by F(a) = |a PP 2a, error analysis of the p-Laplacian.
+ R, which is essential for the
Lemma 11. Let oo) and d E N. Then, there exists constant c(p) 0, depending only p E o0) and not depending on E N, such that the following statements apply:
(i) For every a,b RR, it holds
<(p) c(p)- ((()) F(b)2 (1ap=-2-1160-21))
S c(p) [F(a) - F(b)
(ii) For every a, RRd, holds
dp) F(a)= F(b))2 s (la) + 16)) 2a 612 c(p) [a()) F(6)
Moreover, we can choose c(p) such that (p c(p)) E C* (1,
Proof. See Diening et al. 2007
Appendix) or DDiening & Ettwein 2008 Appendix). Furthermore,
carefully reviewing the proofs in (Diening et al. 2007 Appendix) reveals that the constants c(p) > O. E (1, x)), in Lemma 11 depend continuously
P oo).
Lemma 12. Let p (1, oc) and d E N. Then, there exits constant c(p) > 0, depending only on p E (1, 30) and not depending on d E N, such that for every a, b E R2d with (a [++1 [6]
have that
c(p)~1 - F(b) < S' D²a(Ta+ (11 (1- : (a b) 8 (a b) (1 r) dr
I [((a) - (Fb))))
where & C² density, Moreover, we can choose c(p) > 0 such that (p c(p)) E (1,
for (0)), defined by (ba) = Lalp for all E IR,, denotes the p-Dirichlet
Proof. We introduce the abbreviation 1/2 12d Rd (() Ezo. for every a, E Rd with |a] + |6| > defined by
r7 b) ( D
b) (a b) (1 dr
Using D2 b b min(L,p-1Hoat 73, ineq. (1.35))), for every a. E Rd with |a| + [b| > 0. we obtain
for all a nd \ (0). IRd (cf. Rúžžčča 2004
ra+ (1 - a 612 (1 dr. (a, b) > min(1,p 13 /
(11)
12

---

Under review as a conference paper at ICLR 2023
With the help of Jensen's inequality applied with respect to the measure dj (1 r)dT, i.e., in particular, we use that d0, 1)) 2' for every a, b E R with |a] + |b| > 0, we observe that
(2// Ta + (1 - --) - 7) dr)
p
ITa+ (1 (1-=)bb* b) dr
(12)
Then, continue in equation 11 by incorporating equation
12 and, thus, find that for every a, be Rd
with la| + [6] > O, it holds
(a, b) 2 min 1} / Ta + (1-5)614 (1 7) dr
10 6/2
(1al + 16|}2
(13)
2 mi. 17 2 22/10 Ta + -)}b (1 - 7) dr)
" la 6/2
(/al [b])
For every a, b E Rd, it holds
2 L ITa+ + (1 (1- (1-7)bl (1 - dr dr 2
(la) + (6),
(14)
6
which is based on that for |a| > [b] and E [3) 1J. it holds Ta + (1 -)bl g|aa > [1a + |b)
and for [6] 2 |a| and 7 E [0, $1. it holds Ta + (1 -- ž / |b| 2 (laa + (bl). Using equation 14 in
equation 13 for every a, b E I2d with |a| [6] > 0, we deduce that
1
ri b) Z min[1,p 1} (la +
la b12
6
Resorting Lemma II
we conclude the existence of a constant c(p) > 0, depending only on E oo), with (p + c(p)) E (1, oo), such that for every a, b E IR" with |a| + [b] > 0, it holds
(a, b) 2 c(p)- F(a) F(b)1².
On the other hand, since also (a) : b & 6 < max 1.p - 2a|ap2-11b/ for all a e Rd \ {0}, b E Red, which, again, follows very similarly (Räzička 2004 p. 73, ineq. (1.35),, we find that
r22 b) S max p-a 2} / |ra + (1- (1-7)61--2 (1 7) dr la - b|
(15)
Since, appealing to (Diening et al. 2007 Lemma 6.1), there is a constant c(p) > 0, depending only on p E (1, with (p >> c(p))
(1,, such that for every a, b E IRd with |a| [b] > 0, it holds
L. ra + (1 - dr I c(p) (la) + 1b1)-2
we deduce from equation 15 that 7/² (a, b) s max (1, p (la))) 16) 612 for all a, b Rd with la| + [6] > 0, which, resorting again to Lemma I1 completes the proof of Lemma 12
Proof of Proposition 2 ad (i). The p-Dirichlet energy E U IR is proper, strictly convex, and continuous, hence, weakly lower semi-continuous. In addition, the validity of Poincaré's 's inequality guarantees the coercivity of E cf. Dacorognal (0077). the existence of unique minimizer u² E E of E U - IR.
F, so that the direct method in the calculus of variations yields,
ad (ii). We proceed similar to (Diening & Kreuzer
2008
Lemma 16.). Again, we employ the -pp Ta for all E IR. for the p-Drrichlet
notation o E C (Ted) n (Rd) {O]), defined by Φ(a) density. Since Dø Co (Rd)dd with [DΦ(a)) = Jal- for all E Rd. the p-Dirichlet energy is continuously Fréchet differentiable with
(DE(u), = Do(Vu) Vedr -
for all u, u.u E U. In particular, due to the minimality of u² E U. we have that DE(u*)= =0 in U*, i.., for every U. it holds
(DE(u), v)u = 0.
(16)
However, E: U R is not twice continuously Fréchet differentiable. Therefore, we consider reg- ularizations (Ør)s20 S (Rd(),), defined by (²(a) == (e² + |a|2) for every e > and a E R having the following properties:
13

---

Under review as a conference paper at ICLR 2023
(a) ( (5 0) for all a E R and @e(a) s /2 + + for all a E R and 0,
(8) (D) + (a) (s + 0) for all Rd and (D))) S = (Ia p- + = for all
E Rd and E > 0,
(n) (a) + (D* (s + 0) for all E (0) and ((D (a)) = (p 1)2 (
aap 2) 3) for all a Rd and >
Inasmuch (os)eso S (IR4) satisfies (a), (3) and (7), it is easily checked that for every 0, the regularized p-Dirichlet energy E : + R, for every v E U defined by
E (v)= 인e (V (f, ())))
is twice continuously Fréchet differentiable. In consequence, using Taylor's formula and Fubini's theorem, for every & and U, obtain
(u) - (=²) (DE 1u" ) u²)
Dd (TU (1 (1--7) r)"") IV u², 2- u²] (1 d)
(17)
(Vu" V(v - -=") dx
( vv + 7) V(o -V w"))dr (1 -T) dr
Do V(u (w-w) f(--w))d dx
a
D'V (1 (1- (L-r)Vn"):V( u') dr (1 r) dr.
Next, given both (ox), (3) and (?), it is allowed to apply Lebesgue's dominated convergence theorem in equation 17 Hence, by passing for + in equation 17 using equation [6 in doing so, for every E U, we find that
( Do(Vu") V(v - u²)dr dr
E(v) E(u²)
(18)
dr dr + [/lo D2 (2 V(r w²) V(v - u²)
(1- ((-)))Vu)))))
(DE(u"),u-v) =")u
S. D2 + V(v w²) x V(r u²) (1 r) dr
Salo D'
V(v u²) v w²) 7) dr dæ.
Apart from that, resorting to Lemma 12 we deduce the existence of a constant <(p) > 0, depend- ing only p (1)) with (p + c(p)) E (1, such that for every U, it holds
pf S [.f. D (1-r)++))))) V n²) V(v u²) (1 - T) dr dr
S c(p) (v)
(19)
Eventually, by combining equation
18 and equation 19 we conclude the assertion of Theorem 2
A.2 THE REMAINING PROOFS
Proof of Lemma 7
The following proof is inspired by (Nakov & Toulopoulos 2021
Section 3.1).
ad (i) By referring to Lemma II (ii). we deduce the existence of a constant c(p) 0, depend- ing only on E (1, 20), with (p+ + c(p)) E (1,, oo), such that for every u, u E W PP2), it holds
V Vellippoyy S / Vu- Vol((Vul (ol)) dr S c(p) (u,
14

---

Under review as a conference paper at ICLR 2023
and, using Hölder's inequality with respect to (2 p2pp-2).
-
Vu - (VVl + (ul) dr
Vu- voldr) "
(|Vu) Vol |VVll+VVo)>
2 ([Vuu]lze()) + |[ol| 2 | Vullip(0)a..
ad (ii) By referring to Lemma (ii), deduce the existence of constant c(p) > 0, depending only on E (1, (1),), with (p + c E (1, oo), such that for every u, E W. (9), using Hölder's inequality with respect
it holds
2
( ()( ([vvl + Vul)o ddr) P(V
[VVu++
[[ - W))l &
22
$ ([VullZF(())) + [ 20702
v(uuv))((Va + Vol)a dx
-
((2-))
s c(p) (IVullzzeane + +/Voll/>e
and
dp) pF(,,)) 0) - Vu - Vu Vel) dx
/ V VVu - Vol
Vu Vo/2-P
V Vullie
V Vu|)2-p
Lemma 13 (Coercivity of the p-Dirichlet Energy). Let f E WLP( E (1, oo), be such that (/ooworo for all R.. Moreover, we define E: WIPP (Q) for every WL.(( by
Vule dx (f. U)wi E(u) = p sn
(20)
Then, for every WLPP( we can estimate
Volle> s cpp (IIf + E(O)E)
For minimizer u' E WW-P(s)) of E: (2P((s) + R. this reduces to
| W S c(p, 22) | (ay)
The constant c(p, 2) > 0 depends continuously on p and on the domain Q.
Proof. Using that f E ( vanishes on constant functions, the Wiitiine inequality and the Young inequality, for every WL.P(())) and e > 0, abbreviating (v)a = fa u dar, it holds
F(v) ; V E - 4 + (f, v (v)a)w1r(()
2 V p [ c(p, E) |f f. -elle (1)allww.r())
(21)
2 / ー e Cp) Voll P c(p. 11
where o(p,s) 1-=) Hence, choosing s > 0 sufficiently small depending on the value of the Poincaré constant Cp in equation 21 for every E W.P)( we find that
Ve Foll S c(p,
[(()) + |I II (
(22)
where the dependence of the Poincaré constant leads to dependence the domain ?.
15

---

Under review as a conference paper at ICLR 2023
Remark 14 (On the Constant S2). While the dependence of c(p, Q2) p directly be understood from the proof, the dependence on f2 stems from the constant appearing in the Poincaré inequality - which call the Poincaré constant and denote by Cp. For convex domains, one has that
(p- 1) Tp 2 psin n
Cp
Tp
where
(ia)
we refer to Esposito et al.
(2013) or Koerber (2018).
Finally, we provide the missing proof of the Main Theorem.
Proof of Theorem we estimate
Let 2 2 and v E M be arbitrary. Using Proposition 9 and the inequality 10
/ |V (v(() 2² (T) (T))) dr
|V V.u Vuu'I [P(P
S <d)) /
(T)
U o (O(T), u (+)) 1) = (*). GE
< ((())
inf
We proceed by utilizzig the relation of the natural distance pi. Wl) x W - R to the Sobolev topology from Lemma 7 and,, subsequently, apply Hölder's inequality with the exponents
(+) 6 inf [/ (IV6(())l + (V" ((4())))))) p- (Vö(T) BEM
- Vu (T)())
$ 6 inf
|V V 12
56 3
2 (P(x)) |V "|
inf
| Ü
V v 1| (P(())
26+3 2
inf V BEM
V u (P
(Px))
This implies the assertion in the case p 2 2. The proof the situation of < works similarly and therefore omitted.
B SMOOTHNESS ASSUMPTION
For linear elliptic equations with parametric right-hand side, higher order Sobolev regularity holds true, provided the right-hand side f E L,P(P x Q) and the domain Ω Rda, da E N, are smooth enough. In the following, we denote by (S2), the Sobolev space with vanishing mean value.
Lemma 15. Let & E N be fixed and let P C ddd, dp N. be open and Ω C Rda, da E N, be domain with on E Ck+1.1 boundary, Let, furthermore, f C*+2(P, (P)) be given. Then, the weak solution u¹ E LP (f)) to
Au f in P x Ω 0 0 on P x 0n,
(23)
a member of the space H H*+ x S2).
Proof. We define e: H # (5) x P + H² (S2) for every u E H and T E P by
e(u, -u [(F) ()).
Then, the zero level set
{U,T)) ( P c(uu r) =0}
16

---

Under review as a conference paper at ICLR 2023
is parametrized through the solution map (T + u0 (T)) P H ( In fact, standard elliptic
regularity theory yields that u (T) E H for a.e. T E P. see for instance Grisvard 2011 1).
The implicit function theorem for Banach spaces guarantees that 2 E C H ((2) provided the partial derivative of e with respect to the first component, i.e., dre(u,T) H (9)) (2),
for every (u, T)T f H (12) x P given via
0re(u, Av in (2) for all H
is a linear homeomorphism. However, by using the exact same elliptic regularity result as used above, we see that this is in fact true. Hence, the assertion of the lemma follows.
C MORE GENERAL PARAMETRIC DEPENDENCIES
In the main part of the manuscript, we only considered parametric dependencies that were in- duced through a parameter dependent right-hand side. This was done to keep the technicality minimal, yet does not constitute the full generality of our analysis. In this Section, we outline more general parametric dependencies, including varying integrability exponents, domains and material tensors. In every situation, we guarantee the well cell-possdsnes of the parametric problem, providing an analogue annd, result to Proposition and, consequently, allows to deduce error decay rates. As error decay rates assuming smoothness follow the same pattern as Theorem [
we do not explicitly state them.
C.I PARAMETRIC EXPONENTS AND PARAMETRIC RIGHT-HAND SIDES
We begin with problem where both, the exponent p oo) and the right-hand side f are allowed to vary in a parameter space P. More precisely, we seek u² :P - R. satisfying
div (V (T, ()) ,," f(r,z) fora.e (T,x) E P
(T)
subjected suitable boundary conditions. The precise statement the following.
Proposition 16 (Variable Exponents). Let Ω S Rdo, da N. and P S Rdr, dp E N, be bounded domains and E (P) such that there exist p .pt E (1,o0) oc) with p S p(T) S pt for a.e. T E P. Moreover, we define the variable exponent Lebesgue space
TPO) x 12) = { (.
So |o(TT [o(T,2))(() p(T) dr < <xc
and the variable exponent Bochner-Lebesgue space
u = - uE L x X v(T,-) E W..P(() (0) for a.e. T E Vavl E LP x 0)}
where the gradient for a.. T E E is to be understood with respect to the variable x E Q only. For fixed f E L'OO x !), i.e. f (P x 2) and fp So. ffT,,)))'( dx dr < co, where p' E (P) is defined by (T) = p(r)
- for a.. T P, we define variable exponent p( -Dirichhtt
energy E: + for every E u by
E(v) - Sf (P(T) / (2o(T,))) p(t) do)) dr ( S(T,)) ((T.)) dr.
Then, the following statements apply:
(i) There exists a unique (parametric) minimizer u' Eu of E 4 - R.
(ii) For a.e. T E P. u² E (.) is a unique minimizer of E W...((t)(() + IR for every & E W.lp(t) (I.) defined by
E+(v) p(T) Vopl) - f(T, dz
'Here, T. 22) denotes the space of scalar
functions Ω.
17

---

Under review as a conference paper at ICLR 2023
(iii) For a.e. P and (l) it holds
db(T)) F (T.-))) S Ex(o) - (x,x"(T.))
s (F F (20))*
where Fr: Rdd + RR, P. for every P is defined by Fr(a) |a " for all a E Rddt and c(p(T)) > 0 is the constant from Theorem 2
(iv) Furthermore, for every v E U, it holds
- ess inf (inff(p(T)) (3 u²) S E(o))-(w'') S ess supc(((()))) p3 u²), TEP
TEP
where
/ |1F=(V F+(V (T, (I 12
(²)
Proof. ad (i). The space 44 equipped with the norm where
|| llu == || (l1 + V ( (Px0)
1 f v(T, ((T)
[1112 =
dx dr dr <
X
denotes the Luxembourg norm, cf. Diening et (201T). is reflexive Banach space, ef. (Kaltenbach 2021 Proposition 3.7 & Proposition 3.9) or (Kaltenbach & Ruzička [2021] Proposition 3.6 & Propo- sition 3.7 Apparently, due to Poincaré's inequality applied for a.e. fixed t E ", which is allowed since v(T,,) E W (() for a.c. T P. we have that
4 + IR is strictly convex and continuous. In addition, for every U
V V=v(T,x)))pt dx dr So// dr) dr S S. dia)m
a
(24)
S (1 2d)a pt
[ [=)((T dr
x)
which for every v and E E (0, 1. using for ae. T E P, the g- Gouung inequality with constant cp((T),)) = (p(T)e))-) 1-p implies that
((T)
1
/ V2o(T,-))Ptt dr
E(O)
P(T)
cp(T).=))f(T, ) - =//(T,, (aou(T,))PP ()p(t)) dz dr
p n
(25)
- e(1 + 2diam (d)a) / (e.UTT)))(t dx dr
(p_e) -
(wty J ( (S(T, )p'(t) drdr.
Hence, since Sp Sq |u(T ((T) + v U(T ()(()) dr dr + oo if |ollee + oa (cf. DDening et al. 2011 Lemma 3.2.4)) from equation 24 and equation 25 for & E (0, 1 sufficiently small, we conclude that from |||| Ilee + oc, follows that E(v) + 0o, i.c. E:4 - R. weakly coercive, so that the direct method in the calculus of variations, ef. Dacorognal (007). yields the existence of unique minimizer u² U E 1u R.
ad (ii). A standard calculation shows that E : u + IR is continuously Fréchet differentiable with
(DE (u), v)u (DFi (u(T, )), ())3,, dr
'More precisely, these references prove only the case N = 1, since therein P represents time interval in an unsteady fluid flow problem. However, the proofs can be generalized verhatimly to the case N > 1, so that we will refrain from proving these results again at this point.
18

---

Under review as a conference paper at ICLR 2023
for all u, U. Therefore, due to the minimality of u² E U, for every Ut we have that
0 = (DE((")))))) / (DEr )), (ww dr.
(26)
Inasmuch Wg.pe (2) (I) densely for a.e. T P and Wiip" (2) is separable and, thus, contains countable dense subset (1h))EEN C W (Ω), the subset (%h keN lies even densely in W (.) for a.e. T E P. Next, choosing v = ppu E U in equation 26 arbitrary yp C (P)
and & E N, we further deduce that
J. (DEr(u (T, ( =0
(27)
so that for each fixed & N, the fundamental lemma of calculus of variations implies that for a.e. T EP, it holds ED (m"(T,"" -)), shx) (2) 0. This, since the countable union of sets of zero mea-
sure has still zero measure, we deduce from equation 27 that for a.c. T E P, it holds for all k E N
(DE,(u' (T, )), (2) =0.
(28)
Since (+h) KeN is dense in ((L for a.e. P. from equation 28 we infer that for P. it holds for all v E Wg ((.
(DE, (u'(T, -), u) = 0.
Eventually, since for a.c. T E P. the p(r)--Drichllt energy E, ()) - R strictly convex, for a.c. T E P. the slice u (T.-) Wo (.))(t(((()) is unique minimizer of E, W.!-P(t) (2) + R.
ad (iii) and (iv). Follows from point (ii) and Theorem 2
C.2 PARAMETRIC DOMAINS
We consider parametric domains, where we focus on domains depending only one parameter, as the required function spaces are only studied this case. More precisely, we aim to solve
div (IV u (T, )²1" 1ill--VVupp Vuz (T, x)) = f(T, for a.c. x E (((T), T P.
The precise requirements are given in the following proposition.
Proposition 17 (Variable Domains). Let Ω C Rda, de N. bounded Lipschitz domain and p E (1, ).). Moreover, lel ςr : Ω + ((T), T PEP: = (0. T O, the induced flow of a smooth, compactly supported vector field v R x Rd Rd, cf (Delfour & Zolésio 2017 Chapter 4). For the non-cylindrical domain Q = UtEP [T] x Q((T), we define the variable domain Beadmer-Lebesgue space
4= LP ().P( = (uE (Q) (u(T.)) E (Q(T)) for a.c T TEP, LP(Q))
where the gradient for a.e. T E P be understood with respect the variable x E SI(T) only. For fixed f E L' (Q), we define the variable domain P- p-Dirichlet energy & : u + for every u E U by
E(v) - f [. (V, MI dx S f(T, :) v(T, dz dæ dr (((T)
Then, the following statements apply:
(i) There exists unique (parametric) minimizer u' E U of E u R.
(ii) For a.e. T E P, (T.-) (Q(P()(T)) is unique minimizer of E, W. ()((()()) - R. for every v E (P(T( defined by
{ Pol - [ f(T,
[r(r)= = s
(ii)) For a.e T T and W..-P(T)(S)) (22), it holds
(.))) ((V F(V ".'"(TT)
S Ex(v) Ex (u' (T.
S <((() [[((V)) F(V.u"(T, (T.
where c(p) > 0 is the constant from Theorem
2
19

---

Under review as a conference paper at ICLR 2023
(iv) Furthermore, for every E U. it holds
ess inttc(pp) (z( s ((0)-EE(") S sup c(p) 2 u²), TEP
TEP
where
(z( [I((Ve0(TT)) - (1)
Proof. ad (i). The space U equipped with the norm || 11 11-10o0)) V (ll forms a reflexive Banach space, cf. (Nägele 2015 Proposition 3.17 & Corollary 3.25) or Nägele et al. 2017); Nägele& & Rúzička (018). Apparently, & 2 + strictly convex and continuous. Apart from that, for every U E U. due to Poincaré's inequality applied for each fixed T E P, which is allowed since v(T, E W (P(T() for all T E P, we have that
[. 1o(T, x0(T,)) dx <
/. (eiim(((TT))
( VVo((T x)) dx ((T)
(29)
$ (1 (1+2 sup diam(2(TT)) TEP
( dr
which for any 2 U and & E (0, 1}. using for each T E P, the e- G-Young inequality with constant cp.c)= = p 1,
implies that
{(v) 2 {
1
{
(2 dr - -Ip
(mm c(p,E))f(T,))))) )1 - =llu(T,))) dx dr
p p ((())
1
8 ((1
sup diam( Jo II TEP
(30)
p
(pE)--V/
I (((T) [5(T, )) dx
vf
From equation 29 and equation 30 for e O sufficiently small, using that, by assumption, it holds
sup, diam (?(T)) x we conclude that from |/0|| - co, follows that (vv) oo, i.e., : - IR is weakly coercive, so that the direct method in the calculus of variations, cf. Dacorognal (2007). yields the existence of unique minimizer u* of
R.
ad (ii). A direct calculation shows that E :u + R is continuously Fréchet differentiable with
(DE(U), = (DE, (u((T.))) v(T, ()33)"(((T)) dr
for u, v U. Therefore, due to the minimality of u" E Eu, for every u E U, we have that
0 (DE(u") (e((u)),)) f (DE+ (u" (T, ((. dr.
(31)
Since (S2-)))((0) is separable. there exists countable dense subset (Wu)LEN C ())))))())) Also, appealing to (Nägele 2015, Lemma 2)1), for any E '. the pull-backs ((x=1 " AEN == (the
op DREN S ([(() )) are dense in Wt (rrl-P((2(2 (S((T)) In addition, (Nägele et al. 2017 p. shows
that (Bb)) = (T + (f- " S U. Next, choosing arbitrary &p Cg (P) and N. we further deduce that
you E u in equation 31 for
I (DE+ (T))) (T, ()/ ((T) dr 00
so that, owing the countability of (shc) S U. the fundamental lemma of calculus of variations implies that for a.c. P, it holds for all & E N
(DE (T, 1), (x th)) (h))")///)//oon wh))wg.rr)(a)) = 0.
Here, we exploit that there exists K 0 such that K-1 S det S K in P(T) for all T P. cf. (Nägele et al. 2017 (3.1)).
20

---

Under review as a conference paper at ICLR 2023
As ((p=)) " (007"))"()k)) bx)keN is dense in (((((((T)) for all T E P, we find that for a.e. E P, it holds for all v E W (((T))
(DE,(u" (T, }), (())) 0.
Eventually, since for every T E P. the p-Dirichlet energy E : (9((T) + R is strictly convex, for a.e. P, the slice u² (T, W ((((T) is unique minimizer of E,, (W) R.
ad (iii) and (iv). Follow from point (ii) and Theorem 2
21


Reviews:
[{"review_id": "50NmteHpUJp", "reviewer": "Reviewer_bRj3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provided a Céa-type lemma's lemma for p-laplacian. Based on this, the author provide a breaking the curse of dimensionality proof for solving the PDE.", "summary_of_the_review": "The main contribution of the paper may lies in the area of numerical PDE and is not fit for a machine learning conference. At the same time, I don't think the contribution of this paper is significant enough for ICLR.", "strength_and_weaknesses": "Question 1: What's the application of p-Laplacian equation in high dimension?  ( I don't think clustering is a good application. \nQuestion 2: The main contribution is a Céa's lemma's lemma for PDE. All other part of the proof is not the contribution of the paper, e.g. breaking the curse of dimensionality. It seems this paper is more fittable for a PDE journal but not a machine learning conference. Why this paper will catch ICLRs's audience's interest?\nQuestion 3: The Cea lemma is also not novel. For p-laplacian, it's easy to find a 1989 paper [1] covers it. It can be used for non-linear ansatz for paper [2] is using it for their bound. \nQuestion 4: It would be interesting to see whether this cea lemma will lead to information theoretical optimal bound as [3,4]\n\n\n[1] Finite element error estimates for non-linear elliptic equations of monotone type\n\n[2] Sheng H, Yang C. PFNN: A penalty-free neural network method for solving a class of second-order boundary-value problems on complex geometries. Journal of Computational Physics, 2021, 428: 110085.\n\n[3] Nickl R, van de Geer S, Wang S. Convergence rates for penalized least squares estimators in PDE constrained regression problems[J]. SIAM/ASA Journal on Uncertainty Quantification, 2020, 8(1): 374-413\n\n[4] Lu Y, Chen H, Lu J, et al. Machine learning for elliptic PDEs: fast rate generalization bound, neural scaling law and minimax optimality[J]. arXiv preprint arXiv:2110.06897, 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "3qWGNsGZCfE", "reviewer": "Reviewer_jdZY", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studied the approximation error of neural networks in approximating a parametric p-Laplace equation with the homogeneous Neumann boundary condition. The main result (theorem 1) established an W^{1,p}-error bound for an approximate solution in terms of the energy difference and the minimum W^{1,p}-error of the underlying approximation function class. The authors also discussed how the error bound can be used to prove that neural networks can beat the curse of dimensionality when the exact solution of the PDE is sufficiently smooth or Barron. ", "summary_of_the_review": "I would not accept the paper in ICLR, but it could be published in a math journal on numerical analysis of PDEs. ", "strength_and_weaknesses": "Strength: The error estimate in Theorem 1 seems to be new in the study of p-Laplace equation. The key ingredient of the analysis is a Cea type lemma which was proved for the case p=2 previously. The paper extends this to the general case that p\\geq 1.\n\nWeakness: The results obtained in the paper are all about the p-Laplacian equation alone and no new results were presented from the side of machine learning (or neural networks).  Hence to me the paper is more suitable for a journal publication in the field of numerical PDEs but not in a conference proceeding of machine learning.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "06Y9qGDK0Hj", "reviewer": "Reviewer_iQb8", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The text focuses on estimating solution of the PDE $\\text{div}[|\\nabla u|^{p-2} \\nabla u ] = f$. The solution $u_\\star$ to this PDE can be expressed as minimiser of an energy functional, $u_\\star = \\text{argmin}  \\mathcal{E}(u)$. The main result of the paper, Theorem 1, shows that any function $v$ in a (very general) subset of function $\\widetilde{M}$ is such that $|\\nabla v - \\nabla u_\\star|_{L^p}$ can be upper bounded by a constant multiple of the sum of the energy gap $A = \\mathcal{E}(v) - \\text{inf}\\big(  \\mathcal{E}(v') : v' \\in \\widetilde{M} \\Big)$ and the quantity $B = \\text{inf}\\Big( |\\nabla v' - \\nabla u_\\star| : v' \\in \\widetilde{M} \\Big)$ related to the quality of the approximating subspace $\\widetilde{M}$.\n\nSince the quantity $B$ is quite well studied (i.e. approximation theory of neural network spaces), this shows that any algorithm that can minimise the energy functional leads to good approximation to the solution $u_\\star$.", "summary_of_the_review": "1. **I am NOT knowledgeable enough to comment on the quality / novelty / correctness of the article** \n2. I am inclined to think that ICLR is not the right venue for this type of work that requires serious proof-checking.", "strength_and_weaknesses": "The text is clearly written, with a well written introductory section and a very useful section following the statement of Theorem 1 discussing the consequences of the main results and how it fits within the modern literature. As a non-expert, I have been to easily follow this sections and understand the importance of the stated result.\n\nNow, I need to admit right away that I am not an expert in this area and have not been able to follow the proof of the main result. As a matter of fact, I am not entirely sure that ICLR is a perfect venue for this type of technical work. Most of the details are in the appendix and I am not entirely sure that the ICLR reviewing process is adapted for proof-checking this type of (important) contributions. \n\nI am sorry that I cannot provide useful comments to the authors (and to the AC).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject