Is $L^2$ Physics Informed Loss Always Suitable for Training Physics Informed Neural Network?
Authors: Chuwei Wang
Shanda Li
Di He
Liwei Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Physics-Informed Neural Network
Partial Differential Equation
adversarial training

Abstract: The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The $L^2$ Physics-Informed Loss is the de-facto standard in training Physics-Informed Neural Networks. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero. With this concept, we study an important class of high-dimensional non-linear PDEs in optimal control, the Hamilton-Jacobi-Bellman (HJB) Equation, and prove that for general $L^p$ Physics-Informed Loss, a wide class of HJB equation is stable only if $p$ is sufficiently large. Therefore, the commonly used $L^2$ loss is not suitable for training PINN on those equations, while $L^{\infty}$ loss is a better choice. Based on the theoretical insight, we develop a novel PINN training algorithm to minimize the $L^{\infty}$ loss for HJB equations which is in a similar spirit to adversarial training. The effectiveness of the proposed algorithm is empirically demonstrated through experiments.  Our code is released at https://github.com/LithiumDA/L_inf-PINN.

Is L Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?
Chuwei Wang' Shanda e Di He Liwei Wang
Schooo of Mathematical Sciences, Peking University
Machine Learning Department, School of Computer Science, Carnegie Mellon University National Key Laboratory of General Artificial Intelligence,
School of Intelligence Science and Technology. Peking University
Center for Data Science, Peking University *Zhejiang Lab chuwei wang@pku. edu, en, chandal@cs. cmu edu dihe@pku. edu, en, wanglw@pku. edu. cn
Abstract
The Physics-Inford Neural Network (PINN) approach is new and promising way solve partial differential equations using deep learning. The L Physics- Informed Loss is the de-facto standard in training Physics-Ifformed Neural Net- works. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability the literature of partial differential equation to study the asymptotic behavior of the learned solu- tion as the loss approaches zero. With this concept, we study an important class of high-dimensional non-linear PDEs in optimal control, the Hamilton-Jacobi Bellman (HJB) Equation, and prove that for general T," Physics-Informee Loss, wide class of HJB equation is stable only if p is sufficiently large. Therefore, the commonly used L loss is not suitable for training PINN on those equations, while - loss is better choice. Based on the theoretical insight, we develop a novel PINN training algorithm to minimize the _ loss for HJB equations which is in similar spirit to adversarial training. The effectiveness of the proposed algorithm empirically demonstrated through experiments. Our code is released at https: //github com/ hom/Lithe LithiumDA/ MmmDALLL -PNN.
Introduction
Recently, with the explosive growth of available data and computational resources, there have been growing interests in developing machine learning approaches to solve partial differential equations (PDEs) [14, 13, 33, 28). One seminal work this direction is the Physics-Informe
Neural Network (PINN) approach [28] which parameterizes the PDE's solution as a neural network. By defining differentiable loss functionals that measure how well the model fits the PDE and boundary conditions, the network parameters be efficiently optimized using gradient- based approaches. L distance is one of the most popularly used measures, which calculates the L norm of the PDE and boundary residual the domain and boundary, respectively. Previous works demonstrated that PINN could solve a wide range of PDE problems using the T.² Physics-Informed Loss, such as Poisson equation, Burgers' equation, and Navier-Stokes equation [28, 6].
Although previous works empirically demonstrated promising results using L Physics-Informed Loss, we argue the plausibility of using this loss for (high-dimensionaa) non-linear PDE problems.
Eqqual contribution.
Corresponnence to: Liwei Wang <wanglwOpku. edu. en> and Di He <dihe@pku edu, en>.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

We know the trivial fact that the learned solution will equal the exact solution when its L loss equals zero. However, the quality of learned solution with small but non-zero loss, which is a more realistic scenario in practice, remains unknown to have any approximation guarantees. In this work, we aim at answering fundamental question:
Can we guarantee that a learned solution with a small Inforic- Loss always corresponds to good approximator of the exact solution?
To thoroughly investigate the problem, advocate analyzing the stability of PDE [8] in the PINN framework. Stability characterizes the asymptotic behavior of the distance between the learned solution and the exact solution when the Physics-Informed Loss approaches zero. If the PDE is not stable with respect to certain loss functions, we may not obtain good approximate solutions by minimizing the loss. To show the strength of the theory, we perform comprehensive study on the stability of an important class of nigh-dimensional non-linear PDEs in optimal control, the Mamilton-Jcobi-Bellle
(HJB) equation, which establishes necessary and sufficient condition for a control's 's optimality with regard to the cost function. Interestingly, we prove that for general LP Loss, the HJB equation is stable only if p is sufficiently large. This finding suggests that the most widely used L loss may not be suitable for training PINN on HJB equations as the learned solution can be arbitrarily distant from the exact solution. Empirical observation verifies the theoretical results.
We further show our theory serve principled way to design loss functions for training PINN. For the high-dimensional HJB equation we target in the paper, the theoretical result suggests that 7 loss may be better choice learn approximate solutions. Motivated by this insight, we propose new algorithm for training PINN, which adopts a min-max optimization procedure to minimize the L loss. Our approach resembles the well-known adversarial training framework. In each iteration, we first fix the network parameters and learn adversarial data points to approximate L loss, and then optimize the network parameters minimize the loss. When the training finishes, the learned network will converge to solution with small L losses and is close to the exact solution. We conduct experiments demonstrate the effectiveness of the proposed algorithm. All empirical results show that our method can indeed learn accurate solutions for HJB equations and is much better than several baseline methods.
The contribution of the paper is summarized as follows.
We make the first step towards theoretically studying the loss design in PINN, and formally introduce the concept of stability in the literature of PDE characterize the quality of learned solution with small but non-zero Physics-Informed Loss.
We provide rigorous investigations on an important class of nigh-dimensional non- non-linear PDEs in optimal control, the HJB equation. Our results suggest that the widely used L loss is not a suitable choice for training PINN on HJB equations.
Based on the theoretical insight, we develop a novel PINN training algorithm to minimize the L loss for HJB equations. We empirically demonstrate that the proposed algorithm can significant improve the accuracy of PINN in solving the optimal control problems.
2 Related Works
Physics-Infornee Neural Network approaches [33, 28] learn to find parametric solutions to satisfy equations and boundary conditions with gradient descent. There has been a notable scarcity of papers that rigorously justify why PINNs work. Important works include [31], which prove the convergence of PINN for second-order elliptic and parabolic equations. In [21], the authors study the statistical limit of learning a PDE solution from sampled observations for elliptic equations. In [29], the convergence of PINN with L loss established for Navier-Stokes equations. At the same time, several works observed different failure modes for training PINN in other PDE problems. In [17], researchers discover that PINN sometimes fails to learn accurate solutions to a class of convection and reaction equations. Their analysis shows that this may be attributed to the complicated loss landscape. [34] observed PINN failed to learn the Helmholtz equation due to the incommensurability between PDE and boundary losses.

---

In this work, we mainly experiment with the CommitonnJJcoble Bellmmmm (HJB) equation in optimal control. Previously, there were several works aiming at solving the HJB equation using deep learning methods [12, 13, 25, 39, 24, 1, 27, 5]. [13] is among the first to leverage neural networks to solve particular, [13] targets constructing an approximation to a solution value u at T = 0, which is further transformed into backward stochastic differential equation and learned by neural networks. The main difference between [13] and ours is that [13] only learns the solution on pre-defined time frame, while with our method, the obtained solution can be evaluated for any time frame. Recently, [12] tackled the offline reinforcement learning problem and developed a soft relaxation of the classical HJB equation, which be learned using offline behavior data. The main difference between [12] and ours is that no additional data is required in our setting.
HJB equations.
Stability is one of the most fundamental concepts in studying the well- posedness of PDE problems. Formally speaking, it characterizes the behavior of the solution to a PDE problem when small perturbation modifies the operator, initial condition, boundary condition, or force term. We say the equation is stable if the solution of the perturbed PDE converges to the exact solution as the perturbations approach zero [8]. The problem regarding whether PDE is stable has been intensively studied [8, 19, 11] in literature. There are also some works studying how (regularity) conditions affect stability. [20] and [7] investigate in which topology Couette Flow is asymptotic stable. [4] answers in which Sobolev space defocusing nonlinear Schrodinger equation, real Korteweg- de Vries (KdV), and modified KdV are locally well-posed. Our main focus is akin to the latter works but is settled in the machine learning framework.
Preliminary
In this section, we introduce basic background on Informe- Neural Networks and stochastic optimal control problems. Without loss of generality, we formulate any partial differential equation as:
[u(xx) = (a)) [Bu(x) = (xx
x E Ω C RM x E an,
(I)
where is the partial differential operator and B is the boundary condition. We use x to denote the spatiotemporalldeeeedeen variable, and use S2 and 0N to denote the domain and boundary.
Physics-Informed Neural Networks (PINN) PINN [28] is a popular choice learn the function w(r) automatically by minimizing the loss function induced by the PDE (1). To be concrete, given p E (1, +oc), we define the I,P Physics- Informeed Loss
(u) = Cu(x) - =p(x)( (
(2)
(u) Bu(xx) (x()))12
(3)
The loss term (u. in Eq. (2) corresponds the PDE residual, which evaluates how u(x) fits the partial differential equation on Ωi and (u)) in Eq. (3) corresponds to the boundary residual, which measures how well u(x) satisfies the boundary condition on 0R. L denotes p-norm, where p is usually set to 2, leading to "mean squared error" interpretation of the loss function [33, 28). The goal is to find u² that minimizes linear combination of the two losses defined above. The function u(x)) is usually parameterized by neural network uz(x) with parameter 0 E θ To find 0* efficiently, PINN approaches use bradient-based optimization methods. Note that computing the loss involves integrals over l and an. Thus, Monte Carlo methods are commonly used to approximate (u) and lon, (u) in practice.
Stochastic Optimal Control Stochastic control [9, 3] is an important sub-field in optimal control theory. In stochastic control, the state function (X₂) (XTTor is stochastic process, where T the time horizon of the control problem. The evolution of the state function is governed by the following stochastic differential equation:
dX. = m[s + odW, E [t.7] =
(4)
where m : [1, x T - RR. the control function and (W.) a standard n-dimensional Brownian motion.
3

---

m. s)ds + g(x)), Given a control function m, its total cost is defined as Ja,((m) Efe r(X,,
where r I x R x [0. 7] + R measures the cost rate during the process and g IR"" + RR measures the final cost at the terminal state. The expectation is taken over the randomness of the trajectories.
We are interested in finding a control function that minimizes the total cost for a given initial state. Formally speaking, we define the value function of the control problem (4) as u(x, 1) min (m,) MEM
where M denotes the set of possible control functions that we take into consideration. It can be obtained that the value function will follow particular partial differential equation as stated below.
Definition 3.1 ([38)). The value function u(z.t) t) is the unique solution to the following partial differential equation, which is called Kamitton-Jacobi-elllee Bellman Equation:
@,u(z,t) t) 20 + min [r(2, m(t, + Vu -m] 0 mEM
(5)
u(x;T) = g(x).
-aaiitoon-Jacoii--e Bellman (HJB) equation establishes necessary and sufficient condition for a
control's optimality with regard to the cost functions. It one of the most important high-dimensional PDEs [16] in optimal control with tremendous applications in physics [32], biology [18], and finance [26]. Many well-known equations, including Riccati equation, Linear Linear- Caadratic-Gaussee control problem [36], Merton's 's portfolio problem [22] are special cases of HJB equation [37].
Conventionally, the solution the HJB equation, i.e., the value function u(æ, I). can be computed using dynamic programming (2]. However, the computational complexity of dynamic programming will grow exponentially with the dimension of state function. Considering that the state function in many applications is high-dimensional, solving such HJB equations is notoriously difficult in practice using conventional solvers. As neural networks have shown impressive power learning high-dimensional functions, it's natural to resort to neural- network--wssedd approaches for solving nigh-dimensional HJB equations.
4 Failure Mode of PINN on High-Dimensional Stochastic Optimal Control
Note that u(x) is the exact solution to the PDE (1) if and only if both loss terms (u) and Eoa.p are zero. However, in practice, we usually can only obtain small but non-zero loss values due to the randomness in the optimization procedure or the capacity of the neural network. In such cases, a natural question arises: whether a learned u(x:) with a small loss will correspond a good approximator to the exact solution u" (z)? Such property is highly related to the concept stability in PDE literature, which can be defined as below in our learning scenario:
Definition 4.1. Suppose Z1, Lz, Zz are three Banach spaces. We say a PDE defined as Eq. (I) is (Zi, Zz, Za) -ttbble if [lu((x)) u(x)]lz, = C((]Cu(z)= - <(x)]lz, + [|Bu(x)) vx(x)llz, as [[Cu(x)) p(x))l2,, [[Bu(x) - (2)]ll2 + 0 for any function u.
By definition, if PDE is (L) (30)) -tsabelee with a suitable Banach space Z, we can mini- mize the widely used 2 Physics-Infomee and the learned solution is guaranteed to be close to the exact solution when the loss terms approach zero. However, stability is not always an obvious property for PDEs. There are tremendous equations that are unstable, such as the inverse heat equation. Moreover, even if an equation is stable, it is possible that the equation is not (L)( L'( -tttbbe which suggests that the original 2 Informed Loss might not be good choice for solving it. We will show later that for control problems, some practical nigh-dimmnsiooal HJB equations are stable but not (L² (S2). (3 Z)- stable, and using L Physics-Informe Loss will fail to find approximated solution in practice.
Losses || [] (<=))) and ((x)=)
We consider a class² of HJB equations in which the cost rate function is formulated as r(x, m) = Baaiitton--aobi-Bllllll equation can
a1 |ma ( .... 0n mm, amn ((x t). The corresponding
"The form of cost function we investigate in the paper is representative in optimal control. For example, in financial markets, we often face pawer-law trading cost in optimal execution problems [10, 30J. The cost function in Linear- Quadratic- Gaussian control and Merton's 's portfolio model (constant relative risk aversion utility function in [22]) is also of this form. Therefore, we believe our theoretical analysis for this class of HJB equation is relevant for practical applications.

---

be reformulated as:
" Σ i=1
LHJBU = 0,((x,)) 0 Au(x,t)
1.10z u/ p(x,t) (x,t) R x [0, T
(6)
BHIBU = u(x, T) = ((x)
x E R"
where A, = (a,0;) - as a,(a,))) -4-1 E (0, +00) and C2 = -ل-ه E (1, 0o). See Appendix B for the detailed derivation. For a function f X + R. where X measurable space, we denote by suppf the support set of f. i.c. the closure of {x E X f(z) + 0}.
An important concept for analyzing PDEs is the Sobolev space, which is defined as follows: Definition 4.2. For E N. E [1, +00) and an open set Q C IR, the Sobolev space W is defined as {f(x) LP(D) D LP(0), Vo |a| S m}. The function space Wmpp()) is
equipped with Sobolev norm, which defined as [l =
Σ JalSm
| MIL
The definition above can be extended to functions defined on spatiotemporal domain Q C R" x [0, T]. With a slight abuse of notation, we define Wm,P((Q) = {s(x, E IP(Q) D e (Q),, Va E No, lax| I m). where the differential D is only operated over spatial variable x. The norm (lw)).(QQ can also be defined accordingly.
Stability of the HJB Equation We present our main theoretical result which characterizes the stability of the HJB equation (Eq. (6)). In particular, we show that the HJB equation is (I"(RR" x [0, TD) (R"). W. (R" x [0, (o,T)))-stal -stable when p. q and satisfies certain conditions. We take the Banach space Z3 Definition 4. as Wi. here because it captures the properties of both the value and the derivatives of function, but LP spaces do not. However, as could be seen from Appendix B, for optimal control problems, is essential to obtain an accurate approximator for both the value and the gradient of the value function (the solution of (Eq. (6)). Thus, it is appropriate analyze the quality the approximate solution in W space.
Theorem 4.3. For P.q 2 1, let ro
(n+2)g Assume the following inequalities hold for p, q and ro: n+y
(c- p 2 max { 2. (1- n (2- C)n 2
1
T2
where G₁ Eq. (6). Then for any E (o) and any bounded open set Q C x 0, Tj.
15i5R
Eq. (6) is (L"(R" [0, T],, (R("), (1.r(Q)) for ē 2.
The proof of Theorem 4.3 can be found in Appendix C and an improved theorem with relaxed dependency on č can be found in Appendix E. Intuitively, Theorem 4.3 states that (L", L", W.."), stability of Eq. (6) can be achieved when p. g = I(n).. We further show that this linear dependency on cannot be relaxed in the following theorem:
Theorem 4.4. There exists an instance of Eq. (6). whose exact solution is u², such that for any 0, > 0,r 1,m E N and p [1, T. there exists a function C (R." x (0, which satisfies the following conditions:
[LubBU (R (0.T) <, BHIBU BHIR"", and supp(u u²) is compact, where CHJB and BAлE are defined in Eq. (6).
||u - u² w''lwwm > A.
The proof of Theorem 4.4 can be found in Appendix D.
Discussion. Theorem 4.3 and 4.4 together state that when the dimension of the state function is large, the HJB equation in Eq. (6) cannot be (L", La, --stall if and are small. Furthermore, since L/= by definition, Theorem 4.4 also implies that Eq. (6) not even (L", L3 -ta-stable. Therefore, for high-dimensional HJB problems, if we use classic 1 Physics-Informed Loss for training PINN, the learned solution may be arbitrarily distant from x² even if the loss is very small. Such thcoretical results are verified in our empirical studies in Section 6.
5

---

More importantly, our theoretical results indicate that the design choice of the Physics-Informed Loss plays a significant role in solving PDEs using PINN. In this work, we shed light upon this problem using HJB equations. We believe the relationship between PDE's stability and the Physics-Informn Loss should be carefully investigated in the future, especially for high-dimensional non-linear PDEs whose stability are more complicated than low-dimensional and linear ones [8, 11, 19]. Given the above observations, we further propose a new algorithm for training PINN to solve HJB Equations, which will be presented in the subsequent sections.
Solving HJB Equations with Adversarial Training
The above results suggest that we should use large value of p and q in the loss (a.p(u) and lan.g to guarantee a learned solution u is close to u" for high-dimensional HJB problems. Note that norm and norm behave similarly when p large. We can substitute norm by T norm and directly optimize (u) and oo (u). Overall, the training objective can be formulated as:
min (u) sup [C(()) (o(x) + A sup [Bu(x) (b2)),
zEQ
zeó0
where À > O a hyper-parameter to trade off the two objectives.
It is straightforward to obtain that setting p and q to infinity satisfies the conditions in Theorem 4.3, and thus the quality of the learned solution enjoys theoretical guarantee. Furthermore, Eq. (8) can be regarded as min-max optimization problem. The inner loop is a maximization problem to find data points on Q and a2 where u violates the PDE most, and the outer loop minimization problem to find u (i.e., the neural network parameters) that minimizes the loss on those points.
In deep learning, such min-max optimization problem has been intensively studied, and adversarial training is one of the most effective learning approaches in many applications. We leverage adversarial training, and the detailed implementation is described in Algorithm 1. In each training step, the model parameters and data points are iteratively updated. We first fix the model u and randomly sample data points xz(1
(Ni) E and f(1) ... (Na) E aΩ, serving as a random initialization of the inner loop optimization. Then perform gradient-based methods obtain data points with large point-wise Physics-Infool Informed Losses, which leads to the following inner-loop update rule:
x (k) +- Projects
(2(()) + Vign (Cuo(al(i) vo(x(()))
(9)
(k) + Projectan
((() + in
(Buo( (6(
(10)
Algorithm 1 7. Training for Physics-Informee Neural Networks
Input:
Target PDE (Eq. (1)); neural network ug; initial model parameters 0 Output: Learned PDE solution Ng
Hyper-parameters: Number of total training iterations M; number of iterations and step size of inner loop K, 17, weight for combining the two loss term À
1: for i 1,... M do
2: Sample (1)
x. (N1)
E Ω and (()).... (N2) E an
3: for j 1.... K do
4:
for = 1,... N₂ do
(kk) Projecta (2()) + signnV ( (Lua(x(()) - -p((z()))) )?
5:
6:
for k 1, ..... N₂ do
(k) + Projectan (5(()) + sign (ī(K)) (((((1)))
7:
No
N₂
1
(z) - p(((60)) +1
(Bu) )(Z())))
8: - V
N₂
N₂
t=1
9:
+ Optimizer (0,g)
10: return up

---

Table 1: Experimental results of solving the 100/250-dimensional LQQ control problems. denotes the dimensionality the problem. Performances are measured by I,", I,2, and WI. relative error in [0, 1 x [0. 77. The best performances are indicated in bold.
Relative enror for 100
Relative error for
250
Method
L W
L'
L
W1.1
Original PINN [28]
3,47%
4.25%
11.31%
6.74%
7.67%
17.51% %
Adaptive time sampling [35]
3.05%
3.67%
13.63%
7.18%
7.91%
18.38%
Learning rate annealing [34]
11.09%
11.82%
33.61%
6.94%
8.04%
18.47%
Curriculum regularization [17]
3.40%
3.91%
9.53%
6.72%
7.51%
17.52%
Adversarial training (ours)
0
0.33%
2.22%
0.95% %
1.18% %
4.38%
where Project, (+) and Project (.) project the updated data points the domain. When the inner-loop optimization finishes, we fix the generated data points and calculate the gradient to the model parameter:
N1 Σ
g
(Cuoax!)) -
1 2₂
Σ
-Baa()
(11)
N1
then the model parameter can be updated using any first-order optimization methods. When the training finishes, the learned neural network will converge to a solution with small L losses and is guaranteed be close the exact solution.
Experiments
In this section, we conduct experiments to verify the effectiveness of our approach. Ablation studies on the design choices and syper-parameters are then provided. Our codes are implemented based on PyTorch [23]. All the models are trained on one NVIDIA Tesla V100 GPU with 16GB memory. Due to space limitation, we only showcase our methods on the Linear Quadratic Gaussian control problem in the main body of the paper. More experimental results other PDE problems can be found in Appendix G.
6.1 High Dimensional Linear Quadratic Gaussian Control Problem
We follow [13] to study the classical linear- quaar--u Gaussian (LQG) control problem in n dimen- sions, special case of the HJB equation:
[ru((, t) Au(a, t) - MI| u(x, t) (u(x, T)) = g(x)
x x RER [0
(12)
x R"
As is shown in [13], there is a unique solution to Eq. (12):
u(x, t) = In (f (2m) (2m) 222 11>11/2 po(a- /2(T (2(T-
(13)
We set = 1, 1, 1, and the terminal cost function g(x) = In 11 |
Experimental Design The neural network used for training a 4-layer MLP with 4096 neurons and tanh activation in each hidden layer. To train the models, we use Adam as the optimizer [15]. The learning rate set to 4 in the beginming and then decays linearly to zero during training. The total number of training iterations is set to 5000/1000 10000 for the
problem. In each training iteration, we sample N₁ = 100/50 points from the domain R x (0, T] and N₂ = 100/50 problem.
points from the boundary R" x {T) to obtain mini-batch for the
The number of inner-loop iterations K set to 20, and the inner-loop step size 7 set to 0.05 unless otherwise specified. Evaluations are performed on a hold-out validation set which is unseen during

---

Larnned solution u (L? PINN)
Error [u-u*] (L PINN)
10
0.12
0.8
0.8
0.10
0.5
4.464
0.6
0.08
Exact solution
1.0
4,600
4.462
0.04
0.8
4.598
0.6
4.596
0.2
1.0
-4,460
0
0.4
1.0
1.0 eeamed solution (Ours)
Error [u-u*| (Ours)
4.600
0.12
0.8-
0.8
0.10
4.598
0
0.2
0.4
0.6
0.8
1.0
4.590
0.6
0.6
0.08
0.04
0 0.2
0 0.4 1.0
Figure 1: Visualization for the solutions of Eq. (12). The left panel shows the exact solution u"; the middle panel shows the learned solutions u of the original PINN method with 1 loss and our method with adversarial training; the right panel show the point-wise absolute error |u u"l: Note that the solution is a high dimensional function, and we visualize its snapshot on a two-dimensional domain. Specifically, we visualize a bivariate function u( 22, 0, 0) for I 22 [0, with the horizontal axis and vertical axis corresponding to æ and respectively.
training. We use the L.1, 1,2, and W relative enror in 00, 1j x [0. 7] as evaluation metrics: L and T,2 relative errors are popular evaluation metrics literature. We additionally consider W1.1 relative error since the gradient of the solution to HJB equations plays an important role in applications, and our theory indicates that Eq. (12) is (L L w stable. More detailed descriptions of the experimental setting and evaluation metrics can be found in Appendix F.
We compare our method with few strong baselines: 1) original PINN trained with L Physics- Informed Loss [28]; 2) adaptive time sampling for PINN training proposed in [35]; 3) PINN with the learning rate annealing algorithm proposed [34]; 4) curriculum PINN regularization proposed in [17]. The training recipes for the baseline methods, including the neural network architecture, training iterations, the optimizer, and the learning rate, are the same as those of our method described above. It should be noted that although these approaches modifies the data sampler, training algorithms or the loss function, they all keep the 7,2 norm of the PDE residual and boundary residual unchanged in the training objective.
Experimental Results The experimental results are summarized in Table 1. It's clear that the relative егror of the model trained using the original PINN does not fit the solution well, c.g., the L' relative error larger than 6% when 250. This empirical observation aligns well with our theoretical analysis, i.e., minimizing L loss cannot guarantee the learned solution to be accurate. Advanced methods, e.g., curriculum PINN regularization [17], can improve the accuracy of the learned solution but with marginal improvement, which suggests that these methods do not address key limitation of PINN in solving high-dimensional HJB Equations. By contrast, our proposed method significantly outperforms all the baseline methods in terms of both T,2 relative error and Sobolev relative error, which indicates that both the values and the gradients our learned solutions more accurate than the baselines.
We also examine the quality of the learned solution u(x, t) by visualization. As the solution is high-dimensional function, we visualize its snapshot on two-dimensional space. Specifically, we consider bivariate function u(21, O.... 0) and use a heatmap to show its function value given different 21 and Z2. Figure 1 shows the ground truth u", the learned solutions u of original PINN and our method, and the point-wise absolute error (u - u²1 for each methods. The two axises correspond and 22, respectively. We can see that the point poin error of the learned solution using our algorithm less than 2e on average. contrast, the point-wise error of the learned solution using original PINN method with 2 loss is larger than 1.3e 1 for most areas. Therefore,

---

Learned [0.u] (L? PINN)
Gradient norm error (L"
PINN)
0.8
0.013
0.8
0.012
0.6
0.6
Exact |V,0*
0.011
0.004
1.0
0.010
0.002
0.8
0.010
0.6
0.008
0
0.2
1.0
0.4
1.0
0.006
Learned [V,u] (Ours)
Gradient error (Ours)
0.004
0.002
0.8
0.010
0.8
0
0.2
0.4
0.6
0.8
1.0
0.6
0.6
0.006
0.002
0 0.2
9.004
0
0.4
1.0
Figure 2: Visualization for the gradient norm of the solutions of Eq. (12). The left panel shows the ground truth =V=""; the middle panel shows the learned V of the original PINN method with 2 loss and method with adversarial training; the right panel shows the point-wise absolute error VV (ur - u u²]]. We only visualize snapshot on a two-dimensional domain. Specifically, we visualize bivariate function u(I1, T2 ..... 0; 0) for æ1,22 [0, 1] with the horizontal axis and vertical axis corresponding to I1 and respectively.
the visualization of the solutions clearly illustrate that PINN learned more accurate solution using our proposed algorithm.
Furthermore, visualize the gradient norm V of the learned solution of both our method and the original PINN in Figure 2, to illustrate that not only can the learned solution of our method accurately approximate the exact solution, but also the gradient of the learned solution can accurately approximate the gradient of the exact solution. Again, since V / is high dimensional function, we use heatmap to show its function value given different z and I2. and set the other variables to 0. From the right panel of Figure 2, we can clearly see that the gradient of the learned solution of our method is much more accurate compared with that of the gradient of the learned solution using original PINN. The gradient norm error of the vanilla PINN approach is nearly le in some areas shown in the visualization, while the error our method is less than le 3 for most data points. This empirical observation aligns well with our theory, which states that Eq. (12) is (L -0 Wi.r stable.
6.2 Ablation studies
We conduct ablation studies the 100-dimensional LQG control problem (Eq. (12)) to ablate the main designs in our algorithm.
Adversarial training v.s. Directly optimizing I,P physic-informed loss. Our proposed algorithm introduces a min-max optimization procedure. One may have concerns that such approach may be unnecessarily complicated, and directly minimizing L physic-informer loss with a large would have the same effect. We use these two methods to solve Eq. (12), and compare their performance in the left panel of Table 2. can be seen that directly minimizing L physic-informed loss does not lead to satisfactory results.
We point out that this observation does not contradict our theoretical analysis (Theorem 4.3). Theorem 4.3 focuses on the approximation ability, which indicates that model with a small L" loss can approximate the exact solution well. The empirical results in Table 2 demonstrate difficulty of learning such model with IP loss. By comparison, our proposed adversarial training method is more stable and leads to better performance. More detailed discussions are provided in Appendix H.
optimization

---

Table 2: Experimental results for ablation studies. The left panel compares PINN trained with LP Physic Informed Loss and our method; the right panel compares PINN trained with partial or no adversarial training and method. In the first two columns of the right panel, x indicates using the original I Physics-Informed Loss for the PDE/boundary residual loss term, while / indicates using the proposed adversarial training method for the corresponding loss term, Performances measured by I.a relative eroo. Best performances are indicated in bold.
relative error (%)
Method
Relative error
Adversarial training
Relative error
K=5
7.5
Domain
Boundary
L Loss
2.42%
L Loss
53.55%
X
x
3.47%
K=10-
1.6
0.46
2.5
1 Loss
113.24%
X
2.79%
K=
0.27
8.8
2.5
Ours
0.27% %
0.27% %
n=0.00
0.05
0.1
n=
n
Adversarial training should be applied to both the PDE residual and the boundary residual. Our theoretical analysis suggests that we should use a large value of p and in the loss fg.p (u) and (u) guarantee the quality of the learned solution 1, both the data points inside the domain and the data points on the boundary are learned in the inner-loop maximization. From the right panel of Table 2, applied to one loss term, the performance slightly improved, but its accuracy is still not satisfactory. When both loss terms use adversarial training, the solution is order of magnitude more accurate, indicating that applying adversarial training to the whole loss function is essential.
Thus, in the proposed Algorithm
that when adversarial training is
Hyper-parameters K and for the inner loop maxinization. Our approach introduces additional hyppe-pprraeteen K (the number of inner-loop iterations) and 1j (the inner-loop step size). These two parameters control the accuracy of inner-loop maximization. We conduct ablation studies to examine the effects of different design choices. Specifically, we experiment with K = 5, 10, 20 and " 0.05, 0.1, 0.2, and show the L relative error in the right panel of Table Typically setting the product Kn achieves the best performance. When Kn is fixed, our results suggest that using larger and a smaller 7). i.e., more inner-loop iterations and smaller step sizes, will lead to better performance while being more time-consuming.
find that
7 Conclusions
In this paper, we theoretically investigate the relationship between the loss function and the ap- proximation quality of the learned solution using the concept of stability in the partial differential equation. We study an important class of high-dimensional non-linear PDEs in optimal control, the Banioon--acobb-Bellmaa (HJB) equation, and prove that for general L Physics-Informen HJB equation stable only if p is sufficiently large. Such a theoretical finding reveals that the widely used 2 loss is not suitable for training PINN nigh-dimensional HJB equations, while L loss is a better choice. The theory also inspires us to develop a novel PINN training algorithm to minimize the 0 loss for HJB equations in a similar spirit to adversarial training. One limitation of this work that we only work on the HJB Equation. Theoretical investigation of other important equations can be exciting direction for future works. We believe this work provides important insights into the loss design in Physics-Informed deep learning.
Loss, the
Acknowledgements
We thank Weinan E, Bin Dong, Yiping Lu, Zhifei Zhang, and Yufan Chen for the helpful discussions.
This work is supported by National Science Foundation of China (NSFC62276005), The Major Key Project of PCL. (PCL2021A12), Exploratory Research Project of Zhejiang Lab (No. 2022RCOAN02), and Project 2020BD006 supported by PKUBaidu Fund.
10

---

References
[1] Christian Beck, Sebastian Becker, Patrick Cheridito, Arnulf Jentzen, and Ariel Neufeld. Deep splitting method for parabolic pdes. SIAM Journal on Scientific Computing, 43(5):A3135 A3154, 2021.
[2] Richard Bellman. Dynamic programming. Science, 153(3731):34 1966.
[3] Dimitri Bertsekas and Steven E Shreve. Stochastic optimal control: the discrete-time case, volume 5. Athena Scientific, 1996.
[4] Michael Christ, James Colliander, and Terrence Tao. Asymptotics, frequency modulation, and low regularity ill posedness for canonical defocusing equations. American journal of mathematics, (5(66):135-12299 2003.
[5] Ashley Davey and Harry Zheng. Deep learning for constrained utility maximisation. Methodol- ogy and Computing in Applied Probability, 24(2):661-69) 2022.
[6] Tim De Ryck, Ameya D Jagtap, and Siddhartha Mishra. Error estimates for physics informed neural networks approximating the navier-stokes equations. arXiv preprint arXiv:2203. 093466 2022.
[7] Yu Deng and Nader Masmoudi. Long time instability of the couette flow in low gevrey spaces. arXiv preprini arXiv: 1803.00400 2018. 2018.
[8] Lawrence C Evans. Partial differential equations. Graduate studies in mathematics, 19(4):7,
1998.
[9] Wendell H Fleming and Raymond W Rishel. Deterministic and stochastic optimal control, volume 1. Springer Science & Business Media, 2012.
[10] Peter A Forsyth, J Shannon Kennedy, Shu Tong Tse, and Heath Windeliff. Optimal trade execution: mean quadratic variation approach. Journal of Economic dynamics and Control, 36(12):1971-1991, 2012.
[11] David Gilbarg, Neil Trudinger, David Gilbarg, and NS Trudinger. Elliptic partial differential equations of second order, volume 224. Springer, 1977.
[12] Igor Halperin. Distributional offline continuous-timg time reinforcement learning with neural physics- informed pdes (sciphy rl for doctr-l). arXiv preprint arX1v:2104.010400 2021.
[13] Jiequn Han, Arnulf Jentzen, and E Weinan. Solving high nigh-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 155(3 8510, 2018.
[14] Yuehaw Khoo, Jianfeng Lu, and Lexing Ying. Solving parametric pde problems with artificial neural networks. arXiv preprint (tt.://777.335/// 2017.
[15) Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015.
[16] Donald E Kirk. Optimal control theory: an introduction. Courier Corporation, 2004.
[17] Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34, 2021.
[18] Weiwei Li, Emanuel Todorov, and Dan Liu. Inverse optimality design for biological movement systems. IFAC Proceedings Volumes, ((1):9662-966) 2011.
[19] Gary M Lieberman. Second order parabolic differential equations. World scientific, 1996.
[20] Zhiwu Lin and Chongchun Zeng. Inviscid dynamical structures near couette flow. Archive for rational mechanics and analysis, 107 2011.

---

[21] Yiping Lu, Haoxuan Chen, Jianfeng Lu, Lexing Ying, and Jose Blanchet. Machine learning for elliptic pdes: Fast rate generalization bound, neural scaling law and minimax optimality. arXiv preprini arXiv:21 10.06897, 2021.
[22] Robert C Merton. Optimum consumption and portfolio rules in a continuous-time model. In Stochastic optimization models in finance, pages 621 261. Elsevier, 1975.
[23] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32:8026-8037, 2019.
[24] Mareus Pereira, Ziyi Wang, Tianrong Chen, Emily Reed, and Evangelos Theodorou. Feynman- kac neural network architectures for stochastic control using second-order fbsde theory. In Learningfo Dynamics and Control, pages 728-738. PMLR, 2020.
[25] Marcus Pereira, Ziyi Wang, loannis Exarchos, and Evangelos Theodorou. Learning deep stochastic optimal control policies using forward-backward sdes. arXiv preprint 1rXiv:190..03986, 2019.
[26] Huyên Pham. Continuous time stochastic control and optimization with financial applications, volume 61. Springer Science & Business Media, 2009.
[27] Huyen Pham, Xavier Warin, and Maximilien Germain. Neural networks- backward scheme for fully nonlinear pdes. SN Partial Differential Equations and Applications, 2(1):1-24, 2021.
[28] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
[29] Tim De Ryck, Ameya D. Jagtap, and Siddhartha Mishra. Error estimates for physics informed neural networks approximating the navier-stokes equations. CoRR, abs/2203.09346, 2022.
[30] Alexander Schied and Torsten Schöneborn. Risk aversion and the dynamics of optimal liquida- tion strategies in illiquid markets. Finance and Stochastics, 13(2):181-20)) 2009.
[31] Yeonjong Shin, Zhongqiang Zhang, and George Em Karniadakis. Error estimates of residual minimization using neural networks for linear pdes. arXiv preprint S.XX::2000000011 2020.
[32] Stanislaw Sieniutycz. Gamilton-jacobee ballbi-bellmamn bellman framework for optimal control in multistage energy systems. Physics Reports, 326(4):165 165-258, 2000.
[33] Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339-1364 2018.
[34] Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informe neural networks. SIAM Journal Scientific Computing, 4365):A3055-A3081 2021.
[35] Colby L Wight and Jia Zhao. Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks. arXiv preprint (tte:20000.05442 2020.
[36] Jan Willems. Least squares stationary optimal control and the algebraic riccati equation. IEEE Transactions on automatic control, 16(6):621-633 1971.
[37] William M Wonham. On matrix riccati equation of stochastic control. SIAM Journal on Control, 6(4):681-697) 1968.
[38] Jiongmin Yong and Xun Yu Zhou. Stochastic controls: Hamiltonian systems and HJB equations, volume 43. Springer Science & Business Media, 1999.
[39] Yajie Yu, Bernhard Hientzsch, and Narayan Ganesan. Backward deep bsde methods and applications to nonlinear problems. arXiv preprint arXiv:2006.0760 2020.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No] We don't see any potential negative societal impacts of this work.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] The proofs are provided in the appendix.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "yIgmk8k0DJ", "reviewer": "Reviewer_hCTD", "summary": "The paper questions the status quo of using L2-loss for optimizing PINNs. Theoretical results show that a learned solution to HJB is (Z1, Z2, Z3)-stable iff the physics-informed loss uses Lp loss with sufficiently large p. Empirical results partially support the theory and show that optimizing L-inf loss will result in lower L1 error in the case of a linear LQG problem. As L-inf loss is challenging to optimize with Adam, the paper proposes an adversarial training algorithm that indeed outperforms optimization of L-inf with Adam. \n", "questions": "Results and Impact:\n1) The authors evaluate the approximation quality of their method on L1 error (Table 1). What is the justification for only using L1 to empirically define a \"good approximator of the exact solution\" (L34)? Indeed, the proposed L-inf loss results in a lower L1 error than L2 loss on a high-dim. linear LQG problem. But, it is unclear to me if L-inf loss will also result in a lower L2 error. I would be willing to raise my score with a comparison of L2 error and a discussion of it. \n2) Can a general statement be made that L-inf loss will be a more appropriate loss choice for high-dimensional problems?\n\nRelated Works:\n\n3) What is the broader novelty wrt. prior works that use deep learning for HJB? The related works state that there exists \"several works\" that solve HJB with deep learning, yet the authors only mention two works [L82-90]. A quick Google shows that there is other works by, e.g., Nüsken and Richter, 2021; \n4) There exist works that use GANs instead of L2 loss. What is the advantage of using L-inf over the GAN loss?\n5) Is (Z1, Z2, Z3)-stable a new concept developed in this paper or does it already exist in the literature? If so, what is the citation for (Z1, Z2, Z3)-stable.\n6) It is confusing to me that authors use the word 'stable' to talk about approximation quality in Sec 4. More specifically, how does (Z1, Z2, Z3)-stable relate to the general definition of stability in PDEs [L93-95]? An \"equation is stable if the solution of the perturbed PDE converges to the exact solution as the perturbations approach zero [6]\"\n", "limitations": "- There is no discussion of limitations nor of negative societal impacts.\n- It would be very helpful if the limitation section could list and analyze the assumptions in Definitions/Theorems 4.1-4.4 and discuss what the extra steps would be to adapt the Definitions to other classes of PDEs beyond HJB.\n- Are the theorems truly applicable to *all* variations of the HJB equations?\n- What does (Z1,Z2,Z3)-stable mean and what does it *not* mean?\n- It would be helpful if the authors mention that, while the work is very theoretical, it could be used to improve numerical modeling of applications that violate the NeurIPS ethical guidelines.  \n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- The work is significant. The broader field of learning surrogate models of PDEs can have wide-reaching impacts in physics, chemistry, fluid dynamics, biology, climate modeling and more. Within the field of phyisics-informed machine learning, the authors have identified and addressed a very important research question: Does there exist theorems that connect properties of PDEs to the optimal choice of PINN loss function. The choice of HJB equations is important and developing the theory on HJB equations only should be sufficient for acceptance. \n- The authors provide rigorous and sound theory to prove that a learned solution to HJB is (Z1, Z2, Z3)-stable iff the physics-informed loss uses Lp loss with sufficiently large p.\n- The authors clearly state the research question [L34-35a] and include necessary background math, e.g., Def. 4.2.\n\nWeaknesses: \n- One of the authors main contributions seems to be the notion of (Z1, Z2, Z3)-stable. However, it is still unclear to me 1) if (Z1, Z2, Z3)-stable actually is a novel concept that the authors came up with, 2) what the concept can and cannot be used for, and 3) how the concept relates the 'stability' in PDEs (see Qestuions 5-6). \n- It is challenging to fully evaluate originality of the work, because the related works section is very sparse. I have inluded some questions to evaluate originality and am willing to raise my score.\n- The empirical evaluation does not fully support the theorems, nor does it fully answer the research question. I have included some questions to evaluate the quality of the empirical section.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "w5Wcc7psQBQ", "reviewer": "Reviewer_a7nv", "summary": "This paper studies the choice of loss functions when using neural networks to solve partial differential equations. The authors establish positive and negative results showing the conditions under which the problem of solving a Hamilton-Jacobi-Bellman equation is stable/unstable. Based on this, an $L^\\infty$ training approach is proposed. Experiments demonstrate that the proposed method has better performance than existing methods.\n", "questions": "I suggest that the authors could address the weaknesses mentioned in the strengths and weaknesses section.", "limitations": "This paper does not have any potential negative societal impact.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\nThe results of this paper are solid and comprehensive. \n\nThe paper is well-organized and easy to follow. \n\nThe experiments clearly demonstrate the advantage of the proposed $L^\\infty$ loss over the original PINN methods.\n\n\n\nWeaknesses: \n\nThis work focuses on a quite specific class of HJB equations and requires $\\bar{c} \\leq 2$ for the stability result to hold, which seems to require $\\alpha_i \\geq 2$ in the cost rate function.\n\nTable 2 in the ablation studies seems to contradict the theorem results. Some more discussions may be quite helpful here to explain why finite but larger values of p lead to worse performances.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DTrMs5iM7Mq", "reviewer": "Reviewer_bGoA", "summary": "Te present work is concerned with the application of physics-informed neural networks (PINNS) to the solution of high-dimensional Hamilton-Jacobi-Bellman equations (HJB). They show that in a wide range of cases directly applying the standard squared penalty to HJB and boundary conditions does not result in a stable problem, meaning that the PINN loss can go to zero without the PINN solution approximating the true solution. In order to overcome this problem, the authors propose to use the higher-order $L^p$ penalties. They show that higher-order $L^p$ penalties achieve stability under a wider range of conditions, with $L^\\infty$ being the extreme case. They then design an adversarial training procedure to train with the $L^\\infty$ loss in practice and show that it improves the relative accuracy of the solution compared to existing PINN variants. \n\n### After rebuttal:\nThe authors have addressed my concerns appropriately. I now recommend acceptance. ", "questions": "1. In the theorems 4.3 and 4.4, the target accuracy is measured in the $W^{1,r}$ norm, meaning in terms of the $L^r$ norm of both the solution and its gradient. However, in the experiments, the error is measured in terms of just the $L^p$ norms of the solution. Thus, experiments do not actually seem to illustrate the theoretical results as advertised.\n\n2. Following up on 1, are similar results true when measuring solution accuracy in $L^p$ spaces (instead of sobolev spaces)? If not, the authors should provide more motivation accuracy in sobolev spaces is important. \n\n3. Did you find that different learning rate schedules had an effect on the accuracy of Adam? \n\n4. The error plots of Figure 4 are not very meaningful instead, or complementing them I think it would be important to show learning to show time series of the training and test error. This would allow to distinguish difficulties of the finite batch setting to accurately approximate the infinite-batch-training loss from difficulties in optimizing the finite batch loss, from the problem of stability that is motivating the author's technique.  \n ", "limitations": "As discussed above under \"questions\", there presently seem to be gaps between theory and experiments that are not commented on in the text.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The method is well-motivated and seems to perform well in practice on the problem of interest.\n\nWeakness: As detailed under \"questions\", the present version of the paper leaves doubts regarding some of the claims. The experimental validation seems to be in a setting not covered by the theory and the most common diagnostics, such as error vs time, are missing.\n\nI want to emphasize that I think that this has the potential to be a solid paper suitable for acceptance. I am giving a somewhat lower score right now but hope to be able to raise it if the authors can clarify my concerns.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1WLC2q3EYV_", "reviewer": "Reviewer_RgXD", "summary": "This paper studies the objective function in physics-informed learning for HJB equations. Authors propose the concept of stability to analyze what is a good choice of loss function. The theoretical result challenges the common practice of $L^2$ and suggests that $L^p$ with large $p$ gives stability guarantee. The authors propose an adversarial training algorithm to minimize $L^{\\infty}$ loss based on their theoretical finding.  The proposed method demonstrates superior empirical performance in the simulated high-dimensional problem.", "questions": "1. In the stochastic differential equation given by Eq (4), the output of control function has the same dimension as the state. Also, the control function only depends on time. But I think the control variable usually has different dimension from the state variable and the control function should be a function of both time and state. The setting described in Eq (4) seems too simple compared to what I found in the literature equation 2.10 in section 2.4 [1].  Is this a standard setting in optimal control? \n2. Given the results in Table 2 and what authors said in page 5 -\"$L^p$ and $L^{\\infty}$ -norm behave similarly when $p$ is large\", it seems that minimizing the exact $L^{\\infty}$ loss may have poor performance. The reason why the adversarial training works better than $L^2$ seems to be that adversarial training fails to approximate $L^{\\infty}$ -norm well. Is it possible to check \n3. Given the training objective in Eq (8), shouldn't the objective functions in algorithm line 5,7,8 use absolute value instead of $L^2$? \n\n[1] Lu, Q., and Xu Zhang. \"A mini-course on stochastic control.\" _Control and inverse problems for partial differential equations_ 22 (2016): 171-254.", "limitations": "Listed in the weakness and question sections. ", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper is well-motivated and well-written. Stability analysis is important for using physics-informed learning to solve PDEs. \n2. The theoretical results advance the understanding of loss design in physics-informed learning.  \n3. The proposed method demonstrates superior empirical performance compared to other baselines in the simulated high-dimensional problem. \n\nWeakness:\n1. Adversarial training is time consuming and it introduces more hyperparameters to tune. \n2. The experiment only considers two special cases (including the one in the appendix) of HJB equations which have close form solutions. The one in the appendix doesn't compare with other baselines. The empirical evidence is not strong enough to me. \n3. The theoretical results suggest Lp loss with a large p guarantees better stability than L2. However, the results in Table 2 turn out to be the opposite. Explanation in the Section 6.2 seems vague. \n4. The setting seems restricted. More details are provided in the questions. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept