Deep learning has significantly revolutionized the design of numerical algorithms for solving high-dimensional partial differential equations (PDEs) . Yet the empirical successes of such approaches remains mysterious in theory. In deep learning-based PDE solvers, solving the original PDE is formulated into an expectation minimization problemwith a PDE solution space discretized via deep neural networks . A global minimizer corresponds to a deep neural network that solves the given PDE. Typically, gradient descent-based methods are applied to minimize the expectation. This papershows that gradient descent can identify a global minimizer of the optimization problem with a well-controlled generalization error in the case of two-layer neural networks in the over-parameterization regime (i. e., the network width is sufficiently large). The generalization error of the gradient descent solution does not suffer from the curse of dimensonality if the solution is in a Barron-type space . The theories developed here could form a theoretical foundation of deep learning-based PDE solvers .
The problem of solving partial differential equations (PDEs) can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions . A global minimizer corresponds to a neural network that solves the given PDE. In this paper, we show that the gradient descent method can identify a global minimizer of the least-squares optimization for solving second-order linear PDEs with two-layer neural networks under the assumption of over-parametrization. We also analyze the generalization error of the least-squares optimization for second-order linear PDEs and two-layer neural networks, when the right-hand-side function of the PDE is in a Barron-type space and the least-squares optimization is regularized with a Barron-type norm, without the over-parametrization assumption .
R
Deep learning has significantly revolutionized the design of numerical algorithms for solving high-dimensional
The problem of solving
R
. Yet the empirical successes of such approaches remains mysterious in theory. In deep learning-based PDE solvers, solving the original PDE is formulated into an expectation minimization problemwith a PDE solution space discretized via deep neural networks
can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions
D
deep
R
Typically, gradient descent-based methods are applied to minimize the expectation. This papershows that gradient descent
In this paper, we show that the gradient descent method
R
optimization problem with a well-controlled generalization error in the case of
least-squares optimization for solving second-order linear PDEs with
R
in the over-parameterization regime (i. e., the network width is sufficiently large). The
under the assumption of over-parametrization. We also analyze the
R
gradient descent solution does not suffer from the curse of dimensonality if the solution
least-squares optimization for second-order linear PDEs and two-layer neural networks, when the right-hand-side function of the PDE
R
. The theories developed here could form a theoretical foundation of deep learning-based PDE solvers
and the least-squares optimization is regularized with a Barron-type norm, without the over-parametrization assumption