# Solving Partial Differential Equations Using Deep Learning and Physical Constraints
## 2 Methodology
### Physics-Informed Neural Networks  
In this section, we introduce the physics-informed neural networks (PINNs) and related settings in this study. Traditional neural networks are based entirely on a data-driven approach that does not take into account the physical laws that are contained in the data. Therefore, a large amount of data is often required to train the neural networks to obtain a reasonable model. In contrast, physics-informed neural networks introduce physical information into the network by forcing the network output to satisfy the corresponding partial differential equations. Specifically, by adding regularization about partial differential equations to the loss function, the model is made to consider physical laws during the training process. This processing makes the training process require less data and speeds up the training process. Physics-informed neural networks can be used to solve not only the forward problem, i.e., obtaining approximate solutions to partial differential equations, but also the inverse problem, i.e., obtaining the parameters of partial differential equations from training data [29; 36; 42; 43]. In the following, the physical-informed neural network modified and used in this study is introduced for the forward problem of partial differential equations. In this study, consider the partial differential equation defined on the domain \(\Omega\) with the boundary \(\partial\Omega\). \[\mathcal{D}(\mathbf{u}(\mathbf{x}))=\mathbf{0}\quad\mathbf{x}\in\Omega \tag{1}\]  
\[\mathcal{B}(\mathbf{u}(\mathbf{x}))=\mathbf{0}\quad\mathbf{x}\in\partial\Omega \tag{2}\]  
where \(\mathbf{u}\) is the unknown solution and \(\mathcal{D}\) denotes a linear or nonlinear differential operator (_e.g._, \(\partial/\partial\mathbf{x},\mathbf{u}\circ\partial/\partial\mathbf{x},\mathbf{u}\circ\partial ^{2}/\partial\mathbf{x}^{2},etc.\)), and the operator \(\mathcal{B}\) denotes the boundary condition of a partial differential Equation (e.g., Dirichlet boundary condition, Neumann boundary condition, Robin boundary condition, etc.). A point to note is that, for partial differential equations that contain temporal variables, we treat \(t\) as a special component of \(\mathbf{x}\), i.e., the temporal domain is included in \(\Omega\). At this point, the initial condition can be treated as a special type of Dirichlet boundary condition on the spatio-temporal domain. First, we construct a neural network for approximating the solution \(u(\mathbf{x})\) of a partial differential equation. This neural network is denoted by \(\hat{u}(\mathbf{x};\mathbf{\theta})\), which takes the \(\mathbf{x}\) as input and outputs a vector of the same dimension as \(u(\mathbf{x})\). Suppose this neural network contains an input layer, \(L-1\) hidden layers and an output layer. Specifically, each hidden layer in the neural network receives the output from the previous layer and, in the \(k^{th}\) hidden layer, there are \(N_{k}\) number of neurons. \(\mathbf{\theta}\) is used to represent the neural network parameters, containing the collection of weight matrix \(W^{[k]}\in\mathbf{R}^{n_{k}\times n_{k-1}}\) and bias vector \(b^{[k]}\in\mathbf{R}^{n_{k}}\) for each layer \(k\) with \(n_{k}\) neurons. These parameters will be continuously optimized during the training phase. The neural network \(\hat{u}\) should satisfy two requirements: on the one hand, given a dataset of \(u(\mathbf{x})\) observations, the network should be able to reproduce the observations when \(\mathbf{x}\) is used as input and, on the other hand, \(\hat{u}\) should conform to the physics underlying the partial differential equation. Thus, we next fulfill the requirements of the second part by defining a residual network. \[f(\mathbf{x};\mathbf{\theta}):=\mathcal{N}[\hat{u}(\mathbf{x};\mathbf{\theta})] \tag{3}\]  
To build this neural network, we need to use automatic differentiation (AD). Currently, automatic differentiation techniques have been widely integrated into many deep learning frameworks, such as Tensorflow [44] and PyTorch [45].