# Solving Partial Differential Equations Using Deep Learning and Physical Constraints
## 3 Experiments and Results
### KdV-Burgers Equation
The neural network contains seven hidden layers, each containing 120 neurons, and the hyperbolic tangent tanh is chosen as the activation function. Besides, the physical information neural network \(f(t,x)\) is constructed using the automatic differentiation of the TensorFlow to introduce control information of the equations. Specifically, the physics-informed neural network \(f(t,x)\) is defined, as follows  
\[f(t,x):=u_{t}+uu_{x}-0.075u_{xx}+\pi/1000u_{xxx} \tag{19}\]  
The next main task is to train the parameters of the neural network \(\hat{u}(t,x)\) and \(f(t,x)\). We continuously optimize the parameters by minimizing the mean square error loss to obtain the optimal parameters. \[J(\mathbf{\theta})=MSE_{u}+MSE_{f} \tag{20}\]  
where  
\[MSE_{u}=\frac{1}{N_{u}}\sum_{i=1}^{N_{u}}\left|u\left(t_{u}^{i},x_{u}^{i} \right)-u^{i}\right|^{2} \tag{21}\]  
\[MSE_{f}=\frac{1}{N_{f}}\sum_{i=1}^{N_{f}}\left|f\left(t_{f}^{i},x_{f}^{i} \right)\right|^{2} \tag{22}\]  
Similar to the previous experiment, \(MSE_{u}\) is a loss function constructed while using observations of the initial and boundary conditions. \(MSE_{f}\) is a loss function that introduces physical information of partial differential equations. Specifically, \(\{t_{u}^{i},x_{u}^{i},u^{i}\}_{i=1}^{N_{u}}\) corresponds to the initial and boundary condition data, \(u(t_{f},x_{f})\) and \(\left\{i^{i}_{f^{i}},x^{i}_{f}\right\}_{i=1}^{N_{f}}\) corresponds to the data in the space-time domain. In this experiment, the training data are also randomly selected from the generated dataset, with the number of training data \(N_{f}\) in the space-time domain being 30,000 and the number of training data \(N_{u}\) in the initial and boundary conditions being 400. We use the Glorot normal initializer for initialization. During optimization, we set the learning rate to 0.001 and, to ensure global convergence and speed up the convergence process, we ran L-BFGS for 30,000 epochs and then used Adam to continue optimizing until convergence. In this experiment, the time to train the model was approximately twelve minutes. After completing the training of the neural network model, we also tested the effects of the model. Figure 5 shows the predictions of the neural network model, and we can see that the resulting predictions are quite complex. We choose different moments of the prediction results to compare with the exact solution in order to test the accuracy of this prediction. Figure 6 shows the comparison between the exact solution and prediction at different moments \(t=0.25,0.5,0.75\). From Figure 6, it can be seen that the predictions of the neural network model and exact solutions are highly consistent, indicating that the constructed neural network model can solve the KdV-Burgers equation well. In addition, the relative L2 error of this example was calculated to be \(4.79\cdot 10^{-4}\), which further validates the effectiveness of this method. Despite the high complexity of the KdV-Burgers equation, the neural network model can still obtain results very close to the true solution from the training data, which again shows that the method has great potential and value and it is worthy of further research.