## 3 Theoretical Foundations
### Generalization Guarantees  
**Rademacher complexity analysis. ** Let the hypothesis space of PINNs be defined as \(\mathcal{H}_{W}=\{u_{\theta}:\|\theta\|\leq W\}\). The empirical Rademacher complexity of this hypothesis space satisfies:  
\[\mathcal{R}_{n}(\mathcal{H}_{W})\leq\frac{W}{\sqrt{n}}\bigg{(}\mathbb{E}\bigg{[} \sup_{x}|\
abla_{\theta}\mathcal{L}_{\text{PDE}}(x)|\bigg{]}\bigg{)} \tag{7}\]  
where \(\mathcal{R}_{n}(\mathcal{H}_{W})\) denotes the empirical risk for the hypothesis class \(\mathcal{H}_{W}\) evaluated on a sample of size \(n\). The term \(W\) represents a bound on the complexity of the model, such as the norm of the parameters in the hypothesis class. \(n\) stands for the sample size used to compute the empirical risk. The expression \(\mathbb{E}\) denotes the expectation over the distribution of the data points, and \(\
abla_{\theta}\mathcal{L}_{\text{PDE}}(x)\) represents the gradient of the loss function \(\mathcal{L}_{\text{PDE}}(x)\) with respect to the model parameters \(\theta\) at a specific point \(x\) in the domain. The \(\sup_{x}\) indicates the supremum (maximum value) of the gradient over all points \(x\), providing a measure of the maximum sensitivity of the loss function with respect to the parameters. This leads to the derivation of the generalization error bound:  
\[\mathcal{E}_{g}\leq\sqrt{\frac{\log\mathcal{N}(\mathcal{H}_{W},\epsilon,n)}{n} }+\epsilon \tag{8}\]  
where \(\mathcal{N}\) represents hypothesis space covering number, \(\mathcal{E}_{g}\) denotes generalization error, \(\mathcal{H}\) is the hypothesis space, \(\epsilon\) is a scale parameter of the cover, and \(n\) is the number of training samples [27]. **Extension of information bottleneck theory. ** The information bottleneck objective function incorporating physical constraints is given by:  
\[\min_{\theta}[I(X;U_{\theta})-\beta I(U_{\theta};\mathcal{P})] \tag{9}\]  
the objective is to minimize with respect to the parameters \(\theta\). Here, \(I(X;U_{\theta})\) represents the mutual information between the input data \(X\) and the model's output \(U_{\theta}\), where \(U_{\theta}\) denotes the solution of the neural network parameterized by \(\theta\). The term \(I(U_{\theta};\mathcal{P})\) represents the mutual information between the model output \(U_{\theta}\) and the set of physical constraints \(\mathcal{P}\), which incorporates domain-specific conditions or laws that the model must satisfy. The parameter \(\beta\) is a balancing factor that controls the trade-off between the mutual information terms. By adjusting \(\beta\), the model can balance the data fitting term \(I(X;U_{\theta})\) with the adherence to the physical constraints \(I(U_{\theta};\mathcal{P})\). Experimental results show that this framework reduces the generalization error of the Burgers equation by 37% [37]. In Figure 2, the decomposition of the generalization error of the Physics-Informed Neural Network (PINN) is presented in a pie chart format. The total error is divided into four distinct components:  
* Approximation Error (40%) arises due to the limitations in the neural network's representational capacity. Despite its capability, the network may not fully capture the complexity of the underlying physical problem, leading to an inherent error in approximating the target function. This error can be mitigated by improving the network architecture, such as increasing the depth or width of the network. * Optimization Error (30%) originates from the suboptimal convergence during the training process. Even with a sufficient model capacity, the optimization algorithm, including factors such as learning rate and initialization, might not converge to the global optimum, leading to a non-ideal solution. This error can be reduced through better optimization strategies and fine-tuning hyperparameters. * Physical Constraint Mismatch (20%) refers to the error caused by inaccuracies in the representation of the physical constraints (e.g., partial differential equations, boundaryconditions) within the model.