## 3 Methodology
### Updating Weights Adaptively for the PI_DeepONet  
In contrast to the Dirichlet boundary condition, the Neumann boundary condition is formed with partial derivative terms. Thus, the loss terms of the PI_DeepONet will become fairly complex for GDEs with multiple Neumann boundary conditions. Due to the insufficient understanding of the regular mechanism at present, PI_DeepONet has a tendency to converge to an incorrect solution when the loss terms of the PI_DeepONet are complex. Using gradient descent to update the parameters of the PI_DeepONet, the \(n\)-th step of gradient descent can be expressed as Equation (28). The gradients used to update the parameters are a weighted sum of the gradients of each individual loss. Thus, in situations where the gradients of each individual loss exhibit significant imbalance, the gradients with smaller values are more likely to be underestimated, resulting in poor fitting for the corresponding physics-driven loss. The imbalance of the gradients of each individual loss is quite serious when the PI_DeepONet has complex loss terms. As such, it is essential to employ effective measures to alleviate the imbalance among the gradients of each loss. In fact, the imbalance can be mitigated by selecting appropriate weights for each loss. However, the gradient distributions of the losses change constantly throughout the training process. Therefore, it is not feasible to establish a predetermined set of weights to maintain balanced gradient distributions during the whole training process. To address that issue, an algorithm for updating weights adaptively is proposed, as summarized in Algorithm 1. Algorithm 1 is designed to automatically adjust the weights during model training usingthe back-propagated gradient statistics. The gradient distributions of each loss will remain balanced after the updated weights are applied. ```
Consider a PI_DeepONet \(G_{\theta}(\mathbf{u})(x)\) with parameters \(\theta\) and a loss function \[L(\theta)=\lambda_{s}L_{s}(\theta)+\sum_{i=1}^{M}\lambda_{i}L_{i}(\theta)\] (29) in which \(L_{s}(\theta)\) is the base loss for the updating of weights; \(\{L_{i}(\theta)\}_{i=1}^{M}\) represents all other losses; \(\lambda\) represents the weight of each loss. Then, use \(S\) steps of a gradient descent algorithm to update the parameters \(\theta\) as follows: for\(n=1,\cdots,S\)do  1. Calculate the transit weights \(\hat{\lambda}_{i}\) as follows: \[\hat{\lambda}_{i}=\lambda_{s}\frac{\overline{|\
abla_{\theta}L_{s}(\theta_{n })|}+\overline{|\
abla_{\theta^{b}}L_{s}(\theta_{n})|}}{\overline{|\
abla_{ \theta}L_{i}(\theta_{n})|}+\overline{|\
abla_{\theta^{b}}L_{i}(\theta_{n})|}}, \ i=1,2,\ldots,M\] (30) in which \(\overline{|\
abla_{\theta}L(\theta_{n})|}\) denotes the average of the absolute values of the gradients of \(L(\theta_{n})\) with respect to parameters \(\theta\), \(\theta^{b}\) denotes all parameters of the trunk net, and \(\theta^{b}\) denotes all parameters of the branch net. 2. Update the weights \(\lambda_{i}\) using a weighted average of the following form: \[\lambda_{i}=(1-\alpha)\lambda_{i}+\alpha\hat{\lambda}_{i},\ i=1,2,\ldots,M\] (31)  3. Update the parameters \(\theta\) using the following gradient descent: \[\theta_{n+1}=\theta_{n}-\eta\lambda_{s}\
abla_{\theta}L_{s}(\theta_{n})-\eta \sum_{i=1}^{M}\lambda_{i}\
abla_{\theta}L_{i}(\theta_{n})\] (32) end  Hyper-parameter \(\alpha\) is recommended to be 0.9. ```  
**Algorithm 1**Updating weights adaptively for the PI_DeepONet  
Due to the stochasticity of the gradient descent updates, it is expected that the instantaneous values of the gradients computed above will exhibit high variance. Thus, the hyper-parameter \(\alpha\) is introduced to Algorithm 1, and the actual weights \(\lambda_{i}\) are weighted averages based on their previously calculated values. The updates of the weights in Equations (30) and (31) can occur at every iteration of the gradient descent loop or at a user-specified frequency (e.g., every 100 gradient descent steps).