# Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks
## 5 Discussion  
_Our results:_ In this paper, we have presented a series of illustrative numerical experiments that aim to provide a glimpse of the potential of Fed-DeepONets for learning the solution operator of complex dynamical systems in a distributed manner. We have shown that the Adaptive Fed-DeepONet is robust to several scenarios, including time-dependent available clients or a different number of clients. We remark, however, that depending on the application, one must carefully select the hyper-parameters used for describing federated learning (e.g., \(K\) or \(R\)) or the dynamical system under study. For instance, one may obtain the parameter values for Fed-DeepONets using a hyper-parameter optimization routine that respects the underlying constraints of the problem.  
_The limitations of Fed-DeepONet:_ The proposed Fed-DeepONet framework has the following limitations. (i) Fed-DeepONet cannot assimilate data in real time. In other words, the federated training of DeepONets must be performed offline. This limitation is a direct consequence of using batch stochastic gradient descent and its variants for neural network training. One may use a distributed version of Kalman Filters to alleviate this limitation. However, training neural network parameters with Kalman Filters is a challenging task. (ii) While Fed-DeepONet protects data privacy, it may not protect the client's information. A skillful adversary may infer information about the client's process from observing the parameters transmitted to the central server. A simple solution to alleviate this limitation may be to keep some of the parameters private. (iii) Fed-DeepONet may fail to optimize multi-objective loss functions or constrained problems. This is another limitation inherited from using stochastic gradient descent for unconstrained optimization. Two possible  
Figure 8: Fed-DeepONet \(G_{\theta^{*}}\) prediction of three solution trajectories driven by three out-of-distribution inputs: (**left**) \(u(t)=t\), (**middle**) \(u(t)=\sin(\pi t)\), and (**right**) \(u(t)=t\sin(2\pi t)\) and parameter \(k\) sampled uniformly within the interval \([0.5,1.5]\). The computed \(L_{2}\)-relative errors are: (**left**) \(e_{L_{2}}=1.726\%\), (**middle**) \(e_{L_{2}}=0.419\%\), and (**right**) \(e_{L_{2}}=10.012\%\).  
Figure 7: Fed-DeepONet \(G_{\theta^{*}}\) prediction of three solution trajectories driven by three inputs generated using a Gaussian Random Field but not included in the training dataset, and the parameter \(k\) sampled uniformly within the interval \([0.5,1.5]\). The computed \(L_{2}\)-relative errors are: (**left**) \(e_{L_{2}}=0.442\%\), (**middle**) \(e_{L_{2}}=1.252\%\), and (**right**) \(e_{L_{2}}=1.561\%\).  
solutions are as follows. One may use a penalty method to transform the problem into an unconstrained one. On the other hand, one may design a federated evolutionary strategy that can handle the underlying multi-objective problem.  
_Our future work:_ First, we aim to employ Fed-DeepONet to build digital twins for (i) distributed renewable energy resources and (ii) city-wide predictive models for the transmission of contagious diseases (e.g., COVID-19). Then, we will apply Fed-DeepONet to accelerate material science by enabling training with high-performance distributed and parallel computing platforms. Finally, we aim to extend our federated optimization strategy to a federated sampling strategy that enables quantifying uncertainty for DeepONets.