# Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks
## 2 Background Information  
Let us consider the problem of approximating a nonlinear operator \(G\) from data. Operator \(G\) maps two infinite-dimensional spaces, i.e., it maps an input function \(u\) to an output function \(G(u)\). The input \(u\) may (i) represent, for example, a forcing term, a control input, or a parametric realization and (ii) belong to the space of continuous or measurable functions. Let \(y\in Y\) denote a query location within the output domain \(Y\). Then, our goal in this paper is to employ a Deep Operator Network (DeepONet) \(G_{\theta}\) with trainable parameters \(\theta\in\mathbb{R}^{s}\) to learn from data nonlinear operators \(G(u)(y)\) at query locations \(y\in Y\).  
As a proof of concept, in our numerical experiments (see Section 4), we will study the problem of learning the solution operator \(G\) arising when solving a complex dynamical system described via the initial value problem (IVP):  
\[\begin{split}\frac{d}{dt}x(t)&=f(x(t),u(t);\lambda),\qquad t\in[0,T],\\ x(0)&=x_{0},\end{split} \tag{1}\]  
where \(x(t)\in\mathbb{R}^{d}\) is the vector-valued state function, \(u(t)\in\mathbb{R}^{m}\) is an input function, \(\lambda\in\mathbb{R}^{p}\) is a parameter realization, and \(f:\mathbb{R}^{d}\times\mathbb{R}^{m}\to\mathbb{R}^{d}\) is the unknown vector field. The solution operator of (1), i.e., \(G:u\mapsto G(u)\equiv x\), is given by  
\[G(u)(t)=x_{0}+\int_{0}^{t}f(G(u)(s),u(s);\lambda)ds,\qquad t\in[0,T]. \tag{2}\]  
Let us now review the traditional centralized strategy for training DeepONets.  
_Centralized Training of DeepONets_  
To approximate \(G\), DeepONet [10] employs a neural network architecture (see Figure 1) that uses two subnetworks: the Trunk network and the Branch network.  
The Branch network processes the input function \(u\) information. Let \((x_{1},\ldots,x_{m})\) be the collection of \(m\) interpolation points \((x_{1},\ldots,x_{m})\) (known as sensors in the original paper [10]) that enable us to discretize/encode the input function, i.e., \(u_{m}:=(u(x_{1}),\ldots,u(x_{m}))\). The Branch network maps this discretized input \(u_{m}\) to a vector of trainable coefficients \(b\in\mathbb{R}^{q}\). On the other hand, the Trunk network processes the query location \(y\in Y\). To this end, the Trunk network maps the query location \(y\) to the following collection of trainable basis functions:  
\[\varphi:=(\varphi_{1}(y),\ldots,\varphi_{q}(y))\in\mathbb{R}^{q}.\]  
The output of DeepONet is then computed by fusing the trainable coefficients \(b\) with the trainable basis functions \(\varphi\) using the following dot product:  
\[G_{\theta}(u_{m})(y):=\sum_{i=1}^{q}b_{i}\cdot\varphi_{i}(y). \tag{3}\]  
The traditional DeepONet \(G_{\theta}\) centralized training strategy collects the following training dataset from possibly distributed locations:  
\[\mathcal{D}_{\text{cent}}:=\Big{\{}u_{m}^{(k)},y^{(k)},G(u_{m}^{(k)})(y^{(k)} )\Big{\}}_{k=1}^{N}.\]  
Then, at a centralized location, one trains the DeepONet parameter \(\theta\) by minimizing (e.g., using the gradient-based Adam [12] algorithm) the following mean squared loss function:  
\[\mathcal{L}(\theta)=\frac{1}{N}\sum_{k=1}^{N}\Big{\|}G_{\theta}(u_{m}^{(k)})( y^{(k)})-G(u_{m}^{(k)})(y^{(k)})\Big{\|}_{2}^{2}\]  
over the centralized dataset \(\mathcal{D}_{\text{cent}}\). Such a centralized training strategy may (i) fail to protect the privacy of distributed clients, e.g., when learning digital twins for engineering applications, or (ii) prevent using high-performance parallel and distributed computing frameworks for scientific computing applications. To alleviate the aforementioned limitations of the centralized strategy, we propose, in the following section, a federated/distributed training strategy for DeepONets, which we refer to as Fed-DeepONet.