# Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks
## 3 Federated Deep Operator Networks (Fed-DeepONet)  
Let us start this section by describing some mathematical notations from federated learning [17]. We let \(\lfloor\cdot\rfloor\) denote the floor function, and we use \(\|\cdot\|_{2}\) to denote the standard  
Figure 1: The DeepONet’s architecture. The Branch (resp. Trunk) network processes the input function \(u_{m}\) (resp. query location within the output function domain \(y\in Y\)). The DeepONet’s output uses a dot product (represented via a crossed node) to fuse the trainable coefficients of the Branch’s output (\(b\in\mathbb{R}^{q}\)) with the trainable basis functions of the Trunk’s output (\(\varphi\in\mathbb{R}^{q}\)). Euclidean norm. For any given positive integer \(n\), we let \([n]\) denote the set \(\{1,2,\ldots,n\}\). We let \(\mathcal{C}\) denote the number of clients participating in the federated training of DeepONet and use variable \(c\) to describe the clients within the set of clients \([C]\). For each client \(c\in[C]\), we use \(\mathcal{L}^{c}\) and \(\
abla\mathcal{L}^{c}\) to denote the \(c\)-th client's corresponding DeepONet loss and gradient of the loss \(\mathcal{L}^{c}\). We let \(\
abla\tilde{\mathcal{L}}^{c}\) denote the stochastic gradient of \(\mathcal{L}^{c}\) computed using the \(c\)-th client's dataset. This stochastic gradient \(\
abla\tilde{\mathcal{L}}^{c}\) is an unbiased estimator of the exact gradient \(\
abla\mathcal{L}^{c}\)[25]. We also denote \(p_{c}\) as the weight of the \(c\)-th client such that \(p_{c}=\frac{n_{c}}{\sum_{c\in[C]}n_{c}}\in(0,1)\), where \(n_{c}>0\) is the number of data triplets in the \(c\)-th client. Let \(K\) denote the number of local stochastic gradient updates taken by each client \(c\in[C]\) and let \(R\) denote the number of global synchronization events (also known as the number of communication rounds). To train the proposed Fed-DeepONet, we follow a distributed optimization strategy. For traditional neural networks, this distributed strategy was introduced in [16; 18], and it is known as federated averaging (FedAvg). Formally, we aim to solve the following optimization problem:  
\[\min_{\theta\in\mathbb{R}^{s}}\ \frac{\sum_{c=1}^{C}\mathcal{L}^{c}(\theta)}{ \sum_{c=1}^{C}n_{c}}. \tag{4}\]  
In the above, \(\mathcal{L}^{c}(\theta):=\sum_{j=1}^{n_{c}}\ell\Big{(}\theta;d_{c}^{(j)}\Big{)}\), where \(\theta\in\mathbb{R}^{s}\) and \(\ell\Big{(}\theta;d_{c}^{(j)}\Big{)}\) is the \(c\)-th client's DeepONet loss function based on \(\theta\) and the data triplet \(d_{c}^{(j)}:=\{u_{m,c}^{(j)},y_{c}^{(j)},G(u_{m,c}^{(j)})(y_{c}^{(j)})\}\) collected by the \(c\)-th client, i.e.,  
\[\ell\Big{(}\theta;d_{c}^{(j)}\Big{)}=\Big{\|}G_{\theta}(u_{m,c}^{(j)})(y_{c}^ {(j)})-G(u_{m,c}^{(j)})(y_{c}^{(j)})\Big{\|}_{2}^{2}.\]  
Here, we have used the notations \(u_{m,c},y_{c}\) and \(G(u_{m,c})(y_{c})\) to emphasize that \(u_{m,c},y_{c}\) and \(G(u_{m,c})(y_{c})\) are, respectively, a discretized input, query location, and solution operator value collected by the \(c\)-th client. The proposed Fed-DeepONet framework can work with independent and identically distributed and non-independent and identically distributed (heterogeneous) data. However, most of the applications we seek for Fed-DeepONet (e.g., learning digital twins, scientific computing, or material discovery) will present some level of data heterogeneity. For example, when the clients' data presents slightly different properties while sharing some others. To provide a more detailed explanation for the causes of heterogeneous data, let us use the joint probability density function \(p_{i}(\{u_{m},y\},G(u_{m})(y))\) of the \(i\)-th client input-output data samples. Observe that we can factorize this joint as \(p_{i}(\{u_{m},y\},G(u_{m})(y))=p_{i}(\{u_{m},y\})p_{i}(G(u_{m})(y)|\{u_{m},y\})\). Using this factorization, we can establish three causes of heterogeneous data for Fed-DeepONet. 1.