# Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks
## 3 Federated Deep Operator Networks (Fed-DeepONet)
Run the algorithm for \(R\) DeepONet synchronization rounds. Formally, for each client \(c\in[C]\), the client locally runs:  
\[\gamma_{k+1}^{c}=\theta_{k}^{c}-\eta\
abla\mathcal{L}^{c}(\theta_{k}^{c}), \tag{5}\]  
\[\theta_{k+1}^{c}=\left\{ \begin{array}{ll}\gamma_{k+1}^{c}&\text{if }k+1\text{ mod }K \
eq 0\\ \sum_{c=1}^{C}p_{c}\gamma_{k+1}^{c}&\text{if }k+1\text{ mod }K=0.\end{array} \right. \tag{6}\]  
Theoretically, under some assumptions, one can show (see [21] for details) the convergence of Algorithm 1. In practice, however, not all the clients may be available for federated DeepONet training. To handle this intermittent training scenario, we introduce the set \(\mathcal{A}_{k}\subseteq[C]\), denoting the clients available at global synchronization round \(k\) (note that \(\mathcal{A}_{k}\) is assumed to be fixed for the next \(K\) local updates). Furthermore, we observed that Algorithm 1 might present slow convergence when training Fed-DeepONets (see Section 4.1). One can improve the Fed-DeepONet training convergence by using adaptive (also known as momentum-enhanced) stochastic gradient-based strategies, e.g., [PERSON] [12]. We summarize in Algorithm 2 a federated strategy for handling a variable number of available clients, which collectively trains Fed-DeepONet using a local adaptive stochastic optimization scheme. We call this strategy the Adaptive Fed-DeepONet. **Algorithm 2** _Adaptive Federated Deep Operator Network (Adaptive Fed-DeepONet) learning_. Denote by (i) \(\theta_{k}^{c}\) the DeepONet parameters of the \(c\)-th client, (ii) \(\gamma_{k}^{c}\) the immediate result from \(\theta_{k}^{c}\) via an adaptive local stochastic gradient with momentum update, (iii) \(\eta\) the step-size, (iv) \(\beta_{1},\beta_{2}\in[0,1)\) the exponential decay rates for the moment estimates, (v) \(m_{k}^{c}\) the first-moment vector, (vi) \(v_{k}^{c}\) the second-moment vector, and (vii) \(\epsilon\ll 1\) a small positive number introduced to avoid division by zero during Adaptive Fed-DeepONet. A global synchronization round of the DeepONet parameters is executed by the centralized server every \(K\) steps. Run the algorithm for \(R\) DeepONet synchronization rounds. Formally, for each client \(c\in\mathcal{A}_{k}\), the client locally runs:  
\[t=k+K[k/K]+1,\]  
\[m_{k+1}^{c}=\left\{ \begin{array}{ll}(1-\beta_{1})\
abla\tilde{\mathcal{L}}^{c }(\theta_{k}^{c})&\text{if }k\bmod K\
eq 0\\ \beta_{1}m_{k}^{c}+(1-\beta_{1})\
abla\tilde{\mathcal{L}}^{c}(\theta_{k}^{c}) &\text{if }k\bmod K=0.\end{array} \right.\]  
\[v_{k+1}^{c}=\left\{ \begin{array}{ll}(1-\beta_{2})\big{(}\
abla\tilde{ \mathcal{L}}^{c}(\theta_{k}^{c})\big{)}^{2}&\text{if }k\bmod K\
eq 0\\ \beta_{2}v_{k}^{c}+(1-\beta_{2})\big{(}\
abla\tilde{\mathcal{L}}^{c}(\theta_{k }^{c})\big{)}^{2}&\text{if }k\bmod K=0.\end{array} \right.\]  
\[\tilde{m}_{k+1}^{c}=m_{k+1}^{c}/(1-(\beta_{1})^{t})\;\;\text{and}\;\;\tilde{v} _{k+1}^{c}=v_{k+1}^{c}/(1-(\beta_{2})^{t}),\]  
\[\gamma_{k+1}^{c}=\theta_{k}^{c}-\eta\cdot\frac{\hat{m}_{k+1}^{c}}{\Big{(} \sqrt{\hat{v}_{k+1}^{c}}+\epsilon\Big{)}}, \tag{7}\]  
\[\theta_{k+1}^{c}=\left\{ \begin{array}{ll}\gamma_{k+1}^{c}&\text{if }k+1\bmod K \
eq 0\\ \sum_{c=1}^{C}p_{c}\gamma_{k+1}^{c}&\text{if }k+1\bmod K=0.\end{array} \right. \tag{8}\]