# Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks
## 3 Federated Deep Operator Networks (Fed-DeepONet)
_Different input distributions._ In this situation, we have \(p_{i}(\{u_{m},y\})\
eq p_{j}(\{u_{m},y\})\) and \(p_{i}(G(u_{m})(y)|\{u_{m},y\})=p_{j}(G(u_{m})(y)|\{u_{m},y\})\) for some \(i,j\in[C]\). This situation may occur when learning digital twins for synchronous generators. Suppose all the participating clients have generators with the same parameters and assume the datasets collected by each client are different. For instance, assume most clients have stable trajectories in their datasets, while the rest may have many unstable and disturbance trajectories. Such a situation makes the functional input space of the different clients skewed. However, notice that the output functional space remains the same for all clients. In Section 4.3, we will design a simple experiment that evaluates the performance of Algorithms 1 and 2 to different levels of input distribution heterogeneity. 2. _Different conditional distributions._ In this situation, we have the conditional probabilities satisfy \(p_{i}(G(u_{m})(y)|\{u_{m},y\})\
eq p_{j}(G(u_{m})(y)|\{u_{m},y\})\) and \(p_{i}(\{u_{m},y\})=p_{j}(\{u_{m},y\})\). For an example within the proposed framework, consider the generator digital twin learning task again. Then an example of this situation is when the clients use the same excitation signals for system identification but have generators with different parameters and responses. This paper will not approach such a situation from a federated learning perspective. Instead, we show, in our experiments (see Section 4.4), that Fed-DeepONet can tackle this heterogeneous data situation by learning all parameter realizations (i.e., a library of digital twins) provided each client has access to the generators' parameters. If the client does not know the parameters, then one could, for example, keep some of the Branch network parameters private. Such a strategy will control the bias introduced during each synchronization round. We left the details of this strategy for our future work. 3. _Different joint probabilities._ In this situation, each client's input and output functional spaces may have slightly different characteristics. In Section 4.3, we present a simple experiment that aims at testing such a situation. To train the proposed Fed-DeepONet, we first propose a federated averaging implementation of the stochastic gradient descent algorithm. Such an implementation requires us to loop over the following three steps: 1. _Broadcast to clients:_ The centralized server broadcasts to all the clients \(c\in[C]\) the most up-to-date DeepONet parameters, which we denote as \(\theta_{k}\). Here \(k\) is an integer variable used to denote the local update number for the \(c\)-th client. 2. _Client local updates:_ For any \(c\in[C]\), the \(c\)-th client receives the most up-to-date DeepONet parameters \(\theta_{k}\), sets \(\theta_{k}^{c}=\theta_{k}\), and then executes \(K\geq 1\) local stochastic gradient DeepONet updates over the client's dataset \(\{d_{c}^{(i)}\}_{j=1}^{n_{c}}\), i.e., \[\gamma_{k+1}^{c}=\theta_{k}^{c}-\eta\
abla\mathcal{L}^{c}(\theta_{k}^{c}),\] where \(\eta\) is the learning rate, and \(\gamma_{k+1}^{c}\) is the immediate result of the one-step stochastic gradient update from \(\theta_{k}^{c}\), which implies that \(\theta_{k+1}^{c}=\gamma_{k+1}^{c}\) if \(k+1\) mod \(K\) is not equal to zero. 3. _Global synchronization:_ The centralized server aggregates the local DeepONet parameters into a unique global Fed-DeepONet parameter as follows, \(\theta_{k+K}:=\sum_{c\in[C]}p_{c}\gamma_{k+K}^{c}\). We provide a summary of the aforementioned training strategy for Fed-DeepONet in Algorithm 1. **Algorithm 1**_Federated Deep Operator Network (Fed-DeepONet) learning._ Denote by (i) \(\theta_{k}^{c}\) the DeepONet parameters of the \(c\)-th client, (ii) \(\gamma_{k}^{c}\) the immediate result from \(\theta_{k}^{c}\) via a local stochastic gradient update, and (iii) \(\eta\) the learning rate. A global synchronization round of the DeepONet parameters is executed by the centralized server every \(K\) steps.