SlowGuess/NeurIPS_2023_OCR huggingface dataset:
PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning.
Mingjia  $\mathbf{Sh}^{1*}$  Yuhao Zhou $^{1}$  Kai Wang $^{2}$  Huaizheng Zhang
Shudong Huang
Qing Ye1† Jiangcheng Lv1†
Classical federated learning (FL) enables training machine learning models without sharing data for privacy preservation, but heterogeneous data characteristic degrades the performance of the localized model. Personalized FL (PFL) addresses this by synthesizing personalized models from a global model via training on local data. Such a global model may overlook the specific information that the clients have been sampled. In this paper, we propose a novel scheme to inject personalized prior knowledge into the global model in each client, which attempts to mitigate the introduced incomplete information problem in PFL. At the heart of our proposed approach is a framework, the PFL with Bregman Divergence (pFedBreD), decoupling the personalized prior from the local objective function regularized by Bregman divergence for greater adaptability in personalized scenarios. We also relax the mirror descent (RMD) to extract the prior explicitly to provide optional strategies. Additionally, our pFedBreD is backed up by a convergence analysis. Sufficient experiments demonstrate that our method reaches the state-of-the-art performances on 5 datasets and outperforms other methods by up to  $3.5\%$  across 8 benchmarks. Extensive analyses verify the robustness and necessity of proposed designs. https://github.com/BDeMo/pFedBreD_public
Federated learning (FL) [53] has achieved significant success in many fields [72, 43, 71, 76, 69, 59, 32, 5, 34, 49], which include recommendation systems utilized by e-commerce platforms [71], prophylactic maintenance for industrial machinery [76], disease prognosis employed in healthcare [69]. Data heterogeneity is a fundamental characteristic of FL, leading to challenges such as inconsistent training and testing data (data drift) [36]. An efficient solution to these challenges is to fine-tune the global model locally for adaptation on local data [3, 74, 35]. This solution is straightforward and pioneering, but presents a fundamental limitation when dealing with highly heterogeneous data. For examples, heterogeneous data drift may introduce substantial noise [29] and the resulted model may not generalize well to new sample [20, 12]. Thus, heterogeneous data in FL is still challenging [66].
Recently, personalized FL (PFL) is proposed to mitigate the aforementioned negative impact of heterogeneous data [66]. To improve the straightforward solution mentioned above, Per-FedAvg [20] is introduced to train a global model that is easier to fine-tune. Another paper on the similar topic, FedProx [45], aims to resolve the issue of personalized models drifting too far from the global model during training with a dynamic regularizer in the objective during local training. This issue could occur especially in post-training fine-tuning methods without regularization (e.g., Per-FedAvg [20]). Moreover, pFedMe [65], another regularization method modeling local problems using Moreau
*3101ihs@gmail.com
$^{\dagger}$ Equal Corresponding Authors {yeqing, lvjiancheng} @scu.edu.cn
Figure 1: pFedBreD framework: Global-MLE and Local-MAP. The personalized prior knowledge is injected into the global model of the global problem (MLE) in the  $2^{nd}$  step for local training. The local knowledge is extracted from the local problem (MAP) in the  $4^{th}$  step for aggregation.
envelopes, replaces FedProx's personalized model aggregation method with an interpretable approach for aggregating local models [45]. It also accommodates first-order Per-FedAvg [20].
Although the existing PFL methods have achieved promising results, the prior knowledge from single global model for local training [66] hinders the development of PFL. Specifically, we analyze the shortcomings of current PFL methods as follows: 1) utilizing the same global model for direct local training could potentially disregard the client's sampling information. As shown in Figure 1, a single global model provides global knowledge directly for local training, which overlooks the client-sampling information when the global knowledge is transferred to specific clients. 2) Explicitly extracting prior knowledge can be a challenging task. Most of the insightful works [17, 52] propose assumptions for recovering this incomplete information, but these assumptions are implicit, which limits the way to use the information to develop personalized strategies.
To address the former issue above, we propose framework pFedBreD to inject personalized prior knowledge into the one provided by a global model. As shown in Figure 1, it is injected in the  $2^{nd}$  step and the local knowledge is transferred into global model via local models instead of directly aggregating personalized models [45]. To address the latter, we introduce relaxed mirror descent RMD to explicitly extract the prior for exploring personalized strategies.
Our method is backed up with direct theoretical support from Bayesian modeling in Section 4 and a convergence analysis in Section 5, which provides a linear bound  $\mathcal{O}(1 / TN)$  with aggregation noise and a quadratic speedup  $\mathcal{O}(1 / (TNR)^2)$  without. Meanwhile, the existence and validity of the injection and extraction aforementioned information is verified in Section 6.2. The remarkable performance of the implements of the proposed method is tested on 5 datasets and 8 benchmarks. Consistently, our method reach the state-of-the-art. Especially, the improvement of accuracy on task DNN-FEMNIST [11] is up to  $3.5\%$ . Extensive ablation study demonstrate that parts of the hybrid strategy mh are complementary to each other. Our contributions can be summarized as follows:
- The problem of overlooking client-sampling information at prior knowledge being transferred is introduced in this paper, and we first investigate the possibility of explicitly expressing the prior knowledge of the information and design personalized strategies on it.
- To express the personalized prior, we model PFL into a Bayesian optimization problem, Global-MLE and Local-MAP. A novel framework, pFedBreD, is proposed for computing the modeled problem, and RMD is introduced to explicitly extract prior information.
- Sufficient experiments demonstrate our method surpasses most baselines on public benchmarks, thereby showcasing its robustness to data heterogeneity, particularly in cases involving small aggregation ratios and non-convex local objective settings.
${}^{3}TN$  and  ${TNR}$  : total global / local epoch in FL system. See Appendix A.1.
Regularization Researchers have developed a variety of approaches based on regularization to handle the PFL challenge in recent years (e.g., FedU [18], pFedMe [65], FedAMP [31], HeurFedAMP [31]). All of these approaches' personalized objective functions can be expressed as  $J(\theta) + R(\theta; \mu)$  where  $J(\theta)$  is the loss function of the local problem and  $R(\theta; \mu)$  is the regularization term used to restrict the deviation between  $\theta$  and  $\mu$  (e.g.,  $R(\theta; \mu) = \frac{1}{2} ||\theta - \mu||^2$  in pFedMe).
Meta Learning One of the most representative meta-learning based single-model PFL approach is the well-known Per-FedAvg [20], aiming to find an initialization that is easy to fine-tune. That is, the global model in the FL setting is regarded as a meta model in MAML [21, 19], where the objective function of the local problem is  $J(\theta - \eta \nabla J(\theta))$ . Researchers also show the connections between FedAvg [53] and Reptile [60], another meta learning framework. [35] shows how to improve personalization of FL via Reptile. Proximal updating is also used in meta-learning based algorithms such as [79]. One of our strategies  $\text{meg}$  is the one motivated by MAML.
Expectation Maximization Two EM-based [15] methods are proposed, e.g., FedSparse in [50] and FedEM in [17]. Both of them focus on communication compression. The latter provides a variance reduce version and assumes complete information (or data) of the global model obeys distribution in X-family. Another FedEM [52] combines Bayesian modeling, Federated Multi-task learning (FTML) and EM. Our framework pFedBreD is a expectation maximization and maximum a posteriori estimate (EM-MAP) [17] algorithm with personalized prior specified.
Bayesian FL In recent years, studies of PFL with Bayesian learning have been proposed. In related approaches, FLOA [48] and pFedGP [1] are proposed with KL divergence regularization in the loss function, which is comparable to applying specific assumption of X-family prior in pFedBreD see Appendix B.2 for details. Our implementation doesn't use a Bayesian neural network (BNN) model as an inferential model as others do (e.g., pFedGP uses a Gaussian process tree and pFedBayes [77] uses BNN). Instead, to eliminate weight sampling cost in Bayesian methods, prior knowledge is introduced through regularization term.
Overlooked Information in Prior Knowledge From a Bayesian and info. perspective, the global knowledge transferred in conventional method with single global model has no mutual information (MI) with client sampling  $i$ , i.e., formally,  $w = \mathbf{E}_i w_i = \mathbf{E}_i w_i | i \Rightarrow \mathbf{MI} I(w; i) = 0$ , in particular when applying reg.  $R(w^{(t)}; \ldots)$  or local init.  $w_{i,0}^{(t)} \gets w$  where  $w_i$  is the local model on the  $i^{th}$  client. This makes the specific model on each client have to re-obtain this information from scratch solely from the data during training, especially impacted on hard-to-learn representations and datasets.
Bregman-Moreau Envelope Bregman divergencee [10] is employed as a general regular term in our local objective that exactly satisfies the computational requirements and prior assumption, and is formally defined in Eq. (1).
$$
\mathcal {D} _ {g} (x, y): = g (x) - g (y) - \langle \nabla g (y), x - y \rangle \tag {1}
$$
where  $g$  is a convex function. For convenience,  $g$  is assumed to be strictly convex, proper and differentiable such that Bregman divergence is well-defined. To utilize the computational properties of Bregman Divergence in optimization problems, we introduce the following definition in Eq. (2) [7, 8]: Bregman proximal mapping, Bregman-Moreau envelope, and the relationship between them.
$$
\mathcal {D} \operatorname {p r o x} _ {g, \lambda^ {- 1}} f (x) := \arg \min  _ {\theta} \left\{f (\theta) + \lambda \mathcal {D} _ {g} (\theta , x) \right\},
$$
$$
\mathcal {D} \mathbf {e n v} _ {g, \lambda^ {- 1}} f (x) := \min  _ {\theta} \left\{f (\theta) + \lambda \mathcal {D} _ {g} (\theta , x) \right\}, \tag {2}
$$
$$
\nabla \mathcal {D} \mathbf {e n v} _ {g, \lambda^ {- 1}} f (x) = \lambda \nabla^ {2} g (x) [ x - \mathcal {D} \mathbf {p r o x} _ {g, \lambda^ {- 1}} f (x) ],
$$
where  $\lambda > 0$  denotes the regular intensity in general and the variance of the prior in our modeling.
Exponential Family The regular exponential family (X-family) is a relatively large family that facilitates calculations. Therefore, to yield the prior, we employ the X-family [6]defined in Eq. (3).
$$
\mathbf {P} _ {e f} (\mathcal {V}; s, g) = h (\mathcal {V}) \exp \left\{\langle \mathcal {V}, s \rangle - g (s) \right\} = h (\mathcal {V}) \exp \left\{- \mathcal {D} _ {g ^ {*}} (\mathcal {V}, \mu) + g ^ {*} (\mathcal {V}) \right\}, \tag {3}
$$
where  $g$  is assumed to be convex,  $\mathcal{D}_g(\cdot ,\cdot)$  is the Bregman divergence, and  $g^{*}$  is the Fenchel Conjugate of  $g$ . In Eq. (3),  $s$ ,  $h(\mathcal{V})$  and  $g(s)$  are respectively the natural parameter, potential measure and logarithmic normalization factor, where we have the mean parameter  $\mu = \nabla g(s)$ . Additionally, to highlight the variance, the scaled exponential family (SX-family) is introduced in Eq. (4)
$$
\mathbf {P} _ {s e f} (\mathcal {V}; \lambda , s, g) = h _ {\mathcal {V}} (\mathcal {V}) \exp \left\{\lambda \left[ \langle \mathcal {V}, s \rangle - g (s) \right] \right\} = h _ {\lambda} (\mathcal {V}) \exp \left\{- \lambda \mathcal {D} _ {g ^ {*}} (\mathcal {V}, \mu) + \lambda g ^ {*} (\mathcal {V}) \right\}, \tag {4}
$$
where  $\log h_{\lambda}(\mathcal{V})$  is the scaled potential measure, and the scale parameter  $\lambda$  is employed to highlight the variance. Moreover,  $\mathcal{V}$  is assumed to be the minimal sufficient statistic of the complete information for local inference, details of which can be found in Section 4.
In this section<sup>4</sup>, we introduce missing client-sampling information based on classic FL, use EM to reduce the computational cost of the information-introduced FL problem, and propose RMD, a class of prior selection strategies, based on the E-step in EM. The general FL classification problem with KL divergence could be formulated as Eq. (5) [53, 66].
$$
\arg \min  _ {w} \mathbf {E} _ {i} \mathbf {E} _ {d _ {i}} \mathbf {K L} (\mathbf {P} (y _ {i} | x _ {i}) | | \hat {\mathbf {P}} (y _ {i} | x _ {i}, w)) = \arg \max  _ {w} \mathbf {E} _ {i} \mathbf {E} _ {d _ {i}} \mathbf {E} _ {y _ {i} | x _ {i}} \log \hat {\mathbf {P}} (y _ {i} | x _ {i}, w), (x _ {i}, y _ {i}) \in d _ {i}, \tag {5}
$$
where we rewrite the discriminant model as an maximum likelihood estimation (MLE) problem [41] of  $y_{i}|x_{i}$  in the right hand side (R.H.S.) of Eq. (5).  $(x_{i},y_{i})$  represent the pairs of input and label respectively in dataset  $d_{i}$  on the  $i^{th}$  client, and  $\hat{\mathbf{P}}(y_i|x_i,w)$  is the inferential model parameterized by  $w$ . Each local data distribution is presuppose to be unique, so using the global model with local data for inference and training could overlook the fact that the client has been sampled before transmitting the global model, and the prior knowledge transmitted directly via the global model as the local training prior knowledge (e.g. via initial points, penalty points in dynamic regular terms, etc.) has no mutual information with the client sampling, i.e., the global model  $w = \mathbf{E}_i w_i = \mathbf{E}_i w_i | i$ . Thus, to reduce the potential impact of the overlooked information, the complete information  $\Theta_{i}$  on the  $i^{th}$  client is introduced which turns Eq. (5) into Eq. (6).
$$
\arg \max  _ {w} \mathbf {E} \log \int_ {\Theta_ {i}} \hat {\mathbf {P}} \left(y _ {i} \mid x _ {i}, \Theta_ {i}, w\right) \mathbf {P} \left(\Theta_ {i} \mid x _ {i}, w\right) d \Theta_ {i}, \tag {6}
$$
where  $\mathbf{E} = \mathbf{E}_i\mathbf{E}_{d_i}\mathbf{E}_{y_i|x_i}$  and the direct calculation of this is computationally expensive [14].
**Framework:** Leveraging Expectation Maximization for Prior Parameter Extraction The integral term in Eq. (6) makes direct computation impossible [14], so we employ EM to approximate the likelihood with unobserved variables [15] as shown in Eq. (7), where  $\mathbf{Q}(\Theta_i)$  is any probability measure.
$$
\sum_ {i} \log \hat {\mathbf {P}} \left(y _ {i} \mid x _ {i}, w\right) \geq \sum_ {i} \mathbf {E} _ {\mathbf {Q} (\Theta_ {i})} [ \log \hat {\mathbf {P}} \left(y _ {i} \mid x _ {i}, \Theta_ {i}, w\right) + \mathbf {E} _ {y _ {i} \mid x _ {i}, w} \log \mathbf {P} \left(\Theta_ {i} \mid d _ {i}, w\right) ]. \tag {7}
$$
Assuming that prior  $\Theta_{i}|d_{i},w\sim \hat{\mathbf{P}}_{sef}(\Theta_{i};\lambda ,s_{i}(w;d_{i}),g)^{5}$  and the local loss function on the  $i^{th}$  client  $f_{i}(\Theta_{i},w)$  is  $\mathbf{E}_{d_i}[-\log \mathbf{P}(y_i|x_i,\Theta_i,w)]$  we have the left hand side (L.H.S.) of the Eq. (8) from (7). Here is an assumption for simplification that  $\theta_{i}$  contains all the information for local inference, i.e.  $\theta_{i} = \Theta_{i}$  and  $\mathbf{P}(y_i|x_i,\Theta_i,w) = \mathbf{P}(y_i|x_i,\Theta_i)$ . It happens when  $\theta_{i}$  is all the parameters of the personalized model and we only use the personalized model for inference. Thus,  $f_{i}(\theta_{i}) = \mathbf{E}_{d_{i}}[-\log \hat{\mathbf{P}} (y_{i}|x_{i},\theta_{i})]$ . Thus, we can optimize an upper bound as a bi-level optimization problem as shown in the R.H.S. of the Eq. (8) to solve Eq. (5) approximately, where mean parameter  $\mu_{i} = \nabla g\circ s_{i}^{6}$  [9]. And, we can derive our framework as shown in Section 5.
$$
- \max  _ {w, \left\{\theta_ {i} \right\}} \mathbf {E} _ {i} \left\{- f _ {i} \left(\theta_ {i}\right) - \lambda \mathcal {D} _ {g ^ {*}} \left(\theta_ {i}, \mu_ {i} (w)\right) \right\} \leq \min  _ {w} \mathbf {E} _ {i} \quad \min  _ {\left\{\theta_ {i} \right\}} \left\{f _ {i} \left(\theta_ {i}\right) + \lambda \mathcal {D} _ {g ^ {*}} \left(\theta_ {i}, \mu_ {i} (w)\right) \right\}. \tag {8}
$$
4 More details of equations are in Appendix B.
${}^{5}$  A-posteriori distribution for local client whose prior knowledge is from global model. See Appendix B.
The following  $d_{i}$  is omitted with the same footnote  $i$  in  $\mu_{i}$  for simplification  $(\mu_i(\cdot)\gets \mu_i(\cdot ;d_i))$
Strategies: Relaxing Mirror Descent for Prior Selection To extract the prior strategies and implement  $\mu_{i}$  E-step of EM in close-form, we propose a method called relaxed mirror descent (RMD), where the mirror descent (MD) is EM in X-family [40]. MD can be generally written as Eq. (9) from the old  $\hat{w}$  to the new one  $\hat{w}^{+}$  in each iteration [54, 40].
$$
\hat {w} ^ {+} \leftarrow \underset {\hat {\theta}} {\arg \min } \left\{f (\hat {w}) + \langle \nabla f (\hat {w}), \hat {\theta} - \hat {w} \rangle + \hat {\lambda} \mathcal {D} _ {\hat {g}} (\hat {\theta}, \hat {w}) \right\}. \tag {9}
$$
According to the Lagrangian dual, we rewrite the problem into a more general variant shown in Eq. (10) with relaxed restrictions and superfluous parameter.
$$
\arg \min  _ {\hat {\theta}, \hat {\mu}} \left\{\Psi (\hat {\theta}, \hat {w}) + \left\langle \nabla \Phi (\hat {w}), \hat {\mu} - \hat {w} \right\rangle + \lambda \mathcal {D} _ {g ^ {*}} (\hat {\theta}, \hat {\mu}) + (2 \eta) ^ {- 1} \| \hat {\mu} - \hat {w} \| ^ {2} \right\}. \tag {10}
$$
We can transform Eq. (10) back into Eq. (9) by setting  $\Phi (\hat{w})$  to satisfy  $\nabla \Phi (\hat{w}) = \nabla f(\hat{w})$ , and defining  $\Psi (\hat{\theta},\hat{w})$  as a function with  $f(\hat{w})$  and a penalty term to make  $\hat{\theta}$  and  $\hat{w}$  close as possible (e.g.,  $\hat{\lambda}\mathcal{D}_{\hat{g}}(\hat{\theta},\hat{w})$ ). This provides us a way to extract  $\mu_{\Phi}$  the function to generate mean parameter of the prior, as shown in Eq. (11), which is minimizing an upper bound of the problem in Eq. (10).
$$
\begin{array}{l} \mathcal {D} \mathrm {e n v} _ {g ^ {*}, \lambda^ {- 1}} \Psi (\cdot , w) \left(\mu_ {\Phi} (w)\right) = \min  _ {\theta} \left\{\Psi (\theta , w) + \lambda \mathcal {D} _ {g ^ {*}} \left(\theta , \mu_ {\Phi} (w)\right) \right\} \\ \left. \mu_ {\Phi} (w) \right. = \arg \min  _ {\mu} \left\{\langle \nabla \Phi (w), \mu - w \rangle + (2 \eta) ^ {- 1} | | \mu - w | | ^ {2} \right\}. \tag {11} \\ \end{array}
$$
By optimality condition, we have  $\mu_{\Phi}(w) = w - \eta \nabla \Phi (w)$ , which can be specified by  $\Phi$ . The remaining part is a Bregman-Moreau envelope. Thus, we can optimize the upper bound with an EM-MAP method, alternately computing  $\mu_{\Phi}(w)$  and  $\mathcal{D}\mathbf{prox}_{g^{*},\lambda^{-1}}\Psi (\cdot ,w)(\mu_{\Phi}(w))$
5 Framework Design
Problem Formulation that Highlights Personalized Prior Inspired by the aforementioned motivation, the personalized models  $\theta_{i}$  and mean parameters are respectively the solution of  $\mathcal{D}\mathrm{env}_{g^{*},\lambda^{-1}}f_{i}(\mu_{i}(w))$  and  $\mu_i(w)$  on the  $i^{th}$  client, where  $w$  is the global model. We assume that personalized model contains all the local information required for inference on the  $i^{th}$  client, and satisfies  $\theta_{i}|d_{i},w\sim \mathbf{P}_{\mathrm{se}f}(\theta_{i};\lambda ,s_{i}(w),g)$ . The global problem can be written as Eq. (12).
$$
\min  _ {w} \mathbf {E} _ {i} \left\{F _ {i} (w) := \mathcal {D} \mathbf {e n v} _ {g ^ {*}, \lambda^ {- 1}} f _ {i} (\mu_ {i} (w)) \right\}. \tag {12}
$$
The given  $g$  is strictly convex,  $\lambda > 0$ ,  $f_{i}$  is the local loss function,  $s_i(w)$  is the natural parameter and  $\mu_i(w) = \mathbf{E}_{\theta_i|x_i,w}\theta_i = \nabla g(s_i(w))$  is the mean (or expectation) parameter in Eq. (12).
Framework: pFedBreD To solve the optimization problem in Eq. (12), we use gradient-based methods to solve the global problem using the gradient of  $F_{i}$ :
$$
\nabla F _ {i} (w) = \lambda \mathbf {D} \mu_ {i} (w) \nabla^ {2} g ^ {*} (\mu_ {i} (w)) [ \mu_ {i} (w) - \mathcal {D} \mathbf {p r o x} _ {g ^ {*}, \lambda^ {- 1}} f _ {i} (\mu_ {i} (w)) ], \tag {13}
$$
where  $\mathbf{D}$  is the gradient operator of the vector value function, and  $\nabla^2$  is the Hessian operator. The framework is shown as Algorithm 1, where  $\mathcal{I}$  is the client selecting strategy for global model aggregation;  $w_{init}$  and  $\theta_{init}$  are the initialization strategies on the  $i^{th}$  client;  $\alpha_{m}$  is the main problem step-size;  $T$ ,  $R$ ,  $N$  are respectively the total number of iterations, local iterations, and clients.  $\beta$  is used in the same trick as [37, 65]. The strategies to derive the initialization points of  $w_{i}$  and  $\theta_{i}$  at each local epoch are  $w_{i,0}^{(t)} \gets w^{(t-1)}$  and  $\theta_{i,0}^{(t)} \gets \theta_{i,R}^{(t-1)}$ .
Implementation: Maximum Entropy and Meta-Step Practically, two main parts of the pFedBreD are needed to be implemented:
-  $g$ , the function used to derive the logarithmic normalization factor, determines the type of prior to be used;
-  $\{s_i\}$  or  $\{\mu_i\}$ , the functions used to derive the natural parameter and mean parameter for the personalized local prior, determine which particular prior is used.
The details of first-order methods is in Appendix A.7.
Algorithm 1 Algorithm for pFedBreD
Input:  $\mathcal{I},\{d_i\}$ $i = 1\dots N$    
Parameter:  $\alpha_{m},g,\lambda ,T,R,\{w_{init},\theta_{i},\mu_{i}\} ,i = 1\dots N$    
Output:  $w^{(T)},\{\theta_i^{(T)}\} ,i = 1\dots N$    
1: Initialize  $w^{(0)},\{\theta_i^{(0)}\} ,\{\mathcal{C}_{i,R}^0\}$  .   
2: for t=1...T do   
3: Server sends  $w^{(t - 1)}$  to clients;   
4: for i=1...N in parallel on each clients do   
5: Initialize  $w_{i,0}^{(t)}$  and  $\theta_{i,0}^{(t)}$  with winit and  $\theta_{init}$  .   
6: for r=1...R do   
7: Generate  $\mu_{i,r}^{(t)}\gets \mu_i(w_{i,r - 1}^{(t)},\dots)$  .   
8:  $\theta_{i,r}^{(t)}\gets \mathcal{D}\mathrm{prox}_{g^{*},\lambda^{-1}}f_{i}(\mu_{i,r}^{(t)})$  .   
9:  $w_{i,r}^{(t)}\gets w_{i,r - 1}^{(t)} - \alpha_m\nabla F_i(w_{i,r - 1}^{(t)})$  .   
10: end for   
11: end for   
12: Server collects  $\{w_{i,R}^{(t)}\}$  and calculate  $w^{(t)}\gets ()$    
13: end for   
14: return  $w^T$ $\{\theta_i^T\}$
We propose first-order implementations based on maximum entropy rule [22, 33]. In the SX-family, the Gaussian distribution has the maximum entropy among continuous distributions when  $g$ ,  $\mu_i$  (the first-order moment), and  $\lambda$  (the parameter determining the second moment) are given. Thus, we employ the scaled norm square  $g = g^* = \frac{1}{2} ||\cdot||^2$  to turn the prior into a spherical Gaussian, in order to maximize the entropy of the prior on a particular client. With this assumed prior, we have  $\nabla g = \nabla g^* = I$ , which means  $\mu_i = s_i$ . We can choose a different  $\Phi_i$  as shown in Eq. (14)<sup>8</sup> to generate selection strategies according to Section 4, via  $\mu_i(w) = w - \eta \nabla \Phi_i(w)$  (meta-step).
$$
\Phi_ {i} = \left\{ \begin{array}{l l} f _ {i} & \\ F _ {i} & \\ f _ {i} + F _ {i} & \end{array} \right. \quad \boxed {\mu_ {i, r} ^ {(t)}} \leftarrow \left\{ \begin{array}{l l} w _ {i, r - 1} ^ {(t)} - \eta_ {\alpha} \nabla f _ {i} \left(w _ {i, r - 1} ^ {(t)}\right), & \mathbf {l g} \\ w _ {i, r - 1} ^ {(t)} - \eta \left(w _ {i, R} ^ {(t - 1)} - \theta_ {i, r - 1} ^ {(t)}\right), & \mathbf {m e g} \\ w _ {i, r - 1} ^ {(t)} - \eta_ {\alpha} \nabla f _ {i} \left(w _ {i, r - 1} ^ {(t)}\right) - \eta \left(w _ {i, R} ^ {(t - 1)} - \theta_ {i, r - 1} ^ {(t)}\right), & \mathbf {m h} \end{array} \right. \tag {14}
$$