Rethinking Client Reweighting for Selfish Federated Learning

Abstract: Most federated learning (FL) algorithms aim to learn a model which achieves optimal overall performance across all clients. However, for some clients, the model obtained by conventional federated training may perform even worse than that obtained by local training. Therefore, for a stakeholder who only cares about the performance of a few $\textit{internal clients}$, the outcome of conventional federated learning may be unsatisfactory. To this end, we study a new $\textit{selfish}$ variant of federated learning, in which the ultimate objective is to learn a model with optimal performance on internal clients $\textit{alone}$ instead of all clients. We further propose Variance Reduction Selfish Learning (VaRSeL), a novel algorithm that reweights the external clients based on variance reduction for learning a model desired in this setting. Within each round of federated training, it guides the model to update towards the direction favored by the internal clients. We give a convergence analysis for both strongly-convex and non-convex cases, highlighting its fine-tune effect. Finally, we perform extensive experiments on both synthesized and real-world datasets, covering image classification, language modeling, and medical image segmentation. Experimental results empirically justify our theoretical results and show the advantage of VaRSeL over related FL algorithms.

Reviews and author responses:

 I thank the authors for their detailed response. 

For my first question regarding the differences and similarities between personalized FL and selfish FL, I am not convinced that those two concepts "differ significantly" in the case of M=1. For example, in "Ridge-tuned FedAvg" of [1], one can tune the regularization parameter to achieve the following effect: (1) when the first client is closely related to the other clients, then the first client borrow information from others and get performance boost; when the first client has little relationship with others, then the first client does local training.

I appreciate the authors' detailed explanation on my third question. This is really helpful.

I have also read other reviewers' comment and I am particularly concerned with the issues with the proofs (e.g., the issues raised by Reviewer nesy and 1vSj). Thus, I would like to retain my original score.
 


[1] Cheng, Gary, Karan Chadha, and John Duchi. "Fine-tuning is fine in federated learning." arXiv preprint arXiv:2108.07313 (2021).

 Thanks to the author(s) for the response. I will try to summarize the main criticisms taking into account the other reviewers' comments.


**$\bullet$ Problem setup.** I disagree with the first reason you provided for external clients to participate in the training, which is ***(i) They can get a somehow useful model from the stakeholder.*** I do not see how the model of stakeholder would be anyway useful for external clients given that ultimate objective function includes only the distribution of internal clients' dataset and it can be very different from the distribution of external datasets. Moreover, in response to the *Reviewer zFoE*, you mention that the stakeholder ***doesn't need to make any compromise with the external, non-interested clients***.

My suggestion of online services might motivate the approach, but it does not allow to assume $M \ll N$. Even if you find a large number of online services, would it be reasonable to pay them all? Note that the setup of Selfish-FL is not the same as FL, and one cannot assume $M \ll N$ justifying it ***since the number of clients are typically large in federated learning*** (as you mentioned).


**$\bullet$ On partial participation.** I agree with *Reviewer 1vSj* and do not understand your claim that ***our algorithm actually allows partial participation.*** Based on the description of VaRSeL method, in each iteration the server sends two vectors and receives a float number from all clients. Of course, downlink communication can be fast and uplink communication is used for floats only. However, partial participation means that only some clients will be active to actually provide the server those floats. If it were actually partial participation, then the server would proceed with a small number of floats received from active clients. Of course, we can modify the method in experiments, but both algorithm description and theory are not written to support it.

**$\bullet$ Lemma 1 and assumption on $A$.** Notice that $\Phi(0; \theta) \le \sigma \|\nabla l(\theta)\|^2$ by Assumption 1 and $\Phi(w_*; \theta) \le A \|\nabla l(\theta)\|^2 + B$ by Lemma 1. How do you theoretically claim the "improvement" by Lemma 1 ? Constants $A$ and $B$ cannot be small since they depend on $\kappa_j$ and $\nu_j$ from the Assumption 1, which are allowed to be big as the data distribution of external clients are assumed to be very different form the internal clients' data (see also the comment of *Reviewer pXQG*).

Moreover, the formula of $A$ in (11) depends on $(1^\top e)^{-2}$, where $e$ is the unit eigenvector corresponding to smallest eigenvalue $\lambda_1$. How big this quantity $(1^\top e)^{-2}$ (and hence $A$) can be ? How did you upper bounded this term in **A6** when estimating the parameter $A$ ?  What if $1^\top e = 0$ ?

I am surprised by the response ***"$\lambda_1=0$ is quite unlikely to happen"*** to my question "What if $\lambda_1 = 0?$" related to Lemma 1. Does this imply that Lemma 1 is not valid, at least the way it is presented in the paper?

**$\bullet$ Additive term in Theorem 2.** The work of Cho et al (2020) is not published as far as I can see. In Theorem 3.1 in Gower et al (2019), the additive term *disappears* when full local-subsampling is used. In Theorem 2, the additive term has $\frac{B}{\mu^2}$ (quadratic dependence on $\mu$), which depends on $\nu_j$'s, which cannot all be zero due to the assumptions on data distributions over internal/external clients.


**$\bullet$ "Constants" in Theorem 3 (A9)**. Then what is the point of showing $\eta$ and $\tau$ in the denominators along with $\Gamma_3$ ?




 We thank the review for the useful feedback on our paper.

**Q1:** *The motivation of the proposed new setting Selfish Federated Learning is not well substantiated. There is only one short paragraph on two examples, which are not quite convincing.*

**A1:** In the paper, we introduce the Selfish Federated Learning setting mainly from the perspective of the stakeholder, who can ensure the benefit from federated training. We assume that the external clients engage in federated training mainly for two reasons: (i) They can get a somehow useful model from the stakeholder. (ii) They can get paid by the stakeholder for providing the training interfaces (a.k.a, online services). We will elaborate the incentives of external clients in our revised version.

**Q2-3:** *About some informal notations.* 

**A2-3:** We thank the reviewer for the suggestions on formal definition. We will modify it into a more formal way in our revised version: $S_n \sim D_n$; supplace ultimate objective by $l(\theta) = \frac{1}{M}\sum_{i=1}^M \mathbb{E}_{(x,y)\sim \mathcal{D}_i} \mathcal{L} (f(x,\theta),y)$; and assume $D_1=D_2=\cdots=D_M$.

**Q4:** *Claims before section 4 "we will show both in theory and in experiments that it is probable to improve the estimation of θ by leveraging additional knowledge from external clients without privacy leakage." Which part of the theory supports this claim? Do you have privacy guarantees to prevent privacy leakage?*

**A4:** We support the claim of "improvement" by Lemma 1. With our reweighting parameter $\mathbf w_*$, the variance $\varPhi(\mathbf w; \theta)$ can be bounded by $A|| \nabla l(\theta) ||^2 + B$ ($A$ and $B$ are small constants), based on which we develop the convergence result. So if we do not leverage the external clients, then $\varPhi(\mathbf w; \theta)$ will become larger and the convergence will slow down.  We admit that the claim of "without privacy leakage" is a bit overstated, and we will change it into "without sharing raw data".

**Q5:** *Intuitions behind Our Strategy.*

**A5:** (1) Using only the limited internal data may be not enough and can induce large variance, and since $N \gg M$, the external clients may also bring fruitful knowledge for the task. And as shown in the second half of Assumption 1, the external distributions are allowed be very different from the internal ones, but there may still be some of them with relatively small diffrence (i.e. with smaller $\kappa_j$, $\nu_j$). Suppose the external distributions are all consistent with the internal ones, then it's reasonable to take all of them to reduce the variance. Now that only part of the external distributions are consistent with the internal ones, so we should take them differently by assigning higher weights ($\mathbf w_j$) to the similar distributions and lower weights to the dissimilar ones. In this way, as shown in Lemma 1 and in our experiments, the surrogate objective can reduce the variance and simulate the ultimate objective better. 

(2) To gain some insights, we also give a simple statistical model as an example. Suppose there are internal variables $x_1, \cdots, x_n$ taken i.i.d from an internal Gaussian distribution $N(\mu,\sigma^2)$, then we also have an external variable $x_{n+1}$ taken from a noisier distribution $N(\mu,\gamma \sigma^2)$, $\gamma>1$. Then as long as $\gamma<2+ \frac 1 n$, absorbing $x_{n+1}$ will reduce the variance and better approximate $\mu$ in expectation, i.e. $Var[\frac {x_1+\cdots+x_n + x_{n+1}} {n+1}] < Var [\frac {x_1+\cdots+x_n} n]$. So even if the external distributions are different and noisier, some of them can still bring some fruitful knowledge.

**Q6:** *The restriction of $A< \frac \mu L$.*

**A6:** We want to show that the condition of $A< \frac \mu L$ is not very restrictive. See our derivation of $A$ in Equation (11). Let's consider an extremely "bad" case, which is very unlikely to happen in practice, where all the external clients are giving completely opposite answers, then $\kappa_j=4$ for $M+1 \le j \le M+N$. So even in this bad case, $A=\frac {M\sigma + (\mathbf 1^{\text{T}} \mathbf e)^{-2} \sum _{j=M+1} ^{M+N} \kappa _j } {(M+N)^2} \le \frac {M\sigma + 4N } {(M+N)^2}$, and since the number of clients are typically large in federated learning (given $N \gg M$), $A$ is still a small constant. Therefore, it's in general reasonable to assume that $A< \frac \mu L$.


 **Q7:** *Proof details of Lemma 1.*

**A7:** (1) $\sum_{i=1}^M \mathbb E || \nabla l_i (\theta)-\nabla l(\theta) ||^2 = \mathbb E || \sum_{i=1}^M (\nabla l_i (\theta)-\nabla l(\theta)) ||^2 = M^2 \mathbb E|| \nabla l_{\mathcal I}(\theta) - \nabla l(\theta)||^2 \le M^2 \sigma || \nabla l(\theta) ||^2$, where the first equality follows from $\mathbb E[\nabla l_i (\theta)] = \nabla l(\theta)$, and the last inequality follows from Assumption 1. 

(2) $\lambda_1$ is the smallest eigenvalue of $\Sigma\Sigma^\text T$, which is a semi-positive-definite matrix, so $\lambda_1 \ge 0$. And $\lambda_1=0$ if and only if the row vectors ($\nabla l_j(\theta) - \nabla l(\theta)$) in $\Sigma$ are linearly dependent. Since the dimension $d$ is typically very large (especially in deep learning), and since $\nabla l_j(\theta)$'s are independent gradient estimations over different external distributions, they are very unlikely to be linearly dependent. Therefore, $\lambda_1 =0$ is very unlikely to happen.

**Q8:** *The additive part in Theorem 2.*

**A8:** We admit that the additive part may weaken the convergence result. But the additive part is somehow inevitable due to the biased sampling strategy--the external gradients are always biased estimators of the ultimate gradient. Also, the same additive part is observed in many related works (Theorem 3.1 in Cho et al (2020) https://arxiv.org/pdf/2010.01243.pdf, Theorem 3.1 in Gower et al (2019) http://proceedings.mlr.press/v97/qian19b.html).

**Q9:** *Factors in Theorem 3.*

**A9:** I suppose the factors in Theorem 3 are constants since we use constant learning rate and constant step size.

**Q10:** The name of "Variance Reduction".

**A10:** Since we choose $\mathbf w_{\star}$ to minimize the variance between ultimate gradient and surrogate gradient, I think it's fair to say that this is a variance-reduced approach. 

 We thank the reviewer for the useful feedback. The reviewer mainly comments on (Q1) the motivation of the model; (Q2) the communication requirement and partial participation; (Q3-4) the restriction of $A$; (Q5) the availability of $\mathbf w_{\star}$ in our proof; (Q6) the hyperperameter tuning in our experiment.

**A1:** In our original paper, we introduce the setting from the perspective of the stakeholder, who wants to opimize internal/ultimate objective, finds external collaborators, and applies Selfish-FL algorithm to solve the task. We assume that the external clients engage in federated training mainly for two reasons: (i) They can get a somehow useful model from the stakeholder. (ii) They can get paid by the stakeholder for providing the training interfaces (a.k.a, online services). We will elaborate the incentives of external clients in our revised version. 

**A2:** We want to show that our algorithm actually allows partial participation. In our Algorithm 1 (Line 3-5), it's true that the server broadcasts its model to all the clients. We assume that this not take lots of time since broadcasting is typically fast, as assumed in many previous works [1,2,3]. And when receiving their updates, the server only need to collect several bytes of information and is also very efficient. More importantly, partial participation is allowed since the server does not need to collect all the information, and does not need to collect the gradient updates from all the clients. As in typical FL algorithms (though may not pointed out explicitly), the server can, for example, set a time threshold in our algorithm. Therefore, partial participation is also allowed in our framework.

**A3-4:** We want to show that the restriction of $A<1$ and $A< \frac \mu L$ is not very restrictive. See our derivation of $A$ in Equation (11). Let's consider an extremely "bad" case, which is very unlikely to happen in practice, where all the external clients are giving completely opposite answers, then $\kappa_j=4$ for $M+1 \le j \le M+N$. So even in this bad case, $A=\frac {M\sigma + (\mathbf 1^{\text{T}} \mathbf e)^{-2} \sum _{j=M+1} ^{M+N} \kappa _j } {(M+N)^2} \le \frac {M\sigma + 4N } {(M+N)^2}$, and since the number of clients are typically large in federated learning (given $N \gg M$), $A$ is still a small constant. Therefore, it's in general reasonable to assume that $A< \frac \mu L$.



*[to be continued]*



 We thank the reviewer for the feedback on our paper. The reviewer mainly (i) questioned the difference between our setting and the personalized setting, and (ii) questionted the complexity of solving $\tilde {\mathbf w}$ in Equation (7) and solving $\mathbf w_*$ in Equation (4).

**Q1:** *The idea of only optimizing the performance of a subset of clients is not really new. If we let $M=1$ (i.e., there is only one internal client), then this is precisely the formulation in [1, 2, 3]. If the authors assume that $\mathcal D_1, \cdots, \mathcal D_M$ are all approximately equal to some $\mathcal D_{\mathcal I}$, (the display before Eq. 2), then there is only a marginal difference between $M\ne 1$ and $M =1$.*

**A1:** We claim that (even if $M=1$) our setting **differs significantly from Personalized-FL** [1,2,3]. Personalized-FL allows each client to maintain a personalized model, but the losses of **all clients** are still considered in the overall objectives. As in [1,2,3], their objectives still include the loss functions across all the clients. Use the notations in Table 1 in our paper. When $M=1$, Personalized-FL optimizes $\sum _{n=1} ^{M+N} \mathcal L _{\mathcal D_n}(f_n)$ [1,2,3,4,5], while Selfish-FL optimizes $\mathcal L _{\mathcal D_1}(f)$, which is purely the learning task for the stakeholder and is largely different from the objective of Personalized-FL. When the stakeholder optimizes this objective in Selfish-FL, the main difference is that it **doesn't need to make any compromise** with the external, non-interested clients $j$ ($M+1 \le M+N$), which results in **great performance boost** in practice (see Figure 1, Table 2, Table 3, Table 4 in our paper).

**Q2:** *Eq. (2) looks a bit confusing for me. What is the meaning of $(x, y) \sim D_1, ..., D_M$? Are you taking the expectation over a mixture of $D_1, ..., D_M$?*

**A2:** We apologize for making the confusion. In our revised version, we will change Equation 2 into a clearer formulation: $ l(\theta) = \frac{1}{M}\sum_{i=1}^M \mathbb{E}_{(x,y)\sim \mathcal{D}_i} \mathcal{L} (f(x,\theta),y)$. We thank the reviewer for pointing out this.

**Q3:** *Please correct me if I am wrong, but isn't optimizing either $\varPhi$ or $\tilde \varPhi$ a very hard problem, as the gradient involves the second derivative of $l_j$?*

**A3:** We want to point out that optimizing $\tilde \varPhi$ and $\varPhi$ with respect to $\mathbf w$ are actually simple tasks and don't require second derivatives.

(1) *Solving $\tilde {\mathbf w}$ in Equation (7).* In Equation (7), $|| \nabla l_i(\theta)-\nabla l(\theta) ||^2$ and $||\nabla l_j(\theta)-\nabla l(\theta) ||^2$ are already computed and communicated (Line 5-8 in Algorithm 1), so it's just a simple optimization equation with respect to $\mathbf w$.

(2) *Solving $\mathbf w_{\star}$ in Equation (4).* 
We have $\varPhi(\mathbf w;\theta) = \frac 1 {(M+\mathbf 1^{\text{T}} \mathbf w)^2} \left[ \sum _{i=1}^M \mathbb E|| \nabla l_i(\theta)-\nabla l(\theta) ||^2 + \mathbb E [ \mathbf w^{\text T} (\Sigma\Sigma^\text T) \mathbf w ] \right]$, where $\Sigma$ is defined on page 14 below Equation (9). 

Then 
$\Sigma\Sigma^\text T$
 is a $N\times N$ matrix, and only $K$ clients are involved in each round, so it is reduced to a $K\times K$ matrix for computation. Moreover, $\mathbf w_{\star}$ is along the direction of the eigenvector of $\Sigma\Sigma^\text T$ with smallest eigenvalue, so it's very simple to optimize and doesn't require second derivatives.





[1] Cheng, Gary, Karan Chadha, and John Duchi. "Fine-tuning is fine in federated learning." arXiv preprint arXiv:2108.07313 (2021).

[2] Chen, Shuxiao, et al. "A Theorem of the Alternative for Personalized Federated Learning." arXiv preprint arXiv:2103.01901 (2021).

[3] Hanzely, Filip, and Peter Richtárik. "Federated learning of a mixture of global and local models." arXiv preprint arXiv:2002.05516 (2020).

[4] Tan, Alysa Ziying, et al. "Towards personalized federated learning." *arXiv preprint arXiv:2103.00710* (2021).

[5] Fallah, Alireza, Aryan Mokhtari, and Asuman Ozdaglar. "Personalized federated learning: A meta-learning approach." *arXiv preprint arXiv:2002.07948* (2020).


*Please let us know if you still have any concerns regarding our settings or methods.*

 Thank the reviewer for thoughtful feedback! The reviewer's main concerns are: (Q1) *The second half of Assumption 1 weakens the method, and makes the result insignificant.* (Q2) *Whether the learning algorithm is still robust under "bad clients"?* (Q3) *Direct analysis of the VaRSeL algorithm.*

**A1.** 

- First, we want to show that Assumption 1 is **not that much restrictive** and is a **relatively mild** assumption for two reasons: (i) It's actually weaker than the assumptions of constantly-bounded gradient difference, since it allows the difference to grow with the norm of the gradient. (ii) It's actually weaker than the assumptions of bounded-growth, since it allows non-shrinking gradients near the optimal point of $ l(\theta)$. 
- Furthermore, we want to show that Assumption 1 **does not contradict the current setting** of the paper. (i) We do not assume a uniform bound on all external clients. Instead, we assume different $\kappa_j$ and $\nu_j$, for each client $M+1 \le j \le M+N$, which allows some clients to have largely different distributions. In our setting, the external clients are generally inconsistent, and therefore, we assume non-uniform bounds. (ii) In our setting, some external clients can have different distributions from the internal clients (due to mislabels, distribution shifts, data imbalance, etc), and therefore, we take a relatively weaker assumption by allowing both growing difference and non-shrinking gradient. (iii) In our setting, "some external clients can have different distributions" does not necessarily mean that all of their distributions are different. The stakeholder wants to integrate knowledge from external clients because some of them are useful and are able to bring fruitful knowledge. This is consistent with the second half of Assumption 1, according to which we can identify those useful clients with smaller $\kappa_j$ and $\nu_j$.
- Finally, we want to argue that the existence of clients with very different distributions **does not make the method insignificant**. Quite the opposite, tackling clients with largely different distributions (or so called "bad clients") is the very purpose of our methodology. The existence of these bad clients actually makes the reweighting strategy more important, since we need to eliminate or reduce their bad impacts.

**A2.**

​	We want to argue that this paper mainly focus on data heterogeneity and optimization. In fact, we claim that our learning algorithm is **robust under "bad clients"** with very different gradient estimations. 

1. Theoretically, when solving $\tilde {\mathbf w}$ in Equation (7) and solving $\mathbf w_*$ in Equation (4), the clients with very different gradients are already been assigned lower weights, which significantly reduces their impacts on the learned model. 
2. Practically, in Section 5.1, we had included "bad clients" in our synthesized experiments. We introduced the method of data preprocessing in Appendix C.2. To make it clearer, for instance, when training CNN for CIFAR-10, the internal clients only hold the images of *airplane, automobile, bird*, while the external clients hold the images of all 10 different classes (including *cat, deer, dog, frog, horse, ship, truck*). I think it's fair to say that their distributions and gradients are very different. And our numerical experiments show that our algorithm VaRSeL can still work well in these cases.

  Finally, we agree with the reviewer that system heterogeneity (e.g. malfunctions of hardware, adversarial attacks, or selfishness of the clients) is also an important practical challenge in FL, but is not the main concern of our work and may be left for future works.

**A3.**

​    Well, we did not give a direct analysis of VaRSeL. The practical algorithm VaRSeL considers practical issues like partial participation. In practice, we find that $\tilde {\mathbf w}$ is a decent approximation of $\mathbf w_*$, but sadly but true, in the worst case analysis, $\tilde {\mathbf w}$ can still be very different from $\mathbf w_*$ (this so happens when the received external gradients are orthogonal to one another). Consequently, the worst-case bound may fail to reflect the real-world performance of the algorithms.


*We try to address the reviewer's concerns above. We thank the reviewer for the feedback and suggestions, and hope the reviewer re-evaluates our paper!*

This paper proposed a new client reweighting algorithm for selfish federated learning where the goal is to optimize the performance of a few internal clients not all clients. For strengths: this paper studied a new selfish federated learning setting (i.e., the objective is the loss on a small (internal) subset clients instead of on all clients) and propose a VaRSeL algorithm for solving them.

For weaknesses: 
- Regarding the selfish-FL model in Table 1. Here are some concerns: 1) the datasets are shared with $M$ clients, not stored on their own clients;  2) **if the objective is only to optimize the loss of $M$ internal clients, why all other $N$ clients will participate? what is the incentive/benefit for them?** Note that in Section 3 (Problem formulation), the authors even assume that **$M\ll N$**. 
 
- Regarding the proposed algorithm VaRSeL, in each communication round, they need to communicate with all clients (see Line 3, 4, 5 of Algorithm 1) to compute the reweight parameter ${w}$, which is impractical compared with classical federated learning algorithms such as FedAvg (which only communicates with a subset of clients for each round).

- Regarding the Theorem 2, the step size $\eta \leq \frac{\frac{1}{L}-\frac{A}{\mu}}{2\sqrt{A}}$ is likely unachievable, i.e., $\frac{A}{\mu} \geq \frac{1}{L}$, since the strongly convex parameter $\mu$ usually is much smaller than the smooth parameter $L$, and $A$ is related to the bounded gradient/similarity and also can be very large.

- Similarly, regarding the Theorem 3, the step size $\eta \leq \frac{2(1-\sqrt{A})}{L(A + \sqrt{A})}$ is also likely unachievable, i.e., $A$ can be larger than $1$.

- In the proof of Theorem 3, i.e., the bound for $T_3^{(t+\frac{k}{\tau+1})}$ (on Page 20), it uses Lemma 1 for the last inequality. However, Lemma 1 only holds for $w_*$. Thus the last inequality of bounding $T_3^{(t+\frac{k}{\tau+1})}$ requires the $w_*$ for $\overline{\theta^{t+\frac{k}{\tau+1}}}_{w}$ 

of Eq (4), but $w_*$ is not available for $\overline{\theta^{t+\frac{k}{\tau+1}}}_{w}$ at that time. Also the weighted average of $\theta_i$ is not equivalent to weighted average of $\nabla l_i(\theta)$.

- Regarding the experiments, for Table 2, I did not see the results for local steps $\tau =1$ while the authors wrote the local steps are 1, 3, 5. Also, how do you choose/tune the learning rates?



Some typos:
- Page 14, in Eq (10), $\sum_{j=1}^N \mathbb{E}||\nabla l_j(\theta)-\nabla l(\theta)||^2$ should be $\sum_{j=M+1}^{M+N} \mathbb{E}||\nabla l_j(\theta)-\nabla l(\theta)||^2$.
- Similarly, for Page 15, these summations should be $\sum_{j=M+1}^{M+N}$.
- Page 20, the first inequality is wrong, i.e., both terms within $ \mathbb{E}||\cdot ||^2$ in the second term are the same. It should be changed to $\nabla l_i(\theta_i^{t+\frac{k}{\tau+1}})$. The model is not very appropriate (see the comments above). Besides, there exist some issues in their proofs, and also the conditions for their theorems may not hold. Thus I suggest a reject.

This work studies a variant of the federated learning (FL) problem where the stakeholder (i.e. central planner) only cares about the performance of a trained model over a small proportion of local (internal) clients. The work proposes a first-order federated learning algorithm that weights external clients to obtain a variance-reduced gradient estimate of the ultimate objective (over internal clients), and presents convergence analysis for their algorithm under certain settings. Finally, the work evaluates their proposed algorithm by comparing against existing FL algorithms using synthetic and real datasets. 
 This is a well-written paper, and in particular I think the paper does a good job in the following aspects: 1. The paper positions its proposed methodology well with existing works, and highlights in a rather concise way some key differences in motivation and methodology; 2. The paper presents many interpretations for/insights into relevant results and claims, which makes the proposed methodology rather interpretable from both a theory and practical perspective;3. The paper includes a wide range of experiments that demonstrate the practicality of applying the proposed algorithm in various settings.

The main weakness, from my opinion, is that the methodological and technical contributions seem insignificant. In particular, the convergence analysis for the reweighting strategy relies heavily on the second half of Assumption 1, i.e. the difference between the gradient for external clients and the gradient of the ultimate objective can be bounded by the gradient norm of the ultimate objective. In other words, this basically means the gradient estimates from external servers are rather sufficiently accurate to serve as estimates for the  ultimate objective’s gradient. This seems like a rather stylized assumption that says external servers essentially give us more (somewhat accurate) estimates of the gradient of the ultimate objective, which in turn reduces overall variance of the ultimate objective’s gradient estimate. The proof techniques in the paper for convergence results are rather standard and, to the best of my knowledge, have been introduced in previous works. Also, it is stated at the beginning of Section 3 that the distributions over the data in external clients are very different from those of internal clients. Intuitively, this seems to be against (if not contradicting) the second half of Assumption 1, because why would the gradients be rather close while the distributions are very different? I might be missing something here and it would be nice if the authors can clarify.

In addition to the above, I feel the paper would be much stronger if among the external clients, there exists "bad clients" whose gradient estimates are very different from that of the ultimate objective. In practice, these bad clients may represent, for example, malfunctions of hardware, or adversarial attacks from strategic clients who are indeed "selfish" with the aim to optimize their own local objectives. In this more complex setting, it would be interesting to see if the weighting strategy would eventually underweight the bad clients. That being said, even in the current setting of the paper, it would be nice if the paper can provide insights into the eventual weights for external clients; e.g. do the weights represent some notion of external client importance? Which external clients have more weights and why? What key characteristic of an external client would impact its weight? I believe that such insights would be beneficial to the overall message of the paper.

Other comments:
I am curious why the paper does not present a theoretical convergence analysis for the VaRSeL algorithm (i.e. solving for the approximate weight $\tilde{w}$ in Eq.(8) instead of the actual optimal weight $w_*$)? Intuitively under Assumption 1, the individual gradient differences are bounded by the norm of the ultimate objective’s gradient, so showing convergence for VaRSeL should not be much more difficult than the reweighting strategy. Is there something that I am missing here that prohibits us to obtain a convergence bound for VaRSeL? If not, it would be nice if the authors can also discuss convergence of VaRSeL, since this is the main algorithm of interest.
 The paper is well-written, explains ideas in a concise manner, and includes a wide range of experimentations.
The methodological contribution may be a bit underwhelming, due to the restrictive assumption that external clients’ gradients are bounded around the ultimate objective’s gradient. The paper would be much stronger if it can deal with "bad" external clients, and also provide interpretations for/ insights into the weights of external clients.


This paper proposes selfish federated learning, an FL framework that optimizes the performance of a small subset of clients. The authors further propose a new selfish-FL algorithm, termed VaRSeL, based on minimizing the variance between the surrogate gradient $\nabla l (\theta; w)$ and the ultimate gradient $\nabla l (\theta)$. Under certain conditions, the authors prove the convergence of an idealized variant of VaRSeL. The performance of VaRSeL is demonstrated in a range of simulations and real-data experiments. Strengths: The idea of minimizing the variance is interesting and the performance looks promising.

Weaknesses and questions: 
- The idea of only optimizing the performance of a subset of clients is not really new. If we let $M=1$ (i.e., there is only one internal client), then this is precisely the formulation in [1, 2, 3]. If the authors assume that $D_1, ..., D_M$ are all approximately equal to some $D_I$ (the display before Eq. 2), then there is only a marginal difference between $M \neq 1$ and $M = 1$.
- Eq. (2) looks a bit confusing for me. What is the meaning of $(x, y) \sim D_1, ..., D_M$? Are you taking the expectation over a mixture of $D_1, ..., D_M$?
- Please correct me if I am wrong, but isn't optimizing either $\Phi$ or $\tilde \Phi$ a very hard problem, as the gradient involves the second derivative of $l_j$? 

[1] Cheng, Gary, Karan Chadha, and John Duchi. "Fine-tuning is fine in federated learning." arXiv preprint arXiv:2108.07313 (2021).

[2] Chen, Shuxiao, et al. "A Theorem of the Alternative for Personalized Federated Learning." arXiv preprint arXiv:2103.01901 (2021).

[3] Hanzely, Filip, and Peter Richtárik. "Federated learning of a mixture of global and local models." arXiv preprint arXiv:2002.05516 (2020). The proposed algorithm is interesting and its performance looks promising. However, considering the fact that the proposed framework bears similarities with earlier works on personalized FL, my score is 5: marginally below the acceptance threshold.

This paper proposes a new setting for federated learning -- Selfish Federated Learning -- where all clients (internal and external) participate in the training, while the trained model needs to give low error only for the internal clients. The main idea is to reweight the loss functions and give small weights to external clients so that the overall objective is closer to the true risk of internal clients. The paper provides some theory for methods where the optimal weights are known and proposes a heuristic method VaRSeL to estimate the optimal weights on the fly. **Motivation.** The motivation of the proposed new setting *Selfish Federated Learning* is not well substantiated. There is only one short paragraph on two examples, which are not quite convincing. For instance, what are the incentives of external clients to participate in the training (provide data in the form of gradients, spend resources on computation and communication) if they are not going to benefit from the trained model? To be clear, I am not criticizing the setting and claiming that is unfruitful. Quite the opposite, I suggest devoting more space, building intuition and motivation behind this setting at the very beginning. As an example, I could imagine some online services, which allow training (to some limited extent) on their data without viewing raw data.

**Internal clients' distribution.** On page 5, it is written that "Since the internal gradients are unbised estimators of the ultimate gradients, we have"
$$\mathbb{E}[\nabla l_i(\theta)] = \mathbb{E}[\nabla l_{I}(\theta)] = \nabla l(\theta), \qquad 1\le i \le M.$$

First, I presume that the expectation here is with respect to local subsampling $S_i \sim \mathcal{D}_i$, which is not clarified in the text. Second, to have those equalities, it is necessary to have all internal clients' distributions $\mathcal{D}_1 = \mathcal{D}_2 = \dots = \mathcal{D}_M$ ***the same***.


**Ultimate objective.** What does $\mathbb{E}_{(x,y)\sim \mathcal{D}_1, \dots, \mathcal{D}_M}$ mean in (2), the definition of $l(\theta)$ ? The wording up to (2) suggests that

$ l(\theta) = \frac{1}{M}\sum_{i=1}^M \mathbb{E}_{(x,y)\sim \mathcal{D}_i} \mathcal{L} (f(x,\theta),y).$

**Claims before section 4.** *" we will show both in theory and in experiments that it is probable to improve the estimation of $\theta$ by leveraging additional knowledge from external clients without privacy leakage."*

Which part of the theory supports this claim? Do you have privacy guarantees to prevent privacy leakage?

**Strategy (section 4.1)** *"make our surrogate objective (3) simulate the ultimate objective (2)."*

Objective (3) includes empirical loss functions of external clients, whose distributions are expected to be very different over the internal clients'. On the other hand, objective (2) consists of true risks (not empirical losses as in (3)) of only internal clients. So, you assume that the external clients' distributions are largely different but somehow want to use them to improve the objective composed of only internal clients' data. I failed to see the intuition behind this.


**Proof of Lemma 1**. How do you justify the inequality in (9), namely $\sum_{i=1}^M \mathbb{E}\\|\nabla l_i(\theta) - \nabla l(\theta)\\|^2 \le M^2\sigma\\|\nabla l(\theta)\\|^2$ ?

What if $\lambda_1=0$ ? Then the function $h(\omega)$ is monotonically decreasing and $\omega_* = +\infty$. Does this mean that internal clients should be ignored in those cases ?


**Theorem 2.** I presume that the method analyzed in Theorem 2 is of theoretical interest only since it uses the optimal weights $w_*^t$ in each iteration, which depends on $\nabla l(\theta)$ (full gradient of true risk). Even so, the rate (5) is very weak. First, the upper bound on $\eta$ implies $A<\frac{\mu}{L}$, which is quite restrictive. Second, the noise term has additive part $\frac{B}{\mu^2}$, which is fixed and independent of how we choose step size $\eta$ or sample $S_i\sim\mathcal{D}_i$.

**Theorem 3 (Main convergence).** No discussion of what the main convergence result means and implies. It is claimed that $\Gamma_1, \Gamma_2, \Gamma_3$ are constant, but in the proof, they depend on both $\eta$ and $\tau$.


**VaRSeL method.** I think, the terminology "variance reduction" is not appropriate here. The method is a heuristic, which approximates several steps discussed in the paper (e.g. $\nabla l(\theta)$ is approximated by $\nabla l_I(\theta)$).





 The proposed setting is not well motivated. There are several issues and inconsistencies in theoretical arguments.

Meta-review: The paper proposes an FL framework that optimizes the performance of a subset of clients. Reviewers did appreciate the value of several contributions, but unfortunately consensus is that it remains below the bar, even after the discussion phase. Concerns remained on correctness issues, motivation of assumptions, and distinction of personalized vs selfish FL, even after the author feedback.

We hope the detailed feedback helps to strengthen the paper for a future occasion.

Decision: Reject