Year: 2021

Abstract:
Most federated learning (FL) algorithms aim to learn a model which achieves optimal overall performance across all clients. However, for some clients, the model obtained by conventional federated training may perform even worse than that obtained by local training. Therefore, for a stakeholder who only cares about the performance of a few internal clients, the outcome of conventional federated learning may be unsatisfactory. To this end, we study a new selfish variant of federated learning, in which the ultimate objective is to learn a model with optimal performance on internal clients alone instead of all clients. We further propose Variance Reduction Selfish Learning (VaRSeL), a novel algorithm that reweights the external clients based on variance reduction for learning a model desired in this setting. Within each round of federated training, it guides the model to update towards the direction favored by the internal clients. We give a convergence analysis for both strongly-convex and non-convex cases, highlighting its fine-tune effect. Finally, we perform extensive experiments on both synthesized and real-world datasets, covering image classification, language modeling, and medical image segmentation. Experimental results empirically justify our theoretical results and show the advantage of VaRSeL over related FL algorithms.

heading: 1 INTRODUCTION
text: Federated learning (FL) proposes a privacy-preserving scheme, which enables different clients to cooperatively learn a global model by integrating knowledge from multiple clients. Suppose there are Q clients {Dn}Qn=1 in total. A general objective of FL is given by
min f Q∑ n=1 LDn(f), (1)
where f is the model, and LDn is the generalization loss on client Dn. However, since the general objective (1) fairly optimizes the mean generalization loss across all clients, a model learned by (1) may not perform well on some individual clients. We implemented several state-of-the-art FL algorithms on FeTS 2021 (Pati et al., 2021), a real-world FL dataset for medical image segmentation, and discovered that the federally learned model is not guaranteed to generalize on local client(s) at least as well as the locally learned model. (see Figure 1)
In cross-silo FL (Kairouz et al., 2019), this issue becomes particularly serious since clients typically have sufficient data for training a decent local model. In this case, the clients may become reluctant to participate in federated training. This issue raises an important challenge for FL society as well as practitioners: For a stakeholder who is only interested in the model’s performance on a small portion of clients, is it possible to guarantee its benefits from federated learning?
This is a common challenge in many practical cases. For instance, in a bank network, due to the limited size of internal datasets, a banking group wants to leverage more data and learn a better model through federated learning, while the banking group expects the final model to maximize the performance on its own internal banks, rather than overfitting to other external banks. As another example, to build an AI-assisted diagnostic system, a medical association hopes to integrate knowledge from the datasets of external partner hospitals through federated learning. As the system ultimately serves
its internal hospitals/patient populations, its performance should not be disturbed by external partner hospitals (Rieke et al., 2020). In these cases, to enhance the model perf the stakeholder would like to purchase gradient updates from external collaborators.
Selfish federated learning In this work, we resolve the aforementioned challenge by investigating a novel learning setting called selfish federated learning (Selfish-FL), where the stakeholder who only cares about the performance on a few internal clients can reap the benefits of cooperative learning with external clients. To formalize, the Q clients are partitioned into M internal clients and N external clients (M +N = Q, M N ). Though it is possible to incorporate the knowledge in all Q clients for model training, the stakeholder only focuses on the model’s performance on the M internal clients.
Comparisons with related setups Table 1 lists several FL settings. While centralized and decentralized FL differ in communication network, both of them are looking for a global consensus among all the clients (Wang et al., 2021; Koloskova et al., 2020). Personalized-FL allows each client to maintain a personalized model, but the losses of all clients are still considered in the overall objectives (Fallah
et al., 2020; Tan et al., 2021). In Selfish-FL, only internal clients are the concern of the stakeholder, while the role of external clients is to provide additional information to further improve the model’s performance on internal clients.
Our contributions The main contributions in this paper can be summarized as four aspects: First, we study a novel setting in federated learning, namely “Selfish-FL”, which is the first work (to the best of our knowledge) considering biased objectives and internal/external partitions in FL setting.
Second, we propose a novel algorithm, called VaRSeL, for solving Selfish-FL problem. It leverages knowledge from external clients to improve the model via gradients variance reduction. This is a different perspective compared with previous loss-based clients reweighting strategy (Cho et al., 2020; Chen et al., 2020). Within each round of federated training, it guides the model to update towards the direction favored by the internal clients. Moreover, our strategy is free of hyperparameter-tuning.
Third, we present a convergence analysis of our reweighting strategy for this new setting, both in strongly-convex cases and non-convex cases. Based on the convergence result, we also observe its fine-tune effect.
Finally, we perform extensive experiments on both synthesized and real-world natural language and medical image analysis datasets. The experimental results demonstrate that our proposed VaRSeL
outperforms the classical FedAvg, related client selection methods, as well as the state-of-the-art for non-iid data (FedProx) and personalized FL algorithm (Per-FL) in terms of model accuracy.

heading: 2 RELATED WORKS

heading: 2.1 FEDERATED LEARNING AND CHALLENGES
text: Federated learning (FL) (Yang et al., 2019; Kairouz et al., 2019; Wang et al., 2021) provides a promising way of cooperative learning with privacy protection. In this subsection, we will briefly summarize a few important lines of FL (Some works may lie at intersection):
Relieving clients’ non-iid Proximal terms (Li et al., 2020c), cluster-based methods (Briggs et al., 2020; Ghosh et al., 2020), client-drift regulations (Karimireddy et al., 2020; Acar et al., 2021) are some leading methods to address the heterogeneity of clients’ data distributions. At a high level, all these methods can be summarized as pursuing a compromise solution that can work fairly well on all the clients. Therefore, even though these methods are very effective in classical FL frameworks, none of them fits well into our selfish-FL framework, because the internal clients are in nature reluctant to compromise with any external clients.
Personalized FL Many approaches have been proposed for achieving personalization in FL setting (Kairouz et al., 2019), e.g., mixing the global model and user’s local model, dividing the network into base and personalized layers. Recently, Per-FL (Fallah et al., 2020) linked personalized FL with meta-learning by building a good initial meta-model that can be updated effectively. Dinh et al. (2020) used Moreau envelopes as clients’ regularization loss to achieve personalized FL and Zhang et al. (2020) proposed a flexible FL framework that allows clients to personalize to specific target data distributions. Readers can refer to comprehensive surveys in Li et al. (2020a); Tan et al. (2021). However in personalized FL, the losses of all clients are still considered in the overall objectives.
Alleviating communication cost Lin et al. (2017); Richtárik et al. (2021) provided several model compression schemes to reduce communication cost and proved the convergence of their compression methods. Liang et al. (2020); Li et al. (2021) proposed to learn a combination of local and global models, while only the global parts are synchronized. It’s worth mentioning that our method is orthogonal to these communication-efficient methods and can be combined with theirs directly.

heading: 2.2 SAMPLE REWEIGHTING.
text: Our work is also related to sample reweighting, i.e., assigning different weights to samples in model training to tackle dataset biases. There are two completely opposite strategies on how to set the weights w for external datasets. One line of works Freund et al. (1996); Malisiewicz et al. (2011); Cho et al. (2020) argues that the samples with higher losses should be given more weights, since these difficult samples can help sharpen the decision boundaries and increase the adversarial robustness. The other line of works Ghadikolaei et al. (2019); Song et al. (2021) suggests lowering their weights, since their high losses are attributed to training set biases or mislabels (e.g., nosiy samples). In fact, this contradiction directly points to a fundamental, difficult problem in machine learning:
Dilemma. How to distinguish the difficult samples from the noisy ones?
To the best of our knowledge, the controversy continues, and people are still far from getting a clear resolution. To bypass the aforementioned dilemma, some researchers (Ren et al., 2018; Shu et al., 2019) used meta-learning approach to learn how to assign weights w adaptively. However, their method suffers from computationally expensive second derivatives estimation, and can hardly generalize to federated learning framework since it requires full-client participation throughout training.
Client reweighting in FL In FL framework, since one only has access to limited training examples, clients instead of samples are reweighted to tackle their heterogeneity. Existing FL algorithms reweight clients using the sizes of their datasets (Li et al., 2020c), the numbers of their local steps (Wang et al., 2020), their local losses (Cho et al., 2020; Chen et al., 2020), etc.
As an inevitable issue in FL, communication costs usually restrict the number of clients one can access within each communication round (McMahan et al., 2017). Suppose client j is sampled with probability wj ∈ [0, 1],M+1 ≤ j ≤M+N . Since the communication costs mainly depend on the number of sampled clients, existing algorithms (Lian et al., 2018; Wang et al., 2019) impose hard constraints on wj as 1Tw = ∑M+N j=M+1 wj ≤ CbN = K, whereCb > 0 is called the communication budget, to alleviate the communication bottleneck.

heading: 3 PROBLEM FORMULATION
text: In this section, we give a formal definition of the setting selfish federated learning (Selfish-FL).
Internal and external clients Let X denote the input space (e.g., feature space) and Y the space of outputs (e.g., label space). Each client characterizes a joint distribution D = P (X,Y ) over X × Y . Suppose there are M + N clients {Dn}M+Nn=1 , among which the first M clients are called the internal clients and the remaining N clients are called the external clients (M N ). We assume that internal clients are consistent and the difference between internal and external clients are expected to be larger than that between internal clients. To formalize, we have
D1 = · · · = DM , DI ; Dj 6= DI , M + 1 ≤ j ≤M +N.
In practice, the distributions of clients are not observed directly, but given in the form of sample sets Sn ∼ Dn, for 1 ≤ n ≤ M + N . The stakeholder has direct access to the raw data in S1, . . . , SM , but can only query external clients in a privacy-preserving way as in typical FL scenario.
Ultimate objective While conventional federated learning setting usually aims to minimize the generalization loss across all clients, the objective in Selfish-FL is to minimize the generalization loss on internal clients D1, · · · ,DM alone:
min θ
l(θ) := 1
M M∑ i=1 E(x,y)∼DiL(fθ(x), y), (2)
where fθ denotes a model f parameterized by θ, and L denotes an appropriate loss function. Although the stakeholder can simply learn θ by local training on datasets of internal clients, we will show both in theory and in experiments that it is probable to improve the estimation of θ by leveraging additional knowledge from external clients without sharing raw data.

heading: 4 VARSEL: VARIANCE REDUCTION SELFISH LEARNING

heading: 4.1 REWEIGHTING VIA VARIANCE REDUCTION
text: In Selfish-FL, we incorporate the knowledge of external clients to further improve the performance of model fθ on internal clients. Since external clients are of uneven qualities, it’s natural to assign them different weights and formulate a surrogate objective as
l(θ;w) =
∑M i=1 li(θ) + ∑M+N j=M+1 wj lj(θ)
M + 1Tw , (3)
where w ∈ [0, 1]N , 1Tw ≤ K, and ln(θ) = 1|Sn| ∑
(x,y)∈Sn L(fθ(x), y) is the empirical loss on client n. As a result, how to set the weights w is a key issue in designing Selfish-FL algorithms. In this paper, to bypass the dilemma in section 2.2, we consider the client reweighting strategy from a different perspective compared with previous works:
Strategy. We’re setting w neither to reduce losses nor to increase losses, but to make our surrogate objective (Eq. (3)) simulate the ultimate objective (Eq. (2)).
Since the majority of optimizers applied in deep learning are first-order methods (e.g., SGD, Adam) (Wang et al., 2021), we consider making the estimated gradient as precise as possible. Specifically,
the goal is to reduce the variance (denoted as Φ(w; θ)) between the estimated, surrogate gradient ∇l(θ;w) and the unseen, ultimate gradient∇l(θ)1:
Φ(w; θ) = E‖∇l(θ;w)−∇l(θ)‖2
= 1 (M + 1Tw)2 · E ∥∥∥M(∇lI(θ)−∇l(θ)) + M+N∑
j=M+1
wj(∇lj(θ)−∇l(θ)) ∥∥∥2,
where lI(θ) = 1M ∑M i=1 li(θ). Since the internal gradients are unbiased estimators of the ultimate gradients, we have
E[∇li(θ)] = E[∇lI(θ)] = ∇l(θ), 1 ≤ i ≤M.
Thus, the variance Φ(w; θ) can be simplified as follows:
Φ(w; θ) = 1
(M + 1Tw)2 [ M∑ i=1 E‖∇li(θ)−∇l(θ)‖2 + E ∥∥∥ M+N∑ j=M+1 wj ( ∇lj(θ)−∇l(θ) )∥∥∥2]. (4) The optimal weights w∗ are thus chosen to minimizes the variance in Equation (4):
w∗ = arg min w Φ(w; θ).
In this way, we can head towards the optimal direction for the ultimate objective.

heading: 4.2 CONVERGENCE ANALYSIS
text: In this section, we give a convergence analysis for our reweighting strategy. Our proofs mainly follow Li et al. (2020c), and we refer readers to Appendix A and B for the details of the proofs.
As in Schmidt & Roux (2013); Li et al. (2020b); Vaswani et al. (2019), we assume bounded dissimilarity on internal datasets. For external datasets, we take a weaker assumption by allowing non-vanishing noises (νj) near the optimal point. In general, our assumptions are weaker than the constantly-bounded noise assumptions since the noises are allowed to grow with the gradient norms.
ASSUMPTION 1 For internal datasets, we assume: for some constant σ > 0,
E‖∇lI(θ)−∇l(θ)‖2 ≤ σ‖∇l(θ)‖2;
For external datasets, we take weaker assumptions: for each M + 1 ≤ j ≤ M + N , for some constants κj , νj > 0, E‖∇lj(θ)−∇l(θ)‖2 ≤ κj‖∇l(θ)‖2 + νj . Lemma 1. Under Assumption 1, there exist constants A,B > 0, such that
Φ(w∗; θ) ≤ A‖∇l(θ)‖2 +B.
To ensure convergence, we further assume that the objective functions are well-conditioned.
ASSUMPTION 2a The ultimate objective function is L-smooth (L > 0):
‖∇l(θ1)−∇l(θ2)‖ ≤ L‖θ1 − θ2‖.
ASSUMPTION 2b The empirical loss functions are L-smooth (L > 0):
‖∇ln(θ1)−∇ln(θ2)‖ ≤ L‖θ1 − θ2‖, 1 ≤ n ≤M +N.
ASSUMPTION 3 The ultimate objective is µ-strongly convex (µ > 0):
‖∇l(θ1)−∇l(θ2)‖ ≥ µ‖θ1 − θ2‖. 1The “variance reduction” here is essentially different from that in conventional FL. In conventional FL, it reduces the variance between all clients to speedup the global convergence; in selfish-FL, it can be seen as reducing the variance between internal and external clients, thus alleviating the disturbance of external noises.
ASSUMPTION 4 The estimated gradient norms are uniformly bounded:
E‖∇ln(θ)‖2 ≤ G2, 1 ≤ n ≤M +N.
To gain some insights, we start our analysis from a well-conditioned case and assume local steps τ = 1. Using the bound of Lemma 1, we can derive Theorem 2 on convergence rate. The first term on the right hand side of Eq. (5) is called optimization term, and the second is called noise term. The convergence rate is bounded by the sum of the two terms. Theorem 2 (Basic Convergence). Let θt+1 = θt − η∇l(θt;wt∗), 0 ≤ t ≤ T − 1. Under Assumption 1, Assumption 2a and Assumption 3, for η < 1 L− A µ
2 √ A , we have
E‖θ∗ − θt‖2 ≤ ( 1− ηΞ1 )tE‖θ∗ − θ0‖2︸ ︷︷ ︸ optimization term + Ξ2B︸︷︷︸ noise term , (5)
where Ξ1 and Ξ2 are constants,
The basic convergence result (Eq. (5)) provides useful insights for model training. At the beginning of training, due to random initialization, the optimization term can be arbitrarily large. But as T increases, the optimization term decays in linear rate. Consequently, the dominant term will be gradually shifted to noise term, which is mainly determined by the noises in external datasets. Based on this observation, when training progresses to later stage, in order to get more precise gradients, it will gradually switch back to internal datasets and fine-tune the trained model at the end of learning.
Taking multiple steps between communication rounds is an entrenched convention in communication efficient federated learning (McMahan et al., 2017; Kairouz et al., 2019). Hence we give a more general convergence analysis, in which local steps τ ≥ 1. We formalize it as follows:
θtn = θ t,
θ t+ k+1τ+1 n = θ t+ kτ+1 n − η∇ln(θ t+ kτ+1 n ), 1 ≤ n ≤M +N, for 0 ≤ k ≤ τ − 1,
θt+1 = θt+ τ τ+1 w,
where θw = ∑M i=1 θi+ ∑M+N j=M+1 wjθj
M+1Tw .
Theorem 3 (Main Convergence). For τ ≥ 1, under Assumption 1, Assumption 2a, Assumption 2b and Assumption 4, for η < 2(1− √ A)
L(A+ √ A) , we have
1
τT T−1∑ t=0 τ−1∑ k=0 E ∥∥∥∇l(θt+ kτ+1 wt∗)∥∥∥2 ≤ l ( θ0 ) − l ( θ∗ ) ηΓ3τT + Γ1Γ2 Γ3τ ,
where Γ1, Γ2 and Γ3 are constants.
Algorithm 1 Variance Reduction Selfish Learning (VaRSeL) Input: learning rate η, communication rounds T . Output: final model θfinal.
1: Initialize model θ0. 2: for t = 0, 1 · · · , T − 1 do 3: Compute internal gradients ∇l1(θt), · · · , ∇lM (θt), and then∇lI(θt). 4: Broadcast (θt,∇lI(θt)) to clients [M + 1,M +N ]. 5: for j ∈ [M + 1,M +N ] in parallel do 6: Compute gradient update∇lj(θt). 7: Send back ‖∇lj(θt)−∇lI(θt)‖. 8: end for 9: Solve w̃ according to Equation (8). 10: Sample a subset of clients St according to w̃. 11: for j ∈ St in parallel do 12: Send back ∇lj(θt). 13: end for 14: Solve w∗ according to Equation (4). 15: ∆t = M∇lI(θt)+wT∗[∇lj(θ t)] j∈St M+1Tw∗ 16: θt+1 := θt − η∆t 17: end for 18: return θfinal := θt

heading: 4.3 VARIANCE REDUCTION SELFISH LEARNING
text: Based on our above reweighting strategy, we design an algorithm, called Variance Reduction Selfish Learning (VaRSeL). It is free of hyperparameter-tuning, and moreover, as shown in experiments, it will automatically fine-tune the model when learning progresses to later stage. VaRSeL is not exactly the reweighting strategy in section 4.1, but an approximation algorithm to the reweighting strategy,
The main idea of VaRSeL is to select the external collaborators wisely within each communication round. Within iteration t, suppose the central server has collected the gradient updates on internal datasets. Then we can estimate the ultimate gradient unbiasedly:
E[∇lI(θt)] = l(θt). (6)
Communication bottleneck is a major obstacle that prevents us from accurately solving the optimal weights w∗ in Equation (4). To this end, we propose a heuristic:
Φ(w; θ) = 1
(M + 1Tw)2 [ M∑ i=1 E‖∇li(θ)−∇l(θ)‖2 + E ∥∥∥ M+N∑ j=M+1 wj ( ∇lj(θ)−∇l(θ) )∥∥∥2]
≤ 1 (M + 1Tw)2 [ M∑ i=1 E‖∇li(θ)−∇l(θ)‖2 + 1Tw M+N∑ j=M+1 wj E ∥∥∥∇lj(θ)−∇l(θ)∥∥∥2︸ ︷︷ ︸
communicated term
] , Φ̃(w; θ). (7)
Before the external clients send back their updates, we first estimate an approximate weight:
w̃ = arg min w Φ̃(w; θ). (8)
This approximate weight tightens the upper bound in (7) and can be computed very efficiently, since only a few bytes of communication are required. Then, we sample clients according to w̃, and after receiving their updates, we can solve a more accurate w∗ according to Equation (4).
The pseudo-code of VaRSeL is given in Algorithm 1. This is a simple implementation when local steps τ equals 1. In fact, τ is allowed to be greater than 1, and we almost don’t modify any other parts of the algorithm, except to replace the gradient update with the accumulated model update.

heading: 5 EXPERIMENTS
text: In this section, we present experimental results on both synthesized benchmark datasets and realworld datasets. For synthesized benchmark experiments, we mainly use them to support the claims made in our paper. For real-world dataset experiments, we show the real-world performance of our method and its higher accuracy over the other competitors.
We compared VaRSeL with seven existing methods mentioned in section 2:
• Baseline methods, including Local Training (i.e., training the model with internal datasets) and the original FedAvg (McMahan et al., 2017);
• Non-iid relief methods, FedProx (Li et al., 2020b); • Reweighting methods, including Skew (Cho et al., 2020), SL (Song et al., 2021), and OCS
(Chen et al., 2020); • Personalized FL methods, the MAML-based Per-FL (Fallah et al., 2020).

heading: 5.1 EXPERIMENTS ON SYNTHESIZED BENCHMARK DATASETS
text: Dataset and settings We conducted experiments on three benchmark datasets, Fashion-MNIST, EMNIST, and CIFAR-10 (Xiao et al., 2017; Cohen et al., 2017; Krizhevsky, 2009) to demonstrate the robustness and fine-tune effect of our methods. The datasets were preprocessed in similar ways as McMahan et al. (2017). We used a logistic model for Fashion-MNIST, a 2NN model for EMNIST, and a CNN model for CIFAR-10. For model training, we used cross entropy loss and SGD optimizer, setting batch size b = 32. Every result is taken from the average of 3 independent runs. More details about data preprocessing and model architecture can be found in Appendix C.
Robustness We’ve tested the robustness of VaRSeL under different learning rates and local steps. Learning rates range from 0, 3, 0.1, 0.03, 0.01, and local steps range from 1, 3, 5. As shown in Table 2, our method outperforms the best accuracy of other competitors by at least 3.94% in FashionMNIST, 1.59% in EMNIST, and 2.7% in CIFAR-10.
The fine-tune effect As indicated in our basic convergence analysis, the external weights w should decay gradually. And in VaRSeL, the weights w∗ is computed adaptively within each iteration, so the fine-tuning will take effect automatically without any human intervention or additional hyperparameters. In Figure 2, we plot the sum of external weights (1Tw) in our synthesized experiments, as an indicator of the fine-tune effect. It can be observed that the external weights gradually decrease during the training process, which is consistent with our theoretical analysis.

heading: 5.2 EXPERIMENTS ON SHAKESPEARE DATASET
text: Dataset and settings We used an FL dataset derived from The Complete Works of William Shakespeare (Shakespeare, 2007). There are altogether 715 clients, each of whom corresponds to a role in one of the Shakespeare’s six classic plays—‘All’s Well That Ends Well’, ‘Much Ado About Nothing’, ‘Pericles, Prince of Tyre’, ‘The First Part of King Henry the Fourth’, ‘The Taming of the Shrew’, and ‘The Tragedy of King Lear’. The clients’ datasets are comprised of their lines in the play. Finally, we designate the roles in one of the six plays as internal clients, and those in the remaining five plays as external clients.
Details of implementation and results Following McMahan et al. (2017), we used a 2-layer LSTM for the prediction of next character. For model training, we employed cross entropy loss, learning rate η ∈ [0.5, 5.0], local steps τ ∈ {1, 2, 3}, and batch size b = 4. We took the average results of three independent runs of 200 communication rounds. As shown in Table 3, the average score of VaRSeL outperforms other competitors by at least 0.89%.

heading: 5.3 EXPERIMENTS ON MEDICAL IMAGE DATASET
text: Dataset and settings Tumor segmentation is one of challenging tasks (Hesamian et al., 2019) due to the limitation annotations, the imbalance of data distribution, and the heavy noises in different image scans. In this subsection, we show the real-world performance of our method and other competitors on FeTS 2021 (Pati et al., 2021), a real-world brain tumor segmentation dataset composed of clinically acquired, magnetic resonance imaging (MRI) scans from 17 different medical sites.
To form the internal and external clients, we took one of the three largest sites (1st, 6th, or 16th) as the internal site, and the rest as the external ones. Furthermore, for each setting, the chosen internal site was partitioned equally into 3 parts, which are distributed to 3 internal clients respectively to simulate more internal clients. The MRI scans include various modalities (Bakas et al., 2018), say a) native (T1), b) post-contrast T1-weighted (T1Gd), c) T2-weighted (T2), and d) T2 Fluid Attenuated Inversion Recovery (T2-FLAIR). For whole brain tumor segmentation, T2-FLAIR is reported to better identify the malignant features (Zeineldin et al., 2020), To highlight the nature of internal/external partition, the internal datasets are all organized in T2-FLAIR, while every external dataset randomly picks up from one modality to enlarge the distribution shift of different datasets.
Details of implementation and results We used the 2D U-Net (Ronneberger et al., 2015) with instance normalization (Ulyanov et al., 2016). We took the following measures to ensure fair competitions between all the methods: (i) Try gradient descent with three different learning rates η = 0.3, 0.1, 0.03; (ii) Try three different local steps τ = 1, 3, 5; (iii) For the best learning rate and local steps, we take the average results of three independent runs of 60 communication rounds. The experimental results are summarized in Table 4. It is observed that our VaRSeL outperforms local training on all three sites, show the effectiveness of incorporating external sites to boost the performance of internal sites. Also, the average score of VaRSeL largely outperforms other competitors by over 2.26%, showing the effectiveness of our proposed framework.

heading: 6 CONCLUSIONS AND REMARKS
text: This work proposes a novel setting called “selfish federated learning” that cooperatively optimizes a biased objective towards internal distributions in a communication-efficient and heterogeneous privacy-preserving way. The central idea is to reweight the external datasets via a variance-reduction approach, based on which we develop “VaRSeL” (Algorithm 1). We provide convergence analysis for our reweighting strategy, and highlight its fine-tune effect. In synthesized and realistic experiments, we support the theoretical results made in our paper, and demonstrate its better performance compared to alternative methods. Our code is publicly available and we believe that VaRSeL can find a wide range of applications, such as healthcare (Rieke et al., 2020).

heading: APPENDIX
text: Roadmap of Appendix The Appendix is organized as follows. We provide the theoretical proof of basic convergence results in Section A and main convergence results in Section B. The details of experimental setting (e.g., model architectures and dataset preprocessing) are in Section C.

heading: A PROOF OF BASIC CONVERGENCE RESULTS

heading: A.1 PRELIMINARIES
text: Proposition 4. For γ > 0, 2〈u,v〉 ≤ γ‖u‖2 + 1γ ‖v‖ 2.
Proof. According to Cauchy-Schwarz inequality, we have
2〈u,v〉 = 2〈√γu, 1√ γ v〉 ≤ 2‖√γu‖ · ‖ 1√ γ v‖ ≤ γ‖u‖2 + 1 γ ‖v‖2.
Proposition 5. Let ∑N i=1 pi = 1, pi > 0 for 1 ≤ i ≤ N . Then
E‖ N∑ i=1 pixi‖2 ≤ N∑ i=1 piE‖xi‖2.
Proof. This follows from Jensen’s inequality and the convexity of E‖x‖2.
Lemma 6. For θ ∈ ΘH, we have
l(θ∗)− l(θ) ≤ − 1 2L ‖∇l(θ)‖2.
Proof. Let θ′ = θ − 1L∇l(θ). Then
l(θ′) ≤ l(θ) + 〈∇l(θ), θ′ − θ〉+ L 2 ‖θ′ − θ‖2
≤ l(θ) + 〈∇l(θ),− 1 L ∇l(θ)〉+ L 2 ‖ 1 L ∇l(θ)‖2 = l(θ)− 1 2L ‖∇l(θ)‖2.
Hence l(θ∗)− l(θ) ≤ l(θ′)− l(θ) ≤ − 1
2L ‖∇l(θ)‖2.
Lemma 7. Let {an} be the sequence of non-negative real numbers. If 0 < α < 1, β > 0, and
an+1 ≤ (1− α)an + β,
then an ≤ (1− α)na0 + β
α .
Proof. Since
an − β
α ≤ (1− α)(an −
β α ),
we can simply prove by induction that
an − β
α ≤ (1− α)n[a0 −
β α ],
and then an ≤ (1− α)na0 + [1− (1− α)n] β
α
≤ (1− α)na0 + β
α .

heading: A.2 PROOF OF LEMMA 1
text: Proof. We have:
Φ(w; θ)
= 1
(M + 1Tw)2 [ M∑ i=1 E‖∇li(θ)−∇l(θ)‖2 + E ∥∥∥ M+N∑ j=M+1 wj ( ∇lj(θ)−∇l(θ) )∥∥∥2]
≤ 1 (M + 1Tw)2
[ M2σ‖∇l(θ)‖2 + E ∥∥∥ M+N∑ j=M+1 wj ( ∇lj(θ)−∇l(θ) )∥∥∥2]
= 1
(M + 1Tw)2
( M2σ‖∇l(θ)‖2 + E [ (ΣTw)T(ΣTw) ]) = 1
(M + 1Tw)2
( M2σ‖∇l(θ)‖2 + E [ wT(ΣΣT)w ]) ,
(9)
where
Σ =  ∇lM+1(θ)−∇l(θ)... ∇lM+N (θ)−∇l(θ)  ∈ RN×d. Since the right hand side of Equation (9) is an upper bound of Φ(w; θ), and we want to make this upper bound tight by taking its minimum (w.r.t. w̃):
w̃ = argmin w
1
(M + 1Tw)2
( M2σ‖∇l(θ)‖2 + E [ wT(ΣΣT)w ]) .
Clearly, whatever w̃ you take, RHS(9) will always be no less than Φ(w∗; θ).
Let λ1 ≤ λ2 ≤ · · · ≤ λN denote all the N eigenvalues of ΣΣT. Then
Nλ1 ≤ M+N∑ j=M+1 λj = trace ( ΣΣT ) = M+N∑ j=M+1 E‖∇lj(θ)−∇l(θ)‖2. (10)
Let e be the corresponding unit eigenvector (i.e.eTe = 1) of λ1. Then, in order to minimize the RHS(9), w̃ and e must be collinear vectors. Therefore, we can assume that
w̃ = ωe.
Substituting this into RHS(9), we have
1
(M + 1Tw̃)2
( M2σ‖∇l(θ)‖2 + E [ w̃T(ΣΣT)w̃ ]) = 1
(M + ω1Te)2
( M2σ‖∇l(θ)‖2 + ω2E [ eT(ΣΣT)e ]) = 1
(M + ω1Te)2
( M2σ‖∇l(θ)‖2 + λ1ω2 ) , h(ω)
Take derivative with respect to ω:
∂h ∂ω =
2λ1ω(M + ω1 Te)2 − ( M2σ‖∇l(θ)‖2 + λ1ω2 ) · 21Te(M + ω1Te)
(M + ω1Te)4 = 0.
Thus, ω = λ−11 · 1TeMσ‖∇l(θ)‖2, and then
h(ω) = σ‖∇l(θ)‖2
1 + λ−11 (1 Te)2σ‖∇l(θ)‖2
(10) ≤ σ‖∇l(θ)‖ 2
1 + N(1 Te)2σ‖∇l(θ)‖2∑M+N
j=M+1 E‖∇lj(θ)−∇l(θ)‖2
= 1
1 σ‖∇l(θ)‖2 + N(1Te)2∑M+N j=M+1 E‖∇lj(θ)−∇l(θ)‖2 .
According to Assumption 1, we further have
h(ω) ≤ 1 1
σ‖∇l(θ)‖2 + N(1Te)2∑M+N
j=M+1 E‖∇lj(θ)−∇l(θ)‖2
≤ 1 1
σ‖∇l(θ)‖2 + N(1Te)2 ( ∑M+N j=M+1 κj)‖∇l(θ)‖2+ ∑M+N j=M+1 νj .
Further, due to AM-HM inequality, we have
h(ω) ≤ 1 M
Mσ‖∇l(θ)‖2 + N(1Te)2 ( ∑M+N j=M+1 κj)‖∇l(θ)‖2+ ∑M+N j=M+1 νj
≤ Mσ‖∇l(θ)‖2 + (1Te)−2
[ ( ∑M+N j=M+1 κj)‖∇l(θ)‖2 + ∑M+N j=M+1 νj ] (M +N)2
Therefore, Φ(w∗; θ) can be finally bounded as follows:
Φ(w∗; θ) ≤ h(ω) ≤ Mσ‖∇l(θ)‖2 + (1Te)−2
[ ( ∑M+N j=M+1 κj)‖∇l(θ)‖2 + ∑M+N j=M+1 νj ] (M +N)2
= Mσ + (1Te)−2
∑M+N j=M+1 κj
(M +N)2︸ ︷︷ ︸ A
‖∇l(θ)‖2 + (1Te)−2
∑M+N j=M+1 νj
(M +N)2︸ ︷︷ ︸ B
, A‖∇l(θ)‖2 +B.
(11)
Comments. From Formula (11), we can observe that B is mainly determined by ∑M+N j=M+1 νj , i.e. the noises in the external datasets.

heading: A.3 PROOF OF THEOREM 2
text: Proof. Due to the µ-strongly convexity, we have
l(θ∗) ≥ l(θt) + 〈∇l(θt), θ∗ − θt〉+ µ 2 ‖θ∗ − θt‖2
= l(θt) + 〈∇l(θt;w), θ∗ − θt〉 − 〈∇l(θt;w)−∇l(θt), θ∗ − θt〉+ µ 2 ‖θ∗ − θt‖2 ≥ l(θt) + 〈∇l(θt;w), θ∗ − θt〉+ µ 2 ‖θ∗ − θt‖2
− 1 2
[ γ1‖∇l(θt;w)−∇l(θt)‖2 + 1
γ1 ‖θ∗ − θt‖2 ] = l(θt) + 〈∇l(θt;w), θ∗ − θt〉+ (µ
2 − 1 2γ1 )‖θ∗ − θt‖2 − γ1 2 ‖∇l(θt;w)−∇l(θt)‖2,
(12)
where γ1 > 0.
Since θt+1 = θt − η∇l(θt;w), it yields
〈∇l(θt;w), θ∗ − θt〉 = −1 η 〈θt+1 − θt, θ∗ − θt〉
= − 1 2η
[ ‖θt+1 − θt‖2 + ‖θ∗ − θt‖2 − ‖θ∗ − θt+1‖2 ] = − 1
2η
[ η2‖∇l(θt;w)‖2 + ‖θ∗ − θt‖2 − ‖θ∗ − θt+1‖2 ] = −η
2 ‖∇l(θt;w)‖2 − 1 2η ‖θ∗ − θt‖2 + 1 2η ‖θ∗ − θt+1‖2.
(13)
Based on Lemma 1, we have
E‖∇l(θt;w∗)−∇l(θt)‖2 ≤ AE‖∇l(θt)‖2 +B, (14)
and then E‖∇l(θt;w∗)‖2 = E‖∇l(θt) +∇l(θt;w∗)−∇l(θt)‖2
≤ γ2E‖∇l(θt)‖2 + 1
γ2 E‖∇l(θt;w∗)−∇l(θt)‖2
≤ γ2E‖∇l(θt)‖2 + 1
γ2
( AE‖∇l(θt)‖2 +B ) = (γ2 + A
γ2 )E‖∇l(θt)‖2 + B γ2 ,
where γ2 > 0.
Substituting it into Equation (13), we have
E〈∇l(θt;w∗), θ∗ − θt〉
≥ −η 2
[ (γ2 + A
γ2 )E‖∇l(θt)‖2 + B γ2
] − 1
2η E‖θ∗ − θt‖2 + 1 2η E‖θ∗ − θt+1‖2.
(15)
Then, substituting Formula (14), (15) into Inequality (12), we have
E[l(θ∗)] ≥ E[l(θt)]− η 2
[ (γ2 + A
γ2 )E‖∇l(θt)‖2 + B γ2
] − 1
2η E‖θ∗ − θt‖2 + 1 2η E‖θ∗ − θt+1‖2
+ ( µ 2 − 1 2γ1 )E‖θ∗ − θt‖2 − γ1 2
( AE‖∇l(θt)‖2 +B ) .
Rewrite the above formula as: E‖θ∗ − θt+1‖2 ≤ ( 1− (ηµ− η γ1 ) ) E‖θ∗ − θt‖2 + 2ηE [ l(θ∗)− l(θt) ] + (Aηγ1 + Aη2
γ2 + η2γ2)E‖∇l(θt)‖2 + (
η2 γ2 + ηγ1)B,
and then, it follows from Lemma 6 that E‖θ∗ − θt+1‖2 ≤ ( 1− (ηµ− η
γ1
) E‖θ∗ − θt‖2 − η
L E‖∇l(θt)‖2
+ (Aηγ1 + Aη2
γ2 + η2γ2)E‖∇l(θt)‖2 + (
η2 γ2 + ηγ1)B
≤ ( 1− (ηµ− η γ1 ) ) E‖θ∗ − θt‖2 + (η 2 γ2 + ηγ1)B,
where 1L ≥ Aγ1 + Aη γ2 + ηγ2 and µ > 1γ1 . 2
2It’s easy to verify that γ1 and γ2 exist when η < 1 L −A µ
2 √ A .
Finally, according to Lemma 7, we have
E‖θ∗ − θt‖2 ≤ ( 1− (ηµ− η γ1 ) )tE‖θ∗ − θ0‖2 + ηγ2 + γ1
µ− 1γ1 B.

heading: B PROOF OF MAIN CONVERGENCE RESULTS

heading: B.1 PRELIMINARIES
text: To simplify, we introduce the following notations:
θI = 1
M M∑ i=1 θi,
θw =
∑M i=1 θi + ∑M+N j=M+1 wjθj
M + 1Tw = MθI +
∑M+N j=M+1 wjθj
M + 1Tw .
Thus,
θt+1 = θt+ τ τ+1 w.
Lemma 8. For 0 ≤ k ≤ τ − 1,
E ∥∥∥θt+ kτ+1i − θt+ kτ+1 w∥∥∥2 ≤ 4η2k2G2. (16)
Proof. Prove by induction. Inequality 16 holds for k = 0. Assume that it also holds for k − 1, 0 < k < τ . Then,
E ∥∥∥θt+ kτ+1i − θt+ kτ+1 w∥∥∥
= E ∥∥∥θt+ kτ+1i − θt+ k−1τ+1i + θt+ k−1τ+1i − θt+ k−1τ+1 w + θt+ k−1τ+1 w − θt+ kτ+1 w∥∥∥
≤ E ∥∥∥θt+ kτ+1i − θt+ k−1τ+1i ∥∥∥+ E∥∥∥θt+ k−1τ+1i − θt+ k−1τ+1 w∥∥∥+ E∥∥∥θt+ kτ+1 w − θt+ k−1τ+1 w∥∥∥
≤ ηG+ 2η(k − 1)G+ ηG = 2ηkG.
Hence, E ∥∥∥θt+ kτ+1i − θt+ kτ+1 w∥∥∥2 ≤ 4η2k2G2.

heading: B.2 PROOF OF THEOREM 3
text: Proof. According to Assumption 2a, we have E [ l ( θt+ k+1 τ+1 w )] = E [ l ( θt+ k τ+1 w − η
M + 1Tw ( M∑ i=1 ∇li(θ t+ kτ+1 i ) + M+N∑ j=M+1 wj∇lj(θ t+ kτ+1 j ) ))] ≤ E [ l ( θt+ k τ+1 w
)] − η M + 1Tw E 〈 ∇l ( θt+ k τ+1 w ) , M∑ i=1 ∇li(θ t+ kτ+1 i ) + M+N∑ j=M+1 wj∇lj(θ t+ kτ+1 j ) 〉 ︸ ︷︷ ︸
T1
+ Lη2 2 E ∥∥∥ 1 M + 1Tw ( M∑ i=1 ∇li(θ t+ kτ+1 i ) + M+N∑ j=M+1 wj∇lj(θ t+ kτ+1 j ) )∥∥∥2︸ ︷︷ ︸ T2
, E [ l ( θt+ k τ+1 w )] + T1 + T2.
(17)
Then we are to bound the T1 and T2 in Equation 17 respectively.
For T1, we have
T1 = − η M + 1Tw E [ M∑ i=1 〈 ∇l ( θt+ k τ+1 w ) ,∇li(θ t+ k τ+1 i ) 〉 + M+N∑ j=M+1 wj 〈 ∇l ( θt+ k τ+1 w ) ,∇lj(θ t+ k τ+1 j ) 〉]
= − η M + 1Tw
E [( M + M+N∑ j=M+1 wj )〈 ∇l ( θt+ k τ+1 w ) ,∇l ( θt+ k τ+1 w )〉
− M∑ i=1 〈 ∇l ( θt+ k τ+1 w ) ,∇l ( θt+ k τ+1 w ) −∇li(θ t+ k τ+1 i ) 〉 − M+N∑ j=M+1 wj 〈 ∇l ( θt+ k τ+1 w ) ,∇l ( θt+ k τ+1 w ) −∇lj(θ t+ k τ+1 j ) 〉]
= −ηE ∥∥∥∇l(θt+ kτ+1 w)∥∥∥2 + η M + 1Tw E 〈 ∇l ( θt+ k τ+1 w ) ,
M∑ i=1 ( ∇l ( θt+ k τ+1 w ) −∇li(θ t+ k τ+1 i ) ) + M+N∑ j=M+1 wj ( ∇l ( θt+ k τ+1 w ) −∇lj ( θ t+ k τ+1 j ))〉 ,
and further, since 〈u1,v1〉 ≤ γ1‖u1‖2 + 14γ1 ‖v1‖ 2, it yields T1 ≤ (−η + γ1η)E ∥∥∥∇l(θt+ kτ+1 w)∥∥∥2
+ η
4γ1 E ∥∥∥∥∥ ∑M i=1 ( ∇l ( θt+ k τ+1 w ) −∇li(θ t+ k τ+1 i ) ) + ∑M+N j=M+1 wj ( ∇l ( θt+ k τ+1 w ) −∇lj ( θ t+ k τ+1 j )) M + 1Tw ∥∥∥∥∥ 2
= (−η + γ1η)E ∥∥∥∇l(θt+ kτ+1 w)∥∥∥2
+ η
4γ1 E ∥∥∥∥∥∇l(θt+ kτ+1 w)− ∑M i=1∇li(θ t+ k τ+1 i ) + ∑M+N j=M+1 wj∇lj ( θ t+ k τ+1 j ) M + 1Tw ∥∥∥∥∥ 2
︸ ︷︷ ︸ T3
, (−η + γ1η)E ∥∥∥∇l(θt+ kτ+1 w)∥∥∥2 + η
4γ1 T3,
(18)
where 0 < γ1 < 1.
For T2, we have
T2 = Lη2 2 E ∥∥∥ 1 M + 1Tw ( M∑ i=1 ∇li(θ t+ k τ+1 i ) + M+N∑ j=M+1 wj∇lj(θ t+ k τ+1 j ) )∥∥∥2
= Lη2 2 E ∥∥∥∥∇l(θt+ kτ+1 w)
+
∑M i=1 ( ∇li(θ t+ k τ+1 i )−∇l ( θt+ k τ+1 w )) + ∑M+N j=M+1 wj ( ∇lj(θ t+ k τ+1 j )−∇l ( θt+ k τ+1 w )) M + 1Tw ∥∥∥∥2
and further, since ‖u2 + v2‖2 = ‖u2‖2 + ‖v2‖2 + 2〈u2,v2〉 ≤ (1 + γ2)‖u2‖2 + (1 + 1γ2 )‖v2‖ 2,
it yields
T2 ≤ (1 + γ2)Lη
2 2 E ∥∥∥∥∇l(θt+ kτ+1 w)∥∥∥∥2 + (1 + 1γ2 )Lη22
· E ∥∥∥∥ ∑M i=1 ( ∇li(θ t+ k τ+1 i )−∇l ( θt+ k τ+1 w )) + ∑M+N j=M+1 wj ( ∇lj(θ t+ k τ+1 j )−∇l ( θt+ k τ+1 w )) M + 1Tw ∥∥∥∥2 = (1 + γ2)Lη 2 2 E ∥∥∥∥∇l(θt+ kτ+1 w)∥∥∥∥2
+ (1 + 1 γ2 )Lη2
2 E ∥∥∥∥∥∇l(θt+ kτ+1 w)− ∑M i=1∇li(θ t+ k τ+1 i ) + ∑M+N j=M+1 wj∇lj ( θ t+ k τ+1 j ) M + 1Tw ∥∥∥∥∥ 2
︸ ︷︷ ︸ T (t+ k τ+1 )
3
, (1 + γ2)Lη
2 2 E ∥∥∥∥∇l(θt+ kτ+1 w)∥∥∥∥2 + (1 + 1γ2 )Lη22 T (t+ kτ+1 )3 ,
(19) where γ2 > 0.
Therefore, putting Equation 17, 18, 19 together, we have
E [ l ( θt+ k+1 τ+1 w )] = E [ l ( θt+ k τ+1 w )] + T1 + T2
≤ E [ l ( θt+ k τ+1 w )] − ( (1− γ1)η − (1 + γ2)L 2 η2 ) · E ∥∥∥∇l(θt+ kτ+1 w)∥∥∥2
+ ( η
4γ1 +
(1 + 1γ2 )Lη 2
2
) · T (t+ k τ+1 )
3 .
(20)
Let Γ0 = (1 − γ1) − (1+γ2)L2 η, and Γ1 = 1 4γ1 +
(1+ 1γ2 )Lη
2 . Summing over k from 0 to τ − 1 in Equation 20, we have
τ−1∑ k=0 E ∥∥∥∇l(θt+ kτ+1 w)∥∥∥2 ≤ E
[ l ( θtw )] − E [ l ( θt+ τ τ+1 w )] ηΓ0 + Γ1 Γ0 τ−1∑ k=0 T (t+ kτ+1 ) 3
= E [ l ( θt )] − E [ l ( θt+1 )] ηΓ0 + Γ1 Γ0 τ−1∑ k=0 T (t+ kτ+1 ) 3
(21)
Then, we are to bound ∑τ−1 k=0 T (t+ kτ+1 ) 3 :
T (t+ k τ+1 )
3 = E ∥∥∥∥∥∇l(θt+ kτ+1 w)− ∑M i=1∇li(θ t+ k τ+1 i ) + ∑M+N j=M+1 wj∇lj ( θ t+ k τ+1 j ) M + 1Tw ∥∥∥∥∥ 2
≤ (1 + γ3)E ∥∥∥∥∥∇l(θt+ kτ+1 w)− ∑M i=1∇li ( θt+ k τ+1 w ) + ∑M+N j=M+1 wj∇lj ( θt+ k τ+1 w ) M + 1Tw ∥∥∥∥∥ 2
+ (1 + 1
γ3 )E ∥∥∥∥∥ ∑M i=1 ( ∇li ( θt+ k τ+1 w ) −∇li ( θ t+ k τ+1 i )) M + 1Tw
+
∑M+N j=M+1 wj ( ∇lj ( θt+ k τ+1 w ) −∇lj ( θ t+ k τ+1 j )) M + 1Tw ∥∥∥∥∥ 2
≤ (1 + γ3)E ∥∥∥∥∥∇l(θt+ kτ+1 w)− ∑M i=1∇li ( θt+ k τ+1 w ) + ∑M+N j=M+1 wj∇lj ( θt+ k τ+1 w ) M + 1Tw ∥∥∥∥∥ 2
+ (1 + 1
γ3 ) · 4L2η2k2G2
= (1 + γ3)Φ(w; θ t+ k τ+1 w) + (4 + 4
γ3 )L2η2k2G2
≤ (1 + γ3)A ∥∥∇l(θt+ kτ+1 w)∥∥2 + (1 + γ3)B + (4 + 4
γ3 )L2η2k2G2,
and then τ−1∑ k=0 T (t+ kτ+1 ) 3 ≤ τ−1∑ k=0 (1 + γ3)A ∥∥∇l(θt+ kτ+1 wt∗)∥∥2
+ (1 + γ3)τB + ( 2
3 +
2
3γ3 )L2η2G2(2τ3 − 3τ2 + τ)︸ ︷︷ ︸ Γ2
. (22)
Substituting Formula (22) into Inequality (21), we have
η ( Γ0 − (1 + γ3)AΓ1 )︸ ︷︷ ︸ Γ3 τ−1∑ k=0 E ∥∥∥∇l(θt+ kτ+1 wt∗)∥∥∥2 ≤ E[l(θt)]− E[l(θt+1)]+ ηΓ1Γ2,
or τ−1∑ k=0 E ∥∥∥∇l(θt+ kτ+1 wt∗)∥∥∥2 ≤ E
[ l ( θt )] − E [ l ( θt+1 )] ηΓ3 + Γ1Γ2 Γ3 .
Summing over t from 0 to T − 1 and dividing by τT , we have:3
1
τT T−1∑ t=0 τ−1∑ k=0 E ∥∥∥∇l(θt+ kτ+1 wt∗)∥∥∥2 ≤ l ( θ0 ) − l ( θ∗ ) ηΓ3τT + Γ1Γ2 Γ3τ .

heading: C EXPERIMENTAL DETAILS

heading: C.1 MODEL ARCHITECTURES
text: In synthesized experiments, we mainly follow the model architectures used in McMahan et al. (2017). The logistic model is a 784 × 10 fully connected layer. The 2NN model consists of a
3It’s easy to verify that the constants in the derivation exist when η < 2(1− √ A)
L(A+ √ A) .
784×200 hidden layer, another 200×200 hidden layer, and a final 200×47 output layer. The CNN model consists of a convolution layer with 5× 5 kernel and 32 channels, another convolution layer with same kernel size and 64 channels, a fully connected layer with 512 units with ReLU activation, and a final 512× 10 output layer. For all three model outputs, we use cross entropy loss. In Shakespeare experiment, we use a 8-dimension embedding, a 2-layer character LSTM with 256 hidden units, and a final 256× 90 output layer. We use cross entropy loss that ignores empty words in the truncated sentence.
In FeTS2021 experiment, we follow the U-Net structure in Ronneberger et al. (2015). There are three slight modifications: (1) The input image is resized to 120 × 120; (2) We halves the channel sizes to (32, 64, 128, 256, 512, 256, 128, 64, 32); (3) We replace the batch normalization layer with instance normalization layer.

heading: C.2 DATASET PREPROCESSING
text: In synthesized experiments, we mainly follow the preprocessing methods suggested in McMahan et al. (2017). The Fashion-MNIST dataset is split into 120 shards, each of which has 500 samples of one single digit. Assign each of the 60 clients two shards. Two of the clients are chosen as internal clients. The EMNIST-Balanced dataset is split into 600 shards, each of which has 180 samples of one single category. Assign each client 24 shards with continuous category indices. Two of the clients are chosen as internal clients. The CIFAR-10 dataset is partitioned in the same way as Fashion-MNIST. For all three synthesized experiments, the communication thresholdK is set to 10.
In Shakespeare experiments, there are altogether 715 clients, each of whom corresponds to one role in one of the six classic plays of Shakespeare. Then all the clients are naturally classified into 6 groups, according to which of the six plays they belong to. The clients’ datasets are formed by their lines in the play. To simplify the training, we truncate each line into segments of 80 characters. Each time we designate the clients in one group as internal clients, and the remaining clients as external clients. The communication threshold K is set to 50.
In FeTS2021 experiments, as shown in Figure 3, there are altogether 17 medical sites. There are only 341 different volumes, so we only choose the three largest sites (numbered 1, 6 and 16) as internal sites in our experiments. We did not choose the remaining sites as internal sites since they are too small to separate a test set of appropriate size. Further, we divide the chosen internal site into three balanced parts to form the group of internal clients. Moreover, to simplify the training, all client samples are resized to 120× 120 and all the 3 different lesion labels are merged for whole brain tumor segmentation. Finally, we set the communication threshold K to 4.

Review summaries:
This work studies a variant of the federated learning (FL) problem where the stakeholder (i.e. central planner) only cares about the performance of a trained model over a small proportion of local (internal) clients. The work proposes a first-order federated learning algorithm that weights external clients to obtain a variance-reduced gradient estimate of the ultimate objective (over internal clients), and presents convergence analysis for their algorithm under certain settings. Finally, the work evaluates their proposed algorithm by comparing against existing FL algorithms using synthetic and real datasets.  

This paper proposes selfish federated learning, an FL framework that optimizes the performance of a small subset of clients. The authors further propose a new selfish-FL algorithm, termed VaRSeL, based on minimizing the variance between the surrogate gradient $\nabla l (\theta; w)$ and the ultimate gradient $\nabla l (\theta)$. Under certain conditions, the authors prove the convergence of an idealized variant of VaRSeL. The performance of VaRSeL is demonstrated in a range of simulations and real-data experiments.

This paper proposes a new setting for federated learning -- Selfish Federated Learning -- where all clients (internal and external) participate in the training, while the trained model needs to give low error only for the internal clients. The main idea is to reweight the loss functions and give small weights to external clients so that the overall objective is closer to the true risk of internal clients. The paper provides some theory for methods where the optimal weights are known and proposes a heuristic method VaRSeL to estimate the optimal weights on the fly.