Rethinking Client Reweighting for Selfish Federated Learning

1 INTRODUCTION . Federated learning ( FL ) proposes a privacy-preserving scheme , which enables different clients to cooperatively learn a global model by integrating knowledge from multiple clients . Suppose there are Q clients { Dn } Qn=1 in total . A general objective of FL is given by min f Q∑ n=1 LDn ( f ) , ( 1 ) where f is the model , and LDn is the generalization loss on client Dn . However , since the general objective ( 1 ) fairly optimizes the mean generalization loss across all clients , a model learned by ( 1 ) may not perform well on some individual clients . We implemented several state-of-the-art FL algorithms on FeTS 2021 ( Pati et al. , 2021 ) , a real-world FL dataset for medical image segmentation , and discovered that the federally learned model is not guaranteed to generalize on local client ( s ) at least as well as the locally learned model . ( see Figure 1 ) In cross-silo FL ( Kairouz et al. , 2019 ) , this issue becomes particularly serious since clients typically have sufficient data for training a decent local model . In this case , the clients may become reluctant to participate in federated training . This issue raises an important challenge for FL society as well as practitioners : For a stakeholder who is only interested in the model ’ s performance on a small portion of clients , is it possible to guarantee its benefits from federated learning ? This is a common challenge in many practical cases . For instance , in a bank network , due to the limited size of internal datasets , a banking group wants to leverage more data and learn a better model through federated learning , while the banking group expects the final model to maximize the performance on its own internal banks , rather than overfitting to other external banks . As another example , to build an AI-assisted diagnostic system , a medical association hopes to integrate knowledge from the datasets of external partner hospitals through federated learning . As the system ultimately serves its internal hospitals/patient populations , its performance should not be disturbed by external partner hospitals ( Rieke et al. , 2020 ) . In these cases , to enhance the model perf the stakeholder would like to purchase gradient updates from external collaborators . Selfish federated learning In this work , we resolve the aforementioned challenge by investigating a novel learning setting called selfish federated learning ( Selfish-FL ) , where the stakeholder who only cares about the performance on a few internal clients can reap the benefits of cooperative learning with external clients . To formalize , the Q clients are partitioned into M internal clients and N external clients ( M +N = Q , M N ) . Though it is possible to incorporate the knowledge in all Q clients for model training , the stakeholder only focuses on the model ’ s performance on the M internal clients . Comparisons with related setups Table 1 lists several FL settings . While centralized and decentralized FL differ in communication network , both of them are looking for a global consensus among all the clients ( Wang et al. , 2021 ; Koloskova et al. , 2020 ) . Personalized-FL allows each client to maintain a personalized model , but the losses of all clients are still considered in the overall objectives ( Fallah et al. , 2020 ; Tan et al. , 2021 ) . In Selfish-FL , only internal clients are the concern of the stakeholder , while the role of external clients is to provide additional information to further improve the model ’ s performance on internal clients . Our contributions The main contributions in this paper can be summarized as four aspects : First , we study a novel setting in federated learning , namely “ Selfish-FL ” , which is the first work ( to the best of our knowledge ) considering biased objectives and internal/external partitions in FL setting . Second , we propose a novel algorithm , called VaRSeL , for solving Selfish-FL problem . It leverages knowledge from external clients to improve the model via gradients variance reduction . This is a different perspective compared with previous loss-based clients reweighting strategy ( Cho et al. , 2020 ; Chen et al. , 2020 ) . Within each round of federated training , it guides the model to update towards the direction favored by the internal clients . Moreover , our strategy is free of hyperparameter-tuning . Third , we present a convergence analysis of our reweighting strategy for this new setting , both in strongly-convex cases and non-convex cases . Based on the convergence result , we also observe its fine-tune effect . Finally , we perform extensive experiments on both synthesized and real-world natural language and medical image analysis datasets . The experimental results demonstrate that our proposed VaRSeL outperforms the classical FedAvg , related client selection methods , as well as the state-of-the-art for non-iid data ( FedProx ) and personalized FL algorithm ( Per-FL ) in terms of model accuracy . 2 RELATED WORKS . 2.1 FEDERATED LEARNING AND CHALLENGES . Federated learning ( FL ) ( Yang et al. , 2019 ; Kairouz et al. , 2019 ; Wang et al. , 2021 ) provides a promising way of cooperative learning with privacy protection . In this subsection , we will briefly summarize a few important lines of FL ( Some works may lie at intersection ) : Relieving clients ’ non-iid Proximal terms ( Li et al. , 2020c ) , cluster-based methods ( Briggs et al. , 2020 ; Ghosh et al. , 2020 ) , client-drift regulations ( Karimireddy et al. , 2020 ; Acar et al. , 2021 ) are some leading methods to address the heterogeneity of clients ’ data distributions . At a high level , all these methods can be summarized as pursuing a compromise solution that can work fairly well on all the clients . Therefore , even though these methods are very effective in classical FL frameworks , none of them fits well into our selfish-FL framework , because the internal clients are in nature reluctant to compromise with any external clients . Personalized FL Many approaches have been proposed for achieving personalization in FL setting ( Kairouz et al. , 2019 ) , e.g. , mixing the global model and user ’ s local model , dividing the network into base and personalized layers . Recently , Per-FL ( Fallah et al. , 2020 ) linked personalized FL with meta-learning by building a good initial meta-model that can be updated effectively . Dinh et al . ( 2020 ) used Moreau envelopes as clients ’ regularization loss to achieve personalized FL and Zhang et al . ( 2020 ) proposed a flexible FL framework that allows clients to personalize to specific target data distributions . Readers can refer to comprehensive surveys in Li et al . ( 2020a ) ; Tan et al . ( 2021 ) . However in personalized FL , the losses of all clients are still considered in the overall objectives . Alleviating communication cost Lin et al . ( 2017 ) ; Richtárik et al . ( 2021 ) provided several model compression schemes to reduce communication cost and proved the convergence of their compression methods . Liang et al . ( 2020 ) ; Li et al . ( 2021 ) proposed to learn a combination of local and global models , while only the global parts are synchronized . It ’ s worth mentioning that our method is orthogonal to these communication-efficient methods and can be combined with theirs directly . 2.2 SAMPLE REWEIGHTING .. Our work is also related to sample reweighting , i.e. , assigning different weights to samples in model training to tackle dataset biases . There are two completely opposite strategies on how to set the weights w for external datasets . One line of works Freund et al . ( 1996 ) ; Malisiewicz et al . ( 2011 ) ; Cho et al . ( 2020 ) argues that the samples with higher losses should be given more weights , since these difficult samples can help sharpen the decision boundaries and increase the adversarial robustness . The other line of works Ghadikolaei et al . ( 2019 ) ; Song et al . ( 2021 ) suggests lowering their weights , since their high losses are attributed to training set biases or mislabels ( e.g. , nosiy samples ) . In fact , this contradiction directly points to a fundamental , difficult problem in machine learning : Dilemma . How to distinguish the difficult samples from the noisy ones ? To the best of our knowledge , the controversy continues , and people are still far from getting a clear resolution . To bypass the aforementioned dilemma , some researchers ( Ren et al. , 2018 ; Shu et al. , 2019 ) used meta-learning approach to learn how to assign weights w adaptively . However , their method suffers from computationally expensive second derivatives estimation , and can hardly generalize to federated learning framework since it requires full-client participation throughout training . Client reweighting in FL In FL framework , since one only has access to limited training examples , clients instead of samples are reweighted to tackle their heterogeneity . Existing FL algorithms reweight clients using the sizes of their datasets ( Li et al. , 2020c ) , the numbers of their local steps ( Wang et al. , 2020 ) , their local losses ( Cho et al. , 2020 ; Chen et al. , 2020 ) , etc . As an inevitable issue in FL , communication costs usually restrict the number of clients one can access within each communication round ( McMahan et al. , 2017 ) . Suppose client j is sampled with probability wj ∈ [ 0 , 1 ] , M+1 ≤ j ≤M+N . Since the communication costs mainly depend on the number of sampled clients , existing algorithms ( Lian et al. , 2018 ; Wang et al. , 2019 ) impose hard constraints on wj as 1Tw = ∑M+N j=M+1 wj ≤ CbN = K , whereCb > 0 is called the communication budget , to alleviate the communication bottleneck . 3 PROBLEM FORMULATION . In this section , we give a formal definition of the setting selfish federated learning ( Selfish-FL ) . Internal and external clients Let X denote the input space ( e.g. , feature space ) and Y the space of outputs ( e.g. , label space ) . Each client characterizes a joint distribution D = P ( X , Y ) over X × Y . Suppose there are M + N clients { Dn } M+Nn=1 , among which the first M clients are called the internal clients and the remaining N clients are called the external clients ( M N ) . We assume that internal clients are consistent and the difference between internal and external clients are expected to be larger than that between internal clients . To formalize , we have D1 = · · · = DM , DI ; Dj 6= DI , M + 1 ≤ j ≤M +N . In practice , the distributions of clients are not observed directly , but given in the form of sample sets Sn ∼ Dn , for 1 ≤ n ≤ M + N . The stakeholder has direct access to the raw data in S1 , . . . , SM , but can only query external clients in a privacy-preserving way as in typical FL scenario . Ultimate objective While conventional federated learning setting usually aims to minimize the generalization loss across all clients , the objective in Selfish-FL is to minimize the generalization loss on internal clients D1 , · · · , DM alone : min θ l ( θ ) : = 1 M M∑ i=1 E ( x , y ) ∼DiL ( fθ ( x ) , y ) , ( 2 ) where fθ denotes a model f parameterized by θ , and L denotes an appropriate loss function . Although the stakeholder can simply learn θ by local training on datasets of internal clients , we will show both in theory and in experiments that it is probable to improve the estimation of θ by leveraging additional knowledge from external clients without sharing raw data .

Summary: This paper proposes selfish federated learning, an FL framework that optimizes the performance of a small subset of clients. The authors further propose a new selfish-FL algorithm, termed VaRSeL, based on minimizing the variance between the surrogate gradient $\nabla l (\theta; w)$ and the ultimate gradient $\nabla l (\theta)$. Under certain conditions, the authors prove the convergence of an idealized variant of VaRSeL. The performance of VaRSeL is demonstrated in a range of simulations and real-data experiments.