Rethinking Client Reweighting for Selfish Federated Learning

1 INTRODUCTION . Federated learning ( FL ) proposes a privacy-preserving scheme , which enables different clients to cooperatively learn a global model by integrating knowledge from multiple clients . Suppose there are Q clients { Dn } Qn=1 in total . A general objective of FL is given by min f Q∑ n=1 LDn ( f ) , ( 1 ) where f is the model , and LDn is the generalization loss on client Dn . However , since the general objective ( 1 ) fairly optimizes the mean generalization loss across all clients , a model learned by ( 1 ) may not perform well on some individual clients . We implemented several state-of-the-art FL algorithms on FeTS 2021 ( Pati et al. , 2021 ) , a real-world FL dataset for medical image segmentation , and discovered that the federally learned model is not guaranteed to generalize on local client ( s ) at least as well as the locally learned model . ( see Figure 1 ) In cross-silo FL ( Kairouz et al. , 2019 ) , this issue becomes particularly serious since clients typically have sufficient data for training a decent local model . In this case , the clients may become reluctant to participate in federated training . This issue raises an important challenge for FL society as well as practitioners : For a stakeholder who is only interested in the model ’ s performance on a small portion of clients , is it possible to guarantee its benefits from federated learning ? This is a common challenge in many practical cases . For instance , in a bank network , due to the limited size of internal datasets , a banking group wants to leverage more data and learn a better model through federated learning , while the banking group expects the final model to maximize the performance on its own internal banks , rather than overfitting to other external banks . As another example , to build an AI-assisted diagnostic system , a medical association hopes to integrate knowledge from the datasets of external partner hospitals through federated learning . As the system ultimately serves its internal hospitals/patient populations , its performance should not be disturbed by external partner hospitals ( Rieke et al. , 2020 ) . In these cases , to enhance the model perf the stakeholder would like to purchase gradient updates from external collaborators . Selfish federated learning In this work , we resolve the aforementioned challenge by investigating a novel learning setting called selfish federated learning ( Selfish-FL ) , where the stakeholder who only cares about the performance on a few internal clients can reap the benefits of cooperative learning with external clients . To formalize , the Q clients are partitioned into M internal clients and N external clients ( M +N = Q , M N ) . Though it is possible to incorporate the knowledge in all Q clients for model training , the stakeholder only focuses on the model ’ s performance on the M internal clients . Comparisons with related setups Table 1 lists several FL settings . While centralized and decentralized FL differ in communication network , both of them are looking for a global consensus among all the clients ( Wang et al. , 2021 ; Koloskova et al. , 2020 ) . Personalized-FL allows each client to maintain a personalized model , but the losses of all clients are still considered in the overall objectives ( Fallah et al. , 2020 ; Tan et al. , 2021 ) . In Selfish-FL , only internal clients are the concern of the stakeholder , while the role of external clients is to provide additional information to further improve the model ’ s performance on internal clients . Our contributions The main contributions in this paper can be summarized as four aspects : First , we study a novel setting in federated learning , namely “ Selfish-FL ” , which is the first work ( to the best of our knowledge ) considering biased objectives and internal/external partitions in FL setting . Second , we propose a novel algorithm , called VaRSeL , for solving Selfish-FL problem . It leverages knowledge from external clients to improve the model via gradients variance reduction . This is a different perspective compared with previous loss-based clients reweighting strategy ( Cho et al. , 2020 ; Chen et al. , 2020 ) . Within each round of federated training , it guides the model to update towards the direction favored by the internal clients . Moreover , our strategy is free of hyperparameter-tuning . Third , we present a convergence analysis of our reweighting strategy for this new setting , both in strongly-convex cases and non-convex cases . Based on the convergence result , we also observe its fine-tune effect . Finally , we perform extensive experiments on both synthesized and real-world natural language and medical image analysis datasets . The experimental results demonstrate that our proposed VaRSeL outperforms the classical FedAvg , related client selection methods , as well as the state-of-the-art for non-iid data ( FedProx ) and personalized FL algorithm ( Per-FL ) in terms of model accuracy . 2 RELATED WORKS . 2.1 FEDERATED LEARNING AND CHALLENGES . Federated learning ( FL ) ( Yang et al. , 2019 ; Kairouz et al. , 2019 ; Wang et al. , 2021 ) provides a promising way of cooperative learning with privacy protection . In this subsection , we will briefly summarize a few important lines of FL ( Some works may lie at intersection ) : Relieving clients ’ non-iid Proximal terms ( Li et al. , 2020c ) , cluster-based methods ( Briggs et al. , 2020 ; Ghosh et al. , 2020 ) , client-drift regulations ( Karimireddy et al. , 2020 ; Acar et al. , 2021 ) are some leading methods to address the heterogeneity of clients ’ data distributions . At a high level , all these methods can be summarized as pursuing a compromise solution that can work fairly well on all the clients . Therefore , even though these methods are very effective in classical FL frameworks , none of them fits well into our selfish-FL framework , because the internal clients are in nature reluctant to compromise with any external clients . Personalized FL Many approaches have been proposed for achieving personalization in FL setting ( Kairouz et al. , 2019 ) , e.g. , mixing the global model and user ’ s local model , dividing the network into base and personalized layers . Recently , Per-FL ( Fallah et al. , 2020 ) linked personalized FL with meta-learning by building a good initial meta-model that can be updated effectively . Dinh et al . ( 2020 ) used Moreau envelopes as clients ’ regularization loss to achieve personalized FL and Zhang et al . ( 2020 ) proposed a flexible FL framework that allows clients to personalize to specific target data distributions . Readers can refer to comprehensive surveys in Li et al . ( 2020a ) ; Tan et al . ( 2021 ) . However in personalized FL , the losses of all clients are still considered in the overall objectives . Alleviating communication cost Lin et al . ( 2017 ) ; Richtárik et al . ( 2021 ) provided several model compression schemes to reduce communication cost and proved the convergence of their compression methods . Liang et al . ( 2020 ) ; Li et al . ( 2021 ) proposed to learn a combination of local and global models , while only the global parts are synchronized . It ’ s worth mentioning that our method is orthogonal to these communication-efficient methods and can be combined with theirs directly . 2.2 SAMPLE REWEIGHTING .. Our work is also related to sample reweighting , i.e. , assigning different weights to samples in model training to tackle dataset biases . There are two completely opposite strategies on how to set the weights w for external datasets . One line of works Freund et al . ( 1996 ) ; Malisiewicz et al . ( 2011 ) ; Cho et al . ( 2020 ) argues that the samples with higher losses should be given more weights , since these difficult samples can help sharpen the decision boundaries and increase the adversarial robustness . The other line of works Ghadikolaei et al . ( 2019 ) ; Song et al . ( 2021 ) suggests lowering their weights , since their high losses are attributed to training set biases or mislabels ( e.g. , nosiy samples ) . In fact , this contradiction directly points to a fundamental , difficult problem in machine learning : Dilemma . How to distinguish the difficult samples from the noisy ones ? To the best of our knowledge , the controversy continues , and people are still far from getting a clear resolution . To bypass the aforementioned dilemma , some researchers ( Ren et al. , 2018 ; Shu et al. , 2019 ) used meta-learning approach to learn how to assign weights w adaptively . However , their method suffers from computationally expensive second derivatives estimation , and can hardly generalize to federated learning framework since it requires full-client participation throughout training . Client reweighting in FL In FL framework , since one only has access to limited training examples , clients instead of samples are reweighted to tackle their heterogeneity . Existing FL algorithms reweight clients using the sizes of their datasets ( Li et al. , 2020c ) , the numbers of their local steps ( Wang et al. , 2020 ) , their local losses ( Cho et al. , 2020 ; Chen et al. , 2020 ) , etc . As an inevitable issue in FL , communication costs usually restrict the number of clients one can access within each communication round ( McMahan et al. , 2017 ) . Suppose client j is sampled with probability wj ∈ [ 0 , 1 ] , M+1 ≤ j ≤M+N . Since the communication costs mainly depend on the number of sampled clients , existing algorithms ( Lian et al. , 2018 ; Wang et al. , 2019 ) impose hard constraints on wj as 1Tw = ∑M+N j=M+1 wj ≤ CbN = K , whereCb > 0 is called the communication budget , to alleviate the communication bottleneck . 3 PROBLEM FORMULATION . In this section , we give a formal definition of the setting selfish federated learning ( Selfish-FL ) . Internal and external clients Let X denote the input space ( e.g. , feature space ) and Y the space of outputs ( e.g. , label space ) . Each client characterizes a joint distribution D = P ( X , Y ) over X × Y . Suppose there are M + N clients { Dn } M+Nn=1 , among which the first M clients are called the internal clients and the remaining N clients are called the external clients ( M N ) . We assume that internal clients are consistent and the difference between internal and external clients are expected to be larger than that between internal clients . To formalize , we have D1 = · · · = DM , DI ; Dj 6= DI , M + 1 ≤ j ≤M +N . In practice , the distributions of clients are not observed directly , but given in the form of sample sets Sn ∼ Dn , for 1 ≤ n ≤ M + N . The stakeholder has direct access to the raw data in S1 , . . . , SM , but can only query external clients in a privacy-preserving way as in typical FL scenario . Ultimate objective While conventional federated learning setting usually aims to minimize the generalization loss across all clients , the objective in Selfish-FL is to minimize the generalization loss on internal clients D1 , · · · , DM alone : min θ l ( θ ) : = 1 M M∑ i=1 E ( x , y ) ∼DiL ( fθ ( x ) , y ) , ( 2 ) where fθ denotes a model f parameterized by θ , and L denotes an appropriate loss function . Although the stakeholder can simply learn θ by local training on datasets of internal clients , we will show both in theory and in experiments that it is probable to improve the estimation of θ by leveraging additional knowledge from external clients without sharing raw data .

Summary: This paper proposes a new setting for federated learning -- Selfish Federated Learning -- where all clients (internal and external) participate in the training, while the trained model needs to give low error only for the internal clients. The main idea is to reweight the loss functions and give small weights to external clients so that the overall objective is closer to the true risk of internal clients. The paper provides some theory for methods where the optimal weights are known and proposes a heuristic method VaRSeL to estimate the optimal weights on the fly.