Diverse Client Selection for Federated Learning via Submodular Maximization

1 INTRODUCTION . Federated learning ( FL ) involves collaboratively training of machine learning model across a large number of clients while keeping client data local . Recent approaches to this problem repeatedly alternate between device-local ( stochastic ) gradient descent steps and server-aggregation of the clients ’ model updates ( McMahan et al. , 2017 ) . In cross-device settings , a server and its model usually serves several thousands of devices . Therefore , the communication between clients and the server can be costly and slow , forming a huge impediment to FL ’ s viability . One property of the collection of clients that can mitigate these problems , however , is often not exploited , and that is redundancy . Specifically , many clients might provide similar , and thus redundant , gradient information for updating the server model . Therefore , transmitting all such updates to the server is a waste of communication and computational resources . How best to select a representative and more informative client set while adhering to practical constraints in federated learning is still an open challenge . Although several selection criterion have been investigated in recent literature , e.g. , sampling clients with probabilities proportional to their local dataset size ( McMahan et al. , 2017 ) , sampling clients of larger update norm with higher probability ( Chen et al. , 2020 ) , and selecting clients with higher losses ( Balakrishnan et al. , 2020 ; Cho et al. , 2020 ) , the redundancy and similarity of the clients ’ updates sent to the server is not represented and exploited in these approaches . In particular , communicating multiple clients ’ updates to the server may cause statistical and system inefficiency if too many of them are too similar to each other . The commonly studied modular score/probability for each individual client is incapable of capturing information as a property over a group of clients . Ideally , a diverse set of clients would be selected , thereby increasing the impact of under-represented clients that contribute different information , and thereby improving fairness . This , in fact , is a topic of increasing interest ( Mohri et al. , 2019 ; Cho et al. , 2020 ; Dennis et al. , 2021 ) . In this paper , we introduce diversity to client selection in FL , namely a strategy to measure how a selected subset of clients can represent the whole when being aggregated on the server . Specifically , in each communication round , we aim to find a subset whose aggregated model update approximates the aggregate update over all clients . By doing this , we aim to limit the impact of subset selection which introduces variance in the model updates across round , that could otherwise slow the learning process . Inspired by the CRAIG method of coreset selection for efficient machine learning training ( Mirzasoleiman et al. , 2020 ) , we derive an upper bound of the approximation error as a supermodular set function ( in particular , the min-form of the facility location function ( Cornuéjols et al. , 1977 ) ) evaluated on the selected subset . We can then apply submodular maximization ( Fujishige , 2005 ; Iyer et al. , 2013 ; Wei et al. , 2014 ) on a complement submodular function to ( approximately ) minimize the error upper bound . We employ the greedy selection ( Nemhauser et al. , 1978 ) of a subset of clients according to the marginal gain of the submodular function to achieve a solution with provable approximation guarantee ( Conforti & Cornuejols , 1984 ) . By integrating the diverse client selection into the most commonly studied FL scheme , i.e. , Federated Averaging ( FedAvg ) ( McMahan et al. , 2017 ) , we propose DivFL that applies global model aggregation over a selected subset of clients after multiple local steps on every client . We present theoretical convergence analysis of DivFL and show its tolerance to the heterogeneity of data distributions across clients and large number of local steps . However , our method differs from the CRAIG method where selection is performed based on model updates ( involving multiple epochs at the clients ) . In addition , our approach allows for partial device participation where the server does not have access to all data at any communication round , as is standard in FL ( McMahan et al. , 2017 ) . In experiments , we compare DivFL with other client selection approaches on both synthetic dataset and FEMNIST , wherein our method excels on convergence , fairness and learning efficiency . 2 BACKGROUND AND RELATED WORK . We consider a typical federated learning objective : min w f ( w ) = N∑ k=1 pkFk ( w ) , where for each client k ∈ [ N ] , pk is a pre-defined weight ( such that ∑N k=1 pk = 1 ) that can be set to 1 N or the fraction of training samples , and Fk is the client-specific empirical loss . While there are various possible modeling approaches , we consider this canonical objective of fitting a single global model to the non-identically distributed data across all clients ( McMahan et al. , 2017 ) . Client Selection in Federated Learning . Client1 sampling is a critical problem particularly for cross-device settings where it is prohibitive to communicate with all devices . Two common ( or default ) strategies are ( a ) sampling the clients based on the number of local data points and uniformly averaging the model updates , and ( b ) sampling the clients uniformly at random and aggregating the model updates with weights proportional to the local samples ( Li et al. , 2020 ) . There is also recent work proposing advanced sampling techniques to incorporate dynamic systems constraints , accelerate the convergence of federated optimization , or to obtain a better model with higher accuracy ( Nishio & Yonetani , 2019 ; Ribero & Vikalo , 2020 ; Cho et al. , 2020 ; Lai et al. , 2020 ) . We investigate client selection through the lens of encouraging client diversity at each communication round which largely remains unexplored in previous work . The closest client selection method to ours is based on clustering ( e.g. , selecting representative clients from separate clusters ( Dennis et al. , 2021 ) ) . We note that performing ( private ) clustering in federated settings is still an open problem , and our method can be viewed as a soft version of dynamic clustering at each round ( discussed in the next paragraph ) . The benefits of gradient ( or model ) diversity has been demonstrated in other related contexts , such as scaling up mini-batch stochastic gradient descent ( SGD ) ( Yin et al. , 2018 ) . Enforcing sample or gradient diversity during optimization also implicitly places more emphasis on the underrepresented sub-population of clients , and can promote fairness defined as representative disparity ( Hashimoto et al. , 2018 ) . Similar to previous work ( e.g. , Cho et al. , 2020 ; Balakrishnan et al. , 2020 ) , we observe our approach yields more fair solutions across the network in Section 5 . Diverse Subset Selection via Submodularity . Modular scores have been widely studied for subset selection in machine learning and federated learning , e.g. , a utility score for each sample or client often measured by the loss . However , the diversity of a subset can not be fully captured by such 1Following conventions , we use the term ‘ client ’ for the problem of client selection . Throughout the paper , we use ‘ devices ’ and ‘ clients ’ interchangeably . modular scores since there is no score interaction . Diversity is often well modeled by a diminishing return property , i.e. , the ( marginal ) gain an element brings to a subset diminishes as more elements added to the subset . There exists a rich and expressive family of functions , all of which are natural for measuring diversity , and all having the diminishing returns property : given a finite ground set V of size n , and any subset A ⊆ B ⊆ V and a v /∈ B , a set function F : 2V → R is submodular if F ( v ∪A ) − F ( A ) ≥ F ( v ∪B ) − F ( B ) . ( 1 ) This implies v is no less valuable to the smaller set A than to the larger set B . The marginal gain of v conditioned on A is denoted f ( v|A ) , f ( v ∪A ) − f ( A ) and reflects the importance of v to A. Submodular functions ( Fujishige , 2005 ) have been widely used for diversity models ( Lin & Bilmes , 2011 ; Batra et al. , 2012 ; Prasad et al. , 2014 ; Gillenwater et al. , 2012 ; Bilmes & Bai , 2017 ) . Maximizing a submodular function usually encourages the diversity and reduces the redundancy of a subset . This property has been utilized for data selection in active learning ( Guillory & Bilmes , 2011 ) , curriculum learning ( Zhou & Bilmes , 2018 ) , mini-batch partitioning ( Wang et al. , 2019 ) , gradient approximation ( Mirzasoleiman et al. , 2020 ) , etc . Although the number of possible subsets A is ( n k ) , enumerating them all to find the maximum is intractable . Thanks to submodularity , fast approximate algorithms ( Nemhauser et al. , 1978 ; Minoux , 1978 ; Mirzasoleiman et al. , 2015 ) exist to find an approximately optimal A with provable bounds ( Nemhauser et al. , 1978 ; Conforti & Cornuejols , 1984 ) . Despite its success in data selection , submodularity has not been explored for client selection in federated learning . Encouraging diversity amongst local gradients ( or model updates ) of selected clients can effectively reduce redundant communication and promote fairness . Moreover , it raises several new challenges in the FL setting , e.g. , ( 1 ) it is unclear which submodular function to optimize and in which space to measure the similarity/diversity between clients ; ( 2 ) What convergence guarantee can be obtained under practical assumptions such as heterogeneity among clients , and ( 3 ) What are the effects of outdated client selection due to communication constraints ? 3 DIVERSE CLIENT SELECTION . In this section , we introduce “ federated averaging with diverse client selection ” ( or DivFL ) , a method that incorporates diverse client selection into the most widely studied FL scheme , federated averaging ( FedAvg ) . We will first derive a combinatorial objective for client selection via an approximation of the full communication from all clients , which naturally morphs into a facility location function in the gradient space that can be optimized by submodular maximization . We then present the standard greedy algorithm that optimizes the objective by selecting a diverse subset of clients at every communication round . 3.1 APPROXIMATION OF FULL COMMUNICATION . We aim to find a subset S of clients whose aggregated gradient can approximate the full aggregation over all the N clients V = [ N ] . To formulate this problem , we start by following the logic in Mirzasoleiman et al . ( 2020 ) . Given a subset S , we define a mapping σ : V → S such that the gradient information ∇Fk ( vk ) from client k is approximated by the gradient information from a selected client σ ( k ) ∈ S. For i ∈ S , let Ci , { k ∈ V |σ ( k ) = i } be the set of clients approximated by client-i and γi , |Ci| . The full aggregated gradient can be written as∑ k∈ [ N ] ∇Fk ( vk ) = ∑ k∈ [ N ] [ ∇Fk ( vk ) −∇Fσ ( k ) ( vσ ( k ) ) ] + ∑ k∈S γk∇Fk ( vk ) . ( 2 ) Subtracting the second term from both sides , taking the norms , and applying triangular inequality , we can obtain an upper bound for the approximation to the aggregated gradient by S , i.e. , ∥∥∥∥∥∥ ∑ k∈ [ N ] ∇Fk ( vk ) − ∑ k∈S γk∇Fk ( vk ) ∥∥∥∥∥∥ ≤ ∑ k∈ [ N ] ∥∥∥∇Fk ( vk ) −∇Fσ ( k ) ( vσ ( k ) ) ∥∥∥ . ( 3 ) The above inequality holds for any feasible mapping σ since the left hand side does not depend on σ . So we can take the minimum of the right hand side w.r.t . σ ( k ) , ∀k ∈ [ N ] , i.e. , ∥∥∥∥∥∥ ∑ k∈ [ N ] ∇Fk ( vk ) − ∑ k∈S γk∇Fk ( vk ) ∥∥∥∥∥∥ ≤ ∑ k∈ [ N ] min i∈S ∥∥∇Fk ( vk ) −∇Fi ( vi ) ∥∥ , G ( S ) . ( 4 ) The right hand side provides a relaxed objective G ( S ) for minimizing the approximation error on the left hand . Minimizing G ( S ) ( or maximizing Ḡ , a constant minus its negation ) equals maximizing a well-known submodular function , i.e. , the facility location function ( Cornuéjols et al. , 1977 ) . To restrict the communication cost , we usually limit the number of selected clients to be no greater than K , i.e. , |S| ≤ K. This resorts to a submodular maximization problem under cardinality constraint , which is NP-hard but an approximation solution with 1− e−1 bound can be achieved via the greedy algorithm ( Nemhauser et al. , 1978 ) .

Summary: This paper studies the selection of clients for federated learning under the assumption of full participation via submodular function maximization. Specifically, the goal is to obtain a subset of the clients such that the aggregation of the gradients of their loss functions approximates the full aggregated gradient. The authors show that the error defined by the difference between both aggregations (the full one and the one defined by the subset) can be upper bounded by a supermodular function. Therefore, minimizing this function (subject to a single cardinality constraint, which is motivated by communication costs) gives an upper bound for the problem of minimizing the error. The minimization of this particular supermodular function can be equivalently posed as the maximization of a submodular function, which has been previously studied in the literature (e.g. [1] below). Given this, the authors propose a variant of the federated averaging scheme, called *DivFL*, which uses the standard greedy algorithm to select $K$ clients to communicate the current value $w_t$. They show that under the “gradient approximation error” assumption, the error between $w_t$ and $w^*$ decreases as $O(1/t)+O(\epsilon)$, where $\epsilon$ is a parameter of the assumption. Finally, they test their scheme in synthetic and real-data sets against other methods in the literature.   [1] Ryan Gomes and Andreas Krause. Budgeted Nonparametric Learning from Data Streams, 2015.