Optimality and Stability in Federated Learning: A Game-theoretic Approach

1 Introduction . Recent advances of machine learning techniques has made it possible to apply powerful prediction algorithms to a variety of domains . However , in real-world situations , data is often distributed across multiple locations and can not be combined to a central repository for training . For example , consider patient medical data located at hospitals or student educational data at different schools . In each case , the individual agents ( hospitals or schools ) who hold the data wish to find a model that minimizes their error . However , the data at each location may be insufficient to train a robust model . Instead , the agents may prefer to build a model using data from multiple agents : multiple hospitals or schools . Collectively , the combined data may be able to produce a model with much higher accuracy , providing more powerful predictions to each agent and increasing overall welfare . However , it may be infeasible to transfer the data to some coordinating entity to build a global model : privacy , data size , and data format are all possible reasons that would make transferring data not a reasonable solution . Federated learning is a novel distributed learning paradigm that aims to solve this problem ( McMahan et al . [ 2016 ] ) . Data remains at separate local sites , which individual agents use to learn local model parameters or parameter updates . Then , only the parameters are transferred to the coordinating entity 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . ( for example , a technology company ) , which averages together all of the parameters in order to form a single global model , which all of the agents use . Federated learning is a rapidly growing area of research ( Li et al . [ 2020 ] , Kairouz et al . [ 2019 ] , Lim et al . [ 2020 ] ) . However , research has also noted that federated learning , in its traditional form , may not be the best option for each agent ( Yu et al . [ 2020 ] , Bagdasaryan and Shmatikov [ 2019 ] , Li et al . [ 2019 ] , Mohri et al . [ 2019 ] ) . In the real world , agents may differ in their true distribution : the true model of patient outcomes at hospital A may differ from the true model at hospital B , for example . If these differences are large enough , federating agents may see their error increase under certain situations , potentially even beyond what they would have obtained with only local learning . For example , a player with relatively few samples may end up seeing its model “ torqued ” by the presence of a player with many samples . For this reason , agents may not wish to federate with every other potential agent . Instead , each agent faces a choice : given the costs and benefits of federating with different players , it must determine which of the exponentially many combinations of players it would prefer to federate with . Simultaneously , every other agent is also attempting to identify and join a federating group that it prefers - and agents may have conflicting preferences . Prior work ( Donahue and Kleinberg [ 2021 ] , Hasan [ 2021 ] ) has formulated this problem as a hedonic game , which each player derives some cost ( error ) from the coalition they join . The aim of such research has been to identify partitions of players that are stable against deviations , for varying definitions of stability . A hedonic game in general may not have any stable arrangements , so the area ’ s contributions in the analysis of stability adds valuable insight into the incentives of federating agents . However , this framework also leaves open multiple game theoretic questions . While the federating agents have individual incentives to reduce their error , society as a whole also has an interest in minimizing the overall error . In the school example , individual schools wish to find coalitions that work well on their own sub-populations , while the overall district or state may have an interest in finding an overall set of coalitions that minimizes the overall error . This analysis of a coalition partition ’ s overall cost falls under the game theoretic notion of optimality . One natural question relates to the tension between these two goals : the self-interested goal of the individual actors ( stability ) and the overall goal of reducing total cost ( optimality ) . Given a that set of self-interested agents has found a stable solution , how far from optimal could it be ? This is reflected by the Price of Anarchy of a game , the canonical approach to study optimality and stability jointly ( Papadimitriou [ 2001 ] , Koutsoupias and Papadimitriou [ 1999 ] ) . The Price of Anarchy ( PoA ) is a ratio where the numerator is equal to the highest-cost stable arrangement and the denominator is equal to the lowest-cost arrangement ( the optimal arrangement ) . It is lower bounded by 1 , a bound that it achieves only if all stable arrangements are optimal . A higher Price of Anarchy value implies a greater trade off between stability and optimality , and bounding the Price of Anarchy for a particular game puts a limit on this trade-off . Federated learning is a situation where questions of stability have been analyzed , but to our knowledge there has been no systematic analysis of the Price of Anarchy in a model of federated learning . The present work : A framework for optimality and stability in federated learning In this work , we make two main contributions to address this gap . First , we provide an efficient , constructive algorithm for calculating an optimal federating arrangement . Secondly , we prove the first-ever constant bound on the Price of Anarchy for this game , showing that the worst stable arrangement is no more than 9 times the cost of the best arrangement . We begin Section 4 by defining optimality , drawing on a notion of weighted error derived from the standard objective in federated learning literature . The main contribution of this section is an efficient , constructive algorithm for calculating an optimal arrangement , along with a proof of its optimality . However , as demonstrated in Section 5 , optimality and stability are not always simultaneously achieved . This section analyzes the Price of Anarchy , which measures how far from optimal the worst stable arrangement can be . First , we demonstrate that the optimal arrangement is not always stable . Next , we show that there exist sub-regions where the Price of Anarchy is equal to 1 . Finally , this section proves an overall Price of Anarchy bound of 9 , the first constant bound for this game . It is worth emphasizing that , beyond the Price of Anarchy bound itself , part of the contribution of this work is the optimization and analysis to produce this bound . The proofs for this contribution are modular and illuminate multiple properties about the broader federated learning game under study . As such , these contributions could be useful for further investigating this model . For example , the modular structure of our proof is what enables us to establish stronger bounds for certain sub-cases . The results in this paper are theoretical and do not depend on any experiments or code . However , while writing the paper , we found it useful to write and work with code to check conjectures . Some of that code is publicly available at github.com/kpdonahue/model_sharing_games . 2 Related work . Federated learning As we mentioned previously , federated learning has recently seen numerous advances . In this section , we highlight a few papers in federated learning that are especially related to our work . The idea that agents might differ in their true models ( that data might be generated non-i.i.d . across multiple agents ) is commonly acknowledged in the federated learning literature . For example , Yu et al . [ 2020 ] , Bagdasaryan and Shmatikov [ 2019 ] empirically demonstrate that federated learning , and especially privacy-related additions , can cause a wide disparity in error rates . Some techniques have been developed specifically towards this problem . For example , hierarchical federated learning adds an additional layer of hierarchical structure to federated learning , which could be used to reduce latency or to cluster together similar players ( Lin et al . [ 2018 ] , Liu et al . [ 2020 ] ) . Many other works also relate to clustering , such as ( Lee et al . [ 2020 ] , Sattler et al . [ 2020 ] , Shlezinger et al . [ 2020 ] , Wang et al . [ 2020 ] , Duan et al. , Jamali-Rad et al . [ 2021 ] , Caldarola et al . [ 2021 ] ) . Nagalapatti and Narayanam [ 2021 ] uses tools from cooperative game theory to selectively combine gradient updates from multiple agents . These works , which tend to be more applied than our work , may also differ in that they analyze situations where additional information is known , such as the data distribution at each location . Other work aims to improve accuracy rates by selecting acquiring additional data ( Blum et al . [ 2017 ] , Duan et al . [ 2021 ] ) . Some papers specifically analyze federated learning for high-stakes situations such as medical settings ( Xia et al . [ 2021 ] , Guo et al . [ 2021 ] , Vaid et al . [ 2021 ] , Kumar et al . [ 2021 ] , Zhang et al . [ 2021 ] ) . In general , all of these works have the goal of reducing the average error over all federating agents , which we will use to motivate our definition of optimality in later sections . Game theory in federated learning The closest work to this current paper is our prior paper Donahue and Kleinberg [ 2021 ] , which we discuss in greater detail in Section 3 . Another paper using hedonic game theory to analyze federated learning games is Hasan [ 2021 ] , which gives conditions for Nash stability in federated learning . Other works analyze the incentives of players to contribute resources towards federated learning : Blum et al . [ 2021 ] analyzes fairness and efficiency in sampling additional points for federated learning and Le et al . [ 2021 ] analyzes incentives for agents to contribute computational resources in federated learning . Interestingly , multiple works take a game theoretic approach towards coalition formation in cloud computing , but with the aim of minimizing some cost besides error , such as electricity usage Guazzone et al . [ 2014 ] , Anglano et al . [ 2018 ] . 3 Model and assumptions . We assume that there are M total agents ( sometimes referred to as players ) . Each agent i 2 [ M ] has drawn ni data points from their true local distribution g ( ✓i ) , where ✓i are their true local parameters and g ( · ) is some true labeling function . A player ’ s goal is to learn a model with low expected error on its own distribution . If a player opts for local learning , then it uses its local estimate of these parameters g ( ✓̂i ) to predict future data points , obtaining error erri ( { i } ) . If a set of players C are federating together , we say that they are in a coalition or cluster together . They combine their local estimates of parameters into a single federated estimate , governed by the weighted average of their parameters : ✓̂C = 1P i2C ni · X i2C ni · ✓̂i ( 1 ) A federating player i obtains error erri ( C ) : note that this value may differ between players in the same coalition . For example , if player j has more samples than player k , then ✓̂C will be weighted more towards player j , meaning that player j will have lower expected error than k. The weighted average method in Equation 1 is commonly used in federated learning ( McMahan et al . [ 2016 ] ) . Because it is the most straightforward method , it is sometimes called “ vanilla ” federated learning . Alternative ways of federation might involve customizing the model for individuals , as in domain adaptation . For example , Donahue and Kleinberg [ 2021 ] models three methods of federation : vanilla ( called “ uniform ” ) , as well as two models of domain adaptation . There are multiple reasons why we opted to analyze the federation method in Equation 1 in this work . First of all , this federation method is the most straightforward method , and as such it is the natural candidate to begin analysis . Secondly , this federation method is the most interesting to analyze technically . Domain adaptation serves to increase the incentives of an individual agent to participate in federation : it reduces the tension between an individual ’ s incentives and the optimal overall arrangement . Because of this , for Price of Anarchy it is more valuable and challenging to explore the case in Equation 1 , where incentives are more opposed .

Summary: The paper studies a coalition formation process in federated learning. When agents form a coalition, they aggregate their learned parameters in the hope of a more accurate result. The paper presented an algorithm to obtain the optimal way of partitioning the agents in coalitions, so that the total weighted error of the agents is minimized. In addtion, the paper also analyzed the price of anarchy and provided upper bounds of it.