Title: Incentives in Federated Learning: Equilibria, Dynamics, and Mechanisms for Welfare Maximization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Federated learning

Nash equilibrium

Mechanism design

Welfare maximization
Abstract: Federated learning (FL) has emerged as a powerful scheme to facilitate the collaborative learning of models amongst a set of agents holding their own private data.  Although the agents benefit from the global model trained on shared data, by participating in federated learning, they may also incur costs (related to privacy and communication) due to data sharing. In this paper, we model a collaborative FL framework, where every agent attempts to achieve an optimal trade-off between her learning payoff and data sharing cost. We show the existence of Nash equilibrium (NE) under mild assumptions on agents' payoff and costs. Furthermore, we show that agents can discover the NE via best response dynamics. However, some of the NE may be bad in terms of overall welfare for the agents, implying little incentive for some fraction of the agents to participate in the learning. To remedy this, we design a budget-balanced mechanism involving payments to the agents, that ensures that any $p$-mean welfare function of the agents' utilities is maximized at NE. In addition, we introduce a FL protocol FedBR-BG that incorporates our budget-balanced mechanism, utilizing best response dynamics. Our empirical validation on MNIST and CIFAR-10 substantiates our theoretical analysis. We show that FedBR-BG outperforms the basic best-response-based protocol without additional incentivization, the standard federated learning protocol FedAvg, as well as a recent baseline MWFed in terms of achieving superior $p$-mean welfare.

Incentives in Federated Learning: Equilibria, Dynamics, and Mechanisms for Welfare Maximization

Abstract

Federated learning (FL) has emerged as a powerful scheme to facilitate the collaborative learning of models amongst a set of agents holding their own private data. Although the agents benefit from the global model trained on shared data, by participating in federated learning, they may also incur costs (related to privacy and communication) due to data sharing. In this paper, we model a collaborative FL framework, where every agent attempts to achieve an optimal trade-off between her learning payoff and data sharing cost. We show the existence of Nash equilibrium (NE) under mild assumptions on agents' payoff and costs. Furthermore, we show that agents can discover the NE via best response dynamics. However, some of the NE may be bad in terms of overall welfare for the agents, implying little incentive for some fraction of the agents to participate in the learning. To remedy this, we design a budget-balanced mechanism involving payments to the agents, that ensures that any p -mean welfare function of the agents' utilities is maximized at NE. In addition, we introduce a FL protocol FedBR-BG that incorporates our budgetbalanced mechanism, utilizing best response dynamics. Our empirical validation on MNIST and CIFAR-10 substantiates our theoretical analysis. We show that FedBR-BG outperforms the basic best-response-based protocol without additional incentivization, the standard federated learning protocol FedAvg (McMahan et al. [2017]), as well as a recent baseline MWFed (Blum et al. [2021]) in terms of achieving superior p -mean welfare.

1 Introduction

Federated Learning (FL) has emerged as an effective collaborative training paradigm, where a group of agents can jointly train a common machine learning model. The success of collaborative learning paradigms is visible in the domains of autonomous vehicles (Elbir et al. [2020]), digital healthcare (Dayan et al. [2021], Xu et al. [2021], Nvidia [2019]), multi-devices (Learning [2017]), and biology (Bergen and Petryshen [2012]). Although such collaborative learning is immensely beneficial to the agents, individually, they may not be incentivized to share their data. This is because sharing data may incur costs attributed to bandwidth use, privacy leakage, and the use of computing resources. In turn, high costs and low learning payoffs may cause some of the agents to drop out of FL, resulting in subpar learning. As noted by Blum et al. [2021], the success of FL systems depends on its ability to attract and retain a large number of federating agents.

Thus, it is crucial to achieve fairness and welfare guarantees for all participating agents.

This calls for game-theoretic modeling and analysis of the agent's payoff and costs, and subsequent mechanism design to incentivize participation. Towards the former, Blum et al. [2021] introduce a model, where every agent receives a payoff from the collaboration measuring the 'learning benefit' the agent derives from the total data shared. In particular, when each agent i contributes s i units of data, the payoff for agent i is captured by a i ( s ) , where s = ⟨ s 1 , s 2 , . . . , s n ⟩ is the data contribution

profile, or sample vector . Blum et al. [2021] consider the constrained cost-minimizing model , where each agent minimizes her data contribution ( s i ) subject to the requirement that her payoff should be larger than a threshold ( a i ( s i , s -i ) ≥ µ i ). They show that a Nash equilibrium (NE) (Nash [1951]), arguably the most sought after solution concept in game theory, may not always exist in this model. They derive sufficient conditions for existence of NE, as well as provide novel structural results about the equilibria.

We note that an agent's cost for sharing data may be more complex than just the size of the total data shared. There are studies dedicated to quantifying the losses (attributed to data collection, processing, and privacy) that are incurred with increased sharing of data (Li and Raghunathan [2014], Laudon [1996], Jaisingh et al. [2008]). Furthermore, within game theory (and economics) typically agents are considered as net utility maximizers, i.e., maximizing payoff minus cost, for example value minus payment in auctions (Myerson [1981]), and revenue minus cost in markets (Huber et al. [2001], Börgers [2015]). To capture these aspects, in this paper we propose an unconstrained utilitarian model , where we define the utility u i ( s ) of each agent i as the difference between her payoff and her cost of sharing data, i.e., u i ( s ) = a i ( s ) -c i ( s i ) , where c i ( s i ) denotes the cost incurred by agent i on sharing s i data samples. Each agent aims to maximize their utility. This model is natural in FL settings where there is a natural coupling between payoffs and costs, and there are no hard cost constraints, i.e., having an upper bound on the total cost that an agent can incur. Since utility functions are directly indicative of the welfare for the agents, this facilitates defining global fairness and welfare criteria in terms of the utilities of the agents. This motivates the following questions:

When agents strategize on data contribution to maximize their net utility, does a Nash equilibrium always exist? If yes, then can agents actually discover a Nash equilibrium when acting independently? Can any welfare function of utilities of the agents be optimized at such a Nash equilibrium by designing new rules ( mechanisms )?

The goal of this paper is to address the above questions. Before we describe our contributions, we note that Karimireddy et al. [2022] also consider a framework similar to ours, by defining the utility function to be payoff minus cost. However, their focus is data maximization and avoiding free-riding , while the focus of our paper is to design mechanisms that achieve fairness and welfare for all agents. Furthermore, our model generalizes the model of Karimireddy et al. [2022] as we do not require the agents' data to be identically distributed and in turn to have identical payoff functions.

1.1 Our Contributions

We work under the concavity assumption on the utility functions. In particular, we assume that the payoff functions are concave and cost functions are convex. These assumptions are standard in the literature (Blum et al. [2021], Karimireddy et al. [2022]). Convexity of cost functions is a natural choice since it captures the property of increasing marginal costs. For instance, data sharing through ordered selection , i.e., sharing records in ascending order of costs involved for collecting the records, results in convex cost functions. There are more models that result in strictly convex cost functions (Li and Raghunathan [2014]). Similarly, several important ML models exhibit concave payoff functions; for instance payoffs in linear or random discovery models, random coverage models, and general PAC learning are all concave - see Section 2.1. Furthermore, there is empirical evidence that the accuracy function in neural networks under the cross-entropy loss is also concave (Kaplan et al. [2020]).

Existence and Reachability of Nash Equilibrium: Under the concavity assumptions, we show that a Nash equilibrium (NE) is guaranteed to exist. We note that our existence result holds for more general settings than those of Blum et al. [2021] - we do not assume that the cost functions are linear, or that the utility functions have bounded derivatives. In particular, our result shows the existence of NE under the general PAC learning framework (see Section 2.1); under this setting an equilibrium in the model of Blum et al. [2021] fails to exist. We also demonstrate that if the concavity assumption is relaxed, there exist instances which do not admit NE.

Furthermore, we show that the agents can discover a NE through an intuitive best response dynamics , where agents update their data contribution proportional to the gradient of their utility functions in the direction that increases their utility. We show that for strongly concave utility functions and under the mild assumption of the gradient of the utilities being bounded, the best response dynamics converges

to an approximate NE in time polynomial in O (log( ε -1 )) . This contribution may be of independent interest to equilibrium computation in concave games.

A Fair and Welfare Maximizing Mechanism: In general, a Nash equilibrium need not be fair or maximize any notion of welfare for the agents (see Example 1). Therefore, we next focus on designing mechanisms that optimize welfare of the agents at its NE. The generalized p -mean welfare, defined to be the p th -norm of the utilities of the agents ( 1 n ∑ i ∈ [ n ] u i ( s ) p ) 1 p for p ∈ ( -∞ , 1] , constitutes a family of functions characterized by natural fairness axioms including the Pigou-Dalton principle (Moulin [2003]). This notion encompasses well studied notions of fairness and economic welfare, such as (i) the average social welfare ( 1 n ∑ i ∈ [ n ] u i ( s )) when p = 1 , (ii) the egalitarian welfare (min i ∈ [ n ] u i ( s )) when p = -∞ , which is a fundamental measure of fairness, and (iii) the Nash welfare ( ∏ i ∈ [ n ] u i ( s )) 1 /n in the limiting p → 0 case, which is a popular notion in social choice theory that achieves a balance between welfare and fairness (Varian [1974], Caragiannis et al. [2019]).

As our second main contribution, for linear costs, we design a budget-balanced mechanism that always admits a Nash equilibrium. The mechanism involves payments , however, budget-balancedness ensures that the total payment of all the agents is zero, and thereby the central server operates on no-profit-no-loss. Moreover we show that when the sample vector at NE is positive, i.e., all agents contribute positive amount of data samples, then the NE maximizes the p -mean welfare among all positive sample vectors. Since we can ensure the server does not communicate with an agent who does not contribute any data points, insisting on positive sample vectors is a mild assumption.

Our mechanism builds on ideas from a mechanism of Falkinger et al. [2000] for the efficient provisioning of public goods. The key idea is to compensate an agent who incurs a cost higher than the average cost incurred by other agents via a subsidy proportional to her excess cost; likewise, agents incurring a lower cost than average of others are proportionally taxed. By setting the level of tax/subsidy carefully, we show that the NE of the mechanism are p -mean welfare maximizing. To the best of our knowledge, our work is the first to explore this intimate connection between FL and public goods provisioning. Our results highlight the promise of bridging the fields of machine learning and social choice theory.

Once we have established the mechanism, we show that a corollary of our first main result implies that the best response dynamics under this mechanism will lead to the discovery of the p -mean welfare maximizing Nash equilibrium.

Empirical Evaluation: We design a distributed training protocol for FL based on our mechanism, called FedBR-BG. We compare our algorithm with three other algorithms: the distributed protocol for the vanilla mechanism without budget balancing FedBR, the standard federated learning protocol FedAvg (McMahan et al. [2017]), and a recent adaptation of FedAvg called MWFed (Blum et al. [2021]). We show that our budget-balancing algorithm achieves superior p -mean welfare under different settings on two datasets, MNIST (LeCun et al. [2010]) and CIFAR-10 (Krizhevsky [2009]).

1.2 Related Work

We mention some additional literature on welfare maximization and incentives in FL, as well as other related mechanisms in public goods theory.

Welfare maximization in FL. Typically federated learning protocols compute a model which maximize some weighted sum of agent accuracies (i.e. utilties). Examples of such protocols include the standard FedAvg (McMahan et al. [2017]), AFL (agnostic FL) (Mohri et al. [2019]), and q -FFL (Li et al. [2020]). However, unlike our work, all these methods ignore the strategic aspects arising from costs involved in data sharing, and instead assume agents honestly contribute all their data.

Incentives in FL. This line of work adopts game theory for incentivizing the contribution of data owners. Common models include the Stackelberg game, non-cooperative game, and coalition game. More specifically, the Stackelberg game is employed to optimize the utility of both the server and agents Feng et al. [2019]. On the other hand, in non-cooperative games, the server or the agent seeks to maximize its own utility Zou et al. [2019]. While most previous methods analyze the properties of a certain scenario, we aim to design mechanisms that achieve fairness and welfare for all agents.

Mechanisms for public good provisioning. There is a long line of work for the efficient provisioning of public goods, beginning from Samuelson [1954]. Several works such as Falkinger et al. [2000], Andreoni and Bergstrom [1996] and Bergstrom et al. [1986] use the idea of imposing a tax/subsidy on the agents but differ in the specific manner in which this tax/subsidy is imposed. While our mechanism is inspired by Falkinger et al. [2000], our model is more general than theirs. The design of our mechanism for this general model and the proof of its properties are our novel contributions.

2 Problem formulation

We consider a federated learning problem with n agents who wish to jointly solve a common learning problem. Let D i be the distribution of data points available to agent i . Towards jointly solving the learning problem, each agent i contributes some set T i ∼ D s i i of s i data samples. Under the assumption that agent i 's data is i.i.d. from their distribution D i , each agent's contribution can be captured simply by their contribution level, i.e., amount of data samples they contribute. Let S i ⊆ R ≥ 0 be the set of feasible contribution levels of agent i , and let S := × j S j . Given a sample vector s = ( s 1 , . . . , s n ) ∈ S , the central server returns model(s) trained using the samples ⋃ i T i .

In our model, each agent i derives a payoff from the jointly learned model, e.g. the payoff could be the accuracy of the model. We assume a general framework which models the payoff of agent i as a function a i : S → R ≥ 0 . We typically assume each payoff function a i is bounded, continuous in s , and non-decreasing in the contribution s i of agent i . Moreover each agent i incurs a cost associated with data sharing captured through a non-decreasing cost function c i : S i → R ≥ 0 . The net utility of agent i is the payoff minus cost, i.e.,

Given the above framework, the goal of an agent i is to decide how many samples to contribute so that her net utility u i ( · ) is maximized. Note that the utility of agent i depends on the contributions of other agents as well. Further, we can assume without loss of generality that each set S i = [0 , τ i ] for some threshold τ i > 0 . This is because we can discard contribution levels where an agent gets negative utility. Since the payoff a i ( s ) is bounded above by some constant A i > 0 and costs are increasing, agent i cannot obtain a positive utility by contributing more than τ i := c -1 i ( A i ) samples.

Payoff and cost functions are assumed to be concave and convex respectively (Blum et al. [2021], Karimireddy et al. [2022]). As discussed in Section 1.1, it is natural to assume that cost functions are convex to capture increasing marginal costs (Li and Raghunathan [2014]). Similarly, there are ample justifications to concave payoff functions, as discussed in Section 2.1 where we analyze payoff functions arising from some of the canonical learning paradigms, and their concavity properties .

Remark 1 . Our framework generalizes those of Blum et al. [2021] and Karimireddy et al. [2022]. In the former, an agent's goal is to contribute the fewest number of data samples subject to ensuring that their payoff crosses a certain threshold; and in the latter all agents have a common payoff function that is a function of ∥ s ∥ 1 = s 1 + · · · + s n , and linear cost functions.

Nash Equilibrium (NE). Arguably the most sought after solution concept within game theory is of Nash Equilibrium (Nash [1951]), a stable state or an equilibrium state of the system where no agent gains by unilaterally changing their data contribution level. Formally,

Definition 1 (Nash equilibrium (NE)) . A sample vector s ∈ S is said to be at a Nash equilibrium if for every i ∈ [ n ] , and every s ′ i , we have u i ( s ) ≥ u i ( s ′ i , s -i ) where ( s ′ i , s -i ) = ( s 1 , . . . , s ′ i , . . . , s n ) .

An alternate view of a Nash equilibrium relies on the concept of best response . Given the sample contributions s -i of other agents, the set f i ( s -i ) of contribution levels of agent i that maximize her utility is the best response set of agent i :

The best response correspondence f is then defined as a set-valued function f : S → × i 2 S i , where [ f ( s )] i = f i ( s -i ) . It is then clear that:

Proposition 1. A sample vector s ∈ S is a Nash equilibrium if and only if it is a fixed point of the best response correspondence, i.e., s ∈ f ( s ) .

2.1 Canonical examples of payoff functions

We now discuss a few examples of payoff functions that are captured by our general framework. In all the examples below, the payoff functions a i ( s ) are non-negative, bounded, continuous, nondecreasing, and concave in s i for any fixed strategy profile s -i of the other agents.

Linear or Random discovery. In this model, the payoff is linear in the sample vector and is given by a i ( s ) = ( W s ) i for a matrix W . For example, Blum et al. [2021] consider a setting where each agent has a sampling probability distribution q i over the instance space X and gets a reward equalling q ix whenever the instance x is sampled by any agent. Then the expected payoff to agent i is a i ( s ) = ( QQ T s ) i , where Q is a matrix given by Q [ i, x ] = q ix for i ∈ [ n ] and x ∈ X . Here W = QQ T is a symmetric PSD matrix with an all one diagonal.

Random coverage. In the above setting, agent i obtains a reward q ix each time some agent samples instance x . In the random coverage model arising in binary classification (Blum et al. [2021]), an agent gets this reward only once. Under this model, the payoff given by expected accuracy takes the form:

Generalization bounds from general PAC learning. Consider a general learning setting where we want to learn a model h from a hypothesis class H which minimizes the error over a data distribution D given by R ( h ) = E ( x,y ) ∈D e ( h ( x ) , y ) , for some loss function e ( · ) . Given m i.i.d. data points, the empirical risk minimizer (ERM) can be computed as the model h m = arg min h ∈H ∑ ℓ ∈ [ m ] e ( h ( x ℓ ) , y ℓ ) . Mohri et al. [2018] show the following bound on the error of h m , which holds with high probability:

where (1 -a 0 ) is the accuracy of the optimal model from H , and k is a (constant) measure of the difficulty of the learning problem depending on e ( · ) and H . Using this we can define the agent payoff a i function as the accuracy of the learning task as a i ( s ) = a ( ∥ s ∥ 1 ) 1 .

Empirical evidence. Kaplan et al. [2020] discuss empirical scaling laws relating the cross-entropy loss on neural language models. They observe that the loss scales with the dataset size m as a power law ℓ ( m ) = α · m -β , for some parameters α > 0 and β ∈ (0 , 1] . This naturally defines the payoff function as the accuracy of the learning task as

In addition, the pay-off functions of current large language models (e.g., accuracies) are also nonnegative and non-decreasing as a function of the data size (Henighan et al. [2020]).

3 Nash Equilibrium: Existence and Best Response Dynamics

In this section, we investigate the existence and computation of a Nash equilibrium. We start with two positive results showing the existence of a Nash equilibrium for a broad class of payoff and cost functions.

Theorem 3.1. A Nash equilibrium exists in any federated learning problem where for every agent i the payoff function a i ( s ) is continuous in s and concave in s i , and cost function c i is increasing and convex in s i .

Proof. We will show the existence of a Nash equilibrium by showing that the best response correspondence f has a fixed point. First observe that f is defined over a compact, convex domain S = × j S j since each S j is convex. Next, note that agent i 's utility function u i ( s ) = a i ( s ) -c i ( s i ) is continuous in s due to the continuity of a i and c i . The continuity of u i in s and the concavity of

u i in s i implies the upper semi-continuity of the best response correspondence f i . Moreover, u i is concave in s i for each fixed s -i , since a i and -c i are concave in s i . Thus for each fixed s -i , the best response set f i ( s -i ) ⊆ R ≥ 0 is a non-empty interval, and hence is also convex. Thus f is a upper semi-continuous non-empty and convex valued correspondence defined over a compact, convex domain. By the Kakutani fixed-point theorem (Kakutani [1941]), f admits a fixed point.

The above result can also be proved directly by invoking Rosen [1965], who showed the existence of a Nash equilibrium of n -person concave games, where the utility function of an agent i is defined over closed, compact set, is continuous and concave in i 's own strategy.

Implications. Theorem 3.1 shows that a Nash equilibrium exists when payoffs are concave and costs are convex. All our motivating examples from Section 2.1 lie under this concave/convex regime and therefore admit a NE. In Appendix A we discuss existence and non-existence of Nash equilibrium in our model when we go beyond the concave-convex regime of payoff and cost functions. In particular, we show that Nash equilibrium exists even with decreasing payoff function of an agent as long as the function is separable between her and other agents' data contribution. We also show that a NE need not exist even with linear cost functions if the payoff functions are non-concave.

Best response dynamics. We now turn to computation and consider a natural procedure by which agents can find a Nash equilibrium: best response (BR) dynamics . Agents start with some initial sample vector s 0 . In each step t of the BR dynamics, every agent i updates their sample contribution proportional to the gradient ∂u i ∂s i in the direction of increasing utility. Concretely, for a scalar step size δ t > 0 , the updates take the following form:

where g ( s t , µ t ) i = ∂u i ( s t ) ∂s i + µ t i and µ t is chosen so that the updated sample vector s t +1 lies in the feasible region S . Specifically, µ t ∈ arg min µ ∈ K ∥ g ( s t , µ ) ∥ 2 , where K = { µ : s t + δ t · g ( s t , µ ) ∈ S} . For instance, if 0 ≤ s i ≤ τ i , then:

We measure convergence of the above dynamics to a Nash equilibrium via the L 2 norm of the update direction g ( s t , µ t ) . Under mild assumptions on the utility functions, we show the dynamics (4) converges to an approximate Nash equilibrium where ∥ g ( s t , µ t ) ∥ 2 < ε :

Theorem 3.2. Let G ( s ) be the Jacobian of u : S → R n , i.e., G ( s ) ij = ∂ 2 u i ( s ) ∂s j ∂s i . Assuming agent utility functions u i satisfy

Strong concavity: ( G + λ · I n × n ) is negative semi-definite,

Bounded derivatives: | G ij | ≤ L ,

for constants λ, L > 0 , the best response dynamics (4) with step size δ t = λ n 2 L 2 converges to an approximate Nash equilibrium s T where ∥ g ( s T , µ T ) ∥ 2 < ε in T iterations, where

Below we sketch the proof and defer the full proof to Appendix A.

Proof sketch. We measure convergence of the above dynamics by the error term ∥ g ( s t , µ t ) ∥ 2 . We show the following bound:

Wethen use strong concavity to show g ( s t , µ t ) T G ( s ′ ) g ( s t , µ t ) ≤ -λ ∥ g ( s t , µ t ) ∥ 2 2 , and the bounded derivatives property to show ∥ G ( s ′ ) g ( s t , µ t ) ∥ 2 ≤ nL ∥ g ( s t , µ t ) ∥ 2 . For our choice of the step size δ t = λ n 2 L 2 , we can relate the error in subsequent iterations as follows:

This then allows us to conclude that after T = 2 n 2 L 2 λ 2 log ( ∥ g ( s 0 ,µ 0 ) ∥ 2 ε ) iterations, we will have an approximate Nash equilibrium s T with ∥ g ( s T , µ T ) ∥ 2 ≤ ε .

Implications. The above theorem implies that agents playing the natural best response update strategy will converge quickly (in O (log( ε -1 )) iterations) to a NE. We note that our theorem applies to the payoff functions defined by generalization bounds (eq. (2)) and observed in practice (eq. (3)) as they are strongly concave and have bounded derivatives. Moreover, under the assumptions of Theorem 3.2, our proof also implies the fast convergence of the best response dynamics in the budget-balanced mechanism we discuss in the next section.

4 Welfare Maximization: A Budget-Balanced Mechanism

We first note through an example that Nash equilibria need not be welfare maximizing.

Example 1 . Consider two agents with identical payoff functions a ( s ) = 8 -( s 1 + s 2 ) -1 , and linear cost functions given by c 1 ( s 1 ) = 5 c 1 and c 2 ( s 2 ) = 4 c 2 . The NE is given by s ∗ = (0 , 0 . 5) , i.e., agent 1 does not contribute any data samples. The Nash welfare (which is the p -mean welfare in p → 0 limiting case) of this NE is u 1 ( s ∗ ) · u 2 ( s ∗ ) = 24 . However consider another sample vector s ′ = (0 . 2 , 0 . 4) where agent 1 increases her contribution and agent 2 reduces her contribution. Then s ′ has a higher Nash welfare of u 1 ( s ′ ) · u 2 ( s ′ ) = 25 . 25 > 24 .

To address the issue of NE not being welfare-maximizing, by designing a budget-balanced mechanism for agents with linear cost functions. Our mechanism is inspired from a mechanism for the efficient provisioning public goods (Falkinger et al. [2000]). We show that our mechanism always admits a NE. Moreover, when the sample vector at NE is positive, i.e. every agent participates by contributing data, the NE maximizes the p -mean welfare among all positive sample vectors. For the FL setting, assuming positive sample vectors is natural since the only way for an agent to participate is by making some positive data contribution.

̸

We present our result for a more general model than one discussed so far. Not only does this generalization strengthen our result, it also naturally allows expressing agent utilities in terms of the taxes/subsidies they receive from our mechanism. In this generalization, each agent i has a budget B i of which b i is unspent and the remaining is used towards the cost of sampling s i data points, i.e., b i + c i ( s i ) = B i . We assume agents have arbitrary continuous, concave utility functions of the form u i ( b i , ∥ s ∥ 1 ) . Agents have linear cost functions c i ( s i ) = c i · s i + d i , with c i > 0 . This model already captures some previously discussed settings (Sec 2.1) as follows. For e.g., when payoff functions are derived from generalization bounds (Mohri et al. [2018]) or empirical evidence (Kaplan et al. [2020]), they take the form a i ( s ) = ˆ a i ( ∥ s ∥ 1 ) , for some function ˆ a i which depends on ∥ s ∥ 1 . Then for u i ( b i , ∥ s ∥ 1 ) = b i +ˆ a ( ∥ s ∥ 1 ) , the budget constraint implies the utility takes the form ˆ a i ( ∥ s ∥ 1 ) -c i ( s i ) + B i , which is the same as the utility under the original model with a constant offset. We also assume that for all i , ∂u i ( b i ,S ) ∂S > 0 and ∂u i ( b i ,S ) ∂b i = 0 .

Mechanism M β . We design a mechanism parametrized by a scalar β ∈ (0 , 1) . It uses the following payment scheme. At a data contribution vector s , each agent i is rewarded an amount

̸

Thus if an agent incurs higher (resp. lower) sampling cost than the average cost borne by other agents, we compensate (resp. penalize) her by a subsidy (resp. tax) of β times her excess cost. By design, our mechanism is budget-balanced: at any sample vector s , we have:

̸

Under this mechanism, given a vector of contributions s -i of agents other than agent i , the best response of agent i is any solution to the following optimization problem:

̸

We next define a β value that plays a crucial role in our mechanism.

Definition 2. (Optimal parameter β ∗ ) Let A := ( ∑ i c -1 i ) -1 and C := ∑ i c i . Then we define β ∗ as the solution to the following equation.

which satisfies 0 ≤ β ∗ ≤ 1 -1 /n .

The next lemma shows that (6) indeed has such a root - the proof is deferred to Appendix B.

Lemma 1. The equation Cβ 2 -( An ( n -2) + C ) β + A ( n -1) 2 = 0 of (6) has a real root β ∗ where 0 ≤ β ∗ ≤ 1 -1 /n .

We now state the main result of this section. We show that for every β ∈ [0 , 1] , a Nash equilibrium of M β exists. Additionally, for a specific value of β = β ∗ (Definition 2), our mechanism admits Nash equilibria which are socially efficient: it maximizes the p -mean welfare: W p ( b , s ) = ( ∑ i u i ( b i , ∥ s ∥ 1 ) p ) 1 /p , for p ≤ 1 among all positive sample vectors s .

Theorem 4.1. For each β ∈ [0 , 1] , the mechanism M β admits a Nash equilibrium. For β = β ∗ (Definition 2), whenever the NE s ∗ of M β ∗ satisfies s ∗ > 0 , the NE s ∗ maximizes the p -mean welfare among all vectors s > 0 , for any p ≤ 1 .

We now sketch the proof of the above theorem. The full proof is deferred to Appendix B.

Proof sketch. When 0 ≤ β ≤ 1 , the first constraint in the above program is a convex constraint even for general convex cost functions. Since u i ( · ) is concave, a proof similar to the proof of Theorem 3.1 shows the existence of a Nash equilibrium.

The program maximizing p -mean welfare as follows.

̸

We first show that the above program is convex. With µ i and γ i as the dual variables to the first and second constraints respectively, we write down the KKT conditions of program (7) with s i > 0 . We then use the KKT conditions satisfied by a NE ( b ∗ , s ∗ ) to find values of the dual variables µ ∗ and γ ∗ so that ( b ∗ , s ∗ , µ ∗ , γ ∗ ) satisfy the KKT conditions of (7). Since KKT conditions are sufficient for optimality, this shows that the NE ( b ∗ , s ∗ ) also maximizes p -mean welfare.

Implications. Theorem 4.1 shows that by augmenting the federated learning with a simple payment protocol that is budget-balanced, one can obtain NE that maximize any p -mean welfare function of the net-utilities of the agents. Furthermore, note that the payment augmented utility function is still essentially of the form u i ( s ) = ˆ a i ( s ) -(1 -β ) c i ( s i ) -β n -1 c j ( s j ) , with a constant offset. Since β ∈ [0 , 1] , when a i ( · ) are λ -strongly concave and c i ( · ) are λ -strongly convex, the functions u i ( · ) are λ -strongly concave. Therefore Theorem 3.2 applies, which ensures that the welfare maximizing NE can be reached through the simple best response dynamics quickly. Finally, we remark that when cost functions are identical, the value of the optimal parameter β ∗ is (1 -1 /n ) , which is exactly the value used by Falkinger et al. [2000].

Remark 2 . Our mechanism requires that costs of the agents be publicly known in order to calculate the value of β ∗ by solving (6) in Definition 2. This is a common assumption made in previous work (e.g. Karimireddy et al. [2022] and Blum et al. [2021]) and is justified in practice. Indeed, costs are common knowledge in many real-world applications e.g. (1) in many ML applications, each agent derives their training data from manually labeling a subset of a publicly available dataset like CIFAR or ImageNet, and the cost of labeling dataset is usually known; (2) in autonomous driving, where each data point is a random path taken under random external conditions.

5 Distributed Algorithm and Empirical Evaluation

In this section, we realize our budget-balanced mechanism in a real-world FL system. We perform the evaluation on the MNIST (LeCun et al. [2010]) and CIFAR-10 (Krizhevsky [2009]) datasets. We compare our mechanism with two baselines: the standard FedAvg and MWFed Blum et al. [2021]. We denote the vanilla mechanism without budget balancing as FedBR and the budget-balanced mechanism as FedBR-BG. We demonstrate that compared to the FedBR, FedBR-BG achieves better p -mean welfare for p ≤ 1 . We also show that the standard FL protocol FedAvg gives significantly lower welfare since it does not allow agents to change their contribution.

5.1 Algorithm Details

We first define the concrete forms for payoff and cost functions. According to standard practice in FL, an agents payoff is measured through the accuracy evaluated on her local test data given model θ , which has the form a i ( s , θ ) = 1 | D ∗ i | ∑ ( x,y ) ∈ D ∗ i 1 [ θ ( x ) = y ] , where D ∗ i is the test data for agent i . We note that this form inherently aligns with a i ( s ) = ˆ a i ( ∥ s ∥ 1 ) because the received global model θ is trained on ∥ s ∥ samples. We consider linear cost functions where c ( s ) = c s .

1 i i i i

̸

Now we derive the update rule for agent contributions for FedBR-BG based on best-response dynamics. We rewrite the utility of agent i in the budget-balancing mechanism as u i ( s ) = ˆ a i ( ∑ i s i ) -(1 -β ) c i s i -β n -1 ∑ j = i c j s j . We then compute its gradient with respect to s i : ∂u i ∂s i = ∂ ˆ a i ( ∑ i s i ) ∂s i -(1 -β ) c i . Since the accuracy function is generally unknown in practice, the server can train a public accuracy function ˜ a : ˜ S → R ≥ 0 and broadcast it to all agents. We obtain an estimate of the accuracy ˜ a by evaluating models trained on different numbers of samples, in increasing intervals of ∆ s . For example, if the server trains models on 0 , 2 , 4 , · · · samples and evaluates their accuracy to obtain the estimated accuracy ˜ a , the interval ∆ s = 2 in this case. We assume ( τ 1 , . . . , τ n ) ∈ ˜ S as the server can be a service provider with plenty of data sources. With ˜ a , we can approximate the gradient as ∂u i ∂s i ≈ ˜ a ( ∑ i s i +∆ s ) -˜ a ( ∑ i s i ) ∆ s -(1 -β ) c i . Similarly, ∂u i ∂s i ≈ ˜ a ( ∑ i s i +∆ s ) -˜ a ( ∑ i s i ) ∆ s -c i for FedBR, and u i ( s ) = a i ( ∑ i s i ) -c i s i for FedAvg, MWFed and FedBR.

We further assume that s i is a multiple of ∆ s to ensure that ˜ a is always well-defined for our chosen contributions. Finally, we leverage best-response dynamics to update agent contributions. We present the full description of the algorithm for FedBR-BG and FedBR as Algorithm 1 and Algorithm 2 in Appendix C, respectively.

5.2 Experiment Setup

We conduct the experiments with 10 agents for MNIST and 100 agents for CIFAR-10. For MNIST, we use a CNN as the global model, which has two 5 × 5 convolution layers followed by two fully connected layers with ReLU activation. For CIFAR-10, we use VGG11 (Simonyan and Zisserman [2014]). We consider the i.i.d. setting, i.e., the local data of agents are sampled from the same distribution. For both datasets, each agent has 100 training images and 10 testing images, i.e., τ i = 100 , ∀ i ∈ [ n ] .

We randomly initialize the contributions as a multiplier of 10 in [0 , 100] . In each contribution updating step, we re-initialize the global model and perform FedAvg for 50 communication rounds. We set global learning rate η to 1.0, local learning rate α to 0.01, and momentum to 0.9. We set the number of contribution updating steps to 100 and the sample number interval to 10. For evaluating FedAvg, we simply optimize the global model with the same hyperparameters while keeping individual contribution to τ i .

5.3 Experiment Results

We show the p -mean welfare of our method and baselines on MNIST and CIFAR-10 in Table 1. We observe that FedBR-BG achieves better p -mean welfare on both datasets compared to FedBR and FedAvg, verifying our theoretical results. Note that the p -mean of FedAvg is significantly lower since agents always contribute all their data in FedAvg, which incurs a high cost so that the marginal gain becomes limited.

Table 1: p -mean welfare of our budget-balanced mechanism FedBR-BG and baselines on MNIST and CIFAR10. We report the results for different p . The cost for adding one data sample c i is 0.005 for every agent.

6 Discussion

In this paper, we formulated a federated learning framework which incorporates both payoffs an agent receives from sharing data as well as the cost she incurs due to sharing data. We show the existence of Nash equilibria under the assumption of concave payoffs and convex costs, which are mild assumptions observed in practice. We then note that while NE exist, they may not maximize any notion of welfare for the agents, leaving agents with less incentive to participate. We address this issue via a budget-balanced mechanism with payments whose NE maximize the p -mean welfare of the agent utilities. Our experiments show that FedBR-BG achieves better p -mean welfare compared to FedBR and FedAvg, verifying our theoretical results.

We conclude by discussion some directions for future work. Our mechanism required that costs of the agents be publicly known, or at least verifiable. Studying incentives when costs are not common knowledge is an interesting question. Another assumption of our mechanism was that an agent's payoff depends on the number of data samples. Designing welfare-maximizing mechanisms for general settings where payoff functions take more general forms is another direction for future work.

Acknowledgements. This work is partially supported by the National Science Foundation under grant No. 1750436, No. 1910100, No. 2046726, No. 2229876, DARPA GARD, the National Aeronautics and Space Administration (NASA) under grant no. 80NSSC20M0229, the Alfred P. Sloan Fellowship, and the Amazon research award.

References

James Andreoni and Ted Bergstrom. Do government subsidies increase the private supply of public goods? Public Choice , 88(3-4):295-308, 1996. URL https://EconPapers.repec.org/RePEc: kap:pubcho:v:88:y:1996:i:3-4:p:295-308 .

Sarah E Bergen and Tracey L Petryshen. Genome-wide association studies of schizophrenia: does bigger lead to better results? Current opinion in psychiatry , 25(2):76-82, 2012.

Theodore Bergstrom, Lawrence Blume, and Hal Varian. On the private provision of public goods. Journal of Public Economics , 29(1):25-49, 1986. ISSN 0047-2727. doi: https://doi.org/10.1016/ 0047-2727(86)90024-1. URL https://www.sciencedirect.com/science/article/pii/ 0047272786900241 .

Avrim Blum, Nika Haghtalab, Richard Lanas Phillips, and Han Shao. One for one, or all for all: Equilibria and optimality of collaboration in federated learning. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 1005-1014. PMLR, 18-24 Jul 2021. URL https://proceedings.mlr.press/v139/blum21a.html .

Tilman Börgers. An introduction to the theory of mechanism design . Oxford University Press, USA, 2015.

S.P. Boyd and L. Vandenberghe. Convex Optimization . Number pt. 1 in Berichte über verteilte messysteme. Cambridge University Press, 2004. ISBN 9780521833783. URL https://books. google.com/books?id=mYm0bLd3fcoC .

Ioannis Caragiannis, David Kurokawa, Hervé Moulin, Ariel D. Procaccia, Nisarg Shah, and Junxing Wang. The unreasonable fairness of maximum Nash welfare. ACM Trans. Econ. Comput. , 7(3), sep 2019. ISSN 2167-8375.

Ittai Dayan, Holger R Roth, Aoxiao Zhong, Ahmed Harouni, Amilcare Gentili, Anas Z Abidin, Andrew Liu, Anthony Beardsworth Costa, Bradford J Wood, Chien-Sung Tsai, et al. Federated learning for predicting clinical outcomes in patients with covid-19. Nature medicine , 27(10): 1735-1743, 2021.

Ahmet M Elbir, Burak Soner, and Sinem Coleri. Federated learning in vehicular networks. arXiv preprint arXiv:2006.01412 , 2020.

Josef Falkinger, Ernst Fehr, Simon Gächter, and Rudolf Winter-Ebmer. A simple mechanism for the efficient provision of public goods: Experimental evidence. The American Economic Review , 90 (1):247-264, 2000. ISSN 00028282. URL http://www.jstor.org/stable/117291 .

Shaohan Feng, Dusit Niyato, Ping Wang, Dong In Kim, and Ying-Chang Liang. Joint service pricing and cooperative relay communication for federated learning. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) , pages 815-820. IEEE, 2019.

Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701 , 2020.

Frank Huber, Andreas Herrmann, and Robert E Morgan. Gaining competitive advantage through customer value oriented management. Journal of consumer marketing , 18(1):41-53, 2001.

Jeevan Jaisingh, Jack Barron, Shailendra Mehta, and Alok Chaturvedi. Privacy and pricing personal information. European Journal of Operational Research , 187(3):857-870, 2008.

Shizuo Kakutani. A generalization of Brouwer's fixed point theorem. Duke Mathematical Journal , 8 (3):457 - 459, 1941. doi: 10.1215/S0012-7094-41-00838-4. URL https://doi.org/10.1215/ S0012-7094-41-00838-4 .

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR , abs/2001.08361, 2020. URL https://arxiv.org/abs/2001.08361 .

Sai Praneeth Karimireddy, Wenshuo Guo, and Michael Jordan. Mechanisms that incentivize data sharing in federated learning. In Workshop on Federated Learning: Recent Advances and New Challenges (in Conjunction with NeurIPS 2022) , 2022. URL https://openreview.net/forum? id=Bx4Sz-N5K3J .

Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report TR2009, University of Toronto, Toronto , 2009. URL https://www.cs.toronto.edu/~kriz/ learning-features-2009-TR.pdf .

Kenneth C Laudon. Markets and privacy. Communications of the ACM , 39(9):92-104, 1996.

Federated Learning. Collaborative machine learning without centralized training data. Publication date: Thursday, April , 6, 2017.

Yann LeCun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist , 2, 2010.

Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning. In ICLR . OpenReview.net, 2020.

Xiao-Bai Li and Srinivasan Raghunathan. Pricing and disseminating customer data with privacy awareness. Decision support systems , 59:63-73, 2014.

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics (AISTATS) , 2017.

Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning, second edition . Adaptive Computation and Machine Learning series. MIT Press, 2018. ISBN 9780262039406. URL https://books.google.com/books?id=V2B9DwAAQBAJ .

Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In International Conference on Machine Learning , pages 4615-4625. PMLR, 2019.

Hervé Moulin. Fair Division and Collective Welfare . The MIT Press, 01 2003. ISBN 9780262280297. doi: 10.7551/mitpress/2954.001.0001. URL https://doi.org/10.7551/mitpress/2954. 001.0001 .

Roger B Myerson. Optimal auction design. Mathematics of operations research , 6(1):58-73, 1981.

John Nash. Non-cooperative games. Ann. Math. , 54(2):286-295, 1951.

N Nvidia. Nvidia clara federated learning to deliver ai to hospitals while protecting patient data. NVIDIA Clara Federated Learning to Deliver AI to Hospitals While Protecting Patient Data , 2019.

B. Rosen. Existence and uniqueness of equilibrium points for concave n-person games. Econometrica , 33(3):520-534, 1965. ISSN 00129682, 14680262. URL http://www.jstor.org/ stable/1911749 .

Paul A. Samuelson. The pure theory of public expenditure. The Review of Economics and Statistics , 36(4):387-389, 1954. ISSN 00346535, 15309142. URL http://www.jstor.org/stable/ 1925895 .

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

Hal R Varian. Equity, envy, and efficiency. Journal of Economic Theory , 9(1):63 - 91, 1974.

Jie Xu, Benjamin S Glicksberg, Chang Su, Peter Walker, Jiang Bian, and Fei Wang. Federated learning for healthcare informatics. Journal of Healthcare Informatics Research , 5(1):1-19, 2021.

Yuze Zou, Shaohan Feng, Dusit Niyato, Yutao Jiao, Shimin Gong, and Wenqing Cheng. Mobile device training strategies in federated learning: An evolutionary game approach. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) , pages 874-879. IEEE, 2019.

A Appendix to Section 3

We first show that a Nash equilibrium exists when agent payoff functions are separable , i.e., for every agent i there are functions g i : S i → R ≥ 0 and h i : × j = i S j → R ≥ 0 s.t. for all s ∈ S , a i ( s ) = g i ( s i ) + h i ( s -i ) .

̸

Theorem A.1. In any federated learning problem where agent payoff functions are separable, a Nash equilibrium exists.

Proof. When the payoff function of an agent i is separable, the best response to any contribution vector s -i is independent of s -i :

̸

Let F i := arg max x ∈ S i g i ( x ) -c i ( x ) . Clearly F i = ∅ since S i = ∅ . Then any s ∈ × i F i satisfies s ∈ f ( s ) by definition. By Proposition 1 any such sample vector is a Nash equilibrium.

̸

Next, we present a negative result showing that there are federated learning settings where a Nash equilibrium is not guaranteed to exist.

Theorem A.2. There exists a federated learning problem in which a Nash equilibrium does not exist. Moreover, the instance has three agents with continuous, non-decreasing, non-concave payoff functions and linear cost functions.

Proof. Let ε ∈ (0 , 1 16 ) . Let e : [0 , 1] → [0 , 1] be a function given by:

Essentially the function e is a continuous, piece-wise linear function connecting (0 , 0) , ( 1 2 -ε, 0) , ( 1 2 + ε, 1) and (1 , 1) .

Now consider the following federated learning instance with n = 3 agents, where S 1 = S 2 = S 3 = [0 , 1] . The payoff functions are given by:

and the cost functions are c i ( s i ) = 1 4 s i for all i ∈ [3] . Notice that the payoff functions are increasing in s j for every j ∈ [3] and are continuous since e is continuous.

We now show that this instance does not admit a Nash equilibrium. Let us first evaluate the best response set f 1 ( s 2 , s 3 ) . Note that u 1 ( s ) = e ( s 1 ) · (1 -e ( s 3 )) + e ( s 3 ) -1 4 s 1 . Since u 1 ( s ) is independent of s 2 , f 1 ( s 2 , s 3 ) only depends on s 3 .

Case 1. s 3 ≤ 1 2 -ε . Then u 1 ( s ) = e ( s 1 ) -1 4 s 1 , which is maximized at s 1 = 1 2 + ε and results in a utility of 7 8 -ε 4 .

Case 2. s 3 ≥ 1 2 + ε . Then u 1 ( s ) = 1 -1 4 s 1 , which is maximized at s 1 = 0 and results in a utility of 1 .

Case 3. 1 2 -ε ≤ s 3 ≤ 1 2 + ε . We consider the intervals in which the best response s 1 to such an s 3 can lie:

-s 1 ≤ 1 2 -ε . In this range, u 1 ( s ) = e ( s 3 ) -1 4 s 1 , which is maximized at s 1 = 0 and results in a utility of e ( s 3 ) .

-s 1 ≥ 1 2 + ε . In this range, u 1 ( s ) = 1 -1 4 s 1 , which is maximized at s 1 = 1 2 + ε and results in a utility of 7 8 -ε 4 .

-1 2 -ε ≤ s 1 ≤ 1 2 + ε . In this range, using the definition of e ( s 1 ) (eq. 8) we obtain:

Thus u 1 ( s ) is a linear function in s 1 with slope 1 -e ( s 3 ) 2 ε -1 4 . If the slope is positive, then the best response in the current interval is s 1 = 1 2 + ε , and gives a utility of 7 8 -ε 4 . If the slope is negative, then s 1 = 1 2 -ε is the best response in the current interval and gives a utility of e ( s 3 ) -1 4 ( 1 2 -ε ) . However s 1 = 0 gives a utility of e ( s 3 ) implying that s 1 = 1 2 -ε cannot be a best response. Finally if the slope is zero, then it must mean that e ( s 3 ) = 1 -ε 2 , and the utility is ε 2 ( 1 2 -1 4 ε ) + 1 -ε 2 = 7 8 -ε 4 . However responding with s 1 = 0 gives a utility of e ( s 3 ) = 1 -ε 2 , which exceeds 7 8 -ε 4 , since ε < 1 16 . Thus, the best response does not lie in ( 1 2 -ε, 1 2 + ε ) and s 1 = 0 is the overall best response.

The above discussion shows that the best response f 1 ( s 2 , s 3 ) ⊆ { 0 , 1 2 + ε } . By symmetry, the same holds for f 2 and f 3 . Suppose there exists a Nash equilibrium s ∗ = ( s ∗ 1 , s ∗ 2 , s ∗ 3 ) . By Proposition 1, s ∗ ∈ f ( s ∗ ) . Since the above discussion implies s ∗ 3 ∈ { 0 , 1 2 + ε } , we consider two cases:

Suppose s ∗ 3 = 0 . Then

which is a contradiction.

Suppose s ∗ 3 = 1 2 + ε . Then

which is also a contradiction.

This shows that there is no s ∗ such that s ∗ ∈ f ( s ∗ ) , implying that the above instance does not admit a Nash equilibrium.

We now prove the fast convergence of best response dynamics.

Theorem 3.2. Let G ( s ) be the Jacobian of u : S → R n , i.e., G ( s ) ij = ∂ 2 u i ( s ) ∂s j ∂s i . Assuming agent utility functions u i satisfy

Strong concavity: ( G + λ · I n × n ) is negative semi-definite,

Bounded derivatives: | G ij | ≤ L ,

for constants λ, L > 0 , the best response dynamics (4) with step size δ t = λ n 2 L 2 converges to an approximate Nash equilibrium s T where ∥ g ( s T , µ T ) ∥ 2 < ε in T iterations, where

Proof. Observe that µ t is chosen s.t. ∥ g ( s t , µ t ) ∥ 2 is minimized among all µ s.t. the updated sample vector s t +1 remains in S . Thus:

Using Taylor's expansion, we have:

where H ij ( s ′ , µ t ) = ∂g ( s ′ , µ t ) ∂s j , and s ′ = s t + α ( s t +1 -s t ) for some α ∈ [0 , 1] .

By definition, g ( s t , µ t ) i = ∂u i ( s t ) ∂s i + µ t i . Thus H ij ( s ′ , µ t ) = ∂ 2 u i ( s t ) ∂s j ∂s i = G ij ( s ′ ) , hence H ( s ′ , µ t ) = G ( s ′ ) . The BR dynamics update rule (4) implies s t +1 -s t = δ t · g ( s t , µ t ) . We therefore have g ( s t +1 , µ t ) = ( I n × n + δ t · G ( s ′ )) · g ( s t , µ t ) . Taking the L 2 norm, we get:

By the strong concavity assumption, for a constant λ > 0 , G + λ · I n × n is negative semi-definite, i.e., v T ( G + λ · I n × n ) v ≤ 0 for any v ∈ R n . With v = g ( s t , µ t ) , we have:

Next we use the fact that the L 2 norm ∥ A ∥ 2 of an n × n matrix A is bounded by its Frobenius norm ∥ A ∥ F :

̸

By the bounded derivatives assumption, we have | G ( s ′ ) ij | ≤ L , which implies that ∥ G ( s ′ ) ∥ F = √∑ i ∑ j L 2 = nL . This gives:

Using (12) and (13) in (11), we get:

Since δ t = λ n 2 L 2 , the above equation together with (10) gives:

Using (1 -x ) r ≤ e -xr repeatedly we obtain that:

Thus if we want the error ∥ g ( s t , µ t ) ∥ 2 ≤ ε , T = 2 n 2 L 2 λ 2 log ( ∥ g ( s 0 ,µ 0 ) ∥ 2 ε ) iterations suffice, as claimed.

B Appendix to Section 4

Lemma 1. The equation Cβ 2 -( An ( n -2) + C ) β + A ( n -1) 2 = 0 of (6) has a real root β ∗ where 0 ≤ β ∗ ≤ 1 -1 /n .

Proof. Using the quadratic formula, we see that β ∗ given by:

We first argue β ∗ is real, by showing ( An ( n -2) + C ) 2 -4 AC ( n -1) 2 ≥ 0 . This is equivalent to showing q ( y ) := ( y + n ( n -2)) 2 -4( n -1) 2 y ≥ 0 , where y = C/A . Expanding q , we have q ( y ) = y 2 -2( n 2 -2 n +2) y + n 2 ( n -2) 2 . The roots of q are:

i.e., y 1 = ( n -2) 2 and y 2 = n 2 . Since q ( y ) has a positive leading coefficient, we have that q ( y ) ≥ 0 for all y ≥ y 2 = n 2 . Thus it remains to show that y = C/A ≥ n 2 . To see this, we use the AM-HM inequality:

implying C/A ≥ n 2 as desired. This shows that the root β ∗ of equation (6) is real, hence well-defined.

We now show 0 ≤ β ∗ ≤ 1 -1 /n . From (14), we see:

Further, from (14) we also have:

where we used A/C ≤ 1 /n 2 (15) in the last inequality. This concludes the proof of Lemma 1.

Theorem 4.1. For each β ∈ [0 , 1] , the mechanism M β admits a Nash equilibrium. For β = β ∗ (Definition 2), whenever the NE s ∗ of M β ∗ satisfies s ∗ > 0 , the NE s ∗ maximizes the p -mean welfare among all vectors s > 0 , for any p ≤ 1 .

Proof. When 0 ≤ β ≤ 1 , the program (5) is a convex program for general convex cost functions. Since u i ( · ) is concave, a proof similar to the proof of Theorem 3.1 shows the existence of a Nash equilibrium.

We now show the welfare-maximizing property. For simplicity, we only consider feasible strategies where each agent participates in the mechanism, i.e., s i > 0 . Let ρ i and λ i as the dual variables to the first and second constraints respectively for each i , and let S = ∥ s ∥ 1 . Writing the KKT conditions and eliminating all ρ i , we get that a NE ( b ∗ , s ∗ ) together with dual variables λ ∗ satisfies:

∀

i

:

λ

∗

i

0

Now we turn to the p -mean welfare maximizing solution which is an optimal solution to the following program.

̸

The following lemma establishes that (19) is a convex program. For ease of readability we defer its proof to B.1.

Lemma 2. For β ∈ [0 , 1] and p ≤ 1 , the program (19) is convex.

We can now write the KKT conditions of program (19). By letting µ i and γ i denote the dual variables corresponding to the first and second constraints respectively for each i and S = ∥ s ∥ 1 , the KKT conditions (considering only solutions with s i > 0) are:

̸

Since KKT conditions are sufficient for optimality, to prove Theorem 4.1 it suffices to show that for an NE ( b ∗ , s ∗ ) , there exist dual variables µ ∗ and γ ∗ which satisfy (20)-(23) for β = β ∗ .

̸

Let α := ( ∑ j u j ( b ∗ j , s ∗ ) p ) 1 /p -1 ∑ k u k ( b ∗ j , s ∗ ) p -1 ∂u k ( b ∗ k , s ∗ ) ∂S , i.e., the common value of the equality (20) at the NE ( b ∗ , s ∗ ) . The equation (20) then becomes α · c -1 i = µ i (1 -β ) + β n -1 ∑ k = i µ k . Summing these over all i and letting T = ∑ j µ j , we obtain:

̸

Putting this back in (20), we obtain the following expression for µ ∗ i , which can be computed from the NE ( b ∗ , s ∗ ) with T = α · ( ∑ i c -1 i ) :

Recall that the NE ( b ∗ , s ∗ ) satisfies (16)-(18) for some dual variables λ ∗ . We define γ ∗ i as follows:

The next lemma proves Theorem 4.1.

Lemma 3. A NE ( b ∗ , s ∗ ) with µ ∗ and γ ∗ defined by (24) and (25) satisfy the KKT conditions (20) -(23) of program (19) .

Proof. First observe that at the NE, (1 -β ) c i · ( ∂u i ( b ∗ i ,S ∗ ) ∂b i + λ ∗ i ) = ∂u i ( b ∗ i ,S ∗ ) ∂S > 0 by assumption.

Since β ∈ (0 , 1) and c i > 0 , we have ∂u i ( b ∗ i ,S ∗ ) ∂b i + λ ∗ i > 0 . Together with λ ∗ i ≥ 0 (17), this shows γ ∗ i ≥ 0 thus satisfying dual feasibility (22).

Next we show complimentary slackness (23) holds. For any i , λ ∗ i · b i = 0 due to (18). Then by the definition of γ ∗ i , we have γ ∗ i · b i = 0 for all i .

Finally, we show that equations (20) and (21) are satisfied for a specific choice of β = β ∗ . Together, (20) and (21) imply that an optimal solution to program (19) satisfies:

̸

The choice of γ ∗ i from equation 25 implies that µ ∗ i -γ ∗ i = µ ∗ i · ( ∂u i ( b ∗ i , s ) /∂b i ∂u i ( b ∗ i , s ) /∂b i + λ ∗ i ) . Moreover at the NE, equation (16) implies that:

Using the above in (26), it only remains to be argued that µ ∗ , b ∗ and s ∗ satisfy:

̸

for β = β ∗ . By plugging in the value of µ ∗ i from (24) and using α = T · ( ∑ k c -1 k ) -1 , we get:

Let us define A := ( ∑ i c -1 i ) -1 and C := ∑ i c i . Manipulating the above expression, the above equation then becomes:

which is true for β = β ∗ since it is exactly the definition of β ∗ (Definition 2).

Thus for β = β ∗ , the NE ( b ∗ , s ∗ ) with dual variables µ and γ as defined in (24) and (25) respectively satisfy the KKT conditions of program (19).

B.1 Proof of Lemma 2

Lemma 2. For β ∈ [0 , 1] and p ≤ 1 , the program (19) is convex.

Proof. For β ∈ [0 , 1] the constraints of program 19 are convex since c i ( · ) are convex functions. It remains to be shown that the objective W p ( b , s ) := ( ∑ i u i ( b i , ∥ s ∥ 1 ) p ) 1 /p to be maximized is concave.

We use the following standard fact about the concavity of composition of functions (see e.g. Boyd and Vandenberghe [2004], Page 86).

Proposition 2. Let h : R n → R and g i : R k → R and let f : R n → R be given by f ( x ) = h ( g ( x )) = h ( g 1 ( x ) , . . . , g n ( x )) . Then f is concave if h is concave, h is non-decreasing in each argument and g i are concave.

i p i We now observe that:

h is non-decreasing in each argument. This is because:

h is concave. Using the above, we can compute the Hessian H given by:

̸

Thus for any v ∈ R n , we have:

̸

Algorithm 1 FedBR-BG

Input: Number of iterations in game H , number of iterations of gradient descent T , learning rate α , step size δ , data increasing interval ∆ s

since p ≤ 1 , h ≥ 0 , and by the Cauchy-Schwarz inequality ( ∑ i a i · b i ) 2 ≤ ( ∑ i a 2 i ) · ( ∑ i b 2 i ) with a i = v i x p/ 2 -1 i and b i = x p/ 2 -1 i . Thus H is negative semi-definite and hence h is concave.

For each i , g i ( b , s ) = u i ( b , s ) is concave.

Using Proposition 2 and the fact that W p ( b , s ) = h ( g ( b , s )) we conclude that W p ( b , s ) is concave.

C Distributed Algorithms

In this section, we present the distributed algorithms of our two mechanisms, FedBR and FedBR-BG.

Algorithm 2 FedBR

Input: Number of iterations in game H , number of iterations of gradient descent T , learning rate α , step size δ , data increasing interval ∆ s

Output: Model weights θ T , individual contributions s

Server sends

θ

t

to agents;

i computes ∇ θ t L i ( θ t ) on its local dataset D i ;

i

sends

t

to server;

end for

Server aggregates the gradients following

Server updates θ t +1 following

end for

∇

θ

t

L

i

(

θ

)

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper shows that when every agent has a known concave utility function, the Nash equilibrium exists and can be reached independently via best response updates. Moreover, when all agents have linear costs, the proposed budget mechanism (which modifies the utility function) will lead to Nash equilibrium that maximizes p-mean welfare and fairness across agents.\n\nSTRENGTHS:\n1. The differences with existing works are clearly highlighted. The paper novelly considers (i) the unconstrained setting where agents maximize their concave net utility functions, (ii) optimizing the p-mean welfare of the Nash equilibrium and (iii) the derivation of $\\beta^*$.\n2. Most of the claims and assumptions of the paper (e.g., concave payoff function, convex cost functions) are well justified.\n3. The paper is well-written and easy to follow. The implications described after the theorems aid understanding.\n\nWEAKNESSES:\n1. The paper makes two assumptions: a) the cost and payoff functions are common knowledge, and b) any agent utility (payoff and cost) only depends on the number of samples contributed. These assumptions limit the applicability/significance of the work and should be better justified. \n\nFor a), Line 317 proposes that the server can train a public accuracy function and broadcast it to all agents. However, in many FL applications, the server may lack data to do so, and clients may be unwilling to share due to the cost incurred. Moreover, the agents may disagree on the relative weightage of the cost/utility function (e.g., how much monetary cost should one unit of payoff function improvement be worth?) \n\nFor b), different subsets will lead to models with different performance and costs (e.g., lower if an agent only clones data or one agent may have noisier data). Further justifications (e.g., a trusted data sharing platform ensuring that agent’s i. (or i and j) data is sampled i.i.d) would help. \n\n2. In Sec 4, the definition of $u_i$ is unclear (re-defined on Line 260; 297) and is not explicitly expressed as a function of $p_i$ and $\\beta^*$. The descriptions can be improved. My interpretation is that $p_i$ acts as a cost subsidy; hence $u_i = a_i - (c_i - p_i)$ and $(b_i + c_i - p_i = B_i)$. From Theorem 4.1 onwards, $p_i$ uses $\\beta^*$ instead of any $\\beta$.\n\nQUESTIONS:\n1. Theorem 1 states that the NE using $\\beta^*$ will maximize the $p$-mean welfare for any $p ≤ 1$. Does that require the maximizer of the p-mean welfare for every $p ≤ 1$ to be the same? If so, why and how?\n\nOne potential counterexample with 2 agents is: out of the utility sample vectors (2,2) and (1,4), (2,2) has higher egalitarian welfare while (1,4) have higher utilitarian welfare.\n\n\nMinor suggestions (not affecting scores) \n* Line 217: explain that $||g(.)||_2 = 0$ at a Nash equilibrium\n* Line 255: $c_i$ is used as both the cost function and the per unit cost. The latter should use another notation for less ambiguity.\n* Equation after Line 264: add brackets for the summation\n* Line 271: spelling ‘defferred’\n* Line 289 can be clearer: the maximization problem can be converted to a convex minimization problem.\n\nLIMITATIONS:\nThe conclusion briefly identifies the two limitations: 1) the assumption that cost and payoff functions are common knowledge and 2) agents may not report truthfully. The limitations (and weakness 1) can be further elaborated on in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a collaborative Federated Learning framework. Specifically, the authors proposed a utility model, which is the agent’s payoff function minus the data sharing cost function. Under the assumption that the payoff function is concave and the cost function is convex, the authors show the existence of NE. In addition, they proposed a budget-balanced mechanism, which involves payments to the agents, to maximize the agents’ utilities at NE. Experimental results are provided to demonstrate the proposed algorithm's performance, compared with the previous work FedAvg by Blum et al. [2021].\n\nSTRENGTHS:\nOverall, I found the paper to be well-written, and if you take the model as given, the paper gives a fairly satisfying and complete first investigation – the questions asked are exactly the ones I would hope are answered first.\n\nWEAKNESSES:\nOne major weakness of the paper lies in the utility model proposed. While the paper's key idea of explicitly defining the data-sharing cost is illuminative and contributes to addressing the existence question prevalent in the literature, the utility model itself raises concerns. The critical assumption made in the model is that an agent cannot achieve better utility on its own, even when the cost is zero. Additionally, the assumption suggests that collaboration with the crowd leads to a better utility at the equilibrium state, as opposed to the option of an agent choosing to quit after some rounds of collaboration.\n\nThis assumption poses limitations and potentially oversimplifies the complexities of real-world scenarios. It overlooks the possibility that an agent might achieve better utility by pursuing an independent course of action, disregarding the collaborative approach altogether. By not considering the potential benefits an agent could gain from individual efforts, the utility model fails to capture the nuanced dynamics of decision-making and potential trade-offs that exist in real-world situations. Therefore, this utility model undermines the paper's overall conclusions and hinders its practical applicability. I would have liked a more concrete/precise discussion of the motivation for the utility model, for example, what if the cost is too high and the agent prefers to quit the collaboration?\n\nQUESTIONS:\nCan you provide more justification for why the agents always prefer to participate in the collaboration, even if the data-sharing cost is zero when the agent quits the collaboration?\n\nLIMITATIONS:\nN/A.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a simple and elegant mechanism for incentivizing data sharing in FL. The mechanism is budget balanced, and any p-mean welfare is maximized at Nash equilibrium. We show the existence of Nash equilibrium (NE) under mild assumptions on agents' payoff and costs. They also show that agents can discover the NE via best response dynamics. In addition, they introduce a FL protocol FedBR-BG based on the budget-balanced mechanism, utilizing best response dynamics. They empirically validate that FedBR-BG outperforms the basic best-response-based protocol without additional incentivization, as well as the standard federated learning protocol FedAvg, in terms of achieving p-mean welfare.\n\nSTRENGTHS:\nThe paper introduces public-good-provision approaches to the problem of incentivizing data sharing in FL. They propose a simple and elegant mechanism that achieves desirable properties (based on Falkinger et. al, 2000). The theoretical results are elegant and solid. The presentation is crystal clear.\n\nThe results do not seem to rely on any specific feature of FL (not sure whether this is an advantage or disadvantage). There is one result that seems novel and surprising, which is the characterization of the optimal \\beta^* (Definition 2) and the fact that it optimizes all p-mean welfare. The proof for the optimality of \\beta^* is non-trivial. My evaluation will highly depend on the novelty of this specific result.\n\nWEAKNESSES:\nThe mechanism seems to be largely based on the previous literature on the public good provision, especially (Falkinger et. al 2000). The mechanism is basically the same as the one in (Falkinger et. al 2000).\n\nThe comparison to the existing literature is extremely inadequate. (I didn't see a related work section and the references in the intro barely cover the large literature.) There is a large literature on incentive mechanisms for FL, see [1] for example.\n\n[1] Tu et al., 2021, \"Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspective\"\n\nQUESTIONS:\nIs the derivation of \\beta^* entirely novel, or does it build upon prior work? If it is a novel derivation, why not frame the paper as a study on public good provision? The outcomes seem to have broader implications beyond FL.\n\nDoes \\beta^* depend on the value of p? Falkinger et. al (2000) seem to suggest that \\beta = 1-1/n is optimal for p=1. Does that mean \\beta=1-1/n is optimal for all p<= 1?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

comments: SUMMARY:
The authors study the Nash equilibrium in federated learning, under the assumption of concave utility and convex cost, and design welfare maximizing payment for users. Experiments are conducted to verify the mechanism.

STRENGTHS:
The topic is of important value, and the paper is well written.

WEAKNESSES:
My major concern is technical novelty and contribution.

1. The analysis of equilibrium and best response dynamic is standard, especially under the assumptions such as convex cost and concave utilities.

2. Though the authors discuss the definition of fairness and welfare maximization, there is lack of discussion on literature about welfare maximization in federated learning, let alone fair subsidies in mechanism design. 

3. The design of the payment seems to require the knowledge of the true cost of every user, which is not practical and incentive comptible.

4. The performance of fairness (in Table 1) is compared with methods that are not designed to maximize welfare, which is not meaningful.

QUESTIONS:
1. More details on the payment (knowledge required, performance comparison with state-of-the-art mechanism) would help.

LIMITATIONS:
Not applicable.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-12