Title: Incentivizing Honesty among Competitors in Collaborative Learning and Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Game Theory

Federated Learning

Optimization

Strategic Behavior

Economics

Mechanisms
Abstract: Collaborative learning techniques have the potential to enable training machine learning models that are superior to models trained on a single entity’s data. However, in many cases, potential participants in such collaborative schemes are competitors on a downstream task, such as firms that each aim to attract customers by providing the best recommendations. This can incentivize dishonest updates that damage other participants' models, potentially undermining the benefits of collaboration. In this work, we formulate a game that models such interactions and study two learning tasks within this framework: single-round mean estimation and multi-round SGD on strongly-convex objectives. For a natural class of player actions, we show that rational clients are incentivized to strongly manipulate their updates, preventing learning. We then propose mechanisms that incentivize honest communication and ensure learning quality comparable to full cooperation. Lastly, we empirically demonstrate the effectiveness of our incentive scheme on a standard non-convex federated learning benchmark. Our work shows that explicitly modeling the incentives and actions of dishonest clients, rather than assuming them malicious, can enable strong robustness guarantees for collaborative learning.

Incentivizing Honesty among Competitors in Collaborative Learning and Optimization

Abstract

Collaborative learning techniques have the potential to enable training machine learning models that are superior to models trained on a single entity's data. However, in many cases, potential participants in such collaborative schemes are competitors on a downstream task, such as firms that each aim to attract customers by providing the best recommendations. This can incentivize dishonest updates that damage other participants' models, potentially undermining the benefits of collaboration. In this work, we formulate a game that models such interactions and study two learning tasks within this framework: single-round mean estimation and multi-round SGD on strongly-convex objectives. For a natural class of player actions, we show that rational clients are incentivized to strongly manipulate their updates, preventing learning. We then propose mechanisms that incentivize honest communication and ensure learning quality comparable to full cooperation. Lastly, we empirically demonstrate the effectiveness of our incentive scheme on a standard non-convex federated learning benchmark. Our work shows that explicitly modeling the incentives and actions of dishonest clients, rather than assuming them malicious, can enable strong robustness guarantees for collaborative learning.

1 Introduction

Recent years have seen an increased interest in designing methods for collaborative learning, where multiple participants contribute data and train a model jointly. The premise is that the participants will then be able to obtain a better model than if they were learning in isolation. Most prominently, federated learning (FL) (Kairouz et al., 2021) provides a method for training models in a distributed manner, allowing data to stay with institutions, while still harvesting the benefits of collaboration.

An underlying premise for the success of collaborative learning schemes is that the participants contribute data (or gradient updates) relevant to the learning task at hand. However, when participants are in competition on some downstream task, they may have an incentive to sabotage other participants' models. For instance, firms that are competing on the same market can often improve their machine learning models by having access to their competitors's data, but at the same time will likely benefit from a gap between the quality of their models and those of other firms.

These conflicting incentives raise a concern that collaborative learning may be vulnerable to strategic updates from participants. Previous work has empirically demonstrated that irrelevant or malicious updates can negatively impact collaborative learning (Tolpegin et al., 2020; Kairouz et al., 2021). In

Nikola Konstantinov

Martin Vechev

particular, if a subset of participants is modeled as fully-malicious (Byzantine) agents, that collude in a worst-case manner, it is known that optimal convergence rates contain a leading-order term that is based on the fraction of Byzantine agents and is irreducible as the number of players increases (Yin et al., 2018; Alistarh et al., 2018). This suggests that collaborative learning in the presence of strategic behavior may often not provide asymptotic benefits over learning with one's own data.

Contributions In this work, we study collaborative learning in the presence of strategic behavior by explicitly modeling players' competitive incentives. We consider a game between multiple players that exchange updates via a central server, where players' rewards increase both when they obtain a good model for themselves, and when other players' models perform poorly.

We study two important instantiations of this collaborative learning game: mean estimation with a very general action space and strongly-convex stochastic optimization with attacks that add gradient noise. We show that players are often incentivized to strongly manipulate their estimates, rendering collaborative learning useless. To remedy this, we suggest mechanisms inspired by peer prediction (Miller et al., 2005), that penalize cheating players using side payments. Our results on stochastic optimization rely on a novel recursive bound for the squared norm of differences in SGD-iterates between a clean trajectory and a strategically corrupted trajectory. Meanwhile, we show that side payments can be avoided in the mean estimation case, using a novel communication protocol in which the server sends noisy estimates back to players that are suspected of cheating. Our mechanisms are solely based on observable player behaviour, and recover near-optimal convergence rates at equilbrium. Furthermore, expected payments cancel out when all players are honest, so that players are incentivized to participate in the training, rather than use their own data only, despite the penalties.

Finally, we conduct experiments on the FeMNIST and Twitter datasets from LEAF (Caldas et al., 2018) and demonstrate that our mechanisms can incentivize honesty for realistic non-convex problems

2 Related work

Game theory and collaborative learning Many works that study connections between FL and game theory focus on clients' incentives to participate in the training process at all (see Tu et al. (2021) for a recent survey, and Gradwohl & Tennenholtz (2022) for an analysis of how this relates to competition). Similarly, Karimireddy et al. (2022) study incentives for free-riding, i.e. joining collaborative learning without spending resources to contribute data. In contrast, our setting covers clients that strategically manipulate their updates in order to damage other participants' models.

Another line of work studies FL as a coalitional game theory problem, in which the players need to deal with potential issues of between-client heterogeneity by deciding with whom to collaborate Donahue & Kleinberg (2021a,b). Optimal behavior under data heterogeneity is also studied by Chayti et al. (2021), while Gupta et al. (2022) studies invariant risk minimization games in which FL participants learn predictors invariant across client distributions. In contrast, we study FL as a non-cooperative game and seek to address strategic manipulation, rather than data heterogeneity.

Robustness in federated learning The robustness of FL to noisy or corrupted updates from clients has received substantial recent interest, see Shejwalkar et al. (2022) for a recent overview. One line of work studies federated learning in the context of various data corruption models, e.g. noise or bias towards protected groups (Fang & Ye, 2022; Abay et al., 2020). In contrast, we study strategic manipulation by clients as the source of data corruption.

Clients attacking the training are typically modeled as Byzantine (Blanchard et al., 2017; Yin et al., 2018; Alistarh et al., 2018), adversarially seeking to sabotage training by deviating from the FL protocol in a worst-case manner. The goal of Byzantine-robust learning is to achieve guarantees in that setting. Similar models have been studied in a statistical context, where data is stored at a single location and so communication is not a concern (Qiao & Valiant, 2018; Konstantinov et al., 2020). In contrast to these works, we model manipulation as a consequence of competitive incentives rather than maliciousness and analyze client behaviour using game theory (Osborne & Rubinstein, 1994). Rather than focussing on robustness to manipulations, we aim to prevent manipulation alltogether.

Peer prediction mechanisms Our mechanisms for inducing honesty are closely related to peer prediction that aims to incentivize honest ratings on online platforms. In their seminal paper,

Miller et al. (2005) suggest paying raters based on how much their rating helps to predict other raters' ratings. While they require a common prior shared by all raters and the mechanism designer, Witkowski & Parkes (2012) extend their results relaxing that assumption. Closely related to our work, Cai et al. (2015) suggest to incentivize crowdworkers to produce accurate labels by paying them more, the better their label gets predicted by a model estimate from other workers' data. Meanwhile, Waggoner & Chen (2014) prove that peer prediction elicits common knowledge, rather than truth from participants. Lastly, while Karger et al. (2021) find that peer prediction can elicit subjective forecasts of similar accuracy as scoring based on the ground truth, Gao et al. (2014) demonstrate that human raters can end up with dishonest strategies despite the existence of honest equilibria.

3 Competitive federated learning

In this section, we present our framework on a high level and explain how it models competitive behaviour in FL. In this generality, however, it is impossible to analyze the problem quantitatively. We therefore define specific instantiations of the framework, which we study for the rest of the paper.

3.1 General framework

Overview Throughout the paper, we assume that there are N ≥ 2 players, who each have a private dataset. The players participate in an FL-like procedure, which takes place over multiple rounds and requires them to send messages with information relevant to update a centrally trained model at every step. In our setup, players act strategically and competitive pressures might incentivize them to try to corrupt other players' models. This is done by manipulating updates sent to the central model, while simultaneously keeping track of an unmanipulated private (presumably more accurate) model.

To model the participants' strategic interactions, we need to define a game by specifying their action spaces and rewards. To this end, one needs to specify: their attack strategies , that describe whether and how they will corrupt their messages to the server; their defense strategies , which describes how they postprocess the server's updates to defend themselves against others' manipulations; and their rewards , in a way that reflects the quality of learning and the competition between them. Given these components, we are interested in the Nash equilibria of the corresponding game, in order to understand strategic behavior in FL and how it affects the quality of the players' models.

Formal setup We denote the samples of each player i by x i = { x i 1 , . . . , x i n } , where x ∈ X , and assume that { x i j : i ∈ [ N ] , j ∈ [ n ] } are not necessarily independent samples from an unknown distribution D ∈ P ( X ) . For simplicity, we assume that all players have an equal number of samples n . Players communicate via an FL-like protocol, through which a central server model θ s ∈ R d is updated. The intended goal of this procedure is to find a value for θ s that minimizes a loss function f D ( θ ) . Note that because D is unknown to the players, they can benefit from honest collaboration.

The protocol consists of T rounds and the central model is initialized at some θ s 1 ∈ R d . At time t = 1 , 2 , . . . , T the server sends the model θ s t to all participants. Each agent i is then meant to send an update g t ( θ s t , x i ) to the server, for some function g t : R d ×X n → R d . For example, in the standard FedSGD setting, f D ( θ ) = E x ∈D [ f x ( θ )] for functions f x : R d → R and g = 1 n ∑ j ∇ f x i j ( θ s t ) serves as an estimate of the gradient of f evaluated at the data of player i . Each player i then sends a message m i t ∈ R d to the server (which may or may not be equal to g t ( θ s t , x i ) ). Finally, the server computes a new model θ s t +1 = Agg ( θ i t , m 1 t , m 2 t , . . . , m N t ) , via an aggregating function Agg : R d × R N × d → R d (for example, in FedSGD this will be a gradient update computed as ¯ m t = 1 N ∑ N i =1 m i t ).

Players' strategies At every step, the players take two decisions: how to attack by sending a manipulated estimate to the server, and how to defend themselves from unreliable estimates when updating their locally tracked model based on information received from the server. Formally, we assume that each player i ∈ [ N ] chooses (potentially randomized) functions a i 1 , a i 2 , . . . , a i T and d i 1 , . . . , d i T , that describe their behavior for the attack and defense stages at every time step. These functions are chosen from two respective sets of possible actions A and D . The tuple of chosen actions p i = ( a i 1 , . . . , a i T , d i 1 , . . . , d i T ) ∈ A T × D T represents the player's global strategy.

In the most general case, the attacks and defenses of each player i may take into account all information available to the player, throughout the history of the optimization process. At time t this

include the models θ s 1 , . . . , θ s t received by the server; the local models θ i 1 , . . . , θ i t the player kept at previous iterations; the attack strategies a i 1 , . . . , a i t used up to time t (e.g. to correct for one's own faulty estimate at time t ); as well as additional randomness ξ i 1 , . . . , ξ i t sampled at each round.

Players' rewards Each player aims to obtain a final model θ i T +1 that approximately minimizes f D ( θ i T +1 ) . Crucially, their reward also depends on other players' models. Specifically, we assume that each player i has a reward function R i : R N × d ×P ( X ) → R and receives the reward

Note that the messages m i sent by players and thus θ i T +1 and each player's reward depend not only on players' strategies, but also on the particular realization of their samples x i . Wethus focus on expected rewards , averaging out the effects of particular realizations of players' samples and randomness in their strategies. We study the vector of expected rewards E ( r 1 , . . . , r N ) and its dependence on the strategy profile , that is on the distribution of strategies p = ( p 1 , p 2 , . . . , p N ) chosen by each player.

Assumptions on players' behaviour To analyze players' behaviour, we make two assumptions, giving rise to a classic game-theoretic setup. The first is that players seek to maximize their expected reward, as defined above, i.e. players are rational . In addition, since the reward of each player depends on the actions of the other players, players account for the actions of the others, which means that their behavior is strategic . A natural solution concept in this context is the Nash equilibrium (Nash, 1951). This describes a strategy profile in which no player can improve their reward by unilaterally changing their strategy. In our case, this classic notion translates to the following definition.

Definition 1. Let p = ( p 1 , p 2 , . . . , p N ) ∈ ( P ( A T × D T ) ) N be a strategy profile. Then p is a (mixed) Nash equilibrium if:

where the expectation is taken with respect to the randomness of the data and the players' strategies.

3.2 Specific instantiations: mean estimation and stochastic gradient descent

We study two specific cases of the game, each modeling a fundamental learning problem. The first is single-round mean estimation (Sections 4 and 5), which correspond to the general setup with T = 1 , f D ( θ ) = ‖ θ -µ ‖ 2 , where µ = E X ∼D ( X ) , and the updates g being the sample means of the players. The second is multi-round stochastic optimization of strongly-convex functions f D ( θ ) via SGD (Section 6), in which case the updates g are stochastic gradient estimates based on players' data. In the corresponding sections, we define natural rewards that model the competing incentives.

Strategy spaces To describe the attack strategies , in both cases we model attacks that send a noisy update g t ( θ s t , x i ) + α i t ξ i t , for normalized zero-mean noise ξ i t and an attack parameter α i t ∈ R , to the server. Up to a certain magnitude of α i t , these attacks have a natural interpretation as the act of hiding a random subset of a player's data . In addition, the α i t parameters have a natural interpretation as the of aggressiveness of the player . In the case of mean estimation, we are also able to analyze much more general attack strategies that can adjust α i t based on the players' samples x i and additionally allow for adding a directed bias to the communicated messages.

For the defense strategies in the mean estimation case, we consider a defense strategy that corrects the mean estimate received from the server for the player's own manipulation. The player then computes a weighted average of the result and the their local mean. The weighting parameter β i used then has a natural interpretation as the cautiousness of the player . In the SGD case we directly provide mechanisms that incentivize honesty at the attack stage, making potential defenses redundant.

4 A single-round version of the game: competitive mean estimation

In this section we analyze a single-round version of the game, in which players aim to estimate the mean of a random variable X ∈ P ( R d ) . Specifically, we consider the game defined in Section 3, in the case of T = 1 rounds. Players sample from a distribution D ∈ P ( R d ) by first independently sampling a random mean µ i ∼ D µ and "variance" σ 2 i := E ‖ X i -µ i ‖ 2 ∼ D σ (this models potential

heterogeneity between clients), and then receiving (conditionally) independent samples from a random variable X i with mean µ i and "variance" σ 2 i . We call µ = E µ i , σ 2 = E σ 2 i and σ 2 glyph[star] = E ‖ µ -µ i ‖ 2 . We assume that players do not know the distributions D µ and D σ .

Each player wants to estimate the global mean µ as well as possible. During the single communication round, the players are meant to communicate the mean of their samples: g 1 ( θ s 1 , x i ) = 1 n ∑ n j =1 x i j . Instead, they send messages m i 1 . The server aggregates the received messages by averaging them, so that θ s 2 = 1 N ∑ N i =1 m i 1 . The players then receive the value of θ s 2 from the server and use their defense strategy to arrive at a final estimate θ i 2 = d ( θ s 2 , x i ) of the mean. For simplicity of notation, we ignore the dependence of all values on the time t = 1 in the rest of this and the next section.

Reward functions To model competitive incentives, the reward of each player needs to increase as their own estimate of the mean becomes better, and as the estimates of other players become worse. Therefore, a natural reward function is:

glyph[negationslash]

for some λ i ≥ 0 . The value of λ i quantifies to what extent player i prioritizes the quality of their own estimate over damaging the estimates of the other players.

Attack strategies We assume that players choose what estimates to communicate by deciding how to perturb the empirical mean of their data. Specifically, each player i selects parameters α i ( x i ) ≥ 0 , b i ( x i ) ∈ R d based on their sample, in a potentially randomized manner, and communicates:

where ¯ x i = 1 n ∑ n j =1 x i j , E [ ξ i ] = 0 and E ‖ ξ i ‖ 2 = 1 . Here ¯ x i is the standard empirical mean of x i , while α i represents the magnitude of the noise player i adds to the estimate and b i ( x i ) is an additional bias term. Note that the case of b i ( x i ) = 0 recovers the data-hiding attack discussed in Section 3.2.

In order to prevent "non-general" strategies, such as simply setting m i = µ , that cannot be analyzed properly as their success depends on the true parameter µ , we assume that players do not base their strategies on guesses about µ beyond the information they obtained from ¯ x i . Formally, we assume

This prevents b i from linearly encoding additional knowledge about µ and for example holds whenever b j ( x j ) is independent of the residuals ¯ x j -µ . We also assume that the noise variables ξ i are independent of each other and all x k j and α k ( x k ) , but make no further distributional assumptions about ξ i . Indeed, all of our theorems will hold regardless of any additional assumptions about ξ i .

We denote this set of attack strategies as A . Each element in A can be uniquely identified via the distribution of the noise ξ i and the functions α i ( x i ) and b i ( x i ) . As α = b = 0 can be interpreted as covering the fully collaborative case, while α, b →∞ covers the fully malicious case, the (adaptive) parameters α, b have a natural interpretation as measures of the aggressiveness of a player.

We also note that these attacks are very general : m i ( x i ) -¯ x i can always be written as the sum of a determinstic component ˆ b ( x i ) and zero mean noise ˆ ξ ( x i ) , such that (4) and the fixed distribution of ξ i are the only assumptions separating us from the most general possible set of attacks strategies.

Defense strategies In the defense stage each player uses the received estimate θ s = ¯ m = 1 N ∑ N i =1 m i and their local data x i to compute a final estimate of the unknown mean. Two extreme approaches for player i are being fully cautious and using their local mean ¯ x i only, or fully trusting other players and computing the average of all sent updates, corrected for their own manipulation, that is ¯ m i = 1 N ( Nθ s -m i + ¯ x i ) . We consider defense strategies that take a weighted average of these two extremes: Each player i chooses a parameter β i ∈ [0 , 1] and constructs a final estimate

Denote the described set of defenses, as D . Each element in D is uniquely identified via the corresponding parameter β with β = 0 and β = 1 covering the extreme cases from above. Since β can be used to interpolate between these two extremes, it can be seen as a measure of cautiousness .

We do not cover more complicated defense strategies for two reasons: First, our proposed mechanisms will incentivize players to be honest even without any defenses such that more advanced defense mechanisms are not necessary. Second, as defenses can be seen as a method for mean estimation, analyzing the optimality for general classes of defenses would be fundamentally challenging for d ≥ 3 due to Stein's Paradox (Stein, 1956), even for a single player version of our game.

4.1 Expected rewards and Nash equilibria

We now analyze the game with strategy set ( A × D ) . As the specific distribution of ξ i does not affect the players' rewards, the attack and defense strategies are for all relevant purposes uniquely determined by the functions α , b , and β respectively. We abuse notation and consider ( α i , b i , β i ) as the strategy of player i . First we derive a formula for the MSE of a player, for a fixed strategy profile.

Theorem 4.1. Let D be as described above. Then the expected mean squared error (MSE) of player i ∈ [ N ] for any strategy profile (( α 1 , b i , β 1 ) , . . . , ( α N , b N , β N )) ∈ ( A × D ) N is:

glyph[negationslash]

glyph[negationslash]

This is proven in Appendix C similar to the bias-variance decomposition. This result allows us to analyze the expected rewards defined by equation (2) of the players for any strategy profile.

glyph[negationslash]

One can immediately see that there is no incentive for players to cooperate as long as β i < 1 : other players j can always increase their reward by increasing E ( a j ( x j ) 2 ) (unless it is already infinite). But for finite E ( a j ( x j ) 2 ) and E ‖ ∑ j = i b j ( x j ) ‖ 2 , the optimal β i can be shown to never equal one, such that equilibria are only possible "at infinity":

Corollary 4.2. The game defined by the reward in equation (2) and the set of strategies A × D does not have any (pure or mixed) Nash equilibrium for which E ( α j ( x j ) 2 ) and E ‖ b j ( x j ) ‖ 2 are finite for all players.

For details, see Appendix C. This shows that our defenses are unable to prevent maximal dishonesty by at least some players and formalizes a simple intuitive observation: as long as a player considers other players' updates at all, others are incentivized to reduce the information their updates convey about their samples. As at equilibrium at least one other player will infinitely distort the server estimate ¯ m , no player can benefit from collaborative learning without modifications to the protocol.

5 Mechanisms for incentivizing honesty

Given the impossibility of successful learning with rational competing agents in the simple mean estimation setting, we shift our focus to modifications of the protocol that allow for honest Nash equilibria (that is, equilibria where α j ( x j ) = b j ( x j ) = 0 , ∀ j ). To this end, we design two mechanisms that seek to penalize dishonest players proportionally to the magnitude of their manipulations (and, thus, the damage caused to other players). Note that this is complimentary to robust estimation methods (Diakonikolas et al., 2019) that can reduce but not eliminate the impact of manipulations.

The first relies on explicit side payments and requires transferable utility (that is, the existence of an outside resource R such as money, that is valued equally and on the same scale as the reward R by all players). The second is a modification of the FL protocol, in which the server adds noise to the estimates it sends to players that have sent suspicious updates. Importantly, for both mechanisms, the penalties can be computed by the server without the need for knowing α i , b i or other additional information beyond the players' updates, and without prior knowledge of the true distribution D .

5.1 Efficient solution for fully transferable utility

We first consider the case of transferable utility. In this case, we introduce a more general penalized reward for player i , which is given by

Here p i ( m 1 , . . . , m N ) denotes a penalty paid by player i to the server, measured in terms of the resource R and depending on the messages that the clients sent. As players value the reward and resource equally, they optimize for R i p instead of R i .

Inspired by peer prediction (Miller et al., 2005) we consider a penalty for player i proportional to the squared difference between that player's update and the average update sent by all players:

for some constant C ≥ 0 . This is a natural measure of the 'suspiciousness' of the client's update. In order to prevent excessive payments for honest players, we redistribute the penalties as glyph[negationslash]

This redistribution also makes it possible to implement our mechanism in a decentrally with messages sent publically, if players are able to credibly commit to the implied payments to other players.

Theorem 5.1 establishes that this penalty can incentivize full honesty for the right choice of C :

Theorem 5.1. In the setting of Theorem 4.1 for the penalized game with rewards glyph[negationslash]

the strategy profile α j = b j = β j = 0 for all j is a Nash equilibrium, whenever C > 1 ( N -1) 2 -1 and maximizes the sum of all players' rewards among equilibria whenever λ ≥ 1 for all players.

At this equilbrium, the expected penalty p i ( m 1 , . . . , m N ) paid by each player i is equal to 0 . Each player is incentivized to participate in the penalized game rather than relying on their own estimate, whenever N > 2 , the other N -1 players participate at the honest equilibrium, and λ i > N ( N -1) 2 .

Intuitively, our incentive mechanism is effective because α i and b i only affect the first term of the original reward of player i (equation 2), as well as the penalty, to which they contribute at most as 1 N 2 and -( N -1) 2 -1 N 2 C respectively. At the honest equilibrium every player's MSE is in O ( 1 N ) , such that for large N a player can strongly improve their own error by joining the collaboration, while barely affecting others' errors. For a complete proof consider Appendix D.1.

Theorem 5.1 shows that our mechanism fulfills two desirable properties: (budget) balance and (ex-ante) individual rationality/voluntary participation (Jackson, 2014). The first property means that the server neither makes a profit nor a loss. The second holds as long as λ i > N ( N -1) 2 and the other players take part in the optimization, and means that a player will receive better reward at the game's equilibrium, than when learning with their own data, despite the penalties assigned by the server. While non-honest equilbria exist, these are difficult to coordinate on (as they lack the natural symmetric Schelling point of honesty), while also yielding less total reward summed across players than honesty, such that there is no strong incentive for such coordination.

5.2 Non-transferable utility

We now discuss a way to achieve similar results in the case of non-transferable utility, where players' rewards are not translatable to monetary terms. Instead of modifying players' reward function, we modify the FL protocol, altering the server's messages to players. This effectively results in a robust learning algorithm that the server can implement. We do so by letting the server send noisier versions of its mean estimate to players whose messages suspiciously deviate from the average of all players' updates. The penalization scheme is designed in a way such that players receive expected rewards E ( R i ) similar to the expected penalized rewards E ( R i p ′ ) in the previous section. This is conceptually similar to methods against free-riding (e.g. Karimireddy et al. (2022)), which often tie the accuracy of the model a client receives in an FL setting to the client's overall contribution to model training.

glyph[negationslash]

Theorem 5.2. Consider the modified game with reward R i = ∑ j = i ‖ θ j -µ ‖ 2 N -1 -λ i ‖ θ i -µ ‖ 2 , where player i receives an estimate ¯ m + √ Cglyph[epsilon1] i ‖ m i -¯ m ‖ for independent noise glyph[epsilon1] i with mean E glyph[epsilon1] i = 0 and "variance" E ‖ glyph[epsilon1] i ‖ 2 = 1 , instead of the empirical mean ¯ m , from the server. Then honesty

( α i = 0 , b i = 0 , β i = C C +1 ) is a Nash equilibrium, as long as C > 1 λ i ( N -1) 2 -1 and λ i > 1 ( N -1) 2 . Furthermore, honesty maximizes the sum of all players' rewards among equilibria whenever λ i ≥ 1 for all players.

For fixed λ i = λ, k > 1 and C = k λ ( N -1) 2 -1 , E ( ‖ θ i -µ ‖ 2 ) = O ( σ 2 Nn + σ 2 glyph[star] N ) and players are incentivized to participate in the penalized game rather than relying on their own estimate, whenever N ≥ 2 , the other N -1 players participate at the honest equilibrium and λ ≥ 1 .

Essentially, the noise added by the server increases the expected MSE for player i by C ‖ m i -¯ m ‖ 2 , producing similar incentives as in Theorem 5.1. Theorem 5.2 is proven in Appendix D.2. We obtain voluntary participation if at least two other players participate at the honest equilibrium and λ > 1 .

The benefits of modeling clients' rationality Note that at the equilibrium, players' MSEs are of the same order as if all clients honestly communicated their sample means. In particular, when σ ∗ = 0 (i.e. homogeneous clients), this is in contrast to known negative results for worst-case poisoning attacks Qiao & Valiant (2018) and single-round Byzantine robustness Alistarh et al. (2018). In this sense, our modified protocol acts as a robust and efficient collaborative learning algorithm. This is possible because our data corruption model is derived by explicitly modeling clients' incentives.

6 Beyond mean estimation: stochastic gradient descent

We now extend the ideas from the last section to multi-round collaborative Stochastic Gradient Descent (SGD) in the FL setting. We show that under stronger assumptions, mechanisms similar to those described in Section 5.1 can still provide arbitrary bounds on manipulations by rational players. Again, this is complimentary to methods for robust federated learning such as median-based gradient aggregation that can reduce but not eliminate the impact of existing manipulations.

6.1 The game and rewards

We consider a T -round version of the game described in Section 3. The FL protocol is designed to minimize a loss function f D ( θ ) over a closed and convex set of model parameters θ ∈ W ⊂ R d that contains the global minimizer of f D ( θ ) . At every time step t , each player is meant to communicate an update g t ( θ s t , x i ) with expectation ∇ f ( θ s t ) . We denote by e i t ( θ t ) = g t ( θ s t , x i ) - ∇ f ( θ t ) the difference between the gradient and the estimate, which is a deterministic function of θ t for fixed data x i , but is assumed to fulfill E x i e i t ( θ ) = 0 for all θ , and to be independent across time and players. Intuitively, f D ( θ ) can be thought of as an expected loss E x ∼D f x ( θ ) , for which player i computes approximate stochastic gradients as ∇ f x i t ( θ s t -1 ) using their t -th sample x i t . The message sent by player i is termed m i t . The server averages the received messages to compute a gradient estimate ¯ m t = 1 N ∑ i m i t and updates the parameter θ via θ s t +1 = Π W ( θ s t -γ t ¯ m t ) , using a fixed learning rate schedule γ t and the projection Π W onto W . Finally, the server sends the updated θ s t +1 to all players.

Strategies and rewards We consider attacks of the form m i t = g t ( θ s t , x i ) + α i t ξ i t , that is, the true gradient estimate plus random noise of the form α i t ξ i t , where E ξ i t = 0 , E ‖ ξ i t ‖ 2 = 1 and ξ i t is sampled independent from the other ξ j t ′ and the algorithm's trajectory. The set of attack strategies A g is then described by the sequence of aggressiveness parameters α i t > 0 , which we assume to be selected in advance, independent of the optimization trajectory. Since defenses were already ineffective for mean estimation, we directly focus on mechanisms and only consider adjustments to the server's final estimate for the noise player i added themselves in the final step T : θ i = θ -α i ξ i .

T +1 T T N T The assumption of non-adaptive strategies is needed to avoid complicated dependencies between successive SGD rounds and more sophisticated attack strategies are beyond the scope of our analysis.

We do not consider a bias term for these attacks. Our unbiased attacks have natural interpretations, both in terms of hiding randomly selected data points and differential privacy defenses , and unlike in the mean estimation case, the effects of a fixed-direction attack on the loss f D ( θ ) can strongly depend on the current estimate θ s t and the attack's precise direction, making it substantially harder to analyze such attacks. That said, it is easy to see that if the server aims to optimize f D ( θ T +1 ) and is allowed to shift its estimate θ s t to defend against fixed-direction attacks at every step t , the fixed direction attacks would be neutralized by the server at any equilbrium, unless they inadvertently improved f D ( θ T +1 ) .

Given these strategies and a Lipschitz function U i : R N → R , player i aims to maximize the reward glyph[negationslash]

Our next result establishes that players can be incentivized to be arbitrarily honest, using a penalty scheme similar to the one in Section 5.1. Again, penalties are redistributed such that players' penalties have zero expectation whenever α i t = α j t for all i, j, t . We set ¯ m t = 1 N ∑ j m j t and

It is easy to see that this game does not always have an equilibrium and players are often incentivized to lie aggressively. In particular, we recover the mean estimation setting with β i = 0 when setting U i ( x ) = ∑ j = i ‖ θ -θ j ‖ 2 -λ i ‖ θ -θ i ‖ 2 , T = 1 , θ 1 = 0 and γ 1 = 0 . 5 , as d dθ ‖ θ -x j i ‖ 2 = 2( θ -x j i ) .

glyph[negationslash]

for constants C t . With this penalized reward we prove:

Theorem 6.1. Assume f is B -smooth and L -Lipschitz on W and m -strongly convex on R d (See Appendix 2 for definitions of these properties). Also assume that for all i and t the gradient noise e i t is B ′ -Lipschitz with probability one and that there exist scalars M ≥ 0 and M V ≥ 0 , such that for all t :

Set the learning γ t = 4 ηm + tm for an integer constant η > 1 , such that 4 ηm + m ≤ 1 B ( M V /N +1) .

glyph[negationslash]

Then if U i is l 1 -Lipschitz with constant L U for all players i , all player's best response strategy fulfill α t i ≤ 8 LL U N C t ( N -2) m √ T + η independent of other players' strategies. If α i t = α j t , ∀ i, j, t , each player's expected penalty is 0 . For glyph[epsilon1] > 0 , C t ≥ 8 LL U N glyph[epsilon1] ( N -2) m √ T + η yields α t i ≤ glyph[epsilon1] for rational players. In that case, if in addition W is bounded and we have that P ( ∃ t ≤ T : Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t ) ∈ O ( 1 NT ) , we get E ( f ( θ T +1 ) -f ( θ ∗ )) ∈ O ( 1+ M + glyph[epsilon1] 2 NT ) + O ( 1 T 2 ) .

Intuitively, small perturbations of gradient estimates sent to the server should only have a small effect on the final learnt model. Formally, our assumptions allow us to inductively prove bounds on the difference between the values of E f ( θ j T +1 ) in a clean ( α i t = 0) scenario and a scenario with other values of α i t . By setting C t large enough, we can then ensure that the penalties paid by a dishonest player always outweigh their effect on the final model. The condition on Π W ensures that SGD is not slowed down by projections and holds for "sufficiently large" distances between the boundary of W and θ 1 . See Appendix E.2 for more details on this and the theorem proof. Theorem 6.1 implies that for sufficiently large C t , despite all players acting strategically , the model converges at speed comparable to when all clients share clean updates (where the convergence rate is O ( 1+ M NT )+ O ( 1 T 2 ) ), thereby ensuring full learning benefits from the collaboration. Moreover, as long as all players are equally honest, this is achieved with zero expected penalties for players and thus with budget balance.

7 Experimental results

To verify that our mechanisms can work for SGD in the non-convex case we simulate FedSGD McMahan et al. (2017) with clients corrupting their messages to different degrees, and record how players' rewards and penalties are affected by their aggressiveness α for different penalty constants C . The α i t that empirically maximizes a player's reward is an approximate best response for a given C and fixed α j t for j = i , and should thus be small for a successful mechanism.

First, we simulate FedSGD, treating each writer as a client, to train a CNN-classifier using the architecture provided by Caldas et al. (2018) for the FeMNIST dataset that consists of characters and numbers written by different writers. Second, we train a two-layer linear classifier with 384 hidden neurons on top of frozen "bert-base-uncased" embeddings on the Twitter Sentiment Analysis dataset from Caldas et al. (2018). In both cases, we randomly select m = 3 clients and compute a gradient estimate g i t for the cross entropy loss f using a single batch containing 90% of the data provided by the corresponding writer at time step t < T = 10650 1 . We test on the remaining 10% of the data.

glyph[negationslash]

0

2

4

6

8

Added Noise

a

A

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

0

.

40

Average Reward

R i

p

(

C

)

for group A

C=0

C=0.0001

C=0.0002

0

1

2

3

4

5

Added Noise

a

A

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

Average Reward

R i

p

(

C

)

for group A

C=0

C=0.0005

C=0.001

C=0.002

glyph[negationslash]

For both experiments, we randomly split writers into two groups A and B containing one and two thirds of the writers respectively, and corrupt the gradient estimates ( m i t ) l = ( g i t ) l + α A ( ξ i t ) l sent by players in group A for each weight tensor and bias vector l separately, by adding isotropic normal noise ( ξ i t ) l with "variance" E ‖ ( ξ i t ) l ‖ 2 = 1 . We do the same with α B for group B . At each step, the three corrupted gradients are then averaged (weighted by the corresponding writers' datasets' sizes) to ¯ m t , which is used to update our neural network's parameters θ t with learning rate 0 . 06 , i.e. θ t +1 = θ t -0 . 06 ¯ m t . Unlike in our theorems, writers reuse the same data points whenever they calculate gradients. In the clean case ( α A = α B = 0) , our final models achieve a test accuracy of 86% on FeMNIST 2 and 63% on Twitter. We record both the cross-entropy loss f achieved by the final model θ T on the test set, as well as the sum of the squared deviations ‖ m i t -¯ m t ‖ 2 incurred by each individual client i across all steps. This allows us to calculate the estimated reward R i p ( C ) = f ( θ T ) -∑ T -1 t =0 I t ( i ) C ‖ m i t -¯ m t ‖ 2 + 1 2 ∑ k = i ∑ T -1 t =0 I t ( i ) I t ( k ) C ‖ m k t -¯ m t ‖ 2 received by every player for penalty weights C held constant over time, where I t ( j ) is a binary indicator equal to 1 whenever player i provided an update at time t .

Figure 3 shows the average reward R i p ( C ) received by players in group A for α B = 0 and α A varying on the x-axis for different penalty weights C . It clearly shows that penalization decreases players' gains from adding noise even in the non-convex case, and that near-zero noise is optimal for players given sufficiently large penalty weights C . At the same time, despite client heterogeneity, the penalties paid by honest players are small: In the FeMNIST experiments, if all players are honest, an overwhelming majority (98%) of players end up paying less than 0 . 0031 on average (over 10 rounds), even at C = 0 . 0002 . This is an order of magnitude under the increase in loss from moving from α A = 0 to α A = 9 , which is already disincentivized for the substantially smaller penalty constant C = 5 e -5 . On the Twitter dataset, noise has a larger effect on the loss (the loss is degraded by 0 . 084 already at noise level a A = 5 compared to 0 . 034 at noise level a A = 9 for FeMNIST). This means that larger penalty weights are necessary to incentivize honesty. Correspondingly, the 98 th percentile of penalties paid at the largest considered penalty level C = 0 . 002 is also larger ( 0 . 0243 ) than on FeMNIST. Additional experimental results can be found in Appendix B.2.

8 Conclusion

In this paper, we studied a framework for FL with strategic agents, where players compete and aim to not only improve their model but also damage others' models. We analyzed both the single- and multi-round version of the problem for a natural class of strategies and goals and showed how to design mechanisms that incentivize rational players to behave honestly. In addition, we empirically demonstrated that our approach works on realistic data outside the bounds of our theoretical results.

Acknowledgments

This research was partially funded by the Ministry of Education and Science of Bulgaria (support for INSAIT, part of the Bulgarian National Roadmap for Research Infrastructure). Florian Dorner is grateful for financial support from the Max Planck ETH Center for Learning Systems (CLS). The authors would like to thank Mark Vero and Nikita Tsoy for their helpful feedback on earlier versions of this manuscript.

References

Abay, A., Zhou, Y., Baracaldo, N., Rajamoni, S., Chuba, E., and Ludwig, H. Mitigating bias in federated learning. arXiv preprint arXiv:2012.02447 , 2020.

Alistarh, D., Allen-Zhu, Z., and Li, J. Byzantine stochastic gradient descent. Conference on Neural Information Processing Systems (NeurIPS) , 2018.

Balashov, M. V. and Golubev, M. O. About the lipschitz property of the metric projection in the hilbert space. Journal of Mathematical Analysis and Applications , 394(2):545-551, 2012.

Blanchard, P., El Mhamdi, E. M., Guerraoui, R., and Stainer, J. Machine learning with adversaries: Byzantine tolerant gradient descent. Conference on Neural Information Processing Systems (NIPS) , 2017.

Bottou, L., Curtis, F. E., and Nocedal, J. Optimization methods for large-scale machine learning. SIAM review , 2018.

Cai, Y., Daskalakis, C., and Papadimitriou, C. Optimum statistical estimation with strategic data sources. In Conference on Learning Theory , pp. 280-296. PMLR, 2015.

Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Koneˇ cn` y, J., McMahan, H. B., Smith, V., and Talwalkar, A. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097 , 2018.

Chayti, E. M., Karimireddy, S. P., Stich, S. U., Flammarion, N., and Jaggi, M. Linear speedup in personalized collaborative learning. arXiv preprint arXiv:2111.05968 , 2021.

Chung, K. L. On a stochastic approximation method. The Annals of Mathematical Statistics , 1954.

Diakonikolas, I., Kamath, G., Kane, D., Li, J., Moitra, A., and Stewart, A. Robust estimators in high-dimensions without the computational intractability. SIAM Journal on Computing , 48(2): 742-864, 2019.

Donahue, K. and Kleinberg, J. Model-sharing games: Analyzing federated learning under voluntary participation. AAAI Conference on Artificial Intelligence , 2021a.

Donahue, K. and Kleinberg, J. Optimality and stability in federated learning: A game-theoretic approach. Conference on Neural Information Processing Systems (NeurIPS) , 2021b.

Fang, X. and Ye, M. Robust federated learning with noisy and heterogeneous clients. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2022.

Gao, X. A., Mao, A., Chen, Y., and Adams, R. P. Trick or treat: putting peer prediction to the test. In Proceedings of the fifteenth ACM conference on Economics and computation , pp. 507-524, 2014.

Gradwohl, R. and Tennenholtz, M. Coopetition against an amazon. In International Symposium on Algorithmic Game Theory , pp. 347-365. Springer, 2022.

Grimberg, F., Hartley, M.-A., Karimireddy, S. P., and Jaggi, M. Optimal model averaging: Towards personalized collaborative learning. International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2021 , 2021.

Gupta, S., Ahuja, K., Havaei, M., Chatterjee, N., and Bengio, Y. Fl games: A federated learning framework for distribution shifts. arXiv preprint arXiv:2205.11101 , 2022.

Jackson, M. O. Mechanism theory. Available at SSRN 2542983 , 2014.

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. Advances and open problems in federated learning. Foundations and Trends R © in Machine Learning , 2021.

Karger, E., Monrad, J., Mellers, B., and Tetlock, P. Reciprocal scoring: A method for forecasting unanswerable questions. Available at SSRN , 2021.

Karimireddy, S. P., Guo, W., and Jordan, M. Mechanisms that incentivize data sharing in federated learning. In Workshop on Federated Learning: Recent Advances and New Challenges (in Conjunction with NeurIPS 2022) , 2022.

Konstantinov, N., Frantar, E., Alistarh, D., and Lampert, C. On the sample complexity of adversarial multi-source pac learning. In International Conference on Machine Learing (ICML) , 2020.

McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-efficient learning of deep networks from decentralized data. In Conference on Uncertainty in Artificial Intelligence (AISTATS) , 2017.

Miller, N., Resnick, P., and Zeckhauser, R. Eliciting informative feedback: The peer-prediction method. Management Science , 51(9):1359-1373, 2005.

Nash, J. Non-cooperative games. The Annals of Mathematics , 54(2):286-295, 1951.

Osborne, M. J. and Rubinstein, A. A course in game theory . MIT press, 1994.

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Qiao, M. and Valiant, G. Learning discrete distributions from untrusted batches. Innovations of Theoretical Computer Science (ITCS) , 2018.

Rakhlin, A., Shamir, O., and Sridharan, K. Making gradient descent optimal for strongly convex stochastic optimization. In Proceedings of the 29th International Coference on International Conference on Machine Learning , pp. 1571-1578, 2012.

Shejwalkar, V., Houmansadr, A., Kairouz, P., and Ramage, D. Back to the drawing board: A critical evaluation of poisoning attacks on production federated learning. In IEEE Symposium on Security and Privacy , 2022.

Stein, C. Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics , volume 3, pp. 197-207. University of California Press, 1956.

Tolpegin, V., Truex, S., Gursoy, M. E., and Liu, L. Data poisoning attacks against federated learning systems. In European Symposium on Research in Computer Security , 2020.

Tu, X., Zhu, K., Luong, N. C., Niyato, D., Zhang, Y., and Li, J. Incentive mechanisms for federated learning: From economic and game theoretic perspective. arXiv preprint arXiv:2111.11850 , 2021.

Waggoner, B. and Chen, Y. Output agreement mechanisms and common knowledge. In Second AAAI Conference on Human Computation and Crowdsourcing , 2014.

Witkowski, J. and Parkes, D. C. Peer prediction without a common prior. In Proceedings of the 13th ACM Conference on Electronic Commerce , pp. 964-981, 2012.

Yin, D., Chen, Y., Kannan, R., and Bartlett, P. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning , 2018.

A Ethical considerations

Our theoretical results establish that penalties average out to zero in expectation, but substantial client heterogeneity can still cause large payments for individual honest participants. Despite our experimental results indicating that variance can be manageable even for real-world data, this is problematic for two reasons: The first is about fairness: Paying some honest participants large amounts, while demanding large amounts from others, based on what essentially amounts to luck is problematic, especially when the participants are individuals rather than firms, and when the nature of the problem might make it difficult for participants to gauge the order of magnitude of payments in advance. The second is about diversity: Clients that expect their data to deviate strongly from the mean of the overall data distribution might opt to not participate in FL with our mechanisms, even though underrepresented types of clients can provide data that is crucial to a model's broad performance and generalization. In our formalism, this problem is obscured by the assumption that all participants sample their data independently from the same distribution, and are unable to predict whether or not their data represents outliers.

Correspondingly, it is important to keep penalty weights C as low as possible to reduce the likelihood of overwhelmingly large penalties, only apply our framework in the context of firms rather than individuals (for whom competitive incentives might often play less of a direct role either way), as well as ensure that the our assumptions about a common data distributions are plausible for the problem at hand. The former can be particularly challenging for non-convex problems, or convex problems with unknown problem parameters, for which no strong candidate for C can be established theoretically.

Furthermore, our results do not establish collusion-proofness of our mechanism. While we expect our mechanism to be collusion-proof against small coalitions, there is a problem once the colluding coalition is large enough to significantly affect the mean estimate, as the shifted mean would reduce the penalty paid by each member of the coalition.

B Additional results

B.1 Nash equilibria in the mean estimation game under bounded attacks

The conclusion that no Nash equilibria exist in the mean estimation game described in 4 is rather intuitive, since all participants have an incentive to send as modified an update as possible, therefore damaging the other players' estimates. In practice, however, attacking in an unbounded manner, that is, sending updates very far from the true mean, may not be plausible. Indeed, if most players send their true mean, one expects the variance of the estimates that the server receives to be of order O ( σ 2 n + σ 2 glyph[star] ) . Therefore, players might in practice be reluctant to send estimates that are further than A √ σ 2 n + σ 2 glyph[star] away from there true local mean, for some constant A .

We therefore consider the same game as before, in the case when an upper bound A on the parameters α i is given. Denote the resulting set of attack strategies by A A . Since A A ⊂ A m , Theorem 4.1 holds for the joint set of strategies ( A m A × D ) . Then we have the following

Corollary B.1. In the setup of Theorem 4.1, if the set of available strategies is A A × D for some constant A > 0 , the only Nash equilibria of the game with b i ( x i ) = 0 fixed for all players i are the strategy profiles for which:

Furthermore, at each of these equilibria the value of mean squared error of the estimate of each player i is

As a result, whenever A = O (1) , each player's estimate at any Nash equilibrium achieves a mean squared error of O ( σ 2 Nn + σ 2 glyph[star] N ) , which is of the same order as the MSE of the estimates that would have been obtained in a fully collaborative setting.

B.2 Additional details on experiments

The network we train is based on the network used in the LEAF repository 3 but implemented in pytorch Paszke et al. (2019). It consists of two convolutional layers with relu activations, kernel size 5 , (2 , 2) padding and 32 and 64 filters, respectively, each followed by max pooling with kernel size and stride 2 . After the convolutional layers, there is a single hidden dense layer with 2048 neurons and a relu activation, and a dense output layer. All experiments were conducted using a single GPU each 4 per run.

0

1

2

3

4

5

6

Added Noise

a

A

0

.

28

0

.

30

0

.

32

0

.

34

0

.

36

0

.

38

Average Reward

R i

p

(

C

)

for group A

C=0

C=0.0001

C=0.0002

0

1

2

3

4

5

6

Added Noise

a

B

0

.

36

0

.

38

0

.

40

Average Reward

R i

p

(

C

)

for group B

C=0

C=0.0001

C=0.0002

0

1

2

3

4

5

6

Added Noise

a

B

0

.

35

0

.

36

0

.

37

0

.

38

0

.

39

0

.

40

0

.

41

Average Reward

R i

p

(

C

)

for group B

C=0

C=0.0001

C=0.0002

We downloaded the FeMNIST and Twitter datasets using the code provided at https://github.com/TalwalkarLab/leaf/tree/master/data/femnist, opting not to filter writers that only have produced a small amount of samples. Correspondingly, our FeMNIST dataset contains 817851 examples of handwritten digits and characters written by a total of 3597 writers rather than the 805263 samples from 3550 writers reported in Caldas et al. (2018).

Figures 4, 5, 6 show results similar to 2 for α B fixed to 1 instead of 0 (Figure 4), or α B varying while α A is fixed to 0 (Figure 5), or 1 (Figure 6), respectively. As payments are redistributed, the average payments for players in group B increase slower with α B , as each individual's increase in payment is partially balanced out by an increase in received payments from encountering other members of group B (Figures 5, 6). Meanwhile, figures 4 and 6 hint at honest players receiving money when others are adding noise: In both cases, players of one group receive slightly higher reward for larger penalties when they are honest ( α = 0 ) while the other group slightly cheats ( α = 1 ).

It is worth noting, that we do not perform a projection step in our experiments, such that numerical instabilities become an issue for large values of α . In particular, for α A > 6 or α B > 6 we regularly observed NaN gradients on FeMNIST for one or more of our 10 runs.

Figure 10 show histograms over the total penalty for C = 0 . 0002 (FeMNIST) and C = 0 . 002 (Twitter) paid by each individual client over the whole 10650 steps for the honest case of α A = α B = 0 , averaged over 10 runs. Clearly, most penalties are on the order of 0 . 01 (FeMNIST) or 0 . 1 (Twitter) , which is substantially smaller (FeMNIST) / comparable (Twitter) to the negative effects of larger noise values on the order of 0 . 1 , which are strongly disincentivized by the considered penalty values.

Figure 13 shows additional results using SGD with Median-based rather than Mean-based aggregation as a baseline that is more robust to noisy gradients. We can see that while using Median-based aggregation helps, players can affect the loss as much as before by adding even more noise (see Figure 11), such that adding noise is still incentivized. At the same time, our mechanism still works: As can be seen in Figure 12, rewards still decrease with increased noise for sufficiently large C . Correspondingly, players remain disincentivized to add noise. This empirically suggests that our mechanism can also be applied to more advanced federated learning protocols that go beyond simple SGD with Mean-based aggregation.

0

.

005

0

.

000

0

.

005

0

.

010

0

.

015

0

.

020

0

.

025

0

100

200

300

400

-

-

0

.

3

0

.

2

0

.

1

0

.

0

0

.

1

0

.

2

0

.

3

0

200

400

600

800

1000

-

-

0

2

4

6

8

10

12

14

Added Noise

a

A

0

.

37

0

.

38

0

.

39

0

.

40

0

.

41

0

.

42

0

.

43

Average Reward

R i

p

(

C

)

for group A

Mean

Median

0

2

4

6

8

10

12

14

Added Noise

a

A

0

.

34

0

.

36

0

.

38

0

.

40

0

.

42

Average Reward

R i

p

(

C

)

for group A

C=0

C=2e-06

C=5e-06

C=1e-05

Figure 13: Results on FeMNIST with median-based aggregation: Average reward R i p ( C ) (with penalties calculated in terms of the distance to the median rather than the mean for Median-based aggregation) received by players in group A for α B = 0 and varying α A . Results are averaged over 10 runs and error bars show the standard error.

C Proofs on mean estimation

First we prove Theorem 4.1.

Theorem C.1. Let D be as described in 4. Then, for any strategy profile (( α 1 , b 1 , β 1 ) , . . . , ( α N , b N , β N )) ∈ ( A × D ) N and for any player i ∈ [ N ] , the expected mean squared error of player i is:

glyph[negationslash]

glyph[negationslash]

Proof. Recall that θ i = (1 -β i ) ¯ m i + β i ¯ x i = (1 -β i ) 1 N ( Nθ s -m i + ¯ x i ) + β i ¯ x i . Therefore, θ i -µ = (1 -β i ) ( 1 N ( Nθ s -m i + ¯ x i ) -µ ) + β i ( ¯ x i -µ ) .

Denote

Then we have:

The second term is the mean squared error of the (clean) estimate based on the local data of player i and so:

(11)

Where we can split E ( ‖ ∑ j ( x i j -µ ) ‖ 2 ) because of the tower law of expectation and using that E ( x i j ) = µ i conditional on the value of µ i ; and the squared norm of the last sum factors, as all x i j -µ i terms are independent and have zero mean such that

and so on, allowing us to inductively factor the sum.

Now, to compute the terms that depend on Y , note that by definition θ s = 1 N ∑ N i =1 m i and therefore:

glyph[negationslash]

Define random variables:

so that Y = Y 1 + Y 2 . This implies that:

Since the value of Y 1 only depends on the data of all players j ∈ { 1 , 2 , . . . , N } / { i } and the value of Z depends only on the data of player i , it follows that Z and Y 1 are independent and therefore E ( Z t Y 1 ) = E ( Z ) t E ( Y 1 ) .

Therefore, glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Where the squared norm of the sum factors as all x j i -µ terms have zero mean, while µ is a constant and the last equality follows from the computations in (11).

Now, by definition m j = ¯ x j + α j ( x j ) ξ j + b j ( x j ) for every j ∈ [ N ] , where the ξ j are random variables with zero mean and unit "variance" E ( ‖ ξ j ‖ 2 ) . Note that because Y 1 and Y 2 depend only on the data of all players j ∈ { 1 , 2 , . . . , N } / { i } and on the data of player i respectively, they are independent and we get:

In order to calculate the value of E ( Y 2 ) we have to compute E ( Y 1 ) , E ( Y 2 ) , E ( ‖ Y 1 ‖ 2 ) and E ( ‖ Y 2 ‖ 2 ) . We have:

glyph[negationslash]

glyph[negationslash]

since E ( ξ j ) = 0 and ξ j is independent from all other variables. In addition,

Next,

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Where the first squared norms factors because glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

as the ξ j are independent of all other variables and have zero mean, the E ( ‖ ∑ j = i α j ( x j ) ξ j ‖ 2 ) term factors because all α j ( x j ) and ξ j are independent (and the ξ j have zero mean), the b j ( x j ) terms factor because E < ¯ x j -µ, b j ( x j ) > = 0 by assumption and because the ¯ x j -µ have zero mean and are independent of b i ( x i ) for j = i . The last squared norm factors because all ∑ k x k j -µ terms are independent and have zero mean. Finally,

with the sums factoring for the same reasons as above. Substituting in (13):

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Substituting (11), (12) and (14) into (10), we have:

glyph[negationslash]

glyph[negationslash]

Next we prove a lemma that gives the optimal value of the defense parameter β of a player, assuming that the attack parameters α of all players are fixed.

Lemma C.1. For a fixed set of values α 1 , . . . , α N ∈ [0 , ∞ ) and b 1 , . . . , b N ∈ R d , the value of β i that minimizes the mean squared error of the estimate of player i is given by:

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Proof. Re-writing the statement of Theorem C.1, glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

This is a quadratic function of β i with a positive coefficient in front of the square term. Therefore, the function is minimized over ( -∞ , ∞ ) at the point:

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

This result is closely related to the work of Grimberg et al. (2021), which studies the optimal way of averaging two sample sets from two different distributions, with the goal of minimizing the mean squared error of the estimate on one of these distributions. However, in our case the estimate from the server is an average of manipulated samples coming from multiple distributions , rather than i.i.d. samples from a single one. Now, C.1 allows us to prove Corollary 4.2:

Corollary C.2. The game defined by the expected reward glyph[negationslash]

and the set of strategies A × D does not have a (pure or mixed) Nash equilibrium at which E (( α j ) 2 ) and E ‖ b j ( x j ) ‖ 2 are finite for all players.

Proof. Assume that a strategy profile (( α 1 , b 1 , β 1 ) , . . . , ( α N , b N , β N )) ∈ P ( A × D ) N is a (potentially mixed) Nash equilibrium for which E (( α j ) 2 ) and E ‖ b j ( x j ) ‖ 2 are finite for player j = i glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

(with the expectations taken both over the randomness of the strategy profile and x ). Note that in the definition of the (expected) reward function, the value of the defense parameters of the i -th player β i only affects the MSE of the estimate θ i of that player. Therefore, it follows from Lemma C.1 that each β i must be non-random and defined by equation (15). In particular, it is easy to see that each β i ∈ [0 , 1) . In that case, it follows from Theorem C.1 that the expected reward of each player j is strictly monotonically increasing in E ( α j ( x j ) 2 ) , so that player j can increase their reward by increasing E (( α j ) 2 ) . This contradicts the assumption that the strategy profile is a Nash equilibrium.

Finally, we prove corollary B.1. For this, we consider the same game as before, in the case when an upper bound A on the parameters α i is given. Recall that we denote the resulting set of attack strategies by A A . Since A A ⊂ A m , Theorem 4.1 holds for the joint set of strategies ( A m A × D ) . Then we have the following

Corollary C.3. In the setup of Theorem 4.1, if the set of available strategies is A A × D for some constant A > 0 , the only Nash equilibria of the game with b i ( x i ) = 0 fixed for all players i are the strategy profiles for which:

Furthermore, at each of these equilibria the value of mean squared error of the estimate of each player i is

Proof. Assume that a strategy profile (( α 1 , b 1 , β 1 ) , . . . , ( α N , b N , β N )) is a Nash equilibrium. As in Corollary C.2, it follows that each β i is given by equation (15) and is therefore in the interval [0 , 1) . Therefore, the reward of each player i is increasing with E ( α i ( x i ) 2 ) . It follows that α i ( x i ) = A for all i ∈ [ N ] . Substituting for the value of ( β i ) ∗ we get that for every i ∈ [ N ] :

Substituting into Theorem C.1 and setting ˆ σ 2 = σ 2 + nσ 2 glyph[star] we get:

glyph[negationslash]

D Proofs on mechanisms for mean estimation

Next, we proof a version of Theorem 5.1 without redistribution.

Proposition D.1. In the setting of 4.1, the penalized game with rewards glyph[negationslash]

has a Nash equilbrium consisting of the strategies α j = b j = β j = 0 for all j whenever C > 1 ( N -1) 2 .

At this equilbrium, the expected penalty p i ( m 1 , . . . , m N ) paid by each player i is equal to C ( N -1) Nn σ 2 , thus a player is incentivized to participate in the penalized game rather than relying on their own estimate, whenever N > 2 , the other N -1 players participate at the honest equilibrium and λ i > C + N ( N -1) 2 or N = 2 and λ i > C +1 .

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Correspondingly, we get

To analyze the second term, we calculate:

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

setting d i b = N -1 N b i ( x i ) -∑ j = i b j ( x j ) N . The squared norm again factors because of the independence and zero means of both ξ i and x i k -µ and because E < ¯ x j -µ, b j ( x j ) > = 0 , while b j ( x j ) is independent of ¯ x i -µ for i = j . Now, inserting 18 in 17, we obtain glyph[negationslash]

As β i ∈ [0 , 1] . Thus E d dα i R i p is negative whenever C ( N -1) 2 > 1 or C > 1 ( N -1) 2 . Correspondingly, for such C , player i is incentivized to set α i = 0 , independent of other players' strategies. Now, assuming b j ( x j ) = 0 for all other players j = i , we also get glyph[negationslash]

glyph[negationslash]

Correspondingly, b i = 0 for all players is a Nash Equilbrium. As the penalty p i ( m 1 , ..., m N ) does not dependend on the defense strategies β i , the optimal β i at the equilbrium α i = 0 and b i = 0 can still be calculated using 15 and is equal to zero as well.

The average penalty honest players pay at the Nash equilbrium is then given by C ( N -1) N ( σ 2 n + σ 2 glyph[star] ) , according to 18.

To understand participation incentives, we compare the equilbrium reward R i ( θ 1 , . . . , θ N , µ ) player i receives if they do not participate while all other players do, to the penalized reward R i p player i glyph[negationslash]

would obtain when participating. We first calculate R i ( θ 1 , . . . , θ N , µ ) assuming player i only uses their own estimate and does not send an update to the server, while the other N -1 players participate in the penalized game at equilbrium:

glyph[negationslash]

using 4.1 with α j = b j = β j = 0 and substituting N -1 and 1 for N respectively. We then calculate E R i p :

glyph[negationslash]

using 4.1 with α j = b j = β j = 0 and D.1. The difference between these can the be calculated as

and is positive whenever λ i > C + N ( N -1) 2 , such that in these cases player i is better off participating in data sharing, despite the penalties. If N = 2 , we instead obtain

We now prove 5.1:

Theorem D.1. In the setting of 4.1, the penalized game with rewards glyph[negationslash]

glyph[negationslash]

has a Nash equilbrium consisting of the strategies α j = b j = β j = 0 for all j whenever C > 1 ( N -1) 2 -1 . Furthermore, this equilibrium maximizes the sum of all players' rewards among equilibria whenever λ i ≥ 1 for all players.

At this equilbrium, the expected penalty p i ( m 1 , . . . , m N ) paid by each player i is equal to 0 , such that player i is incentivized to participate in the penalized game rather than relying on their own estimate, whenever N > 2 , the other N -1 players participate at the honest equilibrium, and λ i > N ( N -1) 2 .

Proof. Analogous to the proof of D.1 We have that

by inserting 18 and 19.

Similarly, assuming b j ( x j ) = 0 for all players j = i we get

by inserting 18 and 20

Thus E d d E α i ( x i ) 2 R i p ′ is negative whenever C (( N -1) 2 -1) > 1 or C > 1 ( N -1) 2 -1 and d d E ‖ b i ( x i ) ‖ 2 E R i p ′ under the same conditions as long as b j ( x j ) = 0 for all other players j .

The expected penalty paid by each player is zero by symmetry, as every players' paid penalty gets redistributed equally among all other players, such that the payments cancel out in expectation.

Similarly, the calculations for participation incentives are exactly as in D.1, but E R i p ′ is now equal to (1 -λ i )( σ 2 Nn + σ 2 glyph[star] N ) rather than (1 -λ i )( σ 2 Nn + σ 2 glyph[star] N ) -C ( ( N -1) Nn σ 2 + ( N -1) N σ 2 glyph[star] ) because the expected penalty paid by players is equal to zero at equilbrium.

glyph[negationslash]

Lastly, it is easy to see that E ‖ b i ( x i ) ‖ 2 has to be nonzero for at least two players at any other equilibrium. Compared to the honest equilibrium, that increases every players' MSE. But the (unpenalized) expected reward for player i equals ∑ j = i ‖ θ j -µ ‖ 2 N -1 -λ i ‖ θ i -µ ‖ 2 , such that the sum of all players' (unpenalized) rewards equals ∑ j (1 -λ j ) ‖ θ j -µ ‖ 2 , which is strictly monotonically decreasing in all players' MSEs when λ i ≥ 1 for all players. As all players' penalties add up to zero in expectation, the sum of penalized rewards is equally strictly monotonically decreasing in all players' MSEs.

Next, we prove 5.2

Theorem D.2. Consider the modified game with reward glyph[negationslash]

where player i receives an estimate ¯ m + √ Cglyph[epsilon1] i ‖ m i -¯ m ‖ for independent noise glyph[epsilon1] i with mean E glyph[epsilon1] i = 0 and "variance" E ‖ glyph[epsilon1] i ‖ 2 = 1 , instead of the empirical mean ¯ m , from the server. Then honesty ( α i = 0 , b i = 0 , β i = C C +1 ) is a Nash equilibrium, as long as C > 1 λ i ( N -1) 2 -1 and λ i > 1 ( N -1) 2 . Furthermore, honesty maximizes the sum of all players' rewards among equilibria whenever λ i ≥ 1 for all players.

Furthermore, for fixed constant λ i = λ , E ( ‖ θ i -µ ‖ 2 ) ∈ O ( σ 2 Nn + σ 2 glyph[star] N ) whenever C = k λ ( N -1) 2 -1 for any constant k > 1 , such that players are incentivized to participate in the penalized game rather than relying on their own estimate, whenever N > 2 , the other N -1 players participate at the honest equilibrium, and λ ≥ 1 .

glyph[negationslash]

Proof. In order to calculate the reward R i ( θ 1 , . . . , θ N , µ ) = ∑ j = i ‖ θ j -µ ‖ 2 N -1 -λ i ‖ θ i -µ ‖ 2 for player i , we have a closer look at the mean squared error incurred by players in the modified game. For glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

using the calculations from the proof of C.1 for the first term and D.1 for the second. The last term is equal to zero because:

as glyph[epsilon1] i is independent of all the other terms and has mean zero. Simplifying, we obtain

E ‖ θ i -µ ‖ 2

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

As in C.1, this yields an optimal value for β i of

glyph[negationslash]

for H i ( α, b ) = 1+ C N 2 ∑ j = i E ( α j ( x j ) 2 ) + 1 N 2 E ‖ ∑ j = i b j ( x j ) ‖ 2 + C ( N -1 N ) 2 E ( α i ( x i ) 2 ) + C E ‖ d i b ‖ 2 Now, calculating the derivative of E R i ( θ 1 , . . . , θ N , µ ) with respect to E ( α i ( x i ) 2 ) yields:

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Similarly, assuming b j ( x j ) = 0 for all other players j = i , we get glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Both are negative, whenever β j = β i = 1 and λ i C ( N -1) 2 > 1 + C or C ( λ i ( N -1) 2 -1) > 1 , which is true whenever C > 1 λ i ( N -1) 2 -1 and λ i > 1 ( N -1) 2 .

But for α j = b j = 0 for all players j , the formula for ( β i ) ∗ simplifies to

glyph[negationslash]

for all players. In particular, we have ( β i ) ∗ < 1 for C > 0 , such that β j = β i = 1 and the derivatives with respect to both α and b are negative for all players, turning α j = b j = 0 , ( β i ) = C 1+ C into a Nash equilibrium.

We can now upper bound E ‖ θ i -µ ‖ 2 at the honest equilbrium by considering it at the suboptimal β i = 0 :

which is is O ( ¯ σ 2 N ) as long as C is in O ( 1 N ) , which is the case for constant λ i = λ , C = k λ ( N -1) 2 -1 and any k > 1 .

In terms of participation incentives, we again look at the difference in rewards obtained by player i in both cases:

glyph[negationslash]

It is obvious that E ‖ θ i ( N ) -µ ‖ 2 < E ‖ θ j ( N -1) -µ ‖ 2 ≤ E ‖ θ i (1) -µ ‖ 2 at the honest equilibrium because of symmetry and as players could otherwise improve their reward by setting β i = 1 , which glyph[negationslash]

glyph[negationslash]

was shown to be suboptimal in the proof of 5.2. This has two implications: First, 23 is positive for λ i = 1 . Second, the derivative of 23 with respect to λ i is always positive. Combined, this implies that 23 is positive for all λ i ≥ 1 .

glyph[negationslash]

Lastly, it is easy to see that E ‖ b i ( x i ) ‖ 2 has to be nonzero for at least two players at any other equilibrium. Compared to the honest equilibrium, that increases every players' MSE. But the expected reward for player i equals ∑ j = i ‖ θ j -µ ‖ 2 N -1 -λ i ‖ θ i -µ ‖ 2 , such that the sum of all players' rewards equals ∑ j (1 -λ j ) ‖ θ j -µ ‖ 2 , which is strictly monotonically decreasing in all players' MSEs when λ i ≥ 1 for all players.

E Proofs on stochastic gradient descent

Outlook In this section we prove Theorem 6.1. To this end, we first present the formal definitions of the assumptions on f that we make. Next, we prove two results which bound the difference between the performance of a model resulting from a corrupted optimization scheme (in which players send corrupted estimates) and the performance of a model resulting from honest participation (i.e. from vanilla SGD). In particular, Theorem E.1 provides such a result for a simple penalization scheme, which then easily extends to the penalties presented in Section 6 (Theorem E.2). Next, we combine these results with classic bounds on the distance of the plain SGD trajectory to the minimum value of f (Lemma E.1), to provide an upper bound on the difference between the performance of the corrupted trajectory and the minimum value of f in Theorem E.1. Finally, we show how this last result can be extended to general Lipschitz utilities in Theorem E.2, thereby proving the result from the main text.

Definitions First we formally state our assumptions on the function f .

Definition 2. A function f : W ⊂ R n → R d is called L -Lipschitz with respect to given norms ‖· ‖ n and ‖ · ‖ d if for all x, y ∈ W

Definition 3. A continously differentiable function f : W ⊂ R n → R is called B -smooth if its gradient ∇ f : W → R n is B -Lipschitz with respect to the euclidean norm.

Definition 4. A differentiable function f : W ⊂ R n → R is called m -strongly convex if for all x, y ∈ W

where ‖ · ‖ denotes the euclidean norm.

We start with proving a weaker version of 6.1 with constant learning rates in which player's paid penalties do not get redistributed:

Proposition E.1. Assume f is B -smooth and L -Lipschitz with respect to the euclidean norm on W and m -strongly convex on R d . Also assume that for all i, t the gradient noise e i t is B ′ -Lipschitz with respect to the euclidean norm with probabiltiy one and that the constant learning rate γ fulfills 0 < γ < 2 m B 2 + B ′ 2 . Then for the penalized game with reward glyph[negationslash]

any player's best response strategy fulfills α i t ≤ LNc T -t 2 γ C t ( N -1) 2 ≤ LNγ C t ( N -1) 2 for c = (1+ γ 2 ( B 2 + B ′ 2 ) -2 γm ) , independent of other players' strategies.

Given glyph[epsilon1] > 0 , the expected absolute change in function values f ( θ j T ) due to noise added by players playing best responses compared to full honesty can be bounded by 1 1 - √ c Lγglyph[epsilon1] by choosing C t ≥

LNc T -t 2 γ glyph[epsilon1] ( N -1) 2 < LNγ glyph[epsilon1] ( N -1) 2 . The total penalties paid by player i can then be bounded by 1 1 - √ c Lγ ( N -1) ( G 2 glyph[epsilon1] + glyph[epsilon1] ) ≤ for a global bound on the "variance" of the gradient estimates ‖ e i t ( θ ) ‖ 2 ≤ G 2 .

Proof. We compare two trajectories θ t and θ ′ t starting at the same θ 0 and sharing the same realizations for the noise variables e j t and ξ j t in which player i employs different agressiveness schedules α i t and ( α i t ) ′ with squared difference δ i t = ( α i t -( α i t ) ′ ) 2 . We define ¯ e t = 1 N ∑ i e i t . Then:

glyph[negationslash]

glyph[negationslash]

Where the first inequality follows from the well-known 1 -Lipschitzness of projections onto convex closed sets with respect to the euclidean norm (Balashov & Golubev (2012)) and the ξ i terms factor because of their zero mean and independence of the other variables. Similarly, the last term turns out to equal zero because:

as E ¯ e t ( θ ) = 0 for any fixed θ . Correspondingly,

Where the first inequality follows from strong convexity.

glyph[negationslash]

glyph[negationslash]

Now for a constant learning rate γ = γ and c = (1 + γ 2 ( B 2 + B ′ 2 ) 2 γm ) :

so that

Where the last inequality follows from the general inequality √∑ i x i ≤ ∑ i √ x i for x i ≥ 0 , which inductively follows from

In particular if we set θ ′ t to the trajectory in which player i is honest ( ( α i t ) ′ = 0 ), we obtain

The same inequalities holds for players' final estimates θ j T +1 and ( θ j T +1 ) ′ , as the noise correction step θ i T +1 = θ T +1 -α i T N ξ i is the same in both cases, so that

It is worth noting, that the contribution of noise at early time steps to the sum diminishes exponentially as long as c < 1 which is true for γ 2 ( B 2 + B ′ 2 ) -2 γm < 0 , i.e. γ < 2 m B 2 + B ′ 2 .

Next, we consider the expected difference in penalties received by player i at time t if they use ( α i t ) ′ rather than α i t and thus send message ( m i t ) ′ rather than m i t :

glyph[negationslash]

glyph[negationslash]

for g i t = g t ( θ s t -1 , x i ) .

In particular, we can bound the difference between expected penalized rewards for two trajectories with all α i k fixed but two different values α i t and ( α i t ) ′ varying for k = t as follows:

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

In particular, for ( α t i ) ′ = 0 , we obtain

By the quadratic formula, this is zero at zero and at

and because of the negative quadratic term negative whenever α i t > LNc T -t 2 γ C t ( N -1) 2 . Correspondingly, in terms of penalized reward players are always better off by not adding any noise at all α i t = 0 compared to adding large noise, such that rational players will never choose α i t > LNc T -t 2 γ C t ( N -1) 2 . Therefore, the noise α i t added by any player i at a given time step t can be limited to any fixed constant glyph[epsilon1] > 0 by choosing C t such that LNc T -t 2 γ C t ( N -1) 2 ≤ glyph[epsilon1] , i.e. C t ≥ LNc T -t 2 γ glyph[epsilon1] ( N -1) 2 .

Applying this observation to each player, substituting into Equation (26) and using the triangle inequality, the overall damage caused by all N players compared to full honesty can then be bounded by

where θ ′ t represents the fully honest strategy and θ t represents a strategy in which all players act rationally given the penalty magnitude C t . Using a geometric series bound, we obtain E | f ( θ T +1 ) -f ( θ ′ T +1 ) | ≤ 1 1 - √ c Lγglyph[epsilon1] .

Lastly, for a global bound on the "variance" of the gradients ‖ e i t ( θ ) ‖ 2 ≤ G 2 we get, glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

as the ξ i t are independent with zero mean, and the gradient noise e i t ( θ ) are independent with zero mean, given θ . Correspondingly, for C t = LNc T -t 2 γ glyph[epsilon1] ( N -1) 2 the total expected penalties paid by player i can be bounded as

Next, we prove the budget-balanced version of E.1

Proposition E.2. Under the assumptions of E.1, in the balanced penalized game with reward glyph[negationslash]

glyph[negationslash]

any player's best response strategy fulfills α i t ≤ Lc T -t 2 γ C t ( N -2) ≤ Lγ C t ( N -2) for c = (1 + γ 2 ( B 2 + B ′ 2 ) -2 γm ) , independent of other players' strategies.

Given glyph[epsilon1] > 0 , the expected absolute change in function values f ( θ j T ) due to noise added by players playing best responses compared to full honesty can be bounded by 1 1 - √ c Lγglyph[epsilon1] by choosing C t ≥ Lc T -t 2 γ glyph[epsilon1] ( N -2) ≤ Lγ glyph[epsilon1] ( N -2) . As long as all players i choose the same strategy ( α i t = α j t ∀ i, j, t ), the expected total penalty paid by each player equals zero.

Proof. We begin by considering the expected difference in penalties received by player l at time t if player i uses ( α i t ) ′ rather than α i t and thus send message ( m i t ) ′ rather than m i t :

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Again, we can bound the difference between expected penalized rewards for two trajectories with all α i k fixed but two different values α i t and ( α i t ) ′ varying for k = t as follows:

glyph[negationslash]

glyph[negationslash]

Again, for ( α i t ) ′ = 0 we obtain

which is zero at zero and at glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

and negative for α i t > Lc T -t 2 γ C t ( N -2) as long as N > 2 . Players are thus again incentivized to select α i t that do not fulfill that inequality.

As in E.1, the overall damage caused by all N players compared to full honesty can then be bounded by E | f ( θ T +1 ) -f ( θ ′ T +1 ) | ≤ 1 1 - √ c Lγglyph[epsilon1] by choosing C t ≥ Lc T -t 2 γ glyph[epsilon1] ( N -2) where θ ′ T represents the fully

honest strategy and θ T represents a strategy in which all players act rationally given the penalty magnitude C t , and the same is true for player's estimates θ j T +1 and ( θ j T +1 ) ′ . By symmetry, as long as all players i choose the same strategy ( α i t = α j t ∀ i, j, t ), the expected penalties paid by each player equal 0 .

To prove 6.1, we adapt a classic result from convex optimization to give convergence rates for SGD with bounded perturbations from the clients and with a linearly decaying learning rate.

Lemma E.1. In the settings of E.2, assume that all players use bounded attacks, so that ( α i t ) 2 ≤ glyph[epsilon1] 2 for all i, t . Also, assume that there exist scalars M ≥ 0 and M V ≥ 0 , such that for all t :

glyph[negationslash]

Assume that for some integer constant η > 0 , such that 4 ηm + m ≤ 1 B ( M V /N +1) , the learning rate is set as γ t = 4 ηm + tm . In that case, if P ( ∃ t ≤ T : Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t ) ∈ O ( 1 NT ) we get E ( f ( θ t ) -f ( θ ∗ )) ∈ O ( 1+ M + glyph[epsilon1] 2 Nt ) + O ( 1 t 2 ) , we have:

for any T ≥ η .

Proof. We first condition on the case in which there is no t ≤ T with Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t , so that we do not have to worry about projections. Then, for a random vector g , denote V ( g ) = E ( ‖ g ‖ 2 ) -‖ E ( g ) ‖ 2 . Note that, by the independence of the stochastic gradients and players' noise, it follows that:

glyph[negationslash]

Additionally, since f is strongly convex, it has a unique minimizer θ ∗ ∈ R d . Now since the learning rate is of the form γ t = 4 /m η + t , with 4 m > 1 m , η > 0 and γ 1 = β η +1 ≤ 1 B ( M V /N +1) , the conditions of Theorem 4.7 in Bottou et al. (2018) hold with µ = 1 , M V = M V /N,M = ( M + glyph[epsilon1] 2 ) /N,M G = M V /N +1 . Using equation 4.23 in their proof gives:

for any t ≥ 1 . We now use a classic result by Chung:

LemmaE.2 (Chung (1954)) . Let { b n } n ≥ 1 be a sequence of real numbers, such that for some n 0 ∈ N , it holds that for all n ≥ n 0 ,

We set x t + η := E ( f ( θ t ) -f ( θ ∗ )) for t ≥ 1 and x k = E ( f ( θ 1 ) -f ( θ ∗ )) for k ≤ η . Using 31 and k = t + η we get

Now using d = 4 and c = 8 B ( M + glyph[epsilon1] 2 ) Nm 2 , we have x k +1 ≤ (1 -d k ) x k + c k 2 such that E.2 yields

where c > 1 , c 1 > 0 . Then and thus

glyph[negationslash]

Now, if there is a t with Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t , we can still bound E ( f ( θ t ) -f ( θ ∗ )) by some constant because W is bounded and f is Lipschitz. Correspondingly, as P ( ∃ t ≤ T : Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t ) ∈ O ( 1 NT ) , the total expectation for both cases combined is bounded by 8 B ( M + glyph[epsilon1] 2 ) 3 m 2 NT + O ( 1 NT ) + O ( 1 T 2 ) .

glyph[negationslash]

To prove theorem 6.1, we use a non-asymptotic version of Chung's Lemma Chung (1954) similar to the one used in the proof of Lemma 1 in Rakhlin et al. (2012):

Lemma E.3. For constants c > 0 and d > 1 , whenever t +1 ≥ d and the recursive inequality

holds we get that if for t = k the same is true for t = k +1 .

Proof. Using the condition on x t , we obtain

as by assumption d t +1 ≤ 1 . As d > 1 , 2 d 2 c ( t +1)( d 3 -d 2 ) is positive and we can divide the equation above by it to obtain

for which we want to show that it is bounded above by 1 . Multiplying the equation

which is equivalent to i.e.

and which is true whenever

Theorem E.1. In the settings of E.2, assume that there exist scalars M ≥ 0 and M V ≥ 0 , such that for all t :

Assume that for some integer constant η > 1 , such that 4 ηm + m ≤ 1 B ( M V /N +1) , the learning rate is set as γ t = 4 ηm + tm .

by t ( t +1) for t ≥ 1 we get

glyph[negationslash]

Then any player's best response strategy fulfills α t i ≤ 8 L C t ( N -2) m √ T 1+ η independent of other players' strategies, such that for any given glyph[epsilon1] > 0 , C t ≥ 8 L glyph[epsilon1] ( N -2) m √ T + η yields α t i ≤ glyph[epsilon1] for rational players. In that case, as long as W is bounded and we have that P ( ∃ t ≤ T : Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t ) ∈ O ( 1 NT ) we get E ( f ( θ t ) -f ( θ ∗ )) ∈ O ( 1+ M + glyph[epsilon1] 2 Nt ) + O ( 1 t 2 ) .

Proof. We make use of inequality 25 from the proof of E.1 to analyse the difference between two trajectories that are identical except for the actions of player i .

For t ≥ 0 and η ≥ max { 32( B 2 + B ′ 2 ) 13 m 2 , 1 } we get

by calculating

We now set x t + η := E ‖ θ t +1 -θ ′ t +1 ‖ 2 for t ≥ 0 and x k = 0 for k ≤ η . These definitions are consistent for t = 0 because E ‖ θ 1 -θ ′ 1 ‖ 2 = 0 .

Using 33 and k = t -1 + η , we get

At the same time, x η = E ‖ θ 1 -θ ′ 1 ‖ 2 = 0 ≤ 4 c η -1 for any c > 0 . Correspondingly, E.3 with d = 1 . 5 and c = 16 N 2 m 2 max t { δ t i } implies that for k ≥ η and k +1 ≥ 1 . 5 we get and thus

for t ≥ 1 . This yields

Consequentially, we obtain

Again, we can bound the difference between expected penalized rewards for two trajectories with all α i k fixed but two different values α i t and ( α i t ) ′ varying for k = t by considering δ k = 0 for all k = t . This yields glyph[negationslash]

As in E.2, this allows us to upper bound the gains in penalized reward from changing α i t to ( α i t ) ′ by

In particular, for ( α i t ) ′ = 0 this bound becomes

which is zero at α i t = 0 and at

and negative for α i t larger than that as long as N > 2 .

Again the noise α i t added by any rational player i at a given time step t can therefore be limited to any fixed constant glyph[epsilon1] > 0 by choosing C t such that 8 L C t ( N -2) m √ T + η ≤ glyph[epsilon1] , i.e. C t ≥ 8 L glyph[epsilon1] ( N -2) m √ T + η . We conclude by using E.1 to obtain the convergence rate.

As a final step, we show that a version of E.1 also holds for more general reward functions, thus proving 6.1.

glyph[negationslash]

Theorem E.2. Up to constants, theorems E.2 and E.1 also hold for the reward R i U p ( θ 1 T +1 , . . . , θ N T +1 , f ) = U i ( f ( θ i T +1 ) , . . . , f ( θ N T +1 )) -∑ T t =1 C U t ‖ m i t -1 N ∑ j m j t ‖ 2 + 1 N -1 ∑ k = i ∑ T t =1 C U t ‖ m k t -1 N ∑ j m j t ‖ 2 for arbitrary l 1 -Lipschitz U i with common Lipschitz constant L U . Any bound on α i t can be achieved by setting C U t = L U NC t for the C t achieving the same bound in E.2 or E.1 respectively.

Proof. We first note that for any pointwise bound E | f ( θ j T +1 ) -f (( θ j T +1 )) ′ | ≤ K

This means that the gains | U i ( f ( θ i T +1 ) , . . . , f ( θ N T +1 )) -U i ( f (( θ i T +1 ) ′ ) , . . . , f (( θ N T +1 ) ′ )) | in unpenalized reward player i can achieve by using a given α i t at time t instead of α i t = 0 is multiplied by L U N compared to 28 and 34. Thus, for a given C U t , the bound on α i t for rational players is multiplied by NL U as well, as the quadratic formula solution for α i t is linear in the linear term. Correspondingly, we need to set C U t = L U NC t to achieve the same bounds on α i t as in E.2 or E.1.

Now, for α i t ≤ glyph[epsilon1] , the bound on the expected absolute change in U i ( f ( θ i T +1 ) , . . . , f ( θ N T +1 )) is L U N times higher than for f ( θ j T +1 ) using the bounds above. Because these bounds are linear in glyph[epsilon1] for E.2, we can achieve a bound of δ for U i ( f ( θ i T +1 ) , . . . , f ( θ N T +1 )) by ensuring α i t ≤ glyph[epsilon1] NL U for glyph[epsilon1] achieving a bound of δ for f ( θ i T ) . In total, we thus need to multiply the corresponding C t by NL U twice to achieve a given bound on the gains in unpenalized reward from cheating: Once because we need a smaller bound on α i t to achieve the same bound on the unpenalized reward, and once to ensure that rational players are incentivized to use that smaller bound.

glyph[negationslash]

Discussion on the projection assumptions We note that the assumption P ( ∃ t ≤ T : Π W ( θ s t -γ t ¯ m t ) = θ s t -γ t ¯ m t ) ∈ O ( 1 NT ) in particular holds if W is chosen such that ‖ w -θ s 0 ‖ ∈ Ω( T ) for all w in the boundary of W while ‖ ¯ m t ‖ ∈ O ( ‖ θ s t -θ s 0 ‖ ) with probability one for all t ≤ T . In that case, the linearly decaying learning rate ensures that ( ‖ θ s t -θ s 0 ‖ ) stays in O ( T ) and thus in W (for appropriately chosen constants) with probability one.

Similarly, Lemma 5 in Rakhlin et al. (2012) states that for a probability one bound on the gradient norm ‖ ¯ m t ‖ ≤ G, ∀ t , we have that ‖ θ s t -θ glyph[star] ‖ < 2 G m with probability one, such that the iterates stay in W without the need for any projections, as long as G grows slower in T than inf w ∈ δ ( W ) ‖ w -θ glyph[star] ‖ . In particular, if f grows quadratically in θ -θ glyph[star] and noise is proportional to the gradient norm, G grows linearly in the distance between θ glyph[star] and the boundary of W , such that the condition holds for the right proportionality constants.

Peer reviews:
comments: SUMMARY:
The paper presents an incentive mechanism to encourage honest data reporting in the presence of spiteful behavior aiming to harm other participants.

STRENGTHS:
The paper considers an interesting an novel setting.

It shows that incentive schemes can in principle induce cooperative behavior.

The incentive schemes show both budget-balance and individual rationality (ex ante).

WEAKNESSES:
While the reward scheme in the paper has truthful reporting as an equilibrium, it is well-known that peer-prediction schemes also admit uninformative equilibria; for example in this case all participants could report the same data without any penalty.
*** This was my most important worry and the authors have addressed this weakness in their rebuttal ****

The schemes requires that the participants observe IID data, which is usually not the case in federated/distributed learning.
*** This still remains to be improved ****

Only particular attack and defense strategies are considered.
*** The authors have convinced me that they have gone far enough at least for this paper. ***

There is no consideration of data privacy.
*** This remains future work. ***

QUESTIONS:
How can you scale the penalties without knowing the utilities of the model to the participants?

LIMITATIONS:
I think the fact that utilities have to be known needs to be stated more clearly.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers a federated learning setting with strategic data resources. The authors assume that the entities taking part in the learning process are selfish players incentivized to get the best model but benefit if their competitors receive inaccurate models. This selfish behavior pushes players to lie to the central learning mechanism in their reports.\n\nThe authors consider two cases: Mean estimation and a multi-round SGD on strongly convex objectives. They model the players' strategies as multiplicative/additive factors that could be added to the players' actual local computations. The authors show that even in the straightforward case of mean estimation, a PNE does not exist. They offer two remedies: Monetary payments (via peer-prediction techniques) and punishments (noisy model updates by the central mechanism). Then, they show that a PNE exists and characterize the form of payments/punishments required.\n\nFinally, they conduct an experimental analysis demonstrating that their remedies positively affect the learning procedure.\n\nSTRENGTHS:\n1.\tThe paper deals with a practical issue that is somewhat under-explored.\n2.\tDespite the abundance of notations, the authors have done an excellent job in making the paper read smoothly.\n\nWEAKNESSES:\n1.\tThe paper adopts a game theoretic approach, but many modeling assumptions seem cumbersome and unjustified (see questions below).\n2.\tIt is hard to assess this paper's technical contribution. Particularly, the novelty of the peer prediction-based mechanisms are well-studied ideas. The authors did not explain whether this paper adopts these ideas in a plug-and-play manner or presents new non-trivial derivations. The \"our contribution\" part addresses the paper's content but not its marginal contribution to the line of research, making the technical contribution hard to assess.\n\nQUESTIONS:\n1.\tStrategy spaces: The assumption that players report their updates along with $\\alpha^i \\xi^i$, where $\\alpha^i$ is player $ i$'s strategy and $\\xi^i$ is a random variable seems completely arbitrary and unjustified. Typically, one makes assumptions about what players aim to do (e.g., maximize their payoff) and what they can do (e.g., bounded computation or memory, acting myopically, etc.). Explicit assumptions about players' strategies without proper grounding and reasoning are cumbersome and inconvenient. Could the authors justify what real-world scenarios the strategy spaces in Eq. (3) models? Could the authors present assumptions about player rationality\\behavior that would recover their modeling? \n2.\tThe assumption about $b^i$ in Eq. (4): What is the justification for this? Indeed, it facilitates the analysis but seems entirely out of context.\n3.\tMulti-round: The authors assume that in the multi-round case, players pick their strategies only once initially. What is the rationale for this modeling? Allowing players to change their strategies throughout the execution will be harder to analyze, but this compromise does not make much sense. \n4.\tCorollary 2: The authors show that a pure Nash equilibrium does not exist, concluding that \"without modifications to the protocol, no player can benefit from collaborative learning.\" While this might be true, I do not see how the inexistence of Nash equilibrium implies collaboration is useless. To reach this conclusion, the authors must show that players are better off (i.e., their estimates are more accurate) without the protocol. Where is this analysis located in the paper? Further, I suspect that a mixed Nash equilibrium does exist, so arguing about whether players can benefit from collaboration should at least consider their payoff under some form of a solution concept (be it mixed equilibrium, sink equilibrium, or otherwise). My question: Could the authors describe why the inexistence of PNE suggests that collaboration is useless?\n5.\tThe methods the authors adopt, e.g., scoring rules, seem to treat the most general case where players can report whatever they want, beyond limiting the structure of their message (Eq. (3) for the mean estimation case). This is at least true in the mean estimation case. The same thing applies to the payment case. What does this paper benefit from making the limiting (and, as I argued before, the highly unjustified) assumption of the structured strategy spaces? \n6.\tWhat is the technical modification of peer-prediction\\noise communication required for this paper? How novel is the derivation needed for this paper, and how does it differ from previous papers? Answers to this question could facilitate the assessment of this paper's contribution.\n\n\nMinor:\n\n•\tWhy are the super scripts m and w needed in lines 198 and 211 (they also appear later in the paper)? \n\n•\t173: clients->players\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

comments: SUMMARY:
The paper studies a centralized collaborative learning problem. Authors provide theoretical guarantees for an attack method and a defense method. Further, the paper proposes two mechanisms to incentivize honesty: a method that uses an explicit side payment method and requires transferable utility, a centralized punishment mechanism where a central server adds noise to the estimates it sends to players that have sent suspicious updates. Simulation results are provided supporting the claims.

STRENGTHS:
-- The paper is well written and easy to follow.

-- Authors provide a description of related work and background. The problem formulation considered in the paper is well positioned in the relevant literature. 

-- Authors provide several theoretical results making novel technical contributions. Authors provide discussions around the implications of the theoretical results.

-- The paper provides simulation results supporting the theoretical claims.

WEAKNESSES:
-- Problem formulation is well positioned in the relevant literature. However, authors do not provide a discussion on how their results and methods compare to existing literature.

-- Authors provide numerical simulation results supporting their analysis. However, authors fail to compare their method with existing methods in the simulations.

QUESTIONS:
-- I encourage authors provide a comparison with existing methods.

-- Is it possible to add a proof sketch in the main paper highlighting the technical challenges addressed in the analysis?

-- Can these results be extended to the decentralized setting?

LIMITATIONS:
Authors do not provide a discussion on limitations of their method. Authors include a discussion on societal impacts in the Appendix.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors investigate the issue of  manipulation (in the form of falsifying data or model updates) among agents who mutually contribute to a shared model. Incentives for such behaviors arise when agents possess differing objectives with respect to the shared model. The authors first demonstrate that without external intervention, these incentives are essentially unavoidable. However, the authors propose two methods for inducing incentive compatibility in such settings; namely payments when utility is transferable, and noisy server messages  when utility is non-transferable. The authors derive these mechanisms and provide additional theoretical results for two settings of collaborative online learning, single shot mean estimation and multi shot shared gradient updates. Lastly the authors provide experimental results on the FeMNIST data set demonstrating that their mechanisms dissuade strategic behavior.\n\nSTRENGTHS:\n1. Distributed learning is a rapidly growing area and there is a real danger that strategic agents could disrupt the efficacy of these systems if their incentives are not properly accounted for. As such, the authors’ work is well motivated and helps to fill an important piece which is currently missing from the literature.\n\n2. Manipulations in these types of settings are often framed as being adversarial. The authors model agents as being strategic rather than purely adversarial. As we have seen in areas like supervised learning the differences between adversarial and strategic agents can be highly consequential in terms of designing robust systems; considering both types of behavior is imperative (the former is already covered in prior work).\n\n3. The need for such mechanisms is well motivated both from a narrative perspective and from a theoretical perspective (Corollary 4.2).\n\n- Considering the case of non-transferable utility increases the applicability of the authors results, and the case of transferable utility provides the system more options to incentivize collaboration when payments are feasible.\n\n4. The authors’ results are constructive, rather than simply existential. For example, rather than saying that there exists a $C$ and $\\lambda$ such that players participate honestly, the authors provide specific ranges of these variables for which honest behavior is an equilibrium. This makes it easier for others to implement their methods in real-world scenarios. \n\n5. While some strong assumptions are made for the theoretical results, such as convexity, the inclusion of the experimental results helps demonstrate that the authors’ approach is effective even when such assumptions do not hold.\n\n6. The paper is well written and the authors take the time to outline the intuition and implications of their results.\n\nWEAKNESSES:\n1.  The mechanisms proposed only induce truthfulness as a Nash Eq, implying that other non-truthful equilibria exist. I understand that these types of results are standard throughout the literature, but when deploying these mechanisms in practice it is important to note that we have no guarantee which equilibrium agents will end up in. This is a far weaker result than truthfulness being a dominant strategy.\n\n2. Similar to the last point, the mechanisms do not appear to be collusion proof. In particular one player could pretend to represent multiple clients (i.e., sending multiple updates each round). For example, in the case of single-shot mean estimation, if such an agent monopolizes a sufficiently large position of the data being submitted, they could force the other agents into submitting any desired value for large enough $C$ (since the deviation penalty will outweigh the other parts of their utility). If agents are willing to misreport data, they are probably also willing to collude. Not accounting for this possibility limits the scope of the work. With that said, the authors appear to be the first work to study robustness to strategic behavior in distributed systems, so perhaps asking for additional results on collusion is too much. However, this should be more clearly stated as a limitation of the work.\n\n3. The experiments are somewhat limited. The primary contribution of the paper is theoretical and the point of these experiments is to show that the payment scheme works even when convexity does not hold, but this observation on a single dataset, for a single model, is a bit unconvincing. In particular, I would expect that the average reward received when increasing $\\alpha_A$ would decrease more rapidly for larger $C$, however, it is not clear to me that the small amount of fines paid by honest players would hold across different scenarios.\n\n\n\n\n### Comments and minor issues: no impact on score and need not be addressed in the author response. \n\n1. Line 259: should this say “... at the honest equilibrium [when] ….”? In the supplement this is stated as an [and] rather than a [when].\n\n2. Corollary 4.2 should probably be a Theorem. This is actually quite an interesting result and somewhat non-trivial based on the proofs. Although Theorem 4.1 is doing most of the heavy lifting here the corollary is actually the main results, while the Theorem feels more like a helping lemma. \n\n3. Links to theorems and references are broken in the main body. Looks like this is the result of compiling the document with the supplement and then using a PDF editor to trim the supplement pages.\n\n4. Figure 6 in the supplement takes up its own page.\n\n5. The naming convention of Theorems is not consistent with the main body and makes it difficult to find a specific theorem within the supplement, unless using the reference provided in the main body.\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nSee above", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}