SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Best response dynamics. We now turn to computation and consider a natural procedure by which agents can find a Nash equilibrium: best response (BR) dynamics. Agents start with some initial sample vector  $s^0$ . In each step  $t$  of the BR dynamics, every agent  $i$  updates their sample contribution proportional to the gradient  $\frac{\partial u_i}{\partial s_i}$  in the direction of increasing utility. Concretely, for a scalar step size  $\delta^t > 0$ , the updates take the following form:
$$
\boldsymbol {s} ^ {t + 1} = \boldsymbol {s} ^ {t} + \delta^ {t} \cdot g \left(\boldsymbol {s} ^ {t}, \boldsymbol {\mu} ^ {t}\right), \tag {4}
$$
where  $g(\pmb{s}^t, \pmb{\mu}^t)_i = \frac{\partial u_i(\pmb{s}^t)}{\partial s_i} + \mu_i^t$  and  $\pmb{\mu}^t$  is chosen so that the updated sample vector  $\pmb{s}^{t+1}$  lies in the feasible region  $S$ . Specifically,  $\pmb{\mu}^t \in \arg \min_{\pmb{\mu} \in K} \| g(\pmb{s}^t, \pmb{\mu}) \|_2$ , where  $K = \{\pmb{\mu} : \pmb{s}^t + \delta^t \cdot g(\pmb{s}^t, \pmb{\mu}) \in S\}$ . For instance, if  $0 \leq s_i \leq \tau_i$ , then:
$$
\mu_ {i} ^ {t} = \left\{ \begin{array}{l} - \frac {\partial u _ {i}}{\partial s _ {i}}, \text {i f} s _ {i} = 0 \text {a n d} \frac {\partial u _ {i}}{\partial s _ {i}} <   0, \text {o r} s _ {i} = \tau_ {i} \text {a n d} \frac {\partial u _ {i}}{\partial s _ {i}} > 0 \\ 0, \text {o t h e r w i s e .} \end{array} \right.
$$
We measure convergence of the above dynamics to a Nash equilibrium via the  $L^2$  norm of the update direction  $g(\pmb{s}^t,\pmb{\mu}^t)$ . Under mild assumptions on the utility functions, we show the dynamics (4) converges to an approximate Nash equilibrium where  $\| g(\pmb{s}^t,\pmb{\mu}^t)\|_2 < \varepsilon$ :
Theorem 3.2. Let  $G(s)$  be the Jacobian of  $u: S \to \mathbb{R}^n$ , i.e.,  $G(s)_{ij} = \frac{\partial^2 u_i(s)}{\partial s_j \partial s_i}$ . Assuming agent utility functions  $u_i$  satisfy
1. Strong concavity:  $(G + \lambda \cdot I_{n\times n})$  is negative semi-definite,
2. Bounded derivatives:  $|G_{ij}| \leq L$
for constants  $\lambda, L > 0$ , the best response dynamics (4) with step size  $\delta^t = \frac{\lambda}{n^2L^2}$  converges to an approximate Nash equilibrium  $\mathbf{s}^T$  where  $\| g(\mathbf{s}^T,\boldsymbol{\mu}^T)\|_2 < \varepsilon$  in  $T$  iterations, where
$$
T = \frac {2 n ^ {2} L ^ {2}}{\lambda^ {2}} \log \left(\frac {\| g (\boldsymbol {s} ^ {0} , \boldsymbol {\mu} ^ {0}) \| _ {2}}{\varepsilon}\right).
$$
Below we sketch the proof and defer the full proof to Appendix A.
Proof sketch. We measure convergence of the above dynamics by the error term  $\| g(\boldsymbol{s}^t,\boldsymbol{\mu}^t)\| _2$ . We show the following bound:
$$
\left\| g (\boldsymbol {s} ^ {t + 1}, \boldsymbol {\mu} ^ {t}) \right\| _ {2} ^ {2} \leq \left\| g (\boldsymbol {s} ^ {t}, \boldsymbol {\mu} ^ {t}) \right\| _ {2} ^ {2} + \delta_ {t} ^ {2} \cdot \left\| G (\boldsymbol {s} ^ {\prime}) g (\boldsymbol {s} ^ {t}, \boldsymbol {\mu} ^ {t}) \right\| _ {2} ^ {2} + 2 \delta_ {t} g (\boldsymbol {s} ^ {t}, \boldsymbol {\mu} ^ {t}) ^ {T} G (\boldsymbol {s} ^ {\prime}) g (\boldsymbol {s} ^ {t}, \boldsymbol {\mu} ^ {t}).
$$
We then use strong concavity to show  $g(\pmb{s}^t, \pmb{\mu}^t)^T G(\pmb{s}') g(\pmb{s}^t, \pmb{\mu}^t) \leq -\lambda \| g(\pmb{s}^t, \pmb{\mu}^t) \|_2^2$ , and the bounded derivatives property to show  $\| G(\pmb{s}') g(\pmb{s}^t, \pmb{\mu}^t) \|_2 \leq nL \| g(\pmb{s}^t, \pmb{\mu}^t) \|_2$ . For our choice of the step size  $\delta^t = \frac{\lambda}{n^2 L^2}$ , we can relate the error in subsequent iterations as follows:
$$
\left\| g \left(\boldsymbol {s} ^ {t + 1}, \boldsymbol {\mu} ^ {t + 1}\right) \right\| _ {2} ^ {2} \leq \left(1 - \frac {\lambda^ {2}}{n ^ {2} L ^ {2}}\right) \cdot \left\| g \left(\boldsymbol {s} ^ {t}, \boldsymbol {\mu} ^ {t}\right) \right\| _ {2} ^ {2}.
$$
This then allows us to conclude that after  $T = \frac{2n^2L^2}{\lambda^2}\log \left(\frac{\|g(\pmb{s}^0,\pmb{\mu}^0)\|_2}{\varepsilon}\right)$  iterations, we will have an approximate Nash equilibrium  $s^T$  with  $\| g(\pmb {s}^T,\pmb {\mu}^T)\| _2\leq \varepsilon$ .
Implications. The above theorem implies that agents playing the natural best response update strategy will converge quickly (in  $O(\log (\varepsilon^{-1}))$  iterations) to a NE. We note that our theorem applies to the payoff functions defined by generalization bounds (eq. (2)) and observed in practice (eq. (3)) as they are strongly concave and have bounded derivatives. Moreover, under the assumptions of Theorem 3.2, our proof also implies the fast convergence of the best response dynamics in the budget-balanced mechanism we discuss in the next section.
4 Welfare Maximization: A Budget-Balanced Mechanism
We first note through an example that Nash equilibria need not be welfare maximizing.
Example 1. Consider two agents with identical payoff functions  $a(s) = 8 - (s_1 + s_2)^{-1}$ , and linear cost functions given by  $c_{1}(s_{1}) = 5c_{1}$  and  $c_{2}(s_{2}) = 4c_{2}$ . The NE is given by  $s^{*} = (0,0.5)$ , i.e., agent 1 does not contribute any data samples. The Nash welfare (which is the  $p$ -mean welfare in  $p \to 0$  limiting case) of this NE is  $u_{1}(s^{*}) \cdot u_{2}(s^{*}) = 24$ . However consider another sample vector  $s' = (0.2,0.4)$  where agent 1 increases her contribution and agent 2 reduces her contribution. Then  $s'$  has a higher Nash welfare of  $u_{1}(s') \cdot u_{2}(s') = 25.25 > 24$ .
To address the issue of NE not being welfare-maximizing, by designing a budget-balanced mechanism for agents with linear cost functions. Our mechanism is inspired from a mechanism for the efficient provisioning public goods (Falkinger et al. [2000]). We show that our mechanism always admits a NE. Moreover, when the sample vector at NE is positive, i.e. every agent participates by contributing data, the NE maximizes the  $p$ -mean welfare among all positive sample vectors. For the FL setting, assuming positive sample vectors is natural since the only way for an agent to participate is by making some positive data contribution.
We present our result for a more general model than one discussed so far. Not only does this generalization strengthen our result, it also naturally allows expressing agent utilities in terms of the taxes/subsidies they receive from our mechanism. In this generalization, each agent  $i$  has a budget  $B_{i}$  of which  $b_{i}$  is unspent and the remaining is used towards the cost of sampling  $s_i$  data points, i.e.,  $b_{i} + c_{i}(s_{i}) = B_{i}$ . We assume agents have arbitrary continuous, concave utility functions of the form  $u_{i}(b_{i},\| s\|_{1})$ . Agents have linear cost functions  $c_{i}(s_{i}) = c_{i}\cdot s_{i} + d_{i}$ , with  $c_{i} > 0$ . This model already captures some previously discussed settings (Sec 2.1) as follows. For e.g., when payoff functions are derived from generalization bounds (Mohri et al. [2018]) or empirical evidence (Kaplan et al. [2020]), they take the form  $a_{i}(\boldsymbol {s}) = \hat{a}_{i}(\| \boldsymbol {s}\|_{1})$ , for some function  $\hat{a}_i$  which depends on  $\| s\| _1$ . Then for  $u_{i}(b_{i},\| s\|_{1}) = b_{i} + \hat{a} (\| s\|_{1})$ , the budget constraint implies the utility takes the form  $\hat{a}_i(\| s\|_1) - c_i(s_i) + B_i$ , which is the same as the utility under the original model with a constant offset. We also assume that for all  $i$ ,  $\frac{\partial u_i(b_i,S)}{\partial S} >0$  and  $\frac{\partial u_i(b_i,S)}{\partial b_i}\neq 0$ .
**Mechanism  $\mathcal{M}_{\beta}$ .** We design a mechanism parametrized by a scalar  $\beta \in (0,1)$ . It uses the following payment scheme. At a data contribution vector  $s$ , each agent  $i$  is rewarded an amount
$$
p _ {i} (\boldsymbol {s}) = \beta (c _ {i} (s _ {i}) - \frac {1}{n - 1} \sum_ {j \neq i} c _ {j} (s _ {j})).
$$
Thus if an agent incurs higher (resp. lower) sampling cost than the average cost borne by other agents, we compensate (resp. penalize) her by a subsidy (resp. tax) of  $\beta$  times her excess cost. By design, our mechanism is budget-balanced: at any sample vector  $s$ , we have:
$$
\sum_ {i} p _ {i} (\boldsymbol {s}) = \beta \left(\sum_ {i} c _ {i} (s _ {i}) - \frac {1}{n - 1} \sum_ {j \neq i} c _ {j} (s _ {j})\right) = 0.
$$
Under this mechanism, given a vector of contributions  $s_{-i}$  of agents other than agent  $i$ , the best response of agent  $i$  is any solution to the following optimization problem:
$$
\max  \quad u _ {i} \left(b _ {i}, s _ {i} + \| s _ {- i} \| _ {1}\right)
$$
$$
\text {s . t .} \quad \forall i: b _ {i} + (1 - \beta) c _ {i} \left(s _ {i}\right) + \frac {\beta}{n - 1} \sum_ {j \neq i} c _ {j} \left(s _ {j}\right) = B _ {i} \tag {5}
$$
$$
\forall i: b _ {i} \geq 0
$$
We next define a  $\beta$  value that plays a crucial role in our mechanism.
Definition 2. (Optimal parameter  $\beta^{*}$ ) Let  $A := (\sum_{i} c_{i}^{-1})^{-1}$  and  $C := \sum_{i} c_{i}$ . Then we define  $\beta^{*}$  as the solution to the following equation.
$$
C \beta^ {2} - (A n (n - 2) + C) \beta + A (n - 1) ^ {2} = 0, \tag {6}
$$
which satisfies  $0 \leq \beta^{*} \leq 1 - 1/n$ .
The next lemma shows that (6) indeed has such a root - the proof is deferred to Appendix B.
Lemma 1. The equation  $C\beta^2 - (An(n - 2) + C)\beta + A(n - 1)^2 = 0$  of (6) has a real root  $\beta^*$  where  $0 \leq \beta^* \leq 1 - 1/n$ .
We now state the main result of this section. We show that for every  $\beta \in [0,1]$ , a Nash equilibrium of  $\mathcal{M}_{\beta}$  exists. Additionally, for a specific value of  $\beta = \beta^{*}$  (Definition 2), our mechanism admits Nash equilibria which are socially efficient: it maximizes the  $p$ -mean welfare:  $W_{p}(\boldsymbol{b},\boldsymbol{s}) = (\sum_{i}u_{i}(b_{i},\|s\|_{1})^{p})^{1/p}$ , for  $p \leq 1$  among all positive sample vectors  $\boldsymbol{s}$ .
Theorem 4.1. For each  $\beta \in [0,1]$ , the mechanism  $\mathcal{M}_{\beta}$  admits a Nash equilibrium. For  $\beta = \beta^{*}$  (Definition 2), whenever the  $NE s^{*}$  of  $\mathcal{M}_{\beta^{*}}$  satisfies  $s^{*} > 0$ , the  $NE s^{*}$  maximizes the  $p$ -mean welfare among all vectors  $s > 0$ , for any  $p \leq 1$ .
We now sketch the proof of the above theorem. The full proof is deferred to Appendix B.
Proof sketch. When  $0 \leq \beta \leq 1$ , the first constraint in the above program is a convex constraint even for general convex cost functions. Since  $u_{i}(\cdot)$  is concave, a proof similar to the proof of Theorem 3.1 shows the existence of a Nash equilibrium.
The program maximizing  $p$ -mean welfare as follows.
$$
\max  \quad W _ {p} (\boldsymbol {b}, \boldsymbol {s}) := \left(\sum_ {i} u _ {i} \left(b _ {i}, \| s \| _ {1}\right) ^ {p}\right) ^ {1 / p}
$$
$$
\text {s . t .} \quad \forall i: b _ {i} + (1 - \beta) c _ {i} \left(s _ {i}\right) + \frac {\beta}{n - 1} \sum_ {j \neq i} c _ {j} \left(s _ {j}\right) = B _ {i} \tag {7}
$$
We first show that the above program is convex. With  $\mu_{i}$  and  $\gamma_{i}$  as the dual variables to the first and second constraints respectively, we write down the KKT conditions of program (7) with  $s_i > 0$ . We then use the KKT conditions satisfied by a NE  $(\pmb{b}^{*},\pmb{s}^{*})$  to find values of the dual variables  $\pmb{\mu}^{*}$  and  $\pmb{\gamma}^{*}$  so that  $(\pmb{b}^{*},\pmb{s}^{*},\pmb{\mu}^{*},\pmb{\gamma}^{*})$  satisfy the KKT conditions of (7). Since KKT conditions are sufficient for optimality, this shows that the NE  $(\pmb{b}^{*},\pmb{s}^{*})$  also maximizes  $p$ -mean welfare.
Implications. Theorem 4.1 shows that by augmenting the federated learning with a simple payment protocol that is budget-balanced, one can obtain NE that maximize any  $p$ -mean welfare function of the net-utilities of the agents. Furthermore, note that the payment augmented utility function is still essentially of the form  $u_{i}(\boldsymbol{s}) = \hat{a}_{i}(\boldsymbol{s}) - (1 - \beta)c_{i}(s_{i}) - \frac{\beta}{n - 1}c_{j}(s_{j})$ , with a constant offset. Since  $\beta \in [0,1]$ , when  $a_{i}(\cdot)$  are  $\lambda$ -strongly concave and  $c_{i}(\cdot)$  are  $\lambda$ -strongly convex, the functions  $u_{i}(\cdot)$  are  $\lambda$ -strongly concave. Therefore Theorem 3.2 applies, which ensures that the welfare maximizing NE can be reached through the simple best response dynamics quickly. Finally, we remark that when cost functions are identical, the value of the optimal parameter  $\beta^{*}$  is  $(1 - 1 / n)$ , which is exactly the value used by Falkinger et al. [2000].
Remark 2. Our mechanism requires that costs of the agents be publicly known in order to calculate the value of  $\beta^{*}$  by solving (6) in Definition 2. This is a common assumption made in previous work (e.g. Karimireddy et al. [2022] and Blum et al. [2021]) and is justified in practice. Indeed, costs are common knowledge in many real-world applications e.g. (1) in many ML applications, each agent derives their training data from manually labeling a subset of a publicly available dataset like CIFAR or ImageNet, and the cost of labeling dataset is usually known; (2) in autonomous driving, where each data point is a random path taken under random external conditions.
5 Distributed Algorithm and Empirical Evaluation
In this section, we realize our budget-balanced mechanism in a real-world FL system. We perform the evaluation on the MNIST (LeCun et al. [2010]) and CIFAR-10 (Krizhevsky [2009]) datasets. We compare our mechanism with two baselines: the standard FedAvg and MWFed Blum et al. [2021]. We denote the vanilla mechanism without budget balancing as FedBR and the budget-balanced mechanism as FedBR-BG. We demonstrate that compared to the FedBR, FedBR-BG achieves better  $p$ -mean welfare for  $p \leq 1$ . We also show that the standard FL protocol FedAvg gives significantly lower welfare since it does not allow agents to change their contribution.
5.1 Algorithm Details
We first define the concrete forms for payoff and cost functions. According to standard practice in FL, an agents payoff is measured through the accuracy evaluated on her local test data given model  $\theta$ , which has the form  $a_{i}(\boldsymbol{s},\theta) = \frac{1}{|D_{i}^{*}|}\sum_{(x,y)\in D_{i}^{*}}\mathbb{1}[\theta (x) = y]$ , where  $D_{i}^{*}$  is the test data for agent  $i$ . We note that this form inherently aligns with  $a_{i}(\boldsymbol{s}) = \hat{a}_{i}(\| \boldsymbol{s}\|_{1})$  because the received global model  $\theta$  is trained on  $\| \boldsymbol{s}\| _1$  samples. We consider linear cost functions where  $c_{i}(s_{i}) = c_{i}s_{i}$ .
Now we derive the update rule for agent contributions for FedBR-BG based on best-response dynamics. We rewrite the utility of agent  $i$  in the budget-balancing mechanism as  $u_{i}(s) = \hat{a}_{i}(\sum_{i} s_{i}) - (1 - \beta)c_{i}s_{i} - \frac{\beta}{n-1}\sum_{j \neq i}c_{j}s_{j}$ . We then compute its gradient with respect to  $s_{i}$ :  $\frac{\partial u_{i}}{\partial s_{i}} = \frac{\partial\hat{a}_{i}(\sum_{i} s_{i})}{\partial s_{i}} - (1 - \beta)c_{i}$ . Since the accuracy function is generally unknown in practice, the server can train a public accuracy function  $\tilde{a}: \tilde{S} \to \mathbb{R}_{\geq 0}$  and broadcast it to all agents. We obtain an estimate of the accuracy  $\tilde{a}$  by evaluating models trained on different numbers of samples, in increasing intervals of  $\Delta s$ . For example, if the server trains models on  $0, 2, 4, \dots$  samples and evaluates their accuracy to obtain the estimated accuracy  $\tilde{a}$ , the interval  $\Delta s = 2$  in this case. We assume  $(\tau_{1}, \ldots, \tau_{n}) \in \tilde{S}$  as the server can be a service provider with plenty of data sources. With  $\tilde{a}$ , we can approximate the gradient as  $\frac{\partial u_{i}}{\partial s_{i}} \approx \frac{\tilde{a}(\sum_{i} s_{i} + \Delta s) - \tilde{a}(\sum_{i} s_{i})}{\Delta s} - (1 - \beta)c_{i}$ . Similarly,  $\frac{\partial u_{i}}{\partial s_{i}} \approx \frac{\tilde{a}(\sum_{i} s_{i} + \Delta s) - \tilde{a}(\sum_{i} s_{i})}{\Delta s} - c_{i}$  for FedBR, and  $u_{i}(s) = a_{i}(\sum_{i} s_{i}) - c_{i}s_{i}$  for FedAvg, MWFed and FedBR.
We further assume that  $s_i$  is a multiple of  $\Delta s$  to ensure that  $\tilde{a}$  is always well-defined for our chosen contributions. Finally, we leverage best-response dynamics to update agent contributions. We present the full description of the algorithm for FedBR-BG and FedBR as Algorithm 1 and Algorithm 2 in Appendix C, respectively.
5.2 Experiment Setup
We conduct the experiments with 10 agents for MNIST and 100 agents for CIFAR-10. For MNIST, we use a CNN as the global model, which has two  $5 \times 5$  convolution layers followed by two fully connected layers with ReLU activation. For CIFAR-10, we use VGG11 (Simonyan and Zisserman [2014]). We consider the i.i.d. setting, i.e., the local data of agents are sampled from the same distribution. For both datasets, each agent has 100 training images and 10 testing images, i.e.,  $\tau_{i} = 100, \forall i \in [n]$ .
We randomly initialize the contributions as a multiplier of 10 in [0, 100]. In each contribution updating step, we re-initialize the global model and perform FedAvg for 50 communication rounds. We set global learning rate  $\eta$  to 1.0, local learning rate  $\alpha$  to 0.01, and momentum to 0.9. We set the number of contribution updating steps to 100 and the sample number interval to 10. For evaluating FedAvg, we simply optimize the global model with the same hyperparameters while keeping individual contribution to  $\tau_{i}$ .
We show the  $p$ -mean welfare of our method and baselines on MNIST and CIFAR-10 in Table 1. We observe that FedBR-BG achieves better  $p$ -mean welfare on both datasets compared to FedBR and FedAvg, verifying our theoretical results. Note that the  $p$ -mean of FedAvg is significantly lower since agents always contribute all their data in FedAvg, which incurs a high cost so that the marginal gain becomes limited.
Table 1:  $p$ -mean welfare of our budget-balanced mechanism FedBR-BG and baselines on MNIST and CIFAR-10. We report the results for different  $p$ . The cost for adding one data sample  $c_{i}$  is 0.005 for every agent.
Method	MNIST	CIFAR-10
p=0.2	p=0.4	p=0.6	p=0.8	p=1.0	p=0.2	p=0.4	p=0.6	p=0.8	p=1.0
FedAvg	48985.23	154.99	22.763	8.726	4.909	42386.21	135.92	23.528	8.381	4.582
MWFed	51326.49	158.92	21.648	8.803	5.230	48178.29	142.91	23.981	8.879	4.891
FedBR	53395.21	168.85	24.784	9.495	5.340	58297.23	178.32	26.187	9.675	5.681
FedBR-BG	54589.31	172.63	25.340	9.708	5.459	60385.32	183.23	27.958	9.981	5.891
In this paper, we formulated a federated learning framework which incorporates both payoffs and agent receives from sharing data as well as the cost she incurs due to sharing data. We show the existence of Nash equilibria under the assumption of concave payoffs and convex costs, which are mild assumptions observed in practice. We then note that while NE exist, they may not maximize any notion of welfare for the agents, leaving agents with less incentive to participate. We address this issue via a budget-balanced mechanism with payments whose NE maximize the  $p$ -mean welfare of the agent utilities. Our experiments show that FedBR-BG achieves better  $p$ -mean welfare compared to FedBR and FedAvg, verifying our theoretical results.
We conclude by discussion some directions for future work. Our mechanism required that costs of the agents be publicly known, or at least verifiable. Studying incentives when costs are not common knowledge is an interesting question. Another assumption of our mechanism was that an agent's payoff depends on the number of data samples. Designing welfare-maximizing mechanisms for general settings where payoff functions take more general forms is another direction for future work.
Acknowledgements. This work is partially supported by the National Science Foundation under grant No. 1750436, No. 1910100, No. 2046726, No. 2229876, DARPA GARD, the National Aeronautics and Space Administration (NASA) under grant no. 80NSSC20M0229, the Alfred P. Sloan Fellowship, and the Amazon research award.
Yuze Zou, Shaohan Feng, Dusit Niyato, Yutao Jiao, Shimin Gong, and Wenqing Cheng. Mobile device training strategies in federated learning: An evolutionary game approach. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pages 874-879. IEEE, 2019.
A Appendix to Section 3
We first show that a Nash equilibrium exists when agent payoff functions are separable, i.e., for every agent  $i$  there are functions  $g_{i}:S_{i}\to \mathbb{R}_{\geq 0}$  and  $h_i:\times_{j\neq i}S_j\to \mathbb{R}_{\geq 0}$  s.t. for all  $s\in S$ ,  $a_{i}(\pmb {s}) = g_{i}(s_{i}) + h_{i}(\pmb{s}_{-i})$
Theorem A.1. In any federated learning problem where agent payoff functions are separable, a Nash equilibrium exists.
Proof. When the payoff function of an agent  $i$  is separable, the best response to any contribution vector  $s_{-i}$  is independent of  $s_{-i}$ :
$$
\begin{array}{l} f _ {i} (\boldsymbol {s} _ {- i}) = \arg \max  _ {x \in S _ {i}} a _ {i} (x, \boldsymbol {s} _ {- i}) - c _ {i} (x) = \arg \max  _ {x \in S _ {i}} g _ {i} (x) + h _ {i} (\boldsymbol {s} _ {- i}) - c _ {i} (x) \\ = \operatorname * {a r g} _ {x \in S _ {i}} \max  _ {g _ {i} (x) - c _ {i} (x).} \quad (\text {s i n c e} h _ {i} (s _ {- i}) \text {i s i n d e p e n d e n t o f} x) \\ \end{array}
$$
Let  $F_{i} \coloneqq \arg \max_{x \in S_{i}} g_{i}(x) - c_{i}(x)$ . Clearly  $F_{i} \neq \emptyset$  since  $S_{i} \neq \emptyset$ . Then any  $s \in \times_{i} F_{i}$  satisfies  $s \in f(s)$  by definition. By Proposition 1 any such sample vector is a Nash equilibrium.
Next, we present a negative result showing that there are federated learning settings where a Nash equilibrium is not guaranteed to exist.
Theorem A.2. There exists a federated learning problem in which a Nash equilibrium does not exist. Moreover, the instance has three agents with continuous, non-decreasing, non-concave payoff functions and linear cost functions.
Proof. Let  $\varepsilon \in (0, \frac{1}{16})$ . Let  $e:[0,1] \to [0,1]$  be a function given by:
$$
e (x) = \left\{ \begin{array}{l} 0, \text {i f} 0 \leq x \leq \frac {1}{2} - \varepsilon , \\ \frac {1}{2} + \frac {1}{2 \varepsilon} \left(x - \frac {1}{2}\right), \text {i f} \frac {1}{2} - \varepsilon \leq x \leq \frac {1}{2} + \varepsilon , \\ 1, \text {i f} \frac {1}{2} + \varepsilon \leq x \leq 1. \end{array} \right. \tag {8}
$$
Essentially the function  $e$  is a continuous, piece-wise linear function connecting  $(0, 0), (\frac{1}{2} - \varepsilon, 0), (\frac{1}{2} + \varepsilon, 1)$  and  $(1, 1)$ .
Now consider the following federated learning instance with  $n = 3$  agents, where  $S_{1} = S_{2} = S_{3} = [0,1]$ . The payoff functions are given by:
$$
a _ {1} (\boldsymbol {s}) = e (s _ {1}) + e (s _ {3}) - e (s _ {1}) \cdot e (s _ {3})
$$
$$
a _ {2} (s) = e \left(s _ {2}\right) + e \left(s _ {1}\right) - e \left(s _ {2}\right) \cdot e \left(s _ {1}\right) \tag {9}
$$
$$
a _ {3} (\boldsymbol {s}) = e \left(s _ {3}\right) + e \left(s _ {2}\right) - e \left(s _ {3}\right) \cdot e \left(s _ {2}\right),
$$
and the cost functions are  $c_{i}(s_{i}) = \frac{1}{4} s_{i}$  for all  $i \in [3]$ . Notice that the payoff functions are increasing in  $s_{j}$  for every  $j \in [3]$  and are continuous since  $e$  is continuous.
We now show that this instance does not admit a Nash equilibrium. Let us first evaluate the best response set  $f_{1}(s_{2}, s_{3})$ . Note that  $u_{1}(\boldsymbol{s}) = e(s_{1}) \cdot (1 - e(s_{3})) + e(s_{3}) - \frac{1}{4} s_{1}$ . Since  $u_{1}(\boldsymbol{s})$  is independent of  $s_{2}$ ,  $f_{1}(s_{2}, s_{3})$  only depends on  $s_{3}$ .