SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \left(\| Y _ {2} \| ^ {2}\right) = \mathbb {E} \left(\| \frac {1}{N} \bar {x} ^ {i} - \mu \| ^ {2}\right) \\ = \mathbb {E} \left(\left\| \frac {1}{N n} \sum_ {j} x _ {j} ^ {i} - \mu \right\| ^ {2}\right) \\ = \mathbb {E} \left(\left\| \frac {1}{N n} \sum_ {j} \left(x _ {j} ^ {i} - \mu\right) + \frac {1 - N}{N} \mu \right\| ^ {2}\right) \\ = \mathbb {E} \left(\| \frac {1}{N n} \sum_ {j} \left(x _ {j} ^ {i} - \mu\right) \| ^ {2} + \| \frac {1 - N}{N} \mu \| ^ {2}\right) \\ = \frac {1}{N ^ {2}} \mathbb {E} \left(\| \frac {1}{n} \sum_ {j} \left(x _ {j} ^ {i} - \mu\right) \| ^ {2}\right) + \frac {(1 - N) ^ {2}}{N ^ {2}} \| \mu \| ^ {2} \\ = \frac {1}{N ^ {2}} \left(\frac {\sigma}{n} + \sigma_ {\star} ^ {2}\right) + \frac {(1 - N) ^ {2}}{N ^ {2}} \| \mu \| ^ {2} \\ = \frac {1}{N ^ {2} n} \sigma^ {2} + \frac {1}{N ^ {2}} \sigma_ {\star} ^ {2} + \frac {(1 - N) ^ {2}}{N ^ {2}} \| \mu \| ^ {2} \\ \end{array}
$$
with the sums factoring for the same reasons as above. Substituting in (13):
$$
\begin{array}{l} \mathbb {E} \left(\left\| Y ^ {2} \right\|\right) = \mathbb {E} \left(\left\| Y _ {1} \right\| ^ {2}\right) + \mathbb {E} \left(\left\| Y _ {2} \right\| ^ {2}\right) + 2 \mathbb {E} \left(Y _ {1}\right) ^ {t} \mathbb {E} \left(Y _ {2}\right) \\ = \frac {(N - 1)}{N ^ {2}} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| (N - 1) \mu + \sum_ {j \neq i} b ^ {j} (x ^ {j}) \| ^ {2} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} (x ^ {j}) ^ {2}\right) \\ + \frac {1}{N ^ {2} n} \sigma^ {2} + \frac {1}{N ^ {2}} \sigma_ {\star} ^ {2} + \frac {(1 - N) ^ {2}}{N ^ {2}} \| \mu \| ^ {2} + 2 \left(\frac {N - 1}{N} \mu + \frac {1}{N} \sum_ {j \neq i} \mathbb {E} b ^ {j} (x ^ {j})\right) ^ {t} \frac {(1 - N)}{N} \mu \\ = \frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) \\ + \frac {(N - 1) ^ {2}}{N ^ {2}} \| \mu \| ^ {2} + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} (x ^ {j}) \| ^ {2} + \frac {2 (N - 1)}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} (b ^ {j} (x ^ {j}) ^ {t} \mu) \\ + \frac {(N - 1) ^ {2}}{N ^ {2}} \| \mu \| ^ {2} + 2 \frac {(N - 1) (1 - N)}{N ^ {2}} \| \mu \| ^ {2} + \frac {2 (1 - N)}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} b ^ {j} (x ^ {j}) ^ {t} \mu \\ = \frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2} \tag {14} \\ \end{array}
$$
Substituting (11), (12) and (14) into (10), we have:
$$
\begin{array}{l} \mathbb {E} \left(\| \theta^ {i} - \mu \| ^ {2}\right) = \left(1 - \beta^ {i}\right) ^ {2} \mathbb {E} \left(\| Y \| ^ {2}\right) + (\beta^ {i}) ^ {2} \mathbb {E} \left(\| Z \| ^ {2}\right) + 2 \left(1 - \beta^ {i}\right) \beta^ {i} \mathbb {E} \left(Z ^ {t} Y\right) \\ = \left(1 - \beta^ {i}\right) ^ {2} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2}\right) \\ + (\beta^ {i}) ^ {2} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) + 2 (1 - \beta^ {i}) \beta^ {i} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N}\right) \\ \end{array}
$$
Next we prove a lemma that gives the optimal value of the defense parameter  $\beta$  of a player, assuming that the attack parameters  $\alpha$  of all players are fixed.
Lemma C.1. For a fixed set of values  $\alpha_{1},\ldots ,\alpha_{N}\in [0,\infty)$  and  $b^{1},\ldots ,b^{N}\in \mathbb{R}^{d}$ , the value of  $\beta^i$  that minimizes the mean squared error of the estimate of player  $i$  is given by:
$$
\left(\beta^ {i}\right) ^ {*} = \frac {\frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2}}{\left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2} - \frac {\sigma^ {2}}{N n} - \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2}\right)} \tag {15}
$$
Proof. Re-writing the statement of Theorem C.1,
$$
\begin{array}{l} \mathbb {E} \left(\| \theta^ {i} - \mu \| ^ {2}\right) = (\beta^ {i}) ^ {2} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2} - \frac {\sigma^ {2}}{N n} - \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} (\alpha^ {j} (x ^ {j}) ^ {2}) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} (x ^ {j}) \| ^ {2}\right) \\ \left. - 2 \frac {\beta^ {i}}{N ^ {2}} \left(\sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2}\right)\right) \\ \left. \right. + \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2}\right) \\ \end{array}
$$
This is a quadratic function of  $\beta^i$  with a positive coefficient in front of the square term. Therefore, the function is minimized over  $(-\infty ,\infty)$  at the point:
$$
(\beta^ {i}) ^ {*} = \frac {\frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} (\alpha^ {j} (x ^ {j}) ^ {2}) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} (x ^ {j}) \| ^ {2}}{\left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2} - \frac {\sigma^ {2}}{N n} - \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} (\alpha^ {j} (x ^ {j}) ^ {2}) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} (x ^ {j}) \| ^ {2}\right)}
$$
This result is closely related to the work of Grimberg et al. (2021), which studies the optimal way of averaging two sample sets from two different distributions, with the goal of minimizing the mean squared error of the estimate on one of these distributions. However, in our case the estimate from the server is an average of manipulated samples coming from multiple distributions, rather than i.i.d. samples from a single one. Now, C.1 allows us to prove Corollary 4.2:
Corollary C.2. The game defined by the expected reward
$$
\mathbb {E} \left(\mathcal {R} ^ {i} (\theta^ {1}, \ldots , \theta^ {N}, \mu)\right) = \frac {\sum_ {j \neq i} \mathbb {E} \left(\| \theta^ {j} - \mu \| ^ {2}\right)}{N - 1} - \lambda_ {i} \mathbb {E} \left(\| \theta^ {i} - \mu \| ^ {2}\right)
$$
and the set of strategies  $\mathcal{A} \times \mathcal{D}$  does not have a (pure or mixed) Nash equilibrium at which  $\mathbb{E}((\alpha^j)^2)$  and  $\mathbb{E}||b^j(x^j)||^2$  are finite for all players.
Proof. Assume that a strategy profile  $((\alpha^1, b^1, \beta^1), \ldots, (\alpha^N, b^N, \beta^N)) \in \mathcal{P}(\mathcal{A} \times \mathcal{D})^N$  is a (potentially mixed) Nash equilibrium for which  $\mathbb{E}((\alpha^j)^2)$  and  $\mathbb{E}\|b^j(x^j)\|^2$  are finite for player  $j \neq i$
(with the expectations taken both over the randomness of the strategy profile and  $x$ ). Note that in the definition of the (expected) reward function, the value of the defense parameters of the  $i$ -th player  $\beta^i$  only affects the MSE of the estimate  $\theta_i$  of that player. Therefore, it follows from Lemma C.1 that each  $\beta^i$  must be non-random and defined by equation (15). In particular, it is easy to see that each  $\beta^i \in [0,1)$ . In that case, it follows from Theorem C.1 that the expected reward of each player  $j$  is strictly monotonically increasing in  $\mathbb{E}(\alpha^j (x^j)^2)$ , so that player  $j$  can increase their reward by increasing  $\mathbb{E}((\alpha^j)^2)$ . This contradicts the assumption that the strategy profile is a Nash equilibrium.
Finally, we prove corollary B.1. For this, we consider the same game as before, in the case when an upper bound  $A$  on the parameters  $\alpha^i$  is given. Recall that we denote the resulting set of attack strategies by  $\mathcal{A}_A$ . Since  $\mathcal{A}_A \subset \mathcal{A}^m$ , Theorem 4.1 holds for the joint set of strategies  $(\mathcal{A}_A^m \times \mathcal{D})$ . Then we have the following
Corollary C.3. In the setup of Theorem 4.1, if the set of available strategies is  $\mathcal{A}_{\mathrm{A}} \times \mathcal{D}$  for some constant  $\mathrm{A} > 0$ , the only Nash equilibria of the game with  $b^{i}(x^{i}) = 0$  fixed for all players  $i$  are the strategy profiles for which:
$$
\left| \alpha^ {i} \left(x ^ {i}\right) \right| = A \quad a n d \quad \beta^ {i} = \frac {A ^ {2}}{\left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) N + A ^ {2}} \quad \forall i \in [ N ]. \tag {16}
$$
Proof. Assume that a strategy profile  $((\alpha^1, b^1, \beta^1), \ldots, (\alpha^N, b^N, \beta^N))$  is a Nash equilibrium. As in Corollary C.2, it follows that each  $\beta^i$  is given by equation (15) and is therefore in the interval  $[0, 1)$ . Therefore, the reward of each player  $i$  is increasing with  $\mathbb{E}(\alpha^i (x^i)^2)$ . It follows that  $\alpha^i (x^i) = A$  for all  $i \in [N]$ . Substituting for the value of  $(\beta^i)^*$  we get that for every  $i \in [N]$ :
$$
\beta^ {i} = (\beta^ {i}) ^ {*} = \frac {\frac {N - 1}{N ^ {2}} \mathrm {A} ^ {2}}{\frac {\sigma_ {n} ^ {2}}{n} + \sigma_ {\star} ^ {2} - \frac {\sigma_ {n} ^ {2}}{N n} - \frac {\sigma_ {\star} ^ {2}}{N} + \frac {N - 1}{N ^ {2}} \mathrm {A} ^ {2}} = \frac {\mathrm {A} ^ {2}}{(\frac {\sigma_ {n} ^ {2}}{n} + \sigma_ {\star} ^ {2}) N + \mathrm {A} ^ {2}}.
$$
Substituting into Theorem C.1 and setting  $\hat{\sigma}^2 = \sigma^2 + n\sigma_\star^2$  we get:
$$
\begin{array}{l} \mathbb {E} \left(\| \theta^ {i} - \mu \| ^ {2}\right) = \left(1 - \beta^ {i}\right) ^ {2} \left(\frac {\hat {\sigma} ^ {2}}{N n} + \frac {1}{N ^ {2}} \sum_ {j \neq i} (\alpha^ {j}) ^ {2}\right) + (\beta^ {i}) ^ {2} \frac {\hat {\sigma} ^ {2}}{n} + 2 \left(1 - \beta^ {i}\right) \beta^ {i} \frac {\hat {\sigma} ^ {2}}{N n} \\ = \frac {\frac {N ^ {2} \hat {\sigma} ^ {4}}{n ^ {2}}}{(\frac {N \hat {\sigma} ^ {2}}{n} + A ^ {2}) ^ {2}} \left(\frac {\hat {\sigma} ^ {2}}{N n} + \frac {N - 1}{N ^ {2}} A ^ {2}\right) + \frac {A ^ {4}}{(\frac {N \hat {\sigma} ^ {2}}{n} + A ^ {2}) ^ {2}} \frac {\hat {\sigma} ^ {2}}{n} + 2 \frac {\frac {A ^ {2} N \hat {\sigma} ^ {2}}{n}}{(\frac {N \hat {\sigma} ^ {2}}{n} + A ^ {2}) ^ {2}} \frac {\hat {\sigma} ^ {2}}{N n} \\ = \frac {\frac {N \hat {\sigma} ^ {6}}{n ^ {3}} + \frac {\hat {\sigma} ^ {4}}{n ^ {2}} (N - 1) A ^ {2} + \frac {A ^ {4} \hat {\sigma} ^ {2}}{n} + 2 \frac {A ^ {2} \hat {\sigma} ^ {4}}{n ^ {2}}}{(\frac {N \hat {\sigma} ^ {2}}{n} + A ^ {2}) ^ {2}} \\ = \frac {\hat {\sigma} ^ {2}}{n} \frac {\frac {N \hat {\sigma} ^ {4}}{n ^ {2}} + \frac {\hat {\sigma} ^ {2}}{n} (N - 1) A ^ {2} + A ^ {4} + 2 \frac {\hat {\sigma} ^ {2}}{n} A ^ {2}}{\left(\frac {N \hat {\sigma} ^ {2}}{n} + A ^ {2}\right) ^ {2}} \\ = \frac {\hat {\sigma} ^ {2}}{n} \frac {\frac {\hat {\sigma} ^ {4}}{n ^ {2}} N + \frac {\hat {\sigma} ^ {2}}{n} N A ^ {2} + A ^ {4} + \frac {\hat {\sigma} ^ {2}}{n} A ^ {2}}{(\frac {\hat {\sigma} ^ {2}}{n} N + A ^ {2}) ^ {2}} \\ = \frac {\hat {\sigma} ^ {2}}{n} \frac {N + \frac {n}{\hat {\sigma} ^ {2}} N A ^ {2} + \frac {n ^ {2}}{\hat {\sigma} ^ {4}} A ^ {4} + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2}}{(N + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2}) ^ {2}} \\ = \frac {\hat {\sigma} ^ {2}}{n} \frac {(1 + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2}) (N + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2})}{(N + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2}) ^ {2}} \\ = \frac {\hat {\sigma} ^ {2}}{n} \frac {\left(1 + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2}\right)}{\left(N + \frac {n}{\hat {\sigma} ^ {2}} A ^ {2}\right)} \\ \end{array}
$$
$$
= (\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}) \frac {(1 + \frac {1}{\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}} A ^ {2})}{(N + \frac {1}{\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}} A ^ {2})}
$$
D Proofs on mechanisms for mean estimation
Next, we proof a version of Theorem 5.1 without redistribution.
Proposition D.1. In the setting of 4.1, the penalized game with rewards
$$
\mathcal {R} _ {p} ^ {i} = \frac {\sum_ {j \neq i} \| \theta^ {j} - \mu \| ^ {2}}{N - 1} - \lambda_ {i} \| \theta^ {i} - \mu \| ^ {2} - C \| m ^ {i} - \theta^ {s} \| ^ {2}
$$
has a Nash equilibrium consisting of the strategies  $\alpha^j = b^j = \beta^j = 0$  for all  $j$  whenever  $C > \frac{1}{(N - 1)^2}$ .
At this equilibrium, the expected penalty  $p^i(m^1, \ldots, m^N)$  paid by each player  $i$  is equal to  $C\frac{(N-1)}{Nn}\sigma^2$ , thus a player is incentivized to participate in the penalized game rather than relying on their own estimate, whenever  $N > 2$ , the other  $N-1$  players participate at the honest equilibrium and  $\lambda_i > C + \frac{N}{(N-1)^2}$  or  $N = 2$  and  $\lambda_i > C + 1$ .
Proof. We begin by inserting the equality  $\mathbb{E}\left(\| \theta^i -\mu \| ^2\right)$
$$
\left(1 - \beta^ {i}\right) ^ {2} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} (\alpha^ {j} (x ^ {j}) ^ {2}) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} (x ^ {j}) \| ^ {2}\right) + (\beta^ {i}) ^ {2} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) +
$$
2  $\left(1 - \beta^i\right)\beta^i\left(\frac{\sigma^2}{Nn} +\frac{\sigma_*^2}{N}\right)$  from 4.1 in the first two terms to obtain
$$
\begin{array}{l} \mathbb {E} \mathcal {R} _ {p} ^ {i} = \mathbb {E} \left(\frac {\sum_ {j \neq i} \| \theta^ {j} - \mu \| ^ {2}}{N - 1} - \lambda_ {i} \| \theta^ {i} - \mu \| ^ {2} - C \| m ^ {i} - \frac {1}{N - 1} \sum_ {j \neq i} m ^ {j} \| ^ {2}\right) \\ = \mathbb {E} \frac {\sum_ {j \neq i} \left(1 - \beta^ {j}\right) ^ {2} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {*} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {k \neq j} \mathbb {E} (\alpha^ {k} (x ^ {k}) ^ {2}) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {k \neq j} b ^ {k} (x ^ {k}) \| ^ {2})\right)}{N - 1} \\ + \mathbb {E} \frac {\sum_ {j \neq i} \left((\beta^ {j}) ^ {2} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) + 2 (1 - \beta^ {j}) \beta^ {j} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N}\right)\right)}{N - 1} \\ \left. - \lambda_ {i} \mathbb {E} \left(1 - \beta^ {i}\right) ^ {2} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {j \neq i} b ^ {j} \left(x ^ {j}\right) \| ^ {2}\right) \right. \\ - \lambda_ {i} \left(\left(\beta^ {i}\right) ^ {2} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) + 2 \left(1 - \beta^ {i}\right) \beta^ {i} \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N}\right)\right) \\ - \mathbb {E} \left(C \| m ^ {i} - \theta^ {s} \| ^ {2}\right) \\ \end{array}
$$
Correspondingly, we get
$$
\frac {d}{d \mathbb {E} \alpha^ {j} (x ^ {j}) ^ {2}} \mathbb {E} \mathcal {R} _ {p} ^ {i} = \sum_ {j \neq i} \frac {\left(1 - \beta^ {i}\right) ^ {2}}{(N - 1) N ^ {2}} - C \frac {d}{d \alpha^ {j} (x ^ {j}) ^ {2}} \mathbb {E} \| m ^ {i} - \theta^ {s} \| ^ {2} \tag {17}
$$
To analyze the second term, we calculate:
$$
\begin{array}{l} \mathbb {E} \| m ^ {i} - \theta^ {s} \| ^ {2} = \| m ^ {i} - \theta^ {s} \| ^ {2} \\ = \mathbb {E} \| m ^ {i} - \frac {1}{N} \sum_ {j} m ^ {j} \| ^ {2} \\ = \mathbb {E} \| \alpha^ {i} (x ^ {i}) \xi^ {i} + b ^ {i} (x ^ {i}) + \frac {1}{n} \sum_ {k} x _ {k} ^ {i} - \frac {1}{N} \sum_ {j} (\alpha^ {j} (x ^ {j}) \xi^ {j} + b ^ {j} (x ^ {j}) + \frac {1}{n} \sum_ {k} x _ {k} ^ {j}) \| ^ {2} \\ = \mathbb {E} \| \alpha^ {i} (x ^ {i}) \xi^ {i} + \frac {1}{n} \left(\sum_ {k} x _ {k} ^ {i} - \mu\right) \\ \end{array}
$$
$$
\begin{array}{l} - \frac {1}{N} \sum_ {j} \left(\alpha^ {j} (x ^ {j}) \xi^ {j} + \frac {1}{n} \sum_ {k} \left(x _ {k} ^ {j} - \mu\right)\right) + \mu - \mu + d _ {b} ^ {i} \| ^ {2} \\ = \mathbb {E} \| \frac {N - 1}{N} \left(\alpha^ {i} \left(x ^ {i}\right) \xi^ {i} + \frac {1}{n} \left(\sum_ {k} x _ {k} ^ {i} - \mu\right)\right) \\ - \frac {1}{N} \sum_ {j \neq i} \left(\alpha^ {j} (x ^ {j}) \xi^ {j} + \frac {1}{n} \sum_ {k} \left(x _ {k} ^ {j} - \mu\right)\right) + d _ {b} ^ {i} \| ^ {2} \\ = (\frac {N - 1}{N}) ^ {2} \mathbb {E} \| \alpha^ {i} \left(x ^ {i}\right) \xi^ {i} \| ^ {2} + (\frac {N - 1}{N}) ^ {2} \mathbb {E} \| \frac {1}{n} \sum_ {k} x _ {k} ^ {i} - \mu \| ^ {2} \\ + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \| \alpha^ {j} (x ^ {j}) \xi^ {j} \| ^ {2} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \| \frac {1}{n} \sum_ {k} x _ {k} ^ {j} - \mu \| ^ {2} + \mathbb {E} \| d _ {b} ^ {i} \| ^ {2} \\ = (\frac {N - 1}{N}) ^ {2} \mathbb {E} \left(\alpha^ {i} \left(x ^ {i}\right) ^ {2}\right) + \frac {(N - 1) ^ {2}}{N ^ {2}} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \\ + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) + \frac {(N - 1)}{N ^ {2}} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) + \mathbb {E} \| d _ {b} ^ {i} \| ^ {2} \\ = (\frac {N - 1}{N}) ^ {2} \mathbb {E} \left(\alpha^ {i} \left(x ^ {i}\right) ^ {2}\right) + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} \left(\alpha^ {j} \left(x ^ {j}\right) ^ {2}\right) \\ + \frac {(N - 1)}{N n} \sigma^ {2} + \frac {N - 1}{N} \sigma_ {\star} ^ {2} + \mathbb {E} \| d _ {b} ^ {i} \| ^ {2} \tag {18} \\ \end{array}
$$
setting  $d_b^i = \frac{N - 1}{N} b^i (x^i) - \sum_{j\neq i}\frac{b^j(x^j)}{N}$ . The squared norm again factors because of the independence and zero means of both  $\xi^i$  and  $x_k^i -\mu$  and because  $\mathbb{E} < \bar{x}^{j} - \mu ,b^{j}(x^{j}) > = 0$ , while  $b^{j}(x^{j})$  is independent of  $\bar{x}^i -\mu$  for  $i\neq j$ . Now, inserting 18 in 17, we obtain
$$
\begin{array}{l} \frac {d}{d \mathbb {E} \alpha^ {i} (x ^ {i}) ^ {2}} \mathbb {E} \mathcal {R} _ {p} ^ {i} (\theta^ {1}, \dots , \theta^ {N}, \mu) = \sum_ {j \neq i} \frac {(1 - \beta^ {i}) ^ {2}}{(N - 1) N ^ {2}} - C \left(\frac {N - 1}{N}\right) ^ {2} \\ \leq \frac {1}{N ^ {2}} - C \frac {(N - 1) ^ {2}}{N ^ {2}} \\ = \frac {1}{N ^ {2}} \left((1 - C (N - 1) ^ {2})\right) \tag {19} \\ \end{array}
$$
As  $\beta^i\in [0,1]$ . Thus  $\mathbb{E}\frac{d}{d\alpha^i}\mathcal{R}_p^i$  is negative whenever  $C(N - 1)^2 >1$  or  $C > \frac{1}{(N - 1)^2}$ . Correspondingly, for such  $C$ , player  $i$  is incentivized to set  $\alpha^i = 0$ , independent of other players' strategies.
Now, assuming  $b^{j}(x^{j}) = 0$  for all other players  $j \neq i$ , we also get
$$
\begin{array}{l} \frac {d}{d \mathbb {E} \| b ^ {i} \left(x ^ {i}\right) \| ^ {2}} \mathbb {E} \mathcal {R} _ {p} ^ {i} \left(\theta^ {1}, \dots , \theta^ {N}, \mu\right) = \sum_ {j \neq i} \frac {\left(1 - \beta^ {i}\right) ^ {2}}{(N - 1) N ^ {2}} - C \left(\frac {N - 1}{N}\right) ^ {2} \\ \leq \frac {1}{N ^ {2}} - C \frac {(N - 1) ^ {2}}{N ^ {2}} \\ = \frac {1}{N ^ {2}} \left((1 - C (N - 1) ^ {2})\right). \tag {20} \\ \end{array}
$$
Correspondingly,  $b^{i} = 0$  for all players is a Nash Equilibrium. As the penalty  $p^i (m^1,\dots,m^N)$  does not depend on the defense strategies  $\beta^i$ , the optimal  $\beta^i$  at the equilibrium  $\alpha^i = 0$  and  $b^{i} = 0$  can still be calculated using 15 and is equal to zero as well.
The average penalty honest players pay at the Nash equilibrium is then given by  $C\frac{(N - 1)}{N}\left(\frac{\sigma^2}{n} + \sigma_\star^2\right)$ , according to 18.
To understand participation incentives, we compare the equilibrium reward  $\mathcal{R}^i (\theta^1,\dots ,\theta^N,\mu)$  player  $i$  receives if they do not participate while all other players do, to the penalized reward  $\mathcal{R}_p^i$  player  $i$
would obtain when participating. We first calculate  $\mathcal{R}^i (\theta^1,\dots ,\theta^N,\mu)$  assuming player  $i$  only uses their own estimate and does not send an update to the server, while the other  $N - 1$  players participate in the penalized game at equilibrium:
$$
\begin{array}{l} \mathbb {E} \mathcal {R} ^ {i} \left(\theta^ {1} (N - 1), \dots , \theta^ {i} (1), \dots , \theta^ {N} (N - 1), \mu\right) = \mathbb {E} \left(\frac {\sum_ {j \neq i} \| \theta^ {j} (N - 1) - \mu \| ^ {2}}{N - 1} - \lambda \| \theta^ {i} (1) - \mu \| ^ {2}\right) \\ = \frac {\sigma^ {2}}{(N - 1) n} + \frac {\sigma_ {\star} ^ {2}}{(N - 1)} - \lambda_ {i} \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \tag {21} \\ \end{array}
$$
using 4.1 with  $\alpha^j = b^j = \beta^j = 0$  and substituting  $N - 1$  and 1 for  $N$  respectively. We then calculate  $\mathbb{R}_p^i$ .
$$
\begin{array}{l} \mathbb {E} \mathcal {R} _ {p} ^ {i} = \mathbb {E} \left(\frac {\sum_ {j \neq i} \| \theta^ {j} - \mu \| ^ {2}}{N - 1} - \lambda_ {i} \| \theta^ {i} - \mu \| ^ {2} - C \| m ^ {i} - \theta^ {s} \| ^ {2}\right) \\ = \left(1 - \lambda_ {i}\right) \left(\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N}\right) - C \left(\frac {(N - 1)}{N n} \sigma^ {2} + \frac {(N - 1)}{N} \sigma_ {\star} ^ {2}\right) \tag {22} \\ \end{array}
$$
using 4.1 with  $\alpha^j = b^j = \beta^j = 0$  and D.1. The difference between these can be calculated as
$$
\begin{array}{l} \mathbb {E} \mathcal {R} _ {p} ^ {i} - \mathbb {E} \mathcal {R} ^ {i} (\theta^ {1} (N - 1), \ldots , \theta^ {i} (1), \ldots , \theta^ {N} (N - 1), \mu) \\ = (1 - \lambda_ {i}) (\frac {\sigma^ {2}}{N n} + \frac {\sigma_ {\star} ^ {2}}{N}) - C \frac {(N - 1)}{N} (\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}) - \frac {1}{N - 1} (\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}) + \lambda_ {i} (\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}) \\ = \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \left((1 - \lambda_ {i}) \frac {1}{N} - C \frac {N - 1}{N} - \frac {1}{N - 1} + \lambda_ {i}\right) \\ = \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \left(\frac {1}{N} - C \frac {N - 1}{N} - \frac {1}{N - 1} + \frac {N - 1}{N} \lambda_ {i}\right) \\ = \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \left(\frac {1}{N} - \frac {1}{N - 1} + \frac {N - 1}{N} (\lambda_ {i} - C)\right) \\ \geq \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \left(- \frac {1}{N - 1} + \frac {N - 1}{N} (\lambda_ {i} - C)\right) \\ = \left(\frac {\sigma^ {2}}{n} + \sigma_ {\star} ^ {2}\right) \left(\frac {N - 1}{N} \left(\lambda_ {i} - C - \frac {N}{(N - 1) ^ {2}}\right)\right) \\ \end{array}
$$
and is positive whenever  $\lambda_{i} > C + \frac{N}{(N - 1)^{2}}$ , such that in these cases player  $i$  is better off participating in data sharing, despite the penalties. If  $N = 2$ , we instead obtain
$$
\mathbb {E} \mathcal {R} _ {p} ^ {i} - \mathbb {E} \mathcal {R} ^ {i} (\theta^ {1} (N - 1), \dots , \theta^ {i} (1), \dots , \theta^ {N} (N - 1), \mu) = \frac {\sigma^ {2}}{n} \left(- \frac {1}{2} + \frac {1}{2} (\lambda_ {i} - C)\right)
$$
which is positive whenever  $\lambda_{i} > C + 1$
We now prove 5.1:
Theorem D.1. In the setting of 4.1, the penalized game with rewards
$$
\mathcal {R} _ {p ^ {\prime}} ^ {i} = \frac {\sum_ {j \neq i} \| \theta^ {j} - \mu \| ^ {2}}{N - 1} - \lambda_ {i} \| \theta^ {i} - \mu \| ^ {2} - C \| m ^ {i} - \theta^ {s} \| ^ {2} + \frac {1}{N - 1} \sum_ {j \neq i} C \| m ^ {j} - \bar {m} \| ^ {2}
$$
has a Nash equilibrium consisting of the strategies  $\alpha^j = b^j = \beta^j = 0$  for all  $j$  whenever  $C > \frac{1}{(N - 1)^2 - 1}$ . Furthermore, this equilibrium maximizes the sum of all players' rewards among equilibria whenever  $\lambda_i \geq 1$  for all players.
At this equilibrium, the expected penalty  $p^i(m^1, \ldots, m^N)$  paid by each player  $i$  is equal to 0, such that player  $i$  is incentivized to participate in the penalized game rather than relying on their own estimate, whenever  $N > 2$ , the other  $N - 1$  players participate at the honest equilibrium, and  $\lambda_i > \frac{N}{(N - 1)^2}$ .
Proof. Analogous to the proof of D.1 We have that