SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \| \theta_ {t + 1} - \theta_ {t + 1} ^ {\prime} \| ^ {2} = \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \nabla f (\theta_ {t}) - \nabla f (\theta_ {t} ^ {\prime}) \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \bar {e} _ {t} (\theta_ {t}) - \bar {e} _ {t} (\theta_ {t} ^ {\prime}) \| ^ {2} \\ + \frac {\gamma_ {t} ^ {2}}{N ^ {2}} \mathbb {E} \| \left(\alpha_ {t} ^ {i} - \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) \xi_ {t} ^ {i} \| ^ {2} + 2 \gamma_ {t} \mathbb {E} <   \theta_ {t} - \theta_ {t} ^ {\prime}, \nabla f \left(\theta_ {t} ^ {\prime}\right) - \nabla f (\theta_ {t}) > \\ = \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \nabla f (\theta_ {t}) - \nabla f (\theta_ {t} ^ {\prime}) \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \bar {e} _ {t} (\theta_ {t}) - \bar {e} _ {t} (\theta_ {t} ^ {\prime}) \| ^ {2} + \frac {\gamma_ {t} ^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \\ + 2 \gamma_ {t} \mathbb {E} <   \theta_ {t} - \theta_ {t} ^ {\prime}, \nabla f \left(\theta_ {t} ^ {\prime}\right) > + 2 \gamma_ {t} \mathbb {E} <   \theta_ {t} ^ {\prime} - \theta_ {t}, \nabla f (\theta_ {t}) > \\ \leq \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \nabla f (\theta_ {t}) - \nabla f (\theta_ {t} ^ {\prime}) \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \bar {e} _ {t} (\theta_ {t}) - \bar {e} _ {t} (\theta_ {t} ^ {\prime}) \| ^ {2} + \frac {\gamma_ {t} ^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \\ + 2 \gamma_ {t} \mathbb {E} \left(f \left(\theta_ {t}\right) - f \left(\theta_ {t} ^ {\prime}\right) - \frac {m}{2} \left\| \theta_ {t} - \theta_ {t} ^ {\prime} \right\| ^ {2} + f \left(\theta_ {t} ^ {\prime}\right) - f \left(\theta_ {t}\right) - \frac {m}{2} \left\| \theta_ {t} - \theta_ {t} ^ {\prime} \right\| ^ {2}\right) \\ = \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \nabla f (\theta_ {t}) - \nabla f (\theta_ {t} ^ {\prime}) \| ^ {2} + \gamma_ {t} ^ {2} \mathbb {E} \| \bar {e} _ {t} (\theta_ {t}) - \bar {e} _ {t} (\theta_ {t} ^ {\prime}) \| ^ {2} + \frac {\gamma_ {t} ^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \\ - 2 \gamma_ {t} m \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} \\ \leq \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \gamma_ {t} ^ {2} B ^ {2} \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \gamma_ {t} ^ {2} (B ^ {\prime}) ^ {2} \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \frac {\gamma_ {t} ^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \\ - 2 \gamma_ {t} m \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} \\ = \left(1 + \gamma_ {t} ^ {2} \left(B ^ {2} + B ^ {\prime 2}\right) - 2 \gamma_ {t} m\right) \mathbb {E} \| \theta_ {t} - \theta_ {t} ^ {\prime} \| ^ {2} + \frac {\gamma_ {t} ^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \tag {25} \\ \end{array}
$$
Where the first inequality follows from strong convexity.
Now for a constant learning rate  $\gamma = \gamma_{t}$  and  $c = (1 + \gamma^{2}(B^{2} + B^{\prime 2}) - 2\gamma m)$ :
$$
\begin{array}{l} \left(\mathbb {E} \frac {1}{L} | f \left(\theta_ {T + 1}\right) - f \left(\theta_ {T + 1} ^ {\prime}\right) |\right) ^ {2} \leq \left(\mathbb {E} \| \theta_ {T + 1} - \theta_ {T + 1} ^ {\prime} \|\right) ^ {2} \\ \leq \mathbb {E} \| \theta_ {T + 1} - \theta_ {T + 1} ^ {\prime} \| ^ {2} \\ \leq c ^ {T} \mathbb {E} \| \theta_ {1} - \theta_ {1} ^ {\prime} \| ^ {2} + \sum_ {t = 1} ^ {T} c ^ {T - t} \frac {\gamma^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \\ = \sum_ {t = 1} ^ {T} c ^ {T - t} \frac {\gamma^ {2}}{N ^ {2}} \delta_ {t} ^ {i} \\ \end{array}
$$
$$
\mathbb {E} \left| f \left(\theta_ {T + 1}\right) - f \left(\theta_ {T + 1} ^ {\prime}\right) \right| \leq L \sqrt {\sum_ {t = 1} ^ {T} c ^ {T - t} \frac {\gamma^ {2}}{N ^ {2}} \delta_ {t} ^ {i}} \leq L \sum_ {t = 1} ^ {T} \frac {\gamma}{N} c ^ {\frac {T - t}{2}} \sqrt {\delta_ {t} ^ {i}} = \frac {L \gamma}{N} \sum_ {t = 1} ^ {T} c ^ {\frac {T - t}{2}} \left| \alpha_ {t} ^ {i} - \left(\alpha_ {t} ^ {i}\right) ^ {\prime} \right|.
$$
Where the last inequality follows from the general inequality  $\sqrt{\sum_{i}x_{i}}\leq \sum_{i}\sqrt{x_{i}}$  for  $x_{i}\geq 0$ , which inductively follows from
$$
\sqrt {x + y} = \sqrt {(\sqrt {x} + \sqrt {y}) ^ {2} - 2 \sqrt {x y}} \leq \sqrt {(\sqrt {x} + \sqrt {y}) ^ {2}} = \sqrt {x} + \sqrt {y}.
$$
In particular if we set  $\theta_t^\prime$  to the trajectory in which player  $i$  is honest  $((\alpha_t^i)' = 0)$ , we obtain
$$
\mathbb {E} \left| f \left(\theta_ {T + 1}\right) - f \left(\theta_ {T + 1} ^ {\prime}\right) \right| \leq \frac {L \gamma}{N} \sum_ {t = 1} ^ {T} c ^ {\frac {T - t}{2}} \alpha_ {t} ^ {i}. \tag {26}
$$
The same inequalities hold for players' final estimates  $\theta_{T + 1}^{j}$  and  $(\theta_{T + 1}^{j})'$ , as the noise correction step  $\theta_{T + 1}^{i} = \theta_{T + 1} - \frac{\alpha_{T}^{i}}{N}\xi^{i}$  is the same in both cases, so that
$$
\mathbb {E} \| \theta_ {T + 1} ^ {i} - (\theta_ {T + 1} ^ {i}) ^ {\prime} \| ^ {2} = \mathbb {E} \| \theta_ {T + 1} - (\theta_ {T + 1}) ^ {\prime} \| ^ {2}.
$$
It is worth noting, that the contribution of noise at early time steps to the sum diminishes exponentially as long as  $c < 1$  which is true for  $\gamma^2 (B^2 + B'^2) - 2\gamma m < 0$ , i.e.  $\gamma < \frac{2m}{B^2 + B'^2}$ .
Next, we consider the expected difference in penalties received by player  $i$  at time  $t$  if they use  $(\alpha_{t}^{i})'$  rather than  $\alpha_{t}^{i}$  and thus send message  $(m_t^i)'$  rather than  $m_t^i$ :
$$
\begin{array}{l} \mathbb {E} p _ {i} ^ {t} \left(m _ {t} ^ {1}, \dots , \left(m _ {t} ^ {i}\right) ^ {\prime}, \dots , m _ {t} ^ {N}\right) - p _ {i} ^ {t} \left(m _ {t} ^ {1}, \dots , m _ {t} ^ {i}, \dots , m _ {t} ^ {N}\right) \\ = \mathbb {E} \| \frac {N - 1}{N} \left(m _ {t} ^ {i}\right) ^ {\prime} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} - \mathbb {E} \| \frac {N - 1}{N} m _ {t} ^ {i} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} \\ = \mathbb {E} \| \frac {N - 1}{N} ((\alpha_ {t} ^ {i}) ^ {\prime} \xi_ {t} ^ {i} + g _ {t} ^ {i}) - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} - \mathbb {E} \| \frac {N - 1}{N} (\alpha_ {t} ^ {i} \xi_ {t} ^ {i} + g _ {t} ^ {i}) - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} \\ = \mathbb {E} \| \frac {N - 1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime} \xi_ {t} ^ {i} \| ^ {2} - \mathbb {E} \| \frac {N - 1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime} \xi_ {t} ^ {i} \| ^ {2} \\ + \mathbb {E} \| \frac {N - 1}{N} (g _ {t} ^ {i}) - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} - \mathbb {E} \| \frac {N - 1}{N} (g _ {t} ^ {i}) - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} \\ = \left(\frac {N - 1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \left(\frac {N - 1}{N} \alpha_ {t} ^ {i}\right) ^ {2} \\ \end{array}
$$
for  $g_{t}^{i} = g_{t}(\theta_{t - 1}^{s},x^{i})$
In particular, we can bound the difference between expected penalized rewards for two trajectories with all  $\alpha_{k}^{i}$  fixed but two different values  $\alpha_{t}^{i}$  and  $(\alpha_{t}^{i})'$  varying for  $k = t$  as follows:
$$
\begin{array}{l} \mathbb {E} \left(\mathcal {R} ^ {i} \left(\theta_ {T + 1}\right) - \sum_ {k} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , m _ {k} ^ {i}, \dots , m _ {k} ^ {N}\right) \right. \\ \left. - \mathcal {R} ^ {i} \left(\left(\theta_ {T + 1}\right) ^ {\prime}\right) + \sum_ {k} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , \left(m _ {k} ^ {i}\right) ^ {\prime}, \dots , m _ {k} ^ {N}\right)\right) \\ = \frac {1}{N - 1} \sum_ {j \neq i} (f (\theta_ {T + 1} ^ {j}) - f ((\theta_ {T + 1} ^ {j}) ^ {\prime})) + C _ {t} ((\frac {N - 1}{N} (\alpha_ {t} ^ {i}) ^ {\prime}) ^ {2} - (\frac {N - 1}{N} \alpha_ {t} ^ {i}) ^ {2}) \\ \leq L \frac {\gamma}{N} c ^ {\frac {T - t}{2}} \left| \alpha_ {t} ^ {i} - \left(\alpha_ {t} ^ {i}\right) ^ {\prime} \right| + C _ {t} \left(\left(\frac {N - 1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \left(\frac {N - 1}{N} \alpha_ {t} ^ {i}\right) ^ {2}\right) \\ \end{array}
$$
In particular, for  $(\alpha_{i}^{t})^{\prime} = 0$  we obtain
$$
\begin{array}{l} \mathbb {E} \left(\mathcal {R} ^ {i} \left(\theta_ {T}\right) - \sum_ {k} C _ {k} p _ {i} ^ {t} \left(m _ {k} ^ {1}, \dots , m _ {k} ^ {i}, \dots , m _ {k} ^ {N}\right) - \mathcal {R} ^ {i} \left(\left(\theta_ {T}\right) ^ {\prime}\right) + \sum_ {t} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , \left(m _ {k} ^ {i}\right) ^ {\prime}, \dots , m _ {k} ^ {N}\right)\right) \\ \leq L \frac {\gamma}{N} c ^ {\frac {T - t}{2}} \alpha_ {t} ^ {i} - C _ {t} \left(\left(\frac {N - 1}{N} \alpha_ {t} ^ {i}\right) ^ {2}\right) \\ \end{array}
$$
By the quadratic formula, this is zero at zero and at
$$
\alpha_ {t} ^ {i} = \frac {- 2 L \frac {\gamma}{N} c ^ {\frac {T - t}{2}}}{- 2 C _ {t} (\frac {N - 1}{N}) ^ {2}} = \frac {L N c ^ {\frac {T - t}{2}} \gamma}{C _ {t} (N - 1) ^ {2}} \tag {27}
$$
and because of the negative quadratic term negative whenever  $\alpha_{t}^{i} > \frac{LNc^{\frac{T - t}{2}\gamma}}{C_{t}(N - 1)^{2}}$ . Correspondingly, in terms of penalized reward players are always better off by not adding any noise at all  $\alpha_{t}^{i} = 0$  compared to adding large noise, such that rational players will never choose  $\alpha_{t}^{i} > \frac{LNc^{\frac{T - t}{2}\gamma}}{C_{t}(N - 1)^{2}}$ . Therefore, the noise  $\alpha_{t}^{i}$  added by any player  $i$  at a given time step  $t$  can be limited to any fixed constant  $\epsilon > 0$  by choosing  $C_{t}$  such that  $\frac{LNc^{\frac{T - t}{2}\gamma}}{C_{t}(N - 1)^{2}} \leq \epsilon$ , i.e.  $C_{t} \geq \frac{LNc^{\frac{T - t}{2}\gamma}}{\epsilon(N - 1)^{2}}$ .
Applying this observation to each player, substituting into Equation (26) and using the triangle inequality, the overall damage caused by all  $N$  players compared to full honesty can then be bounded by
$$
\mathbb {E} \left| f \left(\theta_ {T + 1}\right) - f \left(\theta_ {T + 1} ^ {\prime}\right) \right| \leq L \gamma \sum_ {t = 1} ^ {T} c ^ {\frac {T - t}{2}} \epsilon
$$
where  $\theta_t'$  represents the fully honest strategy and  $\theta_t$  represents a strategy in which all players act rationally given the penalty magnitude  $C_t$ . Using a geometric series bound, we obtain  $\mathbb{E}|f(\theta_{T+1}) - f(\theta_{T+1}')| \leq \frac{1}{1-\sqrt{c}} L\gamma \epsilon$ .
Lastly, for a global bound on the "variance" of the gradients  $\| e_t^i (\theta)\| ^2\leq G^2$  we get,
$$
\begin{array}{l} \mathbb {E} p _ {i} ^ {t} \left(m _ {t} ^ {1}, \dots , m _ {t} ^ {i}, \dots , m _ {t} ^ {N}\right) \\ = \mathbb {E} \| \frac {N - 1}{N} m _ {t} ^ {i} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} \| ^ {2} \\ = \mathbb {E} \| \frac {N - 1}{N} \left(g _ {t} ^ {i} + \alpha_ {t} ^ {i} \xi_ {t} ^ {i}\right) - \frac {1}{N} \sum_ {j \neq i} \left(g _ {t} ^ {j} + \alpha_ {t} ^ {j} \xi_ {t} ^ {j}\right) \| ^ {2} \\ = \mathbb {E} \| \frac {N - 1}{N} g _ {t} ^ {i} - \frac {1}{N} \sum_ {j \neq i} g _ {t} ^ {j} \| ^ {2} + (\frac {N - 1}{N}) ^ {2} \left(\alpha_ {t} ^ {i}\right) ^ {2} + \frac {1}{N ^ {2}} \sum_ {j \neq i} \left(\alpha_ {t} ^ {j}\right) ^ {2} \\ \leq \mathbb {E} \| \frac {N - 1}{N} (g _ {t} ^ {i} (\theta) - \nabla f (\theta)) - \frac {1}{N} \sum_ {j \neq i} (g _ {t} ^ {j} (\theta) - \nabla f (\theta)) + (\frac {N - 1}{N} - \frac {N - 1}{N}) \nabla f (\theta) \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} + (\frac {N - 1}{N}) \epsilon^ {2} \\ = \mathbb {E} [ \mathbb {E} [ \| \frac {N - 1}{N} e _ {t} ^ {i} (\theta) - \frac {1}{N} \sum_ {j \neq i} e _ {t} ^ {j} (\theta) \| ^ {2} | \theta ] ] + (\frac {N - 1}{N}) \epsilon^ {2} \\ = (\frac {N - 1}{N}) ^ {2} \mathbb {E} [ \| e _ {t} ^ {i} (\theta) \| ^ {2} | \theta ] + \frac {1}{N ^ {2}} \sum_ {j \neq i} \mathbb {E} [ \| (e _ {t} ^ {j} (\theta) \| ^ {2} | \theta ] + (\frac {N - 1}{N}) \epsilon^ {2} \\ \leq \left(\frac {N - 1}{N}\right) \left(G ^ {2} + \epsilon^ {2}\right) \\ \end{array}
$$
as the  $\xi_t^i$  are independent with zero mean, and the gradient noise  $e_t^i (\theta)$  are independent with zero mean, given  $\theta$ . Correspondingly, for  $C_t = \frac{LNc^{\frac{T - t}{2}\gamma}}{\epsilon(N - 1)^2}$  the total expected penalties paid by player  $i$  can be bounded as
$$
\begin{array}{l} \mathbb {E} \sum_ {t} ^ {T} C _ {t} p _ {i} ^ {t} \left(m _ {t} ^ {1}, \dots , m _ {t} ^ {i}, \dots , m _ {t} ^ {N}\right) \\ \leq \sum_ {t} ^ {T} \frac {L c ^ {\frac {T - t}{2}} \gamma}{\epsilon (N - 1)} (K ^ {2} + \epsilon^ {2}) \\ \leq \frac {1}{1 - \sqrt {c}} \frac {L \gamma}{(N - 1)} (\frac {K ^ {2}}{\epsilon} + \epsilon) \\ \end{array}
$$
Next, we prove the budget-balanced version of E.1
Proposition E.2. Under the assumptions of E.1, in the balanced penalized game with reward
$$
\begin{array}{l} \mathcal {R} _ {p} ^ {i} (\theta_ {T + 1} ^ {1}, \dots , \theta_ {T + 1} ^ {N}, f) = \left(\frac {1}{N - 1} \sum_ {j \neq i} f (\theta_ {T + 1} ^ {j})\right) - \sum_ {t = 1} ^ {T} C _ {t} \| m _ {t} ^ {i} - \frac {1}{N} \sum_ {j} m _ {t} ^ {j} \| ^ {2} \\ + \frac {1}{N - 1} \sum_ {k \neq i} \sum_ {t = 0} ^ {T} C _ {t} \| m _ {t} ^ {k} - \frac {1}{N} \sum_ {j} m _ {t} ^ {j} \| ^ {2}, \\ \end{array}
$$
any player's best response strategy fulfills  $\alpha_{t}^{i}\leq \frac{Lc^{\frac{T - t}{2}}\gamma}{C_{t}(N - 2)}\leq \frac{L\gamma}{C_{t}(N - 2)}$  for  $c = (1 + \gamma^2 (B^2 +B'^2) - 2\gamma m)$ , independent of other players' strategies.
Given  $\epsilon > 0$ , the expected absolute change in function values  $f(\theta_T^j)$  due to noise added by players playing best responses compared to full honesty can be bounded by  $\frac{1}{1 - \sqrt{c}} L\gamma \epsilon$  by choosing  $C_t \geq \frac{Lc^{\frac{T - t}{2}\gamma}}{\epsilon(N - 2)} \leq \frac{L\gamma}{\epsilon(N - 2)}$ . As long as all players  $i$  choose the same strategy ( $\alpha_t^i = \alpha_t^j \forall i, j, t$ ), the expected total penalty paid by each player equals zero.
Proof. We begin by considering the expected difference in penalties received by player  $l$  at time  $t$  if player  $i$  uses  $(\alpha_{t}^{i})'$  rather than  $\alpha_{t}^{i}$  and thus send message  $(m_{t}^{i})'$  rather than  $m_{t}^{i}$ :
$$
\begin{array}{l} \mathbb {E} p _ {t} ^ {l} \left(m _ {t} ^ {1}, \dots , \left(m _ {t} ^ {i}\right) ^ {\prime}, \dots , m _ {t} ^ {N}\right) - p _ {t} ^ {l} \left(m _ {t} ^ {1}, \dots , m _ {t} ^ {i}, \dots , m _ {t} ^ {N}\right) \\ = \mathbb {E} \| m _ {t} ^ {l} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} - \frac {1}{N} (m _ {t} ^ {i}) ^ {\prime} \| ^ {2} - \mathbb {E} \| m _ {t} ^ {l} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} - \frac {1}{N} m _ {t} ^ {i} \| ^ {2} \\ = \mathbb {E} \| m _ {t} ^ {l} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} - \frac {1}{N} \left(\left(\alpha_ {t} ^ {i}\right) ^ {\prime} \xi_ {t} ^ {i} + g _ {t} ^ {i}\right) \| ^ {2} - \mathbb {E} \| m _ {t} ^ {l} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} - \frac {1}{N} \left(\alpha_ {t} ^ {i} \xi_ {t} ^ {i} + g _ {t} ^ {i}\right) \| ^ {2} \\ = \mathbb {E} \| m _ {t} ^ {l} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} - \frac {1}{N} g _ {t} ^ {i} \| ^ {2} + \mathbb {E} \frac {1}{N ^ {2}} \| (\alpha_ {t} ^ {i}) ^ {\prime} \xi_ {t} ^ {i} \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} - \mathbb {E} \| m _ {t} ^ {l} - \frac {1}{N} \sum_ {j \neq i} m _ {t} ^ {j} - \frac {1}{N} g _ {t} ^ {i} \| ^ {2} - \mathbb {E} \frac {1}{N ^ {2}} \| \alpha_ {t} ^ {i} \xi_ {t} ^ {i} \| ^ {2} \\ = (\frac {\left(\alpha_ {t} ^ {i}\right) ^ {\prime}}{N}) ^ {2} - (\frac {\alpha_ {t} ^ {i}}{N}) ^ {2} \\ \end{array}
$$
Again, we can bound the difference between expected penalized rewards for two trajectories with all  $\alpha_{k}^{i}$  fixed but two different values  $\alpha_{t}^{i}$  and  $(\alpha_{t}^{i})'$  varying for  $k = t$  as follows:
$$
\begin{array}{l} \mathbb {E} \left(\mathcal {R} ^ {i} \left(\theta_ {T + 1}\right) - \sum_ {k} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , m _ {k} ^ {i}, \dots , m _ {k} ^ {N}\right) \right. \\ + \frac {1}{N - 1} \sum_ {j \neq i} \sum_ {k} C _ {k} p _ {k} ^ {j} \left(m _ {k} ^ {1}, \dots , m _ {k} ^ {i}, \dots , m _ {k} ^ {N}\right) \\ - \mathcal {R} ^ {i} \left(\left(\theta_ {T + 1}\right) ^ {\prime}\right) + \sum_ {t} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , \left(m _ {k} ^ {i}\right) ^ {\prime}, \dots , m _ {k} ^ {N}\right) \\ - \frac {1}{N - 1} \sum_ {j \neq i} \sum_ {t} C _ {k} p _ {k} ^ {j} \left(m _ {k} ^ {1}, \dots , \left(m _ {k} ^ {i}\right) ^ {\prime}, \dots , m _ {k} ^ {N}\right)) \\ = \frac {1}{N - 1} \sum_ {j \neq i} \left(f \left(\theta_ {T + 1} ^ {j}\right) - f \left(\left(\theta_ {T + 1} ^ {j}\right) ^ {\prime}\right)\right) \\ + C _ {t} \left(\left(\frac {N - 1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \left(\frac {1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \left(\frac {N - 1}{N} \alpha_ {t} ^ {i}\right) ^ {2}\right) + \left(\frac {1}{N} \alpha_ {t} ^ {i}\right) ^ {2}) \\ = \frac {1}{N - 1} \sum_ {j \neq i} \left(f \left(\theta_ {T + 1} ^ {j}\right) - f \left(\left(\theta_ {T + 1} ^ {j}\right) ^ {\prime}\right)\right) \\ + C _ {t} \left(\left(\frac {N - 1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \left(\frac {1}{N} \left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \left(\frac {N - 1}{N} \alpha_ {t} ^ {i}\right) ^ {2}\right) + \left(\frac {1}{N} \alpha_ {t} ^ {i}\right) ^ {2}) \\ \leq L \frac {\gamma}{N} c ^ {\frac {T - t}{2}} \left| \alpha_ {t} ^ {i} - \left(\alpha_ {t} ^ {i}\right) ^ {\prime} \right| + C _ {t} \left(\frac {N - 2}{N} \left(\left(\alpha_ {t} ^ {i}\right) ^ {\prime}\right) ^ {2} - \frac {N - 2}{N} \left(\alpha_ {t} ^ {i}\right) ^ {2}\right) \\ \end{array}
$$
Again, for  $(\alpha_{t}^{i})^{\prime} = 0$  we obtain
$$
\begin{array}{l} \mathbb {E} \left(\mathcal {R} ^ {i} \left(\theta_ {T + 1}\right) - \sum_ {k} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , m _ {k} ^ {i}, \dots , m _ {k} ^ {N}\right) \right. \\ + \frac {1}{N - 1} \sum_ {j \neq i} \sum_ {k} C _ {k} p _ {k} ^ {j} \left(m _ {k} ^ {1}, \dots , m _ {k} ^ {i}, \dots , m _ {k} ^ {N}\right) \\ - \mathcal {R} ^ {i} ((\theta T + 1) ^ {\prime}) + \sum_ {k} C _ {k} p _ {k} ^ {i} \left(m _ {k} ^ {1}, \dots , \left(m _ {k} ^ {i}\right) ^ {\prime}, \dots , m _ {k} ^ {N}\right) \\ - \frac {1}{N - 1} \sum_ {j \neq i} \sum_ {k} C _ {k} p _ {k} ^ {j} \left(m _ {k} ^ {1}, \dots , \left(m _ {k} ^ {i}\right) ^ {\prime}, \dots , m _ {k} ^ {N}\right)) \\ \leq L \frac {\gamma}{N} c ^ {\frac {T - t}{2}} \alpha_ {t} ^ {i} - C _ {t} \left(\frac {(N - 2)}{N} \left(\alpha_ {t} ^ {i}\right) ^ {2}\right) \\ \end{array}
$$
which is zero at zero and at
$$
\alpha_ {t} ^ {i} = \frac {- 2 L \frac {\gamma}{N} c ^ {\frac {T - t}{2}}}{- 2 C _ {t} \frac {N - 2}{N}} = \frac {L c ^ {\frac {T - t}{2}} \gamma}{C _ {t} (N - 2)} \tag {28}
$$
and negative for  $\alpha_{t}^{i} > \frac{Lc^{\frac{T - t}{2}\gamma}}{C_{t}(N - 2)}$  as long as  $N > 2$ . Players are thus again incentivized to select  $\alpha_{t}^{i}$  that do not fulfill that inequality.
As in E.1, the overall damage caused by all  $N$  players compared to full honesty can then be bounded by  $\mathbb{E}|f(\theta_{T + 1}) - f(\theta_{T + 1}^{\prime})|\leq \frac{1}{1 - \sqrt{c}} L\gamma \epsilon$  by choosing  $C_t\geq \frac{Lc^{\frac{T - t}{2}\gamma}}{\epsilon(N - 2)}$  where  $\theta_T^\prime$  represents the fully
honest strategy and  $\theta_T$  represents a strategy in which all players act rationally given the penalty magnitude  $C_t$ , and the same is true for player's estimates  $\theta_{T+1}^j$  and  $(\theta_{T+1}^j)'$ . By symmetry, as long as all players  $i$  choose the same strategy ( $\alpha_t^i = \alpha_t^j \forall i, j, t$ ), the expected penalties paid by each player equal 0.
To prove 6.1, we adapt a classic result from convex optimization to give convergence rates for SGD with bounded perturbations from the clients and with a linearly decaying learning rate.
Lemma E.1. In the settings of E.2, assume that all players use bounded attacks, so that  $(\alpha_{t}^{i})^{2}\leq \epsilon^{2}$  for all  $i,t$ . Also, assume that there exist scalars  $M\geq 0$  and  $M_V\geq 0$ , such that for all  $t$ :
$$
\mathbb {E} _ {s _ {i}} \left(\| e _ {t} ^ {i} \left(\theta_ {t} ^ {s}\right) \| ^ {2}\right) = \mathbb {E} _ {x ^ {i}} \left(\| g _ {t} \left(\theta_ {t} ^ {s}, x ^ {i}\right) \| ^ {2}\right) - \| \mathbb {E} _ {x ^ {i}} g _ {t} \left(\theta_ {t} ^ {s}, x ^ {i}\right) \| ^ {2} \leq M + M _ {V} \| \nabla f \left(\theta_ {t} ^ {s}\right) \| _ {2} ^ {2}. \tag {29}
$$
Assume that for some integer constant  $\eta > 0$ , such that  $\frac{4}{\eta m + m} \leq \frac{1}{B(M_V / N + 1)}$ , the learning rate is set as  $\gamma_t = \frac{4}{\eta m + tm}$ . In that case, if  $P(\exists t \leq T : \Pi_W(\theta_t^s - \gamma_t \bar{m}_t) \neq \theta_t^s - \gamma_t \bar{m}_t) \in O(\frac{1}{NT})$  we get  $\mathbb{E}\left(f(\theta_t) - f(\theta^*)\right) \in O\left(\frac{1 + M + \epsilon^2}{Nt}\right) + O\left(\frac{1}{t^2}\right)$ , we have:
$$
\mathbb {E} \left(f \left(\theta_ {T}\right) - f \left(\theta^ {*}\right)\right) \leq \frac {8 B \left(M + \epsilon^ {2}\right)}{3 m ^ {2} N T} + \mathcal {O} \left(\frac {1}{N T}\right) + \mathcal {O} \left(\frac {1}{T ^ {2}}\right) \tag {30}
$$
for any  $T\geq \eta$
Proof. We first condition on the case in which there is no  $t \leq T$  with  $\Pi_W(\theta_t^s - \gamma_t \bar{m}_t) \neq \theta_t^s - \gamma_t \bar{m}_t$ , so that we do not have to worry about projections. Then, for a random vector  $g$ , denote  $\mathbb{V}(g) = \mathbb{E}\left(\| g\|^2\right) - \| \mathbb{E}(g)\|^2$ . Note that, by the independence of the stochastic gradients and players' noise, it follows that:
$$
\begin{array}{l} \mathbb {V} \left(\frac {1}{N} \sum_ {i = 1} ^ {N} m _ {i} ^ {t}\right) = \mathbb {E} \| \frac {1}{N} \sum_ {i = 1} ^ {N} \left(\nabla f \left(\theta_ {t}\right) + e _ {t} ^ {i} \left(\theta_ {t} ^ {s}\right) + \alpha_ {t} ^ {i} \xi_ {t} ^ {i}\right) \| ^ {2} - \| \mathbb {E} \left(\frac {1}{N} \sum_ {i = 1} ^ {N} \left(\nabla f \left(\theta_ {t}\right) + e _ {t} ^ {i} \left(\theta_ {t} ^ {s}\right) + \alpha_ {t} ^ {i} \xi_ {t} ^ {i}\right)\right) \| ^ {2} \\ = \frac {1}{N ^ {2}} \mathbb {E} \| \sum_ {i = 1} ^ {N} (\nabla f (\theta_ {t}) + e _ {t} ^ {i} (\theta_ {t} ^ {s}) + \alpha_ {t} ^ {i} \xi_ {t} ^ {i}) \| ^ {2} - \frac {1}{N ^ {2}} | | \nabla f (\theta_ {t}) | | ^ {2} \\ = \frac {1}{N ^ {2}} (\mathbb {E} \| \sum_ {i = 1} ^ {N} e _ {t} ^ {i} (\theta_ {t} ^ {s}) \| ^ {2} + \mathbb {E} \| \sum_ {i = 1} ^ {N} \alpha_ {t} ^ {i} \xi_ {t} ^ {i} \| ^ {2} + | | \nabla f (\theta_ {t}) | | ^ {2}) - \frac {1}{N ^ {2}} | | \nabla f (\theta_ {t}) | | ^ {2} \\ = \frac {1}{N ^ {2}} \left(\sum_ {i = 1} ^ {N} \mathbb {E} \| e _ {t} ^ {i} \left(\theta_ {t} ^ {s}\right) \| ^ {2} + \sum_ {i = 1} ^ {N} \left(\alpha_ {t} ^ {i}\right) ^ {2} \mathbb {E} \| \xi_ {t} ^ {i} \| ^ {2}\right) \\ \leq \frac {1}{N ^ {2}} \left(\sum_ {i = 1} ^ {N} M + M _ {V} \| \nabla f \left(\theta_ {t} ^ {s}\right)\right) \| _ {2} ^ {2} + \sum_ {i = 1} ^ {N} \left(\alpha_ {t} ^ {i}\right) ^ {2}) \\ \leq \frac {M + \epsilon^ {2}}{N} + \frac {M _ {V}}{N} \| \nabla F (\theta_ {t} ^ {s}) \| _ {2} ^ {2} \\ \end{array}
$$
Additionally, since  $f$  is strongly convex, it has a unique minimizer  $\theta^{*} \in \mathbb{R}^{d}$ . Now since the learning rate is of the form  $\gamma_{t} = \frac{4 / m}{\eta + t}$ , with  $\frac{4}{m} > \frac{1}{m}$ ,  $\eta > 0$  and  $\gamma_{1} = \frac{\beta}{\eta + 1} \leq \frac{1}{B(M_{V} / N + 1)}$ , the conditions of Theorem 4.7 in Bottou et al. (2018) hold with  $\mu = 1$ ,  $M_V = M_V / N$ ,  $M = (M + \epsilon^2) / N$ ,  $M_G = M_V / N + 1$ . Using equation 4.23 in their proof gives:
$$
\mathbb {E} \left(f \left(\theta_ {t + 1}\right) - f \left(\theta^ {*}\right)\right) \leq \left(1 - \frac {4}{\eta + t}\right) \mathbb {E} \left(f \left(\theta_ {t}\right) - f \left(\theta^ {*}\right)\right) + \frac {8 B \left(M + \epsilon^ {2}\right)}{N m ^ {2} (\eta + t) ^ {2}} \tag {31}
$$
for any  $t \geq 1$ . We now use a classic result by Chung:
Lemma E.2 (Chung (1954)). Let  $\{b_n\}_{n\geq 1}$  be a sequence of real numbers, such that for some  $n_0\in \mathbb{N}$ , it holds that for all  $n\geq n_0$