SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $I_{d}$  denotes the identity matrix of size  $d$ . The above, in conjunction with (30), implies that  $\mathcal{L}_{S_1}$  is  $\left(\frac{\alpha}{1 + B^2}\right)$ -smooth  $\left(\frac{\alpha}{1 + B^2}\right)$ -strongly convex. As  $\frac{\alpha}{1 + B^2} = \mu$  (see (27)), we deduce that  $\mathcal{L}_{S_1}$  is  $\mu$ -smooth  $\mu$ -strong convexity. Recall that  $\mu \leq L$ , therefore  $\mathcal{L}_{S_1}$  is also  $L$ -smooth. Next, by invoking Lemma 2, we show that the local losses for the honest workers in this scenario also satisfy  $(G, B)$ -dissimilarity.
We start by analyzing below the terms  $A_{1}, A_{2}$  and  $A_{3}$  introduced in Lemma 2 in this scenario.
Term  $A_{1}$ . Recall from (14) in Lemma 2 that
Let  $A_1' \coloneqq \frac{(n - f)^2}{f(n - 2f)} A_1$ . Substituting in the above, from (28),  $K = \frac{1 + B^2}{1 - \frac{f}{n - 2f}B^2} \cdot \frac{1}{\alpha}$ , we obtain that
$$
\begin{array}{l} A _ {1} ^ {\prime} = \left(\left(1 - \frac {n - 2 f}{f} B ^ {2}\right) \left(\frac {1 - \frac {f}{n - 2 f} B ^ {2}}{1 + B ^ {2}}\right) ^ {2} \alpha^ {2} - 2 (1 + B ^ {2}) \frac {1 - \frac {f}{n - 2 f} B ^ {2}}{1 + B ^ {2}} \alpha^ {2} + \alpha^ {2} \left(1 - \frac {f}{n - 2 f} B ^ {2}\right)\right) \\ = \alpha^ {2} \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) \left(\frac {\left(1 - \frac {n - 2 f}{f} B ^ {2}\right) \left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{(1 + B ^ {2}) ^ {2}} - 1\right) \\ = \alpha^ {2} \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} \left(\left(1 - \frac {n - 2 f}{f} B ^ {2}\right) \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) - \left(1 + B ^ {2}\right) ^ {2}\right) \\ = \alpha^ {2} \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} \left(\left(1 - \frac {n - 2 f}{f} B ^ {2} - \frac {f}{n - 2 f} B ^ {2} + B ^ {4}\right) - \left(1 + 2 B ^ {2} + B ^ {4}\right)\right) \\ = - \alpha^ {2} \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} B ^ {2} \left(2 + \frac {n - 2 f}{f} + \frac {f}{n - 2 f}\right) = - \alpha^ {2} \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right) B ^ {2}}{\left(1 + B ^ {2}\right) ^ {2}} \frac {(n - f) ^ {2}}{f (n - 2 f)}. \\ \end{array}
$$
Recall that  $A_1' = \frac{(n - f)^2}{f(n - 2f)} A_1$ . Therefore, from the above we obtain that
$$
A _ {1} = - \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} B ^ {2} \alpha^ {2}. \tag {32}
$$
Term  $A_{2}$  .From (14) in Lemma 2, and (30), we obtain that
$$
A _ {2} = \frac {f (n - 2 f) \alpha}{(n - f) ((n - 2 f) \frac {1}{K} + f \alpha)} \left(\frac {1}{K} - \alpha\right) \frac {1}{K} z = \frac {f (n - 2 f)}{(n - f) ^ {2}} \left(1 + B ^ {2}\right) \left(\frac {1}{K} - \alpha\right) \frac {1}{K} z.
$$
From (28), we obtain that  $\alpha -\frac{1}{K} = \alpha -\frac{1 - \frac{f}{n - 2f}B^2}{1 + B^2}\alpha = \frac{n - f}{n - 2f}\frac{B^2}{1 + B^2}\alpha$ . Using this above we obtain that
$$
\left\| A _ {2} \right\| ^ {2} = \left(\frac {f (n - 2 f)}{(n - f) ^ {2}}\right) ^ {2} (1 + B ^ {2}) ^ {2} \left(\frac {n - f}{n - 2 f}\right) ^ {2} \frac {B ^ {4}}{(1 + B ^ {2}) ^ {2}} \alpha^ {2} \frac {\| z \| ^ {2}}{K ^ {2}} = \left(\frac {f}{n - f}\right) ^ {2} B ^ {4} \alpha^ {2} \frac {\| z \| ^ {2}}{K ^ {2}}. \tag {33}
$$
Term  $A_{3}$ . From (14) in Lemma 2, and (30), we obtain that
$$
A _ {3} = \frac {f (n - 2 f) \alpha^ {2}}{\left((n - 2 f) \frac {1}{K} + f \alpha\right) ^ {2}} \frac {\| z \| ^ {2}}{K ^ {2}} = \frac {f (n - 2 f)}{(n - f) ^ {2}} \left(1 + B ^ {2}\right) ^ {2} \frac {\| z \| ^ {2}}{K ^ {2}}. \tag {34}
$$
Substituting in the above, from (29),  $\| z \|^2 = \frac{(n - f)^2}{f(n - 2f)} \frac{\left(1 - \frac{f}{n - 2f}B^2\right)}{(1 + B^2)^2} G^2 K^2$  we obtain that
$$
A _ {3} = \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) G ^ {2}. \tag {35}
$$
**Invoking Lemma 2.** Using the results obtained above we show below that the conditions stated in (15) of Lemma (2) are satisfied, i.e.,  $A_1 \leq 0$ ,  $A_3 \leq G^2$  and  $\| A_2 \|^2 \leq A_1(A_3 - G^2)$ . Hence, proving that the local loss functions for the honest workers in this particular scenario satisfy  $(G, B)$ -gradient dissimilarity.
Since we assumed that  $B^2 < \frac{n - 2f}{f}$ , (32) implies that  $A_1 \leq 0$ . Similarly, (35) implies that  $A_3 \leq G^2$ . Substituting, from (32) and (34), respectively,  $A_1 = -\frac{\left(1 - \frac{f}{n - 2f}B^2\right)}{(1 + B^2)^2}B^2\alpha^2$  and  $A_3 = \frac{f(n - 2f)}{(n - f)^2}\left(1 + B^2\right)^2\frac{\|z\|^2}{K^2}$ , we obtain that
$$
A _ {1} (A _ {3} - G ^ {2}) = \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} B ^ {2} \alpha^ {2} \left(G ^ {2} - \frac {f (n - 2 f)}{(n - f) ^ {2}} \left(1 + B ^ {2}\right) ^ {2} \frac {\| z \| ^ {2}}{K ^ {2}}\right).
$$
Substituting in the above, from (29), i.e.  $\| z \|^2 = \frac{(n - f)^2}{f(n - 2f)} \frac{\left(1 - \frac{f}{n - 2f}B^2\right)}{(1 + B^2)^2} G^2 K^2$ , we obtain that
$$
\begin{array}{l} A _ {1} \left(A _ {3} - G ^ {2}\right) = \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} B ^ {2} \alpha^ {2} \left(G ^ {2} - \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) G ^ {2}\right) \\ = \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} \left(\frac {f}{n - 2 f}\right) G ^ {2} B ^ {4} \alpha^ {2} = \frac {(n - f) ^ {2}}{f (n - 2 f)} \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{\left(1 + B ^ {2}\right) ^ {2}} G ^ {2} \left(\frac {f}{n - f}\right) ^ {2} B ^ {4} \alpha^ {2}. \\ \end{array}
$$
Recall that  $\frac{(n - f)^2}{f(n - 2f)}\frac{\left(1 - \frac{f}{n - 2f}B^2\right)}{(1 + B^2)^2} G^2 = \frac{\|z\|^2}{K^2}$ . Using this above, and then comparing the resulting equation with (33), we obtain that
$$
A _ {1} \left(A _ {3} - G ^ {2}\right) = \left(\frac {f}{n - f}\right) ^ {2} B ^ {4} \alpha^ {2} \frac {\| z \| ^ {2}}{K ^ {2}} = \| A _ {2} \| ^ {2}.
$$
Second scenario. Consider the set of honest workers to be  $S_{2} = \{f + 1,\dots ,n\}$ . Identical to (31) with  $z = 0$ , from (12) and (13), we obtain that
$$
\mathcal {L} _ {S _ {2}} (\theta) = \frac {f}{n - f} \frac {\alpha}{2} \| \theta \| ^ {2} + \frac {n - 2 f}{n - f} \frac {1}{2 K} \| \theta \| ^ {2}.
$$
Similar to the analysis in the first scenario, we deduce that  $\mathcal{L}_{S_2}$  is  $L$ -smooth and  $\mu$ -strongly convex. Moreover, Lemma 3, in conjunction with the deduction in (32) that  $A_{1} \geq 0$  (recall that  $B^{2} < \frac{n - 2f}{f}$ ), implies that the loss functions for the honest workers in this particular scenario satisfy  $(0, B)$ -gradient dissimilarity, thereby also satisfying  $(G, B)$ -gradient dissimilarity.
This concludes the analysis for the second scenario. We have shown that the conditions on smoothness, strong convexity and  $(G,B)$ -gradient dissimilarity hold in this scenario.
Final step: lower bound on  $\varepsilon$  in terms of  $G$  and  $B$ . We have established in the above that the conditions of smoothness, strong convexity and  $(G,B)$ -gradient dissimilarity are satisfied in both the scenarios. Therefore, by assumptions, algorithm  $\mathcal{A}$  must guarantee  $(f,\varepsilon)$ -resilience in either scenario. Specifically, the output of  $\mathcal{A}$ , denoted by  $\hat{\theta}$  must satisfy the following:
Due to Lemma 1, the above holds only if
$$
\varepsilon \geq \frac {\left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2}}{8 \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right)} \| z \| ^ {2}. \tag {36}
$$
Recall, from (30), that  $\frac{n - 2f}{n - f}\frac{1}{K} +\frac{f}{n - f}\alpha = \frac{\alpha}{1 + B^2}$ . Using this in (36), we obtain that
$$
\varepsilon \geq \frac {\left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2}}{8 \left(\frac {\alpha}{1 + B ^ {2}}\right)} \| z \| ^ {2} = \frac {1}{8} \left(\frac {f}{n - f}\right) ^ {2} \alpha \left(1 + B ^ {2}\right) \| z \| ^ {2}
$$
Substituting, from (27),  $\alpha = \mu (1 + B^2)$  in the above implies that
$$
\varepsilon \geq \frac {\mu}{8} \left(\frac {f}{n - f}\right) ^ {2} (1 + B ^ {2}) ^ {2} \| z \| ^ {2}.
$$
Substituting, from (29),  $\| z \|^2 = \frac{(n - f)^2}{f(n - 2f)} \frac{\left(1 - \frac{f}{n - 2f} B^2\right)}{(1 + B^2)^2} G^2 K^2$  in the above implies that
$$
\begin{array}{l} \varepsilon \geq \frac {\mu}{8} \left(\frac {f}{n - f}\right) ^ {2} (1 + B ^ {2}) ^ {2} \frac {(n - f) ^ {2}}{f (n - 2 f)} \frac {\left(1 - \frac {f}{n - 2 f} B ^ {2}\right)}{(1 + B ^ {2}) ^ {2}} G ^ {2} K ^ {2} \\ = \frac {\mu}{8} \left(\frac {f}{n - 2 f}\right) \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) G ^ {2} K ^ {2}. \\ \end{array}
$$
Substituting, from (27), i.e.  $K = \frac{1}{\mu\left(1 - \frac{f}{n - 2f}B^{2}\right)}$ , in the above implies that
$$
\varepsilon \geq \frac {\mu}{8} \left(\frac {f}{n - 2 f}\right) \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) \frac {G ^ {2}}{\mu^ {2} \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) ^ {2}} = \frac {1}{8 \mu} \cdot \frac {\frac {f}{n - 2 f} G ^ {2}}{1 - \frac {f}{n - 2 f} B ^ {2}}
$$
The above completes the proof.
Let us recall the lemma below.
$$
\max \left\{\mathcal {L} _ {S _ {1}} (\theta) - \mathcal {L} _ {*, S _ {1}}, \mathcal {L} _ {S _ {2}} (\theta) - \mathcal {L} _ {*, S _ {2}} \right\} \geq \frac {\left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2}}{8 \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right)} \| z \| ^ {2}.
$$
Proof. We prove the lemma by contradiction. Suppose there exists a parameter vector  $\hat{\theta}$  such that
$$
\left. \right. \max  \left\{\mathcal {L} _ {S _ {1}} (\hat {\theta}) - \mathcal {L} _ {*, S _ {1}}, \mathcal {L} _ {S _ {2}} (\hat {\theta}) - \mathcal {L} _ {*, S _ {2}} \right\} <   \frac {\left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2}}{8 \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right)} \| z \| ^ {2} =: \delta . \tag {37}
$$
From (11), (12) and (13), we obtain that
$$
\mathcal {L} _ {S _ {1}} (\theta) := \frac {1}{n - f} \left(f \frac {\alpha}{2} \| \theta - z \| ^ {2} + (n - 2 f) \frac {1}{2 K} \| \theta \| ^ {2}\right), \text {a n d}
$$
$$
\mathcal {L} _ {S _ {2}} (\theta) := = \frac {1}{n - f} \left(f \frac {\alpha}{2} \| \theta \| ^ {2} + (n - 2 f) \frac {1}{2 K} \| \theta \| ^ {2}\right).
$$
$$
\mathcal {L} _ {S _ {1}} (\hat {\theta}) - \mathcal {L} _ {*, S _ {1}} = \frac {1}{2} \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right) \left\| \hat {\theta} - \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z \right\| ^ {2}, \text {a n d}
$$
$$
\mathcal {L} _ {S _ {2}} (\hat {\theta}) - \mathcal {L} _ {*, S _ {2}} = \frac {1}{2} \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right) \left\| \hat {\theta} \right\| ^ {2}.
$$
Substituting from the above in (37), we obtain that
$$
\delta > \frac {1}{2} \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right) \max  \left\{\left\| \hat {\theta} - \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z \right\| ^ {2}, \left\| \hat {\theta} \right\| ^ {2} \right\}.
$$
As for any real values  $a, b$ , we have  $\max \{a, b\} \geq \frac{1}{2}(a + b)$ , from above we obtain that
$$
\delta > \frac {1}{4} \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right) \left(\left\| \hat {\theta} - \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z \right\| ^ {2} + \left\| \hat {\theta} \right\| ^ {2}\right). \tag {38}
$$
By triangle and Jensen's inequalities, we have
$$
\left\| \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z \right\| ^ {2} = \left\| \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z - \hat {\theta} + \hat {\theta} \right\| ^ {2} \leq 2 \left\| \hat {\theta} - \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z \right\| ^ {2} + 2 \left\| \hat {\theta} \right\| ^ {2}.
$$
4For arbitrary positive real values  $a$  and  $b$ , and an arbitrary  $z \in \mathbb{R}^d$ , consider a loss function  $\mathcal{L}(\theta) := \frac{a}{2}\|\theta - z\|^2 + \frac{b}{2}\|\theta\|^2$ . The minimum point  $\theta_*$  of  $\mathcal{L}(\theta)$  is given by  $\theta_* = \frac{a}{a + b}z$ , and for any  $\theta, \mathcal{L}(\theta) - \mathcal{L}(\theta_*) = \frac{1}{2}(a + b)\|\theta - \theta_*\|^2$ .
Substituting from the above in (38), we obtain that
$$
\begin{array}{l} \delta > \frac {1}{8} \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right) \left\| \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z \right\| ^ {2} \\ = \frac {1}{8} \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right) \left\| \frac {\frac {f}{n - f} \alpha}{\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha} z \right\| ^ {2} = \frac {\left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2}}{8 \left(\frac {n - 2 f}{n - f} \frac {1}{K} + \frac {f}{n - f} \alpha\right)} \| z \| ^ {2} = \delta . \\ \end{array}
$$
The contradiction above proves the lemma.
Proof. Let  $\theta \in \mathbb{R}^d$ . As  $\frac{1}{|S_1|}\sum_{i\in S_1}\|\nabla\mathcal{L}_i(\theta) - \nabla\mathcal{L}_{S_1}(\theta)\|^2 = \frac{1}{|S_1|}\sum_{i\in S_1}\|\nabla\mathcal{L}_i(\theta)\|^2 - \|\nabla\mathcal{L}_{S_1}(\theta)\|^2$ , we obtain that
$$
\begin{array}{l} \frac {1}{| S _ {1} |} \sum_ {i \in S _ {1}} \| \nabla \mathcal {L} _ {i} (\theta) - \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} - \left(G ^ {2} + B ^ {2} \| \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2}\right) \\ = \frac {1}{\left| S _ {1} \right|} \sum_ {i \in S _ {1}} \| \nabla \mathcal {L} _ {i} (\theta) \| ^ {2} - (1 + B ^ {2}) \| \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} - G ^ {2}. \tag {39} \\ \end{array}
$$
We analyze the right-hand side of the above equality. As  $S_{1} = \{1,\dots ,n - f\}$ , from (11) and (12), we obtain that
$$
\begin{array}{l} \frac {1}{| S _ {1} |} \sum_ {i \in S _ {1}} \| \nabla \mathcal {L} _ {i} (\theta) \| ^ {2} = \frac {n - 2 f}{n - f} \| \nabla \mathcal {L} _ {\mathrm {I I}} (\theta) \| ^ {2} + \frac {f}{n - f} \| \nabla \mathcal {L} _ {\mathrm {I}} (\theta) \| ^ {2} \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \| \theta \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \| \theta - z \| ^ {2}. \tag {40} \\ \end{array}
$$
$$
\begin{array}{l} \nabla \mathcal {L} _ {S _ {1}} (\theta) = \frac {1}{| S _ {1} |} \sum_ {i \in S _ {1}} \nabla \mathcal {L} _ {i} (\theta) = \frac {n - 2 f}{n - f} \frac {1}{K} \theta + \frac {f}{n - f} \alpha (\theta - z) \\ = \frac {(n - 2 f) \frac {1}{K} + f \alpha}{n - f} \theta - \frac {f}{n - f} \alpha z = \underbrace {\frac {(n - 2 f) \frac {1}{K} + f \alpha}{n - f}} _ {A _ {0}} \left(\theta - \underbrace {\frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z} _ {\theta_ {*}}\right). \\ \end{array}
$$
Denoting in the above
$$
A _ {0} := \frac {(n - 2 f) \frac {1}{K} + f \alpha}{n - f} \tag {41}
$$
$$
\theta_ {*} := \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z, \tag {42}
$$
$$
\nabla \mathcal {L} _ {S _ {1}} (\theta) = A _ {0} (\theta - \theta_ {*}).
$$
The above implies that  $\theta_{*}$  is the minimizer of the convex function  $\mathcal{L}_{S_1}$ , and
$$
\left\| \nabla \mathcal {L} _ {S _ {1}} (\theta) \right\| ^ {2} = A _ {0} ^ {2} \| \theta - \theta_ {*} \| ^ {2}. \tag {43}
$$
Substituting from (40) and (43) in (39) implies that
$$
\begin{array}{l} \frac {1}{| S _ {1} |} \sum_ {i \in S _ {1}} \| \nabla \mathcal {L} _ {i} (\theta) - \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} - G ^ {2} - B ^ {2} \| \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \| \theta \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \| \theta - z \| ^ {2} - (1 + B ^ {2}) A _ {0} ^ {2} \| \theta - \theta_ {*} \| ^ {2} - G ^ {2}. \\ \end{array}
$$
Now, we operate the change of variables  $X = \theta -\theta_{*}$  , and rewrite the above as
$$
\begin{array}{l} \frac {1}{| S _ {1} |} \sum_ {i \in S _ {1}} \| \nabla \mathcal {L} _ {i} (\theta) - \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} - G ^ {2} - B ^ {2} \| \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \| X + \theta_ {*} \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \| X + \theta_ {*} - z \| ^ {2} - (1 + B ^ {2}) A _ {0} ^ {2} \| X \| ^ {2} - G ^ {2} \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \left(\| X \| ^ {2} + \| \theta_ {*} \| ^ {2} + 2 \langle X, \theta_ {*} \rangle\right) + \frac {f}{n - f} \alpha^ {2} \left(\| X \| ^ {2} + \| \theta_ {*} - z \| ^ {2} + 2 \langle X, \theta_ {*} - z \rangle\right) \\ - \left(1 + B ^ {2}\right) A _ {0} ^ {2} \| X \| ^ {2} - G ^ {2} \\ = \left(\underbrace {\frac {(n - 2 f) \frac {1}{K ^ {2}} + f \alpha^ {2}}{n - f} - (1 + B ^ {2}) A _ {0} ^ {2}} _ {A _ {1}}\right) \| X \| ^ {2} + 2 \left\langle X, \underbrace {\frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \theta_ {*}} _ {A _ {2}} + \frac {f}{n - f} \alpha^ {2} (\theta_ {*} - z) \right\rangle \\ + \underbrace {\frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \left\| \theta_ {*} \right\| ^ {2} + \frac {f}{n - f} \alpha^ {2} \left\| \theta_ {*} - z \right\| ^ {2}} _ {A _ {3}} - G ^ {2}. \tag {44} \\ \end{array}
$$
Next, we show that  $A_{1}$ ,  $A_{2}$  and  $A_{3}$  as defined in (14) can be equivalently written as follows.
$$
A _ {1} = \frac {(n - 2 f) \frac {1}{K ^ {2}} + f \alpha^ {2}}{n - f} - (1 + B ^ {2}) A _ {0} ^ {2}, \quad A _ {2} = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \theta_ {*} + \frac {f}{n - f} \alpha^ {2} (\theta_ {*} - z) \quad \text {a n d}
$$
$$
A _ {3} = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \| \theta_ {*} \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \| \theta_ {*} - z \| ^ {2}.
$$
Term  $A_{1}$ : Substituting from (41), i.e.  $A_0 = \frac{(n - 2f)\frac{1}{K} + f\alpha}{n - f}$ , we obtain that
$$
\begin{array}{l} \frac {(n - 2 f) \frac {1}{K ^ {2}} + f \alpha^ {2}}{n - f} - (1 + B ^ {2}) A _ {0} ^ {2} = \frac {(n - 2 f) \frac {1}{K ^ {2}} + f \alpha^ {2}}{n - f} - (1 + B ^ {2}) \left(\frac {(n - 2 f) \frac {1}{K} + f \alpha}{n - f}\right) ^ {2} \\ = \frac {(n - 2 f) \frac {1}{K ^ {2}} + f \alpha^ {2}}{n - f} - (1 + B ^ {2}) \left(\left(\frac {n - 2 f}{n - f}\right) ^ {2} \frac {1}{K ^ {2}} + \left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2} + \frac {2 f (n - 2 f)}{(n - f) ^ {2}} \frac {\alpha}{K}\right) \\ = \left(\frac {n - 2 f}{n - f}\right) ^ {2} \left(\frac {n - f}{n - 2 f} - (1 + B ^ {2})\right) \frac {1}{K ^ {2}} - 2 (1 + B ^ {2}) \frac {f (n - 2 f)}{(n - f) ^ {2}} \frac {\alpha}{K} \\ + \left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2} \left(\frac {n - f}{f} - (1 + B ^ {2})\right) \\ = \left(\frac {n - 2 f}{n - f}\right) ^ {2} \left(\frac {f}{n - 2 f} - B ^ {2}\right) \frac {1}{K ^ {2}} - 2 (1 + B ^ {2}) \frac {f (n - 2 f)}{(n - f) ^ {2}} \frac {\alpha}{K} + \left(\frac {f}{n - f}\right) ^ {2} \alpha^ {2} \left(\frac {n - 2 f}{f} - B ^ {2}\right) \\ = \frac {f (n - 2 f)}{(n - f) ^ {2}} \left(\left(1 - \frac {n - 2 f}{f} B ^ {2}\right) \frac {1}{K ^ {2}} - 2 (1 + B ^ {2}) \frac {\alpha}{K} + \left(1 - \frac {f}{n - 2 f} B ^ {2}\right) \alpha^ {2}\right). \\ \end{array}
$$
Comparing the above with  $A_{1}$  defined in (14) implies that
$$
\frac {(n - 2 f) \frac {1}{K ^ {2}} + f \alpha^ {2}}{n - f} - (1 + B ^ {2}) A _ {0} ^ {2} = A _ {1}. \tag {45}
$$
Term  $A_{2}$ : Substituting from (42), i.e.  $\theta_{*} = \frac{f\alpha}{(n - 2f)^{\frac{1}{K}} + f\alpha} z$ , we obtain that
$$
\begin{array}{l} \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \theta_ {*} + \frac {f}{n - f} \alpha^ {2} (\theta_ {*} - z) \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z + \frac {f}{n - f} \alpha^ {2} (\frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z - z) \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} z - \frac {f}{n - f} \alpha^ {2} \frac {(n - 2 f) \frac {1}{K}}{(n - 2 f) \frac {1}{K} + f \alpha} z \\ = \frac {f (n - 2 f) \alpha}{(n - f) ((n - 2 f) \frac {1}{K} + f \alpha)} \left(\frac {1}{K} - \alpha\right) \frac {1}{K} z. \\ \end{array}
$$
Comparing the above with  $A_{2}$  defined in (14) implies that
$$
\frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \theta_ {*} + \frac {f}{n - f} \alpha^ {2} \left(\theta_ {*} - z\right) = A _ {2}. \tag {46}
$$
Term  $A_{3}$ : Similarly, substituting  $\theta_{*} = \frac{f\alpha}{(n - 2f)^{\frac{1}{K}} + f\alpha} z$ , we obtain that
$$
\begin{array}{l} \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \| \theta_ {*} \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \| \theta_ {*} - z \| ^ {2} \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \left(\frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha}\right) ^ {2} \| z \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \left(\frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha} - 1\right) ^ {2} \| z \| ^ {2} \\ = \frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \left(\frac {f \alpha}{(n - 2 f) \frac {1}{K} + f \alpha}\right) ^ {2} \| z \| ^ {2} + \frac {f}{n - f} \alpha^ {2} \left(\frac {(n - 2 f) \frac {1}{K}}{(n - 2 f) \frac {1}{K} + f \alpha}\right) ^ {2} \| z \| ^ {2} \\ = \frac {f (n - 2 f) \alpha^ {2}}{\left((n - 2 f) \frac {1}{K} + f \alpha\right) ^ {2}} \frac {\| z \| ^ {2}}{K ^ {2}}. \\ \end{array}
$$
Comparing the above with  $A_{3}$  defined in (14) implies that
$$
\frac {n - 2 f}{n - f} \frac {1}{K ^ {2}} \left\| \theta_ {*} \right\| ^ {2} + \frac {f}{n - f} \alpha^ {2} \left\| \theta_ {*} - z \right\| ^ {2} = A _ {3}. \tag {47}
$$
Substituting from (45), (46) and (47) in (44), we obtain that
$$
\frac {1}{| S _ {1} |} \sum_ {i \in S _ {1}} \| \nabla \mathcal {L} _ {i} (\theta) - \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} - G ^ {2} - B ^ {2} \| \nabla \mathcal {L} _ {S _ {1}} (\theta) \| ^ {2} = A _ {1} \| X \| ^ {2} + 2 \langle X, A _ {2} \rangle + A _ {3} - G ^ {2}.
$$
Therefore, by Assumption 1, the honest workers represented by set  $S_{1}$  satisfy  $(G,B)$ -gradient dissimilarity if and only if the right-hand side of the above equations is less than or equal to 0, i.e.,
$$
A _ {1} \| X \| ^ {2} + 2 \langle X, A _ {2} \rangle + A _ {3} - G ^ {2} \leq 0, \quad \text {f o r a l l} X = \theta - \theta_ {*} \in \mathbb {R} ^ {d}. \tag {48}
$$
To show the above, we consider an auxiliary second-order "polynomial" in  $x \in \mathbb{R}^d$ ,  $P(x) \coloneqq a\|x\|^2 + 2\langle x, b \rangle + c$  where  $b \in \mathbb{R}^d$  and  $a, c \in \mathbb{R}$ . We show below that  $P(x) \leq 0$  for all  $x \in \mathbb{R}^d$  if and only if  $a, c \leq 0$  and  $\Delta \coloneqq \|b\|^2 - ac \leq 0$ .
Case 1. Let  $a \neq 0$ . In this particular case, we have  $P(x) = a\left(\left\| x + \frac{1}{a} \cdot b \right\|^2 - \frac{\Delta}{a^2}\right)$ . Therefore,  $P(x) \leq 0$  for all  $x$  if and only if  $a < 0$  and  $c, \Delta \leq 0$ .
Case 2. Let  $a = 0$ . In this particular case,  $P(x) = 2\langle x,b\rangle +c$ . Therefore,  $P(x)\leq 0$  for all  $x$  if and only if  $c\leq 0$  and  $\Delta = \| b\| ^2 = 0$
Hence, (48) is equivalent to
$$
A _ {1} \leq 0, \quad A _ {3} - G ^ {2} \leq 0, \quad \text {a n d} \quad \| A _ {2} \| ^ {2} - A _ {1} (A _ {3} - G ^ {2}) \leq 0.
$$
C Proofs of Theorem 2 and Corollary 1: Convergence Results
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \| \nabla \mathcal {L} _ {\mathcal {H}} (\theta_ {t}) \| ^ {2} \leq \frac {\kappa G ^ {2}}{1 - \kappa B ^ {2}} + \frac {2 L (\mathcal {L} _ {\mathcal {H}} (\theta_ {0}) - \mathcal {L} _ {\ast , \mathcal {H}})}{(1 - \kappa B ^ {2}) T}.
$$
C.1.1 Non-convex Case
Proof. As  $\mathcal{L}_{\mathcal{H}}$  is assumed  $L$ -smooth, for all  $\theta, \theta' \in \mathbb{R}^d$ , we have (see Definition 2)
$$
\mathcal {L} _ {\mathcal {H}} (\theta^ {\prime}) - \mathcal {L} _ {\mathcal {H}} (\theta) \leq \langle \nabla \mathcal {L} _ {\mathcal {H}} (\theta), \theta^ {\prime} - \theta \rangle + \frac {L}{2} \| \theta^ {\prime} - \theta \| ^ {2}.
$$
Let  $t \in \{0, \dots, T - 1\}$ . From Algorithm 1, recall that  $\theta_{t + 1} = \theta_t - \gamma R_t$ . Hence, substituting in the above  $\theta = \theta_t$  and  $\theta' = \theta_{t + 1}$ , we have
$$
\mathcal {L} _ {\mathcal {H}} \left(\theta_ {t + 1}\right) - \mathcal {L} _ {\mathcal {H}} \left(\theta_ {t}\right) \leq - \gamma \left\langle \nabla \mathcal {L} _ {\mathcal {H}} \left(\theta_ {t}\right), R _ {t} \right\rangle + \frac {1}{2} \gamma^ {2} L \| R _ {t} \| ^ {2}. \tag {49}
$$
As  $\langle a, b \rangle = \frac{1}{2} \left( \|a\|^2 + \|b\|^2 - \|a - b\|^2 \right)$  for any  $a, b \in \mathbb{R}^d$ , we also have