SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {P} \left(\left| \Lambda \right| \leq 1. 7 5 m \mid X \in X _ {\epsilon}\right) = \mathbb {P} \left(\exists \mathcal {J} \subset [ 2 m ] _ {p}, | \mathcal {J} | = 0. 2 5 m: j \notin \Lambda \forall j \in \mathcal {J} \mid X \in X _ {\epsilon}\right) \\ \leq \left( \begin{array}{c} m \\ 0. 2 5 m \end{array} \right) \epsilon^ {0. 2 5 m} \\ \leq \left(4 \epsilon e\right) ^ {0. 2 5 m}. \\ \end{array}
$$
It follows that there exists a sufficiently small positive constant  $c$  such that
$$
4 \epsilon e = \exp (- c n + 1 + \log (4)) \leq \exp (- c n).
$$
Therefore, using Lemma A.4 there exists a sufficiently small positive constant  $c$  such that
$$
\begin{array}{l} \mathbb {P} (| \Lambda | > 1. 7 5 m) \geq \mathbb {P} (| \Lambda | > 1. 7 5 m \mid X \in X _ {\epsilon}) \mathbb {P} (X \in X _ {\epsilon}) \\ \geq (1 - \exp (- c 0. 2 5 m n)) (1 - \exp (- c n)) \\ \geq (1 - \exp (- c n)) \\ \end{array}
$$
from which the result claimed follows.
Lemma A.6. Assume  $\gamma \leq \frac{4}{5n}$ ,  $\gamma \geq 5\rho$  and  $|\Gamma_p| > 0.99m$  for  $p \in \{-1,1\}$ . There exists a positive constant  $C$  such that for any  $\delta \in (0,1)$ , if  $n \geq C\log \left(\frac{1}{\delta}\right)$  and  $m \geq C\log \left(\frac{k}{\delta}\right)$ , then with probability at least  $1 - \delta$  for all  $i \in S_F$  there exists a  $j \in \Theta_{y_i}$  for which  $i \in \mathcal{A}_j^{(0)}$ .
Proof. For convenience here we use  $T_{j}$ ,  $G_{j}$  and  $B_{j}$  for  $T_{j}(0,1)$ ,  $G_{j}(0,1)$  and  $B_{j}(0,1)$  respectively. For a neuron to be in  $\Theta_{p}$  it must satisfy the following condition,
$$
G _ {j} (\gamma + \rho) - B _ {j} (\gamma - \rho) <   1 - \gamma + \rho .
$$
Adding and subtracting  $T_{j}(\gamma - \rho)$  to the left-hand-side gives
$$
G _ {j} 2 \gamma - T _ {j} (\gamma - \rho) <   1 - \gamma + \rho .
$$
Rearranging this inequality it follows that the conditions  $j \in \Gamma_p$  and  $G_j < \frac{\gamma - \rho}{2\gamma} T_j + \frac{1}{2\gamma}$  are sufficient to conclude  $j \in \Theta_p$ . Let  $\Lambda := \{j \in [2m] : G_j < \frac{\gamma - \rho}{2\gamma} T_j + \frac{1}{2\gamma}\}$ . Therefore, in order to prove the desired result it suffices to lower bound the probability that for each corrupt point  $(\mathbf{x}_i, y_i)$ , the intersection between the set of neurons which  $\mathbf{x}_i$  activates, the set of neurons  $\Gamma_{y_i}$  and the set of neurons  $\Lambda$  is nonempty.
By a Chernoff bound there exists a small constant  $c > 0$  such that with probability at least  $1 - \exp(-cm)$  a fixed training point is activated by at least  $1/3$  of the neurons of each sign. Therefore, using the union bound, every corrupt training point is activated by at least  $m/3$  of the neurons with matching sign with probability at least  $1 - 2k\exp(-cm)$ . Conditioning on this event, then under the assumption  $|\Gamma_p| > 0.99m$  for  $p \in \{-1,1\}$ , each corrupt point  $(\mathbf{x}_i,y_i)$  activates at least  $\frac{97}{300}m$  neurons in  $\Gamma_{y_i}$ . Therefore, if for instance,  $|\Lambda| > 1.75m$ , we can conclude for each  $(\mathbf{x}_i,y_i)$  with  $i \in S_F$  that there exists a  $j \in \Theta_{y_i}$  such that  $i \in \mathcal{A}_j^{(0)}$ . Therefore, under the conditions of the lemma, using the union bound and Lemmas A.5 and A.3, we can upper bound the failure probability of this as
$$
2 k \exp (- c m) + \exp (- c n).
$$
Therefore, for  $\delta \in (0,1)$  there exists a positive constant  $C$  such that if  $n\geq C\log (\frac{1}{\delta})$  and  $m\geq$ $C\log \left(\frac{k}{\delta}\right)$  then the probability that for all  $i\in S_F$  there exists a  $j\in \Theta_{y_i}$  such that  $i\in \mathcal{A}_j^{(0)}$  is at least  $1 - \delta$
The final lemma we provide here states, under mild conditions on the network width, that with high probability every point in the training sample activates a neuron whose output weight matches its label in sign. We use this to prove the result on non-benign overfitting, detailed in Section D.
Lemma A.7. Let  $\delta \in (0,1)$ , if  $m \geq \log_2\left(\frac{2n}{\delta}\right)$  then the probability that for all  $i \in [2n]$  there exists a  $j \in [2m]$  such that  $(-1)^j = y_i$  and  $i \in \mathcal{A}_j^{(0)}$  is at least  $1 - \delta$ .
Proof. Observe by the rotational symmetry of the weight distribution that for any  $j \in [2m]$
$$
\mathbb {P} \left(\left\langle \mathbf {w} _ {j}, \mathbf {x} _ {i} \right\rangle <   0\right) = \mathbb {P} \left(\left\langle \mathbf {w} _ {j}, \mathbf {e} _ {1} \right\rangle <   0\right) = 1 / 2.
$$
By construction, for each element in the training sample  $(\mathbf{x}_i,y_i)_{i = 1}^{2n}$  there are  $m$  neurons whose output weight has the same sign. As the preactivations of  $\mathbf{x}_i$  with each neuron are mutually independent from one another, then using the union bound it follows that
$$
\begin{array}{l} \mathbb {P} \left(\bigcap_ {i = 1} ^ {2 n} \{\exists j \in [ 2 m ]: (- 1) ^ {j} = y _ {i}, i \in \mathcal {A} _ {j} ^ {(0)} \}\right) = 1 - \mathbb {P} \left(\bigcup_ {i = 1} ^ {2 n} \{\nexists j \in [ 2 m ]: (- 1) ^ {j} = y _ {i}, i \in \mathcal {A} _ {j} ^ {(0)} \}\right) \\ \geq 1 - 2 n \left(\mathbb {P} \left(\left\langle \mathbf {w} _ {j}, \mathbf {x} _ {i} \right\rangle <   0\right)\right) ^ {m} \\ = 1 - 2 n 2 ^ {- m}. \\ \end{array}
$$
Setting  $\delta \geq 2n2^{-m}$  and rearranging we arrive at the stated result.
Appendix B Supporting Lemmas
B.1 Bounds on activations and preactivations
For any pair of iterations  $t, t_0$  satisfying  $t > t_0$ , unrolling the GD update rule (3) gives
$$
\mathbf {w} _ {j} ^ {(t)} = \mathbf {w} _ {j} ^ {(t _ {0})} + (- 1) ^ {j} \eta \sum_ {\ell = 1} ^ {2 n} T _ {\ell j} (t _ {0}, t) y _ {\ell} \mathbf {x} _ {\ell}.
$$
Using (1) and the fact that  $\mathbf{n}_i\perp \mathbf{v}$  for any  $i\in [2n]$ , then
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle = \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle + (- 1) ^ {j} \eta \sum_ {\ell = 1} ^ {2 n} T _ {\ell j} (t _ {0}, t) y _ {\ell} \langle \mathbf {x} _ {\ell}, \mathbf {x} _ {i} \rangle \\ = \left\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \right\rangle + (- 1) ^ {j + i} \eta \sum_ {\ell = 1} ^ {2 n} T _ {\ell j} (t _ {0}, t) (- 1) ^ {\ell + i} \beta (\ell) \left\langle \mathbf {x} _ {\ell}, \mathbf {x} _ {i} \right\rangle \tag {5} \\ = \left\langle \mathbf {w} _ {j} ^ {\left(t _ {0}\right)}, \mathbf {x} _ {i} \right\rangle + (- 1) ^ {j + i} \beta (i) \eta \sum_ {\ell = 1} ^ {2 n} T _ {\ell j} \left(t _ {0}, t\right) \lambda_ {i \ell}, \\ \end{array}
$$
where we define  $\lambda_{i\ell} \coloneqq (-1)^{\ell + i}\beta(i)\beta(\ell)\langle \mathbf{x}_{\ell}, \mathbf{x}_{i} \rangle$ . Towards the goal of bounding the activation of a neuron with a data point we provide the following results.
Lemma B.1. Assume  $|\langle \mathbf{n}_i,\mathbf{n}_{\ell}\rangle |\leq \frac{\rho}{1 - \gamma}$  for all  $i,\ell \in [2n]$  such that  $i\neq \ell$
1. If  $i = \ell$  then  $\lambda_{i\ell} = 1$
2. If  $i \neq \ell$ ,  $i \in S_T$ , and  $\ell \in S_F$ , then  $-( \gamma + \rho) \leq \lambda_{i\ell} \leq - (\gamma - \rho)$ .
3. If  $i \neq \ell$ ,  $i \in S_F$ , and  $\ell \in S_T$ , then  $-( \gamma + \rho) \leq \lambda_{i\ell} \leq -(\gamma - \rho)$ .
4. If  $i \neq \ell$  and  $i, \ell \in S_T$ , then  $\gamma - \rho \leq \lambda_{i\ell} \leq \gamma + \rho$ .
5. If  $i \neq \ell$  and  $i, \ell \in S_F$ , then  $\gamma - \rho \leq \lambda_{i\ell} \leq \gamma + \rho$ .
Proof. Observe by the data model that
$$
\langle \mathbf {x} _ {i}, \mathbf {x} _ {\ell} \rangle = (- 1) ^ {\ell + i} (\gamma + (1 - \gamma) \beta (i) \beta (\ell) \langle \mathbf {n} _ {i}, \mathbf {n} _ {\ell} \rangle).
$$
$$
\lambda_ {i \ell} = \beta (i) \beta (\ell) \gamma + (1 - \gamma) \langle \mathbf {n} _ {i}, \mathbf {n} _ {\ell} \rangle
$$
from which the results claimed follow.
Lemma B.2. Assume  $|\langle \mathbf{n}_i, \mathbf{n}_{\ell} \rangle| \leq \frac{\rho}{1 - \gamma}$  for all  $i, \ell \in [2n]$  such that  $i \neq \ell$ . Then for any  $j \in [2m]$  the following are true.
1. If  $i \in S_T$ ,  $i \sim j$  then
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \geq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle + \eta \left(T _ {i j} (t _ {0}, t) + G _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho) - B _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho)\right) \\ \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \leq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle + \eta \left(T _ {i j} (t _ {0}, t) + G _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho) - B _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho)\right). \\ \end{array}
$$
2. If  $i \in S_T$ ,  $i \not\sim j$  then
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \geq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle - \eta (T _ {i j} (t _ {0}, t) + G _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho) - B _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho)) \\ \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \leq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle - \eta \left(T _ {i j} (t _ {0}, t) + G _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho) - B _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho)\right). \\ \end{array}
$$
3. If  $i \in S_F, i \sim j$  then
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \geq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle - \eta (T _ {i j} (t _ {0}, t) - G _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho) + B _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho)) \\ \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \leq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle - \eta \left(T _ {i j} (t _ {0}, t) - G _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho) + B _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho)\right). \\ \end{array}
$$
4. If  $i \in S_F$ ,  $i \not\sim j$  then
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \geq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle + \eta (T _ {i j} (t _ {0}, t) - G _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho) + B _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho)) \\ \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \leq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle + \eta \left(T _ {i j} (t _ {0}, t) - G _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho) + B _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho)\right). \\ \end{array}
$$
Proof. Considering (5) we can further separate the summation term as follows,
$$
\langle \mathbf{w}_{j}^{(t)},\mathbf{x}_{i}\rangle = \langle \mathbf{w}_{j}^{(t_{0})},\mathbf{x}_{i}\rangle +(-1)^{j + i}\beta (i)\eta \left(T_{ij}(t_{0},t) + \sum_{\substack{\ell \in \mathcal{S}_{T}\\ \ell \neq i}}T_{\ell j}(t_{0},t)\lambda_{il} + \sum_{\substack{\ell \in \mathcal{S}_{F}\\ \ell \neq i}}T_{\ell j}(t_{0},t)\lambda_{i\ell}\right).
$$
Note, with  $i \in S_T$  and  $i \sim j$ , or  $i \in S_F$  and  $i \not\sim j$ , then  $(-1)^{j + i}\beta(i) = 1$ . On the other hand, with  $i \in S_T$  and  $i \not\sim j$ , or  $i \in S_F$  and  $i \sim j$ , then  $(-1)^{j + i}\beta(i) = -1$ . Substituting the relevant bounds on  $\lambda_{i\ell}$  provided in Lemma B.1, and observing by definition that  $G_j^{(i)}(t_0,t) = \sum_{\ell \in S_T,\ell \neq i}T_{\ell j}(t_0,t)$  and  $B_j^{(i)}(t_0,t) = \sum_{\ell \in S_T,\ell \neq i}T_{\ell j}(t_0,t)$ , one arrives at the results claimed.
We will often make use of the following similar but more pessimistic bounds on the activations. Recall that  $\phi$  is the ReLU function:  $\phi(a) = \max\{a, 0\}$ .
Lemma B.3. For any  $j \in [2m]$  and iterations  $t_0, t$  with  $t_0 \leq t$  the following hold:
1. If  $i \in S_T, i \sim j$  then
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \geq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) + \eta T _ {i j} (t _ {0}, t) - \eta (\gamma + \rho) B _ {j} (t _ {0}, t) - \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, t)
$$
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) + \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) G _ {j} ^ {(i)} (t _ {0}, t) + \eta \phi (\rho - \gamma) B _ {j} (t _ {0}, t).
$$
2. If  $i\in S_T,i\not\sim j$  then
$$
\phi \left(\left\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \right\rangle\right) \geq \phi \left(\left\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \right\rangle\right) - \eta T _ {i j} (t _ {0}, t) - \eta (\gamma + \rho) G _ {j} ^ {(i)} (t _ {0}, t) - \eta \phi (\rho - \gamma) B _ {j} (t _ {0}, t)
$$
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) B _ {j} (t _ {0}, t) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, t) + \eta .
$$
3. If  $i\in S_F,i\sim j$  then
$$
\phi \left(\left\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \right\rangle\right) \geq \phi \left(\left\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \right\rangle\right) - \eta T _ {i j} (t _ {0}, t) - \eta (\gamma + \rho) B _ {j} ^ {(i)} (t _ {0}, t) - \eta \phi (\rho - \gamma) G _ {j} (t _ {0}, t)
$$
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) G _ {j} (t _ {0}, t) + \eta \phi (\rho - \gamma) B _ {j} ^ {(i)} (t _ {0}, t) + \eta .
$$
4. If  $i \in S_F, i \not\sim j$  then
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \geq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) + \eta T _ {i j} (t _ {0}, t) - \eta (\gamma + \rho) G _ {j} (t _ {0}, t) - \eta \phi (\rho - \gamma) B _ {j} ^ {(i)} (t _ {0}, t)
$$
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) + \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) B _ {j} ^ {(i)} (t _ {0}, t) + \eta \phi (\rho - \gamma) G _ {j} (t _ {0}, t).
$$
The  $\eta$  term in the upper bound for cases 2 and 3 is only necessary if  $T_{ij}(t_0,t) > 0$ .
We remark that we will often use this result in a setting where  $\rho \leq \gamma$ . In these cases, the terms that involve  $\phi (\rho -\gamma)$  are zero and will be dropped.
Proof. For each of these results, we make use of Lemma B.2,  $a \leq \phi(a)$  for all  $a \in \mathbb{R}$ , and
$$
0 \leq G _ {j} ^ {(i)} (t _ {0}, t _ {1}) \leq G _ {j} (t _ {0}, t _ {1})
$$
$$
0 \leq B _ {j} ^ {(i)} (t _ {0}, t _ {1}) \leq B _ {j} (t _ {0}, t _ {1})
$$
for all  $i, j, t_0, t_1$ . We will only prove the inequalities for  $i \in S_T$  here, as the inequalities for  $i \in S_F$  are analogous.
For the first inequality in Statement 1 we claim it suffices to show
$$
\phi \left(\left\langle \mathbf {w} _ {j} ^ {(\tau + 1)}, \mathbf {x} _ {i} \right\rangle\right) \geq \phi \left(\left\langle \mathbf {w} _ {j} ^ {(\tau)}, \mathbf {x} _ {i} \right\rangle\right) + \eta T _ {i j} (\tau , \tau + 1) - \eta (\gamma + \rho) B _ {j} (\tau , \tau + 1) - \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (\tau , \tau + 1) \tag {6}
$$
Indeed, if (6) is true then the result claimed follows as
$$
\begin{array}{l} \phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) - \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) = \sum_ {\tau = t _ {0}} ^ {t - 1} \phi (\langle \mathbf {w} _ {j} ^ {(\tau + 1)}, \mathbf {x} _ {i} \rangle) - \phi (\langle \mathbf {w} _ {j} ^ {(\tau)}, \mathbf {x} _ {i} \rangle) \\ \geq \sum_ {\tau = t _ {0}} ^ {t - 1} \left(\eta T _ {i j} (\tau , \tau + 1) - \eta (\gamma + \rho) B _ {j} (\tau , \tau + 1) \right. \\ \left. - \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (\tau , \tau + 1)\right) \\ = \eta T _ {i j} (t _ {0}, t) - \eta (\gamma + \rho) B _ {j} (t _ {0}, t) - \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, t). \\ \end{array}
$$
In order to prove (6) we bound
$$
\begin{array}{l} \phi \left(\left\langle \mathbf {w} _ {j} ^ {(\tau + 1)}, \mathbf {x} _ {i} \right\rangle\right) \geq \left\langle \mathbf {w} _ {j} ^ {(\tau + 1)}, \mathbf {x} _ {i} \right\rangle \\ \geq \langle \mathbf {w} _ {j} ^ {(\tau)}, \mathbf {x} _ {i} \rangle + \eta T _ {i j} (\tau , \tau + 1) - \eta (\gamma + \rho) B _ {j} (\tau , \tau + 1) \\ - \phi (\rho - \gamma) G _ {j} ^ {(i)} (\tau , \tau + 1). \\ \end{array}
$$
This follows from Statement 1 in Lemma B.2. From here, we consider two cases: first, if  $\langle \mathbf{w}_j^{(\tau)},\mathbf{x}_i\rangle \geq 0$  then  $\langle \mathbf{w}_j^{(\tau)},\mathbf{x}_i\rangle = \phi (\langle \mathbf{w}_j^{(\tau)},\mathbf{x}_i\rangle)$  and so (6) clearly holds. Alternatively, if  $\langle \mathbf{w}_j^{(\tau)},\mathbf{x}_i\rangle < 0$  then  $T_{ij}(\tau ,\tau +1) = 0,\phi (\langle \mathbf{w}_j^{(\tau)},\mathbf{x}_i\rangle) = 0$  and as a result the right-hand-side of (6) is non-positive while the left is non-negative. As such (6) holds trivially.
For the second equality in Statement 1 we bound
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \leq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle + \eta (T _ {i j} (t _ {0}, t) + G _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho) - B _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho)) \\ \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) + \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) G _ {j} ^ {(i)} (t _ {0}, t) + \eta \phi (\rho - \gamma) B _ {j} (t _ {0}, t). \\ \end{array}
$$
Since the right-hand side is non-negative, this inequality is true even if we replace the left-hand side by  $\phi (\langle \mathbf{w}_j^{(t)},\mathbf{x}_i\rangle)$ .
We now proceed to Statement 2. For the first inequality, notice that if  $\phi (\langle \mathbf{w}_j^{(t_0)},\mathbf{x}_i\rangle) = 0$  then the right-hand side is non-positive and therefore the inequality trivially holds. Otherwise, it must be the case that  $\phi (\langle \mathbf{w}_j^{(t_0)},\mathbf{x}_i\rangle) = \langle \mathbf{w}_j^{(t_0)},\mathbf{x}_i\rangle$ . Using Statement 2 from Lemma B.2, we obtain the bound
$$
\begin{array}{l} \phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) \geq \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \\ \geq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle - \eta (T _ {i j} (t _ {0}, t) + G _ {j} ^ {(i)} (t _ {0}, t) (\gamma + \rho) - B _ {j} ^ {(i)} (t _ {0}, t) (\gamma - \rho)) \\ \geq \phi (\langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, t) - \eta (\gamma + \rho) G _ {j} ^ {(i)} (t _ {0}, t) - \eta \phi (\rho - \gamma) B _ {j} (t _ {0}, t). \\ \end{array}
$$
We now turn to the second inequality in Statement 2. The corresponding statement from Lemma B.2 yields
$$
\begin{array}{l} \langle \mathbf {w} _ {j} ^ {(t)}, \mathbf {x} _ {i} \rangle \leq \langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle - \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) B _ {j} (t _ {0}, t) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, t) \\ \leq \phi \left(\left\langle \mathbf {w} _ {j} ^ {\left(t _ {0}\right)}, \mathbf {x} _ {i} \right\rangle\right) - \eta T _ {i j} \left(t _ {0}, t\right) + \eta (\gamma + \rho) B _ {j} \left(t _ {0}, t\right) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} \left(t _ {0}, t\right) \tag {7} \\ \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, t) + \eta (\gamma + \rho) B _ {j} (t _ {0}, t) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, t) + \eta , \\ \end{array}
$$
we remark that the reason for the addition of  $\eta$  to the right-hand-side will soon become apparent. The desired inequality holds as long as the right-hand-side is non-negative, we therefore proceed by induction to prove
$$
\phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, \tau) + \eta (\gamma + \rho) B _ {j} (t _ {0}, \tau) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, t) + \eta \geq 0
$$
for  $\tau \geq t_0$ . The base case  $\tau = t_0$  is trivial, assume then that the induction hypothesis holds for some  $\tau \geq t_0$ . For iteration  $\tau + 1$  there are two cases to consider: first, if  $\langle \mathbf{w}_j^{(\tau)}, \mathbf{x}_i \rangle < 0$  then  $T_{ij}(t_0, \tau + 1) = T_{ij}(t_0, \tau)$ . In addition, as  $B_j(t_0, \tau) \leq B_j(t_0, \tau + 1)$  and  $G_j^{(i)}(t_0, \tau) \leq G_j^{(i)}(t_0, \tau + 1)$  then
$$
\begin{array}{l} 0 \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, \tau) + \eta (\gamma + \rho) B _ {j} (t _ {0}, \tau) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, \tau) + \eta \\ \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, \tau + 1) + \eta (\gamma + \rho) B _ {j} (t _ {0}, \tau + 1) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, \tau + 1) + \eta \\ \end{array}
$$
by the induction hypothesis. Alternatively, if instead  $\langle \mathbf{w}_j^{(\tau)},\mathbf{x}_i\rangle \geq 0$  one may use the second inequality from (7) to conclude that
$$
0 \leq \langle \mathbf {w} _ {j} ^ {(\tau)}, \mathbf {x} _ {i} \rangle \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, \tau) + \eta (\gamma + \rho) B _ {j} (t _ {0}, \tau) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, \tau).
$$
In addition, as  $T_{ij}(t_0,\tau +1)\leq T_{ij}(t_0,\tau) + 1$  it follows that
$$
\begin{array}{l} 0 \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, \tau) + \eta (\gamma + \rho) B _ {j} (t _ {0}, \tau) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, \tau) \\ \leq \phi (\langle \mathbf {w} _ {j} ^ {(t _ {0})}, \mathbf {x} _ {i} \rangle) - \eta T _ {i j} (t _ {0}, \tau + 1) + \eta (\gamma + \rho) B _ {j} (t _ {0}, \tau + 1) + \eta \phi (\rho - \gamma) G _ {j} ^ {(i)} (t _ {0}, \tau + 1) + \eta \\ \end{array}
$$
Lastly, we consider the final remark in the statement of the lemma: if  $T_{ij}(t_0,t) = 0$  then the right hand side of the second line in (7) is non-negative trivially, so we do not need the additional  $\eta$  term.
B.2 Convergence of training
We say that GD terminates if it reaches a finite iteration in which a zero update is applied to the network parameters. The following lemmas are used to show that GD terminates by in turn upper bounding the number of clean and corrupt updates. The first lemma facilitates the bounding of the hinge loss of clean and corrupt points.
Lemma B.4. For any iterations  $t, t_0$  satisfying  $t \geq t_0$ ,
1. if  $i\in \mathcal{S}_T$  then
$$
y _ {i} f (t, \mathbf {x} _ {i}) \geq y _ {i} f (t _ {0}, \mathbf {x} _ {i}) + \eta (T _ {i} (t _ {0}, t) - (\gamma + \rho) B (t _ {0}, t) - \phi (\rho - \gamma) G ^ {(i)} (t _ {0}, t) - m),
$$
2. if  $i \in S_F$  then
$$
y _ {i} f (t, \mathbf {x} _ {i}) \geq y _ {i} f (t _ {0}, \mathbf {x} _ {i}) + \eta (T _ {i} (t _ {0}, t) - (\gamma + \rho) G (t _ {0}, t) - \phi (\rho - \gamma) B ^ {(i)} (t _ {0}, t) - m).
$$
Proof. Both statements follow from the bounds provided in Lemma B.3. For Statement 1