SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Note that according to Assumption 8 we have  $\tilde{\gamma}(t_0) \geq b_{\mathfrak{f}} \rho(t_0)^{-(L+1)}$  which means  $\tilde{\gamma}(t_0)$  cannot be arbitrarily close to 0. Thus, by invoking Lemma 20, we obtain that
$$
\lim  _ {t \rightarrow \infty} \kappa^ {2} (t) = \lim  _ {t \rightarrow \infty} \frac {2}{\tilde {\gamma} \left(t _ {0}\right) ^ {2 / (L + 1)}} \left(1 - \left\langle \frac {\underline {{\mathbf {W}}}}{\| \underline {{\mathbf {W}}} \| _ {\mathrm {F}}}, \frac {\dot {\underline {{\mathbf {W}}}}}{\left\| \dot {\underline {{\mathbf {W}}}} \right\| _ {\mathrm {F}}} \right\rangle\right) = 0. \tag {34}
$$
<sup>10</sup>The Lemma C.12 in [30] which was intended for standard training, can be safely extended to the adversarial settings in this paper. This is because by our construction for adversarial training with scale invariant adversarial perturbations, the adversarial training margin are locally Lipschitz and the prediction function  $f(\underline{\mathbf{x}} + \underline{\delta}(\underline{\mathbf{W}}); \underline{\mathbf{W}})$  is positively homogeneous with respect to  $\underline{\mathbf{W}}$ .
Step 2: Check Condition (II) of the  $(\kappa, \iota)$ -approximate KKT conditions. We check the Condition (II) in Problem (31) for all  $t > t_0$  as follows
$$
\begin{array}{l} \sum_ {i = 1} ^ {N} \lambda_ {i} \left(\tilde {q} _ {i} (\underline {{\mathbf {W}}}) - 1\right) \stackrel {(i)} {=} \sum_ {i = 1} ^ {N} \frac {1}{N} \tilde {q} _ {m} ^ {1 - 2 / (L + 1)} \rho e ^ {- \mathrm {f} \left(\tilde {q} _ {i}\right)} \mathrm {f} ^ {\prime} \left(\tilde {q} _ {i}\right) \left\| \underline {{\mathbf {W}}} \right\| _ {\mathrm {F}} ^ {- 1} \cdot \left(\tilde {q} _ {i} (\underline {{\mathbf {W}}}) \tilde {q} _ {m} ^ {- 1} - 1\right) \tag {35} \\ = \rho \left\| \dot {\underline {{\mathbf {W}}}} \right\| _ {\mathrm {F}} ^ {- 1} \tilde {q} _ {m} ^ {- 2 / (L + 1)} \cdot \frac {1}{N} \sum_ {i = 1} ^ {N} e ^ {- \mathfrak {f} (\tilde {q} _ {i})} \mathfrak {f} ^ {\prime} (\tilde {q} _ {i}) (\tilde {q} _ {i} - \tilde {q} _ {m}), \\ \end{array}
$$
where  $(i)$  is due to the definition that  $\check{\mathbf{W}} = \underline{\mathbf{W}} / \tilde{q}_m^{\frac{1}{L + 1}}$  and the fact that  $\tilde{q}_i(\underline{\mathbf{W}}) = y_i f(\underline{\mathbf{x}}_i + \underline{\boldsymbol{\delta}}_i(\underline{\mathbf{W}}); \underline{\mathbf{W}}) = y_i \cdot \tilde{q}_m f(\underline{\mathbf{x}}_i + \underline{\boldsymbol{\delta}}_i(\underline{\mathbf{W}}); \underline{\mathbf{W}}) = \tilde{q}_i(\underline{\mathbf{W}})$  due to the scale invariance of the adversarial perturbation  $\underline{\boldsymbol{\delta}}_i(\underline{\mathbf{W}})$  and the homogenouty of t-NN.
To upper bound Eq. (35), first note that  $\left\| \dot{\underline{\mathbf{W}}}\right\|_{\mathrm{F}}\geq \left\langle \dot{\underline{\mathbf{W}}},\hat{\underline{\mathbf{W}}}\right\rangle = \left\langle \frac{1}{N}\sum_{i = 1}^{N}e^{-\mathsf{f}(\tilde{q}_i)}\mathsf{f}'(\tilde{q}_i)\frac{\partial\tilde{q}_i}{\partial\underline{\mathbf{W}}},\hat{\underline{\mathbf{W}}}\right\rangle =$ $\rho^{-1}(L + 1)\nu /N$  , in which  $\nu$  can be further lower bounded as
$$
\nu \geq \frac {(i)}{\mathfrak {g} ^ {\prime} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)} N \hat {\mathcal {L}} ^ {\mathrm {a d v}} \geq \frac {1}{2 K} \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}} \cdot N \hat {\mathcal {L}} ^ {\mathrm {a d v}} \geq \frac {1}{2 K} e ^ {- \mathfrak {f} (\hat {q} _ {m})} \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}, \tag {36}
$$
where  $(i)$  is due to Lemma 23,  $(ii)$  holds because of Lemma 24, and  $(iii)$  uses  $N\hat{\mathcal{L}}^{\mathrm{adv}} = \sum_{i}e^{-\mathsf{f}(\tilde{q}_i)}\geq$ $e^{-\mathsf{f}(\tilde{q}_m)}$ . Combining Eq. (35) and Eq. (36) yields
$$
\begin{array}{l} \sum_ {i} \lambda_ {i} \left(\tilde {q} _ {i} (\underline {{\mathbf {W}}}) - 1\right) \leq \frac {2 K \tilde {q} _ {m} ^ {- 2 / (L + 1)} \rho^ {2}}{(L + 1) \log \frac {1}{N \hat {\mathcal {L}} _ {\mathrm {a d v}} ^ {\mathrm {a d v}}}} \sum_ {i = 1} ^ {N} e ^ {\mathfrak {f} \left(\tilde {q} _ {m}\right) - \mathfrak {f} \left(\tilde {q} _ {i}\right)} \mathfrak {f} ^ {\prime} \left(\tilde {q} _ {i}\right) \left(\tilde {q} _ {i} - \tilde {q} _ {m}\right) \tag {37} \\ \stackrel {(i)} {\geq} \frac {2 K \tilde {\gamma} ^ {- 2 / (L + 1)}}{(L + 1) \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}} \sum_ {i = 1} ^ {N} e ^ {\mathfrak {f} (\tilde {q} _ {m}) - \mathfrak {f} (\tilde {q} _ {i})} \mathfrak {f} ^ {\prime} (\tilde {q} _ {i}) (\tilde {q} _ {i} - \tilde {q} _ {m}), \\ \end{array}
$$
where  $(i)$  uses  $\tilde{q}_m^{-2 / (L + 1)}\rho^2\leq \tilde{\gamma}^{-2 / (L + 1)}$  by Lemma 19.
In Eq. (37), if  $\tilde{q}_i > \tilde{q}_m$ , then there exists an  $\xi_i \in (\tilde{q}_m, \tilde{q}_i)$  such that  $\mathsf{f}(\tilde{q}_m) - \mathsf{f}(\tilde{q}_i) = \mathsf{f}'(\xi_i)(\tilde{q}_i - \tilde{q}_m)$  by the mean value theorem. Further, we know that  $\mathsf{f}'(\tilde{q}_i) \leq K^{\lceil \log_2(\tilde{q}_i / \xi_i) \rceil} \mathsf{f}'(\xi_i)$  by Assumption 2. Note that  $\lceil \log_2(\tilde{q}_i / \xi_i) \rceil \leq \log_2(2B_0\rho^{L + 1} / \tilde{q}_m) \leq \log_2(2B_0 / \tilde{\gamma})$ , where
$$
B _ {0} (t): = \sup  \left\{\tilde {q} _ {i} \rho^ {- (L + 1)} \mid \underline {{\mathbf {W}}} \neq \mathbf {0} \right\} = \sup  \left\{\tilde {q} _ {i} \mid \| \underline {{\mathbf {W}}} \| _ {\mathrm {F}} = 1 \right\}.
$$
Then, for all  $t > t_0$ , we have
$$
\begin{array}{l} \sum_ {i} \lambda_ {i} (\tilde {q} _ {i} (\underline {{\mathbf {W}}}) - 1) \leq \frac {2 K \tilde {\gamma} ^ {- 2 / (L + 1)} \rho^ {2}}{(L + 1) \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}} K ^ {\log_ {2} (2 B _ {0} / \tilde {\gamma})} \sum_ {i: \tilde {q} _ {i} \neq \tilde {q} _ {m}} e ^ {\mathfrak {f} ^ {\prime} (\xi_ {i}) (\tilde {q} _ {i} - \tilde {q} _ {m})} \mathfrak {f} ^ {\prime} (\xi_ {i}) (\tilde {q} _ {i} - \tilde {q} _ {m}) \\ \stackrel {(i)} {\leq} \frac {2 K \tilde {\gamma} ^ {- 2 / (L + 1)}}{(L + 1) \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}} K ^ {\log_ {2} (2 B _ {0} / \tilde {\gamma})} \cdot N e \\ \stackrel {(i i)} {\leq} \frac {2 K N e \tilde {\gamma} ^ {- 2 / (L + 1)}}{(L + 1) \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}} \left(\frac {B _ {0}}{\tilde {\gamma}}\right) ^ {\log_ {2} (2 K)} \tag {38} \\ \stackrel {(i i i)} {\leq} \frac {2 K N e}{(L + 1) \tilde {\gamma} (t _ {0}) ^ {- 2 / (L + 1)}} \left(\frac {B _ {0}}{\tilde {\gamma} (t _ {0})}\right) ^ {\log_ {2} (2 K)} \cdot \log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}} \\ := \iota (t), \\ \end{array}
$$
where  $(i)$  holds because the function  $x\mapsto e^{-z}z$  on  $(0,\infty)$  has maximum  $e$  at  $z = 1$ ;  $(ii)$  is due to  $a^{\log_c b} = b^{\log_c a}$ ;  $(iii)$  holds by the non-decreasing property of  $\tilde{\gamma} (t)$  for all  $t\in [t_0,\infty)$  in Lemma 22. Note that by Lemma 22, we have  $\lim_{t\to \infty}\hat{\mathcal{L}}^{\mathrm{adv}}(t) = 0$ , which further yields
$$
\lim  _ {t \rightarrow \infty} \iota (t) = 0. \tag {39}
$$
Step 3: Check the condition for convergence to KKT point. According to Eq. (34) and Eq. (39), the limit point of  $\dot{\mathbf{W}}(t)$  satisfy the  $(\kappa, \iota)$ -approximate KKT conditions of Problem 14 along the
trajectory of adversarial training of t-NN with scale invariant adversarial perturbations where  $\lim_{t\to \infty}(\kappa (t),\iota (t)) = 0$ . Then, we need to check the condition between  $(\kappa ,\iota)$ -approximate points and KKT points.
According to Ref. [30], the KKT condition becomes a necessary condition for global optimality of Problem (14) when the Mangasarian-Fromovitz Constraint Qualification (MFCQ) [33] is satisfied. It is straightforward to see that Problem (14) satisfies the MFCQ condition, i.e.,
$$
\left\langle \frac {\partial \tilde {q} _ {i}}{\partial \underline {{\mathbf {W}}}}, \underline {{\mathbf {W}}} \right\rangle = (L + 1) \tilde {q} _ {i} \geq 0.
$$
at every feasible point  $\mathbf{W}$ . Then restating the theorem in Ref. [7] regarding the relation between  $(\kappa, \iota)$ -approximate KKT point and KKT point in our setting yields the following result.
Theorem 21 (Theorem 3.6 in Ref. [7] and Theorem C.4 in Ref. [30]). Let
$\{\underline{\mathbf{W}}(j) \mid j \in \mathbb{N}\}$  be a sequence of feasible points of Problem (14), and  $\underline{\mathbf{W}}(j)$  is a  $(\kappa(j), \iota(j))$ -approximate KKT point for all  $j$  with two sequences $^{11}$ $\{\kappa(j) > 0 \mid j \in \mathbb{N}\}$  and  $\{\iota(j) > 0 \mid j \in \mathbb{N}\}$  and  $\lim_{j \to \infty} (\kappa(j), \iota(j)) = \mathbf{0}$ . If  $\lim_{j \to \infty} \underline{\mathbf{W}}(j) = \underline{\mathbf{W}}^*$ , and MFCQ holds at  $\underline{\mathbf{W}}^*$ , then  $\underline{\mathbf{W}}^*$  is a KKT point of Problem (14).
Recall that  $\underline{\mathbf{W}} = \underline{\mathbf{W}} / \tilde{q}_m^{\frac{t}{L + 1}}$ . Then, it can be concluded that the limit point of  $\{\underline{\mathbf{W}}(t) / \| \underline{\mathbf{W}}(t)\|_{\mathrm{F}} : t > 0\}$  of GF for empirical adversarial risk  $\hat{\mathcal{L}}^{\mathrm{adv}}(\underline{\mathbf{W}}) \coloneqq \frac{1}{N} \sum_{i=1}^{N} e^{-\mathfrak{f}(y_i f(\underline{x}_i + \underline{\delta}_i(\underline{\mathbf{W}}); \underline{\mathbf{W}}))}$  with scale invariant perturbations  $\underline{\delta}_i$  is aligned with the direction of a KKT point of Problem (14).
D.2 Technical Lemmas for Proving Lemma 9
Lemma 22. Under Assumption 2, we have the following statements for GF-based adversarial training in Eq. (13) with scale invariant perturbations:
(I). For a.e.  $t \in (t_0, \infty)$ , the smoothed normalized robust margin  $\tilde{\gamma}(\underline{\mathbf{W}}(t))$  defined in Eq. (29) is non-decreasing, i.e.,
$$
\frac {\mathrm {d} \tilde {\gamma} (\underline {{\mathbf {W}}} (t))}{\mathrm {d} t} \geq 0.
$$
(II). The adversarial objective  $\hat{\mathcal{L}}^{\mathrm{adv}}(\underline{\mathbf{W}}) \coloneqq \frac{1}{N}\sum_{i=1}^{N}e^{-\mathfrak{f}(y_i f(\underline{\mathbf{x}}_i + \underline{\boldsymbol{\delta}}_i(\underline{\mathbf{W}});\underline{\mathbf{W}}))}$  with scale invariant perturbations  $\underline{\boldsymbol{\delta}}_i$  converges to zero as  $t \to \infty$ , i.e.,
$$
\lim  _ {t \rightarrow \infty} \hat {\mathcal {L}} ^ {\mathrm {a d v}} (\underline {{\mathbf {W}}} (t)) = 0, \tag {40}
$$
and the Euclidean norm of the  $t$ -NN weights diverges, i.e.,
$$
\lim  _ {t \rightarrow \infty} \| \underline {{\mathbf {W}}} (t) \| _ {\mathrm {F}} = \infty . \tag {41}
$$
Proof of Lemma 22. We follow the idea of Ref. [30] to prove Lemma 22 as follows.
Step 1: Prove Part (I). We prove (I) by showing the following results for all  $t \geq t_0$ ,
$$
\frac {\mathrm {d} \log \rho}{\mathrm {d} t} > 0 \quad \text {a n d} \quad \frac {\mathrm {d} \log \tilde {\gamma}}{\mathrm {d} t} \geq (L + 1) \left(\frac {\mathrm {d} \log \rho}{\mathrm {d} t}\right) ^ {- 1} \left\| \frac {\mathrm {d} \hat {\mathbf {W}}}{\mathrm {d} t} \right\| _ {\mathrm {F}}.
$$
Recalling the quantity  $\nu(t) = \sum_{i=1}^{N} e^{-\mathfrak{f}(\tilde{q}_i)} \mathfrak{f}(\tilde{q}_i) \tilde{q}_i$ , by chain rule we obtain
$$
\frac {\mathrm {d} \log \rho}{\mathrm {d} t} = \frac {1}{2 \rho^ {2}} \frac {\mathrm {d} \rho^ {2}}{\mathrm {d} t} = \frac {1}{\rho^ {2}} \left\langle \frac {1}{N} \sum_ {i = 1} ^ {N} e ^ {- \mathfrak {f} (\tilde {q} _ {i})} \mathfrak {f} ^ {\prime} (\tilde {q} _ {i}) \frac {\partial \tilde {q} _ {i}}{\partial \underline {{\mathbf {W}}}}, \underline {{\mathbf {W}}} \right\rangle = \frac {\nu (L + 1)}{\rho^ {2} N} \overset {(i)} {\geq} 0,
$$
where  $(i)$  holds due to Lemma 23.
Using the same method to [30, Lemma C.12], we can construct the two sequences, i.e.,  $\{\kappa(j) > 0 \mid j \in \mathbb{N}\}$  and  $\{\iota(j) > 0 \mid j \in \mathbb{N}\}$ , based on Eq. (33) and Eq. (35).
By the chaining rule, we also have
$$
\begin{array}{l} \frac {\mathrm {d} \log \tilde {\gamma}}{\mathrm {d} t} = \frac {\mathrm {d}}{\mathrm {d} t} \left(\log \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right) - (L + 1) \log \rho\right) \\ = \frac {\mathfrak {g} ^ {\prime} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)}{\mathfrak {g} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)} \cdot \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}} \cdot \left(- \frac {\mathsf {d} N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\mathsf {d} t}\right) - (L + 1) ^ {2} \cdot \frac {\nu (t)}{\rho^ {2}} \\ = \frac {1}{\nu (t)} \cdot \left(- \frac {\mathrm {d} N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\mathrm {d} t}\right) - (L + 1) ^ {2} \cdot \frac {\nu (t)}{\rho^ {2}} \\ = \frac {1}{\nu (t)} \cdot \left(- \frac {\mathrm {d} N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\mathrm {d} t} - (L + 1) ^ {2} \cdot \frac {\nu (t) ^ {2}}{\rho^ {2}}\right). \\ \end{array}
$$
Note that according to Eq. (13), we have
$$
\frac {\mathrm {d} \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\mathrm {d} t} = \left\langle \frac {\partial \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\partial \underline {{\mathbf {W}}}}, \frac {\mathrm {d} \underline {{\mathbf {W}}}}{\mathrm {d} t} \right\rangle = - \left\| \frac {\mathrm {d} \underline {{\mathbf {W}}}}{\mathrm {d} t} \right\| _ {\mathrm {F}} ^ {2},
$$
for  $t \geq 0$  almost everywhere. One the other hand, we have
$$
L \nu (t) = \left\langle \underline {{\mathbf {W}}}, \frac {\mathrm {d} \underline {{\mathbf {W}}}}{\mathrm {d} t} \right\rangle .
$$
$$
\frac {\mathrm {d} \log \tilde {\gamma}}{\mathrm {d} t} \geq \frac {1}{N \nu} \left(\left\| \frac {\mathrm {d} \mathbf {W}}{\mathrm {d} t} \right\| _ {\mathrm {F}} ^ {2} - \left\langle \hat {\mathbf {W}}, \frac {\mathrm {d} \mathbf {W}}{\mathrm {d} t} \right\rangle^ {2}\right) \geq \frac {1}{N \nu (t)} \left\| (\mathbf {I} - \operatorname {v e c} (\hat {\mathbf {W}}) \operatorname {v e c} (\hat {\mathbf {W}}) ^ {\top}) \operatorname {v e c} (\frac {\mathrm {d} \mathbf {W}}{\mathrm {d} t}) \right\| ^ {2}.
$$
By the chain rule,
$$
\frac {\mathrm {d} \hat {\mathbf {W}}}{\mathrm {d} t} = \frac {1}{\rho} (\mathbf {I} - \operatorname {v e c} (\hat {\mathbf {W}}) \operatorname {v e c} (\hat {\mathbf {W}}) ^ {\top}) \operatorname {v e c} (\frac {\mathrm {d} \mathbf {W}}{\mathrm {d} t}),
$$
for  $t > 0$  allmost everywhere. So, we have
$$
\frac {\mathrm {d}}{\mathrm {d} t} \log \tilde {\gamma} \geq \frac {\rho^ {2}}{N \nu (t)} \left\| \frac {\mathrm {d} \hat {\mathbf {W}}}{\mathrm {d} t} \right\| _ {\mathrm {F}} ^ {2} = (L + 1) \left(\frac {\mathrm {d}}{\mathrm {d} t} \log \rho\right) ^ {- 1} \left\| \frac {\mathrm {d} \hat {\mathbf {W}}}{\mathrm {d} t} \right\| _ {\mathrm {F}} ^ {2} \geq 0.
$$
Step 2: Prove Part (II). Motivated by [30, Lemma B.8], we prove (II) as follows. First, note that
$$
- \frac {\mathrm {d} \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\mathrm {d} t} = \left\| \frac {\mathrm {d} \mathbf {W}}{\mathrm {d} t} \right\| _ {\mathrm {F}} ^ {2} \geq \left\langle \underline {{\hat {\mathbf {W}}}}, \dot {\underline {{\mathbf {W}}}} \right\rangle^ {2} = \rho^ {- 2} N ^ {- 2} (L + 1) ^ {2} \nu^ {2}.
$$
By lower bounding  $\nu$  with Lemma 23 and replacing  $\rho$  with  $(\mathfrak{g}(\log \frac{1}{N\hat{\mathcal{L}}^{\mathrm{adv}}}) / \tilde{\gamma})^{1 / (L + 1)}$  by the definition smoothed normalized robust margin of  $\tilde{\gamma}$  in Eq. (29), we obtain
$$
\begin{array}{l} - \frac {\mathrm {d} \hat {\mathcal {L}} ^ {\mathrm {a d v}}}{\mathrm {d} t} \geq (L + 1) ^ {2} \left(\frac {\mathfrak {g} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)}{\mathfrak {g} ^ {\prime} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)} N \hat {\mathcal {L}} ^ {\mathrm {a d v}}\right) ^ {2} \cdot \left(\frac {\tilde {\gamma}}{\mathfrak {g} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)}\right) ^ {2 / (L + 1)} \\ \geq (L + 1) ^ {2} \tilde {\gamma} (t _ {0}) ^ {2 / (L + 1)} \cdot \frac {\mathfrak {g} (\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}) ^ {2 - 2 / (L + 1)}}{\mathfrak {g} ^ {\prime} (\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}) ^ {2}} \cdot (N \hat {\mathcal {L}} ^ {\mathrm {a d v}}) ^ {2}, \\ \end{array}
$$
where the last inequality holds due to the non-decreasing property of  $\tilde{\gamma}$  in Part (I). Then, we obtain
$$
\frac {\mathfrak {g} ^ {\prime} (\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}) ^ {2}}{\mathfrak {g} (\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}) ^ {2 - 2 / (L + 1)}} \cdot \frac {\mathsf {d}}{\mathsf {d} t} \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}} \geq (L + 1) ^ {2} \tilde {\gamma} (t _ {0}) ^ {2 / (L + 1)}.
$$
By integrating on both sides from  $t_0$  to  $t$ , we obtain
$$
G \left(\frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right) \geq (L + 1) ^ {2} \tilde {\gamma} (t _ {0}) ^ {2 / (L + 1)} (t - t _ {0}), \tag {42}
$$
$$
G (z) := \int_ {1 / \left(N \hat {\mathcal {L}} ^ {\mathrm {a d v}} \left(t _ {0}\right)\right)} ^ {z} \frac {\mathfrak {g} ^ {\prime} (\log u) ^ {2}}{g (\log u) ^ {2 - 2 / (L + 1)}} \mathrm {d} u.
$$
We use proof by contradiction to show the empirical training risk  $\hat{\mathcal{L}}^{\mathrm{adv}}$  converges to zero. Note that  $\frac{1}{(N\hat{\mathcal{L}}^{\mathrm{adv}})}$  is non-decreasing. If  $\frac{1}{(N\hat{\mathcal{L}}^{\mathrm{adv}})}$  does not grow to  $\infty$ , then neither does  $G\left(\frac{1}{(N\hat{\mathcal{L}}^{\mathrm{adv}})}\right)$ . But the RHS of Eq. (42) grows to  $\infty$ , which is a contradiction. Therefore,  $\lim_{t\to \infty}N\hat{\mathcal{L}}^{\mathrm{adv}} = 0$ . Hence  $\lim_{t\to \infty}\hat{\mathcal{L}}^{\mathrm{adv}}(t) = 0$  and  $\lim_{t\to \infty}\rho (t) = \infty$ .
Lemma 23 (Adapted from Lemma B.5 of Ref. [30]). The quantity  $\nu = \sum_{i}e^{-\mathfrak{f}(\tilde{q}_i)}\mathfrak{f}(\tilde{q}_i)\tilde{q}_i$  has a lower bound for all  $t\in (t_0,\infty)$
$$
\nu (t) \geq \frac {\mathfrak {g} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)}{\mathfrak {g} ^ {\prime} \left(\log \frac {1}{N \hat {\mathcal {L}} ^ {\mathrm {a d v}}}\right)} N \hat {\mathcal {L}} ^ {\mathrm {a d v}}. \tag {43}
$$
Lemma 24 (Lemma D.1 of Ref. [30]). For  $\mathfrak{f}(\cdot)$  and  $\mathfrak{g}(\cdot)$  in Assumption 2, we have
$$
\frac {\mathfrak {g} (x)}{\mathfrak {g} ^ {\prime} (x)} \in \left[ \frac {1}{2 K} x, 2 K x \right], \forall x \in [ b _ {\mathfrak {g}}, \infty) \quad \text {a n d} \quad \frac {\mathfrak {f} (q)}{\mathfrak {f} ^ {\prime} (q)} \in \left[ \frac {1}{2 K} q, 2 K q \right], \forall q \in [ \mathfrak {g} (b _ {\mathfrak {g}}), \infty).
$$
D.3 Proof of Theorem 10
Note that the t-NN  $g(\underline{\mathbf{x}};\underline{\mathbf{V}})$  with weights  $\underline{\mathbf{V}} = (\underline{\mathbf{V}}^{(1)},\dots ,\underline{\mathbf{V}}^{(J)},\mathbf{v})$  in Theorem 6 has the following structure
$$
g (\underline {{\mathbf {x}}}; \underline {{\mathbf {V}}}) = \mathbf {v} ^ {\top} \operatorname {v e c} (\underline {{\mathbf {g}}} (\underline {{\mathbf {x}}}))
$$
$$
\mathbf {g} (\underline {{\mathbf {x}}}) = \mathbf {g} ^ {(J)} (\underline {{\mathbf {x}}}) \in \mathbb {R} ^ {m _ {J} \times 1 \times c}
$$
$$
\mathbf {g} ^ {(j)} (\underline {{\mathbf {x}}}) = \sigma \left(\underline {{\mathbf {v}}} ^ {(j)} * _ {M} \mathbf {g} ^ {(j - 1)} (\underline {{\mathbf {x}}})\right) \in \mathbb {R} ^ {m _ {j} \times 1 \times c}, \forall j = 1, \dots , J
$$
$$
\mathbf {g} ^ {(0)} (\underline {{\mathbf {x}}}) = \underline {{\mathbf {x}}}.
$$
Let  $\alpha = \left(\frac{1}{B_v}\right)^{\frac{L - J}{L + 1}}$  and  $\beta = \left(\frac{1}{B_v}\right)^{-\frac{J + 1}{L + 1}}$ , where  $L$  is a sufficiently large integer greater than  $J$ . We then construct a t-NN  $h(\underline{\mathbf{x}};\underline{\mathbf{H}})$  of  $L$  t-product layers which perfectly realizes  $g(\underline{\mathbf{x}};\underline{\mathbf{V}})$ . Specifically, we construct  $h$  with weights  $\underline{\mathbf{H}} = (\underline{\mathbf{H}}^{(1)},\dots ,\underline{\mathbf{H}}^{(L)},\mathbf{h})$  satisfying the following equation
$$
\underline {{\mathbf {H}}} = (\underbrace {\alpha \underline {{\mathbf {V}}} ^ {(1)} , \cdots , \alpha \underline {{\mathbf {V}}} ^ {(J)}} _ {\text {f i r s t} J \text {t - p r o d u c t l a y e r s}}, \underbrace {\beta \underline {{\mathbf {I}}}, \cdots , \beta \underline {{\mathbf {I}}}} _ {\text {l a s t} (L - J) \text {t - p r o d u c t l a y e r s}}, \mathbf {v}),
$$
or more clearly
$$
h (\underline {{\mathbf {x}}}; \underline {{\mathbf {V}}}) = \mathbf {v} ^ {\top} _ {\operatorname {v e c}} (\mathbf {h} (\underline {{\mathbf {x}}}))
$$
$$
\mathbf {h} (\underline {{\mathbf {x}}}) = \mathbf {h} ^ {(L)} (\underline {{\mathbf {x}}}) \in \mathbb {R} ^ {m _ {J} \times 1 \times c}
$$
$$
\mathbf {h} ^ {(l)} (\underline {{\mathbf {x}}}) = \sigma \left(\beta \underline {{\mathbf {I}}} * _ {M} \mathbf {h} ^ {(l - 1)} (\underline {{\mathbf {x}}})\right) \in \mathbb {R} ^ {m _ {J} \times 1 \times c}, \forall l = J + 1, \dots , L
$$
$$
\mathbf {h} ^ {(j)} (\underline {{\mathbf {x}}}) = \sigma \left(\alpha \underline {{\mathbf {V}}} ^ {(j)} * _ {M} \mathbf {h} ^ {(j - 1)} (\underline {{\mathbf {x}}})\right) \in \mathbb {R} ^ {m _ {j} \times 1 \times \mathsf {c}}, \forall j = 1, \dots , J
$$
$$
\mathbf {h} ^ {(0)} (\underline {{\mathbf {x}}}) = \underline {{\mathbf {x}}},
$$
where  $\underline{\mathbf{I}}$  is the t-identity tensor.
It is easy to prove that for any input  $\underline{\mathbf{x}}$ , the input and output of  $g(\underline{\mathbf{x}};\underline{\mathbf{V}}) = h(\underline{\mathbf{x}};\underline{\mathbf{H}})$ , it can also be proved that
$$
\min  _ {R _ {\mathrm {a}} (\underline {{\mathbf {x}}} - \underline {{\mathbf {x}}} ^ {\prime}) \leq \xi} y g (\underline {{\mathbf {x}}} ^ {\prime}; \underline {{\mathbf {V}}}) = \min  _ {R _ {\mathrm {a}} (\underline {{\mathbf {x}}} - \underline {{\mathbf {x}}} ^ {\prime}) \leq \xi} y h (\underline {{\mathbf {x}}} ^ {\prime}; \underline {{\mathbf {H}}}).
$$
Therefore,  $h(\underline{\mathbf{x}};\underline{\mathbf{H}})$  can also robustly classify  $(\underline{\mathbf{x}}_i,y_i)_{i = 1}^N$  because  $g(\underline{\mathbf{x}};\underline{\mathbf{V}})$  can robustly classify  $(\underline{\mathbf{x}}_i,y_i)_{i = 1}^N$ .
Then, we consider the class of over-parameterized t-NNs  $\mathfrak{F} = \{f(\underline{\mathbf{x}};\underline{\mathbf{W}})\}$  defined in Eq. (14) with dimensionality of weight  $\underline{\mathbf{W}}^{(l)}\in \mathbb{R}^{d_l\times d_{l - 1}\times c}$  satisfying  $d_{l}\gg \max_{j\leq J}\{m_{j}\}$  for all  $l = 1,\dots ,L$ . Specifically, we construct  $f$  with weights
$$
\underline {{\mathbf {W}}} = (\underline {{\mathbf {W}}} ^ {(1)}, \dots , \underline {{\mathbf {W}}} ^ {(L)}, \mathbf {w}),
$$
and structure
$$
f (\underline {{\mathbf {x}}}; \underline {{\mathbf {W}}}) = \mathbf {w} ^ {\top} _ {\operatorname {v e c}} (\mathbf {f} (\underline {{\mathbf {x}}}))
$$
$$
\mathbf {f} (\underline {{\mathbf {x}}}) = \mathbf {f} ^ {(L)} (\underline {{\mathbf {x}}}) \in \mathbb {R} ^ {d _ {L} \times 1 \times c}
$$
$$
\mathbf {f} ^ {(l)} (\underline {{\mathbf {x}}}) = \sigma (\underline {{\mathbf {W}}} ^ {(l)} * _ {M} \mathbf {f} ^ {(l - 1)} (\underline {{\mathbf {x}}})) \in \mathbb {R} ^ {d _ {l} \times 1 \times c}, \forall l = 1, \dots , L
$$
$$
\mathbf {f} ^ {(0)} (\underline {{\mathbf {x}}}) = \underline {{\mathbf {x}}}.
$$
Note that according to our construction there is a function  $f(\underline{\mathbf{x}}; \underline{\mathbf{W}}_h) \in \mathfrak{F}$  with weights  $\underline{\mathbf{W}}_h = (\underline{\mathbf{W}}_h^{(1)}, \dots, \underline{\mathbf{W}}_h^{(L)}, \mathbf{w}_h)$  satisfying
$$
(\underline {{\mathbf {W}}} _ {h} ^ {(l)}) _ {i _ {1}, i _ {2}, i _ {3}} = \left\{ \begin{array}{c l} (\underline {{\mathbf {H}}} ^ {(l)}) _ {i _ {1}, i _ {2}, i _ {3}} & \text {i f} i _ {1} \leq m _ {l}, i _ {2} \leq m _ {l - 1}, l \leq J \\ (\underline {{\mathbf {H}}} ^ {(l)}) _ {i _ {1}, i _ {2}, i _ {3}} & \text {i f} i _ {1} \leq m _ {J}, i _ {2} \leq m _ {J}, l = J + 1, \dots , L \\ 0 & \text {o t h e r w i s e} \end{array} \right.
$$