SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] = \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial h _ {j} ^ {(l)}} \frac {\partial h _ {j} ^ {(l)}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] = \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial h _ {j} ^ {(l)}}\right) ^ {2} \right] \mathbb {E} [ (x _ {k} ^ {(l - 1)}) ^ {2} ]. \tag {A198}
$$
By Lemmas E.11 and G.11,
$$
\begin{array}{l} \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] = \alpha \omega_ {\mathrm {v}} ^ {L - l} \chi^ {(L)} \left(\omega_ {\mathrm {v}} ^ {l - 1} \mathbb {E} \left[ \left(x _ {j} ^ {(0)}\right) ^ {2} \right] + \alpha \sigma_ {b} ^ {2} \sum_ {m = 1} ^ {l - 1} \omega_ {\mathrm {v}} ^ {m - 1}\right) (A199) \\ = \alpha \omega_ {\mathrm {v}} ^ {L - 1} \chi^ {(L)} \mathbb {E} \left[ \left(x _ {j} ^ {(0)}\right) ^ {2} \right] + \alpha^ {2} \omega_ {\mathrm {v}} ^ {L - l} \chi^ {(L)} \sigma_ {b} ^ {2} \sum_ {m = 1} ^ {l - 1} \omega_ {\mathrm {v}} ^ {m - 1}. (A200) \\ \end{array}
$$
Residual. By Asm B.1,
$$
\begin{array}{l} \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] = \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial x _ {j} ^ {(l)}} \frac {\partial x _ {j} ^ {(l)}}{\partial h _ {j} ^ {(l)}} \frac {\partial h _ {j} ^ {(l)}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] (A201) \\ = \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial x _ {j} ^ {(l)}}\right) ^ {2} \phi^ {\prime} \left(h _ {j} ^ {(l)}\right) ^ {2} \left(x _ {k} ^ {(l - 1)}\right) ^ {2} \right] (A202) \\ = \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial x _ {j} ^ {(l)}}\right) ^ {2} \right] \mathbb {E} \left[ \phi^ {\prime} \left(h _ {j} ^ {(l)}\right) ^ {2} \right] \mathbb {E} \left[ \left(x _ {k} ^ {(l - 1)}\right) ^ {2} \right] (A203) \\ = \chi^ {(l)} \mathbb {E} \left[ \phi^ {\prime} \left(h _ {j} ^ {(l)}\right) ^ {2} \right] \mathbb {E} \left[ \left(x _ {k} ^ {(l - 1)}\right) ^ {2} \right]. (A204) \\ \end{array}
$$
By Eq. (A38),
$$
\mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] = \alpha \chi^ {(l)} \mathbb {E} \left[ \left(x _ {k} ^ {(l - 1)}\right) ^ {2} \right]. \tag {A205}
$$
By Eqs. (A19) and (A38),
$$
\chi^ {(l)} = \omega_ {\mathrm {r}} ^ {L - l} \chi^ {(L)}. \tag {A206}
$$
Thus, by Lemma E.11,
$$
\begin{array}{l} \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial W _ {j k} ^ {(l)}}\right) ^ {2} \right] = \alpha \omega_ {\mathrm {r}} ^ {L - l} \chi^ {(L)} \left(\omega_ {\mathrm {r}} ^ {l - 1} \mathbb {E} \left[ \left(x _ {j} ^ {(0)}\right) ^ {2} \right] + \alpha \sigma_ {b} ^ {2} \sum_ {m = 1} ^ {l - 1} \omega_ {\mathrm {r}} ^ {m - 1}\right) (A207) \\ = \alpha \omega_ {\mathrm {r}} ^ {L - 1} \chi^ {(L)} \mathbb {E} [ (x ^ {(0)}) ^ {2} ] + \alpha^ {2} \omega_ {\mathrm {r}} ^ {L - l} \chi^ {(L)} \sigma_ {b} ^ {2} \sum_ {m = 1} ^ {l - 1} \omega_ {\mathrm {r}} ^ {m - 1}. (A208) \\ \end{array}
$$
Now, we can simply represent the mean of the Fisher-Rao norm.
Lemma G.13. Suppose that Asm B.1 is applied to the network output. Suppose  $\left\| \boldsymbol{x}^{\mathrm{in}}\right\| _2 = \sqrt{d}$  and  $\sigma_b^2 = 0$ . The mean of the Fisher-Rao norm is given by:
$$
\mathbb {E} \left[ \| \boldsymbol {w} \| _ {\mathrm {F R}} \right] = L K \alpha \sigma_ {w} ^ {2} \omega^ {L - 1}. \tag {A209}
$$
Proof. The following discussion is based on [49]. We rearrange and expand their discussion for a ReLU-like network. We also consider it for a residual network. Note that  $\boldsymbol{w} := (W_{11}^{(1)},\dots ,W_{NN}^{(L)})^{\top}$ . As a preliminary, see Remark E.9. The Fisher-Rao norm is calculated as follows:
$$
\left\| \boldsymbol {w} \right\| _ {\mathrm {F R}} := \boldsymbol {w} ^ {\top} \boldsymbol {F} \boldsymbol {w} = \sum_ {i = 1} ^ {L N ^ {2}} \sum_ {j = 1} ^ {L N ^ {2}} w _ {i} F _ {i j} w _ {j} = \sum_ {i = 1} ^ {L N ^ {2}} \sum_ {j = 1} ^ {L N ^ {2}} \sum_ {k = 1} ^ {K} \frac {\partial f _ {k}}{\partial w _ {i}} \frac {\partial f _ {k}}{\partial w _ {j}} w _ {i} w _ {j}. \tag {A210}
$$
By Asm B.1,
$$
\begin{array}{l} \mathbb {E} [ \| \boldsymbol {w} \| _ {\mathrm {F R}} ] = \sum_ {i = 1} ^ {L N ^ {2}} \sum_ {j = 1} ^ {L N ^ {2}} \sum_ {k = 1} ^ {K} \mathbb {E} \left[ \frac {\partial f _ {k}}{\partial w _ {i}} \frac {\partial f _ {k}}{\partial w _ {j}} w _ {i} w _ {j} \right] (A211) \\ = \sum_ {i = 1} ^ {L N ^ {2}} \sum_ {j = 1} ^ {L N ^ {2}} \sum_ {k = 1} ^ {K} \mathbb {E} \left[ \frac {\partial f _ {k}}{\partial w _ {i}} \frac {\partial f _ {k}}{\partial w _ {j}} \right] \mathbb {E} \left[ w _ {i} w _ {j} \right] (A212) \\ = K \frac {\sigma_ {w} ^ {2}}{N} \sum_ {i = 1} ^ {L N ^ {2}} \mathbb {E} \left[ \left(\frac {\partial f _ {k}}{\partial w _ {i}}\right) ^ {2} \right]. (A213) \\ \end{array}
$$
By Lemma G.12,  $\mathbb{E}[(\partial f_i / \partial W_{jk}^{(l)})^2 ]$  is invariant to  $l$  if  $\sigma_b^2 = 0$ . In other words,  $\mathbb{E}[(\partial f_k / \partial w_i)^2 ]$  does not depend on  $i$  if  $\sigma_b^2 = 0$ . Thus,
$$
\mathbb {E} [ \| \boldsymbol {w} \| _ {\mathrm {F R}} ] = L N K \sigma_ {w} ^ {2} \mathbb {E} \left[ \left(\frac {\partial f _ {k}}{\partial w _ {i}}\right) ^ {2} \right] = L N K \alpha \sigma_ {w} ^ {2} \omega^ {L - 1} \chi^ {(L)} \mathbb {E} [ (x _ {i} ^ {(0)}) ^ {2} ]. \tag {A214}
$$
$$
\chi^ {(L)} := \mathbb {E} \left[ \left(\frac {\partial f _ {i}}{\partial x _ {j} ^ {(L)}}\right) ^ {2} \right] = \mathbb {E} \left[ \left(\frac {\partial \sum_ {k = 1} ^ {N} P _ {i k} ^ {\mathrm {o u t}} x _ {k} ^ {(L)}}{\partial x _ {j} ^ {(L)}}\right) ^ {2} \right] = \mathbb {E} [ (P _ {i j} ^ {\mathrm {o u t}}) ^ {2} ] = \frac {1}{N}, \tag {A215}
$$
$$
\mathbb {E} \left[ \left(x _ {i} ^ {(0)}\right) ^ {2} \right] = \mathbb {E} \left[ \left(\sum_ {j = 1} ^ {d} P _ {i j} ^ {\text {i n}} x _ {j} ^ {\text {i n}}\right) ^ {2} \right] = \sum_ {j = 1} ^ {d} \mathbb {E} \left[ \left(P _ {i j} ^ {\text {i n}}\right) ^ {2} \right] \left(x _ {j} ^ {\text {i n}}\right) ^ {2} = 1. \tag {A216}
$$
$$
\mathbb {E} \left[ \| \boldsymbol {w} \| _ {\mathrm {F R}} \right] = L K \alpha \sigma_ {w} ^ {2} \omega^ {L - 1}. \tag {A217}
$$
Theorem 5.9 (Adversarial training degrades network capacity). Consider a vanilla network. Suppose that Asms 5.2 and 5.3 hold, Asm B.1 is applied to the network output, and the  $(M,m)$ -trainability condition holds at  $t = 0$  and  $\alpha \sigma_w^2(0) = 1$ . Assume  $\left\| \mathbf{x}^{\mathrm{in}} \right\|_2 = \sqrt{d}$  and  $\sigma_b^2(t) = 0$ . Then, the expectation of the Fisher-Rao norm is given by:
Proof. By Lemma G.13,
$$
\mathbb {E} \left[ \| \boldsymbol {w} \| _ {\mathrm {F R}} \right] = L K \alpha \sigma_ {w} ^ {2} (t) \omega_ {\mathrm {v}} (t) ^ {L - 1} = L K \alpha^ {L} \sigma_ {w} ^ {2} (t) ^ {L}. \tag {A218}
$$
By Thm 5.4,
$$
\mathbb {E} [ \| \boldsymbol {w} \| _ {\mathrm {F R}} ] = L K \alpha^ {L} \sigma_ {w} ^ {2} (0) ^ {L} \left(1 - \frac {\epsilon \alpha \beta_ {p , q} t}{N}\right) ^ {L} = L K \left(1 - \frac {\epsilon \alpha \beta_ {p , q} t}{N}\right) ^ {L}. \tag {A219}
$$
By  $\epsilon \alpha \beta_{p,q}t / N\ll 1$
$$
\mathbb {E} [ \| \boldsymbol {w} \| _ {\mathrm {F R}} ] \approx L K \left(1 - \frac {L \epsilon \alpha \beta_ {p , q} t}{N}\right). \tag {A220}
$$
Theorem G.14 (Adversarial training degrades network capacity). Consider a residual network. Suppose that Asm 5.2 holds, Asm B.1 is applied to the network output, and the  $(M,m)$ -trainability condition holds at  $t = 0$ , e.g.,  $\alpha \sigma_w^2(0) \ll 1$ . Suppose  $\left\| \mathbf{x}^{\mathrm{in}} \right\|_2 = \sqrt{d}$  and  $\sigma_b^2(t) = 0$ . Then, the mean of the Fisher-Rao norm is given by:
$$
\mathbb {E} [ \| \pmb {w} \| _ {\mathrm {F R}} ] = L K \alpha \sigma_ {w} ^ {2} (0) \left(1 + (L - 1) \alpha \sigma_ {w} ^ {2} (0) - (2 (L - 1) \alpha \sigma_ {w} ^ {2} (0) + 1) \left(1 + \left(\frac {L}{2} - 1\right) \alpha \sigma_ {w} ^ {2} (0)\right) \frac {\epsilon \alpha \beta_ {p , q} t}{N}\right).
$$
$$
\begin{array}{l} \mathbb {E} \left[ \| \boldsymbol {w} \| _ {\mathrm {F R}} \right] = L K \alpha \sigma_ {w} ^ {2} (t) \omega_ {\mathrm {r}} (t) ^ {L - 1} (A221) \\ = L K \alpha \sigma_ {w} ^ {2} (t) \left(1 + \alpha \sigma_ {w} ^ {2} (t)\right) ^ {L - 1} (A222) \\ \approx L K \alpha \sigma_ {w} ^ {2} (t) (1 + (L - 1) \alpha \sigma_ {w} ^ {2} (t)). (A223) \\ \end{array}
$$
Temporally, denote  $A \coloneqq (1 + (L / 2 - 1)\alpha \sigma_w^2 (0))\epsilon \alpha \beta_{p,q}t / N$ . By Thm G.9,
$$
\begin{array}{l} \mathbb {E} [ \| \boldsymbol {w} \| _ {\mathrm {F R}} ] = L K \alpha (1 - A) \sigma_ {w} ^ {2} (0) (1 + (L - 1) \alpha (1 - A) \sigma_ {w} ^ {2} (0)) (A224) \\ = L K \alpha \sigma_ {w} ^ {2} (0) (1 - A) (1 + (L - 1) \alpha \sigma_ {w} ^ {2} (0) - (L - 1) A \alpha \sigma_ {w} ^ {2} (0)) (A225) \\ = L K \alpha \sigma_ {w} ^ {2} (0) \binom {1 - A + (L - 1) (1 - A) \alpha \sigma_ {w} ^ {2} (0)} {- (L - 1) A \alpha \sigma_ {w} ^ {2} (0) + \mathcal {O} \left(A ^ {2}\right)}. (A226) \\ \end{array}
$$
By  $\mathcal{O}(1 / N^2)\approx 0,\mathcal{O}(A^2)\approx 0$  . Thus,
$$
\begin{array}{l} \mathbb {E} [ \| \boldsymbol {w} \| _ {\mathrm {F R}} ] \approx L K \alpha \sigma_ {w} ^ {2} (0) (1 - A + (L - 1) (1 - A) \alpha \sigma_ {w} ^ {2} (0) - (L - 1) A \alpha \sigma_ {w} ^ {2} (0)) (A227) \\ = L K \alpha \sigma_ {w} ^ {2} (0) (1 - A + (L - 1) \alpha \sigma_ {w} ^ {2} (0) - 2 (L - 1) A \alpha \sigma_ {w} ^ {2} (0)) (A228) \\ = L K \alpha \sigma_ {w} ^ {2} (0) (1 + (L - 1) \alpha \sigma_ {w} ^ {2} (0) - (2 (L - 1) \alpha \sigma_ {w} ^ {2} (0) + 1) A) (A229) \\ = L K \alpha \sigma_ {w} ^ {2} (0) \binom {1 + (L - 1) \alpha \sigma_ {w} ^ {2} (0)} {- (2 (L - 1) \alpha \sigma_ {w} ^ {2} (0) + 1) \left(1 + \left(\frac {L}{2} - 1\right) \alpha \sigma_ {w} ^ {2} (0)\right) \frac {\epsilon \alpha \beta_ {p , q} t}{N}}. (A230) \\ \end{array}
$$
H Comparison with matrix decomposition approaches
Here, we analyze Ineq. (8) with another approach, which decomposes  $J(\pmb{x}^{\mathrm{in}})$  into products of matrices, and consider norm of each matrix. As related studies based on this approach, we cite the literature on certified adversarial defense [3, 18, 93] and spectral regularization [60, 108]. We consider the case of  $(p,q) = (2,2)$  for simplicity. In comparison to them, our approach is different in two ways. First, their bound is not tractable due to a deterministic approach. For example, they consider the spectral norm of  $J(\pmb{x}^{\mathrm{in}})$  by decomposing  $J(\pmb{x}^{\mathrm{in}})$  and computing the norm of each matrix consisting  $J(\pmb{x}^{\mathrm{in}})$  as follows:
$$
\begin{array}{l} \left\| \boldsymbol {J} \left(\boldsymbol {x} ^ {\text {i n}}\right) \right\| _ {2} = \left\| \boldsymbol {P} ^ {\text {o u t}} \boldsymbol {D} \left(\phi^ {\prime} \left(\boldsymbol {h} ^ {(L)} \left(\boldsymbol {x} ^ {\text {i n}}\right)\right)\right) \boldsymbol {W} ^ {(L)} \dots \boldsymbol {D} \left(\phi^ {\prime} \left(\boldsymbol {h} ^ {(1)} \left(\boldsymbol {x} ^ {\text {i n}}\right)\right)\right) \boldsymbol {W} ^ {(1)} \boldsymbol {P} ^ {\text {i n}} \right\| _ {2} (A231) \\ \leq \left\| \boldsymbol {P} ^ {\text {o u t}} \right\| _ {2} \left\| \boldsymbol {D} \left(\phi^ {\prime} \left(\boldsymbol {h} ^ {(L)} (\boldsymbol {x} ^ {\text {i n}})\right)\right) \right\| _ {2} \left\| \boldsymbol {W} ^ {(L)} \right\| _ {2} \\ \dots \left\| D \left(\phi^ {\prime} \left(h ^ {(1)} \left(x ^ {\text {i n}}\right)\right)\right) \right\| _ {2} \left\| W ^ {(1)} \right\| _ {2} \left\| P ^ {\text {i n}} \right\| _ {2} (A232) \\ \leq \max  (| u |, | v |) ^ {L} \| P ^ {\text {i n}} \| _ {F} \| P ^ {\text {o u t}} \| _ {F} \prod_ {l = 1} ^ {L} \| W ^ {(l)} \| _ {F}. (A233) \\ \end{array}
$$
Because Eqs. (A232) and (A233) are hard to theoretically manage, we cannot derive some theorems in this study such as Thms 5.4, 5.7 and 5.9. Second, because we calculate the spectral norm of  $J(\pmb{x}^{\mathrm{in}})$  directly in Thm 5.1 instead of considering the norm of each matrix and the spectral norm is submultiplicative, our upper bounds are always tighter than those derived from Eq. (A232).
Here, we proceed with the discussion of tightness of their bound using probabilistic theory. One approach is the rearrangement of Eq. (A232) using the Marchenko-Pastur law. As a preliminary, we note the following:
$$
\left\| \boldsymbol {P} ^ {\text {i n}} \right\| _ {2} \leq 1 + \sqrt {\frac {N}{d}}, \quad \left\| \boldsymbol {P} ^ {\text {o u t}} \right\| _ {2} \leq 1 + \sqrt {\frac {K}{N}}, \quad \left\| \boldsymbol {W} ^ {(l)} \right\| _ {2} \leq 2 \sqrt {\sigma_ {w} ^ {2}}. \tag {A234}
$$
Using the above equations, Eq. (A232) can be rearranged as follows:
$$
\left\| \boldsymbol {J} \left(\boldsymbol {x} ^ {\text {i n}}\right) \right\| _ {2} \leq \max  (| u |, | v |) ^ {L} \left(1 + \sqrt {\frac {N}{d}}\right) \left(1 + \sqrt {\frac {K}{N}}\right) 2 ^ {L} \left(\sigma_ {w} ^ {2}\right) ^ {L / 2}. \tag {A235}
$$
This is exponentially looser than Thm 5.1.
Another approach is the rearrangement of Eq. (A233) using the assumption that each matrix is sufficiently large. As a preliminary, we note the following:
$$
\left\| \boldsymbol {P} ^ {\text {i n}} \right\| _ {\mathrm {F}} = \sqrt {N d \frac {1}{N d} \sum_ {i = 1} ^ {N} \sum_ {j = 1} ^ {d} \left(P _ {i j} ^ {\text {i n}}\right) ^ {2}} = \sqrt {N d \mathbb {E} \left[ \left(P _ {i j} ^ {\text {i n}}\right) ^ {2} \right]} = \sqrt {N d \frac {1}{d}} = \sqrt {N}, \tag {A236}
$$
$$
\left\| \boldsymbol {P} ^ {\text {o u t}} \right\| _ {\mathrm {F}} = \sqrt {K N \frac {1}{K N} \sum_ {i = 1} ^ {K} \sum_ {j = 1} ^ {N} \left(P _ {i j} ^ {\text {o u t}}\right) ^ {2}} = \sqrt {K N \mathbb {E} [ \left(P _ {i j} ^ {\text {o u t}}\right) ^ {2} ]} = \sqrt {K N \frac {1}{N}} = \sqrt {K}, \tag {A237}
$$
$$
\left\| \boldsymbol {W} ^ {(l)} \right\| _ {\mathrm {F}} = \sqrt {N ^ {2} \frac {1}{N ^ {2}} \sum_ {i = 1} ^ {N} \sum_ {j = 1} ^ {N} \left(\boldsymbol {W} _ {i j} ^ {(l)}\right) ^ {2}} = \sqrt {N ^ {2} \mathbb {E} [ \left(\boldsymbol {W} _ {i j} ^ {(l)}\right) ^ {2} ]} = \sqrt {N ^ {2} \frac {\sigma_ {w} ^ {2}}{N}} = \sqrt {N \sigma_ {w} ^ {2}}. \tag {A238}
$$
Using the above equations, Eq. (A233) can be rearranged as follows:
$$
\left\| \boldsymbol {J} \left(\boldsymbol {x} ^ {\text {i n}}\right) \right\| _ {2} \leq \max  (| u |, | v |) ^ {L} \sqrt {N} \sqrt {K} \sqrt {N \sigma_ {w} ^ {2}} ^ {L} = \max  (| u |, | v |) ^ {L} K ^ {1 / 2} N ^ {(L + 1) / 2} \left(\sigma_ {w} ^ {2}\right) ^ {L / 2}. \tag {A239}
$$
This is also exponentially looser than Thm 5.1.
I Comparison with  $\ell_2$  weight regularization
As stated in Secs. 5.1 and 5.2, adversarial training serves the role of a weight regularizer. Here, we compare adversarial training with  $\ell_2$  weight regularization. First, we define  $\ell_2$  weight regularization as follows:
$$
\mathcal {L} _ {\mathrm {w}} (\mathcal {W}; \lambda) := \lambda \sum_ {W \in \mathcal {W}} W ^ {2}, \tag {A240}
$$
where  $\lambda > 0$  is a scaling factor. Then, we determine the scaling factor  $\lambda$  based on the concept of gradient vanishing and explosion. The mean of  $\mathcal{L}_{\mathrm{w}}(\mathcal{W};\lambda)$  is calculated as follows:
$$
\mathbb {E} \left[ \mathcal {L} _ {\mathrm {w}} (\mathcal {W}; \lambda) \right] = \lambda \sum_ {W \in \mathcal {W}} \mathbb {E} \left[ W ^ {2} \right] = \lambda L N ^ {2} \mathbb {E} \left[ W ^ {2} \right] = \lambda L N ^ {2} \frac {\sigma_ {w} ^ {2}}{N} = \lambda L N \sigma_ {w} ^ {2}. \tag {A241}
$$
To prevent (gradient) vanishing and explosion of  $\mathbb{E}[\mathcal{L}_{\mathrm{w}}(\mathcal{W};\lambda)]$  under sufficiently large  $L$  and  $N$ ,  $\lambda$  should be  $1 / (LN)$ . Moreover, for simplicity of the derivation, we set  $\lambda \coloneqq 1 / (2LN)$ . Finally,  $\ell_2$  regularized training tries to minimize the following loss function:
$$
\mathcal {L} := \mathcal {L} _ {\mathrm {s t d}} + \frac {1}{2 L N} \sum_ {W \in \mathcal {W}} W ^ {2} \tag {A242}
$$
Next, we consider the time evolution of  $\sigma_w^2$  with  $\ell_2$  weight regularization. Similar to Thms 5.4 and G.9, we can derive the following proposition:
Proposition I.1 (Weight time evolution with  $\ell_2$  weight regularization). Suppose that Asm 5.2 holds. Let  $\mathcal{L}_{\mathrm{std}}$  be the standard loss function and  $\mathcal{L}_{\mathrm{w}}(\mathcal{W}) := 1 / (2LN)\sum_{W\in \mathcal{W}}W^2$  be the  $\ell_2$  regularization loss function. Suppose that Asm B.1 applies to  $\mathcal{L}_{\mathrm{std}}$ , but not to  $\mathcal{L}_{\mathrm{w}}$ . A network is trained by minimizing  $\mathcal{L}_{\mathrm{std}} + \mathcal{L}_{\mathrm{w}}(\mathcal{W})$ . Then, the time evolution of  $\sigma_w^2$  is given by:
$$
\sigma_ {w} ^ {2} (t) = \left(1 - \frac {t}{L N}\right) \sigma_ {w} ^ {2} (0). \tag {A243}
$$
Proof. Similar to Lemma G.8,
$$
\begin{array}{l} \frac {\mathrm {d} \sigma_ {w} ^ {2} (t)}{\mathrm {d} t} = - N \mathbb {E} \left[ W (t) \frac {\partial \mathcal {L} _ {\mathrm {w}} (\mathcal {W})}{\partial W (t)} \right] (A244) \\ = - N \mathbb {E} \left[ W (t) \frac {W (t)}{L N} \right] (A245) \\ = - \frac {1}{L} \mathbb {E} [ W (t) ^ {2} ] (A246) \\ = - \frac {1}{L N} \sigma_ {w} ^ {2} (t). (A247) \\ \end{array}
$$
$$
\sigma_ {w} ^ {2} (t) = \exp \left(- \frac {t}{L N}\right) \sigma_ {w} ^ {2} (0). \tag {A248}
$$
By  $t \leq T \ll N$ , using Maclaurin expansion of the exponential function,
$$
\sigma_ {w} ^ {2} (t) = \left(1 - \frac {t}{L N}\right) \sigma_ {w} ^ {2} (0). \tag {A249}
$$
Here, we consider the adversarial loss defined as  $\mathcal{L}_{\mathrm{adv}}(\pmb{x}^{\mathrm{in}}) \coloneqq \epsilon \beta_{p,q}\omega^{L/2}$  (cf. Thm 5.1). The difference between adversarial training and  $\ell_2$  regularization lies in the scale:  $\epsilon \alpha \beta_{p,q}/N$  in adversarial training (cf. Thm 5.4) and  $1/(LN)$  in  $\ell_2$  regularized training. Adversarial training with strong adversarial examples (e.g.,  $\ell_{\infty}$  constrained ones) reduces  $\sigma_w^2(t)$  more drastically than  $\ell_2$  regularized training. For example, with  $L = 100$ ,  $N = 1,000$ ,  $d = 3\times 224\times 224$ ,  $\epsilon = 0.1$ ,  $\alpha = 1/2$ ,  $p = \infty$ ,  $q = \infty$ , then  $\Theta(\epsilon \alpha \beta_{p,q}/N) = 10^{-2}$  and  $\Theta(1/(LN)) = 10^{-5}$ .
Finally, we compare adversarial training and  $\ell_2$  regularization in terms of stability of gradient descent. The derivations of both losses with respect to the weight  $W\in \mathcal{W}$  are given by (cf. Lemma G.7):
$$
\frac {\partial \mathcal {L} _ {\mathrm {a d v}}}{\partial W} = \frac {\epsilon \alpha \beta_ {p , q} \omega^ {L / 2 - 1} W}{N}, \quad \frac {\partial \mathcal {L} _ {\mathrm {w}}}{\partial W} = \frac {W}{L N}. \tag {A250}
$$
This indicates that the derivation of  $\ell_2$  regularizer is smooth across the entire weight space, whereas the derivation of adversarial loss changes significantly at the  $\omega = 1$  boundary. Note that  $\omega$  evolves during training (cf. the definition in Thm 4.1). The steep derivation in adversarial training prevents the gradient descent from reaching a minimum.
J Revisiting trainability condition in adversarial training
Here, we revisit Defn 5.5. Recall that this condition can be derived exclusively under Asm B.1. This is because  $\chi^{(l)}$  can only be computed under Asm B.1 [81]. In this study, we assume that Asm B.1 applies to the standard loss  $\mathcal{L}_{\mathrm{std}}$ , but not to the adversarial loss  $\mathcal{L}_{\mathrm{adv}}$ . Consequently, the composite loss  $\mathcal{L} \coloneqq \mathcal{L}_{\mathrm{std}} + \mathcal{L}_{\mathrm{adv}}$  does not satisfy Asm B.1, implying that strictly speaking, Defn 5.5 is not valid for  $\mathcal{L}$  and considering Defn 5.5 in adversarial training might not be entirely accurate.
However, we must emphasize that Defn 5.5 continues to be a determining factor in the success of adversarial training, or in other words, training with  $\mathcal{L}$ . This is primarily due to the fact that if Defn 5.5 is not met, networks will be unable to adequately fit the training dataset owing to gradient vanishing or explosion.
Examining this from a mathematical standpoint, the necessity of Defn 5.5 is underlined by the linearity of the differential operator. The update equation for the parameter  $\theta(t)$  is formulated as follows:
$$
\frac {\mathrm {d} \theta (t)}{\mathrm {d} t} := - \frac {\partial \mathcal {L}}{\partial \theta (t)} = - \frac {\partial \mathcal {L} _ {\mathrm {s t d}}}{\partial \theta (t)} - \frac {\partial \mathcal {L} _ {\mathrm {a d v}}}{\partial \theta (t)} \tag {A251}
$$
As demonstrated by the equation above, updates with  $\mathcal{L}_{\mathrm{std}}$  and  $\mathcal{L}_{\mathrm{adv}}$  are conducted independently. Although Defn 5.5 may not be considered for  $\mathcal{L}_{\mathrm{adv}}$ , it can be for  $\mathcal{L}_{\mathrm{std}}$ . In order to forestall gradient vanishing or explosion in the standard loss and to ensure classification ability for clean images, Defn 5.5 remains a necessity. Therefore, it can be concluded that Defn 5.5 continues to dictate the success of training, even within the context of adversarial training.
K Other theoretical results
Scaled effect of input and output dimensions. In Tab. 1, we showed a wide effect of the input dimension  $d$  and the output dimension  $K$  on the upper bounds of the adversarial loss. In practice, the perturbation budget  $\epsilon$  and the adversarial loss are scaled based on the chosen  $p$  and  $q$ . For example, we can rearrange the definition of the adversarial loss (Eq. (3)) as follows:
$$
\mathcal {L} _ {\mathrm {a d v}} \left(\boldsymbol {x} ^ {\mathrm {i n}}\right) := \max  _ {\| \boldsymbol {\eta} \| _ {p} \leq \lambda_ {p} \epsilon} \lambda_ {q} \| \boldsymbol {f} \left(\boldsymbol {x} ^ {\mathrm {i n}} + \boldsymbol {\eta}\right) - \boldsymbol {f} \left(\boldsymbol {x} ^ {\mathrm {i n}}\right) \| _ {q}, \tag {A252}
$$
Table A4: Values of  $\beta_{p,q}^{\prime}$ . The description is the same as Tab. 1
q=1	q=2	q=∞
p=1	\( \sqrt{\frac{2d}{\pi}}^\dagger \)	\( \sqrt{d}^\dagger \)	\( \sqrt{2d\ln K} \)
p=2	\( \sqrt{\frac{d}{K}}^\diamond + 1 \)	\( \sqrt{d}^\dagger \diamond \)
p=∞	\( \sqrt{\frac{2d}{\pi}}^\dagger \diamond \)
where  $\lambda_{p}$  and  $\lambda_{q}$  denote the scaling factors. In this context, the upper bound (Thm 5.1) can be rearranged as follows:
$$
\mathcal {L} _ {\mathrm {a d v}} \left(\boldsymbol {x} ^ {\text {i n}}\right) \leq \epsilon \cdot \lambda_ {p} \lambda_ {q} \beta_ {p, q} \cdot \omega^ {L / 2} = \epsilon \beta_ {p, q} ^ {\prime} \omega^ {L / 2}, \tag {A253}
$$
where  $\beta_{p,q}^{\prime}\coloneqq \lambda_{p}\lambda_{q}\beta_{p,q}$ . For example, we consider the following  $\lambda_{p}$  and  $\lambda_q$ :
$$
\lambda_ {p} := \left\{ \begin{array}{l l} d & (p = 1) \\ \sqrt {d} & (p = 2) \\ 1 & (p = \infty) \end{array} , \quad \lambda_ {q} := \left\{ \begin{array}{l l} 1 / K & (q = 1) \\ 1 / \sqrt {K} & (q = 2) \\ 1 & (q = \infty) \end{array} . \right. \right. \tag {A254}
$$
Under these conditions,  $\beta_{p,q}^{\prime}$  corresponds to Tab. A4. This table reveals that (i) the input dimension impacts the upper bounds with  $\Theta (\sqrt{d})$ , but the output dimension has little or no effects; (ii) although the dimensions generally exert a similar influence on the bounds, the specific factors vary widely; (iii) for  $(p,q) = (1,\infty)$  and  $(2,2)$ , the output dimension behaves in a distinctive manner. In conclusion, our upper bounds, even inclusive of scaling effects, provide several insights into the relationship between the adversarial loss and input/output dimensions.