SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Soichiro Kumano
kumano@cvm.t.u-tokyo.ac.jp
Hiroshi Kera
kera@chiba-u.jp
Toshihiko Yamasaki
yamasaki@cvm.t.u-tokyo.ac.jp
Although adversarial training is known to be effective against adversarial examples, training dynamics are not well understood. In this study, we present the first theoretical analysis of adversarial training in random deep neural networks without any assumptions on data distributions. We introduce a new theoretical framework based on mean field theory, which addresses the limitations of existing mean field-based approaches. Based on the framework, we derive the (empirically tight) upper bounds of  $\ell_q$  norm-based adversarial loss with  $\ell_p$  norm-based adversarial examples for various values of  $p$  and  $q$ . Moreover, we prove that networks without shortcuts are generally not adversariably trainable and that adversarial training reduces network capacity. We also show that the network width alleviates these issues. Furthermore, the various impacts of input and output dimensions on the upper bounds and time evolution of weight variance are presented.
Adversarial training [38, 58] is one of the most effective approaches against adversarial examples [89]. Various studies aimed to improve the performance of adversarial training [28, 94, 113], leading to numerous observations. Adversarial training improves accuracy for adversarial examples but decreases it for clean images [58, 88]. Moreover, it requires additional training data [14, 40, 77] and achieves high robust accuracy in a training dataset but not in a test dataset [58]. To improve the reliability of adversarial training and address the aforementioned challenges, a theoretical analysis is essential. Specifically, it is crucial to gain insight into network evolution during adversarial training, conditions for adversarial trainability, and differences between adversarial and standard training.
However, the theoretical understanding of network training is challenging, even for standard training, due to the non-convexity of loss surface and optimization stochasticity. Recent studies employed the mean field theory and analyzed the early stage of standard training for randomly initialized deep neural networks (random networks) [71, 81]. Some studies explored network trainability regarding gradient vanishing/explosion [81, 104, 105] and dynamical isometry [69, 79, 100]. Others examined network representation power [49, 71, 101]. The theoretical results of the early stage of training have been empirically observed to fit well with fully trained networks [25, 84], partially supported by recent theoretical results [46]. However, existing mean field-based approaches cannot manage the probabilistic properties of an entire network (e.g., the distribution of a network Jacobian) and dependence between network inputs and parameters, which are crucial for analyzing adversarial training.
In this study, we propose a mean field-based framework that addresses the aforementioned limitations (Thm 4.1), and apply it to the adversarial training analysis. While previous studies on adversarial
training relies on strong assumptions (e.g., Gaussian data [14, 80] and linear classifiers [76, 107]), the proposed framework includes various scenarios (e.g.,  $\ell_p$  norm-based adversarial examples and deep neural networks with or without shortcuts, i.e., residual or vanilla networks) without any assumptions on data distributions. Our analysis reveals various adversarial training characteristics that have been only experimentally observed or are unknown. The results are summarized as follows.
Upper bounds of adversarial loss. We derive the upper bounds of adversarial loss, quantifying the adverse effect of adversarial examples for various combinations of  $\ell_q$ -adversarial loss with the  $\ell_p$ -norm  $\epsilon$ -ball  $(p,q\in \{1,2,\infty \})$  (Thm 5.1). Numerical experiments confirm the tightness of these bounds. We also investigate the impacts of input and output dimensions on these bounds, and discover that for the  $(p,q) = (2,\infty)$  combination, the bound is independent of these dimensions.
Time evolution of weight variance. We present the time (training step) evolution of weight variance in training (Thms 5.4 and G.9). Weight variance has been used to assess training properties [49, 71, 81]. Our analysis indicates that adversarial training significantly regularizes weights and exhibits consistent weight dynamics across different norm choices.
Vanilla networks are not adversarially trainable under mild conditions. We show that gradient vanishing occurs in vanilla networks (without shortcuts) with large depths and small widths, making them untrainable via adversarial training, even with careful weight initialization (Thm 5.7). This contradicts standard training, where deep vanilla networks can be trained with proper initialization [81, 100]. However, residual networks are adversarially trainable even without proper initialization (Thm 5.8 and Prop G.10), proving the importance of shortcuts for adversarial training.
Degradation of network capacity and role of network width. As adversarial robustness requires high network capacity [63], deep networks can be used in adversarial training. However, we reveal that network capacity, measured based on the Fisher-Rao norm [56], sharply degrades in deep networks during adversarial training (Thms 5.9 and G.14). Specifically, we confirm that the capacity at training step  $t$  is  $\Theta(L - tL^2 / N)$ , where  $L$  and  $N$  denote the network depth and width, respectively. Interestingly, this result contrasts the roles of depth and width, i.e., the depth increases the initial capacity but decays it as training proceeds, whereas the width preserves it. While our theory is validated only during the initial stages of training, our experiments confirm that the adversarial robustness after full training is significantly influenced by network width (cf. Tabs. A5 and A6) as demonstrated in our theorems.
Other contributions. Furthermore, we show the followings cases: (a) Equality of the adversarial loss is obtained instead of inequality (upper bound) under several assumptions (Props G.2 and G.3). (b) Adversarial training leads to faster weight decay and less stable gradients compared with  $\ell_2$  weight regularization. (c) Capacity degradation is discussed for metrics other than the Fisher-Rao norm. (d) Adversarial risk cannot be mitigated while maintaining trainability and expressivity. (e) Discussion on ReLU-like activations extends to Lipschitz continuous activations under certain conditions. (f) A single-gradient descent attack can find adversarial examples that flip the prediction of a binary classifier (Prop K.1). Contributions (b)-(f) are found in Appx. K.
Although this study focuses on adversarial training, our theoretical framework can be applied to other training methods that consider the probabilistic property of an entire network and dependence between network inputs and parameters. Consequently, we believe that this study can potentially contribute to the theoretical understanding of adversarial training and various deep learning methods.
Here, we summarize the full version in Appx. A. A technical discussion follows in Sec. 5.1.
Mean field theory. Mean field theory in machine learning investigates the training dynamics of random networks in chaotic and ordered phases [71]. Networks can be trained at the boundary between these phases [81]. The theory has been extended to networks with shortcuts [105, 106], recurrent connections [15, 69], and batch normalization [104]. It has been employed to study dynamical isometry [15, 69, 70, 79], and a subsequent study achieved training of 10,000-layer networks without
shortcuts [100]. Moreover, the theory has been applied to analyze network representation power [49, 71, 101]. However, existing mean field-based analysis cannot handle the properties of an entire network and input-parameter dependence, which is a drawback for some deep learning methods, e.g., adversarial training. Thus, we propose a new framework to address these limitations.
Adversarial training. Various questions related to adversarial training have been theoretically addressed by some studies, including the robustness-accuracy trade-off [29, 47, 75, 76, 92, 113], generalization gap [6, 50, 102, 107], sample complexity [1, 14, 62, 80, 110], large model requirement [63], and enhanced transfer learning performance [27]. However, these results are obtained in limited settings (e.g., Gaussian data and linear classifiers) and are not easily extended to deep neural networks or realistic data distributions. To explore more general settings, recent studies used the neural tangent kernel theory [4, 46, 55]. In the kernel regime, adversarial training, even with a heuristic attack, finds a robust network [35, 115]. In our study, we investigate adversarial training dynamics based on a mean field perspective, covering general multilayered networks with or without shortcuts and without assumptions about data distributions.
3.1 Setting
Notations are summarized in Tab. A2. For an integer  $n \in \mathbb{N}$ , let  $[n] := \{1, \dots, n\}$ . In this study, we focus on random deep neural networks with ReLU-like activations, called random ReLU-like networks. This is formally defined as follows:
Definition 3.1 (ReLU-like network). A network is called a ReLU-like network if all its activation functions are  $\phi(z) \coloneqq uz$  for  $z \geq 0$  and  $vz$  for  $z < 0$ , with  $u, v \in \mathbb{R}$ .
ReLU-like activations [33, 57] are widely used in theoretical and practical applications [42, 45, 53, 85, 109]. In Appx. K, we extend our theorems to networks with Lipschitz continuous activations.
A ReLU-like network,  $\pmb{f} : \mathbb{R}^d \to \mathbb{R}^K$ , comprises  $L \in \mathbb{N}$  trainable layers and two non-trainable layers for adjusting input and output dimensions. The input layer projects  $\pmb{x}^{\mathrm{in}} \in \mathbb{R}^d$  to an  $N$ -dimensional vector  $\pmb{x}^{(0)} \in \mathbb{R}^N$  using the random matrix  $P^{\mathrm{in}} \in \mathbb{R}^{N \times d}$ . Subsequently,  $L$  consecutive affine transformations and activations are applied by  $g : \mathbb{R}^N \to \mathbb{R}^N$ . Then,  $g(\pmb{x}^{(0)})$  is multiplied by a random matrix  $P^{\mathrm{out}} \in \mathbb{R}^{K \times N}$  to obtain the output vector  $f(\pmb{x}^{\mathrm{in}})$ . Finally, the network function is provided by  $f(\pmb{x}^{\mathrm{in}}) \coloneqq P^{\mathrm{out}} g(P^{\mathrm{in}}\pmb{x}^{\mathrm{in}})$ . We assume that  $d$  and  $K$  are sufficiently large, and each entry of  $P^{\mathrm{in}}$  and  $P^{\mathrm{out}}$  is i.i.d. and sampled from Gaussians  $\mathcal{N}(0,1 / d)$  and  $\mathcal{N}(0,1 / N)$ , respectively.
An  $L$ -layer neural network  $\pmb{g}$  comprises weights  $\pmb{W}^{(l)} = (W_{ij}^{(l)}) \in \mathbb{R}^{N \times N}$  and biases  $\pmb{b}^{(l)} = (b_1^{(l)}, \dots, b_N^{(l)})^\top \in \mathbb{R}^N$ , where  $l \in [L]$  denotes the layer index. The network is assumed to possess a sufficiently large width (i.e.,  $N$  is sufficiently large). The  $l$ -th pre- and post-activation are defined as  $\pmb{h}^{(l)} := \pmb{W}^{(l)}\pmb{x}^{(l-1)} + \pmb{b}^{(l)}$  and  $\pmb{x}^{(l)} := \phi(\pmb{h}^{(l)})$ , respectively, where ReLU-like activation  $\phi$  operates entry-wise. The weight  $W_{ij}^{(l)}$  and bias  $b_i^{(l)}$  are i.i.d. and sampled from  $\mathcal{N}(0, \sigma_w^2/N)$  and  $\mathcal{N}(0, \sigma_b^2)$ , respectively. The network function is represented by Eq. (1). For a residual network setting, refer to Appx. C.
$$
\boldsymbol {f} \left(\boldsymbol {x} ^ {\text {i n}}\right) := P ^ {\text {o u t}} \phi \left(\boldsymbol {W} ^ {(L)} \phi \left(\dots \phi \left(\boldsymbol {W} ^ {(1)} \boldsymbol {P} ^ {\text {i n}} \boldsymbol {x} ^ {\text {i n}} + \boldsymbol {b} ^ {(1)}\right) \dots\right) + \boldsymbol {b} ^ {(L)}\right). \tag {1}
$$
3.2 Background
Mean field theory. Mean field theory employs probabilistic methods to analyze the properties of random deep neural networks. It assumes that  $\pmb{h}^{(l)}$  follows a Gaussian, justified by the central limit theorem when width  $N$  is sufficiently large [71]. Here, we review the mean field-based approach to analyze the forward and backward dynamics of a network. Let  $\mathcal{L}:\mathbb{R}^d\to \mathbb{R}$  represent the loss function. The mean squared pre-activation  $\mathbb{E}[(h_i^{(l)})^2 ]$  and gradient  $\chi^{(l)}\coloneqq \mathbb{E}[(\partial \mathcal{L}(\pmb{x}^{\mathrm{in}}) / \partial x_i^{(l)})^2 ]$ , where  $i$  denotes the neuron index, are calculated as follows [71, 81]:
$$
\mathbb {E} \left[ \left(h _ {i} ^ {(l)}\right) ^ {2} \right] = \sigma_ {w} ^ {2} \mathbb {E} \left[ \phi \left(h _ {i} ^ {(l - 1)}\right) ^ {2} \right] + \sigma_ {b} ^ {2}, \quad \chi^ {(l)} = \sigma_ {w} ^ {2} \mathbb {E} \left[ \phi^ {\prime} \left(h _ {i} ^ {(l + 1)}\right) ^ {2} \right] \chi^ {(l + 1)}. \tag {2}
$$
These equations represent the dynamics between adjacent layers. We can infer that gradients vanish when  $\sigma_w^2\mathbb{E}[\phi '(h_i^{(l + 1)})^2 ] < 1$  and explode when  $\sigma_w^2\mathbb{E}[\phi '(h_i^{(l + 1)})^2 ] > 1$ , indicating that a network is trainable only if  $\sigma_w^2\mathbb{E}[\phi '(h_i^{(l + 1)})^2 ]\approx 1$ .
Adversarial training. We define adversarial loss as follows:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {a d v}} \left(\boldsymbol {x} ^ {\text {i n}}\right) := \max  _ {\| \boldsymbol {\eta} \| _ {p} \leq \epsilon} \left\| \boldsymbol {f} \left(\boldsymbol {x} ^ {\text {i n}} + \boldsymbol {\eta}\right) - \boldsymbol {f} \left(\boldsymbol {x} ^ {\text {i n}}\right) \right\| _ {q} (3) \\ = \max  _ {\| \boldsymbol {\eta} \| _ {p} \leq \epsilon} \left\| \begin{array}{c} \boldsymbol {P} ^ {\text {o u t}} \phi (\boldsymbol {W} ^ {(L)} \phi (\dots \phi (\boldsymbol {W} ^ {(1)} \boldsymbol {P} ^ {\text {i n}} (\boldsymbol {x} ^ {\text {i n}} + \boldsymbol {\eta}) + \boldsymbol {b} ^ {(1)}) \dots) + \boldsymbol {b} ^ {(L)}) \\ - \boldsymbol {P} ^ {\text {o u t}} \phi (\boldsymbol {W} ^ {(L)} \phi (\dots \phi (\boldsymbol {W} ^ {(1)} \boldsymbol {P} ^ {\text {i n}} \boldsymbol {x} ^ {\text {i n}} + \boldsymbol {b} ^ {(1)}) \dots) + \boldsymbol {b} ^ {(L)}) \end{array} \right\| _ {q}, (4) \\ \end{array}
$$
where  $\epsilon > 0$  and  $p, q \in \{1,2,\infty\}$ . The adversarial loss aims to minimize the difference between the network outputs of natural and adversarial inputs. Networks are trained by minimizing the sum of the standard loss  $\mathcal{L}_{\mathrm{std}}: \mathbb{R}^d \times \mathcal{Y} \to \mathbb{R}$  (e.g., cross-entropy loss), where  $\mathcal{Y}$  denotes a label set, and the adversarial loss  $\mathcal{L}_{\mathrm{adv}}$ . The mean field analysis typically assumes gradient independence for loss functions (Appx. B) [81]. We use this assumption for  $\mathcal{L}_{\mathrm{std}}$ , but not  $\mathcal{L}_{\mathrm{adv}}$ . Although Eq. (3) differs from the standard adversarial loss based on cross-entropy [58], our definition is employed in more robust methods, e.g., TRADES [113] and is theoretically simpler to analyze. Therefore, herein, the aforementioned loss is analyzed. However, even this simplified definition (Eq. (3)) poses a challenge for theoretical analysis due to the complex nested structure of a deep neural network (cf. Eq. (4)).
4 Theoretical framework
4.1 Limitations of existing mean field-based approaches
We propose a new theoretical framework based on mean field theory to analyze adversarial training. Here, we describe two limitations of existing mean field-based approaches, e.g., Eq. (2).
Layer-wise approach. Existing approaches focus on the dynamics between adjacent layers (cf. Eq. (2)). However, analyzing the adversarial loss (Eq. (3)) requires a framework that handles the probabilistic properties of an entire network instead of adjacent layers. This analysis becomes difficult due to the complex nested structure of networks (cf. Eqs. (1) and (4)). For example, there is no clarity on the probabilistic behavior of  $f(x^{\mathrm{in}})$ , distribution of  $f(x^{\mathrm{in}} + \eta) - f(x^{\mathrm{in}})$ , and dependence between  $f(x^{\mathrm{in}})$  and inputs. We need a framework that disentangles the nested structure of networks and manages the probabilistic properties of an entire network. Recent studies on the mean field theory studies [15, 36, 69, 70, 100] have concepts related to ours; the comparative analysis is given in Sec. 5.1.
Difficulty in analyzing input-parameter dependence. The analysis of adversarial training requires consideration of input-parameter dependence since adversarial perturbations are designed based on network parameters (cf. Eq. (3)). However, this cannot be readily addressed using existing approaches because their frameworks (e.g., Eq. (2)) do not offer a clear view of the dependence between perturbation  $\eta$  and network parameters  $W_{1,1}^{(1)}, W_{1,2}^{(1)}, \ldots$ , and  $W_{N,N}^{(L)}$ .
The proposed framework resolves these limitations and provides a simple network representation that allows us to capture the entire network property with clear input-parameter dependence.
4.2 Proposed framework
The current mean field-based approaches cannot capture the probabilistic properties of an entire network due to the complex nested structure of deep neural networks. Moreover, it is difficult to consider the dependence between inputs and numerous number of parameters. To address these limitations, we employ a linear-like representation of a ReLU-like network and propose its probabilistic properties. As ReLU-like networks are piecewise linear, a vanilla ReLU-like network can be represented as:
$$
\boldsymbol {f} \left(\boldsymbol {x} ^ {\text {i n}}\right) = \boldsymbol {J} \left(\boldsymbol {x} ^ {\text {i n}}\right) \boldsymbol {x} ^ {\text {i n}} + \boldsymbol {a} \left(\boldsymbol {x} ^ {\text {i n}}\right), \tag {5}
$$
$$
\boldsymbol {J} (\boldsymbol {x} ^ {\mathrm {i n}}) := \boldsymbol {P} ^ {\mathrm {o u t}} \boldsymbol {D} \left(\phi^ {\prime} \left(\boldsymbol {h} ^ {(L)} (\boldsymbol {x} ^ {\mathrm {i n}})\right)\right) \boldsymbol {W} ^ {(L)} \boldsymbol {D} \left(\phi^ {\prime} \left(\boldsymbol {h} ^ {(L - 1)} (\boldsymbol {x} ^ {\mathrm {i n}})\right)\right) \boldsymbol {W} ^ {(L - 1)} \dots \boldsymbol {W} ^ {(1)} \boldsymbol {P} ^ {\mathrm {i n}}, \tag {6}
$$
Figure 1: Distribution of  $J(\pmb{x}^{\mathrm{in}})_{1,1}$  in the vanilla ReLU network with  $d = 1,000$ ,  $K = 1$ ,  $N = 5,000$ ,  $L = 10$ ,  $\sigma_w^2 = 2$ , and  $\sigma_b^2 = 0.01$ . The blue histogram represents the experimental results (10,000-time samplings), and the orange curve is predicted by Thm 4.1.
Table 1: Values of  $\beta_{p,q}$ . Under further assumptions, we can obtain equality of the adversarial loss rather than inequality (upper bound). Values marked with  $\dagger$  represent the equality when  $\epsilon$  is sufficiently small. Values marked with  $\diamond$  are applicable if  $\epsilon$  is sufficiently small and  $K = 1$ .
q=1	q=2	q=∞
p=1	√2K2/πd†	√K/d†	√2ln K/d
p=2	1◇+√K/d	1†◇
p=∞	√2d/π†◇
where  $D(\cdot)$  denotes a diagonal matrix and  $\pmb{a}(\pmb{x}^{\mathrm{in}})$  is defined similar to  $\pmb{J}(\pmb{x}^{\mathrm{in}})$  (cf. Eq. (A48)). For a residual network, Eq. (A73) can be referred. Importantly, this representation does not rely on approximations, e.g., Taylor expansions. As  $\pmb{W}^{(l)}$  and  $\pmb{b}^{(l)}$  are randomly sampled,  $\pmb{J}^{(l)}(\pmb{x}^{\mathrm{in}})$  and  $\pmb{a}^{(l)}(\pmb{x}^{\mathrm{in}})$  denote a random matrix and vector, respectively. Unlike the original network definition (Eq. (1)), this representation (Eq. (5)) is non-nested and focuses only two parameters, thereby simplifying network analysis. Remarkably, we show the following properties of  $\pmb{J}(\pmb{x}^{\mathrm{in}})$  and  $\pmb{a}(\pmb{x}^{\mathrm{in}})$ :
Theorem 4.1 (Properties and distributions of  $J(\pmb{x}^{\mathrm{in}})$  and  $\pmb{a}(\pmb{x}^{\mathrm{in}})$ ). Suppose that the width  $N$  is sufficiently large. Then, for any  $\pmb{x}^{\mathrm{in}} \in \mathbb{R}^{d}$ , (I)  $J(\pmb{x}^{\mathrm{in}})$  and  $\pmb{a}(\pmb{x}^{\mathrm{in}})$  are independent. (II) each entry of  $J(\pmb{x}^{\mathrm{in}})$  and  $\pmb{a}(\pmb{x}^{\mathrm{in}})$  is i.i.d. and follows the Gaussian below:
$$
J \left(\boldsymbol {x} ^ {\text {i n}}\right) _ {i j} \sim \mathcal {N} \left(0, \frac {\omega^ {L}}{d}\right), \quad a \left(\boldsymbol {x} ^ {\text {i n}}\right) _ {i} \sim \mathcal {N} \left(0, \alpha \sigma_ {b} ^ {2} \sum_ {k = 1} ^ {L} \omega^ {k - 1}\right), \tag {7}
$$
where  $\alpha \coloneqq (u^2 + v^2) / 2$  (cf. Defn 3.1) and  $\omega$  is  $\omega_{\mathrm{v}} \coloneqq \alpha \sigma_w^2$  for vanilla networks and  $\omega_{\mathrm{r}} \coloneqq 1 + \alpha \sigma_w^2$  for residual networks.
A significance of this theorem lies in that despite being functions of  $x^{\mathrm{in}}$ , the distributions of  $J(x^{\mathrm{in}})$  and  $a(x^{\mathrm{in}})$  do not depend on  $x^{\mathrm{in}}$ . In other words, although  $J(x^{\mathrm{in}})$  and  $a(x^{\mathrm{in}})$  are determined by (a fixed)  $x^{\mathrm{in}}$  for an initialized network, they become different for each sampling of weights and biases, and the selection of these values is independent of  $x^{\mathrm{in}}$ . Besides,  $J(x^{\mathrm{in}})$  and  $a(x^{\mathrm{in}})$  are independent and their distributions are Gaussian, which exhibits convenient properties. A sketch of proof is given in Appx. D and formal one is in Appxs. E and F.
To validate Thm 4.1, we conducted a numerical experiment and present the results in Fig. 1. We randomly sampled 10,000 vanilla ReLU networks and computed  $J(\pmb{x}^{\mathrm{in}})_{1,1}$  for each network using the identical input  $\pmb{x}^{\mathrm{in}}$ . Additional experimental results can be found in Appx. L.
Broader applicability. The proposed framework (Thm 4.1) manages an entire network using only two Gaussians. While we focus on adversarial training, Thm 4.1 can be valuable for other deep neural network analyses based on the mean field theory. For example, contrastive learning [16] can be investigated, as it aims to minimize the distance between original and positive samples while maximizing it for negative samples. In this context, instead of considering the adversarial loss,  $\| f(x^{\mathrm{in}} + \eta) - f(x^{\mathrm{in}})\|$ , we can analyze loss functions, e.g.,  $\| f(x_{\mathrm{pos}}^{\mathrm{in}}) - f(x_{\mathrm{ori}}^{\mathrm{in}})\|$  and  $\| f(x_{\mathrm{neg}}^{\mathrm{in}}) - f(x_{\mathrm{ori}}^{\mathrm{in}})\|$ , where  $x_{\mathrm{ori}}^{\mathrm{in}}, x_{\mathrm{pos}}^{\mathrm{in}}$ , and  $x_{\mathrm{neg}}^{\mathrm{in}}$  represent the original, positive, and negative samples, respectively. The complex nested structure of a network makes it challenging to consider the difference between two network outputs; however, Thm 4.1 helps theoretically manageable analyses.
5 Analysis of adversarial training
The proof of each theorem is described in Appx. G.
This does not imply that random variables,  $J(\pmb{x})$  and  $J(\pmb{y})$ , are identical for  $\pmb{x} \neq \pmb{y}$ . In addition,  $J(\pmb{x})$  and  $J(\pmb{y})$  are not always independent. Please also refer to the empirical results in Appx. L.
5.1 Upper bounds of adversarial loss
As the ReLU-like network  $\pmb{f}$  is locally linear and its input Jacobian at  $\pmb{x}^{\mathrm{in}}$  is  $J(\pmb{x}^{\mathrm{in}})$  (cf. Eq. (5)), we can consider a more tractable form of the adversarial loss instead of Eq. (4) as follows:
$$
\mathcal {L} _ {\mathrm {a d v}} (\boldsymbol {x} ^ {\mathrm {i n}}) \leq \max  _ {\boldsymbol {x} \in \mathbb {R} ^ {d}, \| \boldsymbol {\eta} \| _ {p} \leq \epsilon} \| \boldsymbol {J} (\boldsymbol {x}) \boldsymbol {\eta} \| _ {q} = \epsilon \max  _ {\boldsymbol {x} \in \mathbb {R} ^ {d}} \| \boldsymbol {J} (\boldsymbol {x}) \| _ {p, q}, \tag {8}
$$
where  $\| J(\pmb{x})\|_{p,q} \coloneqq \max_{\|\pmb{\eta}\|_p = 1} \| J(\pmb{x})\pmb{\eta}\|_q$  denotes the  $(p,q)$ -operator norm of  $J(\pmb{x})$ . Using Thm 4.1, which describes the property of  $J(\pmb{x})$ , we transform Ineq. (8) and obtain the following: