Year: 2019

Abstract:
The vulnerability to slight input perturbations is a worrying yet intriguing property of deep neural networks (DNNs). Though some efforts have been devoted to investigating the reason behind such adversarial behavior, the relation between standard accuracy and adversarial behavior of DNNs is still little understood. In this work, we reveal such relation by first introducing a metric characterizing the standard performance of DNNs. Then we theoretically show this metric can be disentangled into an information-theoretic non-robust component that is related to adversarial behavior, and a robust component. Then, we show by experiments that DNNs under standard training rely heavily on optimizing the non-robust component in achieving decent performance. We also demonstrate current state-ofthe-art adversarial training algorithms indeed try to robustify DNNs by preventing them from using the non-robust component to distinguish samples from different categories. Based on our findings, we take a step forward and point out the possible direction of simultaneously achieving decent standard generalization and adversarial robustness. It is hoped that our theory can further inspire the community to make more interesting discoveries about the relation between standard accuracy and adversarial robustness of DNNs.

text: The vulnerability to slight input perturbations is a worrying yet intriguing property of deep neural networks (DNNs). Though some efforts have been devoted to investigating the reason behind such adversarial behavior, the relation between standard accuracy and adversarial behavior of DNNs is still little understood. In this work, we reveal such relation by first introducing a metric characterizing the standard performance of DNNs. Then we theoretically show this metric can be disentangled into an information-theoretic non-robust component that is related to adversarial behavior, and a robust component. Then, we show by experiments that DNNs under standard training rely heavily on optimizing the non-robust component in achieving decent performance. We also demonstrate current state-ofthe-art adversarial training algorithms indeed try to robustify DNNs by preventing them from using the non-robust component to distinguish samples from different categories. Based on our findings, we take a step forward and point out the possible direction of simultaneously achieving decent standard generalization and adversarial robustness. It is hoped that our theory can further inspire the community to make more interesting discoveries about the relation between standard accuracy and adversarial robustness of DNNs.

heading: 1 INTRODUCTION
text: Deep neural networks (DNNs) have achieved wide success over the last decade. In literature, the majority of deep learning models pursue boosted performance from different aspects (Krizhevsky et al., 2012; Simonyan & Zisserman, 2014; Szegedy et al., 2015; He et al., 2016; 2015; Ioffe & Szegedy, 2015). However, it is found these powerful models are susceptible to perturbations, even those imperceptible to humans (Szegedy et al., 2013). Therefore beyond the main research stream, there are also works devoted to investigating effective attacks (Goodfellow et al., 2014; MoosaviDezfooli et al., 2016; Kurakin et al., 2016; Zhao et al., 2018) and designing adversarial robust models (Goodfellow et al., 2014; Miyato et al., 2015; Madry et al., 2017; Zhang et al., 2019).
It is observed that the adversarial robustness is usually achieved at the cost of a non-trivial degradation of standard performance. Previous efforts (Schmidt et al., 2018; Tsipras et al., 2018; Nakkiran, 2019) trying to understand this phenomenon are usually based on simple toy models or heavy assumptions, and do not provide a general theoretic framework that explicitly shows how adversarial robustness is related to standard generalization. The work (Ilyas et al., 2019) empirically shows the adversarial samples might be human imperceptible features that could help generalization, but it does not provide any general theoretic framework to properly explain this phenomenon. In another work (Zhang et al., 2019), though a bound on the gap between standard accuracy and adversarial accuracy is derived so that the gap between these two quantities could be explicitly controlled, the underlying reason of such trade-off is still not fully characterized.
In this work, we start from a new perspective and consider the performance of deep classification models with a new metric Cross Category Kullback-Leibler divergence (CCKL), namely the Kullback–Leibler (KL) divergence between the model’s output distributions over input data from different categories, instead of the traditional metric of accuracy. Interestingly, by applying Taylor expansion on CCKL, we show that it can be disentangled into a lower order non-robust component, which is related to causing adversarial behavior, and a higher order robust component. By applying such
disentanglement, we are able to reveal the relation between standard generalization and adversarial robustness of the deep learning model in a relatively general setting.
Furthermore, we demonstrate by experiments that current deep learning models rely heavily on optimizing the lower order non-robust component to generalize, which is a major underlying reason for the adversarial behavior. We also show the state-of-the-art adversarial training algorithms are all in fact trying to constrain the model from using the lower order non-robust component to discriminate data of different categories. Based on these findings, we claim that enabling the model to rely more on the higher order robust component instead of the adversary-prone lower order component might be the key to achieving decent standard accuracy and adversarial robustness simultaneously.
Our contributions are summarized as follows:
• We propose a new metric, Cross Category Kullback-Leibler divergence (CCKL), to characterize the standard performance. This metric can be more naturally connected with the adversarial behavior of DNNs than the traditional metric of accuracy.
• By applying a simple Taylor expansion on CCKL, we theoretically reveal the relation between standard generalization and adversarial behavior of DNNs in a general way without relying on toy models.
• Based on the above novelties, we take a further step and point out the possible direction for simultaneously achieving decent standard accuracy and adversarial robustness.

heading: 2 RELATED WORK
text: Adversarial Attack and Defense To study the adversarial behavior of DNNs, gradient-based algorithms (Goodfellow et al., 2014; Papernot et al., 2016a; Moosavi-Dezfooli et al., 2016; Carlini & Wagner, 2017; Kurakin et al., 2016; Zhao et al., 2018) are developed to find the most effective adversarial perturbations, and on the other hand, model-based defense algorithms (Papernot et al., 2016b; Madry et al., 2017; Miyato et al., 2015; Zhang et al., 2019) are also explored for enhancing model robustness against the adversarial perturbations. For example, in (Miyato et al., 2015; Zhang et al., 2019), the model is regularized by the KL divergence between outputs of clean and adversarial samples, which is also adopted in our theoretic framework.
Fisher Information in Deep Learning Fisher information is used as a tool to study machine learning topics. Many previous works (Chaudhry et al., 2018; Pascanu & Bengio, 2013; Desjardins et al., 2015; Andrychowicz et al., 2016; Achille et al., 2019) view the parameters of the model as those of the Fisher information, while some other (Miyato et al., 2015; Zhao et al., 2018) take the model’s input as the parameters of Fisher information, making it applicable in the context of adversarial robustness. We adopt the same definition for Fisher information as (Miyato et al., 2015; Zhao et al., 2018) in our later analysis.
Explaining Adversarial Behavior of DNNs Previous works trying to explain the robust model’s degradation in standard performance (Tsipras et al., 2018; Schmidt et al., 2018; Nakkiran, 2019) are mostly based on some simple toy models that cannot generalize to real world settings. Recently, (Ilyas et al., 2019) empirically shows that adversarial noise is actually features that can help the generalization of DNNs. However, this work does not provide a general theoretic framework to explain this phenomenon. Zhang et al. (2019) provides a bound that characterizes the gap between standard accuracy and adversarial robustness, but they do not clearly explain the underlying reason of such trade-off. In our work, we analyze the underlying reason for the adversarial behavior of DNNs by directly disentangling the proposed CCKL. In this way, we not only theoretically demonstrate the reasoning behind the trade-off between adversarial robustness and standard generalization, but also point out the possible direction for achieving both objectives simultaneously.

heading: 3 NEW PERFORMANCE METRIC AND ITS DISENTANGLEMENT
text: In this section, we first propose to consider and measure the performance of deep classification models from the perspective of CCKL instead of accuracy. Then we explain our finding that the performance
of a DNN model is determined by two components that are disentangled from the obtained objective based on KL divergence, one of which related to the model’s adversarial behavior and the other not. After that, to help understand our idea, we provide some complementary understandings from the viewpoint of information geometry.

heading: 3.1 PROPOSED PERFORMANCE METRIC
text: Normally, DNNs are trained by minimizing the following cross-entropy loss:
L = E(x,y)∼X×Y [KL(y‖f(x))], (1)
where X × Y is the distribution where the data-label pair (x, y) is sampled from and f(x) is the prediction of the model. Note that y and f(x) are both discrete distribution over categories here. Correspondingly, the prediction accuracy is adopted to measure the performance of a DNN model. However, such a metric hides the connection between the model performance and its adversarial behavior. Since adversarial attack is concerned with changing the prediction of the model on a data point to another, we propose to shift the performance metric from investigating one single data point to comparing the data point with its counterparts from other categories, which can help reveal the above connection. Therefore, we propose the following Cross Category KL Divergence (CCKL) metric: CCKL = E((xi,yi),(xj ,yj))∈S [KL(f(xi)||f(xj))], (2) where S is defined as
S = {((xi, yi), (xj , yj)) | ∀i, j ∈ N , yi 6= yj , xi, xj ∈ X}. (3) With the definition above, we can derive the following theorem:
Theorem 3.1 Denoting JS(p‖q) as the Jensen-Shannon divergence between two distributions p and q, we have CCKL ≥ E((xi,yi),(xj ,yj))∈S [JS(yi‖yj)]− L− coL, (4) where L is the cross-entropy loss defined in Equation (1) and coL = E(x,y)∼(X ,Y)[KL(f(x)‖y)] is the co-object for L.
We defer the proof and more comments of Theorem 3.1 to Appendix A due to space limit.
Taking a closer look at Theorem 3.1, we know that E((xi,yi),(xj ,yj))∈S [JS(yi‖yj)] remains unchanged during training, and L and coL are supposed to decrease when the model performance increases. Therefore, the lower bound on CCKL is supposed to increase with the rise of model performance, which will asymptotically result in the increase of CCKL. Thus, the CCKL, which characterizes how well the model discriminates input data from different categories, can serve as a performance metric.
To further expound on our point, we provide a visualization of how CCKL evolves with test accuracy and test cross entropy loss during training in Figure 2. According to our results, the variation of CCKL correlates well with the test accuracy and test cross entropy loss during training.

heading: 3.2 CONNECTIONS BETWEEN ADVERSARIAL BEHAVIOR AND CCKL
text: We now explain how the above CCKL objective (2) correlates with the model’s adversarial behavior. Consider a single data-label pair (x, y). The following cross entropy loss of the model fθ on adversarial samples (with perturbation η):
Lθ(x, η) = KL(y‖fθ(x+ η)) s.t. ‖η‖ ≤ (5) is adopted to study the adversarial behavior in most previous literature (Goodfellow et al., 2014; Kurakin et al., 2016; Madry et al., 2017). However, it is not easy to build connection between this objective and the standard generalization of the model. Interestingly, in some recent works (Miyato et al., 2015; Zhao et al., 2018; Zhang et al., 2019), another objective is adopted to characterize the model’s adversarial behavior:
Lθ(x, η) = KL(fθ(x)‖fθ(x+ η)) s.t. ‖η‖ ≤ . (6) Based on the above objective (6), (Zhao et al., 2018) reports state-of-the-art results in the task of adversarial attack and (Zhang et al., 2019) reports state-of-the-art results in adversarial robustness.
More importantly, as will be shown later in the paper, the adversarial objective (6) can be easily linked to the standard generalization of the model. Therefore, we adopt the objective (6) in the rest of the paper. The corresponding adversarial training objective is thus formulated as follows:
min θ max η
Lθ(x, η) s.t. ‖η‖ ≤ . (7)
Given the definition of Lθ(x, η), applying Taylor expansion yields the following:
max η Lθ(x, η) = max η
1 2 η>Fxη + ∞∑ k=3 T (k)x (η) s.t. ‖η‖ ≤ , (8)
where Fx is the Fisher information of f(x) w.r.t. x and T (k) x is the following k-th order Taylor expansion term. Let f (j)(x) be the j-th entry of f(x) and n be the dimension of f(x). Then Fx can be calculated by
Fx = n∑ j=0 f (j)(x)(∇x log f (j)(x))(∇x log f (j)(x))>. (9)
When is sufficiently small, the higher order terms in the above would vanish and Equation (8) could be simplified into
max η
Lθ(x, η) = 1
2 max η
η>Fxη s.t. ‖η‖ ≤ . (10)
We could therefore construct the lagrangian of the above problem:
L(x, η) = 1
2 η>Fxη −
1 2 λ(η>η − 2). (11)
By setting∇ηL(x, η) = 0, we obtain Fxη = λmaxη, where λmax is the maximum eigenvalue of Fx. Therefore, the most effective adversarial perturbation corresponds to the leading eigenvector of Fx.
Consequently, we have
max η
Lθ(x, η) = 1
2 λmax
2. (12)
Note that λmax here is also the spectral norm of the Fisher information matrix Fx. The above theory shows that the local adversarial behavior of the model f around the input x is determined by the spectral norm of the Fisher information matrix: the adversarial behavior around x would be more severe if the spectral norm of Fx is larger.
Now we analyze CCKL by considering one tuple ((xi, yi), (xj , yj)) randomly sampled from S . We rewrite KL(f(xi)‖f(xj)) as
KL(f(xi)‖f(xj)) = KL(f(xi)‖f(xi + (xj − xi))) = Lθ(xi, xj − xi). (13)
Then, we apply the same Taylor expansion as above and obtain
KL(f(xi)‖f(xj)) = Lθ(xi, xj − xi) = 1
2 (xj − xi)>Fxi(xj − xi) + ∞∑ k=3 T (k)xi (xj − xi). (14)
Comparing Equation (14)1 and Equation (10), we notice they share the same Fisher information Fxi at data point xi. Here, we show the adversarial behavior at each data point can be connected to the performance objective CCKL via Fisher information, which lays the foundation for the disentanglement in the next subsection.

heading: 3.3 DISENTANGLEMENT OF CCKL
text: As mentioned above, we build the connection between adversarial behavior and CCKL. We here explain how the non-robust and robust component in CCKL can be disentangled. We first denote 1 2 (xj − xi) >Fxi(xj − xi) in Equation (14) as G1 and the following terms ∑∞ k=3 T (k) xi (xj − xi) as G2. Thus KL(f(xi)‖f(xj)) can be formulated as
KL(f(xi)‖f(xj)) = G1 +G2. (15)
Taking a closer look at Equation (15), we notice the increase of G1 and G2 can both contribute to the rise of KL(f(xi)‖f(xj)), which contribute to CCKL. In addition, since λmax is the maximum eigen value of Fxi , we have the following bound:
1 2 λmax‖η‖2 ≥ 1 2
[ ‖η‖
‖xi − xj‖ (xi − xj)
]> Fxi [ ‖η‖
‖xi − xj‖ (xi − xj)
] = G1 ‖η‖2
‖xi − xj‖2 . (16)
With the above bound, we know that the rise of G1 would asymptotically result in the rise of the norm of Fxi , which corresponds to the adversarial behavior.
Therefore, if the model relies heavily on the increase of G1 to boost performance, the norm of Fxi would also increase drastically. According to Equation (12) and our analysis above, the rise in the norm of Fxi means more severe adversarial behavior around xi. The trade-off between standard performance and adversarial behavior is thus characterized as follows: the model can rely on G1 to boost performance, but it comes with the side effect of more severe adversarial behavior; on the other hand, G2 contributes to the CCKL but it is not involved in the adversarial objective, thus it would not cause adversarial behavior to rely on terms in G2 to distinguish xi from data belonging to other categories. In this way, we can disentangle the non-robust and robust component in the overall performance objective CCKL.
To further understand the role of G1 in classification, we visualize how F-norm of Fisher information evolves during training. We visualize F-norm instead of spectral norm because all norms are equivalent and spectral norm is not computationally feasible in our case. We first empirically show how the average F-norm of Fisher information and standard accuracy of VGG13 model on CIFAR-10 test set vary during normal training in Figure 4. We observe that the norm of Fisher information increases drastically with the rise of accuracy, which indicates DNNs rely heavily on the non-robust component G1 to boost performance.
1Note that (xj − xi) is the difference between two input data instead of a small perturbation, so the higher order terms would not vanish here.
Then we compare normal training with the two state-of-the-art adversarial training algorithms (Madry et al., 2017; Zhang et al., 2019) using the same visualization method, and show results in Figure 3. It is clear that during adversarial training, although the Fisher information’s average F-norm also rises with standard accuracy, its value is significantly smaller than its counterpart during normal training. That is, the adversarial training process can effectively constrain the model from relying on Fisher information to boost performance.
Our experiments reveal the widely known but seldom understood fact that the standard accuracy of models under the two adversarial training algorithms are significantly lower than that under normal training. According to our theory, it can be explained as these models trained over adversarial samples are unable to effectively rely on adversary-prone non-robust components such as Fisher information to distinguish the input data from those belonging to other categories.
Till now, we theoretically and empirically demonstrate the relation between standard performance and adversarial robustness using the proposed disentanglement theory.

heading: 3.4 EXPLANATION FROM GEOMETRY POINT OF VIEW
text: We provide some explanations on the above findings from the information geometry viewpoint. We note the model is doing maximum likelihood estimation by learning to fit the label distribution over training data. This can be viewed as a process of the log-likelihood landscape of the model on input data gradually transiting into a state where the model can well distinguish data of different categories. Since the training data are sparsely sampled from the whole distribution, the smoothness prior does not hold during the formation of the model’s log-likelihood landscape. Consequently, the model tends to use lower order local geometric descriptors such as Fisher information —the local curvature of the log-likelihood landscape —to form an overly simplified adversary-prone log-likelihood landscape. When applying adversarial training, a strong smoothness constraint is enforced and the model would have to rely on higher-order global geometric descriptors that vanish locally to form the whole landscape. Thus the landscape would be more robust to adversarial perturbations.

heading: 3.5 ANALYSIS FOR NOT SUFFICIENTLY SMALL
text: Researchers have found that the local linearity does not hold when the norm of perturbation is allowed to be relatively larger (Kurakin et al., 2016; Madry et al., 2017). In this case, some higher order terms may also involve in the adversarial objective. However, since the norm of perturbation is still considered to be small, the Taylor expansion terms of KL(f(x)‖f(x+ η)) would still vanish up to a certain order K . Therefore, the adversarial objective can be rewritten as
max η Lθ(x, η) = K ∑ k=2 T (k)x (η) s.t. ‖η‖ ≤ . (17)
Comparing (14) and (17), we know that the functional of the first K − 1 terms could still jointly connect adversarial objective and CCKL. Also, we conduct numeric experiments in Appendix C to show that the Fisher induced term is the dominant component in KL(f(x)‖f(x+ η)) within a practical range of perturbation. We leave more detailed theoretic analysis on higher order terms involved in the adversarial objective to future work.

heading: 4 TOWARDS SIMULTANEOUS GOOD PERFORMANCE AND ROBUSTNESS
text: With the disentanglement theory introduced above, we now consider how to achieve simultaneous decent standard accuracy and adversarial robustness. According to our analysis, if relying on the robust component alone can effectively distinguish data of different categories, then obtaining an adversarial robust model with high standard accuracy is possible.
Since the expansion terms in the robust component are all higher order terms, we claim that the key to achieving the two desired qualities simultaneously is to increase the expressive power of the model so that it would be able to rely more on higher order terms for prediction. In this way, the model would not have to rely heavily on Fisher information and still have decent standard performance with higher order terms (the robust component). Our disentanglement provides theoretical justification for the importance of model capacity in achieving adversarial robustness and decent standard performance.
We experiment on CIFAR-10 to validate our strategy of achieving the two objectives simultaneously. We train the following models with TRADES algorithm (Zhang et al., 2019): a pure linear VGG13 model without ReLU and with average pooling, a half linear VGG13 model without ReLU but using max pooling, a normal VGG 13 model, a normal resnet20 model and a resnet32 model with 10× more channels. We evaluate these models on standard samples and adversarial samples produced
by L∞ PGD attack (Madry et al., 2017) and CW attack (Carlini & Wagner, 2017). The results are provided in Table 1.
We first compare VGG models. Since adversarial training impedes the model from using the adversary-prone lower order terms to discriminate input data, and the pure linear model is not capable of utilizing higher order information, the standard generalization and adversarial robustness of the pure linear model are both very poor. However, as the capacity of the model increases (half linear and normal VGG13 models), the performance on standard and adversarial samples improves simultaneously. For resnet models, when the model is shallower and narrower (resnet20x1), the performance in both standard and adversarial settings is relatively low. However, with a deeper and wider resnet model (resnet32x10), more higher order information is explored for prediction, so the performance on standard samples and adversarial samples increases significantly.
In addition, we sampled 25896 pairs of images (xi, xj) in CIFAR-10 test set (xi and xj belong to different categories) and visualized the distribution of KL(f(xi)‖f(xj)), G2, and G2/KL(f(xi)‖f(xj)) with the above mentioned adversarially trained resnet20x1 and resnet32x10 model in Figure 5.
According to our results in Figure 5, we have the following observations. 1) The value of CCKL of the resnet32x10 model on CIFAR-10 test set is much higher than that of resnet20x1, which means the resnet32x10 can better distinguish input data from different categories. 2) The value of G2 of resnet32x10 on the CIFAR-10 test set is significantly higher than that of resnet20x1, which shows G2 plays a much more important role in resnet32x10 than resnet20x1. 3) The ratio of G2/KL(f(xi)‖f(xj)) of resnet32x10 on CIFAR-10 test set is much higher than that of resnet20x1, meaning the relative importance of G2 of resnet32x10 is much greater than that of resnet20x1.
These observations validate our claim that a model with better capacity can more effectively leverage higher order information to achieve higher standard accuracy and also adversarial robustness under adversarial training.

heading: 5 CONCLUSION
text: In this work, we provide a novel viewpoint on the relation of standard accuracy and adversarial robustness of deep learning models. We propose a new CCKL metric to measure the model performance instead of accuracy. With CCKL, the overall performance objective can be disentangled into an adversary-prone non-robust component, and a robust component. Based on such disentanglement, we then claim that for achieving both adversarial robustness and decent standard accuracy, a DNN model should rely more on the robust component to generalize. Our findings are well validated. In the future, we will dig deeper along the geometric properties of the log-likelihood landscape formed by DNNs on input data, and try to better characterize the relation between standard accuracy and adversarial robustness.

heading: A PROOF AND COMMENTS OF THEOREM 3.1
text: In practical scenario, one cannot be too confident about a label y due to human annotation errors, and often replaces it during training with smoothed version yLS using the label smoothing technique Szegedy et al. (2016). Namely,
yLS = αy + (1− α) 1 K ,
where K is the number of classes, and α ∈ [0, 1] is the confidence level. The label smoothing version of Theorem 3.1 still holds.
Proof of Theorem 3.1:
∀ data-label pair (xi,yi), (xj ,yj), where yi 6= yj , since JS1/2 satisfies triangular inequality (Endres & Schindelin, 2003), we have:
JS(f(xi)‖f(xj))1/2 + JS(yi‖f(xi))1/2 + JS(yj‖f(xj))1/2
≥ JS(yi‖f(xj))1/2 + JS(yj‖f(xj))1/2 ≥ JS(yi‖yj)1/2 (18)
Also, we have the inequality:
(a+ b+ c)2 ≤ 2(a2 + b2 + c2). (19) With (18) and (19), we have:
2(JS(f(xi)‖f(xj)) + JS(yi‖f(xi)) + JS(yj‖f(xj))) ≥ (JS(f(xi)‖f(xj))1/2 + JS(yi‖f(xi))1/2 + JS(yj‖f(xj))1/2)2 (applying (19)) ≥ JS(yi‖yj). (applying (18))
(20)
In addition, we have Lin’s inequality (Lin, 1991):
JS(p‖q) ≤ 1 4 (KL(p‖q) +KL(q‖p)). (21)
Apply (21) to (20), we have:
JS(yi‖yj) ≤ 2(JS(f(xi)‖f(xj)) + JS(yi‖f(xi)) + JS(yj‖f(xj)))
≤ 1 2 (KL(f(xi)‖f(xj)) +KL(f(xj)‖f(xi)) +KL(yi‖f(xi)) +KL(yj‖f(xj)) +KL(f(xi)‖yi) +KL(f(xj)‖yj))
(22)
Taking the expectation over the data set in (22), we have:
CCKL ≥ E∀yi 6=yj [JS(yi‖yj)]− E[KL(y‖f(x))]− E[KL(f(x)‖y)], (23) which proves the theorem 3.1.

heading: B ADDITIONAL RESULTS ON CCKL AS PERFORMANCE METRIC
text: We further empirically validate the relation between CCKL and standard cross entropy loss on test set in Figure. 6. We trained a ResNet-20 on CIFAR-10 and CIFAR-100. The Pearson Correlation Coefficients of CCKL and cross entropy loss on test set across the whole training period are -0.6744 and -0.8018 in these two settings, which further justify our use of CCKL as performance metric.

heading: C APPROXIMATE KL DIVERGENCE LOCALLY WITH FISHER
text: Experiment in Tab. 2 shows the relative numerical error of approximating KL locally with Fisher induced term with a VGG13 model adversarially trained by (Zhang et al., 2019) on CIFAR-10. We can observe that when is within practical range (≤ 8/255), the relative numeric error is small.

heading: D MORE EXPERIMENTS ON THE ROLE OF FISHER INFORMATION
text: We conduct more visualization experiments about the role of Fisher information in standard performance of DNN. The results are shown in Figure 7 and Figure 8. The experiments are conducted on a resnet20 model. The same conclusion could be drawn according to our statistics.
Also, to better understand the geometry of an adversarially trained model around robust samples and non-robust samples, we visualized the F-norm of Fisher information at correct and robust samples and correct but not robust samples of a VGG13 model. According to our results in Figure 9, we could see that although the overall Fisher information is small, the Fisher information around robust samples are still significantly smaller than that of non-robust samples.
E INTERPRETATION FROM CRAMÉR-RAO BOUND POINT OF VIEW
According to the main text, adversarial training constrains the input-output Fisher information of a DNN model. This constrain is a criteria of a good DNN model due to the following reasons. Recall the well-known Cramér-Rao bound var(x̂)Fx ≥ 1 says that if we try to use the output probability f(x) to a statistics x̂ to reconstruct the input x, the uncertainty in terms of variance var(x̂) is bounded below by the inverse of Fisher information Fx. For a DNN model that represents the reality, when it classifies an image with a correct label, say a dog, the label does not have any information about the environments - what color the dog is, where is the dog, adversarial perturbation, etc. Therefore, one cannot use the information contained in the label to reconstruct the original image. This means that the variance var(x̂) of any statistics x̂ derived from output distribution f(x) is relatively large for a good DNN model. In view of Cramer-Rao bound, this implies that the Fisher information of a DNN is a relatively small value.

Review summaries:
The authors propose a new performance metric, CCKL, as an alternative to standard cross-entropy. At a high level the goal of this objective is to ensure that the classifier is confidently separating samples from different classes. They decompose this objective into its Taylor expansion terms and argue that, since adversarial vulnerability is a local property, it only depends on the first of these terms. They finally propose training models of higher capacity as a way of relying more on higher-order terms of the objective and hence improving adversarial robustness.