Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
Authors: Yilun Jin
Lixin Fan
Kam Woh Ng
Ce Ju
Qiang Yang
Venue:  (2021)
Keywords: Adversarial Robustness
Uncertainty Promotion
Adversarial Training

Abstract: Deep neural networks (DNNs) are known to be prone to adversarial attacks, for which many remedies are proposed. While adversarial training (AT) is regarded as the most robust defense, it suffers from poor performance both on clean examples and under other types of attacks, e.g. attacks with larger perturbations. Meanwhile, regularizers that encourage uncertain outputs, such as entropy maximization (EntM) and label smoothing (LS) can maintain accuracy on clean examples and improve performance under weak attacks, yet their ability to defend against strong attacks is still in doubt. In this paper, we revisit uncertainty promotion regularizers, including EntM and LS, in the field of adversarial learning. We show that EntM and LS alone provide robustness only under small perturbations. Contrarily, we show that uncertainty promotion regularizers complement AT in a principled manner, consistently improving performance on both clean examples and under various attacks, especially attacks with large perturbations. We further analyze how uncertainty promotion regularizers enhance the performance of AT from the perspective of Jacobian matrices $\nabla_X f(X;\theta)$, and find out that EntM effectively shrinks the norm of Jacobian matrices and hence promotes robustness. 

Under review as a conference paper at ICLR 2021
RETHINKING UNCERTAINTY IN DEEP LEARNING: WHETHER AND How IT IMPROVES ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks (DNNs) are known to be prone to adversarial attacks, for which many remedies are proposed. While adversarial training (AT) regarded as the most robust defense, it suffers from poor performance both on clean ex- amples and under other types of attacks, e.g. attacks with larger perturbations. Meanwhile, regularizers that encourage uncertain outputs, such as entropy max- imization (EntM) and label smoothing (LS) can maintain accuracy on clean ex- amples and improve performance under weak attacks, yet their ability to defend against strong attacks still in doubt. In this paper, we revisit uncertainty promo- tion regularizers, including EntM and LS, in the field of adversarial learning. We show that EntM and LS alone provide robustness only under small perturbations. Contrarily, show that uncertainty promotion regularizers complement AT in principled manner, consistently improving performance on both clean examples and under various attacks, especially attacks with large perturbations. We further analyze how uncertainty promotion regularizers enhance the performance of AT from the perspective of Jacobian matrices (X; 0), and find out that EntM effectively shrinks the norm of Jacobian matrices and hence promotes robustness.
INTRODUCTION
Deep neural networks (DNNs) have achieved great success in image recognition (Russakovsky et al. 2,15), audio recognition (Graves & Jaitly| (2014), etc. However, shown by (Szegedy al. (013).
DNNs are vulnerable to adversarial attacks, where slightly perturbed adversarial examples can easily fool DNN classifiers. The ubiquitous existence of adversarial examples (Kurakin et al. 2016)
casts doubts real-world DNN applications. Therefore, many techniques are proposed to enhance
the robustness of DNNs to adversarial attacks (Papernot et al. 2016 Kannan et al. 2018).
Regarding defenses adversarial attacks, adversarial training (AT) (Goodfellow et al. 2014 Madry et al. 2017 commonly recognized as the most effective defense. However, illustrates poor per- formance on clean examples and under attacks stronger than what they trained on (Song al., 2018). Recent works (Rice et al.) 2020) attribute such shortcomings to overfitting, but solutions, even mitigation them remain open problems. Meanwhile, many regularization techniques proposed to improve robustness, among which two closely related regularizers, entropy maximiza- tion (EntM) and label smoothing (LS) show performance improvements under weak attacks (Pang et al. 2019 Shafahi et al. (201)) without compromising accuracy on clean examples. However, as stated by (Uesato et al. 2018 Athalyy et al. 2018), strong and various kinds of attacks should be used to better approximate the adversarial risk of a defense. Therefore, it remains doubtful how they perform under stronger attacks by themselves and how their adversarial risk is.
One appealing property of both EntM and LS is that they both penalize over-confident predictions and promote prediction uncertainty (Pereyra et al. [017]] [Szegedy et al. 2016). Therefore, both of them are used as regularizers that mitigate overfitting in multiple applications (Müller et al.) 2019). However, remains unclear whether EntM and LS can effectively regularize AT, where many other regularizers are ineffective (Rice et al.) 2020). Therefore, in this paper, we perform extensive empirical study on uncertainty promotion regularizers, i.e. entropy maximization and label smooth- ing, in the domain of adversarial machine learning. We carry out experiments both regularizers, with and without AT, on multiple datasets and under various adversarial settings. We found out that, although neither EntM and LS are able to provide consistent robustness by themselves, both regular-

---

Under review as a conference paper at ICLR 2021
izers complement AT and improve its performances consistently. Specifically, we observe not only better accuracy on clean examples, but also better robustness, especially under attacks with larger 26/255 for models trained with 8/ 8/255
perturbations (e.g. 9% improvement under perturbation AT. See Table and 3). In addition, we investigate the underlying mechanisms about how EntM and LS complement AT, and attribute the improvements to the shrunken norm of Jacobian matrix | 0)ll (~10 times. See Sect. 6). which indicates better numerical stability and hence better adversarial robustness than AT.
To summarize, we make the following contributions:
1. We carry out an extensive empirical study about whether uncertainty promotion regulariz- ers, i.e. entropy maximization (EntM) and label smoothing (LS). provide better robustness, and find out that while neither of them provides consistent robustness under strong attacks alone, both techniques serve as effective regularizers that improve the performance of AT consistently on multiple datasets, especially under large perturbations.
2. We provide further analysis on uncertainty promotion regularizers from the perspective of Jacobian matrices, and find out that by applying such regularizers, the norm of Jacobian matrix
is significantly shrunken, leading better numerical stability and adversarial robustness.
RELATED WORK AND DISCUSSIONS
Adversarial Attacks and Defenses. (Szegedy et al. 2013 Goodfellow et al. 2014) pointed out the vulnerability of DNNs adversarial examples, and proposed efficient attack, the FGSM, to generate such examples. Since then, as increasingly strong adversaries [Crlinn & Wagner (2017) proposed, AT (Madry al. 2017 is considered the most effective defense remaining. More recently, Zhang et al. (2019) proposes TRADES, extending AT and showing better robustness than Madry et al. (2017). We refer readers (Yuan al. 2019 Biggio & Roli 2018) for more compre- hensive surveys on adversarial attacks and defenses.
However, all AT methods illustrate common drawbacks. Specifically, Rice et al. (2000) shows that AT suffers from severe overfitting that cannot be easily mitigated, which leads to multiple undesir- able consequences. For example, AT performs poorly on clean examples, and [Song et al. (2018)
shows that AT overfits on the adversary it is trained on and performs poorly under other attacks.
Uncertainty Promotion Techniques in Machine Learning. The principle of maximum entropy is widely accepted in not only physics, but also machine learning, such as reinforcement learning (iebart et al., 2008; Haarnoja et al., 2018). In supervised learning, uncertainty promotion tech- niques, including entropy maximization (Pereyra et al. 2017) and label smoothing (Szegedy et al.) 2016) act on model outputs and can improve generalization lizztio in many applications, such as image classification and language modeling (Pereyra et al. 2017 Müller et al. 2019 Szegedy et al. 2016))
There are only several works that study uncertainty promotion techniques in the field of adversarial machine learning. ADP (Pang al. 2019) claims that diversity promotion in ensembles can promote robustness, for which entropy maximization is used. Essentially, ADP consists of multiple mutually independent models that in total mimic the behavior of entropy maximization. However, ADP only shows improvements under weak attacks, e.g. PGD with 10 iterations. Shafahi et al. (2019) claims that label smoothing can improve robustness beyond AT by itself. yet the claim remains to be further investigated under a wider range of adversaries, e.g. adaptive attacks and black-box attacks. As stated by (Uesato et al. 2018 | Athalye al., 2018) that, one should use strong and various attackers for better approximation of the adversarial risk (Uesato et al. 2018), it remains unknown how much robustness EntM or I.S alone can provide under strong and diverse attacks. Moreover, neither of these works study the relationship between AT and uncertainty promotion, which is important open problem since many regularizers fail to mitigate the overfitting of AT (Rice et al. 20200).
We also discuss knowledge distillation (KD) (Hinton et al. 2015 as another line of work that in- troduces uncertainty. KD is a procedure that trains a student model to fit the outputs of a teacher network, which are also 'soft' labels and introduce uncertainty. However, KD alone has been shown not to improve robustness (Carlini & Wagner 2016). Also, while (Goldblum et al.) 2020) proposed adversarially robust distillation (ARD), combining AT and KD, ARD primarily aims to build robust
2

---

Under review as a conference paper at ICLR 2021
models upon robust pre-trained teacher models, while primarily study uncertainty and do not rely on pre-trained teacher models. Therefore we consider ARD to be orthogonal to our work.
PRELIMINARIES
Notations. We denote a dataset D
(x((9, (00 " 3==11 where X IR are input data, y RC are one-hot vectors for labels. We denote DNN parameterized by f(X;0) R E, which outputs logits on C' classes. We denote fo (X; 0) = a o f(X;)) 0). the network followed by softmax activation. We denote the cross entropy loss CE(
Σi= Y/ log üt the Shannon entropy as (p) = -E pi log p; and the Kullback-Leibler divergence as D (pllg) = Ei= Pi log &
Adversarial Attacks and Adversarial Training. Given a data-label pair (X, y) and neural net- work f(X;0) 0), adversarial attacks aim to craft an adversarial example X X(adv), || - X|| s. so to fool ((; Such goal can be formulated as the problem:
((adv) arg max (fo 0), v), subject tol| XII S. X
where Lack is some loss function, generally taken as CE. Eqn. can generally be solved via iterative optimization, such as Projected Gradient Descent (PGD) (Madry et al. 2017).
Adversarial Training (AT) generally considered the most effective defense against adversarial attacks, where the training set consists of both clean examples and adversarial examples, and the following objective is minimized (Goodfellow et al. 2014).
min y) aCE (X;O)) + +1
a) ||
max Xlls=
CE (X((x))))) 10).)00))))
(2)
where the inner maximization is commonly solved via PGD. We specifically denote which uses PGD to solve the inner problem PAT (Madry et al.] 2017). TRADES (Zhang et al.) 2019) extends PAT by optimizing Eqn. 3 in similar min-max styles, leading to better robustness than PAT.
min B B(x,)>D
CE(J=(X;0),b) + max [[(tad)) -11sse
DKL. ((((x()t))))10(((xx0))
(]]][((x:0)
(3)
METHODOLOGY
4.1 ENTROPY MAXIMIZATION
Given example (X, 3) and neural network f(X; 0), the proposed entropy maximization (EntM) minimizes the cross entropy loss, while maximizing the Shannon entropy of the output probability.
Trnnm X.y) v) = CF (X; λH (X;
(4)
where À is hyperparameter controlling the strength of entropy maximization. For normal training, we minimize over (x. Lenttm (X, y)]; ; for AT, such as PAT and TRADES, we replace CE with Lentm in Eqn. 2 which for PAT yields Eqn. 5 and similarly for TRADES:
min E E(X,)) a abentm (X; .) + +1 a.)
max
Tentm (Ia(x(ad))) :0).)) (5)
[|X (add)) -Xlss
The entropy maximization term penalizes for over-confident outputs and encourages uncertain pre- dictions, which is different from CE encouraging one-hot outputs. Such a formulation is also seen in (Pereyra al. 2017 Dubey et al. 2018) in general deep learning and fine-grained classification.
4.2 CONNECTION WITH LABEL SMOOTHING
Label smoothing is also widely used technique that prevents assigning over-confident predictions. Given a data-label pair (X, y). and denote the uniform distribution over C classes as uc. the objec- tive of label smoothing can be formulated as
ysmooth y 7 uc); Tesmooth (X (X, v) CE ((Fx(X;O,, Usmooth)
(6)
3

---

Under review as a conference paper at ICLR 2021
Label smoothing and entropy maximization are intrinsically similar. Specifically, both label smooth- ing and entropy maximization penalize over-confident outputs by:
Lsmooth (X, y) = (1 7) (E( + -Dx
+ log C.
(7)
Lentm (X, 1) (X 0)]lc) A C.
Therefore EntM and LS are connected, with differences in the asymmetric DK (fo (X; (lc) and DEL (x: In our paper, carry out experiments on LS and EntM for comparison.
QUANTITATIVE EXPERIMENTS
5.1 EXPERIMENTAL SETTINGS AND RESULTS
Datasets and Models We utilize four commonly used datasets for evaluation, CIFAR-10, CIFAR- 100, SVHN and MNIST. We train ResNet18, common selection in adversarial training (Madry
al. 2017) for CIFAR-10, SVHN and CIFAR-100, and four-layer CNN for MNIST. Details about training settings and architectures can be found in Appendix
Comparison Models We study the robustness of the following models without adversarial training.
Normal models trained using standard cross entropy. We denote them as Normal.
ADP (Pang et al. 2019), where they built ensembles of models, regularized by maximizing their ensemble diversity. Entropy maximization is involved in the ensemble diversity term.
We use 3 models as the ensemble, and take the recommended parameters (2, 0.5).
Models trained with EntM Eqn. We set À = 2 for all datasets following ADP who also set the entropy term 2.
Models trained with label smoothing (LS). We choose y such that the non-maximal predic- tions are the same as EntM on each dataset, i.e. 7 0.74 for CIFAR-10 and SVHN, and
7 = 0.84 for CIFAR-100.
We also study the robustness of the following models with adversarial training. All adversaries are taken as T. adversary with & 8/255, step size 2/255 with 7 steps for CIFAR-10, CIFAR-100 and SVHN, and 48/255, step size 6/255 with 10 steps for MNIST.
AT (Eqn. with PGD adversary, (PAT). We take a 0.5 as in (Goodfellow et al. 2014).
PAT with EntM objective (Eqn.
5). PAT-EntM. We set identical as in EntM.
PAT with label smoothing (Eqn.
6 PAT-LS. We set identical 7 values as in LS.
TRADES Zhang et al. 2019). which shows better robustness than PAT. We take 3 = 3.
Threat models. Without explicit mentioning, we focus on untargeted Lxs adversaries. We consider PGD adversaries with CE objective (PGD and FGSM), and also CW adversaries (Carlini & Wagner) 2017 i.e. attacks using CW objective and optimized by PGD3 We consider both white and black- box settings. In black-box settings, adversarial examples are crafted a proxy model trained on identical data and then transferred to the target model (Papernot et al., 2017).
We show test accuracy under various attacks on CIFAR-10, CIFAR-100, SVHN and MNIST in Table 2 3 cf We use FGSMK to denote FGSM with k/255, PGDp - k as p-step PGD attacks with k 6/255, step size k/2550, and CWp-k as similar to PGDp 1 except for the attack objective. We adopt random initialization with Gaussian X' = X + N(0 0.005) with 5 restarts for all attacks. For black-box attacks, adversarial examples are crafted on a Normal model for defenses without AT, and PAT model for those with AT. Best accuracies higher than random are bolded.
and
We also did experiments on TRADES-EmtN i.c. TRADES trained with entropy maximization objective. The results are shown in Table T2 and 13 in Appendix E
"The C& adversary is implemented and carried out exactly as (Zhang et al, 2020 Note that ADP builds ensemble models upon probability outputs, CW attacks operating on logits cannot be carried out.
RRssult MNIST, and results PGD20 are shown in Appendix
4

---

Under review as a conference paper at ICLR 2021
CIFAR-I0
Without AT
With AT
Attacks
Normal
Mode
EntM
LS
ADP
PAT
TRADES
PAT-EntM
PAT-LS
Clean
94.78
94.71
95.15
95.72
86.47
84.01
87.87
88.31
FGSM4
37.05
72.37
74
69.5
69.21
70.33
72.96
72.95
PGDI0-4
58.45
66.07
42.8
68.33
68.72
71.26
71.48
PGDI0-8
37.49
46.12
21.38
48.87
51.35
53.89
53.22
White-Box
PGD10-16
17.06
24.79
6,25
19.63
25.77
33.65
31.17
PGD40-4
21.12
24.6
12.15
67.51
67.4
69.63
69.44
PGD40-8
6.26
1.94
43.03
46.74
45.95
44.79
PGD40-16
0.7
0.93
0.13
10.26
15.27
19.56
18.3
CW40-4
22.76
24.8
67.04
66.31
68.3
68.58
CW40-8
9.28
43.63
45.78
44.52
43.75
CW40-16
1.16
2.35
10.66
14.47
17.79
17.09
PGD10-4
27.5
35.84
32.24
31.79
76.88
75.52
79.32
79.24
Black-Box
PGDI0-8
7.57
13.26
10.2
10.12
63.9
64.8
65.82
65.76
Table I: Performance under various attacks CIFAR-10 (%). 0 denotes < 0.1%. Italics indicate
white-box accuracies higher than corresponding black-box ones.
CIFAR-100
Without AT
With
Attacks
Normal
Mode
EntM
LS
ADP
PAT
TRADES
PAT-EntM
PAT-LS
Clean
75.66
77.17
77.60
80.04
59.65
54.63
62.53
62.83
FGSM4
12.73
28.29
29.8
31.14
38.86
37.94
44.37
43.2
PGD10-4
0.97
9.1
9.91
6.15
37.76
37.61
43.43
42.39
PGD10-8
0
3.32
3.35
1.3
22.41
24.77
29.09
27.13
White-Box
PGD10-16
0
0.7
0
7.47
10.28
13.18
11.21
PGD40-4
0
1.08
1.35
0.87
36.55
36.45
42.45
41.08
PGD40-8
0
0.1
0
0
19.62
22.6
26.1
24.05
PGD40-16 16
0
0
0
0
4.73
7.26
8.22
6.69
CW40-4
0.4
0.5
36.09
34.29
38,5
38.02
CW40-8
0
0
19.33
20.74
22.17
21.02
CW40-16
0
0
0
5.1
6.01
6.17
5.4
PGD10-4
13.77
18.07
18.44
21.36
49.4
46.76
52.25
$2.97
Black-Box
PGD10-8
3.65
5.45
5.6
7.07
38.84
38.55
41.48
41.44
Table 2: Performance under various attacks on CIFAR-100 (%).
5.2
How MUCH DO UNCERTAINTY PROMOTION REGULARIZERS IMPROVE ROBUSTNESS?
5.2.1 WITHOUT ADVERSARIAL TRAINING
We first analyze the case where adversarial training is used, namely, EntM, ADP and LS, whose robustness under diverse and strong attacks is under question. We make the following findings.
Under attacks, EntM and LS perform better than ADP in CIFAR-10 and CIFAR-100. and in SVHN, LS performs better than ADP, while EntM performs similarly as ADP. Therefore we focus our analysis on EntM and LS instead of ADP.
EntM and LS show gradient obfuscation under weak attacks. On CIFAR-10 and SVHN, white-box PGDIO attacks succeed less often than black-box ones (See Table White- and Black-Box PGDIO), which is clear indication of gradient obfuscation (Athalye et al. 2018). i.e. gradient- based attacks perform worse than gradient- free attacks because gradients around data point made useless by the model. Gradient obfuscation on EntM and LS leads us to use adversaries with more steps to better approximate their ad- likely to escape from
and 3
versarial risk (Uesato al. 2018), multi-step adversaries

---

Under review as a conference paper at ICLR 2021
SVHN Mode
Without AT
With AT
Attacks
Normal
EntM
LS
ADP
PAT
TRADES
PAT-EntM
PAT-LS
Clean
97.04
97.18
97.31
97.58
94
93.28
94.08
94.08
FGSM4
65.74
85.39
85.75
83.55
82.99
84.26
83.21
83.04
PGDIO-4
27.54
72.99
79.86
70.76
79.72
80.41
80.11
79.39
White-Box
PGDI0-8
4.73
48.47
59.13
49,91
61.71
64.95
63.88
62.6
PGD10-16
21,61
31.97
25.13
29.33
36.27
38.85
35.77
PGD40-4
8.37
40.85
44.53
42.73
77.71
77.96
78.3
77.06
PGD40-8
10.27
12.4
13.62
52.76
57.02
54.65
52.14
PGD40-16
1.15
1.9
2.25
14.18
17.96
23.8
17.58
PGDIO-4
66.07
70.47
69.76
67.83
83.69
83.78
83.85
84.08
Black-Box
PGDIO-8
49.77
48.25
46.53
68.88
69.4
69.03
69.48
Performance under various attacks on SVHN (%). Italics indicate white-box accuracies higher than corresponding black-box ones.
Table 3:
the area where gradient obfuscation happens and find adversarial examples. We provide more evidence of gradient obfuscation Appendix
EntM and LS improve robustness only under small perturbations. We carry out evaluations on PGD40. It can be shown that improvements over random guessing can only be seen under PGD40-4 on CIFAR-10 and SVHN (Table and 3). Therefore, the claims by (Pang al. 2019 Shafahi et al. (219)) that ADP and LS improve robustness are conditional, and EntM, LS and ADP are only robust under small perturbations.
5.2.2 WITH ADVERSARIAL TRAINING
We study AT regularized by uncertainty promotion regularizers and make the following findings.
PAT-EntM and PAT-LS outperform PAT on clean examples by 1% on CIFAR- 10, 3% on CIFAR-100, and under strong attacks like PGD40, outperform PAT by 2-9% on CIFAR-10, and 3-6% on CIFAR-100. See Clean and PGD40 in Table |
PAT-EntM achieves generally similar or better performance than its counterpart PAT-LS, e.g. on CIFAR-10 and CIFAR-100, PAT-EntM achieves 1% better adversarial accuracy, while PAT-LS achieves <1 % better accuracy on clean examples, see Table and 2 There- fore for the rest of the paper primarily focus PAT-EntM.
Robustness improvements are most evident under large perturbations. For example, accu- racies under PGD40-L 16 improved by 9% on CIFAR-10 10 and SVHN from PAT to PAT-EntM, see Table and which is exactly the opposite from where AT is not used.
PAT-EntM is generally more robust than TRADES by over 3% CIFAR-10 and CIFAR- 100, and similarly robust as TRADES on SVHN, further showing the effectiveness of uncertainty promotion regularizers.
Results under CW (Table and 2) show that PAT-EntM is still more robust than PAT (by 7% on CIFAR-10 and 3% on CIFAR-100), and TRADES (except CIFAR1O, PGD40-8). The results under different adversary further verifies the robustness improvement.
In addition, we train WideResNer34-10 Zagoruyko & Komodakis 2016) following TRADES, and evaluate the robustness of PAT, TRADES and PAT-EntM. We show the results in Appendix D Table
Following (Athalye al. 2018), we perform the following sanity checks to make sure PAT-EntM is not causing gradient obfuscation, and to better approximate and evaluate its adversarial risk.
We expand perturbations to test whether the robust accuracy monotonically decreases, and whether sufficiently large perturbations lead to 0 accuracy. We utilize PGD with k/255, step size 2/255 and & iterations, & [10, 150], on PAT. PAT-EntM and TRADES, on CIFAR-10 and CIFAR-100 We show the results in Fig. T On both datasets and under
Perturbation-AAccurayy Curve.

---

Under review as a conference paper at ICLR 2021
CIFARIO Accuracy curve
CIFAR100 Accuracy E Curve
0.6
0.8
PAT
PAT
PAT-EntM TRADES
PAT EntM
Accuracy
0.6
20.4
TRADES
0.4
0.3
0.0
0.0
20
40
60
80
1000
120
10
20
30
4.0
50
60
70
(a) CIFAR-10
(b) CIFAR-100
Figure I: Accuracy-s curve on CIFAR-10 and CIFAR-100, x-axis values denote pixel changes.
Accuraty-e
Accuracy- steas
Accunicy stops
Accuracy
0.54
0.49
TRADE:
2524
50.40
20.45
0.44
scaps
(a) CIFAR-10,
(b) CIFAR-100.
(c) CIFAR-10, 16 (d) CIFAR-I00, E 16
Figure 2: Accuracy-# # step curve CIFAR-I0 and CIFAR-100. = 8/255, 16/255
all PAT-EntM outperforms PAT and TRADES, Moreover, under sufficiently large e, all accuracies reach 0, which is another evidence that PAT-EntM is not obfuscating gradients.
Attacks with more iterations. We perform attacks with fixed and more iterations to make sure attacks converge, and to reliably evaluate the robustness of PAT-EntM. We take 8/255, 16/255, with # steps [8, 256]. step size max(1/510, 2:/h) kk) on CIFAR- 10 and CIFAR-100.
We show the results in Fig. It can be shown that even when the attacks converge at 200 steps, PAT-EntM still consistently outperform TRADES (over 4% on both CIFAR-10 and CIFAR-100). Moreover, PAT-EntM is consistently similarly robust more robust than TRADES.
Adaptive Attacks. We perform adaptive attacks on PAT-EntM. Due its connection with label smoothing, we leverage CE with smoothed labels in Eqn. 6 as the loss function Lath in Eqn. [ We search for the best parameter 7 for each attack from [0. 1] at an interval of 0.1. We consider E = 8/255, 16/255 on CIFAR-10 and CIFAR-100, and plot corresponding curves also in Fig.
With properly selected smooth parameter 7, adaptive attacks can succeed more often than non- adaptive ones. However, even under strong adaptive attacks, we can still see improvements of PAT- EntM over PAT (7% on CIFAR-IO and 4% on CIFAR- 100, = 16/ 255) and in most cases over TRADES, except for CIFAR-10. are 45.05% and 45.21%, respectively.
8/255, where robust accuracies of PAT-Ente ENT-EntM and TRADES
5.3 HYPERPARAMETER ANALYSIS
We vary the hyperparameter À in Eqn. and to see how the uncertainty level influences aceu- raries on both clean and adversarial examples. We vary X E [0, 10] CIFAR-10 and CIFAR-100.
We plot the accuracies on clean examples and under PGD40-8 in Fig. 3 The results show that on both datasets, accuracies both clean and adversarial examples simultaneously increase with À from 0.1 to 5. This is an appealing property and contrary to the belief that accuracy and robustness odds with each other (Zhang al. 2019 Tsipras et al. 2018). We also observe in Fig, that on both datasets, = and À 5 achieve similar performances (within 0.5%) in terms of both accuracy and robustness, which shows that EntM not highly sensitive hyperparameters.
7

---

Under review as a conference paper at ICLR 2021
CIFAR1O Accuracy A curve
CIFAR100 Accuracy À curve
0.45
0.20
0.000
0.44
0.82
0.24
Cleen
0.875
0.43
0.61
0.22
Clean (Left)
Clean (Left)
0.870
0.60
PGD40-6 (Righ0
0.20
03
10
0.5
10
(a) CIFAR-10
(b) CIFAR-100
Figure Hyperparameter Analysis for
HOW UNCERTAINTY PROMOTION WORKS FURTHER ANALYSIS
In this section we dig deeper into how uncertainty promotion works, which may bring deeper insights towards more robust algorithms. For brevity, we use f(X) as a shorthand for f(X; 0) and omit 0.
We define M f(X,, maxisyy f(X); as the decision margin of f point X, and introduce connecting robustness with M. and the Lipschitz constant of f.
Theorem |
Theorem Tsuzuku et al. (2018). Denote the global Lipschitz constant of ((x;0) as 1.. The following holds for any data point X, and any perturbation 6.
M 2 /21881221 = Mf 2 0.
Theorem shows that, for an arbitrary noise 8, as long as ||0112 S M,x the model f will output V21,"
correct predictions on 6-. Therefore, we focus on the metric normalized margin, defined as My.x for analyzing robustness. However, as l is hard to compute, we then introduce an approximation for the analysis. We focus on the first-order approximation of f(X).
[(X) R f(X) VxI(X) -X'
(10)
If the approximation holds well locally, we have the following inequality:
IIf(x')- - f(x)]l2 (Xx) X)1l2 < [X)( X112,
-
(11)
(I(()) f(X)|2
[Vx](x) >
(12)
IX'-XI)
where Vxf is the spectral norm (also the largest singular value) of the Jacobian matrix. Eqn. 12 shows that Vx f(X)l2 locally upper bounds the local Lipschitz constant of f at point X
(the R.H.S of Eqn 12 Therefore, we compute [Vx((X)laa
My,x via differentiation to analyze robustness.
However, since f(X) is non-linear, we define another term Q, (X', describing how much f fXX deviates from the linear approximation, to account for cases where Eqn I0 approximats poorly.
xx(X',x)= = 15(x') - -((X) - (XX( - X)1l2 ||x X||2 S..
(13)
Vxf((X)(XX -XI1
We sample 2,000 test samples from CIFAR-10, and study the relationship among normalized mar- gins, non-linearity (X',x), and robustness. We use L2 adversaries in correspondence with The- orem I We list related results in Table 4 With EntM, V /Vx)(l))l is shrunken by over 10 times regardless whether AT is used, leading
4-time increase of normalized margin in both EntM and PAT-EntM. However, while EntM alone achieves high regularized margin, it also massively compro-
mises local linearity compared to Normal, as shown by the high (X X). Therefore, the area where the linear approximation in Eqn. 10 12 holds is limited, thereby umable to guarantee robust- ness. By contrast, PAT-EntM only worsens local linearity slightly compared to PAT, and therefore the enlarged normalized margin contributes better robustness.
8

---

Normalized Margin Under review as a conference paper at ICLR 2021
Models
Normal
EntM
PAT
PAT-EntM
MA IV x (XIl2
10.55
1,3054
6.12
1.12
72.26
3.44
10.69
0.84
M xxx2l
0.1919
2.8132
0.6398
2.9913
Accuracy (PGI2200.55)
0.0029
0.166
0.5909
0.6142
Qy(X(adr), X)
4.4
52.51
4.59
5.36
Correlation
Mr.x [(x1(x ||X - -X112)
0.8327
0.4794
0.8216
0.8131
Table 4: Analysis of margin M, Jacobian matrix V x f(x;0) and non-linearity Q Q X).
EntM, Qi(Xindv
PATIERAM
EntM, QrX((avii, X) <
PAT
1.4
Normalized Margin
(a) EntM, PAT, PAT-EntM
(b) EntM, High and low Q2 (X(((a))) X)
Figure 4: Relationship between normalized margin and distortion for EntM, PAT and PAT-EntM.
We further show that normalized margin, along with local linearity explains better robustness. We leverage an infinite L2 attack on each example X until misclassification. We then compute the distortions (ade) X |2. and study the relationship between distortions and normalized margins. A more robust model should require larger distortion successfully attack.
We plot the normalized margins and distortions in Fig 4(a) and show their Pearson correlations in Table 4 On Normal, PAT, PAT-EntM, the correlation much higher than that on EntM. Also, Fig. 4(a) s hows that although EntM has similar normalized margins as PAT-EntM, the distortions required are smaller on EntM than on PAT-E EntM.. Since linearity is not only the key in the derivations of Eqn.
12 but also major difference of PAT-EnnM from EntM, these results indicate that high -iniiinar compromises robustness on EntM.
We further separate points with (X((oo)) xX) 65 on EntM, which is the 40-percentile, and plot them separately in Fig. 4(b) Samples with higher local non-linearity have higher normalized margins as PAT-EntM, but higher distortions and hence no better robustness. All results show that only when combined with local linearity, normalized margins explains robustness.
7 CONCLUSION
In this paper, we revisit uncertainty promotion regularizers in the field of adversarial learning. We find out that uncertainty promotion regularizers alone are causing gradient obfuscation, and that they alone only provide inconsistent robustness against small perturbations. Contrarily, our extensive ex- periments demonstrate that uncertainty promotion regularizers augment AT, improving accuracy clean examples and enhancing robustness, especially under large perturbations. We further demon- strate that both good local linearity and shrunken norm of Jacobian matrices contribute to better robustness shown by PAT-EntM than PAT.
We hope that this paper would again underscore the necessity to evaluate under strong attacks, and raise attention to further insights about why uncertainty promotion regularizers work. We consider theoretical investigations of why they work well alongside AT an important future work.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp. 274-283, 2018.
Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning, Pattern Recognition, 84:317-331, 2018.
Nicholas Carlini and David Wagner. Defensive distillation is robust to adversarial examples. arXiv preprint arXiv: 1607.0431,. 2016.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Abhimanyu Dubey, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik. Maximum-entropy fine grained classification. In Advances in Neural Information Processing Systems, pp. 637-647, 2018.
2018.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3996-4003, 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014.
Alex Graves and Navdeep Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In International Conference on Machine Learning, pp. 1764-1772, 2014.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with stochastic actor. arXiv preprint arXiv:1801.01290 2018.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv: 103.0053, 2015.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06333, 2018.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint wr..iv:1007.25555 2016.
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.0660 06083, 2017.
Rafael Müller, Simon Korbblith, and Geoffrey E Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, pp. 4694-4703, 2019.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970-4997, 2019.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEEE, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506-519, 2017.
Gabriel Pereyra, George Tucker, Jan Chorowski, Eukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.06544 2017.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. arXiv preprint / 2020.
10

---

Under review as a conference paper at ICLR 2021
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-555 2015.
Ali Shafahi, Amin Ghiasi, Furong Huang, and Tom Goldstein. Label smoothing and logit squeezing: A replacement for adversarial training? arXiv preprint arXiv:/910.11580 2019.
Chuanbiao Song, Kun He, Liwei Wang, and John E Hoperoft. Improving the generalization of adversarial training with domain adaptation. arXiv preprint arXiv:1810.00740, 2018.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1r1iiv1312.611 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and 7higniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2818-2822 2016.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. arXiv preprint arXiv: 12152, 2018.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certifi- cation of perturbation invariance for deep neural networks. In Advances in neural information processing systems, pp. 6541-6550, 2018.
Jonathan Uesato, Brendan O' D'Dooggue, Pushmeet Kohli, and Aaron Oord. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning, pp. 5025-5034, 2018.
Xiaoyong Yuan, Pan He, Qile Zhu, and Xiaolin Li. Adversarial examples: Attacks and defenses for deep learning. IEEE transactions on neural networks and learning systems, 309)):805-22244, 2019.
Sergey Zagoruyko and Nikos Komodakis. 1655::1605.07146 2016.
Wide residual
networks.
arXiv preprint
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning, pp. 7472-7482, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. arXiv preprint 122:4 2020.
Brian Ziebart, Andrew Maas, Andrew Bagnell, and Anind Dey. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pp. 1433-1438. Chicago, IL, USA, 2008.
A DETAILED EXPERIMENTAL SETTINGS
A.1 ARCHITECTURES
We train ResNet18 for CIFAR-10, CIFAR-100 and SVHN. We the same architecture of ResNet18 as in TRADESH The input size for ResNet18 8 is 32 x 32 x 3.
We train a four-layer CNN for MNIST. The detailed architecture is:
Input: 28 x 28 x 1 grayscale images.
5x5 convolution with stride 5 and 10 filters. ReLU activation.
Max pooling with stride 2.
5x5 convolution with stride 5 and 20 filters. ReLU activation.

---

Under review as a conference paper at ICLR 2021
Max pooling with stride 2.
Flatten.
FC layer with input shape 320 and output 10. ReLU activation.
FC layer with input shape 50 and output shape 10.
A.2 TRAINING PARAMETER SETTINGS
Settings/Models
Four-layer
ResNet18, without AT'
ResNet18, with AT
Input Scale
[0, 1]
Input Size
x 28 1
32 32 3
x x 3
Total Training Epochs
40
200 4
120
Weight Decay
10-4
10 -4
Optimizer
SGD with momentum 0.9
Initial Learning Rate
0.01
0.1
0.1
Learning Rate Decay Rate
0.1
0.1
0.1
Learning Rate Decay Epoch
20
[100, 150]
[60, 84, 100]
Early Stopping
No
No
No
Batch Size
128
64
64
Table Training Settings
We list all training settings in T For each quantitative result, we train 3 models, and each PGD repeated 3 times for average.
For datasets, we perform standard data augmentation techniques, including random cropping and random horizontal flipping (on CIFAR- 10 and CIFAR-100).
B FULL EXPERIMENTAL RESULTS
CIFAR-10
Without AT
With AT
Attacks
Normal
Mode
EntM
ADP
LS
PAT
PAT-EntM
PAT-LS
TRADES
Clean
94.78
94.71
95.72
95.15
86.47
87.87
88.31
84.01
FGSM4
37.05
72.37
69.5
70.33
72.96
72.95
69.21
PGD10-4
3.27
58.45
42.8
66.07
68.33
71.26
71.48
68.72
PGDI0-8
37.49
21.38
46.17
48.87
53.89
53.22
51.35
White-Box
PGD10-16
17.06
6.25
24.79
19.63
33.65
31.17
25.77
PGD20-4
0.31
36.73
22.38
47.86
67.89
70.1
69.88
67.58
PGD20-8
14.59
5,8
23.11
44.61
48.72
47.4
47.82
PGD20-16
3.86
0.82
7.95
12.76
24.79
22.73
18.27
PGD40-4
21.12
12.15
24.6
67.51
69.63
69.44
67.4
PGD40-8
5.01
1.94
6.26
43.03
45.95
44.79
46.74
PGD40-16
0.7
0.13
0.93
10.26
19.56
18.3
15.27
PGD10-4
27.5
35.84
31.79
32.24
76.88
79.32
79.24
75.52
PGD10-8
7.57
13.26
10.12
10.2
63.9
65.82
65.76
64.8
Black-Boo Box
PGD20-4
19.73
28.53
24.78
26.75
76.73
78.67
78.69
74.84
PGD20-8
3.12
7.43
5.36
6.46
62.24
64.44
64
63.65
Table 6: Performance under various attacks CIFAR-10 (%). denotes < 0.1%. Italics indicate
white-box accuracies higher than corresponding black-box ones.
We show full experimental results in 6 and Although in MNIST. the improvement is less significant (1% or below), and TRADES significantly outperforms both PAT and PAT-EntM, PAT- EntM still achieves improvement over AT in terms of robustness under various attacks.
12

---

Under review as a conference paper at ICLR 2021
CIFAR-100
Without AT
With AT
Attacks
Normal
Mode
EntM
ADP
LS
PAT
PAT-EntM
PAT-LS
TRADES
Clean
75.66
77.17
80.04
77.60
59.65
62.53
62.83
54.63
FGSM4
12.73
29.8
28.29
31.14
38.86
44.37
43.2
37.94
PGD10-4
0.97
9.1
6.15
9.91
37.76
43.43
42.39
37.61
PGD10-8
3.32
1.3
3.35
22.41
27.13
24.77
White-Box
PGD10-16
0
0.7
0
1
7.47
13.18
11.21
10.28
PGD20-4
0
2.9
1.94
3.15
36.89
42.65
41.3
36,83
PGD20-8
0.69
0.15
0.87
20.46
26.89
24.75
23.12
PGD20-16 16
0
0
0
5.43
9.82
8.01
PGD40-4
0
1.08
0.87
1.35
36.55
42.45
41.08
36.45
PGD40-8
0
0.1
0
19.62
26.1
24.05
22.6
PGD40-16 16
0
4.73
8.22
6.69
7.26
PGDI0-4
13.77
18.07
21.36
18.44
49.4
52.25
52.97
46.76
PGD10-8
3.65
5.45
7.07
5.6
38.84
41,48
41,44
38.55
Black-Box
PGD20-4
9,92 1.89
13.31
16.12
13.5
49.34
52.02
52.56
46.45
PGD20-8
2.84
3.7
2.98
38.25
40.66
40.39
38.13
Table 7: Performance under various attacks on CIFAR-100 (%). denotes 0.1%
SVHN
Without AT
With AT
Attacks
Normal
Mode
EntM
ADP
L.
PAT
PAT-EntM
PAT-LS
TRADES
Clean
97.04
97.18
97.58
97.31
94.08
94.08
93.28
FGSM4
65.74
85.39
83.55
85.75
82.99
83.21
83.04
84.26
PGD10-4
27.54
72.99
70,76
79.86
79.72
80.11
79.39
80.41
PGD10-8
4.73
48.47
49.91
59.13
61.71
63.88
62.6
64.95
White-Box
PGD10-16
21.61
25.13
31.97
29.33
38.85
35.77
36.27
PGD20-4
13.04
55.47
54.68
61.34
78.15
78.66
77.55
78.45
PGD20-8
0.81
22.42
25.24
27.35
55.24
57.37
55.56
58.13
PGD20-16
5.0
6.6
7.2
18.65
27.57
23.9
24.49
PGD40-4
40.85
42.73
44.53
77.71
77.06
77.96
PGD40-8
0.36
10.27
13.62
12.4
52.76
54.65
52.14
$7.02
PGD40-16
1.15
2.25
1.9
14.18
23.8
17.58
17.96
PGDIO-4
66.07
70.47
67.83
69.76
83.69
83.85
84.08
83,78
PGDI0-8
43.7
49.77
46.53
48.25
68.88
69.03
69.48
69.4
Black-Box
PGD20-4
60.17
66.33
63.79
65.8
82.67
83.15
83.3
82.94
PGD20-8
36.1
45.3
40.18
43.05
65.00
65.42
65.43
65.32
Table 8: Performance under various attacks on SVHN (%).
denotes < 0.1%. Italics indicate
white-box accuracies higher than corresponding black-box ones.
C GRADIENT OBFUSCATION OF LS AND ENTM WITHOUT AT
In this section we present more evidence regarding gradient obfuscation of EntM and L.S.
C. RANDOM SEARCHING
present other evidence that EntM, LS and ADP alone may sometimes lead to gradient obfusca- tion. We sample 1000 images from the test set of CIFAR-10. We carry out random search attacks for 10,000 times on each image where EntM, LS and ADP succeeded in defending. and list the success rate of random attacks in Table 10 In many seemingly successful defenses (15% LS, 6% on ADP and EntM), gradient- based attackers are obfuscated that they cannot find adversarial examples while such examples do exist.
13

---

Under review as a conference paper at ICLR 2021
MNIST
Without AT
With AT
Attacks
Normal
Mode
EntM
ADP
PAT
PAT-EntM
TRADES
Clean
99.2
99.13
99.37
98.47
98.04
97.78
FGSM32
55.31
62.77
77.61
95.14
95.01
95.18
PGD10-32
26.03
32.73
$7.54
94.15
94.1
94.28
White-Box
PGD10-48
2.35
3.08
17.06
89.82
90.65
90.57
PGD20-32
20.38
23.32
43.76
93.91
93,91
94.2
PGD20-48
1.13
1.28
7.34
88.62
89.77
90.53
PGD40-32
19.04
19.89
39.14
93.54
93.88
94.17
PGD40-48
0.98
0.85
4.52
88.39
89.59
90.52
PGD10-32
83.11
85.76
87.07
96.32
96.16
96.93
PGD10-48
46.94
52.2
52.77
94.6
94.48
95.79
Black-Box
PGD20-32
81.43
83.95
84.88
96.31 94.36
95.96
96.81
PGD20-48
44.68
46.23
46.81
94.23
95,45
Table 9; Performance under various attacks on MNIST (%).
Models
EntM
ADP
LS
PAT
# PGD8 fails
368
289
487
493
Among which # Random succeeds
20
12
70
0
Table 10: Attack success rate of random searching attacks on CIFAR-10.
We also perform exactly the same experiments for PAT and PAT-EntM. For samples where PAT and PAT-EntM succeeded defending, none of them can be successfully attacked via a 10,000-time random search attack. It endorses that PAT-EntM is not suffering from gradient obfuscation.
C.2 Loss SURFACE VISUALIZATION
We choose data-label pairs (X,y) from the CIFAR-10 test set, and plot its local loss surface at X' X eid; E2d2 to visually show the properties of EntM and PAT-EntM. dj sign CE (X;0), y) the gradient sign direction, and d2 is random direction with ||d2]| = 1. We choose E E2 [-0.04, 0.04].
Among the first 8 samples in CIFAR-10 test set, we select of them and plot them in Fig. It can be shown that EntM alone is likely (25% would be rough estimation. ) to create a 'hole in a small neighborhood of X, thereby causing gradient- trasient-based iterative attackers to stuck. This result corresponds with the fact that weak gradient-based attackers cannot achieve high success rate due to gradient obfuscation. It also corresponds to the results in Table 4 that EntM alone may compromise local linearity by creating highly curved surface.
We also plot the decision margin Mf at X' = X + eidj + e2d2 in Fig. It can be shown that although EntM alone creates small neighborhood where the margin is large, the area of the neighborhood is small, leaving large areas where My.x 0. It shows that EntM alone cannot ensure robustness, especially when perturbations are large, and corresponds with results in Table
3
D RESULTS ON WIDERESNET34-10
Following TRADES (Zhang et al. 2019). we train WideResNNe33-100 (Zagoruyko Komodakis, 2016) with PAT, PAT-EntM and TRADES on CIFAR-10 and CIFAR-100 using the same procedure as Appendix(A) We use À = 1 for EntM on CIFAR-10, and 0.5 on CIFAR-1O8 100, which chosen from (0.5, 1, 1,2.5) 5). We report accuracy and robustness results in Table
14

---

Under review as a conference paper at ICLR 2021
LOX
Ch Grea.
So0
Grad.
Grad.
Grad.
(a) Normal
(b) EntM
(c) PAT
(d) PAT-EatM
Normal
EntM
PAT
PAT-EMI
LONG
Grad.
(c)
Normal
(f) EntM
(g)
(h) PAT-EntM
Figure 5: Loss surface visualization at X' = + Ejd1 + E2dg for for two X taken from the first
images in CIFAR-10 test
Grod.
Grad.
Grad.
Grad.
(a) Normal
(b) EntM
(c) PAT
(d) PAT-EntM
Normal
EntM
PAT
PAT-EntM
Grad.
(e) Normal
(I
EntM
(g)
(h) PAT-EnIM
Figure 6: Decision margin visualization at X' + eidi + Ezd2 for two X taken from the first images in CIFAR-10 test set.
15

---

Under review as a conference paper at ICLR 2021
CIFAR-10
CIFAR-100
PAT
TRADES
PAT-EntM
PAT
TRADES
PAT-EntM
Clean
88.11
86.09
88.65
61.96
58.57
63.93
PGD40-4
69.2
68.09
69.98
39.49
38.52
42.85
PGD40-8
45.53
46.7
52.98
22.89
23.49
26.43
PGD40-16
19.03
19.27
33.21
6.2
7.64
10.6
CW40-4
69.42
68.33
69.66
39.44
38.2
40.86
CW40-8
46.23
47.76
50.4
23.47
23.78
24.7
CW40-16
18.9
18.09
31.31
6.64
7.93
7.77
Table 11: Results on WideResNer34-10 of PAT, PAT-EntM and TRADES
CIFAR-10
PAT
TRADES
PAT-EntM
TRADES-ENE
Clean
86.47
84.01
87.87
85.76
FGSM4
70.33
69.21
72.96
70.43
PGD10-4
68.33
68.72
71.48
69.48
PGD10-8
48.87
51.35
53.89
51.56
White-Box
PGDIO-16
19.63
25.77
33.65
26.47
PGD20-4
67.89
67.58
70.1
68.83
PGD20-8
44.61
47.82
48.72
48.18
PGD20-16
12.76
18.27
24.79
19.01
PGD40-4
67.51
67.4
69.63
68.64
PGD40-8
43.03
46.74
45.95
46.43
PGD40-16
10.26
15.27
19.56
15.14
PGD10-4
76.88
75.52
79.32
77.09
PGD10-8
63.9
64.8
65.82
65.10
Black-Box
PGD20-4
76.73
74.84
78.67
76.81
PGD20-8
62.24
63.65
64.44
63.92
Table TRADES-EntM comparison with PAT, TRADES and PAT-EntM on CIFAR-10
We find out that by using large networks, the amount of uncertainty needed decreases. On the optimal level of uncertainty À 1 and 0.5 respectively, we can still observe consistent improvements with respect to both accuracies on clean examples and under attacks.
E TRADES WITH ENTM
We also carry out experiments on CIFAR- 10 and CIFAR- 100 to see how EntM works with TRADES. We take and list the results in Table 12 and It can be shown that TRADES-ERE improves accuracy on clean examples by 6% on CIFAR- 100 and 1.7% on CIFAR-10 compared to TRADES. the robustness improvements are less evident than PAT-EntM. e.g. less than 1% improvement compared to TRADES under attacks on CIFAR-10, and 2-4% improvements CIFAR-100.
16

---

Under review as a conference paper at ICLR 2021
CIFAR-100
PAT
TRADES
PAT-EntM
TRADES-EntM
Clean
59.65
54.63
62.53
60.96
FGSM4
38.86
37.94
44.37
42.84
PGD10-4
37.76
37.61
42.39
41.96
PGD10-8
22.41
24.77
29.09
27.78
White-Box
PGD10-16
7.47
10.28
13.18
12.33
PGD20-4
36.89
36.83
42.65
41.4
PGD20-8
20.46
23.12
26.89
25.99
PGD20-16
5,43
7.26
9.82
9.16
PGD40-4
36.55
36.45
42.45
41.22
PGD40-8
19.62
22.6
26.1
25,34
PGD40-16
4.73
7.26
8.22
7.9
PGD10-4
49.4
46.76
52.25
50.08
PGD10-8
38.84
38.55
41.48
39.37
Black-Box
PGD20-4
49.34
46.45
52.02
48.12 12
PGID20-8
38.25
38.13
40.66
39.01
Table 13: TRADES-EnE EntM in comparison with PAT, TRADES and PAT-En ENT-EntM on CIFAR-100
17


Reviews:
[{"review_id": "obbpodQfdb1", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors combine adversarial training with two methods that increase the entropy of the output distribution of neural networks (label smoothing and entropy maximization). The authors find that this combination of ideas increases adversarial robustness on standard benchmarks, especially in the regime of large perturbation budgets (e.g., 16/255 on CIFAR-10). The authors also investigate the effect of their methods on the classification margin to understand the increase in adversarial accuracy.\n\nWhile the authors provide a detailed experimental evaluation of their defense method, this part of the paper is still my main concern. In particular, I see the following issues:\n\n* It is not clear if the authors choose a sufficiently large steps size for the PGD attacks. Typical values are 2 * eps / k to 10 * eps / k, where eps is the l_inf perturbation budget and k is the step size. However, the authors choose smaller step sizes in at least some cases, e.g., eps / k for the perturbation accuracy curves in Figure 1. This is a concern particularly because Table 1 shows that the model accuracies still decrease substantially when going from 10 to 40 PGD steps.\n\n* Again on the note of step sizes, why did the authors choose max(1 / 510, eps / k) in the attacks with more iterations?\n\n* It would be good to present a single table with the smallest known adversarial accuracies for the various defenses. E.g., Table 1 does not seem to include the results from more adaptive attacks where TRADES is sometimes comparable to the proposed methods.\n\n* How did the authors choose random restarts? They mention sigma = 0.005 - is this for a Gaussian distribution? What happens if the authors pick a point from the appropriately scaled l_inf ball instead?\n\n* It would be good to see attacks with the Carlini-Wagner (margin) loss function.\n\nConsidering the well-known difficulties with evaluating defenses against adversarial attacks, I currently cannot recommend accepting the paper.\n\nAdditional comments:\n\n- The authors attribue Szegedy et al., 2013 with adversarial examples. The authors may be interested in https://arxiv.org/abs/1712.03141 , in particular Figure 8, for a more detailed history of this research direction.\n\n- Trade-offs between adversarial and standard accuracy have been studied before Rice et al., 2020. For instance, see https://arxiv.org/abs/1805.12152 and https://arxiv.org/abs/2002.10716 .\n\n- Could scaling the softmax temperature also work for increasing the entropy of the softmax distribution in a way that leads to increased robustness?\n\n- Could the proposed technique be combined with TRADES or training on additional unlabeled data? (e.g., see https://arxiv.org/abs/1905.13736 and https://arxiv.org/abs/1905.13725 ).\n\n- Is there a \"max\" missing in Equation 5?\n\n- Beginning of Section 6: \"deeper into the how\"\n\n- Equation 12: what is X + delta?\n\n---------------------------------------------------------------------------------------------\n\nThank you for the detailed response. I have updated my score from 4 to 5.", "extra_scores": {}, "extra_text": {}}, {"review_id": "bkGALOX1Z6b", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1) First, this work studies how entropy maximization and label smoothing combined with adversarial training can improve adversarial robustness. Although these two techniques have been shown to prevent model from being over-confident, I still think it is an over-claim that \"rethinking uncertainty in deep learning\" as this work does not truly study uncertainty in deep learning or the correlation between uncertainty and adversarial robustness.\n2) Label smoothing alone does not provide stronger adversarial robustness, this is not a surprising result as pointed out by many existing work. As entropy maximization is similar as label smoothing, it is also in the expectation that they have similar performance without providing stronger adversarial robustness.\n3) \"Combining label smoothing/EntM with adversarial training can further provide stronger adversarial robustness\", this is the main conclusion of this work. However, I am not fully convinced this is novel enough as this has been observed by existing work. In addition, this work only tested the model on the PGD attacks, the same type of attack (although with smaller l infinity norm bound) is used during training. It is very necessary to test the model against different types of attacks, especially decision-boundary based attacks, to support this conclusion as this is the main contribution of this work. \nIn all, I vote for a rejection for this work.\n\n******** After Rebuttal ************\nI carefully read the authors' response and unfortunately they do not address my concerns. Based on my research background in adversarial robustness and uncertainty estimates, I would keep my original rating unchanged as this work has very limited contribution to these two areas.", "extra_scores": {}, "extra_text": {}}, {"review_id": "bitzfW4zgM_", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary\nThis paper investigates the complementary mechanisms of adversarial training and uncertainty promoting regularizers. In the field of adversarial machine learning, adversarial training as proposed by Madry et al. 2017 has been the common method. In the field of uncertainty regularization, maximum entropy and label smoothing have been the accepted methods. However, the combination of both has not been investigated before. The paper provides extensive experiments on these methods. The final section gives insights in the theory behind adversarial training and uncertainty promoting regularization, where they show that the combined method increases a notion of normalized margin and a notion of adversarial robustness. \n\n# Strong & Weak points\n\n## Strong points\n\n  * The related work section contains an extensive overview of related and contemporary literature, where both literature in adversarial ML and uncertainty promotion is being discussed.\n  * Ablation experiments in figure 1 and 2 show that combining adversarial training and uncertainty promoting regularizers have better accuracy under a range of attack settings. \n  * Theoretical insights as to why Entropy Maximization would help improve adversarial robustness, complementary to adversarial training is provided in Section 6\n\n## Weak points\n\n  * The method is studied on small datasets (CIFAR, MNIST, SVHN) using small models (ResNet18). It remains to be seen how these insights translate to larger datasets (ImageNet) and larger models (ResNet50).\n  * In the abstract and introduction, the text speaks of “true” robustness, against “strong” attacks, but these terms are never defined.\n  * The increase in (approximate) normalized margin when using entropy maximization as shown in table 4 provides an important argument for the claim of this paper. However, the numbers were obtained using the CIFAR10 dataset and a ResNet18 model. A stronger case could be made with a larger dataset and a larger model.\n\n# Statement\n\nRecommendation: 6\n\nReasons\n   * The claims for a complementary benefit of adversarial training and uncertainty promoting regularization are backed up with both extensive experiments and theoretical insights.\n   * The fields of adversarial training and uncertainty regularizers have been evolving separately and this paper provides initial insights how these two lines of research can be combined. [Comment: I am not 100% up to date with the related literature, so I'll be looking to other reviewers if they are aware of existing work combining uncertainty regularization and adversarial training.]\n   * Ablation experiments in Figure 3 shows substrates for the complementary actions of Entropy Maximization and Adversarial training. Entropy maximization can be shown to increase a notion of “normalized margin width”, adversarial training can be shown to increase a notion of  adversarial robustness, and when combining the methods, both metrics increase. \n\n# Additional questions:\n\nTable 3a) misses the point cloud for normal training. How do the normalized margins and adversarial distances of normal training compare in this plot? Table 4 already shows that the normalized margin (0.19) is smaller than the three methods, but I miss the numbers for adversarial distances during normal training.\n\n# Minor feedback\n\nThese points are minor feedback and not part of the assessment.\n\n  * KL divergence can be decomposed as $KL(p|q) = H[q] + CE(p|q)$. Could this decomposition explain the differences between TRADES, PAT, and entropy maximization?\n\n  * Figure 1 & 2: please provide labels for the x axes.\n  * None of the derivations on eqn. 9 to 13 depend on $\\theta$. Consider dropping the $\\theta$ variables everywhere to focus the analysis on what really matters: the derivatives w.r.t. $x$. \n  * Equation 12: the $\\delta$ variable is only implicitly defined. It is not clear to me if its direction is parallel or orthogonal to the decision boundary. \n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "FmeoZcVHs5H", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper tries to improve the model robustness by modifying the loss function with the EntM or LS term. Also, they give a further analysis to identify how uncertainty promotion works.\n\n\nStrength:\n\n-- The methods seem rational. And the experiments also demonstrate the effectiveness.\n\n\nWeakness:\n\n--The novelty is limited. Training the model without the one-hot label is not new. Also, it seems that the label smoothing is the only contribution proposed by this work, but it is pretty naive.\n\n--The analysis part is difficult to follow.\n\n\nComments:\n--There are many works that claim the distillation training can improve the robustness of the model, which also adopt the soft label in loss function. Please compare with relative methods.\n\n-- For uncertainty analysis, the explanation is not straightforward. First, I am not sure why Eq. (11) holds. Second, why the L2 norm of the Jacobian matrix is its largest singular value? Also, I found the analysis is finally conducted through empirical studies. I think maybe the equation descriptions can be modified easier to understand and make the empirical study more clearly if possible.\n\n-- I think the performance drop with adversarial training is still high, although the results show the accuracy is better than TRADE. I suggest the authors to provide some experiments that adjust the hyper parameters to identify the trade-off between the clean accuracy and the robustness of the model.\n\n--For Eqs. (7)-(8), what does the term ‘logC’ stand for?\n\n--This work only considers untarget attack. How about the performance under target attack?\n\n--The attack algorithms seem not state-of-the-art. Please try more algorithms if possible.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviews were a bit mixed, with some concerns on the novelty and experimental evaluation. While the authors' efforts during rebuttable were appreciated, the overall sentiment is that this work, in its current form, cannot be accepted to ICLR yet. Please consider revising your work based on the excellent reviews. Some more comments from the AC's independent assessment:
 
(a) Further elaboration on the novelty is needed. Currently the main message appears to be that if we combine two existing approaches (AT and EntM or LS) then we get better results. This is perhaps not too surprising and more elaboration on the significance would be appreciated. 

(b) More comparisons in the experiments, including the SOTA performances and alternative defenses (some below).
 
(c) The analysis in Section 6 adds more confusion than clarification. It is clear that EntM and LS would largely decrease M_f, but why would they also decrease the Lipschitz constant even more sharply? If this explanation is useful, why not directly regularize the Lipschitz constant and maximize the margin M_f? There is in fact a large body of work on this, see for example:

1. Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation

2. Parseval Networks: Improving Robustness to Adversarial Examples

3. L2-Nonexpansive Neural Networks

4. and the many references since.


Decision: Reject