SPROUT: Self-Progressing Robust Training
Authors: Minhao Cheng
Pin-Yu Chen
Sijia Liu
Shiyu Chang
Cho-Jui Hsieh
Payel Das
Venue:  (2020)
Keywords: robustness
robust training
trustworthy machine learning

Abstract: Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy and reliable machine learning systems. Current robust training methods such as adversarial training explicitly specify an ``attack'' (e.g., $\ell_{\infty}$-norm bounded perturbation) to generate adversarial examples during model training in order to improve adversarial robustness. In this paper, we take a different perspective and propose a new framework SPROUT, self-progressing robust training. During model training, SPROUT progressively adjusts training label distribution via our proposed parametrized label smoothing technique, making training free of attack generation and more scalable. We also motivate SPROUT using a general formulation based on vicinity risk minimization, which includes many robust training methods as special cases. Compared with state-of-the-art adversarial training methods (PGD-$\ell_\infty$ and TRADES) under $\ell_{\infty}$-norm bounded attacks and various invariance tests, SPROUT consistently attains superior performance and is more scalable to large neural networks. Our results shed new light on scalable, effective and attack-independent robust training methods.

Under review as a conference paper at ICLR 2020
SPROUT: SELF-PROGRESSING ROBUST TRAINING
Anonymous authors
Paper under double-blind review
ABSTRACT
Enhancing model robustness under new and even adversarial environments is crucial milestone toward building trustworthy and reliable machine learning systems. Current robust training methods such as adversarial training explicitly specify "attack" (e.g., -norm bounded perturbation) to generate adversarial examples during model training in order to improve adversarial robustness. In this paper, we take a different perspective and propose new framework SPROUT, self-progressing robust training. During model training, SPROUT progressively adjusts training label distribution via our proposed parametrized label smoothing technique, making training free of attack generation and more scalable. We also motivate SPROUT using general formulation based on vicinity risk minimization, which includes many robust training methods special cases. Compared with state- of-the-art adversarial training methods (PGD-f and TRADES) under &oo -norm bounded attacks and various invariance tests, SPROUT consistently attains superior performance and is more scalable to large neural networks. Our results shed new light scalable, effective and attack- tutack-independent robust training methods.
INTRODUCTION
While deep neural networks (DNNs) have achieved unprecedented performance variety of datasets and across domains, developing better training algorithms that are capable of strengthening model robustness the next crucial milestone toward trustworthy and reliable machine learning systems. In recent years, DNNs trained by standard algorithms (i.e.. the natural models) are shown to be vulnerable adversarial input perturbations (Biggio al.) 2013) Szegedy et al. 2014). Adversarial examples crafted by designed input perturbations can easily cause erroneous decision making of
natural models (Goodffllow al. 2015) and thus intensify the demand for robust training methods.
sfate-of-the-ary robust training algorithms are primarily based on the methodology of adversarial training (Goodfellow et al. [0155] Madry et al. 2018). which calls specific attacking algorithms to generate adversarial examples during model training in order to learn robust models. Albeit effective, adversarial training based methods have the following limitations: (i) poor scalability the process of generating ad adversarial examples incurs considerable computation overhead. For instance, experiments show that, with the same computation resources, standard adversarial training (with 7 attack iterations per sample in every minibatch) of Wide ResNet CIFAR-10 consumes 10 times more clock time per training epoch when compared with standard training; (ii) attack specificity adversarially trained models are usually most effective against the same attack they trained on, and the robustness may not generalize well to other types of attacks (Tramèr & Boneh 2019 Kang al. 2019;; (iii) preference toward wider network adversarial training are more effective when the networks have sufficient capacity (e.g., having more neurons in network layers) (Madry et al, 2018).
To address the aforementioned limitations, in this paper we propose a new robust training method named SPROUT, which is short for gelf-progressing robust training. We motivate SPROUT by intro- ducing a general framework that formulates robust training objectives via vicinity risk minimization (VRM), which includes many robust training methods as special cases. It is worth noting that the robust training methodology of SPROUT is fundamentally different from adversarial training, as SPROUT features self- adjusted label distribution during training instead of attack generation. In addition to our proposed parametrized label smoothing technique for progressive adjustification of training label distribution, SPROUT also adopts Gaussian augmentation and Mixup (Zhang et al.) 2018) to further enhance robustness. In contrast to adversarial training. SPROUT spares the need for attack generation and thus makes its training scalable by significant factor, while attaining better

---

Under review as a conference paper at ICLR 2020
C&W Acc
C&W Acc
C&W Acc
C&W Acc
Scalibility
Scalibility
Scalibility
Scalibility
Linf Acc
Acc Acc
Acc
L_nff Acc
Invariance
Invariance
Invariance
Invariance
Clean Acc
Clean Acc
Clean Acc
Clean Acc
(a) Natural
(b) Adversarial training
(c) TRADES
(d) SPROUT (ours)
Figure Multi-dimensional performance comparison of four training methods using VGG-16 network and CIFAR-10 dataset. All dimensions are separately normalized by the best-performance
method. The average score of each method is 0.6718 for natural (standard training). 0.6900 for PGD-foc based adversarial training (Madry et al. 2018), 0.7 for PGD- based TRADES (Zhang) et al. 2019), and 0.8798 for SPROUT (ours). The exact numbers are reported in Table in Appendix.
comparable robustness performance on a variety of experiments. We also show that SPROUT can find robust models from either randomly initialized models pretrained models, and its robustness performance is less sensitive to network width.
1.1 CONTRIBUTIONS
Multi-dimensional performance enhancement. To illustrate the advantage of SPROUT over ad- versarial training and its variations, Figure commpares the model performance of different training methods with the following five dimensions summarized from experimental results: (i) Clean Ace standard test accuracy, (ii) L inf Acc accuracy under f -norm projected gradient descent (PGD) attack (Madry al. 2018), (iii) C& W Acc accuracy under f--norm Carlini-Wagner (C&W) attack, (iv) scalability- per epoch clock run-time, and (v) invariance invariant transformation tests includ- ing rotation, brightness, contrast and gray images. Comparing PGD-L based adversarial training (Madry et al. 2018) and TRADES (hang et al. 22019) SPROUT attains at least 20% better L_inf Acc, 2% better Clean Acc, 5x faster run-time (scalability), ), 2% better invariance, while maintaining C&W Acc, suggesting a new robust training paradigm that is scalable and comprehensive.
We further summarize the main contributions of this paper as follows:
We propose SPROUT, self- progresgggss robust training method composed of three modules that are efficient and free of attack generation: parametrized label smoothing, Gaussian augmentation and Mixup (Section B). They altogether attain the state-of-the-ee robustness performance and are scalable to large-scale networks. In Section (.66iinnn46vvve we will show that these modules are complimentary to enhancing robustness. We also perform an ablation study to demonstrate that our proposed parametrized label smoothing technique contributes to the major gain in boosting robustness.
To provide technical explanations for SPROUT, in Section 2 we motivate its training methodology based on the framework of vicinity risk minimization (VRM). We show that many robust training methods, including attack-specific and attack attack-independent approaches,
be characterized specific form of VRM. The superior empirical results of SPROUT provide new insights on developing efficient robust training methods and theoretical justification based on VRM.
We evaluate the malti-ddimensonall performance of different training methods on (wide) ResNet and VGG networks using CIFAR- 10 and ImageNet datasets. Notably, although SPROUT is attack- independent during training, we find that SPROUT significantly outperforms two major adversarial training methods, PGD-L adversarial training (Madry et al. 2018) and TRADES (Zhang et al.
2019), against the same type of attacks they used during training (Section 4.2) Moreover, SPROUT is more scalable and runs least 5x faster than adversarial training methods (Section 4.5 also
attains higher clean accuracy and generalizes better to various invariance tests (Section 4.4 and is less sensitive to network width (Section 4.6).
1.2 RELATED WORK
Attack-specific robust training. The seminal work of adversarial training with first-order attack algorithm for generating adversarial examples (Madry et al. 2018) has greatly improved adversarial robustness under the same threat model (e.g., lo -norm bounded perturbations) as the attack algorithm. has since inspired many advanced adversarial training algorithms with improved robustness. For instance, TRADES (Zhang al. 2019) is designed to minimize theoretically-drive upper bound
2

---

Under review as a conference paper at ICLR 2020
on prediction error in adversarial examples. which led the first-ranked defense in the NeurIPS 2018 Adversarial Vision Challenge. Bilateral adversarial training (Wang 2019) finds robust models by adversarially perturbing the data samples as well as the associated data labels using attack algorithms. A feature-scattering based adversarial training method is proposed in (Zhang & Wang 2019). Different from attack-specific robust training methods, our proposed SPROUT is free of attack generation, yet it can outperform attack-specific methods. Another line of recent works uses an adversarially trained model along with additional unlabeled data (Carmon et al. 2019) Stanforth et al. 2019)
learning with adversarial examples (Hendrycks et al., 2019) to improve robustness, which in principle can also be used in SPROUT but is beyond the scope of this paper.
Atfack-independent robust training, Here we discuss related works on Gaussian data augmentation, Mixup and label smoothing. Gaussian data augmentation during training a commonly used baseline method to improve model robustness (Zantedeschi et al. 2017). It is revisited in (Cohen et al., 2019)
as a scalable and certifialle defense method called random smoothing. Mixup (Zhang et al. 2018)
and its variants (Verma al. 2018: Thulasidasan et al. 2019) are recently proposed approach to improve model robustness and generalization by training a model on convex combinations of data sample pairs and their labels. Label smoothing was originally proposed in (Szegedy et al. 2016
as a regularizer to stabilize model training. The main idea is to replace one-hot encoded labels by assigning non-zero (e.g., uniform) weights to every label other than the original training label. Although label smoothing is also shown to benefit model robustness (Shafahi et al.] [2018] Goibert & Dohmatob 2019), its robustness gain is relatively marginal when compared to adversarial training. In contrast to current static (i.e., pre-defined) label smoothing function, in SPROUT we propose novel parametrized label smoothing scheme, which enables adaptive sampling of training labels from parameterized distribution on the label simplex. The parameters of the label distribution are progressively adjusted according to the updates of model weights.
2 GENERAL FRAMEWORK FOR FORMULATING ROBUST TRAINING
In supervised learning, the task is essentially learning a K-class classification function f E F that has desirable mapping between data sample X and the corresponding label y E y. Consider a loss function L that penalizes the difference between the prediction f()) and the true label from an unknown data distribution P. (x: y) P. The population risk can be expressed
/ L(((x), y) y) ddddy
R()) =
(1)
However, as the distribution P is unknown, in practice machine learning uses empirical risk mini- mization (ERM) with the empirical data distribution of n training data {aa, y/ )/
Po y) = 8(x = ate=;;,330 1-.y=1 yy)
(2)
to approximate P(æ, y). where 8 is a Dirac mass. However, a more principled approach to use Vicinity Risk Minimization (VRM) (Chapelle et al. 2001), which is
Pa 9) = _ Σ vā ğ x., y:)
(3)
where y is a vicinity distribution that measures the probability of finding the virtual sample-label pair (#. 9) in the vicinity of the training sample-label pair (zx, y:). Therefore, ERM can be viewed as special case of VRM when v = 8. VRM has also been used to motivate Mixup training (Zhang et al. 2018). Based on VRM, we propose general framework that encompasses the objectives of many robust training methods as the following generalized cross entropy loss:
K
H H ÿ. f)
Σ [log ((())))))())))
(4)
k=1
where (ā){ is the model's k-th class probability output of the input E, g(-) R- is mapping adjusting the output probability, and h(-) R - R is mapping adjusting the training label
distribution. When ă = x, ÿ = y and g = h = I. where denotes the identity mapping function, the loss in (4) degenerates to the conventional cross entropy loss used in ERM.
3

---

Under review as a conference paper at ICLR 2020
Table 1: Summary of several robust training methods using VRM formulation in 4 PGD. means (multi-step) PGD attack with perturbation budget e and Dirichlet(b) is the Dirichlet distribution parameterized by b. GA stands for Gaussian Augmentation and LS stands for Label Smoothing.
Mathods
xf-)
altack-specifi
Nonural
Zuntedeuchi et 2017
Vi
Advemart 122
2015
PGD,
TRADES Zhigg
nlg (f
PGD, (a)
Subll
fizje ((R)
Min
May Agg
LSKKAA [Scati
Mir A")
Bilater Timing
Watgg
2019
(aD),(a) (ax) tone two stpp (1 (n,, (M)
(1 a/u PGD..(M)
SPROUT (murs
Dinchlet(( - nly + +
(1 11- Apg + Ago
Based on the general VRM loss formulation in 4 in Table 1 we summarize a large body of robust training methods in terms of different expressions of g()). h(-) and (Ø. ÿ). For example, the vanilla adversarial training in (Madry et al., (2018) aims to minimize the loss of adversarial examples generated by the (multi-step) PGD attack with perturbation budget e, demoted by PGD,(-). Its training objective
be rewritten ø PGDD(2). ý y and g h I. In addition to adversarial training only on perturbed samples of æ, Wang 2019) designs adversarial label perturbation where it uses ã PGD, (z)) y = (1 a y + (y) and a E [0, is mixing parameter. TRADES (Zhang et al. 2019) improves adversarial training with an additional regularization on the clean examples, which is equivalent to replacing the label mapping function h(-) from identity to (1 - a)y + fā). Label smoothing (LS) alone is equivalent to the setup that g I, = 2, y = y and h(.)) (1 - a)y + au, where u often set as a uniform vector with value 1/K for a K -class supervised learning task. Joint training with Gaussian augmentation (GA) and label smoothing (LS) as studied in (Shafahi et al., 2018) is equivalent to the case when á = N(x A ÿ y. g I and hyy) (1 a)y + a/ K. We defer the connection between SPROUT and VRM to the next section.
SPROUT: SCALABLE ROBUST AND GENERALIZABLE TRAINING
In this section, we formally introduce SPROUT, a novel robust training method that automatically finds better vicinal risk function during model training in self-progressing manner.
3.1 SELF-PROGRESSING TRAINING VIA PARAMETRIZED LABEL SMOOTHING
To stabilize training and improve model generalization, Szegedy al. (0016) introduces label smoothing that converts "one-hot" label vectors into "one-warm" vectors representing low-confidence classification, in order to prevent model from making over-confident predictions. Specifically, the one-hot encoded label y is smoothed using
ÿ (1- a)y + aru
(5)
where a E (0, 1] is the smoothing parameter. A common choice is the uniform distribution u = K where K is the number of classes. Later works (Wang) 2019 Goibert & Dohmatob] 2019) use an attack-driven label smoothing function u to further improve adversarial robustness. However, both uniform and attack-driven label smoothing disregard the inherent correlation between labels. To address the label correlation, we propose to use the Dirichlet distribution parametrized by B3 E IR for label smoothing. Our SPROUT learns to update ,3 to find training label distribution that most uncertain to current model weights 6, by solving
max L(x, (:
(6)
where أ = Dirichlet(11 a'y + a,3). Notably, instead of using a pre-defined or attack-driven function for in label smoothing, our Dirichlet label smoothing approach automatically finds a label simplex by optimizing B. Dirichlet distribution indeed takes label correlation into consideration as its generated label z [21...., .,.....2. 2x] has the statistical properties
32 Cov|zo, 2x] = 30'
-Ba3e
K
B[z.]
Σ
2 =
32 +
where Bn- = Σk= Br and s, E (1....,K] s * t. Moreover, one-hot label and uniform label smoothing are special cases of our Dirichlet label smoothing when B = y and ( u, respectively.
4

---

Under review as a conference paper at ICLR 2020
Our Dirichlet label smoothing co-trains with the update in model weights during training (see Algorithm I). The advantage of our proposed self progressnn Dirichlet label smoothing over uniform label smoothing will be justified in ablation study (see Figure S Section (46). illustrate the label correlation learned from our Dirichlet label smoothing in Appendix|A. A.2
(.6). In addition,
3.2 INCORPORATING GAUSSTAN DATA AUGMENTATION AND MIXUP
Gaussian augmentation. Adding Gaussian noise to data samples during training (i.e., Gaussian augmentation) a common pracitice to improve model robustness. Its corresponding vicinal function is the Gaussian vicinity function ( D: [z, y:) M(,, A²) = y:), where A is the variance of standard normal random vector. However, the gain of Gaussian augmentation marginal when compared with adversarial training (see our ablation study in Section[4.6] (Saatahi et al. (2018) finds that combining uniform or attack-driven label smoothing with Gaussian smoothing can further improve adversarial robustness. Therefore, we propose to incorporate Gaussian augmentaion with Dirichlet label smoothing. The joint vicinity function becomes
robustness is
v( Db |a,, ys: (3) = Nax,, ) Dirichlet((1 - o)y; + a;3))
(8)
Training with equation (8) means drawing labels from the parameterized Dirichlet distribution for the original data sample 2; and its neighborhood characterized by Gaussian augmentation.
Mixup. To further improve model generalization, SPROUT also integrates Mixup (Zhang et al. 2018 that performs convex combination on pairs of training data samples (in minibatch) and their labels during training. The vicinity function of Mixup is v/tt,, ja, y:) 8( = (1 >)a;; + Xæ,, 9 - -)y, + Xy;). where À ~ Beta(a, is the mixing parameter drawn from the Beta distribution and a >> 0 the shape parameter. The Mixup vicinity function can be generalized to multiple data sample pairs. Unlike Gaussian augmentation which is irrespective of the label (i.e., only adding noise to d, Mixup aims to augment data samples on the line segments of training data pairs and assign them convexly combined labels during training.
Vicinity function of SPROUT. With the aforementioned techniques integrated in SPROUT, the overall vicinity function of SPROUT can be summarized
v(ū y æ, Vo. 3)
(9)
= =0( = AV(x, A + (1 (), A2), Dirichlet((11 - a)((1))11(11111 A)y, + Xyy) + a:3)
In Section 4.6 we will show that Dirichlet label smoothing, Gaussian augmentation and Mixup complimentary to enhancing robustness by showing their diversity in input gradients.
3.3 SPROUT ALGORITHM
Using the VRM framework, the training objective of SPROUT is
min max 3
Liv(ai 2r, yı, 3);
(10)
where 0 denotes the model weights, n is the number of training data, L is the generalized cross entropy loss defined in (4) and v( az, yr (3) the vicinity function defined in (9. Our SPROUT algorithm co-trains 0 and / via stochastic gradient descent/ascent to solve the outer minimization problem on and the inner maximization problem on B.. In particular, for calculating the gradient ga of the parameter B3, we use the Pytorch implementation based on (Figurnov et al. 2018). SPROUT can either train model from scratch with randomly initialized or strengthen pre-trained model. As will be evident in Section 4.2] when evaluated against PGD- attack with different perturbation constraints, we find that training from either randomly initialized or pre-trained natural models using SPROUT can yield substantially robust models that are resilient to large values. The training steps of SPROUT are summarized Algorithm[I]
We also note that min-max training methodology is different from the min-max formulation in adversarial training (Madry et al. 2018), which is ming Σn= max L(x; + 5,, ys: f), where |6 denotes the Ep norm of the adversarial perturbation 5, While the outer minimization step for optimizing 9 can be identical, the inner maximization of adversarial training requires running multi-step PGD attack to find adversarial perturbations } for each data sample in every minibatch (iteration) and epoch, which is attack-specific and time-consuming (see our scalability analysis in
5

---

Under review as a conference paper at ICLR 2020
Algorithm 1 SPROUT algorithm
Training dataset (X, Y), Mixup parameter X, Gaussian augmentation variance A model learning rate Yo. Dirichlet label smoothing learning rate ?/8 and parameter a, cross entropy loss L Initial model 0: random initialization (train from scratch) or pre-trained model checkpoint Initial 3: random initialization for epoch=l...., .N do for minibatch XB C X, YB C Y do
Input:
XB c B., A?)
X Ymiz & Mixup(XB. YB,
Y min t Dirichlet(aY mis + a)
90 VoL(X mix: Yiiir
93 < ValiXmmia Yisz
0
Vege
B + 3 7/93
end for end for return 0
Table 5). On the other hand, our inner maximization is upon the Dirichlet parameter B, which is inneeddddeeedeetttt and only requires single-step stochastic gradient ascent with a minibatch to update B. We have explored multi-step stochastic gradient ascent on B and found no significant performance enhancement but increased computation time.
Advantages of SPROUT. Comparing to adversarial training, the training of SPROUT is more efficient and scalable, as it only requires one additional back propagation to update /3 in each iteration (see for run-time analysis). As highlighted in Figure I SPROUT is also more comprehensive as it automatically improves robustness multiple dimensions owing to its self-proggessing training methodology. Moreover, find that SPROUT significantly outperforms adversarial training and more effective as network width increases (see Figure 7 which makes SPROUT a promising approach support robust training for much larger set of network architectures.
PERFORMANCE EVALUATION
4.1 EXPERIMENT SETUP
Dataset and model structure. We use CIFAR-10 10 (Krizhevsky al.) and ImageNet (Deng al.)
2009) for performance evaluation. For CIFAR-10, we use both standard VGG-16 (Simonyan & Zisserman 2015) and Wide ResNet that is used both vanilla adversarial training (Madry et al. (0188) and TRADES (Zhang et al.) 2019). The Wide ResNet models are pre- pre-trained PGD- robust models given by adversarial training and TRADES. For VGG-16, we implement adversarial training with the standard syper-parameter and train TRADES with the official implementation. For ImageNet, we use ResNet-152. All our experiments were implemented Pytorch- and conducted using dual Intel E5-2640 v4 CPUs (2.40GHz) with 512 GB memory with GTX 1080 GPU.
Implementation details. As suggested Mixup (Zhang et al. 2018). we set the Beta distribution parameter 0.2 when sampling the mixing parameter X. For Gaussian augmentation, we set 0.1, which is within the suggested range (Zantedeschi et al., (017). Also, we set the label smoothing parameter
0.01. A parameter sensitivity analysis À and is given in Appendix A.4. Unless specified otherwise, for SPROUT we set the model initialization to be a natural model. An ablation study of model initialization given in Section 4.6
4.2 ADVERSARIAL ROBUSTNESS UNDER VARIOUS ATTACKS
White-box attacks. On 10, we compare the model accuracy under different strength of white-box &oc- norm bounded non-targen non-targeted PGD attack, which is considered as the strongest first- order adversary (Madry et al. 2018) with lec -orm constraint (normalized between 0 to 1). All PGD attacks are implemented with random starts and we run PGD attack with 20 and 100 steps in our experiments. The (robust) accuracy under different values shown in Figure 2 When 0.03

---

Under review as a conference paper at ICLR 2020
and under PGD attack with 20 steps, find SPROUT achieves 62.24% and 66.23% robust accuracy on VGG16 and Wide ResNet respectively, while TRADES and adversarial training are 10-20% worse than SPROUT. We also find that SPROUT is significantly more robust to PGD-e> attacks with large e values. In addition to the substantially improved robustness, the clean accuracy (i.e., when e = 0) of SPROUT is 5-10 % higher than TRADES and adversarial training, and it is only 2-4% lower than natural model, suggesting SPROUT better balances the robustness-accuaacy trade-off. Similar trends are observed in robust accuracy under PGD attack with 100 steps. On Wide ResNet we also report the robust accuracy of the "free adversarial training" (Free Adv train) method (Shafahi et al. 2019). which features similar robust accuracy as adversarial training but greatly reduced training time.
VGG PGD 20
VGG PGD 100
Accuracy under attakk
Accuracy under attack
80
60-
60
40
SPROUT
SPROUT
TRADES
TRADES
20
Natural
20
Adv train
Adv train
0.04
0.06
0.00
0.02
PGD attack stregth
PGD attack stregth
Wide ResNet
Wide ResNet PGD 100
100
Accuracy under attack
Accuracy under attack
80
60
60-
SPROUT
SPROUT
TRADES
TRADES
Natural
Natural
Adv train
20
Adv train
Free Adv train
Free Adv train
0.00
0.02
0.04
0.06
0.00
0.02
0.04
0.06
PCD attack stregth
attack stregth
Figure 2: Robust accuracy under PGD-E>o attack. SPROUT signigicantly outperforms other methods.
In addition to PGD- attack, we also compare against fg-norm based C& W attack (Carlini & Wagner 2017). We use the default attack setting do 10 binary search steps with 1000 iterations per step to find successful attacks while minimizing l2 -nomm perturbation. Figure 3 showw that the gain in & robustness using SPROUT is less apparent than that fox robustness. SPROUT's performance is similar to TRADES but is better than both natural and adversarial training. The results also suggest that the antee-iindeeedeet and self-progressing training nature of SPROUT can prevent the drawback of failing to provide comprehensive robustness to multiple and simultaneous fy-norm adversarial
attacks in adversarial training (Tamer & Boneh 2019
Kang et al. 2019).
VGG
Wide Resnet
100
Accuracy under attack
Accuracy under attack
80
80
60-
60
40
SPROUT
40
SPROUT TRADES
TRADES
Natural
20
Natural
20
Adv train
Adv train
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.00
0.05
0.10
0.15
0.20
0.25
0.30
attack stregth
C&W attack stregth
Figure 3: Robust accuracy under C&W-l, attack
Transfer attack. We follow the criterion of evaluating transfer attacks as suggested by Athalye et al. (018 inspect whether the models trained by SPROUT will cause the issue of obfuscated gradients
7

---

Under review as a conference paper at ICLR 2020
and give false sense of model robustness. We generate 10,000 adversarial examples of CIFAR-10 from natural models with = 0.03 and evaluate their attack performance on the target model. Table 2 shows SPROUT achieves the best accuracy when compared with adversarial training and TRADES, suggesting the effectiveness SPROUT in defending both white-box and transfer attacks.
Table 2: Robust accuracy under transfer attack on CIFAR-10
Method
VGG 16
Wide ResNet
Adv train 79.13%
85.84%
TRADES 83.53%
83.9%
SPROUT 86.28%
89.1%
ImageNet results. As many ImageNet class labels carry similar semantic meanings, to generate meaningful adversarial examples for robustness evaluation, here we follow the same setup as in (Athalye al. 2018) that adopts PGD-L attacks with randomly targeted labels. Table compares the robust accuracy natural and SPROUT models. SPROUT greatly improves the robust accuracy across different values. For example, when 0.01, SPROUT boosts the robust accuracy of natural model by over 43%. When c 0.015 R 4/255, a considerably large adversarial perturbation on ImageNet, SPROUT still attains about 35% robust accuracy while the natural model merely has about 2% robust accuracy. Moreover, comparing the clean accuracy, SPROUT is about 4% worse than the natural model but is substantially more robust. We omit the comparison to adversarial training methods as we are unaware of any public pre-trained robust ImageNet models prior to the time of our submission, and is computationally demanding for us train and fine-tune such large-scale networks with adversarial training. On our machine, training a natural model takes 31, 11,15877 seconds and training SPROUT takes 59,201.6 seconds. Comparing to the run- run-time analysis in Section 4.5 SPROUT has much better scalability than adversarial training and TRADES.
Table 3: Robust accuracy under PGD-6oc attack on ImageNet and ResNet-
Method
Clean Acc
<= 0.005
c= 0.01
<= 0.015
=
0.02
Natural 78.31%
37.13%
9.14%
2.12%
0.78%
SPROUT
74.23%
65.24%
52.86%
35.04%
12.18%
4.3 LOSS LANDSCAPE EXPLORATION
To further verify the superior robustness using SPROUT, we visualize the loss landscape different training methods in Figure Following the implementation in (Engstrom et al. (018)) we vary the data input along a linear space defined by the sign of the input gradient and a random Rademacher vector, where the and axes represent the magnitude of the perturbation added in each direction and the z-axis represents the loss. One can observe that the loss surface SPROUT is smoother. Furthermore, it attains smaller loss variation compared with other robust training methods. The results provide strong evidence for finding more robust models via SPROUT.
-630
-132a in
-003
(a) Natural
(b) Adv
(c) TRADES
(d) SPROUT
Figure& 4: Loss landscape comparison of different training methods
4.4 INVARIANCE TEST
In addition to l, norm bounded adversarial attacks, here we also evaluate model robustness against different kinds of input transformations using CIFAR-10 and Wide ResNet. Specifically, change
8

---

Under review as a conference paper at ICLR 2020
rotation (with 10 degree), brightness (increase the brightness factor to 1.5), contrast (increase the contrast factor to 2) and make inputs into grayscale (average all RGB pixel values). The model accuracy under these invariance tests is summarized in Table A Results show that SPROUT outperforms adversarial training and TRADES. Interestingly, natural model attains the best accuracy despite the fact that it lacks adversarial robustness, suggesting potential trade-off between accuracy in these invariance tests and Ep -n-morm based ad verssarial robustness.
Table 4: Accuracy under invariance tests
Table 5: Training-time (seconds) for 10 epochs
Method
Rotation
Brightness
Contrast
Gray
CIFAR-10
Natural
88.21 %
93.4%
91.88 %
91.95%
Methods
VGG 16
Wide ResNet
Adv train
82.66%
83.64%
84.99%
81.08%
Natural
146.7
1449.6
TRADES
80.81% %
81.5 %
83.08%
79.27%
Adv train
1327.1
14246.1
SPROUT
85.95%
88.26 %
86.98%
81.64%
TRADES
1932.5
22438.4 4
SPROUT
271.7
2727.8
Free Adv train(m=8)
2053.1
20652.5
4.5 SCALABILITY
As illustrated in Section 3.3
SPROUT enjoys great scalability over adversarial training based
algorithms because its training requires much less number of back-propagattons per iteration, which dominating factor that contributes to considerable run-time in adversarial training. Table benchmarks the run-time of different training methods for 10 epochs. On CIFAR-10 10, the run-time of adverarial training and TRADES is about 5x more than SPROUT. We also report the run-time analysis using the default implementation from the recent work (Free Adv train) in (Shafahi et al. 2019). Its 10-epoch run- uuntimme with the replay parameter m = 8 is similar to TRADES. However, we note that Free Ady train may require less number of epochs when training to covergence.
4.6 ABLATION STUDY
Dissecting SPROUT. Here we perform an ablation study using VGG-16 and CIFAR- to investigate and factorize the robustness gain in SPROUT's three modules: Dirichlet label smoothing (Dirichlet), Gaussian augmentation (GA) and Mixup. We implement all combinations of these techniques and
include uniform label smoothing (LS) (Szegedy et al., 2016) as another baseline. Their accuracies under PGD-6x attack shown in Figure We highlight some important findings as follows.
Dirichlet outperforms uniform LS by a significant factor, suggesting the importance of our proposed peof-progrsssing label smoothing in improving adversarial robustness.
Comparing the performance of individual modules alone (GA, Mixup and Dirichlet), our proposed Dirichlet attains the best robust accuracy, suggesting its crucial role in training robust models.
No other combinations can outperform SPROUT. Moreover, the robust gains from GA, Mixup and Dirichlet appear to be complimentary, as SPROUT's accuracy is close to the sum of their individual accuracy. To justify their diversity in robustness, we compute the cosine similarity of their pairwise input gradients and find that they are indeed quite diverse and thus can promote robustness when used together. The details are given in Appendix A.3
VGG PGD 20
VGG PGD 100
Accuracy under ettack
SPROUT
80-
80-
GA
60
Accurncy under
60-
Mixup
Dirichlet
40-
20-
GA Mixup
Mixup + DDrtchlet
20
20
GA Dirichlet
Uniform LSS
GA Mixup-1..
0.02
0.04
0.06
PGD allack stregth e
PGD allack stregth e
Figure 5: Robust accuracy with different combinations of the modules in SPROUT
PGD attacks with more iterations. To ensure the robustness of SPROUT is not an artifact of running insufficient iterations in PGD attack (Athalye al., 2018), in Figure 6a we show the robust accuracy with the number of PGD- attack steps varying from 10 to 500 on VGG-16 and CIFAR-10.

---

Under review as a conference paper at ICLR 2020
The results show stable performance in SPROUT, TRADES and adversarial training once the number of steps exceeds 100. It is clear that SPROUT indeed outperforms others by large margin.
Model weight initialization. Figure 6b compares the effect of model initialization using CIFAR- 10 and VGG-16 under PGD-& attack, where the legend A + B means using Model A as the initialization and training with Method B. Interestingly, Natural+ attains the best robust accuracy when c 2 0.02. TRADES+SPROUT and Random+SPROUT also exhibit strong robustness since their training objective involves the loss on both clean and adversarial examples. In contrast, Adv train+SPROUT does not have such benefit since adversarial training only aims to mínimize adversarial loss. This finding is also unique to SPROUT, can boost robust accuracy. Our results provide novel perspectives and also indicate that SPROUT is indeed new robust training method that vastly differs from adversarial training based methods.
neither Natural+Ady train nor Tatural+TRADES
Wide Resnet e 0.03
VGG PGD 20
65
SPROUT
Accuracy under attack
80
TRADES
Natural+5PROUT TRADES Sandom-SPROUr Adv SPROUT
60
Adv train
60
55
Accural
Natural+Adv train Natural+TRMDES
50
20
45
100
500
0.00
0.02
0.04
0.06
Num of PGD steps
attack stregth
(a) Robust accuracy with different PGD steps
(b) Robust accuracy with different model initialization
Figure 6: Stability in PGD- attack and the effect of model initialization.
Effect network width. It was shown in Madry et al. 2018) that adversarial training (Adv train) will take effect when network has sufficient capacity, which can be achieved by increasing network width. Figure 7 compares the robust accuracy of SPROUT and Adv train with varying network width on Wide ResNet and CIFAR-10. When the network has width = 1 (i.e. a standard ResNet- 152 network (He et al. 2016)), the robust accuracy of SPROUT and Adv train are both relatively low (less than 47%). However, as the width increases, SPROUT soon attains significantly better robust accuracy than Adv train by a large margin (roughly 15%). Since SPROUT more effective in boosting robust accuracy as network width varies, the results also suggest that SPROUT can better support robust training for a broader range of network structures.
ResNet PGD-20 attack with 0.003
Accurary under attack
600
33
50-
45-
40
Adv train
2
10
ResNet width
Figure 7: Effect of network width against PGD-6oo attack on CIFAR-10 and ResNet.
5 CONCLUSION
This paper introduced SPROUT, a profrssogg robust training method motivated by vicinity risk minimization. When compared with state-of-the-art adversarial training based methods, our extensive experiments showed that the proposed self- eeff-progressing Dirichlet label smoothing technique in SPROUT plays a crucial role in finding substantially more robust models against -nom bounded PGD attacks and simultaneously makes the corresponding model more generalizable to various invariance tests. We also find that SPROUT can strengthen wider range of network structures as less sensitive to network width changes. Moreover, SPOURT's sdlf-adjusted learning methodology not only makes its training free of attack generation but also becomes scalable solutions to large networks. Our results shed new insights devising comprehensive and robust training methods.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of secu- rity: Circumventing defenses adversarial examples. International Coference on International Conference on Machine Learning. 2018.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šındić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402, 2013.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In /EEE Symposium on Security and Privacy, pp. 39-57, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John Duchi. Unlabeled data improves adversarial robustness. Neural Information Processing Systems, 2019.
Olivier Chapelle, Jason Weston, Léon Bottou, and Vladimir Vapnik. Vicinal risk minimization. In Advances in neural information processing systems, pp. 416-422, 2001.
Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv: 10272, 2018.
Mikhail Figurnov, Shakir Mohamed, and Andriy Mnih. Implicit reparameterization gradients. In Advances in Neural Information Processing Systems, pp. 441-452, 2018.
Morgane Goibert and Elvis Dohmatob. Adversarial robustness via adversarial label-smoothing. arXiv preprint arXiv: 190ivv:9006, /567, 2019.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2015.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. Neural Information Processing Systems, 2019.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv:1908.0800 2019.
Sanjay Kariyappa and Moinuddin Qureshi. Improving adversarial robustness of ensembles with diversity training. arXiv preprint arXiv: /0X1v9/901.09981, 2019.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). URL httl::ttp:///tt www.:wwww...oot...ooooooo
cs wow.nto..oooooooo.o wdw/ war.a.aa.r....a.////// html
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
Ali Shafahi. Amin Ghiasi, Furong Huang, and Tom Goldstein. Label smoothing and logit squeezing: A replacement for adversarial training? 2018.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Neural Information Processing Systems, 2019.
11

---

Under review as a conference paper at ICLR 2020
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, 2015.
Robert Stanforth, Alhussein Fawzi, Pushmeet Kohli, et al. Are labels required for improving adversarial robustness? Neural Information Processing Systems, 2019.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-228888 1--8866, 2016.
Sunil Thulasidasan, Gopinath Chennupati, Jeff Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. Neural Information Processing Systems, 2019.
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. Neural Information Processing Systems, 2019.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, loannis Mitliagkas, Aaron Courville, David Lopez-Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. International Conference on Machine Learning, 2018.
Jianyu Wang. Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. International Conference on Computer Vision, 2019.
Valentina Zantedeschi, Maria-Irina Nicolae, and Ambrish Rawat. Efficient defenses against adversar- ial attacks. In ACM Workshop Artificial Intelligence and Security, pp. 39-49, 2017.
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature caatering-based adversarial training. Neural Information Processing Systems, 2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. International Conference on Learning Representations, 2018.
Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. Improving the robustness of deep neural networks via stability training. In Proceedings of the ieee conference on computer vision and patlern recognition, pp. 4480-4488, 2016
12

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1 EXACT PERFORMANCE METRICS FOR FIGURE]
Table 6: Performance comparison between different training methods VGG- and CIFAR-10
Method
Clean Acc
(s Acc (e = 0.03)
C&W Acc
Invariance (Contrast)
Scalibility (10 epochs)
Natural
95,93%
0%
26.95%
91.88%
146.7 (secs)
Adv train
84.92%
36.29%
70.13%
84.99%
1327.1 (secs)
TRADES 88.6%
38.29%
75.08%
83.08%
1932.5 (secs)
SPROUT 90.56%
58.93%
72.7%
86.98%
271.7 (sees)
A.2 LEARNED LABEL CORRELATION FROM SPROUT
Based the statistical properties of Dirichlet distribution in 7 use the final /3 parameter learned from Algorithm D with CIFAR- 10 and VGG-16 to display the matrix of its pair-wise product 3, -r. in Figure The value in each entry is proportional the absolute value of the label covariance in 7 We observe some clustering effect of class labels in CIFAR-I0, 10, such as relatively high values among the group of (airplane, auto, ship. truck] and relatively low values among the group of {bird. cat, deer, dog}. Moreover, since the /3 parameter is progressively adjusted and co-trained during model training, and the final (3 parameter is clearly not uniformly distributed, the results also validate the importance of using parametrized label smoothing to learn to improve robustness.
Correlogram
atrplane 1e+ 021 021e +021++02
98
99
1e 0221 02
100.8
020
1e++02
1e+
10 0210021112211100 02 021e+-02
99
1 02
99
1e 02
truck
1e+ 0210 021:02211+111 +0
99
99
1e 02
99
1e + 10002++0000 02
bidd
99
98
99
-9.22
99
97
98
-9.44
99
1e+021e +021e+02
98
99
99
99
dog
99
99
99
99
98
97
98
98
99
99
-97.6
frog 1e+ 11++0221e002ee 0211:+121++002220000
98
99
1e + 1e+021e+00
1e+021e+02
airlane
Figure 8: Matrix plot of the product B, Be of the learned / parameter on CIFAR-10 and VGG-16.
13

---

Under review as a conference paper at ICLR 2020
A.3 DIVERSITY ANALYSIS
In order to show the three modules (Dirichlet LS, GA and Mixup) in SPROUT lead robustness gains that are complimentary to each other, we perform diversity analysis motivated by (Kariyappal Qureshi 2019) to measure the similarity of their pair-wise input gradients and report the average cosine similarity in Table o 10,000 data samples using CIFAR-I0 and VGG-16. 16. We find that the pair-wise similarity between modules is indeed quite small (< 0.103). The Mixup-GAt GA similarity is the smallest among all pairs since the former performs both label and data augmentation based on convex combinations of training data, whereas the latter only considers random data augmentation. The DS -A similarity is the second smallest (and it is close to the Mixup GA similarity) since the former progressively adjusts the training label y while the latter only randomly adjusts the training sample æ. The Dirichlet LS-Mixup similarity relatively high because Mixup depends on the training samples and their labels while Dirichlet LS also depend on them and the model weights. The results show that their input gradients are diverse they point to vastly different directions. Therefore, SPROUT enjoys complimentary robustness gain and can promote robustness when combining these techniques together.
Table 7: Average pair-wise cosine similarity of the three modules in SPROUT
Dirichilet LS
Mixup
GA
Dirichilet LS
NA
0.1023
0.0163
Mixup GA
0.1023 0.0163
NA
0.0111
0.0111
NA
A.4 PARAMETER SENSITIVITY ANALYSIS
We perform an sensitivity analysis of the mixing parameter Beta(a,a) and the smoothing parameter of SPROUT in Figure 9 When fixing a, we find that setting too large may affect robust accuracy, as the resulting training label distribution could be too uncertain to train a robust model. Similarly, when fixing a, setting too large may also affect robust accuracy.
VGG PGD 20
90
Accuracy under attack
À Beta 0.2), 0.01
80
À Beta(0.55,.5)) a 0.01
À Beta(1.0, 1.0), a 0.01
70
À Beta 0.2), a = 0.1
60
50
40
0.00
0.02
0.04
0.06
PGD attack stregth
Figure 9: Sensitivity of hyperparameters À and in SPROUT under PGD-Lx attack
A.5 PERFORMANCE ON CW-L ATTACK
To further test the robustness on &oo constraint, we replace the cross entropy loss with CW-fas loss
(Carlimi & Wagner 2017) in PGD attack. Similar the PGD-Lxc attack results in Figure 2 Figure I0
shows that although SPROUT has slightly worse accuracy under small e values, it attains much higher robust accuracy when 2 0.03.
14

---

Under review as a conference paper at ICLR 2020
Wide ResNet CW 20
100
Accuracy under attack
SPROUT TRADES
80
Natural
60
Adv train
40
20
0.00
0.02
0.04 PGD attack stregth E
0.06
Figure 10: Robust accuracy under CW- attack.
A.6 PERFORMANCE WITH DIFFERENT NUMBER RANDOM STARTS FOR PGD ATTACK
As suggested by Madry al. 2018), PGD attack with multiple random starts is stronger attack method evaluate robustness. Therefore, Table X we conduct the following experiment on CIFAR-10 and wide ResNet to show that the model trained by SPROUT can still attain at least 61 % accuracy against PGD-C attack (E 0.03) with the number of random starts varying from I to 10 and with 20 attack iterations. The robust accuracy of SPROUT is still clearly higher than other methods as shown in Figure 2
We also perform two additional attack settings: (i) 100-step PGD-L attack with 10 random restarts using the CW loss and 0.03; (ii) 100-step PGD-Lxc attack with 10 random restarts using the cross entropy and = 0.03. We find that SPROUT can still achieve 51.23% robust accuracy in setting (i) and 61.18% robust accuracy in setting (ii).
Table Robust accuracy of SPROUT PGD-L attack with 0.03 using different number of random starts.
# random start
3
5
8
10
Robust accuracy
64.58%
62.53%
61.98%
61. 61.38%
61.00%
A.7 PERFORMANCE COMPARISON WITH FREE ADVERSARIAL TRAINING ON RESNET-50 AND IMAGENET
Here we compare the performance of SPROUT with pre- rre-trained robust ResNet-50 model on ImageNet, which is shared by the authors in (Shafahi al. 2019) proposing the free adversarial training method (Free Adv train). We find that SPROUT obtains similar robust accuracy as Free Adv train when 0.01. As c becomes larger, Free Adv train has better robust accuracy.
Table 9: Robust accuracy under PGD-C random targeted attack on ImageNet and ResNet-50
Method
Clean Acc
0.005
0.01
e 0.015
0.02
Natural
76. 15%
24.37%
3.54%
0.90%
0.40%
Free Ady train
60.49%
51.35%
42.29%
32.96%
24.45%
SPROUT
61.23%
51.69%
38.14%
25.98%
18.52%
15


Reviews:
[{"review_id": "ryxPtG13KB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors proposed a hybrid method for defending against adversarial attacks called SPROUT. The proposed defense method consists of three main ingredients:\n\n1. label smoothing with a learnable Dirichlet distribution\n2. adding Gaussian noise to input examples\n3. mixup: augment training examples with their linear combinations \n\nThe authors' main argument for their method is the speed over adversarial training and its effectiveness. \n\nIndividually, none of these ingredients are known to be strong defense against adversarial examples in the literature. Indeed this is corroborated by Figure 5, when the individual defenses do not have more than 10% accuracy under PGD100 attacks for epsilon=0.4. However, when all three are used together the accuracy jumps to close to 60%. This is very surprising. Another surprising fact is that in Figure 2, the method beats the benchmark adversarial PGD by more than 20% on white-box attacks, given the difficulty of beating adversarial PGD.  \n\nGiven the surprise in these experimental results, I believe the authors should perform a more detailed analysis on how these ingredients for their SPROUT defense interact to produce such a strong predictor, in addition to doing ablation studies. An attempt should be made to explain why they work so well together when they are quite weak individually as defenses. It is difficult for me to recommend acceptance of this paper without an attempt to explain why it works. \n\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "rkgYpift9B", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes training robust models without explicitly training on adversarial examples and by \"smoothing\" the labels in an adversarial fashion and by using Dirichlet label smoothing. Training robust models without adversarial training is indeed an important problem as mentioned by the authors since it can potentially (as the authors demonstrate) result in faster model training and less drop in clean accuracy. Overall the idea is interesting but I have some concerns mainly about evaluations and baselines which I am including below. If the authors can address my concerns, I am willing to increase my score:\n\n1. Based on equations (9) and (10), if we set \\alpha to be large, then the network is not trainable (since the worst-case adversary will increase the loss on the image by flipping the label during training). As a consequence, we can see that the value of hyper-parameters that the authors use is indeed very small (0.01 and 0.1). Even between these small values, the smaller value results in a better model. In the extreme case, where \\alpha is zero there is no regularization and \\beta becomes irrelevant. This illustrates that the performance of the model is very sensitive to \\alpha. On the other hand, we can prevent the model from not learning anything by constraining \\beta in equation (10) — similar to adversarial training where we constrain \\delta. It seems that without constraining \\beta, if the step-size for \\beta is large, \\beta can grow and completely mess up the labels even when \\alpha is tiny. If we constrain \\beta on the other hand, we can make sure that in no case, the top label for any augmented image is an incorrect label. Can the authors elaborate on why they did not set any limit on the value of \\beta?\n2. What is the batch-size used for ImageNet? The reason that I am asking is that you compute the gradient of \\beta for the previous mini-batch but use it for the next mini-batch. Is it possible that the previous mini-batch's \\beta is not accurate for the current mini-batch? For CIFAR, since the number of classes is 10, I would assume that you can update the statistics for the class (\\betas) using the previous mini-batch since you always see examples from all the classes using any reasonable batch-size. What happens if you do the same but for a dataset with more classes but have the mini-batch be smaller than the number of classes. In this case, your \\beta is getting updated only using information from a few classes and not all classes at once. In that case, what happens if you just use a random \\beta every step? \n3. For the white-box attacks, I also have a few questions. Do you use multiple random restarts? It is known that random restarts can be more effective than increasing the number of PGD attacks. See for example the leaderboards for MNIST and CIFAR-10 challenges by Madry. I would like to see a table where you plot how the accuracy changes by doing 100 step PGD attacks and by increasing the number of random restarts from 1 to 10 for example.\n4. Do you do L-infinity CW attacks? I see that you have done L-2 CW attacks but I can't find any L-infinity CW attacks. It would be great to show numbers for that and also compare it with TRADES and PGD adversarial training. In previous smoothing methods, the L-infinity CW attack seems to be a stronger attack compared to PGD.\n5. For the ImageNet task, the authors state that the evaluation of non-targeted attacks can result in label leaking. Label-leaking happens when one trains on adversarial examples built using a single-step attack and it means that the accuracy of adversarial examples is higher than natural examples at test-time. For this, I do not understand why the authors mention that they only evaluate targeted attacks while they are not doing any adversarial training.\n6. Also, for ImageNet, there are recent methods such as Adversarial training for Free! where the authors do adversarial training on ImageNet with no overhead cost compared to natural training. Maybe this could be added as a better base-line than a naturally trained model.\n7. In Figure 4. (a), why is the loss for the validation image illustrated so high? What image is this from the validation set?\n8. In terms of Scalability, its good to mention new scalable methods such as YOPO and Adversarial Training for Free.\n9. In the ablation study, including Dirichlet Smoothing indeed results in a huge boost compared to having no smoothing. However, it would be better to show that Dirichlet smoothing is indeed better than label-smoothing or adversarial smoothing by including results for other smoothing methods in Fig. 5.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HkeBBIr3KS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel training method to build robust models. A new framework SPROUT is introduced to adjust label distribution during training. It also integrates mixup and Gaussian augmentation to further improve the robustness. The proposed method is built upon the Vicinity Risk Minimization (VRM) framework. Experiments show that the proposed method significantly outperforms the existing best methods in terms of robustness against attacks.\n\nOverall, this paper proposes a novel method with good robustness performance. The proposed approach is built upon the VRM framework, and summarizes a lot of existing methods under this framework (Table 1). Experimental results are also very strong to prove the effectiveness of the proposed method. \n\nOn the other hand, I have some concerns about this paper. Since the performance improvement is significantly large over the current best methods, I need to see those concerns addressed to give a final rating.\n\n1. How do you perform inference given testing data? Do you use Gaussian augmentation or mixup during inference?\n\n2. Do you check that whether the robustness comes from obfuscated gradients? It's very important to examine the true robustness of the propose method.\n\n3. What's the final distribution of \\beta? Does it have a semantic meaning?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new training technique to produce a learned model robust against adversarial attacks -- without explicitly training on example attacked images. The core idea being that such a training scheme has the potential to reduce the cost in terms of training time for obtaining robustness, while also potentially increasing the clean performance. The method does so by proposing a version of label smoothing and doing two forms of data augmentations (gaussian noise and mixup). 

The reviewers were mixed on this work. Two recommended weak reject while one recommended weak accept. All agreed that this work addressed an important problem and that the proposed solution was interesting. The authors and reviewers actively engaged in a discussion, in some cases with multiple back and forths. The main concern of the reviewers is the inconclusive experimental evidence. Though the authors did demonstrate strong performance on PGD attacks, the reviewers had concerns about some attack settings like epsilon and how that may unfairly disadvantage the baselines. In addition, the results on CW presented a different story than the results with PGD. 

Therefore, we do not recommend this work for acceptance in its current form. The work offers strong preliminary evidence of a potential solution to provide robustness without direct adversarial training, but more analysis and explanation of when each component of their proposed solution should increase robustness is needed. 

Decision: Reject