Adversarial Robustness without Adversarial Training: A Teacher-Guided Curriculum Learning Approach
Authors: Anindya Sarkar
Anirban Sarkar
Sowrya Gali
Vineeth N. Balasubramanian
Venue: NeurIPS 2021 Poster (2021)
Keywords: Adversarial Robustness
Inner Maximization
Outer Minimization
Attribution Map
Curriculum Learning

Abstract: Current SOTA adversarially robust models are mostly based on adversarial training (AT) and differ only by some regularizers either at inner maximization or outer minimization steps. Being repetitive in nature during the inner maximization step, they take a huge time to train. We propose a non-iterative method that enforces the following ideas during training. Attribution maps are more aligned to the actual object in the image for adversarially robust models compared to naturally trained models. Also, the allowed set of pixels to perturb an image (that changes model decision) should be restricted to the object pixels only, which reduces the attack strength by limiting the attack space. Our method achieves significant performance gains with a little extra effort (10-20%) over existing AT models and outperforms all other methods in terms of adversarial as well as natural accuracy. We have performed extensive experimentation with CIFAR-10, CIFAR-100, and TinyImageNet datasets and reported results against many popular strong adversarial attacks to prove the effectiveness of our method.

Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach
Anindya Sarkar
Anirban Sarkar
Indian Institute of Technology, Hyderabad an anindyasarkar www.eeeaeeeeeee...e.e.m..e.eeeee com
Indian Institute of Technology, Hyderabad cs cs16recch100011thhoooo ac. in
Sowrya Gali
Vineeth N Balasubramanian Indian Institute of Technology. Hyderabad vineethnbSiith.ac.
Indian Institute of Technology, Hyderabad cc18btech110120itth.co
Abstract
Current SOTA adversarially robust models are mostly based on adversarial training (AT) and differ only by some regularizers either at inner maximization or outer minimization steps. Being repetitive nature during the inner maximization step, they take huge time to train. We propose a non-iterative method that enforces the following ideas during training. Attribution maps are more aligned the actual object in the image for adversarially robust models compared to naturally trained models. Also, the allowed set of pixels perturb an image (that changes model decision) should be restricted to the object pixels only, which reduces the attack strength by limiting the attack space. Our method achieves significant performance gains with little extra effort (10-20%) over existing AT models and outperforms all other methods in terms of adversarial as well as natural accuracy. We have performed extensive experimentation with CIFAR-10. 10, CIFAR-IO0 100, and TinyImageNet datasets and reported results against many popular strong adversarial attacks to prove the effectiveness of our method.
1 Introduction
Ever since deep neural network (DNN) models have emerged as the de facto technique to be applied for many vision problems, adversarial robustness has emerged as a critical need. Goodfellow et al. (T] identified this serious issue show very different predictive behavior of DNN model with similar looking images. Many efforts have followed since either to come up with fooling techniques
4
5 6 defend deep models against them 3 7
16 18 19]
20
25 26). Nevertheless, both these research directions are important at this moment and require more attention. In this work, our effort lies on devising an adversarially robust training technique.
21
22
23
24.
Adversarial training (AT) is the most widely used method for adversarial robustness and most of the improvements have since come by adding regularizers without changing the min-max formulation. The regularizers added either in the inner maximization [27] g [5] outer minimization [19 [4] term. Though AT-based methods are shown to perform well, they incur additional cost due to an iterative inner maximization step. Our proposed robust training method circumvents this fundamental issue.
With objective of not following the costly min-mat optimization of AT, revisit adversarial robustness in terms of standard model training. Adversarially robust models are shown to satisfy
cqual contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

better alignment between saliency and object features in 28) We enforce such alignment through training to achieve model robustness. This is accomplished by forcing saliency of the main model to follow the object features, provided by saliency of a pre-trained reference model. We hypothesize that perturbing only the most discriminative part of an image should trigger a model to change its decision about that image. In other words, perturbing other non-important pixels shouldn' 't affect model decision, and such a model can achieve adversarial robustness. This approach is achieved by progressively narrowing the object-discriminative region according to its importance in a curriculum learning sense and forcing an adversary to perturb only those pixels while changing model's decision. A model trained with this approach restricts the perturbation only to the object pixels when attacked. This diminishes the range of possible perturbations and limits the attack strength, reducing the chance of the model to change its decision for the perturbed image. Being
non-iterative method, our approach not only takes considerably less time for training compared to AT, but also outperforms )
iterative methods (such as [3] 7 27 16 13 14 12 17 15]
and non-iterative methods (such as [29 10 9 18) significantly in both natural and adversarial accuracies.
Our key contributions are summarized as below.
We propose non-iterative novel robust training method. which outperforms recently proposed SOTA techniques, irrespective of their type being iterative or non-iterative, in terms of both natural and adversarial accuracies against wide range of attacks.
Being non-iterative method makes it easily applicable to any large dataset to achieve adver- sarially robust model. Our method takes 10-20% of time compared to any adversarial training technique.
Our method attains clean accuracy which is much closer to the performance of naturally trained models compared to other robust models.
We perform extensive experimentation on CIFAR-10, CIFAR-100, TinyImageNet datasets and report comparative results against all other, including recently proposed, adversarial robustness techniques. We also present various studies in detail to analyze the effectiveness of our method.
Background
Adversarial robustness: Naturally trained deep models are shown to be fooled easily by image perturbations which are human-impenn auman-imperceptible [1. Extra measures need be taken to make them adversarially robust when they stick to their decision even image is perturbed. Consider an image classifier F(r; 0)= x - R with parameters 0, that maps input image æ to a c-dimensional output. The network is called adversarially robust if:
argmaxfi(a;) 0) argmaxf:(x + 5: iEe
IEe
where 5 E B(c) i.e. 8: ||81|p S c.
Adversarial training (AT): AT 3 was proposed for achieving robustness against adversarial exam- ples, which is represented as the below loss function:
L(z, = ((r,)) pl max L(x + 8, 8,y)
(2)
BE
5 is found by projected gradient descent (PGD) which is given by the iterative gradient step as below:
8 ベ projlō - (Vosigg(sLL(e + 6,
where (x) = argmin|| | - ell. Effectively, AT is comprised of an inner maximization which BE)
generates a perturbed image within an e ball and the outer minimization tunes the model parameters based on the perturbed images.
3 Related Work
Robust model training. Many efforts on making DNN models adversarially robust have been proposed in the last few years and shown to perform well against adversarial attacks. (27] proposed TRADES, which perturbs an image by maximizing the L2 distance of the logits and used that as a regularizer to smoothen the decision boundary. Misclassification Aware adveRsarial Training (MART) [1], on the other hand, treats the misclassified examples differently and adds a regularizer term to weight them by the misclassification score for better robustness. Recently, [ found strong positive correlation between robust generalization gap and flatness of weight loss landscape. This motivated an Adversarial Weight Perturbation (AWP) mechanism which adversarially perturbs

---

both inputs and weights in the adversarial training framework to regularize the flatness of weight loss landscape. Zhang et proposed Geometry aware Instance Reweighted adversarial training (GAIRAT) [6] method, which assigns larger weights on the losses of adversarial data, whose natural counterparts are closer to the decision boundary. Contrastively, GAIRAT gives smaller weights on the losses of adversarial data, whose natural counterparts are further away from the decision boundary. adversarial training. A few recent works have modified the adversar-
Curriculum learning-basee ial training strategy incorporating curriculum learning strategy. Curriculum Adversarial Training (CAT) [TI proposed curriculum-based adversarial training strategy by progressively increasing the number of PGD steps during training. [12] proposed dynamic training strategy to gradually increase the convergence quality of the generated adversarial examples, and also a method to quan- titatively evaluate the convergence quality of adversarial examples motivated by the -oole optimality gap. [17) tried to employ the fact that, during initial phase of training, fitting with most difficult adversarial data makes the learning extremely hard for DNNs. They hence proposed Friendly Adversarial Training (FAT), adhering to the spirit of curriculum learning, which learns initially from the least adversarial and progressively utilizes increasingly more adversarial data. Another recent method Adversarial Training with Early Stopping (ATES) [I5] proposed curriculum loss as the inner maximization step which depends on difficulty parameter that are gradually increased as the training progresses. Our method is very different from these methods on how the curriculum learning strategy employed.
Non-iterative adversarial robustness training. There very few efforts in devising robust model through a non-iterative approach that differs from AT 3 30 showed that models trained using single-step adversarial training methods are susceptible to multi-step white box attacks, such as PGD [3]. Recently, (22) proposed single-step adversarial training method using Dropout Scheduling, which improves adversarial robustness against multi-step white box attacks and achieves par results compared to B. But all these single-step adversarial robustness methods still lag far behind current state-of-the fi-thee-rt art (SOTA) methods that use the Adversarial Training framework. In terms of methods that use saliency, Jacobian Adversarially Regularized Network (JARN) [IO] improves model robustness by matching the gradient of loss w.r.t. the attacked image to the actual image. A very similar method was proposed by [9] which employs a discriminator to compare between the Jacobian and the image saliency, while JARN [0] compares the image to the transformed version of the Jacobian through an adaptive network. Our method differs from these as we enforce localization of saliency (w.r.t true class score) and restrict the perturbation set to pixels of the most discriminative part of the object. We explain our proposed method in detail below.
4 Methodology
Adversarial robustness targets to improve model robustness against adversarially perturbed images. Also for adversarially robust models, attribution maps tend to align more to actual image compared to naturally trained models. We study this connection below.
Robustness and aligmment: For an n-class classifier
F(z) argmax('(x) where v
(V/2,.,, V*) X
R" be differentiable in x. Then we call V P(x) the saliency map of F
and the alignment (28 with respect to V in x is represented by
(<x) = Kr, VV (()))))
(4)
Connection of robustness with alignment was studied in 28 This connection is specifically formal- ized in Theorem 2, which states that a network's 's linearized robustness (p) around an input is upper bounded by the binarized alignment term a as:
C
(ax) S at (x) + |llll
(5)
Here C' is constant, the linearized robustness ((x) is given by
V (w)) (x)
p(x) == min 3
(6)
| (x) (x)l
Also, g is the Jacobian of the top two logits i.e. g = ((Vi v (z)) and binarized alignment
i.e af is given by
at + (x) = (r, V ye - (x))
(7)
3

---

Here j is the minimizer of Eqn 6 We also have a(x) a (ar) for linear model and binary classifier. Eqníeexplains the deviation of different terms for linearized robustness in case of neural network. Also, a small error term in Eqn S implies that robust networks yield better alignment i.e. more
interpretable saliency maps.
Adversarial robustness can also be viewed from an angle where an image can be perturbed. to change the model decision, only through perturbing the pixels of the object and not through any pixel outside of the object in the image. We incorporate these ideas through our two-phase training method to achieve an adversarially robust model. Current methods for adversarial robustness in literature have two key drawbacks: (a) Adversarially robust models are shown to perform poorly on clean data, There exists a clear trade-off between adversarial and natural accuracies which is explored in [I (b) Almost all SOTA adversarial robustness methods rely on iterative adversarial training frameworks, which makes them very costly to apply. the proposed method, we aim to solve both these issues, as the model, trained by our method, pushes the bar of both adversarial accuracy and natural accuracy. At the same time, our training strategy does not rely on the iterative adversarial training framework, and thus is very fast.
4.1 Teacher-guided Saliency-based Robust Training
First Phase Enforcing Alignment. This phase incorporates the alignment of the attribution map to the object in the image, as explained before. Let's assume, we have a pre-trained Teacher network (represented as fr). We also consider a student network, represented by a neural network fs. parameterized by 0 and discriminator network faise parameterized by 6. Given input image we obtain the saliency map from a pre-trained Teacher Network, which is denoted as J (TCI represents true class index). Now, maximize the true class prediction score of student network w.r.t input pixels and measure the net change in input pixels, which is represented as JT
Now for an image of dimension w with e channels and as per-pixel gradient and represented as:
c, JI can be considered
JT (a) = ViJr(t) = (VDTT(()) Vir((E))
Similarly, JICI is represented as:
g (x) vyds(t)
(v (
We visualize the training process in first phase of figure 1 Here we trying enforce the fact that the set of pixels, responsible to increase the true class prediction score, are same as actual object pixels which are highlighted by the reference teacher saliency map. This implies imposing similarity between the two saliency maps J/CI and J' Inspired from (9) here we consider the concept of discriminator from GAN[32], which tries to differentiate between real and fake images and backpropagate a signal that forces the model to generate realistic looking images. Here, our sole purpose of using a discriminator is to influence our model generate better saliency that matches the teacher saliency. Hence, we minimize the following objective function:
1 Opptimum argmin([Crs + BL
(10)
0
Where cross-entropy loss LCE R. 1oomnlyy2 [s(r) and Robust is defined as below: L = R,,,, (log Seinc (JC1)) Ryges (log(1 - Saise(JT (J]
(11)
Now can be shown that the global minimum of Robust achieved when JTCJ matches J7 [10 which justifies using the discriminator loss for our purpose. We also add loss term to minimize the l/ distance between JeCI and JJCI, which is represented as Laiff and defined as below:
Laif J CT J
(12)
Hence, we minimize the complete objective function below:
Ooptimum argmin|LCr + (BL + Lasss)ll
(13)
alignment loss
At the same time, The discriminator network is trained as follows: Soptimum argmax Robust] [Robustt]
(14)

---

Reference Network
Reference
lp Image
hp image
JTx
Jg CE
Main Network
Maximize loss class
Jg
true
TC
Network
Maximize
class
Discriminator (Adise)
LRobust
Discriminator (Tdisc)
LRbbsst
Discriminator
(fcur-disc)
Leur-Robusst
Phase Alignment
Phase Refinement
Figure 1:
Our proposed training strategy. Phase enforces alignment and phase 2 enforces curriculum style training by reducing k in ! every step.
The model achieved through this phase of training generate saliency maps that localize the object nicely. This model itself can perform on par with AT terms both adversarial and natural accuracy, and are shown in 6 With the target of attaining better performance, we continue with the second phase of training which is explained below.
Second Phase Model Refinement. In this phase, want ensure that the decision of the model can be changed only by perturbing the object pixels. Here we bring curriculum style learning in picture by gradually shortening the set of pixels which are allowed to perturb in order to reduce true class prediction score. The set of pixels are selected based on discriminativeness of the object parts and which is decided by the teacher saliency. the attacker has to change the image by only perturbing among some fixed amount of top pixels from the whole object i.e. highlighted part of the saliency map given by the teacher. From another perspective, if an attacker, in order to change the model's decision, has only very few options to perturb object pixels compared to the pixels in the input image, which drastically reduces the adversarial attack effect input image. Hence this is implied that the search space of pixels will be very limited during each iteration adversarial attack step.
every step of this phase, the training enforces that
Now, during first step of this phase of training, top 90% of the pixels from the teacher saliency are considered ie. the allowed set pixels for modification by the student model reduced that top 90%. Hence, the training will enforce that only these set of pixels are responsible to maximize the loss i.e. decrease the true class prediction score. Here we follow curriculum style training and after training the first step for few epochs, we consider only top 80% of the pixels from the teacher saliency in the second step. We continue in this fashion with training every step for few epochs (predecided and kept same for every step) and stop after training with top 50% of teacher saliency. We explain the training for one step below with as top percentage of pixels from teacher saliency.
As stated in the first phase of training, we obtain J in the exact same way. Then, we keep top 1% salient pixels J.C,, which is denoted by J and make remaining less salient pixels of actual object zero. We obtain J by maximizing the CE loss of student network w.rt. input pixels and then considering the net change in input pixels. Another discriminator network fourr-disc is considered which is parameterized by &.
Now we have LCE = B( (ogss())) Also, JCB can be considered as per-pixel gradient and represented as
JC = = 5LCE 8LCE
(15)
8x
Now, following the similar motive in the first phase, training objective for this phase is presented as follows:
@pptimuum argmin LOE RBo + 7Lais + Bheurr- Robust + -ouurrddff)))
(16)
alignment loss
curriculum loss
where alignment loss is similar as given in eq and Lourr Robusts Lecur--te diff are defined as below:
5

---

Type
Curriculum
Methods
Clean
FGSM
PGD-5
PGD-10
PGD-20
C&W
AA
AT(PGD-7)
87.25
56.22
55.50
47.30
45.90
46.80
44.04
FNT
87.31
NA
NA
46.99
46.65
NA
NA
LAT
NA
NA
53.84
53.71
49.12
NO
TRADES 27
84.92
61.06
NA
NA
56.61
51.98
$3.08
GAIRAT
NA
NA
AWP-AI L3
85.57
62,90
NA
NA
58.14
55.96
54,04
Herative
MART
84.17
67.51
NA
NA
58.56
54.58
NA
Methods
CA
77.43
57.17
NA
NA
46.06
42.28
NA
Dynamic AT
63.53
NA
NA
Y ES
FAT 17
87.00
65.94
NA
NA
49,86
48.65
53.51
ATES
86.84
NA
NA
NA
55.06
50.72
SADS 2V
82.01
51.99
NA
45.66
NA
NA
NA
JARN-AI 10
84.80
67.20
50.00
27.60
15.50
NA
0,26
Nom lterative Methods
NO
1GAM[9
88.70
54.00
52.50
47.60
45.10
NA
AT-Free 85.96
NA
NA
NA
46.82
46.60
41.47
Y EES
OURS
90.63
67.84|
63.81
61.44
59.59
61.83]
54.71
Table I: Results of natural and adversarial accuracies of different adversarial robustness techniques against different attacks on CIFAR-10 10 dataset. Here and with method represents that the corresponding results with WRN34-10 and WRN28- networks respectively.
Leury- Robust
Eggci [log fourr-diso (J.. E Egge [log(1 - fourr- (J
(TCI)
(17)
Ceurr-diff || J
(18)
Please note, set and coefficients with discriminator loss and loss respectively according their importance. Similar to Robust t in 11 we can show that the global minimum of Learr- Robust is achieved when J J [10, which substantiates the of discriminator loss. At this stage, apart from the discriminator training eq 14 the other discriminator network (faurr-disc) is trained as follows:
Saptimum argmax([eern Robust
(19)
The second phase of training strategy is visualized in fig I The curriculum is imparted by gradually pruning the least attributed pixels thus retaining only top k. k reduced in uniform steps of 10 starting from 100. For example, initially it is trained with JT for, say, 10 epochs, and then with JTCI T,80 and so Going in this fashion, ultimately, the model will be trained to perturb the image by modifying only the most discriminative part of the object to lower the class confidence. We now justify the requirement of two phase training and necessity of curriculum style learning.
Why do we need both Robust and Laiss for the alignment loss? Here both the losses i.e. Rotuust and Laifs together help serve the overall objective i.c. generate better saliency map that matches student saliency. They achieve global minima individually when both saliency maps match. A theoretical justification of attaining global minimum for CRobust is shown in Theorem 3.1 of [9]. Adding both losses helped get stronger signals and attain better performance compared to using any one of them, which is also supported by experimental findings shown in See 6
Why JăCr is considered during first phase and including JS F in the second phase? Our robust model training is motivated by alignment of saliency map with the object features. This idea helps student model to learn better object localization through saliency thus improving robustness and should be maintained throughout the training. Once the model has decent knowledge of the object after alignment phase, we include the concept of J in the training. Incorporating this help the model to learn the allowed set of pixels, to be perturbed, to reduce the class score which further boosts robustness. we were to start the curriculum style training from the beginning. it would have disrupted the model from acquiring knowledge about the whole object.
Why curriculum learning? By introducing curriculum style learning, we try to enforce that most of the pixels, which are allowed to change, should belong to the most discriminative parts of the object. Also fewer pixels should be considered from the lesser discriminative parts the object. As the discriminativeness is decided by the teacher saliency, we consider lesser number of pixels which is top most 4% important part of the object. The value of reduced every step of curriculum and the model is trained to learn about the degree of object discriminativeness gradually.

---

accuuney
50
190
100
90
10
100
S4neace of secovry with Madry Teucher-o F2100
Destetion of scrarery with Nadyy Teucher- CIARTOO
Verittee aemenny with Marty Teacher C3100100
200
110
864
10
10g
10p
Figure 2: Effect of different values and top-k% pixels during second phase of training difffrent accuracies for CIFARIO (top row) and CIFARIO0 (bottom row). These plots are for clean. FGSM and PGD-20 accuracy after every for different y values.
Experiments and Results
Here explain differ- ent experimental settings and the evaluation results against various attacks to show the efficacy of method on CIFAR10 and CIFAR 100 datasets [33] Our results on the Tinylma- geNet dataset is included in the Appendix due to space constraints.
Vorizton of acouacy with camcuili
cilimittee stopo Madry Teacher -IFARt
Veratoon of socurcy wtth cumouium Medry Tescher DTRR3EO
10
Hore KCB-10
PCD 33
20-22
28
100
60
70p
70p
Figure 3: Effect of reducing top pixels during second phase of training. Considering adversarial accuracy for PGD-20 attack, hest results achieved after selecting top-50% and top-70% of teacher saliency for CIFARIO and CIFAR 100 respectively.
Experimental Settings. For CIFAR10, continue first phase of training for 100 epochs and 15 epochs for each & during second phase upto k = 50. While we keep 3 = throughout the training, 7 is changed from 10 in first phase 20 in second phase. Learning rate uniformly decays from 0.1 to 0.001 in first phase and from 0,001 to 0.0001 in second phase. For CIFARIO0 100, we train with the same number of epochs CIFAR-10 for both phases. While keep 3 2 throughout the training, is changed from 10 in first phase to 15 in second phase. Learning rate is also considered same as the training with CIFAR-10.
Evaluation Against Adversarial Attacks. Here we show evaluation results of our proposed model against various standard adversarial attacks [ 2 and recently proposed AutoAttack [5) which is used as a strong attack against most of the recently proposed robust models. We consider many
recently proposed methods 3
g
27
16 3
14
[TI 12] [7]
[5
29
10
g 18 baseline and
report our performance on the same setting as their's on CIFAR-10 10 and CIFAR- 100 datasets. For evaluation, we consider Loe attack with e 8/255 and use WRN32-10
the main network to run all the experiments following [3] for both CIFAR 10 and CIFARI00 While most of the methods, considered as baseline, follow same experimental setup and used WRN32-10
34 architecture as their main network, some of them used bigger WRN34-10 10 3d network (which we point out in our result). We report these results in tables and CIFAR-10 10 and CIFAR-100 datasets. These tables show that our method achieves best adversarial accuracy against all the attacks reported without sacrificing much on natural accuracy for both the datasets. We studied our model with random restarts, and observed variations of negligible range +-0.02%) in our results compared to what we achieve without random restart. Our method thus also achieves state-of-the-ar adversarial accuracy even when considering baselines with random restart setup. It is worth mentioning that surpass
7

---

Training Type
Curriculum
Methods
Clean
FGSM
PGD-5
PGD-10
PGD-20
AT(PGD-7) 7
60.40
29.10
29.30
24.30
23.5
FNT
60.27
NA
NA
22.44
NA
LAT
NA
NA
NO
TRADES 27
58.55
NA
NA
NA
25.89
lterative
26.17
Methods
CAT18
66.05
NA
NA
NA
10.91
YES
Dynamic 1
54.71
NA
NA
NA
23.44
ATES [5
62.95
NA
NA
NA
28.05
NO
IGAM
62.39
34.31
29.59
24.05
21.74
Nom- Mon-Iteratiiv
AT Ar-Frre
62.13
NA
NA
NA
25.88
Methods
YES
OURS
69.88
43.71
38.03
30.04
28.32
Table 2: Results of natural and adversarial accuracies of different adversarial robustness techniques against different attacks on CIFAR-100 dataset. Here with method represents that the corresponding results are with WRN34-10 network.
CIFARIO
CIFARIO
CIFARI00
CIFARI00
Figure 4: Visualizing effect different teacher networks while training our student model. each triplet of images above: (Left) Original image: (Middle) Saliency of our model trained with teacher (adversarially trained on another dataset, finetuned on clean data of given dataset); (Right) Saliency of our model trained with teacher 2 (adversarially trained given dataset)
all the other baseline methods including those using WRN34-10 for both CIFARTO and CIFAR100. Please also note that the best model is achieved after training with k: = 50 and k 70 for CIFARIO and CIFAR 100 respectively. We experimented with increasing number of iterations of PGD attack on CIFAR-10 and achieved adversarial accuracy results as 59.59%, 58.50%, 58.22%, 58.04% for PGD-20, PGD-50, PGD-70, PGD-100 respectively; this shows that our method is fairly robust to increasing intensity of the PGD attack.
6 Discussion and Ablation Studies
Effeet of Curriculum
Style
Learning
in
Refinement
Phase.
As explained sec we re- duce percentage of top dis- criminative pixels (k:) from every
Vartation
Teacher
Varietion
Tercter OIAR100
70
teacher saliency step and stop at top 50% of the pixels during refinement phase. We report adversar- ial and natural accuracies af- ter every step with the best y values (20 for CIFARIO 10 and 15 for CIFAR100) in fig 3 which justifies the ef-
80
HoM
50
60
30
101
5.0
7.
10.0
125
15.0
17.5
20.0
22.5
25.2
3.0
13.0
125
15.0
27.5
20.0
225
250
Figure 5: Effect of considering different values of y during second phase of training on final accuracies for CIFARIO and CIFARI00
fectiveness of curriculum style learning. Our method achieves the best results for 50 and CIFARION 100 datasets respectively (considering PGD-20 accuracy). We also provide detailed results for 5 different 7 values with different & for clean, FGSM and PGD-20 accuracies for every 7. Fig represents such result for CIFAR 10 and CIFAR100, Please note that k = 100, for all the above mentioned figures, represents accuracy values by the model after first phase of training. We provide one more argument in support of curriculum learning for our method, where we experiment without curriculum learning and train refinement phase with the same loss function but with only single Here we consider l same as the best performing & for training with curriculum learning (e.g. k = 50 for CIFAR10). Due to space constraints, we present details of this experiment with results in the Appendix.
70 for CIFAR10 and
8

---

A
B
B
Figure 6: Effect of curriculum learning through visualizing perturbed images when the models are attacked by FGSM and PGD-20. For each group of 12 images. the lop-left and bottom-left triplet corresponds to the actual image, perturbed image and net change (delta) in pixels when model obtained after 1st training phase attacked using FGSM and PGD-20 method respectively. Similarly, the top-right and bottom- rright triplets corresponds to the same set of images obtained using a model obtained after 2nd phase of training. Here A and B represent images with models obtained after first and second phase respectively.
Teacher
Nat.
FGSM
PGD5
PGDI0
PGD20
Teacher
Nat.
FGSM
PGDS
PGDI0
PGD20
A
87.6
57.31
56.12
48.20
46.60
C
74.6
46.67
37.43
25.95
24.06
B
90.6
67.84
63.81
61.44
59.59
D
69.8
43.71
38.03
30.40
28.32
Table 3: Result on CIFAR-10
Table 4: Result on CIFAR-100
Robustness with Different teacher:(Note that all other hyperparameters kept same). A=Avvesariallyy trained on TinylmageNet followed by finetuned on CIFAR 10, B=Adversarially Trained on CIFARI0 10, C=Adversarially trained on CIFARIO followed by finetuned on CIFARI0O, D=Advessaiallly Trained on CIFAR100
Effect of Adversarially Trained Teacher. We consider a reference network which generates a nice saliency map that the main network would try match. As already discussed better alignment of saliency maps to the image features for adversarially robust models compared to naturally trained models, we considered two different models as reference networks for comaprison. For CIFAR 10, one of the teachers is PGD-7 3 trained model with TinyImageNet, followed by natural finetuned with CIFAR 10, and the other one is PGD-7 [3] trained with CIFARIO. 10. Similarly. for CIFARIOM 100, we
have results with teacher model which is PGD-7 7
3 trained with CIFARIO followed by naturally finetuned with CIFARI00 compared to PGD-7 3 trained with CIFAR The results for CIFAR- 10 [nd which depicts better accuracy numbers when our model is trained with full adversarially trained teacher. We also generate saliency maps with our model for both the teachers, which are presented in fig 4 As anticipated, we notice better saliency maps with full adversarially trained teacher for both CIFARIO and CIFARTO0.
and CIFAR- 100 datasets are reported in tables 3
Importance of First Phase of Training, This is an important question if we think of our gain by removing the first phase of training and using the AT model 3 directly, i.e. the teacher model for the second phase of training. As the training consists of cross-entropy as well as alignment losses, plugging the teacher model in place of the student model after the first phase idcally would incur zero alignment loss. But cross-entropy loss of such a model would be more compared to the model trained using our method. After the first phase, we achieve 1% to 2% improvement in natural as well as adversarial accuracies using our model compared to the teacher model as shown in Tables 11 and 12 in the Appendix. For comparing with AT, we report each loss term after phase 1 in Table 7 where the teacher model is of the similar architecture as the student i.c. WRN32-10. Moreover, in order to show the importance of the first phase training, we train the second phase, starting from AT model, instead of the model achieved after the first phase. Here the first phase of training sets the stage for curriculum style learning in the second phase. which is supported by our experimental findings suggesting that robustness of the final model (achieved after second phase of training) is much lesser we use the AT model directly. Experimental results on CIFAR-10 10 are presented in Table 8
Effect of y Regularizer Coefficient. We experimented with different y values during our training and reported the variation of accuracies for different ys in 5 Please note that 3 = 2 and y = 10 were kept always same for first phase of training, and different 7 values were tried during second
Align Loss
Nat.
FGSM
PGDS
PGDIO
PGD20
Align Loss
Nat.
FGSM
PGDS
PGDIO
PGD20
Remove
88.9
59.98
59.02
56.90
55.84
Remove
67.3
39.32
34.67
27.14
24.45
Keep
90.6
67.84
63.81
61.44
59,59
Keep
69.8
43.71
38,03
30.04
28.32
Table 5: Results on CIFAR-IO.
10.
Table 6: Results on CIFAR-100
100.
Results with keeping removing the alignment losses i.c. Roobust and Laifs during 2nd training phase.
9

---

Model
CE loss
Alignment loss
Our Model
0.3594
0.009
AT Model
0.4956
0.0
Table Cross-entropy and alignment loss values by our model and AT model [ i.e. the teacher model after first phase of training for CIFAR-IO.
Model
Nat.
FGSM
PGD-20
PI:OUR P2:OR
90.63
67.84
59,59
PI:AT = P2:OUR
84.47
57.19
51.10
Table 8: Effect of starting the second phase training with the model achieved hy our model after first phase training and AT model [3] for CIFAR-10.
Combination
Natural
PGD20
3 2 and
20
90.63
59,59
7 =
87.08
54.83
3 = 2 and 7 =
84.79
41.43
Table 9: Relative effect of & and Natural and adversarial robustness performance for CIFAR-10.
phase of training only. During the second phase, we achieved best results with 7 = 20 and 7 = 15 for CIFARIO 10 and CIFAR 100 respectively and fig variation of values during second phase only. Selecting higher Y values compared to 3 was required to enforce importance towards alignment and curriculum style training, which is also justified by our reported results.
Relative Effect of 3 and y Regularizer Coefficients. We argued the importance of both losses i.e. Lasfy and Robuut in achieving our objective optimally in Sec 4 We support this claim experimentally by studying our model performance considering / and 7 in Table 9 for CIFAR-10. The best result is attained with both 3 and y compared to removing any one of them.
Effect of Continuing Alignment Losses During Refinement Phase. We start our model training with alignment based losses i.e. LRobust and Laif/ during first phase of training. Discontinuing them during second phase forces the model to completely focus on restricting allowed pixels for perturbation only and gradually forget knowledge about the object as a whole given by the teacher saliency. We justify this argument by reporting the final natural and adversarial accuracies with keeping and removing both the alignment based losses during second phase of training. We got better result by keeping both of them the second phase of training which is reported in tables and 6 for CIFAR-10 10 and CIFAR-100 respectively.
Effect of Curriculum Learning in Restricting the Perturbation Set. We visualize the effect of curriculum style learning by considering models before and after second phase of training and generating adversarially perturbed images by both of them. This presented in fig/ō and these models are termed as A and B respectively. For better clarification, we take the difference between original and perturbed images and call it delta, which is an indication of all the perturbations. Please note that the pixels of delta are normalized between 0 and 1, which represents the maximum possible perturbation for a pixel as white pixel for example. In case of B model, clearly the perturbations are much lesser than A model for both FGSM and PGD-20 attack for both the example images. As the model is trained with enforcing restriction of perturbation during the second phase of training, these visualizations are justified. We provide more such visualizations in the Appendix section.
7 Conclusion
In this work, we propose a new non-iterative method to achieve adversarial robustness that operates 10-20% of the training cost of traditional adversarial training methods. Our method significantly outperforms state-of-the-art methods on adversarial accuracy without affecting natural accuracy, which demonstrates the efficiency and practical applicability of our training strategy.
Broader Impact. Pros: Methods like Adversarial Training take a tremendous amount of time and computational resources. Our method is very efficient in training time and produces SOTA results without compromising on natural accuracy. This allows our method to be practically useful for adversarial robustness efforts. Cons: Owing to the different Jacobians used, method's is efficacy in memory consumption may require improvement. There are no other known socially detrimental effects of this work.
Acknovledgements and Funding Transparency Statement. This work has been partly supported by the Govt of India UAY program, Honeywell and Google Research Scholar Award.
10

---

References
[1] Ian Goodfelloww Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. International Conference Learning Representations, 2015.
[2] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pages 39-57. IEEE, 2017.
[3] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference Learning Representations, 2018.
[4] Seyed- Seyed-Mohsen goosavi-Dezfooll Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2582, 2016.
[5] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning. pages 2206-2216. PMLR, 2020.
[6] Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in Neural Information Processing Systems, 2020.
[7] Cihang Xie, Yuxin Wu, Laurens der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 501-509, 2019.
[8] Mayank Singh, Abhishek Sinha, Nupur Kumari, Harshitha Machiraju, Balaji Krishnamurthy, and Vineeth N Balasubramanian. Harnessing the vulnerability of latent layers in adversarially trained models. International Joint Conference on Artificial Intelligence, 2019.
[9] Alvin Chan, Yi Tay, and Yew-Soon Ong. What it thinks is important important: Robustness transfers through input gradients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 332-341, 2020.
[10] Alvin Chan, Yi Tay, Yew Soon Ong, and Jie Fu. Jacobian adversarially regularized networks for robustness. International Conference on Learning Representatiinss 2020.
[11] Qi-Zhi Cai, Min Du, Chang Liu, and Dawn Song. Curriculum adversarial training. International Joint Conference on Artificial Intelligence, 2018
[12] Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, volume 1, page 2, 2019.
[13] Dongxian Wu, Shu-tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 2020.
[14] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019.
[15] Chawin Sitawarin, Supriyo Chakraborty, and David Wagner. Improving adversarial robustness through progressive hardening. arXiv preprint arXiv:2003.09347, 2020.
[16] Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geaaetry-aware restihhe adversarial training. arXiv preprint 1.X1::2010.01736. 2020.
[17] Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks which do not kill training make adversarial learning stronger. In Interna- tional Conference on Machine Learning, pages 1 11278-1 128. PMLR, 2020.
[18] Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Advances in Neural Information Processing Systems, 2020.

---

[19] Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, arXiv preprint arXiv:1803.06373, 2018.
[20] Yogesh Balaji, Tom Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. arXiv preprint arXiv:1910.08051, 2019.
[21] Eric Wong, Leslie Rice, and Zico Kolter. Fast better than free: Revisiting adversarial training. International Conference on Learning Representations, 2020.
[22] BS Vivek and R Venkatesh Babu. Single-step adversarial training with dropout scheduling. In 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 947-956. IEFE, 2020.
[23] Anindya Sarkar and Raghu Iyengar. Enforcing linearity in dnn succours robustness and adversarial image generation. In International Conference on Artificial Neural Networks, pages 52-64. Springer, 2020.
[24] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning pages 4970-4979. PMLR, 2019.
[25] Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhus- sein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. Advances in Neural Information Processing Systems, 2019.
[26] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. International Conference Learning Representations, 2018.
[27] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pages 7472-7482. PMLR, 2019.
(28) Christian Etmann, Sebastian I Lunz, Peter Maass, and Carola-Bibiane Schönlieb. On the connec- tion between adversarial robustness and saliency map interpretability. International Conference on Machine Learning. 2019.
[29] R Venkatesh Babu. Single-step adversarial training with dropout scheduling. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 950-959, 2020.
[30] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning scale. International Conference on Learning Representations, 2017.
[31] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. International Conference Learning Representations, 2019.
[32] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Advances in Neural Information Processing Systems, 2014.
[33] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[34] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. British Machine Vision Conference, 2016.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? Yes, sections 4 and 5
(b) Did you describe the limitations of your work? Yes, section 7
(c) Did you discuss any potential negative societal impacts of your work? N/A, no known negative social impact
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? - Yes
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? N/A, no theoretical results included
(b) Did you include complete proofs of all theoretical results? N/A, no theoretical results included
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or URLY? Yes, in the supplementary material
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? Yes, in the Experiments section and Appendix
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? No, computationally expensive and time-consuning. Also, our results showed negligible statistical fluctuations.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? Yes, in the Appendix
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? - Yes
(b) Did you mention the license of the assets? Yes, all datasets are well-known public datasets (MIT license).
(c) Did you include any new assets either in the supplemental material or as a URL? N/A
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? N/A
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? N/A
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? N/A, no crowd-soureing/uuman subjects
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? N/A, no crowd-sourcing/huma. subjects
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? N/A, no crowd-se www.d--ouuing/uumann subjects
13


Reviews:
[{"review_id": "tafXEg52Ufq", "reviewer": "Reviewer_EoWP", "summary": "The paper proposes a two phase non-iterative approach for efficient training of robust models. \n\nThe first phase is a distillation phase which trains a neural network while enforcing the gradients of the loss w.r.t. to the input image of the student network to be indistinguishable to that same gradient for the teacher network with the goal of enforcing the saliency maps of the teacher and student networks to be “similar”. In addition to the binary cross-entropy GAN-type loss, the paper also explicitly promotes the two saliency maps to be similar by adding an $\\ell_2$ regularization on their differences. \n\nThe second phase, which is called model refinement, is a curriculum learning based adversarial training fine-tuning step. At every step of this phase, the attacker is limited to only perturbing the top X% of pixels from the teachers saliency. As training progresses, the attacker is more limited (i.e., X gets smaller). ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem considered is an important problem and a well motivated problem: How can we efficiently train a robust network which maintains high natural accuracy? The authors have done a good job in phrasing and motivating the problem. The empirical results reported look good. However, I do have some intuition/theoretical questions. \n\nOne of the main premises of the paper is that it achieves robustness very fast (efficiently) since it is a non-iterative method/compared to iterate methods. However, given that during the first phase, the best performing models of this methods utilize iteratively trained robust networks trained on the same dataset (from Tables 3, and 4), it seems like that this method should either contain a phase 0: adversarially train a robust network, or, the claims should be a bit toned down to efficiently improving robustness of robust networks. And for the later case, the method would have a better / fairer comparison if compared to other methods which are based on either pertaining and transfer learning. \n\nThe first phase of this approach seems pretty complicated and while it has motivation, it leaves me with many questions:\n1- Why do we need both L_diff and L_robust for the alignment loss? From the paper (line 163), the global minimum of L_robust is achieved when the saliency maps match, which implies that L_diff is indeed what is needed. I do wonder that, from a theoretical point of view, what does L_robust do? \n\n2- point 1 even pushed further, why do we need phase 1 completely? Doesn’t using the teacher model (specially for cases where the teacher is trained on the same dataset as the student), result in minimum loss for the first phase? If we drop the first phase, then the second phase is similar to a LwF penalty added to robust training. \n\n3- what is the value of loss for each of the terms after phase1 as completed? What would the value of loss be if we simply plug the Madry WRN32-10 robust net for CIFAR-10?\n\nFor the second phase, the curriculum learning is designed to make the attacker weaker, which intuitively *may* result in lower robustness (but at the same time may be beneficial for natural accuracy), do the authors have an intuition why gradually making the attacker weaker during training, can result in an improvement in robustness? \nAlso, what is the intuition behind still using the teachers saliency? The saliency consistency regularizations for phase 2 resembles the learning without forgetting loss --  It seems like that you refine the net but still maintain the saliency map of the robust net from phase 1 (or phase 0)?\n\nFrom results in Table1, it seems that for the curriculum based methods, the strongest attacks are ether AA & CW. I wonder, how the CIFAR-100 models (Table2) would do with these attacks? Also, comparing PGD-5, to PGD-10, and then PGD-10 to PGD-20, it seems like the methods robustness may not have yet plateaued under the PGD attack, I do wonder what is the robustness if we increase the number of iterations further? \n\nOther:\nSmall typo: In line 124, tern -> term. \nFigure 6 contradicts some recent papers which claim that the saliency maps of robust nets are more aligned with human perception. When going to phase 2 (B) from phase (A), if we are improving robustness, it seems like we are making the saliency map more noisy, any intuition on this?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "oBAB2BY6rW2", "reviewer": "Reviewer_hwPL", "summary": "This paper proposes a non-iterative method to improve adversarial robustness. Compared to mainstream AT baselines, the proposed method avoids iterative gradient calculations, so as to be time-saving. By involving attribution maps and perturbing the allowed set of pixels, the proposed method can achieve better performance. After sufficient experiments, the proposed method beats all baselines.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\nCompared to traditional multi-step AT, this paper has a novel solution Teacher-guided Saliency-based Robust Training and clear explanation. \n\nThe paper conducted extensive experiments to validate the effectiveness of the proposed method, including different datasets, attack methods, and baselines.\n\nThis paper has startling performance, especially on natural accuracy in the case of improvement on adversarial accuracy.\n\nCons:\nMy most concern is about the organized and written style. \n-The title ”Adversarial Robustness without Adversarial Training: A Teacher-Guided Curriculum Learning Approach” does not fit the insights. \nThe method that “Teacher-guided Saliency-based Robust Training” is also a solution of training. By the way, although Adversarial Training is the mainstream to achieve robustness, there are still other ways such as Obfuscated Gradients, “Adversarial Robustness without Adversarial Training” need rethinking.\n\n-The main effect mentioned in the abstract is time-saving, however, there is a Teacher model, and regardless of the training time of the teacher model is unfair. Suggest this paper can reorganize its contribution.\n\n-For adversarial-robust models trained by baselines, it is feasible to use any attack method to test its performance, while too much NA is in the experimental tables, bad for comparison. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately addressed the limitations and potential negative social impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "nRMkBi8aE7", "reviewer": "Reviewer_Wty9", "summary": "The paper proposes a non-iterative adversarial defense. The method trains networks to have similar saliency maps as a teacher model. The authors show empirically that their method outperforms baseline defenses while requiring less training time.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**\nThe method proposed in this paper is similar to other non-iterative adversarial training methods as mentioned by the authors in section 3. The exact methodology differs from prior work, which allows the authors to achieve better results that outperform baselines.\n\n**Quality**\nThe method is not motivated theoretically, so its effectiveness must be demonstrated through experiments. The experiments are particularly important, since the paper claims to present a state-of-the-art defense. There are some potential issues with the experiments, although they may be resolvable:\n1) TRADES has a lower accuracy under AutoAttack in Table 1 compared to what is reported in the AutoAttack paper. Is there a reason for this?\n2) How does the proposed method compare to the state-of-the-art defenses on CIFAR-10 (particularly defenses incorporating unlabeled data) reported in the AutoAttack paper (in Table 2 of that paper)? In particular, is the proposed method the state-of-the-art defense on CIFAR-10 for 8/255 $\\ell_\\infty$ perturbation sizes?\n3) The training time and memory consumption of the proposed method vs. baselines do not appear to be quantified anywhere. This is very important to include since computational efficiency is one of the main advantages of this approach.\n4) CIFAR and TinyImagenet are good choices for evaluating adversarial defenses. However, training a robust model on ImageNet would be ideal. \n5) Are both the discriminator-based and L2 saliency losses necessary to achieve robustness? In other words, how does the method perform when $\\beta=0$ or $\\gamma=0$?\n\nResolving these issues would *significantly* improve the quality of this paper.\n\n\n**Clarity**\nThe paper is generally well organized and the writing is clear. Concepts are introduced with the appropriate context and experiments are clearly described.\n\n\n**Significance**\nIf the issues with experimental results are resolved, this paper could represent a major contribution to the research community since it can achieve state-of-the-art performance with an order of magnitude lower computational cost than current methods.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately describe the impact and limitations of the work in the broader impacts section. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "8CgSS-HeSrZ", "reviewer": "Reviewer_b3WR", "summary": "The paper proposes an AT based defense method, which only focuses on the salient image regions to perturb and attack. Saliency is computed from a pertained teacher model; and is gradually reduced during the training (hence the name curriculum learning). A two phase approach is proposed: phase-1 computes the saliency map; and phase-2 generates adversaries by altering the salient pixels only (by gradually reducing the salient pixels from 90% upto 50%). Experiments are given on CIFAR-10/100 and TinyImageNet.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Can authors validate their assumption that perturbing the salient/foreground image pixels results in quick strong adversaries, by considering a pertained segmentation network, and restricting the perturbations to salient regions, and providing a comparison with perturbing all pixels (vs salient pixels)? \n\nAn underlying assumption is that the model trained using the loss in phase 1 (Eq 13) generates saliency map which can localise the object. I was expecting some visualisations of that. Fig4&Fig7 have a few samples, but they’re too few. Can authors show some more examples in the supplementary? Also, can they quantify (e.g., Jaccard Index) on how good are the generated saliency maps; for a dataset with ground truth segmentation masks?\n\nA major claim of the paper is its compute efficiency. I did not see any comparison with existing AT based defense methods for comparison in terms of compute time; number of iterations required. The proposed method has an additional step (phase-1 of the method), which isn’t the case for most sota AT based defences. A fair comparison is required to substantiate the efficiency claims made in the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Conclusion section of the paper briefly mentions pros and cons of the research.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: this works achieves adversarial robustness without adversarial training adding a "refinement" phase to a network trained with an alignment loss. The idea of the refinement is to force the model to be robust to perturbations outside of the detected item and is done using curriculum learning using a threshold on saliencies as a mask. Avoiding iterative training allows to significantly lower the computational cost for Sota robustness. 
I think the discussion with reviewer Wty9 highlights well the situation: yet the paper can be considered as an empirical better way to force a NN to focus on the actually important part of an image, empirical evidence is convincing and an ablation study is performed. I'd love a discussion about the existence or not of a difference wrt the robustness obtained with iterative methods.  
Also, adding results on imageNet would improve the paper and mentioning the low variance in the legends of the tables rather than in text would be good.

Decision: Accept (Poster)