Exploiting Safe Spots in Neural Networks for Preemptive Robustness and Out-of-Distribution Detection
Authors: Seungyong Moon
Gaon An
Hyun Oh Song
Venue:  (2021)
Keywords: adversarial defense
out-of-distribution detection

Abstract: Recent advances on adversarial defense mainly focus on improving the classifier’s robustness against adversarially perturbed inputs. In this paper, we turn our attention from classifiers to inputs and explore if there exist safe spots in the vicinity of natural images that are robust to adversarial attacks. In this regard, we introduce a novel bi-level optimization algorithm that can find safe spots on over 90% of the correctly classified images for adversarially trained classifiers on CIFAR-10 and ImageNet datasets. Our experiments also show that they can be used to improve both the empirical and certified robustness on smoothed classifiers. Furthermore, by exploiting a novel safe spot inducing model training scheme and our safe spot generation method, we propose a new out-of-distribution detection algorithm which achieves the state of the art results on near-distribution outliers.

Under review as a conference paper at ICLR 2021
EXPLOITING SAFE SPOTS IN NEURAL NETWORKS FOR PREEMPTIVE ROBUSTNESS AND OUT-OF- DISTRIBUTION DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Recent advances on adversarial defense mainly focus on improving the classifier's robustness against udversarially perturbed inputs. In this paper, turn atten- tion from classifiers to inputs and explore if there exist safe spots in the vicinity of natural images that are robust to adversarial attacks. In this regard, we introduce novel bi-level optimization algorithm that can find safe spots on over 90% of the correctly classified images for adversarially trained classifiers on CIFAR- 10 and ImageNet datasets. Our experiments also show that they can be used to improve both the empirical and certified robustness on smoothed classifiers. Furthermore, by exploiting novel safe spot inducing model training scheme and our safe spot generation method, we propose a new out-of-distribution detection algorithm which achieves the state of the art results near-distribution outliers.
INTRODUCTION
Deep neural networks have achieved significant performance various artificial intelligence tasks such as image classification, speech recognition, and reinforcement learning. Despite the results, Szegedy et al. (2013) demonstrated that deep neural networks are vulnerable to adversarial examplex, minute input perturbations designed to mislead networks to yield incorrect predictions. There have been large number of studies improve the robustness networks against adversarial perturbations (Song et al. 2017 Guo et al.) (2018), while many of the proposed methods have been shown to fail against stronger adversaries (Athalye et al.. 2018 Tramer et al. 2020p. Adversarial training (Madry et al. (017) and randomized smoothing (Cohen et al. 2019) are some of the few methods that survived the harsh verifications, each focusing on empirical and certified robustness, respectively.
To summarize, the study of adversarial examples has been arms race between adversaries, who manipulate inputs to raise network malfunction, and defenders, who aim to preserve the network performance against the corrupted inputs. In this paper, approach the adversarial robustness problem from a different perspective. Instead of defending networks from already perturbed examples, assume the situation where the defenders can also influence inputs slightly for their interest before the adversaries' incursion. The defenders' goal for this manipulation will be to improve robustness by searching for spots in the input space that are resistant adversarial attacks, given pre-traue rreitraine classifier. We explore methods for finding those safe spots from natural images under a given input modification budget and the degree of robustness achievable by utilizing these spots, which denote as preemptive robustness. Ultimately, we tackle the following question:
Do safe spots always exist in the vicinity of natural images?
One practical example of the proposed framework is the case where user uploads his or her photo from local storage (e.g., mobile device) to social media (e.g., Instagram), as illustrated in Figure Suppose there is an uploader (A) who posts photo on social media, a web user (B) who queries search engine (e.g., Google) for an image, and a search engine that crawls images from social media, indexes them with a neural network, and retrieves the relevant images to B. Our threat model considers adversary (M) that download A's image from social media, perturb it maliciously, and re-upload the perturbed image on the web, where the search engine may crawl and index images from. The classifier the search engine will wrongly index the perturbed image, causing the search

---

Under review as a conference paper at ICLR 2021
Web
Web
Safte
Search Engine
Search Engine
Indea
Uplcacer
Uploxcirr (4)
(4)
(6)
60
Artockker
(M4
Figure 1: Overview of our proposed framework. The left side shows the web users retrieving wrong results due to the ad adversarial example. The right side adopts a safe spot filter on the image uploading process and succeeds in defending the query system from the attacker.
engine to malfunction. Suppose an African-Americat uploader (A) posts photo of him or herself on social media, and racist adversary (M) perturbs to be misclassified as "gorilla" by the search engine. When another person (B) searches "gorilla" on Google, the perturbed image would appear, though the image content shows a photo of A. This attack fools both A and since the perturbed image is used contrary to A purpose and is not the image B wanted. To prevent this, the social media company, cooperating with the search engine company, could ask if A agrees that the images are slightly changed when uploaded to make them robust to such attacks. The purpose of the modification process, corresponding to the "safe spot filter" in Figure [ will be to ensure that the uploaded images are used under A' intention and provide more accurate search results B.
We develop a novel optimization problem for searching safe spots in the vicinity of natural images and observe that over 90% of the correctly classified images have safe spots nearby for adversarially trained models on both CIFAR-10 10 (Krizhevsky & Hinton, (209) and ImageNet (Russakovsky et al.) 2015). We also find that safe spots can enhance both empirical and certified robustness when applied on smoothed classifiers. Furthermore, we propose a novel safe spot inducing model training scheme improve the preemptive robustness. By exploiting these safe spot-aware classifiers along with our safe spot search method, we also propose new algorithm for out-of-distribation detection, which is often addressed together with robustness (Hendrycks et al.) (019a Our algorithm outperforms other baselines near-distribution outlier datasets such as CIFAR-IU 100 (Krizhessky & Hinton 2009).
2 RELATED WORKS
Adversarial training Goodfellow et al. (2015) first show that the robustness of neural network can be enhanced by generating adversarial examples and including them in training set. PGD adversarial training improves the robustness against stronger adversarial attacks by augmenting training data with multi-step PGD adversarial examples (Madry et al. 2017). Some recent works report performance gains PGD adversarial training by modifying the adversarial example generation procedure (Qin jet al. 2019 Zhang & Wang 2019 [Zhang et al.) 2020). However, most of the recent algorithmic improvements can be matched by simply using early stopping with PGD adversarial training (Rice) ct al.) 2020; Croce & Hein 2020). Other line of works achieve performance gains by utilizing
additional datasets (Carmon al.. 2019 Wang et al. 2020 Hendrycks al. 2019a).
Randomized smoothing Injecting random noise during the forward pass can smooth the classifier's decision boundary and improve empirical robustness (Liu et al.) 2018). Using differential privacy, Lecuyer et al. (2019) give theoretical guarantees for & and &g robustness classifiers smoothed with Gaussian and Laplacian noise. Cohen et al. (2019) provide tight bound of &, robustness of networks Pearson lemma. Another proof the robustness bound was given in Salman et al. (2019) using Lipschitz property of smoothed classifiers, where they also propose a new adversarial training scheme for building robust smoothed classifiers.
smoothed with Gaussian noise via the Neyman-Pearson
nutt-f-Distributiin detection with deep networks Although deep networks achieve high perfor- mance various classification tasks, they also tend to yield high confidence in out-o--distribution samples (Nguyen et al. 2013). To filter out the anomalous the maximum value of classifier's softmax distribution as score function, while Lee (2018)
& Gimpel (2017) use
2

---

Under review as a conference paper at ICLR 2021
propose Mahalanobis distance-based metric which spots out-of-distribun samples using hidden features. Hendrycks et al. (2019b) show that leveraging auxiliary datasets disjoint from test-time data improve the detection performance. Recently, (Sattyy & Oore (2020) characterize activity patterns of hidden features by Gram matrices and use the matrix values to identify anomalies.
METHODS
3.1 GENERAL DEFINITION OF SAFE SPOT AND PREEMPTIVE ROBUSTNESS
We first establish formal definition of safe spot and preemptive robustness. Let c X + V be classifier which maps images to class labels. We define the safe region of the classifier c as the set of images that c can output robust predictions in the presence of slight adversarial perturbations.
Definition (e-safe region). Let c X + y be classifier and e E R+ be the perturbation budget of an adversary. The e-safe region of the classifier is defined by Se(c) = {i E X d(')= e(z), Vx' E (z)).
Here, (x) (a' E X d(z, x') S S is the r-centered e-ball. In this paper, we assume l, threat model, ie., ((xxx²)) = || - r'lpps which is the most common setting on adversarial robustness literature, and consider p oc).
Now, suppose a defender can preemptively manipulate a natural image I, under a small modification budget, knowing its ground-trute -tuth label yo- We denote the modified output image I.. Then, the defender's s objective is to make F, be correctly classified as Yo and locate in the safe region Sc(c) to improve the robustness against adversarial attacks. If v, satisfies these two conditions, then say To is preemptively robust and v. is a safe spot of x..
Definition (Preemptive robustness). Let e X + be a classifier and 8, E F be the modifica- tion budgets of the defender and the adversary, respectively. A natural image To with its ground-truth label yo called (6, prreeemptiivy robust on the classifier if there exists a safe spot H, E BB (Ix)) such that (i) c(xx) = Yo and (ii) Is (c(c).
3.2 SAFE SPOT SEARCH ALGORITHM
In this subsection, develop algorithm for searching safe spot with natural image. Given classifier c. finding a safe spot z, from a natural image No can be formulated as the following problem, which directly from the definition of safe spot:
minimize 1e(=,)) + 22,880(
22
subject to || - zollz 6,
where 1 is the 0-1 loss function.
Note in this formulation the defender requires the ground-truth label for the safe spot search. However, images in the real-world (e.g., social media) usually unlabeled, unless uploaders annotate labels their images by hand. So, it is natural assume that the defender cannot access the ground-trus -ruth label yo- In this case, we utilize the classifier's 's prediction c(x,) instead of yo:
minimize (e(e)))e((s)) 1,, (8.(0)
2y
subject to ||æs - xollp S
As v, & Se(c) implies there exists an adversarial example Ta E Be(xs) such that c(za) # c(x,), we can reformulate the optimization problem
minimize Ta(rr))/o(nu)) + sup I(tor))tt((a)))
subject to ||x, -oolp S 5 and I|z0 xellp S.c.
Since the 0-1 loss is not differentiable, we employ the cross-entropy loss :: x y R+ of the classifier as the convex surrogate loss function:
minimize f(x,, c(to)) + sup {1t, c(xx))
(1)
subject to ||x, xolly 56 and || - xallp S
Let h(xx,) denote the objective in Equation II. Instead of minimizing h(xx),, we minimize h(x.) =
supz, ((Ea; c(x:)), since it upper bounds h(..)) when sufficiently minimized by Lemma
3

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Finding a safe spot
input An image and its prediction (Ia, c(ro)), the cross-entropy function Ja Zo for I...., MAXITER do
Generate N adversarial examples for .,...,N do
(0) 2, + where Th ~ (B) for t 1, ..... de
(8) II (J(aa (t-1), ; ;e(oo), E))
end for
end for
N (f (T) c(z)))
Ux
Illustration of the safe spot search process. The shaded region represents the set of points that are misclassified.
Figure 2:
N
Ds.
end for output J2
Lemma 1. 1f)( S - (o0( = 0.6931, then h(x,) S 2h(x.))
Proof. See Supplementary A.I.
Finally, we have the following optimization problem:
minimize sup f(Ta, c(xo)) zu
(2)
Za
subject to ||., xollo S and || xallp Se.
To solve Equation (2. first approximate the inner maximization problem by running T-step PGD (Madry et al. 2017) whose dynamics is given by
(D) +
(random start)
I = f [t 1) ((
(adversarial update)
(6) IT (z!11)
(projection)
where is a random noise uniformly sampled from the l, zero-centered e-ball, f FGSM (Goodfel low et al. 2015) defined by
+ sgn (vz((z if p
f(;
v.l(r, [..l(,,))
x a.
if p==a2
and II,,, is projection operation to B, (xx.).
Then, we iteratively solve the approximate problem To update Fs, need compute the gradient of
given by replacing Fa x!?) in Equation (2) ((xa)) .c(x))) with respect to Is expressed as
De (I. (T) c(xx)) af
of 11 (1
Val (1 ((())).
Oz.
Ox
Or
where 0f f0 is the Jacobian matrix of f = 1I, ff which is easily computed via back tuck-propagation. After computing the gradient, update x, by projected gradient descent method:
3. dl(x!?) ,c(x)))
(i+1) 2
122.6
dx.
Note that the loss ((x6 c(x.)) ) is now a random variable dependent on 77. Therefore, we generate (T) i.v N adversarial examples {aa,n Ju=1 with different noises and optimize the sample mean of the losses instead. Algorithm I shows the overall safe spot search algorithm and Figure 2 illustrates our optimization process.
4

---

Under review as a conference paper at ICLR 2021
3.3 COMPUTING UPDATE GRADIENT WITHOUT SECOND-ORDED DERIVATIVES
Computing the update gradient with respect to involves the use of second-order derivatives of the loss function l since the dynamics f contains the loss gradient V.l(x, y). Standard deep learning libraries, such as PyTorch (Paszke et al.) 2019), support the computation of higher-order derivatives. However, imposes a huge memory burden as the size of the computational graph increases. Furthermore, for the case of 2, computing the update gradient with the second-order derivatives might cause exploding gradient problem if the loss gradient vanishes by Proposition
Lemma 2. Suppose is wrice-differentabll and its second partial derivatives continuous. p the Jacobian of the dynamics f
of I + a. dæ
(1-(1ele) 9-)
)
(11112)
H
(19ll1l2) (19912
|19/112
||al12"
where g Val(z, and H V2((z,1)
Proof. See Supplementary B.1.
Proposition If the maximum eigenvalue of H. in absolute value o, then
05T
{(1
- (1+a a. 1gl12 |al|2-
Ör
Proof. See Supplementary B.2.
As we update Ie, the loss gradients x, and its adversarial examples Ia get reduced to zero, which might cause the update gradient explode and destabilize the update process. To address this problem, we approximate the update gradient by excluding the second-order derivatives, following the practice in Finn et al. (017). We also include experiment in comparison using the exact update gradient in supplementary B.3. For the case of p co, the second-order derivatives naturally vanish since we take the sign on the loss gradient vz, y). Therefore, the approximate gradient is equal to the exact update gradient.
3.4 FINDING A SAFE SPOT FOR CLASSIFIERS
RANDOMIZED SMOOTHING
To further enhance the robustness of our safe spot framework, we leverage the randomized smoothing technique along with our algorithm. Given base classifier e : X + Y, the smoothed classifier y is defined by
g(x) argmax (o(x + 11) v),
wEY
where 7) ~ NN0,002)) To find a safe spot x, of a natural image Ta. we have to find an adversarial example of x, that maximizes the cross- cross-entropy loss ((x,,,((,, for solving the inner maximiza- tion problem in Equation 2 However, crafting adversarial examples for the smoothed classifier is ill-behaved since the argmax is con-differentiable. To address the problem, we follow the approach in Salman al. (2019) and approximate the smoothed classifier g with the smoothed soft classifier G.A. + P(]] defined as
G(x)
E (0,
(C( + 1)),
where P(Y)) is the set of probability distribution over y and C : X + P(Y) is the soft version of the base classifier such that argmax,e>> C(x), c(x). Finally, the adversarial example x. is found by maximizing the cross-entropy loss of G instead:
maximize #e
logg (G(xa)e(rr))
(3)
subject to ||r.
xs plp S
which can be approximated by T-step randomized PGD with M restarts, where random noises are sampled from Gaussian distribution compute the sample mean of the objective at each step. By replacing the inner maximization problem Equation (2) by the randomized PGD, we can update x, similarly.
5

---

Under review as a conference paper at ICLR 2021
CIFAR-II
FPR99: 50.32%
CIEAR 10
FPR95: 32.16%
LSLN
1000
1000
800
800
600
600
100
100
200
200
10 10-3 10-2 10-1 10"
10 10-3 10-2 10-1 109
Figure 3: Histograms for the loss values of images ((To, ((x.:c(xo)) (left) and the loss values of the perturbed safe spot solution e ((a c(x,)) (right). A safe spot-aware adversarially trained model without fine-tuning is used as the classifier. The dotted lines are where the false positive rate is 95%. Detailed settings in Supplementary C.3.
3.5
SAFE SPOT-AWARE ADVERSARIAL TRAINING
we investigated how the defender can find safe spot from a natural image, given pre-trained classifier. this subsection, we explore the defender's training scheme for a classifier on which data points are preemptively robust. Suppose the defender has a labeled training set, which is drawn from true data distribution D. To induce classifier to have safe spots in the vicinity of data points, the defender's optimal training objective should have the following form:
In Section 3.2.
minimize
[K Yo: 0o: 0)]
subject to x argmax M(Ta, yo) and x = argmin sup E(Ta, yo), 2, (B,(())) a. (x.)
(B,)))
where 0 is the set of trainable parameters. Concretely, the defender finds safe spot candidate x; of a datapoint To and generates an adversarial example z from x;, Then, the defender minimizes the cross-entropy loss f(x Yo; 0) so that x becomes an actual safe spot. Note that the ground-truth label yo is used instead of the prediction c(xo), since we assume that the defender can access the ground-truth label during training.
The most direct way to optimize the objective would be to find x; from x. using our safe spot search algorithm and perform k-step PGD adversarial training (Madry al. (0017) with x However, since the safe spot search algorithm requires running T Ttsp PGD dynamics each update, the proposed training procedure would be more computationally demanding than PGD adversarial training. To ease this problem, consider replacing the inner maximization supz, ((Eas Mo) in the safe spot search by (xx,, yo):
x = argmin x. (x(,)) age (n,))
sup
((Fa, vo)
= J argmin E(I.. Jr =,EBg(
Then, 2 can be easily computed by running targeted FGSM or k-step PGD on 2, towards the ground-truth label yo- We denote this training scheme as safe spot-aware adversarial training.
3.6 OUT-OF-DISTRIBUTION DETECTION
The safe spot-aware adversarial training method induces the learned data distribution to have safe spots near its data points. Thus, will have a higher probability of having safe spots compared to the out-of-distribution (OOD) samples, shown in Figure We leverage this conjecture to propose algorithm that jointly utilizes our safe spot generation method and safe spot-aware adversarial training.
naturally conjecture that the samples from the learned distribution
aut-of-distributtin detection
Following the framework of Hendryeks et al. 2019b), which use auxiliary outlier data to tune anomaly detectors, we consider there are three types of data distributions, Pm, Ditii, and Diest Dm refers to the learned distribution, also called the in-distribution. Damin the given distribution of outliers used to tune the detection algorithm, which is orthogonal to out Diest is the distribution we want detect OOD during inference, which is unknown. We include the auxiliary outlier data

---

Under review as a conference paper at ICLR 2021
safe spot-ware training procedure and adapt the training objective as below:
minimize
E (HorBo)~-aa
(K(2a) Yos Vos
+ R by DrL(ū || C(+o: X ((G,, (4)
subject
Fg
fa Mo) and xz- argmin sup B.tB)(
argmax
Bx(r) F.EB,(
argmax (8,, c(āo)) and E argmin sup f(Ea; c(oo)) (.7)
a.EBg(R.) (.EB,
where ÿ is the uniform distribution and C(ăoi 0) is the softmax probability of d.. Since to is unlabeled, we use the prediction c(t,) instead for safe spot searching. Note that if e 2 6, the first term Equation 4 also maximizes the confidence of the original in-distribution samples, since To Bx(x²) (x() and therefore (( y; 0) I f(x; y; 8). Similarly, the second and third terms minimize the prediction confidence and the probability of safe spot existence the outlier samples, respectively.
With the trained classifier, we measure the safe spot objective value from Equation (2) along with the maximum softmax probability (MSP) and use the values as indicators to detect OOD samples. Concretely, we define the score function as a linear combination of the two indicators. Considering they have a different range of possible values, we replace the safe spot objective value with the MSP of the adversarial example for the safe spot solution. Finally, the score function is formulated
D(x.)= == μ max C(Eo)y (1 - (1) max C(x))r DEy
yEV
where x; E Bs(xs)) is the optimal solution the safe spot algorithm for x. and x E (x;) ) the adversarial example of x². We filter inputs with low scores OOD.
4 EXPERIMENTS
As it is natural to assume that the defender and the adversary have the same modification budget, we set = e for all experiments. We evaluate our methods by measuring clean and adversarial accuracies, where adversarial accuracy refers to the prediction score under 20-step untargeted PGD) attack with a step size of c/4. In the experiment tables, None column indicates using original images inputs, and S-Full uses safe spot images from Algorithm [ We also evaluate safe spot search via targeted FGSM and 20-step PGD towards the class inferred from the classifier, each denoted as S-FGSM and S-PGD. Detailed settings listed on supplementary C.
4.1 CIFAR-10
We use Wide-ResNet-334-10 Zagoruyko & Komodakis, 2016) and consider two threat models, foc with 8/255 and &2 with e 0.5. We run our experiments four differently trained models. The natural model is trained in a standard manner without considering adversaries. ADV is PGD adversarially trained model. S-FGSM+ADV and S-PGD+ADV are safe spot-aware adversarially trained models, with safe spot search approximated by FGSM or 10-step PGD with a step size of 8/4.
The foo threat model result in Table I (left) shows our methods can find safe spots on over 85% of the test set images, except for the natural model. This performance is near the upper bound, which is the classifier's clean accuracy since we use predicted labels for safe spot search. We also observe that safe spot search via targeted FGSM or PGD is also feasible for ADV, S-FGSM+ADV, and S-PGD+ADV models, but they still miss on about 10% of correctly classified images. When jointly used with our safe spot search method, the safe spot-aware training achieves the highest adversarial accuracy, along with a clean accuracy much higher than PGD adversarial training.
The & threat model result in Table D (right) shows similar results the foo experiment, except that the adversarial accuracy of safe spots generated by S-Full on the natural model is much higher. However, note that the adversarial accuracy of safe spots the natural model may go down to about 20% when the attack gets stronger, for example, by increasing PGD iterations. The results on stronger PGD attacks and other types of attacks considered in supplementary D.3 and D.4.
7

---

Under review as a conference paper at ICLR 2021
Method
Method
Model
Model
None
S-FGSM
S-PG
S-Full
None
S-FGSM
S-PGD
S-Full
Natural
95.97000.0
82, 40000000
95.97100.00 95 $5.48/09.0 67
Natural
95 95.97/00.53
94.61/00.79
95.97/00.35 95 9..9949900 00
ADV
86.51/77.08 86.51/47.21
86.51/71.22 86.51/85.0 06
ADV
90.26/68.16
90,26/88.82
90..6/86.28
90.26/89.
S-FGSM-ADV
86.83/42.50
86.83/78.23
86.83/69.25
86.83/85.3 35
S-FGSM-ADV
90.92/63.27
90,92/88.82
90.92/84.22
90.92/90.60
S-PGD+ADV
91,32/39.33
91,32/77.01 91,32/63.94
91.32/89.84
S-PGD+ADV
94,10V57.70
94, 10/8030
94, 10/00,4
94.10993.5
Table I: Classification accuracy under &oo threat with 8/255 (left), l2 threat with e = 0.5 (right), on CIFAR-10. (clean acc./adv acc.)
4.2 IMAGENET
We use ResNet-50 and consider three threat models: loo with E (4/255, 8/2555 and &g with 3.0. For safe spot-aware adversarial training experiments. utilize "fast" adversarial training
(Wng et al. 2020) and train the safe spot-aware model S-FGSM+Fast to reduce the training cost.
Table (left) shows results on for attack under 4/255. Similar results on CIFAR-10, our methods are capable of finding safe spots near to original images that are correctly classified on the robust classifiers. Also, our proposed safe spot-aware classifier outperforms the original robust classifier by a large margin in both clean and adversarial accuracies. Table 2 (right) shows results on Loo on 8/255, In this setting, we also apply our algorithm the ADV model trained with érain 4/255. Note that by changing only the émain value on adversarial training, we get 10% gain on our safe spot's adversarial accuracy. Surprisingly, classifiers adversarially trained with smaller Grain performs substantially better in terms of preemptive robustness compared to using more robust classifiers. This implies that the conventional notion robustness does not necessarily translate to preemptive robustness. Experiments on lg attacks show similar results and can be found in Supplementary D.
Method
Model
Nome
S-FGSM
S-PGD
S-Full
Model
Natural
75.63/00.03
74.8700047
75.52A00.27
75.63/08.2 22
Nome
S-EGSM
S-PGD
S-Foll
ADV
61.35/32.57
61.35/56.50
61.35/53.13
61.35/60.06
Natural
75.63/00.01
73.05/00
75.04/00.8 18 75.5402. 40
Natural
70.81/00.01 70.33/0021
10.79/00.8
70.82/07.4 41
(caua 8/258)
17.11/18.35
47.05/39.04
Fast
57.05/29.97
57.07/50.97
57.05/50.20
57,04/56.26
ADV (doue 4/255)
61.35/1161
61,3632.44
61.35/28.58
61.35/55.88
S-FGSM-Fast
64.67/14.51
64.73/42.77 64.67/34.53
64.67/61.97
Table 2: Classification accuracy under foo threat with = 4/255 (left) and
8/255 (right) on
ImageNet. The lower three models on e = 4/255 are trained Fast style. (clean acc./adv acc.)
4.3 RANDOMIZED SMOOTHING
We also evaluate our algorithm for classifiers with randomized smoothing. Here, we consider the f2 threat model, where e 0.5 for CIFAR-I0 and e 3.0 for ImageNet. We run experiments on the smoothed classifiers based on the natural and the Gaussian-noise augmented model, which are considered certifiably robust (Lecuyer al. 2019 Cohen al. 2019). We measure empirical robust- ness against the randomized PGD on both models and measure certified robustness on the Gaussian model. Detailed settings such as noise level a and randomized PGD are listed in supplementary C.2.
Table 3 shows our results on the empirical robustness against the randomized PGD. We observe that our algorithm can find safe spots for the natural model with randomized smoothing on 57% and 37% of correctly classified images of CIFAR-10 and ImageNet, respectively. Furthermore, as shown in supplementary D.3, the adversarial accuracy of the smoothed natural model does not suffer from accuracy drop when the attack becomes stronger, in contrast to the natural model. Also, the smoothed Gaussian model, whose training cost is comparable to standard training much less than PGD adversarial training, achieves higher clean and adversarial accuracy compared to the ADV model. Certified robustness results of smoothed classifiers can be found in Supplementary D.2, where safe spot algorithm also improves the certified robustness both the datasets.
8

---

Under review as a conference paper at ICLR 2021
Method
Method
Model
Model
None
S-Full
None
S-Full
Natural
95.97 00:53
95.94 / 59.00
Natural
75.63 / 00.01
75.63 / 10.14
Antural-Smotthing
72.39/ 03.31
94,92 / 55.02
Satural-Smoothing
48.93 / 00.50
74.76/ 27.84
Gaussian+Smoothing
92.35 56.03
92.30 / 91.35
Gausian-Smoothing
69.90/ 10.03
70.03 62.78
Table 3: Empirical robustness of randomized smoothed networks under & threat with e = 0.5 on CIFAR-10 (left) and with 3.0 ImageNet (right). (clean acc./adv acc.)
4.4 OUT-OF-DISTRIBUTION DETECTION
We evaluate the performance of our proposed detection algorithm on models trained with CIFAR- 10. We consider various OOD datasets including CIFAR-100 100. SVHN (Netzer et al. (011), TinylmageNet (Johnson et a..), LSUN Yu et al. 2015), and synthetic noise. Following the experimental protocol of Hendryeks et al. 2 we evaluate the OOD detection methods on three metrics: area under the receiver operating characteristic curve (AUROC), area under the precision-recal curve (AUPR), and the false positive rate at 95% true positive rate (FPR95), We compare our method's s performance to Mahalanobis (Lee et al. 2018), OE (Hendrycks et al. 2019b), and Gram (Sastry & Oore 2020). Since Lee et al. 2018) utilizes a subset the De data for tuning the detection procedure while our method and OE do not, modify Mahalanobis to tune with Dinin for fair comparison. For detailed descriptions of the datasets and the experiments, refer to Supplementary E. and E.2.
D2
ZyCE
FPR95
AUROC
AUPR
Mahulanobis
Gaussian
0.00
0.52
0.02
0.38
100.00
99.78
99.99
99.91
100.00
99.39
99.98
99.88
SVIIN
15.38
2.26
0.74
2.841
97.06
99.25
99.77
99.25
97.06
98.95
99.88
99.12
CIFAR-10
CIFAR-10U
78.20
24.65
28.47
21.27
72.43
94.34
93.73
95,06
71.66
94,06
93.77
94.74
ThyymuugNNet
59.61
9.46
10:15
7.05
98.06
Averge
27.35
13.63
14.62
11.75
91.28
97.08
96.92
97.43
90.47
96.87
96.92
97.31
Table Out-o--distii detection results. All results are percentages and averaged over 10 runs.
Table 4 shows the evaluation results. While Mahalanobis and Gram works slightly better on synthetic datasets such as Gaussian noise, more near-distribution outliers such as CIFAR-100. TinylmageNet, and LSUN, our method outperforms these baselines by large margin, which leads to a gain in overall performance, Our method also outperforms OE on most metrics including the Gaussian noise.
CONCLUSION
Parting from recent studies on adversarial examples, we present new adversarial framework where the defender preemptively modifies classifier inputs. We introduce a novel optimization algorithm for finding safe spots in the vicinity of original inputs as well as a new network training method suited for enhancing preemptive robustness. The experiments show that our algorithm can find safe spots for robust classifiers on most of the correctly classified images. Further results show that they can be used improve empirical and certified robustness on smooth classifiers. Finally, combine the new network training scheme and the safe spot generation method to devise a new -ut-of-distribution detection algorithm that achieves the state of the art performance near- dear-distribution outliers.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John Duchi. Unlabeled data improves adversarial robustness. In NewrIPS, 2019.
Jeremy M Cohen. Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In ICML, 2019.

---

Under review as a conference paper at ICLR 2021
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In ICML, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. ICML, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using imput transformations. In ICLR, 2018.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribttion examples in neural networks. In ICLR, 2017.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, 2019a.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019b.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. In NeurIPS, 2019c.
Justin Johnson, Andrej Karpathy, and Fei-Fei Li, Tiny imagenet visual recognition challenge. URL com.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In IEEE Symposium on Security and Privacy (SP), 2019.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurlPS, 2018.
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. Towards robust neural networks via random self-ensemble. In ECCV, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NeurlPS-W W, 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, nigh-performance deep learning library. In NeurIPS, 2019.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. In NewrIPS, 2019.
Lestie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. In IJC,, 2015.
10

---

Under review as a conference paper at ICLR 2021
Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. In NeurIPS, 2019.
Chandramouli S. Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matrices. In ICML, 2020.
Yang Song. Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models understand and defend against adversarial examples. ICLR, 2017.
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2013.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In Sttt::20020083477 2020.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. In arXiv:1506.03365, 2015.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In arXiv: 2016.
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature ceattering-basee adversarial training. In NeurIPS, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In arXiv:2002./1242 2020.


Reviews:
[{"review_id": "zlVunJT1_Nj", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \nThis paper aims to improve adversarial robustness of the classifiers in a different perspective than the existing works. Usually, the networks are trained using adversarial examples to improve robustness (adversarial training). This work extend this line of thought and make an input robust to adversarial attacks. Instead of updating the network, they make updates to the input to gain robustness. In other words, this work explore the existence of safe spots near the input samples that are robust against adversarial attacks. Results on CIFAR-10 and ImageNet reveals that there exists such safe spots which are resistant to adversarial perturbations and improve adversarial robustness when combined with adversarial training (the authors term it as safe-spot aware adversarial training). Based on this approach, the authors also propose out-of-distribution detection method that outperforms previous works.\n\nStrengths: \n+ Motivation is clear.\n+ The proposed approach is interesting and different from existing works. The practical application of the proposed framework is elaborated clearly.\n+ Technical details and formulations are clear.\n+ Results show that the proposed approach improves adversarial robustness and clean data performance on both CIFAR-10 and ImageNet. Furthermore, the proposed approach greatly improves the robustness when evaluated with randomized smoothing.\n+ The design of the approach enables out-of-distribution detection that outperforms previous works.\n\nWeaknesses:\n-\tThe major concern lies in the evaluation of the proposed technique. Here, the authors find safe spots and also propose safe-spot aware adversarial training but evaluate on PGD based adversarial attack in a standard  manner. It is important to address the possibility of safe spot aware adversarial attack on the proposed defense and its success rate. In case such attack is infeasible, please provide the rationale behind that.   \n-\tClarify the difference between S-Full and S-PGD from Experiments section. Since S-Full also uses T-step PGD, how it is different than S-PGD?\n-\tThough the out-of-distribution detection results slightly outperforms previous works under FPR95 metric, the performance gains are very minimal and not very significant than the baseline OE (Hendrycks et al., 2019b) under two metrics AUROC and AUPR.\n\nFinal thoughts:\nThe proposed method is clearly motivated. Although the performance gains on adversarial robustness is significant, there are critical points yet to be addressed. Therefore, I marginally accept this paper.\n\n------------------------------------------------------------------------------------------------------------------------------------\nPost rebuttal:\nThe authors have addressed my concerns in the rebuttal. However, I also agree with the other critical points raised by other reviewers (particularly Reviewer 4) that are of major concern. Hence, I retain my initial score and marginally accept the paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "k0iesDvp8A9", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Thank you for your answers.\n\n------\n\nThe paper proposes a new method for making adversarial attacks more difficult. \nIn their method, the defender (not the attacker) modifies the original input $x_o$ to $x_s$ which is guaranteed to be safe in the sense that an attacker modifying $x_s$ will not manage to change the predicted class until large changes to the sample are performed.\nThe defender's budget for modifying the sample is denoted $\\delta$, whereas the attackers budget is $\\epsilon$.\nAfter some relaxations, they arrive at the optimization problem stated in Equation (2):\nFind the modification $x_s$ (subject to budget $\\delta$) which minimizes the risk (measured by cross-entropy) that any modification of $x_s$ by an attacker (subject to budget $\\epsilon$) is misclassified.\nThey also extend the idea to out-of distribution (OOD) detection, though the main contribution seems to be in mitigating adversarial attacks during testing.\n\nStrong Points:\n\n- the basic idea and derivation of the optimization problem is clearly written.\n- the idea of modifying an input image before classification is interesting, apparently new, and effective in mitigating the impact of adverserial attacks (according to the experiments in 4.1, 4.2, 4.3).\n\n\nWeak/Unclear Points:\n\n- Section 3.5 \"SAFE SPOT-AWARE ADVERSARIAL TRAINING\" is a little bit unclear to me.\nIs it that now the training data, not the test data is modified? But then it is not so clear to me, whether the final training objective is still well defined.\nIt might just have a similar effect as adding noise to training samples.\nFurthermore, it appears that the whole thing becomes difficult to train, since during training it necessary to iterate between (A) ordinary model training and (B) modification of training samples.\n\n- Section 3.6 \"OUT-OF-DISTRIBUTION DETECTION\" is not convincing:\nApparently the proposed method is not only hard to train, but also has three important hyper-parameters $\\gamma$, $\\lambda$, and $\\mu$, which need to be carefully tuned. Therefore, even though the authors report improvements over previous methods in Section 4, I am not convinced that this is a practical approach to OOD.\n\n- In Section 4.1, I am not sure what the authors mean with \"our methods can find safe spots on over 85% of the\ntest set images\". My understanding is that, if the class label could not be changed by an attacker, then the method was successful, even if the original sample was misclassified.\nHowever, Table 1 reports only classification accuracy.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "QJB9URcAEkw", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new adversarial framework where the defender could preemptively modify classifier inputs to find safe spots that are robust to adversarial attacks. They then introduce a novel bi-level optimization algorithm that can find safe spots on over 90% of the correctly classified images for adversarially trained classifiers on CIFAR-10 and ImageNet datasets and show that they can be used to improve both the empirical and certified robustness on smoothed classifiers. Besides, they propose a new training scheme based on their conjecture about safe spots for out-of-distribution detection which achieves state-of-the-art results on near-distribution outliers. \n\nOverall, the writing is clear and the idea is interesting. I think they have the following contributions: \n\n1. Propose a novel adversarial framework and motivate it by a real-world application (search engine example); \n\n2. Propose an effective algorithm to find safe spots; \n\n3. Show the usefulness of safe spots for adversarial robustness and out-of-distribution detection. \n\nHowever, I have the following concerns: \n\n1. I am wondering whether the safe spots actually exist. Based on the definition, the classifier should have the same predictions on all data points in the $\\epsilon$-ball around a safe spot. So the classifier has certified robustness on the safe spots. But it might be hard to show that the classifier has certified robustness on the safe spots. Although they have shown that the attacks could not successfully find adversarial examples for the safe spots, it might be due to the “gradient masking” issues. Could the authors try the auto-attack proposed in [1] to see whether the classifier is actually robust on the safe spots? \n\n2. From their results, we can see that the safe spots don’t exist for naturally trained models. We need to use adversarial training to produce safe spots. But it is not surprising that adversarial training could produce safe spots. In fact, if we could solve the standard adversarial training objective optimally, then any natural images from the training distribution should be safe spots. Could the authors explain why we need to find safe spots other than the natural images in this case? \n\n3. If the classifier is not robust on the natural image $x_o$, and the defender finds a safe spot $x_s$ around $x_o$, then from the attacker perspective, why he could not first perturb $x_s$ to be $x_o$, and then perturb $x_o$ to find adversarial examples? If the attacker could not find adversarial examples for $x_s$, then he may try other attack strategies like using larger $\\epsilon$ or other perturbation types. In such cases, the proposed defense framework may not work. Could the authors explain it? \n\n4. For safe spot-aware adversarial training, they mention that the training procedure is more computationally demanding than PGD adversarial training. Then they use targeted FGSM or k-step PGD towards the ground-truth label as a proxy to safe spot search. It is hard for me to understand what they exactly do in this part. Could the authors describe it in detail? Also, why would the safe spot-aware adversarial training be better than the standard adversarial training? I think standard adversarial training can also produce safe spots. Is it because the safe spot-aware adversarial training search for $x_a$ in a larger ball around $x_o$ ($B_{\\delta+\\epsilon}(x_o)$) than the standard adversarial training? Could the authors try standard adversarial training with a perturbation budget of $\\delta+\\epsilon$ to see if this is the case? \n\n5. For out-of-distribution detection, they conjecture that the samples from the learned distribution will have a higher probability of having safe spots compared to the out-of-distribution samples. But I don’t think their results could support this conjecture. In their training objective, they explicitly minimize the probability of safe spot existence of the outlier samples. So they try to train a model such that their conjecture holds. It would be better if they could show whether their conjecture holds for naturally trained models or the models trained using outlier exposure. I suggest they perform an ablation study for objective (4). Also, I think they miss some OOD detection baselines, such as [2] and [3]. Could the authors compare their method to them? \n\n \n[1] Croce, Francesco, and Matthias Hein. \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.\" arXiv preprint arXiv:2003.01690 (2020).\n\n[2] Mohseni, Sina, et al. \"Self-Supervised Learning for Generalizable Out-of-Distribution Detection.\" AAAI. 2020.\n\n[3] Liu, Weitang, et al. \"Energy-based Out-of-distribution Detection.\" arXiv preprint arXiv:2010.03759 (2020).\n\n--------AFTER DISCUSSION WITH AUTHORS---------\n\nThanks for the clarification. Some of my concerns have been addressed and I have raised my score. But I keep the concern that the proposed defense framework may be easily broken in practice given that the attacker can have unlimited power. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "Jxc_u-7b8j", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors argue that there are some safe \"spots\" in the data space that are less prone to adversarial attacks. The authors propose a technique to identify such \"safe spots\". They then leverage them for robust training and observe higher robust accuracy than baseline. Finally, they leverage this observation to identify out of distribution data. \n\nThe application is important and the results look promising. However, I have the following concern:\n\n- The authors propose a new threat model where the adversary may have access to the labeled data. They motivated such a setting with an example of Google image search. However, such a setting is quite limited. There are also existing methods that use supervised learning setting with incorrect labeling. The paper should discuss how they differ from such a line of work. \n\n- The search algorithm requires that a correct predicted label is available. This setting is not quite realistic. How can we find a safe spot when the label is unknown.\n\n- Some of the findings are not quite surprising. For example, a safe spot is more in a robust model with small epsilon. \n\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers recognized that the proposed method is interesting and seems to be useful in some cases, and the authors provided sufficient empirical results to support their claim.
In addition, some comments have already been clarified.
However, some reviewers still concerned that the proposed defence method will be defeated under some conditions, and still have the major concern regarding the issue of adopting some attack strategies to find adversarial examples near the safe spots, even though the authors clarified some critical points of the proposed method. 
These drawbacks led to the decision to not accept. However, this paper has some merit and can be made into a stronger contribution in the future.

Decision: Reject