Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: 
This paper aims to improve adversarial robustness of the classifiers in a different perspective than the existing works. Usually, the networks are trained using adversarial examples to improve robustness (adversarial training). This work extend this line of thought and make an input robust to adversarial attacks. Instead of updating the network, they make updates to the input to gain robustness. In other words, this work explore the existence of safe spots near the input samples that are robust against adversarial attacks. Results on CIFAR-10 and ImageNet reveals that there exists such safe spots which are resistant to adversarial perturbations and improve adversarial robustness when combined with adversarial training (the authors term it as safe-spot aware adversarial training). Based on this approach, the authors also propose out-of-distribution detection method that outperforms previous works.

Strengths: 
+ Motivation is clear.
+ The proposed approach is interesting and different from existing works. The practical application of the proposed framework is elaborated clearly.
+ Technical details and formulations are clear.
+ Results show that the proposed approach improves adversarial robustness and clean data performance on both CIFAR-10 and ImageNet. Furthermore, the proposed approach greatly improves the robustness when evaluated with randomized smoothing.
+ The design of the approach enables out-of-distribution detection that outperforms previous works.

Weaknesses:
-	The major concern lies in the evaluation of the proposed technique. Here, the authors find safe spots and also propose safe-spot aware adversarial training but evaluate on PGD based adversarial attack in a standard  manner. It is important to address the possibility of safe spot aware adversarial attack on the proposed defense and its success rate. In case such attack is infeasible, please provide the rationale behind that.   
-	Clarify the difference between S-Full and S-PGD from Experiments section. Since S-Full also uses T-step PGD, how it is different than S-PGD?
-	Though the out-of-distribution detection results slightly outperforms previous works under FPR95 metric, the performance gains are very minimal and not very significant than the baseline OE (Hendrycks et al., 2019b) under two metrics AUROC and AUPR.

Final thoughts:
The proposed method is clearly motivated. Although the performance gains on adversarial robustness is significant, there are critical points yet to be addressed. Therefore, I marginally accept this paper.

------------------------------------------------------------------------------------------------------------------------------------
Post rebuttal:
The authors have addressed my concerns in the rebuttal. However, I also agree with the other critical points raised by other reviewers (particularly Reviewer 4) that are of major concern. Hence, I retain my initial score and marginally accept the paper.
review_title: Look at a different perspective to improve adversarial robustness

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors argue that there are some safe "spots" in the data space that are less prone to adversarial attacks. The authors propose a technique to identify such "safe spots". They then leverage them for robust training and observe higher robust accuracy than baseline. Finally, they leverage this observation to identify out of distribution data. 

The application is important and the results look promising. However, I have the following concern:

- The authors propose a new threat model where the adversary may have access to the labeled data. They motivated such a setting with an example of Google image search. However, such a setting is quite limited. There are also existing methods that use supervised learning setting with incorrect labeling. The paper should discuss how they differ from such a line of work. 

- The search algorithm requires that a correct predicted label is available. This setting is not quite realistic. How can we find a safe spot when the label is unknown.

- Some of the findings are not quite surprising. For example, a safe spot is more in a robust model with small epsilon. 






review_title: interesting method to identify images that are less prone to adversarial attack

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a new adversarial framework where the defender could preemptively modify classifier inputs to find safe spots that are robust to adversarial attacks. They then introduce a novel bi-level optimization algorithm that can find safe spots on over 90% of the correctly classified images for adversarially trained classifiers on CIFAR-10 and ImageNet datasets and show that they can be used to improve both the empirical and certified robustness on smoothed classifiers. Besides, they propose a new training scheme based on their conjecture about safe spots for out-of-distribution detection which achieves state-of-the-art results on near-distribution outliers. 

Overall, the writing is clear and the idea is interesting. I think they have the following contributions: 

1. Propose a novel adversarial framework and motivate it by a real-world application (search engine example); 

2. Propose an effective algorithm to find safe spots; 

3. Show the usefulness of safe spots for adversarial robustness and out-of-distribution detection. 

However, I have the following concerns: 

1. I am wondering whether the safe spots actually exist. Based on the definition, the classifier should have the same predictions on all data points in the $\epsilon$-ball around a safe spot. So the classifier has certified robustness on the safe spots. But it might be hard to show that the classifier has certified robustness on the safe spots. Although they have shown that the attacks could not successfully find adversarial examples for the safe spots, it might be due to the “gradient masking” issues. Could the authors try the auto-attack proposed in [1] to see whether the classifier is actually robust on the safe spots? 

2. From their results, we can see that the safe spots don’t exist for naturally trained models. We need to use adversarial training to produce safe spots. But it is not surprising that adversarial training could produce safe spots. In fact, if we could solve the standard adversarial training objective optimally, then any natural images from the training distribution should be safe spots. Could the authors explain why we need to find safe spots other than the natural images in this case? 

3. If the classifier is not robust on the natural image $x_o$, and the defender finds a safe spot $x_s$ around $x_o$, then from the attacker perspective, why he could not first perturb $x_s$ to be $x_o$, and then perturb $x_o$ to find adversarial examples? If the attacker could not find adversarial examples for $x_s$, then he may try other attack strategies like using larger $\epsilon$ or other perturbation types. In such cases, the proposed defense framework may not work. Could the authors explain it? 

4. For safe spot-aware adversarial training, they mention that the training procedure is more computationally demanding than PGD adversarial training. Then they use targeted FGSM or k-step PGD towards the ground-truth label as a proxy to safe spot search. It is hard for me to understand what they exactly do in this part. Could the authors describe it in detail? Also, why would the safe spot-aware adversarial training be better than the standard adversarial training? I think standard adversarial training can also produce safe spots. Is it because the safe spot-aware adversarial training search for $x_a$ in a larger ball around $x_o$ ($B_{\delta+\epsilon}(x_o)$) than the standard adversarial training? Could the authors try standard adversarial training with a perturbation budget of $\delta+\epsilon$ to see if this is the case? 

5. For out-of-distribution detection, they conjecture that the samples from the learned distribution will have a higher probability of having safe spots compared to the out-of-distribution samples. But I don’t think their results could support this conjecture. In their training objective, they explicitly minimize the probability of safe spot existence of the outlier samples. So they try to train a model such that their conjecture holds. It would be better if they could show whether their conjecture holds for naturally trained models or the models trained using outlier exposure. I suggest they perform an ablation study for objective (4). Also, I think they miss some OOD detection baselines, such as [2] and [3]. Could the authors compare their method to them? 

 
[1] Croce, Francesco, and Matthias Hein. "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks." arXiv preprint arXiv:2003.01690 (2020).

[2] Mohseni, Sina, et al. "Self-Supervised Learning for Generalizable Out-of-Distribution Detection." AAAI. 2020.

[3] Liu, Weitang, et al. "Energy-based Out-of-distribution Detection." arXiv preprint arXiv:2010.03759 (2020).

--------AFTER DISCUSSION WITH AUTHORS---------

Thanks for the clarification. Some of my concerns have been addressed and I have raised my score. But I keep the concern that the proposed defense framework may be easily broken in practice given that the attacker can have unlimited power. 
review_title: A new adversarial framework for adversarial robustness

final_score: rating: 7: Good paper, accept
review_content: Thank you for your answers.

------

The paper proposes a new method for making adversarial attacks more difficult. 
In their method, the defender (not the attacker) modifies the original input $x_o$ to $x_s$ which is guaranteed to be safe in the sense that an attacker modifying $x_s$ will not manage to change the predicted class until large changes to the sample are performed.
The defender's budget for modifying the sample is denoted $\delta$, whereas the attackers budget is $\epsilon$.
After some relaxations, they arrive at the optimization problem stated in Equation (2):
Find the modification $x_s$ (subject to budget $\delta$) which minimizes the risk (measured by cross-entropy) that any modification of $x_s$ by an attacker (subject to budget $\epsilon$) is misclassified.
They also extend the idea to out-of distribution (OOD) detection, though the main contribution seems to be in mitigating adversarial attacks during testing.

Strong Points:

- the basic idea and derivation of the optimization problem is clearly written.
- the idea of modifying an input image before classification is interesting, apparently new, and effective in mitigating the impact of adverserial attacks (according to the experiments in 4.1, 4.2, 4.3).


Weak/Unclear Points:

- Section 3.5 "SAFE SPOT-AWARE ADVERSARIAL TRAINING" is a little bit unclear to me.
Is it that now the training data, not the test data is modified? But then it is not so clear to me, whether the final training objective is still well defined.
It might just have a similar effect as adding noise to training samples.
Furthermore, it appears that the whole thing becomes difficult to train, since during training it necessary to iterate between (A) ordinary model training and (B) modification of training samples.

- Section 3.6 "OUT-OF-DISTRIBUTION DETECTION" is not convincing:
Apparently the proposed method is not only hard to train, but also has three important hyper-parameters $\gamma$, $\lambda$, and $\mu$, which need to be carefully tuned. Therefore, even though the authors report improvements over previous methods in Section 4, I am not convinced that this is a practical approach to OOD.

- In Section 4.1, I am not sure what the authors mean with "our methods can find safe spots on over 85% of the
test set images". My understanding is that, if the class label could not be changed by an attacker, then the method was successful, even if the original sample was misclassified.
However, Table 1 reports only classification accuracy.

review_title: Interesting new idea for mitigating adversarial attacks, apparently effective, but possibly difficult to apply

Meta-review: The reviewers recognized that the proposed method is interesting and seems to be useful in some cases, and the authors provided sufficient empirical results to support their claim.
In addition, some comments have already been clarified.
However, some reviewers still concerned that the proposed defence method will be defeated under some conditions, and still have the major concern regarding the issue of adopting some attack strategies to find adversarial examples near the safe spots, even though the authors clarified some critical points of the proposed method. 
These drawbacks led to the decision to not accept. However, this paper has some merit and can be made into a stronger contribution in the future.

Decision: Reject