Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Summary:
The paper studies the phenomenon of trade-off between robust and standard accuracies that is usually observed in adversarial training. Many existing studies try to understand this trade-off and show that it is unavoidable. In contrast, this work shows that under a sensible definition of adversarial risk, there is no trade-off between standard accuracy and sensible adversarial accuracy. It is shown that Bayes optimal classifier has optimal standard and sensible adversarial accuracies. The authors then go on to propose a new adversarial training algorithm which tries to minimize the sensible adversarial risk. Experimental results show that models learned through the proposed technique have high adversarial and standard accuracies. 

Comments:
1) Sensible Adversarial Risk: The first contribution of the work is to define a new notion of adversarial risk which the authors call ''sensible adversarial risk'' and study its properties.  There is a recent work[1] which also proposes a new definition of adversarial risk and which is similar in spirit to what the current work tries to achieve.  In [1], the authors define a perturbation as adversarial only if it doesn't change the label of the Bayes optimal classifier, which is similar to what the current paper does. Owing to this similarity, the properties of sensible adversarial risk obtained in Theorems 1,2 in the current work look similar to [1]. So the authors should discuss/compare their results with [1].

2) Sensible Adversarial Training: I believe the major contribution of the paper is to propose an algorithm for minimizing sensible adversarial risk. However, I have some concerns with the proposed algorithm. The authors say that since the Bayes optimal classifier is unknown, they use a reference model f_r (which can be naturally trained). Consider the following scenario. Suppose the true data is given by (x, f^B(x)), for some unknown f^B; that is, the Bayes optimal classifier has perfect standard accuracy. Suppose f_r has perfect standard accuracy, but very bad adversarial accuracy on train and test sets. Suppose f_r is substituted for f^B in the sensible adversarial risk (with 0-1 loss). Then it is easy to see that f_r is a minimizer of the resulting objective. So the proposed algorithm will just output f_r in this scenario. This is clearly not desirable. Given this, I believe a more thorough understanding of the proposed algorithm is needed. When will the algorithm converge to non-robust classifiers? How should one initialize the algorithm to avoid such undesirable behavior?

While the notion of sensible adversarial risk is sensible, it is not clear why it should result in such high adversarial accuracies as reported in Tables 1,5. The adversarial perturbation of epsilon=8/255 on cifar10 is considered so small that sensible adversarial risk (with reference model f^B) at any point will almost always be equal to the existing notion of adversarial risk at that point. So, minimizing sensible adversarial risk (assuming you are given f^B) is exactly equivalent to minimizing adversarial risk. But it is known that minimizing adversarial risk results in models with low standard accuracies. So I feel sensible adversarial risk is not the reason behind such high accuracies. Could the authors explain what is the reason for such good adversarial accuracies reported in the paper?

3) Experiments:  While the experimental results on cifar10 look impressive, I have some concerns about the way the PGD attacks are run. I downloaded the model provided by the authors and ran PGD attack on it.  I ran L_infty attacks with epsilon=8/255, step size=2/255. The results I obtained seem to differ from the results presented in the paper:
PGD Steps | Adversarial accuracy of SENSE
20            62.09
50            60.34
100           59.99

I believe PGD attack with multiple random restarts will reduce the adversarial accuracy even further. Given this, I'd appreciate if the authors perform more careful attacks (with appropriate hyper-parameters) on their model. It'd also be great if the authors report the performance of PGD trained model using the same attacks used to report the performance on their model. 

4) Other comments:  I'm not sure if the toy example (cheese hole distribution) in Section 2 is helpful. What the authors seem to conclude from it is that adversarial training can improve standard accuracy. But I do not agree with these conclusions. What if Figure 1b is the true distribution? Will the same conclusions hold? In general, these toy examples need not be illustrative of the behavior on real datasets. So instead of having these toy examples, I'd suggest the authors have a thorough discussion on theoretical and experimental results.   

[1] Suggala, A. S., Prasad, A., Nagarajan, V., & Ravikumar, P. (2018). Revisiting Adversarial Risk. arXiv preprint arXiv:1806.02924.
review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: The paper proposes the notion of a "sensible" adversary that does not perturb data points on which the Bayes-optimal classifier is incorrect. The authors then provide theory showing that minimizing robust risk against such a sensible adversary yields the Bayes-optimal classifier, which addresses the question about standard vs. robust risk posed in prior work. On the experimental side, the authors then introduce a simple yet effective variation of adversarial training / robust optimization. Instead of maximizing the loss over the perturbation set, the proposed variant stops as soon as the loss exceeds a certain threshold. This can be seen as a variant of gradient clipping that reduces the influence of examples with a very high loss. The authors show that their modification yields an 8 - 9% improvement in robust accuracy on CIFAR-10, which gives state-of-the-art performance.

I find the empirical improvements achieved with their modification of PGD-style adversarial training very interesting and recommend accepting the paper. However, it is not clear to me how well the theory is connected to the empirical findings. Moreover, there are additional experiments the authors can conduct to investigate the performance of their method more thoroughly. Concretely:

- The theory relies on the "reference model" f_r being the Bayes-optimal classifier, while the experiments use the current model as reference model. Especially early in training, the current model performs significantly worse than a Bayes-optimal classifier. Moreover, it is unclear if the proposed training modification is effective if the reference model f_r is a separate classifier. It would be interesting to use a separately trained CNN (standard training without any robustness interventions) as a reference model to see if the training modification still yields improvements.

- If the improvements of the proposed method come from the loss of a few adversarial examples dominating the overall loss in a batch, it would be interesting to measure and plot the loss distribution over examples in a batch with experiments.

- As a baseline, comparing the proposed approach to clipping the loss or gradients of each example would be interesting.

- In the robustness evaluation, have you experimented with randomly-restarted PGD?

- To ensure that PGD works as intended, it would be helpful to see a plot of PGD iteration vs. adversarial accuracy.

- It would be interesting to see accuracy numbers (standard and robust) for models trained with different values for the parameter c. This would also provide information about the sensitivity of this hyperparameter.


Further comments:

- I encourage the authors to release their code and pre-trained models in a format that is easy for other researchers to build on (e.g., PyTorch model checkpoints).

- Page 3, "Note that R_rob(f^B) = P(X_1 [...]" - should this be X instead of X_1?

- Line 9 of Algorithm 1: should the sum go from 1 to m?

- Equation 5: is "x <= log 1/c" in the subscript a typo?

- Page 8: "Our model achieves 91.51natural accuracy.": percent symbol and space missing
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Motivated from the so-called trade-off between robustness and standard accuracy in the existing adversarial learning, this paper has proposed a "sensible" adversarial example framework without losing  significantly  performance in natural accuracy. Some toy examples have been presented, showing its reasonableness of the model. The proposed algorithm looks very simple, but it appears that it could be effective through some experiments on two data sets.

Though the reviewer did not fully understand all the details, the proposed idea  seems reasonable. In general, the paper chose those adversarial examples that won't change the predication class. Such adversarial examples, called sensible adversarial examples, mean the perturbation which may not mislead the decision boundary.

There are two concerns with the paper from the reviewer.

(1) The paper states that there is always a trade-off between between robustness and standard accuracy in the adversarial learning work, this seems arguable to the reviewer. In some adversarial example learning literatures, the adversarial learning appears able to improve the performance of both natural examples and adversarial examples.  For example, the VAT approach published in PAMI. The reviewer would like to see more comments about this.

(2) Though there are some theoretical analysis in the paper, the empirical validation may not be very convincing. MNIST and CIFAR 10 are relatively easier datasets, I am not sure if a same observation can be attained when the algorithm is applied on some more complicated and challenging dataset. It remains unclear to me if the "sensible" way could indeed be consistently useful in practical and more challenging cases.

review_title: Official Blind Review #2

Meta-review: Thanks for your detailed feedback to the reviewers, which clarified us a lot in many respects.
However, there is still room for improvement; for example, convergence to a good solution needs to be further investigated.
Given the  high competition at ICLR2020, this paper is unfortunately below the bar.
We hope that the reviewers' comments are useful for improving the paper for potential future publication.
Decision: Reject