Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper address the problem of adversarial training for delusive attacks with perturbed yet correctly labeled training data. By characterizing delusive adversaries as maximizing the natural risk with perturbed training data (worst-case data) within a specific Wasserstein ball around the original data, the adversarial training (e.g. in [7]) is shown to minimize an upper bound of such adversarial risk (Theorem 1) and hence is principle to defend against such delusive attacks. Furthermore, some settings of perturbing non-robust features are given and discussed to identify several examples of delusive attacks and show how adversarial training is able to achieve lower natural risk than standard models (Theorem 3). To my understanding, the main contribution is the formal analysis of the performance of adversarial training in defending the delusive attacks, with the new metrics employed such as the Wasserstein distance to generate the perturbed data.
main_review: The paper is overall well written in presenting the theoretical analysis of adversarial training against delusive attacks. However, my main concern is in the significance of the contribution in the paper. It seems the main idea is to characterize the delusive attacks using Wasserstein distance as done by [30], and then show the existing adversarial training formulation (e.g. [7]) is also effective in resisting such delusive adversaries by actually minimizing the worst-case adversarial risk. While the results are interesting, I am not sure if such analysis is significantly novel given that no new adversarial training theory nor attacks are proposed to advance the current state of the art adversarial training algorithms, but to verify the performance in different scenarios. To help readers better appreciate the contribution, authors are  encouraged to provide an explicit statement of contribution and specify any actual contributions presented by the theorems.

On the other hand, I have some questions regarding the generality of the adversarial training in resisting against delusive attacks. While Section 3.2 provides some examples and settings to justify how adversarial training will naturally prevent the learner from relying too much on the non-robust features, I was wondering if such training process is able to deal with other classification problems beyond the example of a mixture of two Gaussian distributions? Besides the given binary examples, it would be helpful to give additional discussions on how generalizable the obtained theorem 3 is, e.g. to non-binary classification problems with non-Gaussian data?

Some other questions:

- It is mentioned in the paper that while adversarial training usually would negatively affect its performance on clean data without attacks, the adversarial training using Wasserstein ball is able to achieve good performance even without adversaries. However, I am not able to find explicit discussions to justify this claim. Is this concluded from Theorem 1 that the adversarial training is also minimizing the upper bound of “natural risk” besides the “adversarial risk”, and hence standard training objective is also implied?

- While this paper deals with adversarial training, the reported results in Fig. 2, 5, 6 and 7 do not contain any performance variance information. In order to further justify the results and performance, authors are strongly suggested to provide more quantitative results such as the number of trials and performance variance besides the average results.

limitations_and_societal_impact: The authors have addressed some limitations of the work, but the reviewer has some questions regarding other limitations such as the generality of the adversarial training in classification problems other than the discussed binary classification task with Gaussian data. Authors are suggested to add more discussions to address the comments above.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper shows adversarial training (AT) can resist delusive attacks naturally. 

AT can naturally prevent the learner from overly relying on non-robust features. 
main_review: 1 May I know what the main difference between delusive attacks and poisoning attacks is? 

2 A piece of work [1] has shown that adversarial training may intensify the backdoor attack. 
Would the claim contradict the finding in [1]? 

3 Have authors tried different epsilon_train budgets for adversarial training in defending delusive attacks?

4 AT has a very strong smoothing effect [2], which can smoothen out the non-robust adversarial features. 
If AT encounters robust adversarial features, it may strengthen it.

[1] On the trade-off between adversarial and backdoor robustness, in NeurIPS 2020

[2] Understanding the Interaction of Adversarial Training with Noisy Labels.
limitations_and_societal_impact: See above.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper tackles the problem of delusive attacks. It reveals that adversarial training, traditionally considered as one of the best defense methods against adversarial examples, can serve as a powerful and elegant defense against delusive attacks. This expands the defense ability of adversarial training from test-time evasion attacks to training-time availability attacks, making the technique significantly more applicable in practice.
main_review: ### Originality

The paper pioneers the use of adversarial training to mitigate delusive attacks. It shows that adversarial training on perturbed examples minimizes an upper bound of natural risk on clean examples. Theoretical analyses also reveal the internal mechanisms of the defense method. As far as I know, these points have never been theoretically justified before.

### Quality

The paper is technically sound and is a solid contribution to the community. Theoretical results are novel and experimental results are remarkable. Both results indicate that adversarial training is a promising solution to defend against delusive attacks.

### Clarity

The writing of this paper is overall clear and straightforward. The experiments are thorough and well explained. 

A few minor observations: line 106: "Eq. (3)" -> "Eq. 3", line 128: "good" -> "a good", line 141: "this case" -> "these cases", line 199: "varying d" -> "vary d", line 140: "changes of" -> "under", line 300: "and, show" -> ", and show", line 354: "optimize" -> "optimizes".



### Significance

1. The paper solves an important and challenging problem of defending against delusive attacks. It provides a novel way to utilize all training data in an elegant manner, rather than abandoning the correctly labeled examples. I believe that many practitioners will be glad to adopt the defense method when their training data is collected from untrusted sources. After all, it's better to be safe than sorry.
2. It is well-known that adversarial training usually leads to a drop in natural accuracy. The paper shows that adversarial training instead can improve the test accuracy on clean examples in the settings of delusive threat models. I found this phenomenon very intriguing. At the same time, this provides a new and convincing argument for the application of adversarial training in many real-world applications.
3. Extensive experiments validate the superiority of the proposed method. The authors also show the versatility of the proposed defense with several experiments; in particular, the authors show the possibility of delusive attacks and defense for rotation-based self-supervised learning. I thought this would inspire researchers to explore the vulnerabilities of SSL and find countermeasures. I also noticed some orthogonal works [1] [2] in this direction that you may find helpful.

### Experiments

1. The authors mention that their implementation of adversarial training minimizes a lower bound of adversarial risk (they adopt the popular PGD-AT method). Why not adopt SOTA certification methods such as CROWN-IBP [3] or Linf-dist Net [4] to directly optimize the adversarial risk?
2. There is still a small gap between the natural accuracy of a standard model trained on the clean data and that of a robust model trained on the perturbed data. Is the gap inevitable?
3. The authors may want to discuss the computation cost of adversarial training compared to standard training.
4. From Figure 5, it turns out that the optimal budgets for P1/P2 and P3/P4 are different. Why is this case? The authors may want to give a reasonable explanation.


[1] Saha et al. Backdoor Attacks on Self-Supervised Learning. 2021

[2] Carlini et al. Poisoning and Backdooring Contrastive Learning. 2021

[3] Zhang et al. Towards stable and efficient training of verifiably robust neural networks. ICLR 2020

[4] Zhang et al. Towards Certifying $\ell_{\infty}$ Robustness using Neural Networks with $\ell_{\infty}$-dist Neurons. ICML 2021
limitations_and_societal_impact: Many limitations and negative aspects are discussed in the paper.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a defense method based on adversarial training to defend against delusive adversaries at the training time. The paper shows that adversarial training can largely improve the accuracy under delusive attacks if the defender applies pessimistic budget. Both experiments and theory verify the claims. 
main_review: **Pros**
1. The delusive attacks are indeed a threat to the training of networks. And adversarial training can largely improve the accuracy under the attacks.
2. The analysis and the theory of the methods are good and reasonable, which point out why adversarial training improves the accuracy.
3. The paper is clearly written and easy to follow.

**Cons**
1. It has been pointed out in [1] (Section 4.3, Under Noise Filtering Techniques) that adversarial training can largely improve the accuracy of delusive attacks. Although that paper does not carefully analyze the phenomenon as this paper, it limits the contribution and novelty of this paper.
2. One biggest drawback of the method is the choices of adversarial budget $\varepsilon$. In Figure 5, the accuracy seems to be sensitive to choice of $\varepsilon$ of the defender. Another question is the generalization across different attack norm. If the attacker chooses attack in $\ell_2$ norm and the defend chooses to defend in $\ell_\infty$ norm, will the accuracy still be improved? The transfer across different attack norm is also a big problem in adversarial training.
3. The analysis is based on the Wasserstein distance. However, the experiments seems to be performed in single $\ell_p$ norm ball, which does not allow different perturbations budget for different training samples.


Overall, I think the paper proposes a good way to prevent delusive attacks. I vote for rejection beacause of 
1. Lack of novelty mentioned in point 1 of Cons senction.
2. It also cannot solve more practical problems for different norms in point 2 of Cons section.

I would increase my score if both of the concerns can be resolved.


[1] Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, Yisen Wang. Unlearnable Examples: Making Personal Data Unexploitable, ICLR 2021.
limitations_and_societal_impact: See above.
review_title: null

Meta-review: The paper analyzes in a particular context how adversarial training can help with preventing delusive adversaries affect training. Delusive adversaries are defined to be those that may modify training data (feature, not labels) to make the resulting classifier have bad test error. The analysis is specifically in the case of Gaussian noise. There is also experimental evidence provided. This paper resulted in a long exchange between reviewers and authors and the reviewers turned increasingly positive about the paper. The authors must include all of the additional details that have been included in the discussion in the next iteration of the paper.
Decision: Accept (Poster)