Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper propose a new threat model called stability attack. The goal of stability attack is to hinder model from being robust to adversarial attacks. The author  proposes hypocritical perturbation as a method for stability attack and shows that hypocritical perturbation is harmful in terms of adversarial robustness in a simple gaussian mixture setting. Finally, the author shows that 2 adversarial training is enough for protecting stability attack.
strengths_and_weaknesses: 
Pros: 
- As far as I know, this is the first work that studies the robustness of adversarial training against train data poisoning called stability attack.
- As [1] noted, hypocritical perturbation is known to be a weak attack in terms of the standard accuracy, but this paper illustrates that it can be a strong attack in terms of the robust accuracy.
- The intuition of stability attack for vulnerability of adversarial attack is well demonstrated.
Cons:
- It seems that hypocritical perturbation is main content, but is not well described. Especially, the comparison of adversarial perturbation and hypocritical perturbation lacks.
- It seems that Stability Attack in Table 2 and Adversarial Poisoning and Hyp. and Adv. in Table (3, 4) are identical, respectively. The consistency of naming the attacks is necessary.
- The description of the poisoning methods in Table 2 is not given.
- In Table 3,  Adv. and Hyp are only considered for the methods of data poisoning. It would be more appropriate to do evaluation against the various attacks as in Table 2.
- The final message of the paper is that by using a larger perturbation size, the hypocritical perturbation can be defended. However, this message would be obvious and so the novelty would not be high. 

[1] Lue Tao, Lei Feng, Jinfeng Yi, Sheng-Jun Huang, and Songcan Chen. Better safe than sorry: Preventing delusive adversaries with adversarial training. In NeurIPS, 2021.
questions: I have no question.
limitations: - As mentioned in the manuscript, the standard accuracy of a robustly leanred prediction model based on training data poisoned by the stability attack is  better than the standard accuracy of a robustly trained model based on training data poisoned by other methods. In this sense, the stability attack is less serious than other poisoned methods which degrade the standard accuracy as well as the robust accuracy simultaeously.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents sability attack against the conventional adversarial training process, aiming to reduce the eventual robust accuracy of the resulting model. Specifically, the corresponding hypocritical perturbations are applied into training data as a training-time attack. Theoretical analysis is provided to support the idea of hypocritical perturbations. Experimental results on commonly used classification datasets like CIFAR-10 demonstrate the effeciveness of the proposed method.
strengths_and_weaknesses: # Strengths

1. This paper is technically sound and clear around the theoretical analysis. Experimental results are significant, which well support the theory.

2. The writing quality of the paper is good overall. Specifically, the background of problem is smoothly introduced.

3. The proposed method is sufficiently evaluated. To be specific, key attacks like FGSM, PGD, CW, and AutoAttack are present for robustness evaluation. Meanwhile, the experiments cover four datasets, namely CIFAR-10/100, SVHN, and Tiny-ImageNet, which are sufficient to demonstrate the effectiveness for the proposed method.

4. The proposed method successfully compromised adversarial training methods. However, countermeasure (adaptive defense) is also proposed and evaluated.

# Weaknesses

1. Overall this is a good paper and I did not find many problems.
questions: My questions are summarized in the weaknesses.
limitations: Limitations are discussed as pointed out in the checklist.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper introduces the problem of adversarial training when they face the new type of attacks called stability attacks. The stability attacks aim to compromise the robust availability by slightly manipulating the training data. Most of existing methods neglect that the test robustness of the adversarial trained models when they are under the training-time availability attacks. Under the threat of the stability attacks, they demonstrate that the adversarial trained network with epsilon perturbation budget is not enough to defend against the epsilon bounded adversarial perturbation. The authors argue that it is necessary to enlarge the epsilon perturbation budget when they conduct the adversarial training.
strengths_and_weaknesses: Strengths :
1. Clear writing. Easy to understand and well-organized paper.
2. Important experimental result. The test-robustness of adversarial trained network against evasion attacks when they are under the delusive attacks is intriguing result in the adversarial research.

Weaknesses : 
1. Low originality: Missing comparison with the recent key reference [Ref_1] which is somewhat similar method of the training-time availability attacks. The attack generation algorithm of training-time and test-time perturbation is similar to [Ref_1]. Furthermore, [Ref_1] achieved the state-of-the-art attack performance against adversarial trained network on clean data. Thus, it is unclear that this paper made the fair comparison between current SOTA poisoning attacks on adversarial trained network. In this regard, the novelty of the threat model on the adversarial trained network by perturbing the training data is marginal.
2. Insufficient explanations on the relationship with non-robust features. This paper passes the buck to the non-robust feature for the experiment results of the increase of standard accuracy and the decrease of robust accuracy when they are under the stability attacks. The authors emphasize the responsibility of the non-robust feature in the title, abstract, and throughout the paper. However, I am not convinced that non-robust feature is the only reason for the experiment result in Table 2. Natural test robustness has increased and test robustness under evasion attacks has decreased when they are adversarially trained under the stability attacks. But, considering the fact that there always exists a trade-off between the standard accuracy and the robust accuracy [Ref_2], non-robust feature can’t be solely blamed. As the authors followed the process of theoretical analysis with the [Ref_3], they need to present another empirical evidence, feature-level analysis, or visualizations for the explanations on relationship with non-robust features as [Ref_3] did.
3. Confounding usage of the term for the similar concept. It is very confused when the term ‘Hyp’, ‘stability attacks’, and similar concept appears throughout the paper. 

[Ref_1] Fu, S., He, F., Liu, Y., Shen, L., & Tao, D. (2021, September). Robust unlearnable examples: Protecting data privacy against adversarial learning. In International Conference on Learning Representations.

[Ref_2] Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., & Jordan, M. (2019, May). Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning (pp. 7472-7482). PMLR.

[Ref_3] Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Madry, A. (2018). Robustness may be at odds with accuracy. arXiv preprint arXiv:1805.12152.

questions: According to the paper, stability attack consists of training-time perturbation and test-time perturbation in Table 1. In Table 2, the authors compare the stability attacks with other training-time availability attacks (DeepConfuse, Unlearnable Examples, NTGA …). However, the training-time availability attacks have training-time perturbation only. Therefore, the test-time perturbation of FGSM, PGD-20, PGD-100, CW, and AutoAttack in Table 2 should be different for the stability attacks and for other training-time availability attacks. What is the exact experiment setting for the test-time perturbation under the threat of stability attacks?
limitations: Yes, the authors have addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: ### Summary:

This paper introduces a novel data poisoning attack against adversarial training called *stability attacks*.
The goal is to temper the training data such that the robust performance of adversarial training over this manipulated dataset is degraded.
To construct this attack, a *hypocritical perturbation* is built: unlike *adversarial perturbations*, the aim of *hypocritical perturbations* is to reinforce the non-robust features in the training data.
These perturbations can be generated by negating adversarial example generation objectives.

**Motivation:**
The paper motivates stability attacks from the perspective of robust vs. non-robust features.
Specifically, a simple binary classification task over a mixture of Gaussians is considered.
Statistical analysis on this task shows that adversarial training over *hypocritically perturbed data* is destructive to adversarial robustness.
Moreover, it is shown that a larger perturbation magnitude is needed to guard adversarial training against stability attacks.

**Implementation:**
The effectiveness of stability attacks against adversarial training is demonstrated through extensive experimental results.
strengths_and_weaknesses: ### Strengths:
- The paper is clear, and it guides the reader skillfully.

- The paper is well-motivated.
The statistical analysis of the binary classification task is thorough, and the implications of the theoretical results are discussed comprehensively.

- This paper sheds light on the implications of robust vs. non-robust features from a novel perspective and utilizes these studies to introduce a new threat against adversarial training.

- The experimental settings are discussed in detail, and the effects of different hyper-parameters and architectures on the performance are investigated.

### Weaknesses:
- While the theoretical justifications of *hypocritical perturbations* on the binary classification task are discussed, the relationship of these results with the attack generation process (Eq. (10)) is obscure.
Although the given example for the logistic loss and the binary classification task is appreciated, the origins of the objective function in Eq. (10) need a better justification.

-  Furthermore, a thorough discussion on the relationship of this work with existing works on the trade-off between the clean and robust accuracy of neural networks with adversarial training seems missing.
As the experimental results suggest, the implications align with the observations of Tsipras et al. [63] on the trade-off between the clean and robust accuracy (e.g., see Table 4).
From this perspective, it seems like stability attacks are somehow just exploiting this trade-off to pose their threat on adversarial training.
Thus, a comprehensive discussion on the differences between this work and prior work in this area is required. 
questions: ### Questions:
As mentioned above, there are a few questions:

1. How the *hypocritical perturbation* objective is derived? Are there any other objectives that can be used?
2. What is the relationship between this work and prior work on the trade-off between clean and adversarial accuracy?
3. While increasing the perturbation bound would help combat stability attacks, how can one maintain the clean accuracy as the current solution jeopardizes this?
limitations: A potential discussion on the real-world negative impacts of the current work is missing. This reviewer would encourage the authors to discuss this matter explicitly.
review_title: null

Meta-review: metareview: This paper proposes a new threat model called stability attack, which aims to hinder model from being robust to adversarial attacks. The author proposes hypocritical perturbation as a method for stability attack and shows that hypocritical perturbation can indeed decrease the adversarial robustness of a model trained in a simple gaussian mixture setting. The reviewers agree that the problem being studied is interesting, the proposed method is well motivated, and the experiments are mostly convincing. The authors are encouraged to merge the new results during the rebuttal into the publication and discuss more on the efficiency of the proposed method. 
confidence: Certain
Decision: Accept