Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes a method for poisoning the train dataset of a victim classification model. Even if the poisoner does not know any information about the target victim model training procedure, it can still seriously degrade the classification accuracy of the victim model on clean data. This is done by replacing total or partial training data as corresponding adversarial examples (AEs) with ground truth labels. It is verified that AEs with ground truth labels indeed degrade the classification performance of models.    
main_review: Originality: This paper contributes some new ideas.
Clarity: The organization of this paper needs to be improved. Some experimental details are presented in the technical part, making it difficult to capture the core idea of the technical contributions.
Significance: The problem formulation of this paper is almost the same as that of the work [1]. Although this paper utilizes a surrogate model to approximately solve the problem, the work [1] attacks DNNs by back-gradient without any surrogate model, which is more challenging. Therefore, the novelty or improvement of the proposed work is rather limited.
Weaknesses and my questions about this paper:
1.	The experiment of this paper is not sufficient. Firstly, there is no comparison with other data poison methods, especially with [1], which is very similar to the proposed one. 
2.	This work utilizes existing attack methods on a surrogate model. It is similar to use the transferability of adversarial examples directly. The author needs to further claim the novelty and contribution of the proposed method.
3.	The proposed method might be invalid when adversarial detections are involved. More precisely, the defender can utilize existing detection methods, such as like LID[2], MD[3], and KB[4], to remove those poisoned examples. Thus, there should be some tests on evaluating the robustness of the proposed method against adversarial detections.
4.	The author has pointed out that their method performs unsatisfactorily to the defense of adversarial training techniques. In fact, such an limitation is fatal as  the adversarial training is not so expensive as the authors claimed. Some adversarial training method like FastAdv [5] improves the training speed significantly. 




limitations_and_societal_impact: NA
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper investigates adversarial examples as a data poisoning method and offers insights as to why they are effective. The idea of using adversarial examples for poisoning is motivated by the intractability of standard data poisoning (cast as a bi-level optimization problem) for neural networks. The paper focuses on _availability attacks_ where the goal of poisoning is to maximize the expected loss of the model. A suite of experiments demonstrates the effectiveness of adversarial examples for poisoning. Experiments include: comparisons with baselines on CIFAR-10 and a facial recognition application; testing attack transfer to different model architectures; testing effectiveness on ImageNet (as a large-scale setting); and testing effectiveness for different poisoning ratios. Seven defenses are also tested, and the only effective defense is found to be adversarial training. Experiments are also run to test the authors’ hypothesis that adversarial examples are effective because they contain discriminative features for the “wrong” class.
main_review: This paper is based on a very simple idea: seeing whether a well-known test-time attack (adversarial examples) is effective as a training-time attack. There is essentially no work required to adapt the attack to a training-time setting, however the authors do propose a novel _class targeted_  variant for which the class labels are permuted. Despite the simplicity of the idea – I see it as an obvious attack to try – it does not appear to have been investigated in the literature. The only prior work I could find is the following (not cited):

> CHAN-HON-TONG A. An Algorithm for Generating Invisible Data Poisoning Using Adversarial Noise That Breaks Image Classification Deep Learning. Machine Learning and Knowledge Extraction. 2019; 1(1):192-204. https://doi.org/10.3390/make1010011

Although the paper lacks new technical ideas, I think the empirical study is well done and offers important insights that ought to be shared with the community. I was surprised by the effectiveness of the approach, especially given its relative simplicity compared to existing approaches for poisoning neural networks. There are very few stones left unturned in the experiments – the attack is tested on multiple datasets (including at larger scales), transferability is assessed for different network architectures, defenses are tested, and there are further experiments testing the effect of various parameters in the appendices. It would have been interesting to see how the compute time varies for each type of attack, however this would be difficult to measure fairly on heterogeneous hardware.

For an empirical paper such as this one, I think it’s important that clearly documented scripts and code be made available to support reproducibility. I have not been able to carefully examine the repository included in the supplementary material, however the README in the root directory seems light on detail. 

limitations_and_societal_impact: Yes, this is covered adequately in Sections 3.1 and 5 of the paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors show that test-time adversarial examples, which are originally crafted to fool a well-trained model at inference time, can be used to construct strong availability attacks at training time. The resulting attacks achieve state-of-the-art results on multiple datasets and can successfully transfer across many settings, such as model initialization, architecture, learning rate scheduler, optimizer, etc.
main_review: #### **Originality**

My main concern about the paper is its originality. The idea, adversarial examples can make availability attacks, is not new to the community. It was first proposed by Nakkiran [1]. He simply predicted the success of the idea through permutation-symmetry of the labels, and showed its effectiveness with numerical results. For attribution in academic contexts, please cite his work [1] properly. 

Furthermore, the exact idea was also experimented in previous works, such as the error-maximizing noise attack in Huang et al. [2] and the P1 attack in Tao et al. [3], just with slightly different optimization hyperparameters. These previous efforts about the idea should not be ignored in the main text.

#### **Significance**

Since the idea is not new, the main insight I can tell from the manuscript is that adversarial examples can make "strong" availability attacks. Contradicting to the results reported by Huang et al. [2], where they concluded that the error-maximizing attack is inferior to their proposed error-minimizing attack, this work shows that the same error-maximizing noise attack can actually surpass the error-minimizing attack (6.25% vs 19.85% in Table 2).

I note that one possible crux is the number of PGD steps used for crafting poisons: [2] adopted only 20 steps for the error-maximizing attack, while this work executes 250 steps. From Table 10 in Appendix, it turns out that the number of PGD steps is crucial for achieving high performance. Therefore, I suggest the authors to clarify this earlier in the main text. This is an important point that differentiates this work from previous works.

#### **Quality**

Main claims are experimentally supported. However, neither a new method nor theoretical analysis is proposed.

#### **Clarity**

The writing of this paper is mostly clear and straightforward. A minor concern is that Section 3 is too long. It would be better to separate Section 3 into two sections, one describing the problems and methods, the other showing the experiments.

Reference:

[1] Nakkiran, https://distill.pub/2019/advex-bugs-discussion/response-5/

[2] Huang et al., https://arxiv.org/abs/2101.04898

[3] Tao et al., https://arxiv.org/abs/2102.04716


-----
After rebuttal:

I am glad to see that my suggestions helped to identify the contributing factors for attack effectiveness. It turns out that three strong factors emerge during the rebuttal:
- the variance of the attack effectiveness itself
- the use of differentiable data augmentation
- the number of PGD steps

I would like to suggest the authors to reflect these key factors (and other minor factors) more clearly in the main body. Surely, achieving SOTA performance itself is a good contribution, but for this venue, it alone may not be enough. In my opinion, another contribution of this work would be to sufficiently clarify why prior implementations of adversarial examples cannot achieve SOTA. Thus, when combining the two contributions, I would prefer this paper.

Overall, I am leaning to accept this paper on the condition that the authors can further clarify the confusion about the variance, the use of differentiable data augmentation, and the motivation for the choice of the number of PGD steps.
limitations_and_societal_impact: Although many limitations of the work are studied through experiments, I still have some suggestions, as listed below.

- It would be better if the authors could plot the data in Table 10 using curves, to show the trend of the decreasing accuracy as the number of PGD steps increases, with more fine-grained hyperparameters (e.g., plotting every 20 steps, up to 1000 steps). I'm curious about the limits of this poison.
- [2] and [3] showed that class-wise noises perform better than sample-wise noises, for both error-maximizing attacks and error-minimizing attacks. It would be better if this work can also consider the class-wise noises.
- Section 4 of the manuscript (and Figure 2) only explains why the targeted adversarial examples (i.e., Equation 4) work for poisoning. However, [1] provided a more concise explanation through permutation-symmetry, which directly implies the use of targeted adversarial examples as poisoning attacks.
- It remains unclear why the untargeted adversarial examples (i.e., Equation 3) are also effective (sometimes more effective than targeted adversarial examples as shown in Table 1). Since the originality of this work is limited, it would be helpful to provide a novel and insightful analysis like this.
- I note that, in Table 2, the results of UNLEARNABLE EXAMPLES and DEEPCONFUSE are different from those reported in their original papers. Please clarify this.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a data poisoning approach based on the adversarial attack technique. Specifically, to generate the perturbed samples, the poisoning model aims at maximizing the traditional cross-entropy loss or minimizing the loss with the wrong labels. Adversarial example generation approach PGD is adopted to generate the poisons. Experiments on several datasets show that training on the perturbed samples would significantly reduce the performance of various deep learning models.
main_review: The main contribution of this work is to leverage the idea of adversarial example generation for data poisoning. The perturbation generation is directly based on the existing approaches. The authors also conduct substantial experiments to show the effectiveness of the proposed approach.
limitations_and_societal_impact: As mentioned in the paper, the proposed approach assumes the knowledge of a pre-trained model to generate the perturbation, which makes the approach not very practical in the real world. Many attack strategies, such as backdoor attacks, usually assume to inject limited instances to training samples to damage the model.
review_title: null

Meta-review: This paper presents a technique	to poison datasets with	adversarial examples.
This an	interesting and	new direction, and the results are (often order of magnitude)
more effective than prior work that relied on much more	sophisticated techniques.

While the reviewers are	concerned about	the tone of the	paper over-claiming in
its originality (and I would encourage the authors to compare to the prior work
the reviewers have pointed out), the results are sufficiently strong to	merit
publication.

Decision: Accept (Poster)