Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper adapts adversarial training to build robust models against data poisoning. Specifically, the paper shows how one can effectively apply data-poisoning attacks in the training loop to successfully defend against potential data poisons. The introduced method is rigorously evaluated by (adaptive) known poisoning attacks. The paper demonstrates that the new defense outperforms existing data-poisoning defenses on CIFAR-10. Finally, the paper qualitatively analyzes the feature space to understand the effect of the proposed defenses.
main_review: ### Strengths
- The introduced method is sounds and a simple adaptation of adversarial training to make it more effective against data poisoning attacks.
- The paper is well-written and flows smoothly
- The authors put decent effort in detailing prior work and replicating their results within the same framework to foster reproducibility. The authors attach the code to the submission.
- The experimental setup is detailed in the appendix.

### Weaknesses
- The analysis section S4 and the corresponding qualitative results (Fig 2) are hard to follow and understand. What is the takeaway in this section? Making this clear in the paper/captions is important in my opinion.

### Small changes:
- Typo: “Defineda” -> “defined”
- Fig 2 captions are almost overlapping. Try to increase the space between the subplots for better readability.

summary_of_the_review: Overall, I believe the paper is good in that it introduces a simple yet effective adaptation of adversarial training to make it effective against data poisoning. The presented empirical results are convincing. Though, I don’t quite understand the qualitative assessment of the introduced defense, so it would be great if the authors can improve the presentation there.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to leverage classical adversarial training to defend against data poisoning attacks. Specifically, the paper splits each training batch randomly into two sets: one set for generating the trigger pattern and the other for injecting the pattern. The subject model is then trained on the modified batch. This paper evaluates the proposed training method on a few $L_\infty$ based backdoor attacks and shows that it has a better balance between normal accuracy and defense performance compared to baselines.
main_review: It is interesting to extend adversarial training to defending against data poisoning attacks. I reviewed this paper before and the following problems have not yet been addressed.

1. The key of the proposed training is based on the expectation that the generated poisoned data can counter the effect from the real injected data. However, this relies how many data are polluted in the training set. If only a very small portion is injected, then it is possible that the proposed training may eliminate the effect from poisoned data. What if there is a much larger portion of the training set poisoned, say 50% or even 80%? Can the proposed training still reduce the attack success rate without sacrificing normal accuracy? For those poisoning attacks that change labels of poisoned data, the chosen label $y_t$ in Algorithm 1 may still be the true target label, which does not counter the original effect of poisoned data. The current version of the paper does not seem to study the impact of different poisoning rates on the proposed approach.

2. The results show the proposed training is effective against $L_\infty$ but not against $L_0$ (in Appendix D.4). This may be due to the approximation of generating trigger patterns during training. The paper only uses 5 steps to generate those patterns. This is useful for $L_\infty$ attacks as it is more like classical adversarial training. As long as the perturbation is pointing towards the target direction, the added perturbation can help the model to be insensitive to those types of perturbations. However, for patch-like attacks, a small number of steps may just generate some random pattern and do not help the training. The results on the semantic patch is poor by the proposed approach. The best result is achieved using image patch, which is actually the baseline CutMix. This means the proposed method is no better than the baseline. This limitation should be discussed early in the paper, preferably in the introduction. Or the authors can adjust their threat model and just focus on $L_\infty$ attacks. The introduction of the current version does not clarify the limitation of the proposed method regarding the threat model. The evaluation on $L_0$ attacks does not include more studies.

3. Generating $L_\infty$ poisoned data during the proposed training procedure is similar to crafting universal perturbations. How about using universal adversarial training [1] to defend against data poisoning attacks? This is not investigated in the current version.

4. The evaluation is only conducted on one model (ResNet-18) and one dataset (CIFAR-10). The observations and experimental results may not be general and applicable to other cases. Including more model structures (e.g., VGG, Inception, etc.) and datasets (GTSRB, ImageNet or its subset, etc.) can provide a better understanding of the performance of the proposed training method. There is a public dataset of poisoned models called TrojAI dataset. The original training datasets and the code for generating those models are public available. An extensive evaluation on a larger benchmark can better assess the performance of the proposed approach.

5. There are missing critical results. The accuracies on clean data are missing in Table 1 and Table 2.

6. The writing of the paper needs improvement. This paper categorizes backdoor attacks into backdoor trigger attacks and targeted data poisoning, which is good. But then the paper lists 5 attacks that are evaluated in the experiment. The two parts are not well organized. I would suggest to put the relevant attacks in the corresponding categories and then elaborate their similarities and differences. Subsections can be used to highlight different parts of the related work. For instance, use subtitle "poisoning attacks" to indicate the discussion on attacks and "poisoning defenses" to indicate the discussion on defenses. The references to figures and tables are not consistent. The paper uses "Figure 2a" on page 6 but uses "fig. 2c" on page 7. The paper directs the readers to the supplementary material on page 7 but does not point out the specific section.

[1] Shafahi, Ali, et al. "Universal adversarial training." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 04. 2020.
summary_of_the_review: 1. There is no study on the impact of different poisoning rates.
2. The method is only effective against $L_\infty$ but not $L_0$ and it is not clarified in the introduction.
3. A baseline defense method is not evaluated in the paper.
4. The number of evaluated models and datasets is small and limited.
5. The writing needs improvement.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a defense against backdoor attacks and targeted attacks. Specifically, the proposed defense injects randomly chosen poisoned instances and targeted instances into the training data, and then forces the model to have right behaviors on these instances. Experimental results demonstrate the effectiveness of the proposed method.
main_review: Strengthes:

1. The proposed training objective for poison immunity is reasonable and well-motivated.
2. Experiments are thorough. Many baselines are compared. Results are promising.

Questions/Weaknesses:

1. Is the proposed defense objective in Eq. (4) a natural generalization of the original adversarial training objective in Eq. (1). In other words, is Eq. (1) a special case of Eq. (4)? The authors are suggested to discuss this. If it is not the case, the wording of some parts of the paper, such as the title of Section 3 (i.e., "Generalizing Adversarial Training to Data Poisoning"), may need to be modified to avoid misunderstandings.
2. There are some closely related papers [1, 2, 3], which the authors may want to discuss. In particular, [1] proposed "an adaptive defense", where they can teach a model to correctly classify both unperturbed and perturbed pictures generated by Fawkes and LowKey. Their procedure shares some similarities with this paper. [2] also extended the use of adversarial training for defending against one type of poisoning attacks (they called delusive attacks). Note that those papers are largely concurrent to this paper, so they would not affect the contribution of this paper. Yet, elaborating on the differences from those papers would increase the integrity of this paper.
3. A minor typo. A punctuation mark is missing after "those of clean data" on the third-to-last line on the second page.

[1] Radiya-Dixit et al., Data Poisoning Won’t Save You From Facial Recognition. ICML Workshop, 2021.  
[2] Tao et al., Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training. NeurIPS 2021.  
[3] Li et al., Anti-Backdoor Learning: Training Clean Models on Poisoned Data. NeurIPS 2021.
summary_of_the_review: Overall, I am leaning towards acceptance. The proposed defense outperforms many existing defense strategies including differentially private SGD, adversarial training, various data augmentations, and filter defenses, providing a strong trade-off of robustness and accuracy.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper propose to use the idea of adversarial training to defend data poisoning. The idea is interesting and the numerical results 
also demonstrate the promising performance of this algorithm.
main_review: This paper is clear and easy to understand. The idea is interesting and the arguments are convincing. However, since it is some kind of combination of two existing robust models, I give weak acceptance based on the novelty. It would be great if the authors can provide more evidence on the novelty of this algorithm, e.g. what difficulties you met when designing this algorithm.

Below are some other minor comments:

(1) Could you provide some insights on why adversarial training performs worse than the proposed method in Figure 3? For example, what attack does adversarial training tend to defend, and what is the formula of the attack used in Figure 3?

(2) After adding adversarial training idea, the algorithm involves three levels of optimization. The adversarial training is known to be time consuming, so I expect the proposed algorithm takes an even longer time to run. Please report the running time of your experiment.

(3) It takes me several minutes to understand the layout of Figure 2: there are four groups (a) to (d) and each group contains two subfiguires. Please enhance the display of Figure 2. At the first glance I was thinking there are two groups (upper and lower), and the title of the upper group is "(a) Undefended model, clean (left) and fine-tuned on (b) Defended model, clean (left) and fine-tuned on".


summary_of_the_review: This paper is clear and easy to understand. The idea is interesting and the arguments are convincing. However, since it is some kind of combination of two existing robust models, I give weak acceptance based on the novelty.
review_title: null

Meta-review: Reviewer E8z9 remained with a number of serious concerns, including efficacy of the defense in higher poisoning setting, overclaiming contributions in terms of L0 defenses (which are mostly achieved by the baseline CutMix), as well as novelty and generalizability of the approach. The author responses were unconvincing, and all other reviewers participated in the discussion, conceding that they too were unable to provide compelling arguments against E8z9's comments. Other reviewers claimed that these drawbacks may be "acceptable" for a first step, but were not willing to defend it very strongly. 

We note that E8z9 claims they are a reviewer for a previous version of the paper and these issues were present before. The authors claim that E8z9's key points had been addressed in this version, but the reviewer maintains that the issues still persist in the latest version. The authors are advised to take their comments into account for further versions of this paper.
Decision: Reject