Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This paper claim that poisoned data should lie around the sharp loss regions. Therefore, this paper proposes to add "friendly" noise to drift poisoned data away from the sharp loss regions.
strengths_and_weaknesses: ++ I think it is practical to post-process the training set (e.g., removing the adversarial or noisy data) before feeding it to the learning algorithm. 
Falling in this catogory, this paper proposes adding "friendly noise + random noise" to each training data before updating model parameters.

++ The friendly noises reinforce and highlight salient (robust) features of each training data, suppressing those unnoticeable (non-robust) features that could subvert the predictions. Due to this property of friendly noise, I suspect it will make the triggered-based backdoor attacks even worse because the triggers themselves are salient features.

--A prior study exists that "Anti-Backdoor Learning: Training Clean Models on Poisoned Data" in NeurIPS 2021. 
The prior study seems to defend against poisoning attacks without degrading the training performance. 
Are there empirical comparisons between the friend noise with those effective and recent prior studies?

--This is a very strong claim that "friendly noise" makes it extremely difficult for an adaptive attacker to break the defense. 
The empirical evaluations in this paper are only based on some weak poisoning attacks, such as clean-label targeted/transfer attacks. I encourage the authors to evaluate the fixed-designed friendly noise on the stronger untargeted attacks, such as unlearnable examples and backdoor triggers.

--The claim "powerful poisons fall in the sharp loss regions" is unsubstantiated. First, the loss should be the function of the model, data, and label. What is the exact meaning of that "powerful poisons fall in the sharp loss regions"? Second, the insights and arguments in Section 3.1 are under-supported. For example, why cannot the learning learn and adapt the poisoned data and make the loss smaller and flatter around the poisoned data?
questions: 1 There exist (robust) unlearnable examples [1][2], which can degrade the learning performance. 
Could the friendly noise make the (robust) unlearnable example learnable?
[1]UNLEARNABLE EXAMPLES: MAKING PERSONAL DATA UNEXPLOITABLE, in ICLR 2021
[2] ROBUST UNLEARNABLE EXAMPLES: PROTECTING DATA AGAINST ADVERSARIAL LEARNING, in ICLR 2022

2 Could the poison attacker also design adaptive poisoning attacks that make the fixed-designed friendly noise ineffective?

3 Suppose the training set is clean and benign, would the friendly noise degrades the training performance?
limitations: No. This paper did not discuss the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes to defend against poisoning attacks by adding specific friendly noise to each training example. The friendly noise is simple, effective, optimized before model training, and fixed during model training. Experiments show its effectiveness in defending against various powerful triggerless and backdoor poisoning attacks.
strengths_and_weaknesses: #### Pros
- This paper is motivated by the observation that powerful poisoning attacks exploit sharp loss regions to craft adversarial perturbations, which is reasonable.
- The proposed method is very simple and easy to implement. 
- The visualization of friendly noises is interesting. They seem to have some semantic information.

#### Cons
- The proposed defense relies on the assumption that poisoning attacks craft the poisons to mimic the gradient of the adversarially labeled target. The authors should discuss whether there are attacks with different mechanisms. When applied to such attacks, is the proposed defense still effective?
- The authors criticize that majority of the existing methods are attack specific. However, the proposed method is still specific to the considered attacks that rely on Equation (2).
- The authors criticize that state-of-the-art defense methods are often so expensive that can hardly be applied to even medium-sized datasets. However, experimentally, the authors only verify their method on CIFAR-10. Thus, such criticism can also be used in the proposed method.
questions: - Why use the L2 norm in the penalty term of Equation (3)? Can L1 or Linf norm work?
- When the poisons do not mimic the gradient, is the proposed defense still effective?
- Recent works proposed several methods (such as [1][2]) that can also be applied to defend against various attacks. What are the advantages of this method compared with them?

[1] Li, Yige, et al. "Anti-backdoor learning: Training clean models on poisoned data." Advances in Neural Information Processing Systems 34 (2021): 14900-14912.

[2] Wu, Dongxian, and Yisen Wang. "Adversarial neuron pruning purifies backdoored deep models." Advances in Neural Information Processing Systems 34 (2021): 16913-16925.

limitations: The method only applies to attacks that rely on sharp loss regions such as Witches’ Brew and Gradient Matching. The experiments are only conducted on CIFAR-10.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, the authors propose a new defense against data-poisoning attacks. The authors train the models on the data perturbed with 2 types of noise – (1) An optimized max norm perturbation within an \epsilon bound when added to the sample, the logit distribution is similar to the original sample logit distribution. (2) A random noise was chosen from uniform, Gaussian, or Bernoulli. The authors discuss how the first noise leads the poisons away from the sharp regions and the random noise smooths the loss. The authors show the efficacy of the method against several adaptive attacks. The authors also show that the perturbations generated in this method can transfer across the architectures. Even though the method’s performance is comparable to a few previous defenses, the proposed method is significantly faster.
strengths_and_weaknesses: Strengths:
- The technique is simple to understand and use and fast.
- The paper is well-written and easy to follow.
- The investigations seem thorough.

Weakness:
- L129-135, this paragraph says, a high std of KL term means higher sharpness. Can you provide intuition for this? Or cite any previous work discussing this.
- L136 – I think a citation is needed for this statement. 


questions: Did you try evaluating models on other types of poisoning attacks like Patch-based? 
limitations: n/a
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper considers targeted data poisoning attack (in short, we want to classify x as label y_adv, we feed x+\delta with right label y, but the trained network will classify the original x into targeted wrong label).

This paper considers an interesting approach of using optimization to add friendly noise so as to bring the training outside of the sharp loss region. Further the friendly noise is a randomized mechanism, which makes adaptive attacks harder to apply
strengths_and_weaknesses: Strengths: The approach is clearly motivated and well executed. The paper is well written and the experimental results seem convincing.

Weakness: The paper did an argument to say that "adaptive attacks is impossible", this may be a little weak -- because one can still think of adaptive attack schemes. For example, how about the adversaries trying to simulate a few such training with different randomness, and try to find poison that breaks all these training? While this seems to be time-consuming, it seems to be a plausible attack scheme.  Note that this may not be the case that the adversaries have to try ALL randomness, since the random Gaussian noise are all centered at mean, one may only need a few samples, and "breaking all of them" will ensure transfer. Nevertheless these are very data dependent, so without experiments or careful thinking it's hard to conclude 
questions: Is there actual evidence that adaptive attacks are hard to apply, beyond a heuristic argument of "words" about exhaustive search?
limitations: No limitation identified
review_title: null

Meta-review: metareview: This paper proposes a poisoning defense that unlike existing methods breaks various types of poisoning attacks with a small drop in the generalization. The key claim is that attacks exploit sharp loss regions to craft adversarial perturbations which can substantially alter examples' gradient or representations under small perturbations. The authors then propose to generate noise patterns which maximally perturb examples while minimizing performance degradation.

I think the framing of this paper is very important and the authors have done a good job at it. They are claiming to have a defense that is non attack-specific as long as it is restricted to the class of attacks involving visually imperceptible inputs. I believe this claim, if substantiated, to be of sufficient significance to the NeurIPS community.

Unfortunately, I noticed that the reviewers largely did not respond to the author rebuttal, other than PZZ3. PZZ3's main concerns were with lack of novelty with respect to the Anti-Backdoor Learning paper, different settings (untargeted, backdoor triggers), and substantiation for the sharp loss hypothesis. Having read carefully the authors' rebuttal to these, I believe the authors have a done a good job of alleviating the concerns and/or misunderstandings. For example I've read the ABL paper and agree it is dealing with a different setting. It was nice to see that the authors actually did experiments to show that ABL is not effective in this setting and vice versa.

Reviewer Sp5u was concerned with the attack-specificity of the defense to which the authors rebutted appropriately that is is data specific but not attack-specific as long as it is restricted to the class of attacks involving visually imperceptible inputs.

As far as I can tell, there were no other strong concerns.

Based on my own assessment, I believe that the central claim of the paper has sufficient evaluations to support it. The attacks considered are highly varied in their techniques and are also recent and SOTA.

I therefore recommend accept.  
confidence: Certain
Decision: Accept