Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors reveal the sensitivity of poisoned samples to transformations and propose a sensitivity metric FCT. And the authors  propose two effective backdoor defense methods for training a secure model from scratch and removing backdoor from the backdoored mode. 
strengths_and_weaknesses: The authors  design a simple sensitivity metric to distinguish poisoned samples from clean samples in the untrustworthy training set, and  propose two effective backdoor defense methods. The paper tackles an interesting issue, and the efforts of the authors are clear in investigating the problem and in writing the manuscript.
questions: 1. In the introduction, it is suggested that the authors add the background to the application of the proposed method, thus enhancing the motivation for the paper.
2. The authors only list the researches for the image super-resolution and don't detail the reasons why these approaches are not sufficient for their goal.
limitations: Yes,  the authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper the authors: (a) Attribute the effectiveness of backdoor attacks to overfitting which they measure by proposing a metric called Feature Consistency towards Transformations. Using this metric they partition a given training dataset into potentially clean and potentially polluted samples and (b) They propose two defense algorithms (i) train a secure model from scratch (ii) Training a backdoored model first then removing the backdoor from the model.

To show (a) they pass the original image with the trigger and an augmentation of the triggered image to the same model and calculate the L2 distance in the feature embedding space. They define large distances mean more sensitivity. Using this metric they create a "sample-distinguishment" SD module to partition a given dataset by thresholding the FCT scores.

For defending against backdoor attacks from a potentially polluted dataset they propose two approaches:
1. Secure Training From Scratch: They follow an approach similar to [29] Contrastive Supervised Learning where they have a two term contrastive loss (a) self-supervised loss on clean and polluted samples (b) supervised loss on clean vs polluted designated by the SD module using FCT thresholding. After they train the feature extractor, they train a classifier on top by minimizing a two term cross entropy loss (i) increasing the likelihood of predicting clean samples as their correct class (i) decreasing the likelihood of predicting the identified polluted samples as incorrect.

2. Unlearn and relearn: on the found clean and polluted sets, they iteratively decrease the likelihood of the polluted predictions and increase the likelihood of the clean predictions using cross entropy loss.

They show with extensive ablation studies the effectiveness of the different choices they make in their defense algorithms.

strengths_and_weaknesses: Strengths:
- Novelty of their method and uncovering an interesting fact about the backdoored models and triggers.
- Presentation and clarity.
- Thorough experimentation for supporting their claims on the proposed defense method.

Weaknesses:
- All the experiments are on a single dataset, CIFAR. This raises questions around how easy it is to tune the different hyper parameters in the model on a different dataset and model architecture.

- More experiments missing about the detection rate of the FCT metric. Simple ROC curves could give a sense of the power of their metrics. I understand that the first ablation study is trying to indirectly address this but since this is such an important idea in the paper, it could be isolated and tested.

questions: - Are the chosen hyper parameters easily transferrable to other datasets and network architectures?
- How sensitive is FCT to the dimensionality of embeddings since it's calculating a euclidean distance? Will it become harder to find a threshold for higher dimensions?
- How are the 0.05 and 0.2 thresholds set for the SD module?
- Is it possible to train a trigger in a whitebox setting with all of the hyper parameters of the defense method know to the attacker?


limitations: - An important aspect of the work is setting the threshold for SD. It is not explained how this threshold can be tuned without knowing beforehand which samples contain the trigger and the distribution of distances. Or it's not discussed how sensitive the method is setting these thresholds to a different value.

- A problem can be that in a whitebox setting the attacker can design the trigger such that FCT and its set thresholds are not effective by incorporating FCT in attack generation process by adding the inverse FCT to their optimization objectives for finding the trigger.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: Through the observation that clean and backdoored data with a dissimilar feature representation after data transformation techniques (e.g. rotation, scaling), the author proposed a sensitivity metric, feature consistency towards transformations (FCT), to detect the potential backdoor samples. The author further proposed two backdoor removing modules with inspiration from the existing defenses of semi-supervised learning and backdoor unlearning. Extensive results show that the proposed methods outperform the existing backdoor defenses either in backdoor detection or backdoor removal.

strengths_and_weaknesses: This paper is well-organized and the main insight is the observation of feature separation by data transformations. The defense solution including a secure training (ST) module and a backdoor removal (BR) module seems simple yet effective against various classical backdoor attacks.  
questions: At the same time, I have some concerns about the paper's contributions:

- At a first glance, the proposed methods seem as a unified framework including backdoor detection and backdoor defense forming an integral defense pipeline from training to post-training phase. But after a depth-reading, I consider this work is much like a patchwork combining existing techniques. For instance, the proposed feature detection matrix has been reflected in [1], and the other two defense modules ST and BR are much related to the DBD [2] and ABL [3] respectively. So, please explain clearly the main contribution and what specific contribution\novelty of this work compared with others.

- In addition, I doubt the generality of this method, for which the underlying assumption is that there are differences in feature representation between backdoor samples and clean samples. In fact, this assumption is often related to the design of backdoor models, and trigger types (static trigger, dynamic trigger, optimization-based trigger, etc). For more powerful adaptive attacks such as FC [4], dynamic [5], and latent modification [6], this assumption is no longer valid. In this case, how to ensure adequate defense? The author is invited to give an explanation and further experimental evidence.

[1] Li Y, et al. Rethinking the trigger of backdoor attack. arXiv preprint arXiv:2004.04692, 2020.  
[2] Huang K, et al. Backdoor defense via decoupling the training process. ICLR, 2022.  
[3] Li Y, et al. Anti-backdoor learning: Training clean models on poisoned data. NeurIPS, 2021.  
[4] Shafahi A, Huang W R, Najibi M, et al. Poison frogs! targeted clean-label poisoning attacks on neural networks. NeurIPS, 2018.  
[5] Nguyen T A, Tran A. Input-aware dynamic backdoor attack. NeurIPS 2021.  
[6] Doan K, Lao Y, Li P. Backdoor attack with imperceptible input and latent modification. NeurIPS, 2021.  
limitations: It is still unclear that the underlining assumption behind ‘feature consistency' is supported efficiently by the proposed solutions with only empirical results. It needs some explanation about this result to backup this drawback.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors observe that the features of poisoned samples are more sensitive to image transformations and use this observation to filter out poisoned data from the training set.  They use this for two different defenses, one to filter out training data before training and one to remove a backdoor from an already backdoored model.  They test against a range of attacks and compare to a variety of defenses.
strengths_and_weaknesses: I thank the authors for their interesting work!  This topic is important for security, and I think the NeurIPS community will benefit from new works on defense against poisoning.  The experimental results are pretty strong, and the work compares to other defenses and a range of attacks, so I lean towards accepting.

Feedback can be found below:

For “paradigm 1” did you compare to very strong data augmentations, like mixup/cutmix/maxup, which have been shown to be strong defenses against backdoors or various adversarial training strategies that have been used against backdoor attacks?

It might be good to test against optimization based poisons, such as Sleeper Agent, which don’t have patches on the training samples.  I wonder if these can defeat your defense.  They can also be made adaptive to beat your defense since you could add an objective that makes the features of poisoned samples insensitive to transformations.

Minor point: Full ImageNet does not seem prohibitively expensive here, so it would be good to test there, since it’s a standard setting.

questions: N/A
limitations: The authors refer to Appendix G for limitations which just contains experiments with different percentages of training samples poisoned.  I don't think this fully encompasses limitations of this paper, but I think the limited testing scenarios are clear to a reader, so it's not that big of a deal.
review_title: null

Meta-review: metareview: The authors propose a new method for defending against backdoor attacks which is based on the observation that poisoned samples are more sensitive to transformations than clean samples. They design a metric called \textit{feature consistency towards transformations (FCT)} to distinguish poisoned samples from clean samples in the untrustworthy training set.

The paper received favorable reviews and has made substantial updates during the rebuttal phase to the general satisfaction of the reviewers. I thus recommend accept.

confidence: Certain
Decision: Accept