Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The proposed work introduces a defense which uses an auxiliary reconstruction task along with the classification to ‘trap’ the backdoor within the classification head. In the second stage, a new classification head is trained from scratch to completely remove the effect of the backdoor. The intuition is that the auxiliary task ensures that low-level features of the image is preserved within the backbone or the stem network, hence reducing the effectiveness of the backdoor. Results are shown on CIFAR-10, GTSRB and ImageNet-12 against various attacks.
strengths_and_weaknesses: **Strengths:**

— The defense method introduced is novel and effective. It is also easy to use without requiring too much hyper-parameter tuning compared to previous methods.

— Authors conduct extensive ablation study to illustrate the effectiveness of the method. An Adaptive attack is also considered, against which the defense still remains effective.

— Results are shown on a variety of backdoor attacks, indicating that it can be used to defend against multiple attacks.

**Weaknesses:**

— Although the authors do provide an intuition that low-level features are preserved better with the auxiliary task, it is not clear why the proposed method should be effective. A feature analysis on the difference with and without the reconstruction task can be helpful in identifying why the defense is effective to such a degree.

— The reconstruction task proposed by the authors is not very well understood. An experiment where the quality of reconstructed images vs ASR can improve understanding. Although authors do provide a variant of this experiment in Table 8 of the appendix, a qualitative analysis would be better in this scenario. Another experiment would be to vary the size or capacity of the ‘decoder’ and observe the variation in ASR.

— Similar to previous methods, a clean heldout dataset is required which is typically 10%. This can be difficult to obtain for larger datasets.
questions: — It seems that replacing the classification head without the auxiliary task results in significant drop in ASR. Is it possible to consider an experiment where the learning rate for samples from clean holdout set is 10x the learning rate for the untrusted dataset? There is no auxiliary task in this scenario. Such an experiment would show whether the effectiveness of the defense is purely due to the correct decision boundaries learned by the classification head or if it is due to the trapping mechanism.

— A more detailed ablation study on the size of clean holdout set is necessary. From, Table 9 of the appendix, it seems there is no difference between the 2.5% and 5% setting. It would be interesting to know what is the least amount of samples required to achieve similar results. 


After Rebuttal:
Thanks to the authors for providing the rebuttal. Having gone through the other reviews, I will keep my score unchanged and strongly encourage the authors to include some of the new experiments about the relationship between reconstruction and ASR in the main paper.
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a new defense strategy *Trap and Replace* to protect deep neural networks from the backdoor in the poisoned dataset. This paper presumes that the backdoor pattern is easy to learn, so it trains a standard image classification model consisting of a stem network and a classification head and an extra image reconstruction model, which consists of the stem network and reconstruction head at the same time to encourage stem model to learn the low-level visual features. Then freeze the stem work and initialize the classification head by setting those parameters to random values. And train the new model in a small but clean dataset. The experimental results show that Trap and Replace outperforms SOTA defense strategies in most cases.

strengths_and_weaknesses: Strength:
1. Introducing a reconstruction task to force the stem model to learn visual features is a novel idea. It is self-supervised training that does not need extra labeled samples.
2. Training classification head-on clean dataset can mitigate the attack success rate and keep accuracy at a high level.
3. The paper's summary of relative works is well organized and comprehensive.
4. The experiments are well designed and clearly shown in the tables.
5. Ablation study showing that both Trap and Replace are necessary is convincing.

	 
Weakness:
1. Why using the fully connected layer as classification head fails to defend the backdoor should be discussed.
2. How many layers should be chosen as the classification head in others models that are not included in this paper should be considered.
questions: N/A
limitations: Comments:
1. In previous works, the classification head is considered the fully connected layer. You can explain the definition of classification head and reconstruction head in the introduction session.
2. In line 156, there is a repetitive word "assumptions".
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose to trap the backdoor by a lightweight classification head on top of a low level feature extractor and replace it with a clean classifier to remove the backdoor. Extensive experimental results on different dataset against various attacks show the effectiveness of the proposed method.
strengths_and_weaknesses: Strengths:

a. The idea is interesting, and the motivation is reasonable.

b. The authors provide rich ablation experiments to evaluate the proposed method.

Weaknesses:

a. Most of the previous methods only need access to either the poisoned dataset or a small clean dataset, but the proposed method requires access to both, which limits its practical value.

b. According to Table 1, the degradation on clean accuracy is considerably large. This may because the final classifier is only trained on limited data (the holdout dataset).
questions: a. I am curious about the performance of the other methods. For example, in [11], ANP use 500 images (i.e., 1% of the training data) to purify the model against SIG on CIFAR-10, and the degradation of the clean accuracy is only 0.24% (93.64% to 93.40%). In this paper, the authors use 5% of the training data (2500 images), but the performance of ANP is far worse (89.27% to 85.67%). It is strange that the baseline methods perform such badly.
limitations: No. I think the largest limitation is the threat model. As far as I know, this is the first method that require both a poisoned dataset and a clean holdout set. If so, I suggest the author to discuss the practical scenario of such threat model.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors of this work proposed a defense method against backdoor attacks. The defense method consists of two stages. In the first stage, they trapped the backdoors in a subnetwork. In the second stage, they replace the poisoned subnetwork and retrain the network with clean samples. Consequently, this method outperforms previous state-of-the-art methods.
strengths_and_weaknesses: Strengths:

1. The writing of this paper is very clear and easy to follow.
2. The experimental results show that the method outperforms the other six baseline methods in various attack methods except in some cases (Blend, Trojan SQ, Trojan WM).
3. The ablation study clearly shows the significance of the two stages.

Weaknesses:

1. In my opinion, the threat model is pretty unrealistic (with some holdout clean samples) however you can follow this assumption from previous works. 
2. The auxiliary image reconstruction task encourages the stem network to keep sufficient low-level visual features that are hard-to-learn but semantically correct, protecting the stem network from overfitting to the easy-to-learn but semantically incorrect backdoor correlations. I cannot figure out the relation between this intuition and how the effect of poisoned data can be trapped.

Typos:
1. Line 156 has two “assumptions”.
2. Line 238 “Tojan” --> “Trojan”.

questions: 1. I think the reconstruction feature is pretty different from the classification feature which may cause both tasks to be difficult to train in more complex datasets. Can you show some reconstruction results? Does the reconstruction network reconstruct the trigger as well?
2. It is confusing that in the SIG attack on ImageNet, the clean accuracy becomes higher in Table 3. Why your finetuning is so powerful in this case?
3. Is your framework only suitable for convolutional neural networks? How about vision transformers?
4. As you mentioned in the footnote on page 2, semi-supervised learning and self-supervised learning are not naive solutions, since they are vulnerable to backdoor attacks. Is your work aim to provide a defense mechanism in this scenario?
5. As a suggestion and a question, I think that the poisoned data is aim to fool the classifier hence finetuning will work in this case. In multi-task learning, you can try to create poisoned data that aims to fool any downstream tasks. As finetuning might work for each task, how about putting various purpose attacks at the same time?

limitations: Yes. The authors state that there are no potential negative societal impacts of this work.
review_title: null

Meta-review: metareview: The recommendation is based on the reviewers' comments, the area chair's personal evaluation, and the post-rebuttal discussion. 

This paper proposed a new training method to defend against backdoor attacks. While all reviewers see merits in this paper, some discussions about (1) the practicality of the defense using clean data samples and (2) fair comparisons to existing defenses have been raised and discussed. During the author-reviewer discussion phase, the reviewer had detailed interactions with the authors to clarify different use cases and practical scenarios of the proposed defense and the fairness of the evaluation. So both major concerns are adequately addressed. Another reviewer also champions acceptance in the internal discussion. All in all, I am recommending acceptance. My confidence is lower compared to other submissions simply because this paper has the lowest average rating score of all papers I recommend acceptance.
confidence: Less certain
Decision: Accept