Backdoor Defense via Decoupling the Training Process

Abstract: Recent studies have revealed that deep neural networks (DNNs) are vulnerable to backdoor attacks, where attackers embed hidden backdoors in the DNN model by poisoning a few training samples. The attacked model behaves normally on benign samples, whereas its prediction will be maliciously changed when the backdoor is activated. We reveal that poisoned samples tend to cluster together in the feature space of the attacked DNN model, which is mostly due to the end-to-end supervised training paradigm. Inspired by this observation, we propose a novel backdoor defense via decoupling the original end-to-end training process into three stages. Specifically, we first learn the backbone of a DNN model via \emph{self-supervised learning} based on training samples without their labels. The learned backbone will map samples with the same ground-truth label to similar locations in the feature space. Then, we freeze the parameters of the learned backbone and train the remaining fully connected layers via standard training with all (labeled) training samples. Lastly, to further alleviate side-effects of poisoned samples in the second stage, we remove labels of some `low-credible' samples determined based on the learned model and conduct a \emph{semi-supervised fine-tuning} of the whole model. Extensive experiments on multiple benchmark datasets and DNN models verify that the proposed defense is effective in reducing backdoor threats while preserving high accuracy in predicting benign samples. Our code is available at \url{https://github.com/SCLBD/DBD}.

Reviews and author responses:

 We greatly appreciate your positive feedback and further insightful comments and questions. We totally understand that you may be extremely busy at this time. But we still hope that you could have a quick look at our responses to your concerns. We appreciate any feedback you could give to us. We also hope that you could kindly update the rating if your questions have been addressed. We are also happy to answer any additional questions before the rebuttal ends.

 Thank you very much again for your initial comments. They are extremely valuable for improving our work. We shall be grateful if you can have a look at our response and modifications, and kindly let us know if anything else that can be added to our next version.

 We are so encouraged by your positive comments! Please kindly find our explanations about the remaining two concerns below.

---

**Q1**: "If the attacker knows that you are going to use semi-supervised learning, why they wouldn't be able to apply the semi-supervised attack?".

**R1**: Thanks for this insightful comment. We try to explain it from the following two aspects.
- The original semi-supervised attack proposed in Carlini's work doesn't work to our DBD method. Because the input of our DBD method is fully labeled data. Although the semi-supervised fine-tuning is adopted as the last stage of DBD, the unlabeled data is constructed by ourselves, after filtering low-credible points in the second stage. Thus, at the beginning, we will abandon unlabeled data, such that the interpolated unlabeled data will be removed.
- However, the above point inspires us to consider an adaptive attack against DBD that, the attacker interpolates some points to build a path between a point from the target class and the selected attacked point from the source class, and labels these interpolated points as the target class. Then, after the first stage of self-supervised learning in DBD, the selected point may be close to these points, and they can be seen as boundary points between two classes. In this case, at the second stage of learning the classifier in DBD, we may not expect to see a similarly large gap between the loss of these points and the loss of other points, compared to the loss gap between backdoor-triggered points and benign points. If it happens, then these interpolated points and the selected point cannot be identified as low-credible points. Consequently, the attack goal of misclassifying the selected sample as the target class can be achieved probably, though the attack may also fail because its label is correct. It will be further empirically verified in our future work. If such a single-point attack succeeds in our DBD method, we would not be surprised, as DBD is not designed for this attack paradigm. However, Carlini's work has provided some useful approaches to defend this attack, such as the pairwise influence (see Section 5.3 in Carlini's work). It can be naturally combined with our DBD method at the second stage, to identify triggered and interpolated points simultaneously.

According to the above two aspects, we don't think that either the original or adaptive semi-supervised attack is a severe threat to our DBD method. We hope that the above explanations could provide more clear information about your concern. The discussion with you is very pleasant and helpful, we are willing to discuss with you any further concerns. Thanks again for your insightful comment, which inspires us to explore a more effective adaptive attack against our DBD method in our future work.

---

**Q2**: More details about the speed constraints.

**R2**: Thank you for this constructive suggestion! Following your suggestion, we report the training time of each stage in our DBD as follows:


Table 1. The training time (minutes) of each stage in our DBD on CIFAR-10 dataset.

|     | Stage 1 (Self-supervised Learning) | Stage 2 (Filtering) + Stage 3 (Fine-tuning) |
|:---:|:----------------------------------:|:-------------------------------------------:|
| DBD |                50.8                |                    260.5                    |

As shown in Table 1, using secure pre-trained backbones (to avoid the self-supervised learning stage) can indeed save some time in our DBD. Note that we only trained the backbone with 100 epochs (instead of the 1000 epochs that is widely adopted in self-supervised learning works) in our DBD to save time, as we described in Appendix K of our original manuscript. This is why we can only save 50.8 minutes if we use pre-trained backbones, otherwise, we can save 508 minutes. 


Inspired by your question, we also notice that our DBD can be further accelerated by using fewer epochs in Stage 2-3. As shown in Table 2, our DBD has converged after 115 epochs. In other words, we can save 115.8 more minutes by setting the epoch as 115 (instead of 200) in Stage 2-3.  In practice, one can adaptively set the maximum epoch to reduce the cost, according to the training curve.

Table 2. The benign accuracy (\%) w.r.t. the training epoch in the stage 2-3 of our DBD on CIFAR-10 dataset.


| Epoch |  25   |  40   |  60   |  80   |  85   |  115  |  116  |  117  |  118  |  125  |  126  |  127  |
|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|  BA   | 85.86 | 88.96 | 90.60 | 91.85 | 92.42 | 93.02 | 93.02 | 93.16 | 93.15 | 93.37 | 93.04 | 93.00 |



Thank you again for your insightful question. We will add more details and discussions in Appendix K in our final version.

 I generally like the response and will raise my score, however I'd ask to do a bit more work on clarifying your answers.

> safety of semi-supervised learning.

I am still a bit confused, although my understanding just might be wrong. If the attacker knows that you are going to use semi-supervised learning, why they wouldn't be able to apply the semi-supervised attack? Yes your threat model is different, but since you change your training procedure the attacker can do adaptive attack as well, i.e. treating labeled examples as unlabeled. And yes, it will only misclassify a single example but it might still succeed. I understand that you have a different threat model, but I did not get whether the attack itself is invalid.

> speed constraints

The proposed solution is pretty good and indeed saves plenty of time, probably you can specify how much time needed to train when you already have a backbone model (e.g. stage 2 and stage 3 combined).

But overall great work and thank you for taking time to answer the questions!

 The reviewer thank the authors for their detailed response. Most of my concerns have been addressed. In particular, the DBD seems to be effective against the adaptive attacks with a large trigger size. The experiments also show that DBD works with other self-supervised learning methods. Also, comparison with the baseline methods such as Spectral Signatures have been made. Due to the above, I raise my score.

This paper shows that self-supervised, contrastive learning can give a feature extractor that scatters training data points with backdoor triggers in the feature space. With this observation, the authors propose a novel defense method based on contrastive learning and decouple end-to-end training to defend against backdoor attacks. They first train a feature extractor using self-supervised contrastive learning that turns the poisoned data points into outliers in the feature space. Then they train a cascade classifier that ignores the poisoned data points by leveraging the fact that a neural network tends to capture frequent patterns. Experiments are conducted and the results verify the effectiveness of the defense. The paper proposes a novel defense method named DBD based on contrastive learning (SimCLR) and conduct extensive experiments to show that DBD is effective against  different types of attacks, including BadNets, Blended, WaNet, and Label-Consistent attacks.

However, the author did not compare DBD with some recently works such as "Spectral signatures in backdoor attacks" in NIPS 2018 and "Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering" in AAAI 2019, which both are based on data removal. It is unclear how differently DBD removes the poisoned data as compared with the existing works.

While the authors claim that their major contribution was to decouple the end-to-end training process, I suspect the proposed approach works only with SimCLR. What is the necessary conditions that makes a feature extractor scatter poisoned data points in the feature space? Can you point out other feature extractors having the same properties as SimCLR? Does DBD works with these extractors too?

The effect of the final fine-tuning step is unclear. How does DBD perform without this phase?

I also suggest the authors to move the section about the resistance to adaptive attacks in Appendix into the main paper as the adaptive attacks are becoming a more serious threats today. Please explain your adaptive attack settings more clearly (for example, what trigger size you used and how you tuned the hyper-parameters).

It would also be good if the author discuss how an attack may work around the proposed defense, and how to further defend such workarounds.

Edit after rebuttal:

The reviewer thank the authors for their response. Most of my concerns have been addressed. In particular, the DBD seems to be effective against the adaptive attacks with a large trigger size. The experiments also show that DBD works with other self-supervised learning methods. Also, comparison with the baseline methods such as Spectral Signatures have been made. Due to the above, I raise my score.
 Overall, the paper proposes a novel defense to the backdoor attacks that could benefit the field. However, the current experiments are not convincing enough because some critical baselines are missing. There is no discussion on how general the proposed defense work when different training techniques are used in different phases. And there is no discussion on how a backdoor attack may work around the proposed defense, and how to further defend such workarounds like many other security papers did. Therefore, I give a weak reject.

 We sincerely thank you for your valuable time and comments. We are encouraged by the positive comments on the **simple but effective idea** and **extensive experiments**.

**Q1**: The primary concern of this paper is the extra computation cost of the proposed DBD pipeline. Considering training a self-supervised and semi-supervised model costs way more computational resources than a supervised learning model. Would it be possible to use a public pre-trained feature extractor to replace the self-supervised feature extractor? The authors are welcomed to discuss this.

**R1**: Thank you for the insightful comment and question! Thank you for this insightful comment! As analyzed in Appendix K, our method is roughly 3-4 times slower than the standard supervised training process in general. We think that there are two possible approaches to accelerate our DBD: 
1. If there is a secure pre-trained feature extractor (e.g, the one from trusted sources), people can directly use it and save the time of the self-supervised stage (as you suggested).   

To verify the possibility of using a public pre-trained feature extractor instead of training the self-supervised extractor from scratch, we download a pre-trained ResNet-50 feature extractor from https://github.com/leftthomas/SimCLR, and replace the self-supervised feature extractor with it in our DBD. As shown in Table 1, our DBD is still effective with the pre-trained feature extractor.

However, we need to notice that using public pre-trained feature extractors is still risky when the model source is not secure. For example, if the pre-trained feature extractor is infected (with the same poisoned samples contained in the training set), using it in our DBD will still create hidden backdoors, which can be verified by Table 3 (Line 'DBD without SS') in Section 5.3 in our revision. For your convenience, we pick up these relevant data to form Table 2 below. Please refer to Section 5.3 (Table 3) for more details.

If the security of these backbones can be guaranteed, then we can directly utilize it in the first stage of our DBD method, which can significantly reduce the cost. Besides, we can further verify the quality of the backbone by ourselves. Specifically, we can deliberately insert some trigger patterns into some samples (as did by the adversary), then visualize the features extracted by the backbone. If the triggered samples are separated, then the backbone may be secure enough to satisfy our expectation in defense. Note that although the defender has no information about the real trigger pattern in the poisoned data, we can try multiple different triggers to enhance the confidence of the verification.


Table 1. The BA (%) and ASR (%) of our DBD defending against four attacks with the pre-trained feature extractor on CIFAR-10 dataset.

|     | BadNets | Blended | Label-Consistent | WaNet |
|:---:|:-------:|:-------:|:----------------:|:-----:|
| BA  |  94.53  |  94.81  |      93.22       | 94.08 |
| ASR |  0.54   |  0.73   |        0         | 1.36  |




Table 2. The BA (%) and ASR (%) of our DBD defending against four attacks with or without the malicious pre-trained feature extractor on the poisoned CIFAR-10 dataset.

|            Attack$\rightarrow$            |  BadNets   |   Blended   | Label-Consistent |    WaNet    |
|:-----------------------------------------:|:----------:|:-----------:|:----------------:|:-----------:|
| Method $\downarrow$, Metric $\rightarrow$ |   BA/ASR   |   BA/ASR    |      BA/ASR      |   BA/ASR    |
|                  DBD(w/)                  | 93.66/100  | 93.47/99.93 |   90.70/98.50    | 81.91/98.40 |
|                 DBD (w/o)                 | 92.41/0.96 | 92.18/1.73  |    91.45/0.34    | 91.20/0.39  |

2. One can also adopt existing accelerated methods (e.g., mixed-precision training) towards each stage of DBD to accelerate the whole training process.

The above acceleration tricks will be thoroughly explored in our future work.

 ---

**Q2**: I suspect the proposed approach works only with SimCLR. What is the necessary conditions that makes a feature extractor scatter poisoned data points in the feature space? Can you point out other feature extractors having the same properties as SimCLR? Does DBD works with these extractors too?


**R2**: Thanks for this insightful concern. The desired feature extractor is mapping visually similar inputs to similar positions in the feature space, such that poisoned samples will be separated into their source classes. This goal is compatible with that of self-supervised learning. We believe that any self-supervised learning can be adopted in our method. To further verify this point, we replace the adopted SimCLR with another two self-supervised methods (i.e., MoCo-V2 [1, 2] and BYOL [3]) in our DBD method. As shown in Table 4, all DBD variants have similar performances. Please refer to Appendix M in our revised manuscript for more details.

Table 4. The BA (%) and ASR (%) of our DBD defending against four attacks with different self-supervised methods on CIFAR-10 dataset.

|            Attack$\rightarrow$            |  BadNets   |  Blended   |   WaNet    | Label-Consistent |
|:-----------------------------------------:|:----------:|:----------:|:----------:|:----------------:|
| Method $\downarrow$, Metric $\rightarrow$ |   BA/ASR   |   BA/ASR   |   BA/ASR   |      BA/ASR      |
|                  SimCLR                   | 92.41/0.96 | 92.18/1.73 | 91.20/0.39 |    91.45/0.34    |
|                   MoCo                    | 93.01/1.21 | 92.42/0.24 | 91.69/1.30 |    91.46/0.19    |
|                   BYOL                    | 91.98/0.82 | 91.38/0.51 | 91.37/1.28 |    90.09/0.17    |

---

**Q3**: The effect of the final fine-tuning step is unclear. How does DBD perform without this phase?


**R3**: Thanks for this insightful concern. 
- Actually, we have evaluated the effect of the final (semi-supervised) fine-tuning in Section 5.3 and Table 3 of our original manuscript. For your convenience, we only pick up the results related to the fine-tuning step to form Table 5 below. It shows that fine-tuning can significantly enhance both BA and ASR simultaneously. 
- This effect can be understood from two aspects. Firstly, as explained in Section 4.5 of our original manuscript, the fine-tuning can somewhat mitigate the side-effect of poisoned samples caused in the training of the simple classifier in stage 2 of our DBD. Secondly, the fine-tuning of the feature extractor (*i.e.*, the backbone) and the classifier jointly is expected to encourage them to be more compatible, since they are originally learned separately. We have clarified the effect of the fine-tuning step in Section 4.5 of our revised manuscript.
 


Table 5. The BA (%) and ASR (%) of our DBD defending against four attacks with or without semi-supervised fine-tuning on CIFAR-10 dataset.

|            Attack$\rightarrow$            |  BadNets   |  Blended   | Label-Consistent |   WaNet    |
|:-----------------------------------------:|:----------:|:----------:|:----------------:|:----------:|
| Method $\downarrow$, Metric $\rightarrow$ |   BA/ASR   |   BA/ASR   |      BA/ASR      |   BA/ASR   |
|                 DBD (w/o)                 | 82.34/5.12 | 82.30/6.24 |    81.81/5.43    | 81.15/7.08 |
|                 DBD (w/)                  | 92.41/0.96 | 92.18/1.73 |    91.45/0.34    | 91.20/0.39 |

---



 We sincerely thank you for your valuable time and comments. We are encouraged by the positive comments on the **novelties**, **strong sets of experiments and baselines**, **practicability**, and **good writing**.

---

**Q1**: The proposed method modifies an underlying training procedure multiplying training time, which I think is significant for practitioners. Addressing this issue is essential to support the practicality of the method.

**R1**: Thank you for this insightful comment! As analyzed in Appendix K, our method is roughly 3-4 times slower than the standard supervised training process in general. We think that there are two possible approaches to accelerate our DBD:

1. Recently, some open-sourced self-supervised pre-trained backbones have been publicly released, e.g., [VISSL Model Zoo](https://github.com/facebookresearch/vissl/blob/main/MODEL_ZOO.md). To verify the possibility of using a public pre-trained feature extractor instead of training the self-supervised extractor from scratch, we download a pre-trained ResNet-50 feature extractor from https://github.com/leftthomas/SimCLR, and replace the self-supervised feature extractor with it in our DBD. As shown in Table 1, our DBD is still effective with the pre-trained feature extractor.

However, we need to notice that using public pre-trained feature extractors is still risky when the model source is not secure. For example, if the pre-trained feature extractor is infected (with the same poisoned samples contained in the training set), using it in our DBD will still create hidden backdoors, which can be verified by Table 3 (Line ‘DBD without SS’) in Section 5.3 of our original manuscript. For your convenience, we pick up these relevant data to form Table 2 below. Please refer to Section 5.3 (Table 3) for more details.

If the security of these backbones can be guaranteed, then we can directly utilize it in the first stage of our DBD method, which can significantly reduce the cost. Besides, we can further verify the quality of the backbone by ourselves. Specifically, we can deliberately insert some trigger patterns into some samples (as did by the adversary), then visualize the features extracted by the backbone. If the triggered samples are separated, then the backbone may be secure enough to satisfy our expectation in defense. Note that although the defender has no information about the real trigger pattern in the poisoned data, we can try multiple different triggers to enhance the confidence of the verification.



Table 1. The BA (%) and ASR (%) of our DBD defending against four attacks with the pre-trained feature extractor on CIFAR-10 dataset.
    
|  Metric $\downarrow$, Attack $\rightarrow$   | BadNets | Blended | Label-Consistent | WaNet |
|:-:|:-:|:-:|:-:|:-:|
| BA  |  94.53  |  94.81  |      93.22       | 94.08 |
| ASR |  0.54   |  0.73   |        0         | 1.36  |



    
Table 2. The BA (%) and ASR (%) of our DBD defending against four attacks with or without the malicious pre-trained feature extractor on the poisoned CIFAR-10 dataset.
    
| Attack $\rightarrow$ |  BadNets   |   Blended   | Label-Consistent |    WaNet    |
|:--:|:--:|:--:|:--:|:--:|
| Method $\downarrow$, Metric $\rightarrow$ |   BA/ASR   |   BA/ASR    |      BA/ASR      |   BA/ASR    |
|                  DBD(w/)                  | 93.66/100  | 93.47/99.93 |   90.70/98.50    | 81.91/98.40 |
|                 DBD (w/o)                 | 92.41/0.96 | 92.18/1.73  |    91.45/0.34    | 91.20/0.39  |



2. One can also adopt existing accelerated methods (e.g., mixed-precision training) towards each stage of DBD to accelerate the whole training process.

The above acceleration tricks will be thoroughly explored in our future work.

---

 ---

**Q4**: I do not think excluding the detection-based method is fair since those methods are strong baselines especially for the badnet and blending attack. Also, since the main contribution of the paper is the empirical performance, it is necessary to compare with different kinds of baselines.

**R4**: Thanks for these insightful comments! We do notice these recent works mentioned by you. These detection-based methods (e.g., Spectral Signatures (SS) [3] and Activation Clustering (AC) [4]) filter malicious samples from the training set and train the model on non-malicious samples. While in our DBD, we do not intend to accurately separate poisoned and benign samples as we mentioned in Section 4.4 of our original manuscript.
+ However, we do understand your concern. To alleviate it, we compare the filtering ability of our DBD (stage 2) with these detection-based methods. Since SS filters $1.5 \varepsilon$ malicious samples for each class, where $\varepsilon$ is the key hyper-parameter means the upper bound of the number of poisoned training samples, we adopt different $\varepsilon$ for a fair comparison. For AC, we use the default setting as suggested by the original paper. As shown in Table 1-2, the filtering performance of DBD is on par with that of SS and AC. DBD is even better than those methods when filtering poisoned samples generated by more complicated attacks (i.e., WaNet and Label-Consistent).
+ Besides, we also conduct the standard training on non-malicious samples filtered by SS and AC. As shown in Table 3, the hidden backdoor will still be created in many cases, even though the detection-based defenses are sometimes accurate. This is mainly because these methods may not able to remove enough poisoned samples while preserving enough benign samples simultaneously, i.e., there is a trade-off between BA and ASR.

Please refer to Appendix L in our revised manuscript for more results.



Table 1. The successful filtering rate (# filtered poisoned samples /# all filtered samples, %) under different $\varepsilon$ in the target class on CIFAR-10 dataset.

|          $\epsilon \rightarrow$          |    250     |     500     |    1000     |    1500     |
|:----------------------------------------:|:----------:|:-----------:|:-----------:|:-----------:|
| Attack$\downarrow$, Defense$\rightarrow$ |   SS/DBD   |   SS/DBD    |   SS/DBD    |   SS/DBD    |
|                 BadNets                  | 95.73/100  | 93.20/97.60 | 87.60/90.87 | 80.71/70.09 |
|                 Blended                  | 0.80/97.87 | 10.53/94.67 | 28.87/87.27 | 35.29/75.16 |
|                  WaNet                   |   0/100    |    0/100    | 1.00/99.47  | 7.42/97.46  |
|             Label-Consistent             | 2.40/43.47 | 4.40/37.07  | 9.00/34.47  | 13.78/32.53 |



Table 2. The number of remaining poisoned samples over filtered non-malicious samples on CIFAR-10 dataset. 

|                      |  BadNets   |  Blended   |   WaNet    | Label-Consistent |
|:--------------------:|:----------:|:----------:|:----------:|:----------------:|
| SS ($\epsilon=500$)  | 1801/42500 | 2421/42500 | 2500/42500 |    1217/42500    |
| SS ($\epsilon=1000$) | 1186/35000 | 2067/35000 | 2400/35000 |    1115/35000    |
|          AC          |  0/42500   |  0/37786   | 5000/45546 |    1250/39998    |
|         DBD          |  8/25000   |  6/25000   |  38/25000  |     13/25000     |



Table 3. The BA (%) and ASR (%) of models trained on non-malicious samples filtered by SS and AC on CIFAR-10 dataset.

|           Defense $\rightarrow$           | SS($\varepsilon=500$) | SS($\varepsilon=1000$) |     AC      |
|:-----------------------------------------:|:---------------------:|:----------------------:|:-----------:|
| Attack $\downarrow$, Metric $\rightarrow$ |        BA/ASR         |         BA/ASR         |   BA/ASR    |
|                  BadNets                  |       92.99/100       |      93.27/99.99       |   85.90/0   |
|                  Blended                  |      92.84/99.07      |      92.56/99.18       |   77.17/0   |
|                   WaNet                   |      92.69/98.13      |      91.92/99.00       | 84.60/99.02 |
|             Label-Consistent              |      92.93/99.79      |      92.88/99.86       | 75.95/99.75 |


---
Reference

[1] Zhilu Zhang and Mert R Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurIPS, 2018.

[2] Xingjun Ma, et al. Normalized loss functions for deep learning with noisy labels. In ICML, 2020.

[3] Brandon Tran, et al. Spectral signatures in backdoor attacks. In NeurIPS, 2018.

[4] Bryant Chen, et al. Detecting backdoor attacks on deep neural networks by activation clustering. In AAAI Workshop, 2019.

---


 We sincerely thank you for your valuable time and comments. We are encouraged by the positive comments on the **well generalization** and **effectiveness**.

---

**Q1**: The paper lacks a theoretical analysis of the proposed method. I understand that this paper mainly focused on empirical performance, but it is quite surprising that the proposed methods perform well on label-consistent attacks. This is because that proposed method decouples the label corruption and feature corruption. When label corruption no longer exists, what is the advantage of the proposed method?

**R2**: Thanks for this insightful concern and your encouraging comment on our intuitive, natural, and simple method. The good defense performance of our DBD method against the label-consistent attack can be explained from the following two levels:

- As shown in Figure 1 of our original manuscript, there is a common observation across different backdoor attacks (including both poisoned label and label-consistent attack) that poisoned samples tend to gather together in the feature space learned by the standard supervised learning, such that the mapping between the trigger and the target label is formed. The most intuitive idea of our DBD method is to prevent such a gathering in the learned feature space, which is implemented by self-supervised learning. As shown in Figure 1 (d), the poisoned samples of the label-consistent attack are also separated into different areas in the feature space learned by self-supervised learning. We think that this example gives an intuitive explanation about why our DBD can successfully defend the label-consistent attack. 
- Furthermore, it is interesting to explore why the poisoned samples in the label-consistent attack are separated under self-supervised learning since all poisoned samples are from the same target class, rather than from different source classes in poisoned-label attacks. For each poisoned sample in the label-consistent attack, there are two types of features: the trigger, and the benign feature with (untargeted) adversarial perturbations. From the perspective of DNNs, benign samples with (untargeted) adversarial perturbations are similar to samples from different source classes, though these samples look similar from the human's perspective. Thus, it is not surprising that poisoned samples in the label-consistent attack can also be separated under self-supervised learning, just like those in poisoned-label attacks. 

Hope the above two brief explanations could somewhat alleviate your concern about clean-label attacks. The above analysis has been added in Appendix O of our revised manuscript. We will explore this point more deeply and thoroughly in our future work.


---

**Q2**: For the second step label noise learning, there are also many choices instead of just using the symmetric cross-entropy method. Investigating more noisy-label algorithms might be interesting.

**R2**: Thank you for this constructive suggestion! We examine our DBD with other noisy-label methods (i.e., GCE[1] and APL[2]). As shown in Table 2, all DBD variants have a similar performance. Please refer to Appendix N in our revised manuscript for more details.


Table 2. The BA (%) and ASR (%) of our DBD defending against four attacks with different noisy-label methods on CIFAR-10 dataset.

|           Attack$\rightarrow$           |  BadNets   |  Blended   |   WaNet    | Label-Consistent |
|:---------------------------------------:|:----------:|:----------:|:----------:|:----------------:|
| Method$\downarrow$, Metric$\rightarrow$ |   BA/ASR   |   BA/ASR   |   BA/ASR   |      BA/ASR      |
|                DBD (SCE)                | 92.41/0.96 | 92.18/1.73 | 91.20/0.39 |    91.45/0.34    |
|                DBD (GCE)                | 92.93/0.88 | 93.06/1.27 | 92.25/1.51 |    91.05/0.15    |
|                DBD (APL)                | 92.95/1.00 | 92.65/0.78 | 92.24/1.40 |    91.08/0.14    |

---

**Q3**: The two-step method makes the algorithm not in an end-to-end fashion. It would be interesting to investigate the possibility to make an end-to-end algorithm.

**R3**: Thank you for this constructive and interesting suggestion! The current 3-stage paradigm of our DBD demonstrates that a good feature representation (learned by self-supervised learning in stage 1) could facilitate the task of poisoned data filtering (in stage 2). On the other hand, accurate filtering of poisoned data may help to learn a better feature presentation. From this perspective, a possible end-to-end paradigm may be unifying feature presentation learning and poisoned data filtering iteratively, rather than sequentially, to encourage them to be mutually reinforced. The backbone and the classifier learned by such an end-to-end paradigm are supposed to be more compatible. We will carefully consider and explore this suggestion in our future work.

---


 ---

**Q2**: In Section 5.2, the authors list the results on "No defense" to show the backdoor defenses' impact on the original model's accuracy and the effectiveness of the backdoor mitigation. It is not clear how the model works without a defense. Suppose the authors train the original model in a supervised learning manner. In that case, directly using a self-supervised learning paradigm should also be included to illustrate each step's contribution.

**R2**: Thank you for the constructive suggestion! Actually, we have conducted a detailed ablation study to evaluate the effectiveness of each stage in our DBD, including directly using a self-supervised learning paradigm as you mentioned, as shown in Section 5.3 and Table 3 of our original manuscript. For your convenience, we pick up some important results from the manuscript to form Table 3 below. The results show that only the self-self-supervised learning paradigm (i.e., SS with SCE) is also effective in reducing backdoor threats, though it is less effective compared with our DBD. Please refer to the ablation study (Section 5.3, Table 3) for more details.



Table 3. The (partial) ablation study of our DBD.

|            Attack$\rightarrow$             |  BadNets   |   Blended   | Label-Consistent |    WaNet    |
|:------------------------------------------:|:----------:|:-----------:|:----------------:|:-----------:|
| Defense $\downarrow$, Metric $\rightarrow$ |   BA/ASR   |   BA/ASR    |      BA/ASR      |   BA/ASR    |
|                 No Defense                 | 94.92/100  | 99.14/98.25 |   94.82/99.19    | 94.29/98.64 |
|                SS with SCE                 | 82.34/5.12 | 82.30/6.24  |    81.81/5.43    | 81.15.7.08  |
|                 DBD (ours)                 | 92.41/0.96 | 92.18/1.73  |    91.45/0.34    | 91.20/0.39  |

Note:
1. No Defense: standard end-to-end supervised training on all samples.
1. SS with SCE: freeze the learned backbone and train remaining FC layers on all samples with SCE loss.

---

**Q3**: In the fine-tuning step, the sensitivity of lambda should also be discussed, since it controls the ratios of unlablelled data.

**R3**: Thank you for this constructive suggestion! We agree with you that the sensitivity test of hyper-parameters is very important to evaluate one method. Actually, we have done this test in Section 5.3.1 (also see Figure 5) of our original manuscript. According to your description that "it controls the ratios of unlabeled data", we guess the 'lambda' you mentioned should be $\alpha$, i.e., the filtering rate. As shown in Figure 5, our method has a very stable and good performance (high benign accuracy and low ASR) in a wide range of $\alpha$ from $30\%$ to $60\%$. It demonstrates that our method is not sensitive to the filtering rate in fine-tuning.

---

**Q4**: In Figure 1(a), the poisoned samples' embedding seems far from the target label (label 3) and closer to other un-targeted labels (label 1, 7, and 9). It would be great if the authors could give explanations.

**R4**: Thank you for the insightful observation. Many works have demonstrated that t-SNE is designed for preserving the *local structure* of the data, rather than the *global structure*. As described in this blog (https://distill.pub/2016/misread-tsne/), both the distance between clusters and the cluster sizes might not mean anything. Meanwhile, our method also only cares about the local structure, i.e., whether poisoned samples gather to one cluster or not. Hope your concern has been well answered.

---

 ---

**Q2**: The primary assumption of the paper is that learning in semi-supervised learning is safe. However, Carlini [1] 's recent work demonstrates the attacker's effectiveness under the same threat model, i.e., when the attacker is only allowed to poison data. If the poisoning is efficient, then the proposed defense exposes the model to a different attack.

**R2**: Thank you for this insightful comment and the reminding of this latest work [1]! After carefully reading about this work, we would like to clarify one important misunderstanding and several intrinsic differences between Carlini’s work and our work. 
-  **'Learning in semi-supervised learning is safe' IS NOT our assumption!**  Our main motivation includes two levels: the standard supervised learning is not safe, as the poisoned samples are likely to gather together to insert the backdoor into the trained model; the self-supervised learning is safe, as they can prevent the gathering of poisoned samples, such that the backdoor insertion is blocked. Although semi-supervised learning is also adopted as the 3rd stage in our DBD method, it is not our main motivation. And, this semi-supervised paradigm is constructed by ourselves, through filtering some low-credible samples in stage 2. 
- **The threat models between Carlini's work and our work are DIFFERENT!** As clearly demonstrated in Section 2.4 of Carlini's work, its threat model is *We consider a victim who trains a machine learning model on a dataset with limited labeled examples. ... The adversary's goal is to poison the victim's model so that the model will classify the selected example as the desired target, *i.e.*, $f_{\theta}(x^\*) = y^\*$*. In contrast, the threat model we consider is the poisoning-based backdoor attacks on fully labeled datasets. In short, both the attack objects (unlabelled samples *v.s.* labeled samples) and the attack goals (the specific selected sample *v.s.* any sample with the trigger) are different.

According to the above two points and our current understanding of [1], we don’t believe that the semi-supervised attack studied in Carlini’s work is a threat to our DBD defense. If we miss or misunderstand some points of [1], please tell us, and we are willing to further discuss this interesting concern. 

However, we still very much appreciate this insightful concern, which inspires us to seriously think about the possible attack to evade our DBD defense. Actually, we have provided some thoughts and evaluations in Section 5.4 ‘Resistance to potential adaptive attacks’. We will continue the exploration of the possible adaptive attacks in our future work.

---
Reference

[1] Carlini, Nicholas. Poisoning the Unlabeled Dataset of Semi-Supervised Learning. USENIX Security, 2021.

---

 We sincerely thank you for your valuable time and comments. We are encouraged by the positive comments on the **novelties**, **extensive experiments**, **good performance**, and **benefits to the field**.

---

**Q1**: Did not compare DBD with some recently works such as "Spectral signatures in backdoor attacks" in NIPS 2018 and "Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering" in AAAI 2019, which both are based on data removal. It is unclear how differently DBD removes the poisoned data as compared with the existing works.


**R1**: Thanks for these insightful comments!

+ We do notice these recent works mentioned by you. These detection-based methods (e.g., Spectral Signatures (SS) and Activation Clustering (AC)) filter malicious samples from the training set and train the model on non-malicious samples. While in our DBD, we do not intend to accurately separate poisoned and benign samples as we mentioned in Section 4.4 of our original manuscript.
+ However, we do understand your concern. To alleviate it, we compare the filtering ability of our DBD (stage 2) with your suggested methods. Since SS filters $1.5 \varepsilon$ malicious samples for each class, where $\varepsilon$ is the key hyper-parameter means the upper bound of the number of poisoned training samples, we adopt different $\varepsilon$ for a fair comparison. For AC, we use the default setting as suggested by the original paper. As shown in Table 1-2, the filtering performance of DBD is on par with that of SS and AC. DBD is even better than those methods when filtering poisoned samples generated by more complicated attacks (i.e., WaNet and Label-Consistent).
+ Besides, we also conduct the standard training on non-malicious samples filtered by SS and AC. As shown in Table 3, the hidden backdoor will still be created in many cases, even though the detection-based defenses are sometimes accurate. This is mainly because these methods may not able to remove enough poisoned samples while preserving enough benign samples simultaneously, i.e., there is a trade-off between BA and ASR.

Please refer to Appendix L in our revised manuscript for more results.



Table 1. The successful filtering rate (# filtered poisoned samples /# all filtered samples, %) under different $\varepsilon$ in the target class on CIFAR-10 dataset.

|          $\epsilon \rightarrow$          |    250     |     500     |    1000     |    1500     |
|:----------------------------------------:|:----------:|:-----------:|:-----------:|:-----------:|
| Attack$\downarrow$, Defense$\rightarrow$ |   SS/DBD   |   SS/DBD    |   SS/DBD    |   SS/DBD    |
|                 BadNets                  | 95.73/100  | 93.20/97.60 | 87.60/90.87 | 80.71/70.09 |
|                 Blended                  | 0.80/97.87 | 10.53/94.67 | 28.87/87.27 | 35.29/75.16 |
|                  WaNet                   |   0/100    |    0/100    | 1.00/99.47  | 7.42/97.46  |
|             Label-Consistent             | 2.40/43.47 | 4.40/37.07  | 9.00/34.47  | 13.78/32.53 |



Table 2. The number of remaining poisoned samples over filtered non-malicious samples on CIFAR-10 dataset. 

|                      |  BadNets   |  Blended   |   WaNet    | Label-Consistent |
|:--------------------:|:----------:|:----------:|:----------:|:----------------:|
| SS ($\epsilon=500$)  | 1801/42500 | 2421/42500 | 2500/42500 |    1217/42500    |
| SS ($\epsilon=1000$) | 1186/35000 | 2067/35000 | 2400/35000 |    1115/35000    |
|          AC          |  0/42500   |  0/37786   | 5000/45546 |    1250/39998    |
|         DBD          |  8/25000   |  6/25000   |  38/25000  |     13/25000     |



Table 3. The BA (%) and ASR (%) of models trained on non-malicious samples filtered by SS and AC on CIFAR-10 dataset.

|           Defense $\rightarrow$           | SS($\varepsilon=500$) | SS($\varepsilon=1000$) |     AC      |
|:-----------------------------------------:|:---------------------:|:----------------------:|:-----------:|
| Attack $\downarrow$, Metric $\rightarrow$ |        BA/ASR         |         BA/ASR         |   BA/ASR    |
|                  BadNets                  |       92.99/100       |      93.27/99.99       |   85.90/0   |
|                  Blended                  |      92.84/99.07      |      92.56/99.18       |   77.17/0   |
|                   WaNet                   |      92.69/98.13      |      91.92/99.00       | 84.60/99.02 |
|             Label-Consistent              |      92.93/99.79      |      92.88/99.86       | 75.95/99.75 |

---

 
**Q4**: I also suggest the authors to move the section about the resistance to adaptive attacks in Appendix into the main paper as the adaptive attacks are becoming a more serious threats today. Please explain your adaptive attack settings more clearly (for example, what trigger size you used and how you tuned the hyper-parameters). It would also be good if the author discuss how an attack may work around the proposed defense, and how to further defend such workarounds.

**R4**: Thank you for this constructive suggestion! Since the self-supervised learning stage plays a key role in our DBD defense, one possible adaptive attack is encouraging the gathering of poisoned samples after self-supervised learning. However, in our threat model, the attacker can only manipulate the dataset. Here we relax the setting a bit to assume that the attacker also knows the network architecture used in our DBD. In this case, the attacker can firstly train a model using some self-supervised learning methods, then optimize a trigger pattern $t$ for this model, to ensure the gathering of poisoned samples in the feature space. The optimization objective is presented in Eq. (4) of our original manuscript, which minimizes the distance between poisoned images, while maximizing the distances between the center of poisoned images and cluster centers of benign images. The detailed settings and results are listed below. 
+ **Settings**. We conduct the experiment on CIFAR-10 dataset. We found that the trigger size of $2\times 2$ used in our main experiments failed to achieve the goal that poisoned samples gather to one cluster after self-supervised learning. Thus, we gradually increase the trigger size, and finally, we adopt an extreme trigger size of $32\times 32$ in this experiment to ensure the attack effectively. This size means that the poisoned image is fully covered by the trigger, while its original semantic information and the stealthiness of the attack are ignored. It reveals that gathering poisoned samples together is very difficult in self-supervised learning. 
+ **Results**. As shown in Appendix H of our orginal manuscript, the adaptive attack works well when there is no defense (BA=94.96%, ASR=99.70%). However, this attack still fails to attack our DBD (BA=93.21%, ASR=1.02%). The most probable reason is that the optimized trigger is overfitted to the model trained by the attacker, and the poisoned samples with this optimized trigger will still be separated in the model trained in the first stage of our DBD defense. 

We have moved the adaptive attack from Appendix H to the main manuscript (Section 5.4) and provided more details in our revised manuscript. Thanks again for this insightful suggestion. In our future work, we will keep the exploration of more effective adaptive attacks against DBD, as well as further defenses.

---
Reference

[1] He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning." CVPR. 2020.

[2] Chen, Xinlei, et al. "Improved baselines with momentum contrastive learning." arXiv preprint arXiv:2003.04297 (2020).

[3] Jean-Bastien Grill, et al. Bootstrap your own latent - a new approach to self-supervised learning. In NeurIPS, 2020.

---

Summary: authors propose a modification to the training procedure to prevent backdoor attacks. Instead of performing supervised training, they suggest first training the model in a self-supervised way, then in a supervised way on fully connected layers. Later they propose to remove low-credible samples and fine-tune the whole model on the remaining samples with labels.  They claim that this procedure eliminates the backdoored inputs that have incorrect labels. Strengths: I enjoyed reading the paper as it has a good structure, strong sets of experiments and baselines. Instead of searching for the triggers as NC and other papers are trying to do, this paper proposes to address the label discrepancy introduced by backdoor samples.

Weaknesses: 
1. The proposed method modifies an underlying training procedure multiplying training time, which I think is significant for practitioners. Addressing this issue is essential to support the practicality of the method.
2. The primary assumption of the paper is that learning in semi-supervised learning is safe. However, Carlini [1] 's recent work demonstrates the attacker's effectiveness under the same threat model, i.e., when the attacker is only allowed to poison data. If the poisoning is efficient, then the proposed defense exposes the model to a different attack.

[1] Carlini, N. (2021). Poisoning the Unlabeled Dataset of Semi-Supervised Learning. USENIX Security'21 Overall, I believe the method is useful, however the above questions are important to be addressed.

The paper uses self-supervised learning to get benign representation and then uses a noisy label algorithm (SCE) to optimize the prediction model. The empirical performance shows the effectiveness of the proposed method.  Strengths:
* The proposed method is quite intuitive and easy to be extended to the more advanced methods.
* The experiment part shows the effectiveness of the proposed method.

Weakness:
* The paper lacks a theoretical analysis of the proposed method. I understand that this paper mainly focused on empirical performance, but * it is quite surprising that the proposed methods perform well on label-consistent attacks. This is because that proposed method decouples the label corruption and feature corruption. When label corruption no longer exists, what is the advantage of the proposed method?
* For the second step label noise learning, there are also many choices instead of just using the symmetric cross-entropy method. Investigating more noisy-label algorithms might be interesting.
* The two-step method makes the algorithm not in an end-to-end fashion. It would be interesting to investigate the possibility to make an end-to-end algorithm.
* I do not think excluding the detection-based method is fair since those methods are strong baselines especially for the badnet and blending attack. Also, since the main contribution of the paper is the empirical performance, it is necessary to compare with different kinds of baselines. I think the proposed method is natural and simple, which makes the paper easy to be extended to different settings. I recommend weak acceptance since the experiment part did not fully convince me.

This paper proposes a decoupling-based backdoor defense (DBD) on poisoning-based backdoor attacks where an adversary can modify the dataset only. Specifically, DBD combines a self-supervised feature extractor and a supervised noise-free classifier, with an additional semi-supervised learning fine-tuning step. The core idea is to decouple the feature extractor and the final prediction. The authors evaluate the effectiveness of DBD on three datasets, three backdoor attack models, and four defense baselines.
 This paper proposes decoupling-based backdoor defense (DBD) on poisoning-based backdoor attacks. Specifically, the authors first give a broad view of why backdoor works by visualizing the poisoned and benign data in the embedding space. They found that the supervised learning paradigm tends to cause poisoned data close to its target label rather than the ground truth label. To mitigate this, the DBD first uses a self-supervised learning to learn a purified feature extractor. It splits the training dataset into two disjoint parts by comparing the fully connected layers' training loss. Later, it uses the high-credible and low-credible samples to fine-tune the whole model in a semi-supervised manner. Experiments are performed on three datasets, three backdoor attack models, and four defense baselines. The authors also consider multiple potential scenarios in practice, such as trigger patterns, adaptive backdoor attacks.

The idea of this paper is simple but effective, and the authors' efforts in the experiments are highly appreciated. Details comments are as follows.

The primary concern of this paper is the extra computation cost of the proposed DBD pipeline. Considering training a self-supervised and semi-supervised model costs way more computational resources than a supervised learning model. Would it be possible to use a public pre-trained feature extractor to replace the self-supervised feature extractor? The authors are welcomed to discuss this.

In Section 5.2, the authors list the results on "No defense" to show the backdoor defenses' impact on the original model's accuracy and the effectiveness of the backdoor mitigation. It is not clear how the model works without a defense. Suppose the authors train the original model in a supervised learning manner. In that case, directly using a self-supervised learning paradigm should also be included to illustrate each step's contribution.

In the fine-tuning step, the sensitivity of lambda should also be discussed, since it controls the ratios of unlablelled data.

In Figure 1(a), the poisoned samples' embedding seems far from the target label (label 3) and closer to other un-targeted labels (label 1, 7, and 9). It would be great if the authors could give explanations.
 I like this paper and would support an accept.

Meta-review: Inspired by the observtion that the poisoned samples tend to cluster together in the feature space of the attacked DNN model, which is mostly due to the end-to-end supervised training paradigm, the authors propose a novel defense method based on contrastive learning and decouple end-to-end training to defend against backdoor attacks.
The issues, including training time, difference from certain previous studies, ablation study, and so on, raised by the reviewers have been properly addressed and the reviewers are satisfied with the responses from the authors.
According to the consistent positive opinions from the reviewrs, this manuscript is recommended to accept.

Decision: Accept (Poster)