B ACKDOOR D EFENSE VIA D ECOUPLING THE T RAINING P ROCESS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent studies have revealed that deep neural networks (DNNs) are vulnerable to backdoor attacks, where attackers embed hidden backdoors in the DNN model by poisoning a few training samples.
The attacked model behaves normally on benign samples, whereas its prediction will be maliciously changed when the backdoor is activated.
We reveal that poisoned samples tend to cluster together in the feature space of the attacked DNN model, which is mostly due to the endto-end supervised training paradigm.
Inspired by this observation, we propose a novel backdoor defense via decoupling the original end-to-end training process into three stages.
Speciﬁcally, we ﬁrst learn the backbone of a DNN model via self-supervised learning based on training samples without their labels.
The learned backbone will map samples with the same ground-truth label to similar locations in the feature space.
Then, we freeze the parameters of the learned backbone and train the remaining fully connected layers via standard training with all (labeled) training samples.
Lastly, to further alleviate side-effects of poisoned samples in the second stage, we remove labels of some ‘low-credible’ samples determined based on the learned model and conduct a semi-supervised ﬁne-tuning of the whole model.
Extensive experiments on multiple benchmark datasets and
DNN models verify that the proposed defense is effective in reducing backdoor threats while preserving high accuracy in predicting benign samples.
I NTRODUCTION
Deep learning, especially deep neural networks (DNNs), has been widely adopted in many realms (Wang et al., 2020c; Li et al., 2020a; Wen et al., 2020) for its high effectiveness.
In general, the training of DNNs requires a large amount of training samples and computational resources.
Accordingly, third-party resources ( e.g. , third-party data or servers) are usually involved.
While the opacity of the training process brings certain convenience, it also introduces new security threats.
Backdoor attack 1 poses a new security threat to the training process of DNNs (Li et al., 2020c).
It maliciously manipulates the prediction of the attacked DNNs by poisoning a few training samples.
Speciﬁcally, backdoor attackers inject the backdoor trigger ( i.e. , a particular pattern) to some benign training images and change their labels with the attacker-speciﬁed target label .
The connection between the backdoor trigger and the target label will be learned by DNNs during the training process.
In the inference process, the prediction of attacked DNNs will be changed to the target label when the trigger is present, whereas the attacked DNNs will behave normally on benign samples.
As such, users are difﬁcult to realize the existence of hidden backdoors and therefore this attack is a serious threat to the practical applications of DNNs.
In this paper, we ﬁrst investigate backdoor attacks from the hidden feature space.
Our preliminary experiments reveal that the backdoor is embedded in the feature space, i.e. , samples with the backdoor trigger (dubbed poisoned samples ) tend to cluster together in the feature space.
We reveal that this phenomenon is mostly due to the end-to-end supervised training paradigm .
Speciﬁcally, the excessive learning capability allows DNNs to learn features about the backdoor trigger, while the DNNs can shrink the distance between poisoned samples in the feature space and connect the learned trigger-related features with the target label by the end-to-end supervised training.
Based on this understanding, we propose to decouple the end-to-end training process for the backdoor defense.
Speciﬁcally, we treat the DNNs as two disjoint parts, including a feature extractor ( i.e. , backbone) and a simple classiﬁer ( i.e. , the remaining fully connected layers).
We ﬁrst learn the puriﬁed feature extractor via self-supervised learning (Kolesnikov et al., 2019; Chen et al., 2020a; Jing & Tian, 2020) with unlabeled training samples (obtained by removing their labels), and then learn the simple classiﬁer via standard supervised training process based on the learned feature extractor and all training samples.
The strong data augmentations involved in the self-supervised learning damage trigger patterns, making them unlearnable during representation learning; and the decoupling process further disconnects trigger patterns and the target label.
Accordingly, hidden backdoors cannot be successfully created even the model is trained on the poisoned dataset based on our defense.
Moreover, we further reveal that the representation of poisoned samples generated by the puriﬁed extractor is signiﬁcantly different from those generated by the extractor learned with standard training process.
Speciﬁcally, the poisoned sample lies closely to samples with its ground-truth label instead of the target label.
This phenomenon makes the training of the simple classiﬁer similar to label-noise learning (Wang et al., 2019b; Ma et al., 2020; Berthon et al., 2021).
As such, we ﬁrst ﬁlter high-credible training samples ( i.e. , training samples that are most probably to be benign) and then use those samples as labeled samples and the remaining part to form unlabeled samples to ﬁne-tune the whole model via semi-supervised learning (Rasmus et al., 2015; Berthelot et al., 2019;
Sohn et al., 2020).
This approach is to further reduce the adverse effects of poisoned samples.
The main contributions of this paper are three- fold.
(1) We reveal that the backdoor is embedded in the feature space, which is mostly due to the end-to-end supervised training paradigm.
(2) Based on our understanding, we propose a decoupling-based backdoor defense (DBD) to alleviate the threat of poisoning-based backdoor attacks.
(3) Experiments on classical benchmark datasets are conducted, which verify the effectiveness of our defense.
R ELATED W ORK 2. B ACKDOOR A TTACK
Backdoor attack is an emerging research area, which raises security concerns about training with third-party resources.
In this paper, we focus on the poisoning-based backdoor attack towards image classiﬁcation, where attackers can only modify the dataset instead of other training components ( e.g. , training loss).
This threat could also happen in other tasks (Bagdasaryan et al., 2020; Jia et al., 2021; Zhai et al., 2021) and with different attacker’s capacities (Nguyen & Tran, 2020; Tang et al., 2020; Bagdasaryan & Shmatikov, 2021), which are out-of-scope of this paper.
In general, existing attacks can be divided into two main categories based on the property of target labels, as follows:
Poison-Label Backdoor Attack.
It is currently the most common attack paradigm, where the target label is different from the ground-truth label of poisoned samples.
BadNets (Gu et al., 2019) is the ﬁrst and most representative poison-label attack.
Speciﬁcally, it randomly selected a few samples from the original benign dataset to generate poisoned samples by stamping the backdoor trigger onto the (benign) image and change their label with an attacker-speciﬁed target label .
Those generated poisoned samples associated with remaining benign ones were combined to form the poisoned training dataset , which will be delivered to users.
After that, (Chen et al., 2017) suggested that the poisoned image should be similar to its benign version for the stealthiness, based on which they proposed the blended attack .
Recently, (Xue et al., 2020; Li et al., 2020b) further explored how to conduct poison-label backdoor attacks more stealthily.
Most recently, a more stealthy and effective attack, the WaNet (Nguyen & Tran, 2021), was proposed.
WaNet adopted image warping as the backdoor trigger, which deforms but preserves the image content.
Clean-Label Backdoor Attack.
Although the poisoned image generated by poison-label attacks could be similar to its benign version, users may still notice the attack by examining the image-label relationship.
To address this problem, Turner et al.
(2019) proposed the clean-label attack paradigm , where the target label is consistent with the ground-truth label of poisoned samples.
Speciﬁcally, they ﬁrst leveraged adversarial perturbations or generative models to modify some benign images from the target class and then conducted the standard trigger injection process.
This idea was generalized to attack video classiﬁcation in (Zhao et al., 2020b), where they adopted the targeted universal adversarial perturbation (Moosavi-Dezfooli et al., 2017) as the trigger pattern.
Although clean-label backdoor attacks are more stealthy compared with poison-label ones, they usually suffer from relatively poor performance and may even fail in creating backdoors (Li et al., 2020c).
2. B ACKDOOR D EFENSE
Currently, there are also some approaches to alleviate the backdoor threat.
Existing defenses are mostly empirical , which can be divided into ﬁve main categories, including (1) detection-based defenses (Wang et al., 2020b; Xu et al., 2021; Zeng et al., 2021a), (2) preprocessing based defenses (Doan et al., 2020; Li et al., 2021b;
Zeng et al., 2021b), (3) model reconstruction based defenses (Liu et al., 2018; Zhao et al., 2020a; Li et al., 2021a), (4) trigger synthesis based defenses (Guo et al., 2020;
Dong et al., 2021;
Shen et al., 2021), and (5) poison suppression based defenses (Du et al., 2020; Borgnia et al., 2021).
Speciﬁcally, detection-based defenses examine whether a suspicious
DNN or sample is attacked and it will deny the use of malicious objects; Preprocessing based methods intend to damage trigger patterns contained in attack samples to prevent backdoor activation by introducing a preprocessing module before feeding images into DNNs; Model reconstruction based ones aim at removing the hidden backdoor in DNNs by modifying models directly; The fourth type of defenses synthesize potential trigger patterns at ﬁrst, following by the second stage that the hidden backdoor is eliminated by suppressing their effects; The last type of methods depress the effectiveness of poisoned samples during the training process to prevent the creation of hidden backdoors.
In general, our method is most relevant to this type of defenses.
In this paper, we only focus on the last four types of defenses since they directly improve the robustness of DNNs.
Besides, there were also few works focusing on certiﬁed backdoor defenses (Wang et al., 2020a;
Weber et al., 2020).
Their robustness is theoretically guaranteed under certain assumptions, which cause these methods to be generally weaker than empirical ones in practice.
2. S EMI - SUPERVISED AND S ELF - SUPERVISED L EARNING
Semi-supervised Learning.
In many real-world applications, the acquisition of labeled data often relies on manual labeling, which is very expensive.
In contrast, obtaining unlabeled samples is much easier.
To utilize the power of unlabeled samples with labeled ones simultaneously, a great amount of semi-supervised learning methods were proposed (Gao et al., 2017; Berthelot et al., 2019;
Van Engelen & Hoos, 2020).
Recently, semi-supervised learning was also introduced in improving the security of DNNs (Stanforth et al., 2019; Carmon et al., 2019), where they utilized unlabelled samples in the adversarial training.
Most recently, (Yan et al., 2021) discussed how to backdoor semi-supervised learning.
However, this approach needs to control other training components ( e.g. , training loss) in addition to modifying training samples and therefore is out-of-scope of this paper.
How to adopt semi-supervised learning for backdoor defense remains blank.
Self-supervised Learning.
This learning paradigm is a subset of unsupervised learning , where
DNNs are trained with supervised signals generated from the data itself (Chen et al., 2020a; Grill et al., 2020; Liu et al., 2021).
It has been adopted for increasing adversarial robustness (Hendrycks et al., 2019; Wu et al., 2021; Shi et al., 2021).
Most recently, there were also a few works (Saha et al., 2021; Carlini & Terzis, 2021; Jia et al., 2021) exploring how to backdoor self-supervised learning.
However, these attacks are out-of-scope of this paper since they need to control other training components ( e.g. , training loss) in addition to modifying training samples.
R EVISITING B ACKDOOR A TTACKS FROM THE H IDDEN F EATURE S PACE
In this section, we analyze the behavior of poisoned samples from the hidden feature space of attacked models and discuss its inherent mechanism.
Settings.
We conduct the BadNets (Gu et al., 2019) and label-consistent attack (Turner et al., 2019) on CIFAR-10 dataset (Krizhevsky, 2009) for the discussion.
They are representative of poison-label attacks and clean-label attacks, respectively.
Speciﬁcally, we conduct supervised learning on the poisoned datasets with the standard training process and self-supervised learning on the unlabelled poisoned datasets with SimCLR (Chen et al., 2020a).
We visualize poisoned samples in the hidden feature space generated by attacked DNNs based on the t-SNE (Van der Maaten & Hinton, 2008).
More detailed settings are presented in Appendix A.
[Figure]
Results.
As shown in Figure 1(a)-1(b), poisoned samples (denoted by ‘black-cross’) tend to cluster together to form a separate cluster after the standard supervised training process, no matter under the poison-label attack or clean-label attack.
This phenomenon implies why existing poisoning-based backdoor attacks can succeed.
Speciﬁcally, the excessive learning capability allows DNNs to learn features about the backdoor trigger.
Associated with the end-to-end supervised training paradigm,
DNNs can shrink the distance between poisoned samples in the feature space and connect the learned trigger-related features with the target label.
In contrast, as shown in Figure 1(c)-1(d), poisoned samples lie closely to samples with their ground-truth label after the self-supervised training process on the unlabelled poisoned dataset.
It indicates that we can prevent the creation of backdoors by self-supervised learning, which will be further introduced in the next section.
D ECOUPLING - BASED B ACKDOOR D EFENSE 4. P RELIMINARIES
General Pipeline of Backdoor Attacks.
Let D =
( x i , y i ) }
Ni =1 denotes the benign training set, where x i ∈
X
= { 0 , 1 , . .
, 255 } C × W × H is the image, y i ∈ Y = { 0 , 1 , . . .
, K } is its label, K is the number of classes, and y t ∈ Y indicates the target label .
How to generate the poisoned dataset D p is the cornerstone of backdoor attacks.
Speciﬁcally, D p consists of two subsets, including the modiﬁed version of a subset of D and remaining benign samples, i.e. , D p =
D m ∪ D b , where D b ⊂ D , γ (cid:44) |D
m
| |D| is the poisoning rate , D m = { ( x (cid:48) , y t ) | x (cid:48)
= G ( x ) , ( x , y ) ∈ D\D b } , and G : X → X is an attacker-predeﬁned poisoned image generator.
For example, G ( x ) = ( 1 − λ ) ⊗ x + λ ⊗ t , where λ ∈ [0 , 1] C × W × H , t ∈ X is the trigger pattern, and ⊗ is the element-wise product in the blended attack (Chen et al., 2017).
Once D p is generated, it will be sent to users who will train DNNs on it.
Hidden backdoors will be created after the training process.
Threat Model.
In this paper, we focus on defending against poisoning-based backdoor attacks.
The attacker can arbitrarily modify the training set whereas cannot change other training components ( e.g. , model structure and training loss).
For our proposed defense, we assume that defenders can fully control the training process.
This is the scenario that users adopt third-party collected samples for training.
Note that we do not assume that defenders have a local benign dataset, which is often required in many existing defenses (Wang et al., 2019a;
Zhao et al., 2020a; Li et al., 2021a).
Defender’s Goals.
The defender’s goals are to prevent the trained DNN model from predicting poisoned samples as the target label and to preserve the high accuracy on benign samples.
4. O VERVIEW OF THE D EFENSE P IPELINE
In this section, we describe the general pipeline of our defense.
As shown in Figure 2, it consists of three main stages, including (1) learning a puriﬁed feature extractor via self-supervised learning, (2) ﬁltering high-credible samples via label-noise learning, and (3) semi-supervised ﬁne-tuning.
[Figure]
Speciﬁcally, in the ﬁrst stage, we remove the label of all training samples to form the unlabelled dataset, based on which to train the feature extractor via self-supervised learning.
In the second stage, we freeze the learned feature extractor and adopt all training samples to train the remaining fully connected layers via supervised learning.
We then ﬁlter α % high-credible samples based on the training loss.
The smaller the loss, the credible the sample.
After the second stage, the training set will be separated into two high-credible and lowcredible samples.
We use high-credible samples as labeled samples and the label of all low-credible samples to ﬁne-tune the whole model via semi-supervised learning.
More detailed information about each stage of our method will be further illustrated in following sections.
4. L EARNING P URIFIED F EATURE E XTRACTOR VIA S ELF - SUPERVISED L EARNING
Let D t denotes the training set and f w : X → [0 , 1] K indicates the DNN with parameter w =
[ w c , w f ] , where w c and w f indicates the parameters of the backbone and the fully connected layer, respectively.
In this stage, we optimize w c based on the unlabeled version of D t via self-supervised learning, as follows:
[Equation]
where L 1 ( · ) indicates the self-supervised loss ( e.g. , NT-Xent in SimCLR (Chen et al., 2020a)).
Through the self-supervised learning, the learned feature extractor ( i.e. , backbone) will be puriﬁed even if the training set contains poisoned samples, as illustrated in Section 3.
4. F ILTERING H IGH - CREDIBLE S AMPLES VIA L ABEL - NOISE L EARNING
Once w ∗ c is obtained, the user can freeze it and adopt D t to further optimize remaining w f , i.e. ,
[Equation]
where L 2 ( · ) indicates the supervised loss ( e.g. , cross entropy).
After the decoupling-based training process (1)-(2), even if the model is (partly) trained on the poisoned dataset, the hidden backdoor cannot be created since the feature extractor is puriﬁed.
However, this simple strategy suffers from two main problems.
Firstly, compared with the one trained via supervised learning, the accuracy of predicting benign samples will have a certain decrease, since the learned feature extractor is frozen in the second stage.
Secondly, poisoned samples will serve as ‘outliers’ to further hinder the learning of the second stage when poison-label attacks appear, since those samples lie close to samples with its ground-truth label instead of the target label in the hidden feature space generated by the learned puriﬁed feature extractor.
These two problems indicate that we should remove poisoned samples and retrain or ﬁne-tune the whole model .
[Figure]
Speciﬁcally, we select high-credible samples D h based on the loss L 2 ( · ; [ w c , w f ]) .
The highcredible samples are deﬁned as the % training samples with the smallest loss, where α [0 , 100] is a hyper-parameter.
In particular, we adopt the symmetric cross-entropy (SCE) (Wang et
2019b)
as L 2 ( · ) , inspired by the label-noise learning.
As shown in Figure 3, compared with the CE loss, the
SCE can signiﬁcantly increase the differences between poisoned samples and benign ones, which further reduces the possibility that high-credible dataset D h still contains poisoned samples.
Note that we do not intend to accurately separate poisoned samples and benign samples .
We only want to ensure that the obtained D h contains as few poisoned samples as possible.
4. S EMI - SUPERVISED F INE - TUNING
After the second stage, the third-party training set D t will be separated into two disjoint parts, including the high-credible dataset D h and the low-credible dataset D l (cid:44)
D t \D h .
Let ˆ D l (cid:44) { x | ( x , y ) ∈
D l } indicates the unlabeled version of low-credible dataset D l .
We ﬁne-tune the whole trained model f [ w ∗ c , w ∗ f ] ( · ) with semi-supervised learning as follows:
[Equation]
where L 3 ( · ) denotes the semi-supervised loss ( e.g. , the loss in MixMatch (Berthelot et al., 2019)).
This process can prevent the side-effects of poisoned samples while utilizing their contained useful information, and encourage the compatibility between the feature extractor and the simple classiﬁer via learning them jointly instead of separately.
Please refer to Section 5.3 for more results.
E XPERIMENTS 5. E XPERIMENTAL S ETTINGS
Datasets and DNNs.
We evaluate all defenses on two classical benchmark datasets, including
CIFAR-10 (Krizhevsky, 2009) and (a subset of)
ImageNet (Deng et al., 2009).
We adopt the ResNet18 (He et al., 2016) for these tasks.
More detailed settings are presented in Appendix B.1.
Besides, we also provide the results on (a subset of)
VGGFace2 (Cao et al., 2018) in Appendix C.
Attack Baselines.
We examine all defense approaches in defending against four representative attacks.
Speciﬁcally, we select the BadNets (Gu et al., 2019), the backdoor attack with blended strategy (dubbed ‘Blended’)
(Chen et al., 2017), WaNet (Nguyen & Tran, 2021), and label-consistent attack with adversarial perturbations (dubbed ‘Label-Consistent’) (Turner et al., 2019) for the evaluation.
They are the representative of patch-based visible and invisible poison-label attacks, nonpatch-based poison-label attacks, and clean-label attacks, respectively.
Defense Baselines.
We compared our DBD with two defenses having the same defender’s capacities, including the DPSGD (Du et al., 2020) and ShrinkPad (Li et al., 2021b). We also compare with
[Figure] other two approaches with an additional requirement ( i.e. , having a local benign dataset), including the neural cleanse with unlearning strategy (dubbed ‘NC’)
(Wang et al., 2019a), and neural attention distillation (dubbed ‘NAD’) (Li et al., 2021a).
They are the representative of poison suppression based defenses, preprocessing based defenses, trigger synthesis based defenses, and model reconstruction based defenses, respectively.
We also provide results of DNNs trained without any defense (dubbed ‘No Defense’) as another important baseline for reference.
Attack Setups.
We use a 2 × 2 square as the trigger pattern on CIFAR-10 dataset and the 32 ×
Apple logo on ImageNet dataset for the BadNets, as suggested in (Gu et al., 2019; Wang et al., 2019a).
For Blended, we adopt the ‘Hello Kitty’ pattern on CIFAR-10 and the random noise pattern on ImageNet, based on the suggestions in (Chen et al., 2017), and set the blended ratio λ = 0 .on
all datasets.
The trigger pattern adopted in label-consistent attack is the same as the one used in
BadNets.
For WaNet, we adopt its default settings on CIFAR-10 dataset.
However, on ImageNet dataset, we use different settings optimized by grid-search since the original ones fail.
An example of poisoned samples generated by different attacks is shown in Figure 4.
Besides, we set the poisoning rate γ 1 = 2 .
5% for label-consistent attack (25% of training samples with the target label) and γ 2 = 5% for three other attacks.
More details are shown in Appendix B.2.
Defense Setups.
For our DBD, we adopt SimCLR (Chen et al., 2020a) as the self-supervised method and MixMatch (Berthelot et al., 2019) as the semi-supervised method.
More details about SimCLR and MixMatch are in Appendix I.
The ﬁltering rate α is the only key hyper-parameter in DBD, which is set to 50% in all cases.
We set the shrinking rate to 10% for the ShrinkPad on all datasets, as suggested in (Li et al., 2021b; Zeng et al., 2021b).
In particular, DPSGD and NAD are sensitive to their hyper-parameters.
We report their best results in each case based on the grid-search (as shown in Appendix D).
Besides, we split a 5% random subset of the benign training set as the local benign dataset for NC and NAD.
More implementation details are provided in Appendix B.3.
Evaluation Metrics.
We adopt the attack success rate (ASR) and benign accuracy (BA) to measure the effectiveness of all methods 2 .
Speciﬁcally, let D test indicates the (benign) testing set and C w :
X → Y denotes the trained classiﬁer, we have ASR (cid:44)
Pr ( x ,y ) ∈D test { C w ( G ( x )) =
y t | y (cid:54) =
y t } and BA (cid:44) Pr ( x
,y ) ∈D test { C w ( x ) =
y } , where y t is the target label and G ( · ) is the poisoned image generator.
In particular, the lower the ASR and the higher the BA, the better the defense .
5. M AIN R ESULTS
Comparing DBD with Defenses having the Same Requirements.
As shown in Table 1-2, DBD is signiﬁcantly better than defenses having the same requirements ( i.e. , DPSGD and ShrinkPad) in defending against all attacks.
For example, the benign accuracy of DBD is 20% over while the attack success rate is 5% less than that of DPSGD in all cases.
Speciﬁcally, the attack success rate of models with DBD is less than 2% in all cases (mostly < 0 .
5% ), which veriﬁes that our method can successfully prevent the creation of hidden backdoors.
Moreover, the decreases of benign accuracy are less than 2% when defending against poison-label attacks, compared with models trained without any defense.
Our method is even better on relatively larger dataset where all baseline methods become less effective.
These results verify the effectiveness of our method.
[Table]
[Table]
[Table] [Table]
SS
[Table]
Comparing DBD with Defenses having Extra Requirements.
We also compare our defense with two other methods ( i.e. , NC and NAD), which have an additional requirement that defenders have a benign local dataset.
As shown in Table 1-2, NC and NAD are better than DPSGD and ShrinkPad, as we expected, since they adopt additional information from the benign local dataset.
In particular, although NAD and NC use additional information, our method is still better than them, even when their performances are tuned to the best while our method only uses the default settings.
Speciﬁcally, the BA of NC is on par with that of our method.
However, it is with the sacriﬁce of ASR.
Especially on ImageNet dataset, NC has limited effects in reducing ASR.
In contrast, our method reaches the smallest ASR while its BA is either the highest or the second-highest in almost all cases.
These results verify the effectiveness of our method again.
Results.
As shown in Figure 7, our method can still prevent the creation of hidden backdoors even when the poisoning rate reaches 20%.
Besides, DBD also maintains high benign accuracy.
In other words, our method is effective in defending attacks with different strengths.
5. A BLATION S TUDY
There are four key strategies in DBD, including (1) obtaining puriﬁed feature extractor, (2) using
SCE instead of CE in the second stage, (3) reducing side-effects of low-credible samples, and (4) ﬁne-tuning the whole model via semi-supervised learning.
Here we verify their effectiveness.
Settings.
We compare the proposed DBD with its four variants, including (1) DBD without SS, (2)
SS with CE, (3) SS with SCE, and (4) SS with SCE + Tuning, on the CIFAR-10 dataset.
Speciﬁcally, in the ﬁrst variant, we replace the backbone generated by self-supervised learning with the one trained in a supervised fashion and keep other parts unchanged.
In the second variant, we freeze the backbone learned via self-supervised learning and train the remaining fully-connected layers with cross-entropy loss on all training samples.
The third variant is similar to the second one.
The only difference is that it uses symmetric cross-entropy instead of cross-entropy to train fully-connected layers.
The last variant is an advanced version of the third one, which further ﬁne-tunes fullyconnected layers on high-credible samples ﬁltered by the third variant.
Results.
As shown in Table 3, we can conclude that decoupling the original end-to-end supervised training process is effective in preventing the creation of hidden backdoors, by comparing our DBD with its ﬁrst variant and the model trained without any defense.
Besides, we can also verify the effectiveness of SCE loss on defending against poison-label backdoor attacks by comparing the second and third DBD variants.
Moreover, the fourth DBD variant has relatively lower ASR and
BA, compared with the third one.
This phenomenon is due to the removal of low-credible samples.
It indicates that reducing side-effects of low-credible samples while adopting their useful information is important for the defense.
We can also verify that ﬁne-tuning the whole model via semi-supervised learning is also useful by comparing the fourth variant and the proposed DBD.
5. R ESISTANCE TO P OTENTIAL A DAPTIVE A TTACKS
In our paper, we adopted the classical defense setting that attackers have no information about the defense.
Attackers may design adaptive attacks if they know the existence of our DBD.
The most straightforward idea is to manipulate the self-supervised training process so that poisoned samples are still in a new cluster after the self-supervised learning.
However, attackers are not allowed to do it based on our threat model about adopting third-party datasets. Despite this, attackers may design adaptive attacks by optimizing the trigger pattern to make poisoned samples still in a new cluster after the self-supervised learning if they can know the model structure used by defenders, as follows: Problem Formulation. For a K -classiﬁcation problem, let X (cid:48)
= { x i } Mi =1 indicates the benign images selected for poisoning,
X j = { x i } N j i =1 denotes the benign images with ground-truth label j , and g is a trained backbone.
Given an attacker-predeﬁned poisoned image generator G , the adaptive attack aims to optimize a trigger pattern t by minimizing the distance between poisoned images while maximizing the distance between the center of poisoned images and centers of clusters of benign images with different label, i.e.,
[Equation]
where g (cid:48)
(cid:44) 1 M (cid:80) x ∈X (cid:48) g
( G ( x ; t )) , g i (cid:44) 1 N i (cid:80) x ∈X i g ( x ) , and d is a distance metric.
Settings.
We adopt the CIFAR-10 dataset and use the (cid:96) 2 norm as the distance metric to conduct the experiment.
Speciﬁcally, we assume that attackers have the entire benign dataset, based on which they can train a backbone adopted in the ﬁrst stage of our DBD.
We use the Adam optimizer to solve the above optimization problem for 100 epochs with a learning rate of 0.1.
The trigger size is set to 32 × 32 , which means the attacker can completely modify the content of poisoned samples, regardless of its original semantic information and the stealthiness of the attack.
This setting is to ensure the attack ability, since clustering poisoned samples together is very difﬁcult in self-supervised learning.
Results.
The adaptive attack works well when there is no defense (BA=94.96%, ASR=99.70%).
However, this attack still fails to attack our DBD (BA=93.21%, ASR=1.02%).
In other words, our defense is resistant to this adaptive attack.
It is most probably because the trigger optimized based on the backbone is far less effective when the model is retrained since model parameters are changed due to the random initialization and the update of model weights during the training process.
C ONCLUSION
The mechanism of poisoning-based backdoor attacks is to establish a latent connection between trigger patterns and the target label during the training process.
In this paper, we revealed that this connection is learned mostly due to the end-to-end supervised training paradigm.
Motivated by this understanding, we proposed a decoupling-based backdoor defense, which ﬁrst learns the backbone via self-supervised learning and then the remaining fully-connected layers by the classical supervised learning.
We also introduced the label-noise learning method to determine high-credible and low-credible samples, based on which we ﬁne-tuned the whole model via semi-supervised learning.
Extensive experiments verify that our defense is effective on reducing backdoor threats while preserving high accuracy on predicting benign samples.
E THICS S TATEMENT
DNNs are widely adopted in many mission-critical areas ( e g. , face recognition) and therefore their security is of great signiﬁcance.
The vulnerability of DNNs to backdoor attacks raises serious concerns about using third-party training resources.
In this paper, we propose a general training pipeline to obtain backdoor-free DNNs, even if the training dataset contains poisoned samples.
This work has no ethical issues in general since our method is purely defensive and does not reveal any new vulnerabilities of DNNs.
However, we need to mention that our defense can be adopted only when training with untrusted samples, and backdoor attacks could happen in other scenarios.
People should not be too optimistic about eliminating backdoor threats.
R EPRODUCIBILITY S TATEMENT
The detailed descriptions of datasets, models, and training settings are in Appendix A-D. We also describe the computational facilities and cost in Appendix J-K.
The codes for reproducing main experiments of our DBD are anonymously open-sourced (as described in Appendix P)
A D ETAILED S ETTINGS FOR R EVISITING B ACKDOOR A TTACKS
Attack Setups.
We conduct the BadNets (Gu et al., 2019) and label-consistent attack (Turner et al., 2019) with the target label y t = 3 on the CIFAR-10 dataset (Krizhevsky, 2009).
The trigger patterns are the same as those presented in Section 5.2.
In particular, we implement the label-consistent attack with adversarial perturbations, as suggested in its original paper (Turner et al., 2019).
Specifically, we used the projected gradient descent (PGD) (Madry et al., 2018) to generate adversarial perturbations within the (cid:96) ∞ -ball where the maximum perturbation size (cid:15)
= 16 .
Training Setups.
We conduct supervised learning on the poisoned datasets with the standard training process and the self-supervised learning on the unlabelled poisoned datasets with the SimCLR (Chen et al., 2020a).
The supervised training is conducted based on the open-source code 3 .
Specifically, we use the SGD optimizer with momentum 0.9, weight decay of 5 × 10 − 4 , and an initial learning rate of 0.1.
The batch size is set to 128 and we train the ResNet-18 model 200 epochs.
The learning rate is decreased by a factor of 10 at epoch 100 and 150, respectively.
Besides, we add triggers before performing the data augmentation ( e.g. , random crop and horizontal ﬂipping).
For the self-supervised training, we use the stochastic gradient descent (SGD) optimizer with a momentum of 0.9, an initial learning rate of 0.4, and a weight decay factor of 5 × 10 − 4 .
We use a batch size of 512, and train the backbone for 1,000 epochs.
We decay the learning rate with the cosine decay schedule (Loshchilov & Hutter, 2016) without a restart.
Besides, we also adopt strong data augmentation techniques, including random crop and resize (with random ﬂip), color distortions, and Gaussian blur, as suggested in (Chen et al., 2020a).
All models are trained until converge.
t-SNE Visualization Settings.
We treat the output of the last residual unit as the feature representation and use the tsne-cuda library (Chan et al., 2019) to get the feature embedding of all samples.
To have a better visualization, we adopt all poisoned samples and randomly select 10% benign samples for visualizing models under the supervised learning, and adopt 30% poisoned samples and 10% benign samples for those under the self-supervised learning.
B D ETAILED S ETTINGS FOR M AIN E XPERIMENTS B. M ORE D ETAILS ABOUT D ATASETS AND DNN S
Due to the limitations of computational resources and time, we adopt a subset randomly selected from the original ImageNet.
More detailed information about the datasets and DNNs adopted in the main experiments of our paper is presented in Table 4.
B. M ORE D ETAILS ABOUT A TTACK S ETTINGS
Attack Setups.
We conduct the BadNets (Gu et al., 2019), blended attack (dubbed ‘Blended’)
(Chen et al., 2017), label-consistent attack (dubbed ‘Label-Consistent’)
(Turner et al., 2019), and WaNet (Nguyen & Tran, 2021) with the target label y t = 3 on all datasets.
The trigger patterns are the same as those presented in Section 5.2.
In particular, we set the blended ratio λ = 0 .
1 for the blended attack on all datasets and examine label-consistent attack with the maximum perturbation size (cid:15) ∈ { 16 , 32 } .
Besides, WaNet assumed that attackers can fully control the whole training process in its original paper.
However, we found that WaNet only modiﬁed training data while other training components ( e.g. , training loss, training schedule, and model structure) are the same as those used in the standard training process.
As such, we re-implement its code in the poisoningbased attack scenario based on its ofﬁcial code 4 .
Speciﬁcally, following the settings in its original paper, we set the noise rate ρ n = 0 .
2 , control grid size k = 4 , and warping strength s = 0 .
5 on
[Table] [Table]
[Table]
the CIFAR-10 dataset.
However, we found that the default k and s are too small to make the attack works on the ImageNet dataset (as shown in Table 5-6).
Besides, the ‘noise mode’ also signiﬁcantly reduces the attack effectiveness (as shown in Table 7).
As such, we set k = 224 and s = 1 and train models without the noise mode on the ImageNet dataset.
Training Setups.
On the CIFAR-10 dataset (Krizhevsky, 2009), the settings are the same as those described in Section A;
On the ImageNet dataset (Deng et al., 2009), we conduct experiments based on the open-source code 5 .
Speciﬁcally, we use the SGD optimizer with momentum 0.9, weight decay of 10 − 4 , and an initial learning rate of 0.1.
The batch size is set to 256 and we train the
ResNet-18 model 90 epochs.
The learning rate is decreased by a factor of 10 at epoch 30 and 60, respectively.
Besides, since the raw images in the ImageNet dataset are of different sizes, we resize them to 3 × 224 × 224 before adding triggers.
B. M ORE D ETAILS ABOUT D EFENSE S ETTINGS
Settings for NC.
We conduct reverse engineering and anomaly detection based on its open-source code 6 .
We implement the ‘unlearning’ method to patch attacked models, as suggested in its paper (Wang et al., 2019a).
We randomly select 5% benign training samples as the local benign dataset, which is used in the ‘unlearning’ process.
Unless otherwise speciﬁed, other settings are the same as those used in (Wang et al., 2019a).
Settings for NAD.
We implement this method based on its open-source code 7 .
The origin NAD only conducted experiments on the WideResNet model.
In our paper, we calculate the NAD loss over the last residual group for the ResNet-18.
The local benign dataset is the same as the one adopted in NC, which is used in the ﬁne-tuning and distillation process of NAD.
Unless otherwise speciﬁed, other settings are the same as those used in (Li et al., 2021a).
Settings for DPSGD.
The original DPSGD was conducted on the MNIST dataset implemented by the TensorFlow Framework.
In this paper, we re-implement it based on the differentially private
SGD method provided by the Opacus 8 .
Speciﬁcally, we replace the original SGD optimizer with the differentially private one, as suggested in (Du et al., 2020).
There are two important hyperparameters in DPSGD, including noise scales σ and the clipping bound C .
In the experiments, we set C = 1 and select the best σ by the grid-search.
[Figure]
[Table] [Table]
Settings for ShrinkPad.
We set the shrinking rate to 10% on all datasets, as suggested in (Li et al., 2021b; Zeng et al., 2021b).
Following their settings, we pad 0-pixels at the bottom right of the shrunk image to expand it to its original size.
Settings for our Defense.
In this ﬁrst stage, We adopt SimCLR (Chen et al., 2020a) to perform self-supervised learning.
We train backbones 100 instead of 1,000 epochs to reduce computational costs while preserving effectiveness.
Other settings are the same as those described in Section A.
We use the same settings across all datasets, models, and attacks; In the second stage, we use the Adam optimizer with a learning rate of 0.002 and set the batch size to 128.
We train the fully connected layers 10 epochs with the SCE loss (Wang et al., 2019b).
Two hyper-parameters involved in the
SCE ( i.e. , α and β in the original paper) are set to 0.1 and 1, respectively.
After that, we ﬁlter 50% high-credible samples.
We use the same settings across all datasets, models, and attacks; In the third stage, we adopt the MixMatch (Berthelot et al., 2019) for semi-supervised ﬁne-tuning with settings suggested in its original paper.
Speciﬁcally, we use the Adam optimizer with a learning rate of 0.002, the batch size of 64, and ﬁnetune the model 190 epochs on the CIFAR-10 and 80 epochs on the ImageNet dataset, respectively.
We set the temperature T = 0 .
5 and the weight of unsupervised loss λ u = 15 on the CIFAR-10 and λ u = 6 on the ImageNet dataset, respectively.
Moreover, we re-ﬁlter high-credible samples after every epoch of the third stage based on the SCE loss.
C D EFENDING AGAINST A TTACKS ON VGGF ACE 2 D ATASET
Dataset and DNN.
Due to the limitations of computational resources and time, we adopt a subset randomly selected from the original VGGFace2 (Cao et al., 2018).
More details are in Table 8.
Settings for Attacks.
For the training of models on the VGGFace2 dataset, the batch size is set to 32 and we conduct experiments on the DenseNet-121 model (Huang et al., 2017).
An example of poisoned samples generated by different attacks are in Figure 5.
Other settings are the same as those used on the ImageNet dataset.
Settings for Defenses.
For NAD, we calculate the NAD loss over the second to last layer for the
DenseNet-121.
Other settings are the same as those used on the ImageNet dataset.
[Table]
[Table] [Table]
Results.
As shown in Table 9, our defense still reaches the best performance even compared with
NC and NAD.
Speciﬁcally, the BA of NC is on par with that of our method whereas it is with the sacriﬁce of ASR.
These results verify the effectiveness of our defense again.
D S EARCHING B EST R ESULTS FOR DPSGD AND NAD
The effectiveness of DPSGD and NAD is sensitive to their hyper-parameters.
Here we search for their best results based on the criteria that ‘BA − ASR’ reaches the highest value after the defense.
D. S EARCHING B EST R ESULTS FOR DPSGD
In general, the larger the σ , the smaller the ASR while also the smaller the BA.
The results of
DPSGD are shown in Table 10-12, where the best results are marked in boldface.
[Table] [Table]
[Table] [Table]
D. S EARCHING B EST R ESULTS FOR NAD
We found that the ﬁne-tuning stage of NAD is sensitive to the learning rate.
We search the best initial learning rate from { 0 . 1 , 0 . 01 , 0 . 001 } .
As shown in Table 13-15, a very large learning rate signiﬁcantly reduces the BA, while a very small learning rate can not reduce the ASR effectively.
To keep a relatively large BA while maintaining a small ASR, we set η = 0 .
01 in the ﬁne-tuning stage.
The distillation stage of NAD is also sensitive to its hyper-parameter β .
We select the best β via the grid-search.
The results are shown in Table 16-
[Table] [Table]
[Table] [Table]
E D EFENDING AGAINST L ABEL -C ONSISTENT A TTACK WITH A S MALLER P OISONING R ATE
For the label-consistent attack, except for the 2.5% poisoning rate examined in the main manuscript, 0.6% is also an important setting provided in its original paper (Turner et al., 2019).
In this section, we compare different defenses against the label-consistent attack with poisoning rate γ = 0 .
As shown in Table
[Table]
when defending against label-consistent attack with a 0.6% poisoning rate
[Table]
our method is still signiﬁcantly better than defenses having the same requirements ( i
[Table]
e. , DPSGD and ShrinkPad).
Even compared with those having the additional requirement ( i
[Table]
e. , NC and NAD) under their best settings, our defense is still better or on par with them under the default settings.
These results verify the effectiveness of our method again.
[Table] [Table]
F D EFENDING AGAINST A TTACKS WITH D IFFERENT T RIGGER P ATTERNS
In this section, we verify whether DBD is still effective when different trigger patterns are adopted.
Settings.
For simplicity, we adopt the BadNets on the CIFAR-10 dataset as an example for the discussion.
Speciﬁcally, we change the location and size of the backdoor trigger while keeping other settings unchanged to evaluate the BA and ASR before and after our defense.
Results.
As shown in Table 21, although there are some ﬂuctuations, the ASR is smaller than 2% while the BA is greater than 92% in every cases.
In other words, our method is effective in defending against attacks with different trigger patterns.
G D EFENDING AGAINST A TTACKS WITH D YNAMIC T RIGGERS
In this section, we verify whether DBD is still effective when attackers adopt dynamic triggers.
Settings.
We compare DBD with MESA (Qiao et al., 2019) in defending the dynamic attack discussed in (Qiao et al., 2019) on the CIFAR-10 dataset as an example for the discussion.
This dynamic attack uses a distribution of triggers instead of a ﬁxed trigger.
Results.
The BA and ASR of DBD are 92.4% and 0.4%, while those of MESA are 94.8% and 2.4%.
However, we ﬁnd MESA failed in defending against blended attack (for it can not correctly detect the trigger) whereas DBD is still effective.
These results veriﬁed the effectiveness of our defense.
H D ISCUSSIONS H. E FFECTS OF H YPER - PARAMETERS
Settings.
Here we analyze the effect of ﬁltering rate α , which is the only key method-related hyperparameter in our DBD.
We adopt the results on the CIFAR-10 dataset for discussion.
Except for the studied parameter α those
Section 5.2.
[Figure]
[Figure]
Results. The number of labeled samples used in the third stage increase with the increase of ﬁltering rate α , while the probability that the ﬁltered high-credible dataset contains poisoned samples also increases. As shown in Figure 6, DBD can still maintain relatively high benign accuracy even when the ﬁltering rate α is relatively small ( e.g. , 30%).
It is mostly due to the high-quality of learned puriﬁed feature extractor and the semi-supervised ﬁne-tuning process.
DBD can also reach a nearly 0% attack success rate in all cases. However, we also have to notice that the high-credible dataset may contain poisoned samples when α is very large, which in turn creates hidden backdoors again during the ﬁne-tuning process.
Defenders should specify α based on their speciﬁc needs.
H. D EFENDING A TTACKS WITH V ARIOUS P OISONING R ATES
Settings.
We evaluate our method in defending against attacks with different poisoning rate γ on
CIFAR-10 dataset.
Except for γ , other settings are the same as those used in Section 5.2.
I M ORE D ETAILS ABOUT S IM CLR, SCE, AND M IX M ATCH
NT-Xent Loss in SimCLR.
Given a sample mini-batch containing N different samples, SimCLR ﬁrst applies two separate data augmentations toward each sample to obtain 2 N augmented samples.
The loss for a positive pair of sample ( i, j ) can be deﬁned as:
[Equation]
where sim( · , · ) is the cosine similarity, z i is the feature representation of sample i , τ is the temperature parameter, and I { k (cid:54) = i } ∈ { 0 , 1 } indicating whether k (cid:54) = i .
The NT-Xent Loss is computed across all 2 N positive pairs in this mini-batch.
SCE.
The symmetric cross entropy (SCE) can be deﬁned as:
[Equation]
where H ( p, q ) is the cross entropy, H ( q, p ) is the reversed cross entropy, p is the prediction, and q is the one-hot label (of the evaluated sample).
MixMatch Loss.
For a batch X of labeled samples and a batch U of unlabeled samples ( |X| = |U| ),
MixMatch produces a guessed label ¯ q for each unlabled sample u ∈ U and applies MixUp (Zhang et al., 2018) to obtain the augmented X (cid:48) and U (cid:48) .
The loss L X and L U can be deﬁned as:
[Equation]
where p x is the prediction of x , q is its one-hot label, and H ( · , · ) is the cross entropy.
[Equation]
where p u is the prediction of u , ¯ q is its guessed one-hot label, and K is the number of classes.
By combining L X with L U , the MixMatch loss can be deﬁned as:
[Equation]
where λ U is a hyper-parameter.
J C OMPUTATIONAL F ACILITIES
We conduct all experiments on two Ubuntu 18.04 servers having different GPUs.
One has four
NVIDIA GeForce RTX 2080 Ti GPUs with 11GB memory (dubbed ‘RTX 2080Ti’) and the another has three NVIDIA Tesla V100 GPUs with 32GB memory (dubbed ‘V100’).
Computational Facilities for Attacks.
All experiments are conducted with a single RTX 2080 Ti.
Computational Facilities for Defenses.
Since we do not use a memory-efﬁcient implementation of DenseNet-121, we conduct DPSGD experiments on the VGGFace2 dataset with a single V100.
Other experiments of baseline defenses are conducted with a single RTX 2080 Ti.
For our defense, we adopt PyTorch (Paszke et al., 2019) distributed data-parallel and automatic mixed precision training (Micikevicius et al., 2018) with two RTX 2080 Ti for self-supervised learning on the VGGFace dataset.
Other experiments are conducted with a single RTX 2080 Ti.
K C OMPUTATIONAL C OST
In this section, we analyze the computational cost of our method stage by stage, compared to standrad supervised learning.
Stage 1.
Self-supervised learning is known to have a higher computational cost than standard supervised learning (Chen et al., 2020a;
He et al., 2020).
In our experiments, SimCLR requires roughly four times the computational cost of standard supervised learning.
Since we intend to get a puriﬁed instead of well-trained feature extractor, we train the feature extractor ( i.e. , backbone) lesser epochs than the original SimCLR to reduce the training time.
As described in Section B.3, we ﬁndepochs is enough to preserve effectiveness.
Stage 2.
Since we freeze the backbone and only train the remaining fully connected layers, the computational cost is roughly 60% of standard supervised learning.
Stage 3.
Semi-supervised learning is known to have a extra labeling cost compared with standard supervised learning (Gao et al., 2020).
In our experiments, MixMatch requires roughly two times the computation cost of standard supervised learning.
We will explore a more computational efﬁcient training method in our future work. L C OMPARING OUR DBD WITH D ETECTION - BASED B ACKDOOR D EFENSES
In this paper, we do not intend to ﬁlter malicious and benign samples accurately, as we mentioned in Section 4.4.
However, we notice that the second stage of our DBD can serve as a detection-based backdoor defense for it can ﬁlter poisoned samples.
In this section, we compare the ﬁltering ability of our DBD (stage 2) with existing detection-based backdoor defenses.
Settings.
We compare our DBD with two representative detection-based methods, including, Spectral Signatures (SS) (Tran et al., 2018) and Activation Clustering (AC) (Chen et al., 2019), on the CIFAR-10 dataset. These detection-based methods ( e.g. , SS and AC) ﬁlter malicious samples from the training set and train the model on non-malicious samples.
Speciﬁcally, we re-implement SS in
PyTorch based on its ofﬁcial code 9 and adopt the open-source code 10 for AC, following the settings in their original paper.
In particular, since SS ﬁlters 1 . 5 ε malicious samples for each class, where ε is the key hyper-parameter means the upper bound of the number of poisoned training samples, we adopt different ε for a fair comparison.
Results.
As shown in Table 22-23, the ﬁltering performance of DBD is on par with that of SS and
AC.
DBD is even better than those methods when ﬁltering poisoned samples generated by more complicated attacks ( i.e. , WaNet and Label-Consistent).
Besides, we also conduct the standard training on non-malicious samples ﬁltered by SS and AC.
As shown in Table 24, the hidden backdoor will still be created in many cases, even though the detection-based defenses are sometimes accurate.
[Table]
[Table]
[Table]
[Table]
This is mainly because these methods may not able to remove enough poisoned samples while preserving enough benign samples simultaneously, i.e. , there is a trade-off between BA and ASR.
M DBD WITH D IFFERENT S ELF - SUPERVISED M ETHODS
In this paper, we believe that the desired feature extractor is mapping visually similar inputs to similar positions in the feature space, such that poisoned samples will be separated into their source classes.
This goal is compatible with that of self-supervised learning.
We believe that any selfsupervised learning can be adopted in our method.
To further verify this point, we replace the adopted SimCLR with other self-supervised methods in our DBD and examine their performance.
Settings.
We replace the SimCLR with two other self-supervised methods, including MoCo-V (Chen et al., 2020b) and BYOL (Grill et al., 2020), in our DBD.
Except for the adopted selfsupervised method, other settings are the same as those used in Section 5.2.
Results.
As shown in Table 25, all DBD variants have similar performances.
In other words, our
DBD is not sensitive to the selection of self-supervised methods.
[Table]
N DBD WITH D IFFERENT L ABEL - NOISE L EARNING M ETHODS
In the main manuscript, we adopt SCE as the label-noise learning method in our second stage.
In this section, we explore whether our DBD is still effective if other label-noise methods are adopted.
Settings.
We replace SCE in our DBD with two other label-noise learning methods, including generalized cross entropy (GCE) (Zhang & Sabuncu, 2018) and active passive loss (APL) (Ma et al., 2020).
Speciﬁcally, we adopt the combination of NCE+RCE in APL and use the default hyperparameters suggested in their original paper.
Except for the adopted label-noise learning method, other settings are the same as those used in Section 5.2.
Results.
As shown in Table 26, all DBD variants are effective in reducing backdoor threats ( i.e. , low ASR) while maintaining high benign accuracy.
In other words, our DBD is not sensitive to the selection of label-noise learning methods.
O A NALYZING W HY OUR DBD IS E FFECTIVE IN D EFENDING AGAINST L ABEL -C ONSISTENT A TTACK
In general, the good defense performance of our DBD method against the label-consistent attack (which is one of the clean-label attacks) can be explained from the following aspects:
Firstly, as shown in Figure 1, there is a common observation across different attacks (including both poisoned- and clean-label attacks) that poisoned samples tend to gather together in the feature space learned by the standard supervised learning.
The most intuitive idea of our DBD is to prevent such a gathering in the learned feature space, which is implemented by self-supervised learning.
As shown in Figure 1(d), the poisoned samples of label-consistent attack are also separated into different areas in the feature space learned by self-supervised learning.
This example gives an intuitive explanation about why our DBD can successfully defend against the label-consistent attack.
Furthermore, it is interesting to explore why the poisoned samples in the label-consistent attack are separated under self-supervised learning since all poisoned samples are from the same target class, rather than from different source classes in poisoned-label attacks.
For each poisoned sample in this attack, there are two types of features: the trigger and the benign feature with (untargeted) adversarial perturbations.
From the perspective of DNNs, benign samples with (untargeted) adversarial perturbations are similar to samples from different source classes, though these samples look similar from the human’s perspective.
Thus, it is not surprising that poisoned samples in clean-label attacks can also be separated under self-supervised learning, just like those in poisoned-label attacks.
The codes for reproducing main experiments of our DBD against BadNets and blended attack are anonymously open-sourced 11 .
We will release all related codes after the acceptance of this paper.