Effective Backdoor Defense by Exploiting Sensitivity of Poisoned Samples
Authors: Weixin Chen
Baoyuan Wu
Haoqian Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: backdoor defense
backdoor learning
trustworthy AI
AI security

Abstract: Poisoning-based backdoor attacks are serious threat for training deep models on data from untrustworthy sources. Given a backdoored model, we observe that the feature representations of poisoned samples with trigger are more sensitive to transformations than those of clean samples. It inspires us to design a simple sensitivity metric, called feature consistency towards transformations (FCT), to distinguish poisoned samples from clean samples in the untrustworthy training set. Moreover, we propose two effective backdoor defense methods. Built upon a sample-distinguishment module utilizing the FCT metric, the first method trains a secure model from scratch using a two-stage secure training module. And the second method removes backdoor from a backdoored model with a backdoor removal module which alternatively unlearns the distinguished poisoned samples and relearns the distinguished clean samples. Extensive results on three benchmark datasets demonstrate the superior defense performance against eight types of backdoor attacks, to state-of-the-art backdoor defenses. Codes are available at: https://github.com/SCLBD/Effective_backdoor_defense.

Effective Backdoor Defense by Exploiting Sensitivity of Poisoned Samples
Weixin Chen', Baoyuan Wu²*, Haoqian Wang
Tsinghua Shenzhen International Graduate School, Tsinghua University Schooo of Data Science, Shenzhen Research Institute of Big Data,
The Chinese University of Hong Kong. Shenzhen chenwx20@mails. tsinghua edu. wubaoyuan@cuhk, edu. cn, argghaaqianttsinggn edu. cn
Abstract
Poisoning- backdoor attacks are serious threat for training deep models on data from untrustworthy sources. Given backdoored model, observe that the feature representations of poisoned samples with trigger are more sensitive transformations than those of clean samples. It inspires to design simple sensitivity metric, called feature consistency towards transformations (FCT), to distinguish poisoned samples from clean samples in the untrustworthy training set. Moreover, we propose two effective backdoor defense methods. Built upon a sample-disingguishmen module utilizing the FCT metric, the first method trains a secure model from scratch using a two-stage secure training module. And the second method removes backdoor from a backdoored model with backdoor removal module which alternatively unlearns the distinguished poisoned samples and relearns the distinguished clean samples. Extensive results on three benchmark datasets demonstrate the superior defense performance against eight types of backdoor attacks, state-of-the-an art backdoor defenses. Codes are available at: http:///gthab..oomSCCBB/fffffff bbckdoorrrooroaacooooo dereedsrrdeeeseee
Introduction
Training deep neural networks (DNNs) often requires a large amount of training data, which is sometimes obtained from third- fhird-party untrustworthy source. However, the untrustworthy data may bring serious security threats. One of the typical threats is the poisoning-based backdoor attack [1], which could inject undesired backdoor-the correlation between trigger(s) and target class(es) into the model through maliciously poisoning few training samples. Specifically, as shown in the top left of Fig. each poisoned sample is attached with a trigger (see a small grid patch) at the bottom right corner, and relabelled
target class. Consequently, the trained backdoored model will predict clean samples very well, but is likely to predict any sample with the trigger to be the target class.
It has been observed in [2] that poisoned samples with triggers are likely to gather together in the feature space of a backdoored model, as shown in the top right of Fig. 1. Note that these poisoned samples contain diverse objects (may be from different source classes), but the informa- tion from these objects seems to be ignored by the backdoor model. In other words, the feature representations of poisoned samples are dominated by the triggers, rather than the objects. We conjecture that such domination is mainly due to the overfitting to the triggers by the backdoor model, since triggers across different poisoned samples are much less diverse than objects. To verify this conjecture, we propose to slightly perturb both poisoned and clean samples, such as
"Corresponding authors.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

rotation transformation. As shown in the bottom right of Fig. 1, there is no longer gathering of poisoned samples in the feature space, and they are located close to samples of their source classes, Le., the dominance of triggers over other objects disappears, which verifies the triggers overfitting.
Besides, although the feature representations of clean samples are also affected by transforma- tions, their changes are much smaller than those of poisoned samples. In other words, poi- soned samples are more sen- sitive to transformations than clean samples. It inspires that poisoned samples could be dis- tinguished from clean samples according to the sensitivity to transformations, which is mea- sured by a simple sensitivity metric, called feature consis- tency towards transformations (FCT). In our experiments, the precision of the distinguished clean and poisoned samples is nearly 100% most cases, re- spectively.
Input
Feature Space
Clean Polsoned
(Cass 1) (label 0j
Before
After
Polsoned
Cleanjolase
Cleanjclass2)
4
Cleanjclasad)
Before Transtormations
Before Transtormations
Cteen Polsoned
(Class 1) (lebel Dj
After Transformations
After Transtormations
Union
Figure 1: Poisoned and clean samples from CIFAR-10 [3]. and the t-SNE [4] visualization of their feature representations with backdoored model. As shown in the Union figure, the changes poisoned samples (from to +) are much larger than those of clean samples (from to + in other colors). Note that we present only three classes for clear illustration.
In this work, we aim to obtain secure model (i.e., high-performme teth-perffrmanc and without backdoor) based on an untrustworthy training set. To this end, we consider two defense paradigms: one is training a secure model from scratch, while the other is firstly training backdoored model using standard supervised learning, and then removing backdoor from the backdoored model. Under paradigm 1, we propose an innovative secure training method, called Distinguishmmnnt and Secure Training (D-ST), which consists of two consecutive modules. The first training set into clean, poisoned and uncertain samples, according to the FCT metric. The second two- stage secure training (ST) module firstly learns the feature extractor via cepi-super contrastive learning, and then learns the classifier via minimizing a mixed gross-entropy loss. Under paradigm 2, we propose an innovative backdoor removal method, called Distinguishment and Backdoor Removal (D-BR), which consists of the SD module and a backdoor removal (BR) module. BR module alternatively unlearns the distinguished poisoned samples and learns the distinguished clean samples. Extensive experiments are conducted to verify the superior defense performance of the above two proposed methods, as well as effecctiveness of each individual module.
(SD) module splits the whole
The main contributions of this work are three-folds. (1) We demonstrate the sensitivity of poisoned samples to transformations, which is mainly due to the overfitting to trigger, and propose a simple sensitivity metric distinguish poisoned samples from clean samples. (2) We propose two effective backdoor defense methods for training a secure model from scratch and removing backdoor from the backdoored model, respectively. (3) Extensive experiments on benchmark datasets show the superior performance of the proposed defense methods against 8 widely used backdoor attacks, to 6 state-of-the-art defense methods.
2
Related work
Backdoor attack. In coisoning-based backdoor attacks, the attacker attaches few training samples with trigger(s), and relabel them as target classíes). Existing attacks can be categorized according to variety of criteria as follows. (1) Size of trigger. Patch-based attacks [1, 5, 6] craft patch- triggers while in haand attacks [7, 8), triggers capture the whole image. (2) Visibility of trigger: Visible attacks [1, 6) design visible but not suspicious triggers while invisible attacks [8, 9, 10] propose invisible and still effective ones. (3) Variability of trigger: Triggers are invariant in sumple-agnoste
attacks [1, 7, 11] while vary with samples in spmple-specific atlacks [9, 12]. (4) Label-consistency: If
poisoned samples are chosen from samples with target class, then we call these attacks as clean-label attacks [11, 13, 12, 14]. Otherwise, we name them as dirty-label attacks [1, 6, 7, 8]. (5) Number of

---

Paradigm Secure Training from Scratch
Me imiingg Moxdd Crose-Entropy oee
Commestiv
Module
Secture Festuue Exiractor
Secure Classiivr
Stage
Stage
Clean Samples
Standard Supervieed Traiinng
SD Module
FOT Metric
Backdoored Moowl
D
D
D
Standard Supertoed
Traning
BR Module
earning
Roleaming
Prasoreed Sampiee
Bankdoored Mode
Poisoned Dataset
Paradigm2: Backdoor Removal
Secure Model
Figure 2: Framework of two proposed backdoor defense methods for secure training from scratch (paradigml) and backdoor removal (paradigm2), respectively.
target classes: All2one attacks [1, 6, 7, 8] designate one class as the target class while in all2all attacks [1]. poisoned samples are relabelled as the next class. There are also other attacks [15, 16, 17, 18, 19] that require the attacker to control the training process, which are out of scope of this paper. The attack performance of the methods above can be referred to the Backdoorbench [20].
Backdoor defense. general. there are two types of defense paradigms against poisoning- based backdoor attacks secure training and backdoor removal. Mainstream defense methods belong to the latter one, which leverage the model properties [21, 22, 23] or the feature space characteristics [24, 5, 25, 26, 27, 28, 29] of backdoored model, to remove the hidden backdoor. For instance, FP (21) observes that some neurons are activated by poisoned samples while others are by clean samples. AC [24) notices the difference in size between the cluster of clean target samples and that of poisoned samples. So far, there still few works for the former defense. Secure training attempts to train secure model from scratch, preventing backdoor inserted during training. The core point lies in how to distinguish poisoned samples from clean samples. The first method DBD [2] uses the loss values, the symmetric cross-entropy loss particularly, to distinguish samples during training while proposed secure training method, i.e., D-ST, leverages the property that the feature representations of poisoned samples in backdoored model are more sensitive transformations than those of clean samples, to distinguish samples in advance of the training.
3 Proposed method
3.1 Problem formulation
Threat model. In this paper, we consider the threat model of poisoning-baser backdoor attacks, where the attacker can manipulate a few samples in the original clean training set Dtrain D. U (a);, " with D. {(x;, y/ " indicating the unmanipulated subset. For the remaining mp samples, each sample x; E X' fused with trigger d to form a poisoned sample F; = æ, + 8 with 4 being the fusion operator. Mcanwhile, its label yı y is also changed to target class t. Then, poisoned training set constructed, denoted as D Dtrain D. U Dp, with Dp = {(0,, t- When a > y based on Dtrain using the standard
user downloads Dtrain and trains a DNN classifier go supervised learning algorithm, it may learn a undesired backdoor, i.e., stable mapping from the trigger ò to the target class t. Consequently, for any new sample with the trigger d, it is likely to be predicted as the target class l. Note that the user does not know which sample is poisoned or clean.
Defense goal. Given the poisoned training set Derain the defender aims to obtain seigh-eeformance model ge without backdoor, i.e., secure model. In this work, we consider two different paradigms:
Paradigm 1: a secure model is directly trained from scratch, as described in Section 3.3.
Paradigm 2: a backdoored model firstly trained using the standard supervised learning, then the backdoor is removed from the backdoored model, as described in Section 3.4.
3

---

3.2 Sensitivity of poisoned samples
Sensitivity metric. As illustrated in Section 1 and Fig. 1, we have found that the poisoned samples are much more sensitive to transformations than the clean samples in a backdoored model. To accurately measure such a difference, we propose simple metric. feature consistency towards transformations (FCT). Specificlly, given backdoored model go trained on Dirain with fo, indicating its feature extractor, and set transformations (e.g., rotation, scaling, will be specified in experiments), for any sample æ (poisoned or clean), the FCT metric is formulated as follows:
A Arans (z; T, fe.) = llfe. (a:) fa. (T(2))]
It measures the change of the feature representation due to the transformations T. If Atrans T, J) is large, then means that x sensitive to т, otherwise stable. For clarity, we use A Atrans (x) hereafter.
Sample-distinguishmens module. Utilizing FCT. we develop a sample-disinguushhmnt (SD) module. Specifically, we firstly train back- doored model ge based on Dtraén using the stan- dard supervised learning algorithm with a few epochs (explained in Appendix A.1). Then, we calculate Atrone (2;), Vx,; E Dirain- and plot
clean samples
3449
Clean samples polsoned samples
Nam Sumples
polsoned samples
mm of Stmppis
the histogram. As shown Fig. 3, where two
representative backdoor attacks are evaluated,
there is remarkable difference on the distribu- tion between the poisoned and the clean samples in both histograms. It demonstrates that Atrass a good metric distinguish the poisoned sam- ples from the clean samples in Dieain. Based on the sensitivity histogram, we set two proportion values Cce Op E [0, 1]. The samples with the bottom-a, trans values are separated a subset of clean samples De, while those with the top-Op Atrons values are separated into subset of poisoned samples D,,. while the remaining samples are partitioned as uncertain subset denoted as D,, We have D Derain D. U Dpp U Da. More details are in Algorithm in Appendix A.1
(a) BadNets
(b) Blend
Figure 3: Distribution of clean and poisoned samples
with respect the FCT metric on CIFAR-10.
3.3 Method for paradigm 1: secure training from scratch
Here, we consider the backdoor defense under paradigm 1. We propose an innovative secure training method, called Distinguishment and Secure Training (D-ST) method. As illustrated in Fig. 2, D-ST consists of the SD module (see above) and two-stage secure training (ST) module, which is described as follows. Details of the D-ST method are summarized in Algorithm in Appendix A.2..
learning feature extractor via semi-supervised contrastive learning (SS-CTL), Our method is inspired by recent backdoor defense method called DBD [2], which proposed to learn good feature extractor fe. based on Derain using self-supervised learning algorithm, iee. contrastive learning (CTL). Consequently, the feature representations of samples with similar appearances will be similar, and poisoned samples with triggers cannot gather together to form the backdoor. Note that all labels have been abandoned in DBD before the extractor learning since there no way to identify poisoned samples in advance of the learning, leading to the waste of the valuable information contained in clean samples. Fortunately, the proposed SD module could identify some clean samples. Thus, inspired by the supervised contrastive learning (S-CTL) [30], which has shown to learn a feature extractor with better performance than CTL, we propose a novel learning called supervede contrastive learning (SS-CTL), to learn fe. by minimizing the following loss function:
Stage 1:
Lss-CTI Ce-CTL( (Pe; Derain)
Σ lOTI (So, (z()), (2)
(2)
Σ ls-CTL f. fo. (a. (2 (e,
(o. fo. (Z :;
where the contrastive loss lCTI, encourages the two augmented versions (1) (2) (e.g. cropping, details are introduced in Appendix C) of sample æ; to be close in the feature space, while the

---

supervised contrastive loss És-CTL additionally encourages the feature representations of two clean augmented samples from the same class be close. In this work, we instantiate lCTI as the contrastive loss defined in [31] and CC-CTI as the SupCon loss defined in [30].
Stage 2: learning classifier via minimizing the mixed cross-entropy loss. Given the feature extractor fe. learned in stage 1, we then learn the classifier he. by minimizing the following mixed cross-entropy (MCE) loss:
Ap
LMC = Diel
Σ
logihe (fo. + (2..))
Σ
log[ho, (f),)))))r (3)
1Dpl
(x..))De
where the first term the standard cross-entropy loss defined based on the distinguished clean samples Do. while the second term is the negative cross-entropy loss defined based on the distinguished poisoned samples Dp, which is used to eliminate the effect of poisoned samples. Ap E R is a trade-off parameter between two losses.
3.4 Method for paradigm 2: backdoor removal
Here we consider the backdoor defense under paradigm We propose an innovative backdoor removal method, called Distinguishment and Backdoor Removal (D-BR) method. As illustrated in Fig. 2. D-BR consists of the SD module (see Section 3.2) and backdoor removal (BR) module. The BR module aims to remove the backdoor from the backdoored model, i.e., the backdoor is no longer activated by the trigger, while keeping the high performance on clean samples. To this end, the BR module implements iterative learning algorithm, which consists of two alternating steps, i.e., unlearning and relearning. The D-BR method summarized in Algorithm 4 in Appendix A.3.
Unlearning, This step aims to eliminate the effect of the trigger, through unlearning [32] the poisoned samples in Dr distinguished by the SD module, as follows:
1
Luntearn (0; (p)
Σ Dpl (æN))
loglgo(>)]y-
(4)
Relearning. After conducting the above unlearning step for one epoch, although the effect of poisoned samples is somewhat eliminated, in experiments we find that the performance on clean samples is also degraded to some extent. Thus, we want to relearn the mapping from the clean objects to the ground-truth classes based on clean samples in D. distinguished by the SD module, follows:
Lrelearn (0;
Σ
- [ooggo(())ss
(5)
Del
(m. D.
Note that both unlearning and relearning are run for one epoch in each round.
4 Experiments
4.1 Experimental settings
Attack configurations.
According to the taxonomy described in Section 2, we consider 8 typical
poisoning-basee backdoor attacks by choosing at least one method from each category, including: BadNets [1] using two attack types (BadNets-all2onee, BadNets-alllaal Trojan backdoor attack [6] (Trojan), Blend backdoor attack using two different patterns Belend-Signal. Blend-Kitty) 2 [7], Clean- label backdoor (CL) [13], Sinusoidal signal backdoor attack (SIG) [11], Sample-specific backdoor attack (SSBA) [9]. We evaluate all attacks on benchmark datasets, CIFAR-10 [3], CIFAR- 100 [3] and ImageNet subset [33, 9], with ResNet- 18 [34] as the base model. Poisoning rate is set to 10% in all attacks. Due to the space limit, more implementations details about attacks can be found in Appendix C.3.
Defense configurations. We first compare the proposed D-ST method with DBD [2]. Since studies with this secure-training paradigm are limited, we additionally add baselines for comparison which
If not specified, 'BadNets/Blend" generally stands for the
signal attack.
5

---

are detailed in Section 4.2. We then compare the proposed D-BR method with 5 state-of -tate-of-the-arr methods with the same backdoor-remove paradigm: the standard fine-tuning FT. ANP [22], NAD [35], MCR [36] and ABL [37]. For methods requiring extra clean data, 1% of the clean training samples are provided. Other configurations are set as clarified in the original papers. In summary, we consider 6 state-of-the-arr art defense methods and 2 additional baselines. More implementations details can be found in Appendix C.4. For our proposed methods, we use a = 20%, ap = 5% and =aftee in all experiments. Other details can be seen in Appendix C.5.
Evaluation metries. We evaluate the defense performance adopting two commonly used metries: accuracy on clean samples (ACC) and attack success rate (ASR), i.e., accuracy of predicting poisoned samples as the target label.
4.2 Experimental results
Effectiveness of D-ST method. We first consider paradigm 1-secure training from scratch. Per- formance of different defense methods against various attacks on CIFAR-10 and CIFAR-100 is demonstrated in Table 1. An ideal defense method is supposed to increase ACC while keep ASR as low as possible. Thus, larger ACC-ASR indicates better method. We mark the best result in boldface. Note that we only report results on successful attacks where ASR is higher than 85%.
Table I: Comparisons of the D-ST method with 3 secure-training defense methods (%).
Defense +
Baselinel ACC
Baseline2 ACC
DBD
D-ST
Datases J
Altack
ASR
ASR
ACC
ASR
ACC
ASR
BN -N-aalionne
83.54
2.60
91.32
99.91
92.75
100.00
92.77
0.03
BN-aliizal
2.72
91.59
57.39
92.95
75.21
2,05
Trojan
5.24
93.63
08
92.81
Strip
99.93
94.19
100000
94.21
0
CIFAR-10
Blend-Kiny
99.09
94.31
100.00
93.32
100.00
0
SIG
85.14
99.02
94.37
99.93
94.37
99.71
90.07
0.00
CL
85.79
10.76
94.58
98.5 87
94.32
99.87
90.46
6.40
| Avg
84.65
45.75
93:43
93.73
993.53
96.40
91.66
1.21
BN-allZoone
54.48
10.41
62.62
100000
09
0.12
Trojan
56.17
12.76
71:01
100.00
72.18
99.
0
CIEAR- 100
Blend Kinly
Blend-Strip
57.21
58.01
99.99
99.91
73.36
72.47
99.9
99.99
71.29
72.43
100.00
99.00
67.06
67.63
0.00
0.000
| Avg
56.47
55.77
71.12
100.00
71.24
99.99
67.79
0.05
DBD fails in most attacks on CIFAR-10 10 and CIFAR-100, probably due to the failure of the symmetric cross-entropy to distinguish samples. By comparison, the good performance reached by D-ST illustrates the accurate distinguishment from the FCT-based SD module. We additionally introduce two feasible baselines without requiring special knowledge. Baseline I first uses SimCLR [31] to train the feature extractor and then trains the classifier on the poisoned dataset with standard supervised learning. By comparison, Baseline2 leverages S-CTL [30] to train the feature extractor. We focus on discussing the effect of different extractor-training algorithms defense performance. More discussions are in the later experiments. Bascline reveals that training extractor without labels may result in low ASR (<5% / <20% in some cases on CIFAR-10/ CIF)R-100), but will sacrifice ACC definitely (84.6%% 56.47% on average). While Baseline 2 demonstrates that training extractor with all labels guarantees high ACC (93.43% / 71. 11.12% average), but also brings high ASR 993.73% / 100% on average). By contrast, results of D-ST illustrate the effectiveness of ST module in training the feature extractor in secure way since ACC is high (91.66% 67.79% average) and ASR (1.21% / 0.05% on average) is extremely low.
Effectiveness of D-BR method. Then, we consider paradigm backkooo removal. Defense performance of different defense methods against various attacks on CIFAR- and CIFAR-100 is demonstrated in Table 2. Results on ImageNet is shown in Table 5 in Appendix D.
Results on CIFAR-10. We discover that except for FT and MCR, other selected methods generally reduce ACC markedly. Additionally, they have two common disadvantages. (1) They can not take effect on all attacks. For example, ANP can defend against Trojan and Blend attacks (ASR 1 1%) while fails in clean-label attacks, i.e. SIG and CL (ASR > 10%). (2) There exists at least one attack that can disable the methods (ASR > 50%). By contrast, the proposed D-BR method overcomes these drawbacks. It not only maintains ACC as large as that of backdoored model, but also reduces ASR to less than 1 on all attacks, verifying the effectiveness of the BR module and the high precision of the distinguishment conducted by the SD module.

---

Table 2: Comparisons of the D-BR method with 5 backdoor-remmoal defense methods on CIFAR-I0 R-00 and CIFAR- (%). "Backdoored' refers to the backdoored model. * ddeottes methods which require a few (1%) clean training samples.
Detense- +
Hackdoored
FTY
ANP
NADY
MCK*
ABL
D-BR
Antack J
ACC
ASR
ACC
ASR
ACC
ASR
ACC
ASR
ACC
ASR
ACC
ASR
ACC
ASR
HN alllono
91.64
100.00
$9.39
06.79
90.03
10.54
$4.46
2.15
94.21
8.29
89.55
0.39
92.83
0.40
BN-aliaall
92.70
83.01
90.31
4.95
86.04
1.47
54.97
1.71
92.17
2.96
79.91
78.16
92.61
0156
Trujan
91.91
100.00
89.86
100.00
90.89
83.29
5.04
93.90
2.58
90.18
0.23
92.21
0.76
Blend-Strip
92,07
88.53
83.09
13.50
0.22
92.40
0.06
CIFAR-10
Kilty
28.96
7.49
2.27
SIG
$2.43
76.33
64.72
79.94
92.73
CL
93.20
93.34
90.03
77.44
72.57
10.90
$4.46
2.65
92.13
72.01
84.39
0.31
92.08
0.00
Aug
92.46
97.20
90:05
77.41
84.91
14.30
69
16.95
92.92
30.05
80.21
25.75
92.42
0.51
DN-allZone
28
0.00
72.58
Trojaa
75.75
64.52
72.11
92.21
0.00
74.52
CIFAR-100
Blend Kitty
Pheid-Strip
75.54
75.18
99.97
72.93
73.36
99.96
67.3 38
09.03
0.00
0.00
71
71.73
99.93
95.76
73.93
75.37
20.60
0.07
47.05
0.00
0.00
72.00
74.35
0.00
0.01
Avg
74.43
90.77
72.83
91.4 46
66.59
0.01
71.01
74.83
73,80
5.27
57.69
0.01
73.36
0.07
Results on CIFAR-100. Although FT and NAD have a relatively high ACC 68%), they fail to reduce ASR (91.46% and 74.83% on average). While ANP and ABL can decrease ASR to less than 0.1%, they sacrifice too much ACC (66.59% and 57. 57.69% on average). Among the selected methods, MCR performs the best (ACC = 73.80% on average. ASR < 0.5% in three cases), but it still fails to defend against the Blend-Kitty attack (ASR = 20.60%). Note that MCR requires extra clean data. In contrast, D-BR keeps ACC higher than 72%, while reduces ASR to almost 0%% without any extra clean data.
4.3 Ablation studies
Effectiveness of the SD module. Here, we aim to study the effectiveness of the SD module. Specifically, we will show how our proposed FCT metric, performs better than other metrics, under the backdoor-removal paradigm for illustration. To this end, we select three existing metrics for comparison. Spectral signatures [5] specifies the metric as the correlation with the top singular vector of the covariance matrix of feature representations. DBD [2] assigns symmetric cross-entropy loss as the metrie. The metric used ABL [37] is loss value applied with local gradient ascent. The metric values of clean samples are smaller than those of poisoned samples according to the former two metrics, while larger for the third metric. For fair comparison, we uniformly set Oe = 20%, Op = 5%. We first apply the pectricreeplaee SD module the poisoned training set, and then conduct the BR module based on the distinguished samples. Results are shown in Fig.
$2:27
20.33
02.13
28
22
600
040
(a) BadNets
(b) Blend
(c) SIG
(d) CL
Figure 4: Test ACC and Test ASR of four metric-replaced D-BR methods on the poisoned CIFAR-1O 10.
The height of the blue bar above the orange bar suggests how well the metric could distinguish. As shown in Fig. 4 (a,b,)), orange bars are all higher than blue bars for Spectral signatures and DBD, indicating metrics of which fail to distinguish in BadNets, Blend and CL attacks. In contrast, the metric of ABL is reliable since ABL performs well in most cases except for Blend attack where ASR is 24.72%. By comparison, our proposed FCT metric could distinguish samples stably well, resulting in extremely low ASR (< 0.5%) on all attacks. We attribute the success to that FCT exploits the sensitivity of poisoned samples, which is mainly due to the overfitting to trigger by the backdoored model that exists in all backdoor attacks we have evaluated this paper.
Effectiveness of the BR module. Here, we focus on studying the effectiveness of the BR module. Specifically, we aim to show how the iterative learning algorithm consisting of unlearning and relearning performs better than the pure unlearning adopted by [37] or pure relearning. To this
7

---

end, we first conduct the SD module, and then apply different learning algorithms. For the three algorithms, we run 20 epochs on CIFAR- 10 and record the variations of Test ACC and Test ASR which are illustrated as Fig. 5.
Ean edarssgeneenngg
Leaurrn robering
(a) BadNets
(b) Blend
(c) SIG
(d) CL
Figure 5: Test ACC(top) and Test ASR(bottom) of three learning algorithms on poisoned CIFAR- 10.
Although pure unlearning (red lines) effectively decreases ASR, it could hardy maintain ACC, showing downward trend. The results indicate a strict requirement for choosing the number of unlearning epochs. On the contrary, pure relearning (green lines) can keep ACC stably high, but it takes tiny effect in reducing ASR. By contrast, unlearning+ selearning+rlearnnnnn relearning (blue lines) combines their advantages and successfully diminishes ASR while maintains ACC. ACC and ASR steadily converge to high and low values, respectively, validating the effectiveness and the stability of the BR module.
Effectiveness of the ST module. Here, we aim to study the effectiveness of the ST Module. Backdoor can be injected during training the feature extractor fe. and the classifier he. The final defense performance of depends on how well fe. and he. inhibit backdoor.
Firstly, we want show how SS-CTL performs better than CTL S-CTL in training secure feature extractor fa.. To this end, we train fo. with different learning algorithms and then uniformly leverage LMGE train ho.- Training fo. with CTT,, shown in the first row in Table 3, guarantees low ASR, but the low ACC turns into a tradeoff. Note that the low ASR is the joint effort of CTL and LMCE And the usage of CTL does not indicate low ASR definitely, but it indeed reduces the possibility of backdoor injection in fe. The second row illustrates that S-CTL could bring high ACC and the potentially high ASR, as seen in the SIG and CL attacks. Since all labels (including poisoned) are used in this scenario, ACC is reasonably high. Besides, backdoor is already injected into fe.- But due to the inhibition effect of LMCE, the ASR of ge may not be high. For example, in dirty-label attacks, ie. BadNets. Trojan and Blend attacks, ASR is almost 0%. While in clean-label attacks, i.e. SIG and CL attacks, LMCE can not withstand the backdoor injected in fe.. so the ASR is almost 100%. Hence, in order to establish reliable defense module, fo. should be trained in more secure way. In comparison, the third row demonstrates the superior defense performance of SS-CTL, illustrating that the ST module securely bridges genuinely clean intra-class samples together which are distinguished by the SD module.
Table 3: Performance with fe_ trained with three learning algorithms on the poisoned CIFAR-10.
Attack
BN-allone
BN-olthill
Trijen
Bienk-Signal ACC
Blend-Kitt
fa.
ASR
ASR
ACC
ASR
ASR
ACC
ASR
ASR
ASR
1.32
S-CT1.
93.80
04.18
S5.CTT
89.22
93.72
0.00
Secondly, we explore how LMCE affects the defense performance of he.- For clarity, we denote L1 = Do_ L log/ho, (Jo, (æ))]y and L2 = |b,ा E(EN)ED, log[ho, (x))) We have LMCE = Li + X2 Generally, knowing clean samples, the defender will train ha, with L, So here, we aim to show how our proposed L2 and the trade-off parameter Ap affect ho.- To this end, we first fix fo. learned by SS-CTL. and then apply LMCE with A, = 0, 0.001, 0.01, on he.- Results are shown in Fig. 6. In the previous experiments, adopt A 0.001.
8

---

The comparison between X 0 and A * in the right figure illustrates that L2 can effectively reduce ASR. When comparing different Xp & in the left figure, we discover that as Xp increases, there is trend decrease in ACC. We infer that since L2 drops faster than L1, namely unlearning faster than relearning, adding weights to L2 makes he. focus on unlearn- ing instead of relearning, leading the low ACC. Therefore, we conclude that L2 helps to inhibit backdoor in ho., but its weight should not be too large. Ap = 0.001 is considered to be an appropriate choice.
Dend
0.01
0.231
9m
Figure 6: Test ACC (left) and Test ASR (right) under various Ap the poisoned CIFAR-10.
In summary, we have empirically validated the effectiveness of each individual module, and shown the flexibility of our method to combine with other existing modules.
Appendix. Due to the space limit, more results and analysis will be presented in Appendix. The overall structure of the Appendix is listed as follows.
Appendix A: more algorithmic details and analysis on the proposed method.
Appendix B: more details on cemi-supervised contrastive learning (SS-CTL).
Appendix C: more implementation details.
Appendix D: results on ImageNet.
Appendix performance with different data transformations т.
Appendix F: performance with different proportion values Oee Opp
Appendix G: performance with different poisoning rates.
Appendix H: performance with different model architectures and feature dimensionalities.
Appendix I: complexities of two proposed methods.
5
Conclusions
In this paper, we reveal the sensitivity of poisoned samples to transformations and propose a sensitivity metric, called FCT. Besides, we propose three modules- the SD module to distinguish between clean and poisoned samples, the ST module to train a secure model from scratch and the BR module to remove backdoor- which constitute two defense methods, D-ST and D-BR, to defend under two different defense paradigms. Extensive experiments have demonstrated the effectiveness of each individual module and also the proposed defense methods.
Broader impact
Poisoning based backdoor attacks are severe threats to the learning paradigm of learning DNN model based on the training set from some untrustworthy sources. This work reveals the sensitivity of poisoned samples in the backdoored model, which will help people to better understand the inner mechanism of backdoor attacks. The proposed two effective defense methods can not only significantly mitigate the threat of existing poisoning based backdoor attacks, but also serve as the new baseline for developing more advanced attack methods in future.
Acknowledgments and disclosure of funding
Baoyuan Wu is supported by the NSFC Fund under grant No.62076211 Shenzhen Science and Technology Program under grant No. 169220 103057050, No. GXWG22222201111 105722002- 20200901 110001175500 001, and NSFC Fund under grant 0014 Shenzhen Science and Technology Program under grant No. (..CG/IZDD22000770000000000 and No.JSSGG200208802133315000000
1141555 Haoqian Wang is supported by the

---

References
Tianyu Gu, Kang Liu, Brendan Dolan- and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 1147230-472444 2019.
[2] Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. In ICLR, 2022.
[3] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009.
[4] Laurens Van der Maaten and Geoffrey Hinton, Visualizing data using t-sne, Journal of machine learning research, 2008.
Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. In NIPS, 2018.
[6] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. In NDSS, 2017.
[7] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXi::712.00526. 2017.
[8] Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. In ECCV, 2020.
Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. ICCV, 2021.
[10] Fugene Bagdasaryan and Vitaly Shmatikov. Blind backdoors deep learning models. In USENIX, 2021.
[11] Mauro Bami, Kassem Kallas, and Benedetta Tondi. A new backdoor attack in cnns by training set corruption without label poisoning. In ICIP, 2019.
[12] Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. Hidden trigger backdoor attacks. In AAAI, 2020.
[13] Alexander Turner, Dimitris Tsipras, and Aleksander Madry, Label-consistent backdoor attacks, arXiv preprint arXiv: 2019.
[14] Shihao Zhao, Xingjun Ma. Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. Clean-label backdoor attacks on video recognition models. In CVPR, 2020.
[15] Tuan Anh Nguyen and Anh Tuan Tran. Wanet - imperceptible warping-based backdoor attack. In ICLR, 2021.
[16] Tuan Anh Nguyen and Anh Tran. Input-aware dynamic backdoor attack. In NIPS, 2020.
[17] Junyu Lin, Lei Xu, Yingqi Liu, and Xiangyu Zhang. Composite backdoor attack for deep neural network by mixing existing benign features. In CCS, 2020.
[18] Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. Dynamic backdoor attacks against machine learning models. arXiv preprint arXiv:2003.0 03675, 2020.
[19] Ili Shumailov, Zakhar Shumaylov, Dmitry Kazhdan, Yiren Zhao, Nicolas Papernot, Murat A Erdogdu, and Ross J Anderson. Manipulating sgd with data ordering attacks. In NIPS, 2021.
[20] Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, Chao Shen, and Hongyuan Zha. Backdoorbench: A comprchensive benehmark of backdoor learning, arXiv preprint arXiv:2206.1224 12654, 2022.
[21] Kang Liu, Brendan Dolan-Gavin Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In RAID, 2018.
[22] Dongxian Wu and Yisen Wang. Adversarial neuron pruning purifies backdoored deep models. In NIPS, 2021.
[23] Kota Yoshida and Takeshi Fujino. Disabling backdoor and identifying poison data by using knowledge distillation in backdoor attacks deep neural networks. In ACM Workshop. 2020.
10

---

[24] Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig. Benjamin Edwards. Taesung Lee. Ian M. Molloy, and Biplav Srivastava. Detecting backdoor attacks deep neural networks by activation clustering. In AAAI, 2019.
[25] Ezekiel Soremekun, Sakshi Udeshi, and Sudipta Chattopadhyay. Exposing backdoors in robust machine learning models. arXiv preprini arXiv:2003. 0000000000.0005, 2020.
[26] Alvin Chan and Yew-Soon Ong. Poison cure: Detecting & neutralizing variable-sized backdoor attacks deep neural networks. arXiv preprint arXiv:1911.0804 2019.
[27] Huili Chen, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. Deepinspect: A black-box trojan detection and mitigation framework deep neural networks. IJCAI, 2019.
[28] Haripriya Harikumar, Vuong Le, Santu Rana, Sourangshu Bhattacharya, Sunil Gupta, and Svetha Venkatesh. Scalable backdoor detection neural networks. ECML, 2020.
[29] Zhen Xiang, David J. Miller, and George Kesidis. Detection of backdoors in trained classifiers without access to the training set. IEEE TNL, 2022.
[30] Prannay Khosla, Piotr Teterwak, Chen Wang. Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Liu, and Dilip Krishnan. Supervised contrastive learning. NIPS, 2020.
[31] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. simple framework for contrastive learning of visual representations. ICML, 2020.
[32] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang. David Lie, and Nicolas Papernot. Machine unlearning. IEEE S&P, 2021.
[33] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hicrarchical image datahase. CVPR, 2009.
[34] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[35] Vige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Frasing backdoor triggers from deep neural networks. ICIR, 2021.
[36] Zhao, Pin- Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. Bridging mode connectivity in loss landscapes and adversarial robustness. ICLR, 2020.
[37] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor lcarning: Training clean models on poisoned data. In NIPS, 2021.
[38] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. ICI.R, 2015.
[39] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.


Reviews:
[{"review_id": "cq2blWJobTA", "reviewer": "Reviewer_gz1a", "summary": "Through the observation that clean and backdoored data with a dissimilar feature representation after data transformation techniques (e.g. rotation, scaling), the author proposed a sensitivity metric, feature consistency towards transformations (FCT), to detect the potential backdoor samples. The author further proposed two backdoor removing modules with inspiration from the existing defenses of semi-supervised learning and backdoor unlearning. Extensive results show that the proposed methods outperform the existing backdoor defenses either in backdoor detection or backdoor removal.\n", "questions": "At the same time, I have some concerns about the paper's contributions:\n\n- At a first glance, the proposed methods seem as a unified framework including backdoor detection and backdoor defense forming an integral defense pipeline from training to post-training phase. But after a depth-reading, I consider this work is much like a patchwork combining existing techniques. For instance, the proposed feature detection matrix has been reflected in [1], and the other two defense modules ST and BR are much related to the DBD [2] and ABL [3] respectively. So, please explain clearly the main contribution and what specific contribution\\novelty of this work compared with others.\n\n- In addition, I doubt the generality of this method, for which the underlying assumption is that there are differences in feature representation between backdoor samples and clean samples. In fact, this assumption is often related to the design of backdoor models, and trigger types (static trigger, dynamic trigger, optimization-based trigger, etc). For more powerful adaptive attacks such as FC [4], dynamic [5], and latent modification [6], this assumption is no longer valid. In this case, how to ensure adequate defense? The author is invited to give an explanation and further experimental evidence.\n\n[1] Li Y, et al. Rethinking the trigger of backdoor attack. arXiv preprint arXiv:2004.04692, 2020.  \n[2] Huang K, et al. Backdoor defense via decoupling the training process. ICLR, 2022.  \n[3] Li Y, et al. Anti-backdoor learning: Training clean models on poisoned data. NeurIPS, 2021.  \n[4] Shafahi A, Huang W R, Najibi M, et al. Poison frogs! targeted clean-label poisoning attacks on neural networks. NeurIPS, 2018.  \n[5] Nguyen T A, Tran A. Input-aware dynamic backdoor attack. NeurIPS 2021.  \n[6] Doan K, Lao Y, Li P. Backdoor attack with imperceptible input and latent modification. NeurIPS, 2021.  ", "limitations": "It is still unclear that the underlining assumption behind ‘feature consistency' is supported efficiently by the proposed solutions with only empirical results. It needs some explanation about this result to backup this drawback.\n", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is well-organized and the main insight is the observation of feature separation by data transformations. The defense solution including a secure training (ST) module and a backdoor removal (BR) module seems simple yet effective against various classical backdoor attacks.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "b2yeTSo-9F0", "reviewer": "Reviewer_dNMX", "summary": "In this paper, the authors reveal the sensitivity of poisoned samples to transformations and propose a sensitivity metric FCT. And the authors  propose two effective backdoor defense methods for training a secure model from scratch and removing backdoor from the backdoored mode. ", "questions": "1. In the introduction, it is suggested that the authors add the background to the application of the proposed method, thus enhancing the motivation for the paper.\n2. The authors only list the researches for the image super-resolution and don't detail the reasons why these approaches are not sufficient for their goal.", "limitations": "Yes,  the authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors  design a simple sensitivity metric to distinguish poisoned samples from clean samples in the untrustworthy training set, and  propose two effective backdoor defense methods. The paper tackles an interesting issue, and the efforts of the authors are clear in investigating the problem and in writing the manuscript.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aWqaoddm5gX", "reviewer": "Reviewer_tGWV", "summary": "The authors observe that the features of poisoned samples are more sensitive to image transformations and use this observation to filter out poisoned data from the training set.  They use this for two different defenses, one to filter out training data before training and one to remove a backdoor from an already backdoored model.  They test against a range of attacks and compare to a variety of defenses.", "questions": "N/A", "limitations": "The authors refer to Appendix G for limitations which just contains experiments with different percentages of training samples poisoned.  I don't think this fully encompasses limitations of this paper, but I think the limited testing scenarios are clear to a reader, so it's not that big of a deal.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I thank the authors for their interesting work!  This topic is important for security, and I think the NeurIPS community will benefit from new works on defense against poisoning.  The experimental results are pretty strong, and the work compares to other defenses and a range of attacks, so I lean towards accepting.\n\nFeedback can be found below:\n\nFor “paradigm 1” did you compare to very strong data augmentations, like mixup/cutmix/maxup, which have been shown to be strong defenses against backdoors or various adversarial training strategies that have been used against backdoor attacks?\n\nIt might be good to test against optimization based poisons, such as Sleeper Agent, which don’t have patches on the training samples.  I wonder if these can defeat your defense.  They can also be made adaptive to beat your defense since you could add an objective that makes the features of poisoned samples insensitive to transformations.\n\nMinor point: Full ImageNet does not seem prohibitively expensive here, so it would be good to test there, since it’s a standard setting.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4xjRo3AA04s", "reviewer": "Reviewer_FGt5", "summary": "In this paper the authors: (a) Attribute the effectiveness of backdoor attacks to overfitting which they measure by proposing a metric called Feature Consistency towards Transformations. Using this metric they partition a given training dataset into potentially clean and potentially polluted samples and (b) They propose two defense algorithms (i) train a secure model from scratch (ii) Training a backdoored model first then removing the backdoor from the model.\n\nTo show (a) they pass the original image with the trigger and an augmentation of the triggered image to the same model and calculate the L2 distance in the feature embedding space. They define large distances mean more sensitivity. Using this metric they create a \"sample-distinguishment\" SD module to partition a given dataset by thresholding the FCT scores.\n\nFor defending against backdoor attacks from a potentially polluted dataset they propose two approaches:\n1. Secure Training From Scratch: They follow an approach similar to [29] Contrastive Supervised Learning where they have a two term contrastive loss (a) self-supervised loss on clean and polluted samples (b) supervised loss on clean vs polluted designated by the SD module using FCT thresholding. After they train the feature extractor, they train a classifier on top by minimizing a two term cross entropy loss (i) increasing the likelihood of predicting clean samples as their correct class (i) decreasing the likelihood of predicting the identified polluted samples as incorrect.\n\n2. Unlearn and relearn: on the found clean and polluted sets, they iteratively decrease the likelihood of the polluted predictions and increase the likelihood of the clean predictions using cross entropy loss.\n\nThey show with extensive ablation studies the effectiveness of the different choices they make in their defense algorithms.\n", "questions": "- Are the chosen hyper parameters easily transferrable to other datasets and network architectures?\n- How sensitive is FCT to the dimensionality of embeddings since it's calculating a euclidean distance? Will it become harder to find a threshold for higher dimensions?\n- How are the 0.05 and 0.2 thresholds set for the SD module?\n- Is it possible to train a trigger in a whitebox setting with all of the hyper parameters of the defense method know to the attacker?\n\n", "limitations": "- An important aspect of the work is setting the threshold for SD. It is not explained how this threshold can be tuned without knowing beforehand which samples contain the trigger and the distribution of distances. Or it's not discussed how sensitive the method is setting these thresholds to a different value.\n\n- A problem can be that in a whitebox setting the attacker can design the trigger such that FCT and its set thresholds are not effective by incorporating FCT in attack generation process by adding the inverse FCT to their optimization objectives for finding the trigger.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Novelty of their method and uncovering an interesting fact about the backdoored models and triggers.\n- Presentation and clarity.\n- Thorough experimentation for supporting their claims on the proposed defense method.\n\nWeaknesses:\n- All the experiments are on a single dataset, CIFAR. This raises questions around how easy it is to tune the different hyper parameters in the model on a different dataset and model architecture.\n\n- More experiments missing about the detection rate of the FCT metric. Simple ROC curves could give a sense of the power of their metrics. I understand that the first ablation study is trying to indirectly address this but since this is such an important idea in the paper, it could be isolated and tested.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept