SampDetox: Black-box Backdoor Defense via Perturbation-based Sample Detoxification
Authors: Yanxin Yang
Chentao Jia
DengKe Yan
Ming Hu
Tianlin Li
Xiaofei Xie
Xian Wei
Mingsong Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: Backdoor defense
black-box
Primary area: safety_in_machine_learning

Abstract: The advancement of Machine Learning has enabled the widespread deployment of Machine Learning as a Service (MLaaS) applications. However, the untrustworthy nature of third-party ML services poses backdoor threats. Existing defenses in MLaaS are limited by their reliance on training samples or white-box model analysis, highlighting the need for a black-box backdoor purification method. In our paper, we attempt to use diffusion models for purification by introducing noise in a forward diffusion process to destroy backdoors and recover clean samples through a reverse generative process. However, since a higher noise also destroys the semantics of the original samples, it still results in a low restoration performance. To investigate the effectiveness of noise in eliminating different types of backdoors, we conducted a preliminary study, which demonstrates that backdoors with low visibility can be easily destroyed by lightweight noise and those with high visibility need to be destroyed by high noise but can be easily detected. Based on the study, we propose SampDetox, which strategically combines lightweight and high noise. SampDetox applies weak noise to eliminate low-visibility backdoors and compares the structural similarity between the recovered and original samples to localize high-visibility backdoors. Intensive noise is then applied to these localized areas, destroying the high-visibility backdoors while preserving global semantic information. As a result, detoxified samples can be used for inference, even by poisoned models. Comprehensive experiments demonstrate the effectiveness of SampDetox in defending against various state-of-the-art backdoor attacks.

SampDetox: Black-box Backdoor Defense via Perturbation-based Sample Detoxification
Yanxin Yang', Chentao Jia', DengKe Yan', Ming Hu² Tianlin Li Xiaofei Xie?, Xian Wei Mingsong Chen
1 MoB Eng. Research Center of SW/HW Co-Design Tech. and App., East China Normal University
Singapore Management University, Nanyang Technological University {52275902023, 51 51265902040, 51265902053 @stu.ecnu ecnu.edu.cn. hu.ming. work @ gmail. com, tianlin001 @ att..du..g.. xfxie smu.edu.sg. {xwei, mschen) @ ses..ccnu.eu..c
Abstract
The advancement of Machine Learning has enabled the widespread deployment of Machine Learning as Service (MLaaaS) applications. However, the untrustworthy nature of third-party MI. services poses backdoor threats. Existing defenses in MLaaS are limited by their reliance on training samples or white-box model anal- ysis, highlighting the need for black-box backdoor purification method. In our paper, we attempt to use diff'usion models for purification by introducing noise a forward diffusion process to destroy backdoors and recover clean samples through a reverse generative process. However, since a higher noise also destroys the semantics of the original samples, it still results in low restoration performance. To investigate the effectiveness of noise in eliminating different types of backdoors, conducted preliminary study, which demonstrates that backdoors with low visibility can be easily destroyed by lightweight noise and those with high visibility need to be destroyed by high noise but be easily detected. Based the study, we propose SampDetox, which strategically combines lightweight and high noise. SampDetox applies weak noise eliminate low-visibility backdoors and compares the structural similarity between the recovered and original samples to localize high-visibility backdoors. Intensive noise is then applied these localized areas, destroying the visibiiiyyy backdoors while preserving global semantic informa- tion. As result, detoxified samples can be used for inference even by poisoned models. Comprehensive experiments demonstrate the effectiveness of SampDetox in defending against various state-of-the-ar backdoor attacks. The source code of this work is publicly available at
Introduction
With Artificial Intelligence (AI) technologies showing advantages in tasks such image classifi- cation and target detection, they are widely used safety- and security-critical applications such autonomous processing [1, 21. IoT System [3, 4] and health-care [5, 6]. As Machine Learning as a Service (MLaaS) [7, 8, 9] becomes increasingly popular, AI applications are relying on Al services provided by third parties. However, since users cannot guarantee the trustworthiness of service providers, they face serious threats of backdoor attacks [10, 11, 12]. Typically, adversaries inject backdoors into deep models numerous occasions, especially when the training samples collected from unreliable sources, or the pre-trained deep models are obtained from untrusted third parties. During the inference phase, when fed clean samples, the backdoored models behave normally
Miing Hu and Mingsong Chen the corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

with satisfactory classification performance. However, when processing poisoned samples containing trigger patterns, the backdoored models will be fooled into predicting attack target categories with high confidence.
Although various defense methods have been proposed identify and destroy hackdoors, most of them are white-bos defenses that require additional training samples or the right to analyze model parameters [13, 14, 15J. Since in MLaaS-based applications users are not allowed access the original training data and parameters of models, these backdoor defense methods are strongly limited. In contrast, black-box backdoor defense methods do not have such requirements. So far, black- -box backdoor defense methods can be classified into three categories, i.e., model detection sample detection-basse based [7, 17], and sample purification-base idea of the first two categories is to simply discard training samples or deep models once they are identified as poisoned. In this case, the usability and performance of classification tasks are greatly affected. As an alternative, sample patterns in samples. However, existing sample purification-bant assumption that trigger patterns are small and only located in the corners of samples, which is not always true in practice. As a result, such methods can only be used to defend against specific types of backdoor attacks. Worse still, most of the above defense methods need to repetitively feed samples to deep models and get their continuous feedback, resulting in extra non-neglig computation overhead. Therefore, how to effectively mitigate the impacts of all possible backdoor attacks without deteriorating the overall inference performance is becoming a great challenge in black-box defense.
[16], [18, 19] defense methods. The basic
ceafiicaationbaaeee approaches strive to destroy backdoor trigger
approaches are based on the strong
Intuitively, adding noise to poisoned samples can disrupt the semantics of the backdoor, thereby eliminating it from the sample. To preserve the semantics of the original sample while removing the backdoor, the diffusion model [20, 21] serves as an effective solution to restore the sample. However, different backdoors exhibit varying levels of robustness to noise, which requires different noise amplitudes for effective disruption. In addition, the amplitude and range of noise will affect the quality of samples restored by the diffusion model. We performed a preliminary study on various backdoor attacks to explore the robustness of different types of backdoors. We found that: i) invisible triggers with low robustness (to random noise) can be easily destroyed, and ii) backdoor triggers with high robustness are easily detectable due to their high visibility.
Inspired by these observations, to address the challenge of black-box backdoor defense, we propose novel two-stage method called SampDetox. Based on our first observation, we globally apply coarse-granule grannl-agalllrrtty noises on each given sample in the first stage of SampDetox, where backdoor triggers with low visibility can be easily destroyed. To counteract the effects of such added noises, we then restore the samples by denoising them using diffusion models. Note that for a poisoned sample, its poisoned regions with high visibility, compared with clean regions, will be different from their counterparts in the restored version (see the proof in Theorem 4.2), making them identified easily. Based our second observation, the second stage can quickly locate such robust triggers and destroy them using our dedicatedly designed noises. Similar to the first stage, the second stage then leverages diffusion models to restore the samples. Since our approach does not pose any assumptions or requirements on the models, it accommodates arbitrary backdoor attack scenarios. In summary, this paper makes the following three major contributions:
We perform a preliminary study on a wide spectrum of backdoors to reveal the correlation between the visibility of triggers and the robustness of poisoned samples.
We present a novel perturbation- theoretical foundations. Our approach can effectively destroy all possible triggers with dedicatedly designed noises and does not compromise the overall inference performance.
sample detoxification method together with its
We conduct extensive experiments to show the applicability and superiority of our approach over state-of-ther ahr (SOTA) backdoor defense methods.
2 Related Work
Backdoor Attacks. Existing backdoor attacks can be mainly classified into two categories, i.e., visible attacks and invisible attacks, based on the visibility of trigger patterns during the inference phase. Specifically, visible attacks do not consider the concealment of trigger patterns. For example, BadNets [22] uses visible trigger patterns the form of fixed pixels patched in the corner of the images. Although there exist various visible attacks (c.g., training set corruption without label
2

---

poisoning [23], label-consistent backdoor attacks [24], attacking pre- rre-traine models [25], and specific- sample triggers [26]), few of them can be applied in real scenarios, since the triggers on samples are conspicuous and can be easily detected by manual inspection. To make attacks stealthier, more and more backdoor attacks (e.g., Blended [27]. WaNet [28], and ISSBA [29]) adopt invisible triggers. For example, BPP [30] uses image quantization and dithering as imperceptible backdoor triggers to avoid manual inspection.
Backdoor Defenses. According to the capabilities of defenders in manipulating the training and inference processes on deep models, backdoor defense methods can be classified into two categories, ie., white-box and box defenses. Typically, white-box methods defend against backdoor attacks by changing training processes [31, 32], extract gradients [33], and fine-tuning and -pruning of poisoned models [34, 35]. However, these methods require access to model parameters, which strongly restricts their usage (e.g.. MI.aaS applications)
practice. For black-box defenses, such as model detection-based [16], sample detection- baasd [7, 17], and sample purification-bast 19] approaches, defenders only can rely on the predictions of input samples. For example, as model detection-based approach, CBD [36] can effectively identify whether models are poisoned by analyzing their predictions. However, simply discarding the suspicious model/sample does not apply to real-world classification tasks. Rather than ignoring poisoned samples, sample purification-bant bbase methods aim to destroy the triggers on samples. For example, Sancdifi [37] measures saliency maps and purifies the specific areas of samples. BDMAE [19] reconstructs the local area of the samples using its AutoEncoder. ZIP [18] transforms and reconstructs samples independently
[18,
models
through zero-shot image purification. However, all these purification-bat methods can handle only
small triggers within specific regions.
The robustness of triggers against perturbations modifications plays an important role in deter- mining the performance of backdoor attacks and defenses. For example, CBD [36] analyzes the robustness of triggers with different sizes against random noises to certify its proposed defense scheme. However, CBD only investigates attacks with simple triggers without taking SOTA backdoor attacks into account. To the best of our knowledge, SampDetox is the first attempt consider the correlation between the visibility of backdoor triggers and the robustness of poisoned samples. As a model-indepen independent method, SampDetox enables perturbation-basee sample detoxification, which can effectively defend against complex backdoor attacks within arbitrary application scenarios.
3 Preliminary Study
This section first reviews the process of backdoor attacks. Then, it gives our definitions of the visibility of backdoor triggers and the robustness of poisoned samples and reveals their inherent correlation based on a preliminary study.
3.1 Evaluation Metrics
Assume that M is a model poisoned by some backdoor attack. Let xf be a clean sample, and z represent its poisoned version by adding specific backdoor triggers. If the backdoor attack succeeds, the model M should classify x into its correct category while classifying x" into target category. To defend against backdoor attacks, the findings in [38] show that transforming or perturbing samples will achieve defensive effects against locally patched triggers. However, according to the observations in [18, 19], these operations are unsuitable for defending against other types of triggers. This is mainly because locally patched triggers are unremarkable and susceptible to disturbance, while other triggers are robust (i.e., not sensitive) to such backdoor defenses. To better understand the inherent characteristics backdoor attacks and the trends of their development, we conduct a preliminary study existing attacks to reveal the correlation between the visibility of backdoor triggers and the robustness of their poisoned samples, whose descriptions are as follows.
Visibility, When a backdoor attack is applied to some deep model, the visibility of embedded triggers indicates the stealthiness of the backdoor attack. In this paper, we use the Structural Similarity Index Measure (SSIM) [29, 39] to evaluate the visibility of backdoor triggers. This is because, unlike traditional measurements like Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR), SSIM takes into account the local characteristics of samples, aligning more closely with the visual system of human eyes. Let (a E [-1, 1] be the SSIM between Iº and x", quantifying the visibility of trigger patterns embedded in z". For case of evaluation, we adopt the notation
3

---

== (1 SSIM(=", x²)))/ to normalize the visibility values. Specifically, the higher the value of u, the more notable the triggers are.
Robustness. For given poisoned sample x?, we propose to use the term robustness to quantify its capability resist perturbations modifications imposed by random noise. Assume that the backdoored model classifies x² into a specific attack target category due to the existence of triggers. We apply random noises x" and get modified sample Iym where the modification process can be described by the equation Zm = (1 7) )a + T) (e ~ (0, I)). Since 7) plays an important role in determining whether its modified version I'm belongs to another category, we use reflect the robustness of xº. Typically, the higher the value of 1/r, the more robust the poisoned sample r" is.
3.2 Observations from Attacks
To explore the correlation between the visibility of triggers and the robustness of their host poisoned samples, we conduct an experiment on the dataset CIFAR-10 10 against ten backdoor attacks, including both visible and invisible attacks, where each attack is applied on 200 randomly selected samples from CIFAR-10 10. Note that the experiment, we assume that each poisoned sample is touched by only one kind of backdoor attack. In other words, each poisoned sample is embedded with specific trigger. For each poisoned sample, we calculate the visibility score v of the trigger and the robustness Vr of the poisoned sample. As shown in Figure 1, for different poisoned counterparts of clean sample, their visibility and robustness scores differ significantly under different backdoor attacks.
0.5
BadNett
Blended
Wanet
SIG
LF
0.005/0.072
0.4
LC
WaNett
Clean
TrojanNN
ISSBA
Robustness
0.3
Dynamic
BPP
BadNets
Low visibility
0.045/0.075
0.2
Low robustness
High robustness High visibility
0.1
SIG
0.197/0.432
0.0
0.000
0.025
0.050
0.075
0.100
0.125
0.150
0.175
0.200
Figure 1:
Examples of poi-
Visibility
soned samples and their v/n.-
Figure 2: Correlation between v and T/
Figure 2 illustrates the (v.n,) pairs for all the poisoned samples under ten attacks (detailed in Section 5.1), where each filled pentagram symbol represents poisoned sample. From the figure, we can get the following two observations.
Observation 1. For backdoor triggers with low visibility (u < 0.13), their host samples (i.e., ones with 7)- < 0.18 located in the red box) have low robustness. According to [38], the triggers within these poisoned samples can be easily destroyed by lightweight noises.
Observation 2. For poisoned samples (i.e., those located in the blue box) with high robustness (nr 2 0.18), their backdoor triggers have high visibility, which renders them easily detectable and accurately located.
4
Our SampDetox Approach
4.1 Threat Model
We consider defending against backdoor attacks in black-box scenarios [17, 18], where defenders do not have access to the parameters of deep models and can only obtain predictions for samples. For attackers, we assume that they can completely control the training processes of models, enabling them to poison samples in datasets and modify model components to achieve poisoned models. In our approach, defenders strive to detoxify poisoned samples to classify them into their original correct. categories. For clean samples, defenders aim to maintain their classification accuracy.

---

Sample Poisoning
Diffusion Model
Noises
Diffusion Model
Clean Sample
Trgger
Stage Global Detoxification
Stage
Triggers
Polsoned Samples
Local Detoxification
Delonified
sarppees
Figure 3: Framework and workflow of our SampDetox approach
4.2 Overview of SampDetox
As observed in Section 3.2, there exists a strong correlation between the visibility of triggers and the robustness of poisoned samples, which can be used for backdoor defense. However, due to the lack of prior knowledge of clean samples, it is difficult figure out both the trigger visibility and robustness of poisoned samples. To defend various backdoor attacks, Figure 3 presents a novel perturbatton--baeed backdoor defense framework using our proposed sample detoxification method, which consists of two stages:
Stage 1: Global Detoxification. According to Observation I, backdoor triggers with low visibility typically have low robustness poisoned samples. In this case, such triggers can be easily destroyed by adding lightweight noises to suspected samples. However, due to the low visibility or even invisibility, is hard to determine the existence or location of triggers within a given sample. To destroy all these potential triggers, Stage 1 applies the noise-based perturbation (see Section 4.3) globally to each sample using lightweight noises. Note that the introduction of noises inevitably affects the inference accuracy of deep models. To counteract the impacts of added noises inference performance, then utilize 2DPM-based denoising (see Section 4.4) to denoise the samples. In this way, Stage can destroy possible triggers spread over the poisoned samples with low robustness.
Local Detoxification. According to Observation 2. poisoned samples with high robustness often embedded with highly visibility triggers. Based DDPM-based denoising in Stage the clean regions of poisoned samples can easily be restored to their original states, while the poisoned regions cannot, especially when triggers are highly visible. Due to this fact, by comparing potentially poisoned sample with its detoxified version by Stage 1 pixel by pixel, we can figure out the positions of obvious triggers and the required intensity of local noises. Then, Stage 2 applies the noise-based perturbation locally on each sample using pixel-specific noises to destroy such triggers. Similarly to Stage 1, the added noises in Stage 2 also greatly influence the final inference accuracy for both clean and detoxified poisoned samples, thus requiring diffusion models to perform the denoising. After Stage 2, both visible and invisible triggers can be safely eliminated without degrading the inference performance.
Stage 2:
4.3 Noise-based Perturbation
To quantitatively control the intensity of noises applied on samples in each stage of SampDetox, we resort to the forward Denoising Diffusion Probabilistic Model (DDPM) [20], which can also be used to determine the number of denoising steps required in the DDPM-based denoising process. Assume that 2g is an input sample, and x; is the sample after adding noises for i times (steps). In the forward process of DDPM, the model incrementally adds noises Zo step by step until the sample (denoted by 2rr is completely composed of random Gaussian noises at step T., where T is a diffusion model-specific constant. In each step t, xe is obtained by adding a random Gaussian noise c~N(0, I) to It-1 based on the formula I==VT- Be + Vicc where 3,E 1). For a given Tt-1, the posterior probability of za can be represented I only relies on step 1-1, the forward process can be regarded as a Markov process in the form of q(rx|2o)=[[[= ((1a/11-1) For a given To, we can get the probability of x; as
vI TT,,-1, 3,I). Since step
P(x, (r) = N (ar; Taroo (1 T.) I),
where 0; 1 3, and o = IT (1 B.). According [20], can get the exact relationship between To and H; using the formula
Ft = varzo + a
(2)
where z0, 1) is random Gaussian noise.
5

---

Leveraging the forward DDPM, our approach aims to add proper random Gaussian noise on samples within steps based on Equation 2, which can destroy backdoor triggers samples. Therefore, Theorem 4.1 investigates the potential of DDPM to eliminate triggers within poisoned samples.
Theorem 4.1. Suppose that 2, represents diffusse sample step t of the forward process in DDPM, where t E [0, (). Given a clean sample Tó and its poisoned version z we obtain o; and zt through the forward DDPM, whose distributions are Pt(z) and Q2(x),, respectively. By using the Kullback-Leibe divergence [40, 41] to measure the distance between the two distributions, we have LL
& (PPl)()
(3)
where the equality happens only when P. Pt-Qu.. and the inequality implies that the KL divergence between Pt and Qi monotonically decreases along with the forward DDPM.
Please refer to Appendix B.I for the proof of the theorem. According to Theorem 4.1, for any positive number &. there exists a minimum & E [o, T] such that for any 2 we have (P(::]) S &. Note that this theorem forms the theoretical basis to utilize the forward DDPM to minimize the distribution discrepancy between poisoned and clean samples, thus mitigating the impact of backdoor triggers.
4.4 DDPM- Denoising
We also utilize DDPM to perform denoising on samples within the two stages of SampDetox. For each pixel in sample, the number of denoising steps is determined by its noise-based perturbation steps (i.e., t) as introduced in Section 4.3. In reverse DDPM, by taking the current sample state Te as an input, the diffusion model can be used achieve its previous state IT-1. Given both To and Tt, according to Bayes' Theorem, we can obtain the probability distribution of r2-1 as p(22-1 ( 20)= (x (z))) //ou))/((sss/////) (zo). By resorting to the relationship between To and Ie described by Equation 2, according to [20], we can use re to approximate To as follows:
Fo
(xe
a.2)) 2t ((x,tt).
Va
where 2, is predicted by the pre-trained neural network 0 of DDPM, denoting the estimation of the real noise in step t. Accordingly, we can obtain - from Nz:
1 0 0(1, t)) + 0
It-1
(xt
(5)
Vas
VT
where of
-1- 1-Th,
Br and 2 I).
Unlike existing DDPM- methods that strive to restore samples from Gaussian noises, SampDetox uses diffusion models to denoise samples generated by our noise-based perturbation approach (see Section 4.3). Assuming that the training data used for the pre-trained diffusion model do not involve any knowledge about backdoor triggers, according to the principle of DDPM, the denoised results will restore the original samples with all the triggers cleansed. Theorem 4.2 investigates the difference between clean and poisoned regions of poisoned samples after denoising.
Theorem 4.2. Suppose that R(x) denotes a specific region of the sample x. Given a poisoned sample Sh use Re(x²) and R2 Rx(xx) to represent clean region and poisoned region of the sample, respectively. We perform the noise-based perturbation on 2 for t steps and get the noisy samples xre and then utilize DDPM to get a denoised sample 2g When comparing different regions of the original sample oy with their counterparts of the detoxified sample T.. we have
(x)))) - R2(=z111
n/2 where nn and n2 represent the numbers pixels in R,(xx) and Re(2%) respectively.
11
Please refer to Appendix B.2 for the proof of the theorem. According to Theorem 4.2, we can calculate the similarity between the denoised and original sample identify the positions of trigger patterns. The inequality in Theorem 4.2 implics that for a poisoned sample, its poisoned regions, compared with clean regions, will be more different from their counterparts in the restored version.

---

4.5 Implementation of SampDetox
Algorithm 1 Implementation of SampDetox
Algorithm details the implementation of SampDetox. Lines 1-6 describe the pro- cess of global detoxification, while Lines 8-21 present the local detoxification pro- cess. Specifically, Line applies the noise- based perturbation to the sample follow- ing Equation with fixed noise intensity 41 Lines 2-6 show the process of DDPM- based denoising obtain the denoised sam- ple To. In Line 7, SampDetox calculates
Input: i) Th. original input sample; ii) f1. f2, the numbers of noise-adding and denoising steps in Stage I and Stage 2, respectively; iii) 0, diffusion model:
Output: a detoxified sample:
#7, <- /07,70 2o + V1 07,² 1/2~N(0.1)
2: for t
do
3:
a ~~ N0, ift > 1, clse =0
4: of + 1 Te
1----1 -B1
5: Øt-1 - yoa (xc
L-y VI-a
( 1)) + 022
end for
the SSIM score for each pixel between the original sample Zo and its denoised ver- sion T.. Lines 8-11 describe how a noise with specific intensity (i.e.. m(()) is added to the pixel Lines 12-21 present the DDPM-based denoising process, whether
7: M +- PixeISSIM(xD: To)
8: for each pixel in M do
9: m[i] + fy M(i
10: 37, - m
1/2 N(0,1 To 11-mmlif
11: end for
12: for t Tz to do (0, if Ze ((.e. f)
13:
else
SampDetox iteratively calculates the esti-
14:
mation of the real noise 2, with Te using
15:
for each pixel in do
the diffusion model and then selectively
16:
if ml] > I then
denoises each pixel accordingly. In Lines 16-19, SampDetox only updates pixels with
17:
It- [0] + [ax
1-0+ Vi-ai
2 x1]) + 0.2
18:
else xT- <- me
noise intensity higher than that of the cur-
19:
end if
rent step. Finally, Line 22 returns the re-
20: end for
sulting detoxified sample zo, which is the denoised sample by Stage 2.
21:
end for
22:
return To
5
Experiments
To evaluate the effectiveness of our approach, we implemented our approach, i.e., SampDetox, on top of Pytorch (version 1.13.0). We compared the defense performance between SampDetox and state-of-the-art (SOTA) black-box defense methods against various well-known backdoor attacks. All experiments were carried out on an Ubuntu workstation equipped with one Intel i7-13700K CPU, 64GB memory, and one NVIDIA GeForce RTX4090 GPU.
5.1 Experimental Setup
Dataset and Model Settings. We investigated three classical datasets (i.c., CIFAR-10 [42], GT- SRB [43], and Imng-eeeeeNet [44]) and three models (i.e.,
ResNet34 [45] and
VGG-19 [46]). Due to space limitations, this section only presents the experimental results of PreAct- find similar trends from the experiments using different models and datasets. Please refer to Appendix C for more details.
ResNet18 the CIFAR-10 dataset. Note that
Evaluation Metries. To objectively evaluate the effectiveness of given backdoor defense method, we adopted the following three metrics: i) Clean sample Accuracy (CA), which denotes the inference accuracy of clean samples processed by the defense method; ii) Poisoned sample Accuracy (PA), which indicates the inference accuracy of poisoned samples purified (detoxified) by the defense method based their ground-truth labels; and iii) Attack Success Rate (ASR), which represents the rate of successful attacks by poisoned samples. Note that a higher CA means that the defense method causes less interference in the classification of clean samples. Typically, backdoor defenders strive to achieve both high CA and PA with lowered ASR.
Attack Methods. We conducted defenses against ten SOTA backdoor attacks, i.e., BadNets [22], SIG [23], Label Consistent (LCC) [24], TrojanNN [25], Dynamic [26], Blended [27], Low Frequency (LF) [47], WaNet [28], ISSBA [29], and BPP [30]. Note that the first five are visible attacks, while the last five are invisible attacks. We used the same benehmark settings [48] to configure all these
attack methods. Please refer to Appendix A. for more details.
7

---

Clean
BadNets
SIG
LC
TrojanNN Dynamic Blended
LF
WaNet
ISSBA
BPP
Triggers
Detoxified Samples
Figure 4: Illustration of poisoned samples, backdoor triggers, and their corresponding detoxified samples for the ten attacks, respectively.
Table Defense performance comparison, where Bold fonts and underlines denote the best and the second-best values, respectively.
Defense-
No Defense
Sanediñi
BDMAE
ZIP
SampDetox (Ours)
Atlak
CA(R) PA(E) ASR(S) CA(S) PA(S) ASR(S)CA((M)) PA(K) ASR(E)) CA(R) PA(S) ASR(S) (CA(S) PA(%)
No Attack
93.84
BadNets
92.00
10.18
99.97
76.59
89.55
1.92
89.02
90.10
2.32
88.12
86.52
7.17
89.57
90.15
2.11
SIG
84.94
9.78
98.50
70.68
43.73
29.58
82.77
10.08
96.65
82.15
35.60
36.58
83,71
65.06
11.03
LC
84.34
10.26
99.06
68.44
51.78
3.64
79.75
73.39
2.01
79.85
74.92
2.06
80.72
74.36
1.55
TrojenNN
93,20
11.07
99.03
76.63
90,84
1.81
91.19
89.35
2.47
87.35
86.91
7.10
92.78
89.95
1.86
Dynamic
91.09
10.02
98.19
76.24
68.89
7.92
88.48
75.78
12.57
87.96
80.19
2.75
88.52
88.62
1.45
Blended
93.85
10,93
99.51
77.92
51.12
15.06
87.84
14.88
96.46
88.51
63.81
8.72
90.23
86.65
1.96
LF
93.63
11.13
99.48
71.92
49.95
16.57
87.50
13.63
80.97
88.
86.59
5.85
90.01
87.40
3.02
WaNet
91.43
10.27
91.05
71.87
42.97
14.35
85.95
23.19
50.63
86.91
85.22
8.36
89.34
88.92
5,59
ISSBA
93.57
11.38
95.96
77.70
$2.05
14.20
86.18
53.12
22.39
87.75
85.46 46
1.79
90.74
86.51
1.60
BPP
91.38
9.46
98.40
75.32
50.42
15.25
86.69
21.73
53.46
85.42
82.94
7.20
90.59
84,83
6.15
Defense Baselines. We compared SampDetox with three SOTA sample purification- based backdoor defenses: i) Sancdifi [37], which utilizes the RISE algorithm to measure the saliency maps and purify the specific areas of samples; ii) BDMAE [19], which uses a masked AutoEncoder to defend against backdoor attacks; and iii) ZIP [18], which applies linear transformations samples and restores them under noises by using diffusion models. Note that both Sancdifi and BDMAE need to feed samples into deep models repeatedly to obtain their predictions, while ZIP does not rely on predictions. Please refer to Appendix A.2 for more details.
5.2 Performance Comparison
Figure 4 shows an example of backdoor defense against the ten different attacks using our proposed SampDetox. Here, the first row presents both clean sample and all its poisoned counterparts. The second row presents the triggers identified by stage 2 of SampDetox, while the third row exhibits the detoxified samples of each attack. From this figure, we can find that SampDetox can effectively figure out all visible and invisible triggers embedded by different SOTA backdoor attacks.
Table compares the defense performance between SampDetox and the three baselines on the CIFAR- 10 dataset using &re Please see Appendix C.1 for the results of datasets GTSRB and Tiny- Imagee.. From this table, different attacks, indicating that the impact of SampDetox on clean samples is negligible. Meanwhile, SampDetox has the best PA and ASR for 8 out of 10 attacks, respectively. Even for all the remaining cases (i.c., BadNets, LC, and TrojanNN), our approach can achieve the second-best PA and ASR, whose values are quite close the ones of the best PA and ASR. As an example of the BadNets attack, Sancdifi has the best ASR, which is only 0.19% lower than our approach. However, in this case, our approach significantly outperforms Sanedifi CA by 12. 22.8%. For the TrojanNN attack. though Sancdifi can achieve slightly better PA (by 0.89%) and ASR (by 0.05%) than SampDetox, SampDetox notably outperforms Sancdifi in CA by 16. 15%. For the Label Consistent (L.C attack, ZIP can achieve slightly better PA, but its CA and ASR are worse than SampDetox. It is important to note that Sancdifi and BDMAE require feedback from deep models. As an alternative, our approach does not rely underlying models, showing the applicability and superiority of our approach.
find that SampDetox
achieve the highest CA for all
8

---

Table 2: Ablation study results of our approach against backdoor attacks with different visibility.
Visibility
Noise
Stage I
SampDetox (Stage Stage 2)
Attack
CA(S)
PA(S)
ASR(%)
CA(%)
PA(S)
ASR(%)
CA(%)
PA(S)
ASR(%)
BadNets
0.052
56.25
49.71
3,93
90.12
88.15
6.61
89.57
90.15
2.11
SIG
0.185
51.81
17.16
15.89
83.10
9.48
92.33
83.71
65.06
11.03
LC
0.121
47.75
28.04
1.07
81.50
61.45
34.75
80.72
74.36
1.55
TrojanNN
0.137
58.03
45.89
5.68
92.54
35.51
46.86
92.78
89.95
1.86
Dynamic
0.098
56.26
41.18
1.09
87.58
85.62
3.82
87.52
88
1.45
Blended
0.067
59.42
45.22
1,53
88.65
86.65
1.86
90.23
86.65
1.96
LF
0.005
56.12
40.95
2.55
89.43
87.50
3.02
90.01
87,40
3.02
WaNet
0.005
57.71
43.73
5.38
89.43
88,91
5.60
89.34
88.92
5,59
ISSBA
0.006
57.10
37.35
1.14
90.92
86,50
1.61
90.74
86.51
1.60
BPP
0.009
58.40
42.03
5.75
89.09
84.84
6.15
90.59
84.83
6.15
5.3
Ablation Studies
Impacts of Different Stages and Denoising. The implementation of SampDetox consists of two stages. The first stage strives to destroy triggers globally on a given sample in a coarse manner, while the second stage tries identify fine triggers and remove them locally. To examine the impacts of these two stages, we consider variant (i.e., "Sttge 1") of our approach, which does not take stage 2 of SampDetox into account. Meanwhile, to evaluate the impact of the denoising operations, we considered another variant of SampDetox (i.c., "Noise*"), where denoising is not applied to restore the samples in both stages. Table 2 presents the ablation study results against the ten attacks. Note that to enable an intuitive understanding of the role of each stage in SampDetox, this table provides the average visibility of backdoor triggers for each attack.
For all the defense methods in Table 2, we can find that the smaller the trigger visibility caused by backdoor attacks, the lower PA and higher ASR comparable ASR results to SampDetox, SampDetox outperforms "Noise*" in both CA and PA signif- icantly. It means that simply adding noises samples can effectively destroy the backdoor triggers of the samples. However, the noises themselves will also result in a sharp decline in the inference accuracy of clean and poisoned samples. If we merely adopt the first stage of SampDetox due the denoising operation by diffusion models, both CA and PA can be improved. However, since the first stage only coarsely applies the perturbation on samples, most subtle backdoor triggers still survive. As an example in SIG attack, "Stage 1" can only achieve a PA of 9.48% but with an ASR of 92.33%. By combining both Stages and 2, SampDetox can effectively identify such undetectable triggers and destroy them without affecting the inference of original samples, leading to a better PA and ASR. For example, in SIG attack, by using SampDetox, we can achieve a PA of 65.06% and an ASR of 11.03%.
achieve. Although "Noise*" can achieve
Impact of Hyperparameters T and Tz- In our approach, we use two hyperparam- eters, i.c., & and t2 to control the noise intensity and denoising samples, respee- tively. We investigated two variants of Sam- pDetox to evaluate the impacts of & and
Table 3: Ablation study results for different & and f22
Fixed 7, =
Fixed 7, = 20
2
CA(%)
PA(%)
ASR(
7
(CA())
PA(%)
ASR($))
5
92.07
62.48
27.58
40
92.19
60.90
30.07
10
91.22
78.69
12.96
60
92.02
73.32
19.32
f2. For the first variant, we fixed the value
15
90.92
86.35
5.39
80
91.86
79.38
14.11
20
90.65
86.43
1.73
100
91.72
83.68
6.13
of 12 (i.e., 12=0) while allowing the tun-
25
88.26
85.13
1.80
120
92.02
85.22
2.34
ing of 7,. With different values of Z,, we can evaluate the defense performance of SampDetox against various invisible back-
30
86.77
84.56
1.71
150
91.85
84.92
2.28
35
84.91
83.78
1.75
200
92.26
81.87
2.30
40
82,30
83.01
1.72
250
92.39
77.03
2.29
door attacks (i.c., Blended, Low Frequency, WaNet, ISSBA, and BPP). Similarly, for the second variant, we fixed the value of &1 (i.c., 21= while changing the values of Ez, to evaluate the defense performance SampDetox against visible backdoor attacks (i.e., BadNets, SIG, LC, TrojanNNN and Dynamic). Table 3 shows the ablation study results with varying values of i or &g. Note that the results in the table represent the average results across multiple attack types. From this table, we can observe that when i and & increase, the ASR decreases, indicating the effect of adding noises in destroying backdoor triggers. However, a high value of 1 and la will also lead to significant reduction of CA and PA, showing that the denoising processes are strongly affected by excessive noises. Therefore, we suggest to set = 20 and fg 120 in practice.

---

5.4 Discussion
To show the applicability of SampDetox, we investigated the extra time overheads, explored training sample detoxification, evaluated the defense performance of SampDetox against SOTA adaptive attacks, and analyzed its limitations. Due to the space limitation, this section only presents the overhead caused by inference detoxification. Please refer to Appendix D for more discussions.
Extra time overhead. Based on our pro- posed two-stage perturbation- basdd sample detoxification, SampDetox protect any deep models during their inference phase. However, similar to other backdoor defense approaches, the detoxification process will inevitably lead to extra time overhead. To address this issue, SampDetox resorts to the Denoising Diffusion Implicit Model (DDIM) [21] to accelerate the denoising process. Figure 5 compares the average in- ference time per sample for different back- door defense methods. We find that with the help of DDIM, the inference time of SampDetox is comparable to that of the method without defense.
13917
13.8
(Scemdaa)
0.4953
0.4516
04
0.3526
0.4355
ROMAE
ZIP
ZIP-DDIM
Ours-DDIM
Defenne Methods
Figure 5: Inference time of different defense methods.
Conclusion
Based on our observed correlation between the visibility of backdoor triggers and the robustness of poisoned samples, this paper introduces novel black-box backdoor defense approach named SampDetox. By using our proposed perturbation- based sample detoxification, SampDetox can easily destroy backdoor triggers in samples and restore the samples using diffusion models. Extensive experiments against various complex backdoor attacks show the superiority of SampDetox from both perspectives of effectiveness and applicability to arbitrary scenarios.
Acknowledgement
This work was supported by the Natural Science Foundation of China (62272170). "Digital Silk Road" Shanghai International Joint Lab of Trustworthy Intelligent Software (22510750100), Shanghai Trusted Industry Internet Software Collaborative Innovation Center, the National Research Foundation, Singapore, and the Cyber Security Agency under its National Cybersecurity R&D Programme PCCRP25-P4-TAIICNN) Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore and Cyber Security Agency of Singapore.
References
[1] Hongyi Zhang. Jan Bosch, and Helena Holmström Olsson. End-to-end federated learning for autonomous driving vehicles. In Proceedings of International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2021.
[2] Xiaofeng Hou, Tongqiao Xu, Chao Li, Cheng Xu, Jiacheng Liu, Yang Hu, Jieru Zhao, Jingwen Leng, Kwang- Ting Cheng, and Minyi Guo. A tale of two domains: Exploring efficient architecture design for truly autonomous things. In Proceedings of International Symposium on Computer Architecture (ISCA), 2024.
[3] Ming Hu, F. Cao, Hongbing Huang, Min Zhang, Xiaohong Chen, and Mingsong Chen. Aiotml: unified modeling language for aiot-based syhsr-p systems. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., +2(11))3545-3558 2023.
[4] Chentao Jia, Ming Hu, Zekai Chen, Yanxin Yang, Xiaofei Xie, Yang Liu, and Mingsong Chen. Adaptivefl: Adaptive heterogeneous federated learning resource-constrained aiot systems. In Proceedings of The Chips To Systems Conference (DAC), pages 1-6, 2024.
10

---

[5] Muhammad Imran Razzak, Saeeda Naz, and Ahmad Zaib. Deep learning for medical image processing:
Overview, challenges and the future. Classification in BioApps: Automation of decision making, pages 323-350, 2018.
[6] Riccardo Miotto, Fei Wang. Shuang Wang, Xiaoqian Jiang, and Joel T Dudley. Deep learning for healthcare: review, opportunities and challenges. Briefings in bioinformatics, 19(6):1336-1244 2018.
[7] Junfeng Guo, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, and Cong Liu. Scale-up: An efficient black-box input-level backdoor detection via amalyzing scaled prediction consistency. arXiv preprint wrXiv2330.00325. 2023.
[8] Bao Gia Doan, Ehsan Abbasnejad, and Damith C Ranasinghe. Februus: Input purification defense against trojan attacks on deep neural network systems. In Proceedings of Annual Computer Security Applications Conference (ACSAC), pages 897-912, 2020.
[9] Wenke Huang, Mang Ye, Zekun Shi, Bo Du, and Dacheng Tao. Fisher calibration for backdoor- robust heterogeneous federated learning. In Proceedings of European Conference on Computer Vision (ECCV), 2024.
[10] Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE Transuctions Neural Networks and Learning Systems, 2022.
[11] Emily Wenger, Josephine Passananti, Arjun Nitin Bhagoji. Yuanshun Yao, Haitao Zheng, and Ben Y Zhao. Backdoor attacks against deep learning systems in the physical world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6206 0000-221,,222 2021.
[12] Yanxin Yang, Ming Hu, Yue Cao, Jun Xia, Yihao Huang, Yang Liu, and Mingsong Chen. Protect federated learning against backdoor attacks via data-free trigger generation. arXiv preprint arXiv:2308. / 1133, 2023.
[13] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In Proceedings of IEEE Symposium on Security and Privacy (S&P), pages 707-723, 2019.
[14] Zhihao Yue, Jun Xia, Zhiwei Ling, Ming Hu, Ting Wang, Xian Wei, and Mingsong Chen. Model- contrastive learning for backdoor elimination. In Proceedings of the 31st ACM International Conference on Multimedia (MM), pages 8869-8888 8880, 2023.
[15] Jun Xia, Ting Wang, Jiepin Ding, Xian Wei, and Mingsong Chen. Eliminating backdoor triggers for deep neural networks using attention relation graph distillation. In Proceedings of International Joint Conference Artificial Intelligence (IJCAI), 2022.
[16] Yingqi Liu, Guangyu Shen, Guanhong Tao, Zhenting Wang. Shiqing Ma, and Xiangyu Zhang. Complex backdoor detection by symmetric feature differencing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15003-15013, 2022.
[17] Hanxun Huang, Xingjun Ma. Sarah Erfani, and James Bailey. Distilling cognitive backdoor patterns within an image. arXiv preprint arXiv:2301.10900 2023.
[18] Yucheng Shi, Mengnan Du, Xuansheng Wu, Zihan Guan, and Ninghao Liu. Black-box backdoor defense via zero-shot image purification. arXiv preprint arXiv:2303. 12175, 2023.
[19] Tao Sun, Lu Pang, Chao Chen, and Haibin Ling. Mask and restore: Blind backdoor defense at test time with masked autoencoder. arXiv preprint arxiv:2303./5566 2023.
[20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion prohabilistic models. Advances in Neural Information Processing Systems (NIPS), 33:6840-6885 2020.
[21] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arX1v:2010.02502, 2020.

---

[22] Tianyu Gu, Brendan Dolan-Gavitt and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv: 06733, 2017.
[23] Mauro Barni, Kassem Kallas, and Benedetta Tondi. A new backdoor attack in cnns by training set corruption without label possonnng. In Proceedings of IEEE International Conference on Image Processing (ICIP), pages 101-105, 2019.
[24] Alexander Tumnrr, Dimitris Tsipras, and Aleksander Madry. Label-consistent backdoor attacks. arXiv preprint arXiv:1912.022771, 2019.
[25] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. Proceedings of Annual Network and Distributed System Security Symposium (NDSS), 2018.
[26] Tuan Anh Nguyen and Anh Tran. Input-aware dynamic backdoor attack. Advances in Neural Information Processing Systems (NIPS), 33:3454- 3464, 2020.
[27] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv: 2017.
[28] Anh Nguyen and Anh Tran. Wanet-imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102. 10369, 2021.
[29] Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran Hc, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 16463-16472, 2021.
[30] Zhenting Wang, Juan Zhai, and Shiqing Ma. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15074-15084, 2022.
[31] Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. In Proceedings of International Conference on Learning Representations (ICLR), 2022.
[32] Maurice Weber, Xiaojun Xu, Bojan Karlas, Ce Zhang, and Bo Li. RAB: provable robustness against backdoor attacks. In Proceedings of IEEE Symposium Security and Privacy (S&P), pages 1-12-, 2023.
[33] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor learning: Training clean models on poisoned data. Advances in Neural Information Processing Systems (NIPS), 101144400-49922 2021.
[34] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning Defending against backdooring attacks on deep neural networks. In Proceedings of International Symposium on Research in Attacks, Intrusions, and Defenses (RAID), pages 273-294, 2018
[35] Dongxian Wu and Yisen Wang. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems(NIPS), 14:16913-16925, 2021.
[36] Zhen Xiang, Zidi Xiong, and Bo Li. CBD: certified backdoor detector based on local dominant probability. In Advances in Neural Information Processing Systems(NIPS), 2023.
[37] Brandon B May. N Joseph Tatro, Piyush Kumar, and Nathan Shnidman. Salient conditional diffusion for defending against backdoor attacks. arXiv preprint arXiv:2301. 2023.
[38] Yiming Li. Tongging Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. Rethinking the trigger of backdoor attack. arXiv preprint arXiv:2004.04666 2020.
[39] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, (30) 612, 2004.
12

---

[40] Solomon Kullback and Richard A Leibler. On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79-86, 1951.
[41] Tim Van Erven and Peter Harremos. Rényi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory, 60(7):3777-3820, 2014.
[42] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. In Technical report, 2009.
[43] Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, 32:323-332, 2012.
[44] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211 15:211-220 1-252, 2015.
[45] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Proceedings of European Conference on Computer Vision (ECCV), pages 630-645, 2016.
[46] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
[47] Yi Zeng, Won Park, Z Morley Mao, and Ruoxi Jia. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 16473 16481, 2021.
[48] Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. Backdoorbench: A comprehensive benchmark backdoor learning. Advances in Neural Information Processing Systems (NIPS), 35: 15:16666-1055, 2022.
[49] Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li. and Shu-Tao Xia. Backdoor attack in the physical world. arXiv preprint arXiv:2/04.02:30 2021.
[50] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In Proceedings of European Conference on Computer Vision (ECCV), pages 87-102, 2016.
[51] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 3606-3613, 2014.
[52] Emily Wenger, Josephine Passananti, Arjun Nitin Bhagoji. Yuanshun Yao. Haitao Zheng, and Ben Y. Zhao. Backdoor attacks against deep learning systems in the physical world. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 6206 6215, 2021.
[53] Ruotong Wang, Hongrui Chen, Zihao Zhu, Li Liu, Yong Zhang, Yanbo Fan, and Baoyuan Wu. Robust backdoor attack with visible, semantic, sample-specific, and compatible triggers. arXiv preprint arXiv:2306.008811 2023.
[54] Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. In Proceedings of European Conference on Computer Vision (ECCV), pages 182-199, 2020.
[55] Siyuan Cheng, Yingqi Liu, Shiqing Ma. and Xiangyu Zhang. Deep feature space trojan attack of neural networks by controlled detoxification. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI)), pages 1 1156, 2021.
[56] Junyu Lin, Lei Xu, Yingqi Liu, and Xiangyu Zhang. Composite backdoor attack for deep neural network by mixing existing benign features. In Proceedings of Conference on Computer and Communications Security (CCS), pages 113-131, 2020.
13

---

A Detailed Experimental Setup
A.1 Attack Method Settings
BadNets [22]. BadNets is the first proposed backdoor attack method, which patches a fixed pattern on the corners of benign samples and modifies their labels to the attack target category. We use
3 x 3 black and white checkerboard in the lower right corner of samples as the trigger pattern.
SIG [23]. SIG modifies the pixels in samples based on the sinusoidal signal to generate poisoned samples. We the frequency of the sinusoidal signal 3 as the trigger pattern.
Label Consistent (LC) [24]. LC generates the poisoned samples using adversarial attack methods and keeps the original sample label unchanged. The trigger pattern is identical to BadNets, and other settings follow the original paper.
TrojanNN [25]. TrojanNN injects backdoors into pre-trained model instead of injecting backdoors during the training. We use a white apple image as the trigger pattern.
Dynamic [26]. Dynamic trains trigger generator to generate the specific triggers for each sample. The backdoor attack will only succeed if the specific trigger matches its sample. We follow the settings in the original paper to generate triggers.
Blended [27]. Blended generates poisoned samples by blending benign samples with fixed pattern. Blending with coefficient a, Blended increases the invisibility of the trigger to avoid manual inspection. We use "hello kitty" image the trigger pattern.
Low Frequency (LF) [47]. LF proposes an optimization- based approach to filter high-frequency artifacts to generate smooth triggers. We follow the settings in the original paper to generate triggers.
WaNet [28]. WaNet uses fixed warping functions distort benign samples slightly to construct the poisoned samples, We follow the settings in the original paper to generate poisoned samples with trigger patterns.
ISSBA [29]. ISSBA employs image steganography to embed the string trigger into benign samples, which generates unique triggers that specific to each sample. Following the settings in the original paper, we use the trained image steganography method.
BPP [30]. BPP generates the poisoned samples using image quantization. To inject the backdoor more efficiently, BPP proposes contrastive learning- approach improve the success rate of backdoor attacks. We set the negative ratio to 0.1.
A.2 Defense Baseline Settings
Sancdifi [37]. Sancdifi followed the settings in Section "Numerical Experiment" of its paper. Sancdifi computed RISE maps using 2000 random binary masks and set the saliency threshold to 0.95. For a fair comparison, Sancdifi used the same diffusion models ours.
BDMAE [19]. BDMAE followed the settings in Section "Method Configurations" of its paper, which used masked autoencoders pre-trained on ImageNet with 24 encoder layers.
ZIP [18]. ZIP followed the settings in Section "Purification Implementation" of its paper, which set its hyperparameter to 2. For a fair comparison, ZIP used the same diffusion models as ours.
A.3 Diffusion Model Settings
We utilize the framework of the improved diffusion model provided by [21]. The number of total diffusion steps was set 1000 for all the datasets in the experiments, the noise noise-scheeee schedule was set as "cosine", and the learning rate was set to le-4. In the experiments, we load the parameters of the pre-trained model for each dataset. For DDIM, compared with total steps of the diffusion model (e.g., T 1000), our approach does not require too many steps to add noise and denoise samples (e.g., 7 20 and T = 120). Therefore, when we use DDIM to accelerate the denoising, we set the speed-up pace to 20 to ensure the effecttiveness of our approach.
14

---

A.4 PixeISSIM Settings
In our work, the PixeISSIM denotes an intermediate data structure for calculating the SSIM [39] value of two given samples. Specifically, for each block pair in the two samples, we calculate its SSIM value and record it in PixelSSIM at the same position as one of the central pixels of the block pair. When sliding the block pair along the two dimensions of the samples, SSIM values for the block pairs and form the final PiXeISSIM. Note that by averaging all the values in PixeISSIM, we can achieve the SSIM value for the two samples. We use structural Srruucuurll.imilrittttl in the skimage library of Python to implement the PixeISSIM.
can collect all the
B Theoretical Justification
B.1 Proof of Theorem 4.1
Theorem 4.1. Suppose that at represents diffused sample step t of the forward process in DDPM, where t E [0, (). Given a clean sample ió and its poisoned version zy we obtain z; and xf through the forward DDPM, whose distributions are Pi(z) and Qx(x), respectively. By using the Kullback-Leible divergence [40, 41] to measure the distance between the two distributions, we have
KL(Pe+1llQuus S LLP.IIQ..
(7)
where the equality happens only when Pi=Q,, and the inequality implies that the KI. divergence between P. and Qe monotonically decreases along with the forward DDPM.
Proof:
First, following the definition of the diffused sample probability distribution in DDPM [20].
like Equation I, when given clean sample zū and poisoned sample z we have their distributions:
P(2[(=)) = Naa;; vari (1 11-00 -a) I) Q(=7(=²)) NZ22 Voru (1-r,)))
where a 11 -1-68 3r and ā2 IT ag Therefore, we know that both P( and Q(=/|=²)) are Gaussian distributions. When both terms of the KL divergence are Gaussian distributions, we can simplify the formula as follows:
= = p(z) log p(x)
KL (N (f) [N of))
dx g(x))
(z (z- /20i
=f p(z)
v 2nTq
log
dx
1
(x (z-
2
/ p(z) log
0
1
(x 22)
(x- p1)
}
exp
dr
Vo
2
02
of
1
[1og o
loog of +
p(x)
(x
222)
dx -1)
2
02
1
E (x22
2p12 E (x) 2 -1)
log o
log o
2
o
1
of +
(y) - u2)
log 0/ log 07
2
o2
(9)
Since P: and Q, have the same o, i.c., 0 = 01 = VI
I. we can continue to simplify the
formula:
(MI - u2)2
KL(P.IQA.)
o2
(10)
a (2 xx)
2
1 - Te
15

---

Finally, we take the difference of both sides of the inequality
1
[T+
Tee
x0)
(L(P:+all0+a) (LL)))00-+)KKKK(10))))
2
1
a1+1
2
1 - āt
xx))
0+1
0++1
1 a2
(1I)
(r
xx))²
0111++
010011
1-01
x²))
1
0:04-1
at
Since o 1 3e and Be 1), 02 should be less than 1. As a result, we can get
- S..
(12)
According to Equation 12, we have
KT(Pt+1 < (TT(P:))))
(13)
where the equality happens only when P. = Qr.
B.2 Proof of Theorem 4.2
Theorem 4.2. Suppose that R(x) denotes a specific region of the sample x. Given a poisoned sample de we use Rx(2)) and Ro(2)) to represent clean region and poisoned region of the sample, respectively. We perform the noise-based perturbation on z for i steps and get the noisy samples rr, and then utilize DDPM to get denoised sample Tg- When comparing different regions of the original sample dy with their counterparts the detoxified sample Ty. we have
(7))
||
(14)
71
122
where nn and n2 represent the numbers of pixels in (x())) and Rx(x)), respectively.
Proof: First, we expand the inequality that we need to prove. Therefore, proving Equation 14 is equivalent to proving the following inequality:
Σ (2)(1) +1))²
Σ (Eg(1) =²(
(15)
T1
12
JEL-
According to Equation 5, we have
a 011, t)) + otz,
It-1
(xt
(16)
Vox
v1 ai
where of 1-8--1 8, and z NO I). 0 pre-trained neural network. Assuming that is 1-0, fully trained model that can generate possible noise based the input sample n, and the step I. Therefore, we have
(z,,,) zell [10(2,11, 1) 2-1l1b
(17)
where 2, is the real noise in step t.
Then, we analyze the pixels in the clean and poisoned regions, respectively. We denote a pixel in the clean region of the original sample as Xg. and a pixel in the poisoned region yo- Therefore, we have X; and y; to denote the pixels after our noise-based perturbation. And we have X, and Yo to denote the pixels after our DDPM-based denoising process. Since 0 has a strong ability, we can approximate the entire denoising process with the following:
1
X,
(X
V 07 2)
Va
(18)
1
8o
(y:
- V
T. z).
C
16

---

According to Equation 2 about the relationship between and 27, we have
||::0 xoll =
(x
v1
0 2) X
v1 Ot
= =1xx0 +
(Z) z)
X0
(19)
0
oi (Z= ||
Va
where 2 is the real noise and [2] (0(2)2, t)] According Equation 17, have
|
1 xoll-
(20)
Supposing that the training data used for the pre-trained diffusion model does not contain any knowledge about backdoor triggers. it is reasonable to assume that X belongs to the probability distribution of the training dataset of while yo does not. Therefore, have
|10(y,, t) 2,1l 110(x,,t) -
(21)
According Theorem 4.1, when the sample is fully noised, the sample equivalent Gaussian noise. Additionally, the sample generated by the unguided denoising process from Gaussian noise is completely independent of the original sample. Therefore, we have the equation about and y as follows:
R E-T
lim
xoll = lim [8eo - Yoll-
(22)
8-7
According to Equation 21 and Equation 22, we finally get the following equation:
||000 xoll
11 yoll
lim t-
xoll
lim t-T
18:o voll
(23)
where 7 E (O, Therefore, Equation 15 can be proved. Finally, we have
[R))))) Ra [[Ro(G))
(1
(24)
71
112
C Additional Experimental Results
C.1 Performance Comparison More Datasets
Table 4 and Table 5 show the defense performance of our approach and three defense baselines, i.c., Sancdifi [37], BDMAE [19] and ZIP [18], against ten state-of-the-art (SOTA) attack methods on the GTSRB and Timy-lmageNet dataset, respectively. The experimental results in the table illustrate that our approach outperforms existing SOTA methods on a variety of datasets, which shows the generalizability of our approach.
Table 4: Defense performance comparison on GTSRB, where Bold fonts and underlines denote the best and the second-best values.
Defense-)
No Defeuse
Sancdifi
BDMAE
ZIP
SampDetox (Ours)
Attack
CA(S) PA(%) ASR(S) CA(S)) PA(S) ASR(S) CA(S) PA(%) CA(()))(A()) PA(S) ASR(% ASR(%))CA()))
CA(S)) PA(%) ASR(%)
No Attack
98.12
BadNets
98.18
3.16
62.31
81.55
90.29
1.62
92.33
19
2.60
92.58
85.78
6.44
94.56
93.49
1.95
SIG
97.21
2.13
99.26
83.73
46.03
29.44
2.91
95.85
96.33
37.34
35.64
97.13
68.00
10.85
LC
97.52
2.33
99.12
82.48
53.30
3.86
93.42
77.36
1.22
94.29
74.89
1.99
94.25
77.03
1.20
TrojanNN
97.52
92.60
88.63
2.15
92.05
1.79
Dynamic
95.63
2.27
98.18
79.45
71.73
8.21
93.43
75.03
12.43
92.40
79.45
2.85
93.70
90.07
2.83
Blended
5.47
91.85
LF
97.91
2.33
96.88
81.86
51.17
15.98
92.44
6.57
80.35
93.08
87.82
5.18
94.91
88.64
2.31
WaNet
82.53
43.02
22.38
49.80
89.99
92.63
ISSBA
97.47
2.34
98.20
80.78
53.51
13.92
89.61
$3.29
21.65
91.09
85.25
1.05
95.13
86.83
1.30
BPP
96.27
2.32
99.17
78.90
51.28
15.40
90.53
24.19
53.40
89.46
84.83
7.46
94.85
84.99
5.64

---

Table 5: Defense performance comparison on Tiny-ImageNet, where Bold fonts and underlines denote the best and the second-best values.
Defense
No Defense
Sancdifi
BDMAE
ZIP
SampDetox (Ours)
Attack
CA(S) PA(S) ASR($)/CA($) PA(S) ASR(S) CA(S)) PA(S) CAA()))(A(S)) PA(S) ASR(%) PA(S) ASR(%)
No Attack
56.71
BadNets
56.61
0.51
99.96
40.94
49.17
3.88
51.56
50.84
4.45
$2.28
44.15
7.34
53.95
53.02
4.40
SIG
56.00
43.33
25.77
0.26
96.93
56.08
37.36
55.24
38.07
8.74
LC
56.09
0.51
99.71
41,89
12.81
5.03
53.23
36.63
3.33
$4.00
33.90
2.19
54.44
46.67
2.52
TrojaNNN
56.27
3.04
Dynamic
55.65
0.49
99.34
39.14
29.90
10.52
53.22
33.93
13.04
51.96
39.10
3.81
54.18
50.23
4.76
Blended
40.85
10.69
53.20
51.04
LF
56.82
0.51
99.45
40.92
9.93
18.86
51.75
0.42
82.24
53.07
47.14
6.03
54.11
48,91
4.22
WaNet
55.34
99.10
42.47
11.75
15.81
49.95
6,93
52.36
49.93
46.17
8.24
51.90
50,40
6.49
ISSBA
55.60
0.51
99.48
40.60
13.35
16.61
49.49
13.16
24.00
50.62
44.27
1.07
54.28
47.23
3.11
BPP
57.01
0.52
99.73
38.79
10.14
17.68
48.90
4.52
55.86
49.18
42.94
8.68
53.93
45.34
6,48
C.2
Performance More Models.
Since our approach does not require any interaction with the model, it should have defensive effects different models. Table 6 and Table show the defense performance of our approach the VGG-19 9 and ResNet34 models on CIFAR-10. GTSRB, and Tiny- Siny-ImaeNNet datasets, respectively. The experimental results show that approach has satisfactory defense effects on VGG-19 and ResNet34, which demonstrates that our approach is model-independent.
Table Defense performance of our approach on VGG-19 model.
Dataser-
CIFAR-10
GTSRB
Tiny-ImageNet
Attack +
CA(S)
PA(S) ASR(S)
CA(S)
PA(S) ASR(S)
CA(S)
PA(S)
ASR(S)
No Attack
91.02
94.93
45.54
BadNets
SIG
81.62
61.80
11.39
94.19
64.33
10.72
43.51
24.35
9.01
78.68
70.65
1.57
90.86
73.12
1.47
42.61
32.80
2.45
TrojanNN
90.67
86. 19
1.73
94.42
87.60
1.42
42.17
38.88
2.61
Dynamic
86.06
1.73
90.47
85.68
3.22
42.41
36.93
4.44
Blended
87.91
83.22
1.57
90.29
87.05
4.11
41.34
37.16
5.01
87.82
84.36
3.50
92.41
84,30
2.50
42.22
35.11
3.97
WaNet
86.40
85.76
5.45
90.10
86,80
5.53
39.88
36.43
6.47
ISSBA
88.58
83.39
2.07
92.37
83.21
1.23
42.33
34.18
2.86
BPP
88.31
81.59
6.63
91,38
80.88
6.01
41.79
31.76
6.40
Table 7: Defense performance of our approach on ResNet34 model.
Dataset->
CIFAR-10
GTSRB
Tiny- Net
Attack
CA(S))
PA(R)
ASR(%)
CA(S)
PA(SK)
ASR(3)
CA(S)
PA(S)
ASR(S))
No Attack
85,34
97.62
59.96
BadNets
84.27
83.11
2.67
93.44
56.77
54.72
SIG
78.66
59.10
10.94
96.02
66.15
10.16
59.31
39.78
8.04
1.C
92.03
TrojanNN
88.40
83.17
2.03
96.67
88 88.42
2.20
58.05
53.88
2.96
Dynamic
Blended
84.97
79.47
1.66
91.78
88.58
3.96
55.52
52.78
6.55
LF
85.37
81.03
3,86
94.67
84.47
3.00
56.83
$1.17
2.76
WaNet
84,26
83.33
5.94
92.66
88.19
5.63
54.54
52.84
8.26
ISSBA
85.73
80.90
1.94
95.09
83.60
1.06
58.28
49.53
3.79
BPP
85.65
79.19
6.40
93.86
55.96
48.35
4.72
C.3
Comparison with Sample Transformation Methods
In this section, we compare our approach with existing sample transformation methods on CIFAR-10, including Noising, ShrinkPad, and Blur [49]. Noising attempts to destroy the backdoor trigger structure by adding large amount of noise to the sample without considering the inference accuracy. ShrinkPad shrinks the samples and then randomly pads samples back to their original size. Blur utilizes Gaussian kernel to pool the samples to destroy backdoor trigger patterns. The experimental results shown in Table 8.
From the table, we can find that transforming the samples can break some the backdoor triggers. However, the transformation will cause great damage to the quality of the samples, resulting in sig-
18

---

Table 8: Defense performance comparison with sample transformation methods.
Defense
No Defense
Noising
ShrinkPad
Blur
SampDetox (Ours)
Artack 4
CA(S))
PA(S) ASR(S)) ASR(S))CA(() PA(S) (SR(S) CA(S) PA(S) ASR(S)
( (A(S) PA(S)) ASR( CA(S)) PA(S) ASR(%)
Attack
93.84
BadNets
92.00
10.18
99.97
68.43
61.79
5.42
71.63
66.06
8.42
64.02
49.44
18.15
89,57
90.15
2.11
SIG
84.94
98.50
13.97
91.34
59.75
8.35
55.74
LC
84.34
10.26
99.06
61.26
55.91
4.57
63.71
58.07
6.42
56. 10
50,02
12.52
80.72
74.36
1.55
TrojanNN
93.20
11.07
99.03
69.20
59.40
12.16
73,40
69.17
4.33
64.67
50.90
14,30
92.78
89,95
1.86
Dynamic
91.09
10.02
98.19
67.45
59.94
1.67
72.76
64.31
1.67
63.80
55.76
2.34
88.52
88.62
1.45
Blended
93.85
10.93
99,51
71.14
58.87
25.78
72.93
68.52
2.54
68.61
61.76
7.69
90.23
86.65
1.96
LF
93.63
11.13
99.48
73.43
66.72
4.31
73.67
69.35
5.33
66.33
60.98
2.37
90.01
87.40
3.02
WaNet
91.43
10.27
91.05
68.51
62.95
5.53
72.52
65.23
7.52
65.35
57.25
5.67
89.34
88.92
5.59
ISSBA
93.57
11.38
95.96
68.96
62.88
7.82
73.22
65.40
1.67
65
55.55
2.02
90.74
86.51
1.60
BPP
91.38
9.46
98.40
67,36
5182
10.65
72.25
55.43
7.14
65.86
56,90
8.34
90.59
84,83
6.15
nificant reduction in Clean sample Accuracy (CA) and Poisoned sample Accuracy (PA). Specifically, though Blur has lower ASR than SampDetox when defending against the SIG and LF attacks, our approach has better CA and PA. In addition, our methods have the best defensive performance. The experimental results show that our approach can successfully defend against various attacks while preserving the original semantic information of samples to the greatest extent.
C.4 Performance using Different Diffusion Models
In this section, deploy approach using different pre-trained diffusion models to defend against backdoor attacks on CIFAR- and compare their defense performance. Specifically, we choose three diffusion models with different parameters: 1) pre-trained on CIFAR-10, which is the same setup as our main experiments, 2) pre-trained on ImageNet, and 3) pre-trained on ImageNet and fine-turned on CIFAR- 10. Notably, pre- pre-trained model, and we excluded these data during the experiments. The experimental results are shown in Table 9.
used 5% clean samples in the test dataset of CIFAR- 10 to fine-ture the
From the table, we can observe that SampDetox with different diffusion models has satisfied defensive performance on Clean sample Accuracy (CA) and Attack Success Rate (ASR). For CA, since clean samples do not have any trigger patterns, only a little noise will be added them in Stage 2 of our approach. Therefore, all these three diffusion models can accurately denoise samples. For ASR, the results are generally not related to the denoising process. For Poisoned sample Accuracy (PA), when defending against visible attacks (i.e., BadNets, SIG, LC, TrojanNN, and Dynamic), the model pre-traine pre-trained on ImageNet has much lower accuracy, meaning that the model pre- are-trained on outside distribution datasets not accurately denoise samples, especially when much noise is added. The fine-tuning model achieves satisfactory defensive performance against different attacks. Therefore, when we need detoxify samples without pre-trained models, we fine-tune diffusion model to deploy our defense approach.
Table 9: Defense performance of our approach using different diffusion models.
Diffusion-
Pre-trained (CIFAR-10)
Pre-trained (ImageNet)
Fine-tuning (CIFAR-10)
Attack J
CA%
PAS:
ASRS
CAS
PAS
ASRS
CA%
PA
ASRS
Attack
93.39
93.81
BadNets
89.57
90.15
2.11
89.23
85.02
2.26
89.59
90.77
1.98
SIG
83.27
56.04
83.30
64.68
L.C
80.72
74.36
1.55
80.05
64,83
1.80
80.19
73.62
1.87
TrojanNN
92.78
89.95
92.17
82.
1.69
92.66
90.21
1.58
Dynamic
88.52
88.62
1.45
87.97
80.44
1.80
88.46
88,93
1.01
Blended
90.23
86.65
1.96
89.42
83.64
1.54
89.81
85.92
1.64
LF
90.01
87.40
3.02
89.38
86.94
2.65
89.72
86.83
3.53
WaNet
89.34
88.99 92
5.59
88.94
87.88
5.17
89.43
88.02
6.16
ISSBA
90.74
86.51
1.6
90.25
85.31
1.57
90.29
85,53
1.70
BPP
90.59
84.83
6.15
90.96
84.01
5.95
91.24
82.12
5.34
Additionally, considering the case where pre-training datasets for diffusion models are out-of- distribution from their target classification tasks, we conducted experiments to evaluate the perfor- mance of SampDetox on a subset of the MS-Celeb- IM [50] dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with ratio of 8 2 and adjusted the shape of the samples to
19

---

224 224. We adopted BadNets [22] and WaNet [28] as attack methods, representing a visible and an invisible backdoor attack, respectively. We considered four pre-trained diffusion models: i) pre-trained model (i.e., Model 1) on the subset of MS-Celeb-IM, ii) pre-trained model (i.e., Model 2) on another subset of MS-Celeb- IM, iii) pre-trained model (i.e., Model 3) on ImageNet, and iv) Model 4, which is fine-tuned from Model 3 using subset of MS-Celeb-IM.
Table 10 shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 pre-trained on ImageNet rather than MS-Celeb-IM. resulting in the diffusion model failing to effectively denoise LIS-Celeb-IM 1M samples. However, we can find significant improvements using subset of MS-Celed IM. In other words, SampDetox can defend against attacks using dittr---distiittiionn diffusion models with proper fine-tuning.
the CA and PA of Model 4 since Model 4 is fine-tuned
Table 10: Defense performance of our approach using
diffusion models.
Attack
BadNets
Settingl
CA(S)
PA(S)
ASR(%)
CA(S)
PA(S)
ASR(%)
No Defense
94.38
1.27
99.32
94.61
1.96
97.05
Model
91
1.36
91.12
1.29
Model
90.27
89.73
1.31
89.92
90.05
1.22
52.06
1.57
72.35
62.26
1,25
Model 4
88.56
89.27
1.47
89.43
89.32
1.30
D Additional Discussions
D.1 Performance on High-frequency Classification Task
To evaluate the defense performance of SampDetox on classification tasks that rely on benign high- frequency details, we conducted experiments on the DTD (Describable Textures Dataset) dataset [5 l]. where we adopted a visible backdoor attack, i.c., BadNets [22], and invisible backdoor attack, i.c. WaNet [28], respectively. Table 11 compares SampDetox with six baselines. From this table, we can find that though the CA of SampDetox a little smaller than the CA of "No Defense", SampDetox can achieve the best defense performance on CA, PA, and ASR compared with all baselines, which shows the applicability and superiority SampDetox.
Table 11: Defense performance of our approach on the high-frequency classification task.
Attack+
BadNets
WaNet
Defense
CA(%)
PA(%)
ASR(%)
CA($)
PA(S)
ASR(%)
No Defense
65.42
2.05
90.56
64.72
2.17
88.56
Sancdifi
48.98
2.84
BDMAE
56.25
56.43
1,86
55.57
19.17
47.22
ZIP
43.17
40.95
3.32
44.87
40.56
3,31
Noising
37.55
35.87
5.26
38.46
37.08
3.27
ShrinkPad
41.27
39.36
8.92
43.19
42.48
Blur
33.17
30.92
12.05
33.49
29.50
3.67
SampDetex (Ours)
57.46
57.12
1.73
58.33
58.12
2.21
Table 12: Defense performance of SampDetox using DDPM and DDIM.
SampDetox+DDDM.
SampDetox+DDIN
Attack
CA(%)
PA(%)
ASR(%)
CA(%)
PA(%)
ASR(%)
BadNets
89.57
90.15
2.11
89.49
90.13
2.12
11.03
10.98
LC
80.72
74.36
1.55
80.62
74.22
1.53
TrojanNN
1.69
Dynamic
88.52
88.62
1.45
88.52
88.72
1.42
Blended
90.15
LF
90.01
87.40
3.02
90.09
87.61
3.10
WaNet
89.34
88.92
5,59
89.48
88.82
5,54
ISSBA
90.74
86.51
1.60
90.76
86.65
1,55
BPP
90.59
84.83
6.15
90.42
84.91
6.17
20

---

D.2 Performance using DDIM
Since DDIM [21] was proposed to accelerate the speed of DDPM without affecting the image generation quality, the defense performance of "SampDerox++DDIM" "SampDetox+DDPM
comparable to that of To demonstrate the performance of "SampDetox+DDIIM" we conducted new experiments on CIFAR- 10 against ten SOTA backdoor attacks. The experimental results are shown in Table 12. From the table, we can find that the defense performance of "SSmmDetox+DDDIN is almost the same as that of "SampDetox+DDPM"" which means that "SampDetox+DDIM" can match the defense efficacy of "SammDetto+DDDM""
D.3 Training Sample Detoxification
Before training deep models, we can use SampDetox to conduct perturbation -basd training sample detoxification, protecting them against backdoor injection. Table 13 compares the defense perfor- mance of the trained models by SampDetox. Here, the second column shows the defense performance without using SampDetox. The third and fourth columns present the defense performance of models trained by detoxified samples. Note that for the third column, we did not change the labels of detoxified samples. In other words, the poisoned samples still keep their dirty labels (denoted by D.L.). For the fourth column, we tried to correct the dirty labels based on the inference results of the models generated by the third column. Based on the detoxified samples and corrected clean labels (denoted by C. the fourth column shows the defense performance of our approach. Note that for this table, we did not apply sample detoxification test samples. From this table, SampDetox can effectively defend against various backdoor attacks. Although "SampDetox+D.LL and "SampDetox+C..
observe that
have similar ASR, the CA of "SSmmDDetx++..L." can be notably improved based on the corrected labels of poisoned samples.
Table 13: Defense performance of SampDetox in the training phase.
Defense
No defense
SampDetnx++D...
SampDetox-+CI.
Attack 4
CA(S)
ASR(S)
CA(%)
ASR(S)
CA(S)
ASR(E)
BadNets
91.95
99.97
SIG
85.03
98.39
87.33
1.78
91.53
1.69
84.64
TrjjnNN
93.37
99.99
89.46
2.01
93.10
1.84
Dynamic
91.05
12
19
1.64
90.98
1.89
Blended
93.41
99.96
87.67
1.45
91.59
1.46
93.11
99.31
88,59
1.63
90.77
1.45
WaNet
91.81
90.77
88.18
1.77
90.85
1.85
ISSBA
93.09
96.36
89.79
1.55
93.31
1.60
BPP
91.39
99.18
89.52
1.47
93.21
1,38
D.4 Performance against Adaptive Attacks
Assume that attackers have all the prior knowledge of our proposed approach to design an adaptive attack to bypass our defense approach. Since SampDetox is based on observations about the correlation between the visibility of triggers and the robustness of poisoned samples, the attackers can try two types of adaptive attacks follows.
Adaptive attack 1. The attacker uses attack with both low visibility and high robustness since attacks with high visibility and low robustness are easily defended. To guarantee low visibility, the attacker adopts an invisible backdoor attack, i.c., WaNet. To ensure high robustness, the attacker adaptively adds noises to samples during the training phase to improve the robustness of triggers.
2) Adaptive attack 2. The attacker does not consider visibility issues and tries to maximize robustness to bypass our defense. In this case, the attacker adopts a visible backdoor attack, i.e., BadNets, and adds robust noise during the training phase to improve robustness.
We conducted experiments to defend against the two adaptive attacks on the CIFAR-10 dataset. Table 14 shows the results of the experiments, where the first column denotes the amount of noise added during training to improve the robustness. From the table, we find that our approach can defend against adaptive attacks.
21

---

Table 14: Defense performance of SampDetox against adaptive attacks.
Robust Noise
Attack
(No Defense)
Attack (SampDetox)
Attack 2 (No Defense)
Attack 2 (SampDetox)
CA(S)
PA(S)
ASR(%)
CA(S)
PA(S)
ASR(S)
CA($)
PA(S)
ASR(%)
CA(S))
PA(*)
ASR(S)
92.00
99.97
89.57
0.10
91.62
13.14
87.36
89,45
89.07
5.22
92.16
10.07
99.12
89.62
90.37
2.25
21.18
52.53
3.13
97.12
89.58
89,98
0.30
92.26
89.61
1,36
90.83
90.72
1.13
91.62
12.32
82.12
90.02
90.09
2.33
D.5
Limitation
Although our approach aims defend against backdoor attacks with any kind of trigger, there are still certain triggers that SampDetox cannot remove. Physical attack [52, 53] is backdoor attack with visible but natural triggers, usually physical objects. Since our approach identifies the positions of trigger patterns based on Theorem 4.2 by calculating the similarity between the denoised and original samples, the effectiveness of our approach depends on the pre-trained diffusion model. When the triggers are visible, are part of the sample classification characteristics, and belong to the knowledge contained in the pre-traue treitraind diffusion model, our defense approach cannot destroy the trigger patterns and defend against these attacks.
Table 15: Defense performance of SampDetox against semantics attacks.
Attack
CIFAR-10
GTSRB
Tiny-ImageNet
Defense
CA(S)
PA(S)
ASR(%)
CA(S)
PA(%)
ASR(%)
CA(%)
PA(%)
ASR(%)
Refool
87.46
88.03
1.72
93.79
91.43
2.16
54.52
51.11
4.10
DSFT
86.71
86.55
1.96
90.45
89.20
2.05
50.68
49.85
4.15
To investigate the limitation of SampDetox, we conducted experiments with three semantic backdoor attacks, i.c., Refool [54], DSFT [55] and CBA [56]. Table 15 presents the defense performance of SampDetox against Refool and DSFT. From the table, we can find that SampDetox can effectively defend against such backdoor attacks. This is because the semantic features adopted by Refool and DSFT are neither part of the original classified features nor are their distributions the same as the distribution of the training dataset of the pre- traindrdd diffusion model. Therefore, the triggers based on such semantic features will not be restored by the denoising phase of SampDetox.
Additionally, to explain why SampDetox can defend against these two attacks, we conducted ex- periments calculate the robustness of their poisoned samples, respectively. For each attack, we randomly selected 50 poisoned samples on CIFAR-10 and calculated the robustness value of each sample following our robustness definition in Section 3.1. For Refool, the robustness of its poisoned samples has mean value of 0.0759 and standard deviation of 0.011. For DFST, the robustness of its poisoned samples has a mean value of 0.137 and standard deviation of 0.019. From the experimental results, we find that the robustness of the poisoned samples is low (ones with robustness < 0.18, according to Figure 2). Therefore, when SampDetox adds enough noise to these samples, even if trigger patterns partially restored by diffusion models, backdoors will not be successfully triggered.
However, SampDetox fails to defend against CBA. Since CBA concatenates two natural images as backdoor triggers, is considered the backdoor attack type that SampDetox fails to defend. The experimental results CIFAR-10 are follows: Without defense, CA, PA, and ASR were 87.52% 10. 16%, and 90.17%, respectively. However, when using SampDetox, CA, PA, and ASR were 86.81 %, 38.45%, and 37. 72.2%, respectively. The experimental results show that SampDetox has certain effect in defending against CBA but fails to minimize ASR.
22

---

E
More Examples of Poisoned Samples
Figure shows examples of the poisoned samples with their visibility v and robustness 7
0.102/0.182
0.006/0.063
0.010/0.069
0.007/0.053
0.023/0.071
0.099
0.108/0.105
0.187/0.418
0,043/9071
0.013/0.058
0.057/0.067
0.098/0.081
0.006/0.061
0.131/0.233
0.010/0.062
0.191/0.398
0.149/0.406
0.053/0.083
0.006.0005
0,047/00068
0.044/0.073
0.096/0.071
0.174/0.226
0.112/0.121
0.112/0.113
0.047/0.078
0.169/0.430
0.006/0.059
0.015/0.048
0.167/0.364
Figure Examples of poisoned samples and their v/n..
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations in Section D.5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification:
The paper provides the proof' of theoretical results in Section B.1 and Sec- tion B.2, respectively.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses the information needed to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution primarily new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (c.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
25

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The paper provides code in supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
cablic/guudees/CodeSuubbaa ios. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: ctppp.cccppulic//eegee gwwiiiiggg///////eee /Coddeeumdmamaa iommissiionn sgionooiccy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides the experimental settings in Section 5.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper does not involve error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
26

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper provides the information about compute resources in Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https /ttpss//nnuuu../eeeeeeeee wccppubbcc/.thicsssueeeeee.
Answer: [Yes]
Justification: Ethics.
The research conducted in the paper conforms with the NeurIPS Code of
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper discusses the potential societal impacts of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper uses popular datasets and models. We cites the original paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "ctmYLLkskF", "reviewer": "Reviewer_aKFd", "summary": "This paper proposes SampDetox, a novel black-box backdoor defense method using perturbation-based sample detoxification. The key contributions are:\n\n1. A preliminary study revealing the correlation between trigger visibility and poisoned sample robustness.\n2. A two-stage defense approach combining lightweight global noise and targeted local noise to destroy both visible and invisible backdoor triggers.\n3. Use of diffusion models to denoise and restore samples after perturbation.\n4. Comprehensive experiments demonstrating effectiveness against various state-of-the-art backdoor attacks.", "questions": "1. The paper suggests optimal values for t1 and t2 based on your experiments. How sensitive is the performance of SampDetox to these parameters across different datasets or attack types? Is there a way to automatically determine optimal values for a given scenario?\n2. How does the pixel-wise SSIM calculation in Stage 2 scale with image size? Is there a more efficient way to locate high-visibility triggers without compromising effectiveness? Moreover, how does the method perform on very large images or high-resolution datasets? Is there a practical upper limit on image size due to computational constraints?", "limitations": "The authors have adequately addressed the limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novel approach combining noise perturbation and diffusion models for backdoor defense\n2. Comprehensive evaluation against 10 different backdoor attacks\n3. Theoretical analysis supporting the approach (Theorems 4.1 and 4.2)\n4. Does not require access to model parameters or training data, making it applicable to MLaaS scenarios (black-box models)\n5. Compared to previous work, it addresses both visible and invisible backdoor triggers", "weaknesses": "1. The effectiveness of SampDetox heavily depends on the correlation between trigger visibility and robustness, but this relationship may not always hold for more sophisticated attacks.\n2. The two-stage approach with fixed noise levels (t1 and t2) may be too rigid to adapt to diverse trigger patterns, potentially missing subtle backdoors or over-perturbing clean regions.\n3. The approach may remove benign high-frequency details from images, potentially affecting classification accuracy for tasks that rely on such fine-grained features.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**AW1:**\nIndeed, the effectiveness of SampDetox depends on our observed correlation. However, this does not mean that our approach cannot prevent more sophisticated attacks. This is because our observation has good generalization ability, since our observation is based on ten SOTA backdoor attacks covering most mainstream backdoor attack types. In other words, it can reflect the commonality of most attacks.\nIn addition, in Appendix D.2, we tried to design adaptive attacks with high robustness and low visibility that aim to break the observed correlation. However, the designed adaptive attacks cannot achieve high enough robustness while keeping low visibility, which can be successfully defended by SampDetox. To the best of our knowledge, it is still challenging to propose an effective attack with high robustness and low visibility, which is worthy of further study.\n\n**AW2:**\nSorry for the confusion. Although the value of $\\overline{t}_2$ is fixed, the noise levels imposed on sample pixels are not fixed. Please note the hyperparameter $\\overline{t}_2$ in Stage 2 represents the maximum added noise level of each pixel rather than its actual added noise level. Based on both the original sample and its denoised version obtained in Stage 1, SampDetox calculates different noise levels for different pixels in Stage 2 (Line 9 of Algorithm 1), which aims to adapt to various trigger patterns. From Tables 1, 3, 4, 5, and 6, we can find the superiority of SampDetox in defending against various attacks involving different datasets and models.\n\n**AW3:** \nIndeed, our approach may remove benign high-frequency. However, this does not eclipse the effectiveness of our approach against backdoor attacks. To evaluate the defense performance of SampDetox on classification tasks that rely on benign high-frequency details, we conducted new experiments on the DTD (Describable Textures Dataset) dataset, where we adopted a visible backdoor attack, i.e., BadNets, and an invisible backdoor attack, i.e., WaNet, respectively. The following table compares SampDetox with six baselines. From this table, we can find that though the CA of SampDetox a little smaller than the CA of ``No Defense’’, SampDetox can achieve the best defense performance on CA, PA, and ASR compared with all baselines, which shows the applicability and superiority of SampDetox.\n\n\n| Defense | BadNets | WaNet |\n| :--------------: | :----------: | :-------------------: |\n| | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) |\n| No Defense | 65.42 / 2.05 / 90.56 | 64.72 / 2.17 / 88.56 |\n| Sancdifi | 48.98 / 51.23 / 2.84 | 49.64 / 28.06 / 15.12 |\n| BDMAE | 56.25 / 56.43 / 1.86 | 55.57 / 19.17 / 47.22 |\n| ZIP | 43.17 / 40.95 / 3.32 | 44.87 / 40.56 / 3.31 |\n| Noising | 37.55 / 35.87 / 5.26 | 38.46 / 37.08 / 3.27 |\n| ShrinkPad | 41.27 / 39.36 / 8.92 | 43.19 / 42.48 / 2.92 |\n| Blur | 33.17 / 30.92 / 12.05 | 33.49 / 29.50 / 3.67 |\n| SampDetox (Ours) | **57.46** / **57.12** / **1.73** | **58.33** / **58.12** / **2.21** | \n\n**AQ1:**\nSorry for your confusion. We use the same values of $\\overline{t}_1$ and $\\overline{t}_2$ throughout the paper. In other words, utilizing fixed $\\overline{t}_1=20$ and $\\overline{t}_2=120$, SampDetox can defend against various types of attacks involving different datasets and models, as shown in Tables 1, 3, 4, 5, and 6.\nIn general, given a new scenario, there is no need to change the value of $\\overline{t}_1$ and $\\overline{t}_2$ to achieve satisfactory defense performance.\n\n**AQ2:**\nSorry for your confusion. For two $n * n$ images A and B, we calculate their Pixel-SSIM with a block size of $t*t$ as follows: \n$\\operatorname{Pixel-SSIM}(x, y)=\\frac{\\left(2 \\mu_x \\mu_y+c_1\\right)\\left(2 \\sigma_{x y}+c_2\\right)}{\\left(\\mu_x^2+\\mu_y^2+c_1\\right)\\left(\\sigma_x^2+\\sigma_y^2+c_2\\right)}$, where $x$ and $y$ are the same positional pixels of images A and B, respectively. $\\mu_x$ and $\\sigma_x$ are the mean and variance of the block centered on $x$, respectively. $\\sigma_{xy}$ is the covariance between $block_x$ and $block_{y}$, and $c_1$ and $c_2$ are constants. According to the above formula, the time complexity for the pixel pair is $O(t^2)$. Thus the time complexity of whole Pixel-SSIM is $O(t^2 n^2)$. Considering that $t$ is constant in practice, the computational cost of Pixel-SSIM is determined by the number of pixels. In addition, we conducted new experiments to calculate the overhead of Pixel-SSIM. The experimental results show that the Pixel-SSIM overhead only accounts for less than 1% of the overall overhead of SampDetox. In other words, such overhead can be neglected in practice. \n\nWe conducted new experiments on the ImageNette dataset (a subset of 10 classes from ImageNet) containing larger images with a resolution of $256*256$. From the following table, we can find that SampDetox is still effective in defending against attacks on large images. \n\nThe practical upper limit of image size depends on the denoising capability of diffusion models rather than the computational constraints. When the size of images is too large, it is difficult to train an effective diffusion model to denoise such images. Nevertheless, the experimental results show that SampDetox is still effective in common image sizes (i.e., 256*256).\n\n| Attack Methods | No Defense | SampDetox |\n| :------------: | :--------------------: | :--------------------: |\n| | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) |\n| BadNets | 83.75 / 10.83 / 95.09 | 79.59 / 79.74 / 1.91 |\n| SIG | 77.45 / 10.09 / 96.26 | 75.15 / 74.85 / 6.43 |\n| LC | 76.38 / 10.46 / 90.24 | 73.57 / 73.13 / 1.92 |\n| TrojanNN | 84.61 / 10.91 / 98.86 | 83.04 / 83.11 / 1.66 |\n| Dynamic | 83.73 / 11.58 / 97.02 | 81.16 / 80.63 / 1.64 |\n| Blended | 84.39 / 10.66 / 98.22 | 83.02 / 82.49 / 1.08 |\n| LF | 85.19 / 11.39 / 92.65 | 82.15 / 81.95 / 2.51 |\n| WaNet | 83.34 / 9.49 / 87.59 | 82.16 / 82.22 / 4.23 |\n| ISSBA | 85.26 / 11.49 / 90.37 | 82.88 / 82.45 / 2.33 |\n| BPP | 83.04 / 10.30 / 91.11 | 82.23 / 81.93 / 3.55 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ukbVjIvM3U", "reviewer": "Reviewer_ok4k", "summary": "This paper introduces a novel black-box backdoor purification technique consisting of two stages. In the first stage, it eliminates global perturbation by adding lightweight noise to the inputs and then leverages DDPM for denoising. Subsequently, to address visible and robust triggers, it identifies the local trigger regions, applies intense noise, and then performs diffusion denoising again.\nThe paper provides theoretical analysis to validate the effectiveness of its detoxification process.\nIn the evaluation, extensive experiments are conducted on multiple datasets, model architectures, and attacks, showing that the proposed method outperforms existing baselines. An ablation study justifies the design choices and hyper-parameters, and the paper discusses adaptive attack scenarios.", "questions": "The paper presents an innovative concept for black-box input detoxification, highlighting noteworthy observations and techniques. However, I have several concerns I hope the authors can address:\n\n(1) The evaluation includes 10 known backdoor attacks, which are not particularly robust. More complex backdoors utilize semantic features, for instance, [1] uses natural reflection as a trigger, and [2] employs style transferring. It would be beneficial to see evaluations against these more sophisticated backdoors.\n\n(2) The overhead is a concern. Although Section 5.4 suggests using DDIM to reduce the overhead, it is unclear if this significantly affects the detoxification performance. Additional results on this would be helpful.\n\n(3) The adaptive attacks discussed in Section D.2 seem relatively weak, just introducing some noise during training. A stronger attack might use natural features as triggers to enhance the robustness of BadNets. For instance, [3] concatenates two natural images as the backdoor trigger. It is suggested that the authors discuss this and potentially include stronger adaptive attacks.\n\nReference:\n\n[1] Liu, Yunfei, et al. Reflection backdoor: A natural backdoor attack on deep neural networks. ECCV'20.\n\n[2] Cheng, Siyuan, et al. Deep feature space trojan attack of neural networks by controlled detoxification. AAAI'21\n\n[3] Lin, Junyu, et al. Composite backdoor attack for deep neural network by mixing existing benign features. CCS'20.", "limitations": "Please refer to the question part.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Observes interesting links between backdoor visibility and robustness.\n2. Proposes a novel technique based on these observations.\n3. Provides theoretical proof of efficacy.\n4. Demonstrates support for the technique's effectiveness through extensive experiments.", "weaknesses": "1. Lacks evaluation of some semantic backdoors.\n2. Concerns about computational overhead for deployment.\n3. Evaluations of adaptive attacks are relatively weak.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**AQ1:**\nThanks for your suggestion.\nSince the semantic features adopted by Reflect [1] and DSFT [2] are neither part of the original classified features nor whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, the triggers based on such semantic features will not be restored by the denoising phase of SampDetox. Consequently, these triggers can be destroyed by SampDetox to achieve a defensive effect. To demonstrate the defense performance of SampDetox against Reflect [1] and DSFT [2], we conducted new experiments on datasets CIFAR-10, GTSRB, and Tiny-ImageNet, respectively. From the following table, we can find that SampDetox effectively defends against such backdoor attacks.\n\n| Attack Methods | &emsp;&emsp;CIFAR-10 | &emsp;&emsp;GTSRB | &emsp;Tiny-ImageNet |\n| :----------------: | :-----------------------: | :-----------------------: | :-----------------------: |\n| | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) |\n| Reflect | 87.46 / 88.03 / 1.72 | 93.79 / 91.43 / 2.16 | 54.52 / 51.11 / 4.10 |\n| DSFT | 86.71 / 86.55 / 1.96 | 90.45 / 89.20 / 2.05 | 50.68 / 49.85 / 4.15 |\n\n\n**AQ2:**\nThanks for your suggestion. Since DDIM was proposed to accelerate the speed of DDPM without affecting the image generation quality, the defense performance of **SampDetox+DDIM** is comparable to that of **SampDetox+DDPM**. To demonstrate the performance of **SampDetox+DDIM**, we conducted new experiments on CIFAR-10 against ten SOTA backdoor attacks. From the following table, we can find that the defense performance of **SampDetox+DDIM** is almost the same as that of **SampDetox+DDPM**. Therefore, the utilization of DDIM to reduce the overhead will not deteriorate the detoxification performance.\n\n| Attack Methods | SampDetox + DDPM | SampDetox + DDIM |\n| :----------------: | :-----------------------: | :------------------------: |\n| | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) |\n| BadNets | 89.57 / 90.15 / 2.11 | 89.49 / 90.13 / 2.12 |\n| SIG | 83.71 / 65.06 / 11.03 | 83.82 / 65.13 / 10.98 |\n| LC | 80.72 / 74.36 / 1.55 | 80.62 / 74.22 / 1.53 |\n| TrojanNN | 92.78 / 89.95 / 1.86 | 92.83 / 89.87 / 1.69 |\n| Dynamic | 88.52 / 88.62 / 1.45 | 88.52 / 88.72 / 1.42 |\n| Blended | 90.23 / 86.65 / 1.96 | 90.15 / 86.54 / 2.02 |\n| LF| 90.01 / 87.40 / 3.02 | 90.09 / 87.61 / 3.10 |\n| WaNet | 89.34 / 88.92 / 5.59 | 89.48 / 88.82 / 5.54 |\n| ISSBA | 90.74 / 86.51 / 1.60 | 90.76 / 86.65 / 1.55 |\n| BPP | 90.59 / 84.83 / 6.15 | 90.42 / 84.91 / 6.17 |\n\n**AQ3:** Thanks for your suggestion.\nAs discussed in Appendix D.3 (Limitations of SampDetox), when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.\nSince Composite Backdoor Attack (CBA) [3] concatenates two natural images as backdoor triggers, it is considered the backdoor attack type that SampDetox fails to defend. \n\nWe conducted new experiments to evaluate the performance of SampDetox against the attack CBA [3] on CIFAR-10. Without defense, CA, PA, and ASR were 87.52%, 10.16%, and 90.17%, respectively. But when using SampDetox, CA, PA, and ASR were 86.81%, 38.45%, and 37.28%, respectively. The experimental results show that SampDetox has a certain effect to defend against CBA but fails to minimize the ASR.\n\nThanks again for your suggestion. The results of new experiments in **AQ1** and **AQ3** strongly support the discussion of the limitations in Section D.3. We will add the experiments in the final version of the paper.\n\n\n**Reference**:\n\n[1] Liu, Yunfei, et al. Reflection backdoor: A natural backdoor attack on deep neural networks. ECCV'20.\n\n[2] Cheng, Siyuan, et al. Deep feature space trojan attack of neural networks by controlled detoxification. AAAI'21.\n\n[3] Lin, Junyu, et al. Composite backdoor attack for deep neural network by mixing existing benign features. CCS'20.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VtOnCpw0d3", "reviewer": "Reviewer_gGGP", "summary": "This paper proposes a two-stage black-box method to defend against the backdoor in DNN. The first stage mitigates the backdoor triggers with low  visibility and the second stage mitigates the trigger with higher robustness.", "questions": "Would using another pre-trained model have an influence on your work?", "limitations": "The authors discussed about the main concern I have and acknowledged the limitation for semantic backdoor triggers.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The two-stage design is novel and effective.\n2. The evaluation is comprehensive and convincing.\n3. The paper is easy to read and follow.", "weaknesses": "1. One major concern about this work is that I think this work may not be effective against high-level semantic triggers. For example [1] adds the digital and physical triggers to the image, and make sure they can be preserved after gaussian blur or print and recapture. This work may not be effective against such triggers. Similar works can be found in [2,3]. However, I found that the authors have already mentioned  and acknowledged this in the paper. So I think this is not a big issue.\n2. The use of diffusion model is not well introduced. It may not be easy for the readers to understand this part if not familiar with the diffusion model. Also, in appendix A.2, the diffusion model from previous work is used, but the authors did not provide the details of the diffusion model. I think it would be better if the authors can provide more details about the diffusion model. Also, would it have any impact if this used diffusion model is replaced by other pre-trained models?\n\n\n[1] Versatile Backdoor Attack with Visible, Semantic, Sample-specific and Compatible Triggers\n\n[2] Backdoor Attacks Against Deep Learning Systems in the Physical World\n\n[3] Dangerous Cloak: Natural Backdoor Attacks on Object Detector in Real Physical World", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**AW1:**\nIndeed. In Appendix D.3 (Limitations of SampDetox), we have discussed a type of backdoor attack that SampDetox cannot defend against. Specifically, when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.\n\n**AW2:**\nThanks for your suggestion. We will detail the use of diffusion models in the final version.\n\nIn Appendix A.2, we have mentioned that the diffusion models used the same settings as the ones presented in [17]. In this experiment, the number of total diffusion steps was set to 1000 for all the datasets in the experiments, the noise_schedule was set as ``cosine’’, and the learning rate was set to 1e-4. We will provide such details in the final version.\n\nIn Appendix C.4, we have evaluated the defense performance of SampDetox using different diffusion models. Specifically, for CIFAR-10 classification tasks, we considered three different pre-trained diffusion models: i) a pre-trained model on CIFAR-10, ii) a pre-trained model on ImageNet, and iii) a model pre-trained on ImageNet and then fine-tuned on CIFAR-10. Table 8 in our paper demonstrates the generalization ability of SampDetox to such pre-trained diffusion models. \n\nBesides, we conducted new experiments to evaluate the performance of SampDetox considering the case where pre-training datasets for diffusion models are out-of-distribution from their target classification tasks. The detailed experimental setups are as follows:\n* We conducted the training on a subset of the MS-Celeb-1M dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with a ratio of 8:2 and adjusted the shape of the samples to 224*224.\n* We adopted BadNets and WaNet as attack methods, representing a visible and an invisible backdoor attack, respectively.  \n* We considered four pre-trained diffusion models: i) a pre-trained model (i.e., **Model 1**) on one subset of MS-Celeb-1M, ii) a pre-trained model (i.e., **Model 2**) on another subset of MS-Celeb-1M, iii) a pre-trained model (i.e., **Model 3**) on ImageNet, and iv) **Model 4**, which is fine-tuned from Model 3 using a subset of MS-Celeb-1M.\n\nThe following table shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 is pre-trained on ImageNet rather than MS-Celeb-1M, resulting in the diffusion model failing to effectively denoise MS-Celeb-1M samples. However, we can find significant improvements in the CA and PA of Model 4 since Model 4 is fine-tuned using a subset of MS-Celeb-1M. In other words, SampDetox can defend against attacks using out-of-distribution diffusion models with proper fine-tuning.​\n​\n| Diffusion Models | &emsp;&emsp;BadNets | &emsp;&emsp;WaNet |\n| :----------------:| :------------------------: | :------------------------: |\n| | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) |\n| No Defense | 94.38 / 1.27 / 99.32 | 94.61 / 1.96 / 97.05 |\n| Model 1 | 92.06 / 91.65 / 1.36 | 91.83 / 91.12 / 1.29 |\n| Model 2 | 90.27 / 89.73 / 1.31 | 89.92 / 90.05 / 1.22 |\n| Model 3 | 55.47 / 52.06 / 1.57 | 72.35 / 62.26 / 1.25 |\n| Model 4 | 88.56 / 89.27 / 1.47 | 89.43 / 89.32 / 1.30 |\n\n**AQ1:**\nThanks for your suggestion. Please refer to **AW2** for our answer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oFU4m0wKKp", "reviewer": "Reviewer_2sKD", "summary": "The paper addresses the significant issue of backdoor defense in a black-box setting during the inference phase of MLaaS applications. It proposes a novel technique called SampDetox, where the method uses diffusion models for detoxification by injecting noise to disrupt backdoors and retrieve clean instances.", "questions": "Even though there is a theoretical substantiation for the working defense, I wonder about the universality of this theory in real-world applications. Specifically, in varying datasets and models, would the hyper-parameters \\bar{t}_1 and \\bar{t}_2 require fine-tuning for different noise levels? While **Figure 5** provides an ablation study for different \\bar{t}_1 and \\bar{t}_2, it does not display the specific setting. It would be intriguing to know whether different \\bar{t}_1 and \\bar{t}_2 were utilized for the results in **Tables 1, 3, 4, 5, and 6**.", "limitations": "Although the paper acknowledges its limitations, a major overlooked limitation lies in the pre-trained diffusion model. Would the detoxifying effect still be substantial if the diffusion model used is trained on out-of-distribution samples compared to the target dataset? This question presents an area of potential further exploration to validate the robustness of the proposed SampDetox method.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The work underlines an essential research value and a practical scenario by studying black-box backdoor defense during the inference phase. The proposed two-tier defense method shows promise, backed by theoretical underpinnings, and demonstrates efficacy with relatively minimal overhead.\n\n2. An extensive set of experiments validate the effectiveness of the proposed method, surpassing existing defenses. Notably, the method can lower the Attack Success Rate of backdoor models and maintain a high PA on backdoor samples.", "weaknesses": "1. The research appears to make rather robust assumptions for diffusion models. The authors assume that the clean regions of the poisoned samples can be easily restored, while the poisoned regions, especially highly visible triggers, may fail to do so (as empathized in Line 201-202). This could be because the diffusion model learns only from the clean samples and fails to learn backdoor information. While the paper conducts supplemental experiments demonstrating strong defense performance even under other diffusion models (as shown in **Table 8**), the generalizability from ImageNet to CIFAR-10 seems less convincing. If the diffusion model is backdoored or the pre-trained diffusion model is trained on out-of-distribution samples, it could potentially dampen the final defense performance.\n\n2. Furthermore, the method is less likely to be effective if physical attacks are deployed, where natural objects are used as backdoor triggers, as diffusion could learn features of these natural triggers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**AW1:**\nThanks for your suggestion.\nFirst, it should be clarified that the defender completely controls the design, training, and utilization of diffusion models. Therefore, it is almost impossible for attackers to inject backdoors into diffusion models in practice. \nIn addition, even if the training data for the diffusion model is poisoned, backdoors can not be injected into it because attackers need to modify the training process of the diffusion models to inject backdoors [1]. \nMoreover, even if the diffusion model is backdoored, specific trigger patterns must be added to the input Gaussian noise of the diffusion model to trigger the backdoor.  However, when the input of SampDetox is Gaussian noise, due to the small number of denoising steps of SampDetox (i.e., $\\overline{t}_1=20, \\overline{t}_2=120$), it is extremely hard for a backdoored diffusion model to generate its target image.\n\nIn conclusion, backdoored diffusion models do not compromise the defense performance of SampDetox. Thanks again for your suggestion. We will add the discussion in the final version.\n\n**Reference:**\n[1] Chou, Sheng-Yen, et al. How to Backdoor Diffusion Models? CVPR’23\n\nWe conducted new experiments to evaluate the performance of SampDetox considering the case where pre-training datasets for diffusion models are out-of-distribution from their target classification tasks. The detailed experimental setups are as follows:\n* We conducted the training on a subset of the MS-Celeb-1M dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with a ratio of 8:2 and adjusted the shape of the samples to 224*224.\n* We adopted BadNets and WaNet as attack methods, representing a visible and an invisible backdoor attack, respectively.  \n* We considered four pre-trained diffusion models: i) a pre-trained model (i.e., **Model 1**) on one subset of MS-Celeb-1M, ii) a pre-trained model (i.e., **Model 2**) on another subset of MS-Celeb-1M, iii) a pre-trained model (i.e., **Model 3**) on ImageNet, and iv) **Model 4**, which is fine-tuned from Model 3 using a subset of MS-Celeb-1M.\n\nThe following table shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 is pre-trained on ImageNet rather than MS-Celeb-1M, resulting in the diffusion model failing to effectively denoise MS-Celeb-1M samples. However, we can find significant improvements in the CA and PA of Model 4 since Model 4 is fine-tuned using a subset of MS-Celeb-1M. In other words, SampDetox can defend against attacks using out-of-distribution diffusion models with proper fine-tuning.​\n​\n\n| Diffusion Models | &emsp;&emsp;BadNets | &emsp;&emsp;WaNet |\n| :----------------: | :------------------------: | :------------------------: |\n| | CA(%) / PA(%) / ASR(%) | CA(%) / PA(%) / ASR(%) |\n| No Defense |  94.38 / 1.27 / 99.32 | 94.61 / 1.96 / 97.05 |\n| Model 1 | 92.06 / 91.65 / 1.36 | 91.83 / 91.12 / 1.29 |\n| Model 2 | 90.27 / 89.73 / 1.31 | 89.92 / 90.05 / 1.22 |\n| Model 3 | 55.47 / 52.06 / 1.57 | 72.35 / 62.26 / 1.25 |\n| Model 4 | 88.56 / 89.27 / 1.47 | 89.43 / 89.32 / 1.30 |\n\n**AW2:** \nSorry for your confusion.\nIn Appendix D.3 (Limitations of SampDetox), we have discussed a type of backdoor attack that SampDetox cannot defend against. Specifically, when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.\n\n**AQ1:**\nSorry for the confusion. Based on the ablation study results shown in Figure 5, we suggest to set \n$\\overline{t}_1$ and $\\overline{t}_2 $ to 20 and 120, respectively. Please see lines 383-384 for more details. Please note that we use the same settings, i.e., $\\overline{t}_1=20$ and $\\overline{t}_2 = 120$ throughout the paper. In other words, the experimental results in Tables 1, 3, 4, 5, and 6 are obtained utilizing the same $\\overline{t}_1$ and $\\overline{t}_2$.\n\n**AL1:** Thanks for your suggestion. Please refer to **AW1** for our answer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0WiYeqyZTg", "reviewer": "Reviewer_i7qR", "summary": "The paper addresses the issue of backdoor threats in third-party Machine Learning as a Service (MLaaS) applications. The authors propose a novel two-stage method called SampDetox to defend against both visible and invisible backdoor attacks without requiring access to the model's parameters or training data. The proposed method uses diffusion models to introduce noise into samples to destroy backdoors and recover clean samples. It combines lightweight and high noise strategically to handle backdoors with varying visibility and robustness. The paper provides mathematical proofs to support the effectiveness of SampDetox in eliminating backdoor triggers and restoring original samples. Comprehensive experiments demonstrate SampDetox's effectiveness against various state-of-the-art backdoor attacks. The method is evaluated using different datasets and models, showing its superiority over existing defense methods in terms of clean sample accuracy, poisoned sample accuracy, and attack success rate.", "questions": "The Questions can be summarized as follows:\n\n1. **Question on Figure 6**: The authors compare the additional time overhead for various methods in Figure 6, noting that ZIP's overhead exceeds SampDetox's by more than 20 times. Given that ZIP utilizes DDPM once for image purification, whereas SampDetox employs it twice, this discrepancy seems disproportionate, especially if both are configured for the same denoising step count.  If ZIP and SampDetox use different configurations of DDPM, such comparison in overhead and defense performance may not be fair in some sense, as the configuration of DDPM plays an essential role in the recovery performance. Clarification on this point is requested.\n\n2. **Lack results for SampDetox with DDIM**: In section 5.4, the authors assert that integrating DDIM with SampDetox yields inference times comparable to unguarded methods. However, specific performance metrics for SampDetox, when augmented with DDIM, are absent. To substantiate this claim, it is imperative to demonstrate that SampDetox+DDIM can match the defense efficacy of SampDetox+DDPM.\n\n3. **Question on Table 5**: Table 5 presents ablation study outcomes for varying parameter selections. It remains ambiguous whether the data pertain to a single specific attack scenario or represent an average across multiple attack types.\n\n4. **The stability of parameters across different datasets**: Does the suggest parameter for $\\hat{t}_1=20$ and  $\\hat{t}_2=120$ works consistently across different datasets?\n\nI will raise my score if the author can address my concerns.", "limitations": "The authors have acknowledged certain limitations but have not sufficiently addressed the challenges associated with accessing pre-trained diffusion models. This is a critical issue that requires further exploration. A primary concern is the need for additional experiments to validate the approach using diffusion models trained on varied datasets. For instance, testing a model trained on ImageNet to defend against attacks on the CelebA dataset would provide valuable insights into the model's adaptability and robustness (Just as a suggestion/example).", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The strengths can be summarized as follows:\n\n1. **Innovative Defense Mechanism**: SampDetox introduces a novel two-stage approach to address backdoor attacks in a black-box context, which is a significant contribution to the field of machine learning security.\n\n2. **Theoretical Foundation**: The paper provides a solid theoretical basis with mathematical proofs that support the effectiveness of the proposed method in eliminating backdoor triggers.\n\n3. **Comprehensive Evaluation**: The method is extensively tested against state-of-the-art backdoor attacks using multiple datasets and models, demonstrating its generalizability and reliability.\n\n4. **Thorough Analysis**: The paper includes ablation studies that dissect the impact of different stages and parameters of the SampDetox method, providing insights into its workings and optimization.", "weaknesses": "**Summary of Key Weaknesses:**\n\n1. **Dependency on Pre-trained Models**:\n   - *SampDetox's* effectiveness is contingent upon the quality of pre-trained diffusion models utilized for the denoising process. This poses a limitation, particularly when these models are not finely tuned for the specific data in question. While the authors have successfully demonstrated *SampDetox* using an ImageNet-trained model to counter attacks on CIFAR10, the real-world applicability may be limited due to the difficulty of accessing a pre-trained model that matches the unique dataset of the application. \n\n2. **Trigger Recovery Assumption**:\n   - The paper presupposes that clean regions in poisoned samples can be easily restored, whereas poisoned regions, especially with visible triggers, cannot (as stated on Line 201). This assumption may not hold against sophisticated attacks like physical attacks where triggers are inherent and visible parts of the sample. Furthermore, this assumption could be invalidated if the image features an object that the pre-trained diffusion model is incapable of accurately recovering.\n\n3. **Omission of Experimental Details**:\n   - The paper falls short in providing certain critical experimental details, particularly regarding the baseline defense methods (such as the diffusion model configuration for ZIP) and the proposed method itself ($\\bar{t}_1$ and $\\bar{t}_2$ for different dataset).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**AW1:** \nThanks for your suggestion. We conducted new experiments to evaluate the performance of SampDetox considering the case where pre-training datasets for diffusion models are out-of-distribution from their target classification tasks. The detailed experimental setups are as follows:\n* We conducted the training on a subset of the MS-Celeb-1M dataset. Specifically, we considered the top 100 labels with the largest number of samples and randomly selected 380 samples for each label. We split the training and tests with a ratio of 8:2 and adjusted the shape of the samples to 224*224.\n* We adopted BadNets and WaNet as attack methods, representing a visible and an invisible backdoor attack, respectively. \n* We considered four pre-trained diffusion models: i) a pre-trained model (i.e., **Model 1**) on one subset of MS-Celeb-1M, ii) a pre-trained model (i.e., **Model 2**) on another subset of MS-Celeb-1M, iii) a pre-trained model (i.e., **Model 3**) on ImageNet, and iv) **Model 4**, which is fine-tuned from Model 3 using a subset of MS-Celeb-1M.\n\nThe following table shows the experimental results. We can find that SampDetox with different diffusion models achieves satisfied defensive performance on ASR. Note that, for Model 3, the CA and PA of SampDetox decrease. This is because Model 3 is pre-trained on ImageNet rather than MS-Celeb-1M, resulting in the diffusion model failing to effectively denoise MS-Celeb-1M samples. However, we can find significant improvements in the CA and PA of Model 4 since Model 4 is fine-tuned using a subset of MS-Celeb-1M. In other words, SampDetox can defend against attacks using out-of-distribution diffusion models with proper fine-tuning.​\n\n|Diffusion Models|BadNets|WaNet|\n|:-:|:-:|:-:|\n||CA(%)/PA(%)/ASR(%)|CA(%)/PA(%)/ASR(%)|\n|No Defense|94.38/1.27/99.32|94.61/1.96/97.05|\n|Model1|92.06/91.65/1.36|91.83/91.12/1.29|\n|Model2|90.27/89.73/1.31|89.92/90.05/1.22|\n|Model3|55.47/52.06/1.57|72.35/62.26/1.25|\n|Model4|88.56/89.27/1.47|89.43/89.32/1.30|\n\n**AW2:**\nSorry for your confusion. In Appendix D.3 (Limitations of SampDetox), we have discussed a type of backdoor attack that SampDetox cannot defend against. Specifically, when dealing with triggers derived from semantic features that are part of the original classified features and whose distribution is the same as the distribution of the training dataset of the pre-trained diffusion model, these triggers will be restored by the denoising phase of SampDetox. In this case, the triggers will not be destroyed by SampDetox, resulting in a failure of defense.\n\n**AW3:**\nThanks for your suggestion. The details are as follows.\n* SampDetox (Ours). As mentioned in Appendix A.2, we utilized the improved diffusion model provided by [17], whose diffusion process consists of 1000 steps. We use the same settings $\\overline{t}_1=20$ and $\\overline{t}_2=120$ throughout the paper.\n* Sancdifi followed the settings in Section “Numerical Experiments” of its paper [33]. Sancdifi computed RISE maps using 2000 random binary masks and set the saliency threshold to 0.95. For a fair comparison, Sancdifi used the same diffusion models as ours.\n* BDMAE followed the settings in Section “Method Configurations” of its paper [15], which used masked autoencoders pre-trained on ImageNet with 24 encoder layers.\n* ZIP followed the settings in Section “Purification Implementation” of its paper [14], which set its hyperparameter $\\lambda$ to 2. For a fair comparison, ZIP used the same diffusion models as ours.\n\nWe will detail the above experimental settings in the final version.\n\n**AQ1:**\nSorry for your confusion. \nIt should be clarified that we used the same DDPM configuration for ZIP and SampDetox in our experiments. SampDetox is much faster than ZIP because ZIP relies on the whole diffusion process of DDPM, while SampDetox only needs partial steps of the whole diffusion process. Specifically, ZIP utilizes DDPM to generate samples from completely random noise, requiring the whole diffusion process with 1000 denoising steps. However, since we set $\\overline{t}_1=20$, SampDetox requires only 20 denoising steps in Stage 1. For Stage 2, the number of denoising steps varies from pixel to pixel, but it will not exceed 120 steps at most due to $\\overline{t}_2=120$. In this way, though SampDetox employs DDPM twice, its overhead is still much less than that of ZIP.\n\n**AQ2:**\nThanks for your suggestion. Since DDIM was proposed to accelerate the speed of DDPM without affecting the image generation quality, the defense performance of **SampDetox+DDIM** is comparable to that of **SampDetox+DDPM**. To demonstrate the performance of **SampDetox+DDIM**, we conducted new experiments on CIFAR-10 against ten SOTA backdoor attacks. From the following table, we can find that the defense performance of **SampDetox+DDIM** is almost the same as that of **SampDetox+DDPM**, which means that **SampDetox+DDIM** can match the defense efficacy of **SampDetox+DDPM**.\n\n|Attack Methods|SampDetox+DDPM|SampDetox+DDIM|\n|:-:|:-:|:-:|\n||CA(%)/PA(%)/ASR(%)|CA(%)/PA(%)/ASR(%)|\n|BadNets|89.57/90.15/2.11|89.49/90.13/2.12|\n|SIG|83.71/65.06/11.03|83.82/65.13/10.98|\n|LC|80.72/74.36/1.55|80.62/74.22/1.53|\n|TrojanNN|92.78/89.95/1.86|92.83/89.87/1.69|\n|Dynamic|88.52/88.62/1.45|88.52/88.72/1.42|\n|Blended|90.23/86.65/1.96|90.15/86.54/2.02|\n|LF|90.01/87.40/3.02|90.09/87.61/3.10|\n|WaNet|89.34/88.92/5.59|89.48/88.82/5.54|\n|ISSBA|90.74/86.51/1.60|90.76/86.65/1.55|\n|BPP|90.59/84.83/6.15|90.42/84.91/6.17|\n\n**AQ3:**\nSorry for the confusion. The results in Table 5 represent the average results across multiple attack types. We will add this information in the final version.\n\n**AQ4:**\nSorry for the confusion.\nThe setting of $\\overline{t}_1=20$ and $\\overline{t}_2= 120$ remains consistent across varying datasets and models. In other words, we use the same fixed settings throughout the paper.\n\n**AL1:**\nThanks for your suggestion. Please refer to **AW1** for our answer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work proposes a black-box approach to defend against backdoor attacks, based on a diffusion model that restores the image parts covered by the backdoor trigger. Most of the concerns raised by the reviewers were addressed, even if the paper still falls short in evaluating against adaptive attacks and discussing the limitations of the approach, which (i) strongly relies upon the effectiveness of the pre-trained diffusion model, and (ii) remains specific to image data. Nevertheless, the authors can better discuss and highlight such limitations, while the paper does a good job when it comes to the experimental analysis. Overall, also in light of the positive feedback from the reviewers after the rebuttal, I can recommend acceptance.
Decision: Accept (poster)