Black-box Backdoor Defense via Zero-shot Image Purification
Authors: Yucheng Shi
Mengnan Du
Xuansheng Wu
Zihan Guan
Jin Sun
Ninghao Liu
Venue: NeurIPS 2023 poster (2023)
Keywords: backdoor defense
black-box defense
diffusion model

Abstract: Backdoor attacks inject poisoned samples into the training data, resulting in the misclassification of the poisoned input during a model's deployment. Defending against such attacks is challenging, especially for real-world black-box models where only query access is permitted. In this paper, we propose a novel defense framework against backdoor attacks through Zero-shot Image Purification (ZIP). Our framework can be applied to poisoned models without requiring internal information about the model or any prior knowledge of the clean/poisoned samples. Our defense framework involves two steps. First, we apply a linear transformation (e.g., blurring) on the poisoned image to destroy the backdoor pattern. Then, we use a pre-trained diffusion model to recover the missing semantic information removed by the transformation. In particular, we design a new reverse process by using the transformed image to guide the generation of high-fidelity purified images, which works in zero-shot settings. We evaluate our ZIP framework on multiple datasets with different types of attacks. Experimental results demonstrate the superiority of our ZIP framework compared to state-of-the-art backdoor defense baselines. We believe that our results will provide valuable insights for future defense methods for black-box models. Our code is available at https://github.com/sycny/ZIP.

Black-box Backdoor Defense via Zero-shot Image Purification
Yucheng Shi Mengnan Du Xuansheng Wu' Zihan Guan
Jin Sun
Ninghao Liu
School of Computing, University of Georgia
Department of Data Science, New Jersey Institute of Technology (yucheng.shy shi, xuansheng. wu, zihan. guan, jinsun, ninghao wiw.luuaaaaaaee edu mengnan gongman.du@njit. du@njit.
Abstract
Backdoor attacks inject poisoned samples into the training data, resulting in the misclassification of the poisoned input during model's deployment, Defending against such attacks is challenging, especially for real-world black-box models where only query access is permitted. In this paper, we propose a novel defense framework against backdoor attacks through Zero-shot Image Purification (ZIP). Our framework can be applied to poisoned models without requiring internal information about the model or any prior knowledge of the clean/poisoned samples. Our defense framework involves two steps. First, we apply linear transformation (e.g., blurring) the poisoned image to destroy the backdoor pattern. Then, a pre-trained diffusion model to recover the missing semantic information removed by the transformation. In particular, transformed image to guide the generation of high-fidelity purified images, which works in zero-shot settings. We evaluate ZIP framework multiple datasets with different types of attacks. Experimental results demonstrate the superiority of ZIP framework compared to state-of-the-art backdoor defense baselines. We believe that our results will provide valuable insights for future defense methods for black-box models. Our code available /tttts://ggiiaiiii com/ sycny/ZIP
design
reverse process by using the
Introduction
Machine learning has been increasingly integrated into real-world applications such as healthcare 45 finance (28) and computer vision [66]. Despite great success, machine learning models are susceptible to adversaries such backdoor attacks [1] g 43 54 [7]. which compromises model security and reliability. Backdoor attacks can manipulate model's behavior by injecting malicious samples into the training data altering the model's weights. Although many defense strategies have been proposed to mitigate backdoor attacks, most of them require access to the model's internal structure and poisoned training data [32] Deploying these defenses is challenging in real-world black-box scenarios, where defenders do not have access to verify or audit the inner workings of the model [8 19] For example, many developers and end-users now prefer using machine learning service (MLaaS), relying on models provided by third-party vendors for their applications. However, these models may contain backdoors, and due copyright concerns, the services typically operate in a black-box setting with query-only access. In such scenarios, detecting and mitigating backdoor attacks is very difficult.
Currently, there are only a few backdoor defense methods that work in Mlac-box settings. They can be categorized into two types: detecting-baat based and purification-based Detecting- based methods (65
"Conesponding Author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

18 31 12 |66] can detect the poisoned sample but could not remove the poisoned patterns. These methods are not applicable when critical poisoned samples must be used by downstream classification models. Purification-base given the poisoned images. However, state-of-the-ar purification approaches rely on masking and reconstructing the poisoned area 42 53.. It can only protect against patch-based attacks. Other purification-bal based methods that employ image transformations for defense have the potential to defend against more sophisticated attacks but may result in a reduction in classification accuracy due to the loss of semantic information (35].
methods can address this problem since they aim to retrieve clean images
To overcome these challenges, we propose a novel framework to defend against attacks through Zero- shot Image Purification (ZIP). We define "purification" as the process of maximizing the retention of important semantic information while eliminating the trigger pattern. With this goal, we preserve the classification accuracy while breaking the connection between trigger patterns and poisoned labels. We also define "zero-shot" as the ability to defend against various attacks without relying prior knowledge of attack methods. other words, our approach does not require access to any clean or poisoned image samples (patch or non-patch based), and could be applied directly to unseen attack scenarios. This setting is crucial because real- world users usually have limited information, while new threats continue to emerge. Our proposed framework contains two main stages; we first utilize image transformation to destruct the trigger pattern, and then leverage an off- the-shelf,, pre- pre-trained diffusion generative model to restore the semantic information. Our defense strategy is based on the motivation that the semantic information in a poisoned image (e.g., faces, cars, or buildings) constitutes the majority of the data and typically falls within the training data distribution of pre-trained image generation model. In contrast, engineered trigger patterns (e.g.. mosaic patches and colorful noise) are subtle and unlikely to exist in the pre-training datasets 32 34] Since the diffusion model learns to sample images from the training distribution [22], purified images generated from the diffusion model will retain only their semantic information while eliminating the trigger patterns. As result, our purification approach can effectively defend against various attacks while maintaining high-fidelity in the restored images.
Our main contributions are summarized as follows. (1) can be applied to black-box models without requiring any internal information about the model. Our method is versatile and easy to use without retraining.
We develop a novel defense framework that
(2) The proposed framework is designed for zero-shot settings, which do not require any prior knowledge of the clean or poisoned images. This feature relieves end-users from the need to collect samples, thus enhancing the framework's 's applicability. (3) Our defense framework achieves good classification accuracy the purified images, which were originally poisoned samples, even when using an attacked model as the classifier. This improvement further enhances the framework's effectiveness and usability.
2 Preliminaries
2.1 Problem Definition
This paper addresses the backdoor defense problem in the context of image classification tasks. The goal of image classification is to learn a function fo(x) that maps input images x X' to their correct labels E V, where X' denotes the image space, y denotes the label space, and 0 represents model parameters. Typical backdoor attacks include poisoning training data and perturbing model weights, and we use fattuck to denote the model that has been attacked. During inference, an attacker can take a clean sample x and manipulate it to create backdoor sample xP e.g., adding the trigger pattern p make x" x p. The backdoored model will misclassify x" as the target label fargett = fg fgttack (x") + y. Our threat model is in a challenging black-box setting, where defenders can only query the poisoned model and have no access to the model's internal parameters or training datasets. The attackers can modify model components any other necessary information to implement their attacks. We formally define our backdoor defense problem as below.
Problem Image Purification for Backdoor Defense. Our defense is implemented in the model inference stage. Let fo pattack denote the attacked model, whose parameters are not accessible. Given poisoned image xP, our goal is to obtain purified imuge x" from xP by removing the effect of trigger P. The purified image should be classified as the same category as the original clean image x, i.e., fo(x') = fo(x) # fo(x?).
2

---

Stage
Under-Attack
At
X
P
Linear Transfommtionn
A
II A A
0
Stage
to
XT
X
X
XO
x" ~ (0
X
Figure I: The proposed ZIP backdoor defense framework. In Stage I, we use linear transformation such blurring to destruct the trigger pattern in poisoned image x"". In Stage 2, design guided diffusion process to generate the purified image X₂ with a pre-trained diffusion model. Finally, Xo. the semantic information from is kept while the trigger pattern is destroyed.
2.2 Diffusion Models
We leverage the reverse process of diffusion models to purify images. The denoising diffusion probabilistic model (DDPM) [22 is a powerful generative model for generating high-quality images. It has two processes: forward process and a reverse process. In the forward process, the model iteratively adds noise to an input image Xo until it becomes random Gaussian noises Xy; in the reverse process, the model iteratively removes the added noise from XT to recover the noise-free image Xg. More details can be found in the DDPM paper [22].
Forward Process: A noise-free image Xo is transformed to a noisy image x, with controlled noise. Specifically, Gaussian noise is gradually added to image X in T steps based on variance schedule 3t E [0. 1] such that q (x, x(-1) == N (xe: /1 3, 8,I), (xx.-1.8,1)
where denotes the posterior probability of x, conditioned on Xt-1 A nice property of this process is that the f-th step noisy image could be directly generated by:
q (x xo) = - (xii Va,o, (1 I), x = Va,x0 + V1 ōGE, where (O, 02 = Br, and ā₂
II Oz-
Reverse Process: The noisy input image XT is transformed into a noise-free output image xo over time steps. In each step, the diffusion model takes the current image state input and produces the previous state Xt-1. We aim to obtain clean images X by iteratively sampling X from p(x,11|11 X2, Xu):
18 1- ū
Var(1 (1-ar-1) X, + o.c. c~,(( of 1-- B
Xo +
(2)
ā-
1-
Based on Equation D the original clean image Xo could be approximated based the 1-th step observation X₁ as: Xojt vo. (x: - -1 where E, denotes the estimation of the real e in step t. In each step I, DDPM utilizes neural network (-(-) to predict the noise i.e., E, (xx, t). With this estimation, we can convert Equation into the following form as reverse process:
1
aq
(x:
Ee) + OrE.
(3)
Va
VI
ae
Zero-shot Image Purification (ZIP)
3.1 Overview of Proposed Framework
Our defense framework consists of two primary stages: (1) destruct poisoned images through image transformation, and (2) recover images using diffusion model, as depicted in Figure The first stage based on the observation that the integrity of trigger patterns crucial for backdoor attacks to deceive the model. Thus, destructing the trigger pattern would significantly reduce the effectiveness

---

of backdoor attacks [6] On the other hand, a strong transformation may destroy both the trigger pattern and semantic information. To address this. we introduce the second stage to recover the semantic information through the reverse process of diffusion models. However, traditional diffusion models (e.g.. DDPM (22) or image restoration models (e.g., DDRM (29] and DDNM (58)) can not generate high-fidelity and clean images due to lack of direct control over the generated output.
To bridge the gap, in the following section, we first propose an image generation constraint based on the image transformation in Section 3.2 We then apply this constraint to guide the image reverse process of the diffusion model. In addition, we discuss the guide adaptation when applied in the zero-shot settings, along with its theoretical justification in Section 3.3 Finally, we introduce our efforts to improve the inference speed in Section 3.4
3.2 Image Transformation and Decomposition
In the first stage, apply image transformation to destruct potential trigger patterns, such using average pooling to blur the poisoned images. Formally, we denote the transformation as a linear operator A, and let the transformed image be x Ax A(x p). Previous research [4 47 has used the transformed image directly as the purified result. However, such approaches may result in poor classification accuracy due to the loss of fidelity induced by A.
To recover the lost information, an intuitive way is to apply an image generative model, c.g. the diffusion model, to yield purified image. However, the vanilla diffusion model generates images from random Gaussian noise and lacks control over the fidelity of generated images. Thus, we propose constraint guide the generation process of diffusion models to recover high-fidelity images. Specifically, for an ideally purified image Xy, it should satisfy X = x, so we have:
A (x0 + +p) A(x+p) x
(4)
it is possible decompose image x into two parts using linear operator A (e.g., average pooling) and its pseudo-invers, At (e.g.. upsampling) that satisfies AATA A. The decomposition is expressed part denotes the observable information in the range-space, and the latter part denotes the information in the null-space removed by transformation Bringing this decomposition Equation we have:
Based on the RND theory
488 58
AtAx + (I - AtA) x, where the former
(xo p) = A'x + p) + (I A'A) (xo + p).
This equation derives a constraint for image purification to restore the original x as below:
X A A'Ap ATA) Xo:
(6)
accordingly, the ideally purified image could be decomposed into three parts. The first two parts are in the range space: the observable information stored in the transformed image Al xA, and the intractable information embedded in the transformed trigger pattern A A'Ap; the last part is in the null-space and is unobservable as it is removed by the transformation. To restore the lost information in the null-space, we utilize the observable information in the range-space as references.
3.3 Image Purification with Difftusion Model
3.3.1 Reverse Process Conditioned on Poisoned Images
Our proposed image purification is based on the reverse process of the diffusion model, which takes Gaussian noise Xy input and generates noise-free image Xg. We use to denote the image time step in the reverse process of diffusion. To generate rectified estimation of x₁ as x/, so that it produces Xojt that satisfies the decomposition constraint
images, we propose
in Equation 6 The x is computed using the following equation, which is derived from Equation ] and 6 The detailed proof is in the Supplementary Material A
x = V T V
Vo.AiAp App+ + (I- A!A)x + A'AVT - -G,
'Usually we have A A # I, which implies that this operation is lossy. When applying A A to an image, some information will be removed, making this operation irreversible. More details are in Supplementary Material C

---

where denotes the estimated noise, which is calculated using the pre- rre-trained diffusion model gø: E, = Ia (x,,)) Then, we modify the original reverse process Equation 3 accommodate this rectified estimation. The modified reverse process is expressed as:
X
A ā,c;
- 0t
t. (8)
Va,
v1 at
where (0, denotes Gaussian noise and we have of 1-a 1-6c
Bc.
3.3.2 Adapting Reverse Process to the Zero-shot Setting
In this subsection, we show how our reverse process design can be effectively applied in the zero-shot setting, where the trigger pattern p unknown to defenders. We propose to omit the intractable term vā, A Ap in Equation] 7 and approximate x with & to obtain a new rectified estimation:
Vo,Aix (I-A A'Ax, + A A'AVT O,6
The intractable term A can be omitted due the following reasons:
The value of v at the beginning of the reverse process is very small, making v Ap negligible compared to other terms. We provide an improved approximation in Section 3.3.3 for
later stages when vā, increases.
Backdoor attacks are generally stealthy or use more negligible patterns compared to the original images since the attacker wants minimize the impact the model's accuracy legitimate data [43]. Thus, the destructed pattern Ap is usually negligible compared to x A(x + p).
The effect of A Ap is further reduced by selecting appropriate image transformations. Most back- door attacks are characterized by severe segg-freuuency artifacts [65]. Therefore, transformations
such average pooling can remove the high- gigg-freeenccy information in p.
Nevertheless, approximation an iterative process such as diffusion can be risky because errors from the previous step can accumulate rapidly (e.g., exponentially) and lead to significant inaccuracies. Our method addresses this issue by ensuring the approximation error between each step is well-bounded theoretically. As result, the final recovered image Xo preserves the essential information of the original image, while the trigger pattern undergoes a transformation and is likely to be destroyed.
Lemma 3.1. Suppose the estimated noise oulput by (o(-)) is Gaussian. Given g () = 6t. we have Io (o(((( ((x, + ((d,AtA)) = 6, + é4, where 62 é, are also Gaussian.
The error ef in Lemma 3.1 3.1 is much smaller than the real estimated noise 6t- This is because the trigger pattern V Ap, which weighted by vār, is relatively subtle compared with the intermediate image x,. Additionally, the attacker-engineered trigger pattern p is unlikely to be present within the natural image distribution learned by the pre-trained diffusion model. Therefore, it is unlikely to activate the model (+(+) and generate significant output.
Theorem 3.2. Suppose that (x( + (ā,AtAp) = Ez + E. We define the error at step t between & and (xt + Vā,Al Ap) as 8%,, i.e., 8% - -8- (Xt + Ao:AlAp). Let ò, = Aö, we have the following bound on its norm: ||ō,1| < (1-Ge)
||||||
VI-a++1
This theorem means that our proposed approximation introduces only limited approximation error at each time step after considering the error from the previous step. When paired with a common linear transform A like average pooling, its norm is relatively small [58]. Additionally, at the initial steps of the reverse process, the term ||ع/+a||| || is also small according to Lemma 3.1 and we have (1 (1-6,),ReHi < 1_ which is also a small value. Collectively, the error from our V1-6.11
approximation is well bounded by a small number. According to this theorem, the final purified image exhibits the following property.
Corollary 3.2.1. When = 0, we have &o = X + AtAp 8/- where A6 = =0
According to Corollary 3.2.1] our final purified image contains three parts: the ideally purified image Xy. the altered trigger pattern At Ap, and an approximation error 5/- Therefore, compared with the original poisoned image x" = x + p or the transformed image x A(x + p), our purified image Xn reduces the effect of the trigger pattern p and preserves the high fidelity of the image. This is critical for achieving good performance in the downstream image classification task. The proofs can be found in Supplementary Material
5

---

3.3.3 Improving ZIP with Theoretical Insights
Based on the above theoretical analysis, we propose two further improvements to our guided image purification framework to enhance its effectiveness, which include (1) adding a confidence score to the rectified estimation, and (2) introducing multiple transformations.
Confidence Score. The effectiveness of our framework depends on the confidence of the proposed rectified estimation. As the reverse process proceeds, the vār value increases, making it difficult for vä,A Af Ap to be neglected. Simply omitting this pattern would weaken the confidence in our proposed estimation.
To address this issue, we re-formulate the rectified estimation at step as: x, = -1- + aj X where x denotes the result of a pure diffusion model (e.g.. DDPM) at step t + 1, and X 2 is parameter. Since 0 < ã < 1, when À = 0, we obtain pure reverse diffusion process without any rectification; when À +o0, the reverse process reduces to ZIP. A nice property of x, is that, at increases as the reverse process proceeds (t decreases), making the contribution of x, larger. This is reasonable because the intermediate image becomes increasingly informative over time, reducing the reliance on & in reverse diffusion. We set À value to achieve desirable fidelity scores (e.g.. for the resultant purified images. Finally, our revised reverse process defined as: PSNR 29 58
1- a: Ex) OGE. v1 āi
X
(x+
(10)
Va
Multiple Transformations. To improve the effectiveness of our approach in removing the poisoning effect, we propose including multiple transformations to more effectively destroy the unknown trigger pattern in zero-shot setting. Specifically, given N transformations, the intermediate image x, and estimated noise should all satisfy:
x = Vo,
(I A + A
n = 1,,,.... N.
(11)
Finally, we have x (x) + + x We provide our proposed algorithm in Algorithm We focus on two types of transformation pairs in this paper: blurring, represented by A AJA,, and gray-scale conversion, represented by AAA2.
3.4 Purification Speed-up
The purification speed is crucial when performing defense inference time. Our framework leverages pre-trained diffusion models, and conducts purifica- tion on each test image. Hence, propose several techniques to speed up purification and reduce costs.
Algorithm Zero-shot Image Purification
Require:
Test image for purification x"; lin- ear transformation A1, A2..., A, and their pseudo-inverse A A ..., Ad; diffusion model g: hyperparameter A.
Ensure: x xA,, A.," P.
0, I...,,N
3.4.1 Diffusion Model Inference Speed-up
1: x" I..
2:
T,T 1....
Algorithm requires a large number of steps to gen-
3:
E e~ c~N(( if 1, clse e 0
erate single sample. Each step involves computing
4:
9o (x.,t)
the estimated noise and diffusion process, which can be computationally expensive. To address this issue, we leverage the power of the denoising diffusion im- plicit model (DDIM) [50] improve the inference
5:
for 1,2,... N do
6:
x = A x
+ -I-------AAAlA- AAA,,))
A! A. cee
7:
end for
speed of our reverse process as below:
8:
2, = 1/4 (x) + x3
xNN
9:
x: = (1 at X: + 0 Xt
X Vāz-1 + v 1 ā - ofee o,e, (12)
10:
X:-1 va,
(x: 1-a1
E1) + O7E
11: end for
where Roje also an estimated image based on Rt,
12: return xo
and we have Xp)z = -), this speed-up inference method, we can obtain high-fidelity purified image by sampling only few steps instead of conducting sampling thousands of steps. The modified algorithm based on DDIM
(1-6) By using
is provided in the Supplementary Material D

---

3.4.2 Framework Speed-up
Batch Data Speed-up. We introduce acceleration method for purification, when images can be processed batch during inference. For instance, if the pre-trained model accepts images of size L L as imput, and we wish to purify I x images, can first combine (L/1)² images into single Lx L image by tiling. We then apply our purification method to the tiled image, and split the purified image back to the original images afterwards. Larger images could be resized into / x I to apply this method. In this way, our defense approach can handle images of various sizes and significantly improve the purification speed.
Streaming Data Speed-up. In the streaming data scenarios, defenders may only have access to a single test image a time. In such cases, we use fast zero-shot detection method such as [I9 to quickly assess whether the image is likely to be poisoned. If the detection result indicates a potential poisoning, can apply our purification method to remove the trigger pattern. This approach allows us to quickly and effectively defend against poisoned images without applying our purification on every image, thus saving the average processing time.
Experiments
4.1 Experimental Settings
For defense evaluation, we conducted experiments on three types of backdoor attacks: BadNet [5
Attack in the physical world (PhysicalBA) 34 and Blended [8] The first two attacks are repre- sentatives of patch-based based attacks, while the last one represents non-patch-bbes based backdoor attack. To configure these attack algorithms, we follow the benchmark setting [33 and use the provided codes. We evaluate the effectiveness of our defense framework ZIP three datasets: CIFAR-10 30 GTSRB (52), and Imagenette [24). The poisoned classification network is based on ResNet-34 (2 Specifically, for the CIFAR-10 dataset, we apply both blur and gray-scale conversion linear trans- formations. For the GTSRB and Imagenette datasets, we solely apply blur as the linear transformation.
The additional experiment details are in the Supplementary Material E
Defense methods available for black- purification in zero-shot are rare. To fairly assess the effectiveness of our proposed method, we conduct comparative evaluation against two baseline approaches: ShrinkPad (35) and image blurring (referred to as Blur). The former a state-of-the-ar art image transformation-baat based defense method that can work on black-box models in the zero-shot setting. The latter uses blurred images A|x11 purified images. We apply defense methods to all test samples, and then evaluate the output using a poisoned classification network. In addition to the clean accuracy (CA) and attack success rate (ASR) metrics for assessing defense effectiveness, introduce a new metric called poisoned accuracy (PA). It measures the classification performance of the purified poisoned samples. A higher value of PA indicates that the purified poisoned samples more likely to be correctly classified, even when using an attacked classification model.
4.2 Qualitative Results of Purification
We conduct qualitative case studies (see Figure 2 of our method in purifying poisoned images created by the Blended and BadNet attacks. We show examples of poisoned images x" and purified image Xp. where the trigger pattern is clearly visible in the poisoned images but has been altered/removed in the purified images. For comparison, we also show the blurred image A A and the grayscale images Alxx4,, The transformed images can destruct the trigger pattern, but they also alter a lot of semantic information. These results demonstrate the effectiveness of ZIP removing the effect of trigger patterns from images while maintaining semantic information. More qualitative results for different attacks are in the Supplementary Material B
4.3 Quantitative Results of Defense
We conduct quantitative experiments where the results are Table T follows: (1) Our method effectively defends against different backdoor attacks. This is because the transformations in our framework, which are independent of specific attacks, break the connection between backdoor triggers and backdoor labels. (2) Our method achieves overall better classification
Our observations are as
7

---

X
A
A₂x A2
X
Figure 2: Qualitative analysis of purification. (Row 1-2: Blended; Row 3-4: BadNets. More
qualitative results are listed in Supplementary Material
Table 1:
The clean accuracy (CA %),, the attack success rate (ASR %). and the poisoned accuracy (PA %) of defense methods against different backdoor attacks. None means attacks are applied.
Dataset
Attack
No Defense
ShrinkPad (defense)
Blur (defense)
ZIP (Ours)
CA 1
ASR
PA
CA
ASR J. PA
CA
ASR
PA
CA
ASR
PA
None
80.15
BadNet
82.31
99.98
10.00
62.89
9.34
63.12
58.19
21.78
53.47
78.97
5.53
79. 10
CIFAR-10 (32 x 32) (10 classes)
Rlended
80.26
99.96
10.05
58.97
40.22
55.91
3.04
72.62
7.75
57.98
PhysicalBA
85.30
98.73
11.20
82.84
90.50
18.37
41.84
1.09
41.37
80.10
4.33
80.33
Average
82.62
99.56
10.41
68.23
34.04
40.57
51.98
8.64
48.25
77.23
5.87
72.47
None
96.95
BadNer
5.70
78
96..18
966
GTSRB (32 x 32)
classes)
Blended
96.58
99.89
5.79
76.76
10.54
56.41
93.68
11.07
73.91
95.74
8.53
81.27
PhysicalBA
96.83
100.00
5.70
97.44
100.00
5.70
91:00
5.53
90.53
95.44
6.57
94.91
Average
99.96
5.73
84.17
7.98
86.52
95.79
Noue
BadNet
$1.99
94.53
14.98
71.23
8.56
70.72
81,47
16.45
79.94
84.005
7.55
83.97
Imagenette (256 256)
Blendod
86.14
99.85
10.19
74.06
20.63
36.10
78.95
79:41
25.57
81.42
8.55
78. 36
(10 classes)
PhysicalBA
90.67
72.94
34.29
90.22
90.21
96.81
13.07
84.84
32.40
74.87
87.26
10.91
86.54
Average
19.82
accuracy (CA) compared to baseline methods because our formulation successfully recovers semantic information to ensure accurate downstream classification. For example, on the Imagenette dataset, ZIP reduces the ASR of the BadNet attack from 94.53% (no defense) to just 7.55%, with only a 0.94% drop in CA. Similarly, on the GTSRB dataset, our approach reduces the success rate of the PhysicalBA attack from 100% defense) 6.57%, with only a 1.39% drop in CA. (3) Our method outperforms the baselines in poisoned accuracy (PA), indicating that poisoned samples can still be used for classification even when using an attacked black-box classifier. It demonstrates the robustness and usability of ZIP in real-world scenarios. (4) ShrinkPad performs poorly on the
PhysicalBA attack, which is consistent with the findings in (34] This is because PhysicalBA specifically designed to evade defense methods such as ShrinkPad (34 On the other hand, our
method successfully defends against this attack, demonstrating superior performance, We include additional experiment results of defenses in Supplementary Material

---

Table ZIP with different transformations (CAt ASR// PA1).
Attack
Blura
Blur,
Blura
Grayscale
BadNets
84.05/7.55833.97
82.77/7.15/82.39
72.38/13.4177.500
70.52/10.79/68.0
Blended
84.05/26.11/63.
81.42/8.35/78.36
71.03/7. 11.03/7.69/6..3
99.79/99.40/100
PhysicalBA
90.24/22.88779.89 87.26/10.9188654
76.58/19.43/76.33
76.66/17.51/74.87
4.4
Ablation Studies
4.4.1 Evaluation Transformations
Comparison Different Transformations. Image transformation plays a crucial role our defense. We evaluate the effectiveness of ZIP the Imagenette dataset using four distinct types of transformations: Blurg, Blurg, Blurg, and Grayscale (the results are in Table 2}. The subscript of "Blur" represents the kernel size of average pooling, where larger number indicates stronger transformation. We have the following observations: (1) The blur operation is more effective than grayscale conversion in reducing the ASR while maintaining good CA and PA. (2) Generally, stronger transformations are more effective in destructing the trigger pattern, resulting in a lower ASR. However, they also destroy more semantic information, leading to lower classification accuracy.
Comparison to Masking and Reconstruction. Linear transformations more effective than masking in removing poisoned effects, particu- larly in cases like the Blended attack where the backdoor pattern is distributed across the image. We compare ZIP with recently proposed pu- rification method BDMAE [33] on CIFAR-10.
Table 3: Comparison to BDMAE (CA) / ASRR).
Attack
No Defense
BDMAE
ZIP
BadNet
82.31/99.98
81,44/1.12
78.97/5.53
Blended
80.26/99.96
78.57/99.88
72.62/7.75
BDMAE first identifies the trigger region autoencoder to restore the image. To ensure a fair comparison, we apply the defense stage of BDMAE our benchmarks using its official code. Table 3 shows that BDMAE effectively defends against the BadNet attack, but it fails to defend against the Blended attack. This is because the trigger pattern in Blended attacks is not located in a local region, making it difficult for BDMAE to identify an appropriate mask. In contrast, our ZIP framework does not make assumptions about trigger patterns thus successfully defends against both attacks.
test image, masks the region, and then uses masked
4.4.2 Evaluation on Enhanced Attacks
We consider more challenging scenario, where assume the attacker is aware of our defense and has access to the purified backdoor images, which are then used as enhanced poisoned images to attack classifier. To demonstrate the effectiveness of our proposed method in defending against such enhanced attacks, we apply our method to BadNet and Blended as examples and choose blurring and grayscale conversion together the transformations of ZIP. Other settings remain the same the original attack, and more details about the settings are in the Supplementary Material E.T
Table 4: Defense against enhanced attacks (CA+ / ASR./PA
Attack
Original Trigger
ShrinkPad
Blur
7IP (BayreGry
7IP (Blur)
7IP (Grayscale)
BadNets
69.32/10.34/70.44 80,58/31.417000 82.36/24.0076.05
58.01/79 58.01/79.89/28.02
12.39/18.47/7.22
63.26/73.52/32.91
Blended
85.42/21.83448.0
75. 10/1 13
78.47/63.61/3700
75.79/75.21/33.01
81 81.63/28.83/51.3 1.32 73.80/40.11/36.22
During the inference stage, we continue to use the original trigger pattern for attacks. From the experimental results in Table 4 we observe that the original trigger pattern no longer triggers the attack, but if the purification (Blur+Grayscale) we use has the same transformation as the attacker, the attacks can still be triggered. However, by switching to a different transformation, such as solely Blur or Grayscale, the attacks can be effectively mitigated. It is important to note that in practical scenarios, the attacker may not know which transformation the defender using, therefore the defender should consider using diverse transformations to enhance the defense ability.

---

4.4.3 Evaluation on Purification Speed
We analyze the computational costs of classification and purifi- cation before and after applying speed-up strategies. Figure presents the average time required per image in C0F Our defense method's s efficiency can enhanced with the intro- duction of our proposed tiling and letection-based strategies, resulting in 33x and 39 x speedup, respectively. The results also show that ZIP can complete the purification step faster than the classification step under various scenarios, highlight- ing its efficiency. In addition, than another inference time purification model BDMAE [53]. More details about the speed-up up settings provided in the Supplementary Material F.2
Clastiicatio
0.0388
*Transformation
Olgatraation
Dcatinn 100M
0.8666
9.2095
method is three times faster
second[s] 6
0.5
2.0
Figure Inference time with dif- ferent purification on CIFAR-10.
Related Work: Backdoor Defense
In this section, briefly review backdoor defenses here and provide more detailed discussion in the Appendix G Existing backdoor defense methods are mainly designed for white-box models [32 61 However, these methods [47 10) often require access model parameters or original training data, which is not always feasible in real-world scenarios. To address this challenge, existing black-box defense methods are proposed and can be roughly divided into backdoor detection and backdoor purification. Detection models [8 poisoned images for further inference as defense mechanism. However, this approach can limit the usefulness of these methods in practical settings where users expect results for all of their test samples. On the other hand, backdoor purification methods aim to remove the poison effect from the image defend against attacks. Some methods [42 [33 in this category involve masking the potentially poisoned region and then reconstructing the masked image to obtain poison-free image. However, these strategies may fail when the trigger patterns are distributed throughout the image, rather than in a specific patch-based location [ 43 68 377. Another approach [35] involves applying strong image transformations to the test image to destruct the trigger pattern. While such methods can defend against more advanced attacks, they typically result in a decrease in classification accuracy.
19 65
12
38
[4]
56
16 aim identify and reject any detected
Conclusion
We propose a novel framework called ZIP for defending against backdoor attacks in black-box settings. Our method involves applying strong transformations to the poisoned image to destroy the trigger pattern. It then leverages pre-trained diffusion model recover the removed semantic information while maintaining the fidelity of the purified images. The experiments demonstrate the effectiveness of ZIP in defending against various backdoor attacks, without requiring model internal information or any training samples. ZIP also enables end-users to utilize full test samples, even when using attacked classification model. Some future directions include designing black-box defense for other data domains and exploring other types of diffusion models.
Acknowledgments and Disclosure of Funding
The work is. in part, supported by NSF (#2223768. #2310261). The views and conclusions this paper are those of the authors and should not be interpreted as representing any funding agencies.
10

---

References
[1] Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 18208-18218, 2022.
[2] Ruisi Cai, Zhenyu Zhang, Tianlong Chen, Xiaohan Chen, and Zhangyang Wang. Random- ized channel shuffling: Minimal-overhead backdoor attack detection without clean datasets. Advances in Neural Information Processing Systems, 11:33876-33889 2022.
[3] Nicholas Carlini and Andreas Terzis. Poisoning and backdooring contrastive learning. arXiv preprint arXiv:2106.09665 096676 2021.
[4] Shuwen Chai and Jinghui Chen. One-shot neural backdoor erasing via adversarial weight masking. arXiv preprint (.Xxx:2207.04499 2022.
[5] Kangjic Chen, Xiaoxuan Lou, Guowen Xu, Jiwei Li, and Tianwei Zhang. Clean-image backdoor: Attacking multi-label models with poisoned labels only. In The Eleventh International Conference on Learning Representations.
[6] Kangjie Chen, Yuxian Meng, Xiaofei Sun, Shangwei Guo, Tianwei Zhang. Jiwei Li, and Chun Fan. Badpre: Task-agnostic backdoor attacks to pre-trained nlp foundation models. arXiv preprint I 2021.
[7] Weixin Chen, Baoyuan Wu, and Haoqian Wang. Effective backdoor defense by exploiting sensitivity of poisoned samples. Advances in Neural Information Processing Systems, 35:9727- 9737, 2022.
[8] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.055526 2017.
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
[10] Bao Gia Doan, Ehsan Abbasnejad, and Damith C Ranasinghe. Februus: Input purification defense against trojan attacks on deep neural network systems. In Annual Computer Security Applications Conference, pages 897-912, 2020.
[11] Khoa Doan, Yingjie Lao, and Ping Li. Backdoor attack with imperceptible input and latent modification. Advances in Neural Information Processing Systems, 34: 18957, 2021.
[12] Yinpeng Dong, Xiao Yang, Zhijie Deng, Tianyu Pang, Zihao Xiao, Hang Su, and Jun Zhu. Black-box detection of backdoor attacks with limited information and data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16482-16491, 2021.
[13] Min Du, Ruoxi Jia, and Dawn Song. Robust anomaly detection and backdoor attack detection via differential privacy. arXiv preprint 19 2019.
[14] Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pages 1 113-125, 2019.
[15] Tianyu Gu, Brendan Dolan-Gavitt and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprini urXiv: 06777708.773,, 2017.
[16] Zihan Guan, Mengnan Du, and Ninghao Liu. Xgbd: Explanation-guide graph backdoor detection. arXiv preprint arXiv:2308. 04406, 2023.
[17] Zihan Guan, Lichao Sun, Mengnan Du, and Ninghao Liu. Attacking neural networks with neural networks: Towards deep synchronization for backdoor attacks. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM '23, page 608-618, New York, NY, USA, 2023. Association for Computing Machinery.
[18] Junfeng Guo, Ang Li, and Cong Liu. Aeva: Black-box backdoor detection using adversarial extreme value analysis. International Conference Learning Representations, 2022.
11

---

[19] Junfeng Guo, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, and Cong Liu. Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency. The Fleventh International Conference on Learning Representations, 2023.
[20] Jonathan Hayase and Sewoong Oh. Few-shot backdoor attacks via neural tangent kemelss. arXiv preprint arXiv:2210.05992, 2022.
[21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[22] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840- 181840-6855,, 2020.
[23] Sanghyun Hong, Nicholas Carlini, and Alexey Kurakin. Handcrafted backdoors in deep neural networks. Advances Neural Information Processing Systems, 35:8068
18888688800,,,, 8080, 2022.
[24] Jeremy Howard. Imagenette.
com http://www..ooaaaaaaaaaaaaaagaae
2019.
[25] Xiaoling Hu, Xiao Lin, Michael Cogswell, Yi Yao, Susmit Jha, and Chao Chen. Trigger hunting with topologiaal prior for trojan detection. arXiv preprint arXiv:2110.0833,, 2021.
[26] Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. arXiv preprint arXhv:2202.03433, 2022.
[27] Charles Jin, Melinda Sun, and Martin Rinard. Incompatibility clustering as a defense against backdoor poisoning attacks. In The Eleventh International Conference on Learning Representa-
tions.
[28] Patanjali Kashyap. Machine learning for decision makers: Cognitive computing fundamental for better decision making. Springer, 2017.
[29] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 2022.
[30] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[31] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor learning: Training clean models on poisoned data. Advances in Neural Information Processing Systems, 34:14900-14912, 2021.
[32] Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[33] Yiming Li, Mengxi Ya, Yang Bai, Yong Jiang, and Shu-Tao Xia. Backdoorbox: A python toolbox for backdoor learning. arXiv preprinl arXiv:2302.01762. 2023.
[34] Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor attack in the physical world. arXiv preprint arXiv:2104.02360 2021.
[35] Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang. Zhifeng Li, and Shutao Xia. Rethinking the trigger of backdoor attack. arXiv preprint 27Xi:20004004692, 2020.
[36] Yiming Li, Haoxiang Zhong, Xingjun Ma, Yong Jiang, and Shu-Tao Xia. Few-shot backdoor attacks on visual object tracking. arXiv preprint urXiv:2201.13/76 2022.
[37] Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 16463-16472, 2021.
[38] Guanxiong Liu, Abdallah Khreishah, Fatima Sharadgah, and Issa Khalil. An adaptive black-box defense against trojan attacks (trojdef). IEEE Transactions on Neural Networks and Learning Systems, 2022.
12

---

[39] Ninghao Liu, Mengnan Du, Ruocheng Guo, Huan Liu, and Xia Hu. Adversarial attacks and defenses: An interpretation perspective. ACM SIGKDD Explorations Newsletter, 2361);86- 2021.
[40] Yepeng Liu, Bo Feng, and Qian Lou, Trojtext: Test-time invisible textual trojan insertion. arXiv preprint arXiv: 02244, 2023.
[41] Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang Abs: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 1265 1282, 2019.
[42] Brandon B May, Norman Joseph Tatro, Piyush Kumar, and Nathan Shnidman. Salient condi- tional diffusion for backdoors. In ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning.
[43] Anh Nguyen and Anh Tran. Wanet-imperceppibbec warping-basce backdoor attack. International Conference on Learning Representations, 2021.
[44] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. arXiv preprint arXiv:2205.074660 2022.
[45] Ziad Obermeyer and Ezekiel J Emanuel. Predicting the future-big data, machine learning, and clinical medicine. The New England journal of medicine, 375(13):1216, 2016.
[46] Xiangyu Qi, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. Revisiting the assumption of latent separability for backdoor defenses. In The eleventh international conference learning representations, 2023.
[47] Han Qiu, Yi Zeng. Shangwei Guo, Tianwei Zhang, Meikang Qiu, and Bhavani Thuraisingham. Deepsweep: An evaluation framework for mitigating dnn backdoor attacks using data augmen- tation. In Proceedings of the 2021 ACM Asia Conference Computer and Communications Security, pages 363-377, 2021.
[48] Johannes Schwab, Stephan Antholzer, and Markus Haltmeier. Deep null space learning for inverse problems: convergence analysis and rates. Inverse Problems, 35(2):225008, 2019.
[49] Guangyu Shen, Yingqi Liu, Guanhong Tao, Qiuling Xu, Zhuo Zhang, Shengwei An, Shiqing Ma. and Xiangyu Zhang. Constrained optimization with dynamic bound-scaling for effective nlp backdoor defense. In International Conference on Machine Learning, pages 19879-19892. PMLR, 2022.
[50] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. International Conference on Learning Representations, 2021.
[51] Hossein Souri, Liam Fowl, Rama Chellappa, Micah Goldblum, and Tom Goldstein. Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. Advances in Information Processing Systems, 15:19155--9178 2022.
[52] Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks, 32:323-332, 2012.
[53] Tao Sun, Lu Pang. Chao Chen, and Haibin Ling. Mask and restore: Blind backdoor defense at test time with masked autoencoder. arXiv preprint arXiv:2303. 15564, 2023.
[54] Ruixiang Tang. Mengnan Du, Ninghao Liu, Fan Yang. and Xia Hu. An embarrassingly simple approach for trojan attack in deep neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 218-228. 2020.
[55] Guanhong Tao, Yingqi Liu, Guangyu Shen, Qiuling Xu, Shengwei An, Zhuo Zhang, and Xiangyu Zhang. Model orthogonalization: Class distance hardening in neural networks for better security. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1372-1389. IEEE, 2022.
13

---

[56] Sakshi Udeshi, Shanshan Peng. Gerald Woo, Lionell Loh, Louth Rawshan, and Sudipta Chat- topadhyay. Model agnostic defence against backdoor attacks in machine learning. IEEE Transactions on Reliability, 71(2):880-88) 895, 2022.
[57] Haotao Wang, Junyuan Hong, Aston Zhang, Jiayu Zhou, and Zhangyang Wang. Trap and replace: Defending backdoor attacks by trapping them into an casy-to-replace subnetwork. arXiv preprint arXiv:2210. 06428, 2022.
[58] Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint 1:Xiv:221..004/4 2022.
[59] Zhenting Wang, Hailun Ding, Juan Zhai, and Shiqing Ma. Training with more confidence: Mitigating injected and natural backdoors during training. Advances Neural Information Processing Systems, 35:36396-3644 2022.
[60] Zhenting Wang, Kai Mei, Juan Zhai, and Shiqing Ma. Unicorn: A unified backdoor trigger inversion framework. arXiv preprint arXiv:2304. 02t.22224..0786, 2023.
[61] Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. Backdoorbench: A comprchensive benchmark of backdoor learning. Advances in Neural Information Processing Systems, 35: 10546-10559, 2022.
[62] Dongxian Wu and Yisen Wang. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems, 34:16913-16925, 2021.
[63] Zhen Xiang, David J Miller, and George Kesidis. Post-training detection of backdoor attacks for two-class and multi-attack scenarios. arXiv preprint arXiv:2201.08474, 2022.
[64] Yi Zeng, Si Chen, Won Park, Z Morley Mao, Ming Jin, and Ruoxi Jia. Adversarial unlearning of backdoors via implicit hypergradient. arXiv preprint arXiv:2110.03735, 2021.
[65] Yi Zeng, Won Park, Z Morley Mao, and Ruoxi Jia. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 16473-16481, 2021.
[66] Tao Zhang, Qing Li, Chang-shui Zhang, Hua-wei Liang, Ping Li, Tian-miao Wang, Shuo Li, Yun-long Zhu, and Cheng Wu. Current trends in the development of intelligent unmanned autonomous systems. Frontiers of information technology & electronic engineering, 18:68-85, 2017.
[67] Zhiyuan Zhang. Lingjuan Lyu, Weiqiang Wang, Lichao Sun, and Xu Sun. How to inject back- doors with better consistency: Logit anchoring on clean data. arXiv preprint 2rXiv:2109.011 2021.
[68] Shihao Zhao, Xingjun Ma. Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. Clean-label backdoor attacks video recognition models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14443-14452, 2020.
[69] Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jingang Wang, Wei Wu, al. Moderate-fitting natural backdoor defender for pre-trained language models. Advances in Neural Information Processing Systems, 35: 1086 1099, 2022.
14

---

A Theoretical Justification
A.1 Proof on rectified estimation of x; in Equation 7
Equation 7 X = Axx Ap - A'Ax + A'AVI- ārEt
Proof. Based on Xx 04 v1 dGe, we have nice property 22
X
(x+
(13)
vā=
To ensure that the approximated clean image Xu based on the 1-th step observation x, satisfies the constraint in Equation 6, we have:
Xoje A A'x - ATAP + -I- A Xole.
(14)
Combine the above two equations, we can have:
Vā (x. - -1- = A'x - A + AI-)
(x: - vI - āiE:)).
(15)
Vāz
Taking the derivative, we arrive at:
x = Dilx--VoAAApt (I- ATA)x + -AAVI- -GiE
(16)
A.2 Proof of Lemma 3.1
Lemma 3.1 Suppose the estimated noise output by go is Gaussian. Given go (xt,t) Es, we have gx((x; Vā,A Ap), E, + where Er, E/ are also Gaussian.
Proof. Let us define the output of go((x; + xi), 1) as é,, so we have (x(( + x)), = El. Next, we define e ~ N(Jn,of and é ~ N(u2, 03).
Since é also follows a Gaussian distribution, we can subtract from é, to obtain c such that:
é c N(J2 o oz)
(17)
This confirms that e/ is also Gaussian, thus completing the proof.
A.3 Proof of Theorem 3.2
Theorem 3.2 Suppose that go ((x; + A/ = + e/, We define the error at step between
8; and (x: + VāzAl Ap) as 8/, i.e., 8/ = 81 - (Xt + VazAl Ap). Let ò; Aö we have the following bound on its norm: ||5.1| (1- (
||||||++1I|-
VI-
Proof. We prove the above theorem by induction.
The base case is when t = T, where we have Xy (xT + a = 0. which holds.
2 Suppose for & (x+ + vo,AtAp) = Atö,, ||
(1-ār) Ja
A |llc(411 is true.
VI-de+1
Induction:
1 -VA'A--VA'AP-EE- AIAAA +AAAII -,0
04 c1) + oic,
X,-1 +
Va,
V1 04
(18)
&
1
(VTAA'
A'AA -'A
1-
(E, + (c++ + (19)
Va,
/1 a.
15

---

1 - at
- -
(VAAAAAA---AAAAAAAAAAAAAAAAA - -re
vI Ei)
Va."
(20)
By definition, we have &;-1 - (xt-1 + Val = -8-1- Let 8 = 1 we have
d -A = A (xt-1 + Von, A Ap))). Since A(T - ATA) = 0, we can have:
6 = Va,
(AVT OitY A VI āi
a4 (4)
04. Va A
(21)
/1 ai
Based the Souchy-Schward inequality:
6 (1 0z 1) Acl S
(1 - θ4 1) at
|||||| c|ll
(22)
0z
v1 αz
A.4
Proof of Corollary 3.2.1
Corollary 3.2.1 When = O, we have Xo Xn + AlAp 86. where A =0
Proof. First, we have:
1 - -0161 Vag
1 01
Xg
61 +01,, (23)
V1 al
1 01 (a+t)))+o1e, (24)
& Va
-1-
Then we can have:
A8 (1- an) 00 01 Aéi
(25)
VI σ1
Since ag = 1, then we have A8 =
16

---

B Qualitative Results of Purification
B.1 Qualitative Results of Purification on BadNet Attack
(a) Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
Attacked Image
(I) Purified Image
(g) Attacked Image
(h) Purified Image
Attacked Image
(j) Purified Image
(k) Attacked Image
(l) Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
Figure 4: Comparison of Purified and BadNet Attack Images(Partl).
17

---

Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
Attacked Image
(j) Purified Image
(k) Attacked Image
(l) Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
(q) Attacked Image
(r) Purified Image
(s) Attacked Image
(1) Purified Image
(u) Attacked Image
(v) Purified Image
(W) Attacked Image
(x) Purified Image
Figure 5: Comparison of Purified and BadNet Attack IPages
18

---

PARNEY
PREMVE
Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
AME
Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
X
Attacked Image
(i) Purified Image
(k) Attacked Image
(1) Purified Image
LEL TAVO
L TA
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
(q) Attacked Image
(r) Purified Image
(s) Attacked Image
(1) Purified Image
Sunray
U5
HHC
(u) Attacked Image
(v) Purified Image
(w) Attacked Image
(x) Purified Image
Figure Comparison of Purified and BadNet Attack Images(Part3)..
19

---

A
GULF
GULF
Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
Attacked Image
(i) Purified Image
(k) Attacked Image
(l) Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
(q) Attacked Image
(r) Purified Image
(s) Attacked Image
(1) Purified Image
(u) Attacked Image
(v) Purified Image
(w) Attacked Image
(x) Purified Image
Figure 7: Comparison of Purified and BadNet Attack Images(Part4)..
20

---

B.2 Qualitative Results of Purification on Blended Attack
Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
(c) Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
ii Attacked Image
(j) Purified Image
(k) Attacked Image
(1) Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
Figure 8: Comparison of Purified and Blended Attack Images(Partl).
21

---

Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
Attacked Image
(i) Purified Image
(k) Attacked Image
(1) Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
(q) Attacked Image
(r) Purified Image
(s) Attacked Image
(1) Purified Image
(u) Attacked Image
(v) Purified Image
(w) Attacked Image
(x) Purified Image
Figure 9: Comparison Purified and Blended Attack IPages
22

---

Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
2
Attacked Image
(i) Purified Image
(k) Attacked Image
(1) Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
(q) Attacked Image
(r) Purified Image
(s) Attacked Image
(1) Purified Image
(u) Attacked Image
(v) Purified Image
(w) Attacked Image
(x) Purified Image
Figure 10: Comparison Purified and Blended Attack Images(Part3).
23

---

Attacked Image
(b) Purified Image
(c) Attacked Image
(d) Purified Image
C
Gopoo
Attacked Image
(f) Purified Image
(g) Attacked Image
(h) Purified Image
Attacked Image
(i) Purified Image
(k) Attacked Image
(1 Purified Image
(m) Attacked Image
(n) Purified Image
(o) Attacked Image
(p) Purified Image
(q) Attacked Image
(r) Purified Image
(s) Attacked Image
(1) Purified Image
(u) Attacked Image
(v) Purified Image
(w) Attacked Image
(x) Purified Image
Figure 11: Comparison of Purified and Blended Attack Images(Part4).
24

---

C Details on Linear Transformations
In this section, we discuss details of the linear transformation applied in our paper. Two practical examples are discussed to illustrate the simplicity and effectiveness of linear transformations, along with the corresponding operators used.
To convert an RGB image to gray-scale, the operator A = 11/3,1/3, 1/3] be defined as pixel-wise operation that transforms each RGB channel pixel [r, 9, bj into gray- scale value r3 + g + +b.. In this case, constructing pseudo-inverse A [1, 1J" is straightforward,
Gray-scale Conversion:
satisfying the condition AA = I, where I represents the identity matrix.
Image blurring also involves linear transformations. For a blurring operation with scale n, the operator A is defined as the seraag-ppooling operator [1/n²,.., ...., "/.n",..,,//"..... 1/n²]. This operator aggregates each patch the image into a single value. Similarly, the pseudo-inverse A can be built as A 1, ..... 1 fulfill the condition AAt I.
Image Blurring:
Overall, these examples demonstrate how these two linear transformations, in conjunction with their respective operators, can be employed destruct the trigger pattern without relying complex Fourier transform. In cases where the linear transformation is too complex to solve for its pseudo- inverse, the Singular Value Decomposition (SVD) method can be applied. For more details, please refer to papers 58
D Algorithm for improved ZIP based on DDIM
In this section, we include the modified algorithm based on DDIM, which is proposed speed up the diffusion model inference speed.
Algorithm 2 Zero-shot Image Purification (based on DDIM)
Require: Test image for purification x": liner transformation A1 A2.... An and their pseudo- inverse A, ..., A diffusion model gi hyperparameter speed-up pace S.
Ensure: AA A,,' n 0, I....N
1: x" ~ I)
2: for = T,T 5, .... S, do
3:
E (N, if 1, else 0
4:
6 = (x,,))
5:
for 1,2, N do
6:
81 v Vo A + (I- ([-A),)) + A4A
7:
end for
8:
X N (x} + +,+ +x,)
9:
X,
{x; + āi
α²x; Xt
10: Razt = vae-
Xt /1 Ōx,Et)
11: Xx-1 /r - 1801 + /1 O:-1 of 0,E
12: end for
13: return X0
E Experiments Settings
E.1 Datasets Informaiton
CIFAR-10 (30] The CIFAR-10 dataset is widely-used benchmark in computer vision. It consists of 60,000 color images of size 32x32 pixels, belonging to 10 different classes, with 6,000 images per class. The dataset is divided into 50,000 training images and 10,000 test images, with balanced distribution classes.
The German Traffic Sign Recognition Benchmark (GTSRB) dataset is designed for traffic sign classification tasks. comprises more than 50,000 images of traffic signs captured under various real-world conditions. The images have different sizes and aspect ratios, but they resized
GTSRB S2
25

---

to 32x32 pixels for our model training and evaluation. The dataset is divided into training and test sets, with its official split ratio.
Imagenette [24] The Imagenette a subset of the larger ImageNet dataset and is commonly used smaller-scale alternative for image classification tasks. It consists of 10 classes with total of 13,000 images. The images in Imagenette have varying sizes, but they are resized to 256x256 pixels for consistency. The dataset split into training and validation sets, following predefined split ratio.
Table 5: Properties of datasets.
Dataset
Classes
Image Size
Train Split
Test Split
CIFAR-10
10
32x32
50,000
10,000
GTSRB
43
32x32
39,209
12,630
Imagenette
10
256x256
9,480
3,936
E.2 Attacks Implementation
In this section, we discuss the implementation details of three different backdoor attack methods employed in our study: BadNet, Blended, and PhysicalBA. We implement these backdoor attacks using the Backdoorbox framework
33 which under GNU general public license.
BadNet [5] The BadNet attack injects specific trigger patterns into the training data. In our implementation, we set the poisoned rate to 5%, i.e., 5% of the training samples are selected as attack samples and have the trigger pattern added to them. The trigger pattern size is set to 2x2 for 32x32 pixels images and 9x9 for 256x256 pixels images. The trigger patterns are randomly generated.
The Blended attack is a more sophisticated variant aimed at making the backdoor less conspicuous and harder detect. Following the suggestion in BackdoorBox, to 0.2 and the poisoned rate to 5%. The blended pattern randomly generated, seamlessly blending the trigger pattern into the attack samples.
Blended [8]
the blended rate
The PhysicalBA (Physical Backdoor Attack) is a specific type of attack that introduces variations in the location and appearance of the attack pattern embedded in the test samples during inference time. our implementation, we apply the same attack pattern size as the BadNet attack, using 2x2 pattern for 32x32 pixels images and 9x9 pattern for 256x256 pixels images. The attack patterns are generated randomly. We set the poisoned rate to 5% for this attack.
PhysicalBA 34
All other attack settings follow the default configurations in Backboorbox 33
E.3 Purification Implementation
We utilize a pre-trained model provided by OpenAl under the MIT license. The algorithm described in Algorithm is employed to accelerate the inference process, allowing to generate high-quality images within just 20 steps, and the speed-up pace is set to 50. We set the hyperparameter À to value of for Blended attack defense, and 10 for BadNet and PhysicalBA attack defense.
Specifically, for the CIFAR-I0 dataset, we apply both blur and gray-scale conversion as linear trans- formations. For the GTSRB and Imagenette datasets, we solely apply blur as the linear transformation. Additional implementation details can be found in the code we have provided.
Ablation Study Settings
F. Enhanced Attack Settings
In the enhanced attack settings, our first step is to extract 5% of the training dataset and inject the attack's trigger pattern into these images. We then proceed to purify this subset of data using blur and grayscale as linear transformations during the first stage of our proposed purification. Once the attacked images have been successfully purified, we modify their labels to reflect the attack label. Following this, we introduce these purified images poisoned samples into the training set and train a classification model from scratch. This comprehensive procedure is referred to as the enhanced attack process.
26

---

F.2 Purification Speed Seftings
This paper focuses on defending against backdoor attacks during the inference phase using purification techniques. To evaluate the purification speed, we conduct experiments using a workstation that features an Intel(R) Core(TM) 19- 10900X CPU and an NVIDIA RTX3070 GPU with 8GB of memory.
During our experiments, we measure the classification time, which represents the duration taken by the classifier model to perform inference a single image. Additionally, measure the purification time, which indicates the time required by the purification model to purify a single image.
For the combination of purification with detection, we utilize the Scale-up method [9] our chosen detection technique. Furthermore, the dataset used for speed evaluation consists of 5% poisoned images. Following previous settings [19), we set batch size of one for the classifier model and report the average time based on 640 runs.
G Related Work
G.1 Backdoor Attack
Existing backdoor attack methods involve the injection poisoned samples into the training process of Deep Neural Networks (DNNs). These attacks can target various types of models, including image classification models, object detection models S 36.. contrastive learning models [3] and language models 44 49 69 G The attackers exploit vulnerabilities by embedding adversary-specifee
trigger patterns into carefully selected benign samples. Backdoor attacks are characterized by their stealthiness, as the attacked models behave normally on benign samples, making the hidden triggers difficult detect and purify.
There are mainly two categories of backdoor attacks for image classification tasks: patch-based and non-patch-based attacks. Patch-based attacks attacks with triggers embedded as patches or overlays within the input samples. For example, Souri et al. [51] propose the Sleeper Agent attack, which is sophisticated backdoor attack where an adversary subtly injects hidden triggers into an image classification model during training, remaining dormant until specific conditions activate malicious behavior. Non-patch-basse explicit patching, often relying on specific input sequences or subtle modifications in the feature space 677 23 20). For example, Doan et al. [ introduce Wasserstein backdoor attack, an extension of the imperceptible backdoor concept to the latent representation. Their proposed attack manipulates inputs with imperceptible noise, matching latent representations to achieve high attack success rates while remaining stealthy in both the input and latent spaces.
attacks are attacks where triggers are integrated without
G.2 Backdoor Defense
Existing defense methods for backdoor models can be broadly categorized into two approaches: (1) detection-based methods and (2) purification-based methods.
Detection- based methods focus on identifying the presence of backdoors in trained models. These methods typically involve analyzing the model's behavior and inputs to detect any suspicious patterns or triggers that indicate the existence of a backdoor 22. Various techniques such as anomaly detection [13 25 63), and statistical analysis [18] 7 have been employed to detect backdoors. The goal of detection based methods is to provide an carly warning system to identify and mitigate the risks posed by backdoor attacks.
On the other hand, purification-batt methods aim to remove or neutralize the effects of backdoors from the model. These methods involve modifying the model or its training process to eliminate the influence of the backdoor triggers on the model's behavior (26 [3]. Some purification approaches focus on retraining the model using clean or carefully selected training data to reduce the impact of
the backdoor (64 59 31
27 55 Other methods aim to directly identify and neutralize the backdoor
triggers within the model's 's parameters
hidden representations [57 62 60 41 The objective of
purification- based methods to restore the integrity and reliability of the model by eliminating the malicious behavior induced by the backdoor.
27

---

H
Comparison Results with Image Restoration/Purifcatioo Methods
In this section, we compare our proposed ZIP with existing purification and restoration methods, including DiffPure [4] and DDNM [58]. Specifically, DiffPure is a state-of-the of-the-art purification method designed for adversarial attacks, while DDNM a state-of-the-and image restoration technique to repair corrupted images.
H.1 Quantitative Results Comparison with Image Restoration/Purltiattoon Methods
We first conduct quantitative analysis of the defense performance between our proposed method and DiftPure and DDNM. In order to ensure fair comparison, we implemented DiftPure and DDNM using their official code and applied identical linear transformations, diffusion steps, and schedules both methods. The defense performance is listed in Table 6
Table 6: Defense performance Imagenette.
Attack
No Defense
DifiPure
Biut-DiftPure
BlurADDNM
ZIP (Ours)
CA
ASR
PA
CA
ASR
PA
CA
ASR
PA
CA
ASR
PA
CA
ASR
PA
BadNer
94.53
14.98
75.13
74.31
84.56
Blended
86.14
99.85
10.19
80.63
4382
56.68
73,89
1333
73.98
85.37
93.37
15.46
81.42
8.35
78.36
We can first observe that our ZIP performs better than DifiPure and Blur-Dif in all three metrics regarding defense performance. This is because our ZIP can recover the semantic information removed by the Blur, while Blur-DiffPure can not. Specifically, DiftPure aims remove attack patterns by first diffusing images with noise and then recovering images through an unconditional reverse process. While coupling DiffPure with Blur enhances its trigger removal capability (lower ASR), its unconditional generative process fails to effectively recover the semantic information removed by Blur. This leads drop in clean accuracy (CA). In the above table, Blur+DiffPure exhibits poorer CA compared to both DiffPure and our ZIP. In contrast, our ZIP, after Blur, utilizes conditional reverse process to recover semantic information deleted by Blur through RND theory. Therefore, our ZIPs can outperform DiffPure.
We can also observe that DDNM shows better clean accuracy than ZIP, but it cannot effectively defend against backdoor attacks like Blended. This is because DDNM restores the attack patterns in its diffusion process, which decreases the defense performance.
H.2 Qualitative Results Comparison with Image Restoration Methods
In this subsection, we conduct a qualitative analysis of the purification effect between our proposed method and DDNM.
28

---

(a) Attack Image
(b) Restored by DDNM
(c) Purified by ZIP
(d) Attack Image
(c) Restored by DDNM
(f) Purified by ZIP
(g) Attack Image
(h) Restored by DDNM
ii) Purified by ZIP
Figure 12: Comparison of DDNM and ZIP on defending Blended attack.
29

---

(a) Attack Image
(b) Restored by DDNM
(c) Purified by ZIP
(d) Attack Image
(c) Restored by DDNM
(f) Purified by ZIP
Attack Image
(h) Restored by DDNM
Purified by ZIP
Figure 13: Comparison of DDNM and ZIP on defending BadNet attack.
30

---

I Limitations
Due to our reliance on a pre-trained diffusion model implement zero-shot purification, the effec- tiveness of generating purified images may be weakened when our model is applied to highly specific images that fall outside the distribution of pre- pee-preesssd data. To mitigate this issue, we suggest two possible solutions in future work: 1) replacing the current pre-trained diffusion model with more suitable pre-trained model for such specific images, and 2) collecting a subset of highly specific images to perform fine-tuning on the pre-trained model.
31


Reviews:
[{"review_id": "ngC5vjas4a", "reviewer": "Reviewer_L743", "summary": "The work deals with the challenge of defending against backdoor attacks for black-box models in zero-shot settings using pre-trained diffusion models. Based on some theoretical insights, the proposed method modifies the reverse process of the diffusion model so the recovered images can be more high-fidelity and clean. Experimental results show that the proposed framework is effective in defending against backdoor attacks without prior knowledge of the model or additional retraining.", "questions": "- Regarding the setting of 4.4.2, if the attacker assumes that the defender is using Blur for transformation, will switching to a different transformation, such as Grayscale, be effective?\n    \n- The statement \"even when using an attacked model as the classifier\" on L66 is unclear. Could you provide more context or clarification for this scenario?", "limitations": "The authors have discussed the limitations of their work. Please see the Weaknesses stated above to see other limitations I find.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The model-agnostic nature of the proposed method enhances its practicality as it can be adopted by any model without retraining.\n    \n- The theoretical analysis is clear and comprehensive.\n    \n- The additional techniques for speeding up the algorithm make the framework more realistic and applicable.", "weaknesses": "- Previous works have shown that Diffusion models can be used for image purification [1, 2, 3]. Thus, the paper’s novelty primarily lies in the adaptation of the traditional diffusion model for high-fidelity image recovery. While these improvements are theoretically driven, empirical evidence demonstrating the effectiveness of these enhancements or validating the proposed theories is lacking.  \n    \n- The experiments adopt three backdoor attacks: BadNet, PhysicalBA, and Blended. But more advanced attacks [4, 5] have been proposed and adopted in recent backdoor defense literature [6].\n    \n\n- There is an inconsistency in the statement on L344-345: “However, by switching to a different transformation, such as solely Blur or Grayscale, the attacks can be effectively mitigated.” ([pdf](zotero://open-pdf/library/items/HJZXQTNN?page=9)) In contrast, Table 4 reveals subpar defense performance when solely employing Grayscale.\n    \n\n[1] Nie et al. Diffusion Models for Adversarial Purification.\n\n[2] Wang et al. Guided Diffusion Model for Adversarial Purification.\n\n[3] Sun et al. PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognition.\n\n[4] Bagdasaryan and Shmatikov. Blind Backdoors in Deep Learning Models.\n\n[5] Nguyen and Tran. WaNet -- Imperceptible Warping-based Backdoor Attack.\n\n[6] Doan et al. Defending Backdoor Attacks on Vision Transformer via Patch Processing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***W1: Empirical evidence supporting the proposed enhancements to [1,2,3] and validating theories is currently lacking.***\n\nThanks for the thoughtful question. We would like to clarify the novelty and contribution of our work as follows. \n\n1. **An effective black-box defense against backdoor attacks:**  Existing methods [1, 2, 3] rely on the denoising capability of diffusion models (e.g., DDPM), which is limited against backdoor patterns (e.g., patch-based patterns) as demonstrated in [4]. In contrast, our method focuses on defending against backdoor attacks, especially under **the novel and challenging black-box scenario**.\n2. **A novel reverse diffusion process**: Previous methods [1, 3] employ naive diffusion models that could produce unpredictable purified images, while we innovatively integrate RND theory into the reverse diffusion process, ensuring high-fidelity recovery of images and eliminating backdoor patterns. To empirically demonstrate the advantages, we implement DiffPure [1] and its combination with Blur transformation (Blur+DiffPure) as baselines. As shown in the table below, empirical results show ZIP performs better than DiffPure and Blur+DiffPure.\n\n3. **Linear transformation for backdoor destruction:** Linear transformation is a basic step in the RND theory. We explore the effectiveness of linear transformations, such as Blur and Grayscale, in destroying backdoor patterns. Empirical results of linear transformation effectiveness are provided in Sec 4.4.1.\n\n|                        | No Defense        | DiffPure          | Blur+DiffPure     | ZIP (Ours)           |\n| ---------------------- | ----------------- | ----------------- | ----------------- | -------------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑    |\n| BadNet                 | 84.99/94.53/14.98 | 78.85/91.41/18.11 | 75.13/10.16/74.31 | **84.05/7.55/83.97** |\n| Blended                | 86.14/99.85/10.19 | 80.63/43.82/56.68 | 75.89/13.53/73.98 | **81.42/8.35/78.36** |\n\n[1] Diffusion models for adversarial purification. ICML, 2022.    \n[2]  Guided diffusion model for adversarial purification. arXiv, 2022.\n[3] Pointdp: Diffusion-driven purification against adversarial attacks on 3d point cloud recognition.  arXiv, 2022.\n[4] DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks in the Physical World.  arXiv, 2023.\n\n***W2: Adopt more recent attacks like Blind attack and WaNet attack .***\n\nFor more recent attacks, we have conducted experiments with Blind [1], WaNet [2], and Label-Consistence [3] on Imagenette datasets to further demonstrate our defense effectiveness. The quantitative results are listed below, and the qualitative results are provided in the rebuttal PDF. The results show that our ZIP can successfully defend against various advanced attacks.\n\n|                        | No Defense        | ShrinkPad (defense) | Blur (defense)    | ZIP (Ours)           |\n| ---------------------- | ----------------- | ------------------- | ----------------- | -------------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑   | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑    |\n| WaNet                  | 85.45/97.93/11.84 | 73.63/26.87/60.35   | 62.98/17.88/56.81 | **79.13/7.77/76.78** |\n| Blind                  | 79.15/99.64/10.42 | 65.22/6.34/65.60    | 75.26/28.12/67.54 | **78.80/5.32/79.03** |\n| LabelConsistent        | 77.57/73.17/36.56 | 66.14/0.25/64.53    | 50.36/0.28/50.14  | **73.63/0.12/74.26** |\n\n[1] Blind backdoors in deep learning models. USENIX Security, 2021.  \n[2] Wanet--imperceptible warping-based backdoor attack.  ICLR, 2021.  \n[3] Label-consistent backdoor attacks. arXiv 2019.  \n\n***W3: Inconsistency noted in L344-345***\n\nIn Table 4, the effectiveness of Grayscale is shown in its acceptable performance against enhanced Blended attacks. However, we agree that GrayScale shows subpar performance in some cases. To avoid further confusion, we will revise the expression in the next version. Thank you for pointing this out.\n\nMoreover, we would like to emphasize that Table 4 is the result of **enhanced attacks**, where the purified poison images are selected as new attack images to inject backdoors. Existing backdoor defense methods [1] fail to defend against such enhanced attacks. In comparison, our proposed ZIP can defend against the enhanced attack if proper transformation is applied. \n\n[1] Rethinking the Trigger of Backdoor Attack. arXiv, 2021\n\n***Q1: Regarding the setting of 4.4.2, if the attacker assumes that the defender is using Blur for transformation, will switching to a different transformation, such as Grayscale, be effective?***\n\nYes, the defense is expected to be effective when the attacker uses grayscale to enhance their attack while the defender uses Blur. As shown in Table 4, the defense is effective when the attacker enhances their attack with different transformations as the defender. In practical scenarios, the defender's transformation can vary and be hidden from potential attackers. This characteristic enhances the robustness of our defense.\n\n***Q2: The statement \"even when using an attacked model as the classifier\" on L66 is unclear. Could you provide more context or clarification for this scenario?***\n\nThe “attacked model” is a classifier already poisoned by backdoor attacks (e.g., BadNets). We use the attacked model as the downstream classifier following the typical settings in previous work on purification-based defense [1, 2, 3]. We will revise the expression in the next version to avoid confusion.\n\n[1] Backdoor attack in the physical world. ICLR workshop, 2021.  \n[2] Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems.  ACSAC, 2020.  \n[3] Deepsweep: An evaluation framework for mitigating DNN backdoor attacks using data augmentation. ASIA CCS, 2021.  "}, {"review_id": "oNwZWU7MNj", "reviewer": "Reviewer_mm1a", "summary": "This paper proposes a novel backdoor defense framework called \"zero-shot image purification\" (ZIP) designed to protect against backdoor attacks on real-world black-box models. The proposed ZIP framework consists of two steps: a linear transformation applied to the poisoned image to remove the backdoor pattern and a pre-trained diffusion model used to recover missing semantic information. The reverse process generates high-fidelity purified images without requiring internal information about the poisoned model. The ZIP framework is evaluated on various datasets with different attack types and outperforms state-of-the-art backdoor defense methods. The results are expected to offer valuable insights for future defense methods for black-box models.", "questions": "Please see weaknesses for details.", "limitations": "No potential negative societal impact.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed idea is somewhat novel since the authors leverage the diffusion model to recovery the images. There have been similar works using auto-encoder[1] to reconstruct images to remove backdoor triggers but the exploration of diffusion model is not studied before.\n\n2. The authors provide theoretical analysis of how to modeling image purification with diffusion model.\n\n3. The proposed ZIP shows superior performance than existing defenses.\n\n4. The paper is well-written and easy to follow.\n\n[1]. Y. Liu, Y. Xie, and A. Srivastava, “Neural trojans,” in ICCD, 2017.", "weaknesses": "1. The idea of destroying backdoor trigger and reconstructing the images is very similar to pre-processing defenses.  Both defenses aim to remove the backdoor trigger by reconstructing the images.\n\n2. The threat model is not well defined in the paper. The capabilities of defenders and attackers are unclear.\n\n3. The backdoor attacks evaluated in this paper are developed 4-5 years ago, which is a bit too simple and outdated. Many advanced backdoor attacks such as hidden trigger backdoors [2], WaNet [3], LIRA [4], etc. are proposed after that. It would be desirable to evaluate the ZIP against more recent and advanced attacks.\n\n[2]. A. Saha, A. Subramanya, and H. Pirsiavash, “Hidden trigger backdoor attacks,” in AAAI, 2020.\n[3]. T. A. Nguyen and A. T. Tran, “Wanet-imperceptible warping-based backdoor attack,” in ICLR, 2021.\n[4]. K. Doan, Y. Lao, W. Zhao, and P. Li, “Lira: Learnable, imperceptible and robust backdoor attacks,” in ICCV, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***W1: The idea of destroying backdoor trigger and reconstructing the images is very similar to pre-processing defenses. Both defenses aim to remove the backdoor trigger by reconstructing the images.***\n\nThe contribution of our work is different from existing pre-processing defenses [1,2,3,4,5] in the following aspects:\n\n1. **Zero-shot Capability:** Unlike [1, 2, 3], which requires clean images as training data, our method does not require any prior knowledge of clean/poisoned images.\n2. **Black-Box Adaptability:** In contrast to [2, 4], which requires model internal information (e.g. gradient, weights), our method only requires model output.\n3. **Enhanced Semantic Information Preservation:** In comparison to [5], our method can better preserve semantic information as demonstrated in Table 1.\n4. **Theoretical Justification:** We provide theoretical analyses that justify the efficacy of our RND-based purification approach.\n\nOverall, although sharing similarities with pre-processing defenses, our ZIP framework can work in more challenging and realistic scenarios.\n\n[1]. Y. Liu, Y. Xie, and A. Srivastava, “Neural trojans,” ICCD, 2017.  \n[2] H. Qiu, Y. Zeng, S. Guo, T. Zhang, M. Qiu, and B. Thuraisingham, \"Deepsweep: An evaluation framework for mitigating DNN backdoor attacks using data augmentation.\" ASIA CCS, 2021.  \n[3] S. Udeshi, S. Peng, G. Woo, L. Loh, L. Rawshan, and S. Chattopadhyay, \"Model agnostic defense against backdoor attacks in machine learning.\"  IEEE Transactions on Reliability, 2022.  \n[4] B. G. Doan, E. Abbasnejad, and D. C. Ranasinghe, \"Februus: Input purification defense against trojan attacks on deep neural network systems.\" ACSAC, 2020.  \n[5] Y. Li, T. Zhai, Y. Jiang, Z. Li, and S.-T. Xia, \"Backdoor attack in the physical world.\" ICLR workshop, 2021.  \n\n***W2: The threat model is not well defined in the paper. The capabilities of defenders and attackers are unclear.***\n\nThanks for raising this question. We focus on backdoor defense in the black-box setting, where the **defender** has only access to the poisoned model output, without access to the model’s internal parameters or training datasets. On the other hand, the **attackers** can access and modify model internal components, training datasets, or any other necessary information to implement their attacks. We will add a paragraph introducing the threat model in the next version.\n\n***W3: The backdoor attacks evaluated in this paper are developed 4-5 years ago, which is a bit too simple and outdated. Many advanced backdoor attacks such as hidden trigger backdoors [2], WaNet [3], LIRA [4], etc. are proposed after that. It would be desirable to evaluate the ZIP against more recent and advanced attacks.***\n\nFor more advanced attacks, we conduct experiments with the **WaNet [1], Blind [2], and Label-Consistence attack [3]** on Imagenette datasets to further demonstrate our defense effectiveness. The quantitative results are listed below, while the qualitative results are provided in the rebuttal pdf. The results show that our ZIP can successfully defend against advanced attacks.\n\n|                        | No Defense         | ShrinkPad (defense) | Blur (defense)    | ZIP (Ours)           |\n| ---------------------- | ------------------ | ------------------- | ----------------- | -------------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑  | CA ↑/ ASR ↓/ PA ↑   | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑    |\n| WaNet                  | 85.45 /97.93/11.84 | 73.63 /26.87/60.35  | 62.98/17.88/56.81 | **79.13/7.77/76.78** |\n| Blind                  | 79.15/99.64/10.42  | 65.22/6.34/65.60    | 75.26/28.12/67.54 | **78.80/5.32/79.03** |\n| LabelConsistent        | 77.57/73.17/36.56  | 66.14/0.25/64.53    | 50.36/0.28/50.14  | **73.63/0.12/74.26** |\n\n[1] Nguyen, Anh, and Anh Tran. \"Wanet--imperceptible warping-based backdoor attack.\"  ICLR, 2021.  \n[2] Bagdasaryan, Eugene, and Vitaly Shmatikov. \"Blind backdoors in deep learning models.\" USENIX Security, 2021.  \n[3] Turner, Alexander, Dimitris Tsipras, and Aleksander Madry. \"Label-consistent backdoor attacks.\" arXiv 2019.  "}, {"review_id": "0OubCaWMr0", "reviewer": "Reviewer_xL3D", "summary": "This work proposes backdoor defense for scenarios in which a defender does not need to access the internal model, a.k.a., the black-box backdoor defense. Firstly, a linear image transformation is used to destroy potential trigger patterns. Then, a pre-trained diffusion model is used to reconstruct the missing information induced in the linear transformation procedure, where a reverse process was designed to satisfy the conditional image generation/reconstruction. Defense experiments were conducted against three backdoor attacks to show its defensive effects.", "questions": "I listed the questions in Weakness.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper writes well and is easy to follow.\n2.\tThe reported results indicate the effectiveness of the proposed method.\n3.\tInstead of directly applying diffusion model, adaptive reverse process was proposed.\n", "weaknesses": "1.\tThough it could find certain applications, the concept of black-box defense does not seem very practical and hence not quite attractive to me.\n2.\tFor the image reconstruction stage, there already exist a vast amount of advanced image restoration methods, such as diffusion model based e.g. DDNM in [1] or Restormer in [2]. I am wondering how would these pretrained models work to replace the reconstruction process in the second stage.\n3.\tThe authors used zero-shot image purification, however, it depends on a pretrained diffusion model. Have the authors evaluated how would the performance change if using a different diffusion model, or training on a different dataset? \n3.\tThe only baseline method was from 2020, which is too old in my opinion. More recent baselines should be evaluated.\n\nReferences\n\n[1] Wang, Yinhuai, Jiwen Yu, and Jian Zhang. \"Zero-shot image restoration using denoising diffusion null-space model.\" arXiv preprint arXiv:2212.00490 (2022).\n\n[2] Zamir, Syed Waqas, et al. \"Restormer: Efficient transformer for high-resolution image restoration.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: The concept of black-box defense does not seem very practical and hence not quite attractive to me.**\n\nWe appreciate your concerns and wish to highlight that **the black-box defense setting has been widely explored[1, 2, 3, 4]**. In applications like fraud detection, organizations (e.g., financial institutions) often purchase machine learning services from vendors or directly exploit third-party DNNs downloaded online. These systems could harbor backdoors introduced by malicious code or data poisoning[5, 6]. Due to intellectual property concerns, these systems are typically black-box to end-users, restricting access only to query-based APIs, thus posing challenges to end-users effective defense strategies. In these scenarios, our ZIP model can effectively serve as a \"firewall,\" blocking and purifying malicious samples.\n\nIn addition to its black-box capabilities, our framework is also **model-agnostic** and **zero-shot**. “Model-agnostic” means our framework enables seamless adaptation to new downstream classifiers without retraining, as demonstrated with VGG-16 and ResNet-34 below. “Zero-shot” means our ZIP does not require any prior knowledge of poisoned images, making it more practical since new attacks always emerge while defenders may not have access to new attack samples.\n\n|                        | No defense (ResNet-34) | ZIP (ResNet-34) | No defense (VGG-16) | ZIP (VGG-16 )  |\n| ---------------------- | ------------------------- | ------------------ | ---------------------- | ----------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑         | CA ↑/ ASR ↓/ PA ↑  | CA ↑/ ASR ↓/ PA ↑      | CA ↑/ ASR ↓/ PA ↑ |\n| BadNet                 | 84.99/94.53/14.98         | 84.05/7.55/83.97   | 70.24/91.67/17.35      | 68.71/9.72/69.09  |\n| Blended                  | 86.14/99.85/10.19         | 81.42/8.35/78.36   | 75.33/96.17/12.76      | 74.16/6.31/70.77  |\n| PhysicalBA             | 90.67/72.94/34.29         | 87.26/10.91/86.54  | 88.89/99.54/10.49      | 85.09/11.53/83.10 |\n\n[1] DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks. IJCAI, 2019.  \n[2] Aeva: Black-box backdoor detection using adversarial extreme value analysis. ICLR, 2022.  \n[3] Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency.ICLR, 2023.  \n[4] Black-box detection of backdoor attacks with limited information and data.ICCV, 2021.  \n[5] Badnets: Identifying vulnerabilities in the machine learning model supply chain.arXiv, 2017.  \n[6] Blind backdoors in deep learning models.USENIX Security, 2021.\n\n***W2: Evaluation of image restoration methods, such as diffusion model based e.g. DDNM in [1] or Restormer in [2].***\n\nWe have included the **qualitative** performance comparison with DDNM in **Supplementary Material H**. The results show that DDNM restores both semantic information and the trigger pattern from the transformed images. Our ZIP model can restore semantic information while removing attack patterns**.** \n\nWe provide **quantitative** performance comparison as follows. The results reveal that, while DDNM shows better clean accuracy than ZIP, it cannot effectively defend against backdoor attacks like Blended.\n\n|                        | No Defense        | Blur+DDNM             | ZIP (Ours)           |\n| ---------------------- | ----------------- | --------------------- | -------------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑     | CA ↑/ ASR ↓/ PA ↑    |\n| BadNet                 | 84.99/94.53/14.98 | **84.56**/9.14/82.24  | 84.05/**7.55/83.97** |\n| Blended                | 86.14/99.85/10.19 | **85.37**/93.37/15.46 | 81.42/**8.35/78.36** |\n\n***W3:  Evaluation using a different diffusion model, or training on a different dataset?***\n\nTo address your concerns, we **use a different diffusion model [1] pre-trained on CIFAR datasets** to defend against BadNet/Blended/PhysicalBA attacks. Our results showcase its effectiveness. Additionally, we observe that the model pre-trained on ImageNet performs better than the model pre-trained on CIFAR, highlighting the importance of pre-training data quality and quantity. In our experiments, we select the ImageNet-pre-trained diffusion model as our backbone, which demonstrates excellent zero-shot purification performance on both in-distribution (Imagenette) and out-of-distribution (CIFAR-10, GTSRB) images, affirming our approach's effectiveness.\n\n|                 | No Defense        | ZIP (pre-trained on CIFAR) | ZIP(pre-trained on ImageNet) |\n| --------------- | ----------------- | --------------------------------------------- | ------------------------------------------------ |\n| CIFAR (32 × 32) | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑  | CA ↑/ ASR ↓/ PA ↑   |\n| BadNet          | 82.31/99.98/10.00 | 70.35/12.56/66.04   | 78.97/5.53/79.10   |\n| Blended         | 80.26/99.96/10.03 | 70.96/16.39/49.26  | 72.62/7.75/57.98    |\n| PhysicalBA      | 85.3/98.73/11.2   | 80.30/10.05/78.03    | 80.10/4.33/80.33  |\n\n[1] Denoising diffusion probabilistic models. NeurIPS, 2020.  \n\n***W4: More recent baselines should be evaluated***.\n\nThanks for this comment. We **included contemporaneous work BDMAE [1] as a recent baseline in Section 4.4.1**, published online in March 2023. BDMAE is a black-box defense method that first identifies and masks trigger regions, then restores them using a masked autoencoder. \n\nAs demonstrated in Table 3, BDMAE can defend against patch-based attacks like BadNets (ASR=1.12), but it fails in non-patch attacks like Blended (ASR=99.88). In contrast, **our ZIP method can effectively defend against both attacks** (ASR values are low), making it a more practical defense. \n\nIt is also worth clarifying that black-box purification in the zero-shot setting is a very challenging task, and as a result, baseline methods are indeed rare.\n\n[1]Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder. arXiv, 2023.  "}, {"review_id": "C1yAj3pErv", "reviewer": "Reviewer_8w5v", "summary": "The paper proposes a backdoor defense for real-world black-box models through zero-shot image purification (ZIP). The proposed defense framework includes two stages: (1) applying linear transformation on a poisoned sample (2) using a pre-trained diffusion model to recover semantic information removed by stage (1). The proposed ZIP is utilized against different backdoor attacks on different datasets. The experiments demonstrate the effectiveness of proposed ZIP.", "questions": "1. The proposed method uses a pre-trained diffusion model to recover semantic. Does the diffusion model need addition in-distribution clean and poisoned data to train or fine-tune?\n2. Could the proposed method work for different network architectures?\n3. In Table 3, BDMAE gets a higher CA than ZIP. Could the authors explain this?\n4. If the input is a clean image, is there a possibility that ZIP can misclassify the clean image? Could the authors provide some failure samples?", "limitations": "The authors have addressed the limitations", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed idea is interesting and the authors analyze the zero-shot image purification theoretically. The conducted experiments demonstrate the effectiveness of proposed method. ", "weaknesses": "1. The authors only use three backdoor attacks including BadNet, PhysicalBA and Blended. Please use more attacks such as WaNet, label-clean attacks to verify the effectiveness of proposed method.\n2. In the Introduction Section (line 31 - 36), the authors mention that there are some works about black-box setting. Please compare with these methods.\n3. The paper needs to be proofread. There are some typos in the paper e.g. in Algorithm 1 (line 222) \"liner\" should be \"linear\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***W1: The authors only use three backdoor attacks including BadNet, PhysicalBA and Blended. Please use more attacks such as WaNet, label-clean attacks to verify the effectiveness of proposed method.***\n\nThanks for the suggestion. We conduct further experiments with the **WaNet[1], Blind[2], Label-Consistence[3]** attack on Imagenette datasets to further demonstrate our defense effectiveness. The quantitative results are listed below, while the qualitative results are provided in the rebuttal PDF. The results show that our ZIP can successfully defend against various advanced attacks.\n\n|  | No Defense         | ShrinkPad (defense) | Blur (defense)    | ZIP (Ours)           |\n| ---------------------- | ------------------ | ------------------- | ----------------- | -------------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑  | CA ↑/ ASR ↓/ PA ↑   | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑    |\n| WaNet                  | 85.45 /97.93/11.84 | 73.63 /26.87/60.35  | 62.98/17.88/56.81 | **79.13/7.77/76.78** |\n| Blind                  | 79.15/99.64/10.42  | 65.22/6.34/65.60    | 75.26/28.12/67.54 | **78.80/5.32/79.03** |\n| LabelConsistent        | 77.57/73.17/36.56  | 66.14/0.25/64.53    | 50.36/0.28/50.14  | **73.63/0.12/74.26** |\n\n[1]Wanet--imperceptible warping-based backdoor attack. ICLR, 2021.  \n[2]Blind backdoors in deep learning models. USENIX Security, 2021.  \n[3]Label-consistent backdoor attacks. arXiv 2019.\n\n***W2: Please compare with works under black-box setting.***\n\nThanks for this comment. We provide a comprehensive comparison in **Supplementary Material G.** In brief, the existing defense methods in black-box settings are mainly constrained to:\n1. **Backdoor Detections**: Identifying backdoors in poisoned samples [1, 2, 3, 4], without mitigating the poisoning effect.\n2. **Limited Purification**: Removing backdoors patterns under specific circumstances, such as partially white-box settings [5], or focusing solely on patch-based attacks [6].\n\nCompared to existing work, our ZIP can achieve **zero-shot** backdoor **purification** encompassing various attack patterns (patch-based and non-patch-based) under the **black-box setting**. These features make it a unique and powerful defense that overcomes the limitations of existing methods.\n\n[1]DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks. IJCAI, 2019.  \n[2]Aeva: Black-box backdoor detection using adversarial extreme value analysis.ICLR, 2022.  \n[3]Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency.ICLR, 2023.  \n[4]Black-box detection of backdoor attacks with limited information and data.ICCV, 2021.  \n[5]Februus: Input purification defense against trojan attacks on deep neural network systems. ACSAC, 2020.  \n[6]Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder.arXiv, 2023. \n\n***W3:The paper needs to be proofread. There are some typos in the paper e.g. in Algorithm 1 (line 222) \"liner\" should be \"linear\".***\n\nThank you for bringing this to our attention. We will correct these typos in our next version.\n\n***Q1: The proposed method uses a pre-trained diffusion model to recover semantic. Does the diffusion model need addition in-distribution clean and poisoned data to train or fine-tune?***\n\nOur framework does not require additional in-distribution of clean or poisoned data to train or fine-tune. In our current experiments, we choose a diffusion model [1] pre-trained on ImageNet as our backbone model. The selected model demonstrates great zero-shot purification performance on both in-distribution (Imagenette) and out-of-distribution (CIFAR-10, GTSRB) images.\n\n[1]Diffusion models beat gans on image synthesis.NeurIPS, 2021.\n\n***Q2: Could the proposed method work for different network architectures?***\n\nThank you for this comment. Yes, it can work for classifiers with different network architectures. To verify this, we implemented ZIP with a new classifier using VGG-16 Network. The defense performance with VGG-16 and ResNet-34 are listed as follows:\n\n\n|  | No defense with ResNet-34 | ZIP with ResNet-34 | No defense with VGG-16 | ZIP with VGG-16   |\n| ---------------------- | ------------------------- | ------------------ | ---------------------- | ----------------- |\n| Imagenette (256 × 256) | CA ↑/ ASR ↓/ PA ↑ | CA ↑/ ASR ↓/ PA ↑  | CA ↑/ ASR ↓/ PA ↑| CA ↑/ ASR ↓/ PA ↑ |\n| BadNet  | 84.99/94.53/14.98         | 84.05/7.55/83.97   | 70.24/91.67/17.35      | 68.71/9.72/69.09  |\n| Blended | 86.14/99.85/10.19         | 81.42/8.35/78.36   | 75.33/96.17/12.76      | 74.16/6.31/70.77  |\n| PhysicalBA | 90.67/72.94/34.29 | 87.26/10.91/86.54  | 88.89/99.54/10.49      | 85.09/11.53/83.10 |\n\n***Q3: In Table 3, BDMAE gets a higher CA than ZIP. Could the authors explain this?***\n\nThank you for the thoughtful comment. BDMAE achieves a higher CA by applying smaller masking perturbation, which better preserves semantic information in the purified image. BDMAE first identifies and masks trigger regions, then restores them using a masked autoencoder. The smaller masked regions introduce less perturbation compared to the linear transformation (e.g., Blur) in our method.\n\nWe would also like to point out that, as shown in Table 3, BDMAE cannot defend against non-patch attacks like Blended (ASR after BDMAE is 99.88). In contrast, our ZIP method can effectively defend against various attacks.\n\n***Q4: If the input is a clean image, is there a possibility that ZIP can misclassify the clean image? Could the authors provide some failure samples?***\n\nIt is possible but the probability is low. The experiments in Table 1 show that after the purification, the classification accuracy on clean images (CA) experiences a relatively marginal average drop of 3.09% across all three datasets, showing its effectiveness in maintaining information of clean data."}, {"review_id": "5ZREbP8tXw", "reviewer": "Reviewer_8ejw", "summary": "In this paper, authors proposed a two-stage framework. Based on Range-Null Space Decomposition theory, they first utilize image transformation to destruct the trigger pattern, and then leverage a pre-trained diffusion model to restore the semantic information.", "questions": "What is the motivation of introducing RND?  Without RND, is there any change of the results? As shown in this paper(line 261), “Defense methods available for black-box purification in zero-shot are rare”, why not setting the black-box backdoor defense Salient Conditional Diffusion (Sancdifi) as your baseline?", "limitations": "Line 52-54 of this paper, If there is evidence in other existing literature to support this sentence “the semantic information in a poisoned image (e.g., faces, cars, or buildings) constitutes the majority of the data” , please cite it; if not, this idea should be used as a methodological assumption.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1)\tTo the best of my knowledge, authors propose a novel black-box backdoor defense method in a zero-shot setting.\n(2)\tThe contributions mentioned by the authors in this paper are reasonable.\n", "weaknesses": "The chosen baselines of experiments in Section 4.3 (the results in Table 1) are not convincing and sufficient. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***W1&Q2:The chosen baselines of experiments in Section 4.3 (the results in Table 1) are not convincing and sufficient. As shown in this paper(line 261), “Defense methods available for black-box purification in zero-shot are rare”, why not setting the black-box backdoor defense Salient Conditional Diffusion (Sancdifi) as your baseline?***\n\nThank you for the suggestion. To our best knowledge, there is no publicly available code for Salient Conditional Diffusion (Sancdifi) for replication. In our best effort to include recent baseline methods, **we include contemporaneous work BDMAE[1] in Section 4.4.1**, using its available online code. BDMAE is a black-box defense method that first identifies and masks trigger regions, then restores them using a masked autoencoder.\n\nAs shown in Table 3, BDMAE can defend against patch-based attacks like BadNets (ASR = 1.12), but it fails in non-patch attacks like Blended (ASR = 99.88). In contrast, **our ZIP method can effectively defend against both kinds of attacks,** making it a more practical defense against various types of attacks.\n\nIt is also worth clarifying that black-box purification in the zero-shot setting is very challenging, and as a result, baseline methods are indeed rare.\n\n[1] Sun, Tao, et al. \"Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder.\"  arXiv, 2023\n\n\n\n***Q1: What is the motivation of introducing RND? Without RND, is there any change of the results?***\n\nThe motivation of introducing the RND theory is to **guide the diffusion model in generating high-fidelity purified images.** Without RND, the reverse diffusion process becomes uncontrollable, resulting in random purified images with unpredictable semantic information. Corollary 3.2.1, presented in Section 3.3.2, theoretically validates the rationale for incorporating RND into our approach.\n\n***L1: Line 52-54 of this paper, If there is evidence in other existing literature to support this sentence “the semantic information in a poisoned image (e.g., faces, cars, or buildings) constitutes the majority of the data” , please cite it; if not, this idea should be used as a methodological assumption.***\n\nThanks for the suggestion. This sentence can be supported by  [1,2,3]. We will add these references In our next version.\n\n[1] Li, Yiming, et al. \"Backdoor learning: A survey.\"  TNNLS, 2022\n[2]  Li, Yiming, et al. \"Backdoor attack in the physical world.\" ICLR workshop, 2021.\n[3] Sun, Tao, et al. \"Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder.\"  arXiv, 2023"}]

Author rebuttal:
# Global Response to All Reviewers

Thank you for your time and efforts in reviewing our work. We greatly appreciate reviewers’ recognition of the quality and novelty of our research. Here is a summary of our response.

**Related Work:** 

Our paper addresses the challenge of achieving practical **zero-shot purification defense in a black-box setting.** Our ZIP method introduces novelty compared to existing approaches in the following ways:

1. Compared to **pre-processing defenses**, our ZIP does not require access to poisoned model internals or prior knowledge of clean/poisoned images, while still maintaining better semantic information after purification.
2. Compared to **diffusion-based purification models**, ZIP enhances trigger destruction with linear transformation and incorporates RND theory to improve our reverse diffusion process.
3. Compared to **black-box detection methods**, our ZIP can remove backdoor effects from poisoned images through purification, and provides a theoretical analysis of its effectiveness. 

In summary, our model presents a novel and robust defense against various attacks in more complex scenarios.

**Experiments:**

We have added the following experiments:

1. Defense Performance against **WaNet, Blind, and Lable-Consistent** attacks: In Rebuttal_Table 1, ZIP shows effective defense performance with three attacks, which showcases that our ZIP can defend against recent advanced attacks.
2. Performance comparisons with **image purification method**: DiffPure, and Blur+DiffPure: In Rebuttal_Table 2, ZIP shows superior defense performance compared to DiffPure and Blur+DiffPure, which validates the proposed enhancements.
3. Defense Performance with **different pre-trained diffusion models**: In Rebuttal_Table 3, ZIP shows effective defense using a diffusion model pre-trained on CIFAR, demonstrating that ZIP can work with different diffusion models.
4. Defense Performance with **classifiers in different architectures**: In Rebuttal_Table 2 and 4, ZIP shows its effective defense performance with poisoned classifiers in ResNet-34 and VGG-16 networks.
5. Performance comparisons with **image restoration method**: DDNM: In Rebuttal_Table 2, ZIP shows better defense performance compared to DDNM, validating our superiority. 

We also would like to mention that:

6. We include **contemporaneous work BDMAE** as a recent defense baseline in our Section 4.4.1, and experiments show that our ZIP method can effectively defend against BadNet and Blended attack, while BDMAE cannot.
7. Qualitative performance comparisons with **DDNM** are provided in Supplementary Material H, where ZIP shows better performance in removing trigger patterns.
8. Extensive **visualization results of ZIP** purification are provided in Supplementary B to validate our proposed design.

In summary, the experiments show that **our ZIP can defend against advanced backdoors in diverse forms,** and ZIP shows better defense performance **compared to various baselines** including BDMAE, DiffPure, and DDNM.

We really appreciate your constructive comments, as they help us significantly improve the quality of our paper. We sincerely hope our response can address your concerns.

Meta-review: According to the comments from the reviewers who has engaged in discussions during the rebuttal period, some of the reviewers have raised the scores and most comments have been well-addressed. The main strength of this paper is its sufficient novelty and comprehensive and clear theoretical analysis. Based on the above reasons, the AC tends to recommend the acceptance of this paper.
Decision: Accept (poster)