SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yucheng Shi $^{1}$  Mengnan Du $^{2}$  Xuansheng Wu $^{1}$  Zihan Guan $^{1}$  Jin Sun $^{1}$  Ninghao Liu $^{1*}$
$^{1}$ School of Computing, University of Georgia  
 $^{2}$ Department of Data Science, New Jersey Institute of Technology  
{yucheng.shi, xuansheng.wu, zihan.guan, jinsun, ninghao.liu}@uga.edu  
mengnan.du@njit.edu
Backdoor attacks inject poisoned samples into the training data, resulting in the misclassification of the poisoned input during a model's deployment. Defending against such attacks is challenging, especially for real-world black-box models where only query access is permitted. In this paper, we propose a novel defense framework against backdoor attacks through Zero-shot Image Purification (ZIP). Our framework can be applied to poisoned models without requiring internal information about the model or any prior knowledge of the clean/poisoned samples. Our defense framework involves two steps. First, we apply a linear transformation (e.g., blurring) on the poisoned image to destroy the backdoor pattern. Then, we use a pre-trained diffusion model to recover the missing semantic information removed by the transformation. In particular, we design a new reverse process by using the transformed image to guide the generation of high-fidelity purified images, which works in zero-shot settings. We evaluate our ZIP framework on multiple datasets with different types of attacks. Experimental results demonstrate the superiority of our ZIP framework compared to state-of-the-art backdoor defense baselines. We believe that our results will provide valuable insights for future defense methods for black-box models. Our code is available at https://github.com/syncny/ZIP.
Machine learning has been increasingly integrated into real-world applications such as healthcare [45], finance [28] and computer vision [66]. Despite great success, machine learning models are susceptible to adversaries such as backdoor attacks [15, 8, 43, 54, 17], which compromises model security and reliability. Backdoor attacks can manipulate a model's behavior by injecting malicious samples into the training data or altering the model's weights. Although many defense strategies have been proposed to mitigate backdoor attacks, most of them require access to the model's internal structure and poisoned training data [32]. Deploying these defenses is challenging in real-world black-box scenarios, where defenders do not have access to verify or audit the inner workings of the model [18, 19]. For example, many developers and end-users now prefer using machine learning as a service (MLaaS), relying on models provided by third-party vendors for their applications. However, these models may contain backdoors, and due to copyright concerns, the services typically operate in a black-box setting with query-only access. In such scenarios, detecting and mitigating backdoor attacks is very difficult.
Currently, there are only a few backdoor defense methods that work in black-box settings. They can be categorized into two types: detecting-based and purification-based. Detecting-based methods [65,
18, 31, 12, 56] can detect the poisoned sample but could not remove the poisoned patterns. These methods are not applicable when critical poisoned samples must be used by downstream classification models. Purification-based methods can address this problem since they aim to retrieve clean images given the poisoned images. However, state-of-the-art purification approaches rely on masking and reconstructing the poisoned area [42, 53]. It can only protect against patch-based attacks. Other purification-based methods that employ image transformations for defense have the potential to defend against more sophisticated attacks but may result in a reduction in classification accuracy due to the loss of semantic information [35].
To overcome these challenges, we propose a novel framework to defend against attacks through Zero-shot Image Purification (ZIP). We define "purification" as the process of maximizing the retention of important semantic information while eliminating the trigger pattern. With this goal, we preserve the classification accuracy while breaking the connection between trigger patterns and poisoned labels. We also define "zero-shot" as the ability to defend against various attacks without relying on prior knowledge of attack methods. In other words, our approach does not require access to any clean or poisoned image samples (patch or non-patch based), and could be applied directly to unseen attack scenarios. This setting is crucial because real-world users usually have limited information, while new threats continue to emerge. Our proposed framework contains two main stages: we first utilize image transformation to destruct the trigger pattern, and then leverage an off-the-shelf, pre-trained diffusion generative model to restore the semantic information. Our defense strategy is based on the motivation that the semantic information in a poisoned image (e.g., faces, cars, or buildings) constitutes the majority of the data and typically falls within the training data distribution of a pre-trained image generation model. In contrast, engineered trigger patterns (e.g., mosaic patches and colorful noise) are subtle and unlikely to exist in the pre-training datasets [32, 34]. Since the diffusion model learns to sample images from the training distribution [22], purified images generated from the diffusion model will retain only their semantic information while eliminating the trigger patterns. As a result, our purification approach can effectively defend against various attacks while maintaining high-fidelity in the restored images.
Our main contributions are summarized as follows. (1) We develop a novel defense framework that can be applied to black-box models without requiring any internal information about the model. Our method is versatile and easy to use without retraining. (2) The proposed framework is designed for zero-shot settings, which do not require any prior knowledge of the clean or poisoned images. This feature relieves end-users from the need to collect samples, thus enhancing the framework's applicability. (3) Our defense framework achieves good classification accuracy on the purified images, which were originally poisoned samples, even when using an attacked model as the classifier. This improvement further enhances the framework's effectiveness and usability.
This paper addresses the backdoor defense problem in the context of image classification tasks. The goal of image classification is to learn a function  $f_{\theta}(\mathbf{x})$  that maps input images  $\mathbf{x} \in \mathcal{X}$  to their correct labels  $y \in \mathcal{Y}$ , where  $\mathcal{X}$  denotes the image space,  $\mathcal{Y}$  denotes the label space, and  $\theta$  represents model parameters. Typical backdoor attacks include poisoning training data and perturbing model weights, and we use  $f_{\theta}^{attack}$  to denote the model that has been attacked. During inference, an attacker can take a clean sample  $\mathbf{x}$  and manipulate it to create a backdoor sample  $\mathbf{x}^P$ , e.g., adding the trigger pattern  $\mathbf{p}$  to make  $\mathbf{x}^P = \mathbf{x} + \mathbf{p}$ . The backdoored model will misclassify  $\mathbf{x}^P$  as the target label  $y^{target} = f_{\theta}^{attack}(\mathbf{x}^P) \neq y$ . Our threat model is in a challenging black-box setting, where defenders can only query the poisoned model and have no access to the model's internal parameters or training datasets. The attackers can modify model components or any other necessary information to implement their attacks. We formally define our backdoor defense problem as below.
Problem 1. Image Purification for Backdoor Defense. Our defense is implemented in the model inference stage. Let  $f_{\theta}^{attack}$  denote the attacked model, whose parameters are not accessible. Given a poisoned image  $\mathbf{x}^P$ , our goal is to obtain a purified image  $\mathbf{x}^*$  from  $\mathbf{x}^P$  by removing the effect of trigger  $\mathbf{p}$ . The purified image should be classified as the same category as the original clean image  $\mathbf{x}$ , i.e.,  $f_{\theta}(\mathbf{x}^*) = f_{\theta}(\mathbf{x}) \neq f_{\theta}(\mathbf{x}^P)$ .
Figure 1: The proposed ZIP backdoor defense framework. In Stage 1, we use a linear transformation such as blurring to destruct the trigger pattern in poisoned image  $\mathbf{x}^P$ . In Stage 2, we design a guided diffusion process to generate the purified image  $\mathbf{x}_0$  with a pre-trained diffusion model. Finally, in  $\mathbf{x}_0$ , the semantic information from  $\mathbf{x}$  is kept while the trigger pattern is destroyed.
2.2 Diffusion Models
We leverage the reverse process of diffusion models to purify images. The denoising diffusion probabilistic model (DDPM) [22] is a powerful generative model for generating high-quality images. It has two processes: a forward process and a reverse process. In the forward process, the model iteratively adds noise to an input image  $\mathbf{x}_0$  until it becomes random Gaussian noises  $\mathbf{x}_T$ ; in the reverse process, the model iteratively removes the added noise from  $\mathbf{x}_T$  to recover the noise-free image  $\mathbf{x}_0$ . More details can be found in the DDPM paper [22].
Forward Process: A noise-free image  $\mathbf{x}_0$  is transformed to a noisy image  $\mathbf{x}_t$  with controlled noise. Specifically, Gaussian noise  $\epsilon$  is gradually added to image  $\mathbf{x}_0$  in  $T$  steps based on a variance schedule  $\beta_t \in [0,1]$  such that  $q(\mathbf{x}_t \mid \mathbf{x}_{t-1}) \coloneqq \mathcal{N}(\mathbf{x}_t; \sqrt{1 - \beta_t} \mathbf{x}_{t-1}, \beta_t \mathbf{I})$ , where  $q$  denotes the posterior probability of  $\mathbf{x}_t$  conditioned on  $\mathbf{x}_{t-1}$ . A nice property of this process is that the  $t$ -th step noisy image  $\mathbf{x}_t$  could be directly generated by:
$$
q \left(\mathbf {x} _ {t} \mid \mathbf {x} _ {0}\right) = \mathcal {N} \left(\mathbf {x} _ {t}; \sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0}, (1 - \bar {\alpha} _ {t}) \mathbf {I}\right), \quad \mathbf {x} _ {t} = \sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0} + \sqrt {1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon}, \tag {1}
$$
where  $\epsilon \sim \mathcal{N}(0,\mathbf{I})$ ,  $\alpha_{t} = 1 - \beta_{t}$ , and  $\bar{\alpha}_{t} = \prod_{i = 1}^{t}\alpha_{i}$ .
Reverse Process: The noisy input image  $\mathbf{x}_T$  is transformed into a noise-free output image  $\mathbf{x}_0$  over time steps. In each step, the diffusion model takes the current image state  $\mathbf{x}_t$  as input and produces the previous state  $\mathbf{x}_{t-1}$ . We aim to obtain clean images  $\mathbf{x}_0$  by iteratively sampling  $\mathbf{x}_{t-1}$  from  $p(\mathbf{x}_{t-1}|\mathbf{x}_t,\mathbf{x}_0)$ :
$$
\mathbf {x} _ {t - 1} \leftarrow \frac {\sqrt {\bar {\alpha} _ {t - 1}} \beta_ {t}}{1 - \bar {\alpha} _ {t}} \mathbf {x} _ {0} + \frac {\sqrt {\alpha_ {t}} (1 - \bar {\alpha} _ {t - 1})}{1 - \bar {\alpha} _ {t}} \mathbf {x} _ {t} + \sigma_ {t} \boldsymbol {\epsilon}, \quad \boldsymbol {\epsilon} \sim \mathcal {N} (0, \mathbf {I}), \quad \sigma_ {t} ^ {2} = \frac {1 - \bar {\alpha} _ {t - 1}}{1 - \bar {\alpha} _ {t}} \beta_ {t}. \tag {2}
$$
Based on Equation 1, the original clean image  $\mathbf{x}_0$  could be approximated based on the  $t$ -th step observation  $\mathbf{x}_t$  as:  $\mathbf{x}_{0|t} = \frac{1}{\sqrt{\bar{\alpha}_t}} (\mathbf{x}_t - \sqrt{1 - \bar{\alpha}_t}\pmb {\epsilon}_t)$ , where  $\pmb {\epsilon}_t$  denotes the estimation of the real  $\pmb{\epsilon}$  in step  $t$ . In each step  $t$ , DDPM utilizes a neural network  $g_{\phi}(\cdot)$  to predict the noise  $\pmb{\epsilon}_t$ , i.e.,  $\pmb {\epsilon}_t = g_{\phi}(\mathbf{x}_t,t)$ . With this estimation, we can convert Equation 2 into the following form as reverse process:
$$
\mathbf {x} _ {t - 1} \leftarrow \frac {1}{\sqrt {\alpha} _ {t}} \left(\mathbf {x} _ {t} - \frac {1 - \alpha_ {t}}{\sqrt {1 - \bar {\alpha} _ {t}}} \boldsymbol {\epsilon} _ {t}\right) + \sigma_ {t} \boldsymbol {\epsilon}. \tag {3}
$$
3 Zero-shot Image Purification (ZIP)
3.1 Overview of Proposed Framework
Our defense framework consists of two primary stages: (1) destruct poisoned images through image transformation, and (2) recover images using a diffusion model, as depicted in Figure 1. The first stage is based on the observation that the integrity of trigger patterns is crucial for backdoor attacks to deceive the model. Thus, destructing the trigger pattern would significantly reduce the effectiveness
of backdoor attacks [34, 46]. On the other hand, a strong transformation may destroy both the trigger pattern and semantic information. To address this, we introduce the second stage to recover the semantic information through the reverse process of diffusion models. However, traditional diffusion models (e.g., DDPM [22]) or image restoration models (e.g., DDRM [29] and DDNM [58]) can not generate high-fidelity and clean images due to a lack of direct control over the generated output.
To bridge the gap, in the following section, we first propose an image generation constraint based on the image transformation in Section 3.2. We then apply this constraint to guide the image reverse process of the diffusion model. In addition, we discuss the guide adaptation when applied in the zero-shot settings, along with its theoretical justification in Section 3.3. Finally, we introduce our efforts to improve the inference speed in Section 3.4.
3.2 Image Transformation and Decomposition
In the first stage, we apply image transformation to destruct potential trigger patterns, such as using average pooling to blur the poisoned images. Formally, we denote the transformation as a linear operator  $\mathbf{A}$ , and let the transformed image be  $\mathbf{x}^A = \mathbf{A}\mathbf{x}^P = \mathbf{A}(\mathbf{x} + \mathbf{p})$ . Previous research [34, 47] has used the transformed image directly as the purified result. However, such approaches may result in poor classification accuracy due to the loss of fidelity induced by  $\mathbf{A}$ .
To recover the lost information, an intuitive way is to apply an image generative model, e.g., the diffusion model, to yield a purified image. However, the vanilla diffusion model generates images from random Gaussian noise and lacks control over the fidelity of generated images. Thus, we propose a constraint to guide the generation process of diffusion models to recover high-fidelity images. Specifically, for an ideally purified image  $\mathbf{x}_0$ , it should satisfy  $\mathbf{x}_0 = \mathbf{x}$ , so we have:
$$
\mathbf {A} \left(\mathbf {x} _ {0} + \mathbf {p}\right) = \mathbf {A} \left(\mathbf {x} + \mathbf {p}\right) = \mathbf {x} ^ {A}. \tag {4}
$$
Based on the RND theory [48, 58], it is possible to decompose an image  $\mathbf{x}$  into two parts using a linear operator  $\mathbf{A}$  (e.g., average pooling) and its pseudo-inverse  $\mathbf{A}^{\dagger}$  (e.g., upsampling) that satisfies  $\mathbf{A}\mathbf{A}^{\dagger}\mathbf{A} = \mathbf{A}$ . The decomposition is expressed as  $\mathbf{x} = \mathbf{A}^{\dagger}\mathbf{A}\mathbf{x} + (\mathbf{I} - \mathbf{A}^{\dagger}\mathbf{A})\mathbf{x}$ , where the former part denotes the observable information in the range-space, and the latter part denotes the information in the null-space removed by transformation<sup>1</sup>. Bringing this decomposition to Equation 4, we have:
$$
\left(\mathbf {x} _ {0} + \mathbf {p}\right) = \mathbf {A} ^ {\dagger} \mathbf {A} \left(\mathbf {x} _ {0} + \mathbf {p}\right) + \left(\mathbf {I} - \mathbf {A} ^ {\dagger} \mathbf {A}\right) \left(\mathbf {x} _ {0} + \mathbf {p}\right). \tag {5}
$$
This equation derives a constraint for image purification to restore the original  $\mathbf{x}$  as below:
$$
\mathbf {x} _ {0} = \mathbf {A} ^ {\dagger} \mathbf {x} ^ {A} - \mathbf {A} ^ {\dagger} \mathbf {A} \mathbf {p} + (\mathbf {I} - \mathbf {A} ^ {\dagger} \mathbf {A}) \mathbf {x} _ {0}, \tag {6}
$$
accordingly, the ideally purified image could be decomposed into three parts. The first two parts are in the range space: the observable information stored in the transformed image  $\mathbf{A}^{\dagger}\mathbf{x}^{A}$ , and the intractable information embedded in the transformed trigger pattern  $\mathbf{A}^{\dagger}\mathbf{Ap}$ ; the last part is in the null-space and is unobservable as it is removed by the transformation. To restore the lost information in the null-space, we utilize the observable information in the range-space as references.
3.3 Image Purification with Diffusion Model
3.3.1 Reverse Process Conditioned on Poisoned Images
Our proposed image purification is based on the reverse process of the diffusion model, which takes Gaussian noise  $\mathbf{x}_T$  as input and generates a noise-free image  $\mathbf{x}_0$ . We use  $\mathbf{x}_t$  to denote the image at time step  $t$  in the reverse process of diffusion. To generate high-fidelity images, we propose a rectified estimation of  $\mathbf{x}_t$  as  $\mathbf{x}_t'$ , so that it produces  $\mathbf{x}_{0|t}$  that satisfies the decomposition constraint in Equation 6. The  $\mathbf{x}_t'$  is computed using the following equation, which is derived from Equation 1 and 6. The detailed proof is in the Supplementary Material A.
$$
\mathbf {x} _ {t} ^ {\prime} = \sqrt {\bar {\alpha} _ {t}} \mathbf {A} ^ {\dagger} \mathbf {x} ^ {A} - \sqrt {\bar {\alpha} _ {t}} \mathbf {A} ^ {\dagger} \mathbf {A p} + (\mathbf {I} - \mathbf {A} ^ {\dagger} \mathbf {A}) \mathbf {x} _ {t} + \mathbf {A} ^ {\dagger} \mathbf {A} \sqrt {1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon} _ {t}, \tag {7}
$$
Usually, we have  $\mathbf{A}^{\dagger}\mathbf{A}\neq \mathbf{I}$ , which implies that this operation is lossy. When applying  $\mathbf{A}^{\dagger}\mathbf{A}$  to an image, some information will be removed, making this operation irreversible. More details are in Supplementary Material C
where  $\epsilon_{t}$  denotes the estimated noise, which is calculated using the pre-trained diffusion model  $g_{\phi}$ :  $\epsilon_{t} = g_{\phi}(\mathbf{x}_{t}, t)$ . Then, we modify the original reverse process in Equation 3 to accommodate this rectified estimation. The modified reverse process is expressed as:
$$
\mathbf {x} _ {t - 1} \leftarrow \frac {1}{\sqrt {\alpha} _ {t}} \left(\sqrt {\bar {\alpha} _ {t}} \mathbf {A} ^ {\dagger} \mathbf {x} ^ {A} - \sqrt {\bar {\alpha} _ {t}} \mathbf {A} ^ {\dagger} \mathbf {A p} + (\mathbf {I} - \mathbf {A} ^ {\dagger} \mathbf {A}) \mathbf {x} _ {t} + \mathbf {A} ^ {\dagger} \mathbf {A} \sqrt {1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon} _ {t} - \frac {1 - \alpha_ {t}}{\sqrt {1 - \bar {\alpha} _ {t}}} \boldsymbol {\epsilon} _ {t}\right) + \sigma_ {t} \boldsymbol {\epsilon}, (8)
$$
where  $\epsilon \sim \mathcal{N}(0,\mathbf{I})$  denotes Gaussian noise and we have  $\sigma_t^2 = \frac{1 - \bar{\alpha}_{t-1}}{1 - \bar{\alpha}_t} \beta_t$ .
3.3.2 Adapting Reverse Process to the Zero-shot Setting
In this subsection, we show how our reverse process design can be effectively applied in the zero-shot setting, where the trigger pattern  $\mathbf{p}$  is unknown to defenders. We propose to omit the intractable term  $\sqrt{\overline{\alpha}_t} \mathbf{A}^\dagger \mathbf{A} \mathbf{p}$  in Equation 7, and approximate  $\mathbf{x}_t'$  with  $\hat{\mathbf{x}}_t$  to obtain a new rectified estimation:
$$
\hat {\mathbf {x}} _ {t} = \sqrt {\bar {\alpha} _ {t}} \mathbf {A} ^ {\dagger} \mathbf {x} ^ {A} + (\mathbf {I} - \mathbf {A} ^ {\dagger} \mathbf {A}) \mathbf {x} _ {t} + \mathbf {A} ^ {\dagger} \mathbf {A} \sqrt {1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon} _ {t}. \tag {9}
$$
The intractable term  $\sqrt{\bar{\alpha}_t}\mathbf{A}^\dagger\mathbf{A}\mathbf{p}$  can be omitted due to the following reasons:
- The value of  $\sqrt{\bar{\alpha}_t}$  at the beginning of the reverse process is very small, making  $\sqrt{\bar{\alpha}_t} \mathbf{A}^\dagger \mathbf{A} \mathbf{p}$  negligible compared to other terms. We provide an improved approximation in Section 3.3.3 for later stages when  $\sqrt{\bar{\alpha}_t}$  increases.
- Backdoor attacks are generally stealthy or use more negligible patterns compared to the original images since the attacker wants to minimize the impact on the model's accuracy on legitimate data [43]. Thus, the destructed pattern  $\mathbf{A}\mathbf{p}$  is usually negligible compared to  $\mathbf{x}^A = \mathbf{A}(\mathbf{x} + \mathbf{p})$ .
- The effect of  $\mathbf{A}^{\dagger}\mathbf{A}\mathbf{p}$  is further reduced by selecting appropriate image transformations. Most back-door attacks are characterized by severe high-frequency artifacts [65]. Therefore, transformations such as average pooling can remove the high-frequency information in  $\mathbf{p}$ .
Nevertheless, approximation in an iterative process such as diffusion can be risky because errors from the previous step can accumulate rapidly (e.g., exponentially) and lead to significant inaccuracies. Our method addresses this issue by ensuring the approximation error between each step is well-bounded theoretically. As a result, the final recovered image  $\hat{\mathbf{x}}_0$  preserves the essential information of the original image, while the trigger pattern undergoes a transformation and is likely to be destroyed.
Lemma 3.1. Suppose the estimated noise output by  $g_{\phi}(\cdot)$  is Gaussian. Given  $g_{\phi}(\mathbf{x}_t,t) = \epsilon_t$  we have  $g_{\phi}((\mathbf{x}_t + \sqrt{\overline{\alpha}_t}\mathbf{A}^\dagger \mathbf{A}\mathbf{p}),t) = \boldsymbol {\epsilon}_t + \boldsymbol {\epsilon}_t'$ , where  $\epsilon_t,\epsilon_t^{\prime}$  are also Gaussian.
The error  $\pmb{\epsilon}_{t}^{\prime}$  in Lemma 3.1 is much smaller than the real estimated noise  $\pmb{\epsilon}_{t}$ . This is because the trigger pattern  $\sqrt{\bar{\alpha}_t}\mathbf{A}^\dagger\mathbf{A}\mathbf{p}$ , which is weighted by  $\sqrt{\bar{\alpha}_t}$ , is relatively subtle compared with the intermediate image  $\mathbf{x}_t$ . Additionally, the attacker-engineered trigger pattern  $\mathbf{p}$  is unlikely to be present within the natural image distribution learned by the pre-trained diffusion model. Therefore, it is unlikely to activate the model  $g_{\phi}(\cdot)$  and generate significant output.
Theorem 3.2. Suppose that  $g_{\phi}((\mathbf{x}_t + \sqrt{\bar{\alpha}_t}\mathbf{A}^\dagger\mathbf{A}\mathbf{p})) = \boldsymbol{\epsilon}_t + \boldsymbol{\epsilon}_t'$ . We define the error at step  $t$  between  $\hat{\mathbf{x}}_t$  and  $(\mathbf{x}_t + \sqrt{\bar{\alpha}_t}\mathbf{A}^\dagger\mathbf{A}\mathbf{p})$  as  $\delta_t'$ , i.e.,  $\delta_t' = \hat{\mathbf{x}}_t - (\mathbf{x}_t + \sqrt{\bar{\alpha}_t}\mathbf{A}^\dagger\mathbf{A}\mathbf{p})$ . Let  $\delta_t = \mathbf{A}\delta_t'$ , we have the following bound on its norm:  $\| \delta_t \| \leq \frac{(1 - \bar{\alpha}_t)\sqrt{\alpha_{t+1}}}{\sqrt{1 - \bar{\alpha}_{t+1}}} \| \mathbf{A} \| \| \boldsymbol{\epsilon}_{t+1}' \|$ .
This theorem means that our proposed approximation introduces only limited approximation error at each time step after considering the error from the previous step. When paired with a common linear transform  $\mathbf{A}$  like average pooling, its norm is relatively small [58]. Additionally, at the initial steps of the reverse process, the term  $\| \epsilon_{t + 1}^{\prime}\|$  is also small according to Lemma 3.1, and we have  $\frac{(1 - \bar{\alpha}_t)\sqrt{\alpha_{t + 1}}}{\sqrt{1 - \bar{\alpha}_{t + 1}}}\leq \frac{1 - \bar{\alpha}_{t + 1} + \alpha_t}{2}$ , which is also a small value. Collectively, the error from our approximation is well bounded by a small number. According to this theorem, the final purified image exhibits the following property.
Corollary 3.2.1. When  $t = 0$ , we have  $\hat{\mathbf{x}}_0 = \mathbf{x}_0 + \mathbf{A}^\dagger \mathbf{A}\mathbf{p} + \delta_0'$ , where  $\mathbf{A}\delta_0' = \mathbf{0}$ .