SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Boya Zhang*
Weijian Luo
Zhihua Zhang
Adversarial attacks have the potential to mislead deep neural network classifiers by introducing slight perturbations. Developing algorithms that can mitigate the effects of these attacks is crucial for ensuring the safe use of artificial intelligence. Recent studies have suggested that score-based diffusion models are effective in adversarial defenses. However, existing diffusion-based defenses rely on the sequential simulation of the reversed stochastic differential equations of diffusion models, which are computationally inefficient and yield suboptimal results. In this paper, we introduce a novel adversarial defense scheme named ScoreOpt, which optimizes adversarial samples at test-time, towards original clean data in the direction guided by score-based priors. We conduct comprehensive experiments on multiple datasets, including CIFAR10, CIFAR100 and ImageNet. Our experimental results demonstrate that our approach outperforms existing adversarial defenses in terms of both robustness performance and inference speed.
In recent years, there have been breakthrough performance improvements with deep neural networks (DNNs), particularly in the realms of image classification, object detection, and semantic segmentation, as evidenced by the works of [31, 59, 53, 19, 13, 51]. However, DNNs have been shown to be easily deceived to produce incorrect predictions by simply adding human-imperceptible perturbations to inputs. These perturbations are called adversarial attacks [14, 36, 38, 7], resulting in safety concerns. Therefore, studies of mitigating the impact of adversarial attacks on DNNs, which is referred to as adversarial defenses, are significant for AI safety.
Various strategies have been proposed in order to enhance the robustness of DNN classifiers against adversarial attacks. One of the most effective forms of adversarial defense is adversarial training [36, 71, 16, 32, 71, 44], which involves training a classifier with both clean data and adversarial samples. However, adversarial training has its limitations as it necessitates prior knowledge of the specific attack method employed to generate adversarial examples, thus rendering it inadequate in handling previously unseen types of adversarial attacks or corruptions.
On the contrary, adversarial purification [48, 68, 52, 28, 69, 40, 21] is another form of promising defense that leverages a standalone purification model to eliminate adversarial signals before conducting downstream classification tasks. The primary benefit of adversarial purification is that it obviates the need to retrain the classifier, enabling adaptive defense against adversarial attacks at test time. Additionally, it showcases significant generalization ability in purifying a wide range of adversarial attacks, without affecting pre-existing natural classifiers. The integration of adversarial purification models into AI systems necessitates minor adjustments, making it a viable approach to enhancing the robustness of DNN-based classifiers.
*Academy for Advanced Interdisciplinary Studies; Peking University; zhangboya@pku.edu.cn;
†School of Mathematical Sciences; Peking University; luoweijian@stu.pku.edu.cn;
$^{\ddagger}$ School of Mathematical Sciences; Peking University; zhzhang@math.pku.edu.cn;
Figure 1: Illustration of our proposed adversarial defense framework.
Diffusion models [22, 54, 58], also known as score-based generative models, have demonstrated state-of-the-art performance in various applications, including image and audio generation [9, 30], molecule design [23], and text-to-image generation [39]. Apart from their impressive generation ability, diffusion models have also exhibited the potential to improve the robustness of neural networks against adversarial attacks. Specifically, they can function as adaptive test-time purification models [40, 16, 4, 63, 66].
Diffusion-based purification methods have shown great success in improving adversarial robustness, but still have their own limitations. For instance, they require careful selection of the appropriate hyper-parameters such as forward diffusion timestep [40] and guidance scale [63], which can be challenging to tune in practice. In addition, diffusion-based purification relies on the simulation of the underlying stochastic differential equation (SDE) solver. The reverse purification process requires iteratively denoising samples step by step, leading to heavy computational costs [40, 63].
In the hope of circumventing the aforementioned issues, we introduce a novel adversarial defense scheme that we call ScoreOpt. Our key intuition is to derive the posterior distribution of clean samples given a specific adversarial example. Then adversarial samples can be optimized towards the points that maximize the posterior distribution with gradient-based algorithms at test time. The prior knowledge is provided by pre-trained diffusion models. Our defense is independent of base classifiers and applicable across different types of adversarial attacks, making it flexible enough across various domains of applications. The illustration of our method is presented in Figure 1.
Our main contributions can be summarized in three aspects as follows:
- We propose a novel adversarial defense scheme that optimizes adversarial samples to reach the points with the local maximum likelihood of the posterior distribution that is defined by pre-trained score-based priors.
- We explore effective loss functions for the optimization process, introduce a novel score regularizer and propose corresponding practical algorithms.
- We conduct extensive experiments to demonstrate that our method not only achieves start-of-the-art performance on various benchmarks but also improves the inference speed.
2.1 Score-based Diffusion Models
Score-based diffusion models [22, 58] learn how to transform complex data distributions to relatively simple ones such as the Gaussian distribution, and vice versa. Diffusion models consist of two processes, a forward process that adds Gaussian noise to input  $\mathbf{x}$  from  $\mathbf{x}_0$  to  $\mathbf{x}_T$ , and a reverse generative process that gradually removes random noise from a sample until it is fully denoised. For the continuous-time diffusion models (see Song et al. [58] for more details and further discussions), we can use two SDEs to describe the above data-transformation processes, respectively. The forward-time SDE is given by:
$$
d \mathbf {x} _ {t} = \mathbf {f} (\mathbf {x} _ {t}, t) d t + \mathbf {g} (t) d \mathbf {w} _ {t}, t \in [ 0, T ];
$$
where  $\mathbf{f}:\mathbb{R}^D\to \mathbb{R}^D$  and  $\mathbf{g}:\mathbb{R}\rightarrow \mathbb{R}$  are drift and diffusion coefficients respectively,  $\mathbf{w}$  denotes the standard Wiener process. While new samples are generated by solving the reverse-time SDE:
$$
d \mathbf {x} _ {t} = [ \mathbf {f} (\mathbf {x} _ {t}, t) - \mathbf {g} ^ {2} (t) \nabla_ {\mathbf {x} _ {t}} \log p _ {t} (\mathbf {x} _ {t}) ] d t + \mathbf {g} (t) d \bar {\mathbf {w}} _ {t}, t \in [ T, 0 ];
$$
Optimization Loop
where  $\bar{\mathbf{w}}$  defines the reverse-time Wiener process. The score function term  $\nabla_{\mathbf{x}_t}\log p_t(\mathbf{x}_t)$  is usually parameterized by a time-dependent neural network  $s_\theta (\mathbf{x}_t;t)$  and trained by score-matching related techniques [25, 58, 61, 56, 41].
Diffusion Models as Prior Recent studies have investigated the incorporation of an additional constraint to condition diffusion models for high-dimensional data sampling. DDPM-PnP [17] proposed transforming diffusion models into plug-and-play priors, allowing for parameterized samples, and utilizing diffusion models as critics to optimize over image space. Building on the formulation in Graikos et al. [17], DreamFusion [43] further introduced a more stable training process. The main idea is to leverage a pre-trained 2D image diffusion model as a prior for optimizing a parameterized 3D representation model. The resulting approach is called score distillation sampling (SDS), which bypasses the computationally expensive backpropagation through the diffusion model itself by simply excluding the score-network Jacobian term from the diffusion model training loss. SDS uses the approximate gradient to train a parametric NeRF generator efficiently. Other works [35, 37, 62] followed similar approaches to extend SDS to the latent space of latent diffusion models [46].
2.2 Diffusion-based Adversarial Purification
Diffusion models have also gained significant attention in the field of adversarial purification recently. They have been employed not only for empirical defenses against adversarial attacks [69, 40, 63, 66], but also for enhancing certified robustness [5, 67]. The unified procedure for applying diffusion models in adversarial purification involves two processes. The forward process adds random Gaussian noise to the adversarial example within a small diffusion timestep  $t^*$ , while the reverse process recovers clean images from the diffused samples by solving the reverse-time stochastic differential equation. Implementing the aforementioned forward-and-denoise procedure, imperceptible adversarial signals can be effectively eliminated. Under certain conditions, the purified sample restores the original clean sample with a high probability in theory [40, 67].
However, diffusion-based purification methods suffer from two main drawbacks. Firstly, their robustness performance heavily relies on the choice of the forward diffusion timestep, denoted as  $t^*$ . Selecting an appropriate  $t^*$  is crucial because excessive noise can lead to the removal of semantic information from the original example, while insufficient noise may fail to eliminate the adversarial perturbation effectively. Secondly, the reverse process of these methods involves sequentially applying the denoising operation from timestep  $t$  to the previous timestep  $t - 1$ , which requires multiple deep network evaluations. In contrast to previous diffusion-based purification methods, our optimization framework departs from the sequential step-by-step denoising procedure.
In this section, we present our proposed adversarial defense scheme in detail. Our method is motivated by solving an optimization problem of adversarial samples to remove the applied attacks. Therefore, we start by formally formulating the optimization objective, followed by exploring effective loss functions and introducing two practical algorithms.
Our main idea is to formulate the adversarial defense as an optimization problem given the perturbed sample and the pre-trained prior, in which the solution to the optimization problem is the recovered original sample that we want. We regard the adversarial example  $\mathbf{x}_a$  as a disturbed measurement of the original clean example  $\mathbf{x}$ , and we assume that the clean example is generated by a prior probability distribution  $\mathbf{x} \sim p(\mathbf{x})$ . The posterior distribution of the original sample given the adversarial example is  $p(\mathbf{x}|\mathbf{x}_a) \propto p(\mathbf{x}) p(\mathbf{x}_a|\mathbf{x})$ . The maximum a posteriori estimator that maximizes the above conditional distribution is given by:
$$
\hat {\mathbf {x}} ^ {*} = \underset {\mathbf {x}} {\arg \min } - \log p (\mathbf {x} \mid \mathbf {x} _ {a}). \tag {1}
$$
In this work, we use the data distribution under pretrained diffusion models as the prior  $p_{\theta}(\mathbf{x})$
3.2 Loss Functions for Optimization Process
Following Graikos et al. [17], we introduce a variational posterior  $q(\mathbf{x})$  to approximate the true posterior distribution  $p(\mathbf{x}|\mathbf{x}_a)$  in the original optimization objective. The variational upper bound on the negative log-likelihood  $-\log p(\mathbf{x}_a)$  is:
$$
- \log p (\mathbf {x} _ {a}) \leq \mathbb {E} _ {q (\mathbf {x})} \left[ - \log p (\mathbf {x} _ {a} | \mathbf {x}) \right] + \operatorname {K L} (q (\mathbf {x}) \| p _ {\boldsymbol {\theta}} (\mathbf {x})). \tag {2}
$$
As shown in Song et al. [57] and Vahdat et al. [60], we can further obtain an upper bound on the second Kullback-Leibler (KL) divergence term between the target variational posterior distribution  $q(\mathbf{x})$  and the prior distribution defined by the pre-trained diffusion models  $p_{\theta}(\mathbf{x})$ :
$$
\operatorname {K L} \left(q (\mathbf {x}) \| p _ {\boldsymbol {\theta}} (\mathbf {x})\right) \leq \mathbb {E} _ {q (\mathbf {x})} \mathbb {E} _ {t \sim \mathcal {U} (0, 1), \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I})} \left[ w (t) \| s _ {\boldsymbol {\theta}} (\mathbf {x} _ {t}; t) - \nabla_ {\mathbf {x} _ {t}} \log q (\mathbf {x} _ {t} | \mathbf {x}) \| _ {2} ^ {2} \right], \tag {3}
$$
where  $w(t) = g(t)^2 / 2$  is a time-dependent weighting coefficient,  $\mathbf{x}_t = \mathbf{x} + \sigma_t\epsilon$  denotes the forward diffusion process,  $\sigma_t$  is the pre-designed noise schedule, and  $s_\theta$  represents the pre-trained diffusion models.
The simplest approximation to the posterior is using a point estimate, i.e., the introduced variational posterior  $q(\mathbf{x})$  satisfies the Dirac delta distribution  $q(\mathbf{x}) = \delta (\mathbf{x} - \mathbf{x}_{\mu})$ . Thus, the above upper bound can be rewritten as:
$$
\mathbb {E} _ {t \sim \mathcal {U} (0, 1), \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I})} \left[ w (t) \| s _ {\boldsymbol {\theta}} (\mathbf {x} _ {t}; t) - \nabla_ {\mathbf {x} _ {t}} \log q (\mathbf {x} _ {t} | \mathbf {x} _ {\mu}) \| _ {2} ^ {2} \right], \tag {4}
$$
We simply use notation  $\mathbf{x}$  instead of  $\mathbf{x}_{\mu}$  throughout for convenience. The weighted denoising score matching objective in (4) is also equivalent to the diffusion model training loss [43].
According to Tweedie's formula:  $\mu_z = z + \Sigma_z\nabla_z\log p(z)$ , where  $\Sigma$  denotes covariance matrix, we can obtain  $\mathbf{x} = \mathbf{x}_t + \sigma_t^2\nabla_{\mathbf{x}_t}\log q(\mathbf{x}_t)$ . Defining  $D_{\pmb{\theta}}(\mathbf{x}_t;t)\coloneqq \mathbf{x}_t + \sigma_t^2 s_{\pmb{\theta}}(\mathbf{x}_t;t)$ , the KL term of our optimization objective converts to:
$$
\mathbb {E} _ {t \sim \mathcal {U} (0, 1), \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I})} \left[ \tilde {w} (t) \| D _ {\boldsymbol {\theta}} \left(\mathbf {x} + \sigma_ {t} \epsilon ; t\right) - \mathbf {x} \| _ {2} ^ {2} \right], \tag {5}
$$
where  $\tilde{w}(t) = w(t) / \sigma_t^2$ . Note that  $D_{\theta}$  can be used to estimate the denoised image directly, which is called one-shot denoiser [33, 5].
In our work, we adopt the approach of setting  $\tilde{w}(t) = 1$  for convenience and performance as in previous studies [22, 17]. Since we have no information about the conditional distribution  $p(\mathbf{x}_a|\mathbf{x})$ , we need a heuristic formulation for the first reconstruction term in (2). The simplest method is to initialize  $\mathbf{x}$  by the adversarial sample  $\mathbf{x}_a$  and use the loss in (5) to optimize over  $\mathbf{x}$  directly, eliminating the constraint term. The rationale behind this simplification is that leading  $\mathbf{x}_a$  towards the mode of the  $p_\theta(\mathbf{x})$  with the same ground-truth class label makes it easier for the natural classifier to produce a correct prediction. In this way, the loss function of our optimization process reduces to:
$$
\mathcal {L} _ {\text {D i f f}} (\mathbf {x}, \boldsymbol {\theta}) = \mathbb {E} _ {t \sim \mathcal {U} (0, 1), \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I})} \left[ \| D _ {\boldsymbol {\theta}} (\mathbf {x} + \sigma_ {t} \epsilon ; t) - \mathbf {x} \| _ {2} ^ {2} \right]. \tag {6}
$$
Randomized smoothing techniques typically assume that the adversarial perturbation follows a Gaussian distribution. Suppose the Gaussian assumption holds, we can instantiate the reconstruction term with the mean squared error (MSE) between  $\mathbf{x}$  and  $\mathbf{x}_a$ . The optimization objective converts to the following MSE loss:
$$
\mathcal {L} _ {\mathrm {M S E}} (\mathbf {x}, \mathbf {x} _ {a}, \boldsymbol {\theta}) = \mathbb {E} _ {t \sim \mathcal {U} (0, 1), \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I})} \left[ \| D _ {\boldsymbol {\theta}} (\mathbf {x} + \sigma_ {t} \epsilon ; t) - \mathbf {x} \| _ {2} ^ {2} + \lambda \| \mathbf {x} - \mathbf {x} _ {a} \| _ {2} ^ {2} \right], \tag {7}
$$
where  $\lambda$  is a weighting hyper-parameter to balance the two loss terms.
3.2.1 Score Regularization Loss
However, both Diff and MSE optimizations have their own drawbacks. Regarding the Diff loss, the optimization process solely focuses on the score prior  $p_{\theta}$ , and the update direction guided by the pre-trained diffusion models leads the samples towards the modes of the prior, gradually losing the semantic information of the original samples. As illustrated in Figure 2a, with a sufficient number of optimization steps, both standard and robust accuracies decline significantly. On the other hand,
Figure 2: Robustness performance comparison for Diff, MSE, and SR optimizations.
the MSE loss maintains a high standard accuracy at the cost of a large drop in robust accuracy, as depicted in Figure 2b. Furthermore, Figure 2c demonstrates that the performance of MSE is highly dependent on the weighting hyper-parameter, which controls the intensity of the constraint term.
To address the above-mentioned issues, we propose to introduce a hyperparameter-free score regularization (SR) loss:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {S R}} (\mathbf {x}, \mathbf {x} _ {a}, \boldsymbol {\theta}) = \mathbb {E} _ {t \sim \mathcal {U} (0, 1), \epsilon_ {1}, \epsilon_ {2} \sim \mathcal {N} (\mathbf {0}, \mathbf {I})} \\ \left[ \left\| D _ {\boldsymbol {\theta}} \left(\mathbf {x} + \sigma_ {t} \epsilon_ {1}; t\right) - \mathbf {x} \right\| _ {2} ^ {2} + \left\| D _ {\boldsymbol {\theta}} \left(\mathbf {x} + \sigma_ {t} \epsilon_ {1}; t\right) - D _ {\boldsymbol {\theta}} \left(\mathbf {x} _ {a} + \sigma_ {t} \epsilon_ {2}; t\right) \right\| _ {2} ^ {2} \right]. \tag {8} \\ \end{array}
$$
We use the introduced constraint to minimize the pixel-level distance between the denoised versions of the current sample  $\mathbf{x}$  and initial adversarial sample  $\mathbf{x}_a$ . The additional regularization term can be expanded as:
$$
\left\| D _ {\boldsymbol {\theta}} \left(\mathbf {x} + \sigma_ {t} \epsilon_ {1}; t\right) - D _ {\boldsymbol {\theta}} \left(\mathbf {x} _ {a} + \sigma_ {t} \epsilon_ {2}; t\right) \right\| _ {2} ^ {2} \approx \left\| \mathbf {x} - \mathbf {x} _ {a} + \sigma_ {t} ^ {2} \left(s _ {\boldsymbol {\theta}} \left(\mathbf {x} _ {t}; t\right) - s _ {\boldsymbol {\theta}} \left(\mathbf {x} _ {a, t}; t\right)\right) \right\| _ {2} ^ {2}. \tag {9}
$$
The SR loss encourages consistency with the original sample in terms of not only the pixel values but also the score function estimations at the given noise level  $\sigma_t$ . Since the two parts of SR loss correspond to the same noise magnitude  $t$  of score networks, there is no need to introduce an additional hyperparameter  $\lambda$  as in (7).
Figure 2b and 2c demonstrate the effectiveness of the SR loss. As the number of optimization steps increases, both the standard and robust accuracy converge to stable values, with the latter remaining close to the optimal. In contrast to the MSE loss, the SR loss shows insensitivity to the weighting hyperparameter and significantly outperforms MSE, particularly for larger values of  $\lambda$ .
3.3 Practical Algorithms
Algorithm 1: (ScoreOpt-O) Optimizing adversarial sample towards robustness with score-based prior.
Input: Adversarial image  $\mathbf{x}_a$ , pre-trained score-based diffusion model  $s_\theta$ , noise level range  $[t_{min}, t_{max}]$ , optimization iteration steps M, learning rate  $\eta$ .  
 $\mathbf{x}_0 = \mathbf{x}_a$ ;  
for  $i \in 0, \dots, M-1$  do  
    Sample  $t \sim \mathcal{U}(t_{min}, t_{max})$ ,  $\epsilon_1, \epsilon_2 \sim \mathcal{N}(0; \mathbf{I})$ ;  
     $\mathbf{x}_{i,t} = \mathbf{x}_i + \sigma_t \epsilon_1$ ;  
     $\mathbf{x}_{a,t} = \mathbf{x}_a + \sigma_t \epsilon_2$ ;  
    Calculate the gradient grad of Diff loss (6) or SR loss (8) with respect to  $\mathbf{x}_i$ ;  
     $\mathrm{grad} = \nabla_{\mathbf{x}_i} \left[ \| D_\theta(\mathbf{x}_{i,t}; t) - \mathbf{x}_i \|_2^2 + \| D_\theta(\mathbf{x}_{i,t}; t) - D_\theta(\mathbf{x}_{a,t}; t) \|_2^2 \right]$ ;  
     $\mathbf{x}_{i+1} = \mathbf{x}_i - \eta \cdot \mathrm{grad}$ ;  
end  
return Purified image  $\mathbf{x}_M$ .
Noise Schedule The perturbation introduced by adversarial attacks is often subtle and challenging for human perception. As a result, our optimization loop does not necessarily incorporate the complete
noise levels employed by the original diffusion models, which generate images from pure random noise. In fact, high noise levels will disrupt local structures and remove semantic information from the input image, leading to a decrease in accuracy. Many previous studies have focused on lower noise levels to conduct image editing tasks. Therefore, we center our pre-designed noise schedule  $\sigma_{t}$  around lower noise levels to preserve the details of the original image as much as possible. Previous diffusion-based purification methods iteratively denoise the noisy image step by step (from  $\mathbf{x}_t$  to  $\mathbf{x}_{t-1}$ ), following a predetermined noise schedule. In contrast, our optimization process does not rely on a sequential denoising schedule. Instead, at each iteration, we have the flexibility to randomly select a noise level. This approach allows us to concurrently explore different noise levels during the optimization process.
Update Rule Given a noise level  $\sigma_t$ , we can utilize the aforementioned loss functions to compute the update direction of  $\mathbf{x}$ . Noting that the objectives correspond to one single random chosen noise magnitude, we propose an alternative update rule to further improve inference speed and robustness performance. We can use the loss gradient to optimize over  $\mathbf{x}_t$  directly and then obtain the denoised image  $\mathbf{x}$  using one-shot denoising. The loss gradients with respect to  $\mathbf{x}$  and  $\mathbf{x}_t$  are equivalent in our forward process formulation. Experiments in Section 4 demonstrate that our approach significantly improves one-shot denoising with only a few optimization iterations. These two distinct update rules correspond to Algorithm 1 and Algorithm 2, respectively. Both algorithms are straightforward, effective, and easy to implement.
Algorithm 2: (ScoreOpt-N) Optimizing noisy adversarial samples and one-shot denoising.
Input: Adversarial image  $\mathbf{x}_a$ , pre-trained score-based diffusion model  $s_\theta$ , noise level range  $[t_{min}, t_{max}]$ , optimization iteration steps M and N, learning rate  $\eta$ .  
 $\mathbf{x}_0 = \mathbf{x}_a$ ;  
for  $i \in 0, \dots, M-1$  do  
//  $i$  denotes the  $i$ -th iteration of the outer loop.  
Sample  $t \sim \mathcal{U}(t_{min}, t_{max})$ ,  $\epsilon_1 \sim \mathcal{N}(0; \mathbf{I})$ ;  
 $\mathbf{x}_{0,t} = \mathbf{x}_i + \sigma_t \epsilon_1$ ;  
for  $j \in 0, \dots, N-1$  do  
//  $j$  denotes the  $j$ -th iteration of the inner loop.  
Sample  $\epsilon_2 \sim \mathcal{N}(0; \mathbf{I})$ ;  
 $\mathbf{x}_{a,t} = \mathbf{x}_a + \sigma_t \epsilon_2$ ;  
Calculate the gradient grad of Diff loss (6) or SR loss (8) with respect to  $\mathbf{x}_{j,t}$ ;  
 $\mathrm{grad} = \nabla_{\mathbf{x}_{j,t}} \left[ \| D_\theta(\mathbf{x}_{j,t}; t) - \mathbf{x}_i \|_2^2 + \| D_\theta(\mathbf{x}_{j,t}; t) - D_\theta(\mathbf{x}_{a,t}; t) \|_2^2 \right]$ ;  
 $\mathbf{x}_{j+1,t} = \mathbf{x}_{j,t} - \eta \cdot \mathrm{grad}$ ;  
end  
// One-shot denoising.  
 $\mathbf{x}_{i+1} = \mathbf{x}_{N,t} + \sigma_t^2 s_\theta(\mathbf{x}_{N,t}; t)$ ;  
end  
return Purified image  $\mathbf{x}_M$ .