SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This section presents the experimental evaluations conducted to assess the robustness of our proposed methodology against different adversarial attacks, across a range of datasets. Due to space constraints, we defer comprehensive details of the experimental settings, quantitative analysis, and additional results to the Appendix. For all experiments, we report the average results (and corresponding standard deviations) over 5 trials.
Attacks In this work, we evaluate the robustness performance of adversarial defenses mainly on  $\ell_p$ -norm bounded threat models.  $\epsilon_{p}$  denotes the perturbation budget under the  $\ell_p$ -norm. All of the attacks are based on Projected Gradient Descent (PGD) attack [36], a commonly used gradient-based white-box attack, which iteratively perturbs input images along the direction of the gradient of the classifier
Table 1: Standard and robust accuracy against BPDA+EOT attack under  $\ell_{\infty}(\epsilon = 8 / 255)$  threat model on CIFAR10, compared with other preprocessor-based adversarial defenses and adversarial training methods against white-box attacks.
Type	Architecture	Method	Standard (%)	Robust (%)	Avg. (%)
Base Classifier	WRN-28-10	-	95.32	0.0	-
Adversarial Training	ResNet18	Madry et al. [36]	87.30	45.80	66.55
Zhang et al. [71]	84.90	45.80	65.35
WRN-28-10	Carmon et al. [6]	89.67	63.10	76.39
Gowal et al. [15]	89.48	64.08	77.28
Adversarial Purification	ResNet18	Yang et al. [68]	94.80	40.80	67.80
ResNet62	Song et al. [55]	95.00	9.00	52.00
Hill et al. [20]	84.12	54.90	69.51
Yoon et al. [69]	86.14	70.01	78.08
WRN-28-10	Wang et al. [63]	93.50	79.83	86.67
Nie et al. [40]	89.02	81.40	85.21
Ours(o)	90.78±0.40	82.85±0.26	86.82
Ours(n)	93.44±0.40	90.59±0.08	92.02
loss function and projects into some  $l_{p}\epsilon$ -ball:  $\delta \gets \Pi_{\epsilon}(\delta + \alpha \cdot \mathrm{sign}(\nabla_{x}\mathrm{Loss}(f(x + \delta),y)))$ , where  $f$  represents a classifier.
Datasets and Baselines Three datasets are considered in our experiments: CIFAR10, CIFAR100, and ImageNet. Following previous works, the evaluation is conducted on the test set of each dataset. We adopt two kinds of adversarial defenses for comparison: adversarial training methods and adversarial purification methods, especially those also based on diffusion models [69, 40, 63]. In accordance with the settings in Nie et al. [40], we conduct evaluations against strong adaptive attacks using a fixed subset of 512 randomly sampled images.
Evaluation Metrics We leverage two evaluation metrics to assess the performance of our proposed defense method: standard accuracy and robust accuracy. Standard accuracy measures the performance of adversarial defenses on clean data, while robust accuracy measures the classification performance on adversarial examples generated by various attacks.
Models We consider different architectures for a fair comparison with previous works. We utilize the ResNet [19] and WideResNet (WRN) [70] backbones as our base classifiers. For CIFAR10 and CIFAR100 datasets, we employ WRN-28-10 and WRN-70-16, the two most common architectures on adversarial benchmarks. For the ImageNet dataset, we select WRN-50-2, ResNet-50, and ResNet-152, which are extensively used in adversarial defenses. As for the pre-trained diffusion models, we incorporate two popular architectures, the elucidating diffusion model (EDM) [29] and the guided diffusion model [9]. The diffusion models and classifiers are trained independently on the same training dataset as in prior studies.
In this section, we validate our defense method against two strong adaptive attacks: BPDA+EOT and adaptive white-box attack, i.e., PGD+EOT. The evaluation results of transfer-based attacks are deferred to Appendix B.
4.2.1 BPDA+EOT Attack
The combination of Backward Pass Differentiable Approximation (BPDA) [1] with Expectation over Transformation (EOT) [2] is commonly used for evaluating randomized adversarial purification methods. In order to compare with other test-time purification models, we follow the same settings in Hill et al. [20] with default hyperparameters for evaluation against BPDA+EOT attack. We conduct experiments under  $\ell_{\infty}(\epsilon = 8 / 255)$  threat model on CIFAR10 and CIFAR100 datasets. Our method achieves high robust accuracies, outperforming existing diffusion-based purification methods by a significant margin, while maintaining high standard accuracies.
CIFAR10 Table 1 shows the robustness performance against BPDA+EOT attack under the  $\ell_{\infty}$  ( $\epsilon = 8/255$ ) threat models on CIFAR10 dataset. Mark  $o$  in parentheses denotes the practi
Table 2: Standard accuracy and robust accuracy (%) against BPDA+EOT attack under  $\ell_{\infty}(\epsilon = 8 / 255)$  threat model on the CIFAR100 dataset.
Accuracy	Hill et al. [20]	Yoon et al. [69]	Diffusion-based Purification (step)	Ours
1	10	20	40	80	(o)	(n)
Standard	51.66	60.66	61.52	67.77	69.92	67.58	66.99	69.92	75.98
Robust	26.10	39.72	40.03	47.66	48.83	48.05	47.85	66.99	65.43
Table 3: Standard and robust accuracy against PGD+EOT on CIFAR-10. Left:  $\ell_{\infty}(\epsilon = 8 / 255)$ ; Right:  $\ell_{2}(\epsilon = 0.5)$ . Compared with adversarial training (AT) and purification (AP) methods.
Type	Method	Standard	Robust
WideResNet-28-10
AT	Pang et al. [42]	88.62	64.95
Gowal et al. [15]	88.54	65.10
Gowal et al. [16]	87.51	66.01
AP	Yoon et al. [69]	85.66	37.27
Nie et al. [40]	90.07	51.25
Ours	95.02±0.30	67.68±0.29
WideResNet-70-16
AT	Gowal et al. [16]	88.75	69.03
Wang et al. [64]	92.97	72.46
AP	Yoon et al. [69]	86.76	41.02
Nie et al. [40]	90.43	57.03
Ours	95.18±0.09	71.48±0.20
Type	Method	Standard	Robust
WideResNet-28-10
AT	Sehwag et al. [50]	90.93	83.75
Rebuffi et al. [44]	91.79	85.05
Augustin et al. [3]	93.96	86.14
AP	Yoon et al. [69]	85.66	74.26
Nie et al. [40]	91.41	82.11
Ours	94.99±0.09	86.78±0.33
WideResNet-70-16
AT	Rebuffi et al. [44]	92.41	86.24
Wang et al. [64]	96.09	86.72
AP	Yoon et al. [69]	86.76	75.90
Nie et al. [40]	92.15	84.80
Ours	95.18±0.18	87.11±0.01
cal method ScoreOpt-  $O$  in Algorithm 1, and mark  $n$  denotes the method ScoreOpt-  $N$  in Algorithm 2. Our methods achieve surprisingly good results. Specifically, we obtain  $90.59\%$  robust accuracy, with absolute improvements of  $9.19\%$  over previous SOTA adversarial purification methods. Our method is the first adaptive test-time defense to achieve robust accuracy over  $90\%$  against the BPDA+EOT attack. Meanwhile, the standard accuracy result is on par with the best-performing method.
CIFAR100 We also conduct robustness evaluations against strong adaptive attacks under the  $\ell_{\infty}$  ( $\epsilon = 8/255$ ) threat model on the CIFAR100 dataset. The results of ScoreOpt and other adaptive purification methods are presented in Table 2. To showcase the superiority of our method over previous diffusion-based purification algorithms, we perform experiments using the approach proposed in Nie et al. [40] with varying reverse denoising steps. The optimal hyperparameter  $t^{*}$ , representing the forward timestep, is carefully tuned based on experimental results. As indicated in Table 2, increasing the reverse denoising steps only leads to marginal improvements over the one-shot denoising method. Even when the number of denoising steps is increased to 80, which will incur a large computational cost, no further improvement in robust accuracy is observed. In contrast, our ScoreOpt method achieves a notable improvement of  $18.16\%$  in robust accuracy.
4.2.2 Adaptive White-box Attack
In order to evaluate our method against white-box attacks, it is necessary to compute the exact gradient of the entire defense framework. However, our optimization process involves backpropagation through the U-net architecture of diffusion models, making it challenging to directly apply PGD attack. Taking inspiration from Lee and Kim [34], we approximate the full gradient using the one-shot denoising process in our experiments. The performance against PGD+EOT is shown in Table 3. Most results for baselines are taken from Lee and Kim [34]. Notably, our method significantly outperforms other diffusion-based purification methods. Specifically, compared to Nie et al. [40], our method improves robust accuracy by  $16.43\%$  under  $\ell_{\infty}$  and by  $4.67\%$  under  $\ell_{2}$  on WideResNet-28-10, and by  $14.45\%$  under  $\ell_{\infty}$  and by  $2.31\%$  under  $\ell_{2}$  on WideResNet-70-16, respectively. Compared with previous SOTA adversarial training methods, ScoreOpt achieves better robust accuracies on both WideResNet-28-10 and WideResNet-70-16 under  $\ell_{2}$ . Furthermore, under  $\ell_{\infty}$  threat model, ScoreOpt outperforms AT baselines on WideResNet-28-10 and achieves comparable robust accuracy on WideResNet-70-16 with the top-rank model.
Table 4: Standard and robust accuracy against unseen threat models with WRN-70-16 on CIFAR10. Threat that AT is trained with is marked with an underline, and others are considered unseen.
Defense	Standard (%)	Robust (%)
\( \ell_{\infty} \)	\( \ell_2 \)	StAdv
Base WRN-70-16	95.31	-	-	-
Wang et al. [64] (Trained with \( \ell_{\infty} \))	92.97	72.46	71.68	3.13
Wang et al. [64] (Trained with \( \ell_2 \))	96.09	56.64	86.72	5.27
Ours	95.18	71.48	87.11	93.36
Table 5: Robustness against common corruptions on CIFAR10-C.
Defense	gaussian	shot	impulse	elastic transform	pixelate	jpeg compression	snow	frost	fog	brightness
Base WRN-70-16	25.71	32.19	27.89	73.17	44.93	67.32	77.82	66.49	71.37	91.33
Wang et al. [64] (l∞)	79.85	81.22	62.02	86.14	88.89	89.93	84.75	83.94	37.87	88.04
Wang et al. [64] (l2)	81.91	83.17	63.74	89.96	92.0	93.61	89.26	88.17	48.55	92.88
Ours	89.94	87.96	82.68	83.64	86.36	89.26	83.5	85.73	80.2	91.21
Approximate Gradient. To show the effectiveness of our approximate gradients for sufficient robustness evaluation, we evaluate ScoreOpt (with only one single optimization step) on a fixed 512 CIFAR-10 subset. We use exact gradients and approximate gradients obtained by backpropagating through the one-shot denoising surrogate process, respectively. The attack success rates of PGD+EOT are  $33.99\%$  (w/ exact gradients) vs  $34.11\%$  (w/ approximate gradients) under  $\ell_{2}$ , and  $50.976563\%$  vs  $50.976562\%$  under  $\ell_{\infty}$ . Therefore, this kind of approximate gradients obtained by the one-shot denoising surrogate process can be considered as effective as computing the exact gradient to sufficiently evaluate the robustness of ScoreOpt.
4.3 Generalization to Different Types of Attacks
Defense against Unseen Threats A significant advantage of adversarial purification is that it can defend against unseen threats in a plug-and-play fashion, while most adversarial training methods exhibit robustness only against the specific threat trained with. Following Nie et al. [40], we conduct evaluations with three threat models:  $\ell_{\infty}$ ,  $\ell_{2}$ , and the spatially transformed adversarial examples (StAdv). Table 4 illustrates the poor generalization of AT methods to unseen attacks and the significant generalization ability of our proposed method. The performance of AT baselines drops significantly when confronted with unseen attacks while ScoreOpt demonstrates robustness across all three threat models.
Score-based Black-box Attack We further assess the effectiveness of our method against the score-based black-box attack, SPSA. This evaluation is conducted on CIFAR10 with the WRN-28-10 classifier, adopting the same experimental setup as described in Yoon et al. [69], using 1,280 queries. Notably, our ScoreOpt method achieves a robust accuracy of  $91.797\%$ , in contrast to the reported  $80.8\%$  robust accuracy in Yoon et al. [69]. This demonstrates a significant performance improvement of  $10\%$  over the previous method.
Common Corruption To further highlight the generalization ability of ScoreOpt against various types of perturbations, we test the robustness of ScoreOpt and AT baselines on CIFAR10-C, a dataset comprising 75 frequently encountered visual distortions. The evaluation results are summarized in Table 5. where ScoreOpt consistently outperforms the AT baselines across most corruptions.
4.4.1 Combining with adversarial training methods
ScoreOpt can also be seamlessly combined with existing stronger classifiers in a plug-and-play manner to further improve robustness performance. To see this, we conduct an additional experiment by combining ScoreOpt with adversarially trained classifiers [64]. We evaluate against the PGD+EOT attack and observe improvements under both  $l_{\infty}$  (from 71.48% to 72.57%) and  $l_{2}$  (from 87.11% to 89.06%), respectively.
Figure 3: Accuracy with respect to optimization steps.
Table 6: The inference time cost of ScoreOpt with different optimization steps, compared with previous diffusion-based purification methods with different denoising steps.
Denoising Steps	1	5	10	20	50	100
Time Cost (s)	0.0524	0.1930	0.3271	0.6298	1.7125	3.3351
Optimization Steps	1	5	10	20	50	100
Time Cost (s)	0.1172	0.3632	0.6271	1.1747	3.2978	5.8181
4.4.2 Optimization Steps
Since our defense is an iterative optimization process, we conduct ablation experiments on our ScoreOpt algorithm with different optimization steps under  $\ell_{\infty}$  ( $\epsilon = 8/255$ ) threat model against BPDA+EOT on CIFAR10. Figure 3 shows that the standard accuracy and robust accuracy continuously increase as the number of optimization steps increases. As the number of iterations increases further, the accuracies gradually stabilize. This phenomenon shows the effectiveness of our optimization process.
4.4.3 Inference Speed
We compare the inference speed of ScoreOpt with diffusion purification methods that use a sequential multiple-step denoising process. We compute the inference time cost per image. As shown in Table 6, our time cost is about twice under the same steps. However, ScoreOpt needs only a few steps (about 5) to obtain significantly better results than the multi-step denoising method, which requires nearly 100 denoising steps. Therefore, compared to diffusion-based AP baselines, our method further improves the inference speed substantially in practice.
In this work, we have proposed a novel adversarial defense framework ScoreOpt, which optimizes over the input image space to recover original images. We have introduced three optimization objectives using pre-trained score-based priors, followed by practical algorithms. We have shown that ScoreOpt can quickly purify attacked images within a few optimization steps. Experimentally, we have shown that our approach yields significant improvements over prior works in terms of both robustness performance and inference speed. We would believe our work further demonstrates the powerful capabilities of pre-trained generative models for downstream tasks.
This work has been supported by the National Key Research and Development Project of China (No. 2022YFA1004002) and the National Natural Science Foundation of China (No. 12271011).
[1] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pages 274-283. PMLR, 2018.
[2] Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, pages 284-293. PMLR, 2018.
[3] Maximilian Augustin, Alexander Meinke, and Matthias Hein. Adversarial robustness on in- and out-distribution improves explainability. In European Conference on Computer Vision, 2020.
[57] Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34, 2021.
[58] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[59] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
[60] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. In Neural Information Processing Systems, 2021.
[61] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661-1674, 2011.
[62] Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. arXiv preprint arXiv:2212.00774, 2022.
[63] Jinyi Wang, Zhaoyang Lyu, Dahua Lin, Bo Dai, and Hongfei Fu. Guided diffusion model for adversarial purification. arXiv preprint arXiv:2205.14969, 2022.
[64] Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In International Conference on Machine Learning (ICML), 2023.
[65] Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In Advances in Neural Information Processing Systems, volume 33, pages 2958-2969, 2020.
[66] Quanlin Wu, Hang Ye, and Yuntian Gu. Guided diffusion model for adversarial purification from random noise. arXiv preprint arXiv:2206.10875, 2022.
[67] Chaowei Xiao, Zhongzhu Chen, Kun Jin, Jiongxiao Wang, Weili Nie, Mingyan Liu, Anima Anandkumar, Bo Li, and Dawn Song. Densepure: Understanding diffusion models towards adversarial robustness. arXiv preprint arXiv:2211.00322, 2022.
[68] Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation. In International Conference on Machine Learning, 2019.
[69] Jongmin Yoon, Sung Ju Hwang, and Juho Lee. Adversarial purification with score-based generative models. In International Conference on Machine Learning, pages 12062-12072. PMLR, 2021.
[70] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In *British Machine Vision Conference* 2016. British Machine Vision Association, 2016.
[71] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
A.1 Adversarial attack
Adversarial attacks are aimed at manipulating or deceiving machine learning models by introducing imperceptible perturbations to input data, which can lead to misclassification. These attacks can be categorized into three types: black-box, gray-box, and white-box attacks, with increasing intensity.
In a black-box attack setting, the attacker has no knowledge of the internal structure of both the defender and the classifier. Conversely, white-box attacks have full access to all information about the pre-processor and the target classifier. Defending against white-box attacks is the most difficult task.
And gray-box attacks have partial information about the whole target model. Attackers usually have full access to the classifier but lack knowledge about the pre-processor. Adversarial examples are generated exclusively based on the classifier and evaluated using the entire model. This type of malicious perturbation is inherently limited in its effectiveness compared to white-box attacks. It is important to note that gray-box attacks can be seen as transfer-based black-box attacks, where the raw classifier serves as the source model, and the entire model (including the pre-processor) represents the target model.
A.2 Adversarial defense
To enhance the resistance of classifiers against adversarial attacks, two primary approaches have been employed. The first approach is adversarial training (AT), which has proven to be highly effective in defending against such attacks [36, 32, 71, 44]. AT involves training classifiers using both clean data with ground-truth labels and adversarily perturbed samples. However, a notable limitation of AT is that it can only defend against attacks that the model has been specifically trained on, necessitating retraining the classifier when confronted with new attack types.
Purification, which involves applying a pre-processing procedure to input data before it is fed into classifiers, is another approach for enhancing adversarial robustness. Several existing studies have explored the effectiveness of different purification models, such as denoising auto-encoders [24, 27], sets of image filters [8], and compression-and-recovery models [26], among others. Among these purification methods, generative models [55, 12, 49] have demonstrated significant effectiveness in transforming adversarial data into clean data. The underlying idea of this technique is to learn the underlying distribution of clean data and use it to transform adversarial examples.
B Attack Transferred from Base Classifier
We evaluate our defense against adversarial images generated from the PGD attack only on the base classifier. The attack process does not incorporate the pre-processing model. We conduct experiments against Transfer-PGD attacks for both CIFAR10 and ImageNet datasets. The experimental settings adhere to previous purification methods. In summary, our proposed method exhibits substantial improvements in both standard and robust accuracy measures.
CIFAR10 Table 7 and 8 show the robustness results against Transfer-PGD attack under the  $\ell_{\infty}$  ( $\epsilon = 8/255$ ) and  $\ell_{2}$  ( $\epsilon = 0.5$ ) threat models on CIFAR10 dataset, respectively. We compare our method with two kinds of defenses: adversarial training and adversarial purification methods. The results demonstrate that the robust accuracies of our two methods achieve state-of-the-art performance, with comparable standard accuracies. Specifically, our method outperforms existing adversarial purification methods, including diffusion-based multi-step denoising models. Under the  $\ell_{\infty}$  ( $\epsilon = 8/255$ ) threat models, our proposed method improves robust accuracy by  $2.2\%$ . As for the  $\ell_{2}$  ( $\epsilon = 0.5$ ) threats, our algorithms still achieve a substantial improvement over existing methods in terms of both standard accuracy  $93.1\%$  and robust accuracy  $92.52\%$ .
IMAGENET Table 9 presents the robustness performance of our method on the Imagenet dataset. We evaluate our method under  $\ell_{\infty}(\epsilon = 4 / 255)$  threat model with different perturbation budgets on three different backbones with different natural classification accuracies. Results show that the effectiveness of our method is not affected by the underlying classifier architecture. Even under larger attack budgets, we find that our method still keeps the ability to purify adversarial perturbations and generally achieves over  $60\%$  robust accuracy.
Table 7: Standard and robust accuracy against Transfer-PGD attack under  $\ell_{\infty}(\epsilon = 8/255)$  threat model on CIFAR10, compared with other preprocessor-based adversarial defenses and adversarial training methods against transfer-based attacks.
Type	Architecture	Method	Standard (%)	Robust (%)	Avg. (%)
Base Classifier	WRN-28-10	-	95.19	0.00	-
Adversarial Training	ResNet56	Madry et al. [36]	87.30	70.20	78.75
Zhang et al. [71]	84.90	72.20	78.55
Adversarial Purification	ResNet18	Yang et al. [68]	94.90	82.50	88.70
ResNet62	Song et al. [55]	90.00	70.00	80.00
Du and Mordatch [11]	48.70	37.50	43.10
Grathwohl et al. [18]	75.50	23.80	49.65
Hill et al. [20]	84.12	78.91	81.52
WRN-28-10	Ho and Vasconcelos [21]	89.26	80.80	85.03
Yoon et al. [69]	93.09	85.45	89.27
Wang et al. [63]	93.50	90.10	91.80
Ours(o)	91.88±0.05	90.11±0.05	91.00
Ours(n)	93.11±0.18	92.30±0.15	92.71
Table 8: Standard and robust accuracy against Transfer-PGD attack under  $\ell_2(\epsilon = 0.5)$  threat model on CIFAR10.
Models	Accuracy (%)
Standard	Robust
Base Classifier	95.19	0.30
Rony et al. [47]	89.05	67.60
Ding et al. [10]	88.02	66.18
Rice et al. [45]	88.67	71.60
Wu et al. [65]	88.51	73.66
Gowal et al. [15]	90.90	74.50
Ours(o)	91.85±0.02	90.55±0.08
Ours(n)	93.10±0.20	92.52±0.14