SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} D _ {T V} \left(q _ {t}, q _ {t} ^ {\prime}\right) \leq D _ {T V} \left(q _ {t}, p _ {t}\right) + D _ {T V} \left(p _ {t}, p _ {t} ^ {\prime}\right) + D _ {T V} \left(q _ {t} ^ {\prime}, p _ {t} ^ {\prime}\right) (56) \\ \leq \sqrt {2 - 2 \exp \left\{- \frac {1}{8} \left(1 - \bar {\alpha} _ {t}\right) \| \delta \| _ {2} ^ {2} \right\}} + \epsilon_ {r e} + \sqrt {2} H \left(q _ {t} ^ {\prime}, p _ {t} ^ {\prime}\right) (57) \\ \leq \sqrt {2 - 2 \exp \left\{- C _ {2} \left(C _ {1} \| \delta \| _ {2} + (\sqrt {L _ {u}} + 2 M) \sum_ {k = t + 1} ^ {T} \lambda (k , t) + \sum_ {k = t + 1} ^ {T} \lambda (k , t) \| \boldsymbol {s} _ {\theta} \left(\tilde {\mathbf {x}} _ {k} ^ {\prime} , k\right) - \epsilon \left(\tilde {\mathbf {x}} _ {k} ^ {\prime} , k\right) \| _ {2}\right) ^ {2} \right\}} (58) \\ + \sqrt {2 - 2 \exp \left\{- \frac {1}{8} \left(1 - \bar {\alpha} _ {t}\right) \| \delta \| _ {2} ^ {2} \right\}} + \epsilon_ {r e} (59) \\ = \sqrt {2 - 2 \exp \left\{- C _ {2} \left(\sum_ {k = t + 1} ^ {T} \lambda (k , t) \| \boldsymbol {s} _ {\theta} \left(\tilde {\mathbf {x}} _ {k} ^ {\prime} , k\right) - \epsilon \left(\tilde {\mathbf {x}} _ {k} ^ {\prime} , k\right) \| _ {2} + C _ {1} \| \delta \| _ {2} + (\sqrt {L _ {u}} + 2 M) \sum_ {k = t + 1} ^ {T} \lambda (k , t)\right) ^ {2} \right\}} (60) \\ + \sqrt {2 - 2 \exp \{- \frac {1}{8} (1 - \Pi_ {s = 1} ^ {t} (1 - \beta_ {s})) \| \delta \| _ {2} ^ {2} \}} + \epsilon_ {r e} (61) \\ \end{array}
$$
D Experiment
D.1 Pseudo-code of DiffAttack
Given an input pair  $(\mathbf{x},y)$  and the perturbation budget, we denote  $\mathcal{L} \coloneqq \mathcal{L}_{cls} + \lambda \mathcal{L}_{dev}$  (Equation (8)) the surrogate loss,  $\Pi$  the projection operator given the perturbation budget and distance metric,  $\eta$  the step size,  $\alpha$  the momentum coefficient,  $N_{\mathrm{iter}}$  the number of iterations, and  $W$  the set of checkpoint iterations. Concretely, we select  $\mathcal{L}_{cls}$  as the cross-entropy loss in the first round and DLR loss in the second round following [34]. The gradient of the surrogate loss with respect to the samples is computed by forwarding the samples and backwarding the gradients for multiple times and taking the average to tackle the problem of randomness. We also optimize all the samples, including the misclassified ones, to push them away from the decision boundary. The gradient can be computed with our segment-wise forwarding-backwarding algorithm in Section 3.3, which enables DiffAttack to be the first fully adaptive attack against the DDPM-based purification defense. The complete procedure is provided in Algorithm 2.
D.2 Experiment details
We use pretrained score-based diffusion models [49] on CIFAR-10, guided diffusion models [15] on ImageNet, and DDPM [25] on CIFAR-10 to purify the images following the literature [34, 4, 55, 56]. Due to the high computational cost, we follow [34] to randomly select a fixed subset of 512 images sampled from the test set to evaluate the robust accuracy for fair comparisons. We implement DiffAttack in the framework of AutoAttack [12], and we use the same hyperparameters. Specifically, the number of iterations of attacks  $(N_{\mathrm{iter}})$  is 100, and the number of iterations to approximate the gradients (EOT) is 20. The momentum coefficient  $\alpha$  is 0.75, and the step size  $\eta$  is initialized with  $2\epsilon$  where  $\epsilon$  is the maximum  $\ell_p$ -norm of the perturbations. The balance factor  $\lambda$  between the classification-guided loss and the deviated-reconstruction loss in Equation (8) is fixed as 1.0 and  $\alpha(\cdot)$  is set the reciprocal of the size of sampled time steps in the evaluation. We consider  $\epsilon = 8/255$  and  $\epsilon = 4/255$  for  $\ell_{\infty}$  attack and  $\epsilon = 0.5$  for  $\ell_2$  attack following the literature [11, 12].
We use randomly selected 3 seeds and report the averaged results for evaluations. CIFAR-10 is under the MIT license and ImageNet is under the BSD 3-clause license.
More details of baselines. In this part, we illustrate more details of the baselines 1) SOTA attacks against score-based diffusion adjoint attack (AdjAttack) [34], 2) SOTA attack against DDPM-based
Algorithm 2 DiffAttack
1: Input:  $\mathcal{L} \coloneqq \mathcal{L}_{cls} + \lambda \mathcal{L}_{dev}, \Pi, (\mathbf{x}, y), \eta, \alpha, N_{\mathrm{iter}}, W = \{w_0, \dots, w_n\}$
2: Output:  $\tilde{\mathbf{x}}$
3:  $\tilde{\mathbf{x}}^{(0)}\gets \tilde{\mathbf{x}}$
4:  $\tilde{\mathbf{x}}^{(1)}\gets \Pi \left(\tilde{\mathbf{x}}^{(0)} + \eta \nabla_{\tilde{\mathbf{x}}^{(0)}}\mathcal{L}(\tilde{\mathbf{x}}^{(0)},y)\right)$
5:  $L_{\mathrm{max}} \gets \max \{\mathcal{L}(\tilde{\mathbf{x}}^{(0)}, y), \mathcal{L}(\tilde{\mathbf{x}}^{(1)}, y)\}$
6:  $\tilde{\mathbf{x}}\gets \tilde{\mathbf{x}}^{(0)}$  if  $L_{\mathrm{max}}\equiv \mathcal{L}(\tilde{\mathbf{x}}^{(0)},y)$  else  $\tilde{\mathbf{x}}\gets \tilde{\mathbf{x}}^{(1)}$
7: for  $k = 1$  to  $N_{\mathrm{iter}} - 1$  do
8:  $\mathbf{z}^{(k + 1)}\gets \Pi \left(\widetilde{\mathbf{x}}^{(k)} + \eta \nabla_{\widetilde{\mathbf{x}}^{(k)}}\mathcal{L}\left(\widetilde{\mathbf{x}}^{(k)},y\right)\right)$
9:  $\tilde{\mathbf{x}}^{(k + 1)}\gets \Pi \left(\tilde{\mathbf{x}}^{(k)} + \alpha (\mathbf{z}^{(k + 1)} - \tilde{\mathbf{x}}^{(k)}) + (1 - \alpha)(\tilde{\mathbf{x}}^{(k)} - \tilde{\mathbf{x}}^{(k - 1)})\right)$
0: if  $\mathcal{L}(\tilde{\mathbf{x}}^{(k + 1)},y) > L_{\mathrm{max}}$  then
1:  $\tilde{\mathbf{x}}\gets \tilde{\mathbf{x}}^{(k + 1)}$  and  $L_{\mathrm{max}}\gets \mathcal{L}(\tilde{\mathbf{x}}^{(k + 1)},y)$
2: end if
3: if  $k \in W$  then
4:  $\eta \gets \eta /2$
5: end if
6: end for
diffusion Diff-BPDA attack [4], 3) SOTA black-box attack  $SPSA$  [53] and square attack [1], and 4) specific attack against EBM-based purification joint attack (score/full) [60]. AdjAttack selects the surrogate loss  $\mathcal{L}$  as the cross-entropy loss and DLR loss and leverages the adjoint method [28] to efficiently backpropagate the gradients through SDE. The basic idea is to obtain the gradients via solving an augmented SDE. For the SDE in Equation (4), the augmented SDE that computes the gradients  $\partial \mathcal{L} / \partial \mathbf{x}_T^{\prime}$  of back=propagating through it is given by:
$$
\left( \begin{array}{c} \mathbf {x} _ {T} ^ {\prime} \\ \frac {\partial \mathcal {L}}{\partial \mathbf {x} _ {T} ^ {\prime}} \end{array} \right) = \operatorname {s d e i n t} \left(\left( \begin{array}{c} \mathbf {x} _ {0} ^ {\prime} \\ \frac {\partial \mathcal {L}}{\partial \mathbf {x} _ {0} ^ {\prime}} \end{array} \right), \tilde {\mathbf {f}}, \tilde {\mathbf {g}}, \tilde {\mathbf {w}}, 0, T\right) \tag {62}
$$
where  $\frac{\partial\mathcal{L}}{\partial\mathbf{x}_0'}$  is the gradient of the objective  $\mathcal{L}$  w.r.t. the  $\mathbf{x}_0'$ , and
$$
\tilde {\mathbf {f}} ([ \mathbf {x}; \mathbf {z} ], t) = \left( \begin{array}{c} \mathbf {f} _ {\mathrm {r e v}} (\mathbf {x}, t) \\ \frac {\partial \mathbf {f} _ {\mathrm {r e v}} (\mathbf {x} , t)}{\partial \mathbf {x}} \mathbf {z} \end{array} \right)
$$
$$
\tilde {\mathbf {g}} (t) = \left( \begin{array}{c} - \mathbf {g} _ {\text {r e v}} (t) \mathbf {1} _ {d} \\ \mathbf {0} _ {d} \end{array} \right) \tag {63}
$$
$$
\tilde {\mathbf {w}} (t) = \left( \begin{array}{c} - \mathbf {w} (1 - t) \\ - \mathbf {w} (1 - t) \end{array} \right)
$$
with  $\mathbf{1}_d$  and  $\mathbf{0}_d$  representing the  $d$ -dimensional vectors of all ones and all zeros, respectively and  $\mathbf{f}_{\mathrm{rev}}(\mathbf{x},t) := -\frac{1}{2}\beta (t)\mathbf{x} - \beta (t)\nabla_{\mathbf{x}}\log p_t(\mathbf{x}),\mathbf{g}_{\mathrm{rev}}(t) := \sqrt{\beta(t)}$ .
SPSA attack approximates the gradients by randomly sampling from a pre-defined distribution and using the finite-difference method. Square attack heuristically searches for adversarial examples in a low-dimensional space with the constraints of the perturbation pattern (i.e., constraining the square shape of the perturbation). Joint attack (score) updates the input by the weighted average of the classifier gradient and the output of the score estimation network (i.e., the gradient of log-likelihood with respect to the input), while joint attack (full) leverages the classifier gradients and the difference between the input and the purified samples. The update of the joint attack (score) is formulated as follows:
$$
\tilde {\mathbf {x}} \leftarrow \tilde {\mathbf {x}} + \eta \left(\lambda^ {\prime} \operatorname {s i g n} \left(s _ {\theta} (\tilde {\mathbf {x}})\right) + (1 - \lambda^ {\prime}) \operatorname {s i g n} \left(\nabla_ {\tilde {\mathbf {x}}} \mathcal {L} (F (P (\tilde {\mathbf {x}})), y)\right) \right. \tag {64}
$$
The update of the joint attack (full) is formulated as follows:
$$
\tilde {\mathbf {x}} \leftarrow \tilde {\mathbf {x}} + \eta \left(\lambda^ {\prime} \operatorname {s i g n} (F (P (\tilde {\mathbf {x}})) - \tilde {\mathbf {x}}) + (1 - \lambda^ {\prime}) \operatorname {s i g n} \left(\nabla_ {\tilde {\mathbf {x}}} \mathcal {L} (F (P (\tilde {\mathbf {x}})), y)\right) \right. \tag {65}
$$
where  $\eta$  is the step size and  $\lambda^\prime$  the balance factor fixed as 0.5 in the evaluation.
Table 5: Comparisons of gradient backpropagation time per batch(second)/Memory cost (MB) between [4] and DiffAttack. We evaluate on CIFAR-10 with WideResNet-28-10 with batch size 16.
Method	T=5	T=10	T=15	T=20	T=30	T=1000
[4]	0.45/14,491	0.83/23,735	1.25/32,905	1.80/38,771	—	—
DiffAttack	0.44/2,773	0.85/2,731	1.26/2,805	1.82/2,819	2.67/2,884	85.81/3,941
Table 6: The clean / robust accuracy (\%) of diffusion-based purification with different diffusion lengths  $T$  under DiffAttack. The evaluation is done on ImageNet with ResNet-50 under  $\ell_{\infty}$  attack ( $\epsilon = 4/255$ ).
T = 50	T = 100	T = 150	T = 200
71.88 / 12.46	68.75 / 24.62	67.79 / 28.13	65.62 / 26.83
D.3 Additional Experiment Results
Efficiency evaluation. We evaluate the wall clock time per gradient backpropagation of the segment-wise forwarding-backwarding algorithm for different diffusion lengths and compare the time efficiency as well as the memory costs with the standard gradient backpropagation in previous attacks [4]. The results in Table 5 indicate that the segment-wise forwarding-backwarding algorithm consumes comparable wall clock time per gradient backpropagation compared with [4] and achieves a much better tradeoff between time efficiency and memory efficiency. The evaluation is done on an RTX A6000 GPU with  $49,140\mathrm{MB}$  memory. In the segment-wise forwarding-backwarding algorithm, we require one forward pass and one backpropagation pass in total for the gradient computation, while the standard gradient backpropagation in [4] requires one backpropagation pass. However, since the backpropagation pass is much more expensive than the forward pass [36], our segment-wise forwarding-backwarding algorithm can achieve comparable time efficiency while significantly reducing memory costs.
More ablation studies on ImageNet. We conduct more evaluations on ImageNet to consolidate the findings in CIFAR-10. We evaluate the clean/robust accuracy (\%) of diffusion-based purification with different diffusion lengths  $T$  under DiffAttack. The results in Table 6 indicate that 1) the clean accuracy of the purification defenses negatively correlates with the diffusion lengths, and 2) a moderate diffusion length benefits the robust accuracy under DiffAttack.
Tansferability of DiffAttack. ACA [10] and Diff-PGD attack [58] explore the transferability of unrestricted adversarial attack, which generates realistic adversarial examples to fool the classifier and maintain the photorealism. They demonstrate that this kind of semantic attack transfers well to other models. To explore the transferability of adversarial examples by  $\ell_p$ -norm-based DiffAttack, we evaluate the adversarial examples generated on score-based purification with ResNet-50 on defenses with pretrained WRN-50-2 and DeiT-S. The results in Table 7 indicate that DiffAttack also transfers better than AdjAttack and achieves much lower robust accuracy on other models.
Ablation study of balance factor  $\lambda$ . As shown in Equation (11),  $\lambda$  controls the balance of the two objectives. A small  $\lambda$  can weaken the deviated-reconstruction object and make the attack suffer more from the vanishing/exploded gradient problem, while a large  $\lambda$  can downplay the guidance of the classification loss and confuse the direction towards the decision boundary of the classifier. The results in Table 8 show that selecting  $\lambda$  as 1.0 achieves better tradeoffs empirically, so we fix it as 1.0 for experiments.
D.4 Visualization
In this section, we provide the visualization of adversarial examples generated by DiffAttack. Based on the visualization on CIFAR-10 and ImageNet with different network architectures, we conclude that the perturbation generated by DiffAttack is stealthy and imperceptible to human eyes and hard to be utilized by defenses.
Table 7: Robust accuracy  $(\%)$  with  $\ell_{\infty}$  attack  $(\epsilon = 8 / 255)$  against score-based diffusion purification on CIFAR-10. The adversarial examples are optimized on the diffusion purification defense with pretrained ResNet-50 and evaluated on defenses with other types of models including WRN-50-2 and DeiT-S.
ResNet-50	WRN-50-2	DeiT-S
AdjAttack	40.93	52.37	54.53
DiffAttack	28.13	37.28	39.62
Table 8: The impact of different loss weights  $\lambda$  on the robust accuracy  $\left( \% \right)$  . We perform  ${\ell }_{\infty }$ $\left( {\epsilon  = 8/{255}}\right)$  against score-based diffusion purification on CIFAR-10 with WideResNet-28-10 and diffusion length  $T = {0.1}$  .
λ = 0.1	λ = 1.0	λ = 10.0
54.69	46.88	53.12
Figure 5: Visualization of the clean images and adversarial samples generated by DiffAttack on CIFAR-10 with  $\ell_{\infty}$  attack ( $\epsilon = 8/255$ ) against score-based purification with WideResNet-28-10.
Figure 6: Visualization of the clean images and adversarial samples generated by DiffAttack on CIFAR-10 with  $\ell_{\infty}$  attack ( $\epsilon = 8/255$ ) against score-based purification with WideResNet-70-16.
Figure 7: Visualization of the clean images and adversarial samples generated by DiffAttack on ImageNet with  $\ell_{\infty}$  attack ( $\epsilon = 4/255$ ) against score-based purification with WideResNet-50-2.
Figure 8: Visualization of the clean images and adversarial samples generated by DiffAttack on ImageNet with  $\ell_{\infty}$  attack ( $\epsilon = 4/255$ ) against score-based purification with DeiT-S.
Figure 9: Visualization of the clean images and adversarial samples generated by DiffAttack on ImageNet with a larger perturbation radius:  $\ell_{\infty}$  attack  $(\epsilon = 8 / 255)$  against score-based purification with ResNet-50.