SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbf {v} = \mathbf {z} _ {h} ^ {(u)} - \mathbf {z} _ {h} ^ {(0)} \approx \frac {\left\| \mathbf {z} _ {h} ^ {(u)} - \mathbf {z} _ {h} ^ {(0)} \right\| _ {2}}{L \left(g \left(\mathbf {z} _ {h} ^ {(u)}\right) , y\right) - L \left(g \left(\mathbf {z} _ {h} ^ {(0)}\right) , y\right)} \nabla_ {\mathbf {z} _ {h} ^ {(0)}} L \left(g \left(\mathbf {z} _ {h} ^ {(0)}\right), y\right), \tag {6}
$$
where the Taylor series is adopted to obtain the approximation. Therefore, optimizing Eq. (2) at the  $j$ -th step of an iterative optimizer can be considered as adopting the gradient w.r.t.  $\mathbf{z}_h^{(0)}$  as a replacement of the gradient w.r.t.  $\mathbf{z}_h^{(j)}$ . Without injecting noise or augmenting data at each iteration, ILA++ [26, 16] and NAA [52] can also be approximately regarded as replacing the gradient w.r.t.  $\mathbf{z}_h^{(j)}$  with the gradient w.r.t.  $\mathbf{z}_h^{(0)}$ .
In addition to these efforts, Guo et al. proposed LinBP [15] which back-propagates linearly in later layers. It can be regarded as replacing each  $D_{i}$  with an identity matrix for computing input gradients. Though effective, it mainly takes care of  $\{D_i\}$  and somehow ignores the difference of  $\nabla_{\mathbf{z}_g}L(\mathbf{z}_g,y)$  across models.
Experimental settings will be introduced in Section 5.1. We will compare our solution with a variety of state-of-the-arts to evaluate its effectiveness in Section 5.2. We combine our solution with some existing methods to further improve the adversarial transferability in Section 5.3, and we perform some ablation studies in Section 5.4.
Our experiments were conducted on CIFAR-10 [24] and ImageNet [35] under  $\ell_{\infty}$  constraint in the black-box setting. We focus on untargeted attacks and use I-FGSM attack as the back-end method, just like in previous work [46, 11, 28, 42]. We set the perturbation budget to  $\epsilon = 4 / 255$  and  $8 / 255$  for attacks on CIFAR-10 and ImageNet, respectively. Further, on CIFAR-10, we used a VGG-19 with batch normalization [36] as the substitute model, and a different VGG-19 with batch normalization,
Table 1: Transfer-based attack performance on ImageNet. The attacks are performed under the  $\ell_{\infty}$  constraint with  $\epsilon = 8 / 255$  in the untargeted setting.
Method	ResNet -50	ResNet -152	DenseNet -121	WRN -101	Inception v3	RepVGG -B1	VGG -19	Mixer -B	ConvNeXt -B	ViT -B	DeiT -B	Swin B	BEiT -B	Average
I-FGSM	38.20%	18.88%	49.24%	47.62%	22.14%	33.54%	44.98%	9.00%	9.68%	4.42%	3.96%	5.16%	3.84%	22.36%
MI-FGSM	47.88%	25.98%	59.96%	56.26%	30.42%	43.22%	54.28%	13.36%	13.18%	6.74%	5.80%	7.68%	6.62%	28.57%
DI²-FGSM	74.24%	47.20%	90.28%	84.76%	50.84%	77.44%	83.50%	18.62%	29.76%	12.82%	11.96%	13.92%	14.12%	46.88%
TI-FGSM	38.72%	19.72%	49.60%	49.00%	22.72%	33.90%	45.86%	9.36%	9.20%	4.70%	3.94%	5.12%	4.08%	22.76%
SI-FGSM	51.46%	28.62%	71.42%	65.82%	36.50%	47.60%	59.18%	12.42%	12.22%	6.34%	5.66%	6.28%	6.48%	31.54%
SGM	71.46%	48.72%	73.28%	74.26%	35.86%	65.60%	75.10%	19.08%	22.84%	10.66%	8.64%	12.26%	9.96%	40.59%
LinBP	73.00%	52.86%	81.56%	78.18%	39.40%	68.28%	81.54%	13.90%	15.80%	7.70%	5.82%	8.90%	7.98%	41.15%
Admix	63.52%	37.64%	71.94%	73.68%	31.84%	60.50%	70.92%	11.90%	15.14%	6.62%	5.10%	7.62%	6.24%	35.59%
TAIG-R	66.26%	44.28%	87.50%	84.46%	59.32%	72.36%	80.94%	18.86%	14.54%	10.96%	8.00%	9.56%	13.46%	43.88%
NAA	74.14%	58.84%	86.16%	84.34%	57.40%	76.80%	82.96%	20.96%	19.86%	9.92%	8.42%	11.56%	11.36%	46.36%
ILA++	75.28%	55.56%	84.78%	83.46%	50.98%	72.48%	78.48%	22.22%	30.28%	13.76%	12.18%	16.50%	14.94%	46.99%
ILPD	83.96%	69.86%	90.68%	90.28%	64.70%	85.90%	88.10%	29.82%	44.54%	20.42%	21.52%	26.88%	25.12%	57.06%
a ResNet-18 [18], a PyramidNet [17], GDAS [10], a WRN-28-10 [50], a ResNeXt-29 [47], and a DenseNet-BC [19]  $^2$  as the victim models, following some prior work [15, 26, 16]. All victim models (except for the VGG-19 which was trained by ourselves) were pre-trained by others and directly collected for experiments. On ImageNet, we used a ResNet-50 [18] model as the substitute model, and collect 13 victim models, including a different ResNet-50 [18], a VGG-19 [36], a ResNet-152 [18], a Inception v3 [37], a DenseNet-121 [19], a WRN-101 [50], a RepVGG-B1 [9], a MLP Mixer-B [38], a ConvNeXt-B [31], a ViT-B [13], a DeiT-B [39], a Swin-B [30], and a BEiT-B [3]  $^3$ .
The image size of adversarial examples was  $224 \times 224$  on ImageNet and  $32 \times 32$  on CIFAR-10, without crop, following prior work [28, 42]. We performed adversarial attacks on all test data in CIFAR-10 and 5000 randomly sampled examples from the ImageNet validation data. The adversarial examples obtained at each iteration of the attack should be clipped to [0, 1], in order to ensure that only valid images are fed to DNNs. We run 100 iterations with a step size of  $1/255$  for all attack methods on both CIFAR-10 and ImageNet in this section, and we set a smaller step size of  $\epsilon/100$  for Section 4 as we care more about the trend of overfitting before exhausting the perturbation budget. ILPD was performed at the output of the fourth VGG block for VGG-19 on CIFAR-10 and the output of the last building block of the second ResNet meta layer for ResNet-50 on ImageNet, with  $\gamma$  tuned in the range satisfying  $0.1 \leq 1/\gamma \leq 0.5$ . The advanced version of ILA++ [16] and NAA [52] are adopted as competitors. Since both methods inject some random transformations to the benign examples, when computing gradients, we also add Gaussian noise with a standard deviation of 0.05 to the benign examples at each iteration, for fair comparison. For competitors, we directly adopt their official implementations and hyper-parameters, and we attack on the same CIFAR-10 and ImageNet images for all methods. All experiments are performed on an NVIDIA V100 GPU.
5.2 Comparison with State-of-the-arts
The comparison results are summarized in Table 1 and 2. For ImageNet, 13 victim models, including four vision transformers, one multi-layer perceptron model, and a model that has the same architecture of the substitute model, are adopted. For CIFAR-10, 7 victim models are adopted. We compare our method with a gradient stabilization method (i.e., MI-FGSM [11]), some input augmentation methods (i.e.,  $\mathrm{DI}^2$ -FGSM [46], TI-FGSM [12], SI-FGSM [28], and the recent Admix [42]), compelling intermediate-level attacks (i.e., ILA++ [16] and NAA [52]), some back-propagation modification methods (i.e., SGM [45] and LinBP [15]), and a very recent method called TAIG [21].
Experiment results in the tables show that our method outperforms all these competitors by large margins. Specifically, it can be seen that with the assistance of our ILPD, it is possible to achieve an average success rate of  $57.06\%$  on ImageNet, which outperforms the second best (i.e.,  $\mathrm{ILA} + +$ ) by  $+10.07\%$ . On CIFAR-10, we achieve  $62.82\%$ , which outperforms the second best by  $+3.88\%$ .
Attacking some newly designed models is challenging when using ResNet as the substitute model. The I-FGSM baseline can only achieve a success rate of less than  $10\%$  against these models (see the right half of Table 1) and even  $< 5\%$  against some vision transformers. However, our method
$^{2}$ https://github.com/bearpaw/pytorch-classification
<sup>3</sup>https://github.com/rwrightman/pytorch-image-models
Table 2: Transfer-based attack performance on CIFAR-10. The attacks are performed under the  $\ell_{\infty}$  constraint with  $\epsilon = 4 / 255$  in the untargeted setting.
Method	VGG-19	ResNet-18	WRN	ResNeXt	DenseNet	PyramidNet	GDAS	Average
I-FGSM	71.18%	67.67%	51.28%	52.77%	48.74%	12.44%	36.23%	48.62%
MI-FGSM	73.56%	69.65%	53.55%	55.11%	50.36%	13.03%	38.07%	50.48%
DI²-FGSM	77.40%	72.53%	56.68%	57.58%	53.68%	15.78%	40.70%	53.48%
TI-FGSM	71.32%	67.67%	51.53%	52.76%	48.63%	12.43%	36.44%	48.68%
SI-FGSM	70.51%	72.70%	55.73%	57.51%	53.65%	14.19%	39.77%	52.01%
LinBP	78.19%	75.08%	59.35%	61.92%	57.89%	16.88%	43.56%	56.12%
Admix	78.17%	74.51%	58.18%	59.94%	55.69%	15.05%	42.42%	54.85%
TAIG-R	73.72%	71.25%	51.57%	52.72%	50.01%	12.93%	39.64%	50.26%
NAA	71.16%	71.00%	56.62%	58.67%	56.68%	18.31%	46.25%	54.10%
ILA++	79.77%	78.81%	63.36%	63.91%	60.38%	18.86%	47.51%	58.94%
ILPD	83.52%	81.63%	68.03%	67.97%	65.37%	19.44%	53.79%	62.82%
achieves impressive improvements in attacking these models. For instance, compared with the I-FGSM baseline, when attacking the MLP-Mixer and ConvNeXt models, our ILPD obtains  $+20.82\%$  and  $+34.86\%$  absolute gains, respectively, and when attacking the vision transformers, our ILPD obtains an absolute gain of  $+19.14\%$ .
Attack robust models. We also tested our method on the task of attacking robust models. We collected a robust Inception v3 and a robust EfficientNet-B0 from the timm [44] repository, and obtain a robust ResNet-50 and a robust DeiT-S [40] from Bai et al. [2]'s open-source repository as the victim models. Adversarial examples generated on the ResNet-50 substitute model were applied for this experiment. Our ILPD is compared with
Table 3: Performance in attacking robust ImageNet models. The attacks are performed under the  $\ell_{\infty}$  constraint with  $\epsilon = 8 / 255$  in the untargeted setting.
Method	Inception v3 (robust)	EfficientNet (robust)	ResNet-50 (robust)	DeiT-S (robust)	Average
I-FGSM	11.68%	10.88%	9.10%	10.56%	10.56%
DI²-FGSM	15.84%	34.04%	9.66%	11.02%	17.64%
NAA	22.86%	37.02%	10.80%	11.42%	20.53%
ILA++	18.22%	34.62%	10.46%	11.40%	18.68%
ILPD	25.30%	50.02%	11.42%	11.90%	24.66%
$\mathrm{DI}^2$ -FGSM, NAA, and  $\mathrm{ILA}++$ , which are the most competitive methods in Table 1, and we summarize the results in Table 3. It shows that our method outperforms competitors in attacking these robust models and getting  $+4.13\%$  improvement, compared with NAA, which is the second best method. In addition to the robust models, we also test our method on two advanced defense methods, i.e., random smoothing [7] and NRP [33]. Specifically, our ILPD achieves a success rate of  $12.90\%$  in attacking a smoothed ResNet-50 on ImageNet, while the second best method (NAA) and the baseline show  $11.84\%$  and  $7.20\%$ , respectively. For NRP, the adversarial examples generated by our method can achieve an average success rate of  $20.03\%$  after purification. Meanwhile, the second best method  $(\mathrm{DI}^2$ -FGSM) achieves  $14.54\%$ , and the baseline achieves  $9.39\%$ .
5.3 Combination with Existing Methods
Our method can be readily combined with other attacks. In this section, we evaluate the combination performance. We tried combining our method with different sorts of attacks, including a gradient stabilization method, i.e., MI-FGSM [11] and two input augmentation methods, i.e.,  $\mathrm{DI}^2$ -FGSM [46] and Admix [52]. The results are reported in Table 4. It shows that the adversarial transferability can indeed be further improved by combining our method with these prior arts. Specifically, when combining with  $\mathrm{DI}^2$ -FGSM, the best average success rate, i.e.,  $63.23\%$ , is obtained.
Table 4: Combination of our ILPD with MI-FGSM,  $\mathrm{DI}^2$ -FGSM, and Admix on ImageNet. The attacks use I-FGSM as the back-end attack and are performed under the  $\ell_{\infty}$  constraint with  $\epsilon = 8/255$  in the untargeted setting.
Method	ResNet -50	ResNet -152	DenseNet -121	WRN -101	Inception v3	RepVGG -B1	VGG -19	Mixer -B	ConvNeXt -B	ViT -B	DeiT -B	Swin B	BEiT -B	Average
MI-FGSM	47.88%	25.98%	59.96%	56.26%	30.42%	43.22%	54.28%	13.36%	13.18%	6.74%	5.80%	7.68%	6.62%	28.57%
+ILPD	84.90%	72.22%	91.74%	91.42%	69.44%	87.66%	89.12%	34.62%	48.00%	24.20%	25.36%	29.16%	30.02%	59.84%
DI²-FGSM	74.24%	47.20%	90.28%	84.76%	50.84%	77.44%	83.50%	18.62%	29.76%	12.82%	11.96%	13.92%	14.12%	46.88%
+ILPD	85.26%	74.62%	92.40%	91.36%	74.42%	88.74%	90.10%	37.98%	53.42%	29.50%	31.22%	35.32%	37.70%	63.23%
Admix	63.52%	37.64%	71.94%	73.68%	31.84%	60.50%	70.92%	11.90%	15.14%	6.62%	5.10%	7.62%	6.24%	35.59%
+ILPD	86.08%	75.48%	92.48%	91.66%	70.44%	88.84%	90.52%	33.42%	47.94%	23.24%	24.12%	29.66%	29.78%	60.28%
$^{4}$ https://github.com/ytongbai/ViTs-vs-CNNs
(a) ResNet-50
(b) VGG-19
Figure 5: How the average success rate change with different choices of position on (a) ImageNet trained ResNet-50 and (b) CIFAR-10 trained VGG-19. For the ResNet-50 model, "0" represent the stem layer and "1-1" represent the first building block of the first ResNet meta layer.
The effect of position. As an intermediate-level attack, our method needs to choose a position to split  $f$  in  $h$  and  $g$ . Such a position selection affects the performance of all intermediate-level attacks. We vary the position and compare our method with ILA++ [16], NAA [52], and LinBP [15] in Figure 5. The performance on ImageNet and CIFAR-10 are both tested, using ResNet-50 and VGG-19 as the substitute models, respectively. Apparently, our ILPD demonstrates consistently higher adversarial transferability on all choices of the position. In particular, our method significantly outperforms ILA++, NAA, and LinBP when choosing a lower layer to split  $f$  into  $h$  and  $g$ . When using ResNet-50 as the substitute model, choosing any of the first eleven layers (i.e., from "0" to "3-3") can achieve an average success rate of more than  $53\%$  on ImageNet, even simply choosing the first layer will lead to an average success rate of  $53.12\%$ , which is already superior to all competitors in Table 1.
The effect of  $\gamma$ . We evaluate ILPD with varying  $\gamma$  on ImageNet and CIFAR-10 in Figure 6. The victim models are the same as the ones in Table 1 and Table 2. It can be seen that using  $1 / \gamma < 1$  always leads to improved performance compared with the baseline (which is equivalent to adopting  $1 / \gamma = 1$ ). On ImageNet, the optimal performance is obtained with  $1 / \gamma = 0.1$  (i.e.,  $\gamma = 10$ ) and ResNet-50, while, for CIFAR-10,  $1 / \gamma = 0.3$  (i.e.,  $\gamma = 10 / 3$ ) is more effective. That is, on ImageNet, we aim to maximize the prediction loss of a perturbation which is cut by
(a) ImageNet
Figure 6: How the average success rate changes with  $\gamma$  using (a) ImageNet trained ResNet-50 and (b) CIFAR-10 trained VGG-19 as the substitute models.
$10 \times$  in the middle layer. It is possible that employing  $\gamma > 10$  leads to even better performance on ImageNet, yet we do not aim to carefully tune  $\gamma$  and would like to leave it to future work.
In this paper, we have revisited some intermediate-level attacks and shown that, limited by their two-stage training scheme, the obtained intermediate-level perturbations will deviate from the directional guides and such a deviation largely impairs the attack performance of the crafted adversarial examples. In order to overcome this limitation, we have developed a method that performs attacks through a single stage of optimization and guarantees the obtained intermediate-level perturbation possesses a larger magnitude when compared with a directional guide and exactly follows its direction that can effectively increase the prediction loss of the model. The developed method is formed as adopting intermediate-level perturbation decay when computing gradients. Extensive experimental results on ImageNet and CIFAR-10 have shown that our method outperforms state-of-the-arts by large margins in attacking various victim models, including convolutional models, multi-layer perceptron model, vision transformers, and robust models. Moreover, we have shown that our method can be easily combined with prior methods to craft more transferable adversarial examples.
This material is based upon work supported by the National Science Foundation under Grant No. 1801751 and 1956364.
[42] Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16158-16167, 2021.
[43] Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, and Kui Ren. Feature importance-aware transferable adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7639-7648, 2021.
[44] Ross Wightman. Pytorch image models. https://github.com/rwrightman/pytorch-image-models, 2019.
[45] Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Rethinking the security of skip connections in resnet-like neural networks. In ICLR, 2020.
[46] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In CVPR, 2019.
[47] Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In CVPR, 2017.
[48] Ziang Yan, Yiwen Guo, Jian Liang, and Changshui Zhang. Policy-driven attack: learning to query for hard-label black-box adversarial examples. In ICLR, 2021.
[49] Ziang Yan, Yiwen Guo, and Changshui Zhang. Subspace attack: Exploiting promising subspaces for query-efficient black-box attacks. In NeurIPS, 2019.
[50] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
[51] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
[52] Jianping Zhang, Weibin Wu, Jen-tse Huang, Yizhan Huang, Wenxuan Wang, Yuxin Su, and Michael R Lyu. Improving adversarial transferability via neuron attribution-based attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14993-15002, 2022.
A Experiment Results of Using Other Substitute Models
We compare our method with other intermediate-level attacks using various substitute model on ImageNet. The comparison results are summarized in Table 5. Three vision transformers and one multi-layer perceptron model are considered as the substitute models, including a ViT-B [13], a DeiT-B [39], a Swin-B [30], and a MLP Mixer-B [38]. The victim models are the same as those in Table 1. For ILPD,  $h$  is always split to be the first two blocks of these models and we fix  $1 / \gamma = 0.1$ . For NAA and ILA++, we tested the results of all possible intermediate layer selections and report the best results among them. The average success rates are compared as follows, and the victim models are the same as those in Table 1 of our paper. Experiment results in the table show that our method outperforms all these competitors by large margins.
Table 5: Transfer-based attack performance of using different substitute models on ImageNet. The attacks are performed under the  $\ell_{\infty}$  constraint with  $\epsilon = 8 / 255$  in the untargeted setting. "Average" is obtained on victim models different from the substitute model.
Substitute	Method	ResNet-50	ResNet-152	DenseNet-121	WRN-101	Inception v3	RepVGG-B1	VGG-19	Mixer-B	ConvNeXt-B	ViT-B	DeiT-B	Swin B	BEiT-B	Average
ViT-B	ILA++	22.12%	17.44%	28.20%	21.28%	28.52%	29.30%	32.22%	47.06%	19.00%	97.22%	40.12%	29.50%	43.34%	29.84%
NAA	31.90%	27.64%	45.74%	35.36%	46.56%	44.98%	46.36%	58.32%	26.76%	94.58%	60.62%	40.52%	57.52%	43.52%
ILPD	34.96%	32.64%	43.78%	37.38%	44.02%	44.60%	44.36%	58.82%	36.38%	96.42%	61.92%	54.16%	66.54%	46.63%
DeiT-B	ILA++	20.92%	14.40%	24.76%	18.60%	25.24%	29.46%	33.00%	33.94%	18.24%	27.22%	87.54%	18.38%	19.72%	23.66%
NAA	29.74%	25.04%	36.52%	30.26%	38.26%	39.18%	39.42%	45.78%	31.38%	39.42%	75.32%	31.54%	37.62%	35.35%
ILPD	40.14%	36.20%	42.02%	36.80%	40.82%	44.80%	43.76%	53.34%	45.68%	51.82%	81.74%	46.36%	49.64%	44.28%
Mixer-B	ILA++	22.10%	14.46%	24.52%	19.08%	27.96%	25.48%	27.64%	96.64%	15.08%	22.76%	23.76%	12.68%	14.24%	20.81%
NAA	33.28%	23.24%	42.92%	28.86%	41.56%	41.48%	43.88%	92.62%	26.54%	42.76%	52.88%	24.64%	29.06%	35.93%
ILPD	39.84%	30.84%	42.56%	34.40%	41.88%	44.44%	42.56%	91.56%	40.80%	49.88%	58.26%	36.38%	38.74%	41.72%
Swin-B	ILA++	12.68%	8.36%	13.44%	9.22%	14.30%	18.92%	22.78%	12.64%	19.84%	9.14%	7.64%	83.96%	9.80%	13.23%
NAA	16.36%	13.10%	22.72%	16.54%	22.90%	26.78%	28.84%	22.04%	28.80%	17.70%	19.94%	71.90%	21.16%	21.41%
ILPD	31.04%	27.44%	32.66%	25.88%	28.06%	43.08%	42.52%	32.06%	55.48%	40.14%	35.42%	89.56%	47.42%	36.77%