SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 5: Untargeted attack success rates (%) of our PGN method, when it is integrated with I-FGSM (BIM), MI-FGSM, NI-FGSM, VMI-FGSM, EMI-FGSM, and RAP, respectively. The adversarial examples are generated on Inc-v3. * indicates the white-box model.
Attack	Inc-v3	Inc-v4	IncRes-v2	Res-101	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens	Avg.
BIM	100.0*	28.1	20.6	26.7	11.9	11.9	5.0	29.17
PGN-BIM	100.0*	33.1	25.3	30.4	13.7	13.8	6.1	31.77
MI	100.0*	50.8	46.3	48.9	23.3	22.2	11.7	43.31
PGN-MI	100.0*	56.4	53.3	55.0	24.7	24.7	11.8	46.54
NI	100.0*	62.3	59.8	57.8	22.4	22.3	11.8	48.06
PGN-NI	100.0*	68.6	65.8	61.9	25.7	25.8	12.8	51.51
VMI	100.0*	75.2	70.2	66.0	41.7	40.9	24.8	59.83
PGN-VMI	100.0*	79.8	75.9	69.8	45.1	46.1	27.8	63.50
EMI	100.0*	81.6	77.0	72.0	32.8	32.2	17.5	59.01
PGN-EMI	100.0*	83.6	81.7	75.9	35.2	34.4	17.5	61.19
RAP	99.9	84.6	79.3	76.6	57.2	51.3	32.7	68.80
PGN-RAP	100.0*	90.3	86.4	85.7	59.5	56.3	39.3	73.91
C.2 Skip Gradient Method
Moreover, our method can also be combined with the Skip Gradient Method (SGM) [58]. By using MI-FGSM as the backbone method, we first compare our PGN method with SGM on the
Figure 4: Visualization of adversarial examples with their corresponding loss surfaces along two random directions. Here, we randomly sampled five images and generated the adversarial examples on Inc-v3. These selected images include the three images (i.e., the last three ones) that can be successfully transferred using our PGN method but cannot be transferred using other baseline methods. The loss surfaces are also calculated on Inc-v3.
ResNet-18 model. Since SGM modifies the backpropagation and is compatible with our PGN, we also integrate PGN into SGM to evaluate its generality to other attacks. As shown in Table 6, our PGN consistently exhibits better attack performance than SGM, which further shows its superiority in boosting adversarial transferability. Besides, PGN-SGM outperforms both PGN and SGM, showing its remarkable compatibility with various attacks.
Table 6: Untargeted attack success rates (%) of our PGN method with SGM method in the single model setting. The adversarial examples are crafted on Res-18.
Source:Res-18	Res-101	Res-152	Inc-v3	Inc-v4	IncRes-v2	Inc-v3ens3	Inc-v3ens4	Avg.
MI	82.8	73.3	54.5	48.7	33.9	17.4	18.1	46.96
SGM	89.1	82.5	66.0	58.8	45.4	20.8	20.8	54.77
PGN	95.9	92.9	77.9	75.9	58.7	33.7	36.3	67.33
PGN-SGM	96.7	93.8	80.1	76.3	62.8	33.8	37.0	68.64
D Ablation Studies for More Hyper-parameters
D.1 The Number of Sampled Examples,  $N$
Figure 5: Untargeted attack success rates  $(\%)$  on six black-box models with the different number of sampled samples  $N$ . The adversarial examples are generated by PGN on Inc-v3.
In this study, we employ random sampling of multiple examples and calculate the average gradients of these examples to mitigate the variance resulting from random sampling during the iterative process. To investigate the influence of the number of sampled examples, denoted as  $N$ , we conduct ablation experiments to analyze this parameter. As illustrated in Figure 5, when  $N = 1$ , our method demonstrates the lowest level of transferability. However, as we increase the value of  $N$ , the transferability exhibits rapid improvement until  $N = 12$ , after which it gradually converges for normally trained models. Notably, when  $N > 12$ , a slight performance improvement can still be achieved by increasing the number of sampled examples in our PGN method. To strike a balance between transferability and computational overhead, we set  $N = 20$  in our work. This observation further substantiates that sampling random examples from the vicinity of the adversarial example effectively facilitates neighborhood exploration. Consequently, it stabilizes the gradient update process and encourages the discovery of flatter regions by the adversarial example.
D.2 Finite Difference Step-size  $\alpha$
In this work, we simply set the finite difference step-size to  $\alpha = 1.6 / 255\approx 6.27\mathrm{e - }3$  to avoid redundant hyperparameters. In order to study the effect of the smaller step-size  $\alpha$  on the finite
difference method, we performed more experiments. We investigate the transferability of adversarial examples when the alpha is set to 1e-4 and 1e-5, and the experimental results are reported in Table 7. The results show that smaller step-sizes cannot improve adversarial transferability. This is because the smaller the step-size, the closer the two neighboring gradients are, leading our method to degrade to MI-FGSM.
With the current choice of  $\alpha$ , our method also approximates the actual Hessian/vector product. We compared the cosine similarity of perturbations generated by the Hessian/vector product and the finite difference method. The average cosine similarity of 1,000 images is about 0.9035. Note that existing adversarial attacks typically rely on the sign of the gradient, rather than requiring an exact gradient value. Thus, we can approximate the second-order Hessian matrix by using the finite difference method to accelerate the attack process.
Table 7: Untargeted attack success rate (%) when using smaller step sizes in the finite-difference method. * indicates the white-box model.
Size	Inc-v3	Inc-v4	IncRes-v2	Res-101
6.27e-3	100.0*	91.6	90.6	89.1
1e-4	100.0*	86.4	84.6	82.6
1e-5	100.0*	85.3	85.0	82.4
E Differences between Our method and Related Works
There are two previous works similar to our proposed attack method (i.e., PGN), where one is Reverse Adversarial Perturbation (RAP) [42], which is an adversarial attack method. And the other is Gradient Norm Penalty (GNP) [66], which was proposed to improve generalization. To demonstrate the novelty and significance of the proposed method, we give a detailed discussion about the differences and similarities compared to these two works.
E.1 Differences with RAP
Reverse adversarial perturbation (RAP) is a closely related work that encourages adversarial examples to be located at a region with low loss values. However, there are still many differences between our method and RAP.
Empirical verification. Both RAP and our PGN aim to achieve flat local optima for more transferable adversarial examples. Moreover, this is the first work to provide empirical verification that adversarial examples in flat local optima are more transferable.
Methodology. RAP injects worst-case perturbations into the optimization process to maximize the loss and achieve flat local optima. In contrast, PGN introduces a gradient norm into the loss to achieve a flat local optimum. Hence, the proposed method differs significantly from RAP.
Efficiency. Our method is more computationally efficient than RAP. In RAP, the outer loop is 400, and the inner loop is 8. In contrast, our PGN method employs finite differences to approximate the second-order Hessian matrix and samples multiple points to obtain a more stable gradient, which requires only 10 outer loops and 20 inner loops for the computation. Here we also compared the time consumed for each batch of images between our method and RAP. The experiment was conducted on RTX 2080 Ti with a CUDA environment. We chose Inc-v3 as the source model and set the batch size to 10. Experimental results show that RAP will consume  $207.19s$  in each batch size, while our method only consumes  $29.85s$ , which validates that our PGN is more computationally efficient.
E.2 Compared with Gradient Norm Penalty
Zhao et al. [66] proposed a gradient norm penalty method in deep learning to improve generalization, which is similar to our method. However, there are also some differences between these two works, which are summarized as follows:
Different goals. Zhao et al. primarily focus on computing flat local minima to improve generalization during model training. In contrast, our main objective is to investigate the potential impact of flat
local minima on the transferability of adversarial attacks. Previous research in the field of adversarial attacks has shown limited attention to the relationship between flat local optima and adversarial transferability. Consequently, our work significantly contributes a new insight into the domain of adversarial attacks.
Objective function. A distinguishing feature of our proposed method lies in its emphasis on the gradient information surrounding the adversarial example. Specifically, we approximate the second-order Hessian matrix using the finite difference method, which plays a crucial role in our approach. The work presented by Zhao et al. also supports and provides theoretical foundations for the feasibility of our method within the realm of adversarial attacks.
In summary, our main motivation in this work is to explore whether flat local optimum can improve adversarial transferability. To the best of our knowledge, it is also the first work that penalized the gradient norm and the finite difference method support our motivation in the field of adversarial attacks.
F Additional Experimental Results
F.1 Evaluated on Transformer-based Models
To further validate the effectiveness of our PGN method, we evaluated the performance on the Transformer-based models, i.e. ViT [12], PiT [19], Visformer [7], and Swin [33]. The adversarial examples are generated on Inc-v3 and the attack success rates are summarized in Table 8. It can be seen that our PGN can consistently outperform the baselines on these Transformers, showing its high effectiveness and generality to various architectures.
Table 8: Untargeted attack success rate (%) of adversarial examples in Transformer-based models. The adversarial examples are generated on Inc-v3.
Attack	ViT	PiT	Visformer	Swin	Avg.
MI	18.9	18.1	23.7	22.2	20.73
NI	20.0	19.9	25.5	25.5	22.73
VMI	28.4	34.0	41.1	40.8	36.08
EMI	26.0	27.3	36.9	36.7	31.73
RAP	35.2	41.8	50.6	50.2	44.45
PGN	44.7	53.2	65.5	64.5	56.98
F.2 Attack Defense Models
In this subsection, besides normally trained models and adversarially trained models, we further validate the effectiveness of our methods on other defenses, including Bit-Red [62], ComDefend [22], JPEG [17], HGD [29], R&P [60], NIPs-r3 [39], FD [34], NPR [39], and RS [9]. The adversarial examples are generated on an ensemble of Inc-v3, Inc-v4, and IncRes-v2, and the weight for each model is  $1/3$ .
The experimental results are reported in Table 9. In the context of ensemble models, it is evident that our algorithm can considerably enhance existing attack methods. For instance, VMI, EMI, and RAP achieve average success rates of  $54.91\%$ ,  $61.59\%$ , and  $69.94\%$ , respectively, against the six defense models. In contrast, our proposed PGN method achieves an average success rate of  $77.08\%$ , surpassing them by  $22.17\%$ ,  $15.49\%$ , and  $7.14\%$ , respectively. This notable improvement demonstrates the remarkable effectiveness of our proposed method against adversarially trained models as well as other defense models. Consequently, it poses a more substantial threat to advanced defense models. These findings further validate that the discovery of adversarial examples within flat regions can significantly enhance the transferability of adversarial attacks.
F.3 Attack Success Rates on CIFAR-10
To further illustrate the effectiveness of our PGN method on different datasets, we conduct experiments on CIFAR-10 [25]. we set the hyperparameters as follows: maximum perturbation  $\epsilon = 8 / 255$  , number
Table 9: Untargeted attack success rates (\%) on six defense models. The adversarial examples are crafted on the ensemble models, i.e. Inc-v3, Inc-v4 and IncRes-v2.
Attack	ComDefend	JPEG	NIPs-r3	FD	R&P	HGD	Bit-Red	NPR	RS	AVG.
MI	54.9	49.5	29.9	51.7	22.2	24.8	23.8	36.5	30.3	39.96
NI	56.9	50.8	29.3	53.6	23.1	22.3	23.9	38.1	80.7	36.52
VMI	72.4	71.5	58.1	67.8	50.6	54.3	39.0	44.9	35.6	54.91
EMI	78.2	74.1	69.1	74.8	60.1	64.8	47.6	46.8	38.8	61.59
RAP	89.5	88.1	81.0	79.6	73.1	72.3	54.6	49.6	41.7	69.94
PGN	93.7	91.3	88.3	85.7	83.6	82.5	72.1	51.3	45.2	77.08
of iterations  $T = 10$ , and step size  $\alpha = 1 / 255$ . We compare our PGN method with various gradient-based attacks, including MI-FGSM, NI-FGSM, VMI-FGSM, EMI-FGSM, and RAP. The adversarial examples are generated on the VGG-16, ResNet-50, and DenseNet-121 models, respectively. The results in Table 10 clearly show that our PGN method can enhance the attack transferability on the CIFAR-10 dataset. This verifies our motivation that adversarial examples located in flat local regions tend to exhibit better transferability across diverse models. Moreover, our attack method shows superior performance when applied to other datasets, reinforcing its versatility and effectiveness.
Table 10: Untargeted attack success rates (%) on the CIFAR-10 dataset for the attack methods in the single model setting. The adversarial examples are crafted on VGG-16, ResNet-50 (Res-50), and DenseNet-121, respectively.
Attack	MobileNet	VGG-19	GoogLeNet	Inc-v3	DenseNet-121	DenseNet-169	Res-34	Res-50
MI	52.18	57.56	47.29	52.74	40.96	42.40	41.72	41.93
NI	56.13	61.36	49.19	54.87	37.61	39.74	38.74	38.90
VMI	66.14	68.05	60.89	65.63	55.62	57.21	55.35	56.46
EMI	70.69	74.36	66.78	70.56	59.98	63.04	60.47	61.83
RAP	77.98	78.43	73.41	77.86	68.30	69.74	65.48	66.27
PGN	85.97	86.73	82.82	85.59	72.48	74.66	71.62	72.95
(a) Untargeted attack success rates  $(\%)$  for the adversarial examples crafted on VGG-16.
Attack	MobileNet	VGG16	VGG19	GoogLeNet	Inc-v3	DenseNet-121	DenseNet-169	Res-34
MI	70.42	67.37	65.8	63.06	69.02	72.39	73.34	67.78
NI	71.97	65.57	63.76	63.28	69.13	71.03	72.78	65.02
VMI	77.60	74.27	73.26	71.11	75.99	76.80	77.68	73.54
EMI	80.11	78.66	77.43	76.34	78.12	79.68	80.12	77.24
RAP	86.92	84.24	83.46	80.68	81.75	83.54	84.98	81.36
PGN	90.88	88.68	88.07	85.79	89.53	89.93	90.91	87.19
(b) Untargeted attack success rates  $(\%)$  for the adversarial examples crafted on Res-50.
Attack	MobileNet	VGG16	VGG19	GoogLeNet	Inc-v3	DenseNet-169	Res-34	Res-50
MI	63.47	60.64	60.08	57.39	63.35	71.09	61.99	67.37
NI	66.86	61.80	60.85	60.30	66.54	75.92	61.57	69.25
VMI	71.28	68.49	68.01	65.40	70.62	75.51	68.42	72.50
EMI	74.36	75.66	73.54	70.41	75.23	78.94	73.64	77.45
RAP	79.98	80.22	78.68	76.39	80.55	84.25	78.65	80.03
PGN	86.73	85.12	84.66	81.82	86.26	88.35	83.30	86.21
(c) Untargeted attack success rates  $(\%)$  for the adversarial examples crafted on DenseNet-121.