SlowGuess/NeurIPS_2023_OCR huggingface dataset:
**Evaluations on the single baseline.** As shown in Table A4, model-related methods consistently achieve better attack performance than the attacks on the original models, showing the effectiveness of these methods. Compared with LinBP and Ghost, our proposed BPA exhibits superior performance across all five attacks. On average, BPA outperforms the runner-up method with a remarkable margin of  $14.21\%$ ,  $16.44\%$ ,  $14.15\%$ ,  $11.80\%$ ,  $13.64\%$  for PGD, MI-FGSM, VMI-FGSM, ILA, and SSA, respectively. These results are consistent with the findings reported in Sec. 4.2 for ResNet-50. The superior performance of BPA not only validates its effectiveness but also highlights its generality to different architectures.
Evaluations by combining BPA with the baselines. Similar in Sec. 4.2, we also integrate BPA into LinBP and Ghost to further boost the performance. The results in Table A5 indicate that BPA can significantly improve the attack performance of PGD and MI-FGSM. For instance, considering MI-FGSM attack, integrating BPA results in a clear performance improvement of  $11.42\%$  and  $16.46\%$  for LinBP and Ghost, respectively. These findings are consistent with the results obtained on ResNet-50, as discussed in Sec. 4.2. These results further highlight the effectiveness and superiority of BPA in boosting the adversarial transferability of existing attacks, which are not limited to the surrogate models.
**Evaluations on defense methods.** Finally, we evaluate these model-related approaches on defense methods and report the results in Table A6. Notably, our BPA method consistently enhances the performance of PGD and MI-FGSM attacks, yielding superior results against the defense methods compared to other model-related methods. On average, BPA outperforms the runner-up method with a
Attacker	Method	Inc-v3	IncRes-v2	DenseNet	MobileNet	PNASNet	SENet	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
DIM	N/A	45.00	38.56	71.64	70.08	41.60	48.56	28.52	24.82	16.48
SGM	52.72	45.22	79.42	82.34	49.50	58.66	32.42	28.20	19.26
LinBP	45.74	38.38	76.34	77.58	39.28	50.20	27.40	22.60	15.50
Ghost	45.20	37.86	72.70	72.34	40.32	48.50	27.44	23.98	15.78
BPA	59.20	50.86	87.70	86.92	55.40	62.68	40.32	34.84	24.42
TIM	N/A	32.58	26.66	58.44	55.44	29.26	34.84	21.38	18.76	13.54
SGM	41.18	35.22	71.12	72.20	41.24	47.88	25.50	23.66	16.28
LinBP	43.20	36.30	75.08	74.08	39.14	46.90	27.62	23.62	16.74
Ghost	37.08	29.36	66.48	63.26	33.26	39.74	23.58	21.14	14.54
BPA	59.20	50.86	87.70	86.92	55.40	62.68	40.32	34.84	24.42
SIN	N/A	42.68	33.76	70.12	65.70	34.90	39.90	26.36	23.48	15.22
SGM	52.78	43.04	79.92	80.22	45.10	52.70	31.94	27.44	18.74
LinBP	50.46	41.06	78.22	77.14	38.82	47.64	30.38	25.08	17.14
Ghost	47.46	37.92	77.14	72.82	38.66	46.02	29.14	24.58	16.28
BPA	52.40	43.44	80.12	76.38	45.42	50.28	39.16	36.32	26.06
SGM+BPA	62.26	54.12	88.04	88.28	56.80	63.46	46.08	39.80	30.68
DA	N/A	45.00	38.56	71.64	70.08	41.60	48.56	28.52	24.82	16.48
SGM	52.72	45.22	79.42	82.34	49.50	58.66	28.20	19.26
LinBP	45.74	38.38	76.34	77.58	39.28	50.20	27.40	22.60	15.50
Ghost	45.20	37.86	72.70	72.24	40.32	48.50	27.44	23.98	15.78
BPA	59.20	50.86	87.70	86.92	55.40	62.68	40.32	34.84	24.42
margin of  $5.32\%$  and  $7.78\%$  for PGD and MI-FGSM, respectively. These findings further underscore the high effectiveness of BPA in improving the performance of various attacks and highlight its versatility in enhancing adversarial attacks across different architectural models.
In conclusion, the results obtained for untargeted attacks on VGG-19 align with the findings presented for ResNet-50 in Sec. 4.2. The significant and consistent improvement in performance across various architectures validates our motivation that addressing the gradient truncation issue caused by non-linear layers can enhance adversarial transferability. These findings also strongly support the high effectiveness and utility of our BPA to boost adversarial transferability.
Table A7: Untargeted attack success rates  $(\%)$  of various input-transformation-based attacks on nine models when generating the adversarial examples on ResNet-50 w/wo various model-related methods.
Attacker	Method	HGD	R&P	NIPS-r3	JPEG	RS	NRP
PGD	N/A	5.44	3.16	3.54	8.36	8.45	11.26
LinBP	5.28	3.26	3.88	9.14	9.00	11.76
Ghost	5.68	3.16	3.70	9.10	8.50	10.98
BPA	15.78	7.58	9.46	16.22	12.00	13.18
MI-FGSM	N/A	9.12	5.08	5.76	12.18	8.00	12.86
LinBP	8.06	4.75	5.34	11.56	8.50	12.32
Ghost	9.14	4.92	5.78	12.32	8.50	12.08
BPA	24.36	11.50	14.30	22.38	14.00	13.12
Table A6: Untargeted attack success rates  $(\%)$  of several attacks on six defenses when generating the adversarial examples on VGG-19 w/wo various model-related methods.
A.3 Additional Evaluation on Various Input Transformation based Attacks
In Table 1, we compare our BPA with various model-related attacks when combined with other attacks, including gradient-based (PGD), momentum-based (MI-FGSM, VMI-FGSM), objective-related (ILA) and input transformation based (SSA) attack. Due to the page limit, we only evaluate BPA with one up-to-date transformation based attack (SSA). Here we further evaluate its generality to other input transformation based attacks, namely DIM [57], TIM [7], SIN [26] and DA [18] in Table A7. As we can see, our BPA can significantly boost these input transformation based attacks. Compared with existing model-related attacks, BPA consistently exhibits better transferability for TIM, DIM, and DA. For SIN, BPA exhibits comparable attack performance with SGM on standardly trained models but much better transferability on adversially trained models. To integrate the advantage of SGM and our BPA, we combine BPA with SGM for SIN, which outperforms the baselines with a clear margin. These results further validate its superiority in boosting adversarial transferability.
A.4 Additional Evaluation on Another Widely Adopted Perturbation Budget
Both 8/255 and 16/255 are widely adopted perturbation budgets in adversarial learning. In Sec. 4.2, we mainly adopt  $\epsilon = 8 / 255$  for evaluation. To further validate the effectiveness of our BPA, we also
evaluate BPA using PGD with  $\epsilon = 16 / 255$ . As shown in Table A8, BPA consistently outperforms the baselines with a clear margin, showing its remarkable effectiveness in boosting adversarial transferability with various perturbation budgets.
Attacker	Method	Inc-v3	IncRes-v2	DenseNet	MobileNet	PNASNet	SENet	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
PGD	N/A	36.42	29.90	65.08	62.46	30.40	38.60	21.26	18.70	12.60
SGM	49.62	41.60	77.78	80.14	46.04	56.24	29.28	24.18	17.20
LinBP	60.48	53.12	87.24	86.60	53.84	67.74	36.60	28.86	20.24
Ghost	39.90	31.42	67.98	67.84	32.26	40.74	22.62	19.68	13.76
BPA	72.06	66.56	93.42	92.28	70.26	78.74	49.76	40.80	31.08
Table A8: Untargeted attack success rates (%) of PGD on nine models when generating adversarial examples on ResNet-50 w/wo model-related methods using  $\epsilon = 16 / 255$
A.5 Additional Evaluation on CIFAR-10 Dataset
Existing transfer-based attacks mainly validate their effectiveness on ImageNet dataset. For a fair comparison, we also evaluate our BPA on ImageNet dataset. Here we also conduct experiments on CIFAR-10 using PGD with  $\epsilon = \frac{8}{255}$  on VGG-19. As shown in Table A9, BPA exhibits better transferability than the baselines, showing its high effectiveness and generality to various datasets and models.
Attacker	Method	WRN	ResNeXt	DenseNet	pyramidnet	gdas
PGD	N/A	65.94	65.66	63.56	16.96	50.00
LinBP	67.88	67.46	65.02	18.18	51.78
Ghost	66.46	65.52	62.92	17.32	49.18
BPA	74.38	73.80	69.66	20.74	57.16
Table A9: Untargeted attack success rates  $(\%)$  of several attacks on five models for CIFAR-10 dataset when generating the adversarial examples on VGG-19 w/wo various model-related methods.
A.6 Additional Evaluation on Targeted Attacks
Targeted attacks are more challenging than untargeted attacks. To further validate the effectiveness and generality of BPA, we also perform the targeted attack on VGG-19, following the experimental settings in Sec. 4.3. The results are summarized in Table A10. It is interesting that LinBP decays the targeted attack performance on VGG-19. Since there is no skip connection in VGG-19, LinBP only modifies the derivative of ReLU, which might introduce an imprecise gradient. This highlights the significance that BPA excludes extreme values from consideration when calculating the gradient for better transferability. It is evident that our BPA achieves the best attack performance among various methods. Overall, BPA outperforms LinBP and Ghost by  $8.18\%$  and  $7.90\%$  for PGD, and  $2.43\%$  and  $2.46\%$  for MI-FGSM. These results further validate the effectiveness of BPA in targeted attacks, demonstrating its superiority over the baselines. The improved performance of BPA showcases its potential and generality in enhancing targeted attacks on various models.
Attacker	Method	Inc-v3	IncRes-v2	DenseNet	MobileNet	PNASNet	SENet	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
PGD	N/A	1.26	1.26	3.80	2.72	5.10	4.32	0.10	0.04	0.02
LinBP	1.26	1.22	3.44	2.24	4.26	3.52	0.26	0.12	0.02
Ghost	1.34	1.26	3.88	2.52	5.26	4.40	0.12	0.06	0.02
BPA	6.70	7.30	19.44	12.56	23.34	17.32	1.80	0.76	0.74
MI-FGSM	N/A	0.18	0.10	1.00	0.92	1.02	1.14	0.00	0.00	0.02
LinBP	0.24	0.20	1.18	0.86	0.94	1.02	0.02	0.00	0.02
Ghost	0.22	0.14	0.94	0.74	1.04	1.12	0.00	0.02	0.02
BPA	1.24	1.24	5.60	4.22	7.06	6.80	0.12	0.02	0.04
Table A10: Targeted attack success rates (\%) of various attackers on nine models when generating adversarial examples on VGG-19 w/wo model-related methods using PGD and MI-FGSM.
A.7 Additional Evaluation on Modifications of ReLU Layers with BPA and LinBP
In this work, we find that the truncation of non-linear layers (e.g., ReLU, max-pooling) decays the relevance between the gradient w.r.t. the input and loss. Making the model more linear can enhance such relevance, thus improving the transferability. However, making the model more linear is not optimal. Taking ReLU for example, we compare the transferability of PGD on LinBP and BPA by
solely changing the derivatives of ReLU. Here LinBP makes the model more linear than our BPA. As shown in Table A11, BPA exhibits better transferability than LinBP, which supports our argument.
Attacker	Method	Inc-v3	IncRes-v2	DenseNet	MobileNet	PNASNet	SENet	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
PGD	N/A	16.34	13.38	36.86	36.12	13.46	17.14	10.24	9.46	5.52
LinBP	27.22	23.04	59.34	59.74	22.68	33.72	16.24	13.58	7.88
BPA	29.38	24.00	62.80	61.82	24.98	34.96	17.52	14.38	8.90
Table A11: Untargeted attack success rates (%) of PGD on nine models when generating adversarial examples on ResNet-50 with modifications on ReLU layers.
A.8 Comparison between Random Replacement and BPA on Max-pooling Layers
For max-pooling, the initial replacement of zeros in the gradient with ones results in an increase in relevance and subsequent improvement in attack performance. However, as the number of replaced zeros increases, the vallina BP faces challenges in accurately discerning which elements are critical that are related to the magnitude of input values. This indiscriminate replacement introduces a substantial error, leading to a decay in attack performance. Hence, we also compare randomly replacing the zeros with ones using the probability of 0.3 (the best result in Fig. 2c) and BPA for max-pooling on ResNet-50. As shown in Table A12, BPA exhibits better transferability, which further validates our motivation and indicates that the effectiveness of our BPA is not solely attributed to achieving linearity.
Attacker	Method	Inc-v3	IncRes-v2	DenseNet	MobileNet	PNASNet	SENet	Inc-v3ens3	Inc-v2ens4	IncRes-v2ens
PGD	N/A	16.34	13.38	36.86	36.12	13.46	17.14	10.24	9.46	5.52
Replace(0.3)	14.52	11.94	37.52	36.02	13.84	17.28	10.34	10.56	6.48
BPA	20.26	16.16	44.66	42.82	17.12	21.52	13.20	11.88	7.74
Table A12: Untargeted attack success rates (\%) of PGD on nine models when generating adversarial examples on ResNet-50 with modifications on max-pooling layers.
A.9 Relevance between Gradient w.r.t. Input and Loss Function
The relevance between gradient w.r.t. input and loss function indicates the sensitivity of the loss function to changes in the input when taking a small step in the direction of gradient. This relevance can be formally defined as follows:
Definition 1 (Relevance between gradient w.r.t. input and loss function) Given an input  $x$ , loss function  $J(x)$  and a step size  $\epsilon$ , the relevance between gradient w.r.t. input and objective function can be defined as  $\frac{J(x + \epsilon \cdot \nabla_x J(x)) - J(x)}{\epsilon}$ .
ReLU helps address the vanishing gradient issue during the training of deep models by eliminating some gradients. However, BPA focuses on boosting transferability of adversarial examples generated on such ReLU-activated networks. To effectively generate adversarial examples, it is crucial that the gradient w.r.t. the input provides a reliable direction to maximize the loss. Unfortunately, the truncation of ReLU makes the calculated gradient unable to provide such exactly precise direction, e.g., making the gradient weakened to some extent. Similarly, max-pooling also truncates the gradient during the backpropagation, causing the gradient unable to indicate an exactly precise direction.
As summarized above, the truncation of ReLU and max-pooling drops gradient (introduces zeros) in the backpropagation process, which decays the relevance. We also calculate the Relevance using vallina backpropagation and our backpropagation on ResNet-50 using 1,000 images. BPA achieved the relevance of 240.51, while the gradient calculated by the vallina backpropagation achieves lower relevance(149.23), which supports our hypothesis that non-linear layers can decay the relevance.