SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Assuming  $\mathcal{G} \in \mathcal{B}(\mathcal{G})$  leads to the following lower bound on the adversarial misclassification rate:  $\mathcal{L}_{0/1}(f_{\theta}) \leq \mathcal{L}_{adv}(f_{\theta})$ , valid for all  $\theta$ . This implies (i)  $\min_{\theta} \mathcal{L}_{0/1}(f_{\theta}) \leq \min_{\theta} \mathcal{L}_{adv}(f_{\theta})$ . Now, assuming  $\theta^{*} = \arg \min_{\theta} \mathcal{L}_{0/1}(f_{\theta})$  is given and using our modified learning algorithm  $\tilde{f}_{\theta^{*}}$ , we obtain
$$
\mathcal {L} _ {a d v} \left(\tilde {f} _ {\theta^ {*}}\right) = \min  _ {\theta} \mathcal {L} _ {a d v} \left(\tilde {f} _ {\theta}\right) \leq \min  _ {\theta} \mathcal {L} _ {a d v} \left(f _ {\theta}\right)
$$
The first equality follows from  $\mathcal{L}_{adv}(\tilde{f}_{\theta^{*}}) = \mathcal{L}_{0 / 1}(\tilde{f}_{\theta^{*}}) = \mathcal{L}_{0 / 1}(f_{\theta^{*}}) = \min_{\theta}\mathcal{L}_{0 / 1}(f_{\theta})$  and then, using  $\mathcal{L}_{0 / 1}(f_{\theta}) = \mathcal{L}_{0 / 1}(\tilde{f}_{\theta}) = \mathcal{L}_{adv}(\tilde{f}_{\theta})$  results in  $\min_{\theta}\mathcal{L}_{0 / 1}(f_{\theta}) = \min_{\theta}\mathcal{L}_{adv}(\tilde{f}_{\theta})$ . The last inequality follows from (i). This shows the optimality of  $\tilde{f}_{\theta^{*}}$  w.r.t. the saddle-point problem.
C.2 Empirical Limitations
C.2.1 Adversarial Training Causes Overfitting
Table C.1 shows that GPRGNN adversarially trained (robust diffusion) with  $\epsilon = 1$  on Cora (with self-training) is (almost) perfectly robust against a wide range of attacks, not only PGD. Similar results hold for other common transductive benchmark datasets such as Cora-ML (Table C.2), or Citeseer (Table C.3).
Table C.1: GPRGNN's test-accuracy  $[\%]$  under attack on Cora. This robust diffusion has been adversarially trained with  $\epsilon = 1$  (with self-training) and shows (almost) perfect robustness against a wide range of attacks.
Perturbed Edges [%]	0	1	5	1	25	50	100
PGD	82.0+1.5	81.8+1.4	81.8+1.4	81.8+1.4	81.8+1.4	81.7+1.4	81.7+1.3
FGSM	82.0+1.5	81.8+1.4	81.8+1.4	81.7+1.3	81.3+1.2	81.0+1.1	80.6+1.0
GreedyRBCD	82.0+1.5	81.8+1.4	81.8+1.4	81.7+1.3	81.4+1.2	81.0+1.1	80.7+1.1
PRBCD	82.0+1.5	81.8+1.4	81.8+1.4	81.7+1.4	81.6+1.3	81.5+1.3	81.4+1.2
Table C.2: GPRGNN's test-accuracy  $[\%]$  under attack on Cora-ML. This robust diffusion has been adversarially trained with  $\epsilon = 1$  (with self-training) and shows (almost) perfect robustness against a wide range of attacks.
Perturbed Edges [%]	0	1	5	1	25	50	100
PGD	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9	81.7+1.9	81.7+1.9
DICE	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9
FGSM	81.8+1.9	81.8+2.0	81.7+1.9	81.6+2.0	81.5+2.0	81.2+2.0	80.9+2.0
GreedyRBCD	81.8+1.9	81.8+1.9	81.7+1.9	81.6+1.9	81.5+1.9	81.2+2.0	81.0+2.0
PRBCD	81.8+1.9	81.8+1.9	81.8+1.9	81.8+1.9	81.7+1.9	81.6+1.9	81.5+1.9
Table C.3: GPRGNN's test-accuracy  $[\%]$  under attack on Citeseer. This robust diffusion has been adversarially trained with  $\epsilon = 1$  (with self-training) and shows (almost) perfect robustness against a wide range of attacks.
Perturbed Edges [%]	0	1	5	1	25	50	100
PGD	71.8+0.9	71.8+0.9	71.8+0.9	71.8+0.9	71.7+0.9	71.8+0.8	71.8+0.9
DICE	71.8+0.9	71.8+0.9	71.8+0.9	71.8+0.9	71.8+0.9	71.8+0.9	71.9+0.9
FGSM	71.8+0.9	71.8+0.9	71.7+0.9	71.7+0.9	71.5+0.9	71.3+1.0	71.0+1.0
GreedyRBCD	71.8+0.9	71.8+0.9	71.7+0.9	71.7+0.9	71.5+0.9	71.3+0.9	71.0+1.0
PRBCD	71.8+0.9	71.8+0.9	71.8+0.9	71.7+0.9	71.7+0.9	71.7+0.9	71.4+0.9
C.2.2 Self-Training as the (Main) Cause for Robustness
Figure C.1 shows that self-training is the major cause for robustness not only for GCN but also for other architectures. Figure C.2 highlights that this phenomenon also occurs given different attack strengths in the adversarial training. Figure C.3 shows that these results are not only constrained to the Cora dataset.
(a) GCN
(b) APPNP
(c) Robust Diffusion (GPRGNN)
Figure C.1: Training different architectures with different training schemes on (transductive) Cora with a global training budget of  $\epsilon = 0.1$  ( $10\%$  perturbed edges) and using self-training.
(a) GCN
Figure C.2: Training different architectures with different training schemes on (transductive) Cora with a global training budget of  $\epsilon = 0.05$  ( $5\%$  perturbed edges) and using self-training.
(a) GCN
Figure C.3: Training different architectures with different training schemes on (transductive) Citeseer with a global training budget of  $\epsilon = 0.1$  ( $10\%$  perturbed edges) and using self-training.
D Additional Inductive Results
Here, we present additional results for adversarial training in the inductive setting. We supplement the results of the main part with results on other datasets as well as more extensive results for Citeseer.
D.1 Test Accuracy
Table D.1 supplements Table 1 and additionally shows the results of the investigated models for an adversary with  $\epsilon = 5\%$  and using only self-training without adversarial training. Thereby, we ablate the effects of self-training and adversarial training on the achieved robustness and highlight, that self-training alone already significantly increases robustness, but combining both methods achieves the best results.
Table D.2 and Table D.3 show analogous results for Cora and Cora-ML, respectively and Table D.4 for Pubmed. They again show that robust diffusion achieves the highest robustness values. Table D.5
Table D.1: Comparison on CiteSeer of regularly trained models, the state-of-the-art SoftMedian GDC defense, self-trained models and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the robust accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in robust accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.
Model	Self-trn.	Adv.trn.	A. eval. → A.trn.↓	Clean	LR-BCD ε=5%	PR-BCD	LR-BCD ε=10%	PR-BCD	LR-BCD ε=25%	PR-BCD
GCN	✗	✗	-	72.0 ± 2.5	62.5 ± 1.9	59.3 ± 2.3	54.7 ± 2.8	51.7 ± 2.8	45.3 ± 3.4	38.0 ± 3.8
GAT	✗	✗	-	-3.6 ± 2.7	-3.6 ± 2.4	-0.6 ± 3.0	-3.9 ± 3.4	+0.5 ± 3.5	-15.9 ± 5.3	-2.3 ± 7.3
APPNP	✗	✗	-	+0.2 ± 1.1	-0.9 ± 0.7	+1.6 ± 0.7	+1.7 ± 0.7	+1.9 ± 1.4	+3.0 ± 1.2	+2.2 ± 2.5
GPRGNN	✗	✗	-	+2.2 ± 4.3	+2.0 ± 3.3	+3.1 ± 4.4	+4.2 ± 2.7	+3.6 ± 4.9	+5.5 ± 3.9	+7.9 ± 4.6
ChebNetII	✗	✗	-	+1.1 ± 2.2	+4.0 ± 1.0	+3.4 ± 2.5	+5.8 ± 2.5	+5.0 ± 2.4	+10.4 ± 2.6	+7.6 ± 3.4
SoftMedian	✗	✗	-	+0.9 ± 1.7	+4.4 ± 1.5	+5.9 ± 1.4	+9.5 ± 2.2	+9.3 ± 1.9	+16.2 ± 2.4	+14.6 ± 2.9
GCN	✓	✗	-	+1.1 ± 0.9	+3.0 ± 0.3	+3.4 ± 0.7	+6.4 ± 0.9	+5.1 ± 0.8	+9.2 ± 1.4	+7.5 ± 1.6
GAT	✓	✗	-	+0.2 ± 2.6	+0.8 ± 2.5	+4.8 ± 2.7	+1.6 ± 3.2	+5.5 ± 3.7	-6.7 ± 3.6	+10.0 ± 4.3
APPNP	✓	✗	-	-0.5 ± 2.2	+2.0 ± 0.9	+3.3 ± 1.3	+6.2 ± 2.2	+5.3 ± 2.3	+8.7 ± 2.5	+5.9 ± 3.3
GPRGNN	✓	✗	-	+1.7 ± 3.4	+7.3 ± 2.8	+7.2 ± 3.4	+13.2 ± 3.5	+10.9 ± 3.7	+18.4 ± 4.2	+19.6 ± 5.4
ChebNetII	✓	✗	-	+0.2 ± 1.7	+2.6 ± 0.9	+3.9 ± 1.5	+7.6 ± 2.2	+6.9 ± 1.3	+12.0 ± 1.1	+10.4 ± 2.3
GCN	✓	✓	LR-BCD	-0.2 ± 1.2	+2.3 ± 0.3	+3.0 ± 0.9	+7.8 ± 1.6	+5.9 ± 1.5	+10.9 ± 2.1	+8.1 ± 2.3
PR-BCD	+0.0 ± 1.9	+1.4 ± 0.3	+2.5 ± 0.9	+6.9 ± 0.9	+5.3 ± 1.6	+8.6 ± 2.3	+5.8 ± 2.4
GAT	✓	✓	LR-BCD	+0.8 ± 1.6	+1.7 ± 2.0	+5.1 ± 2.7	+5.9 ± 3.7	+9.0 ± 3.0	+8.4 ± 5.1	+13.1 ± 3.5
PR-BCD	+1.1 ± 2.2	+4.5 ± 1.8	+7.8 ± 1.2	+8.9 ± 2.8	+13.2 ± 3.7	+10.3 ± 2.3	+21.8 ± 4.8
APPNP	✓	✓	LR-BCD	-0.8 ± 1.3	+2.2 ± 1.0	+3.0 ± 1.5	+7.6 ± 1.6	+5.6 ± 1.9	+11.1 ± 1.9	+8.3 ± 3.6
PR-BCD	-0.2 ± 2.2	+2.2 ± 1.2	+3.1 ± 1.8	+6.2 ± 2.3	+5.5 ± 3.1	+9.0 ± 2.9	+6.5 ± 3.8
GPRGNN	✓	✓	LR-BCD	+1.7 ± 3.0	+8.1 ± 2.7	+8.4 ± 3.4	+15.7 ± 3.6	+13.6 ± 3.5	+24.8 ± 4.2	+22.9 ± 4.3
PR-BCD	+0.6 ± 3.6	+8.1 ± 2.8	+10.3 ± 3.4	+15.0 ± 3.6	+15.9 ± 4.2	+23.2 ± 4.3	+26.3 ± 5.4
ChebNetII	✓	✓	LR-BCD	+3.7 ± 1.4	+6.7 ± 0.8	+8.4 ± 1.0	+11.5 ± 1.6	+11.5 ± 1.1	+16.4 ± 1.9	+16.0 ± 2.9
PR-BCD	+3.4 ± 1.7	+7.0 ± 0.9	+10.0 ± 1.3	+13.2 ± 2.2	+14.3 ± 1.2	+19.5 ± 1.6	+19.8 ± 2.3
shows the results for OGB-arXiv. The training split in OGB-arXiv is fully labeled and hence, self-training is not applicable. Adversarial training improves the robustness of the investigated models on most occasions. A learnable, robust diffusion is particularly effective for small  $\epsilon$ , while fixating the diffusion can be advantageous for large  $\epsilon$ . This is most likely explained by the different structure of this large-scale graph compared to the other benchmark graphs, indicated by adversaries with  $\epsilon = 1\%$  or  $\epsilon = 2\%$  being already very effective and corresponding to a large number of adversarial edges.
In Table D.6, we additionally compare the performance using the DICE and PGD attacks. Moreover, we show that our adversarial training approach is a more effective defense than GRAND [30] against LR-BCD.
Table D.2: Comparison on Cora of regularly trained models, the state-of-the-art SoftMedian GDC defense, self-trained models and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the robust accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in robust accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.
Model	Self-trn.	Adv.trn.	A. eval. → A.trn.↓	Clean	LR-BCD ε=5%	PR-BCD	LR-BCD ε=10%	PR-BCD	LR-BCD ε=25%	PR-BCD
GCN	✗	✗	-	79.9 ± 0.9	67.5 ± 0.4	66.7 ± 0.8	60.8 ± 0.2	59.1 ± 0.4	47.9 ± 1.8	44.1 ± 0.7
GAT	✗	✗	-	-2.7 ± 1.6	-2.1 ± 0.9	-1.8 ± 2.3	-5.3 ± 1.0	-1.1 ± 3.2	-11.7 ± 3.9	-0.6 ± 3.1
APPNP	✗	✗	-	+1.6 ± 0.1	+2.4 ± 1.7	+2.2 ± 0.4	+2.4 ± 1.7	+2.7 ± 0.9	+7.2 ± 2.6	+3.1 ± 0.7
GPRGNN	✗	✗	-	+2.0 ± 1.2	+7.4 ± 0.5	+4.0 ± 0.8	+8.5 ± 0.1	+5.3 ± 1.0	+13.8 ± 1.4	+7.9 ± 1.2
ChebNetII	✗	✗	-	+1.0 ± 2.3	+3.3 ± 2.1	+1.1 ± 2.6	+4.0 ± 1.3	+2.1 ± 1.4	+8.2 ± 0.7	+4.9 ± 1.4
SoftMedian	✗	✗	-	-2.9 ± 1.0	+3.8 ± 1.8	+2.2 ± 1.0	+6.6 ± 1.1	+5.9 ± 1.4	+14.8 ± 2.7	+10.4 ± 1.4
GCN	✓	✗	-	+0.6 ± 1.0	+3.5 ± 1.0	+2.0 ± 0.9	+5.4 ± 1.2	+3.2 ± 0.6	+9.4 ± 0.5	+4.6 ± 0.8
GAT	✓	✗	-	-4.2 ± 2.7	-0.5 ± 1.7	-2.2 ± 3.2	-3.7 ± 1.0	-1.7 ± 3.3	-8.8 ± 3.6	-2.4 ± 3.4
APPNP	✓	✗	-	+2.0 ± 0.9	+5.3 ± 1.7	+4.0 ± 0.8	+8.2 ± 1.4	+6.1 ± 1.0	+13.4 ± 2.6	+7.1 ± 0.8
GPRGNN	✓	✗	-	+2.8 ± 1.0	+8.2 ± 0.5	+6.7 ± 0.2	+12.0 ± 0.7	+10.1 ± 1.3	+19.7 ± 2.4	+16.7 ± 3.0
ChebNetII	✓	✗	-	+2.3 ± 0.5	+5.7 ± 1.2	+3.3 ± 0.8	+6.8 ± 1.2	+4.6 ± 0.9	+11.7 ± 2.3	+4.6 ± 1.1
GCN	✓	✓	LR-BCD	+0.5 ± 0.7	+3.3 ± 1.1	+2.1 ± 0.4	+5.7 ± 0.6	+3.1 ± 0.3	+10.4 ± 2.4	+5.4 ± 1.2
PR-BCD	+1.1 ± 1.1	+3.1 ± 0.6	+2.3 ± 0.2	+6.2 ± 0.8	+3.8 ± 0.3	+8.7 ± 1.8	+5.0 ± 0.7
GAT	✓	✓	LR-BCD	-1.5 ± 0.9	+0.4 ± 1.3	+0.6 ± 1.0	+2.4 ± 1.6	+2.3 ± 0.5	+6.5 ± 3.0	+7.3 ± 1.4
PR-BCD	-2.1 ± 0.9	+3.8 ± 0.7	+5.0 ± 1.9	+5.1 ± 1.5	+7.6 ± 1.9	+5.9 ± 3.5	+14.5 ± 3.0
APPNP	✓	✓	LR-BCD	+1.8 ± 1.1	+5.0 ± 1.2	+3.9 ± 0.7	+8.5 ± 0.4	+6.8 ± 0.8	+14.7 ± 2.7	+8.2 ± 1.2
PR-BCD	+1.0 ± 0.7	+3.7 ± 1.1	+3.7 ± 0.6	+6.6 ± 1.3	+5.0 ± 1.1	+12.7 ± 3.5	+7.3 ± 0.2
GPRGNN	✓	✓	LR-BCD	+2.6 ± 0.2	+8.2 ± 0.6	+8.1 ± 0.4	+11.8 ± 1.2	+11.0 ± 0.0	+22.1 ± 2.3	+16.8 ± 1.5
PR-BCD	+0.0 ± 0.6	+8.1 ± 1.1	+8.1 ± 0.4	+13.3 ± 1.3	+12.8 ± 0.7	+24.2 ± 2.4	+22.2 ± 1.4
ChebNetII	✓	✓	LR-BCD	-6.7 ± 9.6	-1.2 ± 7.0	-1.2 ± 6.7	+1.2 ± 6.2	+1.8 ± 4.8	+10.0 ± 5.8	+6.5 ± 1.4
PR-BCD	+1.5 ± 1.5	+3.7 ± 1.8	+4.8 ± 0.4	+7.7 ± 1.5	+6.7 ± 1.4	+11.7 ± 3.3	+7.6 ± 1.4
Table D.3: Comparison on Cora-ML of regularly trained models, the state-of-the-art SoftMedian GDC defense, self-trained models and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the robust accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in robust accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.
Model	Self-trn.	Adv.trn.	A. eval. → A.trn.↓	Clean	LR-BCD ε=5%	PR-BCD	LR-BCD ε=10%	PR-BCD	LR-BCD ε=25%
GCN	✗	✗	-	82.5 ± 1.9	66.0 ± 2.8	66.2 ± 2.5	59.2 ± 2.8	57.4 ± 2.3	48.5 ± 1.6
GAT	✗	✗	-	-1.8 ± 0.4	-5.6 ± 2.1	-6.3 ± 2.1	-11.3 ± 2.1	-6.0 ± 2.2	-21.4 ± 4.3
APPNP	✗	✗	-	+0.6 ± 0.7	+3.3 ± 1.6	+2.3 ± 1.9	+5.3 ± 0.2	+3.1 ± 0.8	+7.0 ± 1.1
GPRGNN	✗	✗	-	+0.9 ± 0.7	+5.0 ± 0.8	+2.5 ± 0.4	+5.6 ± 0.6	+4.5 ± 0.5	+8.6 ± 0.1
ChebNetII	✗	✗	-	-0.2 ± 0.5	+4.2 ± 1.7	+1.4 ± 0.7	+6.0 ± 1.3	+2.6 ± 0.2	+7.0 ± 1.5
SoftMedian	✗	✗	-	-0.8 ± 1.0	+7.7 ± 1.9	+4.8 ± 1.7	+10.7 ± 1.4	+7.7 ± 0.9	+17.0 ± 0.6
GCN	✓	✗	-	+0.7 ± 0.7	+4.5 ± 2.0	+2.6 ± 1.6	+6.1 ± 2.4	+3.9 ± 1.3	+7.3 ± 0.5
GAT	✓	✗	-	-3.6 ± 0.5	-2.8 ± 2.7	-1.9 ± 3.7	-6.9 ± 2.2	-1.3 ± 3.3	-17.0 ± 3.8
APPNP	✓	✗	-	+1.4 ± 0.7	+5.8 ± 2.0	+4.6 ± 1.6	+7.3 ± 1.7	+5.2 ± 0.4	+10.1 ± 0.5
GPRGNN	✓	✗	-	+2.1 ± 1.3	+10.7 ± 1.8	+7.7 ± 1.8	+13.5 ± 1.5	+9.9 ± 2.1	+17.0 ± 1.4
ChebNetII	✓	✗	-	+1.8 ± 0.5	+8.0 ± 1.7	+4.9 ± 2.1	+9.9 ± 2.2	+6.2 ± 1.8	+10.8 ± 1.7
GCN	✓	✓	LR-BCD	+1.2 ± 0.7	+6.1 ± 1.9	+3.5 ± 1.4	+8.6 ± 1.8	+3.1 ± 0.5	+10.9 ± 0.5
PR-BCD	+0.5 ± 0.6	+5.4 ± 1.3	+3.1 ± 1.0	+8.2 ± 1.1	+3.5 ± 1.0	+10.3 ± 0.7
GAT	✓	✓	LR-BCD	-2.7 ± 0.6	+4.3 ± 1.9	+1.8 ± 1.6	+7.0 ± 1.9	+7.2 ± 1.9	+10.7 ± 0.1
PR-BCD	-2.1 ± 1.0	+2.9 ± 2.7	+1.3 ± 2.2	+5.0 ± 3.0	+4.8 ± 2.6	+5.6 ± 1.4
APPNP	✓	✓	LR-BCD	+0.9 ± 0.9	+6.7 ± 2.1	+4.7 ± 1.4	+9.6 ± 1.4	+5.4 ± 1.2	+12.9 ± 0.5
PR-BCD	+1.1 ± 0.4	+7.3 ± 1.6	+5.8 ± 1.7	+10.6 ± 1.6	+6.0 ± 0.9	+13.4 ± 0.9
GPRGNN	✓	✓	LR-BCD	+0.8 ± 1.2	+12.0 ± 3.2	+8.9 ± 2.7	+17.6 ± 3.7	+13.1 ± 2.8	+25.9 ± 3.4
PR-BCD	-0.0 ± 1.6	+13.4 ± 3.5	+10.7 ± 2.9	+18.2 ± 4.4	+16.1 ± 3.9	+28.2 ± 3.6
ChebNetII	✓	✓	LR-BCD	+0.9 ± 0.6	+7.9 ± 1.6	+6.2 ± 1.7	+10.9 ± 1.9	+8.7 ± 0.8	+15.0 ± 0.1
PR-BCD	+1.1 ± 0.4	+8.5 ± 2.0	+6.5 ± 1.2	+13.1 ± 1.7	+8.9 ± 0.9	+15.1 ± 1.1
Table D.4: Comparison on Pubmed of regularly trained models, self-trained models and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the robust accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in robust accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.
Model	Self-trn.	Adv.trn.	A.eval.→A.trn.↓	Clean	LR-BCDε=5%	PR-BCD	LR-BCDε=10%	PR-BCD	LR-BCDε=25%	PR-BCD
GCN	X	X	-	77.4 ± 0.4	64.3 ± 0.9	62.2 ± 0.8	60.4 ± 0.7	54.0 ± 0.5	52.8 ± 1.4	39.3 ± 1.2
GAT	X	X	-	-3.0 ± 1.7	-5.6 ± 4.5	-1.5 ± 2.8	-10.3 ± 6.5	-0.9 ± 3.7	-14.8 ± 8.5	-0.9 ± 6.1
APPNP	X	X	-	+0.5 ± 0.5	+1.9 ± 0.0	+1.8 ± 0.9	+1.7 ± 0.1	+2.5 ± 0.4	+2.4 ± 0.2	+1.1 ± 0.9
GPRGNN	X	X	-	+1.4 ± 1.3	+1.3 ± 2.0	+2.8 ± 1.2	+0.7 ± 2.1	+3.0 ± 1.7	+1.2 ± 2.3	+3.4 ± 1.0
GCN	✓	X	-	+1.8 ± 0.9	+3.4 ± 1.3	+3.1 ± 1.0	+4.0 ± 1.5	+4.1 ± 0.6	+5.1 ± 1.7	+2.9 ± 0.6
GAT	✓	X	-	-0.3 ± 1.0	-0.1 ± 1.4	+0.5 ± 0.3	-2.3 ± 1.9	+0.6 ± 1.9	-3.8 ± 3.4	-1.7 ± 2.5
APPNP	✓	X	-	+2.7 ± 1.1	+5.5 ± 1.3	+5.7 ± 1.2	+6.1 ± 1.2	+6.7 ± 1.1	+7.3 ± 1.8	+6.0 ± 1.3
GPRGNN	✓	X	-	+1.7 ± 1.1	+6.0 ± 1.5	+5.6 ± 0.9	+6.6 ± 1.8	+7.8 ± 1.4	+9.0 ± 2.9	+11.5 ± 2.1
GCN	✓	✓	LR-BCD	+2.2 ± 0.8	+5.3 ± 1.1	+4.2 ± 0.9	+6.1 ± 1.0	+4.4 ± 0.5	+7.9 ± 1.4	+4.5 ± 0.9
PR-BCD	+1.1 ± 1.6	+5.8 ± 1.2	+4.3 ± 1.2	+6.8 ± 0.9	+5.0 ± 1.2	+8.8 ± 1.3	+5.3 ± 0.7
GAT	✓	✓	LR-BCD	-0.1 ± 1.3	+4.0 ± 1.0	+4.7 ± 1.2	+4.9 ± 0.8	+8.2 ± 0.4	+8.4 ± 1.3	+13.0 ± 1.8
PR-BCD	-0.3 ± 1.0	+3.2 ± 1.5	+3.8 ± 1.4	+3.1 ± 1.3	+6.2 ± 0.7	+4.1 ± 1.7	+9.0 ± 2.5
APPNP	✓	✓	LR-BCD	+2.6 ± 1.4	+6.9 ± 1.2	+6.9 ± 1.4	+8.5 ± 1.2	+8.8 ± 1.0	+11.2 ± 1.7	+8.8 ± 1.3
PR-BCD	+2.6 ± 1.3	+7.8 ± 1.8	+7.5 ± 1.9	+9.1 ± 1.5	+9.4 ± 1.4	+12.5 ± 2.0	+9.1 ± 1.9
GPRGNN	✓	✓	LR-BCD	+3.1 ± 1.1	+12.6 ± 1.8	+9.9 ± 1.1	+15.4 ± 1.7	+14.1 ± 1.6	+21.9 ± 2.5	+20.6 ± 2.0
PR-BCD	+3.0 ± 0.8	+13.6 ± 1.5	+13.2 ± 1.0	+16.7 ± 1.2	+19.0 ± 1.6	+23.5 ± 1.9	+29.5 ± 1.7
Table D.5: Comparison on OGB-arXiv of regularly trained models and adversarially trained models (train  $\epsilon = 5\%$ ). All numbers are in  $\%$  . The best model is highlighted in bold. In general, adversarial training is effective in increasing adversarial robustness.
Model	Adv. trn.	A. eval. → A. trn. ↓	Clean	LR-BCD ε=1%	PR-BCD ε=2%	LR-BCD ε=5%	PR-BCD ε=10%	LR-BCD ε=4%
APPNP	✗	-	70.6 ± 0.2	61.6 ± 0.3	62.6 ± 0.2	58.3 ± 0.4	59.0 ± 0.3	52.4 ± 0.5
GPRGNN	✗	-	71.9 ± 0.1	63.9 ± 0.2	64.8 ± 0.2	60.7 ± 0.3	61.0 ± 0.2	54.6 ± 0.4
54.0 ± 0.4
APPNP	✓	LR-BCD	68.2 ± 0.1	63.1 ± 0.1	63.2 ± 0.1	60.4 ± 0.1	60.3 ± 0.1	56.5 ± 0.2
PR-BCD	68.2 ± 0.4	62.8 ± 0.3	62.5 ± 0.3	59.8 ± 0.3	59.0 ± 0.3	55.4 ± 0.2
GPRGNN	✓	LR-BCD	70.9 ± 0.0	64.4 ± 0.1	63.8 ± 0.1	61.1 ± 0.1	59.8 ± 0.1	55.9 ± 0.1
PR-BCD	70.3 ± 0.2	63.8 ± 0.2	63.5 ± 0.2	60.6 ± 0.2	59.6 ± 0.2	55.5 ± 0.2
D.2 Learned Diffusion Coefficients
Figure D.1 shows the learned coefficients  $\gamma$  for GPRGNN on Citeseer, Cora, Cora-ML and Pubmed. The results for Citeseer are the ones reported in Figure 6 and given again for reference. The observation that including local constraints during adversarial training leads to a robust model leveraging the
Table D.6: Further adversarial attacks on Citeseer and additional GRAND defense. The LR-BCD results also used LR-BCD during the adversarial training. For PGD and DICE, we perform adversarial training with PR-BCD to match the set of admissible perturbations.
Attack evaluation	Model	Adv. trn.	Clean	ε=0.1	ε=0.25
LR-BCD	GCN	✗	72.0 ± 1.6	53.2 ± 1.9	41.7 ± 2.8
✓	72.0 ± 1.5	59.3 ± 1.5	48.8 ± 2.4
SoftMedian GDC	✗	72.9 ± 0.8	62.6 ± 1.0	57.1 ± 2.1
GRAND	✗	74.6 ± 2.4	65.7 ± 3.3	59.7 ± 3.3
GPRGNN	✗	74.1 ± 1.1	58.0 ± 1.0	49.1 ± 1.1
✓	72.9 ± 0.6	67.8 ± 0.8	64.6 ± 1.2
PGD	GCN	✗	73.4 ± 1.3	57.9 ± 3.0	46.4 ± 3.1
GPRGNN	✗	74.1 ± 0.2	63.2 ± 1.3	57.6 ± 2.8
✓	73.7 ± 0.6	67.6 ± 1.2	64.8 ± 1.3
DICE	GCN	✗	73.4 ± 1.3	71.8 ± 1.5	68.8 ± 1.1
GPRGNN	✗	74.1 ± 0.2	73.8 ± 1.0	70.9 ± 0.8
✓	73.7 ± 0.6	73.7 ± 1.0	73.1 ± 0.8
(a) Citeseer
(b) Cora
(c) Cora-ML
(d) Pubmed
Figure D.1: Learned robust diffusion (GPRGNN) coefficients  $\gamma$  for different datasets. For each dataset, left figure represents standard training, middle figure represents LR-BCD (adv. training w/ local constraints), right figure PR-BCD (adv. training w/o local constraints). For adversarial training,  $\epsilon = 20\%$  is used.
general graph structure in a robust way compared to training with global constraints only is general for all datasets. Training only with global constraints, for node classification, the information in the graph other than the node's own features are suppressed, as only  $\gamma_0$  and  $\gamma_{2}$  and, to a lesser extent,  $\gamma_{4}$  are pronounced. These mainly correspond to information about the original node itself, as the information about itself propagates to its neighbor and back in 2-hops and similar for the second-hop neighbors. However, information from  $\gamma_{k}$  with  $k$  odd is suppressed through being set close to zero.
A spectral analysis as carried out in Figure 7 can be found for Cora and Cora-ML (and Citeseer for reference) in Figure D.2. The behavior that PR-BCD training leads to high-pass behavior is consistent across datasets.
(a)CiteSeer
(b) Cora
Figure D.2: Spectral filters corresponding to Figure D.1 for Citeseer, Cora and Cora-ML. Low filter magnitude denotes suppression of the respective frequencies associated with eigenvalue  $\lambda$ .
(c) Cora-ML
D.3 Comparison of PR-BCD and LR-BCD
(a) GPRGNN
Figure D.3: Comparison of PR-BCD and LR-BCD on Cora-ML. The x-axis gives the relative global budget. The local budget (if considered) for each node  $u$  of degree  $d_u$  is  $\Delta_u^{(l)} = \lfloor d_u / 2 \rfloor$ . The models are learned using self-training. Results are aggregated over three different data splits.
Figure D.3 compares the efficacy of PR-BCD to LR-BCD on Cora-ML. For a fair comparison, as the admissible perturbations  $\mathcal{B}(\mathcal{G})$  are inherently different for both attacks, we also consider LR-BCD with an unlimited local budget. In this case, LR-BCD's projection defaults to greedily picking the highest-valued perturbations until the global budget is met. We observe that LR-BCD w/o local constraints performs similar to PR-BCD. This suggests that the chosen greedy approach is suitable. Additionally including local constraints, the attack efficacy reduces. Still, even with local constraints, LR-BCD is able to recover adversarial examples that significantly reduce classification performance.
(b) APPNP
(c) GCN
D.4 Certified Accuracy
Table D.7 shows certified accuracies for Citeseer against an extended set of perturbations compared to Table 1. Table D.8 and Table D.9 show analogous results for the benchmark graphs Cora and Cora-ML, respectively. On all datasets, highest certifiable accuracy is achieved through robust diffusion.
Table D.7: Comparison of certifiable accuracies for different perturbations on CiteSeer of regularly trained models, the state-of-the-art SoftMedian GDC defense, and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the certifiable accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in certifiable accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.
Model	Adv. trn.	Training attack	Clean	1 add.	2 add.	Certifiable accuracy / sparse smoothing
3 add.	4 add.	1 del.	3 del.	5 del.	7 del.
GCN	X	-	38.3 ± 7.2	15.6 ± 3.4	1.9 ± 0.5	1.7 ± 0.4	0.0 ± 0.0	25.2 ± 5.5	12.0 ± 2.8	4.8 ± 1.0	1.9 ± 0.5
GAT	X	-	-14.0 ± 7.6	-15.1 ± 3.2	-1.9 ± 0.5	-1.7 ± 0.4	+0.0 ± 0.0	-19.3 ± 5.2	-12.0 ± 2.8	-4.8 ± 1.0	-1.9 ± 0.5
APPNP	X	-	+8.9 ± 5.8	+26.2 ± 3.4	+31.8 ± 4.4	+31.2 ± 4.0	+26.6 ± 2.2	+19.0 ± 3.2	+28.7 ± 3.4	+31.6 ± 4.0	+31.9 ± 4.5
GPRGNN	X	-	+17.9 ± 4.4	+34.6 ± 2.0	+42.5 ± 2.7	+42.4 ± 2.8	+37.5 ± 2.9	+26.8 ± 3.0	+36.6 ± 2.1	+41.3 ± 2.4	+42.5 ± 2.7
ChebNetII	X	-	+24.6 ± 9.9	+45.5 ± 4.4	+55.9 ± 1.3	+55.6 ± 1.2	+54.8 ± 2.0	+36.9 ± 7.3	+47.8 ± 3.2	+54.0 ± 0.9	+56.1 ± 1.2
SoftMedian	X	-	+25.2 ± 10.5	+47.4 ± 4.0	+60.1 ± 1.3	+60.3 ± 1.4	+61.5 ± 2.0	+38.2 ± 7.6	+50.6 ± 3.0	+57.5 ± 0.8	+60.3 ± 1.4
GCN	✓	LR-BCD	-3.0 ± 6.0	+11.2 ± 3.3	+11.4 ± 2.9	+10.7 ± 2.9	+6.1 ± 1.7	+5.1 ± 5.5	+10.3 ± 3.4	+13.1 ± 3.2	+11.5 ± 3.0
PR-BCD	+4.2 ± 9.2	+10.1 ± 3.0	+10.6 ± 3.3	+10.1 ± 3.5	+6.2 ± 2.8	+6.9 ± 6.3	+10.7 ± 2.0	+11.4 ± 2.8	+10.6 ± 3.4
GAT	✓	LR-BCD	-2.0 ± 14.6	+5.1 ± 5.9	+2.6 ± 1.1	+1.4 ± 1.1	+1.6 ± 0.7	+3.0 ± 10.7	+4.2 ± 4.4	+4.0 ± 1.3	+3.0 ± 1.0
PR-BCD	-10.1 ± 8.7	+5.0 ± 2.5	+1.9 ± 1.1	+1.2 ± 1.3	+2.2 ± 0.8	-0.8 ± 5.5	+4.5 ± 1.3	+2.8 ± 0.8	+2.2 ± 0.9
APPNP	✓	LR-BCD	+21.7 ± 6.0	+38.0 ± 1.9	+41.6 ± 1.9	+41.3 ± 1.8	+36.6 ± 1.8	+31.2 ± 4.0	+40.2 ± 1.5	+44.1 ± 0.8	+41.9 ± 2.0
PR-BCD	+19.3 ± 4.6	+37.5 ± 1.2	+41.4 ± 2.2	+41.0 ± 2.4	+37.9 ± 2.6	+29.8 ± 2.4	+38.9 ± 1.5	+41.9 ± 1.9	+41.4 ± 2.2
GPRGNN	✓	LR-BCD	+34.0 ± 7.3	+55.5 ± 3.9	+67.6 ± 1.1	+67.4 ± 0.9	+67.9 ± 0.5	+46.7 ± 5.8	+58.6 ± 3.4	+65.0 ± 1.6	+67.6 ± 1.1
PR-BCD	+32.9 ± 7.5	+55.3 ± 3.9	+68.8 ± 1.0	+69.0 ± 0.9	+70.4 ± 0.4	+45.8 ± 5.9	+58.9 ± 3.3	+65.9 ± 1.5	+68.8 ± 1.0
ChebNetII	✓	LR-BCD	+31.5 ± 12.7	+51.2 ± 6.5	+62.1 ± 1.9	+62.3 ± 1.9	+61.8 ± 2.2	+43.1 ± 9.8	+53.9 ± 5.4	+59.8 ± 2.5	+62.1 ± 1.9
PR-BCD	+30.7 ± 13.2	+52.5 ± 7.0	+65.4 ± 2.7	+65.4 ± 2.6	+66.0 ± 2.1	+43.1 ± 10.4	+56.1 ± 5.9	+62.9 ± 3.7	+65.4 ± 2.7
Table D.8: Comparison of certifiable accuracies for different perturbations on Cora of regularly trained models, the state-of-the-art SoftMedian GDC defense, and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the certifiable accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in certifiable accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.