SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Model	Adv. trn.	Training attack	Clean	1 add.	2 add.	Certifiable accuracy / sparse smoothing
3 add.	4 add.	1 del.	3 del.	5 del.	7 del.
GCN	✘	-	30.9 ± 0.2	19.9 ± 3.2	12.9 ± 4.1	12.3 ± 4.1	5.9 ± 1.7	23.7 ± 2.0	18.4 ± 3.7	15.5 ± 4.1	13.2 ± 4.2
GAT	✘	-	-0.5 ± 4.8	-7.8 ± 8.0	-12.1 ± 3.8	-11.5 ± 3.8	-5.5 ± 1.5	-8.4 ± 6.4	-11.4 ± 5.5	-13.9 ± 4.0	-12.3 ± 4.0
APPNP	✘	-	+26.4 ± 1.2	+30.4 ± 3.5	+29.5 ± 5.2	+28.9 ± 5.3	+30.8 ± 2.5	+29.4 ± 2.2	+30.5 ± 4.1	+28.8 ± 5.2	+29.5 ± 5.3
GPRGNN	✘	-	+20.6 ± 2.4	+25.6 ± 6.1	+25.3 ± 6.3	+25.4 ± 6.1	+27.0 ± 4.0	+24.2 ± 4.9	+26.3 ± 6.2	+25.4 ± 6.2	+25.2 ± 6.4
ChebNetII	✘	-	+32.8 ± 1.8	+40.7 ± 6.9	+46.2 ± 7.8	+46.8 ± 7.8	+52.4 ± 4.2	+38.1 ± 5.2	+42.1 ± 7.6	+44.7 ± 8.2	+46.2 ± 8.1
SoftMedian	✘	-	+36.0 ± 1.2	+46.2 ± 4.6	+51.5 ± 5.9	+51.9 ± 5.8	+57.4 ± 3.0	+42.5 ± 2.9	+47.0 ± 5.3	+49.2 ± 5.7	+51.4 ± 6.2
GCN	✓	LR-BCD	+4.5 ± 2.5	+5.6 ± 3.6	-1.3 ± 6.6	-1.7 ± 6.7	-2.3 ± 3.3	+6.3 ± 2.0	+3.8 ± 4.9	+0.7 ± 6.1	-1.2 ± 6.9
PR-BCD	+13.7 ± 6.2	+7.9 ± 4.5	-2.0 ± 1.0	-3.3 ± 1.5	-2.7 ± 0.7	+10.6 ± 5.1	+4.8 ± 3.0	+0.0 ± 0.6	-2.0 ± 1.2
GAT	✓	LR-BCD	+0.6 ± 0.5	+9.4 ± 3.1	+12.2 ± 3.5	+12.1 ± 3.8	+3.4 ± 3.1	+6.2 ± 2.0	+10.7 ± 3.5	+12.1 ± 3.7	+12.0 ± 3.7
PR-BCD	+6.4 ± 1.4	+1.5 ± 7.4	-4.8 ± 1.5	-4.2 ± 1.6	-2.4 ± 1.0	+4.8 ± 4.7	-2.9 ± 4.2	-5.7 ± 2.0	-4.6 ± 1.6
APPNP	✓	LR-BCD	+25.3 ± 3.5	+31.1 ± 6.6	+30.2 ± 6.9	+30.2 ± 6.9	+32.5 ± 4.0	+28.7 ± 5.5	+30.8 ± 7.0	+30.4 ± 7.3	+29.9 ± 7.1
PR-BCD	+31.5 ± 1.1	+36.4 ± 3.3	+36.6 ± 4.6	+35.8 ± 4.5	+37.4 ± 2.2	+35.4 ± 1.7	+37.1 ± 3.7	+36.9 ± 4.1	+36.6 ± 4.7
GPRGNN	✓	LR-BCD	+43.8 ± 0.5	+51.4 ± 3.4	+54.6 ± 4.9	+55.2 ± 4.9	+59.1 ± 3.0	+49.1 ± 2.3	+52.5 ± 3.8	+53.8 ± 4.5	+54.3 ± 5.1
PR-BCD	+45.5 ± 0.7	+55.7 ± 2.8	+61.9 ± 3.6	+62.5 ± 3.5	+68.6 ± 1.3	+52.1 ± 1.6	+56.8 ± 3.1	+59.5 ± 3.4	+61.7 ± 3.7
ChebNetII	✓	LR-BCD	+24.8 ± 2.8	+34.1 ± 4.5	+38.5 ± 5.3	+38.7 ± 5.5	+41.8 ± 1.7	+30.9 ± 3.1	+35.2 ± 5.2	+36.5 ± 5.3	+38.3 ± 5.5
PR-BCD	+27.1 ± 2.0	+35.8 ± 6.1	+38.6 ± 7.0	+39.1 ± 7.0	+44.7 ± 3.3	+33.9 ± 3.6	+36.5 ± 6.8	+38.3 ± 7.7	+38.3 ± 7.2
Table D.9: Comparison of certifiable accuracies for different perturbations on Cora-ML of regularly trained models, the state-of-the-art SoftMedian GDC defense, and adversarially trained models (train  $\epsilon = 20\%$ ). The first line shows the certifiable accuracy  $[\%]$  of a standard GCN and all other numbers represent the difference in certifiable accuracy achieved by various models in percentage points from this standard GCN. The best model is highlighted in bold.
Model	Adv. trn.	Training attack	Clean	1 add.	2 add.	3 add.	4 add.	1 del.	3 del.	5 del.	7 del.
GCN	✘	-	37.6 ± 0.9	23.6 ± 2.1	9.6 ± 3.8	8.8 ± 3.7	3.8 ± 2.2	29.9 ± 0.8	19.7 ± 2.9	14.3 ± 4.1	10.1 ± 3.8
GAT	✘	-	-12.4 ± 4.1	-3.6 ± 4.4	-2.8 ± 6.0	-4.2 ± 5.3	-3.3 ± 2.3	-7.6 ± 3.8	-0.4 ± 4.8	-2.7 ± 7.5	-3.2 ± 6.1
APPNP	✘	-	+17.3 ± 1.5	+26.1 ± 1.3	+34.5 ± 3.5	+33.8 ± 3.3	+34.9 ± 2.0	+21.9 ± 0.4	+29.0 ± 2.1	+32.3 ± 3.5	+34.0 ± 3.5
GPRGNN	✘	-	+25.4 ± 3.4	+32.0 ± 1.8	+38.1 ± 2.9	+38.6 ± 2.8	+39.3 ± 1.7	+29.2 ± 2.2	+34.5 ± 1.7	+35.6 ± 3.0	+37.8 ± 2.9
ChebNetII	✘	-	+29.9 ± 3.2	+40.8 ± 6.6	+51.6 ± 9.2	+52.5 ± 9.1	+55.3 ± 7.1	+35.9 ± 4.7	+43.8 ± 7.5	+48.0 ± 9.0	+51.4 ± 9.2
SoftMedian	✘	-	+35.6 ± 0.4	+46.7 ± 5.1	+57.2 ± 7.3	+57.7 ± 7.5	+60.8 ± 5.0	+41.8 ± 2.8	+49.8 ± 6.1	+53.8 ± 7.9	+56.9 ± 7.4
GCN	✓	LR-BCD	+11.9 ± 4.3	+7.6 ± 1.9	+1.5 ± 5.4	+0.7 ± 5.5	+0.1 ± 3.5	+8.2 ± 1.6	+6.6 ± 3.4	+3.5 ± 6.4	+1.4 ± 5.6
PR-BCD	+1.8 ± 3.8	+1.2 ± 4.3	+2.8 ± 7.1	+2.8 ± 7.1	+3.5 ± 5.1	+0.0 ± 3.6	+1.3 ± 5.3	+2.0 ± 6.9	+2.8 ± 7.3
GAT	✓	LR-BCD	-5.5 ± 3.1	+5.6 ± 1.4	+11.7 ± 7.4	+11.6 ± 7.7	+10.2 ± 6.7	+0.6 ± 0.7	+8.3 ± 2.8	+10.8 ± 5.8	+11.5 ± 7.4
PR-BCD	-13.0 ± 1.6	-3.3 ± 6.6	+6.7 ± 10.4	+7.4 ± 10.4	+10.0 ± 8.6	-8.1 ± 4.3	-0.9 ± 7.4	+2.1 ± 9.6	+6.2 ± 10.5
APPNP	✓	LR-BCD	+28.4 ± 0.5	+34.3 ± 2.9	+39.1 ± 4.5	+39.0 ± 4.3	+39.4 ± 3.0	+30.9 ± 1.8	+36.0 ± 3.4	+37.2 ± 4.8	+39.0 ± 4.5
PR-BCD	+25.6 ± 1.9	+32.7 ± 3.8	+38.5 ± 5.2	+38.6 ± 5.1	+38.8 ± 3.9	+29.8 ± 3.0	+34.5 ± 4.2	+36.3 ± 5.0	+38.0 ± 5.3
GPRGNN	✓	LR-BCD	+41.2 ± 1.4	+52.9 ± 1.8	+64.9 ± 3.2	+65.3 ± 3.0	+68.7 ± 1.8	+47.8 ± 0.8	+56.5 ± 2.6	+61.0 ± 3.7	+64.4 ± 3.2
PR-BCD	+42.4 ± 1.6	+56.0 ± 1.6	+69.5 ± 3.2	+70.2 ± 3.2	+74.9 ± 1.8	+49.9 ± 0.6	+59.6 ± 2.5	+65.0 ± 3.7	+69.0 ± 3.3
ChebNetII	✓	LR-BCD	+39.6 ± 1.9	+51.4 ± 2.7	+62.1 ± 5.7	+62.8 ± 5.5	+65.7 ± 3.5	+45.7 ± 0.8	+54.6 ± 3.9	+58.7 ± 6.1	+61.6 ± 5.8
PR-BCD	+36.7 ± 1.2	+49.8 ± 3.8	+62.2 ± 6.5	+63.0 ± 6.4	+66.2 ± 3.9	+43.9 ± 1.5	+53.2 ± 5.1	+58.2 ± 6.9	+61.7 ± 6.6
E Further Related Work
Defenses on Graphs. Next to adversarial training, previous works proposed defenses based on smoothing [32], preprocessing the graph structure [50-52], or using robust model structures [46, 48]. For an extended survey, we refer to Gunnemann [12].
Attacks on Graphs. Generally, one distinguishes evasion attacks, perturbing the graph after training [7, 2, 17] and poisoning attacks, perturbing the graph before training [7, 47]. Further, one differentiates global attacks perturbing multiple node predictions at once [17] from local (targeted) attacks perturbing a single node's prediction [7]. Attacks can perturb the nodes attributes [7], the edge structure [7, 47, 49, 2, 17], or insert malicious nodes [53]. An adversarial attack should not change the true classes (unnoticeability) [18]. For certain applications, like combinatorial optimization, it might be known how the true label is affected by perturbation. For example, Geisler et al. [54] study such adversarial attacks (& training) for GNNs for combinatorial optimization. However, in the context of node classification, this relationship is fuzzier. For example, Zügner et al. [7] define an attack that preserves the global degree distribution, or Gosch et al. [6] analyzes semantic preservation in synthetic graphs and show that changing more edges than the degree of nodes is usually never unnoticeable. In this work, we focus on evasion attacks on the edge structure, aiming at globally attacking node predictions with realistic local constraints enforcing unnoticeability.
F Limitations
This work focuses on adversarial robustness against structure perturbations for node classification. While this is the most studied and prevalent case in previous works (see Gunnemann [12] and Table 3), there are many other interesting graph learning tasks or extended settings, out of scope for this work, for which robustness and adversarial training schemes could be of interest, as e.g., link prediction, node feature perturbations or graph injection. Conceptually, the adversarial training procedure and LR-BCD attack can also be applied to the aforementioned tasks as long as the model remains differentiable.
On another note, while we finally put into the hands of practitioners a global attack method aware of local constraints, there is no clear strategy on how to determine the optimal local budget. Indeed, these have to be set application dependent by domain experts. For example, Gosch et al. [6] leverage knowledge about the data generating distribution to determine budgets that correspond to semantic-preserving perturbations. However, in most real-world applications, we lack knowledge about the data generating distribution. As a result, we also use the different interpretability aspects of robust diffusion to collect additional evidence and insights into the learned message passing (see Section 3 & 5).
In Section 4, we report the asymptotic complexity of our LR-BCD attack. In Section 5, we detail the used hardware for the adversarial training experiments and report the computational cost in terms of runtime as well as memory on the arXiv graph, consisting of 170k nodes.