SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[41] for certification. (2) Adversarial training for GNNs under structure perturbations have been studied in [2, 3, 36-39]. However, all prior works study a transductive setting (see Table 3) having serious shortcomings for evasion attacks (see Section 2). Note that only Xu et al. [2, 3] study adversarial training for (global) structure perturbations in its literal sense, i.e., directly work on Equation (1). Further, we are the first to study adversarial training in the inductive setting. (3) Local constraints have been studied for local attacks [7, 17, 42, 12], i.e., if attacking a single node's prediction. They are also well-established for robustness certificates [43-45]. However, surprisingly, there is no global attack, i.e., attack targeting a large set of node predictions at once, that incorporates local constraints. This even led to prior work trying to naively and expensively apply local attacks for each node sequentially to generate locally constrained global perturbations [38]. With LR-BCD, we finally close this gap. (4) GNN Architectures. The robust GNN literature strongly focuses on GCN and variants [7, 17, 46-49, 38, 50, 8]. GAT is perhaps the most flexible studied model [40]. While adversarial training improves the robustness of GAT, in our experiments, an adversarial trained GAT is not more robust than a GCN [4]. A novel aspect of our work is that spectral filters, in the form of polynomials [14, 13], can learn significantly more robust representations, beating a state-of-the-art graph defense. They also reveal insights about the learned robust representation.
Table 3: Works on adversarial training for GNNs.
Publication	Learning setting	Type of attack
Deng et al. [34]	Transductive	evasion (attribute)
Feng et al. [35]	Transductive	evasion (attribute)
Jin and Zhang [36]	Transductive	evasion (structure + attribute)
Xu et al. [2]	Transductive	evasion (structure)
Xu et al. [3]	Transducitve	evasion (structure)
Chen et al. [37]	Transductive	evasion (structure)
Li et al. [38]	Transductive	evasion (structure + attribute)
Guo et al. [39]	Transductive	evasion + poisoning (structure)
7 Broader impact
We are convinced that the benefits outweigh the risks. Having the right tools at hand can further the reliability of graph machine learning. Due to the more fine-grained perturbation models, researchers and practitioners and improve upon defending adversarial attacks. We firmly believe that transparent research into the vulnerabilities of models allows researchers and practitioners to understand potential problems and build strong defenses – as also showcased in our paper. Moreover, due to the studied whitebox setup, our approaches are not directly applicable for real-world malicious actors.
8 Discussion and Conclusion
We show that the transductive learning setting in prior work on adversarial training for GNNs has fundamental limitations leading to a biased evaluation and a trivial solution through memorization. Thus, we revisit adversarial training in a fully inductive setting. Furthermore, we argue that future research on evasion attacks in the graph domain, in general, should focus on the inductive setting to avoid the conceptual limitations inherent to combining transductive learning with test-time attacks. Moreover, we employ more flexible GNNs than before that achieve substantial robustness gains through adversarial training and are interpretable. For more realistic perturbations, we develop LR-BCD - the first global attack able to maintain local constraints.
This research was supported by the Helmholtz Association under the joint research school "Munich School for Data Science - MUDS". Furthermore, this paper has been supported by the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the German Federal Ministry of Education and Research, and the German Research Foundation, grant GU 1409/4-1.
[1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
[2] Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective. In *IJCAI*, 2019.
[3] Kaidi Xu, Sijia Liu, Pin-Yu Chen, Mengshu Sun, Caiwen Ding, Bhavya Kailkhura, and Xue Lin. Towards an efficient and general framework of robust training for graph neural networks. In ICASSP, 2020.
[4] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
[5] Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Gunnemann. Combining neural networks with personalized pagerank for classification on graphs. In ICLR, 2019.
[6] Lukas Gosch, Daniel Sturm, Simon Geisler, and Stephan Gunnemann. Revisiting robustness in graph machine learning. In ICLR, 2023.
[7] Daniel Zügner, Amir Akbarnejad, and Stephan Gunnemann. Adversarial attacks on neural networks for graph data. In KDD, 2018.
[8] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.
[9] Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien. Semi-Supervised Learning (Adaptive Computation and Machine Learning). 2006.
[10] Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In ICML, 2016.
[11] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
[12] Stephan Gunnemann. Graph neural networks: Adversarial robustness. In Graph Neural Networks: Foundations, Frontiers, and Applications. 2022.
[13] Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In ICLR, 2021.
[14] Mingguo He, Zhewei Wei, and Ji-Rong Wen. Convolutional neural networks on graphs with chebyshev approximation, revisited. In NeurIPS, 2022.
[15] Muhammet Balcilar, Guillaume Renton, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. Analyzing the expressive power of graph neural networks in a spectral perspective. In ICLR, 2021.
[16] W.W. Zachary. An information flow model for conflict and fission in small groups. Journal of Anthropological Research, 1977.
[17] Simon Geisler, Tobias Schmidt, Hakan Şirin, Daniel Zügner, Aleksandar Bojchevski, and Stephan Gunnemann. Robustness of graph neural networks at scale. In NeurIPS, 2021.
[39] Jiayan Guo, Shangyang Li, Yue Zhao, and Yan Zhang. Learning robust representation through graph adversarial contrastive learning. In Database Systems for Advanced Applications. 2022.
[40] Qinkai Zheng, Xu Zou, Yuxiao Dong, Yukuo Cen, Da Yin, Jiarong Xu, Yang Yang, and Jie Tang. Graph robustness benchmark: Benchmarking the adversarial robustness of graph machine learning. NeurIPS, 2021.
[41] Yan Scholten, Jan Schuchardt, Simon Geisler, Aleksandar Bojchevski, and Stephan Gunnemann. Randomized message-interception smoothing: Gray-box certificates for graph neural networks. In NeurIPS, 2022.
[42] Marcin Waniek, Tomasz Michalak, Talal Rahwan, and Michael Wooldridge. Hiding individuals and communities in a social network. Nature Human Behaviour, 2, 2018.
[43] Aleksandar Bojchevski and Stephan Gunnemann. Certifiable robustness to graph perturbations. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett, editors, NeurIPS, 2019.
[44] Daniel Züigner and Stephan Gümnmann. Certifiable robustness of graph convolutional networks under structure perturbations. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD, 2020.
[45] Jan Schuchardt, Aleksandar Bojchevski, Johannes Klicpera, and Stephan Gunnemann. Collective robustness certificates: Exploiting interdependence in graph neural networks. In ICLR, 2021.
[46] Simon Geisler, Daniel Zügner, and Stephan Gunnemann. Reliable graph neural networks via robust aggregation. In NeurIPS, 2020.
[47] Daniel Zügner and Stephan Gümnmann. Adversarial attacks on graph neural networks via meta learning. In ICLR, 2019.
[48] Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. Robust graph convolutional networks against adversarial attacks. In KDD, 2019.
[49] Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial attack on graph structured data. In ICML, 2018.
[50] Negin Entezari, Saba A. Al-Sayouri, Amirali Darvishzadeh, and Evangelos E. Papalexakis. All you need is low (rank): Defending against adversarial attacks on graphs. In WSDM, 2020.
[51] Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples for graph data: Deep insights into attack and defense. In *IJCAI*, 2019.
[52] Xiang Zhang and Marinka Zitnik. Gnnguard: Defending graph neural networks against adversarial attacks. In NeurIPS, 2020.
[53] Yongqiang Chen, Han Yang, Yonggang Zhang, MA KAILI, Tongliang Liu, Bo Han, and James Cheng. Understanding and improving graph injection attack by promoting unnoticeability. In ICLR, 2022.
[54] Simon Geisler, Johanna Sommer, Jan Schuchardt, Aleksandar Bojchevski, and Stephan Gunnemann. Generalization of neural combinatorial solvers through the lens of adversarial robustness. In ICLR, 2022.
A Inductive Setting
Here, we formally introduce the inductive node-classification problem and the arising saddle-point problem relevant for adversarial training in the graph domain. The formal presentation follows the setting of a growing graph, i.e., given a training graph  $\mathcal{G}$  of size  $n$  with labels  $y\in \mathcal{V}^m$ ,  $m\leq n$ , we sample a new graph  $\mathcal{G}'$  with  $k$  additional nodes from the underlying data-generating distribution  $\mathcal{D}$  conditioned on  $\mathcal{G}$ , which we denote  $(\mathcal{G}',y')\sim \mathcal{D}(\mathcal{G},y)$ , where  $y'\in \mathcal{Y}^{n + k}$  includes the (unknown) labels of the newly added nodes. The newly added nodes are collected in the set  $\mathcal{I}$ . However, as will be clear later, the given formulation is general and comprises different inductive learning settings described at the end of this section.
For inductive node classification, we are now interested in the expected  $0 / 1$ -loss on the newly added nodes, given as:
$$
\underset {\left(\mathcal {G} ^ {\prime}, y ^ {\prime}\right) \sim \mathcal {D} (\mathcal {G}, y)} {\mathbb {E}} \sum_ {i \in \mathcal {I}} \ell_ {0 / 1} \left(f _ {\theta} \left(\mathcal {G} _ {i} ^ {\prime}, y _ {i} ^ {\prime}\right)\right) \tag {A.1}
$$
Now, assuming an adversary performing a test-time (evision) attack, one can write the robust classification error as
$$
\mathcal {L} _ {a d v} \left(f _ {\theta}\right) = \underset {\left(\mathcal {G} ^ {\prime}, y ^ {\prime}\right) \sim \mathcal {D} \left(\mathcal {G}, y\right)} {\mathbb {E}} \max  _ {\tilde {\mathcal {G}} ^ {\prime} \in \mathcal {B} \left(\mathcal {G} ^ {\prime}\right)} \sum_ {i \in \mathcal {I}} \ell_ {0 / 1} \left(f _ {\theta} \left(\mathcal {G} _ {i} ^ {\prime}, y _ {i} ^ {\prime}\right)\right) \tag {A.2}
$$
with  $\mathcal{B}(\mathcal{G}')$  representing the set of possible perturbed graphs the adversary can choose from.
In adversarial training, the goal now is to solve the following saddle-point problem:
$$
\min  _ {\theta} \mathcal {L} _ {a d v} \left(f _ {\theta}\right) \tag {A.3}
$$
Because of the expectations in the losses (compared to the transductive setting defined by Equation (3) and Equation (4)) neither an optimal solution to the saddle point problem defined by Equation (A.3) nor a perfectly robust model achieving clean test accuracy as in Proposition 1 can be achieved through memorizing information from the training graph. Thus, inductive node classification represents a learning setting without the pitfalls presented in Section 2.1.
The formulation comprises the setting of an evolving graph, which does not necessarily grow but changes the already existing nodes and graph structure over time, by including the indices of the  $n$  nodes present at training in the set  $\mathcal{K}$ . Another inductive learning setting samples  $\mathcal{G}'$  as a completely new graph independent of  $\mathcal{G}$ , then the conditional sampling process reduces to  $(\mathcal{G}', y') \sim \mathcal{D}$ .
This section summarizes the experimental setup, including datasets, models, attacks, and adversarial training. The code can be found at https://www.cs.cit.tum.de/daml/ adversarial-training/.
B.1 Datsets
We perform experiments on the commonly used citation networks Cora, Cora-ML, CiteSeer, Pubmed, and OGB arXiv; as well as WikiCS, Squirrel (with removed duplicates) and (C)SBMs. We always extract the largest connected component for all datasets.
For all datasets, except OGB arXiv and the heterophilic datasets, the data is split as follows. In the transductive setting, we sample 20 nodes per class for both training and validation set. The remaining nodes constitute the test set. Similarly, in the inductive setting, we sample 20 nodes per class for both (labeled) training and validation set. Additionally, we sample a stratified test set consisting of  $10\%$  of all nodes. The remaining nodes are used as additional (unlabeled) training nodes. For OGB arXiv, we use the provided temporal split. For Squirrel, we use the version without duplicated nodes and the split both provided by Platonov et al. [27].
We parametrize the (C)SBMs following Gosch et al. [6]. In a (C)SBM, the labels  $y_{i}$  are sampled uniformly from  $\{0,1\}$ . Then, the respective node features are sampled from a  $d$ -dimensional Gaussian distribution  $\mathcal{N}((2y_i - 1)\mu ,\sigma I)$  with  $\mu = \frac{K\sigma}{2\sqrt{d}}\in \mathbb{R}^d$ . We set  $\sigma = 1$  and  $K = 1.5$ , resulting in a regime, where graph structure is important for node classification [6]. Furthermore, the dimensionality  $d$  is set to 21. The connection probabilities between same-class nodes  $p$  and different-class nodes  $q$  are set to  $p = 0.15\%$  and  $q = 0.63\%$ . (This can be understood as an "inverted" Cora fit, as the maximum likelihood fit to Cora results in  $p = 0.63\%$  and  $q = 0.15\%$ .) We use an  $80\% /20\%$  train/Validation split on the nodes.
B.2 Models
In the following, we present the hyperparameters and architectural details for the models used in this work. The experimental configuration files including all hyperparameters will be made public upon acceptance.
- GPRGNN: We fix the predictive part as a two-layer MLP with 64 hidden units (256 for OGB arXiv). The transition matrix is the symmetric normalized adjacency with self-loops  $\mathring{L} = \mathring{D}^{-1/2}\mathring{\mathbf{A}}\mathring{D}^{-1/2}$ , while the diffusion coefficients are randomly initialized. A total of  $K = 10$  diffusion steps is considered. During training, the MLP dropout is fixed at 0.2 and no dropout is applied to the adjacency. In contrast to Chien et al. [13], the diffusion coefficients are always learned with weight decay. This acts as regularization and prevents the coefficients from growing indefinitely.
- ChebNetII: Similarly, we use a two-layer MLP for the features with 64 hidden units and  $K = 10$ . Following the authors He et al. [14], we use the shifted graph Laplacian without self-loops  $L = -D^{-1/2}AD^{-1/2}$ .  $L$  then has eigenvalues in the range  $[-1,1]$  instead of  $[0,2]$ . This shift is required for the parametrization of the Chebyshev polynomial, however, does not affect eigenvectors. During training, we apply dropout of 0.8 to the features and 0.5 to  $L$  to the propagation.
- GAT: We use two layers with 64 hidden units and a single attention head. During training we apply a dropout of 0.5 to the hidden units. No neighborhood dropout is applied.
- GCN: We use a two-layer GCN with 64 hidden units. For OGB arXiv we increase to a three layer architecture with 256 hidden units. During training, a dropout of 0.5 is applied.
- APPNP: The structure of APPNP mirrors that of GPRGNN. A two-layer MLP with 64 hidden units (256 for OGB arXiv) encodes the node attributes. This is followed by generalized graph diffusion with transition matrix  $\mathring{L} = \mathring{D}^{-1/2}\mathring{\mathbf{A}}\mathring{\mathbf{D}}^{-1/2}$  and coefficient  $\gamma_{K} = (1 - \alpha)K$  and  $\gamma_{l} = \alpha (1 - \alpha)l$  for  $l < K$ . As for GPRGNN, a total of  $K = 10$  steps is done and the MLP dropout is fixed at 0.2, while no dropout is applied to the adjacency. The  $\alpha$  is set to 0.1.
- SoftMedian GDC: We follow the default configuration of Geisler et al. [17] and use a temperature of  $T = 0.2$  for the SoftMedian aggregation with 64 hidden dimensions as well as dropout of 0.5. The Personalized PageRank (PPR) diffusion coefficient is fixed to  $\alpha = 0.15$  and then a top  $k = 64$  sparsification is applied. In the attacks, the model is fully differentiable but the sparsification of the propagation matrix.
- MLP: The MLP follows GPRGNN's prediction module. It has two layers with 64 hidden units (256 for OGB arXiv) and is trained using a dropout of 0.2 on the hidden layer.
Training parameters. We perform adversarial training as described in Algorithm B.2. We train for a maximum of 3000 epochs and optimize the model parameters using ADAM with learning rate 1e-2 and weight decay 1e-3. To avoid overfitting, we apply early stopping with a patience of 200 epochs. For early stopping, we attack all validation nodes with the training attack and consecutively evaluate the loss on all validation nodes w.r.t the resulting perturbed graph. Following insights from Geisler et al. [17], the tanh-margin loss is chosen as both attack and training loss. To increase stability, the first ten epochs are performed without including adversarial examples.
Training attack parameters. During adversarial training, we run a total of 20 attack epochs without early stopping. For PGD and PR-BCD, we stick with the same learning rates as for evaluation. For LR-BCD, we multiply the learning rate by 20 to account for the lower number of epochs. This helps to obtain a sparser solution where local constraints are accounted for.
B.3 Attacks
For evaluating the robustness of GNNs, we use the following attacks and hyperparameters. Following insights from Geisler et al. [17], the tanh-margin loss is chosen as attack objective.
- PGD [3]: We optimize for 400 epochs. In epoch t, we use a learning rate of  $0.1 \cdot \Delta / \sqrt{t}$ , where  $\Delta$  is the global budget (see Section 5).
- PR-BCD [17]: We closely follow the setup from Geisler et al. [17]. We use a block size of 500.000 (3.000.000 for OGB arXiv) and perform 400 epochs. Afterwards, the best epoch's state is recovered and 100 additional epochs with decaying learning rate and without block resampling are performed. Additionally, we scale the learning rate w.r.t.  $\Delta$  and the block size, as suggested by Geisler et al. [17].
- LR-BCD: We use a block size of 500.000 (3.000.000 for OGB arXiv) and perform 400 epochs. We scale the learning rate w.r.t.  $\Delta$  and the block size, identical to PR-BCD. The local budget is always chosen as
- FGSM [17]: The FGSM attack greedily perturbs a single edge each epoch. No hyperparameters are needed.
B.4 Projection of LR-BCD
We next give the pseudo-code for our LR-BCD in Algorithm B.1. To improve performance, we only iterate edges that violate a local budget. Edges that only impact the global budget are handled separately. Additionally, we stop Algorithm B.1 early if the global budget is exhausted.
Note that the algorithm represents the implementation for undirected graphs.
Algorithm B.1 LR-BCD - Projection with local and global constraints
Input: Upper triangular perturbations  $S \in \mathbb{R}^{n \times n}$ , budgets  $\Delta \in \mathbb{N}$ ,  $\pmb{\Delta}^{(l)} \in \mathbb{N}^n$   
Output: projected gradients  $P = \Pi_{\Delta}^{(l)}(S)$ $P \gets 0$  // sparse matrix of shape  $n \times n$ $\hat{S} \gets \Pi_{[0,1]}(S)$   
idx  $\gets$  argsort(triu(S), 'desc', 'indices')  
for  $(u,v)$  in idx do  
if  $\hat{S}_{u,v} = 0$  or  $\Delta - \hat{S}_{u,v} < 0$  then Return end if  
 $\Delta_{u,v} \gets \min(\Delta, \Delta_u^{(l)}, \Delta_v^{(l)})$   
if  $\Delta_{u,v} \geq \hat{S}_{u,v}$  then  
 $P_{u,v} \gets \hat{S}_{u,v}$ $\Delta \gets \Delta - \hat{S}_{u,v}$ $\Delta_u^{(l)} \gets \Delta_u^{(l)} - \hat{S}_{u,v}$ $\Delta_v^{(l)} \gets \Delta_v^{(l)} - \hat{S}_{u,v}$   
end if  
end for
B.5 Adversarial training
In adversarial training, we solve the following objective
$$
\underset {\theta} {\arg \min } \underset {\tilde {\mathcal {G}} \in B (\mathcal {G})} {\max } \sum_ {i = 1} ^ {m} \ell \left(f _ {\theta} \left(\tilde {\mathcal {G}}\right) _ {i}, y _ {i}\right) \tag {B.1}
$$
where  $m$  is the number of labeled nodes and  $\mathcal{G}$  the clean training graph and  $\ell$  is the chosen (surrogate) loss. Since Equation (1) is a challenging (typically) non-convex bi-level optimization problem, we approximate it with alternating first-order optimization. In other words, we train  $f_{\theta}$  not on the clean graph  $\mathcal{G}$  but on adversariably perturbed graphs  $\tilde{\mathcal{G}}$  that are crafted in each iteration. This process is represented by Algorithm B.2.
Input: Training/validation graph  $\mathcal{G}_{t / v}$ , training/validation labels  $\pmb{y}_{t / v}$ , GNN  $f_{\theta_0}$ , adversary  $\mathcal{A}$ , epochs  $E$ , warm-up epochs  $W$ , loss  $\ell$ , learning rate  $\alpha$
Output: GNN  $f_{\theta^{*}}$
$\ell_{min}\gets \infty$
for  $l = 1$  to  $W$  do
$$
\theta_ {l} \leftarrow \theta_ {l - 1} + \alpha * \nabla_ {\theta_ {l - 1}} \ell \left(f _ {\theta_ {l - 1}} (\mathcal {G} _ {t}), \boldsymbol {y} _ {t}\right)
$$
for  $l = W$  to  $E$  do
if  $\ell_{min} > \ell (f_{\theta_l}(\hat{\mathcal{G}}_v),\pmb {y}_v)$  then
$$
\begin{array}{l} \hat {\mathcal {G}} _ {t} \leftarrow \mathcal {A} \left(f _ {\theta_ {t - 1}}, (\mathcal {G} _ {t}), \boldsymbol {y} _ {t}\right) \\ \theta_ {l} \leftarrow \theta_ {l - 1} + \alpha * \nabla_ {\theta_ {l - 1}} \ell \left(f _ {\theta_ {l - 1}} \left(\hat {\mathcal {G}} _ {t}\right), \boldsymbol {y} _ {t}\right) \\ \tilde {\mathcal {G}} _ {v} \leftarrow \mathcal {A} \left(f _ {\theta_ {l}}, \mathcal {G} _ {v}, \boldsymbol {y} _ {v}\right) \\ \ell_ {m i n} \leftarrow \ell \left(f _ {\theta_ {l}} \left(\hat {\mathcal {G}} _ {v}\right), \boldsymbol {y} _ {v}\right) \\ \theta^ {*} \gets \theta_ {l} \\ \end{array}
$$
B.6 Self-Training
For self-training, first pseudo-labels are generated using Algorithm B.3. Then, a new, final GNN is trained on the expanded label set.
Algorithm B.2 Adversarial Training
Algorithm B.3 Self-Training
Input: Graph  $\mathcal{G}$ , labels  $\pmb{y}^{L}$ , GNN  $f_{\theta}$ , epochs  $E$   
Output: Expanded labels  $\hat{\pmb{y}}$ $f_{\theta^{*}} \gets \text{train}(f_{\theta}, \mathcal{G}, \pmb{y}^{L})$  // train an initial GNN  
 $\pmb{y}^{U} \gets f_{\theta^{*}}(\mathcal{G})$  // predict pseudo labels for all unlabeled nodes  
 $\hat{\pmb{y}} \gets \pmb{y}^{L} \cup \pmb{y}^{U}$  // return union
C Transductive Setting
C.1 Proof of Proposition 2
Before proving Proposition 2, note that the assumption  $\mathcal{G} \in \mathcal{B}(\mathcal{G})$  in Proposition 2 is natural and usually fulfilled in practice. This is as the idea behind choosing a perturbation model  $\mathcal{B}(\mathcal{G})$  is to include all graphs, which are close under some notion to the clean graph  $\mathcal{G}$ . As a result, all common perturbation model choices [12] allow the adversary to not change the graph if it would not increase misclassification, i.e.,  $\mathcal{G} \in \mathcal{B}(\mathcal{G})$  holds. Now, we restate Proposition 2 for convenience:
Proposition 2. Assuming we solve the standard learning problem  $\theta^{*} = \arg \min_{\theta}\mathcal{L}_{0 / 1}(f_{\theta})$  and that  $\mathcal{G}\in \mathcal{B}(\mathcal{G})$ . Then,  $\tilde{f}_{\theta^*} = \mathcal{A}(f_{\theta^*})$  is an optimal solution to the saddle-point problem arising in (transductive) adversarial training, in the sense of  $\mathcal{L}_{adv}(\tilde{f}_{\theta^{*}}) = \min_{\theta}\mathcal{L}_{adv}(\tilde{f}_{\theta})\leq \min_{\theta}\mathcal{L}_{adv}(f_{\theta})$ .