SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lukas Gosch<sup>1*</sup>, Simon Geisler<sup>1*</sup>, Daniel Sturm<sup>1*</sup>, Bertrand Charpentier<sup>1</sup>, Daniel Zügner<sup>1,2</sup>, Stephan Gunnemann<sup>1</sup>
$^{1}$ Department of Computer Science & Munich Data Science Institute Technical University of Munich
2Microsoft Research
{1.gosch, s.geisler, da.sturm, s.guennemann}@tum.de | dzuegner@microsoft.com
Despite its success in the image domain, adversarial training did not (yet) stand out as an effective defense for Graph Neural Networks (GNNs) against graph structure perturbations. In the pursuit of fixing adversarial training (1) we show and overcome fundamental theoretical as well as practical limitations of the adopted graph learning setting in prior work; (2) we reveal that flexible GNNs based on learnable graph diffusion are able to adjust to adversarial perturbations, while the learned message passing scheme is naturally interpretable; (3) we introduce the first attack for structure perturbations that, while targeting multiple nodes at once, is capable of handling global (graph-level) as well as local (node-level) constraints. Including these contributions, we demonstrate that adversarial training is a state-of-the-art defense against adversarial structure perturbations.<sup>1</sup>
Adversarial training has weathered the test of time and stands out as one of the few effective measures against adversarial perturbations. While this is particularly true for numerical input data like images [1], it is not yet established as an effective method to defend predictions of Graph Neural Networks
(GNNs) against graph structure perturbations.
Although previous work reported some robustness gains by using adversarial training in GNNs [2, 3], closer inspection highlights two main shortcomings: (i) their learning setting leads to a biased evaluation and (ii) the studied architectures seem to struggle to learn robust representations.
Figure 1: Robust diffusion (GPRGNN) vs. GCN on (inductive) Cora-ML. We report adversarial training and standard training.
(i) Learning setting. In the previously studied transductive learning settings (see Table 3), clean validation and test nodes are known during training. Thus, perfect robustness can be achieved by memorizing
the training graph. This can lead to a false impression of robustness. Indeed, we find that the gains of the adversarial training approach from Xu et al. [2] largely stem from exploiting this flaw. Motivated by this finding, we revisit adversarial training for node classification under structure perturbations in a fully inductive setting (i.e., validation/test nodes are excluded during training). Thus, our results do not suffer from the same evaluation pitfall and pertain to a more challenging and realistic scenario.
<sup>1</sup>Project page: https://www.cs.cit.tum.de/daml/adversarial-training/
(ii) GNN architectures. The studied GNNs like GCN [4] or APPNP [5] all seem to share the same fate as they have a limited ability to adjust their message passing to counteract adversarial perturbations. Instead, we propose to use flexible message-passing schemes based on learnable diffusion. The main motivation behind this choice is the ability to approximate any spectral graph filter. Thus, adversarial training may choose the most robust filter that achieves a competitive training loss. Thereby, we significantly improve robustness compared to previous work, while yielding an interpretable message-passing scheme, and making the evaluation bias in the transductive setting (i) apparent.
More realistic perturbations. Inspecting robust diffusion indicates that previously studied perturbation sets [2, 3] are overly permissive and unrealistic. Prior work only constrained the global (graph-level) number of inserted and removed edges, despite studying node classification, where predictions are inherently local (node-level). As a result, commonly, an adversary has been allowed to add or remove a number of edges that exceeds the average node degree by 100 times or more, providing ample leeway for a complete change of many node neighborhoods through rewiring. E.g., on Cora-ML  $5\%$  edge-changes correspond to 798 changes, but the average degree is 5.68. To prevent such degenerate perturbations [6], we propose Locally constrained Randomized Block Coordinate Descent (LR-BCD), the first attack that, while targeting multiple nodes at once, is able to constrain both local perturbations per node and global ones. Even though local constraints are well-studied for attacks targeting a single node [7], surprisingly, there was no attack incorporating these for jointly attacking a set of nodes at once. Thus, LR-BCD fills an important gap in the literature, while being efficient and effective.
Addressing the aforementioned points, we substantially boost empirical and certifiable adversarial robustness up to the level of state-of-the-art defenses for GNNs. For example, in Figure 1, we show a 4-fold increased robustness over standard training (measured in decline in accuracy after an attack).
Contributions. (1) We show theoretically and empirically that in the transductive learning setting previously studied for adversarial training, one can trivially achieve perfect robustness. We show that the full inductive setting does not have this limitation and, consequently, revisit adversarial training in this setting (see Section 2). (2) By leveraging more flexible GNN architectures based on learnable diffusion, we significantly improve upon the robustness under adversarial training (see Section 3). (3) We implement more realistic adversaries for structure perturbations with a novel attack that constrains perturbations globally and locally for each node in the graph (see Section 4).
2 Learning Settings: Transductive vs. Inductive Adversarial Training
We first give background on adversarial training and self-training. Then, in Section 2.1, we discuss the transductive learning setting and its shortcomings in contrast to inductive learning in the context of robust generalization. Following previous work, we focus on node classification and assume we are given an undirected training graph  $\mathcal{G} = (\mathbf{X},\mathbf{A})$  consisting of  $n$  nodes of which  $m$  are labeled. By  $\mathbf{A}\in \{0,1\}^{n\times n}$  we denote the adjacency matrix, by  $\mathbf{X}\in \mathbb{R}^{n\times d}$  the feature matrix, and by  $y_{i}\in \{1,\dots,C\}$  the label of a node  $i$ , summarized for all nodes in the vector  $\mathbf{y}$ .
Adversarial Training. Adversarial training optimizes the parameters of a GNN  $f_{\theta}$  on an adversarially perturbed input graph with the goal to increase robustness against test-time (evasion) attacks (i.e., attacks against the test graph after training). The objective during training is
$$
\underset {\theta} {\arg \min } \underset {\tilde {\mathcal {G}} \in \mathcal {B} (\mathcal {G})} {\max } \sum_ {i = 1} ^ {m} \ell \left(f _ {\theta} \left(\tilde {\mathcal {G}}\right) _ {i}, y _ {i}\right) \tag {1}
$$
where  $f_{\theta}(\tilde{\mathcal{G}})_i$  is the GNN-prediction for node  $i$  based on the perturbed graph  $\tilde{\mathcal{G}}$ ,  $\ell$  is a chosen loss function, and  $\mathcal{B}(\mathcal{G})$  is the set of allowed perturbed graphs given the clean graph  $\mathcal{G}$ . As in previous work, we assume that an adversary maliciously changes the graph structure by inserting or deleting edges. Then,  $\mathcal{B}(\mathcal{G})$  can be defined by restricting the total number of malicious edge changes in the graph to  $\Delta \in \mathbb{N}_{\geq 0}$  (called a global constraint) and/or restricting the total number of edge changes in each neighborhood of a node  $i$  to  $\Delta_i^{(l)} \in \mathbb{N}_{\geq 0}$  (called local constraints, see Section 4). Solving Equation (1) is difficult in practice. Thus, we approximate it with alternating first-order optimization, i.e., we approximately solve Equation (1) by training  $f_{\theta}$  not on the clean graph  $\mathcal{G}$ , but on a perturbed one  $\tilde{\mathcal{G}}$  that is newly crafted in each iteration through attacking the current model (see Algorithm B.2).
Self-training is an established semi-supervised learning strategy [8] that leverages unlabeled nodes via pseudo-labeling and is applied by previous works on adversarial training for GNNs [2, 3]. For this, first
a model  $f_{\theta'}$  is trained regularly using the  $m$  labeled nodes, minimizing  $\sum_{i=1}^{m} \ell(f_{\theta}(\mathcal{G})_i, y_i)$ . Thereafter, a new (final) model  $f_{\theta}$  is randomly initialized and trained on all nodes, using the known labels for the training nodes and pseudo-labels generated by  $f_{\theta'}$  for the unlabeled nodes (see Algorithm B.3). Thus, if including self-training, Equation (1) changes to
$$
\underset {\theta} {\arg \min } \underset {\tilde {\mathcal {G}} \in \mathcal {B} (\mathcal {G})} {\max } \left\{\sum_ {i = 1} ^ {m} \ell \left(f _ {\theta} \left(\tilde {\mathcal {G}}\right) _ {i}, y _ {i}\right) + \sum_ {i = m + 1} ^ {n} \ell \left(f _ {\theta} \left(\tilde {\mathcal {G}}\right) _ {i}, f _ {\theta^ {\prime}} \left(\mathcal {G}\right) _ {i}\right) \right\} \tag {2}
$$
2.1 Transductive Setting
Transductive node classification is a common and well-studied graph learning problem [9, 4]. It aims to complete the node labeling of the given and partially labeled graph  $\mathcal{G}$ . More formally, the goal at test time is to accurately label the already known  $n - m$  unlabeled nodes, i.e., to achieve minimal
$$
\mathcal {L} _ {0 / 1} (f _ {\theta}) = \sum_ {i = m + 1} ^ {n} \ell_ {0 / 1} \left(f _ {\theta} (\mathcal {G}) _ {i}, y _ {i}\right) \tag {3}
$$
This is in contrast to an inductive setting, where at test time a new (extended) graph  $\mathcal{G}'$  (with labels  $\pmb{y}'$ ) is sampled conditioned on the training graph  $\mathcal{G}$ . Then, the goal is to optimally classify the newly sampled nodes  $\mathcal{I}$  in expectation over possible  $(\mathcal{G}', \pmb{y}')$ -pairs, i.e., to minimize  $\mathbb{E}\left[\sum_{i \in \mathcal{I}} \ell_{0/1}(f_{\theta}(\mathcal{G}')_i, y_i')\right]$ . That is, in transductive learning the  $n - m$  unlabeled nodes known during training are considered test nodes, but in an inductive setting, new unseen test nodes are sampled. For additional details on the inductive setting, see Appendix A.
Many common graph benchmark datasets such as Cora, CiteSeer, or Pubmed [10, 11], are designed for transductive learning. Naturally, this setting has been adopted as a starting point for many works on robust graph learning [7, 2, 12], including all of the adversarial training literature (see Section 6). Here, the latter is concerned with defending against test-time (evasion) attacks in transductive node classification, i.e., it is assumed that after training an adversary can select a maliciously changed graph  $\tilde{\mathcal{G}}$  out of the set of permissible perturbations  $\mathcal{B}(\mathcal{G})$ , with the goal to maximize misclassification:
$$
\mathcal {L} _ {a d v} \left(f _ {\theta}\right) = \max  _ {\tilde {\mathcal {G}} \in B (\mathcal {G})} \sum_ {i = m + 1} ^ {n} \ell_ {0 / 1} \left(f _ {\theta} \left(\tilde {\mathcal {G}}\right) _ {i}, y _ {i}\right) \tag {4}
$$
Since the adversary changes the graph at test time (i.e., the changes are not known during training), this, strictly speaking, corresponds to an inductive setting [9], where the only change considered is adversarial. Now, the goal of adversarial training is to find parameters  $\theta$  minimizing  $\mathcal{L}_{adv}(f_{\theta})$ , corresponding to the optimal (robust) classifier under attack [1].
2.1.1 Theoretical Limitations
Defending against test-time (evasion) attacks in a transductive setting comes with conceptual limitations. In the case of graph learning, the test nodes are already known at training time and the only change is adversarial. Hence, we can design a defense algorithm  $\mathcal{A}$  achieving perfect robustness without trading off accuracy through memorizing the (clean) training graph.
Formally,  $\mathcal{A}$  takes a GNN  $f_{\theta}$  as input and returns a slightly modified model  $\tilde{f}_{\theta}$  corresponding to  $f_{\theta}$  composed with a preprocessing routine (memory) that, during inference, replaces the perturbed graph  $\tilde{\mathcal{G}} = (\tilde{\mathbf{X}},\tilde{\mathbf{A}})\in \mathcal{B}(\mathcal{G})$  with the clean graph  $\mathcal{G} = (\mathbf{X},\mathbf{A})$  known from training, resulting in  $\tilde{f}_{\theta}(\tilde{\mathcal{G}}) = f_{\theta}(\mathcal{G})$ . In other words,  $\tilde{f}_{\theta}$  ignores every change to the graph including those of the adversary. Trivially, this results in the same (clean) misclassification rate  $\mathcal{L}_{0/1}(f_{\theta}) = \mathcal{L}_{0/1}(\tilde{f}_{\theta})$  because  $\tilde{f}_{\theta}$  and  $f_{\theta}$  have the same predictions on the clean graph, but also perfect robustness, in the sense of  $\mathcal{L}_{0/1}(\tilde{f}_{\theta}) = \mathcal{L}_{adv}(\tilde{f}_{\theta})$ , as the predictions are not influenced by the adversary. Thus, we can state:
Proposition 1. For transductive node classification,  $\tilde{f}_{\theta} = \mathcal{A}(f_{\theta})$  is a perfectly robust version of an arbitrary GNN  $f_{\theta}$ , in the sense of  $\mathcal{L}_{0/1}(f_{\theta}) = \mathcal{L}_{0/1}(\tilde{f}_{\theta}) = \mathcal{L}_{adv}(\tilde{f}_{\theta})$ .
However, what is probably most interesting about  $\mathcal{A}$  is that we can use it to construct an optimal solution to the otherwise difficult saddle-point problem  $\min_{\theta} \mathcal{L}_{adv}(f_{\theta}) = \min_{\theta} \max_{\tilde{\mathcal{G}} \in B(\mathcal{G})} \mathcal{L}_{0/1}(f_{\theta})$  arising in adversarial training. Formally, we state (proof see Appendix C.1):
Proposition 2. Assuming we solve the standard learning problem  $\theta^{*} = \arg \min_{\theta}\mathcal{L}_{0 / 1}(f_{\theta})$  and that  $\mathcal{G}\in \mathcal{B}(\mathcal{G})$ . Then,  $\tilde{f}_{\theta^{*}} = \mathcal{A}(f_{\theta^{*}})$  is an optimal solution to the saddle-point problem arising in (transductive) adversarial training, in the sense of  $\mathcal{L}_{adv}(\tilde{f}_{\theta^*}) = \min_\theta \mathcal{L}_{adv}(\tilde{f}_\theta)\leq \min_\theta \mathcal{L}_{adv}(f_\theta)$ .
In other words, at train time, from a GNN  $f_{\theta^*}$  minimizing the clean error, we can construct a perfectly robust classifier  $\tilde{f}_{\theta^*}$  minimizing the maximal misclassification rate under attack solely from memorizing the data available during training.
Note that in a fully inductive setting due to the expectation in the losses, neither Proposition 1 nor Proposition 2 hold, i.e., an optimal solution cannot be found through memorization (see Appendix A). Thus, inductive node classification does not suffer from the same theoretical limitations.
2.1.2 Empirical Limitations
Even though prior work did not achieve perfect robustness, we show below that the reported gains from adversarial training by Xu et al. [2] actually mostly stem from self-training, i.e., the "leakage" of information on the clean test nodes through pseudo-labeling. Then, we close the gap between theory and practice and show that perfect robustness, while preserving clean accuracy, can be achieved empirically through adversarial training – effectively solving the learning setting in prior work.
Self-training is the (main) cause for robustness in transductive learning. In Figure 2, we compare
Figure 2: Robust test-accuracy of a GCN under different training schemes on Cora. Adversarial training uses a PGD-attack (10% pert. edges). Most robustness gains are due to self-training.
Figure 3: Adv. trained GPRGNN following Xu et al. [2] achieves perfect robustness on Cora. GCN baseline uses the same adv. training setup (training budget:  $100\%$  of edges).
the (robust) test accuracies of a GCN using different training schemes:  $(i)$  normal training,  $(ii)$  adversarial training,  $(iii)$  self-training, and  $(iv)$  pairing self- and adversarial training. Indeed, Figure 2 shows that the main robustness gain actually stems from selftraining and not adversarial training. Surprisingly, adversarial training without self-training can even hurt robustness, while adversarial training paired with self-training only slightly increases robust test accuracy compared to using self-training alone. These results are independent of the used architecture, dataset, or attack strength, as we show in Appendix C.2.2.
Adversarial training causes overfitting. For transductive node classification, it is also empirically possible to achieve perfect robustness while maintaining clean test accuracy by combining self- with adversarial training and using a more flexible, learnable diffusion model (GPRGNN) as introduced in Section 3 In Figure 3, we show the test accuracy under severe perturbations for an adversially trained GCN [2] and GPRGNN, both trained using a very strong adversary and pseudo-labels derived from self-training. The performance of the adversially trained GCN reduces rapidly, while GPRGNN achieves (almost constant) perfect robustness. Since the pseudo-labels are derived from the clean graph, the clean graph is leaked in the training process. This allows GPRGNN to mimic the behavior of an MLP and overfit to the pseudo-labels, i.e., memorize the (node-feature, pseudo-label)-pairs it has seen during training. In other words, it finds a trivial solution to this learning setting, making its limitations evident similar to the theoretic solution discussed in Section 2.1.1. In Appendix C.2.1 we show that we achieve perfect robustness not only if using the PGD attack as Xu et al. [2], but also for many different attacks.
These findings question how to interpret the reported robustness gains of previous work, which all evaluate transductively and usually use self-training. Conceptually, being robust against test-time (evasion) attacks is most relevant if there can be a natural change in the existing data. Only then it is realistic to assume an adversary causing some of the change to be malicious and against which we want to be robust without losing our ability to generalize to new data. Therefore, in Section 5 we revisit adversarial training in an inductive setting, avoiding the same evaluation pitfalls. Indeed, we find that using robust diffusion (see Section 3), we are not only capable of solving transductive robustness, but learn to robustly generalize to unseen nodes far better than previously studied models.
3 Robust Diffusion: Combining Graph Diffusion with Adversarial Training
We propose to use learnable graph diffusion models able to approximate any graph filter in conjunction with adversarial training to obtain a robust diffusion. The key motivation is to use more flexible GNN architectures for adversarial training than previous studies. We not only show that a robust diffusion significantly outperforms other models used in previous work (see Section 5), but it also allows for increased interpretability as we can gain insights into the characteristics of such robustness-enhancing models from different perspectives.
In fixed message passing schemes of popular GNNs like GCN or APPNP, each layer can be interpreted as the graph convolution  $g(\Lambda) \circledast H = Vg(\Lambda)V^{\top}H$  between a fixed graph filter  $g(\Lambda)$  and the transformed node attributes are  $H = f_{\theta}(X)$  using MLP  $f_{\theta}$ . This convolution is defined w.r.t. the (diagonalized) eigenvalues  $\Lambda \in \mathbb{R}^{n\times n}$  and eigenvectors  $V \in \mathbb{R}^{n\times n}$  of the Laplacian  $L' = I - D^{-1/2}\mathbf{A}D^{-1/2}$  with diagonal degree matrix  $D$ . Following the common reparametrization, instead of  $L'$ , we use the "normalized adjacency"  $L = D^{-1/2}\mathbf{A}D^{-1/2}$  or, depending on the specific model,  $\mathring{L} = \mathring{D}^{-1/2}\mathring{\mathbf{A}}\mathring{D}^{-1/2}$  where  $\mathring{A} = A + I$  with node degrees  $\mathring{D}$ . Then, many spatial GNNs relate to the  $K$ -order polynomial approximation  $Vg(\Lambda)V^{\top}H \approx \sum_{k=0}^{K}\gamma_kL^kH$  with the  $K+1$  diffusion coefficients  $\gamma \in \mathbb{R}^{K+1}$ . Many GNNs stack multiple convolutions and add point-wise non-linearities.
Crucially, GCN's or APPNP's graph filter  $g(\Lambda)$  is fixed (up to scaling). Specifically, we obtain an MLP with  $\gamma = [1,0,\dots,0]$ . If using  $\hat{\pmb{L}}$ , a GCN corresponds to  $\gamma = [0,-1,0,\dots,0]$ , and in APPNP  $\gamma$  are the Personalized PageRank coefficients.
Robust diffusion. In contrast to prior work on robust graph learning, we do not solely use a static parametrization of  $g(\Lambda)$ . Instead, we learn the graph filter, which corresponds to training diffusion coefficients  $\gamma$ . This allows the model to adapt the graph diffusion to the adversarial perturbations seen during training, i.e., we learn a robust diffusion.
For this, the used architectures consist of two steps: (1) using an MLP to preprocess the node features, i.e.,  $\pmb{H} = f_{\theta}(\pmb{X})$  where  $\theta$  are learnable parameters; and then, (2) using a learnable diffusion computing the logits. For the learnable diffusion, we employ GPRGNN [13] that uses the previously introduced monomial basis:  $\mathrm{softmax}\big(\sum_{k=0}^{K} \gamma_k \mathring{L}^k H\big)$ . Additionally, we study Chebyshev polynomials (see ChebNetII [14]) that are of interest due to their beneficial theoretical properties. ChebNetII can be expressed as  $\mathrm{softmax}\big(\sum_{k=0}^{K} \gamma_k w_k T_k(L) H\big)$  with extra weighting factor  $w_k = 2 / K - 1 \sum_{j=0}^{K} T_k(x_j)$ . The Chebyshev basis is given via  $T_0(L) = I$ ,  $T_1(L) = L$ , and  $T_k(L) = 2 LT_{k-1}(L) - T_{k-2}(L)$ . The Chebyshev nodes  $x_j = \cos\left(\frac{j+1/2}{K+1} \pi\right)$ ,  $j = 0, \dots, K$  for  $w_k$  reduce the so-called Runge phenomenon [14]. Note, the resulting Chebyshev polynomial can be expressed in monomial basis (up to  $L$  vs.  $\mathring{L}$ ) via expanding the  $T_k(L)$  terms and collecting the powers  $L^k$ .
(a) Norm. adj. -L
(b)  $T$  (no adv. trn.)
(c)  $\pmb{T}$  (w/o local con.)
(d)  $T$  (w/ local con.)
Figure 4: Robust diffusion (GPRGNN) on Karate Club where the edge  $(u,v)$  width encodes the diffusion coefficient  $T_{u,v}$  learned during training. In (a) we show the normalized adjacency matrix. The other plots show the robust diffusion transition matrix: (b) w/o adversarial training, (c) w/ adversarial training but w/o local constraints, and (d) w/ adversarial training and w/local constraints.
Interpretability. While chaining multiple layers of a "fixed" convolution scheme (e.g. GCN) might allow for similar flexibility, with our choice of robust diffusion, we can gain insights about the learned robust representation from the  $(i)$  polynomial,  $(ii)$  spectral, and  $(iii)$  spatial perspective.
(i) Polynomial perspective. The coefficients  $\gamma_{k}$  determine the importance of the respective  $k$ -hop neighborhood for the learned representations. To visualize  $\gamma_{k}$ , we can always consider  $\gamma_{0}$  to be positive, which is equivalent to flipping the sign of the processed features  $\pmb{H}$ . Additionally, we normalize the coefficients s.t.  $\sum |\gamma| = 1$  since  $\pmb{H}$  also influences the scale. In Figure 6, we give an example for the polynomial perspective (details are discussed in Section 5).