Are Defenses for Graph Neural Networks Robust?
Anonymous Author(s)
AffiliationAddress email
Abstract
A cursory reading of the literature suggests that we have made a lot of progress in designing effective adversarial defenses for Graph Neural Networks (GNNs).
Yet, thestandard methodology has a serious flaw – virtually all of the defenses are evaluatedagainst non-adaptive attacks leading to overly optimistic robustness estimates.
Weperform a thorough robustness analysis of 7 of the most popular defenses spanningthe entire spectrum of strategies, i e. aimed at improving the graph, the architecture,or the training.
The results are sobering – most defenses show no or only marginalimprovement compared to an undefended baseline.
We advocate using customadaptive attacks as a gold standard and we outline the lessons we learned fromsuccessfully designing such attacks.
Moreover, our diverse collection of perturbedgraphs forms a (black-box) unit test offering a first glance at a model’s robustness
Introduction
The vision community learned a bitter lesson – we need specific carefully crafted attacks to properlyevaluate the adversarial robustness of a defense.
Consequently, adaptive attacks are considered thegold standard [39].
This was not always the case, until recently most defenses were tested only againstrelatively weak static attacks.
The turning point was Carlini & Wagner [3]’s work showing that 10methods for detecting adversarial attacks can be easily circumvented.
Shortly after Athalye et al.[1] showed that 7 out of the 9 defenses they studied can be broken since they (implicitly) rely onobfuscated gradients.
So far, this bitter lesson is completely ignored in the graph domain.
Virtually no existing work that proposes an allegedly robust Graph Neural Network (GNN) evaluatesagainst adaptive attacks, leading to overly optimistic robustness estimates.
[Figure]
distribute
To show the seriousness of this methodological flaw we categorize 45 works that propose a robust
GNN and are published at major conferences/journals.
We then choose one defense per category(usually the most highly cited).
Not surprisingly, we show that none of the assessed models are asrobust as originally advertised in their respective papers.
In Fig.
1 we summarize the results for 7 ofthe most popular defenses, spanning the entire spectrum of strategies (i.e. aimed at improving thegraph, the architecture, or the training, see Table 1).
We see that in both local and global settings, as well as for both evasion and poisoning, the adversarialaccuracy under our adaptive attacks is significantly smaller compared to the routinely used nonadaptive attacks.
Even more troubling is that many of the defenses perform worse than an undefendedbaseline (a vanilla GCN [31]).
Importantly, the 7 defenses are not cherry-picked.
We report the resultsfor each defense we assessed and selected each defence before running any experiments.
Adversarial robustness measures the local generalization capabilities of a model, i.e. sensitivityto (bounded) worst-case perturbations.
Certificates typically provide a lower bound on the actualrobustness while attacks provide an upper bound.
Since stronger attacks directly translate into tighterbounds our goal is to design the strongest attack possible.
Our adaptive attacks have perfect knowledgeof the model, the parameters, and the data, including all defensive measures.
In contrast, non-adaptiveattacks (e.g. transferred from an undefended proxy or an attack lacking knowledge about defensemeasures), only show how good is the defense at suppressing a specific subset of input perturbations.
Tramer et al.
[39] showed that even adaptive attacks can be tricky to design with many subtlechallenges.
The graph domain comes with additional challenges since graphs are typically sparseand discrete and the representation of any node depends on its neighborhood.
For this reason, wedescribe the recurring themes, the lessons learned, and our systematic methodology for designingstrong adaptive attacks for all examined models.
Additionally, we find that defenses are sometimessensitive to a common attack vector and transferring attacks can also be successful.
Thus, the diversecollection of perturbed adjacency matrices resulting from our attacks forms a (black-box) unit testthat any truly robust model should pass before we need an adaptive evaluation.
In summary:
• We survey and categorize 45 defenses published across prestigious machine learning venues.•
We design custom attacks for 7 defenses (16 %) , covering the spectrum of defense techniques.
All examined models forfeit a large fraction of previously reported robustness gains.
• We provide a transparent methodology and guidelines for designing strong adaptive attacks.
• Our collection of perturbed graphs (to be published) can serve as a robustness unit test for GNNs.
Background and preliminaries
We follow the most common setup and assume GNN [18, 31] classifiers f θ ( A , X ) that operate on asymmetric binary adjacency matrix A ∈ { 0 , 1 } n × n with node features X ∈ R n × d and node labelsy ∈ { 1 , 2 , . .
, C } n where C is the number of classes,
n is the number of nodes, and m the numberof edges.
A poisoning attack perturbs the graph (flips edges) prior to training, optimizing
[Equation]
where ℓ attack is the attacker’s loss, which is possibly different from ℓ train (see § 4).
In an evasion attack,θ ∗ is kept fixed and obtained by training on the clean graph min θ ℓ train ( f θ ( A , X ) , y ) .
In both cases,the locality constraint Φ( A ) limits the perturbation to an L 0 -ball around the clean adjacency matrixs.t.
∥
˜
A − A ∥ 0 ≤ 2∆ .
Attacks on X also exist, however, this scenario is not considered by the vastmajority of defenses. For example, only one out of the seven examined defenses also discusses featureperturbations. We refer to § D for more details on adaptive feature attacks.
Threat model.
Our attacks aim to either cause misclassification of the entire test set ( global ) or asingle node ( local ).
To obtain the strongest attack possible (i.e. tightest robustness upper bound)we use white-box attacks.
We do not constrain the attacker beyond a simple budget constraint thatenforces a maximum number of perturbed edges.
For our considerations on unnoticeability see § A.
Greedy attacks.
Attacking a GNN typically corresponds to solving a constrained discrete nonconvex optimization problem that – evident by this work – is hard to solve.
Commonly, greedy algorithms are used to approximate the solution of these optimization problems (e.g. see Nettack [62]or Metattack [61]).
These greedy approaches have the nice side-effect that an attack for a high budget∆ directly gives all attacks for budgets lower than ∆ , but they tend to be relatively weaker.
Projected Gradient Descent (PGD).
Alternatively, PGD [48] has been applied to GNNs wherethe discrete adjacency matrix is relaxed to [0 , 1] n × n during the gradient-based optimization (e.g.∇ A ℓ train ( f θ ∗ ( A , X ) , y ) ) and the resulting weighted change reflects the probability of flipping an edge.
After each gradient update, the changes are projected back such that the budget holds in expectation∥ E [ ˜ A ] − A ∥ 0 ≤ 2∆ .
Finally, multiple samples are obtained and the strongest perturbation ˜
A ischosen that obeys the budget ∆ .
The biggest caveats while applying L 0 -PGD are the relaxation gapand limited scalability (see Geisler et al.
[16] for a detailed discussion and a scalable alternative).
Evasion vs. poisoning.
Evasion can be considered the easier setting from an attack perspective sincethe model is fixed f θ ∗ .
For poisoning, on the other hand, the adjacency matrix is perturbed beforetraining (Eq. 1).
Two general strategies exist for poisoning attacks: (1) transfer a perturbed adjacencymatrix from an evasion attack [62]; or (2) attack directly by e.g. unrolling the training procedureto obtain gradients through training [61].
Xu et al.
[48] propose to solve Eq.
1 with alternatingoptimization which was shown to be even weaker than the evasion transfer (1).
Note that evasion isparticularly of interest for inductive learning and poisoning for transductive learning.
Adversarial defenses
We select the defenses s.t.
we capture the entire spectrum of methods improving robustness againststructure perturbations.
For the selection, we extend the taxonomy proposed in [19].
We selectedthe subset without cherry-picking based on the criteria elaborated below before experimentation.
Taxonomy.
The taxonomy includes the top-level categories Improving Graph (preprocessing graph),
Improving Training (e.g. adv.
training or augmentations), and Improving Architecture .
Many defensesfor structure perturbations either fall into the category of improving the graph or adaptively (down-)weighting edges through an improved architecture.
Thus, we introduce further subcategories.
Similarto [19]’s discussion, unsupervised improvement of the graph finds clues in the structure and nodefeatures, while supervised improvement incorporates gradient information from the learning objective.
Conversely, for adaptive edge weighting, we identify three prevalent approaches: rule-based (e.g. usinga statistical test or a simple metric), probabilistic (e.g. modeling a latent distribution), and robust aggregations (e.g. a median aggregation).
We assign each defense to the most appropriate category (moredetails in § B) but exclude [54, 26, 59] since these defenses are covered by their subsequent works.
Selected defenses.
To evaluate a diverse set of defenses, we select one per leaf taxon, prioritizinghighly cited defenses published at renowned venues with publicly available code.
We implementall defenses in one unified pipeline.
We present the categorization of defenses and our selection in
Table 1.
Similarly to Tramer et al.
[39], we exclude defenses in the “robust training” category (see§ C for a discussion).
The three models in the “miscellaneous” category report some improvementin robustness, but they are not explicitly designed for defense purposes so we exclude them fromour study.
Some works evaluate only against evasion [43], others only poisoning [12, 15, 53], andthe rest tackle both [58, 28, 16].
In some cases the evaluation setting is not explicitly stated.
Forcompleteness we consider each defense in all four settings (local/global and evasion/poisoning).
Next, we provide a short summary of the key ideas behind each defense (details in § E).
[Table]
Improving the graph.
Jaccard GCN [43] uses a preprocessing step to remove all edges betweennodes whose features exhibit a Jaccard similarity below a certain threshold.
This was motivated by thehomophily assumption which is violated by prior attacks that tend to insert edges between dissimilarnodes.
SVD GCN [12] replaces the adjacency matrix with a low-rank approximation prior to pluggingit into a regular GNN.
This defense was motivated by the observation that the perturbations from
Nettack [62] tend to disproportionately affect the high-frequency spectrum of the adjacency matrix.
The key idea in ProGNN [28] is to learn the graph structure by alternatingly optimizing the parametersof the GNN and the adjacency matrix (the edge weights).
The loss for the latter includes the standardcross-entropy loss, the distance to the original graph, and three other objectives designed to promotesparsity, low rank, and feature smoothness.
Improving training.
GRAND [15] relies on random feature augmentations (setting some features to0) coupled with neighbourhood augmentations
¯ X
= ( AX + AAX + · · · ) .
All (randomly) augmentedcopies of ¯ X are passed through the same MLP or GCN unifying the predictions using a custom loss.
Improving the architecture.
GNNGuard [53] filters edges in each message passing aggregationvia cosine-similarity (smoothed over layers).
In the first layer of RGCN [58] we learn a Gaussiandistribution over the feature matrix and the subsequent layers then manipulate this distribution (insteadof using point estimates).
For the loss we then sample from the resulting distribution.
In addition, ineach layer, RGCN assigns higher/lower weights to features with low/high variance.
Soft-Median-GDC[16] replaces the message passing aggregation function in GNNs (typically a weighted mean) with amore robust alternative by relaxing the median using differentiable sorting.
Common themes .
One theme shared by some defenses is to first discover some property that candiscriminate clean from adversarial edges (e.g. high vs. low feature similarity), and then propose astrategy based on that property (e.g. filter low similarity edges).
Often they analyze the edges fromonly a single attack such as Nettack [62].
The obvious pitfall of this strategy is that the attackercan easily adapt by restricting the adversarial search space to edges that will bypass the defense’s(implicit) filter.
Another theme is to add additional loss terms to promote some robustness objectives.
Similarly, the attacker can incorporate the same terms in the attack loss to negate their influence.
Methodology: How to design strong adaptive attacks
In this section, we describe our general methodology and the lessons we learned while designingadaptive attacks.
We hope these guidelines can serve as a reference for testing new defenses.
Step 1 – Understand how the defense works and categorize it.
For example, some defenses relyon preprocessing by filtering out edges that meet certain criteria (e.g. Jaccard-GCN [43]).
Othersintroduce additional losses during training (e.g. GRAND [15]), or change the architecture (e.g. RGCN[58]).
Different defenses might require different attacks or impose extra requirements on the attacks.
Step 2 – Probe for obvious weaknesses .
Some examples include: (a) transfer adversarial edges fromanother (closely related) model (see also § 6); (b) use a gradient-free (black-box) attack.
For example,in our local experiments, we use a greedy brute force attack and in each step we iterate through allpossible single edge flips and choose the one that contributes most to the attack objective.
Step 3 – Launch a gradient-based adaptive attack such as PGD.
If the defense contains nondifferentiable components (or components that obfuscate gradients [1]) then either: (a) replace themwith a differentiable counterpart, e.g. drop “hard” filtering of edges as done in the preprocessingof Soft-Median-GDC [16]; or (b) attack a proxy model that removes them, e.g. for ProGNN [28]we ignore the nuclear norm loss term during training (while of course keeping it intact for the finalevaluation of the defense).
Step 4 – Adjust the attack loss .
In previous works the attack loss is often chosen to be the same asthe training loss, i.e. maximize the cross-entropy (CE).
This is suboptimal since CE is not consistentas defined by Tramer et al.
[39] – higher loss values do not indicate a stronger attack.
Thus, we usethe consistent Carlini-Wagner (CW) loss [4] for local attacks, i.e. the logit difference between thetarget and most-likely non-target class.
However, as discussed by Geisler et al.
[16], for global attacks
CW is also suboptimal since it can “waste” budget on already misclassified nodes.
Their tanh margin(TM) loss resolves this issue.
If not indicated otherwise, we either use TM or the probability margin(PM) loss [4] – a slight variant of TM that computes the margin after the softmax (rather than before).
Step 5 – Tune the (attack) hyperparameters such as the number of PGD steps, the attack learningrate, the optimizer, etc.
For example, for meta attacks (poisoning) we observed that using the ADAMoptimizer [30] can weaken the attack and replacing it with SGD can increase the effectiveness.
Moreover, many defenses combine several distinct components – potentially introducing several lossterms.
For example, for ProGNN [28] we find it important to tune the weights for these new terms.
Remarks on poisoning attacks.
For poisoning we either: (a) transfer the edges obtained from anevasion attack; or (b) unroll the training procedure to obtain a gradient ∇ ˜ A for determining goodedges for a greedy attack.
Not surprisingly, we find that applying PGD [48] on the (unrolled) metagradients often yields even stronger attacks than the greedy Metattack [61] previously proposed.
Werefer to this attack as Meta-PGD.
In some cases, it was helpful to alter the training to obtain betterbehaved gradients ∇ ˜ A , but naturally we always report the results on the original training procedure.
Lessons learned.
We provide a detailed description of each adaptive attack and the necessary actionsto make it as strong as possible in
§ E. Here, we highlight some important recurring challengesthat should be kept in mind when designing adaptive attacks.
(1) Numerical issues, e.g. due todivision by tiny numbers can lead to weak attacks, and we typically resolve them via clamping.
In some cases we observed that for PGD attacks it is beneficial to clip the gradients to stabilizethe adversarial optimization.
For a strong attack it is essential to tune its hyperparameters.
Relaxing non-differentiable components and deactivating operations that filter edges/embeddingsbased on a threshold in order to obtain gradients for every edge is an effective strategy.
(5) If thesuccess of evasion-poisoning transfer depends on a fixed random initialization (see § J), it helps touse multiple clean auxiliary models trained with different random seeds for the PGD attack – in each
PGD step we choose one model randomly.
(6) Components that make the optimization more difficultbut barely help the defense can be safely deactivated.
(7) It is sometimes beneficial to control therandomness in the training loop of Meta-PGD.
(8) For Meta-PGD it can help to initialize the attackwith non-zero perturbations and e.g. use the perturbed graph of a different attack.
Example 1 – SVD-GCN.
To illustrate the attack process (especially step 3) we present a case studyof how we construct an adaptive attack against SVD-GCN.
Gradient-free attacks like Nettack donot work well here as they waste budget on adversarial edges which are filtered out by the lowrank approximation (LRA).
Moreover, to the demise of gradient-based attacks, the gradients of theadjacency matrix are very unstable due to the SVD and thus less useful.
Still, we start with a gradientbased attack as it is easier to adapt, specifically FGSM whose quick runtime enables rapid prototyping.
To replace the LRA with a function whose gradients are better behaved, we first decompose theperturbed adjacency matrix ˜
A =
A + δ A and, thus, only need gradients for δ A .
Next, we noticethat the eigenvectors of A usually have few large components.
Perturbations along those principaldimensions are representable by the eigenvectors, hence most likely are neither filtered out nor impactthe eigenvectors.
Knowing this, we approximate the LRA in a tractable manner by element-wisemultiplication of δ A with weights that quantify how well an edge aligns with the principal dimensions(details in § E).
In short we replace LRA( A + δ A ) with LRA( A ) + δ A ◦
Weight( A ) , which admitsuseful gradients.
This approach carries over to other attacks such as Nettack – we can incorporatethese weights into its score function to avoid selecting edges that will be filtered out.
Example 2 – ProGNN.
While we approached SVD-GCN with a theoretical insight, breaking acomposite defense like ProGNN requires engineering and tinkering.
When attacking ProGNN with
PGD and transferring the perturbations to poisoning we observe that the perturbations are onlyeffective if the model is trained with the same random seed.
This over-sensitivity can be avoided byemploying lesson (5) in § 4.
As ProGNN is very expensive to train due to its nuclear norm regularizer,we drop that term when training the set of auxiliary models without hurting attack strength.
Forunrolling the training we again drop the nuclear norm regularizer since it is non-differentiable.
Sometimes PGD does not find a state with high attack loss, which can be alleviated by randomrestarts.
As Meta-PGD optimization quickly stalls, we initialize it with a strong perturbation found by
Meta-Attack on GCN.
All of these tricks combined are necessary to successfully attack ProGNN.
Effort.
Breaking Jaccard-GCN (and SVD-GCN) required around half an hour (resp.
three days) ofwork for the initial proof of concept.
Some other defenses require various adjustments that need to bedeveloped over time, but reusing those can quickly break even challenging defenses.
It is difficult toquantify this effort, but it can be greatly accelerated by adopting our lessons learned in § 4.
In anycase, we argue that authors proposing a new defense must put in reasonable effort to break it.
Evaluation of adaptive attacks
First, we provide details on the experimental setup and used metrics.
We then report the main resultsand findings.
We provide additional results in § I, we refer to § A for the list of attacks, and § E fortheir adaptive counter parts.
We provide the code, configurations, and a collection of perturbed graphsanonymously: https://figshare.com/s/0ecabc74ad7809a84c8d .
Setup.
We use the two most widely used datasets in the literature, namely Cora ML [2] and Citeseer[17].
We repeat the experiments for five different data splits and we report the mean and the variance.
We use an internal cluster with Nvidia GTX 1080Ti GPUs.
Most experiments can be reproducedwithin a few hours.
However, the experiments with ProGNN and GRAND will likely require several GPU days.
In § N, we discuss scalability and apply an adaptive attack to arXiv (170k nodes) [21].
Defense hyperparameters.
When first attacking the defenses we observed that many exhibit poorrobustness using the hyperparameters provided by their authors.
To not accidentally dismiss a defenseas non-robust, we tune the hyperparameters such that the clean accuracy remains constant but therobustness w.r.t.
adaptive attacks is improved.
We run all experiments on the untuned defenses as wellto confirm we achieve this goal.
In the same way, we also tune the GCN model, which we use as areference to asses whether a defense has merit.
We report the configurations and comparisons in § H.
Attacks and budget.
In the global setting, we run the experiments for budgets ∆ of up to 15% of thetotal number of edges in the dataset.
Due to our (R)AUC metric, we effectively focus on only the lowerrange of evaluated budgets (see below).
We apply the attacks FGSM and PGD [48] for evasion.
Forpoisoning, we transfer the found perturbations and additionally run Metattack [61] and our Meta-PGD.
In the local setting, we first draw sets of 20 target nodes per split with degrees 1, 2, 3, 5, 8-10, and15-25 respectively (total of 120 nodes).
This enables us to study how the attacks affect different typesof nodes – lower degree nodes are often conjectured to be less robust.
We then run the experiments forrelative budgets ∆ of up to 200% of the target node’s degree.
For example, if a node has 10 neighbors,and the budget ∆ = 70% then the attacker can change up to 10 · 0 .
7 = 7 edges.
This commonly usedsetup ensures that we treat both low and high-degree nodes fairly.
We use Nettack [62], FGSM, PGD,and our greedy brute force attack for evasion.
For poisoning, we only transfer the found perturbations.
Recall that where necessary, we adapt the attacks to the defense as discussed in § 4.
In alignment with our threat model, we evaluate each found perturbation by the test set accuracyit achieves ( global ) or the number of target nodes it correctly classifies ( local ).
For each budget,we choose the strongest attack over different attack types also considering lower budgets (i.e. weenforce the curve to be monotonically decreasing).
Choosing the strongest attack per budget amongall attempts (e.g. among PGD, Metattack, Meta-PGD) gives rise to an envelope curve as seen in
Fig.
For all reported numbers the higher the value the more robust is the model (conversely, thelower the value the stronger the attack).
Finally, we generate additional attacks by transferring theperturbations supporting the envelope curve to every other defense for both poisoning and evasion(see § 6 for a breakdown).
They also contribute to the envelope curve of each defense, but in mostcases their contribution is marginal.
By non-adaptive attacks we refer to the best transfer attack fromthe untuned GCN (often Metattack) for the global setting and Nettack for the local setting, mirroringthe attacks used by most defenses.
[Figure]
EnvelopeMLPRAUC
Area Under the Curve (AUC).
An envelope curve gives us adetailed breakdown of the empirical robustness of a defense fordifferent adversarial budgets.
However, it is difficult to quantitatively compare different defenses without visually comparingtheir curves in a figure (e.g. see Fig.
4 or Fig. 5).
Therefore, inaddition to this breakdown per budget, we use the Area Underthe Curve (AUC) to quantify the relations between differentattacks and defenses using a single summary metric.
We usethe standard AUC for local attacks.
For global attacks, we need to be more careful.
We desire ametric that assigns a single number to the curve, and we requireit to: (a) be independent of the specific choice of budget ∆ ; (b)disregard high budgets where the model performs worse thanan MLP; and (c) punish defenses that achieve robustness bytrading off too much clean accuracy.
All three properties are
[Figure]
(a) Global, Poisoning
[Figure] [Figure]
RAUC (b) Global, Evasion 0 .
[Figure] [Figure]
AUC (c) Local, Pois.
[Figure]
AUC
AdaptiveattackNonadaptiveattack
[Figure]
fulfilled by the area between the envelope curve and the line signifying the accuracy of an MLP.
We call this metric Relative AUC (RAUC).
More formally, RAUC( c ) =
(cid:82) b ′
( c ( b ) − a MLP )
d b s.t.b ≶ b ′ =
⇒ c ( b ) ≷ a MLP where c ( · ) is a piecewise linear robustness per budget curve, and a MLP isthe accuracy of the MLP baseline.
We normalize RAUC s.t. 0% is the performance of an MLP and100% is the optimal score (i.e. 100% accuracy).
On Fig.
2 we explicitly show the area that is computed with RAUC.
It captures the robustness relativeto an MLP – a model that is perfectly robust w.r.t.
structure perturbations since it is oblivious to thegraph structure, at the expense of a substantially lower clean accuracy than a GNN.
We argue thatrobustness becomes less relevant when accuracy drops beyond the MLP threshold, as then, discardingthe graph entirely is a better defense.
Nonetheless, since some defenses seem to have the largest gainsfor perturbations beyond this threshold we show more severe budgets in § F for completeness.
Finding 1 – Our adaptive attacks lower robustness by 40% on average.
In Fig.
3 we comparenon-adaptive attacks, the current standard to evaluate defenses, with our adaptive attacks which wepropose as a new standard.
The achieved (R)AUC in each case drops on average by 40% (similarly for
Citeseer, see § F).
In other words, the reported robustness in the original works proposing a defenseis roughly 40% too optimistic.
We confirm a statistically significant drop ( p < 0 . 05 ) with a one-sidedt-test in 85% of all cases.
Soft-Median-GDC is one partial exception showing small gains in 3 out of4 cases.
Regardless, the gains are noticeably smaller than reported.
Considering adversarial accuracy for (small) fixed adversarial budget (Fig. 1) instead of the summary(R)AUC over all budgets tells the same story: non-adaptive attacks are too weak to be reliableindicators of robustness and adaptive attacks massively shrink the alleged robustness gains.
Finding 2 – Structural robustness of GCN is not easily improved.
In Fig.
4 (global) and Fig. 5(local) we provide a more detailed view for different adversarial budgets and different graphs.
Foreasier comparison we show the accuracy relative to the undefended GCN baseline.
We exclude SVD-GCN from Fig.
4 since it is catastrophically broken and plotting would make the other defensesillegible (accuracy <24% already for a budget of 2% on Cora ML).
Overall, the decline is substantial.
Almost half of the examined defenses perform worse than GCN and most remaining defensesneither meaningfully improve nor lower the robustness (see also Fig.
1 and Fig. 2).
GRAND and Soft-Medoid-GCN retain robustness in some settings, but the gains are smaller than reported.
[Figure] [Figure]
Finding 3 – Defense effectiveness depends on dataset.
As we can see in Fig.
4 and Fig.
5, ourability to circumvent specific defenses tends to depend on the dataset.
It appears that some defensesare more suited for different datasets.
For example, GRAND seems to be a good choice for Citeseerwhile it is not as strong on Cora ML.
The results for local attacks (Fig. 5) paint a similar picture, herewe see that Cora ML is more difficult to defend.
This points to another potentially problematic pitfall– most defenses are developed only using these two datasets as benchmarks.
Is the robustness evenworse on other graphs?
We leave this question for future work.
[Figure]
robustness accuracy trade-
, we even find models with higher clean accuracy are more robust.
Finding 4 – No trade-off between accuracy and robustness for structure perturbations.
Instead, Fig. 6 showsthat defenses with high clean accuracy also exhibit high RAUC, i.e. are more robust against our attacks.
This appears to be in contrast to the image domain [40].
However,we cannot exclude that future more powerful defensesmight manifest this trade-off in the graph domain.
Additional analysis.
During this project we generated atreasure trove of data.
We perform a more in-depth analysis of our attacks in the appendix.
First, we study hownode degree affects attacks (see § K).
For local attacks,the required budget to misclassify a node is usually proportional to the node’s degree.
Global attacks tend to beoblivious to degree and uniformly break nodes.
Next, weperform a breakdown of each defense in terms of the sensitivity to different attacks (see § I).
In short, global attacks are dominated by PGD for evasion and
Metattack/Meta-PGD for poisoning with the PM or TM loss.
For local attacks, PGD and especiallyour greedy brute-force attack are most effective.
Finally, we analyze the properties of the adversarialedges in terms of various graph statistics such as edge centrality and frequency spectra (see § L, § M).
Robustness unit test
Next we systematically study how well the attacks transfer between the defenses.
In Fig.
7 we see thatin 14 out of 16 cases the adaptive attack is the most effective strategy (see main diagonal).
However,often there is a model combination for which transfer attacks are very strong as well.
Thus, we canprobe a new defense’s robustness simply by testing against all perturbed graphs.
Recall, if there existsat least one perturbed graph that diminishes the robustness gains we can immediately conclude that adefense is not robust (in the worst-case) without the elaborate process of designing a new attack.
Motivated by the effectiveness of transfer attacks (e.g. from ProGNN [28] to Soft-Median-GDC [16]and GRAND [15]) we suggest this set of perturbed graphs to be used as a bare minimum robustnessunit test.
There is no need for adaptive attacks if a defense does not withstand this black box (ensemble)attack.
We provide this collection in the supp.
material, and we will open source it upon acceptance.
Nevertheless, we cannot stress enough that this collection does not replace a properly developedadaptive attack.
For example, if one would come up with SVD GCN and would use our collection(excluding the perturbed graphs for SVD GCN) the unit test would partially pass.
However, as wecan see in e.g. Fig.
3, SVD GCN can be broken with an – admittedly very distinct – adaptive attack.
G C N J a cc a r d - G
C N
R G C N P r o G N N G N N G u a r d
G R A N D S
o f t - M e d i a n - G D C
S V D - G C N
Transfer from
GCN
Jaccard-GCN
RGCN
ProGNN
GNNGuard
GRAND
Soft-Median-GDC
SVD-GCN
T r a n s f e r t o
[Table]
[Figure]
Poisoning
G C N J a cc a r d - G
C N
R G C N P r o G N N G N N G u a r d
G R A N D S
o f t - M e d i a n - G D C
S V D - G C N
Transfer from
[Table] [Table]
[Figure]
performance after the transfer.
With only two exceptions, adaptive attacks (diagonal) are most effective.
Related work
Excluding attacks on undefended GNNs, previous works studying adaptive attacks in the graphdomain are scarce.
The recently proposed graph robustness benchmark [57] also only studies transferattacks.
Such transfer attacks (typically from a GCN proxy, and typically Nettack or Metattack) areso common in the graph domain that their usage is often not even explicitly stated.
Often the authorsof a defense only state that they use PGD [48] (aka “topology attack”) without further explanations.
In this case, the authors most certainly refer to a PGD transfer attack on a GCN proxy.
They almostnever apply PGD to their actual defense which would yield a possibly weak but adaptive attack.
An exception where the defense authors study an adaptive attack is SVD-GCN [12].
Their attackcollects the edges flipped by Nettack in a difference matrix δ A , replaces its most significant singularvalues and vectors with those from the clean adajcency matrix A , and finally adds it to A .
Notably,this yields a dense continuous perturbed adjacency matrix.
While their SVD GCN is susceptible tothese perturbations, the results however do not appear as catastrophic as with our adaptive attacks,despite the severe violation of the threat model (see § 2).
Geisler et al.
[16] is another exception wheregradient-based greedy and PGD attacks are directly applied to their Soft-Median defense, makingthem adaptive.
Still, our attacks manage to further reduce its robustness estimate.
Discussion
We hope that the adversarial learning community for GNNs will reflect on the bitter lesson thatevaluating adversarial robustness is not trivial.
We show that on average adversarial robustnessestimates are overstated by 40%.
To ease the transition into a more reliable regime of robustnessevaluation for GNNs we share our recipe for successfully designing strong adaptive attacks.
Using adaptive (white-box) attacks is also interesting from a security perspective.
If a model successfully defends such strong attacks it is less likely to have remaining attack vectors for a real-worldadversary.
Practitioners can use our methodology to evaluate their models in hope to avoid an armsrace right with attackers.
Moreover, the white-box assumption lowers the chance that real-worldadversaries can leverage our findings, as it is unlikely that the they have perfect knowledge.
We also urge for caution since the attacks only provide an upper bound (which with our attacks is now40% tighter).
Nevertheless, we argue that the burden of proof that a defense is truly effective shouldlie with the authors proposing it.
Following our methodology the effort to design a strong adaptiveattack is reduced, so we advocate for adaptive attacks as the gold-standard for future defenses.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See §
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See §
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See §
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See § 5, § H and provided code(c)
Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes] All experiments are repeated for five random datasplits
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See beginning of §
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] See beginning of §
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]