Single-Node Attack for Fooling Graph Neural Networks
Authors: Ben Finkelshtein
Chaim Baskin
Evgenii Zheltonozhskii
Uri Alon
Venue:  (2021)
Keywords: graphs
GNN
adversarial
attack

Abstract: Graph neural networks (GNNs) have shown broad applicability in a variety of domains.
Some of these domains, such as social networks and product recommendations, are fertile ground for malicious users and behavior.
In this paper, we show that GNNs are vulnerable to the extremely limited scenario of a single-node adversarial example, where the node cannot be picked by the attacker. 
That is, an attacker can force the GNN to classify any target node to a chosen label by only slightly perturbing another single arbitrary node in the graph, even when not being able to pick that specific attacker node. When the adversary is allowed to pick a specific attacker node, the attack is even more effective. 
We show that this attack is effective across various GNN types (e.g., GraphSAGE, GCN, GAT, and GIN), across a variety of real-world datasets, and as a targeted and non-targeted attack.
Our code is available anonymously at https://github.com/gnnattack/SINGLE .

Under review as a conference paper at ICLR 2021
SINGLE-NODE ATTACK FOR FOOLING GRAPH NEURAL NETWORKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Graph neural networks (GNNs) have shown broad applicability in variety of domains. Some of these domains, such as social networks and product recommen- dations, are fertile ground for malicious users and behavior. In this paper, we show that GNNs are vulnerable to the extremely limited scenario of a single-node adver- sarial example, where the node cannot be picked by the attacker. That is, an attacker can force the GNN to classify any target node to chosen label by only slightly perturbing another single arbitrary node the graph, even when not being able pick that specific attacker node. When the adversary is allowed to pick specific attacker node, the attack is even more effective. We show that this attack is effee- tive across various GNN types (e.g.. GraphSAGE, GCN, GAT, and GIN), across a variety of real-world datasets, and as a targeted and non-targen non-targeted attack. Our code is available anonymously https: / thub. com/ /gnattgnaattiiinnennnn tack/SINGLE.
INTRODUCTION
Graph neural networks (GNNs) (Scarselli et al., 2008; Micheli, 2009) have recently shown sharply increasing popularity due their generality and computation--effccinnyy (Duvenaud al., 2015; Li et al., 2016; Kipf & Welling, 2017; Hamilton et al., 2017; Veličković et al., 2018; Xu et al., 2019b). Graph-structure data underlie plethora of domains such citation networks (Sen al., 2008), social networks (Leskovee & Mcauley, 2012; Ribeiro et al., 2017: 2018), knowledge graphs (Wang al., 2018: Trivedi et al., 2017; Schlichtkrull et al., 2018), and product recommendations (Shchur et al., 2018). Therefore, GNNs are applicable for a variety of real-world structured data.
While most work in this field has focused on improving the accuracy of GNNs and applying them to growing number of domains, only few past works have explored the vulnerability of GNNs to adversarial examples. Consider the following scenario: a malicious user joins a social network such as Twitter Facebook. The malicious user mocks the behavior of a benign user, establishes connections with other users, and submits benign posts. After some time, the user submits a new adversarially crafted post, which might seem irregular but overall benign. Since the GNN represents every user according to all the user's posts, this new post perturbs the representation of the user as seen by GNN. As result, another, specific benign user gets blocked from the network; alternatively, another malicious user submits a hateful post - but does not get blocked. This scenario is illustrated in Figure In this paper, we show the feasibility of such a troublesome scenario: a single attacker node can perturb its own representation, such that another node will be misclassified as label of the attacker's 's choice.
Most previous work on adversarial examples in GNNs required the perturbation to span multiple nodes, which in reality requires the cooperation of multiple attackers. For example, the pioneering work of Züggnr et al. (2018) perturbed set of attacker nodes; Bojchevski & Günnemann (2019) perturb edges that are covered by a set of nodes. Further and in contrast with existing work, we show that perturbing a single node is more harmful than perturbing single edge.
In this paper, we present a first a single-node adversarial attack on graph neural networks. the adversary is allowed to choose the attacker node, for example, by hacking into existing account, the efficiency of the attack significantly increases. We present two approaches for choosing the attacker: white-box gradient-based approach, and a black-box. model-free approach that relies graph topology. Finally, we perform a comprehensive experimental evaluation of our approach on multiple datasets and GNN architectures.

---

Under review as a conference paper at ICLR 2021
Allacker node
Vistim node
Altacker node a
Victim hode V
GNN: valid
GNN: invalid
vision..
vision....
not recist
(a) Before attacking: the victim node (v) classified valid.
After attacking: the victim node (e) is classified invalid.
Figure 1: An partial adversarial example from the test set of the Twitter dataset. An adversarially- crafted post perturbs the representation of the attacker node. This perturbation causes a misclassifica- tion of the target victim node, although they are not direct neighbors.
PRELIMINARIES
Let G = (G:) be set of graphs. Each graph G (V, E, X) has set of nodes V and set of edges & C x V, where (u, )) E & denotes an edge from a node u E V to a node E V. X RRXD
matrix of D-dimensional node features. The i-th row of X is the feature vector of the node E V and is denoted as I. = Xt;: E IR.
Graph neural networks GNNs operate by iteratively propagating neural messages between neigh- boring nodes. Every GNN layer updates the representation of every node by aggregating its current representation with the current representations of its neighbors.
Formally, each node is associated with an initial representation x = h E R... This repre- sentation is considered the given features of the node. Then, a GNN layer updates each node's representation given its neighbors, yielding (2 E Rdd for every v E V. In general, the f-th layer of GNN is function that updates node's 's representation by combining with its neighbors:
h
COMBINE (N(-1), (0,-))((K
(h (6-1)
E N.);;
(1)
where N₁, is the set direct neighbors o: .V,, (u E V (u, E
The COMBINE function is what mostly distinguishes GNN types. For example, graph convolutional networks (GCN) (Kipf & Welling, 2017) deñine layer as:
h ReLU (EnEN
W wOnk!
(2)
Cu.
where Ca.a normalization factor usually set to VN, NNa. After E such aggregation itera- tions, every node representation captures information from all nodes within its ê-hop neighborhood. The total number layers is usually determined empirically as a hyperparameter. In the node classification scenario, use the final representation h to classify
For brevity, we focus our definitions on the semi-supervised transductive node classification goal, where the dataset contains single graph G, and the split into training and test sets is across nodes in the same graph. Nonetheless, these definitions can be trivially generalized to the inductive setting, where the dataset contains multiple graphs, the split into training and test sets is between graphs, and the test nodes are unseen during training.
We associate each node E V with class ye y (1,..., Y). The labels of the training nodes are given during training; the test nodes are seen during training without their labels. The training subset represented as = (G, ((v,, y:) ) Given the training set, the goal is to learn a model So (G, V) + y that will classify the of the nodes correctly. During training, the model fo thus minimizes the loss over the given labels, using J (-, -), which typically is the cross-entropy loss:
Ene
No J (fo (G, vz) v1))
8* a (fo. D) argming ND
(3)
2

---

Under review as a conference paper at ICLR 2021
3 SINGLE-NODE GNN ATTACK
In this section, we describe our Single-node INdirect Gradient adversarial Evasion (SINGLE) attack. While our attack is simple, is the first attack that focuses perturbing nodes (in contrast to edges (Dai et al., 2018)), which works with an arbitrary single attacker node (in contrast to multiple nodes (Zügner et al., 2018)) that is not the node under attack (in contrast "direct" attacks where the attacker perturbs the node under attack directly (Zügner et al., 2018; Li et al., 2020)).
3. PROBLEM DEFINITION
Given a graph G. a trained model fo, a "victim" node y from the test set along with its classification by the model U. = fo (G, v), assume that adversary controls another node in the graph. The goal of the adversary is to modify its own feature vector D. by adding a perturbation vector vy E R of its choice, such that the model's classification of o will change.
We denote by Gæ. 1n the graph G where the row of X that corresponds to the node a was added with the vector 77. non-tonngeen non-targeted attack, the goal of the attacker is to find perturbation vector r that will change the classification to any other class, i.c.. fe (Gæ, tn. v) & fe (G, v). In a targeted attack, the adversary chooses a specific label Yadv E y and the adversary's goal is to force fo (Ga.+ v) = Vadv-
Generally, the classification of node v depends only nodes whose distance to in the graph is lower than or equal L the number of GNN layers. Thus, modification of the features of will affect the classification of only if the distance between and is lower than equal L. Otherwise, will not be contained the receptive field of e, and the attack will result "under-reaching" (Alon & Yahav, 2020)- any perturbation of will not affect the prediction of v (Barceló et al., 2020). Therefore, we require that distanceg (a, v) S L.
In this work, we focus gradient-based attacks. These kinds of attacks assume that the attacker can access a similar model to the model under attack and compute gradients. As recently shown by Wallace et al. (2020), this is reasonable assumption: an attacker can query the original model; using these queries, imitate the model under attack by training an imitation model; find adversarial examples using the imitation model; and transfer these adversarial examples back to the original model. Under this assumption, these attacks are general and are applicable to any GNN and dataset.
3.2 CHALLENGES
Unnoticeable Perturbations. Our first challenge is to find an adversarial example that will allow an imperceptible perturbation of the input. This objective is attainable in continuous domains such as images (Szegedy et al., 2013; Goodfellow et al., 2014) and audio (Carlini & Wagner, 2018) 8) if constrain loc -nrm of the perturbation vector n7. It is, however, unclear what imperceptibility means in graphs. In most GNN datasets, a node's features are a bag-of-words representation of the words that are associated with the node. For example, in Cora (McCallum al., 2000; Sen al., 2008), every node is annotated by many-hot feature vector of words that appear in the paper; in PubMed (Namata et al., 2012), node vectors are TF-IDF word frequencies; in Twitter (Ribeiro et al., 2017), node features are averages of GloVe embeddings, which can be viewed as word frequency vectors multiplied by (frozen) embedding matrix. We argue that an attack would unnoticeable in an academic paper or in set of Tweets if the frequency of some words is slightly modified. For example, a particular word may be repeated few times throughout the text or remain unused.
To constrain the 7 vector, we require that || < 6oo the maximal absolute value of the elements in the perturbation vector - is bounded by 600 E IR+
Perturbing nodes instead of edges. Previous work mostly focused on perturbing graph edges. Zügner et al. (2018) perturb both edges and node features, but conclude that "perturbations in the structure lead to a stronger change in the surrogate loss compared to feature attacks": Wu et al. (2019b) also conclude that "perturbing edges is more effective than modifying the features".. In this paper, counter these conclusions and show that small node feature perturbations removing all the edges of a particular node is a special case of node feature perturbation. There exists perturbation n such that W (æ,, 17) i.e., the modified feature vector F. 17 is in the null
stronger: (i) first,
3

---

Under review as a conference paper at ICLR 2021
space of the first GNN layer'' Such feature perturbation is equivalent to removing all the edges of the node a. (ii) Second, we argue that perturbing the graph structure is not realistic, because a single attacker controls only its own edges, and cannot control the global graph structure in previous work (Dai et al., 2018; Bojchevski & Günnemann, 2019b; Zhang & Zitnik, 2020). (iii) Finally, when successful attack is caused by removing edges, it is unclear whether the misclassification is caused by sensitivity to non-robust features in the data (Ilyas et al., 2019), or simply due to smaller amount of information. Similarly, when a successful attack is caused by inserting edges, is unclear whether this is simply due to incorrect or unrealistic added information.
3.3 FINDING THE PERTURBATION VECTOR
To find the perturbation, we iteratively differentiate the desired loss of with respect the perturba- tion vector v). update 77 according to the gradient, and add it to the feature vector. In non-targeted attacks, we take the positive gradient of the loss of the undesired label to increase the loss; in targeted attacks, we take the negative gradient of the loss of the adversarial label Yadv:
17 + VV,' (fo (G " ;.)
non-targeted attack
++
(4)
17 -V, (Jo (Ga.-n',,)) v) Madu) targeted attack
where y E I is learning rate. We repeat this process for a predefined number of K iterations, or until the model predicts the desired label.
Enforcing the constraints. We treat the node features as continuous throughout the attack iterations, whether they are discrete or continuous. Once the attack succeeds, we try to reset to zero as many perturbation vector elements as possible. We sort the perturbation vector elements in a decreasing order, according to their absolute value: i1,....p. iD. We start with the index n whose absolute value is the largest, 7 and reset the rest of the (i2...., i elements to zero. We then check whether perturbing only the i index is sufficient. If the attack succeeds, we stop. If the attack fails (because of the large number of perturbation vector elements set to zero), we continue perturbing the rest of the elements of 17. In the worst case, we perturb all D vector elements of 17. In most cases, we stop much earlier, practically perturbing only small fraction of the vector elements. If the original node features are discrete, we discretized features after the optimization.
Differentiate by frequencies, not by embeddings. When taking the gradient with respect to the perturbation vector Vn, there is a subtle, but crucial, difference between the way that node rep- resentations are given in the dataset: (a) indicative datasets provide initial node representations x x= [ Ø2,-.. that are word indicator vectors (many-hot) or frequencies such as (weighted) bag- of-words (Sen et al., 2008; Shchur al., 2018); (b) in encoded datasets. initial node representations are given encoded, e.g., as an average of word2vec vectors (Hamilton et al., 2017: Hu et al., 2020). Indicative datasets can be converted to encoded by multiplying every vector by an embedding matrix; encoded datasets cannot be converted to indicative, without the authors releasing the textual data that was used to create the encoded dataset.
In indicative datasets, perturbation of node vector can be realized as perturbation of the original text from which the indicative vector was derived. That is, adding or removing words in the text can result in the perturhed node vector. In contrast, a few-indices perturbation in encoded datasets might be an effective attack, but will not be realistic because there is no perturbation of the original text that will result in that perturbation of the vector. That is, when perturbing nodes, it is crucial to use indicative datasets, or convert encoded datasets to the indicative representation from which they were derived (as we do in Section 4) using their original text.
4 EVALUATION
We evaluate and analyze the effectiveness of our SINGLE attack. In Section 4.1, we show that SINGLE is more effective than alternatives such as single-edge attacks. In Section 4.2, we show that we are allowed to choose the attacker node, SINGLE is significantly more effective.
Setup. Our implementation is based PyTorch Geometric (Fey & Lenssen, 2019) and its provided datasets. We trained each GNN type with two layers (L = 2), using the Adam optimizer, early
This equation demonstrates GCN, but similar equations hold for other GNN types like GAT and GIN.
4

---

Under review as a conference paper at ICLR 2021
Cora
CiteSeer
PubMed
Twitter
Clean (no attack)
80.5 +
68.5 0.7
89.1 0.
EdgeGrad
65.1 = 1.3
48.15+00
59.7
82.7
SINGLE
6.11
3.6
0..
7221
SINGLE-hope 6.
45.1
48.7
77
Table I: Test accuracy (lower is better) under different types of attacks, when the attacker node is chosen randomly. Performed using GCN, Eoo = 1 for the discrete datasets (Cora and CiteSeer), and Ex. 0.1 for the continuous datasets (PubMed and Twitter).
stopped according to the validation set, and applied a dropout of 0.5 between layers. We used up to K 20 attack iterations. All experiments in this section were performed with GCN, except for Section 4.5, where additional GNN types (GAT, GIN, and GraphSAGE) are shown. In Appendix A.2, we show consistent results across additional GNN types: GAT (Veličković al., 2018), GIN (Xu et al., 2019b), GraphSAGE (Hamilton et al., 2017), SGC (Wu et al., 2019a), and RobustGCN (Zügner Günnemann, 2019).
Data. We used Cora and CiteSeer (Sen et al., 2008) which are discrete datasets, i.c., the given node feature vectors are many-hot vectors. Thus, we set foc = the minimal possible perturbation. We also used PubMed (Sen et al., 2008) and the Taitter-Hatefull Hateful-aaelll (Ribeiro et al., 2017) datasets, which are continuous, and node features represent frequencies of words. Continuous datasets allow much more subtle perturbation, and we set E 0.1. An analysis of these values is presented in Section 4.5.
The Twitter-Hateful- Hateful-User dataset is originally provided an average of GloVe vectors (Pennington et al., 2014). We reconstructed this dataset using the original text from Ribeiro et al. (2017), to be able to compute gradients with respect to the weighted histogram of words, rather than the embeddings. We took the most frequent 10,000 words as node features, and used GloVe-Twiie Twitter embeddings multiply by the node features. We thus converted this dataset to indicative rather than encoded. Statistics of all dataset are provided in the supplementary material.
encoded dataset, where every node is
Baselines. In SINGLE (Section 3.3) the attacker node is selected randomly for each victim node, and the attack perturbs this node's 's features according to Eoo. SINGLE-hops is a modification of SINGLE where the attacker node is sampled only among nodes that are not neighbors, i.e., the attacker and the victim are not directly connected ((a, v) & E). We compare to additional approaches from the literature: EdgeGrad follows most previous work (Xu al., 2019a: Li al., 2020; Zügner & Günnemann. 2020): EdgeGrad randomly samples an attacker node as in SINGLE, and either inserts or removes a single edge from or to the attacker node, according to the gradient. If both use a randomly selected attacker node, EdgeGrad is strictly stronger than the GrudArgmax attack of Dai et al. (2018), which only removes edges. We ran each approach 5 times with different random seeds for each dataset, and report the mean and standard deviation.
4.1 MAIN RESULTS
Table 1 shows our main results for non-targeted attacks across various datasets. As shown, SINGLE is more effective than EdgeGrad across all datasets. SINGLEE-hops, which is more unnoticeable than attacking with neighbor node, performs almost as good as SINGLE which attacks using a non-neighboring node, and better than EdgeGrad. On Twitter, SINGLE reduces the test accuracy significantly better than EdgeGrad: 72.1 % compared to 82.7%. Results for targeted attacks are shown in Appendix A.3.
Surprisingly, Table A.5 shows that Robust GCN (Zügner & Günnemann, 2019) is as vulnerable to the SINGLE attack as a standard GCN, showing that there still much room for novel ideas and improvements to the robustness of current GNNs.
As we explain Section 3.3, SINGLE tries find perturbation vector which the number of perturbed elements is minimal. We measured the number of vector elements that the attack had
This can be implemented easily using edge weights: training the GNN with weights of 1 for existing edges, adding all possible edges with weights of O, and taking the gradient with respect the vector of weights.
5

---

Under review as a conference paper at ICLR 2021
Cora
CiteSeer
PubMed
Twitter
GlobalEdgeGrad
29.7 2.4
11.9 0.8
15.3
82.7 2..7000
GinhhicGraCChoce
11.0
19.0
7.0 8.5
SINGLE+Topology
31.
18.1
5.2
6.6
Table 2: Test accuracy when the adversary can choose the attacker node.
perturbed in practice. In PubMed, SINGLE used 76 vector elements on average, which are 15% of the elements in the feature vector. Cora, SINGLE perturbed 717 elements on average, which are 50%. In CiteSeer, SINGLE used 1165 attributes on average, which are 31% of the features. In Twitter, SINGLE used 892 attributes on average, which are 9% of the features. In the experiments shown in Table 1, we used 6oo 0.1 in the continuous datasets (PubMed and Twitter). If we allow larger values of Co.. we can reduce the number of perturbed vector elements: using 6o = 0.5 requires perturbing only 3% of the attributes
average achieve the same effectiveness; using <s 1 requires perturbing only 1.6% of the attributes on average to achieve the same effectiveness (in PubMed, where varying 6oo is meaningful).
4.2 ATTACKER CHOICE
the attacker could choose its node, e.g., by hijacking an existing account in social network, could they increase the effectiveness of the attack? We examine the effectiveness of two approaches for choosing the attacker node.
Gradient Attacker Choice (GradChoice) chooses the attacker node according to the largest gradient with respect to the node representations (for a non-targeted attack): a* eevll (fo (G, v) Bo)lloo The chosen attacker node is never the victim node itself.
Topological Attucker Choice (Topology) chooses the attacker node according to topological properties of the graph. As example, we choose the neighbor of the victim node with the smallest number of neighbors: a" = argmin,eN,, [Wa)- The advantage of this approach that the attacker choice is model-free: if the attacker cannot compute gradients, they at least choose the most harmful attacker node, and then perform the perturhation itself using other non-gradient approaches such as proposed by Waniek al. (2018) and Chang al. (2020).
To perform a fair comparison, we compare these approaches with GlobalEdgeGrad, which is similar EdgeGrad that can insert or remove edge, with the difference that the chosen edge can be chosen from the entire graph.
Results. Results for these attacker choice approaches are shown in Table 2. The main results are that choosing the attacker node significantly increases the effectiveness of the SINGLE attack: for example, in Twitter, from 72.1 (Table to 6.6% accuracy (Table 2).
In datasets where the given initial node features are continuous (PubMed and Twitter), SIN- GLE+Topology and SINGLE+GradChoie show similar results: on Twitter accuracy difference is less than 0.5%; on PubMed SINGLE+Topology outperforms SINGLE+GradChoice by ~33,, even though SINGLE+Topology is model-free free.. Both of those attacks are more efficient than GlobalEdge- Grad, showing the superiority of node perturbation over edge perturbation in the global view. In Appendix A.4, we show that allowing GlobalEdgeGradd to insert and remove multiple edges that belong to the same attacker node does not lead to significant improvement.
Interestingly, GradChoice and Topology agree on the choice of attacker node for 50.3% of the nodes in Cora, 78.7% of the nodes in CiteSeer, 51.0% of the nodes in PubMed, and on 55.0% of the nodes in Twitter, showing that the node selection can sometimes be performed model-free.
In datasets where the initial node features are discrete (Cora and CiteSeer), i.e., many-hot vectors, GlobalEdgeGrad reduces the test accuracy more than GradChoice and Topology. We believe that the reason is the difficulty of two-step optimization in discrete datasets: for example, GradChoice needs to choose the node, and find the perturbation afterwards. Finding a perturbation for a discrete vector more difficult than in continuous datasets, and the choice of the attacker node may not be optimal.

---

Under review as a conference paper at ICLR 2021
Cora
CiteSeer
PubMed
Twitter
SINGLE-two altuckery
7.1
0.2
27.7
SINGLE-direc
25
13.8
0.3
$7.6
SINGLE
60.1 #.1
18.1
45.5
72.1
Table 3: Scenario ablation: test accuracy under different attacking scenarios.
Standard training
Adversarial training
Clean (no attack)
78.5 0.6
76.9 = 0.6
SINGLE SINGLE- hops
45.5
58.5 2.7
48.
62.1
SINGLE+GradChoice
8.5
30.6 6.8
SINGLE+Topology
5.2
21.1 2.1
SINGLE-two attackers
27.7 1.2
40.7 :: 3.4
SINGLE-dire
0.3 -
4.6 L.1
Table 4: Test accuracy while attacking a model that was adversarially trained on PubMed, with different types of attacks.
4.3 SCENARIO ABLATION
The main scenario that we focus on in this paper is SINGLE approach that always perturbs single node, which is not the victim node (a & v). We now examine our SINGLE attack in other, easier but less realistic, scenarios: SINGLE-two attackers follows Zügner et al. (2018) and Zang al. (2020), randomly samples two attacker nodes and perturbs their features using the same approach as SINGLE. SINGLE-ddirect perturbs the victim node directly (i.e., v), approach that found to be the most efficient by Zügner et al. (2018). Table shows the test accuracy of these ablations. In Appendix A.5.3, additionally experiment with more than two attacker nodes.
4.4 ADVERSARIAL TRAINING
In the previous sections, we studied the effectiveness of the SINGLE attack. In this section, we investigate what extent can adversarial training (Madry et al., 2018) defend against SINGLE. For each training step and labeled training node, we perform Kuain adversarial steps to adversarially perturb another randomly sampled node, exactly as SINGLE, but training time. The model is then trained to minimize the original cross-entropy loss and the adversarial loss:
C(Je, D) - E No
2i (Jo (G) yi) + J (fo (Gaa, ;))))))
(5)
2ND
The main difference from Equation (3) is the adversarial term J (fe (G.,, on ex) Vz), where a, is the randomly sampled attacker for the node us In every training step, randomly sample new attacker for each victim node and compute new vectors. After the model is trained, we attack the model with Kass SINGLE adversarial steps. This is similar Feng al. (2019) and Deng et al. (2019), except that they used adversarial training as a regularizer, to improve the accuracy of a model while under attack. contrast,
use adversarial training defend a model against an attack at test time. We used Kurin = 5. as we found it to be the maximal value for which the model's accuracy is not significantly hurt while under attack ("clean"), and Kist 20 as in the previous experiments. As shown in Table 4, adversarial training indeed improves the model's robustness against the different SINGLE attacks. However, the main result of this section that SINGLE, SINGLE+GradCII and SINGLE+Topolee
still very effective attacks, as they succeed in attacking the adversarially trained model, reducing its test accuracy to 58.5%, 30.6% and 21.1%, respectively.
4.5 SENSITIVITY TO 60c
How does the intensity of the adversarial perturbation affect the performance of the attack? Intuitively, we say that the less we restrict the perturbation (i.c., larger values of cxc), the more powerful the attack. We examine whether this holds in practice.
7

---

Under review as a conference paper at ICLR 2021
80
80
70
60
60
50
Acc
40
Acc
40
30
20
20
PubMed
Cora
10
CiteSeer
0 0.1 0..10.2.02 0. 0.0 0.6 7...60.770 7..70.80.9. 0.8 6ec
3
5
7
distance (a, v)
GCN + GAT - GIN
GraphSage
Figure 2: Effectiveness of the attack compared to the allowed Coo (performed on PubMed, be- cause its features are continuous).
Figure 3: Test accuracy compared the dis- tance between the attacker and the victim, when the GCN was trained with L 8 on PubMed.
In our experiments in Sections 4. to 4.4, we used 6oc = 0.1 for the continuous datasets (PubMed and Twitter). In this section, we vary the value of 6ee across different GNN types and observe the effectiveness of the attack. Figure 2 shows the results on PubMed. We used this dataset because it larger than Cora and CiteSeer (Appendix A.1), and most importantly, its features are continuous, thus real-valued perturbations are feasible. As shown in Figure 2, the most significant difference is between performing the perturbation (foo 00 and not attacking at (< = 0). As we increase the value of eco, GCN and GraphSage (Hamilton al., 2017) show natural descent in test accuracy. Contrarily, GAT (Veličković et al., 2018) and GIN (Xu et al., 2019b) are more robust to increased absolute values of perturbations, while GAT is also the most robust compared to the other GNN types.
4.6 DISTANCE BETWEEN ATTACKER AND VICTIM
In Section 4.1, we found that SINGLE performs similarly to SINGLE-hops, although SINGLE-hops samples an attacker node whose distance from the victim node v is at least 2. We further question whether the effectiveness of the attack depend on the distance in the graph between the attacker and the victim. We trained a new model for each dataset using L = 8 layers. Then, for each test victim node, we sampled attackers according to their distance to the test node.
As shown in Figure 3, the effectiveness of the attack increases the distance between the attacker and the victim decreases. At distance of 5, the curve seems to saturate. A possible explanation for this is that apparently more than few layers (e.g., L 2 in Kipf & Welling (2017)) not needed in most datasets. Thus, the rest of the layers can theoretically learn not to pass much of their input starting from the redundant layers, excluding adversarial signals as well.
5 RELATED WORK
Works on adversarial attacks on GNN differ in several main aspects. In this section, we discuss the main criteria, to clarify the settings that we address.
Single vs. multiple attackers All previous works allowed perturbing multiple nodes, or edges that are covered by multiple nodes: Zügner et al. (2018) perturb features set of attacker nodes; Zang et al. (2020) assume "a few bad actors"; other works perturb edges that in realistic settings their perturbation would require controlling multiple nodes (Bojchevski & Günnemann, 2019 9a; Sun et al., 2020: Chen et al., 2018).
Node vs. edge perturbations Most adversarial attacks on GNNs perturb the input graph by modifying the graph structure (Zügner& & Günnemann, 2019; Wang et al., 2020; Xu et al., 2019a). For example, Dai et al. (2018) iteratively remove edges, yet their attack manages to reduce the accuracy by about 10% at most when perturbing single edge. Li al. (2020) also allow the insertion of edges; Waniek
8

---

Under review as a conference paper at ICLR 2021
et al. (2018) and Chang et al. (2020) allow insertion and deletion of edges, using attacks that based on correlations and eigenvalues, and not on gradients. Yefet et al. (2019) perturb one-hot node vectors, in the restricted domain of computer programs. Zügner et al. (2018) and Wu et al. (2019b) perturb both edges and nodes; but they concluded that perturbing edges more effective than perturbing nodes. In this work, we counter these conclusions and show that perturbing node features is more effective than perturbing edges.
Direct influence attacks Another difference between prior works lies in the difference between direct attacks and influence attacks. In direct attacks, the attacker perturbs the target node itself. For example, the attack of Zügner et al. (2018) is the most effective when the attacker and the target are the same node. In influence attacks, the perturbed nodes are at least one hop away from the victim node. In this paper, we show that the strong direct assumption is not required (SINGLE-dived in Section 4.2), and that our attack is effective when the attacker and the target are not even direct neighbors, i.e., they are at least two hops away (SINGLE-hopp Section 4.1).
Poisoning vs. evasion attacks In a related scenario, some work (Zügner & Günnemann, 2019; Bojchevski Günnemann, 2019a; Li et al., 2020; Zhang & Zitnik, 2020) focuses on poisoning attacks that perturb examples before training. Contrarily, we focus on the standard evasion scenario of adversarial examples in neural networks (Szegedy et al., 2013; Goodfellow al., 2014), where the attack operates at test time, after the model was trained, Dai et al. (2018).
Attacking vs. certifying Ziigner Günnemann (2020) foeus on certifying the robustness of GNNs against adversarial perturbations; and Bojchevski & Günnemann (2019b) certified PageRank-style models. In contrast, we study the effectiveness of the adversarial attack itself.
6 CONCLUSION
We demonstrate that GNNs are susceptible even to the extremely limited scenario of single-node indirect adversarial example (SINGLE). The practical consequences of these findings are that single attacker a network can force a GNN to classify any other target node as the attacker's 's chosen label, by slightly perturbing some of the attacker's ' features. We further show that if the attacker can choose its attacker node the effectiveness the attack increases significantly. We study the effectiveness of these attacks across various GNN types and datasets.
We believe that this work will drive research in this field toward exploring novel defense approaches for GNNs. Such defenses can be crucial for real-world systems that modeled using GNNs. Fur- thermore, we believe that the surprising results of this work motivate better theoretical understanding of the expressiveness and generalization of GNNs. To these ends, make all our code and trained models publicly available.
REFERENCES
Uri Alon and Eran Yahav, On the bottleneck of graph neural networks and its practical implications. arXiv preprint wrXi::2006.05205. 2020.
Pablo Barceló, Egor V. Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan Pablo Silva.
The logical expressiveness of graph neural networks. In International Conference on Learning Representations, 2020. URL https: //tp////ppeeeeeeeeeeeeeeeeeeeeee net wwwoooooomniiooooooooerreeeereee
Aleksandar Bojchevski and Stephan Günnemann. Adversarial attacks on node embeddings via graph poisoning. In International Conference on Machine Learning, pp. 695-704, 2019a.
Aleksandar Bojehevski and Stephan Günnemann. Certifiable robustness to graph perturbations. In Advances in Neural Infommtion Processing Systems, pp. 8319-8330, 8330, 2019b.
Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech-to-text. In 2018 IEEE Security and Privacy Workshops (SPW), pp. 1-7. IEEE, 2018
Heng Chang, Yu Rong, Tingyang Xu, Wenbing Huang. Honglei Zhang, Peng Cui, Wenwu Zhu, and Junzhou Huang. A restricted black-box adversarial framework towards attacking graph embedding models. In AAAI, pp. 3389-3396, 2020.

---

Under review as a conference paper at ICLR 2021
Jinyin Chen, Yangyang Wu, Xuanheng Xu, Yixian Chen, Haibin Zheng, and Qi Xuan. Fast gradient attack on network embedding. arXiv preprint Stt.::889902797,, 2018.
Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial attack on
graph structured data. In International Conference on Machine Learning, pp. 1115-1124, 2018.
Zhijie Deng, Yinpeng Dong, and Jun Zhu. Latent adversarial training of graph convolution networks. In ICML Workshop on Learning and Reasoning with Graph-Structuree Representations, 2019. URL https:/ grwphraaso....ccc.. github. ho/papers/ pdf.
David K. Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. Convolutional networks graphs for learning molecular fingerprints. In Advances in neural information processing systems, pp. 2224-2232, 2015.
Fuli Feng, Xiangnan He, Jie Tang, and Tat-Seng Chua. Graph adversarial training: Dynamically regularizing based on graph structure. IEEE Transactions on Knowledge and Data Engineering. 2019.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Will Hamilton, Zhitao Ying, and Jure Leskovee. Inductive representation learning on large graphs. In Advances in neural information processing systems, pp. 1024-1034, 2017.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. arXiv preprint arXiv:2005.00663 2020.
Andrew Ilyss, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems, pp. 125-136, 2019.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks In International Conference Learning Representations, 2017. URL batttp::/oppenaee net/ / forum? i SJSSUUAyY
Jure Leskovec and Julian J. Mcauley. Learning to discover social circles in ego networks. In Advances in neural information processing systems, pp. 539-547, 2012.
Jintang Li, Tau Xie, Liang Chen, Fentang Xie, Xiangnan He, and Zibin Zheng. Adversarial attack on large scale graph. arXiv preprint htth:22009.044883 2020.
Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. In International Conference on Learning Representations, 2016.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction internet portals with machine learning. Information Retrieval, 3(2):127-163, 2000.
Alessio Micheli. Neural network for graphs: A contextual constructive approach. IEEE Transactions on Neural Networks, 20(3):498-511 2009.
Galileo Mark Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven active surveying for collective classification. In Workshop on Mining and Learning with Graphs, 2012.
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. GloVe: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pp. 1532-1543, 2014. URL http::/t/// hwwww/wwww.ooooo////ooo../ acl web. wwwprr//nnntttggnnneeeoeeeeeeeee 62.
10

---

Under review as a conference paper at ICLR 2021
Manoel Horta Ribeiro, Pedro H. Calais, Yuri A. Santos, Virgilio A. F. Almeida, and Wagner Meira Jr. "Like sheep among wolves": Characterizing hateful users on twitter. arXiv preprint arXIv:1801.00317.1 2017.
Manoel Horta Ribeiro, Pedro H. Calais, Yuri A. Santos, Virgilio A. F. Almeida, and Wagner Meira Jr. Characterizing and detecting hateful users on twitter. arXiv preprint (rxxiv:8033.050 2018.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80) 2008
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pp. 593-607. Springer, 2018.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification network data. Al magazine, 29(3):93-93, 2008
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. Relational Representation Learning Workshop, NeurlPS 2018, 2018.
Yiwei Sun, Suhang Wang, Xianfeng Tang, Tsung- Yu Hsieh, and Vasant Honavar. Non-target-specific node injection attacks on graph neural networks: A hierarchical reinforcement learning approach. In Proc. WWW, volume 3, 2020.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In International Conference on Machine Learning, pp. 3462-3471, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018. URL https: wpwp/eveeeeeeeeee//e/ee /penneeee net forum? dtt.1ddrrrr.pp...
Eric Wallace, Mitchell Stern, and Dawn Song. Imitation attacks and defenses for black-box machine translation systems. arXiv preprint arXiv:2004. 15015, 2020.
Binghui Wang, Jinyuan Jia, Xiaoyu Cao, and Neil Zhenqiang Gong. Certified robustness of graph neural networks against adversarial structural perturbation. arXiv preprint arXiv:2008. 10715, 2020.
Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. Cross-lingual knowledge graph alignment via graph convolutional networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 349-357, 2018
Marcin Waniek, Tomasz Michalak, Michael J. Wooldridge, and Talal Rahwan. Hiding individuals and communities in social network. Nature Human Behaviour, 2(2):139-147, 2018.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In International Conference on Machine Learning, pp. 6861-6871, 2019a.
Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples for graph data: deep insights into attack and defense. In Proceedings of the 28th International Joint Conference Artificial Intelligence, pp. 4816-4823. AAAI Press, 2019b.
Kaidi Xu, Hongge Chen, Sijia Liu, Pin- Yu Chen, Tsui- Wei Weng, Mingyi Hong, and Xue Lin. Topol- ogy attack and defense for graph neural networks: an optimization perspective. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3961-3967. AAAI Press, 2019a.
11

---

Under review as a conference paper at ICLR 2021
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful graph neural networks? In International Conference on Learning Representations. 2019b. URL httpa:
//oeenreview net/ formm?dd-ryG 6m.5Km.
Noam Yefet, Uri Alon, and Eran Yahav. Adversarial examples for models of code. arXiv preprint arXiv:1910.07517, 2019.
Xiao Zang, Yi Xie, Jie Chen, and Bo Yuan. Graph universal adversarial attacks: A few bad actors ruin graph learning models. arXiv preprint arXiv:2002.0478: 2020.
Xiang Zhang and Marinka Zitnik. GNNGuard: Defending graph neural networks against adversarial attacks. arXiv preprint arXiv:2006.08/ 04000 2020.
Daniel Züggnrr and Stephan Günnemann. Certifiable robustness and robust training for graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 256-256, 2019.
Daniel Zügner and Stephan Günnemann. Certifiable robustness of graph convolutional networks under structure perturbations. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1656-1665, 2020.
Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In International Conference Learning Representations, 2019. URL https://
openreview iww. n forum? 1x2-0ylx12999
Daniel Zügnerr, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2847-2856, 2018.
12

---

Under review as a conference paper at ICLR 2021
A SUPPLEMENTARY MATERIAL
A.1 DATASET STATISTICS
Statistics of the datasets are shown in Table A.I.
Table A.1: Dataset statistics.
#Training
#Val
#Test
#Unlabeled Nodes
#Classes
Avg. Node Degree
Cora
140
500
1000
2708
3.9
CiteSeer
120
500
1000
3327
6
2.7
PubMed
60
500
1000
19717
3
4.5
Twitter
4474
248
249
95415
2
45.6
A.2 ADDITIONAL GNN TYPES
Tables A.2 to A.4 present the test accuracy of different attacks applied on GAT (Veličković et al., 2018), GIN (Xu et al., 2019b), GraphSAGE (Hamilton al., 2017), RobustGCN (Zügner & Günnemann. 2019), and SGC (Wu et al., 2019a), showing the effectiveness of SINGLE across different GNN types.
Cora
CiteSeer
PubMed
EdgeGrad
66.4 1.2
49.4 14
64.9 1.0
SINGLE
40.0 = 12.5
63
35.7 13.3
SINGLE-hops
42.0 :: 11.5
41.7
35.5 13.6
GlobalGradEdge
67.8
48.3+
6..5
SINGLE+GradChoice
43.1
4.74
36.4
SINGLE++opollgy
32.2
25.5 1.00
27.8
SINGLE-IW attackers
1..7
11.0
26.8
SINGLEE-direc
2.56
14.8
21.8
Table A.2: Test accuracy of GAT under different non-targeted attacks
Cora
CiteSeer
PubMed
EdgeGrad
32.9 3.1
33.3 1.7 38.5
SINGLE
27.1 1.3
12.3 2:9
12.9
SINGLE-hops
32.6
1..5
14.0
GlobalGradEdge
10.7 2.8
4,8 2.1
10.3 1 1.0
SINGLE+GradChoiee
15.9
8.1
10.0
SINoooE+Topolog
16.11117
1.6
6.3
SINGLEE-wo attackers
2.7.10.7
5.4
6.2
SINGL.E-direct
5.7
1.1.
3.1 131.1
Table A.3: Test accuracy of GIN under different non-targ non-targeted attacks
Surprisingly, Table A.5 shows that Robust GCN (Zügner Günnemann, 2019) is as vulnerable to the SINGLE attack as a standard GCN, showing that there is still much room for novel ideas and improvements to the robustness of current GNNs.
A.3 TARGETED ATTACKS
Tables A.6 A.9 show the results of targeted attacks across datasets and approaches. Differently from other tables which show test accuracy, Tables A.6 to A.9 present the targeted attack's s success rate, which the fraction of test examples that the attack managed force specific label prediction
13

---

Under review as a conference paper at ICLR 2021
Cora
CiteSeer
PubMed
EdgeGrad
62.9
34
1.
SINGLE
2.
3..
07.1
SINGLE-hops
30.0
4..
6..9
GlobalGradEdge
48.9
3..4
64.7 1: 1.1
SinhhieGradChoice
3..
18.0
0.7
SINGLE+Topology
3.6
1.22
6.6 0.3
SINGLEE-to attackers
14.4.1..9
11.1 1.111
45.4 +
SINGLE-direct
19.6
13.5
0.0 0.1
Table A.4: Test accuracy of GraphSAGE under different non-targeted attacks
GCN
RobustGGN
SGC
Clean
78.5 4:
73.9 1.6
78.9
EdgeGrad
5..7
65.1
SINGLE
45.5
34.3 14
47.3 1.2
SINGLE+hops
48.7 0.09
29.7 :: 1.1
49.6 1.2
GlobalGradEdge
1..3
0.4
SINGLEAArnuChoice
8.5
19.6 : 0.9
11.3 1.2
SINGLEA+Topoloy
5.2.1
72.5 1.9
5.6
SINGLE+two attackers
27.7
20.0
30,0= 1.8
SINGLFE+direct
0.3
15.8 1.8:1.
0.5 = 0.2
Table A.5: Test accuracy GCN, Robust GCN (Ziigner Günnemann, 2019), and SGC (Wu et al., 2019a) under different non-targeted attacks, on PubMed.
Cora
CiteSeer
PubMed
Twitter
EdgeGrad
8.0
14.8 0.5
20.1
12.6
SINGLE
36.6 + 2.4
60.7 =:2
38.2
14.6
SINGLE-hops
33.6
50.0
35.2
12.6
GlobalGradEdge
59.4
78.7 0.9
80.1
13.0
SINGLE+GradChoice
65.8 1.5
6.22
43.5
42.2 11.6
SINGLE+Topology
2..3
66.3
9..4
5..4
Table A.6: Success rate (higher is better) of different targeted attacks GCN network.
(in these results, higher is better). These results suggest that in targeted attack settings, attacks (such as SINGLE) have even bigger advantage over edge-based attacks (such as EdgeGrad).
Cora
CiteSeer
PubMed
EdgeGrad
6.1
1
17.9
SINGLE
33.7
43,5 13.5+11
50.7 ::
SINGLE-indirect
7..6
29.88
50.3 4 15.7
GlobalGradEdge
6.0
1..8
22.3 1:
SINGLE+GradChoice
25.8
38.6 1.8.
50.5 :: 13.5
SINGLE+Topology
41.3+68.
11.
63.0 = 10.2
Table A.7: Success rate (higher is better) of different targeted attacks on GAT network.
14

---

Under review as a conference paper at ICLR 2021
Cora
CiteSeer
PubMed
EdgeGrad
1.
25.6
26
SINGLE
1.7
49
58.8
SINGLE-hops
2.2
4.1
5.7
GlobalGradEdge
44.7
5.00.17.0
64.9 11.8
SINGLE+GradChoice
4.03
4.5
63.5 499
SINGLE+Topology
2..1
58.7 5.1
73.2 13.3
Table A.8: Success rate (higher is better) of different largeted attacks GIN network.
Cora
CiteSeer
PubMed
EdgeGrad
7.6
1.73
19.1 1.4
SINGLE
24.3
27.9 2..0
SINGLE-hops
18
2..2
24.1 1.0
GlobalGradEdge
9.3 0.9
1.
19.6
SINGLE+GradChoice
49.1 3.0
63.7
36.3
SINGLE+Topology
54.1 1.3
6:.3
89.8 :: 0.3
Table A.9: Success rate (higher is better) of different targeted attacks on GraphSAGB network.
A.4 MULTIEDGE ATTACKS
We strengthened the EdgeGrad attack by allowing it to add and remove multiple edges that are connected the attacker node MultiEdgeGrad. Accordingly, MulttGloallEdgeGnd is equivalent to GlobalEdgeGrad, except that MultiGlobalEdgeGrrd can choose the attacker node.
PubMed
Clean
0..
EdgeGrad
6..1
MultiEdgeGraa
6.2
SINGLE
45.5 0,5
SINGLE-hhops
48.7
SINGLE+Topology
5.2 0.1
SINGLE+GradChoice
8.5
GlohalGradEdge
15.3
MultiGiabalGradEdge
15.3
Table A.10: Test accuracy of GCN using MultiEdge attacks
As shown in Table A0 allowing the attacker node to add and remove multiple edges (MultiEdge- Grad and MuliiGlobllEggeGru) results in a very minor improvement compared to EdgeGrad and GlobalEdge while SINGLE, SINGLE+Topology and SINGLE+GradChoice are much more effective.
A.5 ADDITIONAL BASELINES
A.5.1 ZERO-FEATURES APPROACH
We experimented with a baseline where we set 7) = -2 as the feature perturbation. The objective of experimenting with such attack is illustrate that SINGLE find better perturbations than simply canceling the node feature vector, making the new vector vector zeros (and thus effectively removes the edges of the attacker node in GCN).
As shown, Zero features is barely effective (compared to "Clean"), and SINGLE can find much better perturbations.
15

---

Under review as a conference paper at ICLR 2021
PubMed
Clean SINGLE
78.5
4
Zero features
0.
Table A.11: 1:: Test accuracy of our zero features attack on a GCN network.
A.5.2 INJECTION ATTACKS
We also study an additional type of realistic attack that is based on node injection, In this approach, we insert a new node to the graph with single edge attached to our victim node. The attack is performed by perturbing the injected node's attributes. Since there is initial node feature vector to measure the 6Go distance to, the injected node allowed to find any realistic representation (e.g.. without choosing negative frequencies). This attack is very powerful, reducing the test accuracy down to 0.02% on PubMed.
A.5.3 LARGER NUMBER ATTACKERS
Number of attackers
PubMed
45.5
2
27.7
3
19.0
4
15.3
5
12.9
Table A.12: Test accuracy for different number of attackers on PubMed.
We performed additional experiments with up five randomly sampled attacker nodes simultaneously (Table A.12). As expected, allowing a larger number attackers reduces the test accuracy. However, the main observation in this paper is that even single attacker node is surprisingly effective.
A.6 LIMITING THE ALLOWED EQ
In Section 4.5, we analyzed the effect of the value of co that is, the maximal allowed perturbation in each vector attribute, on the performance of the attack. However, in datasets such as Cora and CiteSeer, the input features are binary (i.e., the input node vector is many-hot), so the possible perturbation to each vector element only "flipping" its value from zero to one, or vice-versa. Thus, in these datasets, it is interesting to analyze the value of cp, the maximal number of allowed perturbed vector elements, on the performance the attack. In this case, measuring the ly norm is equivalent to measuring the / norm: || || and is proportional to the l2 norm.
We performed experiments where we measured the test accuracy of the model while limiting the number of allowed perturbed vector elements. The results shown in Figure A.1.
As shown, when 6g = 0, no attack is allowed, and the test accuracy is equal to the "Clean" value of Table 1. When Eg 100%, the results equal to the SINGLE values of Table - resulting in flipping 50% of the features on average in Cora, and 31 % of the features on average in CiteSeer.
It is important note that in practice, the average number of perturbed features is much lower than the maximal number of allowed features. For example, in CiteSeer, allowing 100% of the features results in actually using only 31% on average.
16

---

Under review as a conference paper at ICLR 2021
70
60
50
Acc
40
Cora - SINGLE
30
Cora SANGLE+GrulChoice
CiteSeer SINGLE
20
CiteSeer SINGLE+GrudChoice
10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Maximal fraction of perturbed features
Figure A.1: Test accuracy compared the maximal allowed co, the number of perturbed features (divided by the total number of features in the dataset). In practice, the average number of perturbed features much lower than the maximal number of allowed features.
17


Reviews:
[{"review_id": "pPogDc5POm", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This submission shows that a single-node attack for GNNs can be surprisingly effective. The discovery is mostly in the form of experimental results, rather than in the form of a new method.\n\nStrengths:\n\nS1: The discovery that a single-node attack can be surprisingly effective is indeed interesting. It can potentially complement our understanding of GNNs and bring novel ideas in the future.\n\nS2: The empirical results are mostly sufficient to support the main insight that a single-node attack can be effective in many cases. Still, it would be even better if Table 2 can include results for models other than the vanilla GNNs (e.g., GAT) (note that there aren’t GAT’s results on Cora, CiteSeer, Twitter, even though GAT’s result on PubMed is shown in Figure 2). \n\nS3: Good literature review. Friendly to even the readers who don’t follow the field of adversarial attacks closely.\n\nWeaknesses:\n\nW1: The abstract is a bit exaggerating the main discovery. The empirical results only suggest that single-node attacks are effective under the following conditions: (1) the model is the vanilla GNN (while GAT is much harder to attack), (2) the attack node is close enough to the victim node (<= 3 hops), and (3) the node features are of the continuous kind (the results on datasets of discretized features are much weaker).\n \nW2: The proposed method seems unrealistic for datasets of discretized (one-hot, multi-hot, integer-valued, etc.) node features. (1) It seems like the optimized perturbation vector \\eta is still of continuous values (rather than being discretized after optimization) even for datasets of discretized node features. (2) The submission states that “In Cora, SINGLE perturbed 717 elements on average, which are 50%...” However, 50% of ALL features seems to be a bit too unrealistic if most of the normal nodes have far fewer than 50% features that are not zeros. \n \nW3: The submission is mostly about new insights. It, however, barely invent any novel techniques. The submission would be stronger if it had explored how to improve the effectiveness of the single-node attack on the discretized datasets or against GNNs with attention.", "extra_scores": {}, "extra_text": {}}, {"review_id": "K9BVKpAvG-i", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# summary #\nThis paper studies a problem of attacking graph neural networks. \nEspecially, the paper focuses on the single node attack while most of the attack studies focusing on edges. \nThe paper claims that attacks on multiple edges (and nodes) are difficult to conduct in practice. \nConsidering the single node attack is important since injecting a new single node, or a manipulation on one vulnerable node is less difficult. \nThe studied problem is an evasion attack on a node feature vector of a single node. \nUntargeted and targetted attacks are both formulated with a gradient-based update of a perturbation vector. The proposed attack method perturbates BoW count vectors, not the embedded continuous vectors. \nExperimental results show that the proposed single-node attack is more effective than a single-edge attack. Also, the single-node attack is not much weaker than the two-nodes attack experimentally. \n\n# comment #\nI am not an expert on the attacks of GNN. However, this manuscript is easy to read for non-expert readers like me. \n\nSingle-node interventions for GNNs sounds realistic than attacks on multiple edges. \nI feel a node injection (add a new malicious node to a graph) is also a possible scenario but the paper does not consider this case. Any comments? \n\nI find no obvious concerns about the methodology. However, I have a few questions concerning the experiments. \n\nTable 1: \nI'm not sure comparisons between the proposed SINGLE method and the baseline EdgeGrad method are fair comparisons, in terms of the amount of perturbations on graphs. \nThe proposed SINGLE method manipulates 15%, 50%, and 31% of the feature attributes for PubMed, Cora, and CiteSeer, respectively. \nHowever, the counterpart EdgeGrad always manipulates a single edge. Is a single edge manipulation a (roughly) same amount of perturbations of MULTIPLE attribute manipulations of the SINGLE method? \nAn average degree (number of edges a node has) will be a good measure to check this issue. Is it possible to add an average degree to Table 4? \n\nTable 3:\nI'm curious about the differences between SINGLE-two attackers and the SINGLE. \nThe two-nodes attacks perform inferior 3 out of 4 datasets than the single-node attacks. \nThis conflicts with a naive guess: multi-node attacks is stronger than single-node attacks. \nSo I think many readers will expect some intuitive explanations. Gradients from two nodes harm each other in optimization? What will happen if we choose three, or more attackers? \n\n\n# Evaluation points #\n(+) Single-node evasion attack on GNN is first in the literature\n\n(+) Readability\n\n(-) I'm not fully sure about the fairness of parts of the experiment Table 1. \n\n(-) No explanation in why two-nodes attacks are not superior to the one-node attacks. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "JVikAF-wTG", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors mainly show that the adversary can force the GNN to classify any target node to a chosen label by perturbing another single arbitrary node’s feature in the graph. The paper is well written and easy to understand. However, there are several concerns about the paper:\n\n1. The novelty of the paper is rather limited. The paper simply uses the gradient attacker method to add continuous perturbations to the node attributes. What is the novelty here compared to other gradient based attacks for the data without graph structure such as images? I don’t see any novelty or contribution from the methodology perspective.\n\n2. The problem setting needs to be well discussed. In the introduction, the authors use the example of crafting adversarial posts to motivate the problem. However, in the problem definition, it becomes adding perturbations to the node features.  Modifying the node’s feature in realistic settings is even harder than modifying the graph structure to me .(They could revise the words or sentences in the post, but they could not add the feature vector directly as the feature vector are usually preprocessed by some other models).\n\n3. The authors should include some robust GNNs such as [1][2] as baselines to test how effective is the proposed method. The authors should also consider add more baselines as the attack methods. \n\n[1] Zügner, Daniel, and Stephan Günnemann. \"Certifiable robustness and robust training for graph convolutional networks.\" In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 246-256. 2019.\n[2] Jin, Hongwei, and Xinhua Zhang. \"Robust Training of Graph Convolutional Networks via Latent Perturbation.\" ECML-PKDD 2020\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "B30wPJg1Qa4", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies the problem of adversarial attacks in graph neural networks. It proposes a new attack strategy called single-node attack where only one node is perturbed.  A gradient-based attack algorithm is proposed to modify features of the attack node and achieve the single-node attack. Experiments have demonstrated the effectiveness of the proposed attack algorithm.\nThe studied problem (adversarial attack on graphs) is important and the single-node attack setting is interesting. However, there are several concerns that need to be addressed.\n1.\tFor the constraint of unnoticeable perturbation discussed in Section 3.2, the authors argue that the perturbation would be unnoticeable if the frequency of some words is slightly modified. This contradicts with the claim that the attack should preserve the feature co-occurrence to make the perturbation unnoticeable in Nettack[1]. According to Section 4.1, SINGLE changes 50%, 31% of the node attributes on Cora and CiteSeer, respectively. With so many perturbed elements, the attack could be easily detected if many pairs of features that never appear together suddenly co-occur in a single node.\n2.\tAlso, it is hard to evaluate whether comparing single-node attack with single-edge attack is fair, since single-edge attack only changes one edge while single-node attack can change a relative larger number of elements in the feature vector. (In Nettack, one element change in the feature matrix or adjacency matrix is considered as one perturbation) It would be more convincing if the number of elements in the feature vector changed by SINGLE is much less.\n3.\tFor the first claim in Section 3.2 \"Perturbing nodes instead of edges\", although this claim shows perturbing features could be better than removing edges, it remains a question whether it is better than adding edges. Also, it might be interesting to include one more baseline where we set $\\eta=-x_a$ as the feature perturbations to illustrate SINGLE can find better perturbations than simply removing the edges of the attacker node.\n4.\tFor datasets with discrete features, SINGLE only uses \"$\\epsilon_{\\infty}=1$\" as the constraint but the discrete features are originally binary values and \"$\\epsilon_{\\infty}=1$\". This basically means no unnoticeable constraints are applied on the attacker for those datasets (Cora and CiteSeer). Hence, it would be more convincing to conduct experiments on how the model performs when using different norms as constraints (e.g., l1 and l2 norm) for discrete datasets.\n5.\tSince the authors claim a single-node attack is as effective as a multiple-node attack, multiple-node attacks, i.e setting the number of attacker nodes to a larger value, are suggested to be included as baselines.\n6.\tGiven the superiority of only changing the features for one attacker node, injecting one attacker node would be of more interest since it is even more practical than modifying features on the existing node (e.g., creating an attacker account on Twitter). So, it would be better to add the experiment of node injecting to further improve the algorithm.\nMinor Comments\n1.\tIt would be better to report the standard deviation of the proposed methods in Table 1/2 considering the performances of other baselines are reported with standard deviation.\n[1] Adversarial Attacks on Neural Networks for Graph Data. KDD'18\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2Bk2Nul-meV", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies how to attack a specific attacker node by perturbing only one single node's feature. The authors proposed a gradient-based attack strategy, namely SINGLE, to learn the perturbation vector with the consideration of unnoticeability. In general, I really like the idea of this work. Compared with other attacking strategies, the attacking scenario in this paper is more realistic, e.g., the access to a single node (hacking the account), etc. And the gradient-based attack is intuitive and straightforward. Overall, this paper is well-written and easy to follow. I have the following concerns that would like the authors to clarify.\n\n- In the paper, the authors claim that it can attacking by perturbing single arbitrary node. I think 'arbitrary' is a little bit over-claimed, since the distance between attacker node $a$ and the victim node $v$ should satisfy $d(a, v) \\leq L$ (# layers).\n- I am a little bit confused about how to process the encoded dataset. In section 4, the authors mention that the most frequent 10000 words are used as node features, then the glove embeddings are used to multiply by the embedding. Does it mean that the update rule in each layer becomes $\\sigma(A X Z W)$, where $A$ = adjacency matrix (shape $n \\times n$), $X$ = many-hot/TD-IDF encoding of 10000 words (shape $n \\times 10000$), $Z$ = $d$-dimensional word embedding (shape $10000 \\times d$) and $W$ = weight matrix (shape $d \\times \\textit{nhid}$)? If I understand it correctly, the derivative of loss wrt $X$ (i.e., many-hot/TD-IDF encoding) is calculated. If so, I think it could be better to make it clear in your manuscript.\n- If the parameter $\\epsilon_{\\inf}$ is set to a very small value (like 0.1 or 1), it is possible that a lot of words in the original text would be perturbed. In this case, is it still reasonable to say the perturbation is unnoticeable? For example, in section 4.1, there are 50% and 31% of the features (I assume it means words) being perturbed. If 50% of words are perturbed in some texts, I think we can notice such changes as a human. A related but different question: what are the ratio of perturbed words if we set $\\epsilon_{\\inf}$ a little bit larger?\n- A general question about your experiments: how does the test accuracy computed? If we only wish to attack one node, is it more reasonable to test the success rate (like waht the authors did in appendix). Or does it mean that the overall test accuracy can drop that much if we only perturb one node in the graph? This is a little bit unclear in the manuscript (or please point it out if there is something I missed).\n- About the analysis on distance between attacker and victim nodes, it seems the test accuracy of cora and pubmed does not change once the # layers reaches to 5. How many different rounds of experiments do the authors conduct in this part? If an attacker node is randomly chosen, you cannot always find a victim node v that has a large $d(a, v)$. If the authors want to study the effect of distance, maybe the attacker node needs to be carefully chosen.\n- In section 4.4, does the attacker still use GCN as base model (or imitation model) to attack GAT, GIN and GraphSage? If so, does the number of layers in imitation model remain the same as the attacked true model? What would happen if the number of layers in imitation model is different from the attacked model? I think all those models have some sort of nonlinearity baked in the model. Just for my curiosity, I wonder what would happen if we use a simpler model like SGC (without any nonlinearity) as the imitation model.\n- Just some minor comments, I sometimes got confused about 'attacker node' and 'victim node'. It would be better if the authors can clearly define it somewhere in the paper or mark them (which one is which) on figure 1.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper deals with adversarial attacks on graph neural networks, a new and promising field in graph representation learning. The paper analyzes a new extreme setting of attack for a single node, and presents important insights, albeit not new algorithms.

The reviewers were not particularly enthusiastic and complained about
- limited novelty in light of Zuegner et al
- missing baselines
- doubts about the attack setting with a selected attacker node

The authors provided an elaborate rebuttal, including specific responses to the above questions, however, the final scores are not quite above the bar, especially having in mind the sheer number of submissions on graph deep learning this year. We, therefore, recommend rejection and encourage the author to publish the paper elsewhere. 

Decision: Reject