Original: S
Revised: S

Original: I N G L E - N O D E A T TA C K F O R
Revised: I N G L E - N O D E A T TA C K F O R

Original: F O O L
Revised: F O O L

Original: I N G
Revised: I N G

Original: G R A P H N E U R
Revised: G R A P H N E U R

Original: A L N E T W O R K S
Revised: A L N E T W O R K S

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A B S T R AC T
Revised: A B S T R AC T

Original: Graph neural networks (GNNs) have shown broad applicability in a variety of domains.
Revised: Graph neural networks (GNNs) have shown broad applicability in a variety of domains.

Original: Some of these domains, such as social networks and product recommendations, are fertile ground for malicious users and behavior.
Revised: Some of these domains, such as social networks and product recommendations, are fertile ground for malicious users and behavior.

Original: In this paper, we show that GNNs are vulnerable to the extremely limited scenario of a single-node adversarial example, where the node cannot be picked by the attacker.
Revised: In this paper, we show that GNNs are vulnerable to the extremely limited scenario of a single-node adversarial example, where the node cannot be picked by the attacker.

Original: That is, an attacker can force the GNN to classify any target node to a chosen label by only slightly perturbing another single arbitrary node in the graph, even when not being able to pick that speciﬁc attacker node .
Revised: That is, an attacker can force the GNN to classify any target node to a chosen label by only slightly perturbing another single arbitrary node in the graph, even when not being able to pick that speciﬁc attacker node .

Original: When the adversary is allowed to pick a speciﬁc attacker node , the attack is even more effective.
Revised: When the adversary is allowed to pick a speciﬁc attacker node , the attack is even more effective.

Original: We show that this attack is effective across various GNN types (e.g., GraphSAGE, GCN, GAT, and GIN), across a variety of real-world datasets, and as a targeted anda non-targeted attack.
Revised: We show that this attack is effective across various GNN types (e.g., GraphSAGE, GCN, GAT, and GIN), across a variety of real-world datasets, and as a targeted and non-targeted attack.

Original: Our code is available anonymously at https://github.com/gnnattack/SINGLE .
Revised: Our code is available anonymously at https://github.com/gnnattack/SINGLE .

Original:  I N T RO D U C T I O N
Revised:  I N T RO D U C T I O N

Original: Graph neural networks (GNNs) (Scarselli et al., 2008; Micheli, 2009) have recently shown sharply increasing popularity due to their generality and computation-efﬁciency (Duvenaud et al., 2015; Li et al., 2016; Kipf & Welling, 2017; Hamilton et al., 2017; Veli ˇ ckovi ´ c et al., 2018; Xu et al., 2019b).
Revised: Graph neural networks (GNNs) (Scarselli et al., 2008; Micheli, 2009) have recently shown sharply increasing popularity due to their generality and computation-efﬁciency (Duvenaud et al., 2015; Li et al., 2016; Kipf & Welling, 2017; Hamilton et al., 2017; Veli ˇ ckovi ´ c et al., 2018; Xu et al., 2019b).

Original: Graph-structured data underlie a plethora of domains such as citation networks (Sen et al., 2008), social networks (Leskovec & Mcauley, 2012;
Revised: Graph-structured data underlie a plethora of domains such as citation networks (Sen et al., 2008), social networks (Leskovec & Mcauley, 2012;

Original: Ribeiro et al., 2017; 2018), knowledge graphs (Wang et al., 2018; Trivedi et al., 2017; Schlichtkrull et al., 2018), and product recommendations (Shchur et al., 2018).
Revised: Ribeiro et al., 2017; 2018), knowledge graphs (Wang et al., 2018; Trivedi et al., 2017; Schlichtkrull et al., 2018), and product recommendations (Shchur et al., 2018).

Original: Therefore, GNNs are applicable for a variety of real-world structured data.
Revised: Therefore, GNNs are applicable for a variety of real-world structured data.

Original: While most work in this ﬁeld has focused on improving the accuracy of GNNs and applying them to a growing number of domains, only a few past works have explored the vulnerability of GNNs to adversarial examples.
Revised: While most work in this ﬁeld has focused on improving the accuracy of GNNs and applying them to a growing number of domains, only a few past works have explored the vulnerability of GNNs to adversarial examples.

Original: Consider the following scenario: a malicious user joins a social network such as Twitter or Facebook.
Revised: Consider the following scenario: a malicious user joins a social network such as Twitter or Facebook.

Original: The malicious user mocks the behavior of a benign user, establishes connections with other users, and submits benign posts.
Revised: The malicious user mocks the behavior of a benign user, establishes connections with other users, and submits benign posts.

Original: After some time, the user submits a new adversarially crafted post, which might seem irregular but overall benign.
Revised: After some time, the user submits a new adversarially crafted post, which might seem irregular but overall benign.

Original: Since the GNN represents every user according to all the user’s posts, this new post perturbs the representation of the user as seen by a GNN.
Revised: Since the GNN represents every user according to all the user’s posts, this new post perturbs the representation of the user as seen by a GNN.

Original: As a result, another, speciﬁc benign user gets blocked from the network; alternatively, another malicious user submits a hateful post – but does not get blocked.
Revised: As a result, another, speciﬁc benign user gets blocked from the network; alternatively, another malicious user submits a hateful post – but does not get blocked.

Original: This scenario is illustrated in Figure 1.
Revised: This scenario is illustrated in Figure 1.

Original: In this paper, we show the feasibility of such a troublesome scenario: a single attacker node can perturb its own representation, such that another node will be misclassiﬁed as a label of the attacker’s choice.
Revised: In this paper, we show the feasibility of such a troublesome scenario: a single attacker node can perturb its own representation, such that another node will be misclassiﬁed as a label of the attacker’s choice.

Original: Most previous work on adversarial examples in GNNs required the perturbation to span multiple nodes, which in reality requires the cooperation of multiple attackers.
Revised: Most previous work on adversarial examples in GNNs required the perturbation to span multiple nodes, which in reality requires the cooperation of multiple attackers.

Original: For example, the pioneering work of Z¨ugner et al.
Revised: For example, the pioneering work of Z¨ugner et al.

Original: (2018) perturbed a set of attacker nodes; Bojchevski & G¨unnemann (2019a) perturb edges that are covered by a set of nodes.
Revised: (2018) perturbed a set of attacker nodes; Bojchevski & G¨unnemann (2019a) perturb edges that are covered by a set of nodes.

Original: Further and in contrast with existing work, we show that perturbing a single node is more harmful than perturbing a single edge .
Revised: Further and in contrast with existing work, we show that perturbing a single node is more harmful than perturbing a single edge .

Original: In this paper, we present a ﬁrst a single-node adversarial attack on graph neural networks.
Revised: In this paper, we present a ﬁrst a single-node adversarial attack on graph neural networks.

Original: If the adversary is allowed to choose the attacker node, for example, by hacking into an existing account, the efﬁciency of the attack signiﬁcantly increases.
Revised: If the adversary is allowed to choose the attacker node, for example, by hacking into an existing account, the efﬁciency of the attack signiﬁcantly increases.

Original: We present two approaches for choosing the attacker: a white-box gradient-based approach, and a black-box, model-free approach that relies on graph topology.
Revised: We present two approaches for choosing the attacker: a white-box gradient-based approach, and a black-box, model-free approach that relies on graph topology.

Original: Finally, we perform a comprehensive experimental evaluation of our approach on multiple datasets and GNN architectures.
Revised: Finally, we perform a comprehensive experimental evaluation of our approach on multiple datasets and GNN architectures.

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Let G =
Revised: Let G =

Original: { G i } N G i =1 be a set of graphs.
Revised: { G i } N G i =1 be a set of graphs.

Original: Each graph G =
Revised: Each graph G =

Original: ( V , E , X ) ∈ G has a set of nodes V and a set of edges E ⊆ V × V , where ( u, v ) ∈
Revised: ( V , E , X ) ∈ G has a set of nodes V and a set of edges E ⊆ V × V , where ( u, v ) ∈

Original: E denotes an edge from a node u ∈ V to a node v ∈ V .
Revised: E denotes an edge from a node u ∈ V to a node v ∈ V .

Original: X ∈ R N × D is a matrix of D -dimensional node features.
Revised: X ∈ R N × D is a matrix of D -dimensional node features.

Original: The i -th row of X is the feature vector of the node v i ∈ V and is denoted as x i
Revised: The i -th row of X is the feature vector of the node v i ∈ V and is denoted as x i

Original: = X i, : ∈ R D .
Revised: = X i, : ∈ R D .

Original: Graph neural networks GNNs operate by iteratively propagating neural messages between neighboring nodes.
Revised: Graph neural networks GNNs operate by iteratively propagating neural messages between neighboring nodes.

Original: Every GNN layer updates the representation of every node by aggregating its current representation with the current representations of its neighbors.
Revised: Every GNN layer updates the representation of every node by aggregating its current representation with the current representations of its neighbors.

Original: Formally, each node is associated with an initial representation x (0) v = h (0) v ∈ R D .
Revised: Formally, each node is associated with an initial representation x (0) v = h (0) v ∈ R D .

Original: This representation is considered as the given features of the node.
Revised: This representation is considered as the given features of the node.

Original: Then, a GNN layer updates each node’s representation given its neighbors, yielding h (1) v ∈ R d 1 for every v ∈ V .
Revised: Then, a GNN layer updates each node’s representation given its neighbors, yielding h (1) v ∈ R d 1 for every v ∈ V .

Original: In general, the (cid:96) -th layer of a GNN is a function that updates a node’s representation by combining it with its neighbors:
Revised: In general, the (cid:96) -th layer of a GNN is a function that updates a node’s representation by combining it with its neighbors:

Original: [Equation]
Revised: [Equation]

Original: where N v is the set of direct neighbors of v : N v = { u ∈ V | ( u, v ) ∈ E} .
Revised: where N v is the set of direct neighbors of v : N v = { u ∈ V | ( u, v ) ∈ E} .

Original: The COMBINE function is what mostly distinguishes GNN types.
Revised: The COMBINE function is what mostly distinguishes GNN types.

Original: For example, graph convolutional networks (GCN) (Kipf & Welling, 2017) deﬁne a layer as:
Revised: For example, graph convolutional networks (GCN) (Kipf & Welling, 2017) deﬁne a layer as:

Original: [Equation]
Revised: [Equation]

Original: where c u,v is a normalization factor usually set to (cid:112)
Revised: where c u,v is a normalization factor usually set to (cid:112)

Original: |N v | · |N u | .
Revised: |N v | · |N u | .

Original: After (cid:96) such aggregation iterations, every node representation captures information from all nodes within its (cid:96) -hop neighborhood.
Revised: After (cid:96) such aggregation iterations, every node representation captures information from all nodes within its (cid:96) -hop neighborhood.

Original: The total number of layers L is usually determined empirically as a hyperparameter.
Revised: The total number of layers L is usually determined empirically as a hyperparameter.

Original: In the node classiﬁcation scenario, we use the ﬁnal representation h
Revised: In the node classiﬁcation scenario, we use the ﬁnal representation h

Original: Lv to classify v .
Revised: Lv to classify v .

Original: For brevity, we focus our deﬁnitions on the semi-supervised transductive node classiﬁcation goal, where the dataset contains a single graph G , and the split into training and test sets is across nodes in the same graph.
Revised: For brevity, we focus our deﬁnitions on the semi-supervised transductive node classiﬁcation goal, where the dataset contains a single graph G , and the split into training and test sets is across nodes in the same graph.

Original: Nonetheless, these deﬁnitions can be trivially generalized to the inductive setting, where the dataset contains multiple graphs, the split into training and test sets is between graphs, and the test nodes are unseen during training.
Revised: Nonetheless, these deﬁnitions can be trivially generalized to the inductive setting, where the dataset contains multiple graphs, the split into training and test sets is between graphs, and the test nodes are unseen during training.

Original: We associate each node v ∈ V with a class y v ∈ Y = { 1 , ..., Y } .
Revised: We associate each node v ∈ V with a class y v ∈ Y = { 1 , ..., Y } .

Original: The labels of the training nodes are given during training; the test nodes are seen during training – without their labels.
Revised: The labels of the training nodes are given during training; the test nodes are seen during training – without their labels.

Original: The training subset is represented as D = (cid:16) G, { ( v i , y i ) } N D i =0 (cid:17) .
Revised: The training subset is represented as D = (cid:16) G, { ( v i , y i ) } N D i =0 (cid:17) .

Original: Given the training set, the goal is to learn a model f θ : ( G , V )
Revised: Given the training set, the goal is to learn a model f θ : ( G , V )

Original: → Y that will classify the rest of the nodes correctly.
Revised: → Y that will classify the rest of the nodes correctly.

Original: During training, the model f θ thus minimizes the loss over the given labels, using J ( · , · ) , which typically is the cross-entropy loss:
Revised: During training, the model f θ thus minimizes the loss over the given labels, using J ( · , · ) , which typically is the cross-entropy loss:

Original: [Equation]
Revised: [Equation]

Original:  S I N G L E - N O D E G N N A T TAC K
Revised:  S I N G L E - N O D E G N N A T TAC K

Original: In this section, we describe our Single-node INdirect Gradient adversariaL Evasion ( SINGLE ) attack.
Revised: In this section, we describe our Single-node INdirect Gradient adversariaL Evasion ( SINGLE ) attack.

Original: While our attack is simple, it is the ﬁrst attack that focuses on perturbing nodes (in contrast to edges (Dai et al., 2018)), which works with an arbitrary single attacker node (in contrast to multiple nodes (Z¨ugner et al., 2018))
Revised: While our attack is simple, it is the ﬁrst attack that focuses on perturbing nodes (in contrast to edges (Dai et al., 2018)), which works with an arbitrary single attacker node (in contrast to multiple nodes (Z¨ugner et al., 2018))

Original: that is not the node under attack (in contrast to “direct” attacks where the attacker perturbs the node under attack directly (Z¨ugner et al., 2018;
Revised: that is not the node under attack (in contrast to “direct” attacks where the attacker perturbs the node under attack directly (Z¨ugner et al., 2018;

Original: Li et al., 2020)).
Revised: Li et al., 2020)).

Original: 3. P ROBLEM D EFINITION
Revised: 3. P ROBLEM D EFINITION

Original: Given a graph G , a trained model f θ , a “victim” node v from the test set along with its classiﬁcation by the model ˆ y v = f θ ( G, v ) , we assume that an adversary controls another node a in the graph.
Revised: Given a graph G , a trained model f θ , a “victim” node v from the test set along with its classiﬁcation by the model ˆ y v = f θ ( G, v ) , we assume that an adversary controls another node a in the graph.

Original: The goal of the adversary is to modify its own feature vector x a by adding a perturbation vector η ∈ R D of its choice, such that the model’s classiﬁcation of v will change .
Revised: The goal of the adversary is to modify its own feature vector x a by adding a perturbation vector η ∈ R D of its choice, such that the model’s classiﬁcation of v will change .

Original: We denote by G x a + η the graph G where the row of X that corresponds to the node a was added with the vector η .
Revised: We denote by G x a + η the graph G where the row of X that corresponds to the node a was added with the vector η .

Original: In a non-targeted attack, the goal of the attacker is to ﬁnd a perturbation vector η that will change the classiﬁcation to any other class, i.e., f θ ( G x a + η , v ) (cid:54) =
Revised: In a non-targeted attack, the goal of the attacker is to ﬁnd a perturbation vector η that will change the classiﬁcation to any other class, i.e., f θ ( G x a + η , v ) (cid:54) =

Original: f θ ( G, v ) .
Revised: f θ ( G, v ) .

Original: In a targeted attack, the adversary chooses a speciﬁc label y adv ∈ Y and the adversary’s goal is to force f θ ( G x a + η , v ) =
Revised: In a targeted attack, the adversary chooses a speciﬁc label y adv ∈ Y and the adversary’s goal is to force f θ ( G x a + η , v ) =

Original: y adv .
Revised: y adv .

Original: Generally, the classiﬁcation of a node v depends only on nodes whose distance to v in the graph is lower than or equal L – the number of GNN layers.
Revised: Generally, the classiﬁcation of a node v depends only on nodes whose distance to v in the graph is lower than or equal L – the number of GNN layers.

Original: Thus, a modiﬁcation of the features of a will affect the classiﬁcation of v only if the distance between a and v is lower than or equal L .
Revised: Thus, a modiﬁcation of the features of a will affect the classiﬁcation of v only if the distance between a and v is lower than or equal L .

Original: Otherwise, a will not be contained in the receptive ﬁeld of v , and the attack will result in “under-reaching”
Revised: Otherwise, a will not be contained in the receptive ﬁeld of v , and the attack will result in “under-reaching”

Original: (Alon & Yahav, 2020) – any perturbation of a will not affect the prediction of v (Barcel ´ o et al., 2020).
Revised: (Alon & Yahav, 2020) – any perturbation of a will not affect the prediction of v (Barcel ´ o et al., 2020).

Original: Therefore, we require that distance G ( a, v ) ≤ L .
Revised: Therefore, we require that distance G ( a, v ) ≤ L .

Original: In this work, we focus on gradient-based attacks.
Revised: In this work, we focus on gradient-based attacks.

Original: These kinds of attacks assume that the attacker can access a similar model to the model under attack and compute gradients.
Revised: These kinds of attacks assume that the attacker can access a similar model to the model under attack and compute gradients.

Original: As recently shown by Wallace et al.
Revised: As recently shown by Wallace et al.

Original: (2020), this is reasonable assumption: an attacker can query the original model; using these queries, imitate the model under attack by training an imitation model; ﬁnd adversarial examples using the imitation model; and transfer these adversarial examples back to the original model.
Revised: (2020), this is reasonable assumption: an attacker can query the original model; using these queries, imitate the model under attack by training an imitation model; ﬁnd adversarial examples using the imitation model; and transfer these adversarial examples back to the original model.

Original: Under this assumption, these attacks are general and are applicable to any GNN and dataset.
Revised: Under this assumption, these attacks are general and are applicable to any GNN and dataset.

Original: 3. C HALLENGES
Revised: 3. C HALLENGES

Original: Unnoticeable Perturbations.
Revised: Unnoticeable Perturbations.

Original: Our ﬁrst challenge is to ﬁnd an adversarial example that will allow an imperceptible perturbation of the input.
Revised: Our ﬁrst challenge is to ﬁnd an adversarial example that will allow an imperceptible perturbation of the input.

Original: This objective is attainable in continuous domains such as images (Szegedy et al., 2013; Goodfellow et al., 2014) and audio (Carlini & Wagner, 2018) if we constrain l ∞ -norm of the perturbation vector η .
Revised: This objective is attainable in continuous domains such as images (Szegedy et al., 2013; Goodfellow et al., 2014) and audio (Carlini & Wagner, 2018) if we constrain l ∞ -norm of the perturbation vector η .

Original: It is, however, unclear what imperceptibility means in graphs.
Revised: It is, however, unclear what imperceptibility means in graphs.

Original: In most GNN datasets, a node’s features are a bag-of-words representation of the words that are associated with the node.
Revised: In most GNN datasets, a node’s features are a bag-of-words representation of the words that are associated with the node.

Original: For example, in Cora (McCallum et al., 2000; Sen et al., 2008), every node is annotated by a many-hot feature vector of words that appear in the paper; in
Revised: For example, in Cora (McCallum et al., 2000; Sen et al., 2008), every node is annotated by a many-hot feature vector of words that appear in the paper; in

Original: PubMed (Namata et al., 2012), node vectors are TF-IDF word frequencies; in Twitter (Ribeiro et al., 2017), node features are averages of GloVe embeddings, which can be viewed as word frequency vectors multiplied by a (frozen) embedding matrix.
Revised: PubMed (Namata et al., 2012), node vectors are TF-IDF word frequencies; in Twitter (Ribeiro et al., 2017), node features are averages of GloVe embeddings, which can be viewed as word frequency vectors multiplied by a (frozen) embedding matrix.

Original: We argue that an attack would be unnoticeable in an academic paper or in a set of Tweets if the frequency of some words is slightly modiﬁed.
Revised: We argue that an attack would be unnoticeable in an academic paper or in a set of Tweets if the frequency of some words is slightly modiﬁed.

Original: For example, a particular word may be repeated a few times throughout the text or remain unused.
Revised: For example, a particular word may be repeated a few times throughout the text or remain unused.

Original: To constrain the η vector, we require that (cid:107) η (cid:107) ∞ ≤ (cid:15) ∞ – the maximal absolute value of the elements in the perturbation vector – is bounded by (cid:15) ∞ ∈ R + .
Revised: To constrain the η vector, we require that (cid:107) η (cid:107) ∞ ≤ (cid:15) ∞ – the maximal absolute value of the elements in the perturbation vector – is bounded by (cid:15) ∞ ∈ R + .

Original: Perturbing nodes instead of edges.
Revised: Perturbing nodes instead of edges.

Original: Previous work mostly focused on perturbing graph edges .
Revised: Previous work mostly focused on perturbing graph edges .

Original: Z¨ugner et al.
Revised: Z¨ugner et al.

Original: (2018) perturb both edges and node features, but conclude that “perturbations in the structure lead to a stronger change in the surrogate loss compared to feature attacks”; Wu et al.
Revised: (2018) perturb both edges and node features, but conclude that “perturbations in the structure lead to a stronger change in the surrogate loss compared to feature attacks”; Wu et al.

Original: (2019b) also conclude that “perturbing edges is more effective than modifying the features”.
Revised: (2019b) also conclude that “perturbing edges is more effective than modifying the features”.

Original: In this paper, we counter these conclusions and show that small node feature perturbations are stronger: (i) ﬁrst, removing all the edges of a particular node is a special case of node feature perturbation.
Revised: In this paper, we counter these conclusions and show that small node feature perturbations are stronger: (i) ﬁrst, removing all the edges of a particular node is a special case of node feature perturbation.

Original: There exists a perturbation η such that W 1 ( x a + η ) = 0 , i.e., the modiﬁed feature vector x a + η is in the null space of the ﬁrst GNN layer.
Revised: There exists a perturbation η such that W 1 ( x a + η ) = 0 , i.e., the modiﬁed feature vector x a + η is in the null space of the ﬁrst GNN layer.

Original: 1 Such a feature perturbation is equivalent to removing all the edges of the node a .
Revised: 1 Such a feature perturbation is equivalent to removing all the edges of the node a .

Original: (ii)
Revised: (ii)

Original: Second, we argue that perturbing the graph structure is not realistic, because a single attacker controls only its own edges, and cannot control the global graph structure as in previous work (Dai et al., 2018;
Revised: Second, we argue that perturbing the graph structure is not realistic, because a single attacker controls only its own edges, and cannot control the global graph structure as in previous work (Dai et al., 2018;

Original: Bojchevski & G¨unnemann, 2019b; Zhang & Zitnik, 2020).
Revised: Bojchevski & G¨unnemann, 2019b; Zhang & Zitnik, 2020).

Original: (iii) Finally, when a successful attack is caused by removing edges, it is unclear whether the misclassiﬁcation is caused by sensitivity to non-robust features in the data (Ilyas et al., 2019), or simply due to smaller amount of information.
Revised: (iii) Finally, when a successful attack is caused by removing edges, it is unclear whether the misclassiﬁcation is caused by sensitivity to non-robust features in the data (Ilyas et al., 2019), or simply due to smaller amount of information.

Original: Similarly, when a successful attack is caused by inserting edges, it is unclear whether this is simply due to incorrect or unrealistic added information.
Revised: Similarly, when a successful attack is caused by inserting edges, it is unclear whether this is simply due to incorrect or unrealistic added information.

Original: 3. F INDING THE P ERTURBATION V ECTOR
Revised: 3. F INDING THE P ERTURBATION V ECTOR

Original: To ﬁnd the perturbation, we iteratively differentiate the desired loss of v with respect to the perturbation vector η , update η according to the gradient, and add it to the feature vector.
Revised: To ﬁnd the perturbation, we iteratively differentiate the desired loss of v with respect to the perturbation vector η , update η according to the gradient, and add it to the feature vector.

Original: In non-targeted attacks, we take the positive gradient of the loss of the undesired label to increase the loss; in targeted attacks, we take the negative gradient of the loss of the adversarial label y
Revised: In non-targeted attacks, we take the positive gradient of the loss of the undesired label to increase the loss; in targeted attacks, we take the negative gradient of the loss of the adversarial label y

Original: adv :
Revised: adv :

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: ) where γ ∈ R + is a learning rate.
Revised: ) where γ ∈ R + is a learning rate.

Original: We repeat this process for a predeﬁned number of K iterations, or until the model predicts the desired label.
Revised: We repeat this process for a predeﬁned number of K iterations, or until the model predicts the desired label.

Original: Enforcing the constraints.
Revised: Enforcing the constraints.

Original: We treat the node features as continuous throughout the attack iterations, whether they are discrete or continuous.
Revised: We treat the node features as continuous throughout the attack iterations, whether they are discrete or continuous.

Original: Once the attack succeeds, we try to reset to zero as many perturbation vector elements as possible.
Revised: Once the attack succeeds, we try to reset to zero as many perturbation vector elements as possible.

Original: We sort the perturbation vector elements in a decreasing order, according to their absolute value: i 1 , ..., i D .
Revised: We sort the perturbation vector elements in a decreasing order, according to their absolute value: i 1 , ..., i D .

Original: We start with the index of η whose absolute value is the largest, η i 1 , and reset the rest of the { i 2 , ..., i D } elements to zero.
Revised: We start with the index of η whose absolute value is the largest, η i 1 , and reset the rest of the { i 2 , ..., i D } elements to zero.

Original: We then check whether perturbing only the i 1 index is sufﬁcient.
Revised: We then check whether perturbing only the i 1 index is sufﬁcient.

Original: If the attack succeeds, we stop.
Revised: If the attack succeeds, we stop.

Original: If the attack fails (because of the large number of perturbation vector elements set to zero), we continue perturbing the rest of the elements of η .
Revised: If the attack fails (because of the large number of perturbation vector elements set to zero), we continue perturbing the rest of the elements of η .

Original: In the worst case, we perturb all D vector elements of η .
Revised: In the worst case, we perturb all D vector elements of η .

Original: In most cases, we stop much earlier, practically perturbing only a small fraction of the vector elements.
Revised: In most cases, we stop much earlier, practically perturbing only a small fraction of the vector elements.

Original: If the original node features are discrete, we discretized features after the optimization.
Revised: If the original node features are discrete, we discretized features after the optimization.

Original: Differentiate by frequencies, not by embeddings.
Revised: Differentiate by frequencies, not by embeddings.

Original: When taking the gradient with respect to the perturbation vector ∇ η , there is a subtle, but crucial, difference between the way that node representations are given in the dataset: (a) indicative datasets provide initial node representations
Revised: When taking the gradient with respect to the perturbation vector ∇ η , there is a subtle, but crucial, difference between the way that node representations are given in the dataset: (a) indicative datasets provide initial node representations

Original: X =
Revised: X =

Original: [ x 1 , x 2 , ... ]
Revised: [ x 1 , x 2 , ... ]

Original: that are word indicator vectors (many-hot) or frequencies such as (weighted) bagof-words (Sen et al., 2008; Shchur et al., 2018); (b) in encoded datasets, initial node representations are given encoded, e.g., as an average of word2vec vectors (Hamilton et al., 2017;
Revised: that are word indicator vectors (many-hot) or frequencies such as (weighted) bagof-words (Sen et al., 2008; Shchur et al., 2018); (b) in encoded datasets, initial node representations are given encoded, e.g., as an average of word2vec vectors (Hamilton et al., 2017;

Original: Hu et al., 2020).
Revised: Hu et al., 2020).

Original: Indicative datasets can be converted to encoded by multiplying every vector by an embedding matrix; encoded datasets cannot be converted to indicative , without the authors releasing the textual data that was used to create the encoded dataset.
Revised: Indicative datasets can be converted to encoded by multiplying every vector by an embedding matrix; encoded datasets cannot be converted to indicative , without the authors releasing the textual data that was used to create the encoded dataset.

Original: In indicative datasets, a perturbation of a node vector can be realized as a perturbation of the original text from which the indicative vector was derived.
Revised: In indicative datasets, a perturbation of a node vector can be realized as a perturbation of the original text from which the indicative vector was derived.

Original: That is, adding or removing words in the text can result in the perturbed node vector.
Revised: That is, adding or removing words in the text can result in the perturbed node vector.

Original: In contrast, a few-indices perturbation in encoded datasets might be an effective attack, but will not be realistic because there is no perturbation of the original text that will result in that perturbation of the vector.
Revised: In contrast, a few-indices perturbation in encoded datasets might be an effective attack, but will not be realistic because there is no perturbation of the original text that will result in that perturbation of the vector.

Original: That is, when perturbing nodes, it is crucial to use indicative datasets, or convert encoded datasets to the indicative representation from which they were derived (as we do in Section 4) using their original text.
Revised: That is, when perturbing nodes, it is crucial to use indicative datasets, or convert encoded datasets to the indicative representation from which they were derived (as we do in Section 4) using their original text.

Original:  E VA L UAT I O N
Revised:  E VA L UAT I O N

Original: We evaluate and analyze the effectiveness of our SINGLE attack.
Revised: We evaluate and analyze the effectiveness of our SINGLE attack.

Original: In Section 4.1, we show that
Revised: In Section 4.1, we show that

Original: SINGLE is more effective than alternatives such as single-edge attacks.
Revised: SINGLE is more effective than alternatives such as single-edge attacks.

Original: In Section 4.2, we show that if we are allowed to choose the attacker node, SINGLE is signiﬁcantly more effective.
Revised: In Section 4.2, we show that if we are allowed to choose the attacker node, SINGLE is signiﬁcantly more effective.

Original: Setup.
Revised: Setup.

Original: Our implementation is based on PyTorch Geometric (Fey & Lenssen, 2019) and its provided datasets.
Revised: Our implementation is based on PyTorch Geometric (Fey & Lenssen, 2019) and its provided datasets.

Original: We trained each GNN type with two layers ( L = 2 ), using the Adam optimizer, early
Revised: We trained each GNN type with two layers ( L = 2 ), using the Adam optimizer, early

Original: [Table]
Revised: [Table]

Original: stopped according to the validation set, and applied a dropout of 0 .
Revised: stopped according to the validation set, and applied a dropout of 0 .

Original: 5 between layers.
Revised: 5 between layers.

Original: We used up to K = 20 attack iterations.
Revised: We used up to K = 20 attack iterations.

Original: All experiments in this section were performed with GCN, except for
Revised: All experiments in this section were performed with GCN, except for

Original: Section 4.5, where additional GNN types (GAT, GIN, and GraphSAGE) are shown.
Revised: Section 4.5, where additional GNN types (GAT, GIN, and GraphSAGE) are shown.

Original: In Appendix A.2, we show consistent results across additional GNN types: GAT (Veliˇckovi´c et al., 2018), GIN (Xu et al., 2019b), GraphSAGE (Hamilton et al., 2017), SGC (Wu et al., 2019a), and RobustGCN (Z¨ugner & G¨unnemann, 2019).
Revised: In Appendix A.2, we show consistent results across additional GNN types: GAT (Veliˇckovi´c et al., 2018), GIN (Xu et al., 2019b), GraphSAGE (Hamilton et al., 2017), SGC (Wu et al., 2019a), and RobustGCN (Z¨ugner & G¨unnemann, 2019).

Original: Data.
Revised: Data.

Original: We used Cora and CiteSeer (Sen et al., 2008) which are discrete datasets, i.e., the given node feature vectors are many-hot vectors.
Revised: We used Cora and CiteSeer (Sen et al., 2008) which are discrete datasets, i.e., the given node feature vectors are many-hot vectors.

Original: Thus, we set (cid:15) ∞
Revised: Thus, we set (cid:15) ∞

Original: = 1 , the minimal possible perturbation.
Revised: = 1 , the minimal possible perturbation.

Original: We also used PubMed (Sen et al., 2008) and the Twitter-Hateful-Users (Ribeiro et al., 2017) datasets, which are continuous , and node features represent frequencies of words.
Revised: We also used PubMed (Sen et al., 2008) and the Twitter-Hateful-Users (Ribeiro et al., 2017) datasets, which are continuous , and node features represent frequencies of words.

Original: Continuous datasets allow a much more subtle perturbation, and we set (cid:15) ∞
Revised: Continuous datasets allow a much more subtle perturbation, and we set (cid:15) ∞

Original: = 0 .
Revised: = 0 .

Original: An analysis of these values is presented in
Revised: An analysis of these values is presented in

Original: Section 4.5.
Revised: Section 4.5.

Original: The Twitter-Hateful-Users dataset is originally provided as an encoded dataset, where every node is an average of GloVe vectors (Pennington et al., 2014).
Revised: The Twitter-Hateful-Users dataset is originally provided as an encoded dataset, where every node is an average of GloVe vectors (Pennington et al., 2014).

Original: We reconstructed this dataset using the original text from Ribeiro et al.
Revised: We reconstructed this dataset using the original text from Ribeiro et al.

Original: (2017), to be able to compute gradients with respect to the weighted histogram of words, rather than the embeddings.
Revised: (2017), to be able to compute gradients with respect to the weighted histogram of words, rather than the embeddings.

Original: We took the most frequent 10,000 words as node features, and used GloVe-Twitter embeddings to multiply by the node features.
Revised: We took the most frequent 10,000 words as node features, and used GloVe-Twitter embeddings to multiply by the node features.

Original: We thus converted this dataset to indicative rather than encoded .
Revised: We thus converted this dataset to indicative rather than encoded .

Original: Statistics of all dataset are provided in the supplementary material.
Revised: Statistics of all dataset are provided in the supplementary material.

Original: Baselines.
Revised: Baselines.

Original: In SINGLE (Section 3.3) the attacker node is selected randomly for each victim node, and the attack perturbs this node’s features according to (cid:15) ∞ .
Revised: In SINGLE (Section 3.3) the attacker node is selected randomly for each victim node, and the attack perturbs this node’s features according to (cid:15) ∞ .

Original: SINGLE -hops is a modiﬁcation of
Revised: SINGLE -hops is a modiﬁcation of

Original: SINGLE where the attacker node is sampled only among nodes that are not neighbors , i.e., the attacker and the victim are not directly connected ( ( a, v ) / ∈ E ).
Revised: SINGLE where the attacker node is sampled only among nodes that are not neighbors , i.e., the attacker and the victim are not directly connected ( ( a, v ) / ∈ E ).

Original: We compare to additional approaches from the literature: EdgeGrad follows most previous work (Xu et al., 2019a;
Revised: We compare to additional approaches from the literature: EdgeGrad follows most previous work (Xu et al., 2019a;

Original: Li et al., 2020; Z ¨ ugner & G ¨ unnemann, 2020): EdgeGrad randomly samples an attacker node as in SINGLE , and either inserts or removes a single edge from or to the attacker node, according to the gradient.
Revised: Li et al., 2020; Z ¨ ugner & G ¨ unnemann, 2020): EdgeGrad randomly samples an attacker node as in SINGLE , and either inserts or removes a single edge from or to the attacker node, according to the gradient.

Original: If both use a randomly selected attacker node, EdgeGrad is strictly stronger than the GradArgmax attack of Dai et al.
Revised: If both use a randomly selected attacker node, EdgeGrad is strictly stronger than the GradArgmax attack of Dai et al.

Original: (2018), which only removes edges.
Revised: (2018), which only removes edges.

Original: We ran each approach 5 times with different random seeds for each dataset, and report the mean and standard deviation.
Revised: We ran each approach 5 times with different random seeds for each dataset, and report the mean and standard deviation.

Original: 4. M AIN R ESULTS
Revised: 4. M AIN R ESULTS

Original: Table 1 shows our main results for non-targeted attacks across various datasets.
Revised: Table 1 shows our main results for non-targeted attacks across various datasets.

Original: As shown, SINGLE is more effective than EdgeGrad across all datasets.
Revised: As shown, SINGLE is more effective than EdgeGrad across all datasets.

Original: SINGLE - hops , which is more unnoticeable than attacking with a neighbor node, performs almost as good as SINGLE which attacks using a non-neighboring node, and better than EdgeGrad .
Revised: SINGLE - hops , which is more unnoticeable than attacking with a neighbor node, performs almost as good as SINGLE which attacks using a non-neighboring node, and better than EdgeGrad .

Original: On Twitter, SINGLE reduces the test accuracy signiﬁcantly better than EdgeGrad : 72.1% compared to 82.7%.
Revised: On Twitter, SINGLE reduces the test accuracy signiﬁcantly better than EdgeGrad : 72.1% compared to 82.7%.

Original: Results for targeted attacks are shown in Appendix A.3.
Revised: Results for targeted attacks are shown in Appendix A.3.

Original: Surprisingly, Table A.5 shows that Robust GCN (Z¨ugner & G¨unnemann, 2019) is as vulnerable to the SINGLE attack as a standard GCN, showing that there is still much room for novel ideas and improvements to the robustness of current GNNs.
Revised: Surprisingly, Table A.5 shows that Robust GCN (Z¨ugner & G¨unnemann, 2019) is as vulnerable to the SINGLE attack as a standard GCN, showing that there is still much room for novel ideas and improvements to the robustness of current GNNs.

Original: As we explain in Section 3.3, SINGLE tries to ﬁnd a perturbation vector in which the number of perturbed elements is minimal.
Revised: As we explain in Section 3.3, SINGLE tries to ﬁnd a perturbation vector in which the number of perturbed elements is minimal.

Original: We measured the number of vector elements that the attack had
Revised: We measured the number of vector elements that the attack had

Original: [Table]
Revised: [Table]

Original: perturbed in practice.
Revised: perturbed in practice.

Original: In PubMed, SINGLE used 76 vector elements on average, which are 15% of the elements in the feature vector.
Revised: In PubMed, SINGLE used 76 vector elements on average, which are 15% of the elements in the feature vector.

Original: In Cora, SINGLE perturbed 717 elements on average, which are 50%.
Revised: In Cora, SINGLE perturbed 717 elements on average, which are 50%.

Original: In CiteSeer, SINGLE used 1165 attributes on average, which are 31% of the features.
Revised: In CiteSeer, SINGLE used 1165 attributes on average, which are 31% of the features.

Original: In Twitter,
Revised: In Twitter,

Original: SINGLE used 892 attributes on average, which are 9% of the features.
Revised: SINGLE used 892 attributes on average, which are 9% of the features.

Original: In the experiments shown in Table 1, we used (cid:15) ∞
Revised: In the experiments shown in Table 1, we used (cid:15) ∞

Original: = 0 .
Revised: = 0 .

Original: 1 in the continuous datasets (PubMed and Twitter).
Revised: 1 in the continuous datasets (PubMed and Twitter).

Original: If we allow larger values of (cid:15) ∞ , we can reduce the number of perturbed vector elements: using (cid:15) ∞
Revised: If we allow larger values of (cid:15) ∞ , we can reduce the number of perturbed vector elements: using (cid:15) ∞

Original: = 0 .
Revised: = 0 .

Original: 5 requires perturbing only 3% of the attributes on average to achieve the same effectiveness; using (cid:15) ∞
Revised: 5 requires perturbing only 3% of the attributes on average to achieve the same effectiveness; using (cid:15) ∞

Original: =requires perturbing only 1.6% of the attributes on average to achieve the same effectiveness (in
Revised: =requires perturbing only 1.6% of the attributes on average to achieve the same effectiveness (in

Original: PubMed, where varying (cid:15) ∞ is meaningful).
Revised: PubMed, where varying (cid:15) ∞ is meaningful).

Original: 4. A TTACKER C HOICE
Revised: 4. A TTACKER C HOICE

Original: If the attacker could choose its node, e.g., by hijacking an existing account in a social network, could they increase the effectiveness of the attack?
Revised: If the attacker could choose its node, e.g., by hijacking an existing account in a social network, could they increase the effectiveness of the attack?

Original: We examine the effectiveness of two approaches for choosing the attacker node.
Revised: We examine the effectiveness of two approaches for choosing the attacker node.

Original: Gradient Attacker Choice ( GradChoice ) chooses the attacker node according to the largest gradient with respect to the node representations (for a non-targeted attack):
Revised: Gradient Attacker Choice ( GradChoice ) chooses the attacker node according to the largest gradient with respect to the node representations (for a non-targeted attack):

Original: [Equation]
Revised: [Equation]

Original: The chosen attacker node is never the victim node itself.
Revised: The chosen attacker node is never the victim node itself.

Original: Topological Attacker Choice ( Topology ) chooses the attacker node according to topological properties of the graph.
Revised: Topological Attacker Choice ( Topology ) chooses the attacker node according to topological properties of the graph.

Original: As an example, we choose the neighbor of the victim node v with the smallest number of neighbors: a ∗ = argmin a ∈N v |N a | .
Revised: As an example, we choose the neighbor of the victim node v with the smallest number of neighbors: a ∗ = argmin a ∈N v |N a | .

Original: The advantage of this approach is that the attacker choice is model-free : if the attacker cannot compute gradients, they can at least choose the most harmful attacker node, and then perform the perturbation itself using other non-gradient approaches such as ones proposed by Waniek et al.
Revised: The advantage of this approach is that the attacker choice is model-free : if the attacker cannot compute gradients, they can at least choose the most harmful attacker node, and then perform the perturbation itself using other non-gradient approaches such as ones proposed by Waniek et al.

Original: and Chang et al.
Revised: and Chang et al.

Original: To perform a fair comparison, we compare these approaches with GlobalEdgeGrad , which is similar to EdgeGrad that can insert or remove an edge, with the difference that the chosen edge can be chosen from the entire graph .
Revised: To perform a fair comparison, we compare these approaches with GlobalEdgeGrad , which is similar to EdgeGrad that can insert or remove an edge, with the difference that the chosen edge can be chosen from the entire graph .

Original: Results.
Revised: Results.

Original: Results for these attacker choice approaches are shown in Table 2.
Revised: Results for these attacker choice approaches are shown in Table 2.

Original: The main results are that choosing the attacker node signiﬁcantly increases the effectiveness of the SINGLE attack: for example, in Twitter, from 72.1% (Table 1) to 6.6% test accuracy (Table 2).
Revised: The main results are that choosing the attacker node signiﬁcantly increases the effectiveness of the SINGLE attack: for example, in Twitter, from 72.1% (Table 1) to 6.6% test accuracy (Table 2).

Original: In datasets where the given initial node features are continuous (PubMed and Twitter), SINGLE + Topology and SINGLE + GradChoice show similar results: on Twitter accuracy difference is less than 0.5%; on PubMed SINGLE + Topology outperforms SINGLE + GradChoice by ∼ 3%, even though SINGLE + Topology is model-free.
Revised: In datasets where the given initial node features are continuous (PubMed and Twitter), SINGLE + Topology and SINGLE + GradChoice show similar results: on Twitter accuracy difference is less than 0.5%; on PubMed SINGLE + Topology outperforms SINGLE + GradChoice by ∼ 3%, even though SINGLE + Topology is model-free.

Original: Both of those attacks are more efﬁcient than GlobalEdgeGrad , showing the superiority of node perturbation over edge perturbation in the global view.
Revised: Both of those attacks are more efﬁcient than GlobalEdgeGrad , showing the superiority of node perturbation over edge perturbation in the global view.

Original: In
Revised: In

Original: Appendix A.4, we show that allowing GlobalEdgeGrad to insert and remove multiple edges that belong to the same attacker node does not lead to a signiﬁcant improvement.
Revised: Appendix A.4, we show that allowing GlobalEdgeGrad to insert and remove multiple edges that belong to the same attacker node does not lead to a signiﬁcant improvement.

Original: Interestingly, GradChoice and Topology agree on the choice of attacker node for 50.3% of the nodes in Cora, 78.7% of the nodes in CiteSeer, 51.0% of the nodes in PubMed, and on 55.0% of the nodes in Twitter, showing that the node selection can sometimes be performed model-free.
Revised: Interestingly, GradChoice and Topology agree on the choice of attacker node for 50.3% of the nodes in Cora, 78.7% of the nodes in CiteSeer, 51.0% of the nodes in PubMed, and on 55.0% of the nodes in Twitter, showing that the node selection can sometimes be performed model-free.

Original: In datasets where the initial node features are discrete (Cora and CiteSeer), i.e., many-hot vectors,
Revised: In datasets where the initial node features are discrete (Cora and CiteSeer), i.e., many-hot vectors,

Original: GlobalEdgeGrad reduces the test accuracy more than GradChoice and Topology .
Revised: GlobalEdgeGrad reduces the test accuracy more than GradChoice and Topology .

Original: We believe that the reason is the difﬁculty of two-step optimization in discrete datasets: for example, GradChoice needs to choose the node, and ﬁnd the perturbation afterwards.
Revised: We believe that the reason is the difﬁculty of two-step optimization in discrete datasets: for example, GradChoice needs to choose the node, and ﬁnd the perturbation afterwards.

Original: Finding a perturbation for a discrete vector is more difﬁcult than in continuous datasets, and the choice of the attacker node may not be optimal.
Revised: Finding a perturbation for a discrete vector is more difﬁcult than in continuous datasets, and the choice of the attacker node may not be optimal.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4. S CENARIO A BLATION
Revised: 4. S CENARIO A BLATION

Original: The main scenario that we focus on in this paper is a SINGLE approach that always perturbs a single node, which is not the victim node ( a (cid:54) = v ).
Revised: The main scenario that we focus on in this paper is a SINGLE approach that always perturbs a single node, which is not the victim node ( a (cid:54) = v ).

Original: We now examine our SINGLE attack in other, easier but less realistic, scenarios: SINGLE - two attackers follows Z ¨ ugner et al.
Revised: We now examine our SINGLE attack in other, easier but less realistic, scenarios: SINGLE - two attackers follows Z ¨ ugner et al.

Original: (2018) and Zang et al.
Revised: (2018) and Zang et al.

Original: (2020), randomly samples two attacker nodes and perturbs their features using the same approach as SINGLE .
Revised: (2020), randomly samples two attacker nodes and perturbs their features using the same approach as SINGLE .

Original: SINGLE - direct perturbs the victim node directly (i.e., a = v ), an approach that was found to be the most efﬁcient by Z¨ugner et al.
Revised: SINGLE - direct perturbs the victim node directly (i.e., a = v ), an approach that was found to be the most efﬁcient by Z¨ugner et al.

Original: Table 3 shows the test accuracy of these ablations.
Revised: Table 3 shows the test accuracy of these ablations.

Original: In
Revised: In

Original: Appendix A.5.3, we additionally experiment with more than two attacker nodes.
Revised: Appendix A.5.3, we additionally experiment with more than two attacker nodes.

Original: A DVERSARIAL T RAINING
Revised: A DVERSARIAL T RAINING

Original: In the previous sections, we studied the effectiveness of the SINGLE attack.
Revised: In the previous sections, we studied the effectiveness of the SINGLE attack.

Original: In this section, we investigate to what extent can adversarial training (Madry et al., 2018) defend against SINGLE .
Revised: In this section, we investigate to what extent can adversarial training (Madry et al., 2018) defend against SINGLE .

Original: For each training step and labeled training node, we perform K train adversarial steps to adversarially perturb another randomly sampled node, exactly as in SINGLE , but at training time.
Revised: For each training step and labeled training node, we perform K train adversarial steps to adversarially perturb another randomly sampled node, exactly as in SINGLE , but at training time.

Original: The model is then trained to minimize the original cross-entropy loss and the adversarial loss:
Revised: The model is then trained to minimize the original cross-entropy loss and the adversarial loss:

Original: [Equation]
Revised: [Equation]

Original: The main difference from Equation (3) is the adversarial term J (cid:0) f θ (cid:0) G x ai + η i , v i (cid:1) , y i (cid:1) , where a i is the randomly sampled attacker for the node v i .
Revised: The main difference from Equation (3) is the adversarial term J (cid:0) f θ (cid:0) G x ai + η i , v i (cid:1) , y i (cid:1) , where a i is the randomly sampled attacker for the node v i .

Original: In every training step, we randomly sample a new attacker for each victim node and compute new η i vectors.
Revised: In every training step, we randomly sample a new attacker for each victim node and compute new η i vectors.

Original: After the model is trained, we attack the model with K test SINGLE adversarial steps.
Revised: After the model is trained, we attack the model with K test SINGLE adversarial steps.

Original: This is similar to Feng et al.
Revised: This is similar to Feng et al.

Original: and
Revised: and

Original: Deng et al.
Revised: Deng et al.

Original: (2019), except that they used adversarial training as a regularizer, to improve the accuracy of a model while not under attack.
Revised: (2019), except that they used adversarial training as a regularizer, to improve the accuracy of a model while not under attack.

Original: In contrast, we use adversarial training to defend a model against an attack at test time.
Revised: In contrast, we use adversarial training to defend a model against an attack at test time.

Original: We used K train = 5 , as we found it to be the maximal value for which the model’s accuracy is not signiﬁcantly hurt while not under attack (“clean”), and K test = 20 as in the previous experiments.
Revised: We used K train = 5 , as we found it to be the maximal value for which the model’s accuracy is not signiﬁcantly hurt while not under attack (“clean”), and K test = 20 as in the previous experiments.

Original: As shown in Table 4, adversarial training indeed improves the model’s robustness against the different
Revised: As shown in Table 4, adversarial training indeed improves the model’s robustness against the different

Original: SINGLE attacks.
Revised: SINGLE attacks.

Original: However, the main result of this section is that SINGLE , SINGLE + GradChoice and SINGLE + Topology are still very effective attacks, as they succeed in attacking the adversarially trained model, reducing its test accuracy to 58.5%, 30.6% and 21.1%, respectively.
Revised: However, the main result of this section is that SINGLE , SINGLE + GradChoice and SINGLE + Topology are still very effective attacks, as they succeed in attacking the adversarially trained model, reducing its test accuracy to 58.5%, 30.6% and 21.1%, respectively.

Original: 4. S ENSITIVITY TO (cid:15) ∞
Revised: 4. S ENSITIVITY TO (cid:15) ∞

Original: How does the intensity of the adversarial perturbation affect the performance of the attack?
Revised: How does the intensity of the adversarial perturbation affect the performance of the attack?

Original: Intuitively, we say that the less we restrict the perturbation (i.e., larger values of (cid:15) ∞ ), the more powerful the attack.
Revised: Intuitively, we say that the less we restrict the perturbation (i.e., larger values of (cid:15) ∞ ), the more powerful the attack.

Original: We examine whether this holds in practice.
Revised: We examine whether this holds in practice.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: In our experiments in Sections 4.1 to 4.4, we used (cid:15) ∞
Revised: In our experiments in Sections 4.1 to 4.4, we used (cid:15) ∞

Original: = 0 .
Revised: = 0 .

Original: 1 for the continuous datasets (PubMed and Twitter).
Revised: 1 for the continuous datasets (PubMed and Twitter).

Original: In this section, we vary the value of (cid:15) ∞ across different GNN types and observe the effectiveness of the attack.
Revised: In this section, we vary the value of (cid:15) ∞ across different GNN types and observe the effectiveness of the attack.

Original: Figure 2 shows the results on PubMed.
Revised: Figure 2 shows the results on PubMed.

Original: We used this dataset because it is larger than Cora and CiteSeer (Appendix A.1), and most importantly, its features are continuous, thus real-valued perturbations are feasible.
Revised: We used this dataset because it is larger than Cora and CiteSeer (Appendix A.1), and most importantly, its features are continuous, thus real-valued perturbations are feasible.

Original: As shown in Figure 2, the most signiﬁcant difference is between performing the perturbation ( (cid:15) ∞
Revised: As shown in Figure 2, the most signiﬁcant difference is between performing the perturbation ( (cid:15) ∞

Original: = 0 . 1 ) and not attacking at all ( (cid:15) ∞
Revised: = 0 . 1 ) and not attacking at all ( (cid:15) ∞

Original: = 0 ).
Revised: = 0 ).

Original: As we increase the value of (cid:15) ∞ , GCN and GraphSage (Hamilton et al., 2017) show a natural descent in test accuracy.
Revised: As we increase the value of (cid:15) ∞ , GCN and GraphSage (Hamilton et al., 2017) show a natural descent in test accuracy.

Original: Contrarily, GAT (Veli ˇ ckovi ´ c et al., 2018) and GIN (Xu et al., 2019b) are more robust to increased absolute values of perturbations, while GAT is also the most robust compared to the other GNN types.
Revised: Contrarily, GAT (Veli ˇ ckovi ´ c et al., 2018) and GIN (Xu et al., 2019b) are more robust to increased absolute values of perturbations, while GAT is also the most robust compared to the other GNN types.

Original: 4. D ISTANCE B ETWEEN A TTACKER AND V ICTIM
Revised: 4. D ISTANCE B ETWEEN A TTACKER AND V ICTIM

Original: In Section 4.1, we found that SINGLE performs similarly to SINGLE -hops, although SINGLE -hops samples an attacker node a whose distance from the victim node v is at least 2.
Revised: In Section 4.1, we found that SINGLE performs similarly to SINGLE -hops, although SINGLE -hops samples an attacker node a whose distance from the victim node v is at least 2.

Original: We further question whether the effectiveness of the attack depend on the distance in the graph between the attacker and the victim.
Revised: We further question whether the effectiveness of the attack depend on the distance in the graph between the attacker and the victim.

Original: We trained a new model for each dataset using L = 8 layers.
Revised: We trained a new model for each dataset using L = 8 layers.

Original: Then, for each test victim node, we sampled attackers according to their distance to the test node.
Revised: Then, for each test victim node, we sampled attackers according to their distance to the test node.

Original: As shown in Figure 3, the effectiveness of the attack increases as the distance between the attacker and the victim decreases.
Revised: As shown in Figure 3, the effectiveness of the attack increases as the distance between the attacker and the victim decreases.

Original: At distance of 5, the curve seems to saturate.
Revised: At distance of 5, the curve seems to saturate.

Original: A possible explanation for this is that apparently more than few layers (e.g., L = 2 in Kipf & Welling (2017)) are not needed in most datasets.
Revised: A possible explanation for this is that apparently more than few layers (e.g., L = 2 in Kipf & Welling (2017)) are not needed in most datasets.

Original: Thus, the rest of the layers can theoretically learn not to pass much of their input starting from the redundant layers, excluding adversarial signals as well.
Revised: Thus, the rest of the layers can theoretically learn not to pass much of their input starting from the redundant layers, excluding adversarial signals as well.

Original:  R E L AT E D W O R K
Revised:  R E L AT E D W O R K

Original: Works on adversarial attacks on GNN differ in several main aspects.
Revised: Works on adversarial attacks on GNN differ in several main aspects.

Original: In this section, we discuss the main criteria, to clarify the settings that we address.
Revised: In this section, we discuss the main criteria, to clarify the settings that we address.

Original: Single vs. multiple attackers All previous works allowed perturbing multiple nodes, or edges that are covered by multiple nodes: Z¨ugner et al.
Revised: Single vs. multiple attackers All previous works allowed perturbing multiple nodes, or edges that are covered by multiple nodes: Z¨ugner et al.

Original: (2018) perturb features of a set of attacker nodes; Zang et al.
Revised: (2018) perturb features of a set of attacker nodes; Zang et al.

Original: (2020) assume “a few bad actors”; other works perturb edges that in realistic settings their perturbation would require controlling multiple nodes (Bojchevski & G ¨ unnemann, 2019a; Sun et al., 2020; Chen et al., 2018).
Revised: (2020) assume “a few bad actors”; other works perturb edges that in realistic settings their perturbation would require controlling multiple nodes (Bojchevski & G ¨ unnemann, 2019a; Sun et al., 2020; Chen et al., 2018).

Original: Node vs. edge perturbations Most adversarial attacks on GNNs perturb the input graph by modifying the graph structure (Z¨ugner & G¨unnemann, 2019; Wang et al., 2020; Xu et al., 2019a).
Revised: Node vs. edge perturbations Most adversarial attacks on GNNs perturb the input graph by modifying the graph structure (Z¨ugner & G¨unnemann, 2019; Wang et al., 2020; Xu et al., 2019a).

Original: For example,
Revised: For example,

Original: Dai et al.
Revised: Dai et al.

Original: (2018) iteratively remove edges, yet their attack manages to reduce the accuracy by about 10% at most when perturbing a single edge.
Revised: (2018) iteratively remove edges, yet their attack manages to reduce the accuracy by about 10% at most when perturbing a single edge.

Original: Li et al.
Revised: Li et al.

Original: (2020) also allow the insertion of edges; Waniek et al.
Revised: (2020) also allow the insertion of edges; Waniek et al.

Original: and Chang et al.
Revised: and Chang et al.

Original: (2020) allow insertion and deletion of edges, using attacks that are based on correlations and eigenvalues, and not on gradients.
Revised: (2020) allow insertion and deletion of edges, using attacks that are based on correlations and eigenvalues, and not on gradients.

Original: Yefet et al.
Revised: Yefet et al.

Original: (2019) perturb one-hot node vectors, in the restricted domain of computer programs.
Revised: (2019) perturb one-hot node vectors, in the restricted domain of computer programs.

Original: Z¨ugner et al.
Revised: Z¨ugner et al.

Original: (2018) and Wu et al.
Revised: (2018) and Wu et al.

Original: (2019b) perturb both edges and nodes; but they concluded that perturbing edges is more effective than perturbing nodes.
Revised: (2019b) perturb both edges and nodes; but they concluded that perturbing edges is more effective than perturbing nodes.

Original: In this work, we counter these conclusions and show that perturbing node features is more effective than perturbing edges.
Revised: In this work, we counter these conclusions and show that perturbing node features is more effective than perturbing edges.

Original: Direct vs. inﬂuence attacks Another difference between prior works lies in the difference between direct attacks and inﬂuence attacks .
Revised: Direct vs. inﬂuence attacks Another difference between prior works lies in the difference between direct attacks and inﬂuence attacks .

Original: In direct attacks, the attacker perturbs the target node itself .
Revised: In direct attacks, the attacker perturbs the target node itself .

Original: For example, the attack of Z¨ugner et al.
Revised: For example, the attack of Z¨ugner et al.

Original: (2018) is the most effective when the attacker and the target are the same node .
Revised: (2018) is the most effective when the attacker and the target are the same node .

Original: In inﬂuence attacks, the perturbed nodes are at least one hop away from the victim node.
Revised: In inﬂuence attacks, the perturbed nodes are at least one hop away from the victim node.

Original: In this paper, we show that the strong direct assumption is not required ( SINGLE - direct in
Revised: In this paper, we show that the strong direct assumption is not required ( SINGLE - direct in

Original: Section 4.2), and that our attack is effective when the attacker and the target are not even direct neighbors , i.e., they are at least two hops away ( SINGLE - hops in Section 4.1).
Revised: Section 4.2), and that our attack is effective when the attacker and the target are not even direct neighbors , i.e., they are at least two hops away ( SINGLE - hops in Section 4.1).

Original: Poisoning vs. evasion attacks In a related scenario, some work (Z¨ugner & G¨unnemann, 2019;
Revised: Poisoning vs. evasion attacks In a related scenario, some work (Z¨ugner & G¨unnemann, 2019;

Original: Bojchevski & G¨unnemann, 2019a; Li et al., 2020; Zhang & Zitnik, 2020) focuses on poisoning attacks that perturb examples before training.
Revised: Bojchevski & G¨unnemann, 2019a; Li et al., 2020; Zhang & Zitnik, 2020) focuses on poisoning attacks that perturb examples before training.

Original: Contrarily, we focus on the standard evasion scenario of adversarial examples in neural networks (Szegedy et al., 2013; Goodfellow et al., 2014), where the attack operates at test time, after the model was trained, as Dai et al.
Revised: Contrarily, we focus on the standard evasion scenario of adversarial examples in neural networks (Szegedy et al., 2013; Goodfellow et al., 2014), where the attack operates at test time, after the model was trained, as Dai et al.

Original: Attacking vs. certifying Z¨ugner & G¨unnemann (2020) focus on certifying the robustness of GNNs against adversarial perturbations; and Bojchevski & G ¨ unnemann (2019b) certiﬁed PageRank-style models.
Revised: Attacking vs. certifying Z¨ugner & G¨unnemann (2020) focus on certifying the robustness of GNNs against adversarial perturbations; and Bojchevski & G ¨ unnemann (2019b) certiﬁed PageRank-style models.

Original: In contrast, we study the effectiveness of the adversarial attack itself.
Revised: In contrast, we study the effectiveness of the adversarial attack itself.

Original:  C O N C L U S I O N
Revised:  C O N C L U S I O N

Original: We demonstrate that GNNs are susceptible even to the extremely limited scenario of a single-node indirect adversarial example ( SINGLE ).
Revised: We demonstrate that GNNs are susceptible even to the extremely limited scenario of a single-node indirect adversarial example ( SINGLE ).

Original: The practical consequences of these ﬁndings are that a single attacker in a network can force a GNN to classify any other target node as the attacker’s chosen label, by slightly perturbing some of the attacker’s features.
Revised: The practical consequences of these ﬁndings are that a single attacker in a network can force a GNN to classify any other target node as the attacker’s chosen label, by slightly perturbing some of the attacker’s features.

Original: We further show that if the attacker can choose its attacker node – the effectiveness of the attack increases signiﬁcantly.
Revised: We further show that if the attacker can choose its attacker node – the effectiveness of the attack increases signiﬁcantly.

Original: We study the effectiveness of these attacks across various GNN types and datasets.
Revised: We study the effectiveness of these attacks across various GNN types and datasets.

Original: We believe that this work will drive research in this ﬁeld toward exploring novel defense approaches for GNNs.
Revised: We believe that this work will drive research in this ﬁeld toward exploring novel defense approaches for GNNs.

Original: Such defenses can be crucial for real-world systems that are modeled using GNNs.
Revised: Such defenses can be crucial for real-world systems that are modeled using GNNs.

Original: Furthermore, we believe that the surprising results of this work motivate better theoretical understanding of the expressiveness and generalization of GNNs.
Revised: Furthermore, we believe that the surprising results of this work motivate better theoretical understanding of the expressiveness and generalization of GNNs.

Original: To these ends, we make all our code and trained models publicly available.
Revised: To these ends, we make all our code and trained models publicly available.

Original: A
Revised: A

Original: S U P P L E M E N TA RY M AT E R I
Revised: S U P P L E M E N TA RY M AT E R I

Original: A L
Revised: A L

Original: A. D ATASET S TATISTICS
Revised: A. D ATASET S TATISTICS

Original: Statistics of the datasets are shown in Table A.1.
Revised: Statistics of the datasets are shown in Table A.1.

Original: [Table]
Revised: [Table]

Original: A. A DDITIONAL GNN T YPES
Revised: A. A DDITIONAL GNN T YPES

Original: Tables A.2 to A.4 present the test accuracy of different attacks applied on GAT (Veli ˇ ckovi ´
Revised: Tables A.2 to A.4 present the test accuracy of different attacks applied on GAT (Veli ˇ ckovi ´

Original: c et al., 2018),
Revised: c et al., 2018),

Original: GIN (Xu et al., 2019b), GraphSAGE (Hamilton et al., 2017), RobustGCN (Z ¨ ugner & G ¨ unnemann, 2019)
Revised: GIN (Xu et al., 2019b), GraphSAGE (Hamilton et al., 2017), RobustGCN (Z ¨ ugner & G ¨ unnemann, 2019)

Original: [Table]
Revised: [Table]

Original: and SGC (Wu et al., 2019a), showing the effectiveness of SINGLE across different GNN types.
Revised: and SGC (Wu et al., 2019a), showing the effectiveness of SINGLE across different GNN types.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Surprisingly, Table A.5 shows that Robust GCN (Z¨ugner & G¨unnemann, 2019) is as vulnerable to the SINGLE attack as a standard GCN
Revised: Surprisingly, Table A.5 shows that Robust GCN (Z¨ugner & G¨unnemann, 2019) is as vulnerable to the SINGLE attack as a standard GCN

Original: [Table]
Revised: [Table]

Original: showing that there is still much room for novel ideas and improvements to the robustness of current GNNs.
Revised: showing that there is still much room for novel ideas and improvements to the robustness of current GNNs.

Original: A. T ARGETED A TTACKS
Revised: A. T ARGETED A TTACKS

Original: Tables A.6 to A.9 show the results of targeted attacks across datasets and approaches.
Revised: Tables A.6 to A.9 show the results of targeted attacks across datasets and approaches.

Original: Differently from other tables which show test accuracy, Tables A.6 to A.9 present the targeted attack’s success rate , which is the fraction of test examples that the attack managed to force a speciﬁc label prediction
Revised: Differently from other tables which show test accuracy, Tables A.6 to A.9 present the targeted attack’s success rate , which is the fraction of test examples that the attack managed to force a speciﬁc label prediction

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: of GCN, Robust GCN (Z ¨ ugner & G ¨ unnemann, 2019), and SGC (Wu et al., 2019a)
Revised: of GCN, Robust GCN (Z ¨ ugner & G ¨ unnemann, 2019), and SGC (Wu et al., 2019a)

Original: under different non-targeted attacks, on PubMed.
Revised: under different non-targeted attacks, on PubMed.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A. M ULTI E DGE ATTACKS
Revised: A. M ULTI E DGE ATTACKS

Original: We strengthened the EdgeGrad attack by allowing it to add and remove multiple edges that are connected to the attacker node – MultiEdgeGrad .
Revised: We strengthened the EdgeGrad attack by allowing it to add and remove multiple edges that are connected to the attacker node – MultiEdgeGrad .

Original: Accordingly, MultiGlobalEdgeGrad is equivalent to GlobalEdgeGrad , except that MultiGlobalEdgeGrad can choose the attacker node.
Revised: Accordingly, MultiGlobalEdgeGrad is equivalent to GlobalEdgeGrad , except that MultiGlobalEdgeGrad can choose the attacker node.

Original: [Table]
Revised: [Table]

Original: As shown in Table A.10, allowing the attacker node to add and remove multiple edges ( MultiEdgeGrad and MultiGlobalEdgeGrad ) results in a very minor improvement compared to EdgeGrad and
Revised: As shown in Table A.10, allowing the attacker node to add and remove multiple edges ( MultiEdgeGrad and MultiGlobalEdgeGrad ) results in a very minor improvement compared to EdgeGrad and

Original: GlobalEdgeGrad , while SINGLE , SINGLE + Topology and SINGLE + GradChoice are much more effective.
Revised: GlobalEdgeGrad , while SINGLE , SINGLE + Topology and SINGLE + GradChoice are much more effective.

Original: A. A DDITIONAL B ASELINES A.5. ZERO - FEATURES A PPROACH
Revised: A. A DDITIONAL B ASELINES A.5. ZERO - FEATURES A PPROACH

Original: We experimented with a baseline where we set η = − x a as the feature perturbation.
Revised: We experimented with a baseline where we set η = − x a as the feature perturbation.

Original: The objective of experimenting with such an attack is to illustrate that SINGLE can ﬁnd better perturbations than simply canceling the node feature vector, making the new vector a vector of zeros (and thus effectively removes the edges of the attacker node in GCN).
Revised: The objective of experimenting with such an attack is to illustrate that SINGLE can ﬁnd better perturbations than simply canceling the node feature vector, making the new vector a vector of zeros (and thus effectively removes the edges of the attacker node in GCN).

Original: As shown, Zero features is barely effective (compared to “Clean”), and SINGLE can ﬁnd much better perturbations.
Revised: As shown, Zero features is barely effective (compared to “Clean”), and SINGLE can ﬁnd much better perturbations.

Original: [Table]
Revised: [Table]

Original: A.5. I NJECTION ATTACKS
Revised: A.5. I NJECTION ATTACKS

Original: We also study an additional type of a realistic attack that is based on node injection.
Revised: We also study an additional type of a realistic attack that is based on node injection.

Original: In this approach, we insert a new node to the graph with a single edge attached to our victim node.
Revised: In this approach, we insert a new node to the graph with a single edge attached to our victim node.

Original: The attack is performed by perturbing the injected node’s attributes.
Revised: The attack is performed by perturbing the injected node’s attributes.

Original: Since there is no initial node feature vector to measure the (cid:15) ∞ distance to, the injected node is allowed to ﬁnd any realistic representation (e.g., without choosing negative frequencies).
Revised: Since there is no initial node feature vector to measure the (cid:15) ∞ distance to, the injected node is allowed to ﬁnd any realistic representation (e.g., without choosing negative frequencies).

Original: This attack is very powerful, reducing the test accuracy down to 0.02% on PubMed.
Revised: This attack is very powerful, reducing the test accuracy down to 0.02% on PubMed.

Original: A.5. L ARGER NUMBER OF ATTACKERS
Revised: A.5. L ARGER NUMBER OF ATTACKERS

Original: [Table]
Revised: [Table]

Original: We performed additional experiments with up to ﬁve randomly sampled attacker nodes simultaneously (Table A.12).
Revised: We performed additional experiments with up to ﬁve randomly sampled attacker nodes simultaneously (Table A.12).

Original: As expected, allowing a larger number of attackers reduces the test accuracy.
Revised: As expected, allowing a larger number of attackers reduces the test accuracy.

Original: However, the main observation in this paper is that even a single attacker node is surprisingly effective.
Revised: However, the main observation in this paper is that even a single attacker node is surprisingly effective.

Revised: A. L IMITING THE ALLOWED (cid:15)

Revised: In Section 4.5, we analyzed the effect of the value of (cid:15) ∞ , that is, the maximal allowed perturbation in each vector attribute, on the performance of the attack.

Revised: However, in datasets such as Cora and

Revised: CiteSeer, the input features are binary (i.e., the input node vector is many-hot), so the possible perturbation to each vector element is only “ﬂipping” its value from zero to one, or vice-versa.

Revised: Thus, in these datasets, it is interesting to analyze the value of (cid:15) 0 , the maximal number of allowed perturbed vector elements, on the performance of the attack.

Original: {+In this case, measuring the l 0 norm is equivalent to measuring the l 1 norm: (cid:107) η (cid:107) 
+}
Revised: {+In this case, measuring the l 0 norm is equivalent to measuring the l 1 norm: (cid:107) η (cid:107) 
+}

Revised: = (cid:107) η (cid:107) 1 , and is proportional to the l 2 norm.

Revised: We performed experiments where we measured the test accuracy of the model while limiting the number of allowed perturbed vector elements.

Revised: The results are shown in Figure A.1.

Revised: As shown, when (cid:15) 0 = 0 , no attack is allowed, and the test accuracy is equal to the “Clean” value of Table 1.

Original: {+When (cid:15) 
+}
Revised: {+When (cid:15) 
+}

Revised: = 100% , the results are equal to the SINGLE values of Table 1 – resulting in ﬂipping 50% of the features on average in Cora, and 31% of the features on average in CiteSeer.

Revised: It is important to note that in practice, the average number of perturbed features is much lower than the maximal number of allowed features.

Revised: For example, in CiteSeer, allowing 100% of the features results in actually using only 31% on average.

Revised: [Figure]

Revised: Maximal fraction of perturbed features

Revised: Acc

Revised: [Figure]

Revised: CiteSeer - SINGLE

Revised: CiteSeer - SINGLE+GradChoice