Attacking Graph Convolutional Networks via Rewiring

1 INTRODUCTION . Graph structured data are ubiquitous in many real world applications . Various data from different domains , such as social networks , molecular graphs and transportation networks can all be modeled as graphs . Recently , increasing effort has been devoted towards developing deep neural networks on graph structured data . This stream of works , which is known as Graph Neural Networks ( GNN ) has shown to enhance the performance in many graph related tasks such as node classification ( Kipf & Welling , 2016 ; Hamilton et al. , 2017 ) and graph classification ( Bruna et al. , 2013 ; Defferrard et al. , 2016 ; Ying et al. , 2018 ; Zhang et al. , 2018 ) . Recent researches have shown that deep neural networks are highly vulnerable to adversarial attacks ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Kurakin et al. , 2016 ; Carlini & Wagner , 2017 ) . In computer vision , performing an adversarial attack is to add deliberately created , but unnoticeable , perturbation to a given image such that the deep model misclassifies the perturbed image . Unlike image data , which can be represented in the continuous space , graph structured data is discrete . Few efforts have been made to investigate the robustness of graph neural networks against adversarial attacks . Only recently , such researches about adversarial attacks on graph structured data started to emerge . A greedy algorithm is proposed to attack the semi-supervised node classification task ( Zügner et al. , 2018 ) . The method deliberately tries to modify the graph structure and node features such that the label of a targeted node can be changed . A reinforcement learning based algorithm is proposed to attack both node classification and graph classification task by only modifying the graph structure ( Dai et al. , 2018 ) . A meta-learning based attack method is designed to impair the overall performance of the node classification task ( Zügner & Günnemann , 2019 ) . For the majority of existing works , the graph structure is modified by adding or deleting edges . To ensure that the difference between the attacked graph and the original graph is “ unnoticeable ” , the number of actions ( adding/deleting edges ) that can be taken by the attacking algorithms is usually constrained by a budget . However , even when this budget is small , adding or deleting edges can still make “ noticeable ” changes to the graph structure ( Miller et al. , 2019 ) . For example , it is evident that many important graph properties are based on eigenvalues and eigenvectors of the Laplacian matrix of the graph ( Chan & Akoglu , 2016 ) ; while adding or deleting an edge can make remarkable changes on the eigenvalues/eigenvectors of the graph Laplacian ( Ghosh & Boyd , 2006 ) . Thus , in this work , we propose a new operation based on graph rewiring . A single rewiring operation involves three nodes ( vfir , vsec , vthi ) , where we remove the existing edge between vfir and vsec and add edge between vfir and vthi . Note that vthi is constraint to be the 2-hop neighbor of vfir in our setting . It is obvious that the proposed rewiring operation preserves some basic properties of the graph such as number of nodes and edges , total degrees of the graph and etc , while operations like adding and deleting edges can not . Furthermore , the proposed rewiring operation affects some of the important measures based on graph Laplacian such as algebraic connectivity in a smaller way than adding/deleting edges , which is theoretically demonstrated in Section 4.1 . In addition , the rewiring operation is a more natural way to modify the graph . For example , in biology , the evolution of DNA and amino acid sequences can lead to pervasive rewiring of protein–protein interactions ( Zitnik et al. , 2019 ) . In this paper , we aim to construct adversarial examples by performing rewiring operations for the task of graph classification . More specifically , we treat the process of applying a series of rewiring operations to a given graph as a discrete Markov decision process ( MDP ) and use reinforcement learning to learn how to make these decisions . We demonstrate the effectiveness of the proposed algorithm on real-world graphs . Then we further analyze how the adversarial changes in the graph structure affect both the graph embedding learned by the graph neural network model and the output label and illustrate the advantages of the rewiring operation . 2 BACKGROUND . In this section , we introduce notations and the target graph convolutional model we seek to attack . We denote a graph as G = { V , E } , where V = { v1 , . . . , v|V| } and E = { e1 , . . . , e|E| } are the sets of nodes and edges , respectively . The edges describe the relations between nodes , which can be described by an adjacency matrix A ∈ { 0 , 1 } |V|×|V| . Aij = 1 means vi and vj are connected , 0 otherwise . Each node in the graph has some features that are associated with it . These features are represented as a matrix X ∈ R|V|×d , where the i-th row of X denotes the node features of node vi and d is the dimension of features . Thus , an attributed graph can be represented as G = { A , X } . 2.1 GRAPH CLASSIFICATION . In the setting of graph classification , we are given a set of graphs G = { Gi } . Each of these graphs Gi is associated with a label yi . The task is to build a good classifier using the given set of graphs such that it can make correct predictions when new unseen graphs are fed into it . A graph classifier parameterized by θ can be represented as f ( G|θ ) = yo , where yo denotes the label of a graph G ∈ G predicted by the classifier . The parameters θ in the classifier f ( ·|θ ) can be learned by solving the following optimization problem minθ ∑ i L ( f ( Gi|θ ) , yi ) , where L ( · , · ) is used to measure the difference between the predicted and ground truth labels . Cross entropy is a commonly adopted measurement for L ( · , · ) . 2.2 GRAPH CONVOLUTION NETWORKS . Recently , Graph Neural Networks have been shown to be effective in graph representation learning . These models usually learn node representations by iteratively aggregating , transforming and propagating node information . In this work , we adopt the graph convolutional networks ( GCN ) ( Kipf & Welling , 2016 ) . A graph convolutional layer in the GCN framework can be represented as Fj = ReLU ( D− 1 2AD− 1 2Fj−1Wj ) ( 1 ) where Fj ∈ RN×dj is the output of the j-th layer and Wj represents the parameters of this layer . A GCN model usually consists of J graph convolutional layers , with F0 = X . The output of the GCN model is FJ , which is denote as F for convenience . To obtain a graph level embedding uG for graph G to perform graph classification , we apply a global pooling over the node embeddings . uG = pool ( F ) ( 2 ) Different global pooling functions can be used , and we adopt the max pooling in this work . A multilayer perceptron ( MLP ) and softmax layer are then sequentially applied on the graph embedding to predict the label of the graph yo = argmax softmax ( MLP ( uG|WMLP ) ) ( 3 ) where MLP ( ·|WMLP ) denotes the multilayer perceptron with parameters as WMLP . A GCNbased classifier for graph classification can be described using eq . equation 1 , equation 2 and equation 3 as introduced above . For simplicity , we summarize it as yo = fGCN ( G|θGCN ) , where θGCN includes all the parameters in the model . 3 PROBLEM FORMULATION . In this work , we aim to build an attacker T that takes a graph as input and modify the structure of the graph to fool a GCN classifier . Modifying a graph structure is equivalent to modify its adjacency matrix . The function of the attacker can be represented as G̃ = T ( G ) = { T ( A ) , X } = { Ã , X } . Given a classifier f ( · ) , the goal of the attacker is to modify the graph structure so that the classifier outputs a different label from what it originally predicted . Note here , we neglect the θ inside f ( · ) , as the classifier is already trained and fixed . Mathematically , the goal of the attacker can be represented as : f ( T ( G ) ) 6= f ( G ) . As described above , the attacker T is specifically designed for a given classifier f ( · ) . To reflect this in the notation , we now denote the attacker for the classifier f ( · ) as Tf . In our work , the attacker Tf has limited knowledge of the classifier . The only information the attacker can get from the classifier is the label of ( modified ) graphs . In other words , the classifier f ( · ) is treated as a black-box model for the attacker Tf . An important constraint to the attacker Tf is that it is only allowed to make “ unnoticeble ” changes to the graph structure . To account for this , we propose the rewiring operation , which is supposed to make more subtle changes than adding or deleting edges . We will show that the rewiring operation can better preserve a lot of important properties of the graph compared to adding or deleting edges in Section 4.1 . We also empirically compare the rewiring operation with the deleting/adding edges in Appendix C. The definition of the proposed rewiring is given below : Definition 1 . A rewiring operation a involves three nodes and it can be denoted as a = { vfir , vsec , vthi } , where vsec ∈ N1 ( vfir ) and vthi ∈ N2 ( vfir ) /N1 ( vfir ) . Nk ( vfir ) denotes the k-th hop neighbors of vfir and the sign / stands for exclusion . The rewiring operation deletes the existing edge between nodes vfir and vsec , while adding an edge to connect nodes vfir and vthi . The attacker Tf is given a budget of K proposed rewiring operations to modify the graph structure . A straightforward way to set K is choosing a small fix number . However , it is likely that graphs in a given data set have various graph sizes . The same number of rewiring operations can affect the graphs of different size in various magnitude . Hence , a more suitable way is to allow flexible number of rewiring operations according to the graph size . Thus , we propose to use K = p · |E| for a given graph G , where p ∈ ( 0 , 1 ) is a ratio . The process of the attacker on a graph G can be now denoted as Tf ( G ) ↔ ( a1 , a2 , . . . , aM ) [ G ] , where the right hand part means to sequentially apply the rewiring operations a1 , . . . , aM to the graph G , and M is the number of rewiring operations taken with M ≤ K .

Summary: This paper proposes the ReWatt method to attack graph classification models by making unnoticeable perturbations on graph. Reinforcement learning was leveraged to find a rewiring operation a = (v1; v2; v3) at each step, which is a set of 3 nodes. In the first step, an existing edge (v1, v2) in the original graph is selected and removed. Then another node v3 that is 2-hop away from v1 and not 1-hop away is selected.  Finally (v3, v1) is connected as a new edge. Some analysis shows that the rewiring operation tends to make smaller changes to the eigenvalues of the graph's Laplacian matrix compared with simply adding and deleting edges, making it difficult to detect the attacks.