Adversarial Attacks on Graph Neural Networks via Meta Learning

1 INTRODUCTION . Graphs are a powerful representation that can model diverse data from virtually any domain , such as biology ( protein interaction networks ) , chemistry ( molecules ) , or social networks ( Facebook ) . Not surprisingly , machine learning on graph data has a longstanding history , with tasks ranging from node classification , over community detection , to generative modeling . In this paper , we study node classification , which is an instance of semi-supervised classification : given a single ( attributed ) network and a subset of nodes whose class labels are known ( e.g. , the topic of a paper in a citation graph ) , the goal is to infer the classes of the unlabeled nodes . While there exist many classical approaches to node classification ( London & Getoor , 2014 ; Chapelle et al. , 2006 ) , recently deep learning on graphs has gained much attention ( Monti et al. , 2017 ; Bojchevski & Günnemann , 2018a ; Battaglia et al. , 2018 ; Perozzi et al. , 2014 ; Bojchevski et al. , 2018 ; Klicpera et al. , 2019 ) . Specifically , graph convolutional approaches ( Kipf & Welling , 2017 ; Pham et al. , 2017 ) have improved the state of the art in node classification . However , recent works have also shown that such approaches are vulnerable to adversarial attacks both at test time ( evasion ) as well as training time ( poisoning attacks ) ( Zügner et al. , 2018 ; Dai et al. , 2018 ) . A core strength of models using graph convolution – exploiting the information in a node ’ s neighborhood to improve classification – is also a major vulnerability : because of these propagation effects , an attacker can change a single node ’ s prediction without even changing any of its attributes or edges . This is because the foundational assumption that all samples are independent of each other does not hold for node classification . Network effects such as homophily ( London & Getoor , 2014 ) support the classification , while on the other hand they enable indirect adversarial attacks . So far , all existing attacks on node classification models are targeted , that is , aim to provoke misclassification of a specific single node , e.g . a person in a social network . In this work , we propose the first algorithm for poisoning attacks that is able to compromise the global node classification performance of a model . We show that even under restrictive attack settings and without access to the target classifier , our attacks can render it near-useless for use in production ( i.e. , on test data ) . Our approach is based on the principle of meta learning , which has traditionally been used for hyperparameter optimization ( Bengio , 2000 ) , or , more recently , few-shot learning ( Finn et al. , 2017 ) . In essence , we turn the gradient-based optimization procedure of deep learning models upside down and treat the input data – the graph at hand – as a hyperparameter to learn . 2 RELATED WORK . Adversarial attacks on machine learning models have been studied both in the machine learning and security community and for many different model types ( Mei & Zhu , 2015 ) . It is important to distinguish attacks from outliers ; while the latter naturally occur in graphs ( Bojchevski & Günnemann , 2018 ) , adversarial examples are deliberately created with the goal to mislead machine learning models and often designed to be unnoticeable . Deep neural networks are highly sensitive to these small adversarial perturbations to the data ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) . The vast majority of attacks and defenses assume the data instances to be independent and continuous . This assumption clearly does not hold for node classification and many other tasks on graphs . Works on adversarial attacks for graph learning tasks are generally sparse . Chen et al . ( 2017 ) have measured the changes in the resulting graph clustering when injecting noise to a bi-partite graph that represent DNS queries . However , their focus is not on generating attacks in a principled way . Torkamani & Lowd ( 2013 ) consider adversarial noise in the node features in order to improve robustness of collective classification via associative Markov networks . Only recently researchers have started to study adversarial attacks on deep learning for graphs . Dai et al . ( 2018 ) consider test-time ( i.e. , evasion ) attacks on graph classification ( i.e. , classification of graphs themselves ) and node classification . However , they do not consider poisoning ( i.e. , trainingtime ) attacks or evaluate transferability of their attacks , and restrict the attacks to edge deletions only . Moreover , they focus on targeted attacks , i.e . attacks designed to change the prediction of a single node . Zügner et al . ( 2018 ) consider both test-time and training-time attacks on node classification models . They circumvent explicitly tackling the bilevel optimization problem underlying poisoning attacks by performing their attacks based on a ( static ) surrogate model and evaluating their impact by training a classifier on the data modified by their algorithm . In contrast to Dai et al . ( 2018 ) , their attacks can both insert and remove edges , as well as modify node attributes in the form of binary vectors . Again , their algorithm is suited only to targeted attacks on single nodes ; the problem of training-time attacks on the overall performance of node classification models remains unexplored . Bojchevski & Günnemann ( 2018b ) propose poisoning attacks on a different task : unsupervised node representation learning ( or node embeddings ) . They exploit perturbation theory to maximize the loss obtained after training DeepWalk . In this work , we focus on semi-supervised learning . Meta-learning ( Thrun & Pratt , 1998 ; Naik & Mammone , 1992 ) , or learning to learn , is the task of optimizing the learning algorithm itself ; e.g. , by optimizing the hyperparameters Bengio ( 2000 ) , learning to update the parameters of a neural network ( Schmidhuber , 1992 ; Bengio et al. , 1992 ) , or the activation function of a model ( Agostinelli et al. , 2014 ) . Gradient-based hyperparameter optimization works by differentiating the training phase of a model to obtain the gradients w.r.t . the hyperparameters to optimize . The key idea of this work is to use meta-learning for the opposite : modifying the training data to worsen the performance after training ( i.e. , training-time or poisoning attacks ) . Muñoz-González et al . ( 2017 ) demonstrate that meta learning can indeed be used to create training-time attacks on simple , linear classification models . On continuous data , they report little success when attacking deep neural networks , and on discrete datasets , they do not consider deep learning models or problems with more than two classes . Like most works on adversarial attacks , they assume the data instances to be independent . In this work , for the first time , we propose an algorithm for global attacks on ( deep ) node classification models at training time . In contrast to Zügner et al . ( 2018 ) , we explicitly tackle the bilevel optimization problem of poisoning attacks using meta learning . 3 PROBLEM FORMULATION . We consider the task of ( semi-supervised ) node classification . Given a single ( attributed ) graph and a set of labeled nodes , the goal is to infer the class labels of the unlabeled nodes . Formally , let G = ( A , X ) be an attributed graph with adjacency matrix A ∈ { 0 , 1 } N×N and node attribute matrix X ∈ RN×D , where N is the number of nodes and D the dimension of the node feature vectors . W.l.o.g. , we assume the node IDs to be V = { 1 , . . . , N } . Given the set of labeled nodes VL ⊆ V , where nodes are assigned exactly one class in C = { c1 , c2 , ... , cK } , the goal is to learn a function fθ , which maps each node v ∈ V to exactly one of the K classes in C ( or in a probabilistic formulation : to the K-simplex ) . Note that this is an instance of transductive learning , since all test samples ( i.e. , the unlabeled nodes ) as well as their attributes and edges ( but not their class labels ! ) are known and used during training ( Chapelle et al. , 2006 ) . The parameters θ of the function fθ are generally learned by minimizing a loss function Ltrain ( e.g . cross-entropy ) on the labeled training nodes : θ∗ = arg min θ Ltrain ( fθ ( G ) ) , ( 1 ) where we overload the notation of fθ to indicate that we feed in the whole graph G . 3.1 ATTACK MODEL . Adversarial attacks are small deliberate perturbations of data samples in order to achieve the outcome desired by the attacker when applied to the machine learning model at hand . The attacker is constrained in the knowledge they have about the data and the model they attack , as well as the adversarial perturbations they can perform . Attacker ’ s goal . In our work , the attacker ’ s goal is to increase the misclassification rate ( i.e. , one minus the accuracy ) of a node classification algorithm achieved after training on the data ( i.e. , graph ) modified by our algorithm . In contrast to Zügner et al . ( 2018 ) and Dai et al . ( 2018 ) , our algorithm is designed for global attacks reducing the overall classification performance of a model . That is , the goal is to have the test samples classified as any class different from the true class . Attacker ’ s knowledge . The attacker can have different levels of knowledge about the training data , i.e . the graph G , the target machine learning model M , and the trained model parameters θ . In our work , we focus on limited-knowledge attacks where the attacker has no knowledge about the classification model and its trained weights , but the same knowledge about the data as the classifier . In other words , the attacker can observe all nodes ’ attributes , the graph structure , as well as the labels of the subset VL and uses a surrogate model to modify the data . Besides assuming knowledge about the full data , we also perform experiments where only a subset of the data is given . Afterwards , this modified data is used to train deep neural networks to degrade their performance . Attacker ’ s capability . In order to be effective and remain undiscovered , adversarial attacks should be unnoticeable . To account for this , we largely follow Zügner et al . ( 2018 ) ’ s attacker capabilities . First , we impose a budget constraint ∆ on the attacks , i.e . limit the number of changes ‖A− Â‖0 ≤ ∆ ( here we have 2∆ since we assume the graph to be symmetric ) . Furthermore , we make sure that no node becomes disconnected ( i.e . a singleton ) during the attack . One of the most fundamental properties of a graph is its degree distribution . Any significant changes to it are very likely to be noticed ; to prevent such large changes to the degree distribution , we employ Zügner et al . ( 2018 ) ’ s unnoticeability constraint on the degree distribution . Essentially , it ensures that the graph ’ s degree distribution can only marginally be modified by the attacker . The authors also derive an efficient way to check for violations of the constraint so that it adds only minimal computational overhead to the attacks . While in this work we focus on changing the graph structure only , our algorithm can easily be modified to change the node features as well . We summarize all these constraints and denote the set of admissible perturbations on the data as Φ ( G ) , where G is the graph at hand .

Summary: This paper studies the problem of learning a better poisoned graph parameters that can maximize the loss of a graph neural network. The proposed using meta-learning to compute the second-order derivatives to get the meta-gradients seems reasonable. The authors also proposed approximate methods to compute the graph as learning parameters, which could be more efficient since the second-order derivatives are no longer computed. The experimental results on three graph datasets show that the proposed model could improve the misclassification rate of the unlabeled nodes.