Ricci-GNN: Defending Against Structural Attacks Through a Geometric Approach

Graph neural networks ( GNNs ) rely heavily on the underlying graph topology and thus can be vulnerable to malicious attacks targeting at perturbing graph structures . We propose a novel GNN defense algorithm against such attacks . In particular , we use a robust representation of the input graph based on the theory of graph Ricci flow , which captures the intrinsic geometry of graphs and is robust to structural perturbation . We propose an algorithm to train GNNs using re-sampled graphs based on such geometric representation . We show that this method substantially improves the robustness against various adversarial structural attacks , achieving state-of-the-art performance on both synthetic and real-world datasets . 1 INTRODUCTION . Recent years we have witnessed the success of graph neural networks ( GNNs ) on many graph applications including graph classification ( Xu et al. , 2019b ) , node classification ( Kipf & Welling , 2016 ; Veličković et al. , 2018 ) , graph generation ( You et al. , 2018 ) and recommendations ( Ying et al. , 2018 ) . As GNNs have shown great potentials , their vulnerability to adversarial attacks ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) becomes a serious concern that hinders their deployment in real life critical applications . For example , a GNN algorithm for fraud detection in financial transaction graphs ( Wang et al. , 2019a ) needs to be robust against attacks aiming at disguising fraud transactions as normal ones . In health informatics , prediction of polypharmacy side effects ( Zitnik et al. , 2018 ) must be robust against attacks that intend to endanger certain patients . In a recommendation system , the developers need to consider potential attacks from spammers who may create fake followers to increase the influence scope of fake news ( Zhou & Zafarani , 2018 ) . One way to attack a GNN model is to modify the graph topology by inserting or deleting edges ( Jin et al. , 2020a ) . A small perturbation of the network topology can significantly impair the graph neural network ’ s performance ( Dai et al. , 2018 ; Zügner & Günnemann , 2019b ) . For example , MetaAttack ( Zügner & Günnemann , 2019a ) can increase the misclassification rate of GCN on a political blog data set by over 18 % with only 5 % perturbed edges . This is not surprising as graph topology is essential for GNNs , both as the backbone of a GNN architecture and as important structural features . In particular , the local neighborhood of each node is commonly used to define receptive fields for the convolution operator . The statistics of local neighborhood , e.g. , node degrees , are important structural information used as additional node features ( Veličković et al. , 2018 ) to re-calibrate the convolutional operation ( Kipf & Welling , 2016 ) . In this paper , we focus on defending against global poisoning adversarial attacks which corrupt the graph topology in the training phase . Some existing approaches assume the graph is true and leverage known robust training techniques , e.g. , enforcing priors on latent representation of data ( Zhu et al. , 2019 ) . These solutions can still be limited by the corrupted graph , considering how critical the underlying graph is for a GNN model . Other methods assume prior knowledge on the graph topology , and perform graph restructuring , e.g. , via low-rank filtering ( Entezari et al. , 2020 ) or graph specification ( Wu et al. , 2019 ) , hoping to remove abnormal edges from the attack . These strong priors , although proven useful , also limit the generality of the method . 1.1 A GEOMETRIC VIEW OF GRAPHS . We take a novel direction to find a robust representation of the graph topology through a geometric lens . We view a discrete graph in a continuous framework , in which nodes stay in an underlying metric space and the connectivity of two nodes has a stochastic nature , depending on the features of the two nodes , their respective neighborhoods and the entire node distribution . The input graph G is replaced by an ensemble of graphs , considered as ( randomized ) discrete realizations of the same underlying metric space in which G is taken . In order to do that , we recover the metric distance between two nodes in the underlying space through the Ricci flow metric on the input graph G. Note that we are not trying to explicitly find an embedding which would involve choices ( e.g , Euclidean vs non-Euclidean , dimensionalities ) that introduce extra and unnecessary distortion . Instead , we represent the underlying metric space via pairwise geodesic distance between nodes . Our geometrical approach is inspired by the Riemmanian geometry in the continuous setting ( Hamilton , 1982 ; Perelman , 2002 ) . On a Riemmanian manifold , one can define Ricci curvature to measure the amount of ‘ bending ’ or ‘ curving ’ at each point . With Ricci curvature , one can define a diffusion process by changing the Riemannian metric ( stretching or shrinking locally ) such that curvature is uniform everywhere . This uniformization process is called Ricci flow . This theory can be extended to a graph setting ( Ollivier , 2009 ) . Generally speaking , edges that are locally well connected have positive curvature while edges that are locally sparsely connected have negative curvature . In Ricci flow , edges of negative curvature are stretched ( with increased edge weight ) and edges of positive curvature are condensed ( with decreased edge weight ) . These new edge weights that uniformize the Ricci curvature of the graph are called the Ricci flow metric . See Figure 1 for an illustration . Graph Ricci curvature and Ricci flow can be used to identify critical edges in a graph ( Ni et al. , 2015 ; Sandhu et al. , 2015 ) and to identify community structures ( Ni et al. , 2019 ; Sia et al. , 2019 ) . We also note that graph Ricci curvature has been used in GNN for node classification task ( Ye et al. , 2020 ) , but not for defending structural attacks to GNN . Robustness against topological perturbation . Ricci flow metric has been shown to be robust to random deletion and addition of edges ( Ni et al. , 2018 ) . This attributes to the fact that Ricci flow is a global process that tries to uncover the underlying metric space supported by the graph topology and thus embraces redundancy . Compared to other graph metrics such as the hop count metric and metric obtained by spectral embedding , Ricci flow metric provides a better trade-off between robustness and representation power of the graph metric , as shown in Figure 3 . When two edges are deleted , the Ricci flow metric is rarely affected ( Figure 3 ( a ) ) , similar to the hop count metric ( Figure 3 ( c ) ) ; while the distance metric by spectral embedding is substantially more sensitive ( Figure 3 ( b ) ) . We note that the hop count metric is also robust to dynamic edge deletions due to the small world phenomena and multiple shortest paths in the graph ; however the hop count metric takes only integer values and generally lacks descriptive power to provide desirable resolution and differentiation . To train a GNN using the Ricci flow metric , we generate an ensemble of sample graphs G1 , G2 , · · · , and use a new sample in each network layer of the GNN of every training epoch ( Figure 2 ) . Therefore the trained model is enforced to focus on the underlying metric information represented by the graph ( which is much more robust ) and not on the particular input graph topology ( which could be corrupted ) per se . Our method is agnostic to both models and attacks , thus can be applied to different GNNs and different structural attacks . We show in both synthetic and real-world datasets that the proposed algorithm effectively defends against various structural attacks , with improved performance compared to other defense schemes . We summarize our contributions as follows . • We are the first to take a geometric view of the GNN defense problem . We propose to train GNNs with the Ricci flow representation of a graph instead of its attacked topology . • We design a new algorithm to sample graphs based on the Ricci flow representation for training GNN . This effectively alleviate the impact of structural attacks by adversaries . • We demonstrate the efficacy of our method on various synthetic and real-world datasets , against state-of-the-art graph topology poisoning methods . 1.2 RELATED WORK . The vulnerability of deep neural network models w.r.t . adversarial attacks is well known . And graph neural networks are not an exception ( Dai et al. , 2018 ; Zügner et al. , 2018 ; Zügner & Günnemann , 2019a ) . Here we briefly review the methods for attacking and defending against GNNs . Adversarial attack on graphs . There are two categories of attacks : evasion attacks and poisoning attacks . Evasion attacks generate fake samples for the trained model in the testing time , while poisoning attacks directly modify the training data . Dai et al . ( 2018 ) employs a reinforcement learning based framework for non-targeted test-time attacks ( i.e . evasion ) on graph classification and node classification . The focus is on the modifications of graph structures , and the attackers are restricted to edge deletions only . Zügner et al . ( 2018 ) consider both training-time ( i.e . poisoning ) and testing-time attacks . The attacks , called nettack , are based on a surrogate model with both edge insertion and deletion . Nettack is a local attack , where the goal is to lower the performance on a target node . Later , a meta-learning poisoning attack is developed by Zügner & Günnemann ( 2019a ) which aims to decrease classification accuracy globally . It treats the the graph structure as a hyper-parameter and conducts training-time attacks through meta learning . Last , Xu et al . ( 2019a ) proposes a gradient-based attack method that directly tackling the dicrete graph data . Since these two are the state-of-the-art non-targeted global attack method , we will mainly focus on developing defense schemes against them . Robustness of GNNs . To defend against these graph attacks , Miller et al . ( 2019 ) seek to increase model robustness by decoupling structure from attributes in the classifier and re-selecting the training data . But their method exhibits a trade-off between robustness and performance , i.e . the performance drops on clean data . Wang et al . ( 2019b ) proposed graph encoder refining and adversarial contrastive learning . They investigate the vulnerabilities in every aggregation layer and the perceptron layer of a GNN encoder , and apply dual-stage aggregation and bottleneck perceptron to address those vulnerabilities . They mainly focus on targeted node attacks ( e.g . Nettack ) instead of global topology attacks . RGCN ( Zhu et al. , 2019 ) treats node features as a Gaussian distribution and encode the hidden representation of nodes by mean and variance matrices . They apply self-attention on the variance matrix to aggregate messages from neighboring nodes . However , this method only focuses on defense against random noise on node features . GCN-Jaccard ( Wu et al. , 2019 ) pre-processes the network by eliminating edges that connect nodes with sufficientely small Jaccard similarity of features . GCN-SVD ( Entezari et al. , 2020 ) proposes to vaccinate GCN with the low-rank approximation of the perturbed graph . Most of these existing methods provide insight of robustness from the perspective of optimization or matrix ranks . DropEdge ( Rong et al. , 2019 ) randomly removes a certain amount of edges from the input graph at each training epoch . It is designed to resolve the over-fitting and over-smoothing issue of developing deeep GCNs . However , it can also be used for improving the graphs robustness . Pro-GNN ( Jin et al. , 2020b ) jointly learns a structural graph and a robust graph neural network model from the perturbed graph guided by exploring the graph properties of sparsity , low rank and feature smoothness to design robust graph neural networks . In this paper , we understand the graph robustness from a geometric view and provide an efficient sampling based model .

Summary: In Ricci-GCN new graphs are resampled in each iteration of the training phase based on the Ricci flow metric. The Ricci flow incorporates curvature information and captures the intrinsic geometry of the graph. Compared to e.g. spectral embedding it is more robust to structural perturbations. This leads to improved robustness against adversarial attacks on the graph structure.