Understanding and Improving Graph Injection Attack by Promoting Unnoticeability

1 INTRODUCTION . Graph Neural Networks ( GNNs ) , as a generalization of deep learning models for graph structured data , have gained great success in tasks involving relational information ( Hamilton et al. , 2017a ; Battaglia et al. , 2018 ; Zhou et al. , 2020 ; Wu et al. , 2021 ; Kipf & Welling , 2017 ; Hamilton et al. , 2017b ; Veličković et al. , 2018 ; Xu et al. , 2018 ; 2019b ) . Nevertheless , GNNs are shown to be inherently vulnerable to adversarial attacks ( Sun et al. , 2018 ; Jin et al. , 2021 ) , or small intentional perturbations on the input ( Szegedy et al. , 2014 ) . Previous studies show that moderate changes to the existing topology or node features of the input graph , i.e. , Graph Modification Attacks ( GMA ) , can dramatically degenerate the performance of GNNs ( Dai et al. , 2018 ; Zügner et al. , 2018 ; Zügner & Günnemann , 2019 ; Xu et al. , 2019a ; Chang et al. , 2020 ) . Since in many real-world scenarios , it is prohibitively expensive to modify the original graph , recently there is increasing attention paid to Graph Injection Attack ( GIA ) , where the adversary can merely inject few malicious nodes to perform the attack ( Wang et al. , 2018 ; Sun et al. , 2020 ; Wang et al. , 2020 ; Zou et al. , 2021 ) . Despite the promising empirical results , why GIA is booming and whether there is any pitfall behind the success remain elusive . To bridge this gap , we investigate both the advantages and limitations of GIA by comparing it with GMA in a unified setting ( Sec . 2.2 ) . Our theoretical results show that , in this setting when there is no defense , GIA can be provably more harmful than GMA due to its relatively high flexibility . Such flexibility enables GIA to map GMA perturbations into specific GIA perturbations , and to further optimize the mapped perturbations to amplify the damage ( Fig . 1a ) . However , according to the principle of no free lunch , we further find that the power of GIA is built upon the severe damage to the homophily of the original graph . Homophily indicates the tendency of nodes connecting to others with similar features or labels , which is important for the success of most existing GNNs ( McPherson et al. , 2001 ; London & Getoor , 2014 ; Klicpera et al. , 2019 ; Battaglia et al. , 2018 ; Hou et al. , 2020 ; Zhu et al. , 2020 ; Yang et al. , 2021b ) . The severe damage to homophily will disable the effectiveness of GIA to evaluate robustness because non-robust models can easily mitigate or even prevent GIA merely through exploiting the property of homophily damage . Specifically , having observed the destruction of homophily , it is straightforward to devise a defense mechanism aiming to recover the homophily , which we term homophily defenders . Homophily defenders are shown to have strong robustness against GIA attacks . Theoretically , they can effectively reduce the harm caused by GIA to be lower than GMA . Empirically , simple implementations of homophily defenders with edge pruning ( Zhang & Zitnik , 2020 ) can deteriorate even the state-ofthe-art GIA attacks ( Zou et al. , 2021 ) ( Fig . 1b ) . Therefore , overlooking the damage to homophily will make GIA powerless and further limit its applications for evaluating the robustness of GNNs . To enable the effectiveness of GIA in evaluating various robust GNNs , it is necessary to be aware of preserving the homophily when developing GIA . To this end , we introduce a novel constraint – homophily unnoticeability that enforces GIA to retain the homophily of the original graph , which can serve as a supplementary for the unnoticeability constraints in graph adversarial learning . To instantiate the homophily unnoticeability , we propose Harmonious Adversarial Objective ( HAO ) for GIA ( Fig . 1c ) . Specifically , HAO introduces a novel differentiable realization of homophily constraint by regularizing the homophily distribution shift during the attack . In this way , adversaries will not be easily identified by homophily defenders while still performing effective attacks ( Fig . 1b ) . Extensive experiments with 38 defense models on 6 benchmarks demonstrate that GIA with HAO can break homophily defenders and significantly outperform all previous works across all settings , including both non-target attack and targeted attack1 . Our contributions are summarized as follows : • We provide a formal comparison between GIA and GMA in a unified setting and find that GIA can be provably more harmful than GMA due to its high flexibility ( Theorem 1 ) . • However , the flexibility of GIA will also cause severe damage to the homophily distribution which makes GIA easily defendable by homophily defenders ( Theorem 2 ) . • To mitigate the issue , we introduce the concept of homophily unnoticeability and a novel objective HAO to conduct homophily unnoticeable attacks ( Theorem 3 ) . 2 PRELIMINARIES . 2.1 GRAPH NEURAL NETWORKS . Consider a graph G = ( A , X ) with node set V = { v1 , v2 , ... , vn } and edge set E = { e1 , e2 , ... , em } , where A ∈ { 0 , 1 } n×n is the adjacency matrix and X ∈ Rn×d is the node feature matrix . We are 1Code is available in https : //github.com/LFhase/GIA-HAO . interested in the semi-supervised node classification task ( Jin et al. , 2021 ) . That is , given the set of labels Y ∈ { 0 , 1 , .. , C − 1 } n from C classes , we can train a graph neural network fθ parameterized by θ on the training ( sub ) graph Gtrain by minimizing a classification loss Ltrain ( e.g. , cross-entropy ) . Then the trained fθ can predict the labels of nodes in test graph Gtest . A GNN typically follows a neighbor aggregation scheme to recursively update the node representations as : H ( k ) u = σ ( Wk · ρ ( { H ( k−1 ) v } |v ∈ N ( u ) ∪ { u } ) ) , ( 1 ) where N ( u ) is the set of neighbors of node u , H ( 0 ) u = Xu , ∀u ∈ V , H ( k ) u is the hidden representation of node u after the k-th aggregation , σ ( · ) is an activation function , e.g. , ReLU , and ρ ( · ) is an aggregation function over neighbors , e.g. , MEAN or SUM . 2.2 GRAPH ADVERSARIAL ATTACK2 . The goal of graph adversarial attack is to fool a GNN model , fθ∗ , trained on a graph G = ( A , X ) by constructing a graph G′ = ( A′ , X ′ ) with limited budgets ‖G′ − G‖ ≤ 4 . Given a set of victim nodes Vc ⊆ V , the graph adversarial attack can be generically formulated as : min Latk ( fθ∗ ( G′ ) ) , s.t . ‖G′ − G‖ ≤ 4 , ( 2 ) where θ∗ = arg minθ Ltrain ( fθ ( Gtrain ) ) and Latk is usually taken as −Ltrain . Following previous works ( Zügner et al. , 2018 ; Zou et al. , 2021 ) , Graph adversarial attacks can be characterized into graph modification attacks and graph injection attacks by their perturbation constraints . Graph Modification Attack ( GMA ) . GMA generates G′ by modifying the graph structure A and the node features X of the original graph G. Typically the constraints in GMA are to limit the number of perturbations on A and X , denoted by4A and4X , respectively , as : 4A +4X ≤ 4 ∈ Z , ‖A′ −A‖0 ≤ 4A ∈ Z , ‖X ′ −X‖∞ ≤ ∈ R , ( 3 ) where the perturbation on X is bounded by via L-p norm , since we are using continuous features . Graph Injection Attack ( GIA ) . Differently , GIA generates G′ by injecting a set of malicious nodes Vatk as X ′ = [ X Xatk ] , A′ = [ A Aatk ATatk Oatk ] , where Xatk is the features of the injected nodes , Oatk is the adjacency matrix among injected nodes , and Aatk is the adjacency matrix between the injected nodes and the original nodes . Let du denote the degree of node u , the constraints in GIA are : |Vatk| ≤ 4 ∈ Z , 1 ≤ du ≤ b ∈ Z , Xu ∈ DX ⊆ Rd , ∀u ∈ Vatk , ( 4 ) where the number and degrees of the injected nodes are limited , DX = { C ∈ Rd , min ( X ) ·1 ≤ C ≤ max ( X ) ·1 } where min ( X ) and max ( X ) are the minimum and maximum entries inX respectively . Threat Model . We adopt a unified setting , i.e. , evasion , inductive and black-box , which is also used by Graph Robustness Benchmark ( Zheng et al. , 2021 ) . Evasion : The attack only happens at test time , i.e. , Gtest , rather than attacking Gtrain . Inductive : Test nodes are invisible during training . Black-box : The adversary can not access the architecture or the parameters of the target model . 3 POWER AND PITFALLS OF GRAPH INJECTION ATTACK . Based on the setting above , we investigate both the advantages and limitations of GIA by comparing it with GMA . While we find GIA is more harmful than GMA when there is no defense ( Theorem 1 ) , we also find pitfalls in GIA that can make it easily defendable ( Theorem 2 ) . 3.1 POWER OF GRAPH INJECTION ATTACK . Following previous works ( Zügner et al. , 2018 ) , we use a linearized GNN , i.e. , H ( k ) = ÂkXΘ , to track the changes brought by attacks . Firstly we will elaborate the threats of an adversary as follows . Definition 3.1 ( Threats ) . Consider an adversary A , given a perturbation budget4 , the threat of A to a GNN fθ is defined as min‖G′−G‖≤4 Latk ( fθ ( G′ ) ) , i.e. , the optimal objective value of Eq . 2 . 2We leave more details and reasons about the setting used in this work in Appendix B . With Definition 3.1 , we can quantitatively compare the threats of different adversaries . Theorem 1 . Given moderate perturbation budgets 4GIA for GIA and 4GMA for GMA , that is , let 4GIA ≤ 4GMA |V | ≤ |E| , for a fixed linearized GNN fθ trained on G , assume that G has no isolated nodes , and both GIA and GMA follow the optimal strategy , then , ∀4GMA ≥ 0 , ∃4GIA ≤ 4GMA , Latk ( fθ ( G′GIA ) ) − Latk ( fθ ( G′GMA ) ) ≤ 0 , where G′GIA and G′GMA are the perturbed graphs generated by GIA and GMA , respectively . We prove Theorem 1 in Appendix E.1 . Theorem 1 implies that GIA can cause more damage than GMA with equal or fewer budgets , which is also verified empirically as shown in Fig . 1a . Intuitively , the power of GIA mainly comes from its relatively high flexibility in perturbation generation . Such flexibility enables us to find a mapping that can map any GMA perturbations to GIA perturbations , leading the same influences to the predictions of fθ . We will give an example below . Definition 3.2 ( Plural MappingM2 ) . M2 maps a perturbed graph G′GMA generated by GMA with only edge addition perturbations,3 to a GIA perturbed graph G′GIA =M2 ( G′GMA ) , such that : fθ ( G′GIA ) u = fθ ( G′GMA ) u , ∀u ∈ V. As illustrated in Fig . 2a , the procedure ofM2 is , for each edge ( u , v ) added by GMA to attack node u , M2 can inject a new node w to connect u and v , and change Xw to make the same effects to the prediction on u . Then GIA can be further optimized to bring more damage to node u . We also empirically verify the above procedure in Fig . 2b . Details about the comparison are in Appendix C .

Summary: This paper first theoretically shows that graph injection attack (GIA) is more powerful than graph modification (GMA), and GIA will lead to great damage to homophily distribution, which makes GIA easily defended by homophily-based defense. To mitigate the issue, the authors further propose to add the homophily unnoticeability constraint to preserve homophily. The authors also theoretically show that with the GIA with homophily unnoticeability constraint is more powerful than GIA without the constraint.