Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient

1 INTRODUCTION . Modern infrastructure systems , such as computer routing and electric power networks , are vulnerable to natural disasters and malicious attacks ( Schneider et al. , 2011 ) . Consider the scenario where the abnormality of one power supply station causes other power supply stations to overload , which cascades more power supply to fail , resulting in a region-wide power outage . Figure 1 visualizes this scenario that the failures of a dozen of nodes could jeopardize the connectivity and utility of the EU power network with 217 nodes . The ability for a system to defend itself from such failures and attacks is characterized by the network resilience . A resilient network should continue to function and maintain an acceptable level of utility when part of the network fails . A network becomes more resilient if some connections could backup the others . It is seemingly straightforward to achieve resilience by adding redundant edges . However , it will not be practically feasible as the nodes are usually already running at their full capacity . For example , a power supply facility can only support a certain number of connections and can not afford additional loads . In such a regime , degree-preserving operations , such as edge rewiring , are desired ( Schneider et al. , 2011 ; Rong & Liu , 2018 ; Yazıcıoğlu et al. , 2015 ; Zhou & Liu , 2014 ) . Let G = ( V , E ) be a graph . An edge rewiring operation alters the graph structure by removing AC and BD and adding AB and CD , where AC , BD ∈ E and AB , CD , AD , BC /∈ E.1 Edge rewiring is empirically effective to maximize network resilience and becomes the prevailing atomic operation in the literature . Despite this , methods for improving network resilience via edge rewiring share several limitations : utility loss , local optimality , and transduction . 1A few works have a different definition for the edge rewiring operation . • Network utility loss . Improving network resilience aims to protect the network utility from degrading under attack and make the network continue functioning . Optimizing network resilience without considering the network utility could jeopardize the functioning without the presence of an attack . • Local optimality . Combinatorially choosing the edges to rewire to optimize the resilience is proved to be NP-hard ( Mosk-Aoyama , 2008 ) . Previous studies predominantly seek approximate optimality through greedy-like algorithms , which yields local optimality in practice . • Transductivity . Traditional resilience optimization methods are transductive since they search the resilience topology on a given graph . This search process has to be executed for every graph and does not transfer between graphs , even if the graphs are only up to a minor structure difference . To tackle the challenges above , in this work , we present ResiNet , the first objective-agnostic learning-based method for inductively discovering resilient network topologies . Frist , by formulating the cumulative gain of the objective into RL , ResiNet is agnostic to arbitrary objective function , which makes resilience optimization practical in real applications by incorporating the utility into the resilience . The nature of delayed gratification in RL also allows the agent to bypass local optimums caused by step-wise optimal actions . Second , the fact that graphs have different sizes and arbitrary node permutations makes it challenging to solve transductivity when selecting edges with deep learning . In this scenario , nodes can no longer be represented by their ID since it is computationally infeasible to train a model to handle all isomorphic graphs . Using an attention-based mechanism ( Vinyals et al. , 2015 ) to identify edges and an auto-regressive space ( Trivedi et al. , 2020 ) to formulate edge selection , an agent trained by ResiNet works on a wide range of graphs with different sizes and permutations . Deep learning and graph neural networks ( GNNs ) have been successfully applied to many combinatorial optimization problems on graphs ( Khalil et al. , 2017 ; Li et al. , 2018a ; Karalias & Loukas , 2020 ) . However , as we empirically observed , the combination of GNNs and RL can not perform well on the task of improving network resilience via edge rewiring . We suspect that this is caused by a dilemma for the expressive power of GNN : If the representation power of the GNN is too weak ( e.g. , the regular GNN ) , then it could not distinguish graphs with minor differences and will choose similar actions in consecutive steps of RL . This causes the process to alternate between two graphs , forming an infinite loop . If the representation power is too strong ( e.g. , SMP , IDGNN ) , it tends to give similar representations to connected vertices , known as the oversmoothing phenomenon ( Vignac et al. , 2020 ; You et al. , 2021 ) . It is then difficult for the agent to distinguish vertices in a connected component , causing large randomness in the output of the policy . This dilemma may lead to the bad performance of selecting edges using regular GNNs and RL . Therefore , to successfully select two edges from a degree-preserving evolving dynamic graph at each step , the implementation of ResiNet is armed with Filtration enhanced GNN ( FireGNN ) , our technical innovation , to solve the above dilemma of the expressive power of GNN . FireGNN creates a series of subgraphs ( the filtration ) by successively removing nodes from the graph , and aggregates the node representations from the subgraphs . It makes the GNN powerful when representing graphs while avoiding oversmoothing as the node information are adequately acquired from a multi-step filtration . This technical innovation is inspired by persistent homology and the approximation of the persistence diagram ( Edelsbrunner & Harer , 2008 ; Aktas et al. , 2019 ; Hofer et al. , 2020 ) . The main contributions of this paper are summarized as follows : 1 ) We propose ResiNet , a data-driven framework to boost network resilience in a degreepreserving way with moderate loss of the network utility by forming resilience optimization into a objective-agnostic sequential decision process of edge rewiring . Extensive experiments show that with a small number of rewiring operations , ResiNet achieves a nearoptimal resilience gain on multiple graphs while balancing network utilities . Existing approaches are outperformed by a large margin . 2 ) FireGNN , our technical innovation , balances the expressive power and the oversmoothing issue through the graph filtration augmentation . FireGNN could distinguish very similar graphs and distinguish connected vertices at the same time , which is essential to make the combination of GNNs and RL work on the network resilience optimization . 3 ) ResiNet is the first to improve network resilience in an inductive way , with the specialized auto-regressive permutation-invariant size-variable policy network . Once an agent is trained , it works on a wide range of graphs . The policy network is general enough to be extended to other problems in constrained graph generation . 2 RELATED WORKS . Network attacks and defenses The problem of attacking a network is characterized by the target resilience measurement and the manipulation type . Depending on the application , network resilience is defined as a corresponding measure to quantify the network functionality , such as graph connectivity ( Grassia et al. , 2021 ; Fan et al. , 2020 ) . For particular networks ( e.g. , scale-free networks ) , attacks appear in destroying critical nodes and critical connections ( Grassia et al. , 2021 ; Fan et al. , 2020 ; Zhao et al. , 2021 ; Zhang et al. , 2017 ; Medya et al. , 2020 ) . Heuristic and learning-based attack methods ( Holme et al. , 2002 ; Iyer et al. , 2013 ; Grassia et al. , 2021 ; Fan et al. , 2020 ) have been proposed to target the critical subset of the network . To defend against potential attacks , various defense strategies have been proposed to protect the network functionality from crashing and preserve some of its topologies . The commonly used defense manipulations include adding additional edges ( Li et al. , 2019 ; Carchiolo et al. , 2019 ) , protecting vulnerable edges ( Wang et al. , 2014 ) and rewiring two edges ( Schneider et al. , 2011 ; Chan & Akoglu , 2016 ; Buesser et al. , 2011 ) . Among these manipulations , edge rewiring fits well to real-world applications as it induces less functionality changes ( e.g . degree preserving ) to the original network and does not impose additional loads to the vertices ( Schneider et al. , 2011 ; Rong & Liu , 2018 ; Yazıcıoğlu et al. , 2015 ; Zhou & Liu , 2014 ) . GNNs for combinatorial optimization problems The idea of applying GNNs to solve combinatorial optimization problems on graphs is becoming a promising yet challenging topic . Abundant algorithms have been developed and can be classified into three categories . Supervised learning approach combines GNNs with heuristics search procedures or solvers to solve classical combinatorial problems , such as graph matching ( Bai et al. , 2018 ) , graph coloring ( Lemos et al. , 2019 ) , TSP ( Li et al. , 2018b ; Joshi et al. , 2019 ) and SAT ( Wang et al. , 2019 ) . These methods require labeled instances and thus are difficult to generalize to large-scale instances directly . Unsupervised learning approach ( Karalias & Loukas , 2020 ) trains GNNs to parametrize a probability distribution over sets . The probabilistic proof achieves the existence of feasible solutions . After that , the derandomized method is applied to decode the desired solutions . However , meeting complex constraints have not been supported yet . Reinforcement learning approach formulates a problem as a Markov decision process with a proper reward function to guide the search towards an optimal solution . Some classical CO problems have achieved remarkable results like TSP ( Fu et al. , 2020 ; Khalil et al. , 2017 ) , Vehicle Routing Problem ( Nazari et al. , 2018 ; Peng et al. , 2020 ; Yu et al. , 2019 ) , SAT ( Yolcu & Póczos , 2019 ) and max-cut ( Khalil et al. , 2017 ) . In this paper , we adopt the RL based approach but with a specialized policy network . Positioning of this work In this paper , we use only the edge rewiring operation to defend against degree-based and centrality-based attacks while preserving the network utility . Extended related works The related works on Network resilience and utility , multi-views graph augmentation for GNNs , and deep graph generation models are deferred to Appendix A . 3 PROPOSED APPROACH : RESINET . In this section , we first present the problem formulation of maximizing network resilience while preserving utility performance . We then discuss the graph resilience-aware environment design and give the graph policy network which guides the process of edge rewiring . The problem of boosting network resilience is formulated as a reinforcement learning task by iteratively rewiring the edges . 3.1 PROBLEM DEFINITION . An undirected graph is defined as G = ( V , E ) , where V = { 1 , 2 , . . . , N } is the set of N nodes , E is the set of M edges , A ∈ { 0 , 1 } N×N is the adjacency matrix , and F ∈ RN×d is the d-dimensional node feature matrix.2 The degree of a node is defined as di = ∑N j=1Aij , and a node with degree 0 is called an isolated node . Let GG denote the set of graphs with the same node degrees as G. Given the network resilience functionR ( G ) and the utility function U ( G ) , the objective of boosting the resilience of G is to find a target graph G ? ∈ GG , which maximizes the network resilience while preserving the network utility . Formally , the problem of maximizing network resilience is formulated as G ? = argmax G′∈GG α · R ( G ′ ) + ( 1− α ) · U ( G ′ ) , where α ∈ R is the scalar weight that balances the resilience and the utility . Two examples of resilience functions , including the graph connectivity-based measurement ( Schneider et al. , 2011 ) and the spectrum-based measurement ( e.g. , adjacency matrix spectrum and Laplacian matrix spectrum ) , and two examples of utility functions , including global efficiency and local efficiency , are given in Appendix B . Our formulation could generalize to other definitions of resilience and utility .

Summary: This paper studies how to improve network resilience by proposing a reinforcement learning-based framework named **ResiNet** and a new GNN architecture called **FireGNN**. The proposed framework is able to directly generalize to unseen graphs. The new GNN architecture applies the graph filtration process, which enhances the expressivity of GNN. The authors conduct experiments on synthetic and real datasets to compare the proposed framework with previous baseline methods.