Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient

1 INTRODUCTION . Modern infrastructure systems , such as computer routing and electric power networks , are vulnerable to natural disasters and malicious attacks ( Schneider et al. , 2011 ) . Consider the scenario where the abnormality of one power supply station causes other power supply stations to overload , which cascades more power supply to fail , resulting in a region-wide power outage . Figure 1 visualizes this scenario that the failures of a dozen of nodes could jeopardize the connectivity and utility of the EU power network with 217 nodes . The ability for a system to defend itself from such failures and attacks is characterized by the network resilience . A resilient network should continue to function and maintain an acceptable level of utility when part of the network fails . A network becomes more resilient if some connections could backup the others . It is seemingly straightforward to achieve resilience by adding redundant edges . However , it will not be practically feasible as the nodes are usually already running at their full capacity . For example , a power supply facility can only support a certain number of connections and can not afford additional loads . In such a regime , degree-preserving operations , such as edge rewiring , are desired ( Schneider et al. , 2011 ; Rong & Liu , 2018 ; Yazıcıoğlu et al. , 2015 ; Zhou & Liu , 2014 ) . Let G = ( V , E ) be a graph . An edge rewiring operation alters the graph structure by removing AC and BD and adding AB and CD , where AC , BD ∈ E and AB , CD , AD , BC /∈ E.1 Edge rewiring is empirically effective to maximize network resilience and becomes the prevailing atomic operation in the literature . Despite this , methods for improving network resilience via edge rewiring share several limitations : utility loss , local optimality , and transduction . 1A few works have a different definition for the edge rewiring operation . • Network utility loss . Improving network resilience aims to protect the network utility from degrading under attack and make the network continue functioning . Optimizing network resilience without considering the network utility could jeopardize the functioning without the presence of an attack . • Local optimality . Combinatorially choosing the edges to rewire to optimize the resilience is proved to be NP-hard ( Mosk-Aoyama , 2008 ) . Previous studies predominantly seek approximate optimality through greedy-like algorithms , which yields local optimality in practice . • Transductivity . Traditional resilience optimization methods are transductive since they search the resilience topology on a given graph . This search process has to be executed for every graph and does not transfer between graphs , even if the graphs are only up to a minor structure difference . To tackle the challenges above , in this work , we present ResiNet , the first objective-agnostic learning-based method for inductively discovering resilient network topologies . Frist , by formulating the cumulative gain of the objective into RL , ResiNet is agnostic to arbitrary objective function , which makes resilience optimization practical in real applications by incorporating the utility into the resilience . The nature of delayed gratification in RL also allows the agent to bypass local optimums caused by step-wise optimal actions . Second , the fact that graphs have different sizes and arbitrary node permutations makes it challenging to solve transductivity when selecting edges with deep learning . In this scenario , nodes can no longer be represented by their ID since it is computationally infeasible to train a model to handle all isomorphic graphs . Using an attention-based mechanism ( Vinyals et al. , 2015 ) to identify edges and an auto-regressive space ( Trivedi et al. , 2020 ) to formulate edge selection , an agent trained by ResiNet works on a wide range of graphs with different sizes and permutations . Deep learning and graph neural networks ( GNNs ) have been successfully applied to many combinatorial optimization problems on graphs ( Khalil et al. , 2017 ; Li et al. , 2018a ; Karalias & Loukas , 2020 ) . However , as we empirically observed , the combination of GNNs and RL can not perform well on the task of improving network resilience via edge rewiring . We suspect that this is caused by a dilemma for the expressive power of GNN : If the representation power of the GNN is too weak ( e.g. , the regular GNN ) , then it could not distinguish graphs with minor differences and will choose similar actions in consecutive steps of RL . This causes the process to alternate between two graphs , forming an infinite loop . If the representation power is too strong ( e.g. , SMP , IDGNN ) , it tends to give similar representations to connected vertices , known as the oversmoothing phenomenon ( Vignac et al. , 2020 ; You et al. , 2021 ) . It is then difficult for the agent to distinguish vertices in a connected component , causing large randomness in the output of the policy . This dilemma may lead to the bad performance of selecting edges using regular GNNs and RL . Therefore , to successfully select two edges from a degree-preserving evolving dynamic graph at each step , the implementation of ResiNet is armed with Filtration enhanced GNN ( FireGNN ) , our technical innovation , to solve the above dilemma of the expressive power of GNN . FireGNN creates a series of subgraphs ( the filtration ) by successively removing nodes from the graph , and aggregates the node representations from the subgraphs . It makes the GNN powerful when representing graphs while avoiding oversmoothing as the node information are adequately acquired from a multi-step filtration . This technical innovation is inspired by persistent homology and the approximation of the persistence diagram ( Edelsbrunner & Harer , 2008 ; Aktas et al. , 2019 ; Hofer et al. , 2020 ) . The main contributions of this paper are summarized as follows : 1 ) We propose ResiNet , a data-driven framework to boost network resilience in a degreepreserving way with moderate loss of the network utility by forming resilience optimization into a objective-agnostic sequential decision process of edge rewiring . Extensive experiments show that with a small number of rewiring operations , ResiNet achieves a nearoptimal resilience gain on multiple graphs while balancing network utilities . Existing approaches are outperformed by a large margin . 2 ) FireGNN , our technical innovation , balances the expressive power and the oversmoothing issue through the graph filtration augmentation . FireGNN could distinguish very similar graphs and distinguish connected vertices at the same time , which is essential to make the combination of GNNs and RL work on the network resilience optimization . 3 ) ResiNet is the first to improve network resilience in an inductive way , with the specialized auto-regressive permutation-invariant size-variable policy network . Once an agent is trained , it works on a wide range of graphs . The policy network is general enough to be extended to other problems in constrained graph generation . 2 RELATED WORKS . Network attacks and defenses The problem of attacking a network is characterized by the target resilience measurement and the manipulation type . Depending on the application , network resilience is defined as a corresponding measure to quantify the network functionality , such as graph connectivity ( Grassia et al. , 2021 ; Fan et al. , 2020 ) . For particular networks ( e.g. , scale-free networks ) , attacks appear in destroying critical nodes and critical connections ( Grassia et al. , 2021 ; Fan et al. , 2020 ; Zhao et al. , 2021 ; Zhang et al. , 2017 ; Medya et al. , 2020 ) . Heuristic and learning-based attack methods ( Holme et al. , 2002 ; Iyer et al. , 2013 ; Grassia et al. , 2021 ; Fan et al. , 2020 ) have been proposed to target the critical subset of the network . To defend against potential attacks , various defense strategies have been proposed to protect the network functionality from crashing and preserve some of its topologies . The commonly used defense manipulations include adding additional edges ( Li et al. , 2019 ; Carchiolo et al. , 2019 ) , protecting vulnerable edges ( Wang et al. , 2014 ) and rewiring two edges ( Schneider et al. , 2011 ; Chan & Akoglu , 2016 ; Buesser et al. , 2011 ) . Among these manipulations , edge rewiring fits well to real-world applications as it induces less functionality changes ( e.g . degree preserving ) to the original network and does not impose additional loads to the vertices ( Schneider et al. , 2011 ; Rong & Liu , 2018 ; Yazıcıoğlu et al. , 2015 ; Zhou & Liu , 2014 ) . GNNs for combinatorial optimization problems The idea of applying GNNs to solve combinatorial optimization problems on graphs is becoming a promising yet challenging topic . Abundant algorithms have been developed and can be classified into three categories . Supervised learning approach combines GNNs with heuristics search procedures or solvers to solve classical combinatorial problems , such as graph matching ( Bai et al. , 2018 ) , graph coloring ( Lemos et al. , 2019 ) , TSP ( Li et al. , 2018b ; Joshi et al. , 2019 ) and SAT ( Wang et al. , 2019 ) . These methods require labeled instances and thus are difficult to generalize to large-scale instances directly . Unsupervised learning approach ( Karalias & Loukas , 2020 ) trains GNNs to parametrize a probability distribution over sets . The probabilistic proof achieves the existence of feasible solutions . After that , the derandomized method is applied to decode the desired solutions . However , meeting complex constraints have not been supported yet . Reinforcement learning approach formulates a problem as a Markov decision process with a proper reward function to guide the search towards an optimal solution . Some classical CO problems have achieved remarkable results like TSP ( Fu et al. , 2020 ; Khalil et al. , 2017 ) , Vehicle Routing Problem ( Nazari et al. , 2018 ; Peng et al. , 2020 ; Yu et al. , 2019 ) , SAT ( Yolcu & Póczos , 2019 ) and max-cut ( Khalil et al. , 2017 ) . In this paper , we adopt the RL based approach but with a specialized policy network . Positioning of this work In this paper , we use only the edge rewiring operation to defend against degree-based and centrality-based attacks while preserving the network utility . Extended related works The related works on Network resilience and utility , multi-views graph augmentation for GNNs , and deep graph generation models are deferred to Appendix A . 3 PROPOSED APPROACH : RESINET . In this section , we first present the problem formulation of maximizing network resilience while preserving utility performance . We then discuss the graph resilience-aware environment design and give the graph policy network which guides the process of edge rewiring . The problem of boosting network resilience is formulated as a reinforcement learning task by iteratively rewiring the edges . 3.1 PROBLEM DEFINITION . An undirected graph is defined as G = ( V , E ) , where V = { 1 , 2 , . . . , N } is the set of N nodes , E is the set of M edges , A ∈ { 0 , 1 } N×N is the adjacency matrix , and F ∈ RN×d is the d-dimensional node feature matrix.2 The degree of a node is defined as di = ∑N j=1Aij , and a node with degree 0 is called an isolated node . Let GG denote the set of graphs with the same node degrees as G. Given the network resilience functionR ( G ) and the utility function U ( G ) , the objective of boosting the resilience of G is to find a target graph G ? ∈ GG , which maximizes the network resilience while preserving the network utility . Formally , the problem of maximizing network resilience is formulated as G ? = argmax G′∈GG α · R ( G ′ ) + ( 1− α ) · U ( G ′ ) , where α ∈ R is the scalar weight that balances the resilience and the utility . Two examples of resilience functions , including the graph connectivity-based measurement ( Schneider et al. , 2011 ) and the spectrum-based measurement ( e.g. , adjacency matrix spectrum and Laplacian matrix spectrum ) , and two examples of utility functions , including global efficiency and local efficiency , are given in Appendix B . Our formulation could generalize to other definitions of resilience and utility .

Summary: The paper proposes a neural approach that increases network resilience by edge wiring. The approach uses a combination of graph neural network (GNN) and policy gradient method to do so. The experiments use a few small networks and several non-neural baselines.