Learning Graph Representations for Influence Maximization

1 INTRODUCTION . Several real-world problems can be cast as a combinatorial optimization problem over a graph . From distributing packages ( Mathew et al. , 2015 ) to improving the general health ( Wilder et al. , 2018 ) and vehicles ’ management ( Touati-Moungla & Jost , 2012 ) , optimization on graphs lies in the core of many real-world applications that are vital to our way of living . Unfortunately , the majority of these problems are NP-hard , and hence we can only approximate their solution in a satisfactory time limit that matches the real world requirements . Recent machine learning methods have emerged as a promising solution to develop heuristic methods that provide fast and accurate approximations ( Bengio et al. , 2020 ) . The general idea is to train a supervised or unsupervised learning model to infer the solution given an unseen graph and the problem constraints . The models tend to consist of Graph Neural Networks ( GNNs ) to encode the graph and the nodes , Q-learning ( Watkins & Dayan , 1992 ; Sutton & Barto , 2018 ) to produce sequential predictions , or a combination of both . The practical motivation behind learning to solve combinatorial optimization problems , is that inference time is faster than running an exact combinatorial solver ( Joshi et al. , 2019 ) . That said , specialized combinatorial algorithms like CONCORDE for the Traveling Salesman Problem ( TSP ) or GUROBI in general , can not be surpassed yet ( Kool et al. , 2018 ) . Though many such methods have been proposed for a plethora of problems , influence maximization ( IM ) has not been addressed yet extensively . IM addresses the problem of finding the set of nodes in a network that would maximize the number of nodes reached by starting a diffusion from them ( Kempe et al. , 2003 ) . The problem is proved to be NP-hard , from a reduction to the set-cover problem . Moreover , the influence estimation ( IE ) problem that is embedded in IM , i.e. , estimating the number of nodes influenced by a given seed set , is # P-hard as it is analogous to counting s-t connectedness and would require 2|E| possible combinations to compute exactly , where |E| is the number of network edges ( Wang et al. , 2012 ) . Typically , influence estimation is approximated using repetitive Monte-Carlo ( MC ) simulations of the independent cascade ( IC ) diffusion model . In general , the seed set is built greedily , taking advantage of the submodularity of the influence function which guarantees an at least ( 1− 1e ) approximation to the optimal . Although the latter lacks of efficiency as one still has to estimate influence for every candidate seed in every step of building the seed sets . Hence , several scalable algorithms Borgs et al . ( 2014 ) ; Tang et al . ( 2015 ) and heuristics Chen et al . ( 2009 ) ; Jung et al . ( 2012 ) were developed capitalizing on sketches or the structure of the graph to produce more efficient solutions . IM can be applied on a plethora of real-world tasks , such as epidemic containment ( Wilder et al. , 2017 ) , diminishing fake news ( Budak et al. , 2011 ) , and running viral marketing campaigns ( Domingos & Richardson , 2001 ) . We address IM using neural networks , to capitalize on the aforementioned advantages as well as their ability to easily incorporate contextual information such as user profiles and topics ( Tian et al. , 2020 ) , a task that remains unsolvable for non-specialized IM algorithms and heuristics . We propose GLIE , a GNN that provides efficient IE for a given seed set and a graph with influence probabilities . It can be used as a standalone influence predictor with competitive results for graphs up to 10 times larger than the train set . Moreover , we leverage GLIE for IM , combining it with CELF ( Leskovec et al. , 2007 ) , that typically does not scale beyond networks with thousands of edges . The proposed method runs in networks with millions of edges in seconds , and exhibits better influence spread than a state-of-the-art algorithm and previous GNN-RL methods for IM . In addition , we develop GRIM , a Q-learning architecture that utilizes GLIE ’ s representations and predictions to obtain seeds sequentially , while minimizing the number of influence estimations throughout steps . Finally , we propose PUN , a method that uses GLIE ’ s representations to compute the number of neighbors predicted to be uninfluenced and uses it as an approximation to the marginal gain . We prove PUN ’ s influence spread is submodular and monotone , and hence can be optimized greedily with a guarantee , in contrast to prior learning-based methods . The experiments indicate that PUN provides the best balance between influence quality and computational efficiency . The paper is organized as follows . Section 2 presents an overview of relevant approaches and clarifies the advantage of the proposed models . Section 3 describes the proposed methods , starting with IE and advancing progressively towards faster methods for IM . Section 4 exhibits and interprets the experimental results for IE and IM . Finally , Section 5 summarizes the contribution and presents future steps . 2 RELATED WORK . The first approach to solving combinatorial optimization ( CO ) using neural networks was based on attention-based NNs for discrete structures , POINTERNETS ( Vinyals et al. , 2015 ) , followed by an architecture that combines POINTERNETS with an actor-critic training to find the best route for TSP ( Bello et al. , 2016 ) . The first architecture that utilized graph-based learning was S2N-DQN ( Dai et al. , 2017 ) , usingSTRUCT2VEC to encode the states of the nodes and the graph , and training a Q-learning model that chooses the right node to add in a solution given the current state . Based on S2V-DQN , a DQN ( Mnih et al. , 2015 ) for the network dismantling problem was recently proposed ( Fan et al. , 2020 ) ( Li et al. , 2019 ) . The model , named FINDER , uses a deep Q-learning architecture where the representations are derived by three GRAPHSAGE layers . The reward is based on size of the giant connected component size , i.e. , every new node ( seed ) chosen , aims to dismantle the network as much as possible . Some of the main advantages of FINDER is that it is trained on small synthetic data , which are easy to make , and can extrapolate to relatively large graphs . On the other hand , one of the core disadvantages is that it can not work with directed graphs and weighted edges . Another recent supervised deep learning approach on IM , GCOMB ( Manchanda et al. , 2020 ) , utilizes a probabilistic greedy to produce scores on graphs and trains a GNN to predict them . A Q-network receives the scores along with an approximate calculation of the node ’ s neighborhood correlation with the seed set , to predict the next seed . This approach , though scalable and comparable to SOTA in accuracy , has to be trained on a large random subset of the graph ( 30 % of it ) and tested on the rest . This makes the model graph-specific , i.e. , it has to be retrained to perform well on a new graph . This imposes a serious overhead , considering the time required for training , subsampling and labeling these samples using the probabilistic greedy method with traditional IE . As shown in ( Manchanda et al. , 2020 ) Appendix G , it takes at least hundreds of minutes and is thus out of our scope . Finally , recent works on learning approximations to submodular policies ? require a large number of ground truth evaluation to create the training trajectories , which rendering the training too time consuming . Moreover , such methods require a novel neural network encoding to capture the state of IM , which has not been developed yet . In this paper , we propose an approach that combines the advantages of the aforementioned methods , in that it is only trained on small simulated data once and generalizes to larger graphs , and it addresses the problem of IM in weighted directed networks . Furthermore , the approach can be broken down to a GNN for influence estimation and three IM methods . The former can act alone as influence predictor and be competitive with relevant methods , such as DMP ( Lokhov & Saad , 2019 ) for graphs up to one scale larger than the train set . GLIE is used to propose CELF-GLIE , CELF Leskovec et al . ( 2007 ) with GLIE as influence estimator , GRIM , a Q-network that learns how to choose seeds using the GLIE ’ s estimations and hidden representations , and PUN , an adaptive IM method that optimizes greedily a submodular influence spread using GLIE ’ s representations . We note here that the majority of the relevant literature on DL for CO address small graphs ( Vinyals et al. , 2015 ; Dai et al. , 2017 ; Kool et al. , 2018 ; Prates et al. , 2019 ) which makes them not applicable to our task . More scalable , unsupervised methods ( Karalias & Loukas , 2020 ) are tailored to specific problems and is non-trivial to adjust them to our problem , with the exception of ( Li et al. , 2018 ) which was found significantly worse than the SOTA algorithm we compare with in Manchanda et al . ( 2020 ) . 3 METHODOLOGY . 3.1 GRAPH LEARNING FOR INFLUENCE ESTIMATION ( GLIE ) . In this section , we introduce GLIE ( GNN Learning Influence Estimation ) , which aims to learn how to estimate the influence of a seed set S over a graph G = ( V , E ) . Let A ∈ Rn×n be the adjacency matrix and X ∈ Rn×d be the features of nodes , representing which nodes belong to the seed set by 1 and 0 otherwise : Xu = { { 1 } d , u ∈ S { 0 } d , u /∈ S . ( 1 ) For the derivation of the corollary , we set d = 1 . More dimensions will become meaningful when we parameterize the problem . If we normalize A by each row , we form a row-stochastic transition matrix , as : Auv = pvu = { 1 deg ( u ) , v ∈ N ( u ) 0 , v /∈ N ( u ) , ( 2 ) where deg ( u ) is the in-degree of node u andN ( u ) is the set of neighbors of u . Based on the weighted cascade ( Kempe et al. , 2003 ) , each row u stores the probability of node u being influenced by each of the other nodes that are connected to it by a directed link v → u . Note that , in case of directed influence graphs , A should correspond to the transpose of the adjacency matrix . The influence probability p ( u|S ) resembles the probability of a node u getting influenced if its neighbors belong in the seed set , i.e. , during the first step of the diffusion . We can use message passing to compute a well-known upper bound p̂ ( u|S ) of p ( u|S ) for node u : p̂ ( u|S ) = Au ·X = ∑ v∈N ( u ) ∩S 1 deg ( u ) = ∑ v∈N ( u ) ∩S pvu ≥ 1− ∏ v∈N ( u ) ∩S ( 1− pvu ) = p ( u|S ) , ( 3 ) where the second equality stems from the definition of the weighted cascade and the inequality from the proof in Zhou et al . ( 2015 ) , App . A . As the diffusion covers more than one-hop , the derivation requires repeating the multiplication to approximate the total influence spread . To be specific , computing the influence probability of nodes that are not adjacent to the seed set requires estimating recursively the probability of their neighbors being influenced by the seeds . If we let H1 = A ·X , and we assume the new seed set St to be the nodes influenced in the step t− 1 , their probabilities are stored in Ht , much like a diffusion in discrete time . We can then recompute the new influence probabilities with Ht+1 = A ·Ht , Corollary 3.0.1 . The repeated product Ht+1 = A·Ht computes an upper bound to the real influence probabilities of each infected node at step t+ 1 . The proof can be found in Appendix A.1 . In reality , due to the existence of cycles , two problems arise with this computation . Firstly , if the process is repeated the influence of the original seeds may increase again , which comes in contrast with the independent cascade model . This can be controlled by minimizing the repetitions , e.g. , four repetitions cause the original seeds to be able to reinfect other nodes in a network with triangles . To this end , we leverage up to three neural network layers . Another problem due to cycles pertains to the probability of neighbors influencing each other . In this case , the product of the complementary probabilities in Eq . ( 3 ) does not factorize for the non-independent neighbors . This effect was analyzed extensively in Lokhov & Saad ( 2019 ) , App . B , and proved that the influence probability computed by p ( u|S ) is itself an upper bound on the real influence probability for graphs with cycles . Intuitively , the product that represents non-independent probabilities is larger than the product of independent ones . This renders the real influence probability , which is complementary to the product , smaller than what we compute . We can thus contend that the estimation p̂ ( u|S ) provides an upper bound on the real influence probability—and we can use it to compute an upper bound to the real influence spread of a given seed set i.e. , the total number of nodes influenced by the diffusion . Since message passing can compute inherently an approximation of influence estimation , we can parameterize it to learn a function that tightens this approximation based on supervision . In our neural network architecture , each layer consists of a GNN with a batchnorm and dropout omitted here , and starting from H0 = X ∈ Rn×d we have : Ht+1 = ReLU ( [ Ht , AHt ] W0 ) . ( 4 ) The readout function that summarizes the graph representation based on all nodes ’ representations is a summation over all the final representations with skip connections : HGS = ∑ v∈V [ Hv0 , H v 1 , . . . , H v t ] . ( 5 ) This representation captures the probability of all nodes being active throughout each layer . The output that represents the predicted influence spread is derived by : σ̂ ( S ) = ReLU ( HGSWo ) . ( 6 ) Note that , the derived representations of each layerHit , untrained , are the upper bound of the influence probability of seed set ’ s the t-hop neighbors . The parameters of the intermediate layers Wt are trained such that the upper bound is reduced and the final layer Wo can combine the probabilities to derive a cumulative estimate for the total number of influenced nodes . We empirically verify this by examining the layer activations which can be seen in Fig . 1 and the heatmaps indicate a difference between columns ( nodes ) expected to be influenced , meaning we could potentially predict not only the number but also who will be influenced . However , since σ̂ is derived by multiple layers , the relationships and thresholds to determine the exact influenced set is not straight forward . Below we experiment with different such sets extracted from H for the purpose of IM .

Summary: In this paper, the authors consider using learning-based method for Influence Estimation and Influence Maximization. The author proposed a GNN-based to estimate influence (as an upper bound). Based on the estimated influence, the authors use CELF optimization to find the optimal seed set. To further improve the efficiency, the authors proposed (1) a RL DQN based method and (2) a simplified influence function with only one layer. The author carries out experiments on several synthetic and real-world datasets.