BCDR: Betweenness Centrality-based Distance Resampling for Graph Shortest Distance Embedding

Along with unprecedented development in network analysis such as biomedical structure prediction and social relationship analysis , Shortest Distance Queries ( SDQs ) in graphs receive an increasing attention . Approximate algorithms of SDQs with reduced complexity are of vital importance to complex graph applications . Among different approaches , embedding-based distance prediction has made a breakthrough in both efficiency and accuracy , ascribing to the significant performance of Graph Representation Learning ( GRL ) . Embedding-based distance prediction usually leverages truncated random walk followed by Pointwise Mutual Information ( PMI ) -based optimization to embed local structural features into a dense vector on each node and integrates with a subsequent predictor for global extraction of nodes ’ mutual shortest distance . It has several shortcomings . Random walk as an unstrained node sequence possesses a limited distance exploration , failing to take into account remote nodes under graph ’ s shortest distance metric , while the PMI-based maximum likelihood optimization of node embeddings reflects excessively versatile local similarity , which incurs an adverse impact on the preservation of the exact shortest distance relation during the mapping from the original graph space to the embedded vector space . To address these shortcomings , we propose in this paper a novel graph shortest distance embedding method called Betweenness Centrality-based Distance Resampling ( BCDR ) . First , we prove in a statistical perspective that Betweenness Centrality ( BC ) -based random walk can occupy a wider distance range measured by the intrinsic metric in the graph domain due to its awareness of the path structure . Second , we perform Distance Resampling ( DR ) from original walk paths before maximum likelihood optimization instead of the PMI-based optimization and prove that this strategy preserves distance relation with respect to any calibrated node via steering optimization objective to reconstruct a global distance matrix . Our proposed method possesses a strong theoretical background and shows much better performance than existing methods when evaluated on a broad class of realworld graph datasets with large diameters in SDQ problems . 1 INTRODUCTION . Shortest Distance Queries ( SDQs ) in graphs focus on determining the minimum distance between arbitrary node pairs , defined as sum of related edge weights . Despite the fact that Shortest Path Queries ( SPQs ) ( Wei , 2012 ; Sommer , 2014 ; Wang et al. , 2021 ) are already renowned in graph structure exploration , SDQs play an essential role in increasing applications , such as social relationship analysis ( Carlton , 2020 ; Melkonian et al. , 2021 ; Chen et al. , 2021 ; Zaki et al. , 2021 ; Parveen & Varma , 2021 ) , biomedical structure prediction ( Yue et al. , 2019 ; Galovičová et al. , 2021 ; Sokolowski & Wasserman , 2021 ) , learning theory ( Yang et al. , 2021 ; Yuan et al. , 2021 ) , optimization ( Melkonian et al. , 2021 ; Rahmad Syah et al. , 2021 ; Jiang et al. , 2021 ) , etc . The key challenge in the SDQ problem is the prohibitive complexity in very large graphs . e.g. , for an undirected dense graph with N nodes and k queries , the time complexity of A * ( Hart et al. , 1968 ) and Dijkstra Algorithm ( Thorup & Zwick , 2004 ) are up toO ( kN2 ) andO ( kN logN ) for unweighted and weighted graph , respectively . Table 1 : Overall comparison of approaches to shortest distance query . Accuracy loss of approximate methods are evaluated on facebook ( A.7.2 ) by mRE ( 2 ) . N -number of nodes , T -walk length , K-window size , l-number of landmarks , d-dimension of embedding space , α , β , c-constant independent of N . Objective Method Model Off-Line Time Space Time Accuracy Loss Exact Compress - O ( N3 ) O ( N2 ) O ( kN ) 0 Exact Index - O ( N2 logN ) O ( N2 ) O ( kN logN ) 0 Exact Cache - − O ( N ) O ( N ) ∼ O ( N2 ) 0 Approximate Oracle - O ( αN1+ 1 α ) O ( αN1+ 1 α ) O ( kα ) 2 ( α− 1 ) Approximate Landmark - O ( lN logN + lN ) O ( lN ) O ( kl ) 0.523 Approximate Embedding Orion O ( αN logN + cN ) O ( dN ) O ( kd ) 0.688 Rigel O ( αN logN + cN ) O ( dN ) O ( kd ) 0.369 DeepWalk O ( TKN logN + ( T + c ) N ) O ( dN ) O ( kd ) 0.329 Node2Vec O ( TKN logN + ( T + c ) N ) O ( dN ) O ( kd ) 0.299 DADL O ( TKN logN + ( T + c ) N ) O ( dN ) O ( kd ) 0.228 BCDR ( ours . ) O ( αTKN logN + ( βT + c ) N ) O ( dN ) O ( kd ) 0.177 To address this issue , a surging number of approximate algorithms in a wide range of fields have been proposed in the past a few years . They can be categorized into oracle-based ( Thorup & Zwick , 2004 ; Baswana & Kavitha , 2006 ) , landmark-based ( Sarma et al. , 2010 ; Gubichev et al. , 2010 ) , embeddingbased distance prediction methods ( Xiaohan et al. , 2010 ; 2011 ; Rizi et al. , 2018 ) . Among these categories , embedding-based methods are of high accuracy with significantly reduced complexity ( see Table1 ) , owing much to rapid advances of representation learning in graphs ( Perozzi et al. , 2014 ; Grover & Leskovec , 2016 ) . Embedding-based methods tackle the SDQ problem with two stages . First , they embed local structural features into a dense vector on each node , which preserves the essential information for indicating where the node is . Then , for arbitrary node pairs in the training set , subsequent predictor extracts mutual shortest distance globally via their embedding tuples and minimizes the mean of square loss between a predicted value and the real one . As predictors serve as a non-linear metric in the embedding space , the performance of embeddingbased models depends highly on the first stage . These methods usually embed graph structures by leveraging truncated random walk to serialize node ’ s neighborhood in a statistical perspective and maximize co-occurrence likelihood of nodes in one walk path to reflect their correlation , which is also proved to implicitly factorize a Pointwise Mutual Information ( PMI ) matrix ( Levy & Goldberg , 2014 ; Shaosheng et al. , 2015 ) . Although existing embedding-based methods have achieved a great success , they have several shortcomings . On one side , random walk is an unstrained node sequence from the root , possessing a limited distance exploration . This is because each transition on nodes is not implied for a specific direction to move towards or beyond the root , especially after several walk steps , which is backbreaking to accommodate correlation with remote nodes under graph ’ s shortest distance metric ( see Figure 1a ) . On the other side , the PMI-based optimization reflects excessively versatile local sim- ilarity , which is not guaranteed for shortest distance-preserved mapping from the original graph space to the embedded vector space . As a matter of fact , it exerts a too-general metric over nodes ’ correlation , wherein the more links or paths exist between two nodes , the stronger correlation they share . That means we have many ways to claim a shorter distance for two nodes ( e.g. , by adding mutual edges , deleting links to other nodes ) even if some of the operations do not change their actual shortest path distance ( see Figures 1c and 1d ) . In this paper , we address the above shortcomings by proposing a novel graph shortest distance embedding method called Betweenness Centrality-based Distance Resampling ( BCDR ) . Here , random walk paths are simulated by considering nodes ’ betweenness centrality on each transition for covering a wider distance range of nodes . Then , a sampling process of nodes based on their mutual shortest distances is performed before optimization of co-occurrence likelihood to preserve pairwise distance relation during the mapping from graph to the embedding space . We summarize our major contributions as follows . • We propose a Betweenness Centrality ( BC ) -based random walk for accommodating correlation with nodes of wider distance range under the intrinsic graph metric ( see Section 3.1 ) . To the best of our knowledge , there is no existing method that combines betweenness centrality and random walk for the graph representation . We prove in a statistical perspective that the transition driven by nodes ’ BC value tends to explore high order proximity of the root due to the awareness of local path structure . • We propose a Distance Resampling ( DR ) strategy to preserve nodes ‘ mutual shortest distance during the mapping from graph to the embedding space ( see Section 3.2 ) . We prove that by sampling node sequences from original walk paths before maximum likelihood optimization instead of PMI-based optimization , the objective can be steered to reconstruct a global distance matrix , and for any calibrated node , it strictly preserves the shortest distance relation to other nodes on the graph . • We evaluate BCDR and compare it with existing embedding-based methods with a broad class of real-world graph datasets with divergent diameters . BCDR shows a much better empirical performance in solving the SDQ problem than existing embedding-based methods ( see Section 4 ) . 2 NOTATION & PROBLEM DEFINITION . 2.1 NOTATION . G = ( V , E ) denotes an undirected graph , with V = { v1 , v2 , ... , v|V | } being the set of nodes and E being the set of edges . An edge eij = ( vi , vj ) represents an undirected edge between nodes vi and vj . A node vi ’ s neighborhood Ni is a set of nodes with an edge with vi , i.e. , Ni = { vj | ( vi , vj ) ∈ E } . We use Z|V |×d to represent an embedding matrix , where d is the embedding size . For any matrix B , the symbol Bi represents the i-th row of B , and Bij represents the element at i-th row and j-th column . A truncated random walk Wi rooted at node vi of length l is a random vector of 〈W1i , W2i , · · · , W li〉 , where Wki is a node chosen from the neighborhood of node W k−1 i for k = 1 , ... , l , withW0i : = vi . Pi is a finite set of walk paths sampled fromWi , and Wi is the multiset of nodes on the walk paths in Pi . 2.2 PROBLEM DEFINITION . An arbitrary path p of length l ∈ N on graph G is an ordered sequence of nodes ( v1 , v2 , ... vl+1 ) , where each node except the last one has an edge with the subsequent node , i.e. , ( vi , vi+1 ) ∈ E for 1 ≤ i ≤ l. The shortest path p̊ij is one of the paths with the minimum length l̊ between two nodes vi and vj , with the shortest distance Dij defined as the length of p̊ij . The global distance matrix D comprises { Dij } . An embedding-based model learns two mappings F and G for structure embedding and mutual distance extraction , respectively , as follows . F : G→ Rd , G : 〈Rd , Rd〉 → R ( 1 ) A shortest distance query Q can be simply defined as a set of node pairs , i.e. , Q = { ( v11 , v12 ) , ( v21 , v22 ) , ... , ( v|Q|,1 , v|Q|,2 ) } . For each node pair ( vi , vj ) , the model intends to find out approximation D̂ij = G ( F ( vi ) , F ( vj ) ) nearest to real Dij . The commonly used metrics of approximation quality are mean of Relative Error ( mRE ) and mean of Absolute Error ( mAE ) . mRE is defined as the relative loss of the prediction value with respect to the real value , while mAE measures the absolute gap between the prediction value and the real value , i.e. , mRE : = 1 |Q| ∑ ( vi , vj ) ∈Q |D̂ij −Dij | Dij , mAE : = 1 |Q| ∑ ( vi , vj ) ∈Q |D̂ij −Dij | ( 2 )

Summary: In this work the author(s) proposed a framework for node embeddings that captures better shortest path distances for undirected graphs. Specifically, the authors propose a new random walk framework based on betweenness centralities, and a distance resampling strategy that uses the shortest path distances, and the betweenness centrality scores and is shown to capture well shortest path distances (Proposition 1).  The authors also evaluate their framework experimentally, verifying that compared to other popular node embedding methods, it can be used to represent shortest path distances faithfully.