Decomposing Huge Networks into Skeleton Graphs by Reachable Relations

We developed new techniques for accelerating critical link detection and distance-based centrality computation by decomposing huge networks into skeleton graphs by reachable relations. The main research results are on three new approaches, 1) efficient detection of critical links in a large network by using bottom-k sketch algorithm and further by employing two new acceleration techniques: marginal-link updating and redundant-link skipping (Saito, et al. 2016), 2) accurate and efficient detection of such critical links by proposing a new method which consists of one existing and two new acceleration techniques: redundant-link skipping, marginal-node pruning and burn-out following (Saito, et al. 2017), and 3) accelerating computation of distance-based closeness and betweenness centrality measures by pruning some nodes and links based on the cut links of a given spatial network (Ohara, et al. 2016). First, we addressed the problem of efficiently detecting critical links in a large network. Critical links are such links that their deletion exerts substantial effects on the network performance. In our research, we defined the performance as being the average node reachability. We tackled this problem by using bottom-k sketch algorithm and further by employing two new acceleration techniques: marginal-link updating (MLU) and redundant-link skipping & pruning (RLS), where RLS decomposes huge networks into skeleton graphs by pruning redundant links. We tested the effectiveness of the proposed method using two real-world large networks and two synthetic large networks and showed that the new method can compute the performance degradation by link removal about an order of magnitude faster than the baseline method in which bottom-k sketch algorithm is applied directly. Further, we confirmed that the measures easily composed by well-known existing centralities, e.g. in/out-degree, betweenness, PageRank, authority/hub, are not able to detect critical links. Links detected by these measures do not reduce the average reachability at all, i.e. not critical at all. Second, we also tackled the same problem of efficiently detecting critical links in a large network problem by proposing a new method which consists of one existing and two new acceleration techniques: redundant-link skipping & pruning (RLS), marginal-node pruning (MNP) and burn-out following (BOF), where RLS and MNP decompose huge networks into skeleton graphs by pruning redundant links and nodes. All of them are designed to avoid unnecessary computation and work both in combination and in isolation. We also tested the effectiveness of the proposed method using two real-world large networks and two synthetic large networks. In particular, we showed that the new method can compute the performance degradation by link removal without introducing any approximation within a comparable computation time needed by the bottom-k sketch which DISTRIBUTION A. Approved for public release: distribution unlimited. is a summary of dataset and can efficiently process approximate queries, i.e., reachable nodes, on the original dataset, i.e., the given network. Finally, by focusing on spatial networks embedded in the real space, we extended the conventional step-based closeness and betweenness centralities by incorporating inter-nodes link distances obtained from the positions of nodes. Then, we proposed a method for accelerating computation of these centrality measures by pruning some nodes and links based on the cut links of a given spatial network, which performs a decomposition into its skeleton graph. In our experiments using spatial networks constructed from urban streets of cities of several types, our proposed method achieved about twice the computational efficiency compared with the baseline method. Actual amount of reduction in computation time depends on network structures. We further experimentally showed by examining the highly ranked nodes that the closeness and betweenness centralities have completely different characteristics to each other.