Paper:
Multi-Scale Spectral Decomposition of Massive Graphs
Computing the k dominant eigenvalues and eigenvectors of massive graphs is a key operation in numerous machine learning applications; however, popular solvers suffer from slow convergence, especially when k is reasonably large. In this paper, we propose and analyze a novel multi-scale spectral decomposition method (MSEIGS), which first clusters the graph into smaller clusters whose spectral decomposition can be computed efficiently and independently. We show theoretically as well as empirically that the union of all cluster's subspaces has significant overlap with the dominant subspace of the original graph, provided that the graph is clustered appropriately. Thus, eigenvectors of the clusters serve as good initializations to a block Lanczos algorithm that is used to compute spectral decomposition of the original graph. We further use hierarchical clustering to speed up the computation and adopt a fast early termination strategy to compute quality approximations. Our method outperforms widely used solvers in terms of convergence speed and approximation quality. Furthermore, our method is naturally parallelizable and exhibits significant speedups in shared-memory parallel settings. For example, on a graph with more than 82 million nodes and 3.6 billion edges, MSEIGS takes less than 3 hours on a single-core machine while Randomized SVD takes more than 6 hours, to obtain a similar approximation of the top-50 eigenvectors. Using 16 cores, we can reduce this time to less than 40 minutes.

Review:
The paper addresses the problem of estimation of eigenvalues and eigenvectors of large sparse graph matrices. It uses a nice divide-and-conquer approach to obtain better estimates of top-k eigenspace. Such an estimate can be used in several classic tasks, such as link prediction and recommender systems.The paper is build upon a divide-and-conquer method, taking advantage of the particular characteristics of some graphs (e.g. social networks): sparsity and cluster structure.The theoretical part is strong, with theoretical bounds provided. The divide part uses a decomposition of the adjacency matrix as a sum of a block-diagonal matrix and an almost zero non-diagonal matrix. A spectral decomposition of block matrices is done, and the k first eigen values are chosen among the resulting union of eigenvalues.In the conquer step, the corresponding eigenvectors are used as initialization step in a block-Lanczos (or random SVD) method on the whole matrix.Theoretical bounds are provided, with respect to the norm of the remaining matrix (the non-block one).Experiments show that the method is quite accurate for the eigenspace identification, compared to classic spectral methods (randomized SVD or Lanczos).In tasks such as label propagation and matrix completion, the methods performs as well as other classical methods.The paper is well written. It uses quite sophisticated methods and the theoretical part is solid. The main weakness of the paper is that the tasks that are presented as highly dependent on the accuracy of eigenspace estimation (link prediction, etc.) reveals to be not so dependent: the experiments show that the method presented in the paper is much more accurate than previous state-of-the-art in eigenspace estimation, but this does not implies significantly better results in the label propagation or matrix completion tasks.In other words, The goal of the method (a good estimate of the eigenspace) is reached, but it appears that it has no effect on the tasks that it was supposed to solve (except for the computing time).Also, the method (and the bound) should highly depend on the quality of the first step (the graph clustering step), and it would have been nice to have an idea of the accuracy of the method with respect to this first step.However, the method is original and could be used in other spectral decomposition problems (provided that they are sparse and likely to be clustered). Pros:- nice divide-and-conquer algorithm- solid theoretical part- theoretical bounds- experiments show that the method reaches good accuracyCons:- for real-world tasks, the algorithm does not perform significantly better that previous methods- dependency between first clustering step and quality of the solution is not studied.