Paper:
Recursive Sampling for the Nyström Method
We give the first algorithm for kernel Nyström approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of s landmark points sampled by their ridge leverage scores, requiring just O(ns) kernel evaluations and O(ns 2 ) additional runtime. While leverage score sampling has long been known to give strong theoretical guarantees for Nyström approximation, by employing a fast recursive sampling scheme, our algorithm is the first to make the approach scalable. Empirically we show that it finds more accurate kernel approximations in less time than popular techniques such as classic Nyström approximation and the random Fourier features method.

Review:
The authors provide an algorithm which learns a provably accurate low-rank approximation to a PSD matrix in sublinear time. Specifically, it learns an approximation that has low additive error with high probability by sampling columns from the matrix according to a certain importance measure, then forming a Nystrom approximation using these kernels. The importance measure used is an estimate of the ridge leverage scores, which are expensive to compute exactly ( O(n^3) ). Their algorithm recursively estimates these leverage score by starting from a set of columns, using those to estimate the leverage scores, sampling a set of columns according to those probabilities, and repeating ... the authors show that when this process is done carefully, the leverage score estimates are accurate enough that they can be used to get almost as good an approximation as using the true ridge leverage scores. The cost of producing the final approximation is O(ns^2) computation time and O(ns) computations of entries in the PSD matrix.
This is the first algorithm which allows touching only a linear number of entries in a PSD matrix to obtain a provably good approximation --- previous methods with strong guarantees independent of the properties of the PSD matrix required forming all n^2 entries in the PSD matrix. The experimental results show that the method provides more accurate kernel approximations than Nystrom approximations formed using uniform column samples, and kernel approximations formed using Random Fourier Features. These latter two are currently the most widely used randomized low-rank kernel approximations in ML, as they are cheap to compute. A heuristic modification of the algorithm brings its runtimes down to close to that of RFFs and uniform sampled Nystrom approximations, but retains the high accuracy.
I see one technical problem in the paper: the proof of Lemma 5, in the display following line 528, the first inequality seems to use the fact that if W is a diagonal matrix and 0 <= W <= Id in the semidefinite sense, then for any PSD matrix A, we have WAW <= A. This is not true, as can be seen by taking A to be the rank one outer product of [1;-1], W to be diag(1, 1/2), and x = [1, 1]. Then x^TWAWx > x^tAx = 0. Please clarify the proof if the first inequality holds for some other reason, or otherwise establish the lemma.
Comments:
-In section 3.2, when you introduce effective dimension, you should also cite "Learning Bounds for Kernel Regression Using Effective Data Dimensionality", Tong Zhang, Neural Computation, 2005. He calls the same expression the effective dimension of kernel methods
-Reference for the claim on lines 149-150? Or consider stating it as a simple lemma for future reference
-line 159, cite "On the Impact of Kernel Approximation on Learning Accuracy", Cortes et al., AISTATS, 2010 and "Efficient Non-Oblivious Randomized Reduction for Risk Minimization with Improved Excess Risk Guarantee", Xu et al., AAAI, 2017
-line 163, the reference to (13) should actually be to (5)
-on line 3 of Alg 2, I suggest changing the notation of the "equals" expression slightly so that it is consistent with the definition of \tilde{l_i^lambda} in Lemma 6
-the \leq sign in the display preceding line 221 is mistyped
-the experimental results with Gaussian kernels are very nice, but I think it would be useful to see how the method performs on a wider range of kernels. 
-would like to see "Learning Kernels with Random Features" by Sinha and Duchi referenced and compared to experimentally
-would like to see "Learning Multidimensional Fourier Series With Tensor Trains", Wahls et al., GlobalSIP, 2014 cited as a related work (they optimize over frequencies to do regression versus 'random' features)