Paper:
Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss functions and training nonconvex deep neural networks. The theory assumes that one can easily compute an unbiased gradient estimator, which is usually the case due to the sample average nature of empirical risk minimization. There exist, however, many scenarios (e.g., graphs) where an unbiased estimator may be as expensive to compute as the full gradient because training examples are interconnected. Recently, Chen et al. (2018)  proposed using a consistent gradient estimator as an economic alternative. Encouraged by empirical success, we show, in a general setting, that consistent estimators result in the same convergence behavior as do unbiased ones. Our analysis covers strongly convex, convex, and nonconvex objectives. We verify the results with illustrative experiments on synthetic and real-world data. This work opens several new research directions, including the development of more efficient SGD updates with consistent estimators and the design of efficient training algorithms for large-scale graphs.

Review:
The analysis of the convergence of SGD with biases gradient estimates dates back to Robins&Monroe, but the authors of this paper focused on a recent original algorithm that shows that once can estimate the approximate gradient of a large GNN network, simply by sampling nodes randomly.
When I first read of the paper, I was enthusiastic because I did not know the FastGCN approach presented at ICLR the previous year, which showed that the gradient of a GCN could be efficiently approximated by sampling a subset of the nodes. After reading FastGCN, I was less enthusiastic as most of the originality relied on the consistent estimate of the gradient, when t (number of sampled in the neighbours of the output nodes) increases.
The main contribution of the paper is the proof that the algorithm converge, but there is no theoretical analysis of the key quantity "t", which is the number of sampled nodes in the neighbours of the output nodes. I would expect to see the number of sample grow as the algorithm converge to the optimal solution since the gradient needs less bias when the algorithm converges. However, the authors do not address this point.
I did not into the details of the proofs, but it seems to me that they are quite loose and several details such as the functional spaces, and the boundedness assumptions, are not mentioned. Here are few examples:
- In the first sentence: P(x, y) of data x and associated label y. The space of x, the space of y and the probability space are not defined. In fact, no set in which variables belong is defined in the paper.
- The Theorem 1 is strange to me. I would assume that one needs some assumption of boundedness of Q and finite moments for P to avoid pathological examples where the integral (for the asymptotic expectation) is infinite, but the finite sum G_{st} is always finite, contradicting the limit in theorem 1. 
Overall, while proving that the FastGCN algorithm is consistent is important, it is hard to understand how useful the results are and how they can be useful in practice. For example, what can we interpret or what can we learn from the bounds given by Theorem 2?
Finally, I might miss something, but the empirical results showed do not seem to show better gains than the Adam algorithm. The theory shows that the more bias we have, the less accurate we should be, why isn't it apparent in Table 1. Is there something such as the computational cost of Adam, that I'm missing, especially when looking at the graphs? I'm sorry if I did not get the main message of the experiments, but even after reading the paper 3 times, I did not understand what the authors wanted the reader to conclude with these experiments.