Paper:
Safe Adaptive Importance Sampling
Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants-using importance values defined by the complete gradient information which changes during optimization-enjoy favorable theoretical properties, but are typically computationally infeasible. In this paper we propose an efficient approximation of gradient-based sampling, which is based on safe bounds on the gradient. The proposed sampling distribution is (i) provably the best sampling with respect to the given bounds, (ii) always better than uniform sampling and fixed importance sampling and (iii) can efficiently be computed-in many applications at negligible extra cost. The proposed sampling scheme is generic and can easily be integrated into existing algorithms. In particular, we show that coordinate-descent (CD) and stochastic gradient descent (SGD) can enjoy significant a speed-up under the novel scheme. The proven efficiency of the proposed sampling is verified by extensive numerical testing.

Review:
The authors present a "safe" adaptive importance sampling strategy for coordinate descent and stochastic gradient methods. Based on lower and upper bounds on the gradient values, an efficient approximation of gradient based sampling is proposed. The method is proven to be the best strategy with respect to the bounds, always better than uniform or fixed importance sampling and can be computed efficiently for negligible extra cost. Although adaptive importance sampling strategies have been previously proposed, the authors present a novel formulation of selecting the optimal sampling distribution as a convex optimization problem and present an efficient algorithm to solve it.
This paper is well written and a nice contribution to the study of importance sampling techniques.
Comments:
Proof of Lemma 2.1 -- seems to be missing a factor of 2 in alpha^*.
Example 3.1 - In (7) you want to maximize, so in example 3.1, it would appear that setting c to the upper or lower bound is better than using uniform sampling. Is that what this example is intending to show? It is confusing with the statement directly prior claiming the naive approach of setting c to the upper or lower bound can be suboptimal.
Line 4 of Algorithm 4 - given that m = max(l^{sort}), will this condition ever be satisfied?
Line 7 of Algorithm 4 - should be u^{sort} instead of c^{sort}?
I think the numerical results could benefit from comparisons to other adaptive sampling schemes out there (e.g., [2], [5],[21]), and against fixed importance sampling with say a non-uniform distribution. 
Why are there no timing results for SGD?
Title of the paper in reference [14] is incorrect. 
Add reference: Csiba and Richtarik. "Importance Sampling for Minibatches" 2016 on arXiv. 
============
POST REBUTTAL
============
I have read the author rebuttal and I thank the authors for addressing my questions/comments. I think this paper is a clear accept.