Paper:
We present a novel approach for adaptively selecting the learning rate in gradient descent methods. Specifically, we impose a regularization term on the learning rate via a generalized distance, and cast the joint updating process of the parameter and the learning rate into a maxmin problem. Some existing schemes such as AdaGrad (diagonal version) and WNGrad can be rederived from our approach. Based on our approach, the updating rules for the learning rate do not rely on the smoothness constant of optimization problems and are robust to the initial learning rate. We theoretically analyze our approach in full batch and online learning settings, which achieves comparable performances with other first-order gradientbased algorithms in terms of accuracy as well as convergence rate.

Review:
This paper presents a method for adaptively tuning the learning rate in gradient descent methods. The authors consider the formulation of each gradient descent update as a quadratic minimization problem and they propose adding a phi-divergence between the learning rate that would be used and an auxiliary vector. The authors also propose adding a maximization over all learning rates in the update.  
The authors study an important problem and propose a novel method. The algorithms suggested by the author are also relatively clear, and it is great that the paper presents both theoretical results as well as numerical experiments.
On the other hand, I didn't find the main idea of hyper-regularization to be well-justified. It is not clear why adding an additional regularization term for the learning rate makes sense , and it is even less clear why this should be presented as a maxmin problem. This can make the update step much more complicated and is probably why the authors also propose a simpler alternating optimization algorithm as an alternative. Unfortunately, the authors do not discuss how this alternating optimization problem relates to the original one, and the theoretical guarantees are only presented for the original algorithm. The authors also do not justify the choice of phi-divergence as the regularizer for the learning rate. The theoretical guarantees in the paper also do not suggest that the algorithm presented in the paper is better than existing state-of-the-art methods, even in specific situations (i.e. the regret bounds don't appear better than the AdaGrad regret bounds). Moreover, without tests for statistical significance, I also didn't find the experimental results sufficiently compelling.
Specific comments and questions:
1) Page 3: Equation (4): The paper would be stronger if the authors motivated why the regularization should be posed as an outer maximization.
2) Page 3: "we use the \phi-divergence as our hyper-regularization". Why is this a good choice of reuglarizer?
3) Page 3: "only a few extra calculations are required for each step". This is a misleading comment, because the maximization can be hard when phi is complicated, even if the problem splits across dimensions.
4) Page 4: "The solution of problem (5) is the same as (7) in unconstrained case". You should provide a reference for this statement as well as discuss the specific assumptions on the objective that allow you to arrive at this claim.
5) Page 4: "while the solution of (7) is more difficult to get. Thus, we choose (5) as our basic problem". This seems like a very bad motivation for choosing the maxmin formulation. For instance, the problem would be even simpler if  you didn't include this extra phi-divergence at all.
6) Page 4: "Although setting \eta-t=\beta_t is our main focus...". Why is smoothness in the learning rate a good property? 
7) Page 5: Equation (11). How do these iterates relate to the ones in equation (5) (e.g. when do they coincide, if ever)?
8) Page 5: "influence the efficient of our algorithms." Grammatical error.
9) Page 6. "our algorithms are robust to the choice of initial learning rates and do not rely on the Lipschitz constant or smoothness constant". I'm not sure why this is a valuable property, since AdaGrad doesn't rely on these parameters either.
10) Page 6: Theorems 6 and 7. How do these results depend on alpha and \beta_t? This paper would be much stronger if the bounds depend on \phi more clearly and if the authors were able to show that there exist choices of phi that make this algorithm better than existing methods.
11) Page 6: Theorem 7: The dependence on G in the regret bound actually makes this worse than the AdaGrad regret bound.
12) Page 7: "KL_devergence". Typo.
13) Page 7: "different update rules were compared in advance to select the specific one for any phi divergence in the following experiments." What does this mean exactly? How much of a difference does the choice of update rule make?
14) Page 7: "growth clipping is applied to all algorithms in our framework". Why is this necessary, and how does it affect the theoretical results?
14) Page 7-8: Figures 1, 2, and 3. It's hard to interpret the significance of these results without error bars.