Title: Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space
Venue: NeurIPS 2025
Accepted: yes
Keywords: Riemannian optimization

Acceleration

Wasserstein space
Abstract: There is extensive literature on accelerating first-order optimization methods in an Euclidean setting. Under which conditions such acceleration is feasible in Riemannian optimization problems is an active area of research. Motivated by the recent success of silver stepsize methods in the Euclidean setting, we undertake a study of such algorithms in the Riemannian setting. We provide the new class of algorithms determined by the choice of vector transport that allows the silver stepsize acceleration on Riemannian manifolds for the function classes associated with the corresponding vector transport. As a core application, we show that our algorithm recovers the standard Wasserstein gradient descent on the 2-Wasserstein space and, as a result, provides the first provable accelerated gradient method for potential functional optimization problems in the Wasserstein space. In addition, we validate the numerical strength of the algorithm for standard benchmark tasks on the space of symmetric positive definite matrices.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the silver step-sizes for Riemannian manifold optimization. Silver step-sizes are a recently proposed technique that can accelerate gradient descent without explicitly incorporating momentum terms. The authors extend this idea to the Riemannian setting and show that silver step-sizes can lead to improved worst-case complexity bounds for optimization on manifolds. The paper also discusses two applications of the proposed method: optimization on the 2-Wasserstein space and Rayleigh quotient maximization.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper addresses an important and fundamental problem: how to accelerate gradient descent methods on Riemannian manifolds. The authors propose using the recently introduced silver step-size to achieve acceleration in this setting. Both Riemannian manifold optimization and gradient-based methods are important areas of research.\n\nIn terms of presentation, I appreciate the clarity with which the paper defines the necessary preliminaries, as well as the additional preliminaries included in the appendix. The exposition is generally clear and accessible. However, as someone not deeply familiar with Riemannian optimization, I am not in a good position to verify the technical correctness of the mathematical arguments. \n\nAssuming the theoretical results are correct, I believe the contribution is significant. While the proposed step-size rule is directly borrowed from the Euclidean silver step-size literature, extending it to the Riemannian setting looks nontrivial. This generalization is meaningful, especially given that optimization on manifolds is less mature. From this perspective, the results presented in the paper are valuable.\n\nOne limitation is that, unlike in the Euclidean setting, the paper does not provide a corresponding lower bound to demonstrate the optimality of the obtained complexity result. Although this is unfortunate, it is understandable given the increased technical complexity of the manifold setting.\n\nAnother strength of the paper is the inclusion of two applications, in which the proposed method demonstrates improved empirical performance. Given that the practical value of the silver step-size in Euclidean settings is still valuable than its theoretical value, these positive experimental results are encouraging.\n\nQUESTIONS:\nThe silver step-size was originally inspired by recent advances in the Performance Estimation Problem (PEP). In this paper, it is not clear whether PEP plays any role in the analysis or proof techniques. Could the authors clarify whether any part of the theoretical results, was informed or assisted by PEP? Is it possible to apply the PEP to the Riemannian setting considered in this paper, in order to derive a lower bound result?\n\nLIMITATIONS:\nYes. This paper raises some open questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes the Riemannian gradient descent method under a varying silver stepsize schedule~\\cite{altschuler2024acceleration}, a strategy recently proposed and successfully applied in Euclidean spaces. The authors prove that, under a newly introduced notion of generalized geodesic convexity, the Riemannian gradient descent algorithm achieves an accelerated convergence rate on manifolds with non-negative curvature—consistent with the Euclidean case, i,e, $O(1/k^{\\alpha})$, $\\alpha \\approx 1.2716$.\n\nMoreover, by incorporating a restarting scheme, the authors extend the silver stepsize analysis to the setting of geodesically strongly convex functions, without requiring any modification to the stepsize itself.\n\nAs an application, the paper develops a theoretically accelerated gradient optimization method on Wasserstein spaces. Numerical experiments further demonstrate the effectiveness of the variable stepsize strategy on the Rayleigh quotient maximization problem over the sphere manifold.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The paper introduces a new notion of generalized geodesic convexity and gives some examples.\n* The proposed Riemannian gradient descent method with silver stepsize is theoretically accelerated under non-negatively curved Riemannian manifolds.\n* The paper includes applications on the Wasserstein space and numerical experiments of Rayleigh quotient maximization on the sphere.\n\t\nWeaknesses:\n* The silver step-size schedule is not clearly explained.\n* Several key theoretical arguments, including Lemma 5.4 and its proof, are not clearly presented.\n* The study focuses on the exponential and logarithmic maps on the manifold, while omitting the consideration of the retraction map.\n\nQUESTIONS:\n* Can the assumption of generalized geodesic convexity be relaxed to standard geodesic convexity?\n* What challenges arise in establishing convergence rate estimates when $n\\neq 2^{k}-1$?\n* Is it possible to achieve acceleration with silver step-size for geodesically $\\alpha$-strongly convex functions without employing a restarting strategy?\n\nLIMITATIONS:\nThe present analysis is confined to manifolds with non-negative curvature and has not been extended to general manifolds. \nAdditionally, the method relies on the exponential and logarithmic maps, which may limit its applicability to some manifold-structured problems.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates Riemannian Gradient Descent (RGD) with silver step sizes. It demonstrates that, for optimization problems defined on non-negatively curved Riemannian manifolds, where the objective function is generalized geodesically convex and geodesically smooth, vanilla RGD can achieve accelerated convergence when the step size sequence is carefully chosen—specifically, using silver step sizes. The derived convergence rate is approximately $\\mathcal{O}(1/n^{1.27})$, improving upon the known $\\mathcal{O}(1/n)$ rate for RGD with constant step sizes.\n\nIn the strongly convex setting, the paper shows that RGD with silver step sizes and restarts converges exponentially fast, similar to RGD with constant step sizes. However, it offers an improved dependence on the condition number: the convergence rate involves a coefficient scaling as $\\kappa^{0.786}$, compared to $\\kappa$ for constant step sizes.\n\nNumerical experiments include minimizing potential energy in the Bures--Wasserstein space and Rayleigh quotient maximization on the sphere. The results confirm that RGD with silver step sizes achieves acceleration over its constant-step-size counterpart.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n\n- The theoretical results are noteworthy. To the best of my knowledge, achieving acceleration on Riemannian manifolds remains an active area of research--primarily focused on extensions of Nesterov-style acceleration--where progress has been limited due to challenges posed by the metric distortion of curved spaces. In contrast, this work explores a complementary direction: demonstrating that acceleration can be attained solely through the careful design of step sizes, similar to recent developments in the Euclidean setting.\n\n\n- The writing is of high quality. \n\n# Weaknesses:\n\n- The theory only guarantees for iteration counts of the form $n=2^k-1$.\n\n- One of the main motivations of the paper is to relax the assumption on upper bound and lower bound of the space curvature -- where the authors argue that the Wasserstein space is such an example. However, when working with the Wasserstein space, **deterministic** RGD usually cannot apply (except for very special and rare examples like the pontential energy), even for the practically important case of  KL divergence. Therefore, the motivation is somewhat limited. \n\n- The assumption of generalized geodesic convexity -- while being mild and standard in the Wasserstein space -- is strong and less standard in general Riemannian manifolds. Maybe this is the reason why this definition has not been introduced in the context of Riemannian optimization in the literature.\n\n- The numerical experiments, while illustrative, are somewhat limited in scope, focusing only on the Bures--Wasserstein space and the sphere. Moreover, the paper does not include comparisons with Nesterov-style Riemannian Gradient Descent, which would provide a valuable benchmark. Finally, it would be informative to include the theoretical convergence curve $y = 1/x^{1.27}$ in the plots as a reference, to better contextualize the empirical performance.\n\nQUESTIONS:\n# Some questions and suggestions:\n- The Wasserstein space is not a bona fide Riemannian manifold. I wondered if the theorems 4.1 and 4.2 still applied.\n- The first experiment in the Wasserstein space: maybe also consider a more challenging form of $V$ instead of a quadratic function.\n- The second experiment on a sphere: why did not you choose another function that is generalized geodesically convex instead?\n\n# Minor points:\n- The second condition of Prop. 3.8: should add \"it holds\" before $z$, otherwise it looks a bit confusing.\n- Line 85: A word should be added at the beginning of the sentence starting with $\\exp_{x}(v)$.\n- Line 90: a sentence should be added to introduce/characterize parallel transport.\n- Equation (2.1): should say explicitly the square bracket represents concatenation.\n- Line 104: $\\eta_0$ should be $\\eta^{(0)}$. In equations after line 104, all indices are off by one unit.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work adapts the recent \"silver stepsize\" analysis for Euclidean convex optimization to the Riemannian setting. In particular, the authors focus on optimization over geodesically convex subsets of non-negatively curved manifolds, and develop some sufficient machinery to be able to extend the analysis to the setting of curved geometry. The authors then discuss the specific case of optimization over the Bures-Wasserstein space as a an example (since it carries a Riemannian structure, modulo some details). \n\nOne of the claimed key technical contributions relies on a crucial assumption (on $Q_{ij}$) made by the authors.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n1. The topic studied seems relevant\n2. The analysis is largely clean, and the authors point out clearly where and how they deal with the metric distortion.\n3. The application presented is relevant  (though, to be a bit critical, one must admit that at  this point using barycenter problems to highlight the performance of a Riemannian method is like continuing to use the  \"MNIST\" dataset as a benchmark).\n4. Localization of the metric distortion challenge to a condition like (3.1) -- more on this in the review below.\n\n# Weaknesses\n\n1. The adaptation from the Euclidean to the Riemannian case seems to not require any true heavy lifting (but this is not that big a weakness, as this view applies to a large body of Riemannian methods anyways). \n\n2. The functions that may satisfy the \"generalized geodesic convexity/smoothness\" inequality seems small (though appendix D does provide some interesting examples)\n\n3. The analysis is (strangely) limited to the nonnegative curvature case, and it would be valuable to see what happens for negative curvature.\n\n4. The empirical results suggest intense oscillations / numerical instability of the Riemannian silver step method, without further control or discussion or amelioration, this behavior can easily render the current contribution to be primarily of theoretical value.\n\nQUESTIONS:\n* One of the key technical assumptions is 3.1. The need for precisely the same inequality has come up several times over the years, and to my knowledge, previous authors left it as an open problem (I feel if one does not assume it, then by paying an extra factor of \"L\" one should be able to get around this assumption, but then the constants might be wrong; however, this feeling is not a theorem, so please don't view it as such), while the current authors put it in as an assumption. That said, it is useful to have identified sufficient conditions (such as generalized geodesic convexity) for this assumption to hold. The authors should discuss the implications and limitations of this assumption more thoroughly, ultimately, because this seems to be one of the few \"truly Riemannian barriers\" that are in the paper.\n\nIn fact, I think if the authors could prove inequality 3.1 without invoking their sufficient condition for some nontrivial settings, that would be a valuable contribution on its own, and would add to the depth and maturity of the current work.\n\n* The numerical instability of the silver step size suggests that there may be a stronger rate for the \"lim inf\" hiding in there? Moreover, is this a problem that also plagues the Euclidean case? How to ameliorate that? Possibly, there is a Lyapunov function that the authors could exhibit, that is actually monotonically decreasing (for instance this Lyapunov function in the authors case might easily come from combining function values with a stepsize x tangent-space distance (because their generalized geodesic convexity relies on an object that relates to such a distance)\n\n*  The hypersphere example, while not geodesically convex, is known to satisfy a Riemannian PL inequality. From Euclidean results, it is known that PL functions cannot be solved at an accelerated rate, so that applies to the Riemannian case too. However, it is still possible that for the sphere, one can bypass that type of lower bound. Might the authors have some thoughts about that?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}