Title: HomM: Homogeneous Momentum Optimizer with Finite-Time Convergence
Primary area: optimization
Keywords: optimization, homogeneity, finite-time, scaling

Abstract: We introduce HomM, a homogeneous momentum optimizer derived from a perspective of continuous-time dynamical systems. HomM integrates homogeneity (scaling) and momentum, achieving finite-time convergence to an optimal solution under standard assumptions for both convex and non-convex objectives. To bridge theory and practice, we propose a semi-implicit discretization of the continuous-time HomM. Additionally, we present a unified framework for understanding adaptive optimizers through the lens of homogeneity, highlighting comparisons with HomM. Empirical evaluations on deep learning benchmarks show that HomM outperforms widely used momentum-based baselines, including SGD with momentum and Nesterov acceleration, as well as adaptive methods such as Adam and Lion.

TL;DR: We developed a new optimizer, HomM, that achieves finite-time convergence by  blending momentum and scaling.