This paper establishes global convergence and provides global bounds of the rate of convergence for the Heavy-ball method for convex optimization. When the objective function has Lipschitz-continuous gradient, we show that the Cesáro average of the iterates converges to the optimum at a rate of O(1/k) where k is the number of iterations. When the objective function is also strongly convex, we prove that the Heavy-ball iterates converge linearly to the unique optimum. Numerical examples validate our theoretical findings.
REF analyze momentum and provide global convergence properties for functions with Lipschitz-continuous gradients.
Global convergence of the Heavy-ball method for convex optimization
2015 European Control Conference (ECC)
2015 European Control Conference (ECC)
Mathematics Computer Science