In this manuscript  we study the properties of a family of a second order differential equations with damping, its discretizations and their connections with accelerated optimization algorithms for m-strongly convex and L-smooth functions. In particular, using the Linear Matrix Inequality framework developed in Fazlyab et. al. (2018), we derive analytically a (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods, which allows for a complete characterization of their convergence rate. We then show that in the appropriate limit  this family of methods may be seen as a discretization of a family of second order ordinary differential equations , which properties can be also understood by a (continuous) Lyapunov function, which can also be obtained by studying the limiting behaviour of the discrete Lyapunov function . Finally, we show that the majority of typical discretizations of this ODE , such as the Heavy ball method, do not possess suitable discrete Lyapunov functions , and hence fail to reproduce the desired limiting behaviour of this ODE, which in turn implies that their converge rates when seen as optimization methods cannot behave in an "accerelated" manner .
In this manuscript , we study the properties of a family of second-order differential equations with damping, its discretizations and their connections with accelerated optimization algorithms for m-strongly convex and L-smooth functions. In particular, using the Linear Matrix Inequality LMI framework developed by Fazlyab et. al. (2018), we derive analytically a (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods, which allows for the complete characterization of their convergence rate. In the appropriate limit , this family of methods may be seen as a discretization of a family of second-order ordinary differential equations for which we construct (continuous) Lyapunov functions by means of the LMI framework. The continuous Lyapunov functions may alternatively, be obtained by studying the limiting behaviour of their discrete counterparts . Finally, we show that the majority of typical discretizations of the family of ODEs , such as the Heavy ball method, do not possess Lyapunov functions with properties similar to those of the Lyapunov function constructed here for the Nesterov method .
A
,
R
a second order
second-order
R
framework developed in
LMI framework developed by
R
a
the
R
We then show that in
In
A
,
R
second order
second-order
R
, which properties can be also understood by a
for which we construct
R
function, which can also
functions by means of the LMI framework. The continuous Lyapunov functions may alternatively,
R
the discrete Lyapunov function
their discrete counterparts
R
this ODE
the family of ODEs
R
suitable discrete Lyapunov functions , and hence fail to reproduce the desired limiting behaviour of this ODE, which in turn implies that their converge rates when seen as optimization methods cannot behave in an "accerelated" manner
Lyapunov functions with properties similar to those of the Lyapunov function constructed here for the Nesterov method