Motivated by recent work of Renegar [21], we present new computational methods and associated computational guarantees for solving convex optimization problems using first-order methods. Our problem of interest is the general convex optimization problem f * = min x∈Q f (x), where we presume knowledge of a strict lower bound f slb < f * . [Indeed, f slb is naturally known when optimizing many loss functions in statistics and machine learning (least-squares, logistic loss, exponential loss, total variation loss, etc.) as well as in Renegar's transformed version of the standard conic optimization problem [21] ; in all these cases one has f slb = 0 < f * .] We introduce a new functional measure called the growth constant G for f (·), that measures how quickly the level sets of f (·) grow relative to the function value, and that plays a fundamental role in the complexity analysis. When f (·) is non-smooth, we present new computational guarantees for the Subgradient Descent Method and for smoothing methods, that can improve existing computational guarantees in several ways, most notably when the initial iterate x 0 is far from the optimal solution set. When f (·) is smooth, we present a scheme for periodically restarting the Accelerated Gradient Method that can also improve existing computational guarantees when x 0 is far from the optimal solution set, and in the presence of added structure we present a scheme using parametrically increased smoothing that further improves the associated computational guarantees. 1 Problem Statement and Overview of Results Motivated by recent work of Renegar [21], we present new computational methods and associated computational guarantees for solving convex optimization problems using first-order methods. Our problem of interest is the following optimization problem:
More recently, REF proposed a new SG method by assuming that a strict lower bound of f * , denoted by f slb , is known and f satisfies a growth condition, w − w * 2 ≤ G · (f (w) − f slb ), where w * is the optimal solution closest to w and G is a growth rate constant depending on f slb .
New Computational Guarantees for Solving Convex Optimization Problems with First Order Methods, via a Function Growth Condition Measure
arXiv: Optimization and Control
Mathematics Computer Science