Online and stochastic optimization beyond Lipschitz continuity: A Riemannian approach

Motivated by applications to machine learning and imaging science , we study a class of online and stochastic optimization problems with loss functions that are not Lipschitz continuous ; in particular , the loss functions encountered by the optimizer could exhibit gradient singularities or be singular themselves . Drawing on tools and techniques from Riemannian geometry , we examine a Riemann–Lipschitz ( RL ) continuity condition which is tailored to the singularity landscape of the problem ’ s loss functions . In this way , we are able to tackle cases beyond the Lipschitz framework provided by a global norm , and we derive optimal regret bounds and last iterate convergence results through the use of regularized learning methods ( such as online mirror descent ) . These results are subsequently validated in a class of stochastic Poisson inverse problems that arise in imaging science . 1 Introduction The surge of recent breakthroughs in machine learning and artificial intelligence has reaffirmed the prominence of first-order methods in solving large-scale optimization problems . One of the main reasons for this is that the computation of higher-order derivatives of functions with thousands – if not millions – of variables quickly becomes prohibitive ; another is that gradient calculations are typically easier to distribute and parallelize , especially in large-scale problems . In view of this , first-order methods have met with prolific success in many diverse fields , from machine learning and signal processing to wireless communications , nuclear medicine , and many others [ 10 , 34 , 37 ] . This success is especially pronounced in the field of online optimization , i.e. , when the optimizer faces a sequence of time-varying loss functions ft , t = 1 , 2 , . . . , one at a time – for instance , when drawing different sample points from a large training set [ 11 , 35 ] . In this general framework , first-order methods have proven extremely flexible and robust , and the attained performance guarantees are well known to be optimal [ 1 , 11 , 35 ] . Specifically , if the optimizer faces a sequence of G-Lipschitz convex losses , the incurred min-max regret after T rounds is Ω ( GT 1/2 ) , and this bound can be achieved by inexpensive first-order methods – such as online mirror descent and its variants [ 11 , 35 , 36 , 41 ] . Nevertheless , in many machine learning problems ( support vector machines , Poisson inverse problems , quantum tomography , etc . ) , the loss landscape is not Lipschitz continuous , so the results mentioned above do not apply . Thus , a natural question that emerges is the following : Is it possible to apply online optimization tools and techniques beyond the standard Lipschitz framework ? And , if so , how ? Our approach and contributions . Our point of departure is the observation that Lipschitz continuity is a property of metric spaces – not normed spaces . Indeed , in convex optimization , Lipschitz continuity is typically stated in terms of a global norm ( e.g. , the Euclidean norm ) , but such a norm is de facto independent of the point in space at which it is calculated . Because of this , the standard Lipschitz framework is oblivious to the finer aspects of the problem ’ s loss landscape – and , in particular , any singularities that may arise at the boundary of the problem ’ s feasible region . On the other hand , in general metric spaces , this is no longer the case : the distance between two points is no longer given by a global norm , so it is much more sensitive to the geometry of the feasible region . For this reason , if the ( Riemannian ) distance dist ( x , x′ ) between two points x and x′ becomes larger and larger as the points approach the boundary of the feasible region , a condition of the form | f ( x ) − f ( x′ ) | = O ( dist ( x′ , x ) ) may still hold even if f becomes singular at the boundary . We leverage this observation by introducing the notion of Riemann–Lipschitz ( RL ) continuity , an extension of “ vanilla ” Lipschitz continuity to general spaces endowed with a Riemannian metric . We show that this metric can be chosen in a principled manner based solely on the singularity landscape of the problem ’ s loss functions – i.e. , their growth rate at infinity and/or the boundary of the feasible region . Subsequently , using a similar mechanism to choose a Riemannian regularizer , we provide an optimal O ( T 1/2 ) regret guarantee through the use of regularized learning methods – namely , “ follow the regularized leader ” ( FTRL ) and online mirror descent ( OMD ) . Our second contribution concerns an extension of this framework to stochastic programming . First , in the context of stochastic convex optimization , we show that an online-to-batch conversion yields an O ( T−1/2 ) value convergence rate . Second , motivated by applications to nonconvex stochastic programming ( where averaging is not a priori beneficial ) , we also establish the convergence of the method ’ s last iterate in a class of nonconvex problems satisfying a weak secant inequality . Finally , we supplement our theoretical analysis with numerical experiments in Poisson inverse problems . Related work . To the best of our knowledge , the first treatment of a similar question was undertaken by Bauschke et al . [ 3 ] who focused on deterministic , offline convex programs ( ft = f for all t ) without a Lipschitz smoothness assumption ( i.e. , Lipschitz continuity of the gradient , as opposed to Lipschitz continuity of the objective ) . To tackle this issue , Bauschke et al . [ 3 ] introduced a second-order “ Lipschitz-like ” condition of the form ∇2 f 4 β∇2h for some suitable Bregman function h , and they showed that Bregman proximal methods achieve an O ( 1/T ) value convergence rate in offline convex problems with perfect gradient feedback . Always in the context of deterministic optimization , Bolte et al . [ 8 ] extended the results of Bauschke et al . [ 3 ] to unconstrained non-convex problems and established trajectory convergence to critical points for functions satisfying the Kurdyka–Łojasiewicz ( KL ) inequality . In a slightly different vein , Lu et al . [ 25 ] considered functions that are also strongly convex relative to the Bregman function defining the Lipschitz-like condition for the gradients , and they showed that mirror descent achieves a geometric convergence rate in this context . Finally , in a very recent preprint , Hanzely et al . [ 17 ] examined the rate of convergence of an accelerated variant of mirror descent under the same Lipschitz-like smoothness assumption . Importantly , all these works concern offline , deterministic optimization problems with perfect gradient feedback and regularity assumptions that can not be exploited in an online optimization setting ( such as the KL inequality ) . Beyond offline , deterministic optimization problems , Lu [ 24 ] established the ergodic convergence of mirror descent in stochastic non-adversarial convex problems under a “ relative continuity ” condition of the form ‖∇ f ( x ) ‖ ≤ G infx′ √ 2D ( x′ , x ) /‖x′ − x‖ ( with D denoting the divergence of an underlying “ reference ” Bregman function h ) . More recently , Hanzely and Richtárik [ 16 ] examined the performance of stochastic mirror descent under a combination of relative strong convexity and relative smoothness / Lipschitz-like conditions , and established a series of convergence rate guarantees that mirror the corresponding rates for ordinary ( Euclidean ) stochastic gradient descent . Except for trivial cases , these conditions are not related to Riemann–Lipschitz continuity , so there is no overlap in our results our methodology . Finally , in a very recent paper , Bécigneul and Ganea [ 5 ] established the convergence of a class of adaptive Riemannian methods in geodesically convex problems ( extending in this way classical results for AdaGrad to a manifold setting ) . Importantly , the Riemannian methodology of [ 5 ] involves the exponential mapping of the underlying metric and focuses on geodesic convexity , so it concerns an orthogonal class of problems . The only overlap would be in the case of flat Riemannian manifolds : however , even though the manifolds we consider here are topologically simple , they are not flat.1 In view of this , there is no overlap with the analysis and results of [ 5 ] . 2 Problem setup We begin by presenting the core online optimization framework that we will consider throughout the rest of our paper . This can be described by the following sequence of events : 1 . At each round t = 1 , 2 , . . . , the optimizer chooses an action Xt from a convex – but not necessarily closed or compact – subset X of an ambient normed space V d. 2 . The optimizer incurs a loss ft ( Xt ) based on some ( unknown ) convex loss function ft : X → . 3 . The optimizer updates their action and the process repeats . Remark 1 . For posterity , we note that if X is not closed , ft ( or its derivatives ) could become singular at a residual point x ∈ bd ( X ) \ X ; in particular , we do not assume here that ft admits a smooth extension to the closure cl ( X ) of X ( or even that it is bounded over bounded subsets of X ) . In this broad framework , the most widely used figure of merit is the minimization of the agent ’ s regret . Formally , the regret of a policy Xt ∈ X , t = 1 , 2 , . . . , is defined as Regx ( T ) = T∑ t=1 [ ft ( Xt ) − ft ( x ) ] , ( 1 ) for all x ∈ X . We then say that the policy Xt leads to no regret if Regx ( T ) = o ( T ) for all x ∈ X . In addition to convexity , the standard assumption in the literature for the problem ’ s loss functions is Lipschitz continuity , i.e. , | ft ( x′ ) − ft ( x ) | ≤ Gt‖x′ − x‖ ( LC ) for some Gt ≥ 0 , t = 1 , 2 , . . . , and for all x , x′ ∈ X . Under ( LC ) , if the agent observes at each stage t an element vt of ∂ ft ( Xt ) , straightforward online policies based on gradient descent enjoy a bound of the form Regx ( T ) = O ( ḠT T 1/2 ) , with Ḡ2T = T−1 ∑T t=1 G 2 t [ 11 , 35 , 41 ] . In particular , if G ≡ lim supT→∞ ḠT < ∞ ( e.g. , if each ft is G-Lipschitz continuous over X ) , we have the bound Regx ( T ) = O ( GT 1/2 ) ( 2 ) which is well known to be min-max optimal in this setting [ 1 ] . A note on notation . Throughout our paper , we make a clear distinction between V and its dual , and we use Dirac ’ s notation 〈v|x〉 for the duality pairing between v ∈ V∗ and x ∈ V ( not to be confused with the notation 〈· , ·〉 for a scalar product on V ) . Also , unless mentioned otherwise , all notions of boundary and interior should be interpreted in the relative ( as opposed to topological ) sense . We also make the blanket assumption that the subdifferential ∂ ft of ft admits a continuous selection ∇ ft ( x ) ∈ ∂ ft ( x ) for all x ∈ dom ∂ ft ≡ { x ∈ X : ∂ ft ( x ) , ∅ } .

Summary: The paper establishes optimal regret bounds of the order O(\sqrt{T}) for Follow The Regularised Leader (FTRL) and Online Mirror Descent (OMD) for convex loss functions and potentials (a.k.a. Riemannian regularizers) that are, respectively, Lipschitz continuous and strongly convex with respect to a given Riemannian metric. These conditions naturally generalize the classical conditions typically considered in the literature, which are defined with respect to a global norm and, as such, are not well-suited to problems where the loss functions and its gradient present singularities at the boundary of the feasibility region. The authors suggest a principled way to choose both the Riemannian metric and the potential function based on the singularity landscape of the gradient of the loss function. Via standard online-to-batch conversion, the authors also address the offline setting and give O(1/\sqrt{T}) error bounds for ergodic averages in convex problems and for last iterates in non-convex problems satisfying a weak secant inequality. The authors include numerical experiments involving a Poisson inverse problem.