O nline and S tochastic O ptimization beyond L ipschitz C ontinuity : A R iemannian A pproach
A bstract
Motivated by applications to machine learning and imaging science, we study a class of online and stochastic optimization problems with loss functions that are not
Lipschitz continuous; in particular, the loss functions encountered by the optimizer could exhibit gradient singularities or be singular themselves.
Drawing on tools and techniques from Riemannian geometry, we examine a Riemann–Lipschitz (RL) continuity condition which is tailored to the singularity landscape of the problem’s loss functions.
In this way, we are able to tackle cases beyond the Lipschitz framework provided by a global norm, and we derive optimal regret bounds and last iterate convergence results through the use of regularized learning methods (such as online mirror descent).
These results are subsequently validated in a class of stochastic Poisson inverse problems that arise in imaging science.
I ntroduction
The surge of recent breakthroughs in machine learning and artiﬁcial intelligence has rea ﬃ rmed the prominence of ﬁrst-order methods in solving large-scale optimization problems.
One of the main reasons for this is that the computation of higher-order derivatives of functions with thousands – if not millions – of variables quickly becomes prohibitive; another is that gradient calculations are typically easier to distribute and parallelize, especially in large-scale problems.
In view of this, ﬁrst-order methods have met with proliﬁc success in many diverse ﬁelds, from machine learning and signal processing to wireless communications, nuclear medicine, and many others [10, 34, 37].
This success is especially pronounced in the ﬁeld of online optimization , i.e., when the optimizer faces a sequence of time-varying loss functions f t , t = 1 , 2 , . .
, one at a time – for instance, when drawing di ﬀ erent sample points from a large training set [11, 35].
In this general framework, ﬁrst-order methods have proven extremely ﬂexible and robust, and the attained performance guarantees are well known to be optimal [1, 11, 35].
Speciﬁcally, if the optimizer faces a sequence of G -Lipschitz convex losses, the incurred min-max regret after T rounds is Ω ( GT 1 / 2 ), and this bound can be achieved by inexpensive ﬁrst-order methods – such as online mirror descent and its variants [11, 35, 36, 41].
Nevertheless, in many machine learning problems (support vector machines, Poisson inverse problems, quantum tomography, etc.), the loss landscape is not Lipschitz continuous, so the results mentioned above do not apply.
Thus, a natural question that emerges is the following: Is it possible to apply online optimization tools and techniques beyond the standard Lipschitz framework?
And, if so, how?
Our approach and contributions.
Our point of departure is the observation that Lipschitz continuity is a property of metric spaces – not normed spaces.
Indeed, in convex optimization, Lipschitz continuity is typically stated in terms of a global norm (e.g., the Euclidean norm), but such a norm is de facto independent of the point in space at which it is calculated.
Because of this, the standard
Lipschitz framework is oblivious to the ﬁner aspects of the problem’s loss landscape – and, in particular, any singularities that may arise at the boundary of the problem’s feasible region.
On the other hand, in general metric spaces, this is no longer the case: the distance between two points is no longer given by a global norm, so it is much more sensitive to the geometry of the feasible region.
For this reason, if the (Riemannian) distance dist( x , x (cid:48) ) between two points x and x (cid:48) becomes larger and larger as the points approach the boundary of the feasible region, a condition of the form | f ( x ) − f
( x (cid:48) )
| =
O (dist( x (cid:48) , x )) may still hold even if f becomes singular at the boundary.
We leverage this observation by introducing the notion of Riemann–Lipschitz (RL) continuity, an extension of “vanilla” Lipschitz continuity to general spaces endowed with a Riemannian metric.
We show that this metric can be chosen in a principled manner based solely on the singularity landscape of the problem’s loss functions – i.e., their growth rate at inﬁnity and / or the boundary of the feasible region.
Subsequently, using a similar mechanism to choose a Riemannian regularizer , we provide an optimal O ( T 1 / 2 ) regret guarantee through the use of regularized learning methods – namely, “follow the regularized leader” (FTRL) and online mirror descent (OMD).
Our second contribution concerns an extension of this framework to stochastic programming.
First, in the context of stochastic convex optimization, we show that an online-to-batch conversion yields an O ( T − 1 / 2 ) value convergence rate.
Second, motivated by applications to nonconvex stochastic programming (where averaging is not a priori beneﬁcial), we also establish the convergence of the method’s last iterate in a class of nonconvex problems satisfying a weak secant inequality.
Finally, we supplement our theoretical analysis with numerical experiments in Poisson inverse problems.
Related work.
To the best of our knowledge, the ﬁrst treatment of a similar question was undertaken by Bauschke et al.
[3] who focused on deterministic, o ﬄ ine convex programs ( f t = f for all t ) without a Lipschitz smoothness assumption (i.e., Lipschitz continuity of the gradient , as opposed to Lipschitz continuity of the objective ).
To tackle this issue, Bauschke et al.
[3] introduced a second-order “Lipschitz-like” condition of the form ∇ 2 f (cid:52) β ∇ 2 h for some suitable Bregman function h , and they showed that Bregman proximal methods achieve an O (1 / T ) value convergence rate in o ﬄ ine convex problems with perfect gradient feedback.
Always in the context of deterministic optimization, Bolte et al.
[8] extended the results of Bauschke et al.
[3] to unconstrained non-convex problems and established trajectory convergence to critical points for functions satisfying the Kurdyka–Łojasiewicz (KL) inequality.
In a slightly di ﬀ erent vein, Lu et al.
[25] considered functions that are also strongly convex relative to the Bregman function deﬁning the Lipschitz-like condition for the gradients, and they showed that mirror descent achieves a geometric convergence rate in this context.
Finally, in a very recent preprint, Hanzely et al.
[17] examined the rate of convergence of an accelerated variant of mirror descent under the same
Lipschitz-like smoothness assumption.
Importantly, all these works concern o ﬄ ine , deterministic optimization problems with perfect gradient feedback and regularity assumptions that cannot be exploited in an online optimization setting (such as the KL inequality).
Beyond o ﬄ ine, deterministic optimization problems, Lu [24] established the ergodic convergence of mirror descent in stochastic non-adversarial convex problems under a “relative continuity” condition of the form (cid:107)∇ f ( x ) (cid:107) ≤ G inf x (cid:48) √ 2 D ( x (cid:48) , x ) / (cid:107) x (cid:48) − x (cid:107) (with D denoting the divergence of an underlying “reference” Bregman function h ).
More recently, Hanzely and
Richtárik [16] examined the performance of stochastic mirror descent under a combination of relative strong convexity and relative smoothness / Lipschitz-like conditions, and established a series of convergence rate guarantees that mirror the corresponding rates for ordinary (Euclidean) stochastic gradient descent.
Except for trivial cases, these conditions are not related to Riemann–Lipschitz continuity, so there is no overlap in our results our methodology.
Finally, in a very recent paper, Bécigneul and Ganea [5] established the convergence of a class of adaptive Riemannian methods in geodesically convex problems (extending in this way classical results for AdaGrad to a manifold setting).
Importantly, the Riemannian methodology of [5] involves the exponential mapping of the underlying metric and focuses on geodesic convexity, so it concerns an orthogonal class of problems.
The only overlap would be in the case of ﬂat Riemannian manifolds: however, even though the manifolds we consider here are topologically simple, they are not ﬂat.
In view of this, there is no overlap with the analysis and results of [5].
P roblem setup
We begin by presenting the core online optimization framework that we will consider throughout the rest of our paper.
This can be described by the following sequence of events:
At each round t = 1 , 2 , . . .
, the optimizer chooses an action X t from a convex – but not necessarily closed or compact – subset X of an ambient normed space V (cid:27) (cid:146) d . 2.
The optimizer incurs a loss f t ( X t ) based on some (unknown) convex loss function f t : X → (cid:146) .
The optimizer updates their action and the process repeats.
Remark 1 .
For posterity, we note that if X is not closed, f t (or its derivatives) could become singular at a residual point x ∈ bd( X ) \ X ; in particular, we do not assume here that f t admits a smooth extension to the closure cl( X ) of X (or even that it is bounded over bounded subsets of X ).
In this broad framework, the most widely used ﬁgure of merit is the minimization of the agent’s regret.
Formally, the regret of a policy X t ∈ X , t = 1 , 2 , . . .
, is deﬁned as
[Equation]
for all x ∈ X .
We then say that the policy X t leads to no regret if Reg x ( T ) =
o ( T ) for all x ∈ X .
In addition to convexity, the standard assumption in the literature for the problem’s loss functions is
Lipschitz continuity , i.e.,
[Equation]
for some G t ≥ 0, t = 1 , 2 , . . .
, and for all x , x (cid:48) ∈ X .
Under (LC), if the agent observes at each stage t an element v t of ∂ f t ( X t ), straightforward online policies based on gradient descent enjoy a bound of the form Reg x ( T ) =
O ( ¯ G T T 1 / 2 ) , with ¯ G 2 T =
T − 1 (cid:80) Tt = 1 G 2 t [11, 35, 41].
In particular, if
G ≡ lim sup T →∞ ¯ G T < ∞ (e.g., if each f t is G -Lipschitz continuous over X ), we have the bound
Reg x ( T ) =
O ( GT 1 / 2 ) (2) which is well known to be min-max optimal in this setting [1].
A note on notation.
Throughout our paper, we make a clear distinction between V and its dual, and we use Dirac’s notation (cid:104) v | x (cid:105) for the duality pairing between v ∈ V ∗ and x ∈ V (not to be confused with the notation (cid:104)· , ·(cid:105) for a scalar product on V ).
Also, unless mentioned otherwise, all notions of boundary and interior should be interpreted in the relative (as opposed to topological) sense.
We also make the blanket assumption that the subdi ﬀ erential ∂ f t of f t admits a continuous selection ∇ f t ( x ) ∈ ∂ f t ( x ) for all x ∈ dom ∂ f t ≡ { x ∈ X : ∂ f t ( x ) (cid:44) ∅ } .
R iemann –L ipschitz continuity
Despite its generality, (LC) may fail to hold in a wide range of problems and applications, ranging from support vector machines to Poisson inverse problems, quantum tomography, etc.
The loss functions of these problems exhibit singularities at the boundary of the feasible region, so the standard regret analysis cited above no longer applies.
Accordingly, our ﬁrst step will be to introduce a family of local norms (cid:107)·(cid:107) x , x ∈ X , such that a variant of (LC) holds even if the derivatives ∂
f /∂ x i of f blow up near the boundary of X .
To achieve this, we will employ the notion of a Riemannian metric .
This is simply a positiondependent scalar product on V , i.e., a continuous assignment of bilinear pairings (cid:104)· , ·(cid:105) x , x ∈ X , satisfying the following conditions for all z , z (cid:48) ∈ V and all x ∈ X :
Symmetry: (cid:104) z , z (cid:48) (cid:105) x
= (cid:104) z (cid:48) , z (cid:105) x . 2.
Positive-deﬁniteness: (cid:104) z , z (cid:105) x ≥ 0 with equality if and only if z = 0.
More concretely, in the standard basis { e i } di = 1 of (cid:146) d , we deﬁne the metric tensor of (cid:104)· , ·(cid:105) x as the matrix g ( x ) ∈ (cid:146)
d × d with components
[Equation]
The norm of z ∈ V at x ∈ X is then deﬁned as
[Equation]
In this way, a Riemannian metric allows us to measure lengths and angles between displacement vectors at each x ∈ X ; for illustration, we provide some notable examples below:
Example 1 (Euclidean geometry) .
The ordinary Euclidean metric on X =
(cid:146) d is g ( x ) =
I .
This yields the standard expressions (cid:107) z (cid:107) 2 x = (cid:80) di
= 1 z
i
and
(cid:104) z , z (cid:48) (cid:105) x
= (cid:80) di
= 1 z i z (cid:48) i , both independent of x .
Example 2 (Hyperbolic geometry) .
The Poincaré metric on the positive orthant X =
(cid:146)
d
++ is
[Equation] [Equation]
leading to the local norm (cid:107) z (cid:107) 2 x = (cid:80) di
=
z
i / x 2 i .
Under (5), (cid:146) d
++ can be seen as a variant of Poincaré’s half-space model for hyperbolic geometry [22]; this will become important later.
Given a Riemannian metric on X , the length of a curve γ : [0 , 1] → X is deﬁned as L g [ γ ] = (cid:82)0 (cid:107) ˙
γ ( s ) (cid:107) γ ( s ) ds , and the Riemannian distance between x 1 , x 2 ∈ X is given by
[Equation]
Under this deﬁnition, it is natural to introduce the following Riemannian variant of (LC):
Deﬁnition 1.
We say that f : X → (cid:146) is Riemann–Lipschitz continuous relative to g if
[Equation]
some G ≥ 0 and all
[Equation]
X
[Equation]
Albeit simple to state, (RLC) may be di ﬃ cult to verify because it requires the computation of the distance function dist g of g – which, in turn, relies on geodesic calculations to identify the shortest possible curve between two points.
Nevertheless, if f is di ﬀ erentiable, Proposition 1 below provides an alternative characterization of Riemann–Lipschitz continuity which is easier to work with:
Proposition 1.
Suppose that f : X → (cid:146) is di ﬀ erentiable.
Then, (RLC) holds if and only if
[Equation]
Remark 2 .
In the above, the Riemannian gradient grad f ( x ) of f at x is deﬁned as follows: First, let
Z = span { x (cid:48) − x : x , x (cid:48)
∈ X } denote the tangent hull of X .
Then, grad f ( x ) ∈
Z is deﬁned by the characteristic property
[Equation]
Existence and uniqueness of grad f ( x ) is due to the fact that g ( x ) is positive-deﬁnite – and, hence, invertible [22].
In particular, if X is full-dimensional (so Z = V ), we have:
[Equation]
The proof of Proposition 1 requires the introduction of further tools from Riemannian geometry; seeing as these notions are not used anywhere else in our paper, we relegate it to the appendix.
Instead, we close this section with a simple example of a singular function which is nonetheless
Riemann–Lipschitz continuous:
Example 3.
Let X =
[0 , 1] d \{ 0 } (so X is convex but neither open nor closed) and let f ( x )
= − log( a (cid:62) x ) for some positive vector
a ∈ (cid:146) d
++ .
If we take g ij ( x ) =
δ ij / ( x 1 + · · · + x d ) 2 , we get (cid:107) grad f ( x ) (cid:107) 2 x
[Equation]
Thus, although f is not Lipschitz continuous in the standard sense, it is Riemann–Lipschitz continuous relative to g ; we will revisit this example in our treatment of Poisson inverse problems in Section 6.
More generally, Example 3 suggests the following rule of thumb: if f exhibits a gradient singularity of the form | ∂ i f ( x ) |
=
O ( φ ( x )) at some residual point x ∈ cl( X ) \ X of X , taking g ij ( x ) =
φ ( x ) 2 δ ij gives (cid:107) grad f ( x ) (cid:107) 2 x = φ ( x ) − 2 (cid:80) di = 1 [ ∂ i f ( x )] 2 =
O (1) .
On that account, f is Riemann–Lipschitz continuous, even though its derivative is singular; we ﬁnd this heuristic particularly appealing because it provides a principled choice of Riemannian metric under which f satisﬁes (RLC).
A lgorithms
In this section, we present the algorithms that we will study in the sequel: “follow the regularized leader” (FTRL) and online mirror descent (OMD).
Both methods have been widely studied in the literature in the context of “vanilla” Lipschitz continuity; however, beyond this basic setting, treating
FTRL / OMD in the Riemannian framework of the previous section is an intricate a ﬀ air that requires several conceptual modiﬁcations.
For this reason, we take an in-depth look into both methods below.
4.1 R egularization
We begin with the idea of regularization through a suitable penalty function.
In our Riemannian setting, we adapt this notion as follows:
Deﬁnition 2.
Let g be a Riemannian metric on X and let h : V → (cid:146) be a proper lower semicontinuous (l.s.c.)
convex function with dom h
= X .
We say that h is a Riemannian regularizer on
X if:
The subdi ﬀ erential of h admits a continuous selection ,
i
e., a continuous function ∇ h such that ∇ h ( x ) ∈
∂ h ( x ) for all x ∈ X ◦ ≡ dom ∂ h . 2.
h is strongly convex relative to g , i.e.,
[Equation]
for some K > 0 and all x ∈ X ◦ , x (cid:48) ∈ X .
The Bregman divergence induced by h is then deﬁned for all p ∈ X , x ∈ X ◦ as
[Equation]
There are two points worth noting in the above deﬁnition.
First, the domain of h is all of X , but this need not be the case for the subdi ﬀ erential ∂ h of h : by convex analysis arguments [33, Chap. 26], we have ri X ⊆
X ◦ ≡ dom ∂ h ⊆ X .
To connect the two, we will say that h is a Riemann–Legendre regularizer when X ◦ =
ri X and D ( p , x n ) → 0 whenever x n → p .
Second, strong convexity in (11) is deﬁned relative to the underlying Riemannian metric.
If the norm in (11) does not depend on x , we recover the standard deﬁnition; however, the dependence of the second-order term in (11) on g can change the landscape signiﬁcantly.
Lemma 1 and Examplebelow provide an illustration of this interplay between g and h :
Lemma 1.
A Riemannian regularizer h is K-strongly convex relative to g if and only if
[Equation]
The proof of Lemma 1 follows from a rearrangement of (11) so we omit it.
Instead, we present below some examples of Riemannian regularizers:
Example 4.
Let X =
[0 , 1] d , and consider the so-called Burg entropy h ( x ) = − (cid:80) di
= 1 log x i .
It is easy to see that h ( x ) is strongly convex relative to the standard Euclidean norm (cid:107)·(cid:107) 2 .
Moreover, we have
[Equation]
and, by Taylor’s theorem with Lagrange remainder, we readily get
[Equation]
where (cid:107) z (cid:107) x
= (cid:80) di
=
z
i / x i denotes the so-called Shahshahani norm on X (i.e., h is also strongly convex relative to (cid:107)·(cid:107) x ).
This regularizer has been used extensively in the setting of Poisson inverse problems and plays a central role in the analysis of Bauschke et al.
[3], Hanzely and Richtárik [16], He et al.
[18], and Lu et al.
For completeness, we revisit it in Section 6.
Example 5.
Let X
and g be as in Example 3, and let h ( x )
= (1 + r 2 ) /
(cid:80) d i = 1 x i with r 2 = (cid:80) d i = 1 x 2 i .
Then, a tedious (but otherwise straightforward) algebraic calculation gives
[Equation]
[Equation]
i.e., h is strongly convex relative to g .
By contrast, due to the singularity of g at 0, it is easy to check that the Euclidean regularizer h ( x ) =
(1 / 2) (cid:80) di
= 1 x 2 i is not strongly convex relative to g .
4.2 A lgorithms and feedback structure
With these preliminaries in hand, we begin with the FTRL algorithm, which we state here as follows:
[Equation]
In the above, γ > 0 is a step-size parameter whose role is discussed below; as for the existence of the arg min, this is justiﬁed by the lower semicontinuity and strong convexity of h together with the fact that dom h
= X
(so the minimum cannot be attained in the residual set cl( X ) \ X of X ).
In terms of feedback, FTRL assumes that the optimizer has access to all the loss functions encountered up to a given round (except, of course, for the current one).
In many cases of practical interest, this assumption is too restrictive and, instead, the optimizer only has access to a ﬁrst-order oracle for each f t .
To model this feedback structure, we assume that once X t has been chosen, the optimizer receives an estimate v t of ∇ f t ( X t ) satisfying the following hypotheses:
a ) Unbiasedness
[Equation]
b ) Finite mean square
[Equation]
In the above, (cid:107)·(cid:107) ∗ denotes the dual norm of (cid:107)·(cid:107) X t , i.e., the Riemannian norm at the point X t where the oracle was called (we suppress here the index X t and write (cid:107)·(cid:107) ∗ instead of (cid:107)·(cid:107)
X t , ∗ to lighten the notation).
In particular, the oracle feedback v t may fail to be bounded in L 2 relative to any global norm on V ∗ ; as such, (17) is considerably weaker than the standard L 2 -boundedness assumption for global norms.
Finally, in terms of measurability, the expectation in (17) is conditioned on the history
F t of X t up to stage t ; since v t is generated randomly from X t , it is not F t -measurable.
To proceed, the main idea of mirror descent is to replace f s ( x ) in (FTRL) with the ﬁrst-order surrogate f s ( x ) ← f s ( X s )
+ (cid:104)∇ f s ( X s )
| x − X s (cid:105) .
In this way, substituting v s for the estimate of ∇ f s ( X s ) received at stage s , we obtain the linearized FTRL scheme
[Equation]
To rewrite this process in recursive form, introduce the auxiliary (dual) variable
[Equation]
so Y t + 1 = − γ
(cid:80) ts = 1 v s , and hence
[Equation]
Therefore, letting
[Equation]
denote the so-called “mirror map” of the method, we obtain the following incarnation of the online mirror descent (OMD) algorithm:
[Equation]
This version of OMD is also known as “ dual averaging ” [26, 29, 30, 38] or “ lazy mirror descent ” [35]; for a “greedy” variant, see [6, 27, 28] and references therein.
A nalysis and results 5.1 R egret analysis
We begin by stating our main results for the regret minimization properties of FTRL and OMD.
Throughout this section, we make the following blanket assumptions:
Both algorithms are initialized at the “prox-center” x c = arg min h of X and are run with (constant) step-size α/ T 1 / 2 for some α > 0 chosen by
the
optimizer.
The t -th stage loss function f t : X → (cid:146) is convex and satisﬁes (RLC) with constant G t .
The optimizer’s aggregate loss (cid:80) Tt = 1 f t attains its minimum value at some x ∗ ∈ X .
The purpose of the last assumption is to avoid cases where the inﬁmum of a loss function is not attained within the problem’s feasible region (such as e − x over (cid:146) + ).
We then have:
Theorem 1.
Let Reg( T ) ≡ Reg x ∗ ( T ) , ¯ G 2 T =
T − 1 (cid:80) Tt = 1 G 2 t , and ¯ M 2 T
= T − 1 (cid:80) Tt = 1 M 2 t .
Then:
a ) The FTRL algorithm enjoys the regret bound
[Equation]
b ) The OMD algorithm with noisy feedback of the form (17) enjoys the mean regret bound
[Equation]
In particular, if sup t G t < ∞ , sup t M t < ∞ , both algorithms guarantee O ( √ T ) regret.
Remark 3 .
We emphasize here that the O ( √ T ) regret bound above is achieved even if X is unbounded or if the “Bregman depth” H ≡ sup x ∈ X D ( x , x c ) =
sup h − inf h of X is inﬁnite.
4 Of course, if H < ∞ and G (or M ) is known to the optimizer, (22) can be optimized further by tuning α .
When these constants are unknown, achieving an optimized constant by means of an adaptive step-size policy is an important question, but one which lies beyond the scope of this paper.
The main idea behind the proof of Theorem 1 is to relate the Riemannian structure of X to the
Bregman regularization framework underlying (FTRL) and (OMD).
A ﬁrst such link is provided by the Bregman divergence (12); however, because of the primal-dual interplay between X t ∈ X and
Y t ∈ V ∗ , the Bregman divergence is not su ﬃ ciently adapted.
To overcome this di ﬃ culty, we employ the Fenchel coupling between a target point p ∈ X and y ∈ V ∗ , deﬁned here as
[Equation]
[Table] [Table]
with h ∗ ( y ) =
max x ∈ X {(cid:104) y
| x (cid:105) − h
( x ) } denoting the convex conjugate of h .
As we show in the appendix, the Fenchel coupling (which is non-negative by virtue of Young’s inequality) enjoys the key property
[Equation]
It is precisely this primal-dual inequality which allows us to go beyond the standard Lipschitz framework: compared to (primal-primal) inequalities of a similar form for global norms [2, 21, 27, 30, 40], the distinguishing feature of (24) is the advent of the Riemannian norm (cid:107) v (cid:107) x , ∗ .
Thanks to the intricate connection between g and h , the second-order term in (24) can be controlled even when the received gradient is unbounded relative to any global norm, i.e., even if the objective is singular.
The main obstacle to achieve this is that the underlying Riemannian metric g , the Fenchel coupling Φ and the Bregman divergence D (all state-dependent notions of distance) need not be compatible with one another.
That this is indeed the case is owed to Lemma 1: tethering the Riemannian norm in (13) to the second argument of the Bregman divergence instead of the ﬁrst (or any other point in-between) plays a crucial role in deriving (24).
Any other relation between g and h along these lines is not amenable to analyzing (FTRL) or (OMD) in this framework.
5.2 A pplications to stochastic optimization
The second part of our analysis concerns stochastic optimization problems of the form minimize
[Equation]
subject to
[Equation]
(Opt
[Equation]
with the expectation taken over some model sample space Ω .
Our ﬁrst result here is as follows:
Theorem 2.
Assume that f is convex and Riemann–Lipschitz continuous in mean square, i.e., sup x (cid:133)
[ (cid:107)∇ F ( x ; ω ) (cid:107) 2 x , ∗ ] ≤ M 2 for some M > 0 .
If (OMD) is run for T iterations with a constant step-size of the form α/ √ T and stochastic gradients v t = ∇ F ( X t ; ω t ) generated by an i.i.d.
sequence ω t ∈ Ω , we have
[Equation]
where ¯ X T = (1 / T ) (cid:80) Tt = 1 X t is the “ergodic average” of X t and D c = inf x ∗ ∈ argmin
f D ( x ∗ , x c ) <
∞ denotes the Bregman distance of the prox-center x c of X to arg min f.
The key novelty in Theorem 2 is that the optimal O ( T − 1 / 2 ) convergence rate of OMD is maintained even if the stochastic gradients of F become singular at residual points x ∈ cl( X ) \ X .
As with the regret guarantee of Theorem 1, this is achieved by the intricate three-way relation between the landscape of f , the underlying Riemannian metric g (which is tailored to the singularity proﬁle of the latter), and the Riemannian regularizer h .
The proof of Theorem 2 likewise relies on an online-to-batch conversion of the regret guarantees of (OMD) for the sequence of stochastic gradients ∇ F ( · ; ω t ) of f ; the details can be found in the appendix.
To go beyond the ergodic guarantees of Theorem 2, we also analyze below the convergence of the “last iterate” of OMD, i.e., the actual sequence of generated points X t .
This is of particular interest for non-convex problems where ergodic convergence results are of limited value (because Jensen’s inequality no longer applies).
To obtain global convergence results in this setting, we focus on a class of functions which satisfy a weak secant inequality of the form
[Equation]
for every closed subset K of X that is separated by neighborhoods from arg min f .
Variants of this condition have been widely studied in the literature and include non-convex functions with complicated ridge structures [9, 13, 19, 20, 23, 31, 39, 40].
In this very general setting, we have:
Theorem 3.
Assume f satisﬁes (SI) and is Riemann–Lipschitz continuous in L 2 .
Suppose further that arg min f is bounded and (OMD) is run with a sequence of stochastic gradients v t = ∇ F ( X t ; ω t ) , a Riemann–Legendre regularizer h, and a variable step-size γ t such that (cid:80) ∞ t = 1 γ t = ∞ , (cid:80) ∞ t = 1 γ 2 t < ∞ .
Then, with probability 1 , X t converges to some ( possibly random ) x ∗ ∈ arg min f.
[Figure]
����� ����� ��
[Figure]
� � � � � �� ������� ���� ������������
The proof for Theorem 3 hinges on combining (quasi-)supermartingale convergence results with the basic inequality (24); we detail the proof in the paper’s appendix.
Seeing as (SI) holds trivially for (pseudo-)convex functions, we only note here that Theorem 3 complements Theorem 2 in an important way: the convergence of X t implies that of ¯ X t , so the convergence of (cid:133)
[ f ( ¯ X t )] to min f follows immediately from Theorem 3; however, the rate of convergence (25) doesn’t.
In practice, the ergodic average converges to interior minimizers faster than the last iterate but lags behind when tracking boundary points and / or in non-convex landscapes; we explore this issue in Section 6 below.
N umerical experiments in P oisson inverse problems
For the purposes of validation, we proceed with an application of our algorithmic results to a broad class of Poisson inverse problems that arise in tomography problems.
Referring the reader to the appendix for the details, the objective of interest here is the Poisson likelihood loss (generalized
Kullback–Leibler divergence):
[Equation]
26) where u ∈ (cid:146) m + is a vector of Poisson data observations (e.g., pixel intensities) and H ∈ (cid:146) m
× d is an ill-conditioned matrix representing the data-gathering protocol.
Since the generalized KL objective of (26) exhibits an O (1 / x ) singularity at the boundary of the orthant, we consider the Poincaré metric g ( x ) =
diag(1 / x 1 , . .
, 1 / x d ) under which the KL divergence is Riemann–Lipschitz continuous (cf.
Example 2).
Going back to Example 3, a suitable Riemannian regularizer for this metric is h ( x ) = (cid:80) mi = 1 1 / x 2 i , which is 1-strongly convex relative to g .
We then run the induced mirror descent algorithm with an online-to-batch conversion mechanism as described in Section 5.2.
For reference purposes, we call the resulting process Riemannian mirror descent (RMD).
Subsequently, we ran RMD on a Poisson denoising problem for a 384 × 384 test image contaminated with Poisson noise (so d ≈ 10 5 in this case).
For benchmarking, we also ran a fast variant of the widely used Lucy–Richardson (LR) algorithm [7], and the recent composite mirror prox (CMP) method of [18]; all methods were run with stochastic gradients and the same minibatch size.
Because of the “dark area” gradient singularities when [ Hx ] j → 0, Euclidean stochastic gradient methods oscillate without converging, so they are not reported.
As we see in Fig.
1, the RMD process provides the sharpest reconstruction of the original.
In particular, after an initial warm-up phase, the last iterate of Riemannian mirror descent consistently outperforms the LR algorithm by 7 orders of magnitude, and CMP by 3.
We also note that the Poisson likelihood loss decreases faster under the last iterate of
RMD relative to the di ﬀ erent algorithmic variants that we tested, exactly because of the hysteresis e ﬀ ect that is inherent to ergodic averaging.
Overall, we note that the introduction of an additional degree of freedom (the choice of Bregman function and that of the local Riemannian norm), makes RMD a particularly ﬂexible and powerful paradigm for loss models with singularities.
We ﬁnd these results particularly encouraging for further investigations on the interplay between Riemannian geometry and Bregman-proximal methods.
C oncluding remarks
Owing to its connections with machine learning (support vector machines, Poisson inverse problems, quantum tomography, etc.)
venturing beyond Lipschitz continuity is a fruitful research direction that has recently generated considerable interest in the literature.
Depending on the type of continuity or smoothness encountered (Lipschitz continuity of the objective or Lipschitz continuity of the objective’s gradients)
the results can be signiﬁcantly di ﬀ erent and it is not a priori clear which surrogate smoothness / continuity condition would be the most appropriate for any given problem.
The present paper provides a complementary, Riemannian-geometric viewpoint which we feel can be fairly promising for the design of e ﬃ cient optimization algorithms in this context.
The precise characterization of the interplay between the di ﬀ erent continuity conditions considered in the literature is an important open issue which we leave for future work.
A cknowledgments
The authors gratefully acknowledge ﬁnancial support from the French National Research Agency (ANR) under grants ORACLESS (ANR–16–CE33–0004–01) and ELIOT (ANR-18-CE40-0030), as well as the FAPESP 2018 / 12579-7 project
B P roperties of mirror mappings and the F enchel coupling
We begin by recalling and clarifying some of the notational conventions used in the paper.
First, let
V (cid:27) (cid:146)
d be a ﬁnite-dimensional real space; then, its dual space will be denoted by Y ≡ V ∗ , and we write (cid:104) y
| x (cid:105) for the duality pairing between y ∈ Y and x ∈ V .
Also, if (cid:107)·(cid:107) is a norm on V , the dual norm on Y is deﬁned as (cid:107) y (cid:107) ∗ ≡ sup {(cid:104) y
| x (cid:105) : (cid:107) x (cid:107) ≤ 1 } .
Given an extended-real-valued convex function f : X → (cid:146) ∪ {∞} , we will write dom f ≡ { x ∈
V : f ( x ) < ∞} for its e ﬀ ective domain .
The subdi ﬀ erential of f at x ∈ dom f is then deﬁned as ∂ f ( x ) ≡ { y ∈ Y :
f ( x (cid:48) ) − f ( x ) + (cid:104) y
| x (cid:48) − x (cid:105) ≥ 0 for all x (cid:48) ∈ V } and the domain of subdi ﬀ erentiability of f is dom ∂ f ≡ { x ∈ dom f : ∂ f (cid:44) ∅ } .
Finally, assuming it exists, the directional derivative of f at x along z ∈ V is deﬁned as f (cid:48)
( x ; z ) ≡ d / dt | t = 0 f ( x + tz ).
We will then say that f is di ﬀ erentiable at x ∈ dom
f if there exists ∇ f ( x ) ∈ Y such that (cid:104)∇ f ( x ) | z (cid:105) =
f (cid:48) ( x ; z ) for all vectors of the form z = x (cid:48) − x , x (cid:48) ∈ dom f .
With these notational conventions at hand, we proceed to prove some auxiliary results and estimates that are used throughout the analysis of Section 5.
To recall the basic setup, we assume throughout what follows that h is a Riemannian regularizer in the sense of Deﬁnition 2.
The convex conjugate h ∗ :
Y → (cid:146) of h is then deﬁned as
[Equation]
Since h is K -strongly convex relative to g , it is also strongly convex relative to the Euclidean norm (recall here that g ( x ) (cid:60) µ I ).
As a result, the supremum in (B.1) is always attained, and h ∗ ( y ) is ﬁnite for all y ∈ Y [4].
Moreover, by standard results in convex analysis [33, Chap. 26], h ∗ is di ﬀ erentiable on Y and its gradient satisﬁes the identity
[Equation]
Thus, recalling the deﬁnition of the mirror map Q : Y → X (cf.. Section 4):
[Equation]
we readily get
[Equation]
Together with the prox-mapping induced by h , all these notions are related as follows:
Lemma B.1.
Let h be a Riemannian regularizer on X .
Then, for all x ∈ dom ∂ h and all y, v ∈ Y , we have:
[Equation]
Finally, if x = Q ( y ) and p ∈ X , we have
[Equation]
Remark.
Note that (B.5b) directly implies that ∂ h ( x + ) (cid:44) ∅ , i.e., x + ∈ dom ∂ h for all v ∈ Y .
An immediate consequence of this is that the update rule x + = Q ( ∇ h
( x ) + v ) is well-posed , i.e., it can be iterated in perpetuity.
Proof of Lemma B.1.
To prove (B.5a), note that x solves (B.2) if and only if y − ∂ h ( x ) (cid:51) 0, i.e., if and only if y ∈
∂ h ( x ).
Eq.
(B.5b) is then obtained in the same manner.
For the inequality (B.6), it su ﬃ ces to show it holds for all p ∈ X ◦ ≡ dom ∂ h (by continuity).
To do so, let
[Equation]
Since h is strongly convex relative to g and y ∈ ∂ h ( x ) by (B.5a), it follows that φ ( t ) ≥ 0 with equality if and only if t = 0.
Moreover, note that ψ ( t ) =
(cid:104)∇ h ( x + t ( p − x ))
− y | p − x (cid:105) is a continuous selection of subgradients of φ .
Given that φ and ψ are both continuous on [0 , 1], it follows that φ is continuously di ﬀ erentiable and φ (cid:48)
=
ψ on [0 , 1].
Thus, with φ convex and φ ( t ) ≥ 0 = φ (0) for all t ∈ [0 , 1], we conclude that φ (cid:48) (0) =
(cid:104)∇ h ( x ) − y | p − x (cid:105) ≥ 0, from which our claim follows.
[Equation]
As we mentioned earlier, much of our analysis revolves around a ”primal-dual” divergence between a target point p ∈ X and a dual vector y ∈ Y , called the Fenchel coupling .
Following [26], this is deﬁned as follows for all p ∈ X , y ∈ Y :
[Equation]
The following lemma illustrates some basic properties of the Fenchel coupling:
Lemma B.2.
Let h be a Riemannian regularizer on X with convexity
modulus K.
Then, for all p ∈ X and all y ∈ Y , we have:
Φ ( p , y ) =
D ( p , Q ( y ))
if Q ( y ) ∈
X ◦ (but not necessarily otherwise).
If x = Q ( y ) , then Φ ( p , y ) ≥ K 2 (cid:107) x − p (cid:107) 2 x
Proof.
For our ﬁrst claim, let x = Q ( y ).
Then, by deﬁnition we have:
[Equation]
Since y ∈ ∂ h ( x ), we have h (cid:48)
( x ; p − x ) = (cid:104) y | p − x (cid:105) whenever x ∈ X ◦ , thus proving our ﬁrst claim.
For our second claim, working in the previous spirit we get that:
[Equation]
Thus, we obtain the result by recalling the strong convexity assumption for h with respect to the
Riemannian norm (cid:107)·(cid:107) x .
[Equation]
We continue with some basic relations connecting the Fenchel coupling relative to a target point before and after a gradient step.
The basic ingredient for this is a primal-dual analogue of the so-called “three-point identity” for Bregman functions [12]:
Lemma B.3.
Let h be a regularizer on X .
Fix some p ∈ X and let y, y + ∈ Y .
Then, letting x = Q ( y ) , we have
[Equation]
Proof.
By deﬁnition
[Equation]
we get:
[Equation]
Then, by subtracting the above we get:
[Equation]
and our proof is complete.
[Equation]
With all this at hand
[Equation]
we have the following key estimate:
Proposition B.1.
Let h be a Riemannian regularizer on X with convexity modulus K, ﬁx some p ∈ X , let x = Q ( y ) for some y ∈ Y .
Then, for all v ∈ Y , we have:
[Equation]
Proof.
By the three-point identity (B.11), we get
[Equation]
and hence, after rearranging:
[Equation]
By Young’s inequality [33], we also have
[Equation]
B
[Equation] [Equation]
Our claim then follows by the fact that Φ ( Q ( y + v ) , y )
≥ K
[Equation]
Q ( y +
[Equation]
) − x (cid:107) 2 x (cf.
Lemmas 1 and B.2).
(cid:4)
C A nalysis of FTRL
Our goal here is to prove the regret bound (22a) of Theorem 1.
The starting point of our analysis is the following basic bound:
Lemma C.1 (35, Lemma 2.3) .
The sequence of actions generated by (FTRL) satisﬁes
[Equation]
Importantly, the above bound does not require any Lipschitz continuity or strong convexity assumptions, so it applies to our setting “as is”.
The importance of Riemann–Lipschitz continuity lies in the following:
Lemma C.2.
If f is convex and Riemann–Lipschitz continuous with constant G, then:
[Equation]
Proof.
By the convexity of f , we have:
[Equation]
where the ﬁrst line follows from the deﬁnition of the Riemannian gradient of f , the second one from the Cauchy-Schwartz inequality, and the last from Proposition 1.
[Equation]
With these preliminary results at hand, we obtain the following basic bound for FTRL:
Proposition C.1.
Suppose that (FTRL) is run against a sequence of loss convex loss functions with assumptions as in Section 5.
Then, for all x ∈ X , we have:
[Equation]
Proof.
Our proof is patterned after Shalev-Shwartz [35], but with an important di ﬀ erence regarding the use of Riemannian norms and Riemann–Lipschitz continuity.
To begin, let
[Equation]
denote the “cumulative” loss faced by the optimizer up to roun t − 1, including the regularization penalty.
By the deﬁnition of the FTRL policy, X t ∈ arg min F t ( x ), so (cid:104)∇ F t ( X t ) | x − X t (cid:105) ≥ 0 and, likewise, (cid:104)∇ F t + 1 ( X t + 1 ) | x − X t + 1 (cid:105) ≥ 0 for all x ∈ X .
Furthermore, since h is K -strongly convex relative to g , F t and F t + 1 will be ( K /γ )
-strongly convex relative to g .
Putting all this together, we obtain:
[Equation]
and hence, after summing the above inequalities:
[Equation]
C.7)
On the other hand, Lemma C.2 gives
[Equation]
so, combining the last two inequalities, we get:
[Equation]
Therefore, plugging this back into (C.8) yields
[Equation]
and our result obtains from Lemma C.1.
[Equation]
The proof of (22a) then follows by applying Proposition C.1 with a step-size of the prescribed form.
D E rgodic analysis of OMD
This appendix is devoted to the proof of our main regret bound for (OMD).
We begin by recalling ther recursive deﬁnition of the (lazy) OMD method:
[Equation]
with Q deﬁned as in Appendix B and oracle feedback subject to the hypotheses (17).
We may then write the oracle feedback received by the optimizer at time t as v t = ∇ f t ( X t )
+ U t + 1 ; hence, by the unbiasedness assumption (17a), it follows that (cid:133)
[ U t + 1 | F t ] = 0, i.e., U t is a martingale di ﬀ erence sequence (MDS) relative to F t .
Now, applying Proposition B.1 to (OMD), we get:
[Equation]
(D.1)
Hence, after rearranging and telescoping, we obtain
[Equation]
where, in the last line, we used the deﬁnition of the Riemannian dual norm (cid:107)·(cid:107) ∗ ≡ (cid:107)·(cid:107) x ∗ , ∗ , and we set ξ t + 1 = (cid:104) U t + 1 | x ∗ − X t (cid:105) .
Our result then follows by taking expectations on both sides.
E L ast - iterate analysis of OMD
In this last section, we will present the convergence analysis for the last iterate of (OMD) to arg min f .
We begin by recalling two important results from probability theory.
The ﬁrst is a version of the law of large numbers for martingale di ﬀ erence sequences that are bounded in L 2 [15]:
Theorem E.1.
Let Y t = (cid:80) ti = 1 ζ
i be a martingale and β t a non-decreasing positive sequence such that lim t →∞ β t = ∞ .
Then,
[Equation]
almost surely (E.1) on the set (cid:80) ∞ t = 1 β − 2 t (cid:133)
[ ζ 2 t | F t − 1 ] < ∞ .
The second is a convergence result for quasi-supermartingales due to Robbins and Sigmund [32]:
Lemma E.1.
Let ( F t ) t ∈ (cid:142) be a non-decreasing sequence of σ − algebras.
Let ( α t ) t ∈ (cid:142) , ( θ t ) t ∈
(cid:142) nonnegative F t − measurable random variables, ( η t ) t ∈ (cid:142) is an F t − measurable non-negative summable random variable and the following inequality holds:
[Equation]
Then, ( α t ) t ∈ (cid:142) converges almost surely towards a [0 , ∞ )
-valued random variable.
An application of this lemma leads us to the following result which is of independent interest:
Proposition E.1.
Let X t be the sequence of iterates generated by (OMD) run with a step-size sequence γ t such that (cid:80) ∞ t = 1 γ 2 t < ∞ and a stochastic oracle as in the statement of Theorems 2 and 3.
Then, for all x ∗ ∈ arg min f, Φ ( x ∗ , Y t ) converges with probability 1 .
Proof.
Let x ∗ ∈ arg min f .
Recalling our main estimation:
[Equation]
and taking conditional expectations on both sides, we get due to F t − measurability arguments:
[Equation]
Since, (2 K ) − 1 (cid:80) ∞ t = 1 γ 2 t (cid:133) [ (cid:107) v t (cid:107) 2 X t , ∗ | F t ] ≤ M (2 K ) − 1 (cid:80) ∞
t = 1 γ 2 t < ∞ .
Thus, by applying the above we get the result.
[Equation]
Having this at hand, we can establish the following proposition:
Proposition E.2.
Let X t be the sequence of iterates generated by (OMD) with assumptions as in
Theorem 3.
Then, for all x ∗ ∈ arg min f, the sequence (cid:107) X t − x ∗ (cid:107) X t is bounded with probability 1 .
Proof.
Recalling our main estimation and taking condition expectations on both sides, we get:
[Equation]
Hence, by the above corollary, we have that the sequence Φ ( x ∗ , Y t ) converges with probability 1 for all x ∗ ∈ arg min f .
Thus, it is also bounded with probability 1 for all x ∗ .
We then get
[Equation]
which concludes our proof.
[Equation]
We continue by showing that X t possesses a subsequence that converges to arg min f :
Proposition E.3.
Let X t be the sequence of iterates generated by (OMD) with assumptions as in
Theorem 3.
Then, with probability 1 , there exists a ( possibly random ) subsequence of X t which converges to arg min f.
Proof.
Assume to the contrary that, with positive probability, the sequence X t generated by (OMD) admits no limit points in arg min f .
Conditioning on this event, there exists a (nonempty) closed set
C ⊂ X which is separated by neighborhoods from arg min f and is such that X t ∈ C for all su ﬃ ently large t .
Then, by relabeling X t if necessary, we can assume without loss of generality that X t ∈ C for all t ∈ (cid:142) .
Thus, by Proposition B.1, we get:
[Equation]
where in the last line we set δ ( C )
= inf {(cid:104)∇ f ( x ) | x − x ∗ (cid:105) : x ∗ ∈ arg min f , x ∈ C } > 0 (by (SI)),
U t + 1 = v t − ∇ f ( X t ), ξ t + 1 = −(cid:104)
U t + 1 | X t − x ∗ (cid:105) and β t = (cid:80) ti =
1 γ i .
Thus, by telescoping and factorizing we get:
[Equation]

[Equation]
By the unbiasedness assumption for U t , we have (cid:133) [ ξ t + 1 | F t ] =
(cid:104) (cid:133)
[ U t + 1 | F t ] | X t − x ∗ (cid:105) = 0.
Moreover, for all x ∗ ∈ arg min f , we have
[Equation]
where the last (strict) inequality is obtained due to the ﬁnite mean square property, the boundness of Φ ( x ∗ , Y t ) and the fact that (cid:80) ∞ t = 1 γ 2 t < ∞ .
Thus, we can apply the law of large numbers for L 2 − martingales stated above and conclude that β − 1 t (cid:80) ts = 1 γ s ξ s + 1 converges to 0 almost surely.
On the other hand, for the term S t + 1 = (cid:80) ts = 1 γ 2 s (cid:107) v s (cid:107) 2 X t , ∗ , since v s + 1 is F s -measurable for all s = 1 , 2 . . .
, t − 1 we have:
[Equation]
so S t is a submartingale with respect to F t .
Furthermore, by the law of total expectation, we also get:
[Equation]
implying that S t is bounded in L 1 .
Thus, due to Doob’s submartingale convergence theorem [15], we coclude that S t converges to some (almost surely ﬁnite) random variable S ∞ so lim t →∞ S t + 1 β t = 0 with probability 1.
Now, by letting t → ∞ in (E.8), we get Φ ( x ∗ , Y t ) →
−∞ , a contradiction.
Going back to our original assumption, this shows that there exists a subsequence of X t which converges to arg min f with probability 1, as claimed.
[Equation]
With all this at hand, we proceed to the proof of our last-iterate convergence result:
Proof of Theorem 3.
By the boundedness (and hence compactness) of arg min f , Proposition E. implies that, with probability 1, there exists some x ∗ ∈ arg min
f such that X t k
→ x ∗ for some (possibly random) subsequence
X t k of X t .
By the Riemann–Legendre property of h , it follows that Φ ( x ∗ , Y t k ) =
D ( x ∗ , X t k ) → 0 as k → ∞ , implying in turn that lim t →∞ D ( x ∗ , X t ) = 0 (by Proposition E.1).
Since
D ( x ∗ , X t ) ≥ K (cid:107)
X t − x ∗ (cid:107) 2 X t ≥ µ (cid:107) X
[Equation]
− x
[Equation]
(cid:107) 2 , we conclude that X t → x ∗ , and our proof is complete.
(cid:4)
F A pplications to P oisson inverse problems and numerical experiments F.1 D etailed
statement of the problem
The class of Poisson inverse problems that we consider stem from linear systems of the form u = Hx + z (F.1) where
• x ∈ (cid:146) d
+ is the object under study (a signal, image, ...).
• u ∈ (cid:146) m
+
+ is the observed data (usually m (cid:28) d ).
• The kernel matrix H ∈
(cid:146) m
× d + is a representation of the data-gathering protocol and is typically highly ill-conditioned (e.g., a Toeplitz matrix in the case of image deconvolution problems).
• z ∈ (cid:146) m is the noise a ﬀ ecting the measurements.
When data points are obtained by means of a counting process, measurements can be modeled as Poisson random variables of the form u j ∼ Pois( Hx ) j .
Then, up to an additive constant, the log-likelihood of x ∈ (cid:146) d given an observation u ∈ (cid:146) m ++ will be
[Equation]
F
[Equation]
Hence, obtaining a maximum likelihood estimate for x leads to the archetypal Poisson inverse problem minimize
[Equation]
subject to
[Equation]
PIP) where D KL ( p , q ) = (cid:80) mj
=
[ p j log( p j / q j ) + q j − p j ] denotes the generalized KL divergence on (cid:146) m
+ .
For an extensive review of Poisson inverse problems, we refer the reader to Bertero et al.
In many cases of practical interest, measurements arrive in distinct batches over time – e.g., as sequential optical sections in microscopy and tomography.
Moreover, due to the large numbers of pixels / voxels involved (a typical range of values for m is between 10 6 and 10 7 ), gradients of f are very costly to compute; as such, optimization methods that rely on accurate gradient data are di ﬃ cult to apply in this setting.
Accordingly, a natural workaround to this obstacle is to exploit the online nature of the measurement process, model (PIP) as an online optimization problem, and then to use an online-to-batch conversion to get a candidate solution [35].
On the downside, this online optimization analysis crucially requires the loss functions faced by the optimizer to be Lipschitz continuous.
However, this assumption does not hold for (PIP): if f j ( x ) = − u j log( u j / ( Hx ) j ) denotes the singular part of the KL divergence for the j -th sample, we readily get
[Equation]
This shows that the gradient of f j exhibits an O (1 / x ) singularity at the boundary of (cid:146) d + , so f cannot be Lipschitz under any global norm on (cid:146) d .
As suggested by Example 3, this singularity can be lifted by considering the local norm
[Equation]
2 i for all v ∈ (cid:146) d . [Equation]
[Equation]
In this case, we have
[Equation]
so (cid:107)∇ f j (cid:107) x is bounded under this modiﬁed norm.
This is the principal motivation for considering the
Riemannian mirror descent method deﬁned with respect to this metric and the regularizer presented in Example 5.
F.2 D etails on the experiments
In the rest of this appendix, we discuss in more detail the algorithms tested in Section 6.
The algorithms we considered are
The accelerated Lucy–Richardson algorithm , as presented in [7] and corresponding to OMD with the entropic regularizer h ( x ) =
(cid:80)
i x i log x i .
The composite mirror prox (CMP) of He et al.
[18], corresponding to OMD with an extra gradient step and the log-barrier (Burg) regularizer h
( x ) = − (cid:80)
i log x i of Example 4. 3.
The Riemannian mirror descent (RMD) algorithm detailed in Section 6, corresponding to the Poincaré-like regularizer of Example 5.
All algorithms were run with stochastic gradients drawn with the same minibatch size ( n = 256) and a step-size of the form γ t ∝ 1 / √ t (corresponding to the stochastic variant of each algorithm).
For comparison purposes, we harvested each algorithm’s last generated sample (“last iterate”) as well as the corresponding ergodic average (deﬁned here as ¯ X T =
(cid:80) Tt = 1 γ t X t (cid:14) (cid:80)
Tt = 1 γ t ).
Overall, the algorithms’ last generated sample provided consistently better results than the ergodic average.
The ground truth and the evolution of the Poisson likelihood loss are all reported in Fig.
Remark.
We should note here that the method of He et al.
[18] can be seen as an “extra-gradient” version of the NoLips algorithm of Bauschke et al.
[3] and the “relative stochastic gradient descent” scheme of Hanzely and Richtárik [16] (the di ﬀ erence between the last two being the step-size policy).
In our experiments, Burg mirror descent with and without an extra-gradient step behaved similarly, with the extra-gradient version (CMP) performing slightly better.
To minimize clutter, and because we are already comparing RMD to CMP above, we do not report this extra set of numerical experiments.