Title: Riemannian stochastic optimization methods avoid strict saddle points
Venue: NeurIPS 2023
Accepted: yes
Keywords: Riemannian optimization

saddle points

stochastic approximation
Abstract: Many modern machine learning applications - from online principal component analysis to covariance matrix identification and dictionary learning - can be formulated as minimization problems on Riemannian manifolds, typically solved with a Riemannian stochastic gradient method (or some variant thereof). However, in many cases of interest, the resulting minimization problem is _not_ geodesically convex, so the convergence of the chosen solver to a desirable solution - i.e., a local minimizer - is by no means guaranteed. In this paper, we study precisely this question, that is, whether stochastic Riemannian optimization algorithms are guaranteed to avoid saddle points with probability $1$. For generality, we study a family of retraction-based methods which, in addition to having a potentially much lower per-iteration cost relative to Riemannian gradient descent, include other widely used algorithms, such as natural policy gradient methods and mirror descent in ordinary convex spaces. In this general setting, we show that, under mild assumptions for the ambient manifold and the oracle providing gradient information, the policies under study avoid strict saddle points / submanifolds with probability $1$, from any initial condition. This result provides an important sanity check for the use of gradient methods on manifolds as it shows that, almost always, the end state of a stochastic Riemannian algorithm can only be a local minimizer.

Riemannian stochastic optimization methods avoid strict saddle points

Abstract

Many modern machine learning applications - from online principal component analysis to covariance matrix identification and dictionary learning - can be formulated as minimization problems on Riemannian manifolds, typically solved with a Riemannian stochastic gradient method (or some variant thereof). However, in many cases of interest, the resulting minimization problem is not geodesically convex, so the convergence of the chosen solver to a desirable solution - i.e., a local minimizer - is by no means guaranteed. In this paper, we study precisely this question, that is, whether stochastic Riemannian optimization algorithms are guaranteed to avoid saddle points with probability 1 . For generality, we study a family of retraction-based methods which, in addition to having a potentially much lower per-iteration cost relative to Riemannian gradient descent, include other widely used algorithms, such as natural policy gradient methods and mirror descent in ordinary convex spaces. In this general setting, we show that, under mild assumptions for the ambient manifold and the oracle providing gradient information, the policies under study avoid strict saddle points / submanifolds with probability 1 , from any initial condition. This result provides an important sanity check for the use of gradient methods on manifolds as it shows that, almost always, the limit state of a stochastic Riemannian algorithm can only be a local minimizer.

1 Introduction

Modern machine learning systems have achieved remarkable success in the efficient optimization of highly non-convex functions using straightforward Euclidean techniques like stochastic gradient descent. A widely accepted hypothesis to explain this phenomenon is that, when the learning system under study - e.g., a neural network - is sufficiently expressive, local minimizers are essentially as good as global ones [21, 35]; by this token, a training algorithm can attain satisfactory performance by simply evading saddle points of the model's loss surface.

This observation has sparked a far-reaching research thread examining the behavior of various algorithms around saddle points in non-convex functions. Informally, these studies aim to tackle two fundamental questions:

Q 1 : When does a given scheme, like stochastic gradient descent, avoid saddle points?

Q 2 : Can we augment a given scheme so that it efficiently escapes saddle points?

Of the above questions, Q 1 focuses on explaining the empirical success of commonly used schemes, while the resolution of Q 2 usually revolves around proposing new schemes with desirable escape guarantees. These complementary perspectives have been extensively studied over the past decade, 37th Conference on Neural Information Processing Systems (NeurIPS 2023).

leading to a fairly complete understanding of how and when a Euclidean (stochastic) algorithm escapes saddle points, cf. [6, 28, 29, 41, 42, 46, 47, 53, 54] and references therein.

In parallel to the above, the recent surge of interest in Riemannian optimization has prompted a closer examination of Riemannian methods, thereby motivating an extension of Q 1 and Q 2 to a manifold setting - itself due to a wide range of breakthrough applications to machine learning and data science, from natural language processing, and signal processing to dictionary learning and robotics [48, 52, 64, 65]. As a result, there is an increasing demand for a comprehensive exploration of various spaces, such as the 𝑑 -dimensional torus, Grassmannian or Stiefel manifolds, hyperbolic spaces, and many others.

Unfortunately, in a proper Riemannian setting, only Q 2 has received sufficient scrutiny thus far. Recent works by Criscitiello & Boumal [22] and Sun et al. [66] have shown that standard Riemannian deterministic algorithms can be augmented via the injection of an infinitesimal amount of noise (proportional to the method's desired accuracy), to achieve comparable escape guarantees in terms of oracle complexity as the corresponding Euclidean methods [30]. To the best of our knowledge, all existing results for Q 1 concern deterministic methods [28, 41, 42, 53] which are significantly limited in scope in large-scale machine learning applications because of their prohibitively high per-iteration cost.

Our results and techniques. In view of the above, our paper aims to provide a general answer to Q 1 for a broad class of Riemannian stochastic optimization methods - including Riemannian stochastic gradient descent, its retraction-based and/or optimistic variants, etc. Concretely, we focus throughout on a flexible template of Riemannian Robbins-Monro (RRM) schemes [34, 59] which, in addition to the specific algorithms of interest mentioned above, also includes a range of Euclidean methods that can be analyzed efficiently from a Riemannian viewpoint.

Informally, our main result may be stated as follows:

Under any stochastic Riemannian Robbins-Monro method, the probability of converging to a strict saddle point ( or a submanifold thereof ) is zero.

This statement provides firm grounds for accepting the output of a stochastic Riemannian optimization method as valid, as it shows that saddle points are avoided with probability 1 . 1 In the context of stochastic methods, our result builds on a series of foundational results by Pemantle [54] and Brandière & Duflo [20] who focused on hyperbolic traps (isolated saddle points with invertible Hessian). These results were subsequently extended by Benaïm & Hirsch [12] to a more general class of unstable sets , but this analysis remained grounded in a flat, Euclidean setting. The connecting tissue of our analysis with these works is the notion of an asymptotic pseudotrajectory (APT), which allows us to couple the long-run behavior of discrete-time RRM methods to that of an associated Riemannian gradient flow. [This discrete-to-continuous comparison is crucial for our analysis in order to apply center stable manifold techniques [63] to the RRM framework.] However, this comes at a significant cost, as establishing the APT property in a Riemannian setting is quite challenging. To achieve this, we employ a set of techniques recently developed by [33] which allow us to make this comparison precise and establish the desired avoidance result.

2 Background on Riemannian Manifolds

We begin with a brief overview of some basic definitions from Riemmanian geometry and optimization, solely intended to set notation and terminology; our presentation roughly follows the masterful account of Lee [43, 44], to which we refer the reader for a comprehensive introduction to the topic.

Le M be a 𝑑 -dimensional, geodesically complete Riemannian manifold. Throughout the sequel, the tangent space to M at a point 𝑥 ∈ M will be denoted by T 𝑥 M , and we will write / 𝛾 ( 𝑡 ) ∈ T 𝛾 ( 𝑡 ) M for the velocity vector to a smooth curve 𝛾 : ℝ → M at time 𝑡 ∈ ℝ . We will also write ⟨· , ·⟩ 𝑥 for the metric at 𝑥 ∈ M , ∥·∥ 𝑥 for the associated norm, and dist (· , ·) for the induced distance function on M , the latter being defined via the minimization of the length functional L [ 𝛾 ] = ∫ ∥ / 𝛾 ( 𝑡 )∥ 𝛾 ( 𝑡 ) 𝑑𝑡 .

Given a point 𝑥 ∈ M and a tangent vector 𝑧 ∈ T 𝑥 M , the (necessarily unique) geodesic emanating from 𝑥 along 𝑧 will be denoted by 𝛾 𝑧 , and we define the exponential map at 𝑥 as exp 𝑥 ( 𝑧 ) = 𝛾 𝑧 ( 1 ) for all 𝑧 ∈ T 𝑥 M (recall here that M is assumed complete, so this map is well-defined for all 𝑥 ∈ M and all 𝑧 ∈ T 𝑥 M ). Whenever well-defined, the inverse of exp 𝑥 will be written as log 𝑥 : M → T 𝑥 M , with the understanding that the domain of log 𝑥 is actually the largest neighborhood of 𝑥 ∈ M on which the restriction of exp 𝑥 is a (global) diffeomorphism; by definition, we have log 𝑥 ( exp 𝑥 ( 𝑧 )) = 𝑧 for all 𝑧 for which the relevant quantities are well-defined. Finally, given a pair of points 𝑥, 𝑥 ′ ∈ M and a tangent vector 𝑧 ∈ T 𝑥 M , we will write Γ 𝑥 → 𝑥 ′ ( 𝑧 ) for the vector obtained by parallel transporting 𝑧 along any minimizing geodesic connecting 𝑥 and 𝑥 ′ .

In this general context, we will be interested in solving the Riemannian optimization problem

for some smooth objective function 𝑓 : M → ℝ . We will also respectively write

for the negative ( Riemannian ) gradient and the ( Riemannian ) Hessian of 𝑓 at 𝑥 . Finally, in terms of solutions of (Opt), we will focus on the avoidance of strict saddle points of 𝑓 , i.e., points ˆ 𝑥 ∈ M for which

where 𝜆 min denotes the minimum eigenvalue of the tensor in question. We will also say that a smooth compact component (in the sense of manifolds) of critical points of 𝑓 is a strict saddle manifold if there exist constants 𝑐 ± > 0 such that all negative eigenvalues of 𝐻 ( ˆ 𝑥 ) , ˆ 𝑥 ∈ S , are bounded from above by -𝑐 -< 0 , and any positive eigenvalues (if they exist) are bounded from below by 𝑐 + > 0 .

To differentiate the above from the Euclidean setting, when M is a real space equipped with the Euclidean metric, we will instead write ∇ 𝑓 and ∇ 2 𝑓 for the (ordinary) gradient and Hessian matrix of 𝑓 . In this case, as is customary, we will not distinguish between primal and dual vectors.

3 Core algorithmic framework

For generality, our avoidance analysis will be carried out in an abstract stochastic approximation framework which includes several popular Riemannian optimization algorithms - from ordinary Riemannian (stochastic) gradient descent, to its retraction-based variants, optimistic methods, etc. For concreteness, we start with the general template below, and we present a (nonexhaustive!) series of representative examples right after.

3.1. The Riemannian Robbins-Monro template. The Riemannian Robbins-Monro (RRM) framework that we will consider for solving (Opt) is an iterative family of methods which directly extends the seminal stochastic approximation scheme of Robbins & Monro [59] to a manifold setting by replacing vector addition with the Riemannian exponential. Roughly following [33], we will focus on the abstract update rule

where

𝑋 𝑛 ∈ M denotes the state of the algorithm at each iteration 𝑛 = 1 , 2 , . . .

ˆ 𝑣 𝑛 ∈ T 𝑋 𝑛 M is a surrogate for the (negative) gradient 𝑣 ( 𝑋 𝑛 ) of 𝑓 at 𝑋 𝑛 (defined in detail below).

𝛾 𝑛 > 0 is the method's step-size (discussed in detail in Section 4).

In the above, the defining element of (RRM) is the sequence of 'surrogate gradients' ˆ 𝑣 𝑛 , 𝑛 = 1 , 2 , . . . , so this will be our first object of interest. Formally, letting F 𝑛 denote the history of 𝑋 𝑛 up to stage 𝑛 (inclusive), we will write

where we have defined

as the random error and the offset of ˆ 𝑣 𝑛 relative to 𝑣 ( 𝑋 𝑛 ) respectively. It will also be convenient to introduce the total error 𝑊 𝑛 = ˆ 𝑣 𝑛 -𝑣 ( 𝑋 𝑛 ) = 𝑈 𝑛 + 𝑏 𝑛 , which captures both random and systematic fluctuations in ˆ 𝑣 𝑛 , and which measures the total deviation of ˆ 𝑣 𝑛 from 𝑣 ( 𝑋 𝑛 ) .

Two points are worth noting here: First, ˆ 𝑣 𝑛 is not adapted to F 𝑛 , so 𝑈 𝑛 is random relative to F 𝑛 ; on the other hand, 𝑏 𝑛 is F 𝑛 -measurable, so it is deterministic relative to F 𝑛 . This brings us to the second important point regarding ˆ 𝑣 𝑛 : given the systematic offset term 𝑏 𝑛 in ˆ 𝑣 𝑛 , the latter should not be seen as the output of a gradient oracle for 𝑣 ( 𝑋 𝑛 ) . In particular, 𝑏 𝑛 is intended to capture possible corrective terms, deviations from the exponential mapping, different algorithmic update structures (such as optimism), etc. We make this distinction precise below.

3.2. Specific algorithms and examples. In the series of examples that follow, we will assume that the optimizer can access 𝑓 via a stochastic first-order oracle (SFO) returning noisy gradients of 𝑓 at the evaluation point. Formally, following Nesterov [50], an SFO is a black-box mechanism which, when queried at 𝑥 ∈ M , returns a (negative) stochastic gradient of the form

where the seed 𝜃 ∈ Θ is a random variable taking values in some complete probability space Θ , and err ( 𝑥 ; 𝜃 ) is an umbrella error term capturing all sources of uncertainty in the model.

The archetypal example of an SFO occurs when 𝑓 is itself a stochastic expectation of the form 𝑓 ( 𝑥 ) = 𝔼 [ 𝐹 ( 𝑥 ; 𝜃 )] for some random function 𝐹 : M × Θ → ℝ - the so-called stochastic optimization framework. In this case, 𝑉 is typically given by 𝑉 ( 𝑥 ; 𝜃 ) = -grad 𝑥 𝐹 ( 𝑥 ; 𝜃 ) , so, under standard assumptions for exchanging differentiation and expectation, we have 𝔼 [ 𝑉 ( 𝑥 ; 𝜃 )] = 𝑣 ( 𝑥 ) . Extrapolating from this basic framework, our only assumption for the moment will be that 𝔼 [ err ( 𝑥 ; 𝜃 )] = 0 ; for a detailed discussion of the required assumptions for (SFO), see Section 4.

In practice, (SFO) will be queried repeatedly at a sequence of states 𝑋 𝑛 , 𝑛 = 1 , 2 , . . . , with a different random seed 𝜃 𝑛 drawn i.i.d. from Θ . In this manner, we obtain the following specific algorithms as special cases of (RRM):

Algorithm 1 (Riemannian stochastic gradient descent) . Following Bonnabel [17], the Riemannian stochastic gradient descent (RSGD) algorithm queries (SFO) at 𝑋 𝑛 and proceeds as

As such, (RSGD) can be seen as an RRM scheme with ˆ 𝑣 𝑛 = 𝑉 ( 𝑋 𝑛 ; 𝜃 𝑛 ) or, equivalently, 𝑈 𝑛 = err ( 𝑋 𝑛 ; 𝜃 𝑛 ) and 𝑏 𝑛 = 0 . ♦

A key factor limiting the applicability of (RSGD) is that the exponential map exp 𝑋 𝑛 (·) could be prohibitively expensive to compute in practice, even for relatively low-dimensional manifolds. On that account, a popular alternative to (RSGD) is to employ a retraction map [2, 19], that is, a smooth mapping R : T M → M that agrees with the exponential map up to first order, namely

With this machinery in hand, we obtain the following retraction-based variant of (RSGD):

Algorithm 2 (Retraction-based stochastic gradient descent) . By replacing the exponential map in (RSGD) with a retraction, we obtain the retraction-based stochastic gradient descent scheme

This algorithm does not seem immediately related to the RRM template - and, indeed, the whole point of introducing a retraction was to get rid of the exponential map in (RRM). The expressive power of (RRM) can be seen in the fact that, despite this apparent disconnect, (Rtr-SGD) can still be expressed as a special case of (RRM) in a fairly straightforward fashion.

To do so, define the 'forward-backward' gradient mapping

with the proviso that the Riemannian logarithm in (5) is well-defined (we discuss the conditions under which this holds later in the paper). Under this definition, (Rtr-SGD) can be recast as a special case of (RRM) by running the latter with the surrogate gradient sequence ˆ 𝑣 𝑛 of Eq. (5). To streamline our presentation, we defer the discussion about the inherent error 𝑊 𝑛 = ˆ 𝑣 𝑛 -𝑣 ( 𝑋 𝑛 ) to Appendix A. ♦

As we mentioned before, retraction-based algorithms typically exhibit significantly lower per-iteration complexity compared to geodesic methods, resulting in their remarkable success in practical machine learning applications [2, 19]. In addition, as we show below, the use of a retraction mapping allows us to provide a unified perspective for several classical algorithms which, at first sight, might seem completely unrelated. An important example is provided by the (stochastic) mirror descent (MD) family of algorithms [49]:

Algorithm 3 (Stochastic mirror descent) . Let M be an open convex subset of ℝ 𝑀 and let ℎ : M → ℝ be a 𝐶 2 -smooth, strongly convex Legendre function on M , that is, ∥∇ ℎ ( 𝑥 )∥ → ∞ whenever 𝑥 → bd ( M ) [cf. 60, Chap. 26]. Then, the stochastic mirror descent (SMD) algorithm unfolds as

where 𝑉 ( 𝑋 𝑛 ; 𝜃 𝑛 ) is the output of an SFO query for ∇ 𝑓 ( 𝑋 𝑛 ) at 𝑋 𝑛 , and P : M × ℝ 𝑀 → M is the so-called prox-mapping associated to ℎ [7-9, 31], viz.

where ⟨· , ·⟩ stands for the ordinary Euclidean inner product in ℝ 𝑀 .

Now, even though the notation in (SMD) is reminiscent of (Rtr-SGD), the definition (6) of P does not bear any resemblance to a geodesic exponential or a retraction - and, indeed, its origins are starkly different. However, as we show below, P can indeed be seen as a retraction relative to a specific Riemannian structure on M , the Hessian Riemannian (HR) metric associated to ℎ [3, 16, 24, 45].

To make this precise, the first step is to note that the basic recursive structure 𝑥 + = P 𝑥 ( 𝑦 ) of (SMD) can be rewritten as

where ℎ ∗ ( 𝑦 ) = max 𝑥 ∈ M {⟨ 𝑦, 𝑥 ⟩ -ℎ ( 𝑥 )} denotes the convex conjugate of ℎ , and we have used Danskin's theorem [62] to write arg max 𝑥 ∈ M {⟨ 𝑦, 𝑥 ⟩ -ℎ ( 𝑥 )} = ∇ ℎ ∗ ( 𝑦 ) . Then, if we endow M with the Hessian Riemannian metric 𝑔 ( 𝑥 ) = ∇ 2 ℎ ( 𝑥 ) , the Riemannian gradient of 𝑓 relative to 𝑔 becomes grad 𝑓 ( 𝑥 ) = [∇ 2 ℎ ( 𝑥 )] -1 ∇ 𝑓 ( 𝑥 ) ; more generally, given a cotangent (dual) vector 𝑦 to M at 𝑥 , the corresponding tangent (primal) vector will be 𝑧 = 𝑔 ( 𝑥 ) -1 𝑦 = [∇ 2 ℎ ( 𝑥 )] -1 𝑦 . In view of this, by inverting the relation 𝑧 = 𝑔 ( 𝑥 ) -1 𝑦 , the abstract mirror descent recursion (7) can be rewritten as

Now, to proceed, consider the curve

so, by definition, 𝛾 ( 0 ) = 𝑥 . In addition, by a direct differentiation, we readily obtain

where we used the standard identity ∇ 2 ℎ ∗ (∇ ℎ ( 𝑥 )) = [∇ 2 ℎ ( 𝑥 )] -1 [9, 61]. This shows that the map R 𝑥 ( 𝑧 ) = P 𝑥 ( 𝑔 ( 𝑥 ) 𝑧 ) is, in fact, a retraction , so (SMD) is a special case of (Rtr-SGD) - and hence, of the general stochastic approximation template (RRM). ♦

Remark. Even though elements of the above ideas are implicit in previous works on mirror descent and Hessian Riemannian metrics [3, 5, 16, 58, 68], to the best of our knowledge, this is the first time that (SMD) is formalized as a retraction-based (Hessian) Riemannian scheme. ♦

Algorithm 4 (Riemannian optimistic gradient) . Moving forward, an important algorithm for solving online optimization problems and games is the so-called optimistic gradient method - originally pioneered by Popov [56] and subsequently popularized by Rakhlin & Sridharan [57]. In the Euclidean case, this method introduces an interim, 'optimistic' correction to gradient dynamics and updates as

where, as usual, 𝑉 is an SFO for the (negative) gradient ∇ 𝑓 of 𝑓 . This idea can then be directly transported to a manifold setting [33], leading to the Riemannian optimistic gradient method

Importantly, the recursion (ROG) may be seen as a special case of (RRM) by setting ˆ 𝑣 𝑛 = ( 1 / 𝛾 𝑛 ) · Γ 𝑋 + 𝑛 → 𝑋 𝑛 ( 𝛾 𝑛 𝑉 ( 𝑋 + 𝑛 ; 𝜃 𝑛 )) or, equivalently 𝑈 𝑛 = Γ 𝑋 + 𝑛 → 𝑋 𝑛 ( err ( 𝑋 + 𝑛 ; 𝜃 𝑛 )) and 𝑏 𝑛 = Γ 𝑋 + 𝑛 → 𝑋 𝑛 ( 𝑣 ( 𝑋 + 𝑛 )) -𝑣 ( 𝑋 𝑛 ) . We defer the details of this calculation to the Appendix A. ♦

Algorithm 5 (Natural gradient descent) . Our last example concerns the influential natural gradient descent (NGD) method of Amari [4], a stochastic optimization scheme for Euclidean spaces, but adapted to the local geometry defined by a strictly convex function ℎ . Specifically, NGD queries an SFO and proceeds as

where grad 𝑓 ( 𝑥 ) : = [∇ 2 ℎ ( 𝑥 )] -1 ∇ 𝑓 ( 𝑥 ) denotes the Riemannian gradient of 𝑓 relative to Hessian Riemannian metric 𝑔 ( 𝑥 ) = ∇ ℎ 2 ( 𝑥 ) on ℝ 𝑀 . It is well known that (NGD) can be seen as a retractionbased Riemannian scheme [17], and may thus be integrated directly within the framework of (RRM); we defer the details to Appendix A. Importantly, (NGD) also includes the celebrated natural policy gradient [32] which plays an important role in reinforcement learning. ♦

The above examples have been chosen to illustrate a range of different update mechanisms that can be integrated within the general algorithmic template provided by (RRM). Of course, it is not possible to be exhaustive but, for illustration purposes, we provide some more examples in Appendix A.

4 Analysis and results

We are now in a position to state and discuss our main result concerning the avoidance of saddle points under (RRM). For concreteness, we begin by discussing the technical assumptions that we will need in Section 4.1; subsequently, we proceed with the formal statement of our result and some direct applications thereof in Section 4.2.

4.1. Technical assumptions. Our technical assumptions concern the four main ingredients of (RRM), namely ( i ) the regularity of 𝑓 ; ( ii ) the method's step-size sequence 𝛾 𝑛 ; ( iii ) the statistics of the surrogate gradients ˆ 𝑣 𝑛 entering (RRM); and ( iv ) the ambient manifold M . Specifically, we will require the following:

Assumption 1 (Regularity of 𝑓 ) . The function 𝑓 is 𝐶 2 and 𝑣 = -grad 𝑓 is (geodesically) 𝐿 -Lipschitz , i.e., for all 𝑥, 𝑥 ′ ∈ M , we have

Assumption 2 (Step-size schedule) . The step-size sequence 𝛾 𝑛 of (RRM) satisfies

Assumption 3 (Surrogate gradients) . The offset and random error components of ˆ 𝑣 𝑛 satisfy

for suitable constants 𝐶, 𝜎, 𝜁 > 0 and for all 𝑧 ∈ T 𝑋 𝑛 M , ∥ 𝑧 ∥ 𝑋 𝑛 = 1 (in the above, all conditions are to be interpreted in the almost sure sense and [ 𝑡 ]+ = max { 0 , 𝑧 } denotes the positive part of 𝑡 ).

Assumption 4 (Injectivity radius) . The injectivity radius of M is bounded from below by 𝜚 > 0 .

Before proceeding, we discuss the implications and range of validity of each of the above assumptions. Since Assumption 1 is standard, we focus on the remaining three below:

On Assumption 2. The step-size conditions typically encountered in the analysis of Robbins-Monro schemes is the 𝐿 2 -𝐿 1 (' square-summabe-but-not-summable ') condition P 𝑛 𝛾 𝑛 = ∞ , P 𝑛 𝛾 2 𝑛 < ∞ , cf. [10, 13, 14, 17, 39, 59] and references therein. This puts a hard threshold on the range of allowed step-size schedules at Ω ( 1 / 𝑛 1 / 2 ) : any step-size that decays at least as slow as 1 / 𝑛 1 / 2 cannot be used under the 𝐿 2 -𝐿 1 assumption. By contrast, the step-size condition (12) is considerably more lax and can tolerate near-constant step-sizes of the form 𝛾 𝑛 ∝ 1 /( log 𝑛 ) 1 + 𝜀 for some 𝜀 > 0 . This is enough to cover all dereasing step-size policies used in practice. [We also recall here that, in stochastic non-convex settings, trajectory convergence cannot be guaranteed in general without a vanishing step-size, cf. [10, 18, 40] and references therein.]

On Assumption 3. Three remarks are in order for the noise and offset requirements (13). First, we should note that the condition 𝑏 𝑛 = O ( 𝛾 𝑛 ) is, a priori, implicit , because it depends on the statistics of the feedback sequence ˆ 𝑣 𝑛 , and these may be difficult to estimate in general. However, in most practical applications, this quantity is under the explicit control of the optimizer: in particular, as we show later in this section, this requirement is satisfied by all the specific algorithms of Section 3.2.

Likewise, the bounded noise requirement is satisfied in many practical cases of interest. For example, when the problem's objective function admits a finite-sum decomposition of the form 𝑓 ( 𝑥 ) = P 𝑁 𝑖 = 1 𝑓 𝑖 ( 𝑥 ) for an ensemble of empirical instances 𝑓 𝑖 , 𝑖 = 1 , . . . , 𝑁 (the standard framework for applications to data science and machine learning), 𝑈 𝑛 is typically generated by sampling a minibatch of 𝑓 , which in turn results in an error term of the form 𝑈 𝑛 = 𝑞 ( 𝑋 𝑛 ) where 𝑞 ( 𝑥 ) : M → T 𝑥 M is bounded on all compact subsets of M . Therefore, ∥ 𝑈 𝑛 ∥ 𝑋𝑛 ≤ ∥ 𝑞 ( 𝑋 𝑛 )∥ 𝑋𝑛 < 𝜎 for some constant 𝜎 for any convergent algorithm { 𝑋 𝑛 } 𝑛 .

Finally, the 'uniform excitability' condition 𝔼 [[⟨ 𝑈 𝑛 , 𝑧 ⟩ 𝑋𝑛 ]+ | F 𝑛 ] ≥ 𝜁 is also standard in the avoidance literature [10, 54], and it is substantially weaker than the isotropic condition, which, roughly speaking, requires the noise to have the same 𝐿 2 magnitude along all directions in space [26, 30, 54]. Instead, (13) only posits that the noise 𝑈 𝑛 has a non-zero component along each direction, and imposes no other restrictions on the statistical profile of the noise.

On Assumption 4. For our last assumption, recall first that the injectivity radius of M at a point 𝑥 ∈ M is the largest radius for which exp 𝑥 is a diffeomorphism onto its image; the injectivity radius of M is then taken to be the infimum over all such radii [43]. In this regard, Assumption 4 simply serves to ensure that the exponential map is invertible at consecutive iterates of (RRM) so no local topological complications can arise. This assumption is automatically satisfied in closed manifolds (independent of curvature), as well as in non-positively curved manifolds - such as Cartan-Hadamard spaces and the like [43, 44]. This assumption (and its variants) is also standard in the literature, cf. [17, 34, 66] and references therein.

4.2. Avoidance of saddle points. We are now in a position to state our main avoidance result:

Theorem 1. Let 𝑋 𝑛 , 𝑛 = 1 , 2 , . . . , be the sequence of states generated by (RRM) , and let S be a strict saddle manifold of 𝑓 . Then, under Assumptions 1-4, we have

where dist ( S , 𝑋 𝑛 ) = inf 𝑥 ∈ S dist ( 𝑥, 𝑋 𝑛 ) denotes the ( Riemannian ) distance of 𝑋 𝑛 from S .

Before discussing the proof of Theorem 1, it is worthwhile to compare our work with its closest antecedents. First, in regard to the general avoidance theory in Euclidean spaces [10, 12], the statement is similar in scope (avoidance of unstable manifolds with probability 1 ), but the techniques and challenges involved are very different. The reason for this is simple: the additive, vector space structure of ℝ 𝑚 is ingrained at every step of the way in the Euclidean analysis of [10], and adapting the various constructions to a manifold setting can be a complicated affair. For an illustration of the technical difficulties involved, see the recent stochastic approximation analysis of [33].

By contrast, the recent results of [22, 66] paint a complementary picture: they concern Riemannian problems but, at their core, they are deterministic results. More precisely, the noise in [22, 66] is actually injected in an otherwise deterministic gradient scheme to facilitate the escape from flat regions in the vicinity of a saddle point; other than that, the magnitude of the noise must be proportional to the solver's desired accuracy, and hence is typically extremely small. As a result, the analysis of [22, 66] cannot be extended to bona fide stochastic schemes - like (RSGD) - which also explains why these results involve a constant step-size (as opposed to a decreasing step-size schedule, which is required to guarantee trajectory convergence in settings with persistent noise). In this regard, Theorem 1 simultaneously complements the stochastic analysis of [10, 12] to Riemannian problems, and the Riemannian analysis of [22, 66] to a stochastic setting.

Proof outline. To facilitate the reading of our proof, we provide below a detailed outline of the main steps and techniques involved therein, deferring the full proof to Appendix B. We begin with a high-level description of our proof strategy and then encode the main arguments in a series of steps right after.

For the purposes of illustration, suppose that M is a subset of ℝ 𝑚 . Then, given a tangent vector 𝑧 ∈ T 𝑥 M , we define the geodesic offset (see Fig. 1) from 𝑥 along 𝑧 as

i.e., as the difference between the geodesic emanating from 𝑥 along 𝑧 and its first-order approximation relative to 𝑥 in the ambient space ℝ 𝑚 (with all differences expressed in the ordinary vector space structure of ℝ 𝑚 ). The offset Δ ( 𝑥 ; 𝑧 ) is readily checked to be second-order in 𝑧 so, while the curve 𝑥 + 𝑡𝑧 does not in general induce a retraction on the target manifold M (in particular, the point 𝑥 + 𝑡𝑧 may not even belong to M ), the converse is true: the map exp 𝑥 ( 𝑧 ) is always a retraction on the ambient, Euclidean space ℝ 𝑚 . In this way, the basic iteration (RRM) can be expressed as

leading to the fundamental question below:

What is the maximum offset 𝜖 𝑛 : = Δ ( 𝑋 𝑛 ; 𝛾 𝑛 ˆ 𝑣 𝑛 ) that can be tolerated by a Euclidean stochastic approximation algorithm to avoid saddle points?

A key technical step in our work is to develop the means to control the offset term 𝜖 𝑛 under (RRM) under a sufficiently broad class of assumptions that includes Algorithms 1-5. Crucially, this step is made possible thanks to the very recent - and technical - stochastic approximation work of [33]. To help the reader navigate our proof strategy, we outline the main steps below, focusing for simplicity on the case of a single saddle point.

Step 1: From discrete to continuous time (and back). Let ˆ 𝑥 be a strict saddle point of 𝑓 . By the stable manifold theorem [63], the set of all initializations such that the Riemannian gradient flow

converges to ˆ 𝑥 is of measure 0. Then, assuming for the moment that the geodesic offset error 𝜖 𝑛 = Δ ( 𝑋 𝑛 ; 𝛾 𝑛 ˆ 𝑣 𝑛 ) in (16) is sufficiently small, the iterates of (RRM) can be seen as a noisy, approximate Euler discretization of (RGF); as such, it is reasonable to expect that the induced trajectories of (RRM) will never converge to ˆ 𝑥 .

To make this intuition precise, our first step will be to show that the iterates of (RRM) comprise an asymptotic pseudotrajectory of (RGF) in the sense of Benaïm [10], i.e., they asymptotically track the orbits of (RGF) with arbitrary precision over windows of arbitrary length. To formalize this, define the 'effective time' variable 𝜏 𝑛 = P 𝑛 -1 𝑘 = 1 𝛾 𝑘 and the associated geodesic interpolation 𝑋 ( 𝑡 ) of 𝑋 𝑛 as

so, by construction, ( a ) 𝑋 ( 𝜏 𝑛 ) = 𝑋 𝑛 for all 𝑛 ; and ( b ) each segment of 𝑋 ( 𝑡 ) is a geodesic. Then, letting Φ : ℝ + × M → M denote the flow of (RGF) - i.e., Φ ℎ ( 𝑥 ) is simply the position at time ℎ ≥ 0 of the solution orbit of (RGF) that starts at 𝑥 ∈ M - we will say that 𝑋 ( 𝑡 ) is an APT of (RGF) if, for all 𝑇 > 0 , we have

This requirement is non-trivial, and our first technical result is to guarantee precisely this:

Theorem 2. Suppose that Assumptions 2-4 hold. Then, with probability 1 , the geodesic interpolation 𝑋 ( 𝑡 ) of the sequence of iterates 𝑋 𝑛 , 𝑛 = 1 , 2 , . . . , generated by (RRM) is an APT of (RGF) .

A version of Theorem 2 was very recently derived by [33] under a different set of assumptions: On the one hand, [33] imposes a much more restrictive step-size schedule for 𝛾 𝑛 (square summability) but, on the other hand, it only posits that the noise increments 𝑈 𝑛 are bounded in 𝐿 2 (as opposed to 𝐿 ∞ in our case). Our proof relies on the same construction of the Picard iteration map as [33], but otherwise diverges significantly in the probabilistic analysis required to establish (APT).

Step 2: From Riemannian to Euclidean schemes (and back). Albeit crucial, the APT property is decidedly not enough to guarantee avoidance: after all, the constant orbit 𝑋 ( 𝑡 ) = ˆ 𝑥 for all 𝑡 ≥ 0 is trivially an APT of (RGF) but, of course, it does not avoid ˆ 𝑥 . To proceed, we will need to exploit the precise update structure of (RRM) in conjunction with the stable manifold theorem applied to (RGF).

In the Euclidean case, this is achieved by means of an intricate Lyapunov function argument, originally due to [11]. Our second step is to devise a new geometric argument to reduce the analysis from an arbitrary intrinsic manifold to an isometrically embedded submanifold of ℝ 𝑚 . This step is carried out by a combination of the celebrated Nash embedding theorem and a (smooth) Tietze extension argument to rewrite (RRM) as a 'corrected' Robbins-Monro scheme on ℝ 𝑚 that actually evolves on M . This construction also requires a 'perturbation analysis' to ensure that certain subtle topological issues do not arise when we invoke the stable manifold theorem; we present the details in Appendix B.

Step 3: Controlling the geodesic offset. As we briefly described in the beginning of the proof overview, this Euclidean reframing of (RRM) introduces an intrinsic offset error 𝜖 𝑛 = Δ ( 𝑋 𝑛 ; 𝛾 𝑛 ˆ 𝑣 𝑛 ) , which is difficult to analyze in detail (the offset incurred by a retraction on M is of similar order, so the exponential-retraction distinction is not important at this stage). Our crucial observation here is that, under our blanket assumptions, 𝜖 𝑛 is small relative to 𝛾 𝑛 and, in particular, 𝜖 𝑛 = O ( 𝛾 2 𝑛 ) . Thanks to this bound, we are able to leverage a series of stochastic bounds originally developed by Pemantle [54] - to show that the probability that these terms will have an adverse effect on exiting the center manifold of ˆ 𝑥 is zero (this is also where Assumption 3 comes in). We formalize this in Appendix B; Theorem 1 then follows by putting everything together.

Δ

(

𝑥

;

𝑧

)

𝑥

𝑥

+

𝑧

𝑧

𝑥

+

=

exp

𝑥

(

𝑧

)

ˆ

𝑥

M

T

𝑥

M

Remark. A concept similar to our geodesic offset Δ ( 𝑥 ; 𝑧 ) has been explored in the reverse direction by the very recent works [15, 23] whose goal was to study of avoidance of Euclidean subgradient methods as an inexact Riemannian gradient scheme. They further show that this inexact Riemannian gradient descent can avoid saddle points if uniform noise is injected. While their core idea bears some similarity to ours, it remains unclear how to apply the analysis in [15, 23] to handle general RRM schemes, such as retraction-based and natural policy gradient methods.

4.3. Applications. As an illustration of the generality of Theorem 1, we now instantiate it to the range of specific algorithms discussed in Section 3.2. Since all these algorithms are run with gradient input generated by (SFO), applying Theorem 1 would require mapping the requirements of Assumption 3 to the primitives of (SFO). A convenient way to achieve this is by means of the proposition below:

Proposition 1. Suppose that Algorithms 1-5 are run with a gradient oracle 𝑉 ( 𝑥 ; 𝜃 ) = 𝑣 ( 𝑥 ) + err ( 𝑥 ; 𝜃 ) such that

for all 𝑧 ∈ T 𝑥 M , ∥ 𝑧 ∥ 𝑥 = 1 , and for suitable functions 𝜁, 𝜎 : M → ℝ + with 𝜎 bounded on bounded subsets of M and inf 𝑥 𝜁 ( 𝑥 ) > 0 . Then, under Assumptions 2 and 4, the conclusion of Theorem 1 holds, that is, Algorithms 1-5 avoid strict saddle manifolds of 𝑓 .

The proof of Proposition 1 is deferred to Appendix B; we only note here that its proof mainly hinges on verifying the bias requirement ∥ 𝑏 𝑛 ∥ 𝑋 𝑛 = O ( 𝛾 𝑛 ) of (13) by means of ( i ) the boundedness of the error function 𝜎 ( 𝑥 ) on bounded subsets of M ; and ( ii ) controlling the maximal deviation between a retraction and the exponential map for input vectors bounded by 𝜚 .

5 Numerical Illustrations

In this section, we aim to demonstrate the practical applicability of the theoretical framework proposed in our paper by providing numerical illustrations. To do so, we utilize a 2-dimensional torus as the optimization landscape, where the complexity and multi-modal nature of the objective function can be easily visualized in Fig. 2.

Our objective function features three saddle points (in black) and one global minimizer (in red). We subject two RRM schemes, i.e., (RSGD) and (RSEG), to initialization in proximity to these saddle points. This strategic choice rigorously tests their ability to navigate and converge to the global optimum. In line with our theoretical predictions, both RRM methods avoid the saddle points and

(a)

Riemannian stochastic gradient descent

(b)

Riemannian stochastic extra-gradient

1

0

0

-1

-1

4

4

4

-2

2

eventually reach the desirable global minimum. This empirical confirmation reinforces the central message of our paper: stochastic Riemannian schemes only converge to local minimizers.

6 Conclusions and future work

In this paper, we addressed the question of when Riemannian stochastic algorithms can effectively evade saddle points, focusing on the broad category of Riemannian Robbins-Monro schemes. We introduced a novel framework for analyzing the avoidance of Riemannian saddle points within the RRM framework, which encompasses many commonly used Riemannian stochastic algorithms, including retraction-based algorithms. Our framework builds upon the notion of strict saddle points and provides a set of easily verifiable conditions that guarantee the avoidance of such traps.

Our work paves the way for several promising research directions in learning with Riemannian methods. One intriguing avenue for exploration is the investigation of whether Riemannian zerothorder methods, such as the Riemannian extension of the work by Kiefer & Wolfowitz [36], can effectively evade strict saddle points. We believe that combining the insights from the asymptotic pseudotrajectory theory with Euclidean analysis can shed light on this question and provide valuable insights into the behavior of these methods in the Riemannian setting.

Furthermore, an interesting direction for future research is the extension of the avoidance of unstable limit cycles in Euclidean min-max optimization, as studied by Hsieh et al. [29], to the realm of Riemannian games. Investigating the avoidance of unstable limit cycles in this context has the potential to uncover novel phenomena specific to the manifold settings, leading to a deeper understanding on the intricate dynamics and strategies involved in Riemannian games.

Acknowledgments and Disclosure of Funding

This work has been partially supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program grant agreement No 815943, by project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program, and by the French National Research Agency (ANR) in the framework of the PEPR IA FOUNDRY project (ANR-23-PEIA-0003), the 'Investissements d'avenir' program (ANR-15-IDEX-02), the LabEx PERSYVAL (ANR-11-LABX-0025-01), and MIAI@Grenoble Alpes (ANR-19-P3IA-0003). PM is also with the Archimedes Research Unit Athena RC - University of Athens. YPH acknowledges funding through an ETH Foundations of Data Science (ETH-FDS) postdoctoral fellowship.

References

Abraham, R. and Marsden, J. E. Foundations of mechanics . Number 364. American Mathematical Soc., 2008.

Absil, P.-A., Mahony, R., and Sepulchre, R. Optimization Algorithms on Matrix Manifolds . Princeton University Press, 2008.

Alvarez, F., Bolte, J., and Brahic, O. Hessian Riemannian gradient flows in convex programming. SIAM Journal on Control and Optimization , 43(2):477-501, 2004.

Amari, S.-I. Natural gradient works efficiently in learning. Neural computation , 10(2):251-276, 1998.

Antonakopoulos, K., Belmega, E. V., and Mertikopoulos, P. Online and stochastic optimization beyond Lipschitz continuity: A Riemannian approach. In ICLR '20: Proceedings of the 2020 International Conference on Learning Representations , 2020.

Antonakopoulos, K., Mertikopoulos, P., Piliouras, G., and Wang, X. AdaGrad avoids saddle points. In ICML '22: Proceedings of the 39th International Conference on Machine Learning , 2022.

Azizian, W., Iutzeler, F., Malick, J., and Mertikopoulos, P. On the rate of convergence of Bregman proximal methods in constrained variational inequalities. http://arxiv.org/abs/2211.08043 , 2022.

Bauschke, H. H., Bolte, J., and Teboulle, M. A descent lemma beyond Lipschitz gradient continuity: First-order methods revisited and applications. Mathematics of Operations Research , 42(2):330-348, May 2017.

Beck, A. and Teboulle, M. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters , 31(3):167-175, 2003.

Benaïm, M. Dynamics of stochastic approximation algorithms. In Azéma, J., Émery, M., Ledoux, M., and Yor, M. (eds.), Séminaire de Probabilités XXXIII , volume 1709 of Lecture Notes in Mathematics , pp. 1-68. Springer Berlin Heidelberg, 1999.

Benaïm, M. and Hirsch, M. W. Dynamics of Morse-Smale urn processes. Ergodic Theory and Dynamical Systems , 15(6):1005-1030, December 1995.

Benaïm, M. and Hirsch, M. W. Asymptotic pseudotrajectories and chain recurrent flows, with applications. Journal of Dynamics and Differential Equations , 8(1):141-176, 1996.

Benveniste, A., Métivier, M., and Priouret, P. Adaptive Algorithms and Stochastic Approximations . Springer, 1990.

Bertsekas, D. P. and Tsitsiklis, J. N. Gradient convergence in gradient methods with errors. SIAM Journal on Optimization , 10(3):627-642, 2000.

Bianchi, P., Hachem, W., and Schechtman, S. Stochastic subgradient descent escapes active strict saddles on weakly convex functions. arXiv preprint arXiv:2108.02072 , 2021.

Bomze, I. M., Mertikopoulos, P., Schachinger, W., and Staudigl, M. Hessian barrier algorithms for linearly constrained optimization problems. SIAM Journal on Optimization , 29(3):2100-2127, 2019.

Bonnabel, S. Stochastic gradient descent on Riemannian manifolds. IEEE Trans. Autom. Control , 58(9): 2217-2229, September 2013.

Borkar, V. S. Stochastic Approximation: A Dynamical Systems Viewpoint . Cambridge University Press and Hindustan Book Agency, 2008.

Boumal, N. An introduction to optimization on smooth manifolds . https://www.nicolasboumal.net/ book/ , 2022.

Brandière, O. and Duflo, M. Les algorithmes stochastiques contournent-ils les pièges ? Annales de l'Institut Henri Poincaré, Probabilités et Statistiques , 32(3):395-427, 1996.

Choromanska, A., Henaff, M., Mathieu, M., Ben Arous, G., and LeCun, Y. The loss surfaces of multilayer networks. In AISTATS '15: Proceedings of the 18th International Conference on Artificial Intelligence and Statistics , 2015.

Criscitiello, C. and Boumal, N. Efficiently escaping saddle points on manifolds. Advances in Neural Information Processing Systems , 32, 2019.

Davis, D., Drusvyatskiy, D., and Jiang, L. Subgradient methods near active manifolds: saddle point avoidance, local convergence, and asymptotic normality. arXiv preprint arXiv:2108.11832 , 2021.

Duistermaat, J. J. On Hessian Riemannian structures. Asian Journal of Mathematics , 5:79-91, 2001.

Ferreira, O. and Oliveira, P. Proximal point algorithm on Riemannnian manifolds. Optimization , 51(2): 257-270, 2002.

Ge, R., Huang, F., Jin, C., and Yuan, Y . Escaping from saddle points - Online stochastic gradient for tensor decomposition. In COLT '15: Proceedings of the 28th Annual Conference on Learning Theory , 2015.

Hirsch, M. W. Differential Topology . Springer-Verlag, Berlin, 1976.

Hou, T. Y., Li, Z., and Zhang, Z. Analysis of asymptotic escape of strict saddle sets in manifold optimization. SIAM Journal on Mathematics of Data Science , 2(3):840-871, 2020.

Hsieh, Y.-P., Mertikopoulos, P., and Cevher, V. The limits of min-max optimization algorithms: Convergence to spurious non-critical sets. In International Conference on Machine Learning , pp. 4337-4348. PMLR, 2021.

Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., and Jordan, M. I. How to escape saddle points efficiently. In International conference on machine learning , pp. 1724-1732. PMLR, 2017.

Juditsky, A., Nemirovski, A. S., and Tauvel, C. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems , 1(1):17-58, 2011.

Kakade, S. M. A natural policy gradient. Advances in neural information processing systems , 14, 2001.

Karimi, M. R., Hsieh, Y.-P., Mertikopoulos, P., and Krause, A. The dynamics of Riemannian RobbinsMonro algorithms. In COLT '22: Proceedings of the 35th Annual Conference on Learning Theory , 2022.

Karimi, M. R., Hsieh, Y.-P., Mertikopoulos, P., and Krause, A. The dynamics of Riemannian RobbinsMonro algorithms. In COLT 2022-35th Annual Conference on Learning Theory , pp. 1-31, 2022.

Kawaguchi, K. Deep learning without poor local minima. In NIPS '16: Proceedings of the 30th International Conference on Neural Information Processing Systems , 2016.

Kiefer, J. and Wolfowitz, J. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics , 23(3):462-466, 1952.

Kobayashi, S. and Nomizu, K. Foundations of Differential Geometry . Wiley Classics Library. Wiley, 2 edition, 1996.

Korpelevich, G. M. The extragradient method for finding saddle points and other problems. Èkonom. i Mat. Metody , 12:747-756, 1976.

Kushner, H. J. and Clark, D. S. Stochastic Approximation Methods for Constrained and Unconstrained Systems . Springer, 1978.

Kushner, H. J. and Yin, G. G. Stochastic approximation algorithms and applications . Springer-Verlag, New York, NY, 1997.

Lee, J. D., Simchowitz, M., Jordan, M. I., and Recht, B. Gradient descent only converges to minimizers. In 29th Annual Conference on Learning Theory , pp. 1246-1257, 2016.

Lee, J. D., Panageas, I., Piliouras, G., Simchowitz, M., Jordan, M. I., and Recht, B. First-order methods almost always avoid strict saddle points. Mathematical programming , 176:311-337, 2019.

Lee, J. M. Riemannian Manifolds: an Introduction to Curvature . Number 176 in Graduate Texts in Mathematics. Springer, 1997.

Lee, J. M. Introduction to Smooth Manifolds . Number 218 in Graduate Texts in Mathematics. SpringerVerlag, New York, NY, 2003.

Mertikopoulos, P. and Sandholm, W. H. Riemannian game dynamics. Journal of Economic Theory , 177: 315-364, September 2018.

Mertikopoulos, P., Hallak, N., Kavis, A., and Cevher, V. On the almost sure convergence of stochastic gradient descent in non-convex problems. In NeurIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems , 2020.

Mertikopoulos, P., Hsieh, Y.-P., and Cevher, V. A unified stochastic approximation framework for learning in games. Mathematical Programming , forthcoming, 2023.

Meyer, G., Bonnabel, S., and Sepulchre, R. Linear regression under fixed-rank constraints: a riemannian approach. In 28th International Conference on Machine Learning , 2011.

Nemirovski, A. S. and Yudin, D. B. Problem Complexity and Method Efficiency in Optimization . Wiley, New York, NY, 1983.

Nesterov, Y. Introductory Lectures on Convex Optimization: A Basic Course . Number 87 in Applied Optimization. Kluwer Academic Publishers, 2004.

Neto, J. C., Santos, P., and Soares, P. An extragradient method for equilibrium problems on Hadamard manifolds. Optimization Letters , 10(6):1327-1336, 2016.

Oja, E. Principal components, minor components, and linear neural networks. Neural networks , 5(6): 927-935, 1992.

Panageas, I., Piliouras, G., and Wang, X. First-order methods almost always avoid saddle points: The case of vanishing step-sizes. In Advances in Neural Information Processing Systems , pp. 6471-6480, 2019.

Pemantle, R. Nonconvergence to unstable points in urn models and stochastic aproximations. Annals of Probability , 18(2):698-712, April 1990.

Pemantle, R. Vertex-reinforced random walk. Probability Theory and Related Fields , 92:117-136, 1992.

Popov, L. D. A modification of the Arrow-Hurwicz method for search of saddle points. Mathematical Notes of the Academy of Sciences of the USSR , 28(5):845-848, 1980.

Rakhlin, A. and Sridharan, K. Online learning with predictable sequences. In COLT '13: Proceedings of the 26th Annual Conference on Learning Theory , 2013.

Raskutti, G. and Mukherjee, S. The information geometry of mirror descent. IEEE Transactions on Information Theory , 61(3):1451-1457, 2015.

Robbins, H. and Monro, S. A stochastic approximation method. Annals of Mathematical Statistics , 22: 400-407, 1951.

Rockafellar, R. T. Convex Analysis . Princeton University Press, Princeton, NJ, 1970.

Rockafellar, R. T. and Wets, R. J. B. Variational Analysis , volume 317 of A Series of Comprehensive Studies in Mathematics . Springer-Verlag, Berlin, 1998.

Shapiro, A., Dentcheva, D., and Ruszczy´ nski, A. Lectures on Stochastic Programming . Society for Industrial and Applied Mathematics, Philadelphia, PA, 2009.

Shub, M. Global Stability of Dynamical Systems . Springer-Verlag, Berlin, 1987.

Sra, S. and Hosseini, R. Conic geometric optimization on the manifold of positive definite matrices. SIAM Journal on Optimization , 25(1):713-739, 2015.

Sun, J., Qu, Q., and Wright, J. Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method. IEEE Transactions on Information Theory , 63(2):885-914, 2016.

Sun, Y., Flammarion, N., and Fazel, M. Escaping from saddle points on riemannian manifolds. Advances in Neural Information Processing Systems , 32, 2019.

Tang, G.-j. and Huang, N.-j. Korpelevich's method for variational inequality problems on Hadamard manifolds. Journal of Global Optimization , 54(3):493-509, 2012.

Zhou, Z., Mertikopoulos, P., Bambos, N., Boyd, S. P., and Glynn, P. W. On the convergence of mirror descent beyond stochastic convex programming. SIAM Journal on Optimization , 30(1):687-716, 2020.

A Further examples of RRM schemes

In this section, we provide two additional algorithmic examples supplementing the range of Algorithms 1-5 to illustrate the applicability of our RRM template.

Algorithm 6 (Riemannian proximal point methods) . The (deterministic) Riemannian proximal point method (RPPM) [25] is an implicit ('backward') update rule of the form

The RRM representation of (RPPM) is then obtained by taking 𝑏 𝑛 = Γ 𝑋 𝑛 + 1 → 𝑋 𝑛 ( 𝑣 ( 𝑋 𝑛 + 1 )) -𝑣 ( 𝑋 𝑛 ) and 𝑈 𝑛 = 0 in the decomposition (4) of the error term 𝑊 𝑛 of (RRM). If, in additional, the true gradient 𝑣 ( 𝑋 𝑛 + 1 ) is replaced by an oracle 𝑉 ( 𝑋 𝑛 + 1; 𝜃 𝑛 + 1 ) , then (RPPM) becomes the stochastic version of RPPM by setting

For a detailed discussion, see [25] and references therein.

♦

Algorithm 7 (Riemannian stochastic extra-gradient) . Inspired by the original work of Korpelevich [38], the Riemannian stochastic extra-gradient (RSEG) method [51, 67] proceeds as

where 𝜃 𝑛 and 𝜃 + 𝑛 are independent seeds for (SFO). Thus, to cast (RSEG) in the RRM framework, it suffices to take 𝑈 𝑛 = Γ 𝑋 + 𝑛 → 𝑋 𝑛 ( err ( 𝑋 + 𝑛 ; 𝜃 + 𝑛 )) and 𝑏 𝑛 = Γ 𝑋 + 𝑛 → 𝑋 𝑛 ( 𝑣 ( 𝑋 + 𝑛 )) -𝑣 ( 𝑋 𝑛 ) . ♦

Under the assumptions in Proposition 1, one can show that Algorithms 6-7 also avoid strict saddle points of 𝑓 ; we provide the relevant details in Appendix B.3.

B Missing Proofs of Section 4

B.1. Proof of Theorem 2. We begin by proving Theorem 2, which will play a crucial role in our proof of Theorem 1. For the reader's convenience, we restate the result below:

Theorem 2. Suppose that Assumptions 2-4 hold. Then, with probability 1 , the geodesic interpolation 𝑋 ( 𝑡 ) of the sequence of iterates 𝑋 𝑛 , 𝑛 = 1 , 2 , . . . , generated by (RRM) is an APT of (RGF) .

Proof. To begin, let { 𝑒 𝑖 ( 𝑛 )} 𝑑 𝑖 = 1 be an arbitrary sequence of orthonormal bases for T 𝑋 𝑛 M , and let 𝑈 q 𝑛 be the (Euclidean) noise vector composed of components of the noise 𝑈 𝑛 in the basis { 𝑒 𝑖 ( 𝑛 )} 𝑖 = 1 ...𝑑 , viz.

It is then easy to see that 𝔼 [ 𝑈 q 𝑛 | F 𝑛 ] = 0 , and, moreover

by Assumption 3. Then, following Benaïm [10], consider the 'continuous-to-discrete' counter

which measures the number of iterations required for the effective time 𝜏 𝑛 = P 𝑛 -1 𝑘 = 1 𝛾 𝑘 to reach a given timestamp 𝑡 ≥ 0 . We further denote the piecewise-constant interpolation of the noise sequence as

and we let

Moving forward, since 𝑋 𝑛 → S by assumption, we will also have

and

for some (possibly random) 𝑅 ≥ 0 . Moreover, since 𝑓 is assumed to be 𝐶 2 , (B.6) implies that

for some (possibly random) non-negative constant 𝐺 ≥ 0 . Moreover, since the manifold M is assumed to be smooth, the sectional curvatures at each 𝑋 𝑛 , 𝑛 = 1 , 2 , . . . , must be likewise bounded by some constant 𝐾 max . Then, by the analysis of [34, Eq. 53], there exists a constant 𝐶 ≡ 𝐶 𝐿,𝐺,𝐾 max ,𝑅 depending only on 𝐿, 𝐺, 𝐾 max and 𝑅 such that

By Assumption 3, we have ∥ 𝑏 𝑛 ∥ 𝑋𝑛 = O ( 𝛾 𝑛 ) . Since 𝛾 𝑛 → 0 , it suffices to show that Δ ( 𝑡 ; 𝑇 ) → 0 with probability 1 under Assumptions 2 and 3. This is equivalent to showing that, for any 𝜀 > 0 , we have

To this end, let 𝑛 = 𝑀 ( 𝑡 ) and recall that, by (B.2), we have

for all 𝑤 ∈ ℝ 𝑑 where 𝑑 is the dimension of M . Therefore, for each 𝑤 ∈ ℝ 𝑑 , the sequence 𝑌 𝑛 ( 𝑤 ) defined by

is a supermartingale. Since 𝑌 𝑛 is a supermartingale, we have

for any 𝛿 > 0 . Now, let 𝑒 𝑖 be the 𝑖 -th basis vector of ℝ 𝑑 . Then, by (B.12), we have

Optimizing (B.13) over 𝛿 , we get

Since 𝛾 𝑛 → 0 , with loss of generality we may assume that 𝛾 𝑛 ≤ 1 , and hence

where, analogously to (B.4), we have defined the piece-wise constant interpolated step-size sequence ¯ 𝛾 ( 𝑡 ) = 𝛾 𝑛 for all 𝑡 ∈ [ 𝜏 𝑛 , 𝜏 𝑛 + 1 ) , 𝑛 ≥ 1 . (B.16)

Since

by the union bound, we have

« ‹ for some 𝑡 ≤ 𝑠 ≤ 𝑡 + 𝑇 . Therefore, by setting 𝜆 : = exp GLYPH<16> -𝜀 2 2 𝜎 2 𝑑 2 𝑇 GLYPH<17> < 1 , we have

by Assumption 2. The Borel-Cantelli Lemma then implies that the following event happens almost surely:

The proof is finished by noting that, for 𝑘𝑇 ≤ 𝑡 < ( 𝑘 + 1 ) 𝑇 ,

■

by triangle inequality.

B.2. Proof of Theorem 1. We are now in a position to present our proof of Theorem 1, which we restate below for convenience:

Theorem 1. Let 𝑋 𝑛 , 𝑛 = 1 , 2 , . . . , be the sequence of states generated by (RRM) , and let S be a strict saddle manifold of 𝑓 . Then, under Assumptions 1-4, we have

where dist ( S , 𝑋 𝑛 ) = inf 𝑥 ∈ S dist ( 𝑥, 𝑋 𝑛 ) denotes the ( Riemannian ) distance of 𝑋 𝑛 from S .

Proof. Assume that 𝑋 𝑛 → S . We will show that this event has zero probability in a series of steps which we outline below.

Step 1: Isometrically embedded Robbins-Monro iterates. Since M is assumed to be smooth, the second Nash embedding theorem [37] implies there exists a smooth and isometric embedding 𝜄 : M → ℝ 𝑀 such that, for all 𝑥 ∈ M and all 𝑧, 𝑤 ∈ T 𝑥 M , we have

Since 𝜄 is an embedding, it is surjective. Since it is isometric, it preserves distance and hence must be one-to-one. Therefore, 𝜄 is an diffeomorphism since it is also smooth. We can therefore define the pushforward of the vector field 𝑣 on M to a vector field on the image M 𝐸 ⊂ ℝ 𝑀 in the usual way as

We also set S 𝐸 : = 𝜄 ( S ) .

By the Tietze extension theorem and the smooth manifold extension lemma [44], 𝑣 𝐸 0 ( 𝑥 𝐸 ) can be extended to a Lipschitz continuous vector field on all of ℝ 𝑀 , which we still denote by 𝑣 𝐸 0 ( 𝑥 𝐸 ) . To avoid trivialities, we will also need to ensure that 𝑣 𝐸 0 ( 𝑥 𝐸 ) is not 0 in a neighborhood of S 𝐸 : If this is the case, then we set our target field 𝑣 𝐸 ( 𝑥 𝐸 ) : = 𝑣 𝐸 0 ( 𝑥 𝐸 ) ; otherwise, let 1 denote the vector of 1's in all coordinates, and define a new vector field 𝑣 𝐸 on ℝ 𝑀 as

where dist 𝐸 ( 𝑥 𝐸 , S 𝐸 ) : = inf 𝑦 𝐸 ∈ S 𝐸 ∥ 𝑥 𝐸 -𝑦 𝐸 ∥ . Obviously, this new vector field agrees with 𝑣 𝐸 0 ( 𝑥 𝐸 ) on M 𝐸 and therefore is still the pushforward of 𝑣 under 𝜄 . Moreover, it is not uniformly 0 in a neighborhood of S 𝐸 . [It is worth noting that the so-defined vector field 𝑣 𝐸 is in general not the (Euclidean) gradient of any function, a fact which presents significant difficulty to our analysis.]

Step 2: S 𝐸 is an unstable invariant set. Our next goal is to show that there exists an unstable neighborhood U 𝐸 around S 𝐸 in the following sense: First, for each ˆ 𝑥 ∈ S , consider its image ˆ 𝑥 𝐸 = 𝜄 ( ˆ 𝑥 ) ∈ S 𝐸 . Since 𝑣 𝐸 agrees with the pushforward of 𝑣 ≡ grad 𝑓 under 𝜄 , and since 𝜄 is an isometry, we have the following relation for all tangent vector 𝑧 ∈ T ˆ 𝑥 M :

Since ˆ 𝑥 is a strict saddle point, (B.25) implies that 𝜆 min GLYPH<16> D 𝑣 𝐸 ˆ 𝑥 𝐸 GLYPH<17> < -𝑐 -< 0 for all ˆ 𝑥 𝐸 ∈ S 𝐸 . By an established series of arguments [10, 12, 46], using the stable manifold theorem for a strict saddle [63] and the transversality of the strict saddle manifold [1], there exists a ( 𝑀 -𝑚 ) -dimensional embedded submanifold Q 𝐸 in ℝ 𝑀 that contains S 𝐸 . [Here, 1 ≤ 𝑚 ≤ 𝑀 , and 𝑀 -𝑚 represents the dimension of the unstable manifold of 𝑣 𝐸 .] Moreover, writing Φ 𝐸 for the flow generated by 𝑣 𝐸 , it follows that Q 𝐸 is locally invariant under Φ 𝐸 . Hence, there exists a neighborhood N 𝐸 of S 𝐸 in ℝ 𝑀 and a positive time 𝑡 0 such that for all | 𝑡 | ≤ 𝑡 0 , the following inclusion holds:

To proceed, note that ℝ 𝑀 can be decomposed further as the direct sum of the tangent space to Q 𝐸 at 𝑥 𝐸 , denoted by T 𝑥 𝐸 Q 𝐸 , and an additional complementary subspace denoted by E 𝑢 𝑥 𝐸 :

The mapping 𝑥 𝐸 → E 𝑢 𝑥 𝐸 is continuous, where 𝑥 𝐸 varies over S 𝐸 and E 𝑢 𝑥 𝐸 belongs to the Grassmanian manifold 𝐺 ( 𝑚, 𝑀 ) . It is important to note that E 𝑢 𝑥 𝐸 contains at least one direction in T 𝑥 𝐸 M 𝐸 due to (B.25). Then, for all 𝑡 ∈ ℝ and 𝑥 ∈ S 𝐸 , the Jacobian of Φ 𝐸 𝑡 evaluated at 𝑥 𝐸 maps E 𝑢 𝑥 𝐸 to E 𝑢 Φ 𝐸 ( 𝑥 𝐸 ) , i.e.,

𝑡

Finally, and most importantly, we have the following characterization that formalizes the idea that all directions in the unstable manifold should diverge under Φ 𝐸 𝑡 : There exist positive constants 𝑐 and 𝐶 such that for all 𝑥 𝐸 ∈ S 𝐸 , 𝑤 𝐸 ∈ E 𝑢 𝑥 𝐸 , and 𝑡 ≥ 0 , the following inequality holds:

The above verifies all the conditions for a unstable invariant set for S 𝐸 in the sense of Benaïm [10]. A deep result by Benaïm & Hirsch [11] then asserts the existence of a local Lyapunov function near a neighborhood of S 𝐸 , whose construction we outline below.

Step 3: Local Lyapunov function 𝜂 𝐸 . For a right-differentiable function 𝜂 𝐸 : ℝ 𝑀 → ℝ we define its right derivative D 𝜂 𝐸 applied to a vector ℎ 𝐸 ∈ ℝ 𝑀 by

If 𝜂 𝐸 is differentiable, then (B.30) is simply ⟨∇ 𝜂 𝐸 ( 𝑥 𝐸 ) , ℎ 𝐸 ⟩ . In view of all this, Benaïm [10] provides the following crucial result:

Proposition B.1 (Benaïm, 1999, Prop. 9.5) . There exists a compact neighborhood U 𝐸 ( S 𝐸 ) of S 𝐸 , positive numbers 𝑙, 𝛽 > 0 , and a map 𝜂 𝐸 : U 𝐸 ( S 𝐸 ) → ℝ + such that 𝜂 𝐸 ( 𝑥 𝐸 ) = 0 if and only if 𝑥 𝐸 ∈ Q 𝐸 , and the following holds:

(i) 𝜂 𝐸 is 𝐶 2 on U 𝐸 ( S 𝐸 ) \ Q 𝐸 .

(ii) For all 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 ) ∩ Q 𝐸 , 𝜂 𝐸 admits a right derivative D 𝜂 𝐸 ( 𝑥 𝐸 ) : ℝ 𝑀 → ℝ 𝑀 which is Lipschitz, convex and positively homogeneous.

(iii) There exists 𝑘 > 0 and a neighborhood W 𝐸 ⊂ ℝ 𝑀 of 0 such that for all 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 ) and 𝑧 𝐸 ∈ W 𝐸 ,

(iv) There exists 𝑐 1 > 0 such that for all 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 ) \ Q 𝐸

where Π is the projection of a neighborhood of S 𝐸 onto Q 𝐸 . (v) For all 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 ) ∩ Q 𝐸 , 𝑤 𝐸 ∈ T 𝑥 𝐸 Q 𝐸 and 𝑧 𝐸 ∈ ℝ 𝑀 ,

(vi) For all 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 ) we have

The function 𝜂 𝐸 will serve as a local 'energy function' that plays an instrumental role in our analysis; the well-posedness of Π is guaranteed by [27, Chap 4].

Step 4: Geodesic offset. Consider the image of an RRM scheme 𝑋 𝐸 𝑛 : = 𝜄 ( 𝑋 𝑛 ) . If 𝑋 𝐸 𝑛 ∉ U 𝐸 ( S 𝐸 ) for all 𝑛 , then there is nothing to prove. Otherwise, without loss of generality we may assume that 𝑋 𝐸 1 ∈ U 𝐸 ( S 𝐸 ) . Accordingly, define the first exit time 𝑇 from U 𝐸 ( S 𝐸 ) as

Evidently, 𝑇 is a stopping time adaptive to F 𝑛 , so it suffices to show that 3

To this end, a notion that plays a central role in our analysis is the geodesic offet , defined as follows. Define the pushforward of the respective noise and bias vectors in the RRM scheme 𝑋 𝑛 by

It is important to remember that 𝑈 𝐸 𝑛 ∈ T 𝑋 𝐸 𝑛 M 𝐸 , a fact that we will use freely in the sequel.

We now formally define the geodesic offset Δ ( 𝑥 ; 𝑧 ) ∈ ℝ 𝑀 as, for any 𝑥 ∈ M and 𝑧 ∈ T 𝑥 M ,

By Assumption 4, there exists 𝜚 > 0 such that, for all ∥ 𝑧 ∥ 𝑥 < 𝜚 , the exponential mapping is the unique minimizing geodesic. Furthermore, for all such 𝑧 's, define the curve 𝛾 𝐸 ( 𝑡 ) : = 𝜄 ( 𝑥 ) + 𝑡 D 𝜄 𝑥 ( 𝑧 ) , then

so that 𝛾 𝐸 ( 𝑡 ) agrees with the image of the geodesics 𝜄 ( exp 𝑥 ( 𝑡𝑧 )) . As a result, for any ∥ 𝑧 ∥ 𝑥 < 𝜚 , we have Δ ( 𝑥 ; 𝑧 ) = O (∥ 𝑧 ∥ 2 𝑥 ) . Now, setting 𝑥 ← 𝑋 𝑛 and 𝑧 ← 𝛾 𝑛 ( 𝑣 ( 𝑋 𝑛 ) + 𝑈 𝑛 + 𝑏 𝑛 ) , we have

where 𝜖 𝐸 𝑛 : = Δ ( 𝑋 𝑛 ; 𝛾 𝑛 ( 𝑣 ( 𝑋 𝑛 ) + 𝑈 𝑛 + 𝑏 𝑛 )) . By Assumption 3, we know that 𝑈 𝐸 𝑛 + 𝑏 𝐸 𝑛 = O ( 1 ) almost surely. Moreover, since 𝑣 is smooth, on the event 𝑇 = ∞ , 𝑋 𝐸 𝑛 ∈ U 𝐸 ( S 𝐸 ) for all 𝑛 and therefore sup 𝑛 ≥ 1 ∥ 𝑣 𝐸 ( 𝑋 𝐸 𝑛 )∥ < ∞ . Since 𝛾 𝑛 → 0 , for any 𝑛 large enough, we get

Now, define two sequences of random variables { 𝑌 𝑛 } 𝑛 ≥ 1 and { 𝑆 𝑛 } 𝑛 ≥ 1 as

The importance of these sequences is that the event { 𝑇 = ∞} is contained in the event { 𝑆 𝑛 → 0 } . To see this, assume 𝑇 = ∞ . Then we have 𝑌 𝑛 + 1 = 𝜂 𝐸 ( 𝑋 𝐸 𝑛 + 1 ) -𝜂 𝐸 ( 𝑋 𝐸 𝑛 ) and 𝑆 𝑛 = 𝜂 𝐸 ( 𝑋 𝑛 ) by Eqs. (B.43a) and (B.43b). In addition, since { 𝑋 𝐸 𝑛 } remains in U 𝐸 ( S 𝐸 ) by definition of the stopping time 𝑇 , Theorem 2 combined with [10, Theorem 5.7] asserts that the limit set 𝐿 ({ 𝑋 𝐸 𝑛 }) of { 𝑋 𝐸 𝑛 } is a nonempty compact invariant subset of U 𝐸 ( S 𝐸 ) , so that for all 𝑦 𝐸 ∈ 𝐿 ({ 𝑋 𝐸 𝑛 }) and 𝑡 ∈ ℝ , Φ 𝐸 𝑡 ( 𝑦 𝐸 ) ∈ U 𝐸 ( S 𝐸 ) . But then Proposition B.1(vi) implies that 𝜂 𝐸 ( Φ 𝐸 𝑡 ( 𝑦 𝐸 )) ≥ 𝑒 𝛽𝑡 𝜂 𝐸 ( 𝑦 𝐸 ) for all 𝑡 > 0 , forcing 𝜂 𝐸 ( 𝑦 𝐸 ) to be zero. Since 𝜂 𝐸 ( 𝑥 𝐸 ) = 0 if and only if 𝑥 𝐸 ∈ Q 𝐸 , we have 𝐿 ({ 𝑋 𝐸 𝑛 }) ⊂ Q 𝐸 , which implies 𝑆 𝑛 = 𝜂 𝐸 ( 𝑋 𝐸 𝑛 ) → 0 .

Therefore, the rest of the proof is devoted to showing that ℙ ( lim 𝑛 →∞ 𝑆 𝑛 = 0 ) = 0 .

Step 5: Probabilistic estimates. To this end, we will need two technical lemmas, originally due to Pemantle [55], and extended to their current form by Benaïm & Hirsch [12].

Lemma B.1. Let 𝑆 𝑛 be a nonnegative stochastic process, 𝑆 𝑛 = 𝑆 0 + P 𝑛 𝑘 = 1 𝑌 𝑘 where 𝑌 𝑛 is F 𝑛 -measurable. Let 𝛼 𝑛 : = P ∞ 𝑘 = 𝑛 𝛾 2 𝑘 . Assume there exist a sequence 0 ≤ 𝜀 𝑛 = 𝑜 ( √ 𝛼 𝑛 ) , constants 𝑎 1 , 𝑎 2 > 0 and an integer 𝑁 0 such that for all 𝑛 ≥ 𝑁 0 ,

(i) | 𝑌 𝑛 | = 𝑜 ( √ 𝛼 𝑛 ) .

(ii) 1 { 𝑆 𝑛 >𝜀 𝑛 } 𝔼 [ 𝑌 𝑛 + 1 | F 𝑛 ] ≥ 0 .

(iii) 𝔼 [ 𝑆 2 𝑛 + 1 -𝑆 2 𝑛 | F 𝑛 ] ≥ 𝑎 1 𝛾 2 𝑛 .

(iv) 𝔼 [ 𝑌 2 𝑛 + 1 | F 𝑛 ] ≤ 𝑎 2 𝛾 2 𝑛 .

Then ℙ ( lim 𝑛 →∞ 𝑆 𝑛 = 0 ) = 0 .

Lemma B.2. Let 𝑆 𝑛 be a nonnegative stochastic process, 𝑆 𝑛 = 𝑆 0 + P 𝑛 𝑘 = 1 𝑌 𝑘 where 𝑌 𝑛 is F 𝑛 -measurable and | 𝑌 𝑛 | ≤ 𝐶 almost surely for some constant 𝐶 . Assume that P 𝑛 𝛾 2 𝑛 = ∞ , and there exists 𝑐 > 0 , 𝑁 ′ ∈ ℕ such that for all 𝑛 ≥ 𝑁 ′ ,

Then

Our proof will be concluded by verifying all the premises of Lemmas B.1-B.2. To that end, note first that the sequence 𝑆 𝑛 defined in (B.43b) is nonnegative by construction. We will then separate the analysis into two cases:

Case 1. Square-summable step-sizes , i.e., P 𝑛 𝛾 2 𝑛 < ∞ . In this case, we have

so 𝛾 𝑛 = 𝑜 GLYPH<18> √︃ P ∞ 𝑘 = 𝑛 𝛾 2 𝑘 GLYPH<19> . This fact will be used in the proof when we invoke Lemma B.1 below with 𝜀 𝑛 = O ( 𝛾 𝑛 ) and 𝛼 𝑛 = P ∞ 𝑘 = 𝑛 𝛾 2 𝑘 therein. The verification process then proceeds as follows:

Verifying Lemma B.1(i) and (iv): By the Lipschitz continuity of 𝜂 𝐸 , we know that

where 𝐿 ′ is the Lipschitz constant of 𝜂 𝐸 . Wehave seen in the analysis of (B.42) that ∥ 𝑣 𝐸 ( 𝑋 𝐸 𝑛 ) + 𝑈 𝐸 𝑛 + 𝑏 𝐸 𝑛 ∥ = O ( 1 ) almost surely by Assumption 3 on the event 𝑇 = ∞ . Therefore, | 𝑌 𝑛 + 1 | = O ( 𝛾 𝑛 ) = 𝑜 ( √ 𝛼 𝑛 ) which implies both Lemma B.1(i) and (iv).

Verifying Lemma B.1(ii): Let 𝑘 ′ = 𝑘 ∥ 𝑣 𝐸 ∥ + 𝜎 where 𝑘 is given by Proposition B.1(iii) and ∥ 𝑣 𝐸 ∥ : = sup { 𝑣 𝐸 ( 𝑥 𝐸 ) : 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 )} and 𝜎 is the uniform bound of 𝑈 𝑛 . If 𝑛 ≤ 𝑇 , using Proposition B.1(ii), (iii), (v) and (vi) we have

By Assumption 3, there exists a constant 𝑐 ′ > 0 such that -∥ 𝑏 𝐸 𝑛 ∥ ≥ -𝑐 ′ 𝛾 𝑛 (a.s.). Combining this with the Lipschitz continuity of 𝜂 𝐸 and (B.42), we can merge the last four terms in (B.48) as

for some constant 𝑘 ′′ > 0 . We thus get

By Proposition B.1(ii) again, we have

since we have assumed the noise to be zero mean. Combining (B.50) and (B.51), we then get

If 𝑛 > 𝑇 , 𝑌 𝑛 + 1 = 𝛾 𝑛 so trivially

Combining (B.52) with (B.53), we see that Lemma B.1(ii) is satisfied with 𝜀 𝑛 = 𝑘 ′′ 𝛽 𝛾 𝑛 .

Verifying Lemma B.1(iii): We begin by observing that

If 𝑆 𝑛 ≥ 𝜀 𝑛 , then the last term on the right-hand side of (B.54) is non-negative by Lemma B.1(ii) that we just verified above. If 𝑆 𝑛 < 𝜀 𝑛 , (B.52) with (B.53) imply that 𝑆 𝑛 𝔼 [ 𝑌 𝑛 + 1 | F 𝑛 ] ≥ -𝜀 𝑛 𝑘 ′′ 𝛾 2 𝑛 = -O ( 𝛾 3 𝑛 ) . In other words, (B.54) can be rewritten as

Below, we shall prove that 𝔼 [ 𝑌 2 𝑛 + 1 | F 𝑛 ] ≥ 𝑏 1 𝛾 2 𝑛 for some 𝑏 1 > 0 and 𝑛 large enough. Combining this with (B.55) proves Lemma B.1(iii).

From (B.49), we deduce

We now claim that

where 𝑐 1 is given by Proposition B.1(iv) and 𝜁 is defined in Assumption 3. To see this, recall that ∥ 𝑈 𝑛 ∥ 𝑋𝑛 < 𝜎 by Assumption 3. Moreover, we have 𝑋 𝐸 𝑛 ∈ U 𝐸 ( S 𝐸 ) on the event 𝑇 = ∞ . Proposition B.1(i) then implies 𝜂 𝐸 is differentiable on U 𝐸 ( S 𝐸 ) \ Q 𝐸 , and Proposition B.1(iv) further shows that

where we have used the fact that 𝑈 𝐸 𝑛 ∈ T 𝑋 𝑛 M 𝐸 and Assumption 3.

If 𝑋 𝐸 𝑛 ∈ Q 𝐸 , we can choose a unit vector 𝑣 𝐸 𝑛 ∈ ker ( 𝐼 -D Π ( 𝑋 𝐸 𝑛 )) ⊥ ∩ T 𝑋 𝐸 𝑛 M 𝐸 where Π denotes the projection operator onto Q 𝐸 ; note that 𝑣 𝐸 𝑛 ∈ ker ( 𝐼 -D Π ( 𝑋 𝐸 𝑛 )) ⊥ ∩ T 𝑋 𝐸 𝑛 M 𝐸 ≠ ∅ since E 𝑢 𝑥 𝐸 contains at least one direction in T 𝑥 𝐸 M 𝐸 for all 𝑥 𝐸 ∈ U 𝐸 ( S 𝐸 ) ∩ M 𝐸 . By the definition of 𝑣 𝐸 𝑛 , we have ⟨ 𝑈 𝐸 𝑛 , 𝑣 𝐸 𝑛 ⟩ = ⟨ 𝑈 𝐸 𝑛 -D Π ( 𝑋 𝐸 𝑛 ) 𝑈 𝐸 𝑛 , 𝑣 𝐸 𝑛 ⟩ . Let H = { 𝑛 ≤ 𝑇 } ∩ { 𝑋 𝐸 𝑛 ∈ Q 𝐸 } . By Proposition B.1(iv), Cauchy-Schwartz, and Assumption 3, we get

where 𝑣 𝑛 is the pullback of 𝑣 𝐸 𝑛 under 𝜄 . Since 𝜄 is an isometry, the pullback preserves the inner product, and therefore

by Assumption 3. Combining Eqs. (B.53), (B.56) and (B.60) and Item Case 1. then gives

On the other hand, we always have 𝔼 [ 𝑌 2 𝑛 + 1 | F 𝑛 ] ≥ 𝔼 [[ 𝑌 𝑛 + 1 ] + | F 𝑛 ] 2 by Jensen. It then follows that 𝔼 [ 𝑌 2 𝑛 + 1 | F 𝑛 ] ≥ 𝑏 1 𝛾 2 𝑛 for some 𝑏 1 > 0 and large enough 𝑛 as desired.

We have now verified conditions (i)-(iv) in Lemma B.1. Thus, Lemma B.1 concludes that

which finishes the proof for the case of P 𝑛 𝛾 2 𝑛 < ∞ .

Case 2. When P 𝑛 𝛾 2 𝑛 = ∞ , the same proof above shows that 𝔼 [ 𝑌 2 𝑛 + 1 | F 𝑛 ] ≥ 𝑏 1 𝛾 2 𝑛 for some 𝑏 1 > 0 and large enough 𝑛 . Combining this with (B.55) yields

for some 𝑐 > 0 . Lemma B.2 then concludes:

as claimed.

■

B.3. Proof of Proposition 1. We conclude this appendix with the application of Theorem 1 to Algorithms 1-5 under the explicit oracle assumptions of Proposition 1. For convenience, we restate the relevant result below:

Proposition 1. Suppose that Algorithms 1-5 are run with a gradient oracle 𝑉 ( 𝑥 ; 𝜃 ) = 𝑣 ( 𝑥 ) + err ( 𝑥 ; 𝜃 ) such that

for all 𝑧 ∈ T 𝑥 M , ∥ 𝑧 ∥ 𝑥 = 1 , and for suitable functions 𝜁, 𝜎 : M → ℝ + with 𝜎 bounded on bounded subsets of M and inf 𝑥 𝜁 ( 𝑥 ) > 0 . Then, under Assumptions 2 and 4, the conclusion of Theorem 1 holds, that is, Algorithms 1-5 avoid strict saddle manifolds of 𝑓 .

Remark. In additional to the claimed Algorithms 1-5, we will further prove the same conclusion for the two algorithms considered in Appendix A.

Proof. By Theorem 1, it suffices to verify Assumption 3 under (17) and the event dist ( 𝑋 𝑛 , S ) → 0 . We proceed method by method.

Algorithm 1. Since 𝑏 𝑛 = 0 in Algorithm 1, Assumption 3 holds trivially by (17).

Algorithms 2, 3 and 5. By definition, R 𝑥 ( 𝑧 ) is a smooth map and hence satisfies lim 𝑧 → 0 R 𝑥 ( 𝑧 ) = 𝑥 . On the event dist ( 𝑋 𝑛 , S ) → 0 , we have 𝑣 ( 𝑋 𝑛 ) + 𝑈 𝑛 + 𝑏 𝑛 = O ( 1 ) , and therefore 𝑋 𝑛 + 1 lies in the injectivity radius of 𝑋 𝑛 with probability 1 for 𝑛 large enough. As a result, the mapping log 𝑋 𝑛 ( 𝑋 𝑛 + 1 ) is well-define for all 𝑛 large enough.

We first consider Algorithms 2 and 5 whose proofs are identical since they are both are the form:

Let ˜ 𝑣 𝑛 ∈ T 𝑋 𝑛 M be the vector such that exp 𝑋𝑛 ( 𝛾 𝑛 ˜ 𝑣 𝑛 ) = 𝑋 𝑛 + 1 , i.e.,

Then (B.64) is an RRM scheme with 𝑊 𝑛 = ˜ 𝑣 𝑛 -𝑣 ( 𝑋 𝑛 ) where ˜ 𝑣 𝑛 is defined in (B.65). Consider the curve 𝑐 ( 𝑡 ) : = R 𝑋 𝑛 ( 𝑡𝑉 ( 𝑋 𝑛 ; 𝜃 𝑛 )) . By (17), on the event dist ( 𝑋 𝑛 , S ) → 0 , the curve 𝑐 ( 𝑡 ) lies in the injectivity radius of 𝑋 𝑛 almost surely for all 𝑡 ∈ [ 0 , 𝛾 𝑛 ] and all 𝑛 large enough. Let ˆ 𝑐 ( 𝑡 ) be the smooth curve of 𝑐 ( 𝑡 ) in the normal coordinate with base 𝑋 𝑛 and an arbitrary orthonormal frame, and let ˆ 𝑋 𝑛 + 1 be the normal coordinate of 𝑋 𝑛 + 1 . Also, let ˜ 𝑣 N 𝑛 be the (Euclidean) vector of ˜ 𝑣 𝑛 expanded in the chosen orthonormal basis, and define 𝑉 N ( 𝑋 𝑛 ; 𝜃 𝑛 ) and err N ( 𝑋 𝑛 ; 𝜃 𝑛 ) similarly. By definition, ˆ 𝑋 𝑛 + 1 is nothing but 𝛾 𝑛 ˜ 𝑣 N 𝑛 . In addition, by (17), we have

for some 𝜎 < ∞ .

Since 𝑋 𝑛 = 𝑐 ( 0 ) and 𝑋 𝑛 + 1 = 𝑐 ( 𝛾 𝑛 ) , by properties of a retraction map we must have

where ˜ 𝑏 𝑛 = O GLYPH<0> 𝛾 𝑛 ∥ 𝑉 ( 𝑋 𝑛 ; 𝜃 𝑛 )∥ 2 𝑋𝑛 GLYPH<1> = O ( 𝛾 𝑛 ) . Therefore,

which proves the condition for 𝑏 𝑛 in Assumption 3. On the other hand, (B.67) shows that

since ∥ 𝑉 N ( 𝑋 𝑛 ; 𝜃 𝑛 )∥ = O ( 1 ) by (17) and ˜ 𝑏 𝑛 = O ( 𝛾 𝑛 ) . Finally, for any unit vector 𝑧 ∈ T 𝑋 𝑛 M , (B.67) implies

Since 𝛾 𝑛 → 0 , this finishes the proof of Algorithms 2 and 5. For Algorithm 3, an Euclidean oracle of the form (17) translates to a Riemannian oracle with err ′ ( 𝑥 ; 𝜃 ) : = ∇ 2 ℎ ( 𝑥 ) -1 err ( 𝑥 ; 𝜃 ) . It then suffices to note that, on the event dist ( 𝑋 𝑛 , S ) → 0 , ∇ 2 ℎ ( 𝑋 𝑛 ) is both upper and lower bounded.

Algorithms 4, 6 and 7. For (RSEG), 𝑈 𝑛 = Γ 𝑋 + 𝑛 → 𝑋𝑛 ( err ( 𝑋 + 𝑛 ; 𝜃 + 𝑛 )) so

by (17) and the fact that the parallel transport map is a linear isometry. For the bias term, the definition of (RSEG) yields

by the same argument as for Algorithms 2, 3 and 5.

For (ROG), we have 𝑈 𝑛 = Γ 𝑋 + 𝑛 → 𝑋𝑛 ( err ( 𝑋 𝑛 ; 𝜃 + 𝑛 )) and 𝑏 𝑛 = Γ 𝑋 + 𝑛 → 𝑋𝑛 ( 𝑣 ( 𝑋 + 𝑛 )) -𝑣 ( 𝑋 𝑛 ) , so Assumption 3 can be checked exactly as in the case of Algorithm 7 above. The analysis for Algorithm 6 is similar so we omit the details. ■

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study saddle point avoidance of stochastic gradient algorithms formulated on Riemannian manifolds. They prove a number of results on saddle avoidance for which the Euclidean analogs, possibly under slightly different technical assumptions, are known. These results show that stochastic gradient descent algorithms also avoid strict saddle points even if working in a geometric setting. This is a purely theoretical paper and does not include any experiments or empirical examples.\n\nEDIT: I've read the rebuttal, and am happy to keep my score where it was.\n\nSTRENGTHS:\nThis paper addresses an important topic and is very-well-written, particularly its main text. It is very readable to those with theoretical training, but not necessarily experts in optimization - a common pitfall among works written at a similar technical level.\n\nDue to field's historical emphasis on convex optimization at the expense of non-convex optimization, theory on saddle point avoidance is only beginning to become available. At the same time, Riemannian optimization is a rapidly growing area, and has the potential to help bring improved understanding to optimization at a time when doing so is increasingly important due to the rise of deep learning. Therefore, the contribution is timely and valuable\n\nThe technical appendix reads as if it was written carefully. I spotted no hints of sloppiness that would call the results into question, but at the same time this work is heavy enough that typos - including potentially in references the work relies on - are a concern. I did not do a full check, since my primary work is in a different (albeit related) area, and this paper is technical enough that it needs an expert familiar other saddle-avoidance papers for the verification to be trustworthy. I encourage the authors to do a careful read-over after not thinking about this paper for at least a month, so as to go over the logic and spot typos. With this caveat stated, the paper clearly meets the NeurIPS bar from a technical viewpoint.\n\nThe introduction is particularly well put together, makes clear what is already done, what the authors are adding, and what the key takeaways are. I would use this paper as an example to students for how to write a good introduction.\n\nWEAKNESSES:\nNo experiments. This paper would have been made significantly stronger with even a minimal empirical study to illustrate how a few examples of RSGD behave on common manifolds. It would have been particularly interesting to see a case or two where RSGD does not get stuck, confirming the theory, and one which violates the theory's assumptions where it does get stuck, illustrating the limitations.\n\nSec 2: consider a more informative title\n\nOptimization-theoretic assumptions involving the setting should be stated more clearly clearly all in one location, so that optimization theory people can easily see how strong the obtained results are:\n* Lipschitz gradient\n* Avoidance defined in terms of strict saddle points\n* Geodesically complete manifold\n* Maybe consider adding \\paragraph called \"Key Assumptions\"?\n\nPaper takes a bit too long to get to the point, page 5 is still describing the setting. Even though the description is good, consider simplifying by having less examples:\n* Consider splitting the examples into one or two key examples, then have the results more in the middle of the paper, then having a final section which describes additional examples\n\nThe paper talks way too many times about how intricate the analysis is. My experience is that most publishable forms of analysis are either intricate or they are obvious, so stating this just makes it feel like the authors want to make themselves look fancy and important to the readers. This is distracting and takes away from the paper's actual content. Rather than saying it's intricate, maybe instead say what specific elements of the analysis make it so.\n\nQUESTIONS:\nCertain points that describe what is new could be given in more detail:\n* Line 336: why does this work's probabilistic analysis diverge significantly?\n\nStep 2: would it be fair to say this part of the proof is effectively via a reduction to the Euclidean case?\n\nGeneral question: why is the defined \"(strict) saddle manifold\" actually a manifold? If the saddle points are mere points, and not ridges, this'll (trivially) be a zero-dimensional manifold, as it is a finite or countable point set, but what if there are ridges? Does something assumption here that I've missed prevent singularities from appearing? May help to include some intuition here.\n\nAm I right to think of the RRM scheme as traveling some distance in a tangent space, then mapping back onto the manifold? Thought in this way, it starts to feel a bit like mirror descent, which also projects back onto the space.\n\nAm I right to think about the retraction version as \"viewing the difference between the retraction and exponential map as noise, which we then simply merge with the gradient noise\"?\n\nTypos:\n* 48: \"scopein\" -> typo\n* Line 626: wrong \\eqref, this should instead refer to (13), not to (12)\n* Appendix B: title in the PDF table of contents is \"Proof of thm:repeller\", use \\texorpdfstring to fix this (you can look up what it does)\n\nLIMITATIONS:\nYes, this is a theory paper and the technical assumptions are clearly stated throughout", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-02"}

comments: SUMMARY:
This paper proves that under a rather general RRM scheme (akin to vanilla GD in euclidean space), strict saddles can be avoided when stochastic approaches are used.

STRENGTHS:
1. Proves that Riemannian Optimization, much like its euclidean counterpart, can have its strict saddles easily avoided by using a simple update rule (RRM in this case) coupled with perturbations.

2. This paper analyzed a number of different stochastic algorithms (algorithms 1-5) and shows that they all can escape saddle points.

3. The proof outline is nice. It provides readers a new proof technique that can deal with Riemanian optimization.

WEAKNESSES:
1. For unfamiliar readers, it is hard to gauge how limiting the "our assumptions 1-3" are, and I would like to see the authors provide a bit more analysis on the limitations of these assumptions.

2. Maybe it's better to include the convergence analysis of the RRM update rule to let users understand better the limitations of this result (it doesn't need to be self-derived, it's perfectly fine to quote well-known results).

QUESTIONS:
I want to know how important the noise perturbation is to this analysis. For the RRM, if no random error is observed, will the theorems still work? Also I want to know whether random perturbations is more important for saddle-escaping or the stochastic oracles are more important.

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a focused study on the avoidance of saddle points for stochastic Riemannian optimization algorithms. To tackle this issue, the authors introduce the Riemannian Robbins-Monro (RRM) schemes to the context of Riemannian manifolds, where includes fundamental Riemannian stochastic optimization methods as special cases, such as Riemannian stochastic Gradient Descent (GD), Riemannian proximal GD, and Riemannian optimistic GD.\n\nIn their analysis, the authors have modeled the RRM within the context of a Euclidean stochastic approximation framework by famous Nash embedding theorem, introducing an element of geodesic offset. The authors make the assumption that this offset is $\\mathcal O(\\gamma_n^2)$, a condition found to be met in the case of a bounded-variance stochastic gradient estimator as indicated in Proposition 1. Based on this assumption, they have been able to indicate that RRM schemes have the capacity to avoid saddle points with a probability of 1.\n\nSTRENGTHS:\n- Originality: The analysis presented in Theorem 1 and Theorem 2 is highly innovative. In the analysis, the author apply the Nash embedding theorem to isometrically embed the Riemannian manifold into a Euclidean space. This approach allows them to approximate the intrinsic geometry of the manifold using its extrinsic (now Euclidean) geometry, an approach that seems highly original and may potentially shed light on further development in Riemannian optimization methods.\n\n- Quality: The quality of the paper is commendable. The paper is well-written, and the results are convincing.\n\n- Clarity: The paper is well-structured. The assumptions made are either standard in the field of Euclidean optimization or are accompanied by discussions regarding their implications,  making the paper easy to follow for readers.   \n\n- Significance: The paper is the first to generalize saddle point avoidance for stochastic optimization methods on Riemannian manifolds. Additionally, the proposed Riemannian Robbins-Monro (RRM) schemes encompass numerous state-of-the-art Riemannian stochastic optimization algorithms, illustrating potential applications in tackling Riemannian machine learning problems.\n\nWEAKNESSES:\n- Possible typo: l52 shcemes -> schemes.\n\n- It would be better if there are numerical experiments to validate the findings.\n\nQUESTIONS:\n- In the conclusion section of your paper, you mention 'zeroth-order optimization' as a potential avenue for future work. There have been several studies related to the zeroth-order (or so-called 'bandit') gradient descent method ([1,2]). I am interested to know whether such zeroth-order methods fit within your proposed Riemannian Robbins-Monro (RRM) scheme. If not, could you please clarify the primary gap or obstacle in incorporating these methods into the RRM framework?\n\n- A lot of work on Riemannian optimization tends to focus on complexity analysis in terms of iteration rounds, instead of asymptotic behavior. I was wondering if it would be possible to extend your proposed RRM framework to a non-asymptotic setting. In other words, can the avoidance probability be expressed in terms of the number of iteration rounds, like some approaches in Euclidean space? If this is not feasible, could you elaborate on the main hindrances or challenges in doing so?\n\nRef: [1] J LI et.al., Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization  \n[2] X Wang et.al., Online Optimization over Riemannian Manifolds\n\nLIMITATIONS:\nThe authors have outlined the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of when Riemannian first-order optimization algorithms evade strict saddles points. The proof leverages a connection to the Euclidean case, to the continuous-time Riemannian gradient flow, and shows that the deviation from them is bounded. Intuitions are given in the main paper to facilitate readership.\n\nSTRENGTHS:\nThe paper has done a great job in writing, connecting with existing literature, and giving intuitions on how the proof is done. I agree with the claimed contribution in Lines 42-48 and 276-292.\n\nWEAKNESSES:\nI don't see major problems. I do have a few questions and minor comments below.\n\n## Questions\n1. For the definition of a stable manifold: why do we need positive eigenvalues lower bounded by c+>0? What goes bad if the positive eigenvalues could get arbitrarily close to 0?\n1. Equation 6: Consider any update rule whose output is in the range of the exponential map. We can cast the update as (RRM), as is also done in (6). I understand that the gain is that we can then analyze just the exponential map such as the arguments around (15). However, the difficulty is then transferred to checking whether the logarithm of the update satisfies assumption 2, which is much more indirect than checking whether V satisfies assumption 2 as in Algorithm 1. Is this a problem? Could you comment more on this? \n1. Related to the above, for algorithm 3, why do I view it as a special case of algorithm 2 and hence of RRM, rather than saying oh, algorithm 3 is a special case of RRM, because one can just take logarithm and exponential map of (SMD)?\n1. Does (RGF) always stay within the manifold? If so, why? If not, doesn’t the ‘grad’ become not defined outside the manifold?\n\n## Comments\n1. Line 124-125: The definition looks recursive hence confusing: to understand vn_hat I need to understand Un, bn, both of which depend on vn_hat. \n1. Line 18: “end state of a stochastic Riemannian algorithm can only be a local minimizer.” Do you mean something like the limiting state? Even for convex problems in Euclidean space, gradient descent is only guaranteed to approach a solution - the more iterations, the closer. So, in general, the end state is something close to a local minimizer.\n1. Line 102: the “S” is not defined - I suppose you mean a stable manifold?\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}