SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The above examples have been chosen to illustrate a range of different update mechanisms that can be integrated within the general algorithmic template provided by (RRM). Of course, it is not possible to be exhaustive but, for illustration purposes, we provide some more examples in Appendix A.
4 Analysis and results
We are now in a position to state and discuss our main result concerning the avoidance of saddle points under (RRM). For concreteness, we begin by discussing the technical assumptions that we will need in Section 4.1; subsequently, we proceed with the formal statement of our result and some direct applications thereof in Section 4.2.
4.1. Technical assumptions. Our technical assumptions concern the four main ingredients of (RRM), namely  $(i)$  the regularity of  $f$ ;  $(ii)$  the method's step-size sequence  $\gamma_{n}$ ;  $(iii)$  the statistics of the surrogate gradients  $\hat{v}_{n}$  entering (RRM); and  $(iv)$  the ambient manifold  $\mathcal{M}$ . Specifically, we will require the following:
Assumption 1 (Regularity of  $f$ ). The function  $f$  is  $C^2$  and  $v = -\operatorname{grad}f$  is (geodesically)  $L$ -Lipschitz, i.e., for all  $x,x^{\prime}\in \mathcal{M}$ , we have
$$
\left\| \Gamma_ {x \rightarrow x ^ {\prime}} \left(v (x)\right) - v \left(x ^ {\prime}\right)\right\| _ {x ^ {\prime}} = \left\| v (x) - \Gamma_ {x ^ {\prime} \rightarrow x} \left(v \left(x ^ {\prime}\right)\right)\right\| _ {x} \leq L \operatorname {d i s t} \left(x, x ^ {\prime}\right). \tag {11}
$$
Assumption 2 (Step-size schedule). The step-size sequence  $\gamma_{n}$  of (RRM) satisfies
$$
\sum_ {n = 1} ^ {\infty} \gamma_ {n} = \infty \quad \text {a n d} \quad \sum_ {n = 1} ^ {\infty} \lambda^ {1 / \gamma_ {n}} <   \infty \quad \text {f o r a l l} \lambda \in (0, 1). \tag {12}
$$
Assumption 3 (Surrogate gradients). The offset and random error components of  $\hat{v}_n$  satisfy
$$
\left\| b _ {n} \right\| _ {X _ {n}} \leq C \gamma_ {n}, \quad \left\| U _ {n} \right\| _ {X _ {n}} \leq \sigma , \quad \mathbb {E} \left[ \left[ \langle U _ {n}, z \rangle_ {X _ {n}} \right] _ {+} \mid \mathcal {F} _ {n} \right] \geq \zeta \tag {13}
$$
for suitable constants  $C, \sigma, \zeta > 0$  and for all  $z \in \mathcal{T}_{X_n}\mathcal{M}$ ,  $\|z\|_{X_n} = 1$  (in the above, all conditions are to be interpreted in the almost sure sense and  $[t]_+ = \max\{0, z\}$  denotes the positive part of  $t$ ).
Assumption 4 (Injectivity radius). The injectivity radius of  $\mathcal{M}$  is bounded from below by  $\varrho >0$
Before proceeding, we discuss the implications and range of validity of each of the above assumptions. Since Assumption 1 is standard, we focus on the remaining three below:
On Assumption 2. The step-size conditions typically encountered in the analysis of Robbins-Monro schemes is the  $L^2 - L^1$  ("square-summable-but-not-summable") condition  $\sum_{n} \gamma_{n} = \infty$ ,  $\sum_{n} \gamma_{n}^{2} < \infty$ , cf. [10, 13, 14, 17, 39, 59] and references therein. This puts a hard threshold on the range of allowed step-size schedules at  $\Omega(1/n^{1/2})$ : any step-size that decays at least as slow as  $1/n^{1/2}$  cannot be used under the  $L^2 - L^1$  assumption. By contrast, the step-size condition (12) is considerably more lax and can tolerate near-constant step-sizes of the form  $\gamma_{n} \propto 1 / (\log n)^{1 + \varepsilon}$  for some  $\varepsilon > 0$ . This is enough to cover all deleasing step-size policies used in practice. [We also recall here that, in stochastic non-convex settings, trajectory convergence cannot be guaranteed in general without a vanishing step-size, cf. [10, 18, 40] and references therein.]
On Assumption 3. Three remarks are in order for the noise and offset requirements (13). First, we should note that the condition  $b_{n} = \mathcal{O}(\gamma_{n})$  is, a priori, implicit, because it depends on the statistics of the feedback sequence  $\hat{v}_{n}$ , and these may be difficult to estimate in general. However, in most practical applications, this quantity is under the explicit control of the optimizer: in particular, as we show later in this section, this requirement is satisfied by all the specific algorithms of Section 3.2.
Likewise, the bounded noise requirement is satisfied in many practical cases of interest. For example, when the problem's objective function admits a finite-sum decomposition of the form  $f(x) = \sum_{i=1}^{N} f_i(x)$  for an ensemble of empirical instances  $f_i$ ,  $i = 1, \dots, N$  (the standard framework for applications to data science and machine learning),  $U_n$  is typically generated by sampling a minibatch of  $f$ , which in turn results in an error term of the form  $U_n = q(X_n)$  where  $q(x): \mathcal{M} \to \mathcal{T}_x\mathcal{M}$  is bounded on all compact subsets of  $\mathcal{M}$ . Therefore,  $\|U_n\|_{X_n} \leq \|q(X_n)\|_{X_n} < \sigma$  for some constant  $\sigma$  for any convergent algorithm  $\{X_n\}_n$ .
Finally, the "uniform excitability" condition  $\mathbb{E}[[\langle U_n,z\rangle_{x_n}]_+|\mathcal{F}_n]\geq \zeta$  is also standard in the avoidance literature [10, 54], and it is substantially weaker than the isotropic condition, which, roughly speaking, requires the noise to have the same  $L^2$  magnitude along all directions in space [26, 30, 54]. Instead, (13) only posits that the noise  $U_{n}$  has a non-zero component along each direction, and imposes no other restrictions on the statistical profile of the noise.
On Assumption 4. For our last assumption, recall first that the injectivity radius of  $\mathcal{M}$  at a point  $x\in \mathcal{M}$  is the largest radius for which  $\exp_x$  is a diffeomorphism onto its image; the injectivity radius of  $\mathcal{M}$  is then taken to be the infimum over all such radii [43]. In this regard, Assumption 4 simply serves to ensure that the exponential map is invertible at consecutive iterates of (RRM) so no local topological complications can arise. This assumption is automatically satisfied in closed manifolds (independent of curvature), as well as in non-positively curved manifolds - such as Cartan-Hadamard spaces and the like [43, 44]. This assumption (and its variants) is also standard in the literature, cf. [17, 34, 66] and references therein.
4.2. Avoidance of saddle points. We are now in a position to state our main avoidance result:
Theorem 1. Let  $X_{n}$ ,  $n = 1,2,\ldots$ , be the sequence of states generated by (RRM), and let  $S$  be a strict saddle manifold of  $f$ . Then, under Assumptions 1-4, we have
$$
\mathbb {P} \left(\operatorname {d i s t} \left(\mathcal {S}, X _ {n}\right)\rightarrow 0 \text {a s} n \rightarrow \infty\right) = 0 \tag {14}
$$
where  $\mathrm{dist}(\mathcal{S},X_n) = \inf_{x\in \mathcal{S}}\mathrm{dist}(x,X_n)$  denotes the (Riemannian) distance of  $X_{n}$  from  $\mathcal{S}$ .
Before discussing the proof of Theorem 1, it is worthwhile to compare our work with its closest antecedents. First, in regard to the general avoidance theory in Euclidean spaces [10, 12], the statement is similar in scope (avoidance of unstable manifolds with probability 1), but the techniques and challenges involved are very different. The reason for this is simple: the additive, vector space structure of  $\mathbb{R}^m$  is ingrained at every step of the way in the Euclidean analysis of [10], and adapting the various constructions to a manifold setting can be a complicated affair. For an illustration of the technical difficulties involved, see the recent stochastic approximation analysis of [33].
By contrast, the recent results of [22, 66] paint a complementary picture: they concern Riemannian problems but, at their core, they are deterministic results. More precisely, the noise in [22, 66] is actually injected in an otherwise deterministic gradient scheme to facilitate the escape from flat regions in the vicinity of a saddle point; other than that, the magnitude of the noise must be proportional to the solver's desired accuracy, and hence is typically extremely small. As a result, the analysis of [22, 66] cannot be extended to bona fide stochastic schemes – like (RSGD) – which also explains why these results involve a constant step-size (as opposed to a decreasing step-size schedule, which is required to guarantee trajectory convergence in settings with persistent noise). In this regard, Theorem 1 simultaneously complements the stochastic analysis of [10, 12] to Riemannian problems, and the Riemannian analysis of [22, 66] to a stochastic setting.
Proof outline. To facilitate the reading of our proof, we provide below a detailed outline of the main steps and techniques involved therein, deferring the full proof to Appendix B. We begin with a high-level description of our proof strategy and then encode the main arguments in a series of steps right after.
For the purposes of illustration, suppose that  $\mathcal{M}$  is a subset of  $\mathbb{R}^m$ . Then, given a tangent vector  $z \in \mathcal{T}_x\mathcal{M}$ , we define the geodesic offset (see Fig. 1) from  $x$  along  $z$  as
$$
\Delta (x; z) = \exp_ {x} (z) - x - z \tag {15}
$$
i.e., as the difference between the geodesic emanating from  $x$  along  $z$  and its first-order approximation relative to  $x$  in the ambient space  $\mathbb{R}^m$  (with all differences expressed in the ordinary vector space structure of  $\mathbb{R}^m$ ). The offset  $\Delta(x;z)$  is readily checked to be second-order in  $z$  so, while the curve  $x + tz$  does not in general induce a retraction on the target manifold  $\mathcal{M}$  (in particular, the point  $x + tz$  may not even belong to  $\mathcal{M}$ ), the converse is true: the map  $\exp_x(z)$  is always a retraction on the ambient, Euclidean space  $\mathbb{R}^m$ . In this way, the basic iteration (RRM) can be expressed as
$$
X _ {n + 1} = \exp_ {X _ {n}} \left(\gamma_ {n} \hat {v} _ {n}\right) = X _ {n} + \gamma_ {n} \hat {v} _ {n} + \Delta \left(X _ {n}; \gamma_ {n} \hat {v} _ {n}\right) \tag {16}
$$
leading to the fundamental question below:
What is the maximum offset  $\epsilon_{n} \coloneqq \Delta(X_{n}; \gamma_{n}\hat{v}_{n})$  that can be tolerated by a Euclidean stochastic approximation algorithm to avoid saddle points?
A key technical step in our work is to develop the means to control the offset term  $\epsilon_{n}$  under (RRM) under a sufficiently broad class of assumptions that includes Algorithms 1-5. Crucially, this step is made possible thanks to the very recent - and technical - stochastic approximation work of [33]. To help the reader navigate our proof strategy, we outline the main steps below, focusing for simplicity on the case of a single saddle point.
Step 1: From discrete to continuous time (and back). Let  $\hat{x}$  be a strict saddle point of  $f$ . By the stable manifold theorem [63], the set of all initializations such that the Riemannian gradient flow
$$
\dot {x} (t) = - \operatorname {g r a d} f (x (t)) \tag {RGF}
$$
converges to  $\hat{x}$  is of measure 0. Then, assuming for the moment that the geodesic offset error  $\epsilon_{n} = \Delta(X_{n};\gamma_{n}\hat{v}_{n})$  in (16) is sufficiently small, the iterates of (RRM) can be seen as a noisy, approximate Euler discretization of (RGF); as such, it is reasonable to expect that the induced trajectories of (RRM) will never converge to  $\hat{x}$ .
To make this intuition precise, our first step will be to show that the iterates of (RRM) comprise an asymptotic pseudotrajectory of (RGF) in the sense of Benaim [10], i.e., they asymptotically track the orbits of (RGF) with arbitrary precision over windows of arbitrary length. To formalize this, define the "effective time" variable  $\tau_{n} = \sum_{k=1}^{n-1} \gamma_{k}$  and the associated geodesic interpolation  $X(t)$  of  $X_{n}$  as
$$
X (t) = \exp_ {X _ {n}} \left(\left(t - \tau_ {n}\right) \hat {v} _ {n}\right) \quad \text {f o r a l l} t \in \left[ \tau_ {n}, \tau_ {n + 1}\right), n \geq 1 \tag {GI}
$$
so, by construction,  $(a)X(\tau_n) = X_n$  for all  $n$ ; and  $(b)$  each segment of  $X(t)$  is a geodesic. Then, letting  $\Phi \colon \mathbb{R}_+ \times \mathcal{M} \to \mathcal{M}$  denote the flow of (RGF) - i.e.,  $\Phi_h(x)$  is simply the position at time  $h \geq 0$  of the solution orbit of (RGF) that starts at  $x \in \mathcal{M}$  - we will say that  $X(t)$  is an APT of (RGF) if, for all  $T > 0$ , we have
$$
\lim  _ {t \rightarrow \infty} \sup  _ {0 \leq h \leq T} \operatorname {d i s t} (X (t + h), \Phi_ {h} (X (t))) = 0. \tag {APT}
$$
This requirement is non-trivial, and our first technical result is to guarantee precisely this:
Theorem 2. Suppose that Assumptions 2-4 hold. Then, with probability 1, the geodesic interpolation  $X(t)$  of the sequence of iterates  $X_{n}$ ,  $n = 1, 2, \ldots$ , generated by (RRM) is an APT of (RGF).
A version of Theorem 2 was very recently derived by [33] under a different set of assumptions: On the one hand, [33] imposes a much more restrictive step-size schedule for  $\gamma_{n}$  (square summability) but, on the other hand, it only posits that the noise increments  $U_{n}$  are bounded in  $L^2$  (as opposed to  $L^{\infty}$  in our case). Our proof relies on the same construction of the Picard iteration map as [33], but otherwise diverges significantly in the probabilistic analysis required to establish (APT).
Step 2: From Riemannian to Euclidean schemes (and back). Albeit crucial, the APT property is decidedly not enough to guarantee avoidance: after all, the constant orbit  $X(t) = \hat{x}$  for all  $t \geq 0$  is trivially an APT of (RGF) but, of course, it does not avoid  $\hat{x}$ . To proceed, we will need to exploit the precise update structure of (RRM) in conjunction with the stable manifold theorem applied to (RGF).
In the Euclidean case, this is achieved by means of an intricate Lyapunov function argument, originally due to [11]. Our second step is to devise a new geometric argument to reduce the analysis from an arbitrary intrinsic manifold to an isometrically embedded submanifold of  $\mathbb{R}^m$ . This step is carried out by a combination of the celebrated Nash embedding theorem and a (smooth) Tietze extension argument to rewrite (RRM) as a "corrected" Robbins-Monro scheme on  $\mathbb{R}^m$  that actually evolves on  $\mathcal{M}$ . This construction also requires a "perturbation analysis" to ensure that certain subtle topological issues do not arise when we invoke the stable manifold theorem; we present the details in Appendix B.
Step 3: Controlling the geodesic offset. As we briefly described in the beginning of the proof overview, this Euclidean reframing of (RRM) introduces an intrinsic offset error  $\epsilon_{n} = \Delta(X_{n}; \gamma_{n}\hat{v}_{n})$ , which is difficult to analyze in detail (the offset incurred by a retraction on  $\mathcal{M}$  is of similar order, so the exponential-retraction distinction is not important at this stage). Our crucial observation here is that, under our blanket assumptions,  $\epsilon_{n}$  is small relative to  $\gamma_{n}$  and, in particular,  $\epsilon_{n} = \mathcal{O}(\gamma_{n}^{2})$ . Thanks to this bound, we are able to leverage a series of stochastic bounds – originally developed by Pemantle [54] – to show that the probability that these terms will have an adverse effect on exiting the center manifold of  $\hat{x}$  is zero (this is also where Assumption 3 comes in). We formalize this in Appendix B; Theorem 1 then follows by putting everything together.
Figure 1: The geodesic offset  $\Delta (x;z)$
Remark. A concept similar to our geodesic offset  $\Delta(x;z)$  has been explored in the reverse direction by the very recent works [15, 23] whose goal was to study of avoidance of Euclidean subgradient methods as an inexact Riemannian gradient scheme. They further show that this inexact Riemannian gradient descent can avoid saddle points if uniform noise is injected. While their core idea bears some similarity to ours, it remains unclear how to apply the analysis in [15, 23] to handle general RRM schemes, such as retraction-based and natural policy gradient methods.
4.3. Applications. As an illustration of the generality of Theorem 1, we now instantiate it to the range of specific algorithms discussed in Section 3.2. Since all these algorithms are run with gradient input generated by (SFO), applying Theorem 1 would require mapping the requirements of Assumption 3 to the primitives of (SFO). A convenient way to achieve this is by means of the proposition below:
Proposition 1. Suppose that Algorithms 1-5 are run with a gradient oracle  $V(x; \theta) = v(x) + \mathrm{err}(x; \theta)$  such that
$$
\left\| \operatorname {e r r} (x; \theta) \right\| _ {x} \leq \sigma (x) \quad a n d \quad \mathbb {E} \left[ \left[ \left\langle \operatorname {e r r} (x; \theta), z \right\rangle_ {x} \right] _ {+} \right] \geq \zeta (x) \tag {17}
$$
for all  $z \in \mathcal{T}_x\mathcal{M}$ ,  $\|z\|_x = 1$ , and for suitable functions  $\zeta, \sigma: \mathcal{M} \to \mathbb{R}_+$  with  $\sigma$  bounded on bounded subsets of  $\mathcal{M}$  and  $\inf_x \zeta(x) > 0$ . Then, under Assumptions 2 and 4, the conclusion of Theorem 1 holds, that is, Algorithms 1-5 avoid strict saddle manifolds of  $f$ .
The proof of Proposition 1 is deferred to Appendix B; we only note here that its proof mainly hinges on verifying the bias requirement  $\|b_n\|_{X_n} = \mathcal{O}(\gamma_n)$  of (13) by means of  $(i)$  the boundedness of the error function  $\sigma(x)$  on bounded subsets of  $\mathcal{M}$ ; and  $(ii)$  controlling the maximal deviation between a retraction and the exponential map for input vectors bounded by  $\varrho$ .
5 Numerical Illustrations
In this section, we aim to demonstrate the practical applicability of the theoretical framework proposed in our paper by providing numerical illustrations. To do so, we utilize a 2-dimensional torus as the optimization landscape, where the complexity and multi-modal nature of the objective function can be easily visualized in Fig. 2.
Our objective function features three saddle points (in black) and one global minimizer (in red). We subject two RRM schemes, i.e., (RSGD) and (RSEG), to initialization in proximity to these saddle points. This strategic choice rigorously tests their ability to navigate and converge to the global optimum. In line with our theoretical predictions, both RRM methods avoid the saddle points and
(a) Riemannian stochastic gradient descent
(b) Riemannian stochastic extra-gradient
Figure 2: Stochastic RRM schemes on torus.
eventually reach the desirable global minimum. This empirical confirmation reinforces the central message of our paper: stochastic Riemannian schemes only converge to local minimizers.
In this paper, we addressed the question of when Riemannian stochastic algorithms can effectively evade saddle points, focusing on the broad category of Riemannian Robbins-Monro schemes. We introduced a novel framework for analyzing the avoidance of Riemannian saddle points within the RRM framework, which encompasses many commonly used Riemannian stochastic algorithms, including retraction-based algorithms. Our framework builds upon the notion of strict saddle points and provides a set of easily verifiable conditions that guarantee the avoidance of such traps.
Our work paves the way for several promising research directions in learning with Riemannian methods. One intriguing avenue for exploration is the investigation of whether Riemannian zeroth-order methods, such as the Riemannian extension of the work by Kiefer & Wolfowitz [36], can effectively evade strict saddle points. We believe that combining the insights from the asymptotic pseudotrajectory theory with Euclidean analysis can shed light on this question and provide valuable insights into the behavior of these methods in the Riemannian setting.
Furthermore, an interesting direction for future research is the extension of the avoidance of unstable limit cycles in Euclidean min-max optimization, as studied by Hsieh et al. [29], to the realm of Riemannian games. Investigating the avoidance of unstable limit cycles in this context has the potential to uncover novel phenomena specific to the manifold settings, leading to a deeper understanding on the intricate dynamics and strategies involved in Riemannian games.
PM is also with the Archimedes Research Unit – Athena RC – Department of Mathematics, National & Kapodistrian University of Athens. This work has been partially supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program grant agreement No 815943, and project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program. YPH acknowledges funding through an ETH Foundations of Data Science (ETH-FDS) postdoctoral fellowship. PM also acknowledges financial support by the French National Research Agency (ANR) in the framework of the “Investissements d'avvenir” program (ANR-15-IDEX-02), the LabEx PERSYVAL (ANR-11-LABX-0025-01), and MIAI@Grenoble Alpes (ANR-19-P3IA-0003)
[56] Popov, L. D. A modification of the Arrow-Hurwicz method for search of saddle points. Mathematical Notes of the Academy of Sciences of the USSR, 28(5):845-848, 1980.
[57] Rakhlin, A. and Sridharan, K. Online learning with predictable sequences. In COLT '13: Proceedings of the 26th Annual Conference on Learning Theory, 2013.
[58] Raskutti, G. and Mukherjee, S. The information geometry of mirror descent. IEEE Transactions on Information Theory, 61(3):1451-1457, 2015.
[59] Robbins, H. and Monro, S. A stochastic approximation method. Annals of Mathematical Statistics, 22: 400-407, 1951.
[60] Rockafellar, R. T. Convex Analysis. Princeton University Press, Princeton, NJ, 1970.
[61] Rockafellar, R. T. and Wets, R. J. B. Variational Analysis, volume 317 of A Series of Comprehensive Studies in Mathematics. Springer-Verlag, Berlin, 1998.
[62] Shapiro, A., Dentcheva, D., and Ruszczyński, A. Lectures on Stochastic Programming. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2009.
[63] Shub, M. Global Stability of Dynamical Systems. Springer-Verlag, Berlin, 1987.
[64] Sra, S. and Hosseini, R. Conic geometric optimization on the manifold of positive definite matrices. SIAM Journal on Optimization, 25(1):713-739, 2015.
[65] Sun, J., Qu, Q., and Wright, J. Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method. IEEE Transactions on Information Theory, 63(2):885-914, 2016.
[66] Sun, Y., Flammarion, N., and Fazel, M. Escaping from saddle points on riemannian manifolds. Advances in Neural Information Processing Systems, 32, 2019.
[67] Tang, G.-j. and Huang, N.-j. Korpelevich's method for variational inequality problems on Hadamard manifolds. Journal of Global Optimization, 54(3):493-509, 2012.
[68] Zhou, Z., Mertikopoulos, P., Bambos, N., Boyd, S. P., and Glynn, P. W. On the convergence of mirror descent beyond stochastic convex programming. SIAM Journal on Optimization, 30(1):687-716, 2020.
A Further examples of RRM schemes
In this section, we provide two additional algorithmic examples supplementing the range of Algorithms 1-5 to illustrate the applicability of our RRM template.
Algorithm 6 (Riemannian proximal point methods). The (deterministic) Riemannian proximal point method (RPPM) [25] is an implicit ("backward") update rule of the form
$$
\log_ {X _ {n + 1}} \left(X _ {n}\right) = - \gamma_ {n} v \left(X _ {n + 1}\right). \tag {RPPM}
$$
The RRM representation of (RPPM) is then obtained by taking  $b_{n} = \Gamma_{X_{n + 1}\to X_{n}}(v(X_{n + 1})) - v(X_{n})$  and  $U_{n} = 0$  in the decomposition (4) of the error term  $W_{n}$  of (RRM). If, in additional, the true gradient  $v(X_{n + 1})$  is replaced by an oracle  $V(X_{n + 1};\theta_{n + 1})$ , then (RPPM) becomes the stochastic version of RPPM by setting
$$
b _ {n} = \mathbb {E} \left[ \Gamma_ {X _ {n + 1} \rightarrow X _ {n}} \left(V \left(X _ {n + 1}; \theta_ {n + 1}\right)\right) - v \left(X _ {n}\right) \mid \mathcal {F} _ {n} \right]
$$
$$
U _ {n} = \Gamma_ {X _ {n + 1} \rightarrow X _ {n}} (V (X _ {n + 1}; \theta_ {n + 1})) - v (X _ {n}) - b _ {n}.
$$
For a detailed discussion, see [25] and references therein.