No-regret Online Learning over Riemannian Manifolds

1 Introduction . The online optimization has been widely studied in the past decades in online routing , spam filtering , and machine learning [ 4 , 23 , 8 ] . Without a prior knowledge of loss functions , an online convex optimization algorithm predicts solutions before the loss function is revealed . In this paper , we consider the following Riemannian online convex optimization ( R-OCO ) problem , min xt∈K⊂M ft ( xt ) , t = 1 , 2 , . . . , T , ( 1 ) where M is a complete Riemannian manifold equipped with a Riemannian metric g and K is a geodesically convex ( g-convex ) subset ofM . Here , { ft } t=1,2 , ... , T is a sequence of unknown loss functions and every ft is a geodesically convex ( g-convex ) function with sufficient smoothness . The R-OCO problem ( 1 ) extends the online convex optimization in Euclidean spaces with potential applications in machine learning , such as online principal component analysis ( PCA ) , dictionary learning , and neural networks [ 28 , 19 , 25 ] . ∗Correspondence author ( Y. Hong , +86-10-82541888 ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . The R-OCO problem ( 1 ) can be understood as a learning process of T rounds . At each round t = 1 , 2 , 3 , . . . , T , an online learner chooses a strategy xt from the g-convex subset K. Later or simultaneously , the adversary ( or nature ) produces a g-convex loss function ft : K → R of which the learner has no prior knowledge . Finally , the learner receives the feedback and suffers the loss ft ( xt ) . Generally , there are two types of information feedback . One is the full information feedback , where the entire function ft is revealed to the learner ; the other is the bandit feedback , where only the value ft ( xt ) is revealed . The goal of the R-OCO is to minimize the regret , defined as R ( T ) = T∑ t=1 ft ( xt ) −min x∈K T∑ t=1 ft ( x ) , which measures the difference between the cost by { xt } t=1 , ... , T and the best-fixed point chosen in hindsight . An algorithm is called no-regret [ 32 ] , if the regret of the algorithm goes sublinearly with the time horizon T . For carrying out optimization on a manifold , some classical methods treat the manifold as a subset of an ambient Euclidean space and employ Euclidean constrained optimization techniques . For instance , [ 30 ] presented an algorithm for the online PCA problem , where the variables were updated in an embedding Euclidean space and then projected onto a manifold . However , in practical applications , the dimension of an embedding Euclidean space can be too high ( e.g. , the Grassmann manifold [ 13 ] ) , and the projection can be expensive to compute ( e.g. , the manifold of symmetric positive definite ( SPD ) matrices [ 37 ] ) . An alternative approach termed Riemannian optimization makes use of intrinsic geometry of manifolds so that Riemannian optimization can optimize directly on the manifold as an unconstrained problem , and thus avoiding high dimension embedding and high computing cost for the projection . Furthermore , this viewpoint has shown benefits from the g-convexity , by which a nonconvex optimization problem can be converted into a g-convex one [ 6 ] . Consequently , it is important to take a Riemannian approach in our problem ( 1 ) . Although there were many existing algorithms for offline manifold optimization problems [ 2 , 31 , 5 ] , very few results were obtained about the Riemannian online optimization problem . [ 35 ] proposed an online algorithm for estimating hidden Markov chains on Hadamard homogeneous spaces and [ 9 ] analyzed Riemannian adaptive methods on products in the regret sense . More recently , [ 29 ] studied a zeroth-order online optimization problem on Hadamard manifolds with a sublinear assumption . Contribution This paper aims to design no-regret algorithms for the R-OCO problem in both full information feedback and bandit feedback . The contribution of this paper is summarized as follows : • We propose a Riemannian online gradient descent algorithm ( R-OGD ) for the R-OCO problem in the full information feedback , and then establish upper regret bounds of the R-OGD algorithm for g-convex and strongly g-convex functions . • We introduce a Riemannian bandit algorithm ( R-BAN ) for the R-OCO problem in the bandit feedback and then establish an upper regret bound for g-convex functions . Moreover , we develop a key technique to analyze the derivative of a local integration on homogeneous manifolds , which can be applied to estimate gradients in Riemannian optimization and beyond . • We focus on the worst-case regret and present a universal lower regret bound of R-OCO algorithms with g-convex losses on Hadamard manifolds , which matches the upper bound achieved by the R-OGD algorithm for g-convex functions . The established lower and upper bounds on the achievable bounds of R-OCO match their counterparts for Euclidean online convex optimization e.g. , [ 38 , 24 , 20 , 1 ] . We briefly list our results in Table 1 . Related Work The Euclidean online convex optimization was introduced in [ 38 ] . Inspired by the gradient descent method , [ 38 ] proposed the online gradient descent algorithm ( OGD ) of which the upper regret bound was proven to be O ( √ T ) . Then [ 24 ] proceeded with the study of the OGD algorithm and established an upper regret boundO ( log T ) for strongly convex functions . In addition , [ 1 ] gave a universal lower bound of O ( √ T ) for online algorithms , which indicated that the bounds in [ 38 ] and [ 24 ] are essentially optimal . In the bandit setting , [ 20 ] provided a detailed exposition of a one-point bandit algorithm . By modifying the gradient in the OGD algorithm to a randomized estimator , the upper regret bound attained O ( T 3 4 ) and O ( T 2 3 ) for convex loss functions and strongly convex loss functions , respectively . By extending the one-point bandit algorithm , [ 3 ] developed a multi-point bandit algorithm and presented upper regret bounds O ( √ T ) and O ( log T ) for convex and strongly convex loss functions . The Riemannian online algorithms proposed in this paper in the full information feedback and the one-point bandit feedback settings are extensions of the algorithms in ( [ 38 , 24 , 20 , 1 ] ) to Riemannian manifolds . Riemannian optimization has drawn much research attention in the past decades . Many basic algorithms in Euclidean spaces such as the gradient descent method , Newton ’ s method , and trustregion methods have been adapted into a Riemannian setting [ 2 , 5 , 31 ] . Some research of Riemannian stochastic optimization ( R-SO ) was intended to deal with time-varying optimization problems [ 12 , 36 , 37 , 35 ] . Among them , [ 36 ] provided the first global complexity analysis for the R-SGD algorithm on geodesically convex problems over Hadamard manifolds , and [ 35 ] proposed an online algorithm to deal with hidden Markov chains on Hadamard homogeneous spaces . When performed in batch , R-SO methods are to minimize the average regret in the case of knowing the prior distribution of the loss functions . In these sense , the R-SO can be viewed as a kind of R-OCO problems and R-OCO algorithms can handle settings without prior knowledge . The results about the R-OCO problem are quite limited . [ 7 ] proposed regularized online optimization methods via a Riemann–Lipschitz continuity condition , which focused on convex functions from an ambient Euclidean space . In the full information setting , [ 9 ] constructed regret upper bounds of Riemannian adaptive methods for g-convex functions , which required a product manifold structure . When the form of loss function was not available , [ 29 ] proposed a zeroth-order online algorithm on Hadamard manifolds for the tracking error in asymptotic sense as well as regret bounds by assuming the sublinearity of the term VT , which is a summation of distance between the minimizer of ft and ft+1 . In contrast , the regret bounds established for our online gradient-based/bandit Riemannian optimization algorithms are sublinear for any time , matching those for Euclidean online optimization . 2 Preliminaries . Riemannian manifolds A manifoldM is a topological space locally diffeomorphic to Euclidean spaces . The tangent space TxM is a linearization of manifoldM at point x . A vector field X is a map assigning every point x ∈ M with a tangent vector X ( x ) ∈ TxM , which can be also viewed as differential operators over smooth functions onM , i.e. , the operation X ( f ) defines a function X ( f ) ( x ) = limt→0 1 t ( f ( ξ ( t ) ) − f ( x ) ) onM , where ξ is a curve that starts at x with tangent vector X ( x ) . A Riemannian manifold is a smooth manifoldM equipped with a metric tensor g ( or called Riemannian metric ) , which defines an inner product 〈· , ·〉x in every tangent space TxM of x ∈ M. The Riemannian metric g brings a distance structure onM . A curve is a geodesic if it locally minimizes the length , which is an analog of a straight line in Euclidean spaces . On Riemannian manifolds , a geodesic is uniquely determined by the start point and initial tangent vector . In this way , the exponential map expx : TxM→M is defined by mapping a vector X ∈ TpM to γ ( 1 ) ∈ M for the geodesic γ such that γ ( 0 ) = x and γ̇ ( 0 ) = X . Curvature reflects the geometry of manifolds . We focus on sectional curvature , which is the Gauss curvature of a two-dimensional submanifold . Following [ 36 ] , we consider the Hadamard manifold , which is a simply connected and complete manifold with nonpositive sectional curvature . The Cartan-Hadamard theorem [ 11 ] shows that the exponential map expx ( · ) is a diffeomorphism from the tangent space TxM∼= Rn to the manifoldM . Therefore , the exponential map has an global inverse exp−1x ( · ) on Hadamard manifolds , and the distance d ( x , y ) can be expressed as ‖ exp−1x ( y ) ‖x . Isometries of Riemannian manifolds have been widely studied in differential geometry [ 11 , 10 ] . An isometry φ : M→M is a diffeomorphism preserving distance , i.e. , d ( x , y ) = d ( φ ( x ) , φ ( y ) ) for all x , y ∈ M. It is remarked that all isometries of a Riemannian manifold form a Lie group G. A Riemannian manifold is a homogeneous manifold if the group of isometries G acts onM transitively , i.e. , for any points x , y ∈ M there exists an isometry such that φ ( x ) = y . Some properties of homogeneous manifolds are used in our work , for example , the Killing field that represents the infinitesimal symmetry of isometries . We leave those properties in the appendix , due to the space limitation . Function Classes A set K is called geodesically convex ( g-convex ) if , for any points x , y ∈ K , there admits a geodesic γ ⊂ K connecting x and y . A function f : K → R is called geodesically convex ( or g-convex ) if for any geodesic γ : [ 0 , 1 ] →M , f ( γ ( t ) ) ≤ ( 1− t ) f ( γ ( 0 ) ) + tf ( γ ( 1 ) ) , ∀t ∈ [ 0 , 1 ] . The g-convexity has some equivalent conditions . When f is differentiable , which means that there exists a gradient ∇f ( x ) such that 〈∇f ( x ) , X〉 = X ( f ) ( x ) for any vector field X , the g-convexity is equivalent to f ( y ) ≥ f ( x ) + 〈∇f ( x ) , exp−1x ( y ) 〉 , ∀x , y ∈M . Furthermore , if f is twice differentiable , the g-convexity is equivalent to Hess ( f ) ( x ) ( X , X ) = X ( X ( f ( x ) ) − ( ∇XX ) f ( x ) ≥ 0 , ∀x ∈M , ∀X ∈ TxM , where ∇ is the Levi-Civita connection ofM , which is an analog of the differential of vector fields in Euclidean spaces ( see [ 17 ] ) . A differentiable function f : M→ R is geodesically µ-strongly convex ( or µ-strongly g-convex ) if there exists a constant µ > 0 such that for any x , y ∈M , f ( y ) ≥ f ( x ) + 〈∇f ( x ) , exp−1x ( y ) 〉+ µ 2 d2 ( x , y ) . We term a function to be geodesically L-Lipschitz ( or g-L-Lipschitz ) if there exists a constant L > 0 such that , |f ( y ) − f ( x ) | ≤ L · d ( x , y ) , ∀x , y ∈M , When f is differentiable , the g-L-Lipschitzness is equivalent to ‖∇f ( x ) ‖ ≤ L , ∀x ∈M .

Summary: The paper considers the problem of online optimization over manifolds of negative curvature (Hadamard) in the full-information and bandit setting, providing upper bounds for the regret which match the Euclidean case. Prior work has considered the same problem in a zero-order oracle regime and provided only asymptotic results. The theoretical analysis is an adaptation of [40] in the convex case and [18] in the strongly-convex case, using the law of cosines for geodesic triangles on Hadamard manifolds developed in [37] and new geometric results based on the extra assumption of homogeneity. Experimental results in the simple problem of Karcher mean estimation are also included.