SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zihao Hu†, Guanghui Wang†, Jacob Abernethy†,
College of Computing, Georgia Institute of Technology†
Google Research*
{zihaohu, gwang369}@gatech.edu, abernethyj@google.com
The projection operation is a critical component in a wide range of optimization algorithms, such as online gradient descent (OGD), for enforcing constraints and achieving optimal regret bounds. However, it suffers from computational complexity limitations in high-dimensional settings or when dealing with ill-conditioned constraint sets. Projection-free algorithms address this issue by replacing the projection oracle with more efficient optimization subroutines. But to date, these methods have been developed primarily in the Euclidean setting, and while there has been growing interest in optimization on Riemannian manifolds, there has been essentially no work in trying to utilize projection-free tools here. An apparent issue is that non-trivial affine functions are generally non-convex in such domains. In this paper, we present methods for obtaining sub-linear regret guarantees in online geodesically convex optimization on curved spaces for two scenarios: when we have access to (a) a separation oracle or (b) a linear optimization oracle. For geodesically convex losses, and when a separation oracle is available, our algorithms achieve  $O(T^{1/2})$ ,  $O(T^{3/4})$  and  $O(T^{1/2})$  adaptive regret guarantees in the full information setting, the bandit setting with one-point feedback and the bandit setting with two-point feedback, respectively. When a linear optimization oracle is available, we obtain regret rates of  $O(T^{3/4})$  for geodesically convex losses and  $O(T^{2/3} \log T)$  for strongly geodesically convex losses.
Online convex optimization (OCO) offers a framework for modeling sequential decision-making problems (Hazan et al., 2016). The standard setting depicts the learning process as a zero-sum game between a learner and an adversary. At round  $t$ , the learner selects a decision  $\mathbf{x}_t$  from a convex set  $\mathcal{K}$  and observes the encountered convex loss function  $f_t$ . The learner's goal is to minimize regret, defined as
$$
\operatorname {R e g r e t} _ {T} := \sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x} _ {t}) - \min  _ {\mathbf {x} \in \mathcal {K}} \sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x}).
$$
In Euclidean space, OCO boasts a robust theoretical foundation and numerous real-world applications, such as online load balancing (Molinaro, 2017), optimal control (Li et al., 2019), revenue maximization (Lin et al., 2019), and portfolio management (Jézquel et al., 2022). The standard approach for OCO is online gradient descent (OGD), which performs
$$
\mathbf {x} _ {t + 1} = \Pi_ {\mathcal {K}} (\mathbf {x} _ {t} - \eta_ {t} \nabla f _ {t} (\mathbf {x} _ {t})),
$$
where  $\Pi_{\mathcal{K}}$  represents the orthogonal projection onto  $\mathcal{K}$ , ensuring the sequence  $\{\mathbf{x}_t\}_{t=1}^T$  remains feasible. However, the projection operation can be computationally expensive in high-dimensional or complex feasible sets. Projection-free online learning provides a reasonable way to handle this situation. At the heart of this approach is the understanding that, in many cases, the complexity of
Table 1: Summary of main results. Our approach allows us to invoke either a Separation Oracle (SO) or a Linear Optimization Oracle (LOO) for  $O(T)$  times throughout the  $T$  rounds. Notably, our results match those presented by Garber & Kretzu (2022).
Oracle	Losses	Feedback	Measure	Regret
SO	gsc-convex	full information	adaptive regret	\( O(T^{1/2}) \), Thm. 1
gsc-convex	bandit, one-point	expected adaptive regret	\( O(T^{3/4}) \), Thm. 2
gsc-convex	bandit, two-point	expected adaptive regret	\( O(T^{1/2}) \), Thm. 3
LOO	gsc-convex	full information	adaptive regret	\( O(T^{3/4}) \), Thm. 4
strongly gsc-convex	full information	regret	\( O(T^{2/3}\log T) \), Thm. 5
implementing an optimization oracle can be significantly lower than that of the orthogonal projection. As a result, both practical and theoretical interests lie in replacing the projection operation with these optimization oracles. The two most well-known projection-free optimization oracles are:
$$
\text{(Linear optimization oracle)} \underset {\mathbf{x}\in \mathcal{K}}{\operatorname {argmin}}\left\langle \mathbf{g},\mathbf{x}\right\rangle
$$
$$
(S e p a r a t i o n o r a c l e) \mathbf {g}: \langle \mathbf {y} - \mathbf {x}, \mathbf {g} \rangle > 0, \quad \forall \mathbf {x} \in \mathcal {K}.
$$
Whereas much attention has been given to the development of sub-linear regret guarantees using optimization oracles in Euclidean space (Hazan & Kale, 2012; Levy & Krause, 2019; Hazan & Minasyan, 2020; Wan et al., 2022; Garber & Kretzu, 2022; Mhammedi, 2022a; Wan et al., 2023), there is considerably less research on projection-free optimization on Riemannian manifolds. There are numerous scenarios where the space of feasible parameters is not a convex subset of Euclidean space, but instead has a manifold structure with a Riemannian metric; examples include non-negative PCA (Montanari & Richard, 2015),  $K$ -means clustering (Carson et al., 2017) and computing Wasserstein-Barycenters (Weber & Sra, 2022b). However, there has been significantly less research on efficiently computing projections or projection-free optimization in the Riemannian setting, even for highly-structured geodesically convex (gsc-convex) feasible sets like:
$$
\mathcal {K} = \{\mathbf {x} | \max  _ {1 \leq i \leq m} h _ {i} (\mathbf {x}) \}
$$
where each  $h_i(\mathbf{x})$  is gsc-convex. In this paper we focus on projection-free optimization on Riemannian manifolds in the online setting. A Riemannian version of Online Gradient Descent (R-OGD) was given by Wang et al. (2021):
$$
\mathbf {x} _ {t + 1} = \Pi_ {\mathcal {K}} \mathrm {E x p} _ {\mathbf {x} _ {t}} \left(- \eta_ {t} \nabla f _ {t} (\mathbf {x} _ {t})\right),
$$
where  $\mathcal{K}$  is now a gsc-convex set, and  $\Pi_{\mathcal{K}}$  is the metric projection onto  $\mathcal{K}$ . This method achieves sub-linear regret guarantees for Riemannian OCO. The metric projection is often the most expensive operation, and it is not always clear how to implement it on a manifold. In the present work, we propose the use of two alternative operations: a separation oracle (SO) or a linear optimization oracle (LOO) on the Riemannian manifold, with definitions deferred to Section 3. It is important to note that, in Euclidean space, both oracles rely on the definition of a hyperplane. In the realm of Riemannian manifolds, there are two natural extensions of a hyperplane: one is the sub-level set of a Busemann function, known as a horosphere (Bridson & Haefliger, 2013); the other relies on the inverse exponential map, as outlined in (2). While a horosphere is gsc-convex, the corresponding separation oracle exists if every boundary point of the feasible gsc-convex set has a locally supporting horosphere (Borisenko, 2002). The existence of a separation oracle is assured for any gsc-convex sets on Hadamard manifolds (Silva Louzeiro et al., 2022), if defined via the inverse exponential map. Hence, we adopt this latter definition. It is worth noting that this object is typically non-convex (Kristály et al., 2016), leading to geometric complexities that necessitate careful management. Also, our work builds upon the findings of Garber & Kretzu (2022) and inherits the adaptive regret guarantee for gsc-convex losses, as defined by Hazan & Seshadhri (2009):
$$
\text {A d a p t i v e} \operatorname {R e g r e t} _ {T} := \sup  _ {[ s, e ] \subseteq [ T ]} \left\{\sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {t}) - \min  _ {\mathbf {x} \in \mathcal {K}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x}) \right\}.
$$
Our main contributions are summarized in Table 1. More specifically,
- Given a separation oracle, we attain adaptive regret bounds of  $O(T^{1/2})$ ,  $O(T^{3/4})$  and  $O(T^{1/2})$  for gsc-convex losses in the full information setting, the bandit convex optimization setting with one-point feedback $^1$ , and the bandit convex optimization setting with two-point feedback, respectively.
- Assuming access to a linear optimization oracle, we provide algorithms that enjoy  $O(T^{3/4})$  adaptive regret for gsc-convex losses and  $O(T^{2/3} \log T)$  regret for strongly gsc-convex losses.
- We highlight some key differences between convex sets on a Hadamard manifold and in Euclidean space. In particular, shrinking a gsc-convex set towards an interior point does not preserve convexity, and the Minkowski functional on a Hadamard manifold is non-convex. These results, which may not be well-known within the machine learning community, could be of independent interest.
The technical challenges of this paper can be divided into two parts.
Firstly, for the separation oracle, we need to bound the "thickness" of part of the feasible set cut by the separating hyperplane. While in Euclidean space, this can be achieved through a convex combination argument (Garber & Kretzu, 2022), the task becomes challenging on manifolds due to the varying metric at different points and the non-convex nature of the separating hyperplane. Fortunately, this problem can be addressed using the Jacobi field comparison technique. Also, unlike in Euclidean space, one cannot directly construct a separation oracle for  $(1 - \delta)\mathcal{K}$  using a separation oracle for  $\mathcal{K}$ . This poses significant challenges in the bandit setting. Nonetheless, we have identified a novel solution to this issue in the two-point feedback setting.
Secondly, in Euclidean space, the linear optimization oracle is typically invoked by online Frank-Wolfe (OFW) (Hazan & Kale, 2012; Kretzu & Garber, 2021) to achieve no-regret online learning. The analysis of OFW relies on the fact that the Hessian of a linear function is zero everywhere. But on Hadamard manifolds, such functions' existence implies that the manifold has zero sectional curvature everywhere (Kristály et al., 2016). The algorithms in Garber & Kretzu (2022) do not require affinity and serve as a starting point for our results. However, the analysis still needs to be conducted carefully due to the non-convexity of the separating hyperplane on manifolds.
In this section, we briefly review previous work on projection-free online learning in Euclidean space as well as online and projection-free optimization on Riemannian manifolds.
2.1 Projection-free OCO in Euclidean Space
Linear Optimization Oracle. The pioneering work of Hazan & Kale (2012) first introduced an online variant of the Frank-Wolfe algorithm (OFW) and achieved  $O(T^{3/4})$  regret for convex functions. Hazan & Minasyan (2020) proposed a randomized algorithm that leverages smoothness to achieve  $O(T^{2/3})$  expected regret. The insightful analysis of Wan & Zhang (2021) and Kretzu & Garber (2021) demonstrated that OFW indeed attains  $O(T^{2/3})$  regret for strongly convex functions. Garber & Kretzu (2022) showed that it is possible to achieve  $O(T^{3/4})$  adaptive regret and  $O(T^{2/3} \log T)$  regret for convex and strongly convex functions, respectively. Mhammedi (2022b) illustrated how to obtain  $\tilde{O}(T^{2/3})$  regret for convex functions, where  $\tilde{O}(\cdot)$  hides logarithmic terms. Our results are in line with those of Garber & Kretzu (2022) and inherit the adaptive regret guarantee.
Separation Oracle and Membership Oracle. Levy & Krause (2019) demonstrated that it is possible to achieve  $O(\sqrt{T})$  and  $O(\log T)$  regret bounds for convex and strongly convex functions when the feasible set is a sublevel set of a smooth and convex function. Mhammedi (2022a) generalized the idea of Levy & Krause (2019) and showed how to obtain  $O(\sqrt{T})$  and  $O(\log T)$  regret guarantees for general convex sets. Garber & Kretzu (2022) provided algorithms that ensure  $O(\sqrt{T})$  and  $O(T^{3/4})$  adaptive regret guarantees for convex losses in the full information and bandit settings, respectively.
For the one-point feedback setting, we allow the point at which we play and the point where we receive feedback to differ, thereby bypassing a fundamental challenge posed by Riemannian geometry. In the case of two-point feedback, however, we eliminate this non-standard setting.
2.2 Online and Projection-free Optimization on Manifolds
Online Optimization on Manifolds. Bécigneul & Ganea (2019) demonstrated that a series of adaptive optimization algorithms can be implemented on a product of Riemannian manifolds, with each factor manifold being assigned a learning rate. Antonakopoulos et al. (2020) proposed using Follow the Regularized Leader with a strongly gsc-convex regularizer to achieve  $O(\sqrt{T})$  regret when the loss satisfies Riemannian Lipschitzness. Wang et al. (2021) introduced Riemannian OGD (R-OGD) and showed regret guarantees in full information and bandit convex optimization settings. Hu et al. (2023) considered achieving optimistic and dynamic regret on Riemannian manifolds.
Projection-free Optimization on Manifolds. Rusciano (2018) provided a non-constructive cutting hyperplane method on Hadamard manifolds. By comparison, our algorithms are constructive and deterministic. Weber & Sra (2022b) proposed Riemannian Frank-Wolfe (RFW) for gsc-convex optimization and showed some practical applications on the manifold of SPD matrices. In a subsequent work, Weber & Sra (2022a) generalized RFW to the stochastic and non-convex setting. We use RFW as a subroutine to invoke the linear optimization oracle and establish sub-linear regret guarantees. Hirai et al. (2023) implemented the interior point method on Riemannian manifolds and used a self-concordant barrier to enforce the constraint.
3 Preliminaries and Assumptions
In this section, we lay the groundwork for our study by presenting an overview of Riemannian manifolds, the separation oracle and the linear optimization oracle within these spaces. Additionally, we establish key definitions and assumptions that will be integral to the following sections.
Riemannian Manifolds. We provide key notations in Riemannian geometry that will be employed throughout this paper. Readers looking for a more comprehensive treatment are encouraged to consult Petersen (2006); Lee (2018). Our proof also relies on the concept of the Jacobi field, and we provide some backgrounds in Appendix E.1. We consider an  $n$ -dimensional smooth manifold  $\mathcal{M}$  equipped with a Riemannian metric  $g$ . This metric confers a point-wise inner product  $\langle \mathbf{u}, \mathbf{v} \rangle_{\mathbf{x}}$  at every point  $\mathbf{x} \in \mathcal{M}$ , where  $\mathbf{u}$ ,  $\mathbf{v}$  are vectors in the tangent space  $T_{\mathbf{x}}\mathcal{M}$  at  $\mathbf{x}$ . This tangent space, a vector space of dimension  $n$ , encompasses all vectors tangent to  $\mathbf{x}$ . The Riemannian metric also determines the norm of a tangent vector  $\mathbf{u}$  as:  $\| \mathbf{u} \| := \sqrt{\langle \mathbf{u}, \mathbf{u} \rangle}$ . A geodesic  $\gamma(t): [0, c] \to \mathcal{M}$  is a piecewise smooth curve with a constant velocity that locally minimizes the distance between its endpoints, say,  $\mathbf{x}$  and  $\mathbf{y}$ . The Riemannian distance between these two points is given by  $d(\mathbf{x}, \mathbf{y}) := \int_0^c \| \dot{\gamma}(t) \| dt = \int_0^c \| \dot{\gamma}(0) \| dt$ . It's important to note that the Riemannian distance remains invariant under reparameterizations of  $\gamma(t)$ . Consider a geodesic  $\gamma(t): [0, 1] \to \mathcal{M}$  with  $\gamma(0) = \mathbf{x}$ ,  $\gamma(1) = \mathbf{y}$  and  $\dot{\gamma}(0) = \mathbf{v}$ . The exponential map  $\mathrm{Exp}_{\mathbf{x}}(\mathbf{v})$  transforms  $\mathbf{v} \in T_{\mathbf{x}}\mathcal{M}$  to  $\mathbf{y} \in \mathcal{M}$ , and the inverse exponential map  $\mathrm{Exp}_{\mathbf{x}}^{-1}\mathbf{y}$  performs the inverse operation, mapping  $\mathbf{y} \in \mathcal{M}$  to  $\mathbf{v} \in T_{\mathbf{x}}\mathcal{M}$ . The inverse exponential map also offers a handy way to express the Riemannian distance:  $d(\mathbf{x}, \mathbf{y}) = \| \mathrm{Exp}_{\mathbf{x}}^{-1}\mathbf{y} \|$ .
The sectional curvature at a point  $\mathbf{x} \in \mathcal{M}$  is contingent on two-dimensional subspaces of  $T_{\mathbf{x}}\mathcal{M}$ , and describes the curvature near  $\mathbf{x}$ . Generally, geodesics diverge on manifolds with negative sectional curvature, converge on manifolds with positive sectional curvature, and manifolds with zero sectional curvature are locally isometric to Euclidean space. In line with Zhang & Sra (2016); Wang et al. (2021), we primarily explore Hadamard manifolds, which are simply connected manifolds with non-positive curvature that admit a unique global minimizing geodesic between any pair of points. A set  $\mathcal{K} \subseteq \mathcal{M}$  is geodesically convex (gsc-convex) if it includes the geodesic connecting  $\mathbf{x}$  and  $\mathbf{y}$  for any  $\mathbf{x}, \mathbf{y} \in \mathcal{K}$ . A  $\mu$ -strongly gsc-convex (or gsc-convex, when  $\mu = 0$ ) function  $f: \mathcal{M} \to \mathbb{R}$  fulfills
$$
f (\mathbf {y}) \geq f (\mathbf {x}) + \left\langle \nabla f (\mathbf {x}), \operatorname {E x p} _ {\mathbf {x}} ^ {- 1} \mathbf {y} \right\rangle + \frac {\mu}{2} d (\mathbf {x}, \mathbf {y}) ^ {2}, \tag {1}
$$
for any  $\mathbf{x},\mathbf{y}\in \mathcal{M}$ , where  $\nabla f(\mathbf{x})\in T_{\mathbf{x}}\mathcal{M}$  is the Riemannian gradient.
Optimization Oracles on Riemannian Manifolds. In this part, we introduce the concept of a separation oracle and a linear optimization oracle on Riemannian manifolds.
A separation oracle, given a point  $\mathbf{y}$  not in the gsc-convex set  $\kappa$ , returns a non-convex separating hyperplane that satisfies the following condition:
$$
\left\langle - \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \mathbf {x}, \mathbf {g} \right\rangle > 0, \quad \forall \mathbf {x} \in \mathcal {K}, \tag {2}
$$
where  $\mathbf{g} \in T_{\mathbf{y}}\mathcal{M}$ . Even with the non-convexity of the separating hyperplane, for certain gsc-convex sets like  $\mathcal{K} = \{\mathbf{x}|\max_{1 \leq i \leq m}h_i(\mathbf{x}) \leq 0\}$  where each  $h_i(\mathbf{x})$  is gsc-convex, a separation oracle can be efficiently implemented by Lemma 18 and Remark 4.2
On the other hand, a linear optimization oracle is responsible for solving the following problem:
$$
\operatorname *{argmin}_{\mathbf{x}\in \mathcal{K}}\left\langle \mathbf{g},\operatorname{Exp}^{-1}_{\mathbf{x}_{0}}\mathbf{x}\right\rangle
$$
on a gsc-convex set  $\mathcal{K}$ , where  $\mathbf{x}_0 \in \mathcal{K}$  and  $\mathbf{g} \in T_{\mathbf{x}_0}\mathcal{M}$ . Although this objective is not gsc-convex, it can still be solved in closed form for certain problems. Examples include computing the geometric mean and the Bures-Wasserstein barycenter on the manifold of SPD matrices (Weber & Sra, 2022b).
In this paper, we rely on a series of definitions and assumptions, which we introduce here for clarity and reference in subsequent sections.
Assumption 1. The manifold  $\mathcal{M}$  is Hadamard with sectional curvature bounded below by  $\kappa$ , so the sectional curvature of  $\mathcal{M}$  lies in the interval  $[\kappa, 0]$ .
Assumption 2. The manifold  $\mathcal{M}$  is a homogeneous Hadamard manifold with sectional curvature bounded below by  $\kappa$ . For every  $t \in [T]$ , the inequality  $|f_t(\mathbf{x})| \leq M$  holds. For the bandit setting with two-point feedback, we additionally require  $\mathcal{M}$  to be symmetric.
Assumption 3. The set  $\mathcal{K} \subseteq \mathcal{M}$  is a gsc-convex decision set and satisfies  $\mathbb{B}_{\mathbf{p}}(r) \subseteq \mathcal{K} \subseteq \mathbb{B}_{\mathbf{p}}(R)$ . Here,  $\mathbb{B}_{\mathbf{p}}(r)$  represents the geodesic ball centered at  $\mathbf{p} \in \mathcal{K}$  with radius  $r$ .
Assumption 4. For every  $t \in [T]$  and  $\mathbf{x} \in \mathbb{B}_{\mathbf{p}}(R)$ , the function  $f_t(\mathbf{x})$  is gsc-convex (or strongly gsc-convex), and the norm of its gradient is bounded by  $G$ , i.e.,  $\| \nabla f_t(\mathbf{x}) \| \leq G$ .
Let us make two important comments. First, the homogeneity and the symmetry of  $\mathcal{M}$  allows us to employ the unbiased estimator presented in Wang et al. (2023) for the bandit setting. It should be noted that homogeneous and symmetric Hadamard manifolds include two of the most commonly used ones: the hyperbolic space and the manifold of SPD matrices. Second, the projection onto a geodesic ball, denoted as  $\Pi_{\mathbb{B}_{\mathfrak{p}}(R)}(\cdot)$ , is considered projection-free as it can be computed to an  $\epsilon$  precision using  $\log(1/\epsilon)$  bisections<sup>3</sup>.
Definition 1. We define a geometric constant  $\zeta$  as  $\zeta \coloneqq 2R\sqrt{-\kappa}\coth (2R\sqrt{-\kappa})$
Definition 2. Fixing  $\mathbf{p} \in \mathcal{K}$ , for any  $c \in (0, \infty)$ , we define  $c\mathcal{K} = \{\mathrm{Exp}_{\mathbf{p}}(c\mathrm{Exp}_{\mathbf{p}}^{-1}\mathbf{x})|\mathbf{x} \in \mathcal{K}\}$ .
Definition 3. We call  $\tilde{\mathbf{y}}\in \mathcal{M}$  an infeasible projection of  $\mathbf{y}\in \mathcal{M}$  onto a simply connected closed set  $\tilde{\mathcal{K}}\subseteq \mathcal{M}$  if for every point  $\mathbf{z}\in \tilde{\mathcal{K}}$  , the inequality  $d(\tilde{\mathbf{y}},\mathbf{z})\leq d(\mathbf{y},\mathbf{z})$  holds. We define  $\mathcal{O}_{IP}(\tilde{\mathcal{K}},\mathbf{y})$  as an infeasible oracle for  $\tilde{\kappa}$  which, given any  $\mathbf{y}$  , returns an infeasible projection of  $\mathbf{y}$  onto  $\tilde{\kappa}$
We note that, in Euclidean space where the sectional curvature is zero everywhere, we have  $\zeta = \lim_{\kappa \to 0}2R\sqrt{-\kappa}\coth (2R\sqrt{-\kappa}) = 1$ . We also observe that the definition of the infeasible projection in Garber & Kretzu (2022) requires  $\tilde{\mathcal{K}}$  to be convex, which is indeed unnecessary. This distinction is essential because, in the case of the separation-oracle-based OCO, we construct an infeasible projection oracle onto  $\tilde{\mathcal{K}} = (1 - \delta)\mathcal{K}$ , which may be non-convex (Theorem 6).
4 Warm-up: the High-level Idea
We briefly illustrate the overarching strategy of achieving regret guarantees. Our basic algorithm is Algorithm 1, which generates a sequence  $\{\mathbf{y}_t\}_{t=1}^T$  by R-OGD that does not necessarily fall within the feasible set. A key insight is that we can build an infeasible projection oracle using either a separation oracle or a linear optimization oracle, resulting in a sequence  $\{\hat{\mathbf{y}}_t\}_{t=1}^T$  that exhibits a desirable regret guarantee (as shown in Lemma 1). The design of the infeasible projection oracle rests on a straightforward fact: whenever  $\mathbf{y}_t$  deviates significantly from  $\mathcal{K}$ , we can call upon either oracle to produce a descent direction and then apply Lemma 2 to gauge the progress. Additional error terms, arising from the fact that  $\tilde{\mathbf{y}}_t$  may not necessarily lie in  $\mathcal{K}$ , can be quantified by leveraging the boundedness of the gradient in Assumption 4.
This fact is well-known in Euclidean space (Grötschel et al., 2012), but it appears we are the first to observe its counterpart on manifolds.
<sup>3</sup>A comprehensive explanation of this observation can be found in Remark 5.
Algorithm 1: Infeasible Riemannian OGD
Data: horizon  $T$  , feasible set  $\tilde{\mathcal{K}}$  , step-sizes  $\{\eta_t\}_{t = 1}^T$  , infeasible projection oracle  $\mathcal{O}_{IP}(\tilde{\mathcal{K}},\cdot)$  for  $t = 1,\dots ,T$  do Play  $\tilde{\mathbf{y}}_t$  , and observe  $f_{t}(\tilde{\mathbf{y}}_{t})$  Update  $\mathbf{y}_{t + 1} = \mathrm{Exp}_{\tilde{\mathbf{y}}_t}(-\eta_t\nabla f_t(\tilde{\mathbf{y}}_t))$  , and set  $\tilde{\mathbf{y}}_{t + 1} = \mathcal{O}_{IP}(\tilde{\mathcal{K}},\mathbf{y}_{t + 1})$    
end
We have the following guarantee for Algorithm 1.