SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 1. (Proof in Appendix A.1) Assume  $\tilde{\mathbf{y}}_t\in \mathbb{B}_{\mathbf{p}}(R)$  and let  $\nabla_{t} = \nabla f_{t}(\tilde{\mathbf{y}}_{t})$  and  $\kappa \subseteq \mathbb{B}_{\mathbf{p}}(R)$  be a gsc-convex subset of  $\mathcal{M}$ . Consider  $\tilde{\mathcal{K}}\subseteq \mathcal{K}$  as a simply connected and compact set, and  $\mathcal{O}_{IP}(\tilde{\mathcal{K}},\cdot)$  be an infeasible projection oracle as in Definition 3.
1) Suppose all losses are gsc-convex on  $\mathbb{B}_{\mathbf{p}}(R)$ . Fix some  $\eta > 0$  and let  $\eta_t = \eta$  for all  $t \geq 1$ . Algorithm 1 guarantees that the adaptive regret is upper-bounded by:
$$
\forall I = [ s, e ] \subseteq [ T ]: \sum_ {t = s} ^ {e} f _ {t} (\tilde {\mathbf {y}} _ {t}) - \min _ {\mathbf {x} _ {I} \in \tilde {\mathcal {K}}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {I}) \leq \frac {d (\tilde {\mathbf {y}} _ {s} , \mathbf {x} _ {I}) ^ {2}}{2 \eta} + \frac {\eta \zeta \sum_ {t = s} ^ {e} \| \nabla_ {t} \| ^ {2}}{2}.
$$
2) Suppose all losses are  $\alpha$ -strongly gsc-convex on  $\mathbb{B}_{\mathbf{p}}(R)$  for some  $\alpha > 0$ . Let  $\eta_t = \frac{1}{\alpha t}$  for all  $t \geq 1$ . Algorithm 1 guarantees that the static regret is upper bounded by:
$$
\sum_ {t = 1} ^ {T} f _ {t} \left(\tilde {\mathbf {y}} _ {t}\right) - \min  _ {\mathbf {x} \in \mathcal {K}} \sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x}) \leq \sum_ {t = 1} ^ {T} \frac {\zeta \| \nabla_ {t} \| ^ {2}}{2 \alpha t}.
$$
Remark 1. To apply Lemma 1, we need to ensure that  $\tilde{\mathbf{y}}_t\in \mathbb{B}_{\mathbf{p}}(R)$  for any  $t\in [T]$ . In the case of a separation oracle, we have  $\tilde{\mathcal{K}} = (1 - \delta)\mathcal{K}$  for some  $\delta \in (0,1)$ , and  $\tilde{\mathbf{y}}_t\in \mathcal{K}\subseteq \mathbb{B}_{\mathbf{p}}(R)$  by Lemma 4. With a linear optimization oracle, we have  $\tilde{\mathcal{K}} = \mathcal{K}$ , and  $\tilde{\mathbf{y}}_t\in \mathbb{B}_{\mathbf{p}}(R)$  is guaranteed by Lemma 6.
Lemma 2. (Proof in Appendix A.2) Consider  $\tilde{\mathcal{K}}\subseteq \mathcal{K}\subseteq \mathbb{B}_{\mathbf{p}}(R)$  as a simply connected and compact subset of  $\mathcal{M}$ . If  $\mathbf{y}\notin \tilde{\mathcal{K}}$  and  $\mathbf{g}\in T_{\mathbf{y}}\mathcal{M}$  satisfies  $-\left\langle \mathrm{Exp}_{\mathbf{y}}^{-1}\mathbf{z},\mathbf{g}\right\rangle \geq Q$ , where  $Q > 0$ , then consider  $\tilde{\mathbf{y}} = \mathrm{Exp}_{\mathbf{y}}(-\gamma \mathbf{g})$ . For  $\gamma = \frac{Q}{\zeta C^2}$  and  $\| \mathbf{g}\| \leq C$ , assume  $d(\mathbf{y},\mathbf{z})\leq 2R$ , then we have
$$
d (\tilde {\mathbf {y}}, \mathbf {z}) ^ {2} \leq d (\mathbf {y}, \mathbf {z}) ^ {2} - \frac {Q ^ {2}}{\zeta C ^ {2}}.
$$
Unlike Garber & Kretzu (2022), in Lemma 2, we also do not require  $\tilde{\mathcal{K}}$  to be gsc-convex.
5 Riemannian OCO with a Separation Oracle
In this section, we show how to use a separation oracle to construct an infeasible projection oracle and achieve sublinear regret guarantees. We note that we rely on an infeasible projection oracle onto  $(1 - \delta)\mathcal{K}$  rather than directly onto  $\mathcal{K}$ . While we have a separation oracle that results in  $\langle -\mathrm{Exp}_{\mathbf{y}}^{-1}\mathbf{x},\mathbf{g}\rangle >0$ , using Lemma 2 on this separating hyperplane may lead to minuscule progress, given that  $Q$  can be arbitrarily small. Consequently, achieving sublinear regret with only  $O(T)$  oracle calls becomes unfeasible. In contrast, constructing an infeasible projection onto  $(1 - \delta)\mathcal{K}$  always ensures meaningful progress, as quantified by Lemmas 3 and 4.
Algorithm 2: Infeasible Projection onto  $(1 - \delta)\mathcal{K}$  with a Riemannian Separation Oracle
Data: feasible gsc-convex set  $\kappa$  radius  $r$  , squeeze parameter  $\delta$  , initial point  $\mathbf{y}_0$ $\mathbf{y}_1 = \Pi_{\mathbb{B}_{\mathbf{p}}(R)}\mathbf{y}_0$    
for  $i = 1,\ldots$  do if  $\mathbf{y}_i\notin \mathcal{K}$  then SOK returns  $\mathbf{g}_i$  satisfying  $\langle -\mathrm{Exp}_{\mathbf{y}_i}^{-1}\mathbf{x},\mathbf{g}_i\rangle >0$ $\mathbf{y}_{i + 1} = \mathrm{Exp}_{\mathbf{y}_i}(-\gamma_i\mathbf{g}_i)$  where  $\gamma_{i} = \frac{\delta\bar{r}}{\|\mathbf{g}\|}$  and  $\bar{r}$  is defined in Equation (3) else return  ${\bf y} = {\bf y}_i$  end
Lemma 3. (Proof in Appendix B.1) Let  $\mathbf{y} \in \mathbb{B}_{\mathbf{p}}(R) \setminus \mathcal{K}$  and let  $\mathbf{g} \in T_{\mathbf{y}}\mathcal{M}$  be the output of the separation oracle for  $\mathbf{y}$ . Then, under Assumptions 1 and 3, we have that  $\langle -\mathrm{Exp}_{\mathbf{y}}^{-1}\mathbf{z}, \mathbf{g} \rangle > \delta \bar{r} \| \mathbf{g} \|$  for any  $\mathbf{z} \in (1 - \delta)\mathcal{K}$ , where
$$
\bar {r} := \frac {\sqrt {- \kappa} (2 R + r)}{\sinh (\sqrt {- \kappa} (2 R + r))} \cdot \frac {\sqrt {- \kappa} (R + r)}{\sinh (\sqrt {- \kappa} (R + r))} \cdot r. \tag {3}
$$
In Euclidean space, we can establish that  $\langle \mathbf{y} - \mathbf{z},\mathbf{g}\rangle >\delta r\| \mathbf{g}\|$  (Garber & Kretzu, 2022, Lemma 11). However, as indicated in Lemma 3, the result on manifolds is significantly worse with respect to  $R$ , given the exponential nature of  $\sinh$ . It is an interesting line of inquiry to explore whether this dependence is unavoidable.
Based on Lemma 3, to implement an infeasible projection oracle onto  $(1 - \delta)\mathcal{K}$ , the number of calls to the separation oracle is bounded in Lemma 4.
Lemma 4. (Proof in Appendix B.2) Under Assumptions 1 and 3. Let  $0 < \delta < 1$  and set  $\gamma_{i} = \frac{\delta\bar{r}}{\|\mathbf{g}\|}$ . Algorithm 2 executes at most  $\frac{\zeta(d(\mathbf{y}_0,(1 - \delta)\mathcal{K})^2 - d(\mathbf{y},(1 - \delta)\mathcal{K})^2)}{\delta^2\bar{r}^2} + 1$  iterations and returns  $\mathbf{y} \in \mathcal{K}$  such that  $d(\mathbf{y},\mathbf{z})^2 \leq d(\mathbf{y}_0,\mathbf{z})^2$  holds for any  $\mathbf{z} \in (1 - \delta)\mathcal{K}$ .
In the full information setting, with a separation oracle, infeasible R-OGD is shown in Algorithm 3.
Algorithm 3: Infeasible R-OGD with a separation oracle
Data: feasible gsc-convex set  $\mathcal{K}$  radius  $r$  , step-size  $\eta$  and squeeze parameter  $\delta$ $\tilde{\mathbf{y}}_1 = \mathbf{p}\in (1 - \delta)\mathcal{K}$    
for  $t = 1,\ldots ,T$  do Play  $\tilde{\mathbf{y}}_t$  and receive  $f_{t}(\tilde{\mathbf{y}}_{t})$ $\mathbf{y}_{t + 1} = \mathrm{Exp}_{\tilde{\mathbf{y}}_t}(-\eta \nabla f_t(\tilde{\mathbf{y}}_t))$  Update  $\tilde{\mathbf{y}}_{t + 1}$  as the output of Algorithm 2 with set  $\kappa$  , radius  $r$  , squeeze parameter  $\delta$  and initial point  $\mathbf{y}_{t + 1}$
We can show the following regret guarantee for Algorithm 3.
Theorem 1. (Proof in Appendix B.3) Under Assumptions 1, 3 and 4. Set  $\eta = \frac{2R}{G\sqrt{\zeta T}}$  and  $\delta = \frac{1}{2\sqrt{T}}$ , then the regret of Algorithm 3 is upper bounded by
$$
\sup  _ {[ s, e ] \subseteq [ T ]} \left\{\sum_ {t = s} ^ {e} f _ {t} (\tilde {\mathbf {y}} _ {t}) - \min  _ {\mathbf {x} _ {I} \in \mathcal {K}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {I}) \right\} \leq \frac {5}{2} G R \sqrt {\zeta T},
$$
and the number of calls to the separation oracle is  $O(T)$ .
Moving on, we demonstrate how to achieve a sublinear regret guarantee in the bandit convex optimization setting. A major challenge is that, while in Euclidean space, we can construct a separation oracle on  $(1 - \delta)\mathcal{K}$  using the separation oracle on  $\mathcal{K}$  (Garber & Kretzu, 2022, Lemma 11.). On Hadamard manifolds,  $(1 - \delta)\mathcal{K}$  can even be non-convex (Theorem 6), thus a separation oracle for  $(1 - \delta)\mathcal{K}$  may not exist. For Riemannian BCO with one-point feedback, in Algorithm 4, we address this by resorting to a non-standard setting: we play  $\widetilde{\mathbf{z}}_t\in \mathcal{K}$  but we receive feedback at  $\mathbf{z}_t$  where  $\mathbf{z}_t$ ,  $\widetilde{\mathbf{z}}_t$  are nearby points. We present the algorithm and the corresponding regret guarantee in Algorithm 4 and Theorem 2.
Algorithm 4: One-point bandit convex optimization on manifolds with a separation oracle
Data: feasible gsc-convex set  $\mathcal{K}$ , radii  $(R,r)$ , step-size  $\eta$ , squeeze parameters  $(\delta, \delta', \tau)$ ,  $\tilde{\mathbf{y}}_1 = \mathbf{p}$ . for  $t = 1, \dots, T$  do
Sample  $\mathbf{z}_t\sim \mathbb{S}_{\tilde{\mathbf{y}}_t}(\delta ')$  play  $\tilde{\mathbf{z}}_t\coloneqq \mathrm{Exp}_{\mathbf{p}}\left(\frac{\mathrm{Exp}_{\mathbf{p}}^{-1}\mathbf{z}_t}{1 + \tau}\right) / /\delta^{\prime} = \frac{\sqrt{-\kappa}(R + r)}{\sinh\left(\sqrt{-\kappa}(R + r)\right)}\tau r$
Observe  $f_{t}(\mathbf{z}_{t})$ ;  $\mathbf{g}_t = f_t(\mathbf{z}_t)\cdot \frac{\mathrm{Exp}_{\tilde{\mathbf{y}}_t}^{-1}\mathbf{z}_t}{\|\mathrm{Exp}_{\tilde{\mathbf{y}}_t}^{-1}\mathbf{z}_t\|};\mathbf{y}_{t + 1} = \mathrm{Exp}_{\tilde{\mathbf{y}}_t}(-\eta \mathbf{g}_t)$
$\tilde{\mathbf{y}}_{t + 1}\gets$  Output of Algorithm 2 with  $\kappa$  radius  $r$  , squeeze parameter  $\delta$  and initial point  $\mathbf{y}_{t + 1}$
An exponential dependence on the diameter is common on manifolds with strictly negative curvature. For instance, the volume of a geodesic ball grows exponentially with its radius in hyperbolic space, whereas this dependence is only polynomial in Euclidean space. This property has been leveraged to construct lower bounds for convex optimization on manifolds (Crisciello & Boumal, 2022).
Theorem 2. (Proof in Appendix B.4) Under Assumptions 2, 3 and 4. Set  $\eta = T^{-\frac{1}{2}}$ ,  $\delta = T^{-\frac{1}{4}}$ ,  $\tau = T^{-\frac{1}{4}}$ . Then regret of Algorithm 4 is upper bounded by
$$
\sup _ {[ s, e ] \subseteq [ T ]} \left\{\sum_ {t = s} ^ {e} f _ {t} (\widetilde {\mathbf {z}} _ {t}) - \min _ {\mathbf {x} _ {I} \in \mathcal {K}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {I}) \right\} = O \left(T ^ {\frac {3}{4}}\right),
$$
Remark 2. An acute reader may notice a discrepancy between the step-size  $\eta = O\left(T^{-\frac{1}{2}}\right)$  in our work and the step-size  $\eta = O\left(T^{-\frac{3}{4}}\right)$  in Garber & Kretzu (2022, Theorem 15). It is important to highlight that our  $\eta$  is equivalent to  $\frac{n\eta}{\delta'}$  as per Garber & Kretzu (2022), ensuring that the parameters are in fact consistent. This reasoning is also applicable to Theorem 3.
Interestingly, in the context of the two-point feedback setting, we can get rid of the non-standard setting by adapting the algorithm. In Algorithm 5, we adhere to the loop invariants:  $\mathbf{x}_t \in \beta \mathcal{K}$  and  $\mathbf{y}_t \in \mathcal{K}$ . Each round involves constructing an unbiased gradient estimator  $\mathbf{g}_t$  at  $\mathbf{x}_t$  using the two-point feedback. Subsequently, we map  $\mathbf{g}_t$  to the tangent space of  $\mathbf{y}_t$  and execute online gradient descent in that space. A key advantage of this parallel transport is the ability to employ the separation oracle on  $\mathcal{K}$  to construct an infeasible projection onto  $(1 - \delta)\mathcal{K}$ . Upon a meticulous analysis, we observe that, at each round, an additional distortion arises from this parallel transport:
$$
\frac {S _ {\delta^ {\prime}}}{V _ {\delta^ {\prime}}} \mathbb {E} [ \left\langle \mathbf {g} _ {t}, \Gamma_ {\mathbf {y} _ {t}} ^ {\mathbf {x} _ {t}} \mathrm {E x p} _ {\mathbf {y} _ {t}} ^ {- 1} \mathbf {x} - \mathrm {E x p} _ {\mathbf {x} _ {t}} ^ {- 1} \mathbf {x} \right\rangle ] \leq \frac {S _ {\delta^ {\prime}}}{V _ {\delta^ {\prime}}} \mathbb {E} [ \| \mathbf {g} _ {t} \| ] \cdot O (\delta^ {\prime}) = O (\delta^ {\prime}),
$$
This distortion accumulates to  $O(\sqrt{T})$  when choosing  $\delta' = O\left(\frac{1}{\sqrt{T}}\right)$ , ensuring the regret bound remains unaffected. The regret assurance of Algorithm 5 is detailed in Theorem 3.
Algorithm 5: Two-point bandit convex optimization on manifolds with a separation oracle
Data: feasible gsc-convex set  $\mathcal{K}$  radii  $(R,r)$  , step-size  $\eta$  parameters  $(\delta ,\delta^{\prime},\beta)$  ..  $\delta \in (0,1),\beta \in (0,1),\delta^{\prime} = (1 - \beta)\frac{\sqrt{-\kappa(R + r)}}{\sinh\left(\sqrt{-\kappa}(R + r)\right)} r.$  Initialize  $\mathbf{x}_1\in \beta \mathcal{K}$ $\mathbf{y}_1 = \mathrm{Exp}_{\mathbf{p}}\left(\frac{\mathrm{Exp}_{\mathbf{p}}^{-1}\mathbf{x}_1}{\beta}\right) / /\mathbf{y}_1\in \mathcal{K}$    
for  $t = 1,\ldots ,T$  do Sample  $\mathbf{z}_t\sim \mathbb{S}_{\mathbf{x}_t}(\delta ')$  Play  $\mathbf{z}_t$  and its antipodal point  $\tilde{\mathbf{z}}_t$  Observe  $f_{t}$  at  $\mathbf{z}_t$  and  $\tilde{\mathbf{z}}_t$  Construct  $\mathbf{g}_t$  by  $\mathbf{g}_t = \frac{1}{2} (f_t(\mathbf{z}_t) - f_t(\tilde{\mathbf{z}}_t))\frac{\mathrm{Exp}_{\mathbf{x}_t}^{-1}\mathbf{z}_t}{||\mathrm{Exp}_{\mathbf{x}_t}^{-1}\mathbf{z}_t||}$ $\mathbf{y}_{t + 1}^{\prime} = \mathrm{Exp}_{\mathbf{y}_t}(-\eta \Gamma_{\mathbf{x}_t}^{\mathbf{y}_t}\mathbf{g}_t)$ ${\bf y}_{t + 1}\gets$  Output of Algorithm 2 with  $\kappa$  , radius  $r$  , squeeze parameter.  $\delta$  and initial point  $\mathbf{y}_{t + 1}^{\prime}$ $\mathbf{x}_{t + 1} = \mathrm{Exp}_{\mathbf{p}}\left(\beta \mathrm{Exp}_{\mathbf{p}}^{-1}\mathbf{y}_{t + 1}\right) / / \mathbf{x}_{t + 1}\in \beta \kappa$    
end
Theorem 3. (Proof in Appendix B.5) Under Assumptions 2, 3 and 4. Set  $\eta = 1$ ,  $\delta = 1 - \beta = T^{-\frac{1}{2}}$ , then the regret of Algorithm 5 is upper bounded by
$$
\sup _ {[ s, e ] \subseteq [ T ]} \left\{\mathbb {E} \left[ \sum_ {t = s} ^ {e} f _ {t} (\mathbf {z} _ {t}) - \min _ {\mathbf {x} ^ {*} \in \mathcal {K}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} ^ {*}) \right] \right\} = O \left(\sqrt {T}\right)
$$
6 Riemannian OCO with a Linear Optimization Oracle
In this section, we focus on performing projection-free OCO on Riemannian manifolds utilizing a linear oracle. The linear oracle is invoked inside the Riemannian Frank-Wolfe (RFW) algorithm (Weber & Sra, 2022b). We outline how to obtain a separating hyperplane by RFW, as detailed in Algorithm 6 and Lemma 5.
Algorithm 6: Separating Hyperplane via RFW
Data: feasible gsc-convex set  $\mathcal{K}$  error tolerance  $\epsilon$  , initial point  $\mathbf{x}_1\in \mathcal{K}$  , target vector y.   
for  $i = 1,\ldots$  do   
 $\mathbf{v}_i = \mathrm{argmin}_{\mathbf{x}\in \mathcal{K}}\{\langle -\mathrm{Exp}_{\mathbf{x}_i}^{-1}\mathbf{y},\mathrm{Exp}_{\mathbf{x}_i}^{-1}\mathbf{x}\rangle \}$    
if  $\{\langle \mathrm{Exp}_{\mathbf{x}_i}^{-1}\mathbf{y},\mathrm{Exp}_{\mathbf{x}_i}^{-1}\mathbf{v}_i\rangle \} \leq \epsilon$  or  $d(\mathbf{x}_i,\mathbf{y})^2\leq 3\epsilon$  then return  $\tilde{\mathbf{x}}\gets \mathbf{x}_i$    
end   
 $\sigma_{i} = \mathrm{argmin}_{\sigma \in [0,1]}\{d(\mathbf{y},\mathrm{Exp}_{\mathbf{x}_{i}}(\sigma \mathrm{Exp}_{\mathbf{x}_{i}}^{-1}\mathbf{v}_{i}))^{2}\}$ $\mathbf{x}_{i + 1} = \mathrm{Exp}_{\mathbf{x}_i}(\sigma_i\mathrm{Exp}_{\mathbf{x}_i}^{-1}\mathbf{v}_i)$    
end
Lemma 5. (Proof in Appendix C.1) Under Assumptions 1 and 3. For any  $\mathbf{y} \in \mathbb{B}_{\mathbf{p}}(R)$ , Algorithm 6 terminates after at most  $\zeta \lceil (27R^2/\epsilon) - 2 \rceil$  iterations and returns  $\tilde{\mathbf{x}} \in \mathcal{K}$  satisfies:
1)  $d(\tilde{\mathbf{x}},\mathbf{y})^2\leq d(\mathbf{x}_1,\mathbf{y})^2.$
2) At least one of the following holds:  $d(\tilde{\mathbf{x}},\mathbf{y})^2\leq 3\epsilon$  or  $\forall \mathbf{z}\in \mathcal{K}:\left\langle \mathrm{Exp}_{\mathbf{y}}^{-1}\mathbf{z},\mathrm{Exp}_{\mathbf{y}}^{-1}\tilde{\mathbf{x}}\right\rangle \geq 2\epsilon .$
3) If  $d(\mathbf{y},\mathcal{K})\leq \epsilon$  then  $d(\tilde{\mathbf{x}},\mathbf{y})^2\leq 3\epsilon$
Remark 3. Note that the second item of Lemma 5 provides a separating hyperplane between  $\mathbf{y}$  and  $\mathcal{K}$ . One of the challenges in its proof is to find an analog of the Euclidean identity  $\langle \mathbf{z} - \mathbf{y},\tilde{\mathbf{x}} -\mathbf{y}\rangle =$ $\| \tilde{\mathbf{x}} -\mathbf{y}\| _2^2 -\langle \mathbf{z} - \tilde{\mathbf{x}},\mathbf{y} - \tilde{\mathbf{x}}\rangle$  on manifolds, which initially appears to be a daunting task. However, a clever application of Lemma 30 (Appendix E) provides a solution.
Algorithm 7: Closer Infeasible Projection via LOO
Data: feasible gsc-convex set  $\mathcal{K}$ $\mathbf{x}_0\in \mathcal{K}$  , initial point  $\mathbf{y}_0$  , error tolerance  $\epsilon$  , step size  $\gamma$ $\mathbf{y}_1 = \Pi_{\mathbb{B}_{\mathbf{p}}(R)}\mathbf{y}_0$    
if  $d(\mathbf{x}_0,\mathbf{y}_0)^2\leq 3\epsilon$  then return  $\mathbf{x}\gets \mathbf{x}_0,\mathbf{y}\gets \mathbf{y}_1$    
end   
for  $i = 1,\ldots ,T$  do  $\begin{array}{rl} & {\mathbf{x}_i\leftarrow \mathrm{Output~of~Algorithm~6~with~set~}\mathcal{K},\mathrm{~feasible~point~}\mathbf{x}_{i - 1},\mathrm{~initial~point~}\mathbf{y}_i\mathrm{~and~tolerance~}\epsilon .}\\ & {\mathrm{if~}d(\mathbf{x}_i,\mathbf{y}_i)^2 >3\epsilon \mathrm{~then}}\\ & {\mathrm{|}\mathbf{y}_{i + 1} = \mathrm{Exp}_{\mathbf{x}_i}((1 - \gamma)\mathrm{Exp}_{\mathbf{x}_i}^{-1}\mathbf{y}_i)}\\ & {\mathrm{else}}\\ & {\mathrm{|}\mathbf{return~x}\leftarrow \mathbf{x}_i,\mathbf{y}\leftarrow \mathbf{y}_i.}\\ & {\mathrm{end}} \end{array}$    
end
Algorithm 7 demonstrates how to "pull" an initial point  $\mathbf{y}_0$  towards  $\kappa$  using RFW, while Lemma 6 verifies that the output of Algorithm 7 is indeed an infeasible projection onto  $\kappa$ .
Lemma 6. (Proof in Appendix C.2) Under Assumptions 1 and 3. Fix  $\epsilon >0$ . Setting  $\gamma = \frac{2\epsilon}{d(\mathbf{x}_0,\mathbf{y}_0)^2}$ , Algorithm 7 stops after at most  $\max \left\{\frac{\zeta d(\mathbf{x}_0,\mathbf{y}_0)^2(d(\mathbf{x}_0,\mathbf{y}_0)^2 - \epsilon)}{4\epsilon^2} +1,1\right\}$  iterations, and returns  $(\mathbf{x},\mathbf{y})\in \mathcal{K}\times \mathbb{B}_{\mathbf{p}}(R)$  such that
$$
\forall \mathbf {z} \in \mathcal {K}: d (\mathbf {y}, \mathbf {z}) ^ {2} \leq d (\mathbf {y} _ {0}, \mathbf {z}) ^ {2} \quad a n d \quad d (\mathbf {x}, \mathbf {y}) ^ {2} \leq 3 \epsilon .
$$
Given that Lemma 6 provides an infeasible projection oracle, we can combine Algorithms 1 and 7 to achieve sublinear regret by setting the error tolerance as  $\epsilon = o(1)$ . However, RFW requires  $\Theta (1 / \epsilon) = \omega (1)$  iterations in the worst-case scenario (Lemma 5), and the resulting algorithm necessitates  $\omega (T)$  calls to the linear optimization oracle. Garber & Kretzu (2022) utilize a block trick to address this challenge: the time horizon  $T$  is broken into  $B$  blocks, and the infeasible projection is computed once in each block. We demonstrate that this trick can be implemented on Riemannian manifolds in Algorithm 8. We present the regret guarantees for gsc-convex and strongly gsc-convex losses in Theorem 4 and Theorem 5, respectively.
Theorem 4. (Proof in Appendix C.3) Under Assumptions 1, 3 and 4. Fixing  $\eta_{i}$  and  $\epsilon_{i}$  as  $\eta = \frac{R\zeta}{G} T^{-\frac{3}{4}}$  and  $\epsilon = 60R^{2}\zeta^{2}T^{-\frac{1}{2}}$ , respectively, for any  $i\in \{1,\dots ,\frac{T}{B}\}$ . Setting  $B = 5T^{1 / 2}$ . Then the regret of
Algorithm 8: Block OGD on manifolds with a linear optimization oracle
Data: horizon  $T$  , feasible gsc-convex set  $\kappa$  , number of blocks  $B$  , step-sizes  $\{\eta_i\}_{i = 1}^T$  , error tolerances  $\{\epsilon_i\}_{i = 1}^T$  .   
Choose  $\mathbf{x}_0,\mathbf{x}_1\in \mathcal{K}$ $\tilde{\mathbf{y}}_0 = \mathbf{x}_0,\mathbf{y}_1 = \mathbf{x}_0,\tilde{\mathbf{y}}_1 = \mathbf{x}_1,\nabla_1 = \mathbf{0}\in T_{\tilde{\mathbf{y}}_0}\mathcal{M}$    
for  $t = 1,\ldots ,B$  do Play  $\mathbf{x}_0$  ; observe  $f_{t}(\mathbf{x}_{0})$ $\nabla_{1} = \nabla_{1} + \nabla f_{t}(\tilde{\mathbf{y}}_{0})$    
end   
 ${\bf y}_{B + 1} = \mathrm{Exp}_{\tilde{\bf y}_0}(-\eta_1\nabla_1)$    
for  $i = 2,\dots ,\frac{T}{B}$  do   
 $(\mathbf{x}_i,\tilde{\mathbf{y}}_i)\in \mathcal{K}\times \mathbb{B}_{\mathbf{p}}(R)\gets$  output of Algorithm 7 with input  $\kappa$  ,  $\mathbf{x}_{i - 2},\mathbf{y}_{(i - 1)B + 1}$  and  $\epsilon_{i}$ $\mathbf{y}_{(i - 1)B + 1} = \tilde{\mathbf{y}}_{i - 1};\nabla_{i} = \mathbf{0}\in T_{\tilde{\mathbf{y}}_{i - 1}}\mathcal{M}.$  for  $s = 1,\ldots ,B$  do Play  $\mathbf{x}_{i - 1}$  and observe  $f_{t}(\mathbf{x}_{i - 1})$ $\nabla_{i} = \nabla_{i} + \nabla f_{(i - 1)B + s}(\tilde{\mathbf{y}}_{i - 1})$    
end   
 $\mathbf{y}_{iB + 1} = \mathrm{Exp}_{\tilde{\mathbf{y}}_{i - 1}}(-\eta_i\nabla_i)$
Algorithm 8 for gsc-convex losses is bounded by
$$
\sup _ {I = [ s, e ] \subseteq [ T ]} \left\{\sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {t}) - \min _ {\mathbf {x} _ {I} \in \mathcal {K}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {I}) \right\} \leq G R \left(\frac {5}{2} T ^ {3 / 4} \zeta^ {2} + \zeta \sqrt {1 8 0} T ^ {3 / 4} + 4 T ^ {3 / 4} / \zeta + 2 0 T ^ {1 / 2}\right),
$$
and the number of calls to the linear optimization oracle is bounded by  $T$ .
Theorem 5. (Proof in Appendix C.4) Under Assumptions 1, 3 and 4. Suppose all losses  $\{f_t\}_{t=1}^T$  are  $\alpha$ -strongly gsc-convex for some known  $\alpha > 0$  and  $T \geq 3B$ . Choosing  $\epsilon_i = \left(\frac{20G}{\alpha(i+3)}\right)^2\zeta$  and  $\eta_i = \frac{2}{\alpha iB}$  for any  $i = 1, \ldots, \frac{T}{B}$ . With  $B = \left(\frac{\alpha R}{G}\right)^{\frac{2}{3}}T^{2/3}$  and assume  $T \geq 3B$ , the regret guarantee of Algorithm 8 is bounded by
$$
\sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x} _ {t}) - \min  _ {\mathbf {x} \in \mathcal {K}} \sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x}) \leq (2 0 \sqrt {3} \zeta + 1) \left(G ^ {4} R ^ {2} / \alpha\right) ^ {\frac {1}{3}} T ^ {2 / 3} \left(1 + \frac {2}{3} \ln \left(\frac {\sqrt {T} G}{\alpha R}\right)\right).
$$
And the number of total calls to the linear optimization oracle is bounded by  $\zeta T$ .
7 Conclusion and Perspective
This paper pioneers the exploration of projection-free online optimization on Riemannian manifolds. The primary technical challenges originate from the non-convex nature of the Riemannian hyperplane and the variable metric. These challenges are tackled effectively with the aid of the Jacobi field comparison, enabling us to establish a spectrum of sub-linear regret guarantees. Interested readers may question the difficulty of generalizing these techniques from Hadamard manifolds to  $\mathrm{CAT}(\kappa)$  manifolds. Some hints toward this generalization are provided in Appendix E.3.
There are several promising directions for future research. First, there exists the potential to refine the regret bounds, particularly for strongly gsc-convex losses via a separation oracle. In the context of the separation oracle, reducing the dependence on the number of calls about the diameter of the decision set would be an intriguing objective. Moreover, devising an efficient method to optimize the linear optimization oracle objective,  $\mathrm{argmin}_{\mathbf{x} \in \mathcal{K}} \left\langle \mathbf{g}, \mathrm{Exp}_{\mathbf{x}_0}^{-1} \mathbf{x} \right\rangle$ , remains a notable open problem. This paper does not discuss the membership oracle, primarily because related work (Mhammedi, 2022a; Lu et al., 2023) heavily relies on the convexity of the Minkowski functional in Euclidean space, a property not guaranteed to hold on Hadamard manifolds (Theorem 7). However, this does not rule out the potential for executing OCO or convex optimization on manifolds using a membership oracle. Thus, uncovering alternative strategies to tackle this issue remains a compelling research question.
We would like to thank five anonymous referees for constructive comments and suggestions. We gratefully thank the AI4OPT Institute for funding, as part of NSF Award 2112533. We also acknowledge the NSF for their support through Award IIS-1910077. GW would like to acknowledge an ARC-ACO fellowship provided by Georgia Tech. We would also like to thank Xi Wang from UCAS and Andre Wibisono from Yale for helpful discussions.