SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
N _ {c a l l s} \leq \frac {T}{B} \left(1 + \frac {1 5}{2} \zeta + \left(\frac {1 1 B ^ {2} G ^ {2} \eta^ {2}}{2 \epsilon} + \frac {B ^ {4} G ^ {4} \eta^ {4}}{\epsilon^ {2}}\right) \zeta\right) \cdot \frac {2 7 \zeta R ^ {2}}{\epsilon}.
$$
Plugging in the values of  $B, \eta$  and  $\epsilon$ , we find  $N_{calls} \leq T$  as claimed.
Proof. Based on a similar argument as in Equation (35), we have
$$
\begin{array}{l} \sum_ {i = 1} ^ {\frac {T}{B}} \sum_ {t \in \mathcal {T} _ {i}} \left(f _ {t} \left(\tilde {\mathbf {y}} _ {i - 1}\right) - f _ {t} (\mathbf {x})\right) \\ \leq \sum_ {i = 1} ^ {\frac {T}{B}} \left(\sum_ {t \in \mathcal {T} _ {i}} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i - 1}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} ^ {- 1} \mathbf {x} \right\rangle - \frac {\alpha B}{2} d \left(\tilde {\mathbf {y}} _ {i - 1}, \mathbf {x}\right) ^ {2}\right) \tag {43} \\ \leq \sum_ {i = 1} ^ {\frac {T}{B}} \left(\frac {d (\tilde {\mathbf {y}} _ {i - 1} , \mathbf {x}) ^ {2} - d (\tilde {\mathbf {y}} _ {i + 1} , \mathbf {x}) ^ {2}}{2 \eta_ {i}} + \frac {\zeta \eta B ^ {2} G ^ {2}}{2} - \frac {\alpha B}{2} d (\tilde {\mathbf {y}} _ {i - 1}, \mathbf {x}) ^ {2}\right) \\ \leq \sum_ {i = 1} ^ {\frac {T}{B}} \frac {\zeta B G ^ {2}}{\alpha i}, \\ \end{array}
$$
where we apply the strong gsc-convexity on the  $i$ -th block under the help of Lemma 28; the last inequality follows from the definition of  $\eta_{i}$ . If we consider the sum of regret over each block as in Equation (43),
$$
\sum_ {t = 1} ^ {T} \left(f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right) - f _ {t} (\mathbf {x})\right) \leq \frac {\zeta B G ^ {2}}{\alpha} \sum_ {i = 1} ^ {\frac {T}{B}} \frac {1}{i}. \tag {44}
$$
On the other hand, since  $f_{t}(\cdot)$  is  $G$ -Lipschitz, by Lemma 6,
$$
\begin{array}{l} \sum_ {t = 1 \atop T} ^ {T} \left(f _ {t} \left(\mathbf {x} _ {i (t) - 1}\right) - f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right)\right) \tag {45} \\ \leq \sum_ {i = 1} ^ {\frac {T}{B}} \sum_ {t \in \mathcal {T} _ {i}} \left\langle \nabla f _ {t} (\mathbf {x} _ {i (t) - 1}), - \mathrm {E x p} _ {\mathbf {x} _ {i (t) - 1}} ^ {- 1} \tilde {\mathbf {y}} _ {i (t) - 1} \right\rangle \leq \sqrt {3} B G \sum_ {i = 2} ^ {\frac {T}{B}} \sqrt {\epsilon_ {i - 1}}, \\ \end{array}
$$
where we use the fact that for the first block,  $i(t) = 1$  and thus  $\mathbf{x}_{i(t)-1} = \tilde{\mathbf{y}}_{i(t)-1}$ . Now as we combine Equations (44) and (45), we have
$$
\sum_ {t = 1} ^ {T} \left(f _ {t} (\mathbf {x} _ {i (t) - 1}) - f _ {t} (\mathbf {x})\right) \leq \sqrt {3} B G \sum_ {i = 1} ^ {\frac {T}{B}} \sqrt {\epsilon_ {m}} + \frac {\zeta B G ^ {2}}{\alpha} \sum_ {i = 1} ^ {\frac {T}{B}} \frac {1}{i}.
$$
By plugging in  $\{\epsilon_i\}_{i=1}^{\frac{T}{B}}$ , we get the stated regret guarantee.
Now it remains to bound the number of calls to the linear optimization oracle. Note that for each block  $i \in \left[\frac{T}{B}\right]$ , by Lemma 6,  $d(\mathbf{x}_i, \tilde{\mathbf{y}}_i)^2 \leq 3\epsilon_i$ . In view of Algorithm 8, we also have
$$
\mathbf {y} _ {(i + 1) K + 1} = \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i}} \left(- \eta_ {i + 1} \sum_ {t = i B + 1} ^ {(i + 1) B} \nabla f _ {t} (\tilde {\mathbf {y}} _ {i})\right).
$$
$$
d \left(\mathbf {x} _ {i - 1}, \mathbf {y} _ {i B + 1}\right) \leq d \left(\mathbf {x} _ {i - 1}, \tilde {\mathbf {y}} _ {i - 1}\right) + d \left(\tilde {\mathbf {y}} _ {i - 1}, \mathbf {y} _ {i B + 1}\right) \leq \sqrt {3 \epsilon_ {i - 1}} + \eta_ {i} B G. \tag {46}
$$
Squaring both sides and making use of  $(a + b)^2\leq 2(a^2 +b^2)$  , we have
$$
d \left(\mathbf {x} _ {i - 1}, \mathbf {y} _ {i B + 1}\right) ^ {2} \leq 6 \epsilon_ {i - 1} + 2 B ^ {2} G ^ {2} \eta_ {i} ^ {2}. \tag {47}
$$
By Lemma 6, the number of iterations in Algorithm 7 for the  $i + 1$ -th block is at most
$$
\begin{array}{l} \max  \left\{\frac {\zeta d \left(\mathbf {x} _ {i - 1} , \mathbf {y} _ {i B + 1}\right) ^ {2} \left(d \left(\mathbf {x} _ {i - 1} , \mathbf {y} _ {i B + 1}\right) ^ {2} - \epsilon_ {i + 1}\right)}{4 \epsilon_ {i + 1} ^ {2}} + 1, 1 \right\} \\ \leq \max  \left\{\zeta \cdot \frac {3 0 \epsilon_ {i - 1} ^ {2} + 2 2 B ^ {2} G ^ {2} \eta_ {i} ^ {2} \epsilon_ {i - 1} + 4 B ^ {4} G ^ {4} \eta_ {i} ^ {4}}{4 \epsilon_ {i + 1} ^ {2}} + 1, 1 \right\} \tag {48} \\ \leq \max \left\{\zeta \frac {6 1 \epsilon_ {i - 1} ^ {2} + 8 B ^ {4} G ^ {4} \eta_ {i} ^ {4}}{4 \epsilon_ {i + 1} ^ {2}} + 1, 1 \right\} \\ \leq \max  \left\{\frac {1 6 \epsilon_ {i - 1} ^ {2}}{\epsilon_ {i + 1} ^ {2}} \zeta + \frac {2 B ^ {4} G ^ {4} \eta_ {i} ^ {4}}{\epsilon_ {i + 1} ^ {4}} \zeta + 1, 1 \right\} \\ \end{array}
$$
Thus, the number of calls to the linear optimization oracle is bounded by
$$
\begin{array}{l} N _ {c a l l s} := \sum_ {i = 2} ^ {\frac {T}{B}} \left(\frac {1 6 \epsilon_ {i - 1} ^ {2}}{\epsilon_ {i + 1} ^ {2}} \zeta + \frac {2 B ^ {4} G ^ {4} \eta_ {i} ^ {4}}{\epsilon_ {i + 1} ^ {4}} \zeta + 1\right) \cdot \frac {2 7 R ^ {2} \zeta}{\epsilon_ {i}} \\ \leq \sum_ {i = 2} ^ {\frac {T}{B}} \left(\frac {1 6 (i + 3) ^ {4}}{(i + 1) ^ {4}} + \frac {2 (i + 3) ^ {4}}{4 0 0 ^ {2} (i - 1) ^ {4} \zeta^ {2}} + \frac {1}{\zeta}\right) \cdot \left(\frac {R \alpha}{2 0 G}\right) ^ {2} (i + 3) ^ {2} \zeta \tag {49} \\ \leq 0. 3 2 \left(\frac {R \alpha}{G}\right) ^ {2} \sum_ {i = 2} ^ {\frac {T}{B}} (i + 3) ^ {2} \zeta \\ \end{array}
$$
where the last line follows from  $\zeta \geq 1$ ,  $\frac{i + 3}{i + 1} \leq \frac{5}{3}$  and  $\frac{i + 3}{i - 1} \leq 5$  holds for any  $i \geq 2$ . On the other hand, we have
$$
\sum_ {i = 2} ^ {\frac {T}{B}} (i + 3) ^ {2} \leq \sum_ {i = 5} ^ {\frac {T}{B} + 3} i ^ {2} \leq \sum_ {i = 5} ^ {\frac {2 T}{B}} i ^ {2} \leq \frac {8}{3} \left(\frac {T}{B}\right) ^ {2} \tag {50}
$$
Combining Equations (49), (50) and plugging in  $B = \left(\frac{\alpha R}{G}\right)^{\frac{2}{3}}T^{2 / 3}$ , we get the claimed  $N_{calls} \leq \zeta T$ .
D Some Basic Facts about Convex Sets on Hadamard Manifolds
In this section, we highlight two significant characteristics of convex sets in Euclidean space which do not apply to Hadamard manifolds, emphasizing the fundamental differences between these two spaces. Our counterexamples are constructed within the Poincaré half-plane,  $\mathbb{H}^2$ . Therefore, understanding some basic properties of this unique manifold will prove beneficial.
Lemma 10. (Udriste, 2013; Wang et al., 2016; Kristály et al., 2016) The Poincaré half plane  $\mathbb{H}^2$  can be described by
$$
\mathcal {M} := \left\{\left(t _ {1}, t _ {2}\right) \in \mathbb {R} ^ {2} | t _ {2} > 0 \right\}
$$
with metric
$$
g _ {i j} (u, v) = \frac {\delta_ {i j}}{v ^ {2}} \qquad i = 1, 2 \quad j = 1, 2.
$$
We have the following conclusions about geodesics and the distance on the Poincaré half-plane.
1) For the geodesic connecting  $x = (t_1, t_2)$  and  $y = (s_1, s_2)$ , let
$$
b _ {x y} := \frac {s _ {1} ^ {2} + s _ {2} ^ {2} - t _ {1} ^ {2} - t _ {2} ^ {2}}{2 (s _ {1} - t _ {1})} \qquad r _ {x y} = \sqrt {(s _ {1} - b _ {x y}) ^ {2} + s _ {2} ^ {2}},
$$
then the geodesic  $\gamma_{xy} \coloneqq (\gamma_{xy}^1, \gamma_{xy}^2)$  connecting  $x$  and  $y$  is,
$$
\gamma_ {x y} ^ {1} (s) := \left\{ \begin{array}{l l} t _ {1}, & \text {i f} t _ {1} = s _ {1} \\ b _ {x y} - r _ {x y} \tanh  \left((1 - s) \cdot \operatorname {a r t a n h} \frac {b _ {x y} - t _ {1}}{r _ {x y}} + s \cdot \operatorname {a r t a n h} \frac {b _ {x y} - s _ {1}}{r _ {x y}}\right), & \text {i f} t _ {1} \neq s _ {1} \end{array} \right. \tag {51}
$$
$$
\gamma_ {x y} ^ {2} (s) := \left\{ \begin{array}{l l} e ^ {(1 - s) \cdot \ln t _ {2} + s \cdot \ln s _ {2}}, & i f t _ {1} = s _ {1} \\ \frac {r _ {x y}}{\cosh \left(\left(1 - s\right) \cdot \operatorname {a r t a n h} \frac {b _ {x y} - t _ {1}}{r _ {x y}} + s \cdot \operatorname {a r t a n h} \frac {b _ {x y} - s _ {1}}{r _ {x y}}\right)}, & i f t _ {1} \neq s _ {1}. \end{array} \right. \tag {52}
$$
Indeed, we have  $(\gamma_{xy}^{1}(s) - b_{xy})^{2} + (\gamma_{xy}^{2}(s))^{2} = r_{xy}^{2}$  for any  $s \in [0,1]$  when  $t_1 \neq s_1$ .
2) The distance between two points  $(u_{1}, v_{1})$  and  $(u_{2}, v_{2})$  can be calculated by
$$
d _ {\mathbb {H}} \left(\left(u _ {1}, v _ {1}\right), \left(u _ {2}, v _ {2}\right)\right) = \operatorname {a r c o s h} \left(1 + \frac {\left(u _ {2} - u _ {1}\right) ^ {2} + \left(v _ {2} - v _ {1}\right) ^ {2}}{2 v _ {1} v _ {2}}\right).
$$
D.1 Shrinking Gsc-convex Sets on Hadamard Manifolds Does Not Preserve Convexity
In the context of projection-free online learning on manifolds, the set  $c\mathcal{K} = \mathrm{Exp}_{\mathbf{p}}(c\mathrm{Exp}_{\mathbf{p}}^{-1}\mathbf{x})|\mathbf{x}\in \mathcal{K}$  takes on a pivotal role, particularly when  $c < 1$ . Consider a convex set  $\mathcal{K}$  in Euclidean space, then the convexity of  $c\mathcal{K}$  is guaranteed for any positive  $c$ . Nonetheless, this property does not necessarily hold on Hadamard manifolds. We can construct a counterexample within the Poincaré half-plane  $\mathbb{H}^2$ . To achieve this, we rely on the following lemma.
Lemma 11. On the Poincaré half-plane  $\mathbb{H}^2$ , the mid-point of the geodesic  $\gamma_{xy}$  which connects  $x = (-\lambda, \mu)$  and  $y = (\lambda, \mu)$  is  $(0, \sqrt{\lambda^2 + \mu^2})$ .
Proof. By symmetry we know  $\gamma_{xy}^{1}(\frac{1}{2}) = 0$ . To compute  $\gamma_{xy}^{2}(\frac{1}{2})$ , note that  $b_{xy} = 0$  and  $r_{xy} = \sqrt{\lambda^2 + \mu^2}$ , then by Lemma 10,
$$
\gamma_ {x y} ^ {2} \left(\frac {1}{2}\right) = \frac {\sqrt {\lambda^ {2} + \mu^ {2}}}{\cosh \left(\frac {1}{2} \operatorname {a r c t a n h} \frac {\lambda}{\sqrt {\lambda^ {2} + \mu^ {2}}} + \frac {1}{2} \operatorname {a r c t a n h} \frac {- \lambda}{\sqrt {\lambda^ {2} + \mu^ {2}}}\right)} = \sqrt {\lambda^ {2} + \mu^ {2}},
$$
where we use the fact that  $\operatorname{arctanh}(\cdot)$  is an odd function and  $\cosh(0) = 1$ .
Now we prove the following theorem.
Theorem 6. There exists a counter-example on the Poincaré half plane  $\mathbb{H}^2$  such that the radially contracted set  $(1 - \alpha)\mathcal{K}$  where  $\alpha \in (0,1)$  is non-convex.
Proof. Let  $p = (0,1)$ ,  $u = (-1,\sqrt{2})$ ,  $v = (1,\sqrt{2})$ ,  $w = (0,\sqrt{3})$ . Observe that  $\gamma_{uv}$  is an arc satisfying  $\{(\lambda ,\mu)|\lambda^2 +\mu^2 = 3\}$  and, by Lemma 11,  $w$  is its midpoint. Following this, we calculate  $\gamma_{pu},\gamma_{pw}$ , and  $\gamma_{pv}$ , and shrink these three geodesics by a factor of  $(1 - \alpha)$ . Let's denote the resulting geodesics as  $\gamma_{pu'}$ ,  $\gamma_{pw'}$ , and  $\gamma_{pv'}$ . Ultimately, we demonstrate that for some  $\alpha$ ,  $w'$  is positioned beneath  $\gamma_{u'v'}$ , implying that the radial contraction of  $\mathcal{K}$  does not encompass a geodesic, hence proving
its non-convexity. We first compute  $\gamma_{pu}$  and  $\gamma_{pv}$  to facilitate this. Note that  $b_{pu} = -1$  and  $r_{pu} = \sqrt{2}$ , by Lemma 10, we can write down
$$
\gamma_ {p u} ^ {1} (s) = - 1 - \sqrt {2} \tanh  \left((1 - s) \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)
$$
$$
\gamma_ {p u} ^ {2} (s) = \frac {\sqrt {2}}{\cosh \left((1 - s) \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)}.
$$
By symmetry, we have
$$
\gamma_ {p v} ^ {1} (s) = 1 + \sqrt {2} \tanh \left((1 - s) \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)
$$
$$
\gamma_ {p v} ^ {2} (s) = \frac {\sqrt {2}}{\cosh \left((1 - s) \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)}.
$$
We can also compute  $\gamma_{pw}(s) = \left(0, e^{s\ln \sqrt{3}}\right)$ . Now we have
$$
\begin{array}{l} u ^ {\prime} = \gamma_ {p u} (1 - \alpha) = \left(- 1 - \sqrt {2} \tanh \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right), \frac {\sqrt {2}}{\cosh \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)}\right) \\ v ^ {\prime} = \gamma_ {p v} (1 - \alpha) = \left(1 + \sqrt {2} \tanh \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right), \frac {\sqrt {2}}{\cosh \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)}\right) \\ \end{array}
$$
and  $w' = \left(0, e^{(1 - \alpha)\ln \sqrt{3}}\right)$ . We just need to demonstrate that for some  $\alpha \in (0, 1)$ ,  $w'$  is positioned below  $\gamma_{u'v'}\left(\frac{1}{2}\right)$ , where
$$
\gamma_ {u ^ {\prime} v ^ {\prime}} \left(\frac {1}{2}\right) = \left(0, \sqrt {3 + 2 \sqrt {2} \tanh  \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)}\right).
$$
by Lemma 11. Specifically, we need to show  $\exists \alpha \in (0,1)$
$$
e ^ {(1 - \alpha) \ln \sqrt {3}} <   \sqrt {3 + 2 \sqrt {2} \tanh \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)}
$$
We set  $\alpha = .1$ , then
$$
\sqrt {3 + 2 \sqrt {2} \tanh  \left(\alpha \operatorname {a r c t a n h} \left(- \frac {1}{\sqrt {2}}\right)\right)} - e ^ {(1 - \alpha) \ln \sqrt {3}} = 0. 0 3 6 \dots > 0.
$$
Therefore, the set  $(1 - \alpha)\mathcal{K}$  fails to contain the geodesic  $\gamma_{u'v'}(s)$  and is thus non-convex.
D.2 Non-convexity of the Minkowski Functional on Hadamard Manifolds
In this part, we show the Minkowski functional (Gauge function), which is defined as
$$
\gamma_ {\mathcal {K}} (\mathbf {x}) = \inf  \left\{\lambda > 0: \operatorname {E x p} _ {\mathbf {p}} \left(\frac {\operatorname {E x p} _ {\mathbf {p}} ^ {- 1} \mathbf {z}}{\lambda}\right) \in \mathcal {K} \right\} \tag {53}
$$
can be non-convex on Hadamard manifolds, where  $\mathcal{K}$  is a gsc-convex set of a Hadamard manifold  $\mathcal{M}$  and  $\mathbf{p}$  is an interior point in  $\mathcal{K}$ .
Theorem 7. The Minkowski functional defined in Equation (53) can be non-convex on Hadamard manifolds.
Proof. We again construct a counterexample on the Poincaré plane  $\mathbb{H}^2$ :
$$
\mathcal {M} := \left\{\left(t _ {1}, t _ {2}\right) \in \mathbb {R} ^ {2} \mid t _ {2} > 0 \right\}.
$$
To verify the geodesic convexity, we can verify an equivalent statement: a function  $f$  is gsc-convex iff  $f$  is convex when restricted to any geodesic. That is  $f(\gamma(t)) \leq (1 - t)f(\gamma(0)) + tf(\gamma(1))$  holds for any geodesic  $\gamma(t): [0,1] \to \mathcal{M}$ . The counterexample is as follows.
Assume  $\mathcal{K}$  is the gsc-convex set consists of all points  $(u_{1},u_{2})$  such that  $u_{1}^{2} + u_{2}^{2}\leq 3$  and  $u_{2} > 0$ .  $x = (-1,\sqrt{3}),y = (-2,2)$  and  $p = (0,1)$ . We show that  $\gamma_{\mathcal{K}}(x)$  is non-convex on the geodesic  $\gamma_{xy}$ .
We first compute  $b_{xy} = -2$  and  $r_{xy} = 2$ . By Lemma 10, we have
$$
\begin{array}{l} \gamma_ {x y} (s) = \left(- 2 - 2 \tanh  \left((1 - s) \operatorname {a r c t a n h} \left(- \frac {1}{2}\right)\right), \frac {2}{\cosh \left((1 - s) \operatorname {a r c t a n h} \left(- \frac {1}{2}\right)\right)}\right) \tag {54} \\ := (- 2 - 2 \cos \theta (s), 2 \sin \theta (s)), \\ \end{array}
$$
where we let  $\theta(s) \coloneqq \tanh\left((1 - s)\operatorname{arctanh}\left(-\frac{1}{2}\right)\right)$  to ease the elaboration. We denote  $z \coloneqq \gamma_{xy}(s) = (-2 - 2\cos \theta(s), 2\sin \theta(s))$ . Then we compute the geodesic  $\gamma_{pz}$  connecting  $z$  and  $p = (0,1)$ , and we have
$$
b _ {p z} (s) = \frac {(2 + 2 \cos \theta (s)) ^ {2} + (2 \sin \theta (s)) ^ {2} - 1}{2 (- 2 - 2 \cos \theta (s))} = - \frac {7 + 8 \cos \theta (s)}{4 (1 + \cos \theta (s))}
$$
and  $r_{pz}(s) = \sqrt{b_{pz}(s)^2 + 1}$ . Now we can compute the intersection of  $\gamma_{pz}(s)$  with  $\mathcal{K}$  as the solution  $(q_1,q_2)$  of the following system of equations
$$
\left\{ \begin{array}{l} (q _ {1} - b _ {p z} (s)) ^ {2} + q _ {2} ^ {2} = b _ {p z} (s) ^ {2} + 1 \\ q _ {1} ^ {2} + q _ {2} ^ {2} = 3 \end{array} \right.
$$
The intersection point is  $\left(\frac{1}{b_{pz}(s)},\sqrt{3 - \frac{1}{b_{pz}(s)^2}}\right)$ . For each  $s\in [0,1]$ , we can evaluate the Minkowski functional at  $\gamma_{xy}(s)$  w.r.t.  $\mathcal{K}$  as
$$
h (s) = d _ {\mathbb {H}} ((- 2 - 2 \cos \theta (s), 2 \sin \theta (s)), (0, 1)) / d _ {\mathbb {H}} \left(\left(\frac {1}{b _ {p z} (s)}, \sqrt {3 - \frac {1}{b _ {p z} (s) ^ {2}}}\right), (0, 1)\right),
$$
where  $d_{\mathbb{H}}$  is the hyperbolic distance defined in the second item of Lemma 10. If the Minkowski functional is gsc-convex, then we should have  $h(s) \leq (1 - s)h(0) + s \cdot h(1)$  for any  $s \in [0,1]$ . However, by numerical computation, we find
$$
h (1 / 2) - \frac {1}{2} h (0) - \frac {1}{2} h (1) = 0. 0 0 5 7 \dots > 0,
$$
which refutes the assumption that  $h(s)$  is convex. Thus our conclusion is established.
E.1 Background on the Jacobi Field
The role of the Jacobi field as an essential tool for understanding the impact of curvature on the behavior of neighboring geodesics cannot be overstated. To make our discussion more comprehensive and self-contained, we will introduce related definitions and properties sourced from Lee (2006) and Lee (2018).
We define an admissible curve as a continuous map,  $\gamma(t): [a, b] \to \mathcal{M}$ , representing a piecewise regular curve segment. An admissible family of curves, on the other hand, is a continuous map  $\Gamma(t, s): [a, b] \times (-\epsilon, \epsilon) \to \mathcal{M}$  which is smooth on each rectangle  $[a_{i-1}, a_i] \times (-\epsilon, \epsilon)$  for a finite subdivision  $a = a_0 < \dots < a_k = b$ . Here,  $\Gamma_s(t) \coloneqq \Gamma(t, s)$  is recognized as an admissible curve for each  $s \in (-\epsilon, \epsilon)$ . When  $\gamma(t)$  is admissible and  $\Gamma(t, 0) = \gamma(t)$ , we refer to  $\Gamma$  as a variation of  $\gamma(t)$ .
Upon an admissible curve  $\Gamma$ , we can delineate two sets of curves: the principal curves  $\Gamma_s(t)$ , which hold  $s$  as constant, and the transverse curves  $\Gamma^{(t)}(s) \coloneqq \Gamma(t,s)$ , which consider  $t$  as constant. We
consider  $\Gamma$  as a variation through geodesics if each principal curve  $\Gamma_s(t) = \Gamma(t, s)$  is a geodesic segment. For an admissible family of curves  $\Gamma$ , we define a vector field along  $\Gamma$  as a map  $V: [a, b] \times (-\epsilon, \epsilon) \to T\mathcal{M}$  such that  $V(t, s) \in T_{\Gamma(t, s)}\mathcal{M}$  and for each  $s$ ,  $V(s, t)$  is piecewise smooth. The Jacobi field is a vector field along a variation through geodesics and is formally defined as follows.
Lemma 12. (Lee, 2018, Theorem 10.1, The Jacobi Equation) Let  $(\mathcal{M},g)$  be a Riemannian manifold,  $\gamma$  be a geodesic in  $\mathcal{M}$ , and  $J$  be a vector field along  $\gamma$ . If  $J$  is the variation field of a variation through geodesics, then  $J$  needs to satisfy the following Jacobi Equation:
$$
D _ {t} ^ {2} J + R (J, \dot {\gamma}) \dot {\gamma} = 0,
$$
where  $R$  is the Riemann curvature endomorphism. A smooth vector field  $J$  satisfying the Jacobi Equation is called a Jacobi field.
Given initial values of  $J$  and  $D_{t}J$  at one point, the Jacobi Equation can be uniquely determined by the following proposition.
Lemma 13. (Lee, 2018, Prop. 10.2, Existence and Uniqueness of Jacobi Fields) Let  $(\mathcal{M},g)$  be a Riemannian manifold and  $\gamma :I\to \mathcal{M}$  is a geodesic where  $I$  is an interval. Fix  $a\in I$ , let  $p = \gamma (a)$ ,  $u,v\in T_p\mathcal{M}$ , there is a unique Jacobi field  $J$  along  $\gamma$  which satisfies
$$
J (a) = u, \quad D _ {t} J (a) = v.
$$
The Jacobi field seems to be a complicated object to study, but the following lemma shows we can write down the initial conditions of the Jacobi field for certain variations through geodesics.
Lemma 14. (Lee, 2018, Lemma 10.9) Let  $(\mathcal{M},g)$  be a Riemannian manifold,  $I$  be an interval containing 0, and  $\gamma :I\to \mathcal{M}$  be a geodesic. Suppose  $J:I\rightarrow \mathcal{M}$  be a Jacobi field with  $J(0) = 0$ . If  $\mathcal{M}$  is geodesically complete or  $I$  is compact, then  $J$  is the variation field of the following variation of  $\gamma$ :
$$
\Gamma (t, s) = \operatorname {E x p} _ {p} (t (u + s v)),
$$
where  $p = \gamma (0)$ $u = \dot{\gamma} (0)$  and  $v = D_{t}J(0)$
The advantage of introducing the Jacobi field is the norm of  $J(t)$  can be bounded by the initial value  $\| D_t J(0) \|$ , as shown in the following Lemma 15.
Definition 4. We define
$$
\mathbf {s} (\kappa , t) = \left\{ \begin{array}{l l} t, & i f \kappa = 0 \\ \frac {1}{\sqrt {\kappa}} \sin (\sqrt {\kappa} t), & i f \kappa > 0 \\ \frac {1}{\sqrt {- \kappa}} \sinh (\sqrt {- \kappa} t), & i f \kappa <   0. \end{array} \right.
$$
Lemma 15. (Lee, 2018, Theorem 11.9) Suppose  $(\mathcal{M},g)$  is a Riemannian manifold,  $\gamma :[0,b]\to \mathcal{M}$  is a unit-speed geodesic segment, and  $J$  is any Jacobi field along  $\gamma$  such that  $J(0) = 0$ . For each  $c\in \mathbb{R}$ , let  $s(\cdot ,\cdot)$  be the function defined by Definition 4.
1) If all sectional curvatures of  $M$  are bounded above by a constant  $\kappa_{2}$ , then
$$
\| J (t) \| \geq s (\max  \{0, \kappa_ {2} \}, t) \| D _ {t} J (0) \|
$$
for all  $t \in [0, b_1]$ , where  $b_1 = b$  if  $\kappa_2 \leq 0$ , and  $b_1 = \min \left(b, \frac{\pi}{\sqrt{\kappa_2}}\right)$  if  $\kappa_2 > 0$ .
2) If all sectional curvatures of  $M$  are bounded below by a constant  $\kappa_{1}$ , then
$$
\left\| J (t) \right\| \leq s \left(\min  \{0, \kappa_ {1} \}, t\right) \left\| D _ {t} J (0) \right\|
$$
for all  $t \in [0, b_2]$ , where  $b_2$  is chosen so that  $\gamma(b_2)$  is the first conjugate point to  $\gamma(0)$  along  $\gamma$  if there is one, and otherwise  $b_2 = b$ .
E.2 Miscellaneous Technical Lemmas
Lemma 16. (Bacák, 2014, Section 2.1) Let  $\mathcal{M}$  be a Hadamard manifold,  $f:\mathcal{M}\to (-\infty ,\infty)$  be a gsc-convex lower semicontinuous function. Then any  $\beta$  -sublevel set of  $f$  ..
$$
\{\mathbf {x} \in \mathcal {M}: f (\mathbf {x}) \leq \beta \}
$$
is a closed gsc-convex set.
Lemma 17. (Bacák, 2014, Section 2.2) Let  $\mathcal{M}$  be a Hadamard manifold,  $f_{i}:\mathcal{M}\to (-\infty ,\infty)$  for  $i = 1,\ldots ,m$  be a series of gsc-convex lower semicontinuous functions. Then  $\sup_{i\in [m]}f_i(\mathbf{x})$  is gsc-convex.
Lemma 18. For a set  $\mathcal{K} \coloneqq \{\mathbf{x} | \max_{1 \leq i \leq m} h_i(\mathbf{x}) \leq 0\} \subseteq \mathcal{M}$  where each  $h_i(\mathbf{x})$  is gsc-convex,  $\mathcal{M}$  is Hadamard and any  $\mathbf{y} \notin \mathcal{K}$ , there exists  $\mathbf{g} \in T_{\mathbf{y}}\mathcal{M}$  such that
$$
\left\langle - \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \mathbf {x}, \mathbf {g} \right\rangle > 0, \quad \forall \mathbf {x} \in \mathcal {K}.
$$
Proof. By Lemma 17,  $\max_{1\leq i\leq m}h_i(\mathbf{x})$  is gsc-convex. Further, by Lemma 16,  $\kappa$  is a gsc-convex subset of  $\mathcal{M}$ . For some  $\mathbf{y}\notin \mathcal{K}$ , there exists  $j\in [m]$  such that  $h_j(\mathbf{y}) > 0$ . If not, we have  $h_i(\mathbf{y})\leq 0$  holds for any  $i = 1,\ldots ,m$ , which implies  $\mathbf{y}\in \mathcal{K}$  and leads to a contradictory. Applying the gsc-convexity for any  $\mathbf{x}\in \mathcal{K}$ , we have
$$
- \left\langle \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \mathbf {x}, \nabla h _ {j} (\mathbf {y}) \right\rangle \geq h _ {j} (\mathbf {y}) - h _ {j} (\mathbf {x}) > 0,
$$
where the second inequality is because  $h_j(\mathbf{y}) > 0$  and  $h_j(\mathbf{x}) \leq 0$ . This is a valid separation oracle between  $\mathbf{y}$  and  $\mathcal{K}$ .
Remark 4. Given  $\mathcal{K} = \{\mathbf{x}|\max_{1\leq i\leq m}h_i(\mathbf{x})\leq 0\}$  where each  $h_i(\mathbf{x})$  is gsc-convex and  $\mathbf{y}\notin \mathcal{K}$ , we can check the sign of  $h_i(\mathbf{y})$  for  $i = 1,\ldots ,m$  until we find  $i^{*}$  such that  $h_{i^*}(\mathbf{y}) > 0$ , then using the proof of Lemma 18, we obtain
$$
- \left\langle \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \mathbf {x}, \nabla h _ {i ^ {*}} (\mathbf {y}) \right\rangle > 0
$$