SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xiran Fan
fanxiran@ufl.edu
Chun-Hao Yang
Institute of Statistics and Data Science
chunhaoy@ntu.edu.tw
Baba C. Vemuri
Department of CISE
University of Florida vemuri@ufl.edu
Hyperbolic spaces have been quite popular in the recent past for representing hierarchically organized data. Further, several classification algorithms for data in these spaces have been proposed in the literature. These algorithms mainly use either hyperplanes or geodesics for decision boundaries in a large margin classifiers setting leading to a non-convex optimization problem. In this paper, we propose a novel large margin classifier based on horospherical decision boundaries that leads to a geodesically convex optimization problem that can be optimized using any Riemannian gradient descent technique guaranteeing a globally optimal solution. We present several experiments depicting the competitive performance of our classifier in comparison to SOTA.
Hyperbolic space, a non-Euclidean space with constant negative curvature, has been shown [25, 23, 29, 24] to be effective for representing hierarchically organized data. For example, authors in [25] showed that a tree can be embedded in a hyperbolic space with arbitrarily small distortion. The main reason for this is that a hyperbolic space can be regarded as a continuous version of trees – the volume of the space grows exponentially as one moves away from the center in hyperbolic space. This matches the growth pattern of the number of nodes in a tree which grows exponentially as the depth of the tree increases. Hyperbolic space embedding has been shown to be a promising approach for representing data with a (latent) hierarchical structure [23, 24, 29, 15].
Recently, representation of data in hyperbolic space for the fundamental tasks of unsupervised and supervised learning has been popularized in various contexts, e.g., dimensionality reduction [10, 13], clustering [22], large-margin classifier [12, 32, 11], regression [20], etc. Existing 'linear' classifiers in hyperbolic spaces are predominantly based on geodesics i.e., using geodesics as decision boundaries. In [12], the decision boundary is chosen to be the intersection of the hyperboloid model and a hyperplane in the ambient space, which in this case is the Minkowski space. Then, the support vector machine (SVM) in hyperbolic space is formulated as a nonconvex optimization problem. In [32], authors followed the same parameterization of the hyperbolic geodesic decision plane and provided a series of algorithms to provably learn large margin classifiers in hyperbolic space. However, as pointed out by [11], the algorithm in [32] fails to converge in practice. Authors in [11] used the Poincaré ball model and parameterized the geodesic decision plane as a hyperplane mapped using the exponential map from the tangent space at some reference point. They first constructed convex hulls for each data cluster in the hyperbolic space and the reference point is then chosen to be the midpoint
between different convex hulls. Then they apply a Euclidean perceptron/SVM algorithm to data lifted into the tangent space at the aforementioned reference point. Although the optimization problem in the tangent space is convex, the procedure of the tangent space approximation introduces inaccuracies and distortions. Moreover, convex hull learning is highly unstable and their implementation is only applicable to the 2-dimensional hyperbolic space.
Finally, it is worth mentioning that linear classification within hyperbolic space, which can be considered as the last layer, referred to as the hyperbolic logistic regression (LR), is a fundamental component of hyperbolic neural networks (HNNs) [16, 27]. The calculation of logits in this layer is based on the distances between samples and the geodesic decision boundary. Notably, this hyperbolic LR employs a geodesic decision boundary but is not a large-margin classifier.
1.1 Horospherical Decision Boundaries for Classification in Hyperbolic Sapce
Horospheres, which are the level sets of the Busemann function in hyperbolic spaces, are the analogs of Euclidean hyperplanes [4]. Horospheres (horocycles) are contained in the Poincaré ball (disk) and are tangential to the ball (disk) at an ideal point as shown in Figure 1. A collection of horospheres centered at the same ideal point are parallel to each other and the lengths of geodesic segments between two horospheres are all equal, just as the lengths of line segments between parallel hyperplanes in Euclidean space are all equal. This property of horospheres was explored by [9] to develop a dimensionality reduction method for data in hyperbolic space. By using horospherical projection, they are able to preserve the distance information in the original data. However, there is no literature on constructing a 'linear' classifier in hyperbolic space using horospheres as the decision boundaries although the horospheres are the hyperbolic equivalent of Euclidean hyperplanes. Therefore, it is natural to consider the use of horospheres as decision boundaries for classifi-
Figure 1: A 2-d Poincaré disk model  $\mathbb{B}^2$  and its boundary  $\partial \mathbb{B}^2 = \mathbb{S}^1$ . Given an ideal point  $\omega \in \partial \mathbb{B}^2$ , the black lines and curves are hyperbolic geodesics starting (ending) at  $\omega$  and the red circles are horocycles centered at  $\omega$ .
cation in hyperbolic spaces. In this work, we propose a novel hyperbolic large-margin classifier using horospheres as decision boundaries in the Poincaré model. We term this classifier as a HoroSVM. The horospheres are well-defined in the Poincaré ball model. A toy example as shown in Figure 2 demonstrates the advantage of horospherical decision boundaries over geodesic decision boundaries. As the tree-structured data grows in depth, leaf nodes are embedded closer to each other within a subtree and among different subtrees. One of the classification problems in hyperbolic space is to determine whether a node belongs to a chosen subtree given the embedding. For comparison purposes, the decision boundaries of HoroSVM (Figure 2(a)) and hyperboloid SVM [12] (Figure 2(b)) are shown in the figure. As evident, the horospherical decision boundary perfectly separates (the root node is excluded in training) the data while the geodesic decision boundary makes several mistakes on both positive and negative samples. We present a novel formulation of the classification problem in the hyperbolic space as a geodesically convex optimization problem on a Riemannian manifold. This optimization problem can be easily solved using any Riemannian gradient descent technique guaranteeing global optimality. Gradient-based optimizations for geodesically convex problems guaranteeing global optimal solutions are the topic of investigation in optimization literature and we refer the reader to [35] for detailed convergence analysis of several such optimization methods. Further, we empirically validate our method on several real and synthetic data sets.
It should be noted that a horosphere decision boundary has been used in some recent works [31, 28] in constructing HNNs. For example, authors in [31] proposed hyperbolic neuron models using the Busemann function as a generalization of the Euclidean inner product to extract horosphere features from data. Authors in [28] proposed a shallow fully-connected continuous network spanned by (hyperbolic) neurons, on noncompact symmetric space (including hyperbolic space) using the Helgason-Fourier transform. Authors in [33] produced Euclidean features from hyperbolic embeddings via the eigenfunctions of the Laplace operator in the hyperbolic space where the eigenfunctions involve horosphere features. Note that none of the above works developed a large margin classifier using the horosphere as a decision boundary. To the best of our knowledge, our work is the first in the
(a) Horospherical decision boundary
(b) Geodesic decision boundary
Figure 2: A balanced tree with depth 6 and spread 4 embedded in a 2-d Poincaré ball model using [15] is depicted in the figure. The orange plus node is the root of a chosen subtree, the blue dots are positive samples (nodes of the subtree) and the red dots are negative samples. (a) depicts a HoroSVM performance on the classification of positive and negative samples/nodes along with a zoomed-in version on its right. (b) depicts the geodesic boundary from the competing method, Hyperboloid SVM [12], along with the zoomed-in version to its right.
literature to present a convex optimization formulation of a large-margin classifier using a horosphere decision boundary in a hyperbolic space.
The rest of this paper is organized as follows. In Section 2, we present some background on hyperbolic geometry pertinent to the work presented here. In Section 3, we present our horospherical boundary-based classification methods. Experimental results are presented in Section 4 to demonstrate the advantage of our HoroSVM over competing hyperbolic classifiers. Finally, we conclude in Section 5.
In this section, we review some basic concepts of hyperbolic geometry including the generalization of the Euclidean hyperplane to the hyperbolic space namely, the horosphere.
2.1 Hyperbolic Space and the Poincaré Ball Model
There are five isometric models of the hyperbolic space: the Poincaré ball model, the Lorentz model, the Klein model, the upper-half space model, and the Hemisphere model [7]. We choose the Poincaré Ball model in this paper as it is easy to visualize and the Busemann function has a nice closed-form expression in this model. Note that the decision boundary of choice in our work is the level-set of the Busemann function namely, the horosphere.
An  $n$ -dimensional Poincaré Ball model, denoted by  $(\mathbb{B}^n,g_{\mathbb{B}})$ , consists of all points in an open ball of radius 1, i.e.,  $\mathbb{B}^n = \{\pmb {x}\in \mathbb{R}^n:\| \pmb {x}\| < 1\}$ , and equipped with the Riemannian metric  $g_{\mathbb{B}}(\pmb {x}) = 4(1 - \| \pmb {x}\| ^2)^{-2}g_{\mathbb{R}}$ , where  $\| \cdot \|$  is the Euclidean  $L_{2}$  norm, and  $g_{\mathbb{R}}$  is the Euclidean metric.
The geodesic distance between points  $\pmb{x}, \pmb{y} \in \mathbb{B}^n$  is  $d_{\mathbb{B}}(\pmb{x}, \pmb{y}) = \cosh^{-1}\left(1 + 2\frac{\|\pmb{x} - \pmb{y}\|^2}{(1 - \|\pmb{x}\|^2)(1 - \|\pmb{y}\|^2)}\right)$ .
2.2 Horospheres
Geodesics, geodesic rays, and ideal points The shortest path that connects two points in the Poincaré Ball model is called a geodesic segment. A geodesic ray is a geodesic segment that can be infinitely extended in one direction. We call the endpoint at infinity of a geodesic ray an ideal point. For  $\mathbb{B}^n$ , ideal points form the boundary of the ball:  $\partial \mathbb{B}^n = \mathbb{S}^{n-1} = \{x \in \mathbb{R}^n : \|x\| = 1\}$ , where  $\mathbb{S}^{n-1}$  is the  $(n-1)$ -dimensional hypersphere. The hypersphere  $(\mathbb{S}^{n-1}, g_{\mathbb{S}})$  is a Riemannian manifold equipped with the Riemannian metric  $g_{\mathbb{S}} = 4(1 + \|x\|^2)^{-2} g_{\mathbb{R}}$ .
Busemann function [6] Let  $\omega \in \partial \mathbb{B}^n$  be an ideal point and  $\gamma_{\omega}:[0,\infty)\to \mathbb{B}^n$  a geodesic ray pointing  $\omega$ . The Busemann function is defined as
$$
b _ {\boldsymbol {\omega}} (\boldsymbol {x}) = \lim  _ {t \rightarrow \infty} (d (\gamma_ {\boldsymbol {\omega}} (t), \boldsymbol {x}) - t), \quad \boldsymbol {x} \in \mathbb {B} ^ {n}. \tag {1}
$$
In the Poincaré Ball model, Eq. (1) has a closed form:  $b_{\omega}(\pmb{x}) = -\log \frac{1 - \| \pmb{x} \|^2}{\|\pmb{\omega} - \pmb{x}\|^2}$ .
Horospheres [6] In  $\mathbb{B}^n$ , a horosphere is a  $(n - 1)$ -dimensional sphere that is internally tangent to  $\partial \mathbb{B}^n$  at an ideal point. For a given  $\omega \in \partial \mathbb{B}^n$ , the level sets of Busemann function  $b_{\omega}(x)$  in the Poincaré ball model is a series of horospheres tangent at  $\omega$ . Horospheres are hyperbolic hyperplanes in the sense that the corresponding construction in Euclidean space gives a hyperplane.
Given an ideal point  $\omega \in \partial \mathbb{B}^n$ , the function defined by  $\langle \omega, x \rangle_B : x \mapsto -b_\omega(x)$  is constant over horosphere tangent at  $\omega$ . Hence any horosphere can be parameterized with an ideal point  $\omega$  and an offset value  $b$  of the level set.
Let  $\Pi$  denote the set of horospheres of  $\mathbb{B}^n$ . A horosphere  $\pi \in \Pi$  can be parameterized by  $0 < \mu \in \mathbb{R}^{+}$ ,  $\omega \in \mathbb{S}^{n - 1}$ , and  $b \in \mathbb{R}$  as
$$
\pi_ {\mu , \omega , b} := \left\{\boldsymbol {z} \in \mathbb {B} ^ {n} \mid \mu \langle \boldsymbol {\omega}, \boldsymbol {z} \rangle_ {\mathbb {B}} - b = 0 \right\}. \tag {2}
$$
We will use  $\pi$  or  $\pi$ . to represent a horosphere in different parameterizations hereafter.
3 Horospherical Boundary-based Classification
In this section, we present the key theoretical contributions of our work namely, a horosphere-based SVM classifier that involves formulating and solving a geodesically convex optimization problem. First, we present some preliminary facts and results about the horospheres. Then, we present the optimization problems for the horospherical perceptron and SVM along with analysis.
3.1 Point to Horosphere Distance
While the distance from the origin  $\pmb{o}$  to a horosphere has been known for decades [18] (Introduction 4.1, p.31), we present a natural generalization of previous results by providing a closed-form expression for measuring the hyperbolic distance from any arbitrary point  $\pmb{x} \in \mathbb{B}^n$  to a given horosphere  $\pi_{\mu,\omega,b}$ . The following remark provides this result.
Proposition 3.1. Let  $\pi_{\mu, \omega, b}$  be a horosphere. The hyperbolic distance of a point  $x \in \mathbb{B}^n$  to a horosphere  $\pi_{\mu, \omega, b}$  is given by
$$
d _ {\mathbb {B}} (\boldsymbol {x}, \pi_ {\mu , \boldsymbol {\omega}, b}) = \frac {| \mu \langle \boldsymbol {\omega} , \boldsymbol {x} \rangle_ {\mathbb {B}} - b |}{\mu}. \tag {3}
$$
Notice that it shares a similarity to the Euclidean distance of a point to a hyperplane. Before presenting the proof for Proposition 3.1, we recall the following Fact 3.2 and Lemma 3.3 from [31].
Fact 3.2. Given an ideal point  $\omega$  and a point  $x \in \mathbb{B}^n$ , there is a unique horosphere passing through  $x$  and tangent at  $\omega$ .
Lemma 3.3. [31] Let  $\Pi_{\omega}$  be the set of horocycles of  $\mathbb{B}^n$  tangent at  $\omega$ . Given  $\lambda \in \mathbb{R}$ , let  $\pi_{\lambda,\omega}$  be the unique horosphere that passes through  $\tanh(\lambda/2)\cdot\omega$  and tangent at  $\omega$ . Note that  $\Pi_{\omega} = \cup_{\lambda \in \mathbb{R}}\{\pi_{\lambda,\omega}\}$ . We have the following two results: (i) the hyperbolic lengths of geodesic (that pass through  $\omega$ ) segments between  $\pi_{\lambda_1,\omega}$  and  $\pi_{\lambda_2,\omega}$  are equal to  $|\lambda_1 - \lambda_2|$ ; (ii)  $\langle \omega, x \rangle_{\mathbb{B}} = \lambda$  for any  $x \in \pi_{\lambda,\omega}$ .
Figure 3 shows a 2D Poincaré disk model  $\mathbb{B}^2$  and its boundary  $\mathbb{S}^1$ . The point  $o$  is the origin of the disk,  $\pmb{x} \in \mathbb{B}^2$  is a point, and  $\omega \in \mathbb{S}^1$  is a point at infinity (an ideal point). Two geodesics ending at the same  $\omega$  from  $\pmb{x}$  and  $o$  respectively are shown in the figure (black solid line/curve). The circle  $\pi_{\mu,\omega,b}$  is a given horocycle tangent (red solid circle) at  $\omega$ . The hyperbolic distance  $d_{\mathbb{B}}(\pmb{x}, \pi_{\mu,\omega,b})$  between  $\pmb{x}$  and  $\pi_{\mu,\omega,b}$  is identified as the distance between  $\pmb{x}$  and  $\pmb{y}_x$  where  $\pmb{y}_x$  is the projection of  $\pmb{x}$  to  $\pi_{\mu,\omega,b}$  along the geodesic ending at  $\omega$ . Let  $\pi^{\pmb{x}}$  (red dashed circle) be the unique horocycle that passes through  $\pmb{x}$  and is tangent at  $\omega$ . Note that the lengths of all geodesic segments between two horocycles are the same. That is,  $d_{\mathbb{B}}(\pmb{x}, \pi_{\mu,\omega,b}) = d_{\mathbb{B}}(\pmb{x}, \pmb{y}_x) = d_{\mathbb{B}}(\pmb{x}_0, \pmb{y}_0)$ , where  $\pmb{x}_0, \pmb{y}_0$  are horocyclic projections [9] of  $\pmb{x}, \pmb{y}_x$  along  $\pi^{\pmb{x}}$  and  $\pi_{\mu,\omega,b}$  respectively.
<sup>1</sup>Note that the Euclidean inner product  $\langle \pmb{w}, \cdot \rangle$  is constant over a hyperplane that is perpendicular to a given direction  $\pmb{w}$ .
Parameter  $\mu$  is included for simplicity in analysis later on (Eq 3, 12)
Proof of Proposition 3.1. Given a point  $\pmb{x} \in \mathbb{B}^n$  and an ideal point  $\omega \in \mathbb{S}^{n-1}$ , let  $\pi \in \Pi_{\omega}$  be a horosphere tangent at  $\omega$  and  $\pi^x \in \Pi_{\omega}$  be the unique horosphere that passes through  $\pmb{x}$ . Write  $\langle \omega, \pmb{x} \rangle_{\mathbb{B}} = \lambda_x$  and then  $\pmb{x}_0 = \tanh(\lambda_x / 2) \cdot \omega$  is the horospherical projection of  $\pmb{x}$  along  $\pi_x$ . For consistency in notation, we write  $\pi^x$  as  $\pi_{\lambda_x, \omega}$ . Since the horosphere  $\pi_{\mu, \omega, b}$  can be reparameterized as  $\pi_{\lambda, \omega}$ , where  $\lambda = \frac{b}{\mu}$ , the hyperbolic distance from  $\pmb{x}$  to  $\pi_{\mu, \omega, b}$  is the hyperbolic distance between  $\pi_{\lambda_x, \omega}$  and  $\pi_{\lambda, \omega}$  which is  $|\lambda_x - \lambda| = \frac{|\mu\langle\omega, x\rangle_{\mathbb{B}} - b|}{\mu}$  (by Lemma 3.3). This completes the proof.
3.2 Horospherical Decision Boundaries
We consider classification problems in hyperbolic space of the following form:  $\mathcal{X} \subset \mathbb{B}^n$  denotes the feature space and  $\mathcal{Y} = \{\pm 1\}$  denotes the binary label space. In the following, we denote the training set by  $S \subset \mathcal{X} \times \mathcal{Y}$ . The decision rule using a horosphere as its decision boundary can be written as the following function  $f: \mathcal{X} \mapsto \mathcal{Y}$  where
$$
f (\boldsymbol {x}; \mu , \boldsymbol {\omega}, b) = \operatorname {s i g n} \left(\mu \langle \boldsymbol {\omega}, \boldsymbol {x} \rangle_ {\mathbb {B}} - b\right). \tag {4}
$$
The positive samples are expected to lie inside a horosphere while the negative samples are expected to lie outside a horosphere. This is analogous to the linear decision boundary in Euclidean space and we will build a horospherical perceptron and a horospherical SVM based on this decision boundary.
It should be noted that in  $\mathbb{R}^n$  the hyperplane  $\xi_{a,\pmb{w},b} = \{\pmb{z}\in \mathbb{R}^n |a\langle \pmb {w},\pmb {z}\rangle -b = 0\}$  where  $a\in \mathbb{R}^{+},b\in \mathbb{R}$ ,  $\pmb {w}\in \mathbb{S}^{n - 1}$  is the hyperplane  $\xi_{a, - w, - b}$ . However, the horospheres  $\pi_{\mu ,\omega ,b}$  and  $\pi_{\mu , - \omega , - b}$  respectively represent two distinct horospheres, centered at  $\pmb{\omega}$  and  $-\omega$  respectively. Let  $\Pi^{+} = \{\pi_{\mu ,\omega ,b}\in \Pi |b > 0\}$  and  $\Pi^{-} = \{\pi_{\mu ,\omega ,b}\in \Pi |b < 0\}$ . Thus,  $\Pi^{+},\Pi^{-}\subset \Pi$  and the radius of  $\pi \in \Pi^{+}$  is less than  $1 / 2$  and the radius of  $\pi \in \Pi^{-}$  is greater than  $1 / 2$ . In most cases of classification in hyperbolic space, the positive samples are clustered near the boundaries. Hence, we restrict ourselves to finding a horosphere  $\pi \in \Pi^{+}$  that separates data, instead of searching over  $\Pi$ . Intuitively, we are looking for a 'small' horosphere that captures the positive samples. We are now ready to present the Horospherical Perceptron followed by the Horospherical SVM.
Figure 3: The relationship between horocycles, geodesic, and the horocyclic projections in  $\mathbb{B}^2$ .
3.3 Horospherical Perceptron
The loss function for the proposed horospherical perceptron is given by
$$
l (\mu , \omega , b; \boldsymbol {x}, y) = \max  (0, - y \cdot (\mu \langle \boldsymbol {\omega}, \boldsymbol {x} \rangle_ {\mathbb {B}} - b)), \quad (\mu , \omega , b) \in \mathbb {R} ^ {+} \times \mathbb {S} ^ {n - 1} \times \mathbb {R} ^ {+}, \tag {5}
$$
which is zero when the instance is classified correctly and is proportional to the signed distance of the instance from the horosphere when it is misclassified. The empirical loss for a given data set  $S$  is
$$
L (\mu , \boldsymbol {\omega}, b) = \frac {1}{| S |} \sum_ {\{\boldsymbol {x}, y \} \in S} l (\mu , \boldsymbol {\omega}, b; \boldsymbol {x}, y) \tag {6}
$$
Hence, the optimal horosphere is learned by solving the above optimization problem on the manifold  $\mathbb{R}^{+}\times \mathbb{S}^{n - 1}\times \mathbb{R}^{+}$ , i.e.,
$$
\mu^ {*}, \omega^ {*}, b ^ {*} = \arg \min  _ {(\mu , \omega , b)} L (\mu , \omega , b) \tag {7}
$$
To further analyze this optimization problem, we first recall some facts about geodesic convexity.
Definition 3.4. (Geodesically convex sets [30]). Let  $(\mathcal{M},g)$  be a Riemannian manifold. A set  $A\subseteq \mathcal{M}$  is said to be a geodesically convex set if, for any two points  $\pmb {p},\pmb {q}\in A$  , the geodesic  $\gamma_{pq}$  that connects them is contained in  $A$
Definition 3.5. (Geodesically convex/concave functions [30]) Let  $A \subseteq \mathcal{M}$  be a geodesically convex set. A function  $f: A \to \mathbb{R}$  is said to be a geodesically convex function if, for any  $\pmb{p}, \pmb{q} \in A$  the composition  $f \circ \gamma_{pq}: [0,1] \to \mathbb{R}$  is a convex function, where  $\gamma_{pq}: [0,1] \to \mathcal{M}$  is a geodesic that connects  $\pmb{p}, \pmb{q}$ .  $f$  is said to be a geodesically concave function if  $-f$  is a geodesically convex function.
Theorem 3.6. [30] Let  $\mathbf{A} \subseteq \mathcal{M}$  be a geodesically convex set. A function  $f: \mathbf{A} \to \mathbb{R}$  is geodesically convex if and only if its epigraph  $\operatorname{epi}(f) = \{(p, c) | f(p) \leq c\} \subset \mathbf{A} \times \mathbb{R}$  is a convex set.
Now we present the main theoretical result of this paper.
Theorem 3.7. For a given training data sample  $\{\pmb{x},\pmb{y}\} \in S,0 < \| \pmb{x}\| < R < 1$  (the hyperbolic feature  $\pmb{x}$  neither lie on the center nor lie on the boundary),  $l(\mu ,\omega ,b;\pmb {x},\pmb {y})$  is a geodesically convex function on  $\mathbb{R}^+ \times \pmb {A}\times \mathbb{R}^+$  and is a geodesically concave function on  $\mathbb{R}^+ \times \pmb {B}\times \mathbb{R}^+$ , where
$$
\boldsymbol {A} = \left\{\boldsymbol {\nu} \in \mathbb {S} ^ {n - 1} \middle | y \cdot \frac {\boldsymbol {x} ^ {T} \boldsymbol {\nu}}{\| \boldsymbol {x} \|} > 0 \right\} \subset \mathbb {S} ^ {n - 1}, \quad \boldsymbol {B} = \left\{\boldsymbol {\nu} \in \mathbb {S} ^ {n - 1} \middle | y \cdot \frac {\boldsymbol {x} ^ {T} \boldsymbol {\nu}}{\| \boldsymbol {x} \|} <   0 \right\} \subset \mathbb {S} ^ {n - 1}. \tag {8}
$$
Note that both  $\mathbf{A}$  and  $\mathbf{B}$  are geodesically convex sets.
Proof. Let  $l(\mu, \omega, b; \boldsymbol{x}, y) = \max(0, -g(\mu, \omega, b; \boldsymbol{x}, y))$  where  $g(\mu, \omega, b; \boldsymbol{x}, y) = y \cdot (\mu \langle \omega, \boldsymbol{x} \rangle_{\mathbb{B}} - b)$ . Since  $\max(0, -a)$  is a convex function in  $a \in \mathbb{R}$  and  $g(\cdot)$  is linear in  $\mu$  and  $b$ , we only need to show that  $g(\cdot)$ , as a function of  $\omega \in \mathbb{S}^{n-1}$ , is geodesically convex (concave). Without loss of generality, let  $\mu = 1$  and  $b = 0$ . Also note that  $y$  is the label of data that takes values from  $\{-1, +1\}$  which may flip the inequality. It suffices to validate results for the positive sample, i.e.,  $y = 1$ .