Hyperbolic Procrustes Analysis Using Riemannian Geometry
Authors: Ya-Wei Eileen Lin
Yuval Kluger
Ronen Talmon
Venue: NeurIPS 2021 Poster (2021)
Keywords: Label-free data alignment
hyperbolic space
Riemannian geometry
Procrustes analysis
batch correction
domain adaptation

Abstract: Label-free alignment between datasets collected at different times, locations, or by different instruments is a fundamental scientific task. Hyperbolic spaces have recently provided a fruitful foundation for the development of informative representations of hierarchical data. Here, we take a purely geometric approach for label-free alignment of hierarchical datasets and introduce hyperbolic Procrustes analysis (HPA). HPA consists of new implementations of the three prototypical Procrustes analysis components: translation, scaling, and rotation, based on the Riemannian geometry of the Lorentz model of hyperbolic space. We analyze the proposed components, highlighting their useful properties for alignment. The efficacy of HPA, its theoretical properties, stability and computational efficiency are demonstrated in simulations. In addition, we showcase its performance on three batch correction tasks involving gene expression and mass cytometry data. Specifically, we demonstrate high-quality unsupervised batch effect removal from data acquired at different sites and with different technologies that outperforms recent methods for label-free alignment in hyperbolic spaces.

Hyperbolic Procrustes Analysis Using Riemannian Geometry
Ya-Wei Eileen Lin
Yuval Kluger
Ronen Talmon
Viterbi Faculty of Electrical and Computer Engineering, Technion
*Program in Applied Mathematics, Yale University
Interdepartmental Program in Computational Biology and Bioinformatics, Yale University Department of Pathology, Yale University (lin. ya- jaa-we@ccmpuss ronendee} technion. aww.iinnncc..... ac.il, i1, yuval. hal..kkluerrraee.. edu
Abstract
Label-free alignment between datasets collected at different times, locations, by different instruments is a fundamental scientific task. Hyperbolic spaces have recently provided fruitful foundation for the development of informative repre- sentations of hierarchical data. Here, we take a purely geometric approach for label-free alignment of hierarchical datasets and introduce hyperbolic Proerustes analysis (HPA). HPA consists of new implementations of the three prototypical Procrustes analysis components: translation, scaling, and rotation, based the Riemannian geometry of the Lorentz model of hyperbolic space. We analyze the proposed components, highlighting their useful properties for alignment. The efficacy of HPA, its theoretical properties, stability and computational efficiency are demonstrated in simulations. In addition, we showcase its performance on three batch correction tasks involving gene expression and mass cytometry data. Specifically, we demonstrate high-quality unsupervised batch effect removal from data acquired at different sites and with different technologies that outperforms recent methods for label-free alignment in hyperbolic spaces.
Introduction
A key scientific task in modern data analysis is the alignment of data. The need for alignment often arises since data are acquired in multiple domains, under different environmental conditions, using various acquisition equipment, and at different sites. This paper focuses on the problem of fabel-free alignment of data embedded in hyperbolic spaces. Recently, hyperbolic spaces have accentuated in geometric representation learning. These -un-uulideen spaces have become popular since they provide a natural embedding of hierarchical data thanks to the exponential growth of the lengths of their geodesic paths [4]] 42
B0
6 32
The problem of alignment of data embedded in hyperbolic spaces has been extensively studied, e.g., in the context of natural language processing A9) ontology matching (10), matching two data modalities [40] and improving the embedding in hyperbolic spaces [ A few of these studies are based optimal transport (OT) [2] 22). classical problem in mathematics Bا that has recently reemerged modern data analysis, e.g., for domain adaptation [7]. Despite its increasing usage, OT for unsupervised alignment is fundamentally limited [54], since OT (as any density matching approach) cannot recover volume preuee--eeerving maps
17 18 that is based on purely geometric considerations. PA has been widely used for aligning datasets by eliminating the shift, scaling, and rotational factors. Over the years, it has been successfully applied to various applications, e.g.
In this paper, we resort to Procrustes analysis (PA)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

image registration 341 manifold alignment [52], shape matching [31, domain adaptation 47 and manifold learning
name but few. Here, we address the problem of label-free matching of hierarchical data embedded in hyperbolic spaces. We present hyperbolic Procrustes analysis (HPA), a new PA method in the Lorentz model of hyperbolic geometry. The main novelty lies in the introduction of new implementations of the three prototypical PA components based on Riemannian geometry. Specifically, translation is viewed as a Riemannian mean alignment, implemented using parallel transport (PT). Scaling is determined with respect to geodesic paths. Rotation is considered as moment alignment on mapping of the tangent space of the manifold to a Euclidean vector space. Our analysis provides new derivations in the Riemannian geometry of the Lorentz model and specifies the commuting properties of the HPA components. We show that HPA, compared to existing baselines and oT-based methods, achieves improved alignment in a purely unsupervised setting. In addition, it has a natural and stable out-of-sample extension, it supports both small and big data, and it is computationally efficient.
We show application to batch correction in bioinformatics tasks. We present results on both gene expression and mass cytometry (CyTOF) data, exemplifying the generality and broad scope of our method. In contrast to recent works 28 50 our method does not require landmark correspondence. which is often unavailable in many datasets or hard to obtain. Specifically, we show that batch effects caused by acquisition using different technologies, at different sites, and at different times can be accurately removed, while preserving the intrinsic structure of the data.
Our main contributions are as follows. (i) We present new implementation of PA using the Riemannian geometry of the Lorentz model for unsupervised label-free hierarchical data alignment. (ii) We provide theoretical analysis and justification of our alignment method based on new derivations of Riemannian geometry operations in the Lorentz model. These derivations have their own merit as they could be used in other contexts. (iii) We show experimental results of accurate batch effect removal from several hierarchical bioinformatics datasets without landmark correspondence.
2 Background on hyperbolic geometry
Hyperbolic space is a non-Euclidean space with a negative constant sectional curvature and an underlying geometry that describes tree-like graphs with small distortions [6 'hhre exist four commonly-used models for hyperbolic spaces: Poincaré disk model, Lorentz model (hyperboloid model), Poincaré half-plane model, and Beltrami-Klein model. These four models equivalent and
there exist transformations between them. Here, we consider the Lorentz model, and specifically, the upper sheet of the hyperboloid model, because its basic Riemannian operations have simple closed-form expressions and the computation of the geodesic distances is stable 42 30
Formally, the upper sheet of the hyperboloid model in a d-dimensional hyperbolic space is defined by L = { E dd (x, x) = -1 x11 > 0}. where (I, x) = x H is the Lorentzian inner product and H E R [dd+1)x(d+1) is defined by H = [-1.0 :0, Ia]. The Lorentzian norm of a V(x,
hyperbolic vector æ E Ld is denoted by ||0||c Let TaLd be the tangent space at x E Ld. defined by TaLd = (v| v) = 0}. Consider x E I and u E TaL,d, the geodesic path Φ : TR.jt + Ld is defined by s(t) = cosh(] (|llct)))) - (iuh((/o)ee() ालîlz with (0)) = x and initial velocity s0(0) = v. where o'(1) = (l)). In addition, the associated geodesic distance is (x, (t)) = cosh -1 (-(xz, de (t))
with the origin ps = [1, 0 IT E Ld.
The Exponential map. projecting a point E T to the manifold Ldd, is given by Expa (v) (11 = cosl([|e)lcc::: + sinh([[o|le))ppiz The Logarithmic map, projecting a point y E LL to the tangent space TgLd at æ, is defined by Log (v) = cosh ((A) (y- - Aæ), where A = - (w. v)
-x=-1
The PT of a vector v E TaLL along the geodesic path from x E L to y E Ld is defined by (T)- = + - xw. (x + y). where À = ((x v)c. keeping the metric tensor unchanged. X+1
The Riemannian mean x and the corresponding dispersion dx of set X' (2i/20 E Ld) are defined using the Fréchet mean 13 53 by
n
xx == m(x) = arg min
Σ dh (x, 2.) and dx == r(x)= = min Σ
H. x:), (1)
rELd
xELd nI
1=1
i=1
2

---

Our HPAA
Other methods
Given data
Feey Reraetin
Shop RIPNERS
Sip Reeate
PAB
HOTF (54)
HOT-L[[2]]
HOT-ME [22]
Illustration of our alignment method (HPA). Two sets of points in hyperbolic space are given, depicted in dark and light colors. Each point is associated with one of two labels: (i) blue/cyan, and (ii) red/pink. Left: Alignment results after each step our HPA implemented in T.2: Riemannian translation, Riemannian scaling, and Riemannian wrapped rotation. Note that these Riemannian operations in hyperbolic space are different than their Euclidean counterparts. Right: Alignment results of four methods: PAH [51] and HOT-F [4l) applied in L and HOT-L 22 and HOT-ME 22 applied in the 2D Poincaré disk. For visualization, all points in LL are transformed to the 2D Poincaré disk. The alignment after Step 3 of HPA (circled in black) is more accurate than the alignments obtained by the other methods.
Figure 1:
where m Idd and r: X RR+ Note that the Fréchet mean of samples on connected and compact Riemannian manifolds of non-positive curvatures, such as hyperbolic spaces, is guaranteed to exist, and it unique 24 M.. The Fréchet mean is commonly computed by the Karcher Flow
24 20 which is computationally demanding. Importantly, in the considered hyperbolic space, the Fréchet mean can efficiently obtained using the accurate gradient formulation 33
Given vector E E L and symmetric and positive-definic (SPD) matrix E E Raxd, d the wrapped normal distribution G Σ) provides generative model of hyperbolic samples as follows [39 [.. First, a vector o' is sampled from (O, Σ). Then, 0 is concatenated to the vector e' such that
= [0, v'JT E L. Finally, PT from the origin μ40 = [1, 0 J to w is applied to v, and the resulting point mapped the manifold using the Exponential map at The probability density
function of this model is given by log 9 (yla, Σ) = log. NO, Σ) (n 1) log( sinh 1 1ol
1
3 Hyperbolic Procrustes analysis
Existing methods for data alignment typically seek a function that minimizes a certain cost. large body of work attempts to match the empirical densities of two datasets. e.g., by minimizing the maximum mean discrepancy (MMD) 48 2
or solving OT problems 45 D2] Finding an
effective cost function without labels or landmarks is challenging, and minimizing such costs directly often lead to poor alignment in practice (see illustration in Fig. D. different well-established approach that applies indirect alignment based on geometric considerations is PA. While preparing this manuscript, another method of PA in hyperbolic spaces (PAH) was presented for matching two sets, assuming that they consist of the same number of points and that there exists point-wise isometric map between them [T)). We remark that the analysis we present here applies to broader settings and makes such assumptions. See Appendix D for details on classical PA well for comparisons to [5 and to the application of Euclidean PA in the tangent space.
We consider two sets of points ( = (h, (1) y J and H [h, J in Ld, Here, we aim to find a function ( [d + Ldd consisting of three components: translation, scaling, and rotation, that aligns H(2) with ((1) in an unsupervised label-free manner as depicted in Fig. Finding such a function can be viewed as extension of classical PA from the Euclidean space g+++ to the Lorentz model Ld. A natural extension to multiple sets is described Section 3.5 We remark that the statements written in the context of the problem at hand. In Appendix present their proofs.
restate them more generally and
3.1 Riemannian translation
Let h and (2) denote the Riemannian means of the sets H(1 and 2(2) respectively. In this translation component, we find map 1 (1)) Ld + Ld that aligns the Riemannian means of
_

---

the sets. In the spirit of 5 53 we propose to construct T A _ (h!²²) the composition of three
Riemannian operations in _ the Logarithmic map applied to h at h (2) PT h from h (2)
to
h along the geodesic path, and the Exponential map applied to the transported point at h ht")
P R
F (h (1= = Exp (PT (Loggr))) (h. 2 ))).
(2)
See Fig. B.I in Appendix
B for illustration. Since the geodesic path between any two points in T - a is well-defined. The rationale behind the combination of these three
L is unique 46 Riemannian operations twofold. First, PT is map that aligns the means of the sets, while preserving their internal structure. Second, the Logarithmic and Exponential maps compose a map whose domain and range the Lorentz model I,d rather than the tangent space, as desired. We make these claims formal in the following results.
Proposition 1. The map I h -h defined in Eq.
Q aligns the means of the sets, i.e., it satisfies
m([ 19
-h (2) (h: (2)) )
(3)
where m. is the function defined in Eq. Proposition 2. The map I 12 -/ (2) for all h
(2)
(2) (2)) can be recast as:
T T K (h. (h = h 3(h
(1) (2) + +
(2) h h
(4)
where the functions B and y are positive, defined by 0 < B(h
(2)
h
(2) - n th (2)
a+l h
and 0 < 7(,
h h2 =
( (20+
1
h (2)
respectively, and 0 < a =
a+1
(h (2)
In addition to providing a compact closed-form expression, Prop. gives the proposed translation based on Riemannian geometry an interpretation of standard mean alignment in linear vector spaces. It implies that the alignment is nothing but subtracting the mean of the source set (2)2) from each vector in 2222), and adding the mean of the target set F (with the appropriate scales). Proposition 3. The map I (2 -p preserves distances (i.e., is an isometry):
dh (2) h dr. R 0 (3) x - (h,
(5)
for any two points h (2) h (2)
E (222)
Let (() be the unique geodesic path from h (2) to h such that (0)) = h (2) let (0) E T aa and (1) E T be the corresponding velocities, respectively.
and (1) = h2) and
Proposition 4. The map I (p(")="""" aligns geodesic velocities, i.e., given the mapping of the geodesic
velocities to the manifold Ld vo Exp, (bb (0)) T and L 3 v2 " (1)), we have
" (wo) 01-
(6)
Isometry determined up to rotation, a fact that can be problematic for alignment. For example, any H-unitary matrix [16] can be an isometric function in I.d. When landmarks are given, they can be
used to alleviate this redundancy. However, in the purely unsupervised setting we consider, other
data-driven ques required. Prop. iimples that the proposed translation based on PT fixes some
of these rotational degrees of freedom by aligning the geodesic velocities. In Section 3.3
we revisit
this issue. Now, with a slight abuse of notation, let 72 r - (H(
Proposition 5. Consider two subsets A, B C 22)) and their translations Ā = N LR au (B) C. 122)) Let a = m(A), = m(B), ā = a and b = mB3) be the Riemannian means of the subsets. Then,
(A),
I ( o _ aa-05 = L -Ta 1 __))) ())

---

such that In the context of the alignment problem, the importance of Prop. is the following, Suppose the two sets correspond to data measured at two labs (denoted with and without tilde), and suppose each set was acquired by two types of equipment (denoted by A and B). Prop. implles that aligning data from the different labs and then aligning data acquired using the different equipment is equivalent to first aligning the different equipment and then the different labs, i.e., any order of the two alignments generates the same result. Seemingly, this is a natural property in Euclidean spaces. However, in a Riemannian manifold, it is not trivial result, and it holds for the transport along the geodesic path. See Appendix for counter-examples.
3.2 Riemannian scaling
Let d and d(²) denote the Riemannian dispersions of 7((1) and 22(2) By propositions and h (
and d(3) are the mean and dispersion of (2(2), Here, our goal is align the Riemannian dispersions of H(1) and H2 For this purpose, we propose the scaling function r 0 :Ld + L given by
r m (h; ( = (i(s),
(8)
where v d0)) /d(2) the scaling factor and (b) is the geodesic path from h (1)
(2) h
(0( = h and (1) = h, See Fig. B.2 in Appendix B for illustration.
Proposition 6. The dispersion of the rescaled set H2))
- (H((2) ) d(2))
3.3 Riemannian wrapped rotation
The purpose of this component to align the orientation of the distributions of the two sets after translation and scaling, namely, after aligning their first and second moments. The proposed rotation function A,0 : I + La consists of (i) mapping the points from the manifold L to the tangent space T, L2, (ii) mapping to Rd, (iii) rotating in RR and (iv) mapping back to the tangent space and then to the manifold. We perform the rotation in Ed which we term wrapped rotation, rather than direct rotation on the manifold L or on the tangent space T ILd for the following reasons.
First, the frequently used rotation map in Ld 51 does not necessarily preserve the Riemannian mean, and in our context, it might reverse the mean alignment. Second, rotation applied directly to the tangent space T Ld does not guarantee that the rotated points remain on the same tangent space. Third, applying rotation in the Euclidean vector space that is isometric to the tangent space is less efficient and stable, and it obtains slightly worse empirical results (see Appendix D.4 for details). Last, applying the rotation in IRd allows to use the standard Euclidean rotation using S SVD. In Section M alternatives.
we empirically demonstrate the advantage of the proposed rotation compared to the
Definition 1. Let the mapping function PrO) : :T Bdd defined on the langent space at h (1)
and its inverse map be the following functions defined by
hh (T
(s,
P
P (v) [v(()..... ...2).......... v(d+1) 1)]_ Rd and P n
(s)
(1) ,
h
where 8 E Rd and (...) is the standard Euclidean inner product.
Note that removing the first element of v is valid due to the constraint imposed on the vector elements in the tangent space by definition. Indeed, no information is lost and the mapping is invertible.
The first step in our rotation component is to map the points in (1) and in 72)2 to the tangent space (2) for 1...., N₂. In
at h (1), ²
= Log, (1) (h(") for i = 1,..., N1, and of = Log, (1) (h.
the second step, we map the points by the mapping function in Definition I and re-center them:
(1) P (0 (1) s1)) i ...... N₂ and s(²) PP,.' (x: (v( (2) 1,..., N₂. where
((k) NE DN, ( (b) for 1, is the mean vector of the projections. Then, the mapped
and centered points (in Rd) are collected into matrices:
((k) [s] (k) (k)
8
NoJ E dd
(10)
5

---

In the third step, for each set k = 1.2, we compute the rotation matrix U(A) E Rdxd by applying
(8) ( Since the left-singular vectors are determined up to SVD to the matrix S( = ( (
sign, we propose align their signs as follows: u sign((u, (2) ( )u, ( where u!!h and u
the i-th left-singular vector of the two sets, resulting in modified rotation matrices Utk), Finally, we apply the rotation to 7((2) by
O h
(h
EXp (P-d) (UT(PR (Log (h, s(0) x(1))
(11)
where = U (U)
Proposition? 7. The wrapped rotation is bijective, and the inverse given by
(0 (0 = 6 (0)
(12)
3.4 Analysis
Putting all three components together, the proposed HPA that aligns 72 with H() culminates in the composition of translation, scaling, and rotation:
OU F
rs 5
+h (1).
(13)
As in most PA schemes, the order of the three components is important. Yet, the proposed components allow certain degree of freedom, indicated in the following results. Proposition 8. The Riemannian translation and the Riemannian scaling commute w.r.f the Rieman-
nian means h (1) and 22
r- R
-T r T
(2)
(14)
Note that Y (1)) and I (00 (2) m1 do not necessarily commute: Ts Proposition 9. The Riemannian scaling and the wrapped rotation commute:
a 0) # 19 ( r
T aa o UU a) = = a1 A a0)
r
(15)
We note that the rotation does not commute with the translation, because PT only preserves the local covariant derivative on the tangent space but might cause rotation and distortion along the transportation. Therefore, the rotation is required to be the last component of our HPA.
Thus far, we did not present a model for the discrepancy between the two sets, nor we presented the proposed HPA as optimal with respect to some criterion. In the following result, we show that if the discrepancy between the sets be expressed a composition of translation, scaling, and rotation, then the two sets can be perfectly aligned using HPA.
Proposition 10. Let v) L - La be a mup, given by 1) UU (h (((h N3, then,
r h o T TT . I
(2 = (OU'
A o r R
o h m (1) i= I,...,,22
(16)
where U' Od).
Note that HPA consists of the sequence of Riemannian translation, Riemannian scaling and wrapped- rotation. The domain and range of each component is the manifold Ld. Yet, the first and last operations of each component are the Logarithmic and Exponential maps that project a point from the manifold to the tangent space, and vice versa, respectively. This allows us to propose an efficient implementation of the sequence without the back and forth projections as described in Appendix C
3.5 Extension to multiple sets
We naturally scale up the setting to support the alignment of K > 2 sets, denoted by H(R) {h
(8) NN where & E (1, K}. Let h (k) and (k)) be the Riemannian mean and dispersion of

---

Algorithm 1 Hyperbolic Procrustes analysis
Input: K sets of hyperbolic points H(1) (h: (h, ) :.. Hik) (h1 ),1
ji=1....,
Output: K aligned sets of hyperbolic points 7i(11) ( ji=l.-.. H((K) {h₂ (
1: for each set 7k(h) do
2: compute the Riemannian mean h and dispersion (k))
3: end for
(k)
4: compute h, the global Riemannian mean of {h sk=1
5: for each set (()) do
6: apply the Riemannian translation h:!
A - (1)
// Eq.
7: apply the Riemannian scaling (2( = r- h
(1) with 8 == 1/ ddl))
Eq.
8: apply the wrapped rotation h (k)
0 (K)
with = (U(6)T
// Eq.
9: end for
(8)
the k-th dataset, respectively. In addition, let h be the global Riemannian mean of (h" ===1 We propose to transport the points of the k-th set using T T Next, the Riemannian dispersion of each set is set to by applying T with s 1/)d Finally, the wrapped rotation is applied to all the data sets on the mapping of the tangent space T and then mapped back to the manifold Ld. The first set is designated as the reference set, and all other rotation matrices (+) are updated
according to u; (k) +- sign((u. (4) u. ((()))) The proposed HPA for multiple sets summarized in Algorithm 1 and some implementation remarks appear in Appendix C
Experimental results
We apply HPA to simulations and to three biomedical datasets] In addition, we test HPA on MNIST
BI and USPS [23] datasets, which arguably do not have a distinct hierarchical structure. Nonetheless. we demonstrate in Appendix D that our HPA is highly effective in aligning these two datasets. All the experiments are label-free. We compare the obtained results the following alignment methods: (i) PAH [51] which is applied only to the simulated data since it requires the existence of a one-to-one correspondence between the sets, (ii) only the Riemannian translation (RT), (iii) OT in hyperbolic space with the weighted Fréchet mean (HOT-F) extended to an unsupervised setting according to [44, (iv) OT with W-linear map (HOT-L) [22], and (v) hyperbolic mapping estimation (HOT-ME)
22 As baseline, we present the results obtained before the alignment (Baseline). For more details on the experimental setting, see Appendix
4.1 Simulations
The synthetic data in Ld is generated using the sampling scheme described in Section 2 based on 39
Given an arbitrary point μa E I.dd and an arbitrary SPD matrix Σ Rddd, we generate set of N points
Qa) = [q (1) L centered at fa by Ld e q;
(1) = Exp,, (PT (0, (o(")) )), where fu [1, 0j is the
origin, v = [0, "." and o. ~ 00, Σ). Next, we generate three noisy and distorted versions
of (11) The first noisy set Q(2) (q( 2 N | is generated as proposed in by (,2) LT. (1) where T,, is a hyperbolic translation defined by Te, [V1+ e, 61, 6T Es, (I + 6,E, ) e, is sampled from (O0 o2 r2T), o² is the variance, and L is random H-unitary matrix [16] Another noisy
set, denoted as (3) {q.
(3) 1==1 is generated by q. (3) = L(Exp,, (PT pm (u ()), where u,
[0, (1)) + T Here, the noise is added to the tangent space at μgg Finally, let Q(4) = {q) (4) 1 be distorted set, given by g. f (3) where fye (x) = cosh(I|u [|t)// + sinh(l [|]l |u]le and u Log.: (x), for arbitrary (fixed) μı' E I and t > O.
Mi=1
OOr code is available httpp:://gitttbccoo////e////
7

---

@ and Q
2 and @
@ And @
307
Decorpancyy
Discrepancy
201
207
10"
30
20
30
30
Dimersion
Dimonskon
Dimerston
Figure 2: Alignment results of the simulated data.
Before alignment
After alignment
Figure 3: The visualizations of HPA applied to the two breast cancer gene expression datasets.
We apply Algorithm 1 to align the three pairs of sets { (Q(1, ) (o Q(3), and (Q(1) Q setting N = 100, - 1, and d E (3,5, 5, 10, 20, ..... 40). Each experiment is repeated 10 times with different values μa, Σ. μı' and I. evaluate the alignment, we use the pairwise discrepancy based on the hidden one-to-one correspondence, given by E E((")) (()) = N E (g. q: q.")), where j E (2, 4).. The discrepancy as a function of the dimension is shown in Fig. 2 We observe that the proposed HPA has lower discrepancy relative to the other label-free methods. Specifically, it outperforms OT-based methods that are designed to match densities. Furthermore, the proposed HPA is stable, in contrast to HOT-L, which is highly sensitive to the noise and distortion introduced in ((3) and @(4), Interestingly, we remark that the discrepancies of RT and PAH are very close, empirically showing that RT alone comparable PAH. addition, note that HPA permutation- invariant and does not require one-to-one correspondence as PAH. We report the running-ting time in Appendix and demonstrate that HPA more efficient than HOT-F and HOT-ME.
4.2 Batch effect removal
We consider bioinformatics datasets consisting of gene expression data and CyTOF. Representing such data in hyperbolic spaces was shown to be informative and useful (25]. implying that such data have an underlying inherent hierarchical structure. Batch effects 43 arise from experimental variations that can be attributed to the measurement device or other environmental factors. Batch correction is typically a critical precursor to any subsequent analysis and processing.
Three batch effect removal tasks are examined. The first task involves breast cancer (BC) gene expression data. We consider two publicly available datasets: METABRIC [8] and TCGA 266 consisting of samples from five breast cancer subtypes. The batch effect stems from different profiling techniques: gene expression microarray and RNA sequencing. the second task, three cohorts of lung cancer (LC) gene expression data 21 are considered, consisting of samples from three lung cancer subtypes. The data were collected using gene expression microarrays at three different sites likely source of batch effects): Stanford University (ST), University of Michigan (UM), and Dana-Farber Cancer Institute (D-F). The last task involves CyTOF data 4% consisting of peripheral blood mononuclear cells (PBMCs) collected from two multiple sclerosis patients during four days: two day before treatment (BT) and two days after treatment (AT). These 8 = 2 x 2 x 2 batches were collected with or without PMA/ionomycin stimulated PBMCs. We aim to remove the batch effects between two different days from the same condition (BT/AT) and from the same patient. In each batch removal task, we first learn an embedding the data from all the batches into the Lorentz model d 42 Then, HPA is applied to the embedded points in Ld

---

Table 1: The k-NN AUC-ROC from the best k in each method.
Dansers
Space
S-Raseline
Roseline
HPA
RT
HOTT-F
ROTL
HOTEME
BC
1
0 a 0A088
115254 a 0.0891
0.7410 + 0.0354
011000
010538
0.58I1 ai 0.0121
1,4504
0,0228
1.5000 0.01477
LCC
1,2
0.9137 a. 0.0632
0.5521 1 0.099]
0.8316 + 0.0004
0,5318 1 0.0070
0.5400 1 0.0006
0,5150 1 0.0654
0,0901 1. 0.0516
P1 BT
1.30
0.9723
= 0.0088
0.6646 + 0.1596
0.9401 + 0.0068
(190120 + 0.0085
0.5003 + 0.0142
0.8855 = 0.0057
0,8919 + 0.0089
P1 AT
1,50
0,9600
04050
0.7606 + 0.1564
0.91229
+: 0.0011
(18270
0 # 0.0573
0.9014
0.0175
U.8848 (1,0305
P2
1,30
0.9086
0.0114
118971 0,1335
0.9329 0.0186
0.0142
0,8702
0,8506 0,1721
118810
AT
1,50
0.0070
0.5688
0,0688
0.8453
0,0039
0,7012 0.0812
0.7136
0.0012
0.7291
Table 2: The mean and std of MMD computed over ten random subsets of size 300 for BC, 15 for LC, and 1000 for CyTOF from the two datasets.
Damasets
Space
Raseline
HPA
RT
HOT-F
HOT-L
HOT-MR
BC
2.00
0,2089 0.0027
0.0013 a 0,0001
0.0072 1: 0.0011
0,0009 0.0001
0.0019 d: 0.0004
0.0007 : 0.0002
STRUM
1,20
0.1072 + 0.0061
0.0162 + 0.0048
0.0250 + D.0049
0.0023 + 0.0002
0.0014 0.0007
0.1011 = 0.1006
ST&D-F
2,50
0.3213
0.0152
0.0122 0.0042
0.01550 + D.01166
0.0019
0.0002
0.0010 :: 0.0004
0.0015 0.0001
UM&D-M
0.0790 0.0071
0.0168 4: 0,0050
0.0168 4: 0.0032
0,0017
0.0003
0.0012 1: 0.0006
0.0029 =: 0.0005
P1 BT
0.0638
0.0024
0.0012 + 0.0002
0.00220 +: 0.0002
0,0004 :: 0.0002
0.0002 + 0.0001
D.0009 =: 0.0002
P1 AT
0.0598
0.0014
0.0036 0.0001
0.0015
D.0001
0.0001
0.0001 0.0001
0.0002 0.0011
P2 BT
0.0824 0.0021
0.0012
+ 0.0001
0.0015 +: 0.0001
0.0002
0.0001
0.0020 = 0.0008
D.0000 = 0.0003
P2 AT
1.20
0,0738
0.0053
0.0011 + 0.0002
0.0013 + 0.0002
0.0002 0.0001
0.0015 + 0.0008
0.0007 - 0.0003
Fig. shows a visualization of the embedding of the two breast cancer datasets before and after HPA. For visualization, we project the points in 3 to the 3D Poincaré ball. Before the alignment, the dominant factor separating between the patients' samples (points) is the batch. In contrast, after the aligmment, the batch effect is substantially suppressed (visually) and the factors separating the points are dominated by the cancer subtype.
We evaluate the quality of the alignment in two aspects using objective measures: (i) k-NN clas- sification, with battc-on-batch-out coossvalldation, is utilized for assessing the alignment of the intrinsic structure, and (ii) MMD 19] is used for assessing the distribution alignment quality. For the classification, we view the five subtypes of BC, the three subtypes of LC, and the presence of stimulated cells in CyTOF as the labels in the respective tasks. In addition to the results of the different alignment methods, we report the k-NN classification based only on a single batch (S Baselline). which indicates the adequacy of the representation in hyperbolic space to the task at hand.
Table I depicts the k-NN classification obtained for the best k per method, and Table shows the MMD. In each task, we set the dimension of the Lorentz model d to the dimension in which the best empirical single-task performance is obtained (S-Baseline). We note that similar results and trends are obtained for various dimensions. Additional results for various i values and an ablation study, showing that the combination of all three components yields the best classification results, are reported in Appendix D
Although the OT-based methods obtain the best matching between the distributions of the batches, HPA outperforms in all three tasks in terms of classification (see Table I In the two gene expression tasks, where the data have multiple labels and we align multiple batches, the advantage of HPA compared to the other methods is particularly significant. In Appendix D we demonstrate HPA's out-of-sample capabilities the CyTOF data by learning the batch correction map between the different days from one patient and applying it to the data of the other patient.
4.3 Discussion
Alignment methods based on density matching, such as OT-based methods, often overlook an important aspect in purely unsupervised settings. Although sample density is the main data property that can be and need to be aligned, preserving the intrinsic structure/geometry of the sets is important, as it might be tightly related to the hidden labels. Indeed, we see in our experiments that OT- based methods provide a good density aligmment (reducing the inter-set variability), as demonstrated by small MMD values (see Table D However, the intrinsic structure of the sets (the intra-set variability) not preserved, as evident by the resulting poor (hidden) label matching, conveyed by the k-NN classification performance (see Table This is also illustrated in the right panel of Fig. [ There it visible that the three OT-based methods provide good global alignment of the sets, yet the intrinsic structure is not kept, as implied by the poor color matching.

---

In contrast to OT-based methods, HPA does not explicitly aim to match densities, and thus, it obtains slightly worse MMD performance compared to OT-based methods. However, HPA matches the first two moments of the density and includes the rotation component, which was shown to be one of the fundamental limitations of OT-based methods for alignment as OT cannot recover volume-presere preserving As seen in the simulation and experimental results and illustrated in the right panel of Fig. D we still obtain a good global alignment and simultaneously preserve the intrinsic structure, allowing for high classification performance. We remark that in the synthetic examples, there is (hidden) one-to-one correspondence between the sets, and therefore one-to-one discrepancy can be computed (instead or in addition to MMD). When there is such a correspondence, OT still cannot recover volume- volume-preservine preserving maps, while HPA can mitigate noise and distortions.
maps
4 KG
5
Conclusion
We introduced HPA for label-free alignment of data in the Lorentz model. Based on Riemannian geometry, we presented new translation and scaling operations that align the first and second Rie- mannian moments as well as wrapped rotation that aligns the orientation in the hyperboloid model. Our theoretical analysis provides further insight and highlights properties that may be useful for practitioners. We empirically showed in simulations that HPA is stable under noise and distortions. Experimental results involving purely unsupervised batch correction of multiple bioinformatics datasets with multiple labels is demonstrated. Beyond alignment and batch effect removal, our method can viewed as type of domain adaptation or a precursor transfer learning that relies on purely geometric considerations, exploiting the geometric structure of data as well as the geometric properties of the space of the data. In addition, it can be utilized for multimodal data fusion and geometric registration of shapes with hierarchical structure.
Acknowledgments and Disclosure of Funding
We thank the reviewers for their important comments and suggestions, and thank Thomas Dagès for the helpful discussion. The work of YEL and RT was supported by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 802735-ERC-DIFFOP The work of YK was supported by the National Institutes of Health RO1GM131642, UMIPA05141, P50CA121974, and U0IDA053628.
References
[1] B. Afsari. Riemannian 1l center mass: existence, uniqueness, and convexity. Proceedings of the American Mathematical Society, 139(2):655-6733 2011.
[2] D. Alvarez-Melis, Y. Mrouch, and T. Jaakkola. Unsupervised hierarchy matching with optimal transport over hyperbolic spaces. In International Conference on Artificial Intelligence and Statistics, pages 1606-1617. PMLR, 2020.
[3] Y. Brenier. Décomposition polaire et réarrangement monotone des champs de vecteurs. CR Acad. Sci. Paris Sér. I Math., 305:805-8000 808, 1987.
[4] Y. Brenier. Polar factorization and monotone rearrangement of vector-valued functions. Com- munications on pure and applied mathematics, 44(4):375 417, 1991.
[5] R. Chakraborty, D. Seo, and B. C. Vemuri. An efficient exact-pga algorithm for constant curvature manifolds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3976 9976-3984, 2016.
[6] I. Chami, Z. Ying, C. Ré, and J. Leskovec. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 0184868-4877 2019.
[7] N. Courty, R. Flamary, D. Tuia, and A. Rakotomamonjy. Optimal transport for domain adaptation, IEEE transactions on pattern analysis and machine intelligence, 1853 2016.
10

---

[8] C. Curtis, S. P. Shah, S.-F. Chin, G. Turashvili, O. M. Rueda, M. J. Dunning, D. Speed, A. G. Lynch, Samarajiwa, Y. Yuan, et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486(74003) 2012.
[9] M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26:2292-2300 2013.
[10] Euzenat and Shvaiko. Ontology matching: State of the and future challenges. IEEE Transactions on knowledge and data engineering, 25(1):158-176, 2013.
[11] L. Falorsi, P. de Haan, T. R. Davidson, and P. Forré. Reparameterizing distributions lie groups. In The 22nd International Conference on Artificial Intelligence and Statistics, pages $244-3253. PMLR, 2019.
[12] R. Flamary and N. Courty. Pot python optimal transport library. con/rftamaryPOOT, page 144, 2017.
GitHub: https://githu.
[13] M. Fréchet. Les éléments aléatoires de nature quelconque dans un espace distancié. In Annales de I' l'institut Henri Poincaré, volume 10, pages 215-310, 1948.
[14] O. Ganea, G. Bécigneul, and Hofmann. Hyperbolic entailment cones for learning hierarchical embeddings. In International Conference on Machine Learning, pages 1646-1655. PMLR, 2018.
[15] O. Ganea, G. Becigneul, and T. Hofmann. Hyperbolic neural networks. In Advances in Neural Information Processing Systems (NIPS), page 5350-5360, 2018.
[16] I. Gohberg, P. Lancaster, and L. Rodman. Matrices and indefinite scalar products. 1983.
[17] C. Gower. Generalized procrustes analysis. Psychometrika, 40(1):33-51, 1975.
[18] J. C. Gower, G. B. Dijksterhuis, et al. Procrustes problems, volume 30. Oxford University Press on Demand, 2004.
[19] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
[20] D. Groisser. Newton's 's method, zeroes of vector fields, and the riemannian center of mass. Advances in Applied Mathematics, 33(1):95-133) -135, 2004.
[21] D. N. Hayes, S. Monti, G. Parmigiani, C. B. Gilks, K. Naoki, A. Bhattacharjee, M. A. Socinski. C. Perou, and M. Meyerson. Gene expression profiling reveals reproducible human lung adenocareinoma subtypes in multiple independent patient cohorts. Journal of Clinical Oncology, 24(31):5079-509) 2006.
[22] A. Hoyos-Idrobo. Aligning hyperbolic representations: an optimal transport-baue based approach. arXiv preprint arXi::20/2.0/08) 2020.
[23] J. Hull. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.
[24] H. Karcher. Riemannian center of mass and mollifier smoothing. Communications on pure and applied mathematics, 30(5):509 5411, 1977.
[25] A. Klimovskaia, D. Lopez- Paz, L. Bottou, and M. Nickel. Poincaré maps for analyzing complex hierarchies in single-cell data. Nature communications, 11(1):1-9, 2020.
[26] D. Koboldt, R. Fulton, M. McLellan, H. Schmidt, J. Kalicki- Veizer, J. McMichael. L. Fulton,
D. Dooling, L. Ding, E. Mardis, et al. Comprehensive molecular portraits of human breast tumours. Nature, 490(7418):61-70, 2012.
[27] D. Kohli, Cloninger, and G. Mishne. LDLE: Low distortion local eigenmaps. In ICLR 2021 Workshop Geometrical and Topological Representation Learning, 2021.

---

[28] I. Korsunsky, N. Millard, J. Fan, K. Slowikowski, F. Zhang, K. Wei, Y. Baglaenko, M. Brenner, P.-I. Loh, and Raychaudhuri. Fast, sensitive and accurate integration of single-cell data with harmony. Nature methods, 16(12):1289-122: 2019.
[29] A. Kumagai and T. Iwata. Unsupervised domain adaptation by matching distributions based on the maximum mean discrepancy via unilateral transformations. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pages 1106-41133 2019.
[30] M. Law, R. Liao, J. Snell, and R. Zemel. Lorentzian distance learning for hyperbolic rep- resentations. In International Conference on Machine Learning, pages 3672-3681. PMLR, 2019.
[31] Y. LeCun, L. Bottou, Y. Bengio, and Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
[32] Q. Liu, M. Nickel, and D. Kiela. Hyperbolic graph neural networks. In Advances in Neural Information Processing Systems (NIPS), pages 8228-8239, 2019.
[33] A. Lou, I. Katsman, Jiang, S. Belongie, S.-N. Lim, and C. De Sa. Differentiating through the fréchet mean. In International Conference on Machine Learning (ICML), pages PMLR, 2020.
[34] B. Luo and E. Hancock. Feature matching with procrustes alignment and graph editing. 1999.
[35] H. Maron, N. Dym, Kezurer, S. Kovalsky, and Y. Lipman. Point registration via efficient convex relaxation, ACM Transactions on Graphics (TOG), 35 2016.
[36] R. J. McCann. Polar factorization of maps on riemannian manifolds. Geometric & Functional Analysis GAFA, 11(3):589-608, 2001.
[37] H. H. Milioli, R. Vimieiro, I. Tishchenko, C. Riveros, R. Berretta, and P. Moscato. Iteratively refining breast cancer intrinsic subtypes in the metabric dataset. BioData mining, 911::1-8, 2016.
[38] G. Monge. Mémoire sur la théorie des déblais et des remblais. Histoire de l'Académie Royale des Sciences de Paris, 1781.
[39] Y. Nagano, S. Yamaguchi, Y. Fujita, and M. Koyama. A wrapped normal distribution on hyperbolic space for gradient-based learning. In International Conference Machine Learning (ICML), pages 4693-4702. PMLR, 2019.
[40] I. Naim, Y. Song, Q. Liu, H. Kautz, J. Luo, and D. Gildea. Unsupervised alignment of natural language instructions with video segments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
[41] M. Nickel and D. Kiela. Poincaré embeddings for learning hierarchical representations. In Advances in Neural Information Processing Systems (NIPS), page 6341-6350, 2017.
[42] M. Nickel and D. Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In International Conference on Machine Learning (ICML), pages 3779-3788. PMLR, 2018.
[43] V. Nygaard, E. A. Rødland, and E. Hovig. Methods that remove batch effects while retaining group differences may lead to exaggerated confidence in downstream analyses. Biostatisties, 17(1):29-33 -9,999999999 2016.
[44] X. Pennec. Barycentric subspace analysis manifolds. The Annals of Statisties, 46(6A):2711- 2746, 2018
[45] G. Peyré and M. Cuturi. Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning, 1105-6):355-077 2019.
[46] G. Ratcliffe, Axler, and K. Ribet. Foundations of hyperbolic manifolds, volume 149. Springer, 1994.
12

---

[47] P. L. C. Rodrigues, C. Jutten, and M. Congedo. Riemannian procrustes analysis: Trans- fer learning for brain-come computer interfaces. IEEE Transactions on Biomedical Engineering. (0(8):2390-2401, 2018.
[48] U. Shaham, K. P. Stanton, J. Zhao, H. Li, K. Raddassi, R. Montgomery, and Y. Kluger. Removal of batch effects using distribution-matee matching residual networks. Bioinformatics, 33(16):2539- 2546, 2017.
[49] D. Spohr, L.. Hollink, and P. Cimiano. A machine learning approach to multilingual and cross-lingual ontology matching. In International Semantic Web Conference, pages 665-680. Springer, 2011.
[50] T. Stuart, A. Butler, P. Hoffman, C. Hafemeister, E. Papalexi, W. M. Mauck III, Y. Hao, M. Stoeckius, P. Smibert, and R. Satija. Comprehensive integration of single-cell data. Cell, 177(7):1888 1902, 2019.
[51] Tabaghi and Dokmanic. On procrustes analysis in hyperbolic space. arXiv preprint 2021.
[52] C. Wang and S. Mahadevan. Manifold alignment using procrustes analysis. In In International Conference on Machine Learning (ICML), pages 1120-1127, 2008
[53] O. Yair, M. Ben-Chen, and R. Talmon. Parallel transport on the cone manifold of spd matrices for domain adaptation. IEEE Transactions Signal Processing, 67(7): 2019.
[54] O. Yair, F. Dietrich, Talmon, and I. G. Kevrekidis. Domain adaptation with optimal transport on the manifold of spd matrices. arXiv e-prints, pages arXiv- 2019.
13


Reviews:
[{"review_id": "xDqOxISbz7E", "reviewer": "Reviewer_ywh5", "summary": "The paper proposes to adapt Procrust Analysis (PA) to Hyperbolic spaces. Hyperbolic spaces have shown their efficiency for hyperbolic data in previous works. The aim of the work is to use hyperbolic geometry to compute data alignment for hierarchical data. More specifically, the authors derive the three tools used for PA to Lorentz model of hyperbolic space: translation - using parallel transport, logarithmic and exponential map to project to and from the tangent space - , rotation - using a projection to euclidean space - and scaling. Experiments are conducted on artificial datasets in order to evaluate precisely the benefits of the method - knowing the one-to-one correspondence between the two sets to align - and on real datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: To the best of my knowledge, this is the first study concerning Procrust Analysis and hyperbolic space - except from [46] cited by the authors and simultaneously submitted to another review (the approach is quite different). \n\nQuality: I have no review all the proofs but according to my understanding the developed techniques and the results seem sound. The protocol setting is judicious, the experiments on artificial data are interesting to assess the quality of the approach and the experiments on real data are convincing. However, from what I understand, the results on real datasets depend on the computed embeddings using [38]. There is no discussion on the effect of this initial embedding of the data and I am curious it is matters or not.\n\nClarity: The paper is written for specialists of information geometry, it is quite difficult for a non-expert to understand even the purpose of the work. The authors should have introduced better the problematic, at least by introducing PA in the classical setting and discussing more in detail the impact of their work. \n\nSignificance: The work is clearly interesting, but the writing of the article - not educational enough - hinders the potential impact of the paper outside of the information geometry community.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "x9QQ2mQ5Sm", "reviewer": "Reviewer_EAV6", "summary": "This paper deals with data alignment, and more precisely, of unsupervised alignment of data embedded in hyperbolic spaces. The authors propose a purely geometric approach related to Procrustes analysis (PA), and based on Riemannian geometry. Concerning hyperbolic space, the authors consider the Lorentz model for its numerical efficiency concerning the basic Riemannian calculation. \n \nThe hyperbolic PA (HPA) idea is to transpose the classical procrustean analysis to the Lorentzian framework. Hence, the authors generalize the notion of translation using parallel transport, the one of rotation by proposing a \"Riemannian wrapped rotation\" and finally realizing a geodesic scaling. With these generalizations, the authors can develop their HPA analysis  (which they do).  Theoretical justifications are provided (mainly in supplementary material), as well as numerical illustrations. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method seems to me innovative and relevant. It is based on strong theoretical arguments (the Lorentzian geometry studied since...) and benefits from a robust algorithmic.\n \nThe structure of the article is relevant; the writing of the article is clear and effective. I appreciated the pedagogical effort of the authors to try to bring their method to people not necessarily familiar with Riemannian geometry.\n \nI have several relatively minor questions/comments, which I list below. My main complaint is with the writing in Appendix A. The proofs are awful to follow: successions of calculations without justification, without even a single non-mathematical symbol. We deal more with calculations written in rough drafts than with publishable proofs (especially in a very selective context). I highly recommend a thorough rewrite of these demonstrations.\n\n- The figures are not very legible when printed in black and white. The choice of colors in the legends in Figure 2 and its counterparts in the appendix is not consistent.\n- Line 87: Who is $\\mu_0$?\n- Line 97 about the PAH: In addition to the non-limiting nature, I would have liked to find a brief comparison between the two approaches on a technical level.\n- Line 113, about the construction of the Riemannian translation (Equation (2)): A figure illustrating the formula and relating the different objects to each other would be a valuable aid to intuition.\n- Line 163 (and following): The tangent space is topologically equivalent to $\\mathbb{R}^d$; talking about projection seems to me very improper here.\n- Line 117: $\\hat{\\mathcal{H}}^{(2)}$ is not defined.\n- Equation (12): Please improve the parenthesis (different sizes/shapes) to make this equation more readable.\n- Line 207: Is there a relationship between $U'$ and $U$?\n- The authors mention the hierarchical structure several times (for example, in the last sentence of the article). I would like to understand what they mean by this: the connection with the Lorentzian manifold is unclear to me, unless I don't have the same meaning as the authors.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This work is very theoretical. I cannot therefore conclude on the use that could be made of it. As it stands, it is mainly about proposing new tools.\n", "review_text": "", "extra_scores": {"ethical_concerns": "I did not identify any ethical issues in this document. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "YGjDZIB6WFG", "reviewer": "Reviewer_aJ5V", "summary": "The authors introduce Hyperbolic Procrustes Analysis (HPA) for unsupervised alignment of hierarchical data in the Lorentz model of hyperbolic space. The proposed method is tested on biomedical data sets, which are assumed to embed well into hyperbolic space due to possessing a hierarchical structure, and additional data sets, which are not assumed to have a hierarchical structure. Some of the experiments indicate improvements over state-of-the-art optimal transport (OT)-based methods.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Assessment*\n\nOverall, this is an interesting paper. My main concerns are about the experimental setup (choice of data sets for assessment) and the comparison with OT-based methods. At present, to me the later does not demonstrate that HPA is superior to OT-based methods. I am open to raising my score if the authors could address these two concerns in the rebuttal.\n\n\n*Comments*\n\n- Procrustes analysis and the problem of (unsupervised) alignment of data sets is a topical problem and of interest to the ML community. The recent surge of interest in hyperbolic spaces makes the paper relevant.\n- Related work: \n   -  The paper only sparsely refers to the extensive body of literature on the Procrustes analysis in the field of Riemannian optimization.\n  - Consider giving more context on the increasing body of literature on ML in hyperbolic spaces.\n  - Recent work [46] seems to have proposed a similar method, albeit with limitations. Please be more specific about the differences between this work and yours. Granted that it seems to be concurrent work, consider adding an experimental comparison. \n- While I appreciate the detailed definitions of the tools from Riemannian geometry underlying the proposed method, I do not think that sections 2 and parts of 3 are very accessible to the broader ML community. Consider moving details to the supplemental material and provide a more high-level overview instead. I would illustrate the Lorentz model and the respective geometric tools in a schematic figure.\n- Experiments:\n  - When using MMD to evaluate the alignment quality the OT-based methods consistently outperform HPA. Can you comment on why this is? It seems to me that the MMD analysis is more general than evaluating classification performance, so I am not convinced that HPA is a better method. If the main argument is computational efficiency, please make this clearer.\n  - Hyperbolic ML methods are usually motivated by and demonstrated on data that embeds well into hyperbolic space (vs. Euclidean space) due to a true hierarchical structure. For the data sets analyzed, the authors should provide concrete evidence on good embeddability in hyperbolic space, e.g., by computing embeddings into both hyperbolic and Euclidean spaces and reporting distortion. \n  - How does the performance of HPA for data with and without a true hierarchical structure compare? You indicate that it performs well, even on data without a true hierarchical structure. Can you comment on that? \n  - How does HPA compare to the Euclidean SOTA for data with and without a true hierarchical structure?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "  "}, "extra_text": {}}, {"review_id": "4wHJuGiNyQI", "reviewer": "Reviewer_XtJE", "summary": "In this paper, the authors rebuild the three components of Procrustes analysis (i.e. translation, scaling, and rotation) using the Riemannian geometry of the Lorentz model, thus practically reimplementing PA for hyperbolic spaces (HPA). Their methodology follows a theoretical analysis which, as the authors state, has its own merit. Furthermore, experiments show how the approach, in addition to being computationally more efficient than previous ones, also is able to better tackle the problem of \"batch effect\" on hierarchical datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \nThe presented method looks novel to the extent of my knowledge. Some of the methods employed to derive the new HPA components are not new (e.g. Chami et Al. \"Hyperbolic Graph Convolutional Neural Networks\" shows how to employ the composition of Exp and Log maps to allow operators which cannot work directly in the hyperbolic space), however here they are used differently and better grounded into theory.\n\nQuality: \nThe submission looks theoretically sound to me: the method is both theoretically justified and all the steps are properly derived. \n\nExperiments have been conducted to compare HPA's alignment with other methods on increasingly misaligned datasets (with synthetic data); to qualitatively show how it can help reduce the batch effect; and, finally, to quantitatively compare its performances wrt other methods both in terms of k-NN classification and of maximum mean discrepancy.\nIt is not clear to me why HPA has a worse MMD performance compared to other methods, considering how better it performed both in the synthetic examples case and in the classification task (i.e. the same data where it had bigger discrepancy). I think the authors should provide some reasoning about why this happens and why their approach can still be considered better than others.\n\nClarity:\nI think the paper is well written and clear enough to be replicated. Proofs are made available in Appendix and code is provided with supplementary material for replication.\n\nSignificance: \nThe presented approach has a value per se, regardless of the very specific application shown, in all those cases where the data representation, whether provided or learned, has some intrinsic hierarchical nature and different modalities. The experiments in the paper are rather limited in nature, but in my opinion they serve the purpose of showing the method's quality and making it available to the wider research community.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors properly addressed the method's limitations. No societal impacts have been considered.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {"ethical_concerns": "To my understanding, the paper does not raise particular ethical concerns.\nThe bioinformatic datasets used are public and the data is completely anonymised, without information which would make the original subjects identifiable."}}]

Author rebuttal:

Meta-review: This was a very borderline paper. The original submission appears to be technically sound and novel, but reading the paper without difficulty requires a fair amount of prior expertise in PA. It is dense to the point that it might impact the significance of the work in the general NeurIPS community. However, the reviewers greatly appreciated the enthusiastic interaction with the authors during the rebuttal, and have generally agreed to accept the paper under the assumption that the authors make significant improvements to clarity in the camera ready. As just one example, the reviewers strongly recommend including a high-level introduction of PA in the final submission. Please read through all of the reviewer feedback carefully and follow through on promised adjustments.
Decision: Accept (Poster)