Hyperbolic Procrustes Analysis Using Riemannian Geometry
Authors: Ya-Wei Eileen Lin
Yuval Kluger
Ronen Talmon
Venue: NeurIPS 2021 Poster (2021)
Keywords: Label-free data alignment
hyperbolic space
Riemannian geometry
Procrustes analysis
batch correction
domain adaptation

Abstract: Label-free alignment between datasets collected at different times, locations, or by different instruments is a fundamental scientific task. Hyperbolic spaces have recently provided a fruitful foundation for the development of informative representations of hierarchical data. Here, we take a purely geometric approach for label-free alignment of hierarchical datasets and introduce hyperbolic Procrustes analysis (HPA). HPA consists of new implementations of the three prototypical Procrustes analysis components: translation, scaling, and rotation, based on the Riemannian geometry of the Lorentz model of hyperbolic space. We analyze the proposed components, highlighting their useful properties for alignment. The efficacy of HPA, its theoretical properties, stability and computational efficiency are demonstrated in simulations. In addition, we showcase its performance on three batch correction tasks involving gene expression and mass cytometry data. Specifically, we demonstrate high-quality unsupervised batch effect removal from data acquired at different sites and with different technologies that outperforms recent methods for label-free alignment in hyperbolic spaces.

Hyperbolic Procrustes Analysis Using Riemannian Geometry
Ya-Wei Eileen Lin
Yuval Kluger
Ronen Talmon
Viterbi Faculty of Electrical and Computer Engineering, Technion
*Program in Applied Mathematics, Yale University
Interdepartmental Program in Computational Biology and Bioinformatics, Yale University Department of Pathology, Yale University (lin. ya- jaa-we@ccmpuss ronendee} technion. aww.iinnncc..... ac.il, i1, yuval. hal..klluerrraee.. edu
Abstract
Label-free alignment between datasets collected at different times, locations, by different instruments is a fundamental scientific task. Hyperbolic spaces have recently provided fruitful foundation for the development of informative repre- sentations of hierarchical data. Here, we take a purely geometric approach for label-free alignment of hierarchical datasets and introduce hyperbolic Proerustes analysis (HPA). HPA consists of new implementations of the three prototypical Procrustes analysis components: translation, scaling, and rotation, based the Riemannian geometry of the Lorentz model of hyperbolic space. We analyze the proposed components, highlighting their useful properties for alignment. The efficacy of HPA, its theoretical properties, stability and computational efficiency are demonstrated in simulations. In addition, we showcase its performance on three batch correction tasks involving gene expression and mass cytometry data. Specifically, we demonstrate high-quality unsupervised batch effect removal from data acquired at different sites and with different technologies that outperforms recent methods for label-free alignment in hyperbolic spaces.
Introduction
A key scientific task in modern data analysis is the alignment of data. The need for alignment often arises since data are acquired in multiple domains, under different environmental conditions, using various acquisition equipment, and at different sites. This paper focuses on the problem of fabel-free alignment of data embedded in hyperbolic spaces. Recently, hyperbolic spaces have accentuated in geometric representation learning. These -un-uulideen spaces have become popular since they provide a natural embedding of hierarchical data thanks to the exponential growth of the lengths of their geodesic paths [4]] 42
B0
6 32
The problem of alignment of data embedded in hyperbolic spaces has been extensively studied, e.g., in the context of natural language processing A9) ontology matching (10), matching two data modalities [40] and improving the embedding in hyperbolic spaces [ A few of these studies are based optimal transport (OT) [2] 22). classical problem in mathematics Bا that has recently reemerged modern data analysis, e.g., for domain adaptation [7]. Despite its increasing usage, OT for unsupervised alignment is fundamentally limited [54], since OT (as any density matching approach) cannot recover volume preuee--eeerving maps
17 18 that is based on purely geometric considerations. PA has been widely used for aligning datasets by eliminating the shift, scaling, and rotational factors. Over the years, it has been successfully applied to various applications, e.g.
In this paper, we resort to Procrustes analysis (PA)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

image registration 341 manifold alignment [52], shape matching [31, domain adaptation 47 and manifold learning
name but few. Here, we address the problem of label-free matching of hierarchical data embedded in hyperbolic spaces. We present hyperbolic Procrustes analysis (HPA), a new PA method in the Lorentz model of hyperbolic geometry. The main novelty lies in the introduction of new implementations of the three prototypical PA components based on Riemannian geometry. Specifically, translation is viewed as a Riemannian mean alignment, implemented using parallel transport (PT). Scaling is determined with respect to geodesic paths. Rotation is considered as moment alignment on mapping of the tangent space of the manifold to a Euclidean vector space. Our analysis provides new derivations in the Riemannian geometry of the Lorentz model and specifies the commuting properties of the HPA components. We show that HPA, compared to existing baselines and oT-based methods, achieves improved alignment in a purely unsupervised setting. In addition, it has a natural and stable out-of-sample extension, it supports both small and big data, and it is computationally efficient.
We show application to batch correction in bioinformatics tasks. We present results on both gene expression and mass cytometry (CyTOF) data, exemplifying the generality and broad scope of our method. In contrast to recent works 28 50 our method does not require landmark correspondence. which is often unavailable in many datasets or hard to obtain. Specifically, we show that batch effects caused by acquisition using different technologies, at different sites, and at different times can be accurately removed, while preserving the intrinsic structure of the data.
Our main contributions are as follows. (i) We present new implementation of PA using the Riemannian geometry of the Lorentz model for unsupervised label-free hierarchical data alignment. (ii) We provide theoretical analysis and justification of our alignment method based on new derivations of Riemannian geometry operations in the Lorentz model. These derivations have their own merit as they could be used in other contexts. (iii) We show experimental results of accurate batch effect removal from several hierarchical bioinformatics datasets without landmark correspondence.
2 Background on hyperbolic geometry
Hyperbolic space is a non-Euclidean space with a negative constant sectional curvature and an underlying geometry that describes tree-like graphs with small distortions [6 'hhre exist four commonly-used models for hyperbolic spaces: Poincaré disk model, Lorentz model (hyperboloid model), Poincaré half-plane model, and Beltrami-Klein model. These four models equivalent and
there exist transformations between them. Here, we consider the Lorentz model, and specifically, the upper sheet of the hyperboloid model, because its basic Riemannian operations have simple closed-form expressions and the computation of the geodesic distances is stable 42 30
Formally, the upper sheet of the hyperboloid model in a d-dimensional hyperbolic space is defined by L = { E dd (x, x) = -1 x11 > 0}. where (I, x) = x H is the Lorentzian inner product and H E R [dd+1)x(d+1) is defined by H = [-1.0 :0, Ia]. The Lorentzian norm of a V(x,
hyperbolic vector æ E Ld is denoted by ||0||c Let TaLd be the tangent space at x E Ld. defined by TaLd = (v| v) = 0}. Consider x E I and u E TaT,d, the geodesic path Φ : TR.jt + Ld is defined by s(t) = cosh(] (|llct)))) - (iuh((/o)ee() ालîlz with (0)) = x and initial velocity s0(0) = v. where o'(1) = (l)). In addition, the associated geodesic distance is (x, (t)) = cosh -1 (-(xz, de (t))
with the origin ps = [1, 0 IT E Ld.
The Exponential map. projecting a point E T to the manifold Ldd, is given by Expa (v) (11 = cosl([|e)lcc::: + sinh([[o|le))ppiz The Logarithmic map, projecting a point y E LL to the tangent space TgLd at æ, is defined by Log (v) = cosh ((A) (y- - Aæ), where A = - (w. v)
-x=-1
The PT of a vector v E TaLL along the geodesic path from x E L to y E Ld is defined by (T)- = + - xw. (x + y). where À = ((x v)c. keeping the metric tensor unchanged. X+1
The Riemannian mean x and the corresponding dispersion dx of set X' (2i/20 E Ld) are defined using the Fréchet mean 13 53 by
n
xx == m(x) = arg min
Σ dh (x, 2.) and dx == r(x)= = min Σ
H. x:), (1)
rELd
xELd nI
1=1
i=1
2

---

Our HPAA
Other methods
Given data
Feey Reraetin
Shop RIPNERS
Sip Reeate
PAB
HOTF (54)
HOT-L[[2]]
HOT-ME [22]
Illustration of our alignment method (HPA). Two sets of points in hyperbolic space are given, depicted in dark and light colors. Each point is associated with one of two labels: (i) blue/cyan, and (ii) red/pink. Left: Alignment results after each step our HPA implemented in T.2: Riemannian translation, Riemannian scaling, and Riemannian wrapped rotation. Note that these Riemannian operations in hyperbolic space are different than their Euclidean counterparts. Right: Alignment results of four methods: PAH [51] and HOT-F [4l) applied in L and HOT-L 22 and HOT-ME 22 applied in the 2D Poincaré disk. For visualization, all points in LL are transformed to the 2D Poincaré disk. The alignment after Step 3 of HPA (circled in black) is more accurate than the alignments obtained by the other methods.
Figure 1:
where m Idd and r: X RR+ Note that the Fréchet mean of samples on connected and compact Riemannian manifolds of non-positive curvatures, such as hyperbolic spaces, is guaranteed to exist, and it unique 24 M.. The Fréchet mean is commonly computed by the Karcher Flow
24 20 which is computationally demanding. Importantly, in the considered hyperbolic space, the Fréchet mean can efficiently obtained using the accurate gradient formulation 33
Given vector E E L and symmetric and positive-definic (SPD) matrix E E Raxd, d the wrapped normal distribution G Σ) provides generative model of hyperbolic samples as follows [39 [.. First, a vector o' is sampled from (O, Σ). Then, 0 is concatenated to the vector e' such that
= [0, v'JT E L. Finally, PT from the origin μ40 = [1, 0 J to w is applied to v, and the resulting point mapped the manifold using the Exponential map at The probability density
function of this model is given by log 9 (yla, Σ) = log. NO, Σ) (n 1) log( sinh 1 1ol
1
3 Hyperbolic Procrustes analysis
Existing methods for data alignment typically seek a function that minimizes a certain cost. large body of work attempts to match the empirical densities of two datasets. e.g., by minimizing the maximum mean discrepancy (MMD) 48 2
or solving OT problems 45 D2] Finding an
effective cost function without labels or landmarks is challenging, and minimizing such costs directly often lead to poor alignment in practice (see illustration in Fig. D. different well-established approach that applies indirect alignment based on geometric considerations is PA. While preparing this manuscript, another method of PA in hyperbolic spaces (PAH) was presented for matching two sets, assuming that they consist of the same number of points and that there exists point-wise isometric map between them [T)). We remark that the analysis we present here applies to broader settings and makes such assumptions. See Appendix D for details on classical PA well for comparisons to [5 and to the application of Euclidean PA in the tangent space.
We consider two sets of points ( = (h, (1) y J and H [h, J in Ld, Here, we aim to find a function ( [d + Ldd consisting of three components: translation, scaling, and rotation, that aligns H(2) with ((1) in an unsupervised label-free manner as depicted in Fig. Finding such a function can be viewed as extension of classical PA from the Euclidean space g+++ to the Lorentz model Ld. A natural extension to multiple sets is described Section 3.5 We remark that the statements written in the context of the problem at hand. In Appendix present their proofs.
restate them more generally and
3.1 Riemannian translation
Let h and (2) denote the Riemannian means of the sets H(1 and 2(2) respectively. In this translation component, we find map 1 (1)) Ld + Ld that aligns the Riemannian means of

---

the sets. In the spirit of 5 53 we propose to construct T A _ (h!²²) the composition of three
Riemannian operations in _ the Logarithmic map applied to h at h (2) PT h from h (2)
to
h along the geodesic path, and the Exponential map applied to the transported point at h ht")
P R
F (h (1= = Exp (PT (Loggr))) (h. 2 ))).
(2)
See Fig. B.I in Appendix
B for illustration. Since the geodesic path between any two points in T - a is well-defined. The rationale behind the combination of these three
L is unique 46 Riemannian operations twofold. First, PT is map that aligns the means of the sets, while preserving their internal structure. Second, the Logarithmic and Exponential maps compose a map whose domain and range the Lorentz model I,d rather than the tangent space, as desired. We make these claims formal in the following results.
Proposition 1. The map I h -h defined in Eq.
Q aligns the means of the sets, i.e., it satisfies
m([ 19
-h (2 (h: (2)) )
(3)
where m. is the function defined in Eq. Proposition 2. The map I 12 -/ (2) for all h
(2)
(2) (2)) can be recast as:
T T K (h. (h = h 3(h
(1) (2) + +
(2) h h
(4)
where the functions B and y are positive, defined by 0 < B(h
(2)
h
(2) - n th (2)
a+l h
and 0 < 7(,
h h2 =
( (20+
1
h (2)
respectively, and 0 < a =
a+1
(h (2)
In addition to providing a compact closed-form expression, Prop. gives the proposed translation based on Riemannian geometry an interpretation of standard mean alignment in linear vector spaces. It implies that the alignment is nothing but subtracting the mean of the source set (2)2) from each vector in 2222), and adding the mean of the target set F (with the appropriate scales). Proposition 3. The map I (2 -p preserves distances (i.e., is an isometry):
dh (2) h dr. R 0 (3) x - (h,
(5)
for any two points h (2) h (2)
E (222)
Let (() be the unique geodesic path from h (2) to h such that (0)) = h (2) let (0) E T aa and (1) E T be the corresponding velocities, respectively.
and (1) = h2) and
Proposition 4. The map I (p(")="""" aligns geodesic velocities, i.e., given the mapping of the geodesic
velocities to the manifold Ld vo Exp, (bb (0)) T and L 3 v2 " (1)), we have
" (wo) 01-
(6)
Isometry determined up to rotation, a fact that can be problematic for alignment. For example, any H-unitary matrix [16] can be an isometric function in I.d. When landmarks are given, they can be
used to alleviate this redundancy. However, in the purely unsupervised setting we consider, other
data-driven ques required. Prop. iimples that the proposed translation based on PT fixes some
of these rotational degrees of freedom by aligning the geodesic velocities. In Section 3.3
we revisit
this issue. Now, with a slight abuse of notation, let 72 r - (H(
Proposition 5. Consider two subsets A, B C 22)) and their translations Ā = N LR au (B) C. 122)) Let a = m(A), = m(B), ā = a and b = mB3) be the Riemannian means of the subsets. Then,
(A),
I ( o _ aa-05 = L -Ta 1 __))) ())

---

such that In the context of the alignment problem, the importance of Prop. is the following, Suppose the two sets correspond to data measured at two labs (denoted with and without tilde), and suppose each set was acquired by two types of equipment (denoted by A and B). Prop. implles that aligning data from the different labs and then aligning data acquired using the different equipment is equivalent to first aligning the different equipment and then the different labs, i.e., any order of the two alignments generates the same result. Seemingly, this is a natural property in Euclidean spaces. However, in a Riemannian manifold, it is not trivial result, and it holds for the transport along the geodesic path. See Appendix for counter-examples.
3.2 Riemannian scaling
Let d and d(²) denote the Riemannian dispersions of 7((1) and 22(2) By propositions and h (
and d(3) are the mean and dispersion of (2(2), Here, our goal is align the Riemannian dispersions of H(1) and H2 For this purpose, we propose the scaling function r 0 :Ld + L given by
r m (h; ( = (i(s),
(8)
where v d0)) /d(2) the scaling factor and (6) is the geodesic path from h (1)
(2) h
(0( = h and (1) = h, See Fig. B.2 in Appendix B for illustration.
Proposition 6. The dispersion of the rescaled set H2))
- (H((2) ) d(2))
3.3 Riemannian wrapped rotation
The purpose of this component to align the orientation of the distributions of the two sets after translation and scaling, namely, after aligning their first and second moments. The proposed rotation function A,0 : I + La consists of (i) mapping the points from the manifold L to the tangent space T, L2, (ii) mapping to Rd, (iii) rotating in RR and (iv) mapping back to the tangent space and then to the manifold. We perform the rotation in Ed which we term wrapped rotation, rather than direct rotation on the manifold L or on the tangent space T ILd for the following reasons.
First, the frequently used rotation map in Ld 51 does not necessarily preserve the Riemannian mean, and in our context, it might reverse the mean alignment. Second, rotation applied directly to the tangent space T Ld does not guarantee that the rotated points remain on the same tangent space. Third, applying rotation in the Euclidean vector space that is isometric to the tangent space is less efficient and stable, and it obtains slightly worse empirical results (see Appendix D.4 for details). Last, applying the rotation in IRd allows to use the standard Euclidean rotation using S SVD. In Section M alternatives.
we empirically demonstrate the advantage of the proposed rotation compared to the
Definition 1. Let the mapping function PrO) : :T Bdd defined on the langent space at h (1)
and its inverse map be the following functions defined by
hh (T
(s,
P
P (v) [v(()..... ...2).......... v(d+1) 1)]_ Rd and P n
(s)
(1) ,
h
where 8 E Rdd and (...) is the standard Euclidean inner product.
Note that removing the first element of v is valid due to the constraint imposed on the vector elements in the tangent space by definition. Indeed, no information is lost and the mapping is invertible.
The first step in our rotation component is to map the points in (1) and in 72)2 to the tangent space (2) for 1...., N₂. In
at h (1), ²
= Log, (1) (h(") for i = 1,..., N1, and of = Log, (1) (h.
the second step, we map the points by the mapping function in Definition I and re-center them:
(1) P (0 (1) s1)) i ...... N₂ and s(²) PP,.' (x: (v( (2) 1,..., N₂. where
((k) NE DN, ( (b) for 1, is the mean vector of the projections. Then, the mapped
and centered points (in Rd) are collected into matrices:
((k) [s] (k) (k)
8
NoJ E dd
(10)
5

---

In the third step, for each set k = 1.2, we compute the rotation matrix U(A) E Rdxd by applying
(8) ( Since the left-singular vectors are determined up to SVD to the matrix S( = ( (
sign, we propose align their signs as follows: u sign((u, (2) ( )u, ( where u!!h and u
the i-th left-singular vector of the two sets, resulting in modified rotation matrices Utk), Finally, we apply the rotation to 7((2) by
O h
(h (1))))
EXp (P-d) (UT(PR (Log (h, s(0) x(1))
(11)
where = U (U)
Proposition? 7. The wrapped rotation is bijective, and the inverse given by
(0 (0 = 6 (0)
(12)
3.4 Analysis
Putting all three components together, the proposed HPA that aligns 72 with H() culminates in the composition of translation, scaling, and rotation:
OU F
rs 5
+h (1).
(13)
As in most PA schemes, the order of the three components is important. Yet, the proposed components allow certain degree of freedom, indicated in the following results. Proposition 8. The Riemannian translation and the Riemannian scaling commute w.r.f the Rieman-
nian means h (1) and 22
r- R
-T r T
(2)
(14)
Note that Y (1)) and I (00 (2) m1 do not necessarily commute: Ts Proposition 9. The Riemannian scaling and the wrapped rotation commute:
a 0) # 19 ( r
T aa o UU a) = = a1 A a0)
r
(15)
We note that the rotation does not commute with the translation, because PT only preserves the local covariant derivative on the tangent space but might cause rotation and distortion along the transportation. Therefore, the rotation is required to be the last component of our HPA.
Thus far, we did not present a model for the discrepancy between the two sets, nor we presented the proposed HPA as optimal with respect to some criterion. In the following result, we show that if the discrepancy between the sets be expressed a composition of translation, scaling, and rotation, then the two sets can be perfectly aligned using HPA.
Proposition 10. Let v) L - La be a mup, given by 1) UU (h (((h N3, then,
r h o T TT . I
(2 = (OU'
A o r R
o h m (1) i= I,...,,22
(16)
where U' Odd.
Note that HPA consists of the sequence of Riemannian translation, Riemannian scaling and wrapped- rotation. The domain and range of each component is the manifold Ld. Yet, the first and last operations of each component are the Logarithmic and Exponential maps that project a point from the manifold to the tangent space, and vice versa, respectively. This allows us to propose an efficient implementation of the sequence without the back and forth projections as described in Appendix C
3.5 Extension to multiple sets
We naturally scale up the setting to support the alignment of K > 2 sets, denoted by H(R)
(8) NN where & E (1, K}. Let h (k) and (k)) be the Riemannian mean and dispersion of {h

---

Algorithm 1 Hyperbolic Procrustes analysis
Input: K sets of hyperbolic points H(1) (h: ) :.. Hik) (h1 ),1
ji=1....,
Output: K aligned sets of hyperbolic points 7i(11) ( ji=l.-.. H((K) {h₂ (
1: for each set 7k(h) do
2: compute the Riemannian mean h and dispersion (k))
3: end for
(k)
4: compute h, the global Riemannian mean of {h sk=1
5: for each set (()) do
6: apply the Riemannian translation h:!
A - (1)
// Eq.
7: apply the Riemannian scaling (2( = r- h
(1) with 8 == 1/ ddl))
Eq.
8: apply the wrapped rotation h (k)
0 (K)
with = (U(6)T
// Eq.
9: end for
(8)
the k-th dataset, respectively. In addition, let h be the global Riemannian mean of (h" ===1 We propose to transport the points of the k-th set using T T Next, the Riemannian dispersion of each set is set to by applying T with s 1/)d Finally, the wrapped rotation is applied to all the data sets on the mapping of the tangent space T and then mapped back to the manifold Ld. The first set is designated as the reference set, and all other rotation matrices (+) are updated
according to u; (k) +- sign((u. (4) u. ((()))) The proposed HPA for multiple sets summarized in Algorithm 1 and some implementation remarks appear in Appendix C
Experimental results
We apply HPA to simulations and to three biomedical datasets] In addition, we test HPA on MNIST
BI and USPS [23] datasets, which arguably do not have a distinct hierarchical structure. Nonetheless. we demonstrate in Appendix DittDDDDihhe that our HPA is highly effective in aligning these two datasets. All the experiments are label-free. We compare the obtained results the following alignment methods: (i) PAH [51] which is applied only to the simulated data since it requires the existence of a one-to-one correspondence between the sets, (ii) only the Riemannian translation (RT), (iii) OT in hyperbolic space with the weighted Fréchet mean (HOT-F) extended to an unsupervised setting according to [44, (iv) OT with W-linear map (HOT-L) [22], and (v) hyperbolic mapping estimation (HOT-ME)
22 As baseline, we present the results obtained before the alignment (Baseline). For more details on the experimental setting, see Appendix
4.1 Simulations
The synthetic data in Ld is generated using the sampling scheme described in Section 2 based on 39
Given an arbitrary point μa E I.dd and an arbitrary SPD matrix Σ Rddd, we generate set of N points
Qa) = [q (1) L centered at fa by Ld e q;
(1) = Exp,, (PT (0, (o(")) )), where fu [1, 0j is the
origin, v = [0, "." and o. ~ 00, Σ). Next, we generate three noisy and distorted versions
of (11) The first noisy set Q(2) (q( 2 N | is generated as proposed in
by (,2) LT. (1)
where T,, is a hyperbolic translation defined by Te, [V1+e, e, 6i, 6T Es, (I + 6,E, ) e, is sampled from (O0 o2 r2T), o² is the variance, and L is random H-unitary matrix [16] Another noisy
set, denoted as (3) {q.
(3) 1==1 is generated by q. (3) = L(Exp,, (PT pm (u ()), where u,
[0, (1)) + T Here, the noise is added to the tangent space at μgg Finally, let Q(4) = {q) (4) 1 be distorted set, given by g. f (3) where fye (x) = cosh(I|u [|t)// + sinh(l [|]l |u]le and u Log.: (x), for arbitrary (fixed) μı' E I and t > O.
Mi=1
OOr code is available httpp:://gitttbccoo///oe///
7

---

@ and Q
2 and @
@ And @
307
Decorpancyy
Discrepancy
201
207
10"
30
20
30
30
Dimersion
Dimonskon
Dimerston
Figure 2: Alignment results of the simulated data.
Before alignment
After alignment
Figure 3: The visualizations of HPA applied to the two breast cancer gene expression datasets.
We apply Algorithm 1 to align the three pairs of sets { (Q(1, ) (o Q(3), and (Q(1) Q setting N = 100, - 1, and d E (3,5, 5, 10, 20, ..... 40). Each experiment is repeated 10 times with different values μa, Σ. μı' and I. evaluate the alignment, we use the pairwise discrepancy based on the hidden one-to-one correspondence, given by E E((")) (()) = N E (g. q: q.")), where j E (2, 4).. The discrepancy as a function of the dimension is shown in Fig. 2 We observe that the proposed HPA has lower discrepancy relative to the other label-free methods. Specifically, it outperforms OT-based methods that are designed to match densities. Furthermore, the proposed HPA is stable, in contrast to HOT-L, which is highly sensitive to the noise and distortion introduced in ((3) and @(4), Interestingly, we remark that the discrepancies of RT and PAH are very close, empirically showing that RT alone comparable PAH. addition, note that HPA permutation- invariant and does not require one-to-one correspondence as PAH. We report the running-ting time in Appendix and demonstrate that HPA more efficient than HOT-F and HOT-ME.
4.2 Batch effect removal
We consider bioinformatics datasets consisting of gene expression data and CyTOF. Representing such data in hyperbolic spaces was shown to be informative and useful (25]. implying that such data have an underlying inherent hierarchical structure. Batch effects 43 arise from experimental variations that can be attributed to the measurement device or other environmental factors. Batch correction is typically a critical precursor to any subsequent analysis and processing.
Three batch effect removal tasks are examined. The first task involves breast cancer (BC) gene expression data. We consider two publicly available datasets: METABRIC [8] and TCGA 266 consisting of samples from five breast cancer subtypes. The batch effect stems from different profiling techniques: gene expression microarray and RNA sequencing. the second task, three cohorts of lung cancer (LC) gene expression data 21 are considered, consisting of samples from three lung cancer subtypes. The data were collected using gene expression microarrays at three different sites likely source of batch effects): Stanford University (ST), University of Michigan (UM), and Dana-Farber Cancer Institute (D-F). The last task involves CyTOF data 4% consisting of peripheral blood mononuclear cells (PBMCs) collected from two multiple sclerosis patients during four days: two day before treatment (BT) and two days after treatment (AT). These 8 = 2 x 2 x 2 batches were collected with or without PMA/ionomycin stimulated PBMCs. We aim to remove the batch effects between two different days from the same condition (BT/AT) and from the same patient. In each batch removal task, we first learn an embedding the data from all the batches into the Lorentz model d 42 Then, HPA is applied to the embedded points in Ld

---

Table 1: The k-NN AUC-ROC from the best k in each method.
Dansers
Space
S-Raseline
Roseline
HPA
RT
HOTT-F
ROTL
HOTEME
BC
1
0 a 0A088
115254 a 0.0891
0.7410 + 0.0354
011000
010538
0.58I1 ai 0.0121
1,450
0,0228
1.5000 0.01477
LCC
1,2
0.9137 a. 0.0632
0.5521 1 0.099]
0.8316 + 0.0004
0,5318 1 0.0070
0.5400 1 0.0006
0,5150 1 0.0654
0,0901 1. 0.0516
P1 BT
1.30
0.9723
= 0.0088
0.6646 + 0.1596
0.9401 + 0.0068
(190120 + 0.0085
0.5003 + 0.0142
0.8855 = 0.0057
0,8919 + 0.0089
P1 AT
1,50
0,9600
04050
0.7606 + 0.1564
0.91229
+: 0.0011
(18270
0 # 0.0573
0.9014
0.0175
U.8848 (1.1305
P2
1,30
0.9086
0.0114
118971 0,1335
0.9329 0.0186
0.0142
0,8702
0,8506 0,1721
118810
AT
1,50
0.0070
0.5688
0,0688
0.8453
0,0039
0,7012 0.0812
0.7136
0.0012
0.7291
Table 2: The mean and std of MMD computed over ten random subsets of size 300 for BC, 15 for LC, and 1000 for CyTOF from the two datasets.
Damasets
Space
Raseline
HPA
RT
HOT-F
HOT-L
HOT-MR
BC
2.00
0,2089 0.0027
0.0013 a 0,0001
0.0072 1: 0.0011
0,0009 0.0001
0.0019 d: 0.0004
0.0007 : 0.0002
STRUM
1,20
0.1072 + 0.0061
0.0162 + 0.0048
0.0250 + D.0049
0.0023 + 0.0002
0.0014 0.0007
0.1011 = 0.1006
ST&D-F
2,50
0.3213
0.0152
0.0122 0.0042
0.01550 + D.01166
0.0019
0.0002
0.0010 :: 0.0004
0.0015 0.0001
UM&D-M
0.0790 0.0071
0.0168 4: 0,0050
0.0168 4: 0.0032
0,0017
0.0003
0.0012 1: 0.0006
0.0029 =: 0.0005
P1 BT
0.0638
0.0024
0.0012 + 0.0002
0.00220 +: 0.0002
0,0004 :: 0.0002
0.0002 + 0.0001
D.0009 =: 0.0002
P1 AT
0.0598
0.0014
0.0036 0.0001
0.0015
D.0001
0.0001
0.0001 0.0001
0.0002 0.0011
P2 BT
0.0824 0.0021
0.0012
+ 0.0001
0.0015 +: 0.0001
0.0002
0.0001
0.0020 = 0.0008
D.0000 = 0.0003
P2 AT
1.20
0,0738
0.0053
0.0011 + 0.0002
0.0013 + 0.0002
0.0002 0.0001
0.0015 + 0.0008
0.0007 - 0.0003
Fig. shows a visualization of the embedding of the two breast cancer datasets before and after HPA. For visualization, we project the points in 3 to the 3D Poincaré ball. Before the alignment, the dominant factor separating between the patients' samples (points) is the batch. In contrast, after the aligmment, the batch effect is substantially suppressed (visually) and the factors separating the points are dominated by the cancer subtype.
We evaluate the quality of the alignment in two aspects using objective measures: (i) k-NN clas- sification, with battc-on-batch-out coossvalldation, is utilized for assessing the alignment of the intrinsic structure, and (ii) MMD 19] is used for assessing the distribution alignment quality. For the classification, we view the five subtypes of BC, the three subtypes of LC, and the presence of stimulated cells in CyTOF as the labels in the respective tasks. In addition to the results of the different aligmment methods, we report the k-NN classification based only on a single batch (S Baselline). which indicates the adequacy of the representation in hyperbolic space to the task at hand.
Table I depicts the k-NN classification obtained for the best k per method, and Table shows the MMD. In each task, we set the dimension of the Lorentz model d to the dimension in which the best empirical single-task performance is obtained (S-Baseline). We note that similar results and trends are obtained for various dimensions. Additional results for various i values and an ablation study, showing that the combination of all three components yields the best classification results, are reported in Appendix D
Although the OT-based methods obtain the best matching between the distributions of the batches, HPA outperforms in all three tasks in terms of classification (see Table I In the two gene expression tasks, where the data have multiple labels and we align multiple batches, the advantage of HPA compared to the other methods is particularly significant. In Appendix D we demonstrate HPA's out-of-sample capabilities the CyTOF data by learning the batch correction map between the different days from one patient and applying it to the data of the other patient.
4.3 Discussion
Alignment methods based on density matching, such as OT-based methods, often overlook an important aspect in purely unsupervised settings. Although sample density is the main data property that can be and need to be aligned, preserving the intrinsic structure/geometry of the sets is important, as it might be tightly related to the hidden labels. Indeed, we see in our experiments that OT- based methods provide a good density aligmment (reducing the inter-set variability), as demonstrated by small MMD values (see Table D However, the intrinsic structure of the sets (the intra-set variability) not preserved, as evident by the resulting poor (hidden) label matching, conveyed by the k-NN classification performance (see Table This is also illustrated in the right panel of Fig. [ There it visible that the three OT-based methods provide good global alignment of the sets, yet the intrinsic structure is not kept, as implied by the poor color matching.

---

In contrast to OT-based methods, HPA does not explicitly aim to match densities, and thus, it obtains slightly worse MMD performance compared to OT-based methods. However, HPA matches the first two moments of the density and includes the rotation component, which was shown to be one of the fundamental limitations of OT-based methods for alignment as OT cannot recover volume-presere preserving As seen in the simulation and experimental results and illustrated in the right panel of Fig. D we still obtain a good global alignment and simultaneously preserve the intrinsic structure, allowing for high classification performance. We remark that in the synthetic examples, there is (hidden) one-to-one correspondence between the sets, and therefore one-to-one discrepancy can be computed (instead or in addition to MMD). When there is such a correspondence, OT still cannot recover volume- volume-preservinn preserving maps, while HPA can mitigate noise and distortions.
maps
4 KG
5
Conclusion
We introduced HPA for label-free alignment of data in the Lorentz model. Based on Riemannian geometry, we presented new translation and scaling operations that align the first and second Rie- mannian moments as well as wrapped rotation that aligns the orientation in the hyperboloid model. Our theoretical analysis provides further insight and highlights properties that may be useful for practitioners. We empirically showed in simulations that HPA is stable under noise and distortions. Experimental results involving purely unsupervised batch correction of multiple bioinformatics datasets with multiple labels is demonstrated. Beyond alignment and batch effect removal, our method can viewed as type of domain adaptation or a precursor transfer learning that relies on purely geometric considerations, exploiting the geometric structure of data as well as the geometric properties of the space of the data. In addition, it can be utilized for multimodal data fusion and geometric registration of shapes with hierarchical structure.
Acknowledgments and Disclosure of Funding
We thank the reviewers for their important comments and suggestions, and thank Thomas Dagès for the helpful discussion. The work of YEL and RT was supported by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 802735-ERC-DIFFOP The work of YK was supported by the National Institutes of Health RO1GM131642, UMIPA05141, P50CA121974, and U0IDA053628.
References
[1] B. Afsari. Riemannian 1l center mass: existence, uniqueness, and convexity. Proceedings of the American Mathematical Society, 139(2):655-6733 2011.
[2] D. Alvarez-Melis, Y. Mrouch, and T. Jaakkola. Unsupervised hierarchy matching with optimal transport over hyperbolic spaces. In International Conference on Artificial Intelligence and Statistics, pages 1606-1617. PMLR, 2020.
[3] Y. Brenier. Décomposition polaire et réarrangement monotone des champs de vecteurs. CR Acad. Sci. Paris Sér. I Math., 305:805-8000 808, 1987.
[4] Y. Brenier. Polar factorization and monotone rearrangement of vector-valued functions. Com- munications on pure and applied mathematics, 44(4):375 417, 1991.
[5] R. Chakraborty, D. Seo, and B. C. Vemuri. An efficient exact-pga algorithm for constant curvature manifolds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3976 9976-3984, 2016.
[6] I. Chami, Z. Ying, C. Ré, and J. Leskovec. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 0184868-4877 2019.
[7] N. Courty, R. Flamary, D. Tuia, and A. Rakotomamonjy. Optimal transport for domain adaptation, IEEE transactions on pattern analysis and machine intelligence, 1853 2016.
10

---

[8] C. Curtis, S. P. Shah, S.-F. Chin, G. Turashvili, O. M. Rueda, M. J. Dunning, D. Speed, A. G. Lynch, Samarajiwa, Y. Yuan, et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486(74003) 2012.
[9] M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26:2292-2300 2013.
[10] Euzenat and Shvaiko. Ontology matching: State of the and future challenges. IEEE Transactions on knowledge and data engineering, 25(1):158-176, 2013.
[11] L. Falorsi, P. de Haan, T. R. Davidson, and P. Forré. Reparameterizing distributions lie groups. In The 22nd International Conference on Artificial Intelligence and Statistics, pages $244-3253. PMLR, 2019.
[12] R. Flamary and N. Courty. Pot python optimal transport library. con/rftamaryPOOT, page 144, 2017.
GitHub: https://githu.
[13] M. Fréchet. Les éléments aléatoires de nature quelconque dans un espace distancié. In Annales de I' l'institut Henri Poincaré, volume 10, pages 215-310, 1948.
[14] O. Ganea, G. Bécigneul, and Hofmann. Hyperbolic entailment cones for learning hierarchical embeddings. In International Conference on Machine Learning, pages 1646-1655. PMLR, 2018.
[15] O. Ganea, G. Becignnul, and T. Hofmann. Hyperbolic neural networks. In Advances in Neural Information Processing Systems (NIPS), page 5350-5360, 2018.
[16] I. Gohberg, P. Lancaster, and L. Rodman. Matrices and indefinite scalar products. 1983.
[17] C. Gower. Generalized procrustes analysis. Psychometrika, 40(1):33-51, 1975.
[18] J. C. Gower, G. B. Dijksterhuis, et al. Procrustes problems, volume 30. Oxford University Press on Demand, 2004.
[19] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
[20] D. Groisser. Newton's 's method, zeroes of vector fields, and the riemannian center of mass. Advances in Applied Mathematics, 33(1):95-133) -135, 2004.
[21] D. N. Hayes, S. Monti, G. Parmigiani, C. B. Gilks, K. Naoki, A. Bhattacharjee, M. A. Socinski. C. Perou, and M. Meyerson. Gene expression profiling reveals reproducible human lung adenocareinoma subtypes in multiple independent patient cohorts. Journal of Clinical Oncology, 24(31):5079-509) 2006.
[22] A. Hoyos-Idrobo. Aligning hyperbolic representations: an optimal transport-baue based approach. arXiv preprint arXi::20/2.0/08) 2020.
[23] J. Hull. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.
[24] H. Karcher. Riemannian center of mass and mollifier smoothing. Communications on pure and applied mathematics, 30(5):509 5411, 1977.
[25] A. Klimovskaia, D. Lopez- Paz, L. Bottou, and M. Nickel. Poincaré maps for analyzing complex hierarchies in single-cell data. Nature communications, 11(1):1-9, 2020.
[26] D. Koboldt, R. Fulton, M. McLellan, H. Schmidt, J. Kalicki- Veizer, J. McMichael. L. Fulton,
D. Dooling, L. Ding, E. Mardis, et al. Comprehensive molecular portraits of human breast tumours. Nature, 490(7418):61-70, 2012.
[27] D. Kohli, Cloninger, and G. Mishne. LDLE: Low distortion local eigenmaps. In ICLR 2021 Workshop Geometrical and Topological Representation Learning, 2021.

---

[28] I. Korsunsky, N. Millard, J. Fan, K. Slowikowski, F. Zhang, K. Wei, Y. Baglaenko, M. Brenner, P.-I. Loh, and Raychaudhuri. Fast, sensitive and accurate integration of single-cell data with harmony. Nature methods, 16(12):1289-122: 2019.
[29] A. Kumagai and T. Iwata. Unsupervised domain adaptation by matching distributions based on the maximum mean discrepancy via unilateral transformations. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pages 1106-41133 2019.
[30] M. Law, R. Liao, J. Snell, and R. Zemel. Lorentzian distance learning for hyperbolic rep- resentations. In International Conference on Machine Learning, pages 3672-3681. PMLR, 2019.
[31] Y. LeCun, L. Bottou, Y. Bengio, and Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
[32] Q. Liu, M. Nickel, and D. Kiela. Hyperbolic graph neural networks. In Advances in Neural Information Processing Systems (NIPS), pages 8228-8239, 2019.
[33] A. Lou, I. Katsman, Jiang, S. Belongie, S.-N. Lim, and C. De Sa. Differentiating through the fréchet mean. In International Conference on Machine Learning (ICML), pages PMLR, 2020.
[34] B. Luo and E. Hancock. Feature matching with procrustes alignment and graph editing. 1999.
[35] H. Maron, N. Dym, Kezurer, S. Kovalsky, and Y. Lipman. Point registration via efficient convex relaxation, ACM Transactions on Graphics (TOG), 35 2016.
[36] R. J. McCann. Polar factorization of maps on riemannian manifolds. Geometric & Functional Analysis GAFA, 11(3):589-608, 2001.
[37] H. H. Milioli, R. Vimieiro, I. Tishchenko, C. Riveros, R. Berretta, and P. Moscato. Iteratively refining breast cancer intrinsic subtypes in the metabric dataset. BioData mining, 911::1-8, 2016.
[38] G. Monge. Mémoire sur la théorie des déblais et des remblais. Histoire de l'Académie Royale des Sciences de Paris, 1781.
[39] Y. Nagano, S. Yamaguchi, Y. Fujita, and M. Koyama. A wrapped normal distribution on hyperbolic space for -radient-based learning. In International Conference Machine Learning (ICML), pages 4693-4702. PMLR, 2019.
[40] I. Naim, Y. Song, Q. Liu, H. Kautz, J. Luo, and D. Gildea. Unsupervised alignment of natural language instructions with video segments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
[41] M. Nickel and D. Kiela. Poincaré embeddings for learning hierarchical representations. In Advances in Neural Information Processing Systems (NIPS), page 6341-6350 2017.
[42] M. Nickel and D. Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In International Conference on Machine Learning (ICML), pages 3779-3788. PMLR, 2018.
[43] V. Nygaard, E. A. Rødland, and E. Hovig. Methods that remove batch effects while retaining group differences may lead to exaggerated confidence in downstream analyses. Biostatisties, 17(1):29-33 -9,999999999 2016.
[44] X. Pennec. Barycentric subspace analysis manifolds. The Annals of Statisties, 46(6A):2711- 2746, 2018
[45] G. Peyré and M. Cuturi. Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning, 1105-6):355-077 2019.
[46] G. Ratcliffe, Axler, and K. Ribet. Foundations of hyperbolic manifolds, volume 149. Springer, 1994.
12

---

[47] P. L. C. Rodrigues, C. Jutten, and M. Congedo. Riemannian procrustes analysis: Trans- fer learning for brain-come computer interfaces. IEEE Transactions on Biomedical Engineering. (0(8):2390-2401, 2018.
[48] U. Shaham, K. P. Stanton, J. Zhao, H. Li, K. Raddassi, R. Montgomery, and Y. Kluger. Removal of batch effects using distribution-matee matching residual networks. Bioinformatics, 33(16):2539- 2546, 2017.
[49] D. Spohr, L.. Hollink, and P. Cimiano. A machine learning approach to multilingual and cross-lingual ontology matching. In International Semantic Web Conference, pages 665-680. Springer, 2011.
[50] T. Stuart, A. Butler, P. Hoffman, C. Hafemeister, E. Papalexi, W. M. Mauck III, Y. Hao, M. Stoeckius, P. Smibert, and R. Satija. Comprehensive integration of single-cell data. Cell, 177(7):1888 1902, 2019.
[51] Tabaghi and Dokmanic. On procrustes analysis in hyperbolic space. arXiv preprint 2021.
[52] C. Wang and S. Mahadevan. Manifold alignment using procrustes analysis. In In International Conference on Machine Learning (ICML), pages 1120-1127, 2008
[53] O. Yair, M. Ben-Chen, and R. Talmon. Parallel transport on the cone manifold of spd matrices for domain adaptation. IEEE Transactions Signal Processing, 67(7): 2019.
[54] O. Yair, F. Dietrich, Talmon, and I. G. Kevrekidis. Domain adaptation with optimal transport on the manifold of spd matrices. arXiv e-prints, pages arXiv- 2019.
13


Reviews:
[{"review_id": "twrTDiNBpSo", "reviewer": "Reviewer_2HGp", "summary": "The need for alignment often arises since data are acquired in multiple domains, under different environmental conditions, using various acquisition equipment, and at different sites. This paper focuses on the problem of unsupervised alignment of data embedded in hyperbolic spaces. ", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper presents the Lorentz model for unsupervised label-free hierarchical data alignment.\nThey also provide theoretical analysis and justification of the alignment method. The theoretical analysis provides further insight and highlights properties that may be useful. Their experiments also indicate that HPA is stable under noise and distortions.\n\nThe most interesting thing is that the authors also experiment on two datasets that arguably do not have a distinct hierarchical structure but are highly effective.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors has adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "kFLx8B3G8kD", "reviewer": "Reviewer_rvxG", "summary": "The authors presented algorithm based on Riemannian geometric tools to align densities on Hyperbolic space. Experimental results have been demonstrated on synthetic and breast cancer data. The results have shown efficiency of the proposed method in terms of lower discrepancy and less computational time requirement. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Post rebuttal and discussion, I want to recommend acceptance.\n\n\n\nThe overall paper is an easy read. although my main concern is\n(a) The existence and uniqueness of FM should be addressed to ensure validity of Algorithm 1.\n(b) Most of the propositions are true due to the choice of metric and property of PT. So why Algorithm 1 does not valid for other Riemannian manifolds? The authors should point out what is so special about Hyperbolic spaces? \n(c) The experimental results are limited only to a synthetic dataset and the batch effect removal on the breast cancer dataset. Also the comparison is subpar.\n\nDetailed Comments:\n(a) line 83, the authors should cite many other older papers like [Groisser Riemannian Newton .., 2003], [Chakraborty et al., Exact PGA ..., 2014].\n(b) As mentioned earlier, most of the propositions like Proposition 5, 8 follow directly from the property of PT.\n(c) The steps for Riemannian wrapped rotation should be presented as bulleted lists or algorithm for better clarity.\n(d) The main comparison with [18] is not enough, few obvious baselines including doing aligning in the ambient space, using Log Euclidean framework should be presented.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors mentioned they did not have any potential negative societal impact which I agree.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "hA0FIf_xFPM", "reviewer": "Reviewer_Mb2Y", "summary": "This paper proposes Hyperbolic Procrustes Analysis (HPA) for aligning point sets that live in hyperbolic spaces. The proposed algorithm uses the tools from Riemannian geometry to correctly and efficiently generalize the (generalized) Procrustes alignment to Riemannian manifolds. The paper deploys this algorithm to perform unsupervised, label-free, and hierarchical data alignment. The experiments demonstrate that HPA can yield higher accuracy when compared to optimal transport based alignment.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "PROs\n\n- I think generalizing Procrustes alignment to Riemannian manifolds carries fundamental importance. The approach presented in this paper is intuitive, principled and makes sense. \n- The paper is clearly written explaining different stages of the algorithm. It is easy to follow and understand for readers with certain background.\n- The paper conducts an appropriate evaluation on hierarchical datasets and problems where such an alignment is superior to the existing optimal transport based ones.\n\nCONs\n\n1. Procrustes alignment only considers translation, scale and rotation to align two datasets. For a high dimensional problem, I seem to lose the motivation why (and in which cases) this would be superior to say a hierarchical OT? Can we have a general recipe in how to choose which algorithm to use? In other words, intuitively, why would this perform better than OT for a given dataset? It would be good to clarify this point. \n2. It seems like a large portion of the material addresses a general Riemannian manifold. However, when we dig to the specifics of the proofs the Lorenz model comes along. Is it possible to consider a similar alignment procedure not for hyperbolic spaces but for general manifolds? \n3. A lot of ablation studies seem to be missing here. First, as mention in Sec. 3.3, there could be multiple ways to solve the rotation problem. The paper argues for the *wrapped rotation*. I would agree, but this choice is not experimentally validated. Can we see a study of different rotational alignment strategies?\n4. Prop. 10 seems to assume that the dataset is generated by an inverse Procrustes. Why should this be the case?\n5. For the alignment of multiple point clouds (the generalized problem) : would it not make sense to fix one of the point clouds and align the rest? Yes or no, it would be good to discuss why.\n6. Can we see visualizations from other methods as well? This might help in grasping certain problems with the state of the art.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As this work is a theoretical one, the authors avoid including a broader impact statement. However, in general, the paper really fails to discuss the limitations from an empirical perspective. Besides, I also do not think that Section 3 does a good job of discussing thoroughly the theoretical limitations. What are the theoretical cases to break the algorithm? Can we speak of a convergence basin, at least for the hyperbolic geometry? ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "GXCx2duoimk", "reviewer": "Reviewer_VwLT", "summary": "This paper studies the unsupervised alignment of hierarchical datasets embedded in hyperbolic spaces, by proposing a hyperbolic procrustes analysis  (HPA) method. HPA uses new hyperbolic translations, rotations and scaling methods to align datasets. Experiments on synthetic and real datasets show that HPA is able to remove noise from data collection to align datasets. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe paper is well-written. The theorems and propositions in this paper are precisely formulated and clearly written. The authors clearly explained why each theorem is important to the paper.\nExperiments are convincing, and the downstream tasks in 4.2 are good experiments to show the quality and utility of alignment.\n\nSuggestions/Comments:\nLine 167 (\"Second, rotation applied directly to the tangent space [...] does not guarantee that the rotated points remain on the same tangent space\"). I don't understand this sentence. The tangent space of L^d at \\bar{h}^{(1)}, just like the tangent space of any Riemannian manifold at any point, is isometric to the Euclidean space R^d. Thus, we can identify it with R^d using an isometry then perform Euclidean rotations there (then identify things back to the tangent space). If rotations in Euclidean space do not throw points out of the space, why would that happen for rotations in the tangent space T_{\\bar{h}^{(1)}} L^d ? \n\nIf, instead of using the wrapped rotations, we (1) map points to the tangent space T_{\\bar{h}^{(1)}} L^d using the log map, (2) perform Euclidean rotations on this tangent space around the center \\bar{h}^{(1)}, and (3) map the points back to L^d using the exponential map, then the resulting map have the advantage of being an isometry of L^d: In fact, it will just be a rotation of L^d around the point \\bar{h}^{(1)}. In particular, it preserves the mean \\bar{h}^{(1)}.\n\nNote that the authors' comment in line 165 (about rotation maps in L^d not necessarily preserving the mean) does not really apply here: Rotations in L^d can totally preserve the mean, as long as we rotate around the mean itself, as suggested above. While the rotation used in [46] indeed might not preserve the mean \\bar{h}, that is only because that rotation is based at the \"bottom\" of the hyperboloid and not the mean \\bar{h}. I don't see why we have to stick with that particular rotation, given that the hyperbolic space is homogenous so there is nothing intrinsically special about the \"bottom\" point.\n\nTo summarize: I feel that the rotation of L^d around \\bar{h} has a few advantages over the wrapped rotations suggested in 3.3: It is (1) an isometry, (2) quite natural, and (3) intrinsic to the geometry of the hyperbolic space (i.e. not depends on any specific model or coordinate systems). Meanwhile, it still preserves the mean \\bar{h} and has a closed-form expression, as the authors desire. Therefore, I do not understand why the remarks in line 165-168 are valid reasons to discard this rotation. Perhaps the authors have tried this and found that it performed worse empirically? If so, it would be nice to see that analysis.\n\nPS. Note that if we replace the wrapped operations by the usual rotation above, then the whole alignment method is essentially just (1) map points to tangent spaces using logarithm maps, (2) translate them to the same tangent space using parallel transport, (3) perform Euclidean alignment on the tangent space, and (4) map points back to L^d using exponential map. This is a simple baseline that can be efficiently computed and can leverage whatever Eucllidean alignment techniques we want.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper looks at aligning different hyperbolic datasets (which consists of elements in the hyperbolic space). In particular, it proposes to align the means, scale the dispersions, and rotate to align the elements. The proposed approach shows good performance on different datasets. It also rigorously compares with different baselines. A minor comment. How does the analysis extent to cases when the datasets are in different dimensions? 


Decision: Accept (Poster)