L EARNING H YPERBOLIC R EPRESENTATIONS OF T OPOLOGICAL F EATURES
Anonymous authors
Paper under double-blind review
A BSTRACT
Learning task-speciﬁc representations of persistence diagrams is an important problem in topological data analysis and machine learning.
However, current state of the art methods may bottleneck their representational capacity as they are focused on Euclidean representations.
Persistence diagrams often contain features of inﬁnite persistence (i.e., essential features ) and Euclidean spaces shrink their importance relative to non-essential features because they cannot assign inﬁnite distance to ﬁnite points.
To deal with this issue, we propose a method to learn representations of persistence diagrams on hyperbolic spaces, more speciﬁcally on the Poincare ball.
By representing features of inﬁnite persistence inﬁnitesimally close to the boundary of the ball, their distance to non-essential features approaches inﬁnity, thereby their relative importance is preserved.
This is achieved without utilizing extremely high values for the learnable parameters, thus the representation can be fed into downstream optimization methods and trained efﬁciently in an endto-end fashion.
We present experimental results on graph and image classiﬁcation tasks and show that the performance of our method is on par with or exceeds the performance of other state of the art methods.
I NTRODUCTION
Persistent homology is a topological data analysis tool which tracks how topological features (e.g. connected components, cycles, cavities) appear and disappear as we analyze the data at different scales or in nested sequences of subspaces (1; 2).
These nested subspaces are known as ﬁltrations .
As an informal example of a ﬁltration consider an image of variable brightness.
As the brightness is increased, certain features (edges, texture) may become less or more prevalent.
The birth of a topological feature refers to the "time" (i.e., the brightness value) when it appears in the ﬁltration and the death refers to the "time" when it disappears.
The lifespan of the feature is called persistence .
Persistent homology summarizes these topological characteristics in a form of multiset called persistence diagram .
Persistence diagrams are highly robust and versatile descriptors of the data due to their stability and injectivity properties.
Stability ensures that the diagrams of two similar objects are similar (3) and injectivity implies that one can approximately reconstruct the data given diagrams obtained under certain ﬁltrations (4).
However, despite their strengths, the space of persistence diagrams lacks structure as basic operations, such as addition and scalar multiplication, are not deﬁned.
The only imposed structure is induced by the Bottleneck and Wasserstein metrics, which are notoriously hard to compute, thereby preventing us from leveraging them for machine learning tasks.
Related Work .
To address these issues, several vectorization methods have been proposed.
Some of the earliest approaches are based on kernels , i.e., generalized products that turn persistence diagrams into elements of a Hilbert space.
In (5) Kusano et al.
propose a persistence weighted Gaussian kernel which allows them to explicitly control the effect of persistence.
Alternatively, Carrière et al.
(6) leverage the sliced Wasserstein distance to deﬁne a kernel that mimics the distance between diagrams.
The approaches by Bubenik (7) based on persistent landscapes, by Reininghaus et al.
(8) based on scale space theory and by Le et al.
(9) based on the Fisher information metric are along the same line of work.
The major drawback in utilizing kernel methods is that they suffer from scalability issues as the training scales poorly with the number of samples.
In another line of work, researchers have constructed ﬁnite-dimensional embeddings , i.e., transformations turning persistence diagrams into vectors in a Euclidean space.
Adams et al.
(10) map the diagrams to persistence images and discretize them to obtain the embedding vector.
In (11) Carrière et al.
develop a stable vectorization method by computing pairwise distances between points in the persistence diagram.
An approach based on interpreting the points in the diagram as root of a complex polynomial is presented by Di Fabio (12).
Adcock et al.
(13) identify an algebra of polynomials on the diagram space that can be used as coordinates and the approach is extended by Kališnik in (14) to tropical functions which guarantee stability.
The common drawback of these embeddings is that the representation is pre-deﬁned, i.e., there exist no learnable parameters, therefore, it is agnostic to the speciﬁc learning task.
This is clearly sub-optimal as the eminent success of deep learning has demonstrated that it is preferable to learn the representation.
The more recent approaches aim at learning the representation of the persistence diagram in an end-to-end fashion.
These approaches are broadly based on permutation-invariant input layers for learning tasks deﬁned on unstructured sets of ﬁxed cardinality, such as the Deep Sets developed by
Zaheer et al.
(15) and the PointNet by Qi et al.
Nonetheless, the space of persistence diagrams is equipped with a metric and the diagrams could be of varying cardinality, which renders the previous approaches not applicable.
Hofer et al.
(17) present a ﬁrst input layer based on a parameterized family of Gaussian-like functionals, with the mean and variance learned during training.
They extend their method in (18) allowing for a broader class of parameterized function families to be considered.
It is quite common to have topological features of inﬁnite persistence (1), i.e., features that never die.
Such features are called essential and in practice are usually assigned a death time equal to the maximum ﬁltration value.
This may bottleneck their representational capacity because it shrinks their importance relative to non-essential features.
The work by Carrière et al.
(19), which introduces a network input layer the encompasses several vectorization methods, emphasizes the importance of essential features and is the ﬁrst one to introduce extended persistence as a way to deal with them.
In this paper, we approach the issue of essential features from the geometric viewpoint.
We are motivated by the recent success of hyperbolic geometry and the interest in extending machine learning models to hyperbolic spaces or general manifolds.
We refer the read to the review paper by Bronstein et al.
(20) for an overview of geometric deep learning.
Here, we review the most relevant and pivotal contributions in the ﬁeld.
Nickel et al.
(21; 22) propose Poincaré and Lorentz embeddings for learning hierarchical representations of symbolic data and show that the representational capacity and generalization ability outperform Euclidean embeddings.
In (23), Ganea et al.
generalize neural networks to the hyperbolic space and show that hyperbolic sentence embeddings outperform their
Euclidean counterparts on a range of tasks.
Gulcherhe et al.
(24) introduce hyperbolic attention networks which show improvements in terms of generalization on machine translation and graph learning while keeping a compact representation.
In the context of graph representation learning, hyperbolic graph neural networks (25) and hyperbolic graph convolutional neural networks (26) have been developed and shown to lead to improvements on various benchmarks.
However, despite this success of geometric deep learning, little work has been done in applying these methods to topological features, such as persistence diagrams.
The main contribution of this paper is to bridge the gap between topological data analysis and hyperbolic representation learning.
We introduce a method to represent persistence diagrams on a hyperbolic space, more speciﬁcally on the Poincare ball.
We deﬁne a learnable parameterization of the Poincare ball and leverage the vectorial structure of the tangent space to combine (in a manifoldpreserving manner)
the representations of individual points of the persistence diagram.
Our method learns better task-speciﬁc representations than the state of the art because it does not shrink the relative importance of essential features.
In fact, by allowing the representations of essential features to get inﬁnitesimally close to the boundary of the Poincare ball, their distance to the representations of non-essential features approaches inﬁnity, therefore preserving their relative importance.
To the best of our knowledge, this is the ﬁrst approach for learning representations of persistence diagrams in non-Euclidean spaces.
B ACKGROUND
In this section, we provide a brief overview of persistent homology leading up to the deﬁnition of persistent diagrams.
We refer the interested reader to the papers by Edelsbrunner et al.
(1; 2) for a detailed overview of persistent homology.
An overview of homology can be found in the Appendix.
Persistent Homology .
Let K be a simplicial complex.
A ﬁltration of K is a nested sequence of subcomplexes that starts with the empty complex and ends with K ,
[Equation]
A typical way to construct a ﬁltration is to consider sublevel sets of a real valued function, f : K → R .
Let a 1 < · · · < a d be a sorted sequence of the values of f ( K ) .
Then, we obtain a ﬁltration by setting
K 0 = ∅ and K i =
f − 1 (( −∞ , a i ]) for 1 ≤ i ≤ d.
[Equation]
We can apply simplicial homology to each of the subcomplexes of the ﬁltration.
When 0 ≤ i ≤ j ≤ d , the inclusion K i ⊂ K j induces a homomorphism
[Equation]
on the simplicial homology groups for each homology dimension n .
We call the image of f i,jn a n -th persistent homology group and it consists of homology classes born before i that are still alive at j .
A homology class α is born at K i if it is not in the image of the map induced by the inclusion
K i − 1 ⊂ K i .
Furthermore, if α is born at K i , it dies entering K j if the image of the map induced by
K i − 1 ⊂ K j − 1 does not contain the image of α but the image of the map induced by K
i − 1 ⊂ K j does.
The persistence of the homology class α is j − i .
Since classes may be born at the same i and die at the same j , we can use inclusion-exclusion to determine the multiplicity of each ( i, j ) ,
[Equation]
where the n -th persistent Betti numbers β i,jn are the ranks of the images of the n -th persistent homology group, i.e., β i,jn = rank
( im ( f i,jn )) , and capture the number of n -dimensional topological features that persist from i to j .
By setting µ i, ∞ n
[Equation]
we can account for features that still persist at the end of the ﬁltration ( j = d ), which are known as essential features .
Persistence Diagrams .
Persistence diagrams are multisets supported by the upper diagonal part of the real plane and capture the birth/death of topological features (i.e., homology classes) across the ﬁltration.
Deﬁnition 2.1 (Persistence Diagram) .
Let ∆ = { x ∈ R ∆ : mult ( x ) = ∞} be the multiset of the diagonal R ∆ = { ( x 1 , x 2 ) ∈ R 2 : x 1 = x 2 } , where mult ( · ) denotes the multiplicity function and let R∗
= { ( x 1 , x 2 ) ∈ R ∪
( R ∪ ∞ ) : x 2 > x 1 } .
Also, let n be a homology dimension and consider the sublevel set ﬁltration induced by a function f : K → R over the complex K .
Then, a persistence diagram , D n ( f ) , is a multiset of the form D n ( f ) =
{ x : x ∈ R 2 ∗ } ∪ ∆ constructed by inserting each point ( a i , a j ) for i < j with multiplicity µ i,jn (or µ i, ∞ n if it is an essential feature).
We denote the space of all persistence diagrams with D .
Deﬁnition 2.2 (Wasserstein distance and stability) .
Let D n ( f ) , E n ( g ) be two persistence diagrams generated by the ﬁltration induced by the functions f, g : K → R , respectively.
We deﬁne the
Wasserstein distance
[Equation]
[Equation]
where p, q ∈ N and the inﬁmum is taken over all bijections η : D n ( f ) → E n ( g ) .
The special case p = ∞ is known as Bottleneck distance .
The persistence diagrams are stable with respect to the
Wasserstein distance
if and only if w qp ( D n ( f ) , E g ( g )) ≤ (cid:107) f − g (cid:107) ∞ .
Note that a bijection η between persistence diagrams is guaranteed to exist because their cardinalities are equal, considering that, as per Def. 2.1, the points on the diagonal are added with inﬁnite multiplicity.
The strength of persistent homology stems from the above stability deﬁnition, which essentially states that the map taking a sublevel function to the persistence diagram is Lipschitz continuous.
This implies that if two objects are similar then their persistence diagrams are close.
P ERSISTENT P OINCARE R EPRESENTATIONS
In this section, we introduce our method (Fig. 1) for learning representations of persistence diagrams on the Poincare ball.
We refer the reader to the Appendix for some fundamental concepts of differential geometry.
C o m p l e x K , F ilt r a ti on f
D n ( f ) birth death ∆ • x• x• • • • φ ◦ ρ ( x ) yy
+ θ yy
+ θ
T x 0 B xy y z
Φ( D , θ )
The Poincare ball is an m -dimensional manifold ( B , g B x ) , where B = { x ∈ R m : (cid:107) x
(cid:107) < 1 } is the open unit ball.
The space in which the ball is embedded is called ambient space and is assumed to be equal to R m .
The Poincare ball is conformal (i.e., angle-preserving) to the Euclidean space but it does not preserve distances.
The metric tensor and distance function are as follows
[Equation]
where g E =
I m
is the Euclidean metric tensor.
Eq.
6 highlights the beneﬁt of using the Poincare ball for representing persistence diagrams.
Contrary to Euclidean spaces, distances in the Poincare ball can approach inﬁnity for ﬁnite points.
This space is ideal for representing essential features appearing in persistence diagrams without squashing their importance relative to non-essential features.
Informally, this is achieved by allowing the representations of the former ones to get inﬁnitesimally close to the boundary, thereby their distances to the later ones approach inﬁnity.
Fig.
2 provides an illustration.
We gradually construct our representation through a composition of 3 individual transformations.
The ﬁrst step is to transfer the points to the ambient space (i.e., R m ) of the Poincare ball.
Let D 1 be a persistence diagram.
We introduce the following auxiliary transformation
[Equation]
Even though our formalism does not prohibit us from placing learnable parameters in Eq.
7, our main focus is to learn a hyperbolic representation rather than a higher dimensional Euclidean embedding.
Therefore, we assume that this embedding is parameter-free.
The second step is to project the embedded points from the ambient space to the Poincare ball.
When referring to points on a manifold, it is important to deﬁne a coordinate system.
A homeomorphism ψ : B → R m is called coordinate chart and gives the local coordinates on the manifold.
The inverse map φ : R m → B , is called a parameterization of B and gives the ambient coordinates .
The main idea is to inject learnable parameters into this parameterization.
The injected parameters could be any form of differentiable functional that preserves the homomorphic property.
Differentiability is needed such that our representation can be fed to downstream optimization methods.
In our construction, we utilize a variant of the generalized spherical coordinates.
Let θ ∈ Θ be a vector of m parameters.
We deﬁne the learnable parameterization φ : R m × Θ → B as follows
[Equation]
where r 2 i
= x 2 m + · · · + x 2 i +1 + x 2 i + (cid:15) .
The small positive constant (cid:15) is added to ensure that the denominator in Eq.
8 is not zero.
Intuitively, Eq.
8 corresponds to scaling the radius of the point by a factor θ 1 and rotating it by θ i radians across the angular axes.
The scaling and rotation parameters are learned during training.
Note that the form of y 1 ensures that representation belongs in the unit ball for all values of θ 1 .
The coordinate chart is not explicitly used in our representation; it is provided in the Appendix for the sake of completeness.
The third step is to combine the representations of each individual point of the persistence diagram into a single point in the hyperbolic space.
Typically, in Euclidean spaces, this is done by concatenating or adding the corresponding representations.
However, in non-Euclidean spaces such operations are not manifold-preserving.
Therefore, we project the points from the manifold to the tangent space, combine the vectors via standard vectorial addition and project the resulting vector back to the manifold.
This approach is based on the exponential and logarithmic maps
[Equation]
and
[Equation]
The exponential map allows us to project a vector from the tangent space to the manifold and its inverse (i.e., the logarithmic map) from the manifold to the tangent space.
For a general manifold, it is hard to ﬁnd these maps as we need to solve for the minimal geodesic curve (see Appendix for more details).
Luckily, for the Poincare ball case, they have analytical expressions, given as follows
[Equation]
where ⊕ denotes the Möbius addition, which is a manifold-preserving operator (i.e., for any x, y ∈ B = ⇒ x ⊕ y ∈ B ).
The analytical expression is given in the Appendix.
The projections given by these maps are norm-preserving, i.e., for example, the geodesic distance from x to the projected point exp x ( v ) coincides with the metric norm (cid:107) v (cid:107) g induced by the metric tensor g .
This is an important property as we need the distance between points (and therefore the relative importance of topological features) to be preserved when projecting to and from the tangent space.
We now combine the aforementioned transformations and deﬁne the Poincare hyperbolic representation followed by its stability theorem.
Deﬁnition 3.1 (Poincare Representation) .
Let D ∈ D be the persistent diagram to be represented in an m -dimensional Poincare ball ( B , g B x ) embedded in R m and x 0 ∈ B be a given point.
The representation of D on the manifold B is deﬁned as follows:
[Equation]
where the exponential and logarithmic maps are given by Eq.
10 and the learnable parameterization and the auxiliary transformation by Eq.
8 and Eq. 7, respectively.
Theorem 1 (Stability of Hyperbolic Representation) .
Let D , E be two persistence diagrams.
Assume that the auxiliary transformation ρ is Lipschitz continuous and that its kernel is the diagonal R ∆ , i.e., ρ ( x ) = 0 for all x ∈ R ∆ .
Additionally, assume that x 0 = 0 .
Then, the hyperbolic representation given by Eq.
11 is stable w.r.t the Wasserstein distance when p
= 1 , i.e., there exists constant K >such that
[Equation]
(12) where d B is the geodesic distance and w g 1 is the Wasserstein metric with the q -norm replaced by the induced norm (cid:107)·(cid:107) g (i.e., the norm induced by the metric tensor g , see Appendix A.2).
The proof of Theorem 1 (given in the Appendix) results from a general stability theorem (3) and is on par with similar results for other vectorizations (10) or representations (17) of persistence diagrams.
One subtle difference is that Theorem 1 uses the induced norm rather than the q -norm appearing in the Wasserstein distance.
However, since the induced norm implicitly depends on the chosen point x 0 , which, per requirements of Theorem 1, is assumed to be equal to the origin, there is no substantial difference.
The fact that we require the auxiliary transformation ρ to be zero on the diagonal is important to theoretically guarantee stability.
Intuitively, this can be understood by recalling (Def.
2.1) that all (inﬁnite) points on the diagonal are included in the persistence diagram.
By mapping the diagonal to the zero and taking x 0 = 0 , we ensure that the summation in Eq.
11 collapses to zero when summing over the diagonal.
Finally, we note that the assumptions of Theorem 1 are not restrictive.
In fact, we can easily ﬁnd Lipschitz continuous transformations that are zero on the diagonal R ∆ , such as the exponential and rational transformations proposed in (17; 18).
[Figure]
H 0 H( B ,g B x ) g B x = λ 2 x Id ∼ (cid:15) −(cid:15)
≈
E XPERIMENTS
In this section, we present experiments with datasets of diverse type and size.
We focus on persistence diagrams extracted from graphs and grey-scale images.
In both cases, the learning task is classiﬁcation and we compare the performance of our method against other methods.
To show the versatility of our approach, we utilize a common network architecture across all of our simulations.
The architecture as well as other training details are discussed in the Appendix.
We implemented all methods in
TensorFlow 2.2 using the TDA-Toolkit 2 and the Scikit-TDA 3 for extracting persistence diagrams and run all experiments on the Google Cloud AI Platform.
We provide the code to reproduce the results in the supplementary material.
4. G RAPH C LASSIFICATION
In this experiment, we consider the problem of graph classiﬁcation.
We evaluate our approach using social graphs from (27).
The REDDIT-BINARY dataset contains 1000 samples and the graphs correspond to online discussion thread.
The task is to identify to which community a given graph belongs ( question/answer -based community or a discussion -based community).
The REDDIT-5K and the REDDIT-12K are larger variant of the former dataset that contain 5K and ∼ 12K graphs from 5 and 11 subreddits, respectively.
The task is to predict to which subreddit a given discussion graph belongs.
The IMDB-BINARY contains 1000 ego-networks of actors that have appeared together in any movie and the task is to identify the genre ( action or romance ) to which an ego-graph belongs.
Finally, the IMDB-MULTI contains 1500 ego-networks belonging to 3 genres ( action , romance , sci-ﬁ ).
For each dataset, we train our model using 80% of the sample graphs and use the remaining 20% for validation.
Graphs are special cases of simplicial complexes, therefore, deﬁning ﬁltrations is straightforward.
We use two methods: The ﬁrst one captures global topological properties and is based on shortest paths.
In this case, the graph G =
( V, E ) is lifted to a metric space ( V, d ) using the shortest path distance d : V × V → R ≥ 0 between two vertices, which can be easily proved to be a valid metric.
Then, we deﬁne the Vietoris-Rips complex of G as the ﬁltered complex V R s ( G ) that contains a subset of V as a simplex if all pairwise distances in that subset are less than or equal to s , or, formally,
[Equation]
This approach essentially interprets the vertices of the graph as a point cloud in a metric space, with the distance between points given by the shortest path between the corresponding vertices.
In the case of unweighted graph, we assign unit weight across edges.
In Fig.
2 we show a sample persistence diagram extracted with the Vietoris-Rips ﬁltration and the corresponding representation of the features on the Poincare ball.
The second method captures local topological properties and is based on vertex degree.
Given a graph G =
( V, E ) , a simplicial complex can be deﬁned as the union of the vertex 2 https
https:
-tda
[Table]
and edge sets, i.e., K = K 0 ∪ K 1 ,
where K
= { v : v ∈ V } and K 1 = { ( u, v ) : ( u, v ) ∈ E } .
The sublevel function is deﬁned as follows:
f ( v ) =
deg ( v ) for v ∈ K 0 and f ( u, v ) =
max { f ( u ) , f ( v ) } for ( u, v ) ∈ K 1 , where deg ( v ) is the vertex degree v .
Then, the ﬁltration is given by Eq. 2.
We compare our method against several state of the art methods for graph classiﬁcation.
In particular, we compare against : (1) the Weisfeiler-Lehman (WL) graph kernel by Shervashidze et al.
(28), (2) the Graphlet Kernel (GK) by (29), (3) the Deep Graph Kernel by Yanardag et al.
(27), (4) the
Patchy-SAN (PSCN) by Nieper et al.
(30), and (5) the Anonymous Walk Embeddings (AWE) by
Ivanov et al.
Additionally, we compare against the PersLay input layer by Carrière et al.
(19) which utilizes extended persistence as an alternative way to deal with essential features and the
Learnable Representation based on Gaussian-like structure elements (GLR) presented by Hofer et al.
in (17).
We consider two variants of our method: (1) the Persistent Poincare (P-Poinc), which is the original method as presented in Sec. 3, and (2) the Persistent Euclidean (P-Eucl), which is same as the previous method with the Poincare ball replaced with the Euclidean space.
Note that in the P-Eucl case the learnable parameterization (Eq.
8) reduces to a simple addition of the learnable parameters (because the coordinate chart is the identity map) and the exponential and logarithmic maps (Eq.10) reduce to the identity maps.
We run simulations for different manifold dimensions (ranging from m = 2 to m = 12 ) and report the mean and standard deviation of the accuracy on
Table 1.
We observe that the performance of our method is on par or exceeds the performance of the state of the art.
Also, notice that the performance of the P-Eucl method is poor which supports our initial motivation for representing persistence diagrams on the Poincare ball.
4. I MAGE C LASSIFICATION
Even though it is well known that convolutional neural network have achieved unprecedented success as feature extractors for images, we present an image classiﬁcation case-study using topological features as a proof-of-concept for our method.
Contrary to graphs, images are not inherently equipped with the structure of a simplicial complex.
In theory, we could construct Vietoris-Rips complexes, as in Eq. 13, by interpreting pixels as a point cloud.
However, this is not the most natural representation of an image, which has a grid structure.
Therefore, we exploit this structure by constructing cubical complexes , i.e., unions of cubes aligned on a 2D grid.
As in the case of graphs, we use two methods.
For the ﬁrst method, called cubical ﬁltration , we use the grey-scale image directly and represent each pixel as a 2 − cube.
Then, all of its faces (adjacent lower-dimensional cubes) are added to the complex K .
We get a sublevel function by extending the grey-scale value I ( v ) of pixel a v to all cubes in K as follows:
[Equation]
.epoch
t r a i n l o ss m =3 m
= 0 1 2 3 4 5 6 7 8 9 0 .0 .
98epoch v a li d a ti on acc u r ac y m =3 m
=
[Table]
In the second experiment, we consider the problem of image classiﬁcation.
We utilize two standardized datasets: the MNIST, which contains images of handwritten digits, and the Fashion-MNIST, which contains shape images of different types of garment (e.g., T-shirt, trouser, etc.).
Each dataset contains a total of 70K (60K train, 10K validation) grey-scale images of size 28 × 28 .
Both datasets are balanced and categorized into 10 classes.
Finally, a grey-scale ﬁltration is obtained using Eq. 2.
The second method is called height ﬁltration and uses the binarized version of the original image.
We deﬁne the height ﬁltration by choosing a direction v ∈ R 2 of unit norm and by assigning to each pixel p of value 1 in the binarized image a new value equal to (cid:104) p, v (cid:105) , i.e., the distance of pixel p to the plane deﬁned by v .
This creates a new grey-scale image which is then fed to the aforementioned cubical ﬁltration.
We note that the height ﬁltration deserves special attention in persistent homology because complexes ﬁltered with it along can be approximately reconstructed from their persistence diagrams (4).
On Fig.
3, we demonstrate how the height ﬁltration affects a gray-scale image and accentuates different parts.
In practice, we use direction vectors v that are distributed uniformly across the unit cycle.
We chose 30 and 50 directions for the MNIST and Fashion-MNIST, respectively.
We compare our method against three baselines that encompass all methods for handling persistence diagrams: (1) the Persistence Weighted Gaussian Kernel (PWGK) approach proposed by Kusano et al.
in (5), (2) the Persistent Images (PI) embedding developed by Adams et al.
in (10), and (3) the
Learnable Representation based on Gaussian-like structure elements (GLR) by Hofer et al.
As in the previous case-study, we consider the original (P-Poinc) and the Euclidean (P-Eucl) variant of our method.
We run our simulations for manifold dimensions equal to m = 3 and m = 9 and report the best results on Table 2.
Our method outperforms all other methods, in some cases by a considerable margin.
We also study how the manifold dimension affects the train loss and validation accuracy.
The results are shown in Fig.
Observe that for m = 9 the train loss decreases more rapidly and the validation accuracy increases more rapidly in the ﬁrst few epochs.
This suggest that a higher manifold dimension may be slightly better for representing persistence diagrams.
Additionally, we observed that the Poincare representation tends to generalize better than its Euclidean counterpart.
In fact, the validation accuracy of the P-Eucl method started decreasing after the ﬁrst 2-3 epochs, whereas the P-Poinc method showed a saturation rather than a decrease.
This was observed without modifying the dropout rate or any other hyper-parameters and is a strong empirical ﬁnding that demonstrates the superiority of Poincare representations.
C ONCLUSION
We presented the ﬁrst, to the best of our knowledge, method for learning representations of persistence diagrams in the Poincare ball.
Our main motivation for introducing such method is that persistence diagrams often contain topological features of inﬁnite persistence (i.e., essential features) the representational capacity of which may be bottlenecked when representing them in Euclidean spaces.
This stems from the fact that Euclidean spaces cannot assign inﬁnite distance to ﬁnite points.
The main beneﬁt of using the Poincare space is that by allowing the representations of essential features to get inﬁnitesimally close to the boundary of the ball their distance to non-essential features approaches inﬁnity, therefore preserves their relative importance.
Our work is along the lines of the recent trend to extend deep learning methods to non-Euclidean spaces and we hope that it will foster further research in the ﬁeld of hyperbolic representation learning.
A H OMOLOGY AND D IFFERENTIAL G EOMETRY A. H OMOLOGY
Homology is a general method for associating a chain of algebraic objects (e.g abelian groups) to other mathematical objects such as topological spaces.
The construction begins with a chain group , C n , whose elements are the n -chains, which for a given complex are formal sums of the n -dimensional cells.
The boundary homomorphism , ∂ n : C n → C n +1 , maps each n -chain to the sum of the ( n − 1) -dimensional faces of its n -cell, which is a ( n − 1) -chain.
Combining the chain groups and the boundary maps in the sequence, we get the chain complex:
[Equation]
[Equation]
The kernels, Z n = ker ( ∂ n ) , and the images, B n = im ( ∂ n +1 ) , of the boundary homomorphisms are called cycle and boundary groups, respectively.
A fundamental property of the boundary homomorphism is that its square is zero, ∂ n ∂ n +1 = 0 , which implies that, for every n , the boundary forms a subgroup of the cycle, i.e., im ( ∂ n +1 ) ⊆
ker ( ∂ n ) .
Therefore, we can create their quotient group
[Equation]
(16) which is called n -th homology group and its elements are called homology classes .
The rank, β n = rank ( H n ) , of this group is known as the n -th Betti number .
Even though homology can be applied to any topological space, we focus on simplicial homology , i.e., homology groups generated by simplicial complexes.
Let K be a simplicial complex and K n its n -skeleton.
The chain group C n ( K ) is the free abelian group whose generators are the n -dimensional simplexes of K .
For a simplex σ = [ x 0 , x n ] ∈ K n , we deﬁne the boundary homomorphism as
[Equation]
and linearly extend this to C n ( K ) , i.e., ∂ n ( (cid:80) σ i )
= (cid:80) ∂ ( σ i ) .
A. R IEMMANIAN M ANIFOLDS
An m -dimensional manifold can be seen as a generalization of a 2D surface and is a space that can be locally approximated by R m .
The space in which the manifold is embedded is called ambient space .
We assume that the ambient space is the standard Euclidean space of dimension equal to the dimension of the manifold, i.e, R m .
For x ∈ M , the tangent space T x M of M at x is the linear vector space spanned by the m linear independent vectors tangent to the curves passing through x .
A
Riemannian metric g =
( g x ) x ∈M on M is a collection of inner products g x : T x M × T x M → R varying smoothly across M and allows us to deﬁne geometric notions such as angles and the length of a curve.
A Riemannian manifold is a smooth manifold equipped with a Riemannian metric g .
The
Riemannian metric g gives rise to the metric norm (cid:107)·(cid:107) g and induces global distances by integrating the length of a shortest path between two points x, y ∈ M
[Equation]
(18) where γ ∈ M ∞ ([0 , 1] , H ) is such
that γ (0) =
x and γ (1) =
y .
The unique smooth path γ of minimal length between two points x and y is called a geodesic and the underlying shortest path length is called geodesic distance .
Geodesics and can be seen as the generalization of straight-lines.
For a point x ∈ M and a vector v ∈ T x M , let γ v be the geodesic starting from x with an initial tangent vector equal to v
, i.e., γ v (0) =
x and ˙ γ v (0) = v .
The existence, uniqueness and differentiability of the exponential map are guaranteed by the classical existence and uniqueness theorem of ordinary differential equations and Gauss’s Lemma of Riemannian geometry (32).
A. P OINCARE B ALL
We deﬁne the Möbius addition for any x, y on the Poincare ball:
[Equation]
The Möbius addition is a manifold preserving operator that allows us to add point belonging in the
Poincare ball without leaving the space.
Finally, we provide the coordinate chart ψ : M → R m , i.e., the inverse of the parameterization in Eq. 8, which is deﬁned as follows:
[Equation]
i ) , for i = 1 , . . . , m − 1 and x m = x
[Equation]
m − 1 (cid:89)
[Equation]
[Equation]
j ) sin ( y m )
[Equation]
[Equation]
Note that we have omitted the learnable parameters.
The coordinate chart is not explicitly needed for constructing the representation.
However, it could be useful as an intermediate layer between our representation and standart deep neural network architecture.
Its purpose would be to transfer the representation from the manifold to a Euclidean space which can be fed into a standart DNN without having to deﬁne its operations on the manifold.
B T HEORETICAL R ESULTS
Proof of Theorem 1.
Let D , E be two persistence diagrams and η : D → E bijection that achieves the inﬁnum in Eq.5.
Consider the subset D
= D \ R ∆ , which is essentially the original diagram without the points in the diagonal.
Then, we have the following sequence of inequalities
[Equation]
[Equation] [Equation]
where w g 1 is the Wasserstein distance with the q -norm replaced with the norm induced by the metric tensor of the Poincare ball.
Eq.
23 follows from the smoothness of the exponential map (assuming that it has Lipschitz constant equal to K e ).
Eq.
24 follows by substituting the bijection η and because, as per requirements of Theorem 1, the auxiliary transformation ρ is zero on the diagonal R ∆ and x 0 = 0 .
Note that the requirements of Theorem 1, combined with Eq. 8, 10 and 19, imply that the summation over x ∈ R ∆ collapses to zero.
Eq.
25 follows from the triangle inequality.
Eq.follows from the smoothness of the logarithmic map and the parameterization (assuming K l and K φ Lipschitz constants, respectively).
Finally, Eq. 27 follows from the assumed, as per Theorem 1, smoothness of ρ and by using the deﬁnition of Wasserstein distance.
This concludes the proof.
C I MPLEMENTATION D ETAILS
In this section, we present the neural network architecture that we used for the experiments as well as the details of the training.
We used a consistent network architecture across all of our simulations and our proposed representation acts as an input.
In more detail, for each individual ﬁltration that we extract from the data (images or graphs), we create a different input layer.
The input to this layer is the persistence diagrams of all homology dimension given by the corresponding ﬁltration.
Given the nature of the data (graphs and images), the homology dimensions with non-trivial persistence diagrams are only the ﬁrst two (i.e., H 0 and H 1 ).
Our input layer processes the persistence diagrams of each homology class independently and outputs the representations (i.e., Eq. 11) for each one of them.
Following that, we concatenate and ﬂatten the previous outputs and fed the resulting vector to a batch normalization layer.
Then, we use two dense layers of 256 and 128 neurons, respectively, with a Rectiﬁed Linear (ReLu) activation function.
This is followed by a dropout layer to prevent overﬁtting and a ﬁnal dense layer as an output.
We consider two different implementations of the the above architecture.
The ﬁrst one is based on the hyperbolic neural networks introduced by Ganea et al.
in (23).
It re-deﬁnes key neural network operations in the Poincare ball and, therefore, allows us to directly utilize the output of our representation in the neural network.
The second implementation uses the coordinate chart given by Eq.
20 to project the representation back to a Euclidean space and feed it to a standardized implementation of the aforementioned layers.
This method is more convenient in practice as it maintains compatibility with existing backbone networks trained in Euclidean geometry.
Another important practical consideration is the information compression introduced by our input layer.
Since our method essentially represents each persistence diagram on a m − dimensional
Poincare ball, it may introduce substantial information compression when the points in the diagrams are not of the same order as m .
A trivial approach to counteract this issue is to use a high m .
However, we experimentally observed that this did not give the optimal results.
Another solution, which we preferred, is to keep m at moderate values (in the range m = 3 to m = 12 ), replicate the representation K times and concatenate the outputs.
Each replica is called a projection base and their number is determined via the validation set.
Hyperparameters and Training : To train the neural network, we use the Adam optimizer ( β 1 = 0 .
9 , β 2 = 0 .
with an initial learning rate of 0.001 and batch size equal to 64.
We use a random uniform initializer in the interval [ − 0 .
for all learnable variables.
For all graph datasets we trained the network for 100 epochs and halved the learning rate every 25 epochs.
For the MNIST and Fashion-MNIST datasets we used 10 and 20 epochs, respectively, and no learning rate scheduler.
We tune the dropout rate manually by starting from really low values and monitoring the validation set accuracy.
In general, we noticed that the network did not tend to overﬁt, therefore, we kept the rate at low values (0-0.2) for all experiments.
Finally, regarding the choice of K , we explored values dependant on the number of points in the persistence diagram.
Diagrams given by ﬁltrations of images tend to have substantially fewer points that diagrams obtained from graphs.
Therefore, for images we explored moderate values for K , i.e., 5 − 10 , whereas for graphs we increased K to the range 200 − 500 .