Manifold Topology Divergence: a Framework for Comparing Data Manifolds. 
Authors: Serguei Barannikov
Ilya Trofimov
Grigorii Sotnikov
Ekaterina Trimbach
Alexander Korotin
Alexander Filippov
Evgeny Burnaev
Venue: NeurIPS 2021 Poster (2021)
Keywords: data manifolds
point clouds
persistent homology
topology
generative models
generative adversarial networks
mode-dropping
3D-shapes
time-series

Abstract: We propose a framework for comparing data manifolds, aimed, in particular, towards the evaluation of deep generative models. We describe a novel tool, Cross-Barcode(P,Q), that, given a pair of distributions in a high-dimensional space, tracks multiscale topology spacial discrepancies between manifolds on which the distributions are concentrated. Based on the Cross-Barcode, we introduce the Manifold Topology Divergence score (MTop-Divergence) and apply it to assess the performance of deep generative models in various domains: images, 3D-shapes, time-series, and on different datasets: MNIST, Fashion MNIST, SVHN, CIFAR10, FFHQ, market stock data, ShapeNet. We demonstrate that the MTop-Divergence accurately detects various degrees of mode-dropping, intra-mode collapse, mode invention, and image disturbance. Our algorithm scales well (essentially linearly) with the increase of the dimension of the ambient high-dimensional space. It is one of the first TDA-based methodologies that can be applied universally to datasets of different sizes and dimensions, including the ones on which the most recent GANs in the visual domain are trained. The proposed method is domain agnostic and does not rely on pre-trained networks.

Manifold Topology Divergence: a Framework for Comparing Data Manifolds
Serguei Barannikov
Ily Trofimov
Skolkovo Institute of Science and Technology
Skolkovo Institute of Science and Technology Moscow, Russia
Moscow, Russia
CNRS, IMJ, Paris University, France
Grigorii Sotnikov
Ekaterina Trimbach
Skolkovo Institute of Science and Technology Moscow, Russia
Moscow Institute of Physics and Technology
Moscow, Russia
Alexander Korotin
Alexander Filippovv
Skolkovo Institute of Science and Technology, Artificial Intelligence Research Institute (AIRI), Moscow, Russia
Huawei Noah's 's Ark Lab
Evgeny Burnaev
Skolkovo Institute of Science and Technology, Artificial Intelligence Research Institute (AIRI), Moscow, Russia
Abstract
We develop framework for comparing data manifolds, aimed, in particular, towards the evaluation of deep generative models. We describe a novel tool, Cross-Barcode(P.) that, given pair of distributions in a high-dimensional space, tracks multiscale topology spacial discrepancies between manifolds on which the distributions concentrated. Based on the Cross-Barcode
introduce the Manifold Topology Divergence score -ivve-eevegg)))) and apply it to assess the performance of deep generative models in various domains: images, 3D-shapes, time-series, and on different datasets: MNIST, Fashion MNIST, SVHN, CIFAR 10, FFHQ, chest X-ray images, market stock data, ShapeNet. We demonstrate that the MToop-Dvvegeenee accurately detects various degrees of mode-dropping, intra- mode collapse, mode invention, and image disturbance. Our algorithm scales well (essentially linearly) with the increase of the dimension of the ambient high- dimensional space. It is one of the first TDA-based practical methodologies that can be applied universally to datasets of different sizes and dimensions, including the ones on which the most recent GANs in the visual domaín are trained. The proposed method is domain agnostic and does not rely on pre-trained networks.
1 Introduction
Geometric perspective in working with data distributions has been pervasive in machine learning [5, 8, 12, 10, 23, 20]. Reconstruction of the data from observing only a subset of its points has made significant step forward since the invention of Generative Adversarial Networks (GANs) [13].
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Despite the exceptional success that deep generative models achieved, there still exists longstanding challenge of good assessment of the generated samples quality and diversity [7].
For images, the Fréchet Inception Distance (FID) [16] is the most popular GAN evaluation measure. However, FID is limited only to 2D images since it relies on pre-trained ImageNet "Inception" network. FID unrealistically approximates point clouds by Gaussians in embedding space; also, FID is biased [6]. Surprisingly, FID can't 't be applied compare adversarial and non-adversarial generative models since it is overly pessimistic to the latter ones [28].
The evaluation of generative models is about comparing two point clouds: the true data cloud Paata and the model (generated) cloud Qmodel. In view of the commonly assumed Manifold Hypothesis [5, 12], develop a topology- Maata and the model manifold Mmodel, by analysing samples Paata C Maata and Q model C Mmddl--
measure for comparing two manifolds: the true data manifold
Contribution. In this work, we make the following contributions:
1. We introduce new tool: Cross-Barcode(P,, Q). For a pair of point clouds P and Q. the Cross-Barcode P, Q) records the differences in multiscale topology between two manifolds approximated by the point clouds;
2. We propose a new measure for comparing two data manifolds approximated by point clouds: Manifold Topology Divergence (MTop-Div)) Div);
3. We apply the MTop-Div to evaluate performance of GANs in various domains: 2D images, 3D shapes, time-series. We show that the MTop-Div correlates well with domain- -emaiie-peccfic measures and can be used for model selection. Also it provides insights about evolution of generated data manifold during training:
4. We have compared the Mioo--Dvv against 7 established evaluation methods: FID, discriminative score, MMD, JSD, 1-coverage, IMD and Geometry score and found that MTop-Div is able to capture subtle differences in data geometry;
5. We have essentially overcame the known TDA scalability issues and in particular have carried out the MTop-Div calculations most recent datasets such FFHQ. with dimensions D up to 107.
The source code is available at https: : /t///ggtttttttt cww htw.lyyyarrooooooooooooooooooo
Related work. GANs try to recover the true data distribution via a min-max game where two players, typically represented by deep neural networks, called Discriminator and Generator, com- pete by optimizing the common objective. Training curves are not informative since generator and discriminator counter each other, thus, the loss values are often meaningless. Several other measures were introduced to estimate the quality of GANS. However, there is no consensus which of them best captures strengths and limitations of the models and can be used for the fair model selection. The likelihood in Parzen window works poorly in high-dimensional spaces and does not correlate with the visual quality of generated samples [30]. The Inception Score (IS) [29] measures both the discriminability and diversity of generated samples. It relies on the pre-trained Inception network and is limited to 2D images domain. While IS correlates with visual image quality. ignores the true data distribution, doesn't detect mode dropping and is sensitive to image resolution. The Fréchet Inception Distance (FID) [16] distance between two multivariate Gaussians. These Gaussians approximate the features of generated and true data extracted from the last hidden layer of the pretrained Inception network. Similar variant is KID [6] which computes MMD distance between two distributions of features. The work [14] proposed to use the Duality Gap, notion from the game theory, as a domain agnostic measure for GAN evaluation. Another variant of a measure inspired by game theory was proposed in [27]. Conventional precision and recall mea- sures can be also used [22, 21]. Accurate calculation of precision and recall is limited to simple datasets from low-dimensional manifolds. The Geometry Score (GScore) [19] is the L2-distance between mean Relative Living Times (RLT) of topological features calculated for the model dis- tribution and the true data distribution. The GScore is domain agnostic, does not involve auxiliary pretrained networks and is not limited to 2D images. However, GScore is not sensitive even to some simple transformations like constant shift, dilation, reflection (see our Fig. 2, 4). The barcodes in GScore are calculated approximately, based only on the approximate witness complexes on 64 landmark points sampled from each distribution. That's why the procedure is stochastic and should be repeated several thousand times for averaging. Thus, the calculation of GScore can be prohibitively long for large datasets. We also refer reader to the comprehensive survey [7].
2

---

2 Cross-Barcode and Manifold Topology Divergence
2.1 Multiscale simplicial approximation of manifolds
According to the well-known Manifold Hypothesis [12] the support of the data distribution Pdata is often concentrated on low-dimensional manifold Maata. We construct a framework for comparing numerically such distribution Paata with a similar distribution @model concentrated on a manifold Mmodel Such distribution Qmodel is produced, for example, by generative deep neural network in one of applications' scenarios. The immediate difficulty here is that the manifold Maata is unknown and is described only through discrete sets of samples from the distribution Pdata- One standard approach to resolve this difficulty is to approximate the manifold Maata by simplices with vertices given by the sampled points. The simplices approximating the manifold are picked based on proximity information given by the pairwise distances between sampled points [5. 25J. The standard approach is to fix threshold choice of threshold is essential here since if it is too small, then only the initial points, i.e., separated from each other O-dimensional simplices, are allowed. And if the threshold is too large, then all possible simplices with sampled points as vertices are included and their union is simply the big blob representing the convex hull of the sampled points. Instead of trying to guess the right value of the threshold, the standard recent approach is to study all thresholds at once. This can be achieved thanks to the mathematical tool, called barcode [2, 10], that quantifies the evolution of topological features over multiple scales. For each value of the barcode describes the topology, namely the numbers of holes or voids of different dimensions, of the union of all simplices included up to the threshold
0 and to take the simplices whose edges do not exceed the threshold r. The
2.2 Measuring the differences in simplicial approximation of two manifolds
However, to estimate numerically the degree of similarity between the manifolds Mmodel, Maata C RD, it is important not just know the numbers of topological features across different scales for simplicial complexes approximating Mmodel, Maata but to be able to verify that the similar topological features are located at similar places and appear similar scales.
Our method measures the differences in the simpli- cial approximation of the two manifolds, represented by samples P and Q. by constructing sets of sim- plices, describing discrepancies between the two man- ifolds. To construct these sets of simplices we take the edges connecting P pointss with Q Q-points, and also P-points between them, ordered by their length, and start adding these edges one by one, beginning from the smallest edge and gradually increasing the threshold, see Figure 1. We add also the triangles and k simplicess at the threshold when all their edges have been added. It is assumed that all edges between Q-points were already in the initial set. We track in this process the births and the deaths of topological features, where the topological features are allowed here have boundaries on any simplices formed by Q-points. The longer the lifespan of the topological feature across the change of threshold the bigger the described by this feature discrepancy between the two manifolds.
0.2
a 0.6
a D.8
Figure 1:
Edges(red) connecting P- points(redd with Q- points(blue) and also P poipisns between them, are added for three thresholds: a = 0.2, 0.4, 0.6
Homology is a tool that permits to single out topological features that are similar, and to decompose any topological feature into sum of basic topological features. More specifically, in our case, a k-cycle is a collection of k s-simplices formed by P- and Q points, such that their boundaries cancel each other, perhaps except for the boundary simplices formed only by Q-points. For example, cycle of dimension k- = 1 corresponds to path connecting pair of Q-points and consisting of edges passing through a set of P-points. A cycle which is boundary of a set of (k: + 1) simplicess is considered trivial. Two cycles are topologically equivalent if they differ by a boundary, and by collection of simplices formed only by Q-points. can be represented by vector in the vector space where each simplex corresponds to generator. In practice, the vector space over (0, 1} used most often. The union of cycles corresponds to the sum of vectors. The homology vector space H3 is defined as the factor of the vector space of all k k-cycles
union of cycles is again a cycle. Each cycle
3

---

Let P = [p:), Q = {4p). Ps. E RD are two point clouds sampled from two distributions P, Q. To define Cross- (aoss-Barcod((P.() we construct first the following filtered simplicial complex. Let (T'uoq, m(ruQ)/Q) be the weighted graph with the distance-like weights on edges defined as the complete graph on the union of point clouds P Q with the distance matrix given by the pairwise modulo the vector space of boundaries and cycles consisting of simplices formed only by Q-points. set of basic topologiaa features,
set of vectors forming a basis in this factor-space corresponds so that any other topological feature is equivalent to sum of some basic features.
The homology are also defined for manifolds and for arbitrary topological spaces. This definition is technical and we have to omit it due to limited space, and to refer to c.g. [15, 24] for details. The relevant properties for us are the following. For each topological space X the vector spaces (X),),, k = 0, 1.... are defined. The dimension of the vector space Ha equals to the number of independent k di-menssional topological features (holes, etc). An inclusion Y C X induces a natural map HY) + (X)
In terms of homology, we would like to verify that not just the dimensions of homology groups H. Mmodel) and H. (. (Masaa) are the same but that more importantly the natural maps:
Φr: : H. (Mmodel n Maltaa + H. (M
(1)
Pp : (- n Maata) + (Maata)
(2)
induced by the embeddings are as close as possible to isomorphisms. The homology of a pair the tool that measures how far such maps are from isomorphisms. Given a pair of topological spaces Y C X, the homology of a pair H. (X, Y) counts the number of independent topological features in X that cannot be deformed to a topological feauture in Y plus independent topological features in Y that, after the embedding to X, become deformable to a point. An equivalent description, the homology of pair H. Y) counts the number of independent topological features in the factor-space X/Y. where all points of Y are contracted to a single point. The important fact for us is that the map, induced by the embedding, HY) - (X) is isomorphism if and only if the homology of the pair Ha (X, Y) are trivial. Moreover the embedding of simple simplicial complexes Y c X is equivalence in homotopy category, if and only if the homology of the pair H, (X, Y) are trivial [33].
To define the counterpart of this construction for manifolds represented by point clouds, we employ the following strategy. Firstly, we replace the pair
C M by the equivalent pair Mmodel C (Masta U MModee) with the same factor-space. Then, we represent (Matta U Mmodel) by the union of point clouds PUQ, where the point clouds P, Q are sampled from the distributions Paata. @model Our principal claim here is that taking topologically the quotient of (Marza U Mmodel) by Mmodel is equivalent in the framework of multiscale analysis of topological features to the following operation on the matrix muso of pairwise distances of the cloud P U Q: set to zero all pairwise distances within the subcloud Q C (P (PUQ).
2.3 Cross-Barcode (P.Q)
distance in IR' for the pairs of points (pr. (5) or (p:.9) (j) and with all pairwise distances within the cloud Q that we set to zero Our filtered simplicial complex is the Vietoris-Rips complex of (TPuQ, (PPQ)/OO)
Recall that given such a graph I with matrix m of pairwise distances between vertices and parameter a > 0, the Vietoris-Rips complex R. (T, m)
(m] is the abstract simplicial complex with simplices that correspond to the non-empty subsets of vertices of I whose pairwise distances are less than a as measured by m. Increasing parameter a adds more simplices and this gives a nested family of collections of simplices know as filtered simplicial complex. Recall that simplicial complex is described by a set of vertices V = {Ui...., UNN) J. and a collection of k -implicces S, i.c. (k eleemeetts subsets of the set of vertices V. k > 0. The set of simplices S should satisfy the condition that for each simplex & E S all the (A: - 1) -iimplices obtained by the deletion of a vertex from the subset of vertices of belong also to S. The filtered simplicial complexes is the family of simplicial complexes S, with nested collections of simplices: for aj < a2 all simplices of S,, are also in S.g.
At the initial moment, a = O, the simplicial complex Ra (T PUQ; ((UQ)/2) /Q) has trivial homology Hy for all k > 0 since it contains all simplices formed by Q-points. The dimension of the 0-th

---

homology equals at a 0 to the number of P P-points, since no edge between them or between P-point and Q Qo-point added at the beginning. As we increase a, some cycles, holes or voids appear in our complex R.. Then, some combinations of these cycles disappear. The persistent homology principal theorem [2, 37] implies that it possible to choose the set of generators in the homology of filtered complexes H& (Ra) across all the scales such that each generator appears at its specific "birth" time and disappears at its specific "death" time. These sets of "birth" and "deaths" of topological features in Ra are registered in Barcode of the filtered complex.
Definition. The Cross-Barcodee Barcode, (P Q) is the set of intervals recording the "births" and "deaths" times of i- dideennsional topological features in the filtered simplicial complex R. (TPQ. PuQ: M(PU))/Q)
Examples of Cross-Barcode, (P; Q) are shown on Fig. 2, 11, 14, 17, 18. Topological features with longer "lifespan" are considered essential. The topologiaa features with "birth"="dd "deeth" are trivial by definition and do not appear in Cross-Barcode..( (P, Q).
2.4 Basic properties of Cross-Barcode.dee (P,QQ
Proposition 1. Here is list of basic properties of Cross-Barcodes (P, Q):
if the two clouds coincide then Baoss--Brrode,d (P, P) = Ø; for Q = Ø, the norm of
(P, 2) = Barcode, (P), the barcode of the single point cloud P itself; Q). i 2 0, is bounded from above by the Hausdorff distance
Pooos-Barccode (P,Q)IlB - du (P,Q).
(3)
The proof is given in appendix.
2.5 The Manifold Topology Divergence (MTop-Div)
The bound from e3.(3) and the equality
= 2 imply that the closeness of Cross-Barcode, (P, Q) to the empty set a natural measure of discrepancy between P and Q Each Cross-Barcode; (P, Q) is a list of intervals describing the persistent homology H, To measure the closeness to the empty set, one can use segments' statisties: sum of lengths, sum of squared lengths, number of segments, the maximal length (H, max) or specific quantile. We assume that various characteristics of different H, could be useful in various cases, but the cross-barcodes for Ho and Hi can be calculated relatively fast.
Our Diver-Divereence(P... Q) is based on the sum lengths of segments in Cross-Barcode (P. Q). see section 2.6 for details. The sum of lengths of segments in Cross-Barcode; (P., Q) has an interesting interpretation via the Earth Mover's Distance. Namely, it easy to prove (see Appendix B.2) that EM- Distance between the Relative Living Time histogram for Cross-Barcode; (P, Q) and the histogram of the empty barcode, multiplied by the parameter Omar from the definition of RLT, see e.g. [19], coincides with the sum of lengths of segments in H1. This ensures the standard good stability properties this quantity.
Our metrics can be applied in two settings: to a pair of distributions Pdaats @model. in which case we denote our score MTTo--Di(D..M) and pair of distributions Qmodel. Pasta in which case our score is denoted Divergence are related to the concepts of precision and recall. These two variants can be analyzed separately or combined together, e.g. averaged.
(D,,-D) These two variants of the Cross-Barcodds Barcode, and of the MTop-
2.6 Algorithm
To calculate the score that evaluates the similitude between two distributions, we employ the following algorithm. First, we compute Cross-Bare Barcode, (P, Q) on point clouds P, Q of sizes by, by sampled from the two distributions P. Q. For this we calculate the matrices mp, mp: of pairwise distances within the cloud P and between clouds P and Q. Then the algorithm constructs the Vietoris- filtered simplicial complex from the matrix ii (UQ))Q which is the matrix of pairwise distances in P U Q with the pairs of points from cloud Q block replaced by zeroes and with other blocks given by mp, m...Q Next step is to calculate the barcode of the constructed filtered simplicial complex. This step and the previous step constructing the filtered complex from the matrix M(PQ)/QQ can
5

---

Algorithm 1 Cross-Barcode; (P,
Algorithm 2 MTop-Divergence(P, 2) see sec- tion 2.6 for details, default suggested values: bp = 1000, bo = 10000, = 100
Input: m mIP; P), m[P,, Q] matrices of pairwise distances within point cloud P. and between point clouds P and Q
Input: Xp.Xo. Np x D. No x D arrays rep-
Require: (R(M): function computing filtered complex from pairwise distances matrix M Require: B i): function computing persis-
resenting datasets for j = 1 to do
Pj f- random choice Xp.bp)
tence intervals of filtered complex C' in dimen- sion i
random choice(Xo.All
B <- list of intervals Cross-
bo < number of columns in matrix m(P.Q)
Barcode: (Pj, Q2) calculated by Algorithml mld, + sum of lengths of all intervals in B,
m Q) < zeroes(bq (bq; bq)
m P. PI m Q]
M +-
m m
m(Q, Q
end for MTop-Divergeneee Q) mean(mtd) Return: number Mioo--ivergenceeff @) rep- resenting discrepancy between the distributions P.Q
Cross-Barcode; + ((VR(M),
Return:
list of intervals Cross-Barcode; Q)
representing "births" and "deaths" of topologi- cal discrepancies
be done using one of the fast scripts', some of them are optimized for GPU acceleration, see e.g. [35, 4J. The calculation of barcode from the filtered complex is based on the persistence algorithm bringing the filtered complex to its "canonical form" ([2]). Next, sum of lengths or one of other numerical characteristes of Cross-Barcode; (P, Q) is computed. Then this computation is run a sufficient number of times obtain the mean value of the picked characteristic. In our experiments we have found that for common datasets the number of times from 10 to 100 is generally sufficient. Our method is summarized in the Algorithms and 2.
Complexity. The Algorithm 1 requires computation of the two matrices of pairwise distances m[P, P). m[P. for a pair of samples P E RR Q E ReexD involving ( D) and O(bpbo D) operations. After that, the complexity of the computation of barcode does not de- pend on the dimension D of the data. Generally the persistence algorithm is at worst cubic in the number of simplices involved. In practice, the boundary matrix is sparse in our case and thanks also to the GPU optimization, the computation of cross-barcode takes similar time in the previous step on datasets of big dimensionality. Since only the discrepancies in manifold topology are calculated, the results are quite robust and a relatively low number of iterations is needed to obtain accurate results. Since the algorithm scales linearly with D it can be applied to the most recent datasets with D up 10". For example, for D- 3. 15 10% and batch sizes bp 10 bo 10 NVIDIA TITAN RTX the time for GPU accelerated calculation of pairwise distances was 15 seconds, and GPU-accelerated calculation of Cross-Barcodd: (P, Q) took 30 seconds.
3 Experiments
We examine the ability of MToo-Div to measure quality of generative models trained on various datasets. Firstly. we illustrate the behaviour of MTop-Div on simple synthetic datasets (rings (Fig.2), disks (Fig. 13)). Secondly, we show that MTop-Div is superior to the GScore, another topology- GAN quality measure. We carry out experiments with a mixture of Gaussians, MNIST, CIFAR10 10, X-ray images, FFHQ The performance of M'Top-Div is on par with FID. For images, M'Top-Div is always calculated in pixel space without utilizing pre-trained networks. Thirdly, we apply MTop-Div to GANs from non-image domains: 3D shapes and time-series, where FID is not applicable. We show that MTop-Dii Div agrees with domain-specific measures, such as JSD, MMD, Coverage, discriminative score, but -Tov-Div better captures evolution of generated data manifold during training
3.1 Simple synthetic datasets in 2D
As illustrated on Fig. 2 the GScore does not respond to shifts of the distributions' relative position.
I Persistent Homology Computation (wiki)
2 we additionally calculated IMD [31] for the pairs of point clouds from our experiments, see Appendix E.

---

GScore HO max
H1
Mlop Div
200
100
0
0.1
0.2
distance
dislance
Figure 2:
MTop-Div and HO max compared with GScore, for two ring clouds of 1000 points, as function of distance between ring centers, the Cross- pooss-Barcooe (P, Q) is shown at dl = 0.5
3.2 Mode-Droping on Gaussians
Geomenny Score
Dersity Function
One of common tasks to assess GAN's performance measuring it's ability to uncover the variety of
modes in the data distribution. A common bench- mark for this problem a mixture of Gaussians, see Fig. 3. We trained two generators with very different performance: original GAN, which managed cap- ture all 5 modes of the distribution and WGAN-GP, which have only covered poorly two. However, the Geometry score is not sensitive to such a difference since two point clouds have the same RLT histogram. While the -io is sensitive to such a difference.
GAN
G. ScOkE MTor
WGAN-UP
0.562
GAN
Figure 3: Difficulty of the Geometry Score to detect the mode dropping.
3.3 Digit flipping on MNIST
S_ ?_
2 2
5
Geometry Score 0.0 MTTo-Div = 6154.0
Figure shows an experiment with MNIST dataset. We compare two point clouds: "5"s vs. vertically flipped "5"s (resembling rather "2"s). These two clouds are indistinguishable for Geome- try Score, while the MTop-Div is sensi- tive to such flip since it depends on the relative position of the two clouds.
J 5
VS
R. R. 2
2
5 S SS5
2 2 2
2
Figure 4: Two point clouds:"5's from MNIST vs. vertically flipped "5"s from MNIST (resembling rather "2"s). The two clouds are indistinguishable for Geometry Score, while the MToo--Dv is sensitive to such flip as depends on the positions of clouds with respect to each other.
3.4 Synthetic modifications of CIFAR10
Geemeny Sooy
Deamatry Scare 15
Geomeryaare
Scand
Ceereeyy foord
orgint
inghe)
Mrg
4.0
0400
X0CG 8100
040
Figure 5: Experiment with modifications of CIFAR10. The disturbance level rises from zero to a maximum. Ideally, the quality score should monotonically increase with the disturbance level.
We evaluate the proposed We take CIFARIO and apply the following modifications. Firstly, we emulate common issues in GAN: mode drop, mode invention and mode collapse by doing class drop, class addition and intra-class collapse (removal of objects within a class). Secondly, we apply two disturbance types: erasure of a random rectangle and Gaussian noise. As 'real' images we use the test set from CIFAR10, as 'generated' images subsample of the train set with applied modifications. The size of the later always equals the test set size. Figure 5 shows the results. Ideally, the quality measure should monotonically increase with the disturbance level. We conclude that Geometry Score monotone only for 'mode invention" and 'intra-mode collapse' while (TTopDDvvD,,M) is almost monotone for all the cases. The average Kendall-tau rank correlation between MTop-Div(D,M and disturbance level is 0.89, while for Geometry Score the rank correlation is only 0.36. FID performs well on this
using a benchmark with the controllable disturbance level.
7

---

10
Kilip-D v/DAM Discrimnnttvv scone
SyleGer 2, 0.7
24
800
MTop- Div
StyleGent,
600
#
spach 690
10
30
200
Sty 3.0
StyleGanl, 1.0
10
15
20
FID
Figure 7: Comparison the qual- ity measures, FID vs MToo-Div, on StyleGAN, StyleGAN2 trained FFHQ with different truncation levels. MToo-DDi(MM.. is mono- tonically increasing in good corre- lation with FID.
Figure 6: Training process of GAN applied to chest X-ray data. Top: normalized quality measures FID, MTop-Div, Disc, score vs. epoch. Each measure is divided by it's maximum value: max. FID 304, max MTop-Div 21, max. Disc.score 0.5. Lower is better. Dashed horizontal lines depict comparison of real COVID COVVID-positive and COVID-neg, chest X-rays. Bottom: PCA projections of real objects (red) and generated objects (green).
benchmark, not shown on Figure 5 for ease perception. Additionally, we calculated MTop-Div for higher order
see Appendix D.
3.5 GAN model selection
We trained WGAN and WGAN-GP on various datasets: CIFARIO, SVHN, MNIST, Fashion- MNIST and evaluated their quality, see Table 1. Experimental data show that the ranking between WGAN and WGAN-GP consistent for FID and MToo-Div.
Table MTop- Div is consistent with FID for model selection GAN's trained on various datasets.
Dataset
FID
MTop-Div(M.D)
WGAN
WGAN-GP
WGAN
WGAN-GP
CIFAR10
154.6
399.2
353.1
1637.4
SVHN
101.6
154.7
332.0
963.2
MNIST
31.8
22.0
2042.8
1526.1
PaSHionMNIST
52.9
35.1
919.6
660.4
3.6 Experiments with StyleGAN
We evaluated the performance of StyleGAN [17] and StyleGAN2 [18] generators trained on the FFHQ dataset?. We generated 20 x 10 samples with two truncation levels: s 0.7, 1.0 and compared them with 20 x 103 samples from FFHQ. The truncation trick is known to improve average image quality but decrease image variance. Figure 7 show the results (see also Table in Appendix for more data). Thus, the ranking via MTop-Div(M, D) is consistent with FID. We also tried to calculate Geometry Score but found that it takes a prohibitively long time.
3.7 Chest X-rays generation for COVID-19 detection
Waheen et al. [32] described how to apply GANs improve COVID-19 detection from chest X-ray images. Following [32], we trained an ACGAN [26] on dataset consisting chest X-rays of COVID-19 positive and healthy patients 4, Next, we studied the training process of ACGAN. Every 10'th epoch we evaluated the performance of ACGAN by comparing real and generated COVID-19 positive chest X-ray images. That is, calculated FID, MTop-Div(D.N.) and baseline measure-
'https://githun wwwmW/aaas//ffggdaaaaaaa
(CC-BY 2.0 License)
*We used the ACGAN implementation https://githi: wwwc htmm///vraa/AAAAA htt.:/rrra/AACAN-Pyoorrooo (MIT License) and chest X-ray data was from https://wwe kaggle e. covid19-radiographe database (Kaggle Data License)

---

MND
Tierimiattlve cram
1-roueeae
15
16
KThp--
12
2002
opch epech
1ooch
Figure 8: Training process of GAN applied 3D
Training dynamics of TimeGAN ap- plied to market stock data. Top: discriminative score
Figure 9:
shapes. Top, middle: quality measures MMD,
JSD, 1-Coverage, MTop-Div vs. epoch. Each quality measure normalized, that is, divided by its value the first epoch. Lower better. Bot- tom: PCA projection of real objects (red) and gen-
epoch, MTop-Div epoch. Lower is better. Bottom: PCA projection of real time- series (red) and generated time-series (green). Vertical red line (epoch 2000) depicts the mo- ment when manifolds of real and generated ob-
erated objects (green). Vertical red line (epoch
50) depicts the moment, when the manifold of generated objects "explodes" and becomes much more diverse.
jects become close.
discriminative score of a CNN trained to distinguish real vs. generated data. The M'Top-Div agrees with FID and the discriminative score. PCA projections show that generated data approximates real data well. Figure 20 in Appendix presents real and generated images.
Additionally, we compared real COVID-positive and COVID-negative chest X-ray images, see horizontal dashed lines at Fig. 6. Counterintuitively, for FID real COVID-positive images are closer real COVID-negative
than generated COVID-positive images; probably because FID is overly sensitive to textures. At the same time, evaluation by MTop-Dir "ivo--Dvv is consistent.
3.8 3D shapes generation
use the proposed M'Top-Div score to analyze the training process of the GAN applied to 3D shapes, represented by 3D point clouds [1]. For training, used 6778 objects of the "chair" class from ShapeNet [9]. We trained GAN for 1000 epochs and tracked the following standard quality measures: Minimum Matching Distance (MMD), Coverage, and Sensen-Shannon Divergence (JSD) between marginal distributions. To understand the training process in more details, we computed PCA decomposition of real and generated objects (Fig. 8, bottom). For computing PCA, each object (3I) point cloud) was represented by a vector of point frequencies attached to the 3-dimensional 28 grid. Figure 8, top, shows that conventional metrics (MMD, JSD, Coverage) doesn't represent the training process adequately. While these measures steadily improve, the set of generated objects dramatically changes. At epoch 50, the of generated objects (green) "explodes" and becomes much more diverse, covering a much larger space than real objects (red). Conventional quality measures (MMD, JSD, Coverage) ignore this shift while MTop-Div has a peak at this point. Next, we evaluated the final quality of GAN by training classifier to distinguish real and generated object. A simple MLP with hidden layers showed accuracy 98%, indicating that the GAN poorly approximates the manifold of real objects. This result is consistent with MToo-Div: at epoch 1000 it is even larger than at epoch 1.
3.9 Time series generation
Next, we analyze training dynamics of TimeGAN [34] tailored to multivariate time-series generation. We followed the experimental protocol from [34] and used the daily historical market stocks data
Discriminative score equals accuracy minus 0.5. MTop-Di Div better correlates with the discriminative score than FID: 0.75 0.66.

---

from 2004 to 2019, including as features the volume, high, low, opening, closing, and adjusted closing prices. The baseline evaluation measure is calculated via a classifer (RNN) trained to distinguish real and generated time-series. Particularly, the discriminative score equals to the accuracy of such a classifer minus 0.5. Fig. 9, top, shows the results. We conclude that the behaviour of -iv is consistent with the discriminative score: both of them decrease during training. To illustrate training in more details we did PCA projections of real and generated time-series by flattening the time dimension (Fig. 9, bottom). At 2000-th epoch, the point clouds of real (red) and generated (green) time-series became close, which captured by a drop of MTop-Div score, At the same time, discriminative score is not sensitive enough this phenomena.
Conclusions
We have proposed a tool, Cross- Booss-Bareode. (P, Q). which records multiscale topology discrepancies between two data manifolds approximated by point clouds. Based on the have introduced the Manifold Topology Divergence and have applied it to evaluate the performance of GANs in various domains: 2D images, 3D shapes, time-series. We have shown that the M MTop-Div correlates well with domain-specinf measures and can be used for model selection. Also, it provides insights about the evolution of generated data manifold during training and can be used for early stopping. The MTop-Div score is domain agnostic and does not rely on pre-trained networks. We have compared the MTop-Div against 7 established evaluation methods: FID, discriminative score, MMD, JSD, I -coverage, IMD, and Geometry score and found that MTop-Div outperforms many of them and captures well subtle differences in data manifolds. Our methodology permits to overcome the known TDA scalability issues and to carry out the MTop-Div calculations on the most recent datasets such FFHQ, with the size of up to 10s and the dimension of up to 10".
(P. Q), we
Acknowledgements
Evgeny Burnaev acknowledges the support of the Ministry of Science and Higher Education grant No. 075-10-2021-068. Authors are thankful to Alexei Artemov for help with the 3D shapes experiment.
References
[1] Panos Achlioptas. Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning repre- sentations and generative models for 3d point clouds. In International conference on machine learning, pages 40-49. PMLR, 2018.
[2] Serguei Barannikov. Framed Morse complexes and its invariants. Adv. Soviet Math., 21:93-115. 1994.
[3] Serguei Barannikov. Canonical Forms = Persistence Diagrams. Tutorial. In European Workshop on Computational Geometry (EuroCG 2021), 2021.
[4] Ulrich Bauer. Ripser: efficient computation of Vietoris-Rips persistence barcodes. Journal of Applied and Computational Topology, pages 1-33, 2021.
[5] Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic, pages 585-591, 2001.
[6] Mikolaj Birikowski, Dougal Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. arXiv preprint arXiv: 180100040 2018.
[7] Ali Borji. Pros and cons of GAN evaluation measures. Computer Vision and Image Under- standing, 179:41-65, 2019.
[8] Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
[9] Angel X Chang. Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information rich 3d model repository. arXiv preprint arXiv:1512.03012. 2015.
10

---

[10] Frédéric Chazal and Bertrand Michel. An introduction to topological data analysis: fundamental and practical aspects for data scientists. arXiv preprint arXiv: 2017.
[11] Alon Efrat, Alon Itai, and Matthew Katz. Geometry helps bottleneck matching and related problems. Algorithmica, 31)) 2001.
[12] Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1.
MIT press Cambridge, 2016.
[13]
Ian J. Goodfellow, Jean Pouget-Abadis Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014.
[14] Paulina Gmarova, Kfir Y Levy, Aurelien Lucchi, Nathanael Perraudin, Ian Goodfellow, Thomas Hofmann, and Andreas Krause, A domain agnostic measure for monitoring and evaluating GANs. In Advances in Neural Information Processing Systems, pages 12092-12102, 2019.
[15]
Allen Hatcher. Algebraic topology. 2005.
[16] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems, pages 6626-6637, 2017.
[17] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4401-4410, 2019.
[18) Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8110-8119, 2020.
[19] Valentin Khrulkov and Ivan Oseledets. Geometry score: A method for comparing generative adversarial networks. In International Conference on Machine Learning, pages 2621-2629.
PMLR, 2018.
[20] Kwangho Kim, Jisu Kim, Manzil Zaheer, Joon Kim, Frédéric Chazal, and Larry Wasserman. PLLay: Efficient topological layer based on persistence landscapes. In 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020.
[21] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
[22] Mario Lucic, Karol Kurach, Marcin Michalski. Sylvain Gelly, and Olivier Bousquet. Are gans created equal? A large-scale study. In Advances in Neural Information Processing Systems, pages 700-709, 2018.
[23] Michael Moor, Max Hom, Bastian Rieck, and Karsten Borgwardt. Topological autoencoders. In International Conference on Machine Learning, pages 7045-7054. PMLR, 2020.
[24] Rodrigo Rojas Moraleda, Nektarios A Valous, Wei Xiong, and Niels Halama. Computational Topology for Biomedical Image and Data Analysis: Theory and Applications. CRC Press, 2019.
(25) Partha Niyogi, Stephen Smale, and Shmuel Weinberger. Finding the homology of submanifolds with high confidence from random samples. Discrete Computational Geometry, 39(1-3):419
441, 2008
[26] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In International conference on machine learning, pages 2642-2651. PMLR, 2017.
[27] Catherine Olsson, Surya Bhupatiraju, Tom Brown, Augustus Odena, and Ian Goodfellow. Skill rating for generative models. arXiv preprint
04888, 2018.
[28] Suman Ravuri and Oriol Vinyals. Classification accuracy score for conditional generative models. arXiv preprini arXiv: 190X5. 10887, 2019.
[29] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. arXiv preprint arXiv: 1606.633988 2016.
[30] Lucas Theis, Aäron van den Oord, and Mathias Bethge. A note on the evaluation of generative models. arXiv preprint ar
2015.

---

[31] Anton Tsitsulin, Marina Munkhoeva, Davide Mottin, Panagiotis Karras, Alex Bronstein, Ivan Oseledets, and Emmanuel Mueller. The shape of data: Intrinsic distance for data distributions. In International Conference on Learning Representations, 2019.
[32] Abdul Waheed, Muskan Goyal, Deepak Gupta, Ashish Khanna, Fadi Al-Turjman, and Plá- cido Rogerio Pinheiro. CovidGAN: data augmentation using auxiliary classifier GAN for improved covid-19 detection. leee Access, 8:91916-91923, 2020.
[33] George W Whitchead. Elements of homotopy theory, volume 61. Springer Science & Business Media, 1968.
[34] Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. Time-series generative adversarial networks. 2019.
[35] Simon Zhang, Mengbai Xiao, and Hao Wang. Gpu-accelerated computation of Vietoris-Rips persistence barcodes. In 36th International Symposium on Computational Geometry (SoCG 2020). Schloss
Seaahl-LLeiniii-eeeeti
für Informatik, 2020.
[36] Afra Zomorodian and Gunnar Carlsson. Computing persistent homology. Discrete & Computa- tional Geometry, 33(2):249-274, 2005.
[37] Afra Zomorodian. Computing and comprehending topology: Persistence and hierarchical
Morse complexes (Ph.D. University of Illinois at Urbana-Chana Chhana-Chaampain, 2001.
12


Reviews:
[{"review_id": "fTqMDrz_mkw", "reviewer": "Reviewer_ABSW", "summary": "This work introduces a novel method based on relative persistent homology (because the term of persistent relative homology has been occupied before, referring to completely different concept) for measuring the similarity between manifolds, especially the sample point clouds of the manifolds. The distance uses the bar-code of persistent homology, and the homology groups are relative groups H_k(P,Q). The relative between the cross-bar codes and the Hausdorff distance between the point clouds   is proved in the appendix.  Based on the cross-barcode, the manifold topology divergence score (MTop-Divergence) is introduced and applied to assess generative models in various domains. The MTop-Div can detect mode collapsing , mode invention more effectively.  This is a first TDA-based practical method, and has many merits, scalability, domain agnostic, independence of pre-trained networks.", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work uses topological data analysis (TDA) method for deep learning, especially for measuring the distance between point clouds, therefore it is suitable for studying the similarities between distributions on manifolds. From theoretical point of view, the work is novel and inspiring, it shows the rigor and elegance of algebraic topology for data analysis. From practical point of view, the work has been tested on various generative models on different domains, the experiments show the effectiveness for discovering mode dropping, collapsing and mode invention. The way of combing powerful applications with solid theories should be promoted in deep learning.\n\nThe manuscript is well written. The introduction is highly motivated, and explains the key ideas clearly. The previous work part focuses on the literature in deep learning, and completely omits the large literature for persistent homology. It will be helpful to give some important references in TDA there. \n\nThe theoretic section 2 represents the key theoretic results and the idea of the methodology. It is challenging to explain the whole picture to general audience without the knowledge of persistent homology. The authors  give many intuitive examples to explain the idea, and postpone the formulation to the later part, and give formal definitions and prove the claims in the appendix.  \n\nThe \"spider web\" metaphor is very intuitive and interesting. The proof for the position 1,  the upper bound of cross-barcode is given by the Hausdorff distance, is convincing. There are some theoretical considerations. \n1. All the distances among the points are using the Euclidean metric of the embedding space. Since the distributions are defined on the data manifolds, it is natural to consider the Riemannian metric of the manifolds. It is possible that the geodesic distance is very big but the Euclidean distance is very small. How to address this in the current framework ?\n\n2. It is well known in geometric field that Hausdorff distance is the most preliminary distance to measure the shape similarities.  There are many cases that a sequence of shapes converge to a target shape in Hausdorff distance, but the curvature, area diverge. A more refined method may need to considered.\n\n3. In order to measure the distance between distributions, the most common metric is the Wasserstein distance under Lp norm. The proposed method is sensitive to the topological changes,  therefore easily capture the multiple modes; but optimal transport map emphasizes more on distances. Some quantitative comparisons will be valuable.\n\nThe experiments are thorough and convincing, it might be helpful to compare with the methods based on optimal transportation. \n\nIn summary, the work introduces rigorous persistent homology method for measuring the distance between distributions defined on manifolds embedded in Euclidean spaces. The work combines mathematical rigor with practical usage. The manuscript is well written, and work is inspiring. I advocate the acceptance. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The  persistent homology is constructed using the Euclidean distance, for manifold data analysis, it is natural to consider geodesic distance.\n\nThe method focuses on the topology of the simplicial complex, most of the times, the regularity of the manifold also plays an important role. The Hausdorff distance is not sufficient to ensure the convergence of manifolds with Riemannian metrics.\n\n\n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "GmEM7ga917v", "reviewer": "Reviewer_8Pta", "summary": "The authors propose a method to compare two data manifolds based on topology. A persistent diagram is constructed using both sets of samples (cross-barcode), and the proposed metric is computed as the average length of segments in the cross-barcode. The efficiency of the method is demonstrated in several experimental settings.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nFirst of all, I am not an expert in topological data analysis. However, it seems that the cross-barcode is new idea, which is shown to be potentially useful. Also, I think that the proposed idea is a sufficient contribution compared to the most related method (Geometry Score). As regards the related work, I believe that the paper \"The Shape of Data: Intrinsic Distance for Data Distributions\", A. Tsitsulin, et. al., ICLR 2020, should be cited and probably included in the experiments.\n\n\nQuality:\nAs far as I can tell, the technical part of the paper seems to be correct. In addition, the claims are supported by the experiments. Maybe some toy data with nonlinear manifolds could be used in order to show the behavior in complicated scenarios. The authors do not mention any weakness of the methodology, and I could not think of such a case (see Clarity).\n\n\nClarity:\nIn my opinion, the clarity of the paper should be improved. In particular, it is quite difficult for a non-expert reader (like me) to understand the intuition behind the proposed approach. Especially, in Sec 2.2 - 2.6 there is a lot of text that is not easily accessible. I think that some images could help the reader to understand better the proposed idea. Also, I am aware that the used terminology is in accordance to the topological data analysis field, but instead, I would like the authors to provide a manuscript easier to understand from a non-expert reader. Anyways, the proposed method aims researches in machine learning that potentially need a tool to compare generative models, and not only researchers familiar with topology.\n\n\nSignificance:\nI think that the proposed idea is potentially quite useful, since it provides a tool to assess the performance of generative modelling. This is, in general, rather useful for machine learning applications. From the experiments, it seems that the method is efficient and better from the related models. However, my main concern is that as a non-expert reader, I cannot understand in depth the proposed idea, so probably I cannot judge it objectively and find the potential weaknesses. In my opinion this limits the significance of the method.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do not discuss potential negative societal impact, neither the limitations of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "67dv0Obty3V", "reviewer": "Reviewer_kTLP", "summary": "The manuscript introduces the cross-barcode between two point clouds, and uses it to construct the manifold topology divergence (MTD). It is proposed that this can be used for performance evaluation of GANs. A selection of experiments are run to show that the MTD is robust where other evaluation metrics fail. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "It is a shame that most of the cross-barcode is thrown away in favour of just the 1st order one in the construction of the MTD, I assume that this is because computation of the higher order ones are infeasible for large data sets, but it would be nice with some comparisons on smaller sets or at least some more commentary on it.\n\nAs a domain agnostic tool I think it has applications, and the construction of the cross-barcode is interesting and could be the object of further research.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It is unclear to me how exactly the cross-barcode is constructed. In section 2.3 it says that the simplicial complex is constructed from the metric space consisting of P and Q where the pairwise distances in Q are set to zero. As this space is not Hausdorff, or a metric space, this looks slightly odd at first, and could probably be explained better.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Congratulations, the paper is accepted to NeurIPS 2021!
Please make an honest effort making this paper more accessible to general ML audience (non-experts in TDA). Clarify the barcode construction. Include persistent homology literature background. Elaborate limitations. Please include other clarifications, edits and additions as discussed in rebuttal and reviews. 
Decision: Accept (Poster)