Towards Scalable Topological Regularizers
Authors: Wong Hiu Tung
Darrick Lee
Hong Yan
Venue: ICLR 2025 Poster (2025)
Keywords: topological data analysis
persistent homology
generative adversarial network
latent space matching
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We use a subsampling to approximate persistent homology, which is a topological summary of point clouds, to obtain a parallelizable and smooth topological regularizer.

Abstract: Latent space matching, which consists of matching distributions of features in latent space, is a crucial component for tasks such as adversarial attacks and defenses, domain adaptation, and generative modelling.
    Metrics for probability measures, such as Wasserstein and maximum mean discrepancy, are commonly used to quantify the differences between such distributions.
    However, these are often costly to compute, or do not appropriately take the geometric and topological features of the distributions into consideration.
    Persistent homology is a tool from topological data analysis which quantifies the multi-scale topological structure of point clouds, and has recently been used as a topological regularizer in learning tasks.
    However, computation costs preclude larger scale computations, and discontinuities in the gradient lead to unstable training behavior such as in adversarial tasks. 
    We propose the use of principal persistence measures, based on computing the persistent homology of a large number of small subsamples, as a topological regularizer.
    We provide a parallelized GPU implementation of this regularizer, and prove that gradients are continuous for smooth densities.
    Furthermore, we demonstrate the efficacy of this regularizer on shape matching, image generation, and semi-supervised learning tasks, opening the door towards a scalable regularizer for topological features.

Published as a conference paper at ICLR 2025
TOWARDS SCALABLE TOPOLOGICAL REGULARIZERS
Hiu-Tung Wong": Darrick Lee Hong Yan 1.3
I Centtee for Intelligent Multidimensional Data Analysis, Science and Technology Park, Hong Kong 2School of Mathematics, University of Edinburgh, UK
DDepartment of Electrical Engineering. City University of Hong Kong, Kowloon, Hong Kong hi hiutunge i innocimda com, darrick, 1 ac. h.yanecityu. edu hk
ABSTRACT
Latent space matching, which consists of matching distributions of features in latent space, is crucial component for tasks such as adversarial attacks and defenses, domain adaptation, and generative modelling. Metrics for probability measures, such Wasserstein and maximum mean discrepancy, quantify the differences between such distributions. However, these are often costly compute, do not appropriately take the geometric and topological features of the distributions into consideration. Persistent homology is tool from topological data analysis which quantifies the multi-scale topological structure of point clouds, and has recently been used as a topological regularizer in learning tasks. However, computation costs preclude larger scale computations, and discontinuities in the gradient lead to unstable training behavior such as in adversarial tasks. We propose the use of principul persistence measures, based on computing the persistent homology of a large number of small subsamples, as a topological regularizer. We provide a parallelized GPU implementation of this regularizer, and prove demonstrate continuous for smooth densities. Furthermore,
commonly used
that gradients the efficacy of this regularizer on shape matching, image generation, and semi- supervised learning tasks, opening the door towards scalable regularizer for topological features.
INTRODUCTION
Latent space matching is fundamental task in deep learning. Quantifying differences in latent representations and optimizing the network accordingly enables applications such as adversarial attack and defenses (Yu et al., 2021; Madaan et al., 2020; Lin et al., 2020), domain adaptation (Sun et al., 2016; Sun & Saenko, 2016; Long et al., 2017; Xu et al., 2019) and few-shot learning (Schonfeld et al., 2019; Xu et al., 2022; Mondal et al., 2023). Unsupervised training frameworks such as Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are fundamentally built on this concept, which is the primary framework considered throughout this article.
Topological Features of Latent Representations. The manifold hypothesis states that real-world high dimensional data sets are often concentrated about lower dimensional submanifolds. Recent work has empirically verified that image datasets such CIFAR-10 and ImageNet satisfy union of manifolds hypothesis (Brown et al., 2022), where the intrinsic dimension of connected components may be different. In the context of GANs, correctly learning the geometric and topological properties of these lower-dimensions network latent spaces. Such properties are crucial to generalization ability (Zhou al., 2020; Wang et al., 2021b) and generation quality (Zhu et al., 2023; Katsumata et al., 2024). Furthermore, topological metrics based on persistent homology provide highly effective evaluation metrics for GANs (Zhou et al., 2021; Khrulkov & Oseledets, 2018; Barannikov et al., 2021; Charlier et al., 2019).
structures enable meaningful interpolation in data space by traversing
Standard approaches to latent space matching use metrics on probability measures such as the Wasser- stein distance and maximum mean discrepancy (MMD) metrics, which implicitly take topological features into consideration. In particular, the measures (and thus all topological properties) become
1 Equal contribution.

---

Published as a conference paper at ICLR 2025
equivalent when the distance is trivial. However, GANs not guaranteed to reach a global minimum, and often converge to local saddle points (Berard et al., 2019; Liang & Stokes, 2019). When measures a finite distance apart, their topological properties may be distinct. Motivated by the above work, which demonstrates that topological similarity between real and generated distributions is a critical component of GAN performance, propose the use of topological regularizer which explicitly measures the difference between topological features at non-equilibrimm states.
Persistent Homology. Persistent homology (PH) is a tool which summarizes the multi-scale topologcacl features of a dataset in an object called a persistence diagram. Such topological summaries have been applied in machine learning tasks (Hensel et al., 2021), such as image segmentation (Hu al., 2019; Clough et al., 2022, Shit et al., 2021; Waibel et al., 2022), and graph learning (Hom et al., 2021; Ballester & Rieck, 2024). The standard way to quantify the topological differences between datasets to compute the Wasserstein distance between their persistence diagrams. However, there are two difficulties in directly applying persistence-based methods in adversarial deep learning tasks.
1. Scalability. Persistent homology of large point cloud is prohibitively expensive to com- pute'. Even worse, the persistent homology algorithm is highly nontrivial to parallclize. Modern PH packages are either pure CPU implementations (Bauer, 2021; Pérez et al., 2021) or use a CPU-GPU hybrid algorithm (Zhang et al., 2020).
2. Smoothness. Persistent homology is differentiable almost everywhere, which allows us compute backpropagate through PH layers; in fact, stochastic subgradient descent is provably convergent with respect to persistence- based functions (Carriere et al., 2021). However, in adversarial tasks, where the loss function is constantly changing, discontinuities in the gradient leads to highly unstable training dynamics (Wiatrak et al., 2019).
Contributions. We address these two issues by modifying the two central parts of the classical persistence pipeline: the topological summary itself, as well as the metric used to compare them.
Principal Persistence Measures. To reduce the computational cost, we compute the persistent homology of many small batches of subsamples in parallel. By choosing a specific number points depending on the homology dimension, the persistence computation significantly simplifies, and we obtain an object called the principal persistence measure (PPM) (Gómez& & Mémoli, 2024). We provide a pure GPU implementation of the PPM, which enables scalable methodology to incorporate topological features in larger-scale ML tasks. Moreover, subsampling results a smoother features (Solomon et al., 2021), resulting in more stable training behavior.
Topological Summary:
Topological Metric: Maximum Mean Discrepancy for PPMs. The Wasserstein distance is the primary metric used to compare PPMs (Gómez & Mémoli, 2024). In practice, one often uses entropic regularization to lower the computational cost (Cuturi, 2013; Lacombe et al., 2018). Despite this, it is still computationally expensive, and we use maximum mean discrepancy (MMD) metrics to compare PPMs. This coincides with the persistence weighted kernels introduced in (Kusano et al., 2016) for persistence diagrams. Our main theoretical results deals with establishing this metric in PPM framework.
Theorem builds characteristic kernels for PPMs from kernels on R.2
Theorem shows that these MMD metries induce the same topology as Wasserstein.
- Theorem 3 shows that gradients with respect to this metric are continuous.
Theorem I and Theorem 2 adapt results from (Kusano et al., 2016; Divol & Lacombe, 2021) to the setting of PPMs, while the authors' knowledge, Theorem is novel.
These theoretical results imply that we can use PPM-Reg as an alternative to computationally expensive Wasserstein (or Sinkhorn) metrics, which produces stable gradient for training deeper networks. particular, the proposed methods allow us to incorporate topological features into large-scale machine learning tasks in stable manner (Papamarkou et al., 2024, Section 4.2). We
'The worst-case time complexity (mm²). where m is the number of simplices. Computing dimension k persistent homology of point cloud with n points can have up to m O()++1) simplices. However, for practical data sets, the complexity is often much lower; see (Otter al., 2017) for further discussion.
2

---

Published as a conference paper at ICLR 2025
demonstrate this empirically in Section 6, where we provide extensive experiments to demonstrate the efficacy of PPM-Reg in the GAN framework.
Related Work. The application of persistent homology in machine learning has been enabled by theoretical studies into the differentiability properties of PH (Carriere et al., 2021; Leygonie et al., 2022), which have also been extended the multiparameter setting (Scoccola et al., 2024). However, large-scale computation of PH remains a challenge, though recent work has considered computational strategies for optimization problems (Nigmetov & Morozov, 2024; Luo & Nelson, 2024). Our MMD metric for PPMs is also related to work on kernels for persistence diagrams Kusano et al. (2016) and linear representations of persistence diagrams Divol Lacombe (2021); Divol & Polonik (2019).
Subsampling methods for PH of metric measure spaces was introduced in (Blumberg et al., 2014), and used to approximate PH for point clouds (Chazal et al., 2015; Cao & Monod, 2022; Stolz, 2023). Furthermore, distributed approaches for computing the true PH of point clouds have been proposed in (Yoon & Ghrist, 2020; Torras-Casas, 2023) via spectral sequence methods. More recently, (Solomon al., 2021) used subsampling methods for topological function optimization, motivated by the same issues of computational cost and instability of gradients (Bendich et al., 2020), and (Solomon et al., 2022) showed that such distributed persistence methods interpolate between geometric and topological features based on the number of subsamples. The starting point of this article is (Gómez& & Mémoli, 2024), which introduces principal persistence measures.
2 LATENT SPACE MATCHING IN GENERATIVE ADVERSARIAL NETWORKS
Our primary consideration is the latent space matching in generative adversarial networks (GANs). GAN is an unsupervised training framework consisting of a generator Iw : RN + RR discriminator dg IRM - R² and value function V ((R P(R*) - R (Goodfellow et al., 2014). Consider a set of training data, such as a collection of images, which we view as a probability measure y R.M, the data space. The generator 9u is parameterized by E IRG and its goal is to map a given noise measure v on NR (the noise space) to IR MM such that Iw (v) can be interpreted as novel examples of μ. The discriminator dg. parametrized by 0 E RR, performs dimensionality reduction, sending the data space to the latent space Rt, Finally, the value function is used to quantify the difference between the real data μ and the generated data (w) by (do do (g (v))).
The generator is optimized such that it minimizes the value function, while the goal of the discrimina- tor is to maximize it. Training algorithms (Goodfellow et al., 2014; Arjovsky al., 2017; Gulrajani et al., 2017) have been proposed to find an equilibrium of the minimax problem, given by
max min V (p),, de (v)))
In practice, parameters in dg and Iw updated alternatively. Common value functions used are metrics between probability distributions such as the Wasserstein distance, which has better theoretical properties in solving the minimax problem with gradient descent (Arjovsky et al., 2017), and the Cramer distance (Bellemare et al., 2017), which has unbiased gradients with mini-batch training. However, these metries do not explicitly take topological features of the distributions into consideration. This motivates the use of a topological regularizer, which explicitly accounts for topological features non- couuiuuiii state. In particular, we consider value functions of the form
V = L + XT,
(2)
where L is the main loss function, À 0 is hyperparameter, T is our proposed topological regularizer which will be introduced in the following sections.
3 PRINCIPAL. PERSISTENCE MEASURES
We provide a streamlined exposition of the notion of principal persistence measures (Gómez & Mémoli, 2024). As there already exists several excellent references for persistent homology, we refer the reader to (Edelsbrunner & Harer, 2010: Dey & Wang, 2022: Hensel et al., 2021) for further background. Furthermore, we highlight the fact that only consider persistent homology for simple point clouds, with an explicit definition in Equation (4), For a topological space X, we use P(2) (resp. P.())) to denote the Borel probability measure (resp. with compact support) on X.
3

---

Published as a conference paper at ICLR 2025
Throughout this article, we consider persistent homology of point clouds with the Vietoris-Rips filtration.
(n)
(b)
0.4
0.8
1.4
(c)
PD
-80
(d)
(c)
15
1.58
1.82
(
PPM
15
Figure 1: An illustration of PH and PPMs. (a) An example point cloud X. (b) Snapshots of the
Vietoris-Rips filtration X, of X at various Edges are added between 2; and z when d(z;, Zz) > c and higher simplices are added when all pairwise distances are greater than c. (c) The dimension 1 persistence diagram of X in birth-lifeting coordinates. The one point with large lifetime represents the fact that there a hole in the dataset which persists through multiple scales. (d) An example of a
subsampling (in red) of 4 = 24- 2 points when g = 1. (e) Snapshots of the Vietoris-Rips filtration of the subsample, where the distances of the bold lines are to and t.- (f) The dimension I principal persistence measure of X. where the point given by the example subsample is shown in red.
Persistent Homology. Let X (2:) where x; R Persistent homology (Edelsbrunner & Harer, 2010) of dimension q E N builds a multi-scale topological summary of X three steps:
1. Construct sequence of topological spaces X, representing the point cloud at scale parameter > 0, equipped with inclusion maps X, + X, for c <<< e' (see Figure 1(b)).
2. Compute the dimension q homology of X. to obtain topological properties at each scale.
3. Track the birth, b, and lifetime, f, of topological features across scales by using the induced maps Hx,X)) - Hx())), and summarize this information as a multi-set (X)(X) =
{(be, (=) called persistence diagram? (see Figure 1(c)).
The points (b, E) PH2()) in persistence diagram are valued in the quotient of the half plane
Ω = {(b,e) E l 2 0)/{e=10 0},
as topological features (b, l) PH2(X) where points with trivial lifetime f = 0 are equivalent the feature not existing. We view this as a pointed quotient metric space (52, d, *), where d' is the quotient of the Euclidean metric on R² and represents the collapsed point {e - 0}.
Persistent Homology of Small Point Clouds. It is shown in (Gómez & Mémoli, 2024, Theorem 4.4) that PH, of point cloud S with exactly 2q + 2 points has at most single topological feature, and can be explicitly computed follows. Given a point & E S. let (1)), 2x(2) S denote the points such that d(x,x(1)) ()) 2 d(x, (2) 2 d(x; a) for all a E S - {x(1) (2))). Then,
PH = (tt,, tu t,), to == max d(x, x(21) ta := min (() xES
(4)
ZES
whenever tu 2 t.. and (S) {*} otherwise (see Figure ((e)). As we will exclusively consider PH, of 2q points, we will consider this as map PH2 (R+)2++2 Ω. We emphasize that Equation (4) is a significant simplification of the full persistent homology computation (Otter al., 2017), and is the key to parallelized computations discussed in Section 6.
Note that bifth-lifetime coordinates are linear transformation of the more standard birth-death coordinates. We use lifetime coordinates simplify the kemel expressions in the following section.
4

---

Published as a conference paper at ICLR 2025
Principal Persistence Measures. Principal persistence measures (PPMs) of dimension q (Gómez & Mémoli, 2024) contains PHg of all subsamples S a point cloud X with exactly |S| 29 + points. More formally, we will consider the more general setting of probability measures R with compact support rather than point clouds' on RR.. Then, the PPM of dimension is defined as
3 Pe(RR) + P(22, (PMG(H) == (4
(5)
where y.8n is the product measure on (TR*) ) and (PH₂), the pushforward map. In other words, we take 2q + i..d. samples from j and compute PH, on each collection to obtain probability measure on Qi (see Figure I(f).
Metrics and Stability. Let 1, and let W, denote the p> Wasserstein metric on R" and Ω. A key property shown in (Gómez & Mémoli, 2024, Theorem 3.8, Theorem 4. 1 is that PPMs are stable:
Wp (H) PPM, (v)) C,,,(u,PP,, for all J4, (R)
(6)
where C2 > O a constant which depends on q. In particular, p- Wasserstein metrics on 12 for PPMs is the analogue of the partial p- p-Wasserstein distance for persistence diagrams.
4 MAXIMUM MEAN DISCREPANCY FOR PPMS
In order to further reduce the computational cost and obtain smoothness properties, we will use maximum mean discrepancy (MMD) metrics to compare PPMs. The kernels defined here adapted from the persistence weighted kernels of Kusano et al. (2016). We assume basic familiarity with kernels and refer the reader to Appendix A for background.
Bounded PPMs and Notation. Throughout this section, work with bounded PPMs valued in
Dr = {(b, E) E [0, 2 0}/{e=1) 0}
for some T > 0. We continue to denote the collapsed point by *. Note that for μ E PR(R)", where the support of μ has diameter T, we have PPM (p) P(Dr). In order to simplify notation, we use Ω = 22 throughout this section. We use the notation (b, E) for elements in both [0, T)2 and Ω.
Kernels on Ω. Following the construction in Kusano et al. (2016), we introduce a procedure to turn kernel & on j0, 722 into kernel on Ω. Suppose k [0, 2 x [0, 7)² + R is kernel, where H is its reproducing kernel Hilbert space (RKHS), and let 4 [], - H. be the associated feature map given by (2) k(2,-). We define feature map Φ f2 - H. into the same RKHS by
(2) = E. ((2) = k(2, when E
and (x(())= = 0.
(8)
Then, for 21, E Ω {+], the associated kernel kn Ω x Ω + R. satisfies
kn 22) == (#2(21). (2(2))) = l l k(21,22)
by the reproducing kernel property of H. and ko(*,2) = kn x) = O. Note that kn is continuous on Q Q2 We denote the RKHS of kn by Ha, where we have an embedding Hy + H by definition.
Characteristic Kernels on Ω. Recall that kernel k X x X + R (with associated feature map & x + H) is characteristic with respect to probability measures P(X)) the kernel mean embedding, also denoted by tb P(x)) H.
((1) = Ezzap (x)),
(10)
injective. The following result shows that if we start with a characteristic kernel on (0, TJ2 the above procedure yields a characteristic kernel on Ω. This can be shown using similar methods as (Kusano et al., 2016, Section 3.1) in the current setting, but we provide an independent proof of slightly stronger statement in Theorem 5 of Appendix B.
Theorem 1. Let k [o. T x 2 + R be a kernel which is universal with respect to C((0, TT?) (or equivalently. characteristic with respect to P((O, TP?). Then, ką Ω x Ω - R is characteristic with respect to (S)).
Thhis is special case of the metric measure spaces used in (Gómez & Mémoli, 2024). Furthermore, we associate point cloud X {2x N C IR.", with the uniform probability measure μax on X.
5

---

Published as a conference paper at ICLR 2025
MMD for Principal Persistence Measures. A characteristic kernel ko ! induces metric on P(2) via the norm, called the maximum mean discrepancy (MMD),
MMD = [ - (
(11)
Let i1 = N (Ei=1 6n; + (N - n)).) and 1/2 = M
(Σ/=
j=1 9nj + (M- - m)))) discrete measures
in P(St), with n and m nontrivial points Iz, Uソ E Ω2
{+) respectively. The MMD is given by
MMD
1 N2
kn) NM
2
Σ ka Uj)
"
Σ M 2,j=1
By) (12)
The normalization is with respect the total (including numbers of points in V1 and 12, but only compute kernels between nontrivial points since kn (+, (= *) O. This enables the use of computable MMD metrics for PPMs. While the stability property in Equation (6) may no longer hold, MMD metries yield the same topology on the space of probability measures (see also (Kusano al., 2016, Theorem 3.2) in the finite setting). While related result in different context given in (Divol & Lacombe, 2021, Proposition 5.1), we provide an independent proof in Appendix C.
Theorem 2. Let k be a characteristic kernel Q. The p- Wassersttiin metric Wp and the MMD metric MMDs induce the same topology on P(2).
Remark 1. By viewing persistence diagrams measures (Divol & Lacombe, 2021; Giusti & Lee,
2023; Bubenik & Elchesen, 2022). persistence diagrams can be viewed as elements in Min(S)) defined in Equation (30). This includes persistence diagrams with possibly infinite cardinality (with finite total persistence). While in Kusano et al. (2016), analogous kernels are defined for finite persistence diagrams, our results hold for persistence measures in Min(S.).
5 TOPOLOGICAL REGULARIZATION WITH PPMs
In this section, we introduce our proposed topological regularizer based on computing the PPM of probability measures and comparing the PPMs using MMD. Let kn be characteristic kernel as defined in the previous section. Returning to the notation of Section 2, we define our dimension
topological regularizer, PPM-Reg, on the latent space R& by T Pa (IR Pa( + RR by
Ta(p, v/ == MMD kn
(KPM, (PPM, ma
(13)
When we apply this in the GAN setting, we consider Ty (do(M), (do()),dlgga do where v Pc(RN) is the noise measure, and E (R is the data measure. Let T, R x Rd + R be Ig(w, 0) Ty (do dows)()))))
(14)
Our main result of this section is to show that Fy is smooth with respect to w and 0 given sufficient smoothness conditions the underlying measures and the discriminator and generator. Recall that function f R" + Rm is a C'l function if all first derivatives of f are continuous. The following is our main theoretical result, proved in Appendix D.
Theorem 3. Let kn be a characteristic kernel. Suppose E P.(RM and v E (P((R have C densities. Suppose the joint functions G RG RRN RM desined by G(w,,) x) (x) and D : RD x IR + RR_ defined by D(O,y) = do(u) be C² functions. Then, Fg is a C² function wherever the PPM is not the trivial measure at the origin.
6 EXPERIMENTS AND RESULTS
We provide empirical experiments which demonstrates the efficacy of PPM-Reg as a topological regularizer. First, in Section 6.2, we provide an expository shape matching experiment to illustrate the behavior of PPM-Reg, and provide computational comparisons. Next, in Section 6.3, we apply PPM-Reg GAN-based generative modelling problem, consistently improving the generative quality of GANs. Finally, in Section 6.4, we consider GAN-based semi-supervised learning problem, which demonstrates the effectiveness of PPM-Reg in improving the discriminative ability of GANs. Due to space limitations, we have placed implementation details and additional experiments for each of the three settings in Appendix E, Appendix F, and Appendix G.*
*Code supp.: https: / /gi thub. can /htt//ggtwntoeooeeeeeeee/ooo ai /Sca 1 abl e'Topol og Regul ari arizers.

---

Published as a conference paper at ICLR 2025
6.1 COMPUTATIONAL SETUP AND IMPLEMENTATION OVERVIEW
Cramer Distance. The Wasserstein and Cramer distance are both probability metrics that are sensitive the geometry of the change in distribution (Bellemare et al., 2017). Moreover, the Cramer distance does not depend on hyperparameters which simplifies our comparison. We primarily use the Cramer metric as our main loss function L. Following the definition of (Bellemare et al., 2017), for H., ). The Cramer Distance {1, v) is defined as
E(u,) v) Ezzp [D(x) - y (vy), D(=) Ey [||
v/112] Beg |||= x' 21112]
where 2, x² (resp. v/) are independent random variables with law (resp. v). We do not take the gradient estimation (Bellemare al., 2017, Appendix C.3) we obtain sufficient samples.
Implementation of PPM. For all experiments, we use subsamples from μ 21(2)))) approximate the PPM (using the same number of subsamples for dimension O and 1). The persistent homology of each subsample is computed using Equation (4) in parallel on the GPU. Throughout these experiments,
our base kemne is the radial basis function (RBF) kernel kRBF [21 22) = exp (-ll21 22 /2/l
where the width a > 0 is hyperparameter. Thus, the induced kernel kn f2 R from Equa- tion (9) evaluated on 2; (b;, is
(o(=1, 22) = l exp (-1|21 221l2/20).
(15)
We use Equation (12) to compute the MMD metric between PPMs. Furthermore, we use a weighted combination of dimension 0 and PPM in our topological regularizer, such that
T To T1
(16)
where the weights 2o A are hyperparameters. We will call this PPM-Reg.
6.2 SHAPE MATCHING
Initial Condition
Initial Condition
05
03
-0
40
415
Cramer
MMD
Cramer
MMD
Deea
Demer
1903
R23
350
180
10%
300
1000
1500
1200
Figure 2: Visual example of PPM-Reg in a shape matching experiment using Cramer or MMD the main loss function. 1st row: Plots of reference point cloud (in blue) and the initial condition of a random point cloud (in orange). 2nd Row: Plots of 2-Wasserstein distance between I-dimensional persistent homology between the reference shape and training shape over optimization steps.
Our task is to optimize the individual points of a point cloud to match the "shape" of reference point cloud using a loss function of the form L + T, which operates directly on the ambient space of the point clouds. We choose L to either be the Cramer distance (Bellemare et al., 2017) or an MMD metric using an RBF kernel (with width 0.1). Our aim in this expository experiment twofold.
1. Demonstrate the ability of PPM-Reg to regularize topological features in point clouds by comparing the true sunssuussapled) persistence diagrams of the trained and reference shapes. Our focus is showing that this occurs near the beginning of the optimization, since in GAN settings, the regularization is important away from global minima (see Introduction).
2. Show the computational efficiency of PPM-Reg, which enables its use in later experiments.
7

---

Published as a conference paper at ICLR 2025
Shape Matching Experiment. Our main results summarized in Figure 2. We choose two reference shapes in R for visualization purposes: a circle, and the union of two intersecting circles. In the second row of Figure 2, we plot the 2- 2-Wasserstein distance between the 1 1-dimensional full (non-subsampled) persistence diagrams between the fixed reference and the trained point cloud, as a function of optimization steps. In each case, see that adding PPM-Reg significantly reduces the topological distance. An interesting feature of each of these plots is that there is an initial spike in the PD distance when using PPM-Reg. Empirically, this is due to the fact that the point cloud must first move through a regime with trivial topological structure before PPM-Reg can faithfully match the topology of the reference. The training behavior best understood by observing the dynamics of the optimization, and we provide animations of these experiments in the supplementary material.
Computational Comparisons. The efficiency of the PPM-Reg is derived from two major compo- nents: the parallelizable PPM and the iterative-free MMD. Table empirically shows the compu- tational benefit of each component as the size of the point cloud and the number of subsamples s varied. We use Cramer the main loss, and consider the computational cost of using PPM-Reg, W-PPM-Reg and PD-Reg. PD-Reg computes the 2- Wasserstein distance between dimension 0 and 1 full persistent homology with Vietoris-Rips filtration. W-PPM-Reg computes the 2- Wasssersteein distance between PPMs of dimension 0 and 1. We use the torch topoolgical package (Lab) compute persistent homology and Wasserstein distances.
Our aim is to compare the real-world usage of these methods using the circle experiment. Thus, PPM-Reg computations are performed on GPU, while PD-Reg and W-PPM-Reg uses hybrid CPU-GPU methods. The computational cost of PD-Reg grows exponentially with respect to the size of the point cloud. While the computational cost of W-PPM-Reg is sublinear with respect to the size of the point cloud, the cost is exponential with respect to the number of subsamples s. Remarkably, due to parallelization, PPM-Reg sublinear with respect to the number of subsamples & and is nearly constant as the size of the point cloud increases. In the following experiments, we find that s = 1024 and s = 2048 performs well in practice. In summary, using MMD mediates the drawback of the increased number of features extracted by PPM. resulting in our significantly faster PPM-Reg With parallelization, our pure PyTorch implementation of PPM-Reg outperforms highly optimized low-level CPU implementations used in torch logicol.
Cramer + PPM-Reg
Cramer W-PPM-Reg
Cramer PD-Reg
No. points
512
1024
2048
51512
1024
5= 2048
128
0.55
0.005 0.61 0.007 0.98 0.001
3.06 0.033 13.28
0.198
73.00
3.323
1.99 + 0.048
256
0.56
+ 0.008
0.61 4 0.005 0.99 + 0.005
3.25 : 0.083
14.04 +: 0.187
80.11
2 2.545
10.43 1 0.075
512
0.010
3.43
0.111
16.43
0.458
91.29
3.081
107.11 2.837
1024
0.57 4 0.005 0.61 + 0.005 1.00 0.007 3.90 + 0.092 19.45 + 0.468
121.76
4 3.424
655.58 + 11.823
Table 1: Running time of 100 gradient steps (in seconds) in matching circle form randomly initialize gaussian in g² with GPU computation enable. The averages are computed over 10 runs.
Imperfect Convergence. In Appendix E.2, we observe the same trends in additional modified experiments, which prevent the centroid of the trained shape from converging to the centroid of the reference, This done mimic the GAN setting where training algorithms often converge to saddle points rather than global minima (Berard et al., 2019; Liang & Stokes, 2019).
6.3 UNCONDITIONAL IMAGE GENERATION
Next, we consider the use of PPM-Reg in an unconditional image generation task, which is the standard benchmark evaluate GANs (Goodfellow et al., 2014; Arjovsky et al., 2017).
Network Architecture and Implementation Details. We use ResNet based CNN as the generator Iws which takes a 128- 228dimensionnl noise vector as input. We use a CNN as the discriminator de and the output of the network is a 228-iimensional latent vector. We compare the Cramer value function V = L. with the use of PPM-Reg V = L ++7 T. As our network differs from (Bellemare et al., 2017), retrain both regularized and unregularized networks for fair comparison.
8

---

Published as a conference paper at ICLR 2025
Dataset and Evaluation Metrics. We consider the CelebA (Liu et al., 2015) and Anime- Face (Churchill & Chao, 2019) datasets. Images are centered and resized to 32 x 32. While the Frechét Inception Distance (FID) (Heusel et al., 2017) is a popular metric to evaluate the distance between generated and real images, recent empirical work has thoroughly investigated several draw- backs of FID (Horak al., 2021; Stein al., 2024; Jayasumana al., 2024). Instead, we adopt three metrics:
1. CMMD (Jayasumana et al., 2024): MMD of CLIP embeddings (Radford et al., 2021),
2. Dinnov22 (Stein et al., 2024): Frechét Distance of Dinov2 (Oquab et al., 2024) embeddings
3. WDittemt Wasserstein distance of CLIP embeddings (Radford et al., 2021)
In Table 2, these are computed by sampling/ / generating 10K images from the data set / network. We report CMMD, Dinovv22 and WDttt using the epoch with the smallest CMMD,
AnimeFace
CelebA
CMMD
FD
WDisent
CMMD
FDD
WDate
Cramer (Bellemare et al., 2017)
0.73
953.99
0.6294
0.72
722.86
0.6795
Cramer + PPM-R-- -egg
0.56
780.68
0.6080
0.58
700.73
0.6666
Table Quantitative evaluation on 32 32 image generation, values are reported the epoch with the smallest CMMD,
CMMD
FD
WD ptent
Anime
Cumer
Camer
Ang
Comare
Face
3000
2230
1.00
3040
300
4003
5000
000
3000
2000
3000
500
5000
(8)
0pch
Dm:hh
1pmch
3000
0.70
Corte
CelebA
connel Ang
Crawe LAMAA
0.00
505
750
0pcch
Occh
Figure 3: CMMD (a,d), FDpmov2 (b,e) and WD (c,f) versus training epochs for the AnimeFace (a-c) and CelebA (d-f) dataset. 10K samples are randomly generated to compute distances; moving averages with a window of are used to smooth the values. Distances recorded every 160 epochs.
Results. Figure 3 tracks these three metrics during training. Figure 3 (a.b,d,e) shows that using PPM- Reg improves image generation quality for both AnimeFace and CelebA. The Wasserstein distance can better detect geometric information in embedding space. Tracking WD in Figure 3 (c.f) shows that adding PPM-Reg provides more information and helps discover geometric structures in the latent space in an unsupervised way. This reinforces work that shows that persistence-bies based methods are able to effectively measure image generation quality (Zhou al., 2021; Khrulkov & Oseledets, 2018; 88; Barannikov et al., 2021; Charlier et al., 2019). As training progresses, improved Cramer loss does not always lead to improved evaluation metrics (Figure 3 (a,c)). Our reported results use CMMD an early stopping criterion which is prohibitively expensive to compute in practice. In contrast, the evaluation metrics consistently decrease with respect to training time, and this implies that may not need to compute additional metries for early stopping. In Appendix F.2, consider larger (64 x 64) image generation experiments with the CelebA and LSUN Kitchen datasets, and find similarly improved results, demonstrating the efficacy of PPM-Reg in larger-scale experiments.
9

---

Published as a conference paper at ICLR 2025
6.4 SEMI-SUPERVISED LEARNING
Semi- superviie learning (SSL) methods use unlabeled data alongside small amount of labeled data to train a classification network (Yang al., 2022). SSL often assumes that classification problems are supported on low-dimensional manifolds, which allows network to learn the classification problem with limited labels (Niyogi, 2013). In practice, knowledge of the low-dimensional manifold can be learned by encoding the unlabeled data to latent representations. With few labeled data points, a simple classifier is trained using those latent representations (Wang et al., 202 Decourt & Duong, 2020: Truong et al., 2019; Das et al., 2021). Here, demonstrate PPM-Reg can help encode more informative latent representations, and significantly reduce classification error in SSL.
Network Architecture and Implementation Details. We use a deconvolutional network as yu. which takes 64 dimension noise vector input. We use CNN as do and the output of the network is a 64 dimension latent vector. We use an MLP as a classifier parameterized by y termed Cy. We first learn the latent representations using a Cramer GAN (Bellemare al., 2017) framework with all available data, and compare it against the addition of PPM-Reg. After training the GAN, the discriminator dg is frozen and its output is used as the features train the classifier c,, with the subset of training samples. For comparison without latent representations learning, we consider "Baseline", where de and C, are trained together as a classifier (without the generative part).
Dataset and Evaluation Metrics. We compare the SSL performance with Fashion-MNNN (Xiao, 2017), Kuzushiji-MNISS (Clanuwat et al., 2018) and MNIST. In these experiments, 200 and 400 labels are randomly sampled from the data set. Due to the inherent randomness in sampling few labels, experiments repeated ten times and the statistics of the best test-set accuracy reported.
Fashion- Fashion-MNIST MNIST
Kuzushiji-Mi MNIST
MNIST
Number of labels
200
400
200
400
200
400
Baseline
67.18 + 0.95
71.00
+ 0.83 48.40
+ 1.79 55. + 1.55 80.52
+ 1.49 86.39 + 1.14
Cramer
62.70 + 1.25 68.58 + 1.08
47.77 + 1.40
56.06 + 1.88
71.10 + 1.52 78.26 + 1.30
Cramer + PPM-Reg 76.84 + 1.23 80.59 + 0.69 75.78 + 1.99 79.33 + 1.69 96.62 + 0.39 97.33 + 0.21
Table 3: Test-set classification accuracy (%) on Fashion-MNIST, KuzushijiMNNIST and MNIST. with 200 and 400 labeled examples. The average and the error bar are computed over 10 runs.
Result. Table 3 shows the test classification accuracy, where we use only 0.33% (200) and 0.66% (400) of the total number of labels. Compared with Baseline, only using Cramer does not significantly improve the classification accuracy SSL. Note that while the Cramer GAN (without PPM-Reg) has reasonable generative ability, shown in Figure 7, this does not imply strong discriminator performance in SSL. Remarkably, using PPM-Reg significantly improves the classification accuracy. For example, compared with the Baseline, PrzzushjjiMNNIST has gain 27.38% improvement with 200 labels. Notably, with the latent representations learned with PPM-Reg, we can get a good accuracy using only 0.66% of the labels. This section demonstrates that discovering topological structures in latent space is not only useful in generative tasks, but can be leveraged to massively improve classification accuracy when very few labels are available. In Appendix G.2, we observe similar performance gains in additional experiments on the SVHN dataset.
CONCLUSION
In this article, we propose a novel method for stable and scalable topological regularization based on the subsampling principle of PPMs, opening up the possibility of detecting topological information in larger-scale machine learning problems. We introduced a theoretical framework for using kernel methods and MMD metrics for PPMs, and demonstrated the efficacy of this methodology in variety of experimental settings. This work suggests several directions for future study. From the develop parallelizable approximate computations
theoretical and computational perspective, in more general settings? From an applied perspective, how can we leverage approximate topological summaries in further machine learning tasks such classification regression?
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by the Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA).
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Interaational conference on machine learning, pp. PMLR, 2017.
N. Aronszajn. Theory of reproducing kernels. Trans. Amer. Math. Soc., 68(3):337-404, 1950. ISSN 0002-9947. doi: 10.2307/1990404.
Rubén Ballester and Bastian Rieck. On the Expressivity of Persistent Homology in Graph Learning, June 2024.
Serguei Barannikov, Ilya Trofimov, Grigorii Sotnikov, Ekaterina Trimbach, Alexander Korotin, Alexander Filippov, and Evgeny Burnaev. Manifold topology divergence: A framework for comparing data manifolds. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021.
Ulrich Bauer. Ripser: Efficient computation of Vietoris-Rips persistence barcodes. Journal of
Applied and Computational Topology, 5(3):391 September 2021. ISSN 2367-1734. doi:
Marc G Bellemare, Ivo Danihelka, Will Dabney, Shakir Mohamed, Balaji Lakshminarayanan, Stephan Hoyer, and Rémi Munos. The cramer distance as solution to biased wasserstein gradients. arXiv preprint 1705v:1705.1777 2017.
Paul Bendich, Peter Bubenik, and Alexander Wagner. Stabilizing the unstable output of persistent homology computations. Journal of Applied and Computational Topology, 4(2):309-338, June 2020. ISSN 2367-1734. doi: 10. 1000/0//4411 //t///41166--01 468-001900004-99
Hugo Berard, Gauthier Gidel, Amjad Almahairi, Pascal Vincent, and Simon Lacoste-Julien. A closer look at the optimization landscapes of generative adversarial networks. In International Conference Learning Representations, 2019.
Andrew J. Blumberg, Itamar Gal, Michael A. Mandell, and Matthew Pancia. Robust Statistics, Hypoth- esis Testing, and Confidence Intervals for Persistent Homology on Metric Measure Spaces. Found. (00007//12000014922000
Comput. Math, (4(4):745-78, August 2014. ISSN 1615-3383. doi:
Bradley CA Brown, Anthony L Caterini, Brendan Leigh Ross, Jesse C Cresswell, and Gabriel Loaiza-Ganem. Verifying the union of manifolds hypothesis for image data. arXiv preprint arXhv:2207.02862. 2022.
Peter Bubenik and Alex Elchesen. Virtual persistence diagrams, signed measures, Wasserstein distances, and Banach spaces. Journal of Applied and Computational Topology, 6(4):429 December 2022. ISSN 2367-1734. doi: 10.1007/s41468- 0004444468-2222-000 022-00091-99
Yueqi Cao and Anthea Monod. Approximating Persistent Homology for Large Datasets, May 2022.
Mathieu Carriere, Frederic Chazal, Mare Glisse, Yuichi Ike, Hariprasad Kannan, and Yuhei Umeda. Optimizing persistent homology based functions. In Proceedings of the 38th International Confer- ence on Machine Learning, pp. 1294-1303. PMLR, July 2021.
Jeremy Charlier, Radu State, et al. PHom-GeMr Persistent homology for generative models. The 6th Swiss Conference on Data Science (SDS), 2019 IEEE International Conference. IEEE, 2019.
Frédéric Chazal, Brittany Terese Fasy, Fabrizio Lecci, Bertrand Michel, Alessandro Rinaldo, and Larry Wasserman. Subsampling methods for persistent homology. In Proceedings of the 32nd International Conference on International Conference on Machine Learning Volume 37, ICML'IS 15, pp. 2143-2151, Lille, France, July 20155 JMI..org.

---

Published as a conference paper at ICLR 2025
Spencer Churchill and Brian Chao. Anime face dataset, 2019. URL https://www. com/ds/ 3797797
Tarin Clanuwat, Mikel Bober-Irizat Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv:1812.01778, 2018.
J. R. Clough, N. Byrne, I. Oksuz, V. A. Zimmer, A. Schnabel, and A. P. King. A topologiaal loss function for deep-learning based image segmentation using persistent homology. IEEE Transactions on Pattern Analysis &amp: Machine Intelligence, 44(12):876688778, December 2022. ISSN 1939-3539. doi: 10. 10.1109/PAMM22200000000
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
Asha Das, Vinod Kumar Devarampati, and Madhu S Nair. Nas-sgan: semi-supervised generative adversarial network model for atypia scoring of breast cancer histopathological images. IEEE Journal of Biomedical and Health Informatics, 2665):277--2887, 2021.
Colin Decourt and Luc Duong. Semi-supervised generative adversarial networks for the segmentation of the left ventricle in pediatric mri. Computers in Biology and Medicine, 123:103884, 2020.
Tamal Krishna Dey and Yusu Wang. Computational Topology for Data Analysis. Cambridge University Press, March 2022. ISBN 978-1-00-910311--0.
Dieudonne. Foundations of Modern Analysis. Academic Press, New York, 1969. ISBN 978-1- 4465-4751-9.
Vincent Divol and Théo Lacombe. Understanding the topology and the geometry of the space of persistence diagrams via optimal partial transport. Appl. Comput. Topol., 5(1):1-53, March 2021.
ISSN 2367-1734. doi: 10.1007/s41466 468-000-00001--22
Vincent Divol and Wolfgang Polonik. On the choice of weight functions for linear representations of persistence diagrams. J. Appl. Comput. Topol., 3(3):249-283, September 2019. ISSN 2367-1734.
doi: 10.1007/s41468. 148--11--0002----
Herbert Edelsbrunner and John L. Harer. Computational Topology: An Introduction. American Mathematical Society, 2010. ISBN 978-1-4704-6766-22
Chad Giusti and Darrick Lee. Signatures, lipschitz-free spaces, and paths of persistence diagrams.
SIAM Journal Applied Algebra and Geometry, 7(4):828 (266666,666, 2023, doi: 10.1137/22M1528771
Mario Gómez and Facundo Mémoli. Curvature Sets Over Persistence Diagrams. Discrete & Compu- tational Geometry, 72(1):91-180 July 2024. ISSN 1432-0444. doi: 10.10077/500454-024400066444
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Faries Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. Mach. Learn. Res., 13:723-773, March 2012. ISSN 1533-7928.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
Felix Hensel, Michael Moor, and Bastian Rieck. A Survey of Topological Machine Learning Methods. Frontiers in Artificial Intelligence, 4, May 2021. ISSN 2624-8212. doi: 00.3389/frai.2221.661 108.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Berhaard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
12

---

Published as a conference paper at ICLR 2025
Danijela Horak, Simiao Yu, and Gholamreza Based Approach for Evaluating Generative Adversarial Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9):7721-7725 May 2021. ISSN 2374-3468. doi: 10.
Topology Distance: A Topology-
1609/aaai. v35i9. 16943.
Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, and Karsten Borgwardt. Topological Graph Neural Networks. In International Conference on Learning Representations, October 2021.
Xiaoling Hu, Fuxin Li, Dimitris Samaras, and Chao Chen. mentation. In Advances in Neural Information Processing Systems, volume 32, Curran Associates, Inc., 2019.
Deep Image Seg-
Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2024.
Kai Katsumata, Duc Minh Vo, Bei Liu, and Hideki Nakayama. Revisiting latent space of gan inversion for robust real image editing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5313-5322 5322,, 2024.
Valentin Khrulkov and Ivan Oseledets. Geometry score: A method for comparing generative adver- sarial networks. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp- 2621-2629, PMLR, 2018
Genki Kusano, Yasuaki Hiraoka, and Kenji Fukumizu. Persistence weighted Gaussian kernel for topological data analysis. In Maria-Florina Balean and Kilian Q. Weinberger (eds.). Proceedings of the 33nd International Conference Machine Learning, volume 48 of JMLR Workshop and Conference Proceedings, pp. 2004-2013. JMLR.org, 2016.
AIDOS Lab. Pytorch-topological:: topological machine learning framework for pytorch. https: //github..r cotmm/aioos-lee syytorch-tope ogi ca ?ttt= ov- foieeiloo-lleee
Théo Lacombe, Marco Cuturi, and Steve Oudot. Large scale computation of means and clusters for persistence diagrams using optimal transport. Advances in Neural Information Processing Systems, 31, 2018.
Jacob Leygonie, Steve Oudot, and Ulrike Tillmann. A Framework for Differential Calculus on Persistence Barcodes. Foundations of Computational Mathematics, 1269:1109 August 2022. ISSN 1615-3383, doi: 10. (00000/10000000-0099009
Tengyuan Liang and James Stokes. Interaction matters: A note non-asymptotic local convergence of generative adversarial networks. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 907-915. PMLR, 2019.
Wei-An Lin, Chun Pong Lau, Alexander Levine, Rama Chellappa, and Soheil Feizi. Dual manifold adversarial robustness: Defense against Ip and non-lp adversarial attacks. Advances in Neural Information Processing Systems, 33:3487-3498, 2020.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael Jordan. Deep transfer learning with joint adaptation networks. In International conference machine learning, pp. 2208-222 2217. PMLR, 2017.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint /608 htt.://080.038.... 2016.
Yuan Luo and Bradley J. Nelson. Accelerating iterated persistent homology computations with warm starts. Computational Geometry, 120: 102089, June 2024. ISSN 0925-7721. doi: 10.1016/j.
2024ge..022.1020883
13

---

Published as a conference paper at ICLR 2025
Divyam Madaan, Jinwoo Shin, and Sung Ju Hwang. Adversarial neural pruning with latent vulner- ability suppression. In International Conference on Machine Learning, pp. 6575-6585, PMLR, 2020.
Arnab Kumar Mondal, Piyush Tiwary, Parag Singla, and AP Prathosh. Few-shot cross-domain image generation via inference-time latent-code learning. In The Eleventh International Conference on Learning Representations, 2023.
Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf. Kernel Mean Embedding of Distributions: A Review and Beyond. 2017. doi: 10.1561/2200000060.
Arnur Nigmetov and Dmitriy Morozov. Topological Optimization with Big Steps. Discrete & Computational Geometry, 72(1):310-344, July 2024. ISSN 1432-0444. doi: 10.1007/
500454--023-06613xx
Partha Niyogi. Manifold regularization and semi-supervised learning: Some theoretical analyses. Journal of Machine Learning Research, 14(5), 2013.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alazeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research Journal, pp. 1-31, 2024.
Nina Otter, Mason A. Porter, Ulrike Tillmann, Peter Grindrod, and Heather A. Harrington, roadmap for the computation of persistent homology. EPJ Data Sci., 6(1):1-38, December 2017. ISSN 2193-1127 127. doi: 10.1
Theodore Papamarkou, Tolga Birdal, Michael M. Bronstein, Gunnar E. Carlsson, Justin Curry, Yue Gao, Mustafa Hajij. Roland Kwitt, Pietro Lio, Paolo Di Lorenzo. Vasileios Maroulas, Nina Miolane, Farzana Nasrin, Karthikeyan Natesan Ramamurthy, Bastian Rieck, Simone Scardapane, Michael Schaub, Petar Veličkovié, Bei Wang, Yusu Wang, Guowei Wei, and Ghada Zamzmi. Position: Topological deep learning the new frontier for relational learning. In Forty-first International Conference on Machine Learning, 2024. URL https:/ /tpps////////eeeeww net fofumrrrmdd N13RG5XWAt.
Julián Burella Pérez, Sydney Hauke, Umberto Lupo, Matteo Caorsi, and Alberto Dassatti. Giotto-ph: A Python Library for High-Performance Computation of Persistent Homology of Vietoris-Rip Filtrations, August 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Darrell, and Zeynep Akata. Generalized zero-and few-shot learning via aligned variational autoencoders. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8247-8255, 2019.
Luis Scoccola, Siddharth Setlur, David Loiseaux, Mathieu Carrière, and Steve Oudot. Differentiability and Optimization of Multiparameter Persistent Homology. In Proceedings of the 41 International Conference on Machine Learning, pp. 43986-44011. PMLR, July 2024.
Suprosanna Shit, Johannes C. Paetzold, Anjany Sekuboyina, Ivan Ezhov, Alexander Unger, Andrey Zhylka, Josien P. W. Pluim, Ulrich Bauer, and Bjoern H. Menze. clDice Novel Topology- Preserving Loss Function for Tubular Structure Segmentation. In 202/ IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16555-16564, June 2021. doi: 10.1109/
CVPR46437.2021..012
Carl-Johann Simon Gabriel and Bernhard Schölkopf. Kernel distribution embeddings: Universal kernels, characteristic kernels and kernel metrics on distributions. Mach. Learn. Res., 19(44): 1-29, 2018. ISSN 1533-7928.
14

---

Published as a conference paper at ICLR 2025
Carl-Johann Gabron, Alessandro Barp, Bernhard Schölkopf. and Lester Mackey. Metrizing Weak Convergence with Maximum Mean Discrepancies. Journal of Machine Learning Research, 24(184):1-20, 2023. ISSN 1533-7928.
Elchanan Solomon, Alexander Wagner, and Paul Bendich. From geometry to topology: Inverse theorems for distributed persistence. In Xavier Goaoc and Michael Kerber (eds.). 38th International Symposium Computational Geometry (SoCG 2022), volume 224 of Leibniz International
Proceedings in Informatics (LIPlcs), pp. 61:1-61:16, Dagstuhl, Germany, 2022. Schloss Dagstuhl- Leibniz-Zentrum für Informatik. ISBN 978-3-95977-227-3. doi: 1.42300L.PPcs.SSCG..000
Yitzchak Solomon, Alexander Wagner, and Paul Bendich. A Fast and Robust Method for Global Topological Functional Optimization. In Proceedings The 24th International Conference on Artificial Intelligence and Statistics, pp. 109-117. PMLR, March 2021.
Bharath Sriperumbudur. On the optimal estimation of probability measures in weak and strong topologies. Bernoulli, 1233):1839-1893, August 2016. ISSN 1350-7265. doi:
George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony I. Caterini, Eric Taylor, and Gabriel Loaiza-Ganemm. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Bernadette J. Stolz. Outlier-Robust Subsampling Techniques for Persistent Homology. Journal of Machine Learning Research, 24(90):1 2023. ISSN 1533-7928.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14, pp. 443-450. Springer, 2016.
Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Álvaro Torras-Casan Casas. Distributing Persistent Homology via Spectral Sequences. Discrete & Computational Geometry, 70(3):580-619, October 2023. ISSN 1432-0444. doi: 10.1007/ s00454-003-00549922
Nhan Duy Truong, Luping Zhou, and Omid Kavehei. Semi-supervised seizure prediction with generative adversarial networks. In 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp. 2369-2372. IEEE, 2019.
Cédric Villani. Optimal Transport: Old and New. Grundlehren Der Mathematischen Wis- senschaften. Springer-Verlag. Verlag. Berlin Heidelberg, 2009. ISBN 978-3-540-71049-3. doi: 10.1007/978- 0-77778--540-71500-0
Dominik J. E. Waibel, Scott Atwell, Matthias Meier, Carsten Marr, and Bastian Rieck. Capturing Shape Information with Multi-scale Topological Loss Terms for 3D Reconstruction. In Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, and Shuo Li (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2022, Lecture Notes in Computer Science, pp. 150-159, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-16440-8 doi: 10.1007. 1-6-00011-644-81111 15.
Lei Wang, Xin Yan, Zhu- Ahu-Hong You, Xi Zhou, Yuan Li, and Yu-An Huang. Sganrda: semi- supervised generative adversarial networks for predicting circrna-disease associations. Briefings in Bioinformatics, 22(5):bba 2021a
Zijian Wang, Yadan Luo, Ruihong Qiu, Zi Huang, and Mahsa Baktashmotlagh. Learning diversify for single domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 834-843, 2021b.
Maciej Wiatrak, Stefano V Albrecht, and Andrew Nystrom. Stabilizing generative adversarial networks: A survey. arXiv preprint arXiv: 19X0000/000002.., 2019.
H Xiao. Fashion-mnist a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: itai:/0088777777 2017.
15

---

Published as a conference paper at ICLR 2025
Jian Xu, Bo Liu, and Yanshan Xiao. A multitask latent feature augmentation method for few-shot learning. IEEE Transactions on neural networks and learning systems, 2022.
Rujjia Xu, Guanbin Li, Jihan Yang, and Liang Lin. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1426-1435, 2019.
Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, 35(9):8934 8954, 2022.
Hee Rhang Yoon and Robert Ghrist. Persistence by parts: Multiscale feature detection via distributed persistent homology. arXiv: 2001.01623, 2020.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506 03366, 2015.
Yunrui Yu, Xitong Gao, and Cheng-Zhong Xu. Lafeat: Piercing through adversarial defenses with latent features. In Proceedings of the IEEF/CVF conference computer vision and paltern recognition, pp. 5735-5745, 2021.
Simon Zhang. Mengbai Xiao, and Hao Wang. GPU-Accelerated Computation Vietoris-Rip Persistence Barcodes. In DROPS- WDWPSS//DD/D////e///////// http://aeeeee.10.43////......... Sww.IPPPes.SCG..2000 w0..c.00200000000 70. Schloss 2ttp:////0044000.....s////000000
Dagstuhl- Leibniz-Zeettumm für Informatik, 2020. doi: 111.33//..Pccc.SS0G..000000
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part XVI 16, pp. 561-578. Springer, 2020.
Sharon Zhou, Eric Zelikman, Fred Lu, Andrew Y. Ng, Gunnar E. Carlsson, and Stefano Ermon. Eval- uating the disentanglement of deep generative models through manifold topology. International Conference Learning Representations, 2021.
Jiapeng Zhu, Ceyuan Yang, Kecheng Zheng, Yinghao Xu, Zifan Shi, and Yujun Shen. Exploring sparse moe in gans for text-conditioned image synthesis. arXiv preprint arXiv:2309.03900 2023.
16

---

Published as a conference paper at ICLR 2025
A BACKGROUND ON KERNELS AND MMD
In this section, we provide a brief overview of kernel methods, leading towards the maximum mean discrepancy. For further background, we refer the reader to (Muandet et al., 2017).
Kernels and Feature Maps Suppose X' is a topological space on which we wish to study either functions f X IR or measures μ E (()). A kcommon way consider such objects is by using a feature map
: - H
(17)
into some Hilbert space H. Heuristically, we can consider
functions on X' via linear functionals (t, (.)) : X R where E H. and
measures X' by considering the kernel mean embedding (which we also denote by $),
defined by
4 P(X) - H. ((1)) = Sxx ((z)du(x))
(18)
Given this feature map, we can define positive-definite kernel k X' x X + R defined by k<ar, y) = (.(x), +(y))
(19)
Reproducing Kernel Hilbert Spaces. In fact, we can also go in the other direction and start with a continuous positive definite kernel k X' X - I. and obtain feature map from X' into a Hilbert space of functions. In particular, we define H to be the completion of the linear span of functions (k(z.-) X'
R I E } equipped with the inner product
(k(2, A(y, +)) k(x,, v)-
(20)
By the Arose-Aro theorem (Aronszajn, 1950), H is a Hilbert space with the reproducing kernel property: for any f E HH and x E X, we have
(f.k(z,) ))) f(r).
(21)
Thus, H is reproducing kernel Hilbert space (RKHS), Note that this is Hilbert space of functions, H C C(XRR) Then, we can define a feature map 4 X + H by
$ (xx) = k(z,)) -).
(22)
Universality and Characteristicness We wish to consider feature maps (or kernels) which satisfy additional properties such that they can approximate functions and characterize measures. Let F C C(X' R) be topological vector space, and suppose M is space of measures on X. A feature map & : X + H (with associated kernel k. x X' + R), where H. C F (Simon-Gabriel & Schölkopf, 2018)
universal with respect to F if H is dense in F (we can approximate functions in F using functions in H); and
characteristic with respect to M if the kernel mean embedding in Equation (18) is injective.
Maximum Mean Discrepancy Given a feature map $ + H (with kernel k) characteristic to the space of probability measures P(X)), we can use the Hilbert space norm to define metric on this space of measures. In fact, this is equivalent to the notion of maximum mean discrepancy (MMD) from statistics. In particular, given a function class F C(X we define the MMD with respect to J by
MMD v) = sup (E (E By-r(((())) fEF
(23)
Now, by (Gretton al., 2012, Lemma 4), if we choose F be the unit ball of the RKHS H. (with respect to characteristic kernel k), the MMD with respect to F is exactly the Hilbert space norm,
(M,)(H,,)) (μ) ((v)llx MMD
(24)
where the right hand side is how define MMD, in Equation (11).
17

---

Published as a conference paper at ICLR 2025
B CHARACTERISTIC KERNELS ON Ω.
In this appendix, we provide a detailed discussion of the proof of Theorem 1. We begin by characterizing some of the elements in the RKHS Ha. Lemma 1. The RKHS Hy satisfies
Ha P (g : 52 -2 :(b,))))))))) = -0,,0)=--(000,
(25)
Proof. By Moore-Aronszajn (Aronszajn, 1950), an element of g E Hy is defined by convergent series
g(b,6) = Σ ((bi (b, (b. Σ cli (.),0))
(26)
where f(b, Enm clik((b,, (b. Note that if the coefficient series for given by
Ee la (,b((bb (i), (,) convergent, then the coefficient series for 9 given by
loy (n( 6,), (b,, (.)
Σ (1) 4)) ST
(i) (27)
also convergent, since 4 T.
Universality and Characteristicness. Our first main result concerns the transfer of universal and characteristic properties from ls to kn. First, we define the space of linear gheerr continuous functions on 2ep by
(2r) = (E f(b, E C(Sr) : f(b, E c((,
(28)
where we equip with the norm defined on g = f by
Ilalltin == 1f
(29)
Note that for all 9 E (12) have the property that g(+) 0, and (Cin(Sr))))) Il and (C{10, loo) are isometric Banach spaces.
Theorem 4. Let k [0, [0. 2 + R be a kernel on [0, 7)2 universal to C( TJ3). Then, ky Q x Ω + universal with respect to Cun(S)r)
Proof. Let g E Cha (Irr and suppose = f for some f E C(O0,T Because i is universal, there exists fn H such that fn floo -> 0. Then, by Lemma 1, In Ê- Hgg and furthermore, by the definition of || Ilin in Equation (29), we have l|ga glliin + 0. Thus, kn is universal with respect Cii (l2()
The main result we wish to obtain is characteristicness with respect to measures on Ω. We begin by applying the duality theorem of (Simon-Gabries & Schölkopf, 2018, Theorem 6) which immediately implies characteristicness with respect to the topological dual of Cin which we equip with the weak.* & topology with respect Cun (f)).
Corollary 1. Let k [o, T) x 10, 2 + R be kernel on [0, T) universal to C((O,T]3) Then, kin characteristic with respect to
Our next task to show that our desired measures are contained in this dual space.
Theorem Let q : [0, 2J2 - Ω denote the quotient map, and let s [0, TJ2 - [0, oo) be defined by ((b,)) = for f > 0 and s(b, 0) = 0. Define
Mi) = no E M(Ω) E(S) μ M((O,TT3) ulte 0}) =0, =0.1/ pomp ldp < <xx (30)
then (S) (
18

---

Published as a conference paper at ICLR 2025
Proof. Let qupl E (S). Then, for l. CIn( have
1,8 t. f(b 0)..vv
e f(b, E)dz S 1g
Ig
Edp
(31)
10.7J
where we use ul(e = 0}) = in the first equality. Thus, gale is a bounded (hence continuous) linear functional.
As we wish to use this kernel to study PPMs. we are interested in probability measures on Q. However, the elements the dual only contain measures which are trivial on E (whereas PPMs may have nontrivial mass on *), and thus we first define a different representation of probability measures. In particular, we define
Pin (I2) (v Min( |/| S 1) {v M( v({+)) = 0, |l| 1 C (in (32)
Note that the equality holds since the moment condition in Equation (30) is immediately satisfied since the measures finite with bounded support. With the following two lemmas, show that this coincides with ((2)).
Lemma 2. The space Can (Q) is dense Co( equipped with the uniform topology.
Proof. Let E (9)), and since g is uniformly continuous (since Ω2 is compact) and g()) 0, for every e > O, there exists some l, such that g(b, E) < whenever & < le. Now, define fu. E C((0, by
g(b,e) for e2 f221/ and (n) g(b, (1/m) for e< E</nn- (33) E
fn(
f1/0
Then, define gn (n(b, = - f(b, f), where gn (b, () = g(b, E) whenever l lijn. When < li/n we have
2
Ig. (b, 9(b, 0) (gn(b, Ei/n g(b, S n
(34)
so In. converges uniformly g.
Lemma 3. There exists homeomorphism
+ P()) + (1)
(35)
where P(l)) is equipped with the weak. topology with respect to C(S2) and (a) is equipped with the weak-* * topology with respect to Cnn(())..
Proof. We define vb and its inverse by
(()) = μ m(*)6, and (D)
= v
(1-v(1)))
(36)
where 8_ denotes the Dirac measure on * E Ω. By definition of (12) in Equation (32), this map bijection. It remains to show that μn, + in P())) if and only if ((()) + v(μ) in (in())).
Note that for any f E C(2) and μ E P(L)). we can decompose the integral as
u(f) [ fdw /
(2-(-) fdu + f()))((+)).
(37)
Then, for f E (9)) we have
u(f) 1 (0-(4)
fdu [ 12
fd((())
(38)
since f(+) 0 and μ (1)) Ω (x}. Thus, if pn / in P(I)) then (k())) - (()) in (I)).
Next, suppose that vn in (I). Now, note that ( M(2). and thus are also continuous functionals on C( with respect to the uniform topology. By Lemma 2, we have va(s) v(f) for all f E Co()). However, this also implies it holds for all E C(2), since
19

---

Published as a conference paper at ICLR 2025
obtain functions in ((2)) by adding a constant to function in Co( and v({+)) for all V/ E (Q). This implies that
Sa-11) fd fdu (vn) + Sa-(c)) fdi
(39)
(+)
since v = (v) on Ω (*) for all E Pin (9)). Furthermore, this implies that (Q. v(2). and thus by definition of v in Equation (36), we have & (v,) v ( in P(22).
This result allows us to work wiih P(2) and (2) interchangeably. In particular, note that (n(() = 0 Ho. This implies that for any μ E P(22), we have
Pn(u) (n((())).
(40)
Proof of Theorem 1. By Corollary kn characteristic with respect to Min(St), and Pin (S2 C Min (S2) by Theorem 5. Next, by Lemma 3, we can apply the identity Equation (40) to conclude that kg is characteristic with respect to P(2).
C METRIZING WEAK TOPOLOGY
In this appendix, we provide a detailed discussion of the proof of Theorem 2.
Maximum Mean Discrepancy (MMD). Because kn is a characteristic kernel on Q the kernel mean embedding, defined (by an abuse of notation) on q4 E Min(S)) by
Φ Mun (22 - Ha,
(vv) == R (+)
(41)
is injective. This induces a metric on Min (S2), called the maximum mean discrepancy (MMD),
MMD == (v11) (12)
(42)
Our next result shows that the MMD metrizes the weak-* topology Pin (12), where can directly apply (Sriperumbudur, 2016, Theorem 3.2). See also (Simon-Gabriel et al., 2023) for related results.
Theorem 6. The MMD metric metrizes the weak topology on P())). In other words, given measures Pa, V E P(2),, have MMD,,Vun.)) v) - 0 if and only if
(r)()) v(g) +
(43)
for all g E C(.).
Proof. This follows from (Sriperumbudur, 2016, Theorem 3.2) Q is compact Polish space and kn is a continuous bounded kernel.
Corollary 2. The Wasserstein metric W, and the MMD metric MMD, induce the same topology on P(2).
Proof. This a direct consequence the above, since the p- -assserrseiinn distance also metrizes the weak topology on P(2)) (Villani, 2009, Theorem 6.9).
D PPM REGULARIZER HAS CONTINUOUS GRADIENTS
We say that function f _ + Td is C² function if all first derivatives of f are continuous. We will begin with a more general statement which will immediately imply Theorem 3. Fix μ E NR to be measure, and let he R.M I L. be mapping from R to the latent space RR. Suppose that the mapping Ag is parametrized by E IR and define H. : R x N + RL by H x) ho(r). We aim to show that R Hn, defined by
560) (holl()))
(44)
20

---

Published as a conference paper at ICLR 2025
smooth function. In particular, the pipeline for computing this feature map is
PT(IRN) (hol,, P.(RR) (-)0(2++2)
Pa(( (2(((1)22+3))
(PHle),, P(Ω) 2 H.
(45)
Let F - (( be the map from the parameter space to the product measure in latent space. We assume that the resulting product measure has Ct density, which is if H is C², and has a C' density. Then, F has the form
F())
(46)
where :RP x (R2)22+22
a C" function.
Theorem 7. Let μ E P(RN) be a probability measure with a C density, suppose H R. RR R' is a C' fimction. Then 5) Rº + Ho is C' function (where derivatives are Fréchet derivatives).
Proof. Expanding out the definition of 8) we have
Pa (PPPPM()
(47)
Dn o (z) (B4)2212
(48)
$ o 2H f x)dd,
(49)
(Roja4t2
where the definition of the pushforward in the second line, and the definition of the density of F in Equation (46). Now, this integral is Bochner integral since it is valued in Hilbert space, and we can still differentiate under the integral (by Hille's theorem; see (Dieudonne, 1969, Paragraph 8.1 8.11.2)). Then, we have
ae; En (PPM,,())(()) = S(ccypue Φ O (x) 0f(0,x) x) dx,
(50)
00,
which is continuous since f is C
Proof of Theorem 3. By direct application of Theorem 7, both
+
2g o PPMg o de(u) and (0, w) + Dn o PPM, dg o gw(
(51)
C functions. Then, since the norm is continuously differentiable away from the origin, we obtain the desired result.
E SHAPE MATCHING
E. IMPLEMENTATION DETAILS FOR SHAPE MATCHING
Shape Matching Experiment. The task is to match the "shape" of two point clouds by optimizing a loss function of the form L + T in ambient space. Throughout the experiment, we use gradient descent with momentum the optimization algorithm. The value of the momentum parameter is 0.9 and the step size is 0.05. Two simple reference shapes in R22 are chosen for visualization purposes. They are unit circle (left), and the union of two intersecting unit circles (right), shown in Figure 2. A noisy point cloud with 512 points is first initialized with a normally distributed at the origin with a standard deviation of 0.3, we call training shape. The reference shapes are also sampled as a point cloud wiht 512 points. The reference shape is fixed and the training shape changes towards the reference, guided by some loss function. We test for the effectiveness of four loss function, they are Cramer, MMD, Cramer PPM-Reg and MMD PPM-Reg Reg. The MMD metric using an RBF kernel (with width a = 0.1). Throughout the experiment, the hyperparameter PPM-Reg is fixed as X = Ay 1, A₁ 6000, = 0.1 and 2000. For Cramer PPM-Reg, the weight of the cramer loss 1.6. For MMD+ + PPM- -egg the weight of the MMD loss In Figure 4, we show the the shapes after 16000 training steps. Even at this stage, we observe that there several "leftover" points. This partially due to the choice of the underlying loss functions, and we observe that in the trained shapes using PPM-Reg largely capture the topological features of the reference shape.
(b), (d) and (h),
21

---

Published as a conference paper at ICLR 2025
Plan
Hoir
20
(b)
(c)
(d)
ioleccaad
-3
43
43
43
30 -15 12 21 20
31 31
(h)
Figure 4: Plots of the shape matching experiment at convergence after 16000 steps. (a) and (e) use only the Cramer loss. (b) and (f) use Cramer + PPM-Reg. (c) and (g) only the MMD loss. (d) and (h) use MMD + PPM-Reg.
16
MWD PPM-Reg
22
MMC FPM FPM-Reg
14-
MMD PPN Reg. 0.004
20
MMC PPM-Reg C. 0.04
MMD PPN
MMC PPM.Rey c2 0.12
12-
MMD
MND
MMD. C= 0.04
MMC, C2 0.04
MMD, c; 0.32
16
MMC, Co 0.12
12
8-
2000
4000
6000
8000
10000
12000
14000
16000
4000
6000
8000
10030
12000
14000
16000
Step
Step
(a) Unit circle
(b) Union of two intersecting unit circles
Figure 5: Illustrative example of the PPM-Reg in shape matching using MMD with increasing handicap contain. Plotting 2-Wassersteinn distance upto I-dimensional persistent homology between the reference shape and training shape (PDais)) over optimization steps. The reference shape of (a) unit circle and (b) union of two intersecting unit circles. сş indicate the strength of the handicap (detail provided in Appendix E.2). Showing that the ability to matching topological features as the handicap contains increase.
Computational Comparison. Results are computed on Nvidia Geforce RTX 3060 with Intel Core 17-10700.
E.2 IMPERFECT CONVERGENCE
This section demonstrates the efficacy of PPM-Reg when the primary loss function is optimized imperfectly. As discussed in Section 1, most GAN training algorithms converge to local saddle point. simpler latent-space matching tasks, is often optimized alongside with other tasks. To summarize, converging imperfect L value is common occurrence in machine learning.
22

---

Published as a conference paper at ICLR 2025
To restrict the solution of the shape matching problem, impose penalty term that prevents the centroid of the reference shapes and the training shape from being smaller than a user-defined value cg. The penalty term fp is given by
Ap In(1 + ( (8css-lectain Graf la)), Jp B
(52)
where Cerain and Cref refers to the centroid of the training shape and reference shapes respectively. The term Ap is the penalty strength, 3 is a tuning parameter. For the MMD case, = C + fp- For the MMD + PPM-Reg case, V = XT XT fr.
Implementation Details. We largely follow the setup on Section 6.2 and Appendix E.I, but make few minor changes. We reduced the step size to 0.01 and performed 16,000 gradient steps. The hyperparameter PPM-Reg fixed as À = 1, A = 0.3, Å = 6000, o = 0.1 and s 2000, For the penalty function fp. set B 80. The Ap value remains the same when vary сæ and compare against adding IPM-Reg. The Ap value is determined such that |Crrasn Crefl12 converges to co when ca 0.04.
Evaluation Metrics. We consider the case where cs E (0,0.04, 0.12). For reference shape normalized to [-1, 1]. having cs 0.12 is very small. We compare the main loss with the addition of PPM-Reg and track the 2- 2-Wasserstein distance up-to the 1-dimensional persistence diagrams along gradient step.
Result. Figure shows the change in PD distance as сj increases with the circle Figure 5(a) and union of two circles Figure 5(b). Compared with only using MMD main loss, adding PPM-Reg consistently converges to a smaller PD distance regardless of the G value. In contrast, when only using MMD, the PD distance increases es value increases. Figure 5 illustrates the benefit of explicitly comparing the difference between topological features with PPM-Reg compared with implicitly considering topological features with only MMD when the optimization problem may not converge to near zero.
F UNCONDITIONAL IMAGE GENERATION
F.1
IMPLEMENTATION DETAILS FOR UNCONDITIONAL IMAGE GENERATION
This section fills in the details of the unconditional image generation experiment in Section 6.3.
0 = 0.05
0.5
1.0
CMMD
FD
WDattt
CMMD
FD
WD
CMMD
FDpine22
WD
1.0
0.74
945.27
0.6330
0.68
846.33
0.6228
0.56
780.68
0.6080
5.0
0.73
928.60
0.6310
0.72
884.40
0,6282
0.74
894.77
0.6308
10.0
0.74
880.68
0.6332
0.77
922.58
0.6379
0.71
923.50
0.6274
Table 4: Ablation study on AnimeFace dataset.
0.05
0.5
1.0
X
CMMD
FD
WD
CMMD
FDpinow2
WDateet
CMMD
FDpinec2
WDunnt
1.0
0.87
737.09
0.6945
0.65
704.74
0.6744
0.81
745.39
0.6886
5.0
0.72
733.66
0.6815
0.58
700.73
0.6666
0.68
695.36
0.6768
10.0
0.61
690.01
0.6691
0.69
683.35
0.6781
0.71
719.39
0.6795
Table 5: Ablation study on CelebA dataset.
23

---

Published as a conference paper at ICLR 2025
Implementation Details. The network architectures used in the unconditional image generation in Section 6.3 are shown in Figure 8. As illustrated in Figure 8, Iw is a ResNet based CNN that takes 228-dimensional noise vector input. dg is CNN that outputs 128-dimensional latent vector.
We compare the basic Cramer (Bellemare et al., 2017) value function V = L. with the use of PPM- Reg V = L T. For the addition of PPM-Reg case, fix A 0.001, X1 = 0.6, and
1024.
= (1.0, 5.0, 10.0) and o = {0.05, 0.1, 0.5) are the tuning parameter. In both cases, the standard Adam optimizer with learning rate 1 x 10 is used to train the network. For Iw. 31 0.0 and 82 = 13=99, 99. For da. 31 0.5 and B3 0.99. The batch size is 192. For the CelebA dataset, the GAN training run for 5440 epochs. For AnimeFace data set, the GAN training run for 7000 epochs.
During training, we compute CMMD for every 160 epochs. We report the CMMD (Jayasumana al., 2024) and WDaassnt and Dinow2 (Stein et al., 2024) with the smallest CMMD value across training epochs. Those quantitative metrics are computed by sampling 10K images from the data set and generating 10k images from the network.
Ablation Study. There are two primary parameters in our topological regularizer. The parameter À controls the strength of the regularizer, while a controls the width of the RBF kernel in defining the MMD for PPMs. We provide an ablation study to show how the evaluation metrics change as we vary these parameters in Table 5 for AnimeFace and Table 4 for CelebA.
F.2 FURTHER UNCONDITIONAL. IMAGE GENERATION EXPERIMENT
This section introduces supplementary unconditional image generation experiments with in- crease in image resolution as well as additional datasets in conjunction with appropriate network architectures.
CelebA
LSUN Kitchen
CMMD
FD
WDissent
CMMD
FD
WD
Cramer (Bellemare et al., 2017)
0.52
902,09
0.7335
1.56
1592.06
0.7690
Cramer + PPM- Reg
0.46
826.04
0.7296
1.31
1381.22
0.7502
Table 6: Quantitative evaluation on 64 x 64 image generation, values are reported at the epoch with the smallest CMMD.
Implementation Details. Similar to Section 6.3, 9w is a ResNet based CNN that takes 128- dimensional noise vector as input. de is a CNN that outputs a 128-dimensional latent vector. The major difference is instead of just generating 32 x 32 images as in Section 6.3, we test our PPM-Reg with higher resolution. Specifically, we consider the 64 x 64 image generation task. To accommodate the increase in modeling complexity, we introduce a new network architecture shown in Figure
To avoid confusion, we will write out our implementation details. We are comparing Cramer (Belle- mare et al., 2017) and the addition of PPM-Reg. For the addition of PPM- case, we only fix A = 0.001 and s = 1024. The value of X1 changes as the training epoch runs. Specifically, we adapt cosine annealing (Loshchilov Hutter, 2016) on Å1. the A value at epoch term X given by
Apmin + 1. (Ammoz - (1 + cos( (.l. T)), if S fend Ammin
A/
(53)
if > fend:
where Xmrnn Amar and tend are user-defined variable. Ammnn and Xmrar are the range of the X fend is the ending epoch for the cosine annealing. We fix Agnin = 0.1, a = 0.5 for both dataset. For CelebA, A 1, X = 1 and lend = 1920. For LSUN Kitchen, X 10, Xmaar = 0.8, and hend 260. The standard Adam optimizer with learning rate 1 x 10-4 is used to train the network. For gw- 31 = 0.0 and Bg 99. For de. 31 0.5 and 82 0.99. The batch size is 192. For the CelebA dataset, the GAN training is run for 2560 epochs, while for the LSUN Kitchen data set, the GAN training is run for 300 epochs.
Dataset and Evaluation Metrics. We add a new dataset LSUN Kitchen (Yu et al., 2015) and also use CelebA (Liu et al., 2015) higher resolution. Images are centered and resized to 64 x 64.
24

---

Published as a conference paper at ICLR 2025
0.00
Chenar
3000
Cranar Feld
Damer
0.0
Conterr FIrE Ro3
3000
0750
100
0.00
900
0.55
000
0750
326
1200
2043
2533
000
9.0
1,00
1582
200
2900
0725
900
1558
1540
2002
2500
(b)
(C)
Ouner
Dammar- 70M-00
Conner 990 Arg
Daner
082
Osne
C400
1700
0.00
6.79
3000
0:10
2500
0:77
1230
25
120
125
150
115
200
25
15
100
115
300
153 375
200
2007
Epach
Eaash
(d)
(c)
(f)
Figure 6: CMMD (a,e), DDnnovv2 (b,f) and WD (c.g) versus training epochs for the CelebA (a-c) and LSUN Kitchen (d-f) dataset for the 64 x 64 image generation. 10K samples are randomly generated to compute the distances, and moving averages with window of 5 are used to smooth the values. For CelebA, distances are recorded every 160 epochs. For LSUN Kitchen, distances are recorded every 20 epochs.
During training, we compute CMMD for every 160 epochs for CelebA and 20 epochs for LSUN Kitchen. We report the CMMD (Jayasumana al., 2024) and WDitte and FD (Stein et al., 2024) with the smallest CMMD value across training epochs. For justification for using the chosen quantitative evaluation metrics, readers can refer to Section 6.3. Those quantitative metrics are computed by sampling 10K images from the data set and generating 10k images from the network.
Result. Figure 6 tracks the three quantitative metrics during training and Table 6 reports the three metries with the smallest CMMD. While the FD metric has comparable values in Figure 6(b), at the point of the smallest CMMD, Cramer + PPM-Reg has a smaller FD for CelebA, shown in Table 6. Our guantitative results in Table 6 reinforce the fact that using PPM-Reg improves the generative ability of GANs. Compared to CelebA dataset, using PPM- Reg in the LSUN Kitchen dataset results a more significant improvement. We conjecture that since LSUN Kitchen is a much larger dataset (2,212,27) training samples) that contains diverse images (i.e. different color tones, layouts), its underlying lower dimensional submanifold has more complex topological features compared with CelebA. The significant improvement gives evidence that PPM-Reg useful in discovering more complex topological structures in latent space.
G SEMI-SUPERVISED LEARNING
G.1 IMPLEMENTATION DETAILS FOR SEMI-SUPERVISED LEARNING
This section fills in the details of the semi-supervised learning experiment Section 6.4.
Network Architecture and Implementation Details. The network architectures used in the semi- supervised learning experiment in Section 6.4 are shown in Figure 10. The input noise vector of gw has dimension of 64. We set the dimension of the output latent vector of dg as 64.
Specifically, Iw and de are trained with the GAN framework for 4,000 epochs using the standard Adam optimizer with learning rate 1 x 10-4 For 9w, we set 3 0.0 and B3 = 99; for dg. we set 31 = 0.5 and B2 0.99. The batch size is 192. For Cramer + PPM-Reg. fix A 1, A1 90
25

---

Published as a conference paper at ICLR 2025
6
3
7
5
9 7
4
/
3
3
/
2
9
6
4
7
2
9
0
8
776
0
5
9
0 3 4
0 4
9
9
/
4
6
3
3
6
/
9
0
9
9
1
33 000
8
7
73
1
6
3
0
0
9
9
3
6
9
0 9 8
7
1
4
8
7
3
9
9 3
4
5
9
9 2
5
9
7
9
2
5 5
0
0
2
4
8
0
/ 2
8
0
5
7
8
1
7
(a)
(b)
Generated images from SSL experiment from trained Iw for MNIST using (a) Cramer Distance and (b) PPM-Reg.
Figure 7:
and = 1024. A {0.025, 0.05, 0.1) and o = {0.05, 0.1, J.5} are the tuning parameter. After the GAN framework completes the training of fw and dg, the weights dø are frozen. Using de as feature extraction, the output of de is fed into the classifier. The classifier C. is train with the standard Adam optimizer with learning rate x 10 where 31 = 0.1 and 3g = 0.99 for 1000 epochs.
"Baseline" connects de and C, and trains classifier without first pertaining dg with GANs. The standard Adam optimizer with learning rate 1 x 10 where B1 0.1 and B2 0.99 the "Baseline" " network for 4,000 epochs.
train
F MNIST (200 labels)
Fashion-MNIST (400 labels)
0.05
== 0.1
a = 0.5
0.05
a= 0.1
0 == 0.5
0.1
75.39 + 1.12
74.96
0.96
75.24 + 1.20
75.39 + 1.12
74.96 = 0.96
75.24 1.20
0.05
76.35
+ 1.09
76.25
+ 1.30
76.81 + 0.88
76.35 = 1.09
76.25 = 1.30
76.81
+ 0.88
0.025
76.42
0.85
76.77
1.29
76.84 + 1.23
76.42 = 0.85
76.77 1.29
76.84 1.23
Table 7: Ablation study on Fashion-MNIS, dataset with 200/400 labels. Test-set classification accuracy (%) is shown averaged over 10 runs.
Kuzushiji-MNIST (200 labels)
Kuzushij-MNIST (400 labels)
a 0.05
0 0.1
a = 0.5
0.05
0.1
O 0.5
0.1
73.70
2.58
70.84
2.56
70.16 + 1.96
78.35 + 1.81
76.18
1.87
76.67 1.31
0.05
74.35
1 2.45
73.65
1.70
73.97
+ 1.24
78.99 + 1.41
79.92 - 1.88
78.77
1.48
0.025
74.47
+ 1.65
75.78 1 1.99
75.41 + 2.83
79.40 + 1.65
79.33 = 1.69
80.04 + 1.37
Table Ablation study KuZUShijiMMNIST with 200/400 labels. Test-set classification accuracy (%) is shown averaged over 10 runs.
We show how the classification accuracy varies with respect to the topological regularization strength À and the RBF width in Table 7 for Fashion-MNIST, Table for Kuzushiji- MNIST, and Table 9 for MNIST.
Ablation Study.
G.2 FURTHER SEMI-SUPERVISED LEARNING EXPERIMENT
This section introduces supplementary semi-supervised learning experiments with additional datasets in conjunction with appropriate network architectures.
Network Architecture and Implementation Details. Similar to the setup in Section 6.4, 9w is a deconvolutional network with 64 dimension noise vector as input. de is CNN with 64 dimension latent vector as output. The additional SVHM dataset is more intricate than the MNIST variants evaluated in Section 6.4, is color image dataset with higher resolution. To tackle the increase in
26

---

Published as a conference paper at ICLR 2025
MNIST (200 labels)
MNIST (400 labels)
0.05
0.1
0.5
0.05
0.1
0.5
0.1
96.34 + 0.29
96.32
+ 0.20
96.18 0.19 0.19
97.19 = 0.28
97.06 = 0.28
97.07 + 0.26
0.05
96.61 0.37
96.62
0.39
96.22
0.59
97.29
= 0.20
97.33
0.21
97.16 0.25
0.025
96. + 0.42
95.97
0.37
95.91 = 0.55
97.04 = 0.27
96.92 = 0.20
97.2 = 0.25
Table 9: Ablation study on MNIST with 200/400 labels. Test-set classification accuracy (%) is shown averaged over 10 runs.
input dimension and the complexity of modeling the dataset, we introduce a new network architecture shown in Figure 11.
The training setup largely follows the previous experiment; the major difference is the number of training epochs. Specifically, 9w and dæ are trained with the GAN framework for 1,200 epochs. The standard Adam optimizer with learning rate 1 10 is used to train the network. For gw. 31 0.0 and B2 99. For de, 81 = 0.5 and B2 = 0.99. The batch size is 192. For Cramer + PPM-Reg, we fix Au 1, A1 90 and s 1024. À {0.025, 0.05, 0.1} and o {0.05, 0.1, 0.5) are the tuning parameter. After the GAN framework completes the training of Iw and dg. the weights of de are frozen. Using de as feature extraction, the output of de fed into the classifier. The classifier c, is trained with the standard Adam optimizer with learning rate x 10- where 3 0.1 and B3 0.99 for 1000 epochs. The standard Adam optimizer with learning rate 1 x 10 - where 31 = 0.1 and (3 = 0.99 are use to train the "Baseline" network for 4,000 epochs.
Dataset and Evaluation Metries. We compare the SSL performance with the dataset SVHN. In this experiment, 400 and 600 labels are randomly sampled from the data set. Because of the random nature involved in selecting few labels, we conducted the experiments ten times and provided the statistics of the highest test-set accuracy achieved.
SVHN
Number of labels
400
600
Baseline
44.68 + 2.44
53.68
4.15
Cramer
38.12 1 1.75
43.23
1.14
Cramer PPM-Reg
57.20 + 1.51
61.39 1 0.66
Table 10: Test-set classification accuracy (%) on SVHN with 400 and 600 labeled examples. The average and the error bar are computed over 10 runs.
SVHN (400 labels)
SVHN (600 labels)
0.05
== 0.1
0.5
0.05
0.1
0.5
0.1
55.66 + 1.78
57.20
+ 1.51
54.84 + 1.21
59.48 + 1.26
61.39 + 0.66
59.13 + 0.95
0.05
47.56
+ 1.67
56.24
+ 1.25
52.41
+ 1.49
50.73 + 1.06
60.08
0.84
60.91 1.19
0.025
55.96
+ 1.03
56.96
1.92
55.35
+ 1.06
59.13 + 0.95
56.45 + 1.41
58.97 1.00
Table 11: Ablation study on SVHN with 400/600 labels. Test-set classification accuracy (%) is shown averaged over 10 runs.
Result. Table 10 shows the test classification accuracy. An ablation study provided in Table 11. SVHN contains 72,657 training samples and 400 and 600 labels constitute only 0.55% and 0.82% of the original datasets, respectively. The results follow the same characteristic Section 6.4, only using Cramer does not improve the classification accuracy in SSL. In contrast, the use of PPM-Reg results in a notable improvement in classification accuracy. Specifically, using PPM-Reg with 400 labels yields a 12.52% increase in accuracy compared to the Baseline. The consistent outcome with more complex datasets and network architecture reinforces our claim that PPM-Reg can help learn a more informative latent encoding thereby improving SSL performance.
27

---

Published as a conference paper at ICLR 2025
H NETWORK ARCHITECTURE
apec
14-120
inpul-traon depth.
11..33233
caper
Sequantial
Aosho
orpee
Agul
[I. 2000
139
Conrd depthut
Inpat:
11 XD
(L0% 4.4)
leskeRfelJJ depth2
Siqueriial
oupest
Restbbelblack
SpaspliCor
Supartal
Concdd deptt
Impa:
euppon:
Nt.
+lt
LeakyfieL depik
ingin
[1. (1.322
Beakk
ostpue
32
Legall
Convad depthz
inpin
(1 AL.
Iiper
EBG de to.
Aepiul
LeakyReLL depih
(1 (1,0444)
ontpot
Seperaaal
Courad deptiiz
inpuh
ald
cutput
2.2)
Inpal
LeakyRilL depth22
impad: oxtguti
(1.
121
Bdil draki
Agm:
Imput (1. 12k 2.2)
deptliz
outpet
Igga:
(164 12
Jukk
[LA.R
LealyRell depth22
impet:
1.1)
Spradt
Epst
8.1.8.01
sepal
111:0:00
view deptilel
isput:
oxtpot:
230)
edp::
(13.32
Lincar deyihi
inpad:
ontgat:
120
(13.3233
uudpas- texeee-eerr
depela
120
(b)
Figure 8: Network architecture of generator (a), discriminator (b) for 32 x 32 unconditional image generation.
28

---

Published as a conference paper at ICLR 2025
Sptieeccx
Input temor depth
Agei:
Sequential
regui
Ingeet
(1.3664
depth22
outpet
0.332.32
Siauo
amm
LeakyRetL depth.2
inpuat
output
(t. 32, $2,32.33
Sequesial
ResidnalBieck
Convid depthi2
Input:
(1,32,32 32
ApuneteCCar
0-124
11.12
output
(1.64, 16, 16)
11.12
LeakyRet.t depth
Input:
(1. 66, 16,10)
outgut
(1.
Spenes Coeey
Iest
D.
engutt
EL DK
enigatt
Convid
Irput
depth
outpat:
Book
Leak yRELL depth:
Ingrat:
128
output
128, 8,8)
EUER IN IO
engust
EL UULR SO
Conviul depth
Impot:
(1.128 &
output:
(1.256
RacidualBoock
Sperss
ELEROS
LeakyRett depth:2
Inprst
254, 4.4)
oegust
enteeti
output
fgfllc
Lavos
Convad depti
impott
outpet
(1. (1.512
Eqpe
Argin
outer
LeakyRetii
Inprst:
512 2.20
output
(1. 2.22
apps
MaxPoolZd depth.2
Inprut
(1. 512 2.2)
outpat
(1. 512, 1.1)
sumee
view depth:3
Ingrat:
1
autput:
opti
Ingrat:
512
Iepal
depthal
output:
128)
output tenoor
0.300.00
depth:
(1.128)
(a)
(b)
Figure 9: Network architecture of generator (a), discriminator (b) for 64 x 64 unconditional image generation.
29

---

Published as a conference paper at ICLR 2025
inpal Searer depito
(1 1.3,200)
Inpet exppet
0.123
depthi
Irpas
10
Inpu lepul Ageie
1.:00
depitic
eetput
32, 26,30
Impat
22.24:2
deptbs
cutpst:
52 20.28
daep carpol
[1 1.:
[1 col
Irpet: setpu:
12 20.00)
Arytts
Aignt
Imput-tmmor depthin
(4)
(1.32 26, 30
depitcl
oupett
(1.32 3t, 301
Lincar depth
Inpich
(1,
Ixpet
[1.3 10:30
antpuz:
Hipe:
depth3
cupest
(L 44 (.8)
ogtin
Impost
[1.
Learyfetl
Inpun
(1 44, 9
deptiic
Contrrppera
setpet
(I. 44,
11.4 9.91
(LSLS
Linear depth
input:
oegee:
curget
(L4E6.0)
output:
[1. 512)
1:441.8
Leakytitt
11.94..
drpit
(1.44.6.
LeakyBot22
deptiit
outpat:
[t. (0.50
disey
impptt
[1.30
ArgPoates
culpt:
B.44.
isput:
(1.322)
depth
apu
[1:2 i..
Appili
fattes
Impa: exput
(164...9
0.25%)
impat:
[t. (00
CarThrrppnn Appil
34.00
deptittt
outpun
[1. $1.0
Lince deptt
0.55%)
eutput:
lipa: uaqur
1. 24.00
Appic
Linsar degth
$1.)
Lealyfuu
octpot
(I. Ir)
degits
Cuprrhapercal Arpit
log vollmau depth
Ingun
(1.19)
apand
Lok.0
Lance degtts
outgut
sedpen
temant dryito
(144
cutput-tenace deptlee
10)
(a)
(b)
Figure 10: Network architecture of generator (a), discriminator (b) and classifiacter (c) for semi- supervised learning for MNIST variants.
30

---

Published as a conference paper at ICLR 2025
Aeunce Aeptta
Canval degiii
Inpat:
3.3230
exput
LeakyTutt deythi
Liber
celpal
Concat
ippt: ouput
(4.642420
ligut
(1.20)
appii
apet suppe:
0.44 24,
dquit
Arpisi
ispur depthu
Legut:
20,30)
deptto
serpat
Linoor depths
imput:
Cupp deplist
repet
cutput
iqpe:
11 4.4
aghht
LeakyBuct
LeakyRetii deprlit
Inguzt
Coo Tumparat
output
(1. (0.0000
0. 1.
Lipa
Inpet orlpet:
(I. 1ax
Ixput
(I. (820
depitil
128,
cupput:
(1. 3820
Leaky/i appitl
Iqpa::
Indee
seteut
InakyRottt
20240
deptis
eniputt
siepy
lqpan inter:
Anghoatza depths
apat eatpet
125
Ispon
Epu
0.44 is
depticl
cuppst:
(1. (.282)
dryiti
Baticn Cepiiic
Confhempeccl
Leakeytect
iaput:
Di (02200))
rddee:
14.4
ostputt
3024)
Ingin euppst
20:0 (L.120
lepa:
1.
dgiii
Linoor deythi
1830
lgpue ettpu:
[1. 139
cutpes
deptis
CarThuppeca
Ingish
dgiiti
oeteet
appat:
deythct
octput:
(1.10 200
culpel imeer Appit
depibo
(L.40
cutpol txxsot
dept2:0
(1.
(a)
(b)
(c)
Figure 11: Network architecture of generator (a), discriminator (b) and classifiacter (c) for semi- supervised learning for SVHN dataset.
31


Reviews:
[{"review_id": "A6fX9sV1CW", "reviewer": "Reviewer_BKqM", "summary": "The paper proposes a regularizer to be used in Generative Adversarial Networks (GANs) in addition to the “standard” distance between distributions (e.g. Wasserstein distance). The regularizer consists in computing the probability of persistent homology diagrams through Principal Persistent Measures (PPM) on both real and generated data and comparing them through MMD. The paper provides a few theorems proving the smoothness of the MMD applied to compare PPM under mild conditions and experimentally shows the benefits of this regularizer on both generative metrics and semi-supervised classification tasks.", "questions": "I would like the authors to clarify the points I have highlighted on the weaknesses, especially the contribution and the relation with other GAN regularizers.\n\nMoreover, the method proposes the use of MMD rather than WD to compare the PPMs. Is there any theoretical and/or practical reason? Would still be possible to use WD and experimentally compare the performance of WD and MMD?\n\nMinor comments:\n- The citation format is weird and interferes with the reading.\n- At row 66, what would be the classic PH pipeline?\n- In row 155, you provide 5 introductory references to PH, these are far too many, and this does not really help the reader.\n- Row 383: “consider provide”\n- Table 1: since you computed averages over 10 runs, it would make sense to report also the standard deviation.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper pairs the proposed regularizer with a solid theoretical analysis of the smoothness of the resulting loss function, which justifies its use as an additional term to the (W)GAN loss.\n\nExperimental results show that the proposed regularizer provides a significant advantage in training GANs, especially for self-supervised tasks. \n\nAlthough the paper is theoretically dense, it can still be followed by non-experts in the PH fields. Nevertheless, it would be helpful to remark the purpose of each theorem, or sequence of theorems, before introducing them.", "weaknesses": "The main weakness of the paper is the introduction and motivation of the problem and the relation of the proposed method to the current literature. From the introduction (and also the title), it isn’t really clear what problem the paper is going to tackle. While the introduction talks about general regularizers for latent space representations, most of the methodological development and the experimental section tackle specifically the problem of regularizing GANs. I would be clearer about this from the beginning of the paper or show applications beyond GANs (beyond the shape-matching toy example).\n \nI would also better frame the work in the context of GANs, and regularization strategies for GANs. A few methods are mentioned in the literature, but none is discussed in depth. For instance, PHom-GeM seems to share objectives and strategies similar to those of the proposed work. It would be ideal to compare also with some of these methods.\n\nAlso, it is not clear what theorems are novel, and which ones are just reported or adapted from results in the existing literature. Maybe they are all novel, but specifying the current state of the literature and which gaps need to be filled (from a theoretical point of view) would help to better understand the contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q2eoaulf3U", "reviewer": "Reviewer_MDvZ", "summary": "This paper proposes to introduce a topological regularization term to the objective function used in the context of GAN. \n\nAs standard topological descriptors (known as persistence diagrams) are typically too expensive to compute, they propose to rely on the (recently introduced) notion of \"principal persistence measure\". Losely speaking, the principal persistence measure is a distribution in the space of persistence diagrams, but its elements have (at most) one point, and can thus be (roughly speaking) identified with distribution on the open half-plane $\\{b < d,\\ (b,d) \\in R^2\\}$ augmented with an additionnal virtual point $\\star$. \n\nThe work then introduce a notion of MMD distance between PPM. The MMD has several benefits: it is fairly easy to compute and induce smooth gradients. \n\nThe authors showcase their method in a series of experiments.", "questions": "See question 1. and 2. in the Weaknesses section. \n\nAdditionnally : \n- I did not have time to read the appendix in details so I may be wrong, but regarding the Remark 1: in theory, PDs may have infinite total mass (here in the work, everything simplify as one consider PDs with at most 1 point), in which case $n,m, N = +\\infty$. And even if one restrict to finite diagrams, without putting a uniform bound on their cardinalitty, one may still have $n,m \\to \\infty$ and (I guess) peculiar behavior of the metric. Does the remark implicitely consider a space of diagrams with uniformly bounded mass/number of points? From a quick glance, $M_{lin}$ only incorporate a standard integrability constraint but nothing about the cardinality of the diagrams. \n\n- The animations show that optimization let few \"leftovers\", can you comment on this? Is it due to a mix of (gaussian-kernel based) MMD which indeed tends to let leftovers behind + the subsampling in the PPM which needs to be \"lucky\" to put some gradient on this points? Would using the Energy-distance MMD $k(x,y) = - |x-y|$ improve on this particular aspect? (note: gradient may no longer be smooth using this MMD)", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Good introduction / preliminaries (section 1 and 2). \n- The work mixes results from different ideas (mix of MMD, persistence measures, etc.), and may be useful beyond the scope covered by this paper (improving GAN training). It showcases the use of PPM which are interesting objects in their own, introduce novel possibly useful metric between these topological descriptors, etc. \n- interesting experiments, mixing pedagogical Proof-of-Concept and more advanced experiments. \n- Nice animation in the supplementary material!", "weaknesses": "1. While well written, the introduction and section 2 somewhat fail to motivate the need to account for the geometry/topology when comparing distributions in the latent space. This is just given as a fact (line 142-143), but basically, why should one care about such information? Is there some situations in which this clearly lacks in GAN models? I understand that this needs is empirically confirmed in the experiment sections, but is there a good _a priori_ reason? \n\n2. Somewhat in the same vein: Wasserstein-like distances between topological descriptors are discarded because \"gradients are not smooth\", but is it a real problem in practice? I would expect stochasting GD to not actually bother which such details. \n\n3. One may argue that GAN tend to be outdated in comparison to more modern generative models (DDPM, etc.). I nonetheless believe that this does not significantly impede the contributions of the work, whose interest is not limited to GAN only. \n\n### Minor remarks (not real weaknesses)\n\n- I believe that the weighting by a factor $\\ell$ in the \"induced kernel\" from $R^2$ to $\\Omega$ resemble the PWK kernel of Kusano et al., and more generally the kind of weighting needed in linear representation of persistence diagrams (see Divol&Lacombe, along with Obayashi et al., \"Persistence diagrams with linear machine learning models\", and related works).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MF7SPNixmx", "reviewer": "Reviewer_iUNk", "summary": "One of the prohibitive factors of Topological Data Analysis are computational\nconstraints imposed on the calculation of Persistent Homology (PH). While\nhighly optimized CPU-GPU algorithms exist, the computation already become \nunfeasible for small point clouds in low dimensional settings. A remedy is\nproposed through the use of a GPU implementation of the Principal Persistence\nMeasure (PPM) (Gómez & Mémoli 2024). This addresses both difficulties related\nto smoothing as well as scalability. Given the scalable backbone the authors\nthen provide a theoretical framework to turn the PPM into a regularizer that\ncan be used in generative tasks where the topology of the dataset is of\nimportance. They provide a strong theoretical foundation for their regularizer\nand show that the gradients are continuous.\n\nThe authors provide two experiments as empirical evidence of the efficacy of\ntheir method. First is an optimization task where a point cloud is\nbackpropagated to match a reference shape. Faster convergence to the final\nsolution is shown, thereby showing topological regularization to be helpful in\nconvergence. In a final suite of experiments the authors show that the\nregularization significantly increases the generative quality of the trained\nmodels and this is demonstrated in both unsupervised and semi-supervised\nlearning tasks. The computational advantages are shown to have a significant\nincrease in training time over the traditional PH calculation and not suffer from the\nexponential cost.", "questions": "- A question that would also be interesting to consider is how the method\n    scales with dimension, based on the paper it looks like it might scale\n    exponentially in the dimension. \n- What causes loss not to converge to zero in the point cloud optimization\n    task? \n\nRemark:\nDuring the review, some typo's were found. Please give the paper another read\nand correct them.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "In general it is great to see a push towards scalable topological methods and\nthis entails the core strengths of the paper, as it an impactful and relevant\nproblem to work on. Often in TDA, the focal point lies in theoretical\njustification of the methods with only limited emperical evidence, mostly on\nsmall datasets. Lack of scalability of PH being the main driver of this\nphenomenon and therefore this paper is of importance in building the bridge in\napplying TDA in large scale applications.\n\nThe theoretical justification is both thorough and sound and adopts a novel\nview on using PH in machine learning applications with the empirical evidence\nshowing good improvements when using the regularization term.", "weaknesses": "Where the theoretical contributions have good exposition, the implementation\ndetails are less thoroughly addressed. As one of the two cornerstones of the\ncontributions, it would be nice to see where the authors differ in their\nimplementation from previous work that allowed them to compute the PH on the\nGPU. The details seem to be lacking in the paper and together with the release\nof the code, would strengthen the paper considerably.\n\nFor the experimental section the optimization of point clouds is not\nparticularly convincing. In the figure the results could be improved by showing\nhow the final output has converged and in two dimensions, which would be\nnatural as other metrics are not provided. Especially since the method (I\nassume) optimizes only a subset of the whole point cloud at a time I would like\nthe doubt of only a subset of the point cloud converging to the reference point\ncloud to be taken away. Second, only the loss of the training set over the\niterations is shown. An independent set distance to quantify the results would\nstrengthen the results. The Wasserstein distance between the point clouds or\nthe Chamfer Distance would be great examples. Finally, an argument why the\nnon-zero convergence is correct would also be in place, since none of the loss\nfunctions converge to zero.\n\nThe experimental section also heavily relies on the Cramer loss, and although\nthe paper presenting it was rejected at ICLR 2018, although the paper is\nwell-cited. The reviewer is not an expert on training GAN's and evaluating\nthem, hence it would be nice to provide an argument as to why this is the main\nloss metric to use and also the main model to use and compare to.\n\nIf the reviewer would have to show to the reader that a certain regularization\nterm is effective, a good strategy would be to take a variety of datasets and a\nvariety of models (perhaps with certain properties) and show that the\nregularization consistently improves the scores across the board. As the paper\nis currently presented, the scope of the experiments is rather limited.\nMoreover, there is also comparable work on topological regularizers for\nclassification problems which would also provide a nice comparison partner. A\nlast potential point of improvement is the connection with other topological\nmethods and regularization terms. \nThe application of persistent homology is not new and a comparison with previous work, \nboth in terms of computational performance and increase in accuracy would also be \nvery nice. \n\nAlthough the reviewer commends the line of work the authors have decided to\npursue, a considerable amount of work would have to be done to create a logical\nand convincing experimental suite. Multiple reference architectures of GAN's\nwould have to be implemented with more standard metrics. Perhaps other tasks\nwould also benefit from this regularization term, such as classification and\nregression tasks. The non-standard set up for the point cloud optimization\nexperiment would also have to be convincingly revised. Once these elements are\naddressed, the work has great potential impact on the Topological Data Analysis\ncommunity and machine learning in general.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)