TopoDiffusionNet: A Topology-aware Diffusion Model
Authors: Saumya Gupta
Dimitris Samaras
Chao Chen
Venue: ICLR 2025 Poster (2025)
Keywords: Topology
Diffusion Models
Persistent Homology
Counting
Primary area: generative models
TL;DR: Diffusion models are great at creating visually appealing images but struggle to maintain specific topologies. This work introduces TopoDiffusionNet, a diffusion model designed to generate images that adhere to given topological constraints.

Abstract: Diffusion models excel at creating visually impressive images but often struggle to generate images with a specified topology. The Betti number, which represents the number of structures in an image, is a fundamental measure in topology. Yet, diffusion models fail to satisfy even this basic constraint. This limitation restricts their utility in applications requiring exact control, like robotics and environmental modeling. To address this, we propose TopoDiffusionNet (TDN), a novel approach that enforces diffusion models to maintain the desired topology. We leverage tools from topological data analysis, particularly persistent homology, to extract the topological structures within an image. We then design a topology-based objective function to guide the denoising process, preserving intended structures while suppressing noisy ones. Our experiments across four datasets demonstrate significant improvements in topological accuracy. TDN is the first to integrate topology with diffusion models, opening new avenues of research in this area.

Published as a conference paper at ICLR 2025
TOPODIFFUSIONNET: A TOPOLOGY-AWARE DIFFUSION MODEL
Saumya Gupta, Dimitris Samaras & Chao Chen Department of Computer Science Stony Brook University Stony Brook, NY 11794, USA auungupta, samaras 0cs. stonybrook.edu; chac. chen. 1estonybrook..ee
ABSTRACT
Diffusion models excel creating visually impressive images but often struggle generate images with specified topology. The Betti number, which represents the number of structures in an image, is a fundamental measure in topology. Yet, this basic constraint. This limitation restricts
diffusion models fail to satisfy their utility applications requiring exact control, like robotics and environmen- tal modeling. To address this, propose TopoDiffusionNe (TDN), novel approach that enforces diffusion models to maintain the desired topology. We leverage tools from topological data analysís, particularly persistent homology, extract the topological structures within an image. We then design topology- based objective function guide the denoising process, preserving intended structures while suppressing noisy ones. Our experiments across four datasets demonstrate significant improvements in topological accuracy. TDN the first to integrate topology with diffusion models, opening new avenues of research in this area. Code available at httttt:://gtthte com Saumya- -uppa-uppaa 266 TopoDiffusionNett
INTRODUCTION
Over the past few years, diffusion models have become prominent for image generation tasks (Shhl-
Dickstein et al. 2013 Song Ermon 2019 Ho et al. 2020 Song et al. 2020a by Nichol & Dhari-
2021 D Dhariwal & Nichol (021). Naturally, text-to-image (T21) diffusion models are a popular
choice for creating high-quality images based on textual prompts (Saharia et al.) 2022a Rombach
cet al. 2022
Avrahami et al. 2022 Ruiz al. 2023 Nichol et al. 2021] Kim et al. 2022 Ramesh
et al. 2021 Midjourney| OpenAl a). Despite their ability to generate visually impressive images, T2I models are still far from the desired intelligence level. In particular, they often struggle to interpret textual prompts that involve basic reasoning and logic. This includes preserving global and semantic constraints, such consistent number of objects well as structural patterns (e.g., enclosed regions or loops). Improving this capability would be step forward in the control and precision of diffusion models, moving beyond qualitative attributes such style and texture.
Topology, in general sense, defines how different parts of an image interact with each other, dic- tating their overall layout within an image. Preserving topology is essential for generating images that not only look realistic but also adhere to correct semantics. The simplest measure in topology the Betti number, which is equivalent the number of connected components (O-dimension) and holes/loops (I-dimension). In natural images, O-dimensional topology corresponds to the number of distinct objects, while I-dimensional topology refers to the number of enclosed regions. Yet, current diffusion models fail to preserve in applications such as urban planning, robotics, and environmental modeling, where is crucial to generate scenes with specific topology number of entities (like animals road intersections). Fig. [[-bb shows instances where popular T2I diffusion models fail to generate images with the specified topology. such specific numbers of animals, holes/regions in road layouts.
these basic topological properties. This is particularly evident
Recognizing the limited spatial reasoning of T2I models, existing methods use spatial maps (such object masks, edge maps, etc) control the generated images (Zhang et al.! 2023) Bar- Tal al.

---

Published as a conference paper at ICLR 2025
Text protege "ae image wilh emacil Faigftr
Prompti S. "wirsfiss
Pronpes: 5 "aoutee"
0-dim topology
Text "uovial asteord
Promps::
read
1-dim topology
Stable Diclusion
ADM-T ControlNet
(6) ToyooDiMasionNe ConntolNNr
Figure I: Comparison of existing diffusion models in preserving topological constraints. Top
row: O-dim topological constraint to generate exactly five giraffes. Bottom row: 1-dim topological constraint to generate road layout with exactly four distinct regions. Text-to-image methods like (a) Stable Diffusion (Rombach et al,, 2022) and (b) DALL.E 3 (OpenAl a) struggle to respect both
O-dim and 1-dim constraints. (c) Attention Refocusing (AR) (Phung al.. 2024) requires bounding boxes for each object but struggles with higher object counts and often creates partitioned images. (d)-(e) shows a two-step process: mask generation followed by ControlNet (Zhang al., (023) rendering. (d) ADM-T generates masks by fine-tuning ADM (Dhariwal & Nichol 2021) with the topological constraint as a condition, but this alone is insufficient. (e) Our TipoDiffusionNet trained with a topology-
objective function, generates masks with the precise number of objects or regions, which when fed to ControlNet generates the desired image of five giraffes (top row) and four regions (bottom row). Giraffe/region counts are noted in the bottom-right inset of each image.
2023 Bashkirova et al. 2023 Huang et al. 2023 Mou et al., 2023 Li et al. 2023). These meth- ods have indeed shown promise, offering more guided approach to image generation that aligns closely with the provided controls. Nonetheless, generating the spatial map itself is bottleneck: an automatic way generate spatial maps with desired topology is unaddressed problem.
In this work, we address the challenge of generating topologically faithful images using diffusion models. Our focus is generating images with specific topology, defined by properties such as the number of connected components (O-dimension) or loops/holes (1-dimension). These topologi- cal structures, quantified by Betti numbers, serve as our topological constraints. Spatial maps such as masks have shown success in guiding the semantics of the generated image. Leveraging this, propose using diffusion models to automate generating masks that satisfy the desired topological constraint. A straightforward approach is to condition the diffusion model on the constraint informa- tion and fine-tune it on enough samples. However, as shown in Fig. dd). we find that conditioning on the constraint alone falls short of effectively preserving the topology of the generated masks.
We propose TopoDiffusionNet (TDN), novel approach that incorporates topology guide the mask generation process. To ensure the final mask satisfies the topological constraint, topology- aware objective function is necessary for steering the denoising process so that each timestep is one step closer to preserving the desired topological constraint. However, designing such a function is not straightforward. The intermediate timesteps are very noisy especially at larger timesteps extracting meaningful information from them is challenging. We thus need tools that robust to noise. This leads us to persistent homology (Edelsbrunner et al.) 2002; Edelsbrunner & Harer] 2010), mathematical theory that can, amidst the noise, extract the topological structures within an image. Using persistent homology, we can partition an image in terms of topology: separating out the significant structures from the noisy ones. We can thus design a dedicated objective function to preserve the significant structures and suppress the rest, The function guides the denoising process to progress in such a way so as ultimately preserve the topology the final timestep, as we see in Fig. Te). In summary, our contributions are as follows:
To the best of our knowledge, we are the first to integrate topology with diffusion models to address topologically faithful image generation in both O-dimension and 1-dimension. Specifically, we generate spatial maps (masks) to tackle the challenge of generating an image with specific number of structures.
2

---

Published as a conference paper at ICLR 2025
We present TopoDiffusionNet (TDN), which utilizes topology- basod objective function to improve diffusion models' ability to follow simple topologiaal constraints. It serves as a denoising loss, guiding the diffusion denoising process in topology- awwae manner.
We evaluate TDN on four datasets to demonstrate its versatility and effectiveness. TDN ex- hibits large improvements maintaining the topological integrity of the generated image.
The success of our method suggests a surprising harmony between diffusion models and topology. Diffusion models are trained to denoise but are rather hard to control for preserving global semantics. Meanwhile, topological methods such as persistent homology provide principled solution to extract global structural information from noisy input, and can thus successfully guide the diffusion model. We hope the coupling of diffusion models and topology, as well as the techniques developed in this paper, will shed light on more sophisticated control of these generative models in the near future.
2 RELATED WORK
Diffusion models. Diffusion models, first introduced by SSohh-Dicksteinn et al. (2015). are now preva-
lent in image generation (Ho et al. 2020 Song et al. 22200a b Nichol & Dhariwal 2021 Dhariwal
& Nichol| 202 ), evolving from unconditional models, to conditional models using class labels (Ho
& Salimans| 2022 Dhariwal & Nichol, 2021 and later to eox--o-image models (Saharia et al.
2022a Rombach et al. 2022 Avrahami et al. 2022, Ruiz et al., 2023 Nichol et al. 2021 Kim
al. 2022; Ramesh et al., 202 1 Midjournyy OpenAl a). However, textual prompts have limi- tations in controlling spatial composition, like layouts and poses. Recognizing this, several works propose to use spatial maps (such masks, edge maps, etc) as condition guide the image genera-
tion process (Zhang et al. 2023 Qin et al. 2023 Zhao et al. [0244] Bar- Tal et al.] [202]] Bashkirova
et al. 2023 Huang al. 2023 Mou et al. 2023) These approaches aim to overcome the limitations
of text-based conditioning by providing more explicit spatial guidance.
Numeric control in diffusion models. Betti numbers quantify topological structures, such as the number of connected components (O-dimension) or loops/holes (1-dimension). which is conceptu- ally related to the task of counting. Recent works have explored approaches to enhance the counting
performance in diffusion models. Paiss et al. (2023) enhances CLIP''s (Radford et al., 2021] text
embeddings for counting-awand -ware text-to-image (T2I) generation via Imagen (Saharia et al. 2022b). While this results in some improvement, the performance is still limited, supporting our motivation that T2I models often struggle with textual prompts involving semantic reasoning. Layout-based methods (Chen et al.) 2024 Phung et al.] 2024] Farshad et al. (023) use layout maps, that is, maps containing bounding boxes of each object/entity, to guide the reverse diffusion process. While these methods show promise. they are not scalable as their complexity increases with the number of ob- jects. Furthermore, focusing on these boxes often results in images that appear partitioned, as shown in Fig. (c)) top row, where the backgrounds of each giraffe differ. Finally, all of the methods men- tioned above are limited to O-dimensional topological structures (i.c., discrete objects) and do not extend to higher-dimense dimee--iimelii topological constraints.
Deep learning with topology. Methods from algebraic topology, under the name of topological data analysis (TDA) (Carlsson, 2009), have found use in various machine learning problems owing
to their versatility (handling data such as images, time-series, graphs, etc.) and robustness to noise.
The most widely-used tool from TDA, persistent homology (PH) (Edelsbrunner al.] 2002 Edels-
brunner & Harer 2010),, has been applied to several image classification (Peng et al.
2024: Wang
2021 Du al. [2022] Hoter et al. 2017 Chen et al. (2099) and segmentation tasks (Abousamra
et al.
2021 , Clough et al. 2019 Hu et al. [2199] et al. 2020 Stucki et al. 2023 Byrne et al.,
2022
He al. [023] Xu al. 2024) as can track topological changes at multiple intensity val-
ues. Other TDA theories like discrete Morse theory (Dey et al. 2019 Hu et al. 2021 2023 Cupta
let al. 2024; Banerjee et al.
2020), topological interactions (Gupta et al. 2002). and center-line Wang al. 2022a), have also enhanced performance in these areas.
transforms (Shit et al. 2021
In the realm of generative models, TDA has been used with generative adversarial networks (GANs) (Goodfellow et al.] 2014) to evaluate performance through topology comparisons (Khrulkov) & Oseledets 2018), and enhance image quality via topological priors (Bbiie-Ga et al.] 2019) and PH-based loss functions (Wang al. (020). These methods have mainly focused on quality enhancement without directly controlling the topological characteristics of the images. In the case of diffusion models, incorporating TDA has not yet been explored. Our work, TopoDif-
3

---

Published as a conference paper at ICLR 2025
fusionNet, represents unique effort in this direction, employing PH within diffusion models to control the topology of the generated images. This is a sigmificant shift from existing applications of TDA in generative models, moving beyond quality improvement to precise topological control.
3 METHODOLOGY
Given a topological constraint e, our goal is to generate mask contain- ing e number of structures. A structure can either correspond to an object or hole/region. We illustrate these structures in Fig. 2 In (a), the four objects are in white. In (b), the four holes correspond to the four black regions/partitions the white lines create with the border. In the formal language of algebraic topology (Munkres) 2018). e is the Betti number, that is, it the rank of the homology group, which objects (connected components) correspond to the O-dimensional (0-dim) homology classes and holes/noossrregions correspond to the I-dimensional (I-dim) homology classes.
Z
(a) Four objects (b) Four holes
Figure 2: Illustration of topological structures.
To enforce the topological constraint the diffusion model needs to be conditioned on it. However, e alone is not sufficient to control the topology of the final generated image") Therefore, we propose topology-ber based objective function Lup to guide the reverse denoising diffusion process at each timestep during training. Ltop uses persistent homology (Edelsbrunner et al. 2002 Edelsbrunner & Harer 2010) to distinguish between the desired and the spurious structures, aiming enhance the former and reduce the latter to ensure the topology matches e closely. Fig. provides an overview.
The rest of this section is organized follows. We briefly discuss diffusion models in Sec. B.1 followed by quick background on persistent homology in Sec. 3.2 We tie these concepts together finally introduce our method TopoDiffusionNet (TDN) in Sec. 3.3
3.1 DIFFUSION MODELS
Diffusion models (Ho et al., 2020 are able to sample images from the training data distribution p(20) by iteratively denoising random Gaussian noise in T timesteps. The framework consists of forward and a reverse process.
In the forward process, at every timestep & E T, Gaussian noise is added to the clean image To p(zo) until the image becomes an isotropic Gaussian. The forward noising process is denoted by q zo) == N (Ias Vāzzo. (1 ai) I). which can be rewritten as,
Zt v aazo + v(1 (1=aa)e
(1)
<~ I) is noise variable, a, is the noise scale at timestep t, and V is the normal distribution.
The reverse process aims to learn the posterior distribution q (x,-1 Ø2, X2, zn). using which we recover 2t-1 given Zt. This is typically done by training denoising neural network (U-
Net (Ronneberger et al.) 2015)) with network parameters 6. The denoising model c0 t) takes the noisy input Iy timestep í and predicts the noise added in Eq. ① of the forward pro- cess. The model is trained using the simplified objective function Lximple (Ho et al. 2020):
Lsimple = zu, [1), t) e <l In our case, we additionally provide the topological constraint e as a condition control the topology of the generated image. Thus, the denoising model becomes co c, t), where e is injected into the denoising neural network.
During training, although the denoising model predicts the noise cg c, t) at timestep t, we can deterministically (without iterative sampling) recover the predicted noiseless image 20 (an estimate of the true σi)) from Eq. ① as,
[2. - - c)
This alternate form of the prediction will enable to compute Loop as we will see in Sec. 3.3
3.2 PERSISTENT HOMOLOGY
Persistent homology (PH) (Edelsbrunner et al. 2002 Edelsbrunner Harer 2010). owing to its differentiable nature, is an attractive candidate for integrating topological information into the train- ing of deep learning methods. In the case of image data, can detect the changes in topological
In this section, we 'image" to mean the mask.
4

---

Published as a conference paper at ICLR 2025
(a) TopoDiffusionNet (TDN) Overview
(b) Lup
Clean image aip
Predicted clean image 46
Persistence
diagram (
Noise e (Trem forward process)
Topological constraint
Extoll
Lup
Forward process
[ zo)
Predicted noise ea(#,, c,4)
Predicted clean 4
Noisy image 24
Denoising U-Net
Salient P (P)
Noisy set
Eq. (2)
Cuach
Topological
Encoding
Timestep
network
embedding
Ddmeio
We condition the diffusion model on the topological constraint (here = 2). During training. we first add noise to the input Zo using the forward process (Eq. ( to obtain F,, where t is sampled uniformly. The U-Net trained as part the reverse process, predict- ing the added noise c0(I2. C, 1). with which we obtain the noiseless image z (Egq. (2)). Alongside
Figure 3: (a) TDN overview:
standard loss Simple we propose Lop to enforce the topological integrity of si. (b) To compute
Cup. the persistence diagram (aa)) captures all the topological structures in 2 partitioning them
into salient/desired structures P and noisy ones Q. Terms Epreserve and Ldenoise respectively amplify P and suppress Q. guiding the training eventually satisfy
structures (connected components and holes) across varying threshold (also called the filtration value). More importantly, persistent homology is robust to noise, that is, it can extract these struc- tures even in noisy scenarios. Structures that exist for a wide range of thresholds are salient, while the remaining structures deemed noise in the image.
In our setting, during training, we can apply persistent homology to every intermediate image ø (from Eq. 2 predicted by the diffusion model timestep For ease of reference, denote Ai by having size h x w. In practice, has continuous probability values in a normalized range, say, [0, We now consider super-level sets i.e., the set pixels (i, for which L.j above some threshold value u. Let S denote the super-level set, then, S(u) == {i,j) E [1, h] [I, w] 4ij 2 u). This is nothing but thresholding, and we call the resulting binary image the super-level
at u, S(u). Decreasing continuously generates a sequence of sets, i.e. a filtration, which grows as the threshold
parameter u is brought down: C S(1) S S(u,) S S(u2) S 5.
[1, h] [1, w). We
demonstrate this filtration in Fig.
When u is high, only few pixels can exceed the threshold, and hence the size of S(1) is small (an almost black image). As decreases, new pixels join the set, and topological structures in S are created and destroyed. Eventually, at u = 0, the entire image in the super-level set. In this manner, persistent homology can track the evolution of all the topological structures.
The output of the persistent homology algorithm includes the birth and death threshold values for each topological structure. We can keep track of the birth (creation) and death (destruction) d thresholds of all the topological structures and put the tuples (6, d) in a diagram the persistence diagram ' where the y-axis represents birth and the z-axis death [] The persistence diagram is thus a graphical representation of topological structures throughout the filtration process, consisting of multiple dots in a 2-dimensional plane (see Fig. 4. These dots are called persistent dots, where each dot corresponds to one topological structure. The persistence, or the lifetime of a structure, is given by the difference between its death and birth times. Structures that persist over wide range of thresholds are considered significant or salient, indicating stable and prominent structures within the image, while short-lived structures are the noise. The diagonal zero persistence and dots far from the diagonal represent salient structures with high persistence.
represents structures of
The range is typically [-1, 1] implementation. If using sub-level instead of super-level set filtration, the diagram would have the z-axis as birth, and the y-axis death.
5

---

Published as a conference paper at ICLR 2025
O-dimeusional topology (connected components)
I= 0
Persistence diagram D
Filtration, sequence of super-level sets S(k)
$1.00
.10.811
stiaad
5QL221
AT6.951
5005
Figure 4: Illustration of persistent homology and persistence diagrams of O-dim topological struc- tures (connected components). Despite the noise, we can visually see three prominent structures 01, 022 03 in J. In the topological space, (1,, (22, thus appear the top-left corner of the per- sistence diagram D(I), persisting through most of the filtration S. All the remaining connected components are noisy, persisting over a short threshold in S, thus appearing closer to the diagonal in D().. Persistence diagrams useful distinguish between salient and noisy structures in image. The equivalent illustration for 1-dim topological structures (holes) is in Appendix A
With this information, we can identify noisy structures due to their low persistence and proximity to the diagonal, allowing us to filter them out and retain the salient topological structures. The number of salient structures we retain is precisely the number of structures we desire in the final image. As we show in the next subsection, we compute the persistence diagram of the predicted image 26 to optimize it from topological perspective.
3.3 PROPOSED TOPODIFFUSIONNET (TDN)
Conditioning. We condition the diffusion model on to enable it to generate a mask containing exactly number of structures. We follow Nichol & Dhariwal (2021) to inject the condition in- few linear
formation. We first obtain an embedding of с from a trainable network composed layers. Next, inject the embedding through the same pathway the timestep embedding of t. Consequently, both embeddings are passed to residual blocks throughout the denoising model.
Objective function Loop- Conditioning alone is not sufficient to control the topology of the gen- crated image. To address this, we introduce Lup. topology- based objective function to force the predicted image every timestep to preserve as closely as possible. Since the diffusion model is parameterized to predict in the noise space, directly analyzing topology from this noise is not meaningful. We need to map the prediction from the noise space back to the image space in order to infer the topology. From Eq. (2). we obtain #6 an estimate of the noiseless image To at timestep t from ea (Zt, c, t). The estimate át noisy, especially when é large, making it ideal to the theory of persistent homology to separate out its salient structures from the noisy ones.
Given the prediction di. we compute its persistence diagram D(ag) containing either O-dim 1- dim information based on the desired topological structure (object or regions). Recall that we desire e topological structures in the predicted image. For persistent dot p E D. with birth b, and death dp. its persistence value, 1bp dpl. measures its significance, according to the theory. We rank all dots in by their persistence values in descending order. The top e dots are the structures aim to preserve, reflecting our desired topology, whereas the rest denote noisy structures to be suppressed/denoised. Thus, decompose the diagram D into two disjoint sets, D PÚQ. where P contains the e largest persistence dots (IP) = c), while Q contains all the remaining dots.
To constrain a to have structures in the image space, in the topological space need to maximize the persistence or saliency of the dots p E P, and suppress all the noisy dots E Q. To achieve this, introduce loss terms:
Cpreserve
Σ 1bp
dz1
and Litenoiee
Σ 1bp PER
d
(3)
PEP
Coo Lpeserve + Laenoise
(4)
Minimizing Lup is equivalent to maximizing the saliency of the top structures (via Cpreserve) and suppressing the saliency of the rest (via Ldenoise). In the ideal case, Lup -c, as each of the top structures will have persistence of while all the noisy structures will have persistence.

---

Published as a conference paper at ICLR 2025
dimetsioral topology (connested componetts)
Predicted clesn image af
Persistence diagram D
Clan image
Tersin view g
(cubllc)
Salient set ?
Noiey set Q
Figure 5: Illustration of Lereserve and dennisee for O-dim connected components, with c 2 seen in Zo. After computing D( ). we partition it into sets P (the top e structures) and Q (remaining ones). For each dot p D(ag)) the birth and death values respectively correspond to local max- ima my and saddles Sp in dg. In the terrain view of d structures (m1, 81) and (m2, S2) belong to P: hence optimizing Eresserve increases their saliency by increasing (g(m). [6 (m2) and decreasing ( All the remaining n structures (my, $3), (Ma, s1).... (m., 8n) belong to Q. Op-
timizing Ldenoise suppresses these noisy structures by decreasing z (ma)) ), 2ig (ma),.. 2i (17)) and
increasing (53), (5()s)... (s.). The equivalent figure for 1-dim holes is in Appendix B
The image i will then have exactly e topological structures as desired. In the absence of Loo the denoising process were to originally result in structures, now Ldenoise will suppress all the extra/noisy structures, preventing them from appearing in the final clean image. One concern is whether there could be less than e structures in total. In practice, at large timesteps t, & always has several thousand noisy topological structures. Thus, if the denoising process were to originally proceed with c structures, preserre will now increase the persistence of less salient dot to ensure the final image has exactly structures.
Implementation and differentiabilityy For every topological structure, the birth and death values b and d correspond function values of a maximum-saddi: pair; b and d function values of local maximum m and saddle point s, respectively. These pairs can be determined by the almost linear union-find find algorithm (Edelsbrunner & Harer 2010: Ni et al. (017)) which locates and pairs all local maxima and saddle points to reflect the birth and death of topological structures. For every persistent dot p E D, let mp and 8p respectively denote the 2D coordinates of the corresponding local maximum and saddle point in the prediction 2/. Then, Eq. 3 and Eq. (4) can be rewritten as,
Lup
Σ (m))m)) - (tp)) + Σ (=)(mp) - (xp) PEp
(5)
PEO
We illustrate this in Fig. 5 During training, for every topological structure to preserve, i.e., p E P. the function Epreserve increases the intensity value a (mp) at the local maximum mp and decreases (sp)) the saddle point Sy. This strengthens the saliency of the desired structures. At the same time, to prevent exceeding c structures in the final image, Ldenoise suppresses structures E Q by reducing the intensity value (m,p)) the local maximum mp while increasing a (sp) at the saddle point 8p- Litemoise forces # (mp) to be equal to i (sp), leading to a homogenous region. This effectively eliminates the noisy structure,
was neither born nor died, rendering it non-existent.
With this, we see that Loop is differentiable, as Eq. 5)) is written as polynomials of the prediction #6 at certain pixels. From Eq. (5) we can compute the gradient of Loop with respect to 4 and via chain rule, we can ultimately compute the gradient with respect to the denoising model's parameters 0. The training optimization adjusts to ensure that the topological space, i.e. the persistence diagram D(f)), has exactly persistent dots, in turn resulting structures in the image space 2/.
End-to-end training. The overall training objective Liaal of TDN is formulated Liotal Csimpl + XLup where À is the loss weight. The standard denoising loss Eximple produces visually good results, whereas Luop helps respect the topological constraint
7

---

Published as a conference paper at ICLR 2025
4 EXPERIMENTS
Datasets. We train ADM-T and TDN on four datasets: Shapes, COCO (Caesar et al. 2018 CREMI (Funke et al. 2016), and Google Maps (Isola et all The Shapes dataset is a synthetic dataset created by us that contains objects such as circles, triangles, and/or rectangles. For CREMI, COCO, and Google Maps, we train the diffusion models on their segmentation masks. For COCO, we select masks which contain at least one instance of the super category 'animal'. For COCO and Shapes, we use O-dim, the number of connected components, as the topological constraint. For COCO, we also use the animal class as condition to generate masks of specific animals. CREMI an Electron Microscopy dataset, and Google Maps contains acrial photos from New York City. For CREMI and Google Maps, we use 1-dim, the number of holes, as the topological constraint. Each of the datasets contains masks consisting of up to ten structures. More details are in Appendix C
Baselines. Stable Diffusion (Rombach et al.] 2022) and DALLE (OpenAl a) are popular T2I diffusion models. Attention Refocusing (AR) (Phung et al. 2024 uses layout maps (bounding boxes for each object) generated by GPT-4 (Achiam et al. 2023) to guide the reverse process.
Implementation details. Our work extends the ADM Dhhriial & Nichol 2021 diffusion model. We use 'ADM-T' to denote the modification of using a topological constraint as a condition. We obtain an embedding of the constraint using encoding network. Following the approach in Nichol & Dhariwal (202T), (202), , we then feed this embedding to all the residual blocks in the network by adding to the timestep embedding. For COCO, we similarly inject animal class embedding to further control the generated mask. For every dataset, we use 256 x 256 as the image resolution. Our diffusion models use cosine noise scheduler (Nichol & Dhariwal, (021), with T 1000 timesteps for training. During inference, however, we use only 50 steps of DDIM (Song et al., 2020a) sampling. For the ADM-T baseline, we load a pretrained checkpoint (OpenAl b) and then fine-tune on our datasets using the constraint information as condition. For TDN, we follow the same approach but additionally use Cup in the training. To compute persistent homology, we use the Cubical Ripser (Kaji et al., 2020) library. More details are listed in Appendix
Evaluation metries. To evaluate whether the generated image satisfies the input constraint, we use metrics such as Accuracy, Precision, and F1. We report the mean and standard deviation of the results across different constraint values. To measure the performance, for O-dim, we generate 50 samples per constraint E [1, 10] per animal/shape category (resulting in 5K images for COCO). We similarly generate 50 images per constraint E [1, 10] for the 1-dim datasets. We perform the unpaired t-test (Student 1908) (95% confidence interval) to determine the statistical significance of the improvement. In all the tables, performances that are statistically significantly better are bold.
4.1 RESULTS
Qualitative and quantitative results for 0-dim. In Fig. 6 and Tab. D. we present qualitative and quantitative comparisons, respectively, of pretrained Stable Diffusion, DALL-E 3, AR, ADM-T, and our proposed TDN. In Appendix [R we provide constraint-wise results. As ADM-T and TDN pro- duce masks, we employ pretrained ControlNet (Zhang et al., 2023 L.mmi Zhang) SS115 backbone) to create textured images from these masks. In practice, any of the methods (Qin et al. 2023
Zhao
eet al., 2024 Bar- Tal et al.) 2023: Bashkirova et al. 2023 Huang et al. 2023 Mou et al. 2023)
could also be used for this purpose, but we chose ControlNet for its simplicity. From Fig. we that T2I models Stable Diffusion and DALL-E 3 are unable to respect the explicit topological constraint of generating e objects. Due to limited spatial and semantic reasoning, both methods have the lowest performance in Tab. [ AR uses layouts generated by GPT-4 which improves over T2I models, as seen in Tab. T In general, however, GPT-44 does not have much spatial awareness and when the count is high, the bounding boxes tend either be too small or highly overlap with each other. This leads to incorrect counts in the generated image as seen in Fig. 6 Additionally, when the object count is high, AR often produces fragmented results, with objects isolated within assigned areas, leading to a divided and visually disjointed image. ADM-T, despite being conditioned on the constraint e, also falls short of satisfying the constraint. This indicates that the constraint alone is not powerful enough to influence the global reasoning of the model. In contrast, TDN preserves the constraint better than ADM-T, as evident from the quantitative results in Tab. I TDN demonstrates significant improvements across all metries. Lpreserve aims to retain at least objects, while Ldenoise aims to maintain atmost e objects. Thus optimizing both simultancously via training with Luo helps the diffusion model to preserve e objects in the generated mask.
8

---

Published as a conference paper at ICLR 2025
Stable Dirlusion
DALL
AR
ADM-T ConcrolNet
TDN (Ours) ControlNet
Text prompt: "oundyy 4 otjjectss
ADM-T TDN condition 4; ControlNet text pronpt: "odyeets"
prompi:
prompt
Text promptt "exacily can""
ADM-T TDN condition: 6, "car"; ControlNer text promppt. "curv"
Text prompt: "esarity 7 "hinde"
ADM-T TDN condition "hint" NentrolNer text prompt:: "hinde"
X
Figure 6: Qualitative results for O-dim topological constraint. Row 1 Shapes dataset; Rows 2-4: COCO dataset. ADM-T and TDN take constraint as the condition (purple box), and also the animal class for COCO. Stable Diffusion, DALL-E 3, and AR take the equivalent text prompt as input. Object/animal counts are noted in the bottom-right right inset of each image/mask.
ADM-T TDN arndiion:
ADM-T TTN orddrimn 6 Coemo Nel peorett Toologr atame
ADM-T TDN condition Contrilin prerpe: Twons Taonss of
ADM- TDN cunditon
ConretNer proepee "Nivlggy avsnost
Commeoleet proepe: "oonsl stre moad"
ConzolNet
:DN:
CommotNet
Figure 7: Qualitative results for the I-dim topological constraint. ADM-T and TDN take the con- straint (in the purple box) the condition. Columns 1-2: CREMI. Columns 3-4: Google Maps. Number of holes within each mask is noted in its bottom-right inset.
Qualitative and quantitative results for 1-dim. In Fig. 7 we provide the qualitative compari- for ADM-T and TDN. We exclude results from Stable Diffusion, DALL-E 3, and AR these methods are limited to generating distinct objects (0-dim topology) and currently cannot handle generating distinct holes/regions (1-dim topology). As holes are a complex topological constraint, they are challenging to describe in words, and hence the performance of such methods is limited (see Fig. and Appendix G Similar to the O-dim case, ADM-T struggles with the topological con- straint. The quantitative results in Tab. [ highlight that preserving 1-dim topology, which involves boundaries spanning across the mask, is more challenging than O-dim topology. This complexity necessitates powerful global reasoning capabilities from the diffusion model, an area where ADM-T shows limited performance. However, with Lup- TDN achieves substantial improvements across all metrics, with Epreserve and Ldenoise both working to retain exactly holes in the generated mask.
Additional comparison, Paiss et al. (2023) enhances CLIP's text embeddings for counting and uses them to show counting- aware T21 generation via Imagen. We report the Accuracy of TDN results in their setting in Tab, 2 showing significant performance improvement. This supports our motivation that T2I models often struggle with textual prompts involving semantic reasoning.

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative comparison on preserving the topological constraint
Dataset
Method
Accuracy
Prevision
F1
Stble Dillugunn Rumnwyy
0.6361
02339
0.637
DALLE 3 (Owen
0.6857
0.2561
0.7009
0.3198
0.9566
0.2649
Shapes
AR (Pungg at 2046
0,7384 1: 0.2178
0,7596 1 0.2039
0,7474
0,2165
ADMFT
0,7500 0.1889
0,7808
0. 0.1582
0.7651
0.1210
Stable Diffusion RRnseay
0.46% = 0.2361
0.5154 4 0.1747
0,4909 0.1976
DMLLEE (Opena)
0,1162 0.5674
0.5491 0.2724
0.1994 1.3256
(Animals)
37
06379
06611
0.1851
0,668 0.1485
0.7999
0.1931
TDN (Oursi
0,857 = 0,0805
0.8670 1 0.0636
0,8613 0,0970
Google Maps
ADM T
0.339 0.1356
0,5642 0.1861
0.3567 0.11KS
CREMI
TDN (Our))
ADM-T
0.7785 0.1901
0.5357 = 0.7979
0,777 1: 1797
0.8142 + 0.1925
0.4881 0.157
0.7959 0.1699
Table Additional baseline
Table 3: Ablation on loss terms
Table 4: Ablation on À
Method
Accuracy
Leude
Accuracy
Loss weight A
Paiss
07500 = 0.1889
0,7500 0.189
0.796
x
0.826 = 0.1821
le-3
0.9066 + 0/0122
/
0.186 0.1129
le-5
0,9478 + 0,6420
0.9478 0,0420
0.176 0.407
Min-SNR (5)
0.9286 + 0.030
Effect across timesteps. While we use 50 steps of DDIM (Song et al.) 2020a) sampling for in-
ference, the training was conducted with T 1000 timesteps using DDPM (Ho et al.. (020). To understand the effect of Loo throughout training, we analyze intermediate results from the DDPM inference procedure. In Fig. g. we visualize # across different timesteps on the Shapes dataset, highlighting the impact of Lup on denoising efficiency. Notably. TDN achieves closer approxima- tion the true Ty by timestep 750, nearly 200 timesteps before ADM-T. Furthermore, we plot the average Fréchet Inception Distance (FID) (Heusel et al.) [017] for 1000 samples per timestep. The plot shows that Luo accelerates denoising and enhances the quality of i at earlier stages, working intended. This demonstrates that Ctop improves both speed and accuracy of the denoising process.
4.2 ABLATION STUDIES
To demonstrate the efficacy of TDN, we conduct ablation studies on the loss components and the effect of hyperparameter value changes. Appendix H includes an ablation study on the 'Encoding Network' for the topological constraint All analyses on O-dim Shapes dataset.
Ablation study on Presseve and Ldenoise- In Tab. we show the contribution of Ldenoise and ppeserve in meeting the topological constraint. While both terms individually improve performance, Ldenoise has more significant effect. This because d especially at larger timesteps, has several spuri- ous structures. Ldenoise works by suppressing the birth of these structures, leaving only the desired number of structures the mask. Meanwhile, Lpreserve crucial when the model generates fewer structures than expected. Naturally, combining both terms yields the optimal performance.
Ablation study loss weight >. In Tab. 4 show experiments with different weights for Loop- including the Min-SNR (y 5) (Hang et al. 2023) strategy where the loss weight is function of timestep 6. When 1e TDN achieves the best performance. Nonetheless, reasonable range of X always results in improvement, demonstrating the efficacy and robustness of Ltop-
CONCLUSION
We propose TopoDiffusionNet, the first method to integrate topology with diffusion models. Our approach generates images that preserve topology by producing masks with a specified number of structures (Betti number). Empirical results show significant improvement in preserving this topological constraint, demonstrating that our method guides the denoising process in a topology- manner. This paves the way for further research topological control in image generation.
Constraint: 1000
/- 850
750
650
550
y- 450
FID of 2 vs timestep r
106
ADM-T
IDN
Figure 8: TDN has better FID of äf at larger timesteps compared to ADM-T.
10

---

Published as a conference paper at ICLR 2025
Reproducibility Statement. We provide experimental details regarding the datasets, baselines, evaluation metrics, and implementation in Sec. Additional details of the dataset are provided in Appendix In Appendix provide additional details about the baselines, the implementation of our method, and the computation resources used. Code available at wtt.ttt:////.. / thub. com/ Saumya-Gupta-26.
f ffus onNet.
Acknowledgments. would like to express my gratitude to Eisha Gupta for her valuable feedback on the writing and presentation of this paper. also sincerely appreciate the anonymous reviewers for their time and effort in providing constructive critiques during the review and rebuttal process. This research was partially supported by the National Science Foundation (NSF) grants CCF-2144901, IIS 221 2212046 and SCH 2123920, the National Institute of Health (NIH) grants BUINS143144 and R01CA297843, and the Stony Brook Trustees Faculty Award.
REFERENCES
Shahira Abousamra, Minh Hoai, Dimitris Samaras, and Chao Chen. Localization in the crowd with topological constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 872-881, 2021.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.066 088744 2023.
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended difffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18208-18218, 2022.
Samik Banerjee, Lucas Magee, Dingkang Wang, Xu Li, Bing-Xing Huo, Jaikishan Jayakumar, Katherine Matho, Meng-Kuan Lin, Keerthi Ram, Mohanasankar Sivaprakasam, et al. Seman- tic segmentation of microscopic neuroanatomical data by combining topological priors with encoder-decoder deep networks. Nature machine intelligence, 2(10):585-59, 2020.
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. 2023.
Dina Bashkirova, José Lezama, Kihyuk Sohn, Kate Saenko, and Irfan Essa. Masksketch: Unpaired structure-guider masked image generation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 1879-1889, 2023.
Rickard Brübel-Gabiilssonn, Bradley Nelson, Anjan Dwaraknath, Primoz Skraba, Leonidas J Guibas, and Gunnar Carlsson. A topology layer for machine learning. arXiv preprint 1rXi::1905.12200. 2019.
Nick Byrne, James Clough, Israel Valverde, Giovanni Montana, and Andrew King. A persistent homology-base based topological loss for cnn-based multiclass segmentation of cmr. IEEE transac- tions on medical imaging, (2(1):3-14, 2022.
Holger Caesar, Jasper Uijlings, and Vittorio Ferraii. Coco-stuff: Thing and stuff classes in context. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1209- 1218, 2018
Gunnar Carlsson. Topology and data. Bulletin of the American Mathematical Society, 46(2):255- 308. 2009.
Chao Chen, Xiuyan Ni, Qinxun Bai, and Yusu Wang. topological regularizer for classifiers via persistent homology. In The 22nd International Conference Artificial Intelligence and Statis- fics, pp. 2573-2582. PMLR, 2019.
Minghao Chen, Iro Laina, and Andrea Vedaldi. Training- free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5343-5353 2024.
11

---

Published as a conference paper at ICLR 2025
James R Clough, Iikay Oksuz, Nicholas Byrne, Julia A Schnabel, and Andrew P King. Explicit topological priors for deep-learning based image segmentation using persistent homology. In International Conference on Information Processing in Medical Imaging, pp. 16-28. Springer, 2019.
James R Clough, Nicholas Byree, Ilkay Oksuz, Veronika A Zimmer, Julia A Schnabel, and An- drew P King. A topological loss function for deep-leaming based image segmentation using persistent homology. TPAMI, 2020.
Tamal K Dey, Jiayuan Wang, and Yusu Wang. Road network reconstruction from satellite im- ages with machine learning supported by topological methods. In Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp- 520-523, 2019.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8799 8894,, 2021.
Shiyi Du, Qicheng Lao, Qingbo Kang, Yiyue Li, Zekun Jiang, Yanfeng Zhao, and Kang Li. Distill- ing knowledge from topological representations for pathological complete response prediction. In International Conference Medical Image Computing and Computer-Assisted Intervention, pp. 56-65. Springer, 2022.
Edelsbrunner, Letscher, and Zomorodian. Topological persistence and simplification. Discrete & Computational Geometry, 28:511-5333 2002.
Herbert Edelsbrunner and John Harer. Computational Topology: An Introduction. American Math- ematical Soc., 2010.
Azade Farshad, Yousef Yeganeh, Yu Chi, Chengzhi Shen, Böjm Ommer, and Nassir Navab. Scene- genie: Scene graph guided diffusion models for image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 88 38-98, 2023.
J Funke, S Saalfeld, DD Bock, SC Turaga, and E Perlman. Miccai challenge on circuit reconstruction from electron microscopy images, 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Saumya Gupta, Xiaoling Hu, James Kaan, Michael Jin, Mutshipay Mpoy, Katherine Chung, Gagan- deep Singh, Mary Saltz, Tahsin Kure, Joel Saltz, et al. Learning topological interactions for multi-class medical image segmentation. In ECCV, 2022.
Saumya Gupta, Yikai Zhang, Xiaoling Hu, Prateek Prasanna, and Chao Chen. Topology-aware uncertainty for image segmentation. Advances in Neural Information Processing Systems, 36, 2024.
Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffusion training via min-snr weighting strategy. arXiv preprint arXiv:2303.09555. 2023.
Hongliang He, Jun Wang, Pengxu Wei, Fan Xu, Xiangyang Ji, Chang Liu, and Jie Chen. Toposeg: Topology-aware nuclear instance segmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 21307-21316. 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. arXiv:2207. 12598, 2022.
Classifier-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-685.. 2020.
12

---

Published as a conference paper at ICLR 2025
Christoph Hofer, Roland Kwitt, Marc Niethammer, and Andreas Uhl. Deep learning with topologi- cal signatures. Advances in neural information processing systems, 30, 2017.
Xiaoling Hu, Fuxin Li, Dimitris Samaras, and Chao Chen. eopology-preserviie deep image seg- mentation. In NeurIPS, 2019.
Xiaoling Hu, Yusu Wang, Li Fuxin, Dimitris Samaras, and Chao Chen. Topology-aware segmenta- tion using discrete morse theory. In ICLR, 2021.
Xiaoling Hu, Dimitris Samaras, and Chao Chen. Learning probabilistic topological representations using discrete morse theory. In ICLR, 2023.
Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable image synthesis with composable conditions. arXiv preprint arXiv:3302.09778 2023.
Phillip Isola, Yan Zhu, Tinghui Zhou, and Alexci A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134 2017.
Shizuo Kaji, Takeki Sudo, and Kazushi Ahara. Cubical ripser: Software for computing persistent homology of image and volume data. arXiv preprint arXiv:2005. /2692, 2020.
Valentin Khrulkov and Ivan Oseledets. Geometry score: A method for comparing generative ad- versarial networks. In International conference on machine learning, pp. 2621-2622 PMLR, 2018.
Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2022.
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-imase -maae generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 22511 12511-,,52,, 2023.
Maneesh Agrawala Lvmin Zhang. Controlnet-seg model. CottrolNet-Seg model card, https:
seg
2023.
Inc Midjoumny. Midjourney.
2024.
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.0885.. 2023.
James R Munkres. Elements of algebraic topology. CRC press, 2018.
Xiuyan Ni, Novi Quadrianto, Yusu Wang, and Chao Chen. Composing tree graphical models with persistent homology features for clustering mixed-type data. In International Conference on Machine Learning, pp. 2622-2631. PMLR, 2017.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.107411 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffuuson probabilistic models. In International Conference on Machine Learning, pp. 8162-81777 PMLR, 2021.
OpenAl. Dall-e http:://ppenaa..co//prrootooto/t
2023.
OpenAl. Lsun bedroom model.
core windows. .nt/ htt.//aaahhhh/aaaa 2021 Isun uncond 100M 110___0000022200022002 11440KKKba64.00 2021.
Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel. Teaching clip to count to ten. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3170-3180, 2023.
13

---

Published as a conference paper at ICLR 2025
Yaopeng Peng, Hongxiao Wang, Milan Sonka, and Danny Z Chen. Phg-net: Persistent homology guided medical image classification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 7583-7599 2024.
Quynh Phung. Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refocusing. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 7932-7942, 2024.
Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang. Yingbo Zhou, Huan Wang, Juan Car- los Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffusion model for controllable visual generation in the wild. arXiv preprint arXiv:2305.1 11105.1114,, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pp. 8821-8831. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
the IEEE/CVF confer-
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical Image Computing and Computer-Assisted Intervention- MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceed- ings, Part 18, pp. 241-241. Springer, 2015.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-iman to-it diffusion models for subject-driver generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22500- 22510, 2023.
Runway. Stability ai. Stable diffusion v1.5 model card, Eurwayml/stable delle--iffuue wwffffiiof-v-lll---------------- 2022.
co/
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Bureu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Pho- torealistic text-to-image diffusion models with deep language understanding. URL https://arxiv. org/abs/2205.11453 4, 2022a.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image difffusion models with deep language understanding. Advances in neural informa- tion processing systems, 55:36479-36994, 2022b.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. In European Conference Computer Vision, pp. 87-103. Springer, 2025.
Suprosanna Shit, Johannes C Paetzold, Anjany Sekuboyina, Ivan Ezhov, Alexander Unger, Andrey Zhylka, Josien PW Pluim, Ulrich Bauer, and Bjoern Menze, cldice-a novel topology topology-preservigg preserving loss function for tubular structure segmentation. In CVPR, 2021.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan. and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing, pp. 2256-2265. PMLR, 2015.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances in newral information processing systems, 28, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020a.
14

---

Published as a conference paper at ICLR 2025
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint 1:Xiv:2011.134550 2020b.
Nico Stucki, Johannes C Paetzold, Suprosanna Shit, Bjoern Menze, and Ulrich Bauer. Topologically faithful image segmentation via induced matching persistence barcodes. In ICML, 2023.
Student. The probable error mean. Biometrika, 6(1):1-25, 1908.
Robert Endre Tarjan. Efficiency of good but not linear set union algorithm. Journal of the ACM (JACM), 22(2):215-225, 1975.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- lion processing systems, 30, 2017.
Fan Wang, Huidong Liu, Dimitris Samaras, and Chao Chen. Topogan: A topology-aware generative adversarial network. In Computer ECvV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part 16, pp. 118-136. Springer, 2020.
Fan Wang, Saarthak Kapse, Steven Liu, Prateek Prasanna, and Chao Chen. Topotxr: a topological biomarker for predicting treatment response in breast cancer. International Conference on Information Processing in Medical Imaging, pp. 386-397. Springer, 2021.
Haotian Wang, Min Xian, and Aleksandar Vakanski. Ta-net: Topology-aware network for gland
segmentation. In WACV, 2022a.
Tengfei Wang, Ting Zhang, Bo Zhang, Hao Ouyang, Dong Chen, Qifeng Chen, and Fang Wen. Pretraining is all you need for image-to-image translation. arXiv preprint arXiv:2205. /2952, 2022b.
Meilong Xu, Xiaoling Hu, Saumya Gupta, Shahira Abousamra, and Chao Chen. Semi- Semi-supervised segmentation of histopathology images with noise-aware topological consistency. European Conference on Computer Vision, pp. 271-289. Springer, 2024.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings the the IEEF/CVF International Conference on Computer Vision, pp. 3836-3847, 2023.
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
15


Reviews:
[{"review_id": "ujICMwi63S", "reviewer": "Reviewer_1o9Y", "summary": "The paper proposes TopoDiffusionNet, a method to control some topological properties of 2D image generation, such as the number of distinct objects and separated regions.\n\nTopoDiffusionNet is trained to take as input a condition $c$, representing the number of the structures, and to output a mask with exactly $c$ separate objects or regions. The network is optimized with a loss based on persistent homology; after computing the persistent diagram, the features are separated into \"preserve\" (the $c$ features with higher persistence), and \"noise\" (all the others). The network is trained to promote the first while suppressing the others. The generated mask can be used as a condition of a 2D generative ControlNet.\n\nThe method is tested on four datasets (Shapes, COCO, CREMI, Google Maps), showing significantly more control than previous methods.", "questions": "Please refer to the weaknesses of the main points to clarify.\n\nSome minor curiosities are:\n\n1) Would it be possible to combine 0-dim and 1-dim constraints?\n2) Does the computation of persistent homology have an impact on the computational performance of the method?\n3) While the method provides control only on the number $c$, is it possible to tune homology parameters to promote also bigger or smaller components?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The posed problem is interesting, and it is relevant for a wide community. Defining new ways to control generation and especially incorporating this knowledge into the diffusion process itself provide interesting insights. The paper is also informative and well-organized: The text is well-written, and it is pretty straightforward. The proposed methodology is clear, and the figures are quite insightful. Experiments consider diverse datasets and control results are convincing.", "weaknesses": "My main concern is that the paper does not offer much insight into the trade-off between control and quality. It is pretty natural to wonder how the control impacts the quality of the generation compared with other methods. The qualitative examples shown seem ok, but it is also evident that the quality is quite degraded compared to alternatives -- this might be a cause of the generative 2D ControlNet itself or that the control does not incorporate enough domain knowledge and provide a mask difficult to respect (e.g., the mask of cats and birds do not look realistic, and I guess it requires quite some work for the ControlNet generation to compensate) . The only FID score is reported in an ablation study on the Shape dataset, which is not particularly insightful. I think reporting (at least) the same metric for other datasets and comparing it with other methods would be important to assess how much the provided approach destroys the generative capability of the generative backbone.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TYjhgbb4io", "reviewer": "Reviewer_GDUJ", "summary": "This paper proposes a procedure for conditioning text-to-image diffusion models on topological constraints. In particular, an auxiliary diffusion model is trained that generates binary segmentation masks that conform to a specified zeroth and first homology ranks (i.e., the number of connected components and holes). This is done both by providing the Betti numbers as input conditions to the network and by imposing a loss on the homology of the predicted image. Because the predicted images at early timesteps are very noisy, an approximation of the denoised image is computed, and the persistence module of that image is what is used in the loss objective.", "questions": "It would be very helpful if the authors could respond to my two points noted above---the second one in particular. At the very least, some simple ablation studies can be conducted to help justify the overall algorithm. For instance, what if the conditioning masks simply contain circles or other basic shapes for the desired objects (even in the case of animals)? Would the ControlNet still understand the object type from the text prompt and use the simplified mask as the cardinality/topology conditioning?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Maintaining desired compositionality is a key challenge for diffusion models, and getting a model to output the desired quantity of discrete objects is often difficult. Framing this problem through the lens of persistent homology makes a lot of sense and ties nicely to the difficulty of dealing with noisy samples at intermediate timesteps. Additionally, being able to condition on first homology is also a nice feature, albeit it seems a bit more contrived (and indeed the main demonstrated use case is in generating aerial road images).", "weaknesses": "My main concerns with this paper have to do with the overall novelty as well as the overall justification for the approach.\n\nWhile this paper is indeed the first to consider diffusion-based image generation through the lens of TDA and persistent homology, the actual machinery proposed to accomplish this is very similar to prior work that combines TDA with deep learning --- a key insight is differentiating through the persistence computation, but this is not unique to the diffusion setting. I do agree that the observation that the a persistence module can well represent the homology of a noisy $x_0$ approximation is interesting, but this brings me to my second concern, which is that I am not convinced by the results in this paper that training a diffusion model is critical to the success of this method.\n\nOne of the main reasons for the success of image diffusion models is their stability and performance in generating high-quality complex natural images. The masks produced by the diffusion model in this paper, however, are quire primitive and lack detail or high-frequency detail. I wonder if a much simpler generative model, e.g., a VAE, could be just as effective at producing these masks, that could then equally serve as input to a ControlNet. Moreover, I am not convinced that an ML model is necessary to generate these mattes in the first place. Given that the only conditions that the model accepts are the Betti numbers and an object class (nothing more about the actual layout can be specified), it seems to me that such masks could be procedurally generated to obey the desired topology. In fact, looking at Figure 6, it appears that the object class condition is superfluous for the mask model, given that it also appears in the ControlNet prompt. If we look, e.g., at the birds (bottom right), the contour of the second bird from the right is completely different from the input mask. This suggests that the ControlNet largely ignores any higher-frequency context in the mattes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HUcYmPhJOA", "reviewer": "Reviewer_veux", "summary": "This paper proposes a generative method for images that can be constrained in its resulting topology - number of objects, as well as the connectivity between objects. The core idea is to use persistent homology which essentially constructs, for each image produced in a diffusion process, a 3D embedding is created by marking all pixels that exceed each z>0 - this enables the authors to produce a topological signature for the image (for different z's) which enables to ignore noisy structures and evaluate the topological invariants of the resulting clean image. The authors then devise a simple loss that uses this information and regularize the neural network to respect the topological constraints.", "questions": "- Empirically (not in theory), can your code generate more complex specific shapes by choosing the right topological invariants as input?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I am not an expert in image generation, but I like the idea of adding topological constraints to image generation. It could be very interesting to be able to accurately control resulting images' topology.  I found the approach to be rather straightforward and thus easy to understand and implement. Considering I could not find a prior work that aims to achieve the same result, I find this approach novel.", "weaknesses": "Again, I am not an expert in image generation, however it seems to me that in the end, the paper focuses mainly on number of objects and less so on more important topological invariants such as connectivity. There are many interesting tests one could cook up, e.g., choose Betti numbers that force a very specific topology (double annulus) and show the method can achieve that. These more interesting topological invariants are also not directly measured in the quantitative evaluations, as far as I understand. I am also uncertain how far can this type of topological control be effective for practical uses, beyond, again, the trivial case of controlling number of objects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "38nNeE4OMk", "reviewer": "Reviewer_mfGQ", "summary": "This paper proposes a method to control the topology of images generated by the diffusion model. By combining the persistent homology theory, a topological mask is generated with noise, and it is fed to the diffusion model as a condition together with the time steps, achieving more accurate topological control of image generation.", "questions": "In Figure 4, the lower part of \\alpha_3 seems to be cut off by a straight line? Is it because it is a cube? If so, why doesn't it appear on the other three boundaries?\n\nAlthough I know this is not the task of this paper, it would be nice if higher-dimensional topological control, such as 3D, could be discussed as future work.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors claim that they are the first to combine topological and diffusion models on image generation. Although I cannot be sure of this, I think it is indeed a problem that needs to be solved urgently, and this paper provides a possible solution to it.\n\nThe combination of persistent homology and diffusion modeling is new to me, and although the combination way is simple, it appears to be effective.\n\nThis paper is well written, with rich comparative data and ablation experiments on corresponding parameters.", "weaknesses": "Should authors explain in more detail how persistent homology is calculated? I think the part about how it is integrated into the diffusion model and how the loss between the two sets of P and Q defined is well explained, but the specific calculation of persistent homology seems a bit lacking in details.\n\nThe persistence diagram could be more explicit, for example by using different point sizes and colors to show which P sets are more important.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)