TopoDiffusionNet: A Topology-aware Diffusion Model

## Abstract

Abstract Diffusion models excel at creating visually impressive images but often struggle to generate images with a specified topology . The Betti number, which represents the number of structures in an image, is a fundamental measure in topology. Yet, diffusion models fail to satisfy even this basic constraint. This limitation restricts their utility in applications requiring exact control, like robotics and environmental modeling. To address this, we propose TopoDiffusionNet (TDN), a novel approach that enforces diffusion models to maintain the desired topology. We leverage tools from topological data analysis, particularly persistent homology, to extract the topological structures within an image. We then design a topology-based objective function to guide the denoising process, preserving intended structures while suppressing noisy ones. Our experiments across four datasets demonstrate significant improvements in topological accuracy. TDN is the first to integrate topology with diffusion models, opening new avenues of research in this area.

## 1 Introduction

page\cref@result
Over the past few years, diffusion models have become prominent for image generation tasks (Sohl-Dickstein et al., ; Song & Ermon, ; Ho et al., ; Song et al., , ; Nichol & Dhariwal, ; Dhariwal & Nichol, ). Naturally, text-to-image (T2I) diffusion models are a popular choice for creating high-quality images based on textual prompts (Saharia et al., ; Rombach et al., ; Avrahami et al., ; Ruiz et al., ; Nichol et al., ; Kim et al., ; Ramesh et al., ; ; OpenAI, ). Despite their ability to generate visually impressive images, T2I models are still far from the desired intelligence level. In particular, they often struggle to interpret textual prompts that involve basic reasoning and logic.
This includes preserving global and semantic constraints, such as consistent number of objects as well as structural patterns (e.g., enclosed regions or loops).
Improving this capability would be a step forward in the control and precision of diffusion models, moving beyond qualitative attributes such as style and texture.

Topology, in a general sense, defines how different parts of an image interact with each other, dictating their overall layout within an image. Preserving topology is essential for generating images that not only look realistic but also adhere to correct semantics. The simplest measure in topology is the Betti number, which is equivalent to the number of connected components (0-dimension) and holes/loops (1-dimension). In natural images, 0-dimensional topology corresponds to the number of distinct objects, while 1-dimensional topology refers to the number of enclosed regions. Yet, current diffusion models fail to preserve even these basic topological properties. This is particularly evident in applications such as urban planning, robotics, and environmental modeling, where it is crucial to generate scenes with a specific topology or number of entities (like animals or road intersections). \creffig:teaser(a-b) shows instances where popular T2I diffusion models fail to generate images with the specified topology, such as specific numbers of animals, or holes/regions in road layouts.

Recognizing the limited spatial reasoning of T2I models, existing methods use spatial maps (such as object masks, edge maps, etc) to control the generated images (Zhang et al., ; Bar-Tal et al., ; Bashkirova et al., ; Huang et al., ; Mou et al., ; Li et al., ). These methods have indeed shown promise, offering a more guided approach to image generation that aligns closely with the provided controls. Nonetheless, generating the spatial map itself is a bottleneck: an automatic way to generate spatial maps with a desired topology is an unaddressed problem.

In this work, we address the challenge of generating topologically faithful images using diffusion models. Our focus is on generating images with a specific topology, defined by properties such as the number of connected components (0-dimension) or loops/holes (1-dimension). These topological structures, quantified by Betti numbers, serve as our topological constraints. Spatial maps such as masks have shown success in guiding the semantics of the generated image. Leveraging this, we propose using diffusion models to automate generating masks that satisfy the desired topological constraint. A straightforward approach is to condition the diffusion model on the constraint information and fine-tune it on enough samples. However, as shown in \creffig:teaser(d), we find that conditioning on the constraint alone falls short of effectively preserving the topology of the generated masks.

Figure: Figure 1: Comparison of existing diffusion models in preserving topological constraints. Top row: 0-dim topological constraint to generate exactly five giraffes. Bottom row: 1-dim topological constraint to generate a road layout with exactly four distinct regions. Text-to-image methods like (a) Stable Diffusion (Rombach et al., ) and (b) DALL·E 3 (OpenAI, ) struggle to respect both 0-dim and 1-dim constraints. (c) Attention Refocusing (AR) (Phung et al., ) requires bounding boxes for each object but struggles with higher object counts and often creates partitioned images. (d)-(e) shows a two-step process: mask generation followed by ControlNet (Zhang et al., ) rendering. (d) ADM-T generates masks by fine-tuning ADM (Dhariwal & Nichol, ) with the topological constraint as a condition, but this alone is insufficient. (e) Our TopoDiffusionNet, trained with a topology-based objective function, generates masks with the precise number of objects or regions, which when fed to ControlNet generates the desired image of five giraffes (top row) and four regions (bottom row). Giraffe/region counts are noted in the bottom-right inset of each image.\cref@constructprefixpage\cref@result
Refer to caption: x2.png

We propose TopoDiffusionNet (TDN), a novel approach that incorporates topology to guide the mask generation process. To ensure the final mask satisfies the topological constraint, a topology-aware objective function is necessary for steering the denoising process so that each timestep is one step closer to preserving the desired topological constraint. However, designing such a function is not straightforward. The intermediate timesteps are very noisy – especially at larger timesteps – so extracting meaningful information from them is challenging. We thus need tools that are robust to noise. This leads us to persistent homology (Edelsbrunner et al., ; Edelsbrunner & Harer, ), a mathematical theory that can, amidst the noise, extract the topological structures within an image. Using persistent homology, we can partition an image in terms of topology: separating out the significant structures from the noisy ones. We can thus design a dedicated objective function to preserve the significant structures and suppress the rest. The function guides the denoising process to progress in such a way so as to ultimately preserve the topology at the final timestep, as we see in \creffig:teaser(e). In summary, our contributions are as follows:

- •
To the best of our knowledge, we are the first to integrate topology with diffusion models to address topologically faithful image generation in both 0-dimension and 1-dimension. Specifically, we generate spatial maps (masks) to tackle the challenge of generating an image with a specific number of structures.
- •
We present TopoDiffusionNet (TDN), which utilizes a topology-based objective function to improve diffusion models’ ability to follow simple topological constraints. It serves as a denoising loss, guiding the diffusion denoising process in a topology-aware manner.
- •
We evaluate TDN on four datasets to demonstrate its versatility and effectiveness. TDN exhibits large improvements in maintaining the topological integrity of the generated image.

The success of our method suggests a surprising harmony between diffusion models and topology. Diffusion models are trained to denoise but are rather hard to control for preserving global semantics. Meanwhile, topological methods such as persistent homology provide a principled solution to extract global structural information from a noisy input, and can thus successfully guide the diffusion model. We hope the coupling of diffusion models and topology, as well as the techniques developed in this paper, will shed light on more sophisticated control of these generative models in the near future.

## 2 Related Work

page\cref@result

### 2.1 Diffusion models

Diffusion models, first introduced by Sohl-Dickstein et al. (), are now prevalent in image generation (Ho et al., ; Song et al., , ; Nichol & Dhariwal, ; Dhariwal & Nichol, ), evolving from unconditional models, to conditional models using class labels (Ho & Salimans, ; Dhariwal & Nichol, ), and later to text-to-image models (Saharia et al., ; Rombach et al., ; Avrahami et al., ; Ruiz et al., ; Nichol et al., ; Kim et al., ; Ramesh et al., ; ; OpenAI, ). However, textual prompts have limitations in controlling spatial composition, like layouts and poses. Recognizing this, several works propose to use spatial maps (such as masks, edge maps, etc) as condition to guide the image generation process (Zhang et al., ; Qin et al., ; Zhao et al., ; Bar-Tal et al., ; Bashkirova et al., ; Huang et al., ; Mou et al., ). These approaches aim to overcome the limitations of text-based conditioning by providing more explicit spatial guidance.

### 2.2 Numeric control in diffusion models

Betti numbers quantify topological structures, such as the number of connected components (0-dimension) or loops/holes (1-dimension), which is conceptually related to the task of counting. Recent works have explored approaches to enhance the counting performance in diffusion models. Paiss et al. () enhances CLIP’s (Radford et al., ) text embeddings for counting-aware text-to-image (T2I) generation via Imagen (Saharia et al., ). While this results in some improvement, the performance is still limited, supporting our motivation that T2I models often struggle with textual prompts involving semantic reasoning. Layout-based methods (Chen et al., ; Phung et al., ; Farshad et al., ) use layout maps, that is, maps containing bounding boxes of each object/entity, to guide the reverse diffusion process. While these methods show promise, they are not scalable as their complexity increases with the number of objects. Furthermore, focusing on these boxes often results in images that appear partitioned, as shown in \creffig:teaser(c) top row, where the backgrounds of each giraffe differ.
Finally, all of the methods mentioned above are limited to 0-dimensional topological structures (i.e., discrete objects) and do not extend to higher-dimensional topological constraints.

### 2.3 Deep learning with topology

Methods from algebraic topology, under the name of topological data analysis (TDA) (Carlsson, ), have found use in various machine learning problems owing to their versatility (handling data such as images, time-series, graphs, etc.) and robustness to noise. The most widely-used tool from TDA, persistent homology (PH) (Edelsbrunner et al., ; Edelsbrunner & Harer, ), has been applied to several image classification (Peng et al., ; Wang et al., ; Du et al., ; Hofer et al., ; Chen et al., ) and segmentation tasks (Abousamra et al., ; Clough et al., ; Hu et al., ; Clough et al., ; Stucki et al., ; Byrne et al., ; He et al., ) as it can track topological changes at multiple intensity values. Other TDA theories like discrete Morse theory (Dey et al., ; Hu et al., , ; Gupta et al., ; Banerjee et al., ), topological interactions (Gupta et al., ), and center-line transforms (Shit et al., ; Wang et al., ), have also enhanced performance in these areas.

In the realm of generative models, TDA has been used with generative adversarial networks (GANs) (Goodfellow et al., ) to evaluate performance through topology comparisons (Khrulkov & Oseledets, ), and enhance image quality via topological priors (Brüel-Gabrielsson et al., ) and PH-based loss functions (Wang et al., ). These methods have mainly focused on quality enhancement without directly controlling the topological characteristics of the images. In the case of diffusion models, incorporating TDA has not yet been explored. Our work, TopoDiffusionNet, represents a unique effort in this direction, employing PH within diffusion models to control the topology of the generated images. This is a significant shift from existing applications of TDA in generative models, moving beyond quality improvement to precise topological control.

## 3 Methodology

page\cref@result

Figure: Figure 2: Illustration of topological structures.\cref@constructprefixpage\cref@result
Refer to caption: x3.png

Given a topological constraint $c$, our goal is to generate a mask containing $c$ number of structures. A structure can either correspond to an object or a hole/region. We illustrate these structures in \creffig:holescc. In (a), the four objects are in white. In (b), the four holes correspond to the four black regions/partitions the white lines create with the border. In the formal language of algebraic topology (Munkres, ), $c$ is the Betti number, that is, it is the rank of the homology group, in which objects (connected components) correspond to the 0-dimensional (0-dim) homology classes and holes/loops/regions correspond to the 1-dimensional (1-dim) homology classes.

To enforce the topological constraint $c$, the diffusion model needs to be conditioned on it. However, $c$ alone is not sufficient to control the topology of the final generated image(^1^11In this section, we use ‘image’ to mean the mask.). Therefore, we propose a topology-based objective function $\mathcal{L}_{\text{top}}$ to guide the reverse denoising diffusion process at each timestep during training. $\mathcal{L}_{\text{top}}$ uses persistent homology (Edelsbrunner et al., ; Edelsbrunner & Harer, ) to distinguish between the desired and the spurious structures, aiming to enhance the former and reduce the latter to ensure the topology matches $c$ closely. \creffig:overview provides an overview.

The rest of this section is organized as follows. We briefly discuss diffusion models in \crefmethod:diff, followed by a quick background on persistent homology in \crefmethod:ph. We tie these concepts together to finally introduce our method TopoDiffusionNet (TDN) in \crefmethod:ours.

### 3.1 Diffusion Models

page\cref@result
Diffusion models (Ho et al., ) are able to sample images from the training data distribution $p(x_{0})$ by iteratively denoising random Gaussian noise in $T$ timesteps. The framework consists of a forward and a reverse process.

In the forward process, at every timestep $t\in T$, Gaussian noise is added to the clean image $x_{0}\sim p(x_{0})$ until the image becomes an isotropic Gaussian. The forward noising process is denoted by $q(x_{t}\mid x_{0})\coloneqq\mathcal{N}\left(x_{t};\sqrt{\bar{\alpha_{t}}}x_{0}
,\left(1-\bar{\alpha_{t}}\right)\bm{I}\right)$,
which can be rewritten as,

$$ $\displaystyle x_{t}=\sqrt{\bar{\alpha_{t}}}x_{0}+\sqrt{\left(1-\bar{\alpha_{t} }\right)}\epsilon\cref@constructprefix{page}{\cref@result}$ (1) $$

$\epsilon\sim\mathcal{N}(\bm{0},\bm{I})$ is a noise variable, $\bar{\alpha_{t}}$ is the noise scale at timestep $t$, and $\mathcal{N}$ is the normal distribution.

Figure: Figure 3: (a) TDN overview: We condition the diffusion model on the topological constraint $c$ (here $c=2$). During training, we first add noise $\epsilon$ to the input $x_{0}$ using the forward process (\crefeq:rforward) to obtain $x_{t}$, where $t$ is sampled uniformly. The U-Net is trained as part of the reverse process, predicting the added noise $\epsilon_{\theta}(x_{t},c,t)$, with which we obtain the noiseless image $\hat{x}_{0}^{t}$ (\crefeq:x0approx). Alongside the standard loss $\mathcal{L}_{\text{simple}}$, we propose $\mathcal{L}_{\text{top}}$ to enforce the topological integrity of $\hat{x}_{0}^{t}$. (b) To compute $\mathcal{L}_{\text{top}}$, the persistence diagram $\mathcal{D}(\hat{x}_{0}^{t})$ captures all the topological structures in $\hat{x}_{0}^{t}$, partitioning them into salient/desired structures $\mathcal{P}$ and noisy ones $\mathcal{Q}$. Terms $\mathcal{L}_{\text{preserve}}$ and $\mathcal{L}_{\text{denoise}}$ respectively amplify $\mathcal{P}$ and suppress $\mathcal{Q}$, guiding the training to eventually satisfy $c$. \cref@constructprefixpage\cref@result
Refer to caption: x4.png

The reverse process aims to learn the posterior distribution $q\left(x_{t-1}\mid x_{t},x_{0}\right)$, using which we can recover $x_{t-1}$ given $x_{t}$. This is typically done by training a denoising neural network (U-Net (Ronneberger et al., )) with network parameters $\theta$. The denoising model $\epsilon_{\theta}(x_{t},t)$ takes the noisy input $x_{t}$ at timestep $t$ and predicts the noise $\epsilon$ added in \crefeq:rforward of the forward process. The model is trained using the simplified objective function $\mathcal{L}_{\text{simple}}$ (Ho et al., ): $\mathcal{L}_{\text{simple}}=\mathbb{E}_{t,x_{0},\epsilon}\left[\|\epsilon_{
\theta}(x_{t},t)-\epsilon\|^{2}_{2}\right]$.
In our case, we additionally provide the topological constraint $c$ as a condition to control the topology of the generated image. Thus, the denoising model becomes $\epsilon_{\theta}(x_{t},c,t)$, where $c$ is injected into the denoising neural network.

During training, although the denoising model predicts the noise $\epsilon_{\theta}(x_{t},c,t)$ at timestep $t$, we can deterministically (without iterative sampling) recover the predicted noiseless image $\hat{x}_{0}^{t}$ (an estimate of the true $x_{0}$) from \crefeq:rforward as,

$$ $\displaystyle\hat{x}_{0}^{t}=\frac{1}{\sqrt{\bar{\alpha_{t}}}}\left(x_{t}- \sqrt{1-\bar{\alpha_{t}}}\epsilon_{\theta}(x_{t},c,t)\right) \cref@constructprefix{page}{\cref@result}$ (2) $$

This alternate form of the prediction will enable us to compute $\mathcal{L}_{\text{top}}$ as we will see in \crefmethod:ours.

### 3.2 Persistent Homology

page\cref@result
Persistent homology (PH) (Edelsbrunner et al., ; Edelsbrunner & Harer, ), owing to its differentiable nature, is an attractive candidate for integrating topological information into the training of deep learning methods. In the case of image data, it can detect the changes in topological structures (connected components and holes) across a varying threshold (also called the filtration value). More importantly, persistent homology is robust to noise, that is, it can extract these structures even in noisy scenarios. Structures that exist for a wide range of thresholds are salient, while the remaining structures are deemed as noise in the image.

In our setting, during training, we can apply persistent homology to every intermediate image $\hat{x}_{0}^{t}$ (from \crefeq:x0approx) predicted by the diffusion model at timestep $t$. For ease of reference, we denote $\hat{x}_{0}^{t}$ by $I$, having size $h\times w$. In practice, $I$ has continuous probability values in a normalized range, say, $[0,1]$(^2^22The range is typically $[-1,1]$ in implementation.). We now consider super-level sets of $I$, i.e., the set of pixels $(i,j)$ for which $I_{ij}$ is above some threshold value $u$. Let $\mathcal{S}$ denote the super-level set, then, $\mathcal{S}(u)\coloneqq\{(i,j)\in[1,h]\times[1,w]\mid I_{ij}\geq u\}$.
This is nothing but thresholding, and we call the resulting binary image the super-level set at $u$, $S(u)$. Decreasing $u$ continuously generates a sequence of sets, i.e. a filtration, which grows as the threshold parameter $u$ is brought down: $\varnothing\subseteq\mathcal{S}(1)\subseteq\mathcal{S}(u_{1})\subseteq\mathcal
{S}(u_{2})\subseteq\cdots\subseteq\mathcal{S}(0)=[1,h]\times[1,w]$. We demonstrate this filtration in \creffig:ph.

When $u$ is high, only a few pixels can exceed the threshold, and hence the size of $\mathcal{S}(1)$ is small (an almost black image). As $u$ decreases, new pixels join the set, and topological structures in $\mathcal{S}$ are created and destroyed. Eventually, at $u=0$, the entire image is in the super-level set. In this manner, persistent homology can track the evolution of all the topological structures.

The output of the persistent homology algorithm includes the birth and death threshold values for each topological structure. We can keep track of the birth (creation) $b$ and death (destruction) $d$ thresholds of all the topological structures and put the tuples $(b,d)$ in a diagram – the persistence diagram $\mathcal{D}$ – where the $y$-axis represents birth and the $x$-axis death.(^3^33If using a sub-level instead of super-level set filtration, the diagram would have the $x$-axis as birth, and the $y$-axis as death.) The persistence diagram is thus a graphical representation of topological structures throughout the filtration process, consisting of multiple dots in a 2-dimensional plane (see \creffig:ph). These dots are called persistent dots, where each dot corresponds to one topological structure. The persistence, or the lifetime of a structure, is given by the difference between its death and birth times. Structures that persist over a wide range of thresholds are considered significant or salient, indicating stable and prominent structures within the image, while short-lived structures are the noise. The diagonal $b=d$ represents structures of zero persistence and dots far from the diagonal represent salient structures with high persistence.

With this information, we can identify noisy structures due to their low persistence and proximity to the diagonal, allowing us to filter them out and retain the salient topological structures. The number of salient structures we retain is precisely the number of structures we desire in the final image. As we show in the next subsection, we compute the persistence diagram of the predicted image $\hat{x}_{0}^{t}$ to optimize it from a topological perspective.

Figure: Figure 4: Illustration of persistent homology and persistence diagrams of both types of topological structures, 0-dim connected components and 1-dim holes. (a) Despite the noise, we can visually see three prominent structures $\alpha_{1},\alpha_{2},\alpha_{3}$ in $I$. In the topological space, $\alpha_{1},\alpha_{2},\alpha_{3}$ thus appear in the top-left corner of the persistence diagram $\mathcal{D}(I)$, persisting through most of the filtration $\mathcal{S}$. Similarly in (b), $\beta_{1},\beta_{2},\beta_{3}$ denote the prominent holes. All the remaining connected components and holes are noisy, persisting over a short threshold in $\mathcal{S}$, thus appearing closer to the diagonal in $\mathcal{D}(I)$. Persistence diagrams are useful to distinguish between salient and noisy structures in an image.\cref@constructprefixpage\cref@result
Refer to caption: x5.png

Figure: Figure 5: Illustration of $\mathcal{L_{\text{preserve}}}$ and $\mathcal{L_{\text{denoise}}}$ for 0-dim connected components and 1-dim holes, with $c=2$ as seen in $x_{0}$. (a) After computing $\mathcal{D}(\hat{x}_{0}^{t})$, we partition it into sets $\mathcal{P}$ (the top $c$ structures) and $\mathcal{Q}$ (remaining ones). For each dot $p\in\mathcal{D}(\hat{x}_{0}^{t})$, the birth and death values respectively correspond to local maxima $m_{p}$ and saddles $s_{p}$ in $\hat{x}_{0}^{t}$. In the terrain view of $\hat{x}_{0}^{t}$, structures $(m_{1},s_{1})$ and $(m_{2},s_{2})$ belong to $\mathcal{P}$; hence optimizing $\mathcal{L_{\text{preserve}}}$ increases their saliency by increasing $\hat{x}_{0}^{t}(m_{1}),\hat{x}_{0}^{t}(m_{2})$ and decreasing $\hat{x}_{0}^{t}(s_{1}),\hat{x}_{0}^{t}(s_{2})$. All the remaining $n$ structures $(m_{3},s_{3}),(m_{4},s_{4}),\cdots,(m_{n},s_{n})$ belong to $\mathcal{Q}$. Optimizing $\mathcal{L_{\text{denoise}}}$ suppresses these noisy structures by decreasing $\hat{x}_{0}^{t}(m_{3}),\hat{x}_{0}^{t}(m_{4}),\cdots,\hat{x}_{0}^{t}(m_{n})$ and increasing $\hat{x}_{0}^{t}(s_{3}),\hat{x}_{0}^{t}(s_{4}),\cdots,\hat{x}_{0}^{t}(s_{n})$. (b) mirrors this process for holes, where $\mathcal{L_{\text{preserve}}}$ enhances the saliency of the two holes $(m_{1},s_{1})$ and $(m_{2},s_{2})$, and $\mathcal{L_{\text{denoise}}}$ suppresses the appearance of all the remaining holes like $(m_{3},s_{3})$.\cref@constructprefixpage\cref@result
Refer to caption: x6.png

### 3.3 Proposed TopoDiffusionNet (TDN)

page\cref@result
Conditioning. We condition the diffusion model on $c$ to enable it to generate a mask containing exactly $c$ number of structures.
We follow Nichol & Dhariwal () to inject the condition information. We first obtain an embedding of $c$ from a trainable network composed of a few linear layers. Next, we inject the embedding through the same pathway as the timestep embedding of $t$. Consequently, both embeddings are passed to residual blocks throughout the denoising model.

Objective function $\mathcal{L}_{\text{top}}$. Conditioning alone is not sufficient to control the topology of the generated image. To address this, we introduce $\mathcal{L}_{\text{top}}$, a topology-based objective function to force the predicted image at every timestep to preserve $c$ as closely as possible. Since the diffusion model is parameterized to predict in the noise space, directly analyzing topology from this noise is not meaningful. We need to map the prediction from the noise space back to the image space in order to infer the topology. From \crefeq:x0approx, we obtain $\hat{x}_{0}^{t}$ – an estimate of the noiseless image $x_{0}$ at timestep $t$ – from $\epsilon_{\theta}(x_{t},c,t)$. The estimate $\hat{x}_{0}^{t}$ is noisy, especially when $t$ is large, making it ideal to use the theory of persistent homology to separate out its salient structures from the noisy ones.

Given the prediction $\hat{x}_{0}^{t}$, we compute its persistence diagram $\mathcal{D}(\hat{x}_{0}^{t})$ containing either 0-dim or 1-dim information based on the desired topological structure (object or regions). Recall that we desire $c$ topological structures in the predicted image. For a persistent dot $p\in\mathcal{D}$, with birth $b_{p}$ and death $d_{p}$, its persistence value, $\lvert b_{p}-d_{p}\rvert$, measures its significance, according to the theory. We rank all dots in $\mathcal{D}$ by their persistence values in descending order. The top $c$ dots are the structures we aim to preserve, reflecting our desired topology, whereas the rest denote noisy structures to be suppressed/denoised. Thus, we decompose the diagram $\mathcal{D}$ into two disjoint sets, $\mathcal{D}=\mathcal{P}\dot{\bigcup}\mathcal{Q}$, where $\mathcal{P}$ contains the $c$ largest persistence dots ($\lvert\mathcal{P}\rvert=c$), while $\mathcal{Q}$ contains all the remaining dots.

To constrain $\hat{x}_{0}^{t}$ to have $c$ structures in the image space, in the topological space we need to maximize the persistence or saliency of the dots $p\in\mathcal{P}$, and suppress all the noisy dots $p\in\mathcal{Q}$. To achieve this, we introduce two loss terms:

$$ $\displaystyle\mathcal{L}_{\text{preserve}}=-\sum_{p\in\mathcal{P}}\lvert b_{p} -d_{p}\rvert^{2}$ $\displaystyle\hskip 14.22636pt\text{and}\hskip 14.22636pt\mathcal{L}_{\text{ denoise}}=\sum_{p\in\mathcal{Q}}\lvert b_{p}-d_{p}\rvert^{2} \cref@constructprefix{page}{\cref@result}$ (3) $\displaystyle\mathcal{L_{\text{top}}}$ $\displaystyle=\mathcal{L}_{\text{preserve}}+\mathcal{L}_{\text{denoise}} \cref@constructprefix{page}{\cref@result}$ (4) $$

Minimizing $\mathcal{L_{\text{top}}}$ is equivalent to maximizing the saliency of the top $c$ structures (via $\mathcal{L_{\text{preserve}}}$) and suppressing the saliency of the rest (via $\mathcal{L_{\text{denoise}}}$). In the ideal case, $\mathcal{L_{\text{top}}}=-c$, as each of the top $c$ structures will have a persistence of 1, while all the noisy structures will have zero persistence. The image $\hat{x}_{0}^{t}$ will then have exactly $c$ topological structures as desired. In the absence of $\mathcal{L_{\text{top}}}$, if the denoising process were to originally result in $>c$ structures, now $\mathcal{L_{\text{denoise}}}$ will suppress all the extra/noisy structures, preventing them from appearing in the final clean image. One concern is whether there could be less than $c$ structures in total. In practice, at large timesteps $t$, $\hat{x}_{0}^{t}$ always has several thousand noisy topological structures. Thus, if the denoising process were to originally proceed with $<c$ structures, $\mathcal{L_{\text{preserve}}}$ will now increase the persistence of a less salient dot to ensure the final image has exactly $c$ structures.

Implementation and differentiability. For every topological structure, the birth and death values $b$ and $d$ correspond to function values of a maximum-saddle pair; $b$ and $d$ are function values of a local maximum $m$ and a saddle point $s$, respectively. These pairs can be determined by the almost linear union-find algorithm (Edelsbrunner & Harer, ; Ni et al., ) which locates and pairs all local maxima and saddle points to reflect the birth and death of topological structures. For every persistent dot $p\in\mathcal{D}$, let $m_{p}$ and $s_{p}$ respectively denote the 2D coordinates of the corresponding local maximum and saddle point in the prediction $\hat{x}_{0}^{t}$. Then, \crefeq:ver-1 and \crefeq:toploss-1 can be rewritten as,

$$ $\displaystyle\mathcal{L_{\text{top}}}=-\sum_{p\in\mathcal{P}}\lvert\hat{x}_{0} ^{t}(m_{p})-\hat{x}_{0}^{t}(s_{p})\rvert^{2}+\sum_{p\in\mathcal{Q}}\lvert\hat{ x}_{0}^{t}(m_{p})-\hat{x}_{0}^{t}(s_{p})\rvert^{2}\cref@constructprefix{page}{\cref@result}$ (5) $$

We illustrate this in \creffig:losswork. During training, for every topological structure to preserve, i.e., $p\in\mathcal{P}$, the function $\mathcal{L_{\text{preserve}}}$ increases the intensity value $\hat{x}_{0}^{t}(m_{p})$ at the local maximum $m_{p}$ and decreases $\hat{x}_{0}^{t}(s_{p})$ at the saddle point $s_{p}$. This strengthens the saliency of the desired structures. At the same time, to prevent exceeding $c$ structures in the final image, $\mathcal{L_{\text{denoise}}}$ suppresses structures $p\in\mathcal{Q}$ by reducing the intensity value $\hat{x}_{0}^{t}(m_{p})$ at the local maximum $m_{p}$ while increasing $\hat{x}_{0}^{t}(s_{p})$ at the saddle point $s_{p}$. $\mathcal{L_{\text{denoise}}}$ forces $\hat{x}_{0}^{t}(m_{p})$ to be equal to $\hat{x}_{0}^{t}(s_{p})$, leading to a homogenous region. This effectively eliminates the noisy structure, as it was neither born nor died, rendering it non-existent.

With this, we see that $\mathcal{L}_{\text{top}}$ is differentiable, as \crefeq:toploss-2 is written as polynomials of the prediction $\hat{x}_{0}^{t}$ at certain pixels. From \crefeq:toploss-2 we can compute the gradient of $\mathcal{L}_{\text{top}}$ with respect to $\hat{x}_{0}^{t}$, and via chain rule, we can ultimately compute the gradient with respect to the denoising model’s parameters $\theta$. The training optimization adjusts $\theta$ to ensure that the topological space, i.e. the persistence diagram $\mathcal{D}(\hat{x}_{0}^{t})$, has exactly $c$ persistent dots, in turn resulting in $c$ structures in the image space $\hat{x}_{0}^{t}$.

End-to-end training.
The overall training objective $\mathcal{L_{\text{total}}}$ of TDN is formulated as: $\mathcal{L_{\text{total}}}=\mathcal{L}_{\text{simple}}+\lambda\mathcal{L}_{
\text{top}}$
where $\lambda$ is the loss weight. The standard denoising loss $\mathcal{L}_{\text{simple}}$ produces visually good results, whereas $\mathcal{L}_{\text{top}}$ helps respect the topological constraint $c$.

## 4 Experiments

page\cref@result
Datasets. We train ADM-T and TDN on four datasets: Shapes, COCO (Caesar et al., ), CREMI (Funke et al., ), and Google Maps (Isola et al., ). The Shapes dataset is a synthetic dataset created by us that contains objects such as circles, triangles, and/or rectangles. For CREMI, COCO, and Google Maps, we train the diffusion models on their segmentation masks.
For COCO, we select masks which contain at least one instance of the super category ‘animal’. For COCO and Shapes, we use 0-dim, the number of connected components, as the topological constraint. For COCO, we also use the animal class as a condition to generate masks of specific animals. CREMI is an Electron Microscopy dataset, and Google Maps contains aerial photos from New York City. For CREMI and Google Maps, we use 1-dim, the number of holes, as the topological constraint. Each of the datasets contains masks consisting of up to ten structures. More details are in \crefsupp:dataset.

Baselines. Stable Diffusion (Rombach et al., ) and DALL·E 3 (OpenAI, ) are popular T2I diffusion models. Attention Refocusing (AR) (Phung et al., ) uses layout maps (bounding boxes for each object) generated by GPT-4 (Achiam et al., ) to guide the reverse process.

Implementation details. Our work extends the ADM (Dhariwal & Nichol, ) diffusion model. We use ‘ADM-T’ to denote the modification of using a topological constraint as a condition. We obtain an embedding of the constraint using an encoding network. Following the approach in Nichol & Dhariwal (), we then feed this embedding to all the residual blocks in the network by adding it to the timestep embedding. For COCO, we similarly inject animal class embedding to further control the generated mask. For every dataset, we use $256\times 256$ as the image resolution. Our diffusion models use a cosine noise scheduler (Nichol & Dhariwal, ), with $T=1000$ timesteps for training. During inference, however, we use only $50$ steps of DDIM (Song et al., ) sampling. For the ADM-T baseline, we load a pretrained checkpoint (OpenAI, ) and then fine-tune on our datasets using the constraint information as condition. For TDN, we follow the same approach but additionally use $\mathcal{L}_{\text{top}}$ in the training. More details are listed in \crefsupp:impl.

Evaluation metrics. To evaluate whether the generated image satisfies the input constraint, we use metrics such as Accuracy, Precision, and F1. We report the mean and standard deviation of the results across different constraint values. To measure the performance, for 0-dim, we generate $50$ samples per constraint $c\in[1,10]$ per animal/shape category (resulting in 5K images for COCO).
We similarly generate $50$ images per constraint $c\in[1,10]$ for the 1-dim datasets. We perform the unpaired t-test (Student, ) (95% confidence interval) to determine the statistical significance of the improvement. In all the tables, performances that are statistically significantly better are in bold.

Figure: Figure 6: Qualitative results for 0-dim topological constraint. Row 1: Shapes dataset; Rows 2-4: COCO dataset. ADM-T and TDN take the constraint as the condition (purple box), and also the animal class for COCO. Stable Diffusion, DALL·E 3, and AR take the equivalent text prompt as input. Object/animal counts are noted in the bottom-right inset of each image/mask.\cref@constructprefixpage\cref@result
Refer to caption: x7.png

Figure: Figure 7: Qualitative results for the 1-dim topological constraint. ADM-T and TDN take the constraint (in the purple box) as the condition. Columns 1-2: CREMI. Columns 3-4: Google Maps. Number of holes within each mask is noted in its bottom-right inset. \cref@constructprefixpage\cref@result
Refer to caption: x8.png

### 4.1 Results

page\cref@result

Qualitative and quantitative results for 0-dim. In \creffig:qual-0dim and \creftab:quant, we present qualitative and quantitative comparisons, respectively, of pretrained Stable Diffusion, DALL·E 3, AR, ADM-T, and our proposed TDN. In \crefsupp:cwise, we provide constraint-wise results. As ADM-T and TDN produce masks, we employ pretrained ControlNet (Zhang et al., ; ) to create textured images from these masks. In practice, any of the methods (Qin et al., ; Zhao et al., ; Bar-Tal et al., ; Bashkirova et al., ; Huang et al., ; Mou et al., ) could also be used for this purpose, but we chose ControlNet for its simplicity. From \creffig:qual-0dim, we see that T2I models Stable Diffusion and DALL·E 3 are unable to respect the explicit topological constraint of generating $c$ objects. Due to limited spatial and semantic reasoning, both methods have the lowest performance in \creftab:quant. AR uses layouts generated by GPT-4, which improves over T2I models, as seen in \creftab:quant. In general, however, GPT-4 does not have much spatial awareness and when the count is high, the bounding boxes tend to either be too small or highly overlap with each other. This leads to incorrect counts in the generated image as seen in \creffig:qual-0dim. Additionally, when the object count is high, AR often produces fragmented results, with objects isolated within assigned areas, leading to a divided and visually disjointed image.
ADM-T, despite being conditioned on the constraint $c$, also falls short of satisfying the constraint. This indicates that the constraint alone is not powerful enough to influence the global reasoning of the model. In contrast, TDN preserves the constraint better than ADM-T, as evident from the quantitative results in \creftab:quant. TDN demonstrates significant improvements across all metrics. $\mathcal{L_{\text{preserve}}}$ aims to retain at least $c$ objects, while $\mathcal{L_{\text{denoise}}}$ aims to maintain atmost $c$ objects. Thus optimizing both simultaneously via
training with $\mathcal{L_{\text{top}}}$ helps the diffusion model to preserve $c$ objects in the generated mask.

Qualitative and quantitative results for 1-dim. In \creffig:qual-1dim, we provide the qualitative comparison for ADM-T and TDN. We exclude results from Stable Diffusion, DALL·E 3, and AR as these methods are limited to generating distinct objects (0-dim topology) and currently cannot handle generating distinct holes/regions (1-dim topology). As holes are a complex topological constraint, they are challenging to describe in words, and hence the performance of such methods is limited (see \creffig:teaser and \crefsupp:qual). Similar to the 0-dim case, ADM-T struggles with the topological constraint. The quantitative results in \creftab:quant highlight that preserving 1-dim topology, which involves boundaries spanning across the mask, is more challenging than 0-dim topology. This complexity necessitates powerful global reasoning capabilities from the diffusion model, an area where ADM-T shows limited performance. However, with $\mathcal{L_{\text{top}}}$, TDN achieves substantial improvements across all metrics, with $\mathcal{L_{\text{preserve}}}$ and $\mathcal{L_{\text{denoise}}}$ both working to retain exactly $c$ holes in the generated mask.

**Table 1: Quantitative comparison on preserving the topological constraint $c$\cref@constructprefixpage\cref@result**
| Dataset | Method |  | Accuracy $\uparrow$ |  | Precision $\uparrow$ |  | F1 $\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Shapes | Stable Diffusion () |  | 0.6381 $\pm$ 0.2559 |  | 0.6660 $\pm$ 0.1759 |  | 0.6537 $\pm$ 0.2198 |
| DALL·E 3 (OpenAI, ) |  | 0.6857 $\pm$ 0.2561 |  | 0.7059 $\pm$ 0.3198 |  | 0.6956 $\pm$ 0.2649 |  |
| AR (Phung et al., ) |  | 0.7384 $\pm$ 0.2178 |  | 0.7596 $\pm$ 0.2039 |  | 0.7474 $\pm$ 0.2165 |  |
| ADM-T |  | 0.7500 $\pm$ 0.1889 |  | 0.7809 $\pm$ 0.1582 |  | 0.7651 $\pm$ 0.1210 |  |
| TDN (Ours) |  | 0.9478 $\pm$ 0.0420 |  | 0.9499 $\pm$ 0.0492 |  | 0.9488 $\pm$ 0.0370 |  |
| COCO (Animals) | Stable Diffusion () |  | 0.4686 $\pm$ 0.2361 |  | 0.5154 $\pm$ 0.1747 |  | 0.4909 $\pm$ 0.1976 |
| DALL·E 3 (OpenAI, ) |  | 0.5162 $\pm$ 0.3674 |  | 0.5491 $\pm$ 0.2724 |  | 0.5194 $\pm$ 0.3256 |  |
| AR (Phung et al., ) |  | 0.6379 $\pm$ 0.2062 |  | 0.7360 $\pm$ 0.1658 |  | 0.6611 $\pm$ 0.1851 |  |
| ADM-T |  | 0.6685 $\pm$ 0.1485 |  | 0.6917 $\pm$ 0.1079 |  | 0.6799 $\pm$ 0.1931 |  |
| TDN (Ours) |  | 0.8557 $\pm$ 0.0805 |  | 0.8670 $\pm$ 0.0636 |  | 0.8613 $\pm$ 0.0970 |  |
| Google Maps | ADM-T |  | 0.5494 $\pm$ 0.1386 |  | 0.5642 $\pm$ 0.1861 |  | 0.5567 $\pm$ 0.1185 |
| TDN (Ours) |  | 0.8318 $\pm$ 0.1159 |  | 0.8471 $\pm$ 0.1797 |  | 0.8394 $\pm$ 0.1969 |  |
| CREMI | ADM-T |  | 0.5357 $\pm$ 0.1879 |  | 0.4777 $\pm$ 0.1797 |  | 0.4881 $\pm$ 0.1571 |
| TDN (Ours) |  | 0.7785 $\pm$ 0.1901 |  | 0.8142 $\pm$ 0.1925 |  | 0.7959 $\pm$ 0.1659 |  |

Additional comparison. Paiss et al. () enhances CLIP’s text embeddings for counting and uses them to show counting-aware T2I generation via Imagen. We report the Accuracy of our TDN results in their setting in \creftab:clipcomp, showing significant performance improvement. This supports our motivation that T2I models often struggle with textual prompts involving semantic reasoning.

Effect across timesteps. While we use $50$ steps of DDIM (Song et al., ) sampling for inference, the training was conducted with $T=1000$ timesteps using DDPM (Ho et al., ). To understand the effect of $\mathcal{L_{\text{top}}}$ throughout training, we analyze intermediate results from the DDPM inference procedure. In \creffig:interm, we visualize $\hat{x}_{0}^{t}$ across different timesteps on the Shapes dataset, highlighting the impact of $\mathcal{L_{\text{top}}}$ on denoising efficiency. Notably, TDN achieves a closer approximation to the true $x_{0}$ by timestep $750$, nearly 200 timesteps before ADM-T. Furthermore, we plot the average Fréchet Inception Distance (FID) (Heusel et al., ) for $1000$ samples per timestep. The plot shows that $\mathcal{L_{\text{top}}}$ accelerates denoising and enhances the quality of $\hat{x}_{0}^{t}$ at earlier stages, working as intended. This demonstrates that $\mathcal{L_{\text{top}}}$ improves both speed and accuracy of the denoising process.

**Table 2: Additional baseline\cref@constructprefixpage\cref@result**
| Method | Accuracy $\uparrow$ |
| --- | --- |
| Paiss et al. () | 0.5018 |
| TDN (Ours) | 0.7969 |

### 4.2 Ablation Studies

To demonstrate the efficacy of TDN, we conduct ablation studies on the loss components and the effect of hyperparameter value changes. \crefsupp:abl includes an ablation study on
the ‘Encoding Network’ for the topological constraint $c$. All analyses are on 0-dim Shapes dataset.

Ablation study on $\mathcal{L_{\text{preserve}}}$ and $\mathcal{L_{\text{denoise}}}$. In \creftab:ablation-comp, we show the contribution of $\mathcal{L_{\text{denoise}}}$ and $\mathcal{L_{\text{preserve}}}$ in meeting the topological constraint. While both terms individually improve performance, $\mathcal{L_{\text{denoise}}}$ has a more significant effect. This is because $\hat{x}_{0}^{t}$, especially at larger timesteps, has several spurious structures. $\mathcal{L_{\text{denoise}}}$ works by suppressing the birth of these structures, leaving only the desired number of structures in the mask. Meanwhile, $\mathcal{L_{\text{preserve}}}$ is crucial when the model generates fewer structures than expected. Naturally, combining both terms yields the optimal performance.

Ablation study on loss weight $\lambda$. In \creftab:ablation-wt, we show experiments with different weights for $\mathcal{L}_{\text{top}}$, including the Min-SNR ($\gamma=5$) (Hang et al., ) strategy where the loss weight is a function of timestep $t$. When $\lambda=1e-5$, TDN achieves the best performance. Nonetheless, a reasonable range of $\lambda$ always results in improvement, demonstrating the efficacy and robustness of $\mathcal{L}_{\text{top}}$.

Figure: Figure 8: TDN has better FID of $\hat{x}_{0}^{t}$ at larger timesteps compared to ADM-T.\cref@constructprefixpage\cref@result
Refer to caption: x9.png

## 5 Conclusion

We propose TopoDiffusionNet, the first method to integrate topology with diffusion models. Our approach generates images that preserve topology by producing masks with a specified number of structures (Betti number). Empirical results show significant improvement in preserving this topological constraint, demonstrating that our method guides the denoising process in a topology-aware manner. This paves the way for further research on topological control in image generation.

## Appendix A Dataset Details

page\cref@result
We provide the number of images per topological constraint $c$ used for training on each dataset in \creftab:data. For COCO (Caesar et al., ), since we also consider the animal class, each animal is distributed unequally across the different constraint values. For example, there were more images for ‘birds’ having $c=10$ than compared to, say, ‘elephant.’ All the $10$ animal classes are present in the dataset; they are bear, bird, cat, cow, dog, elephant, giraffe, horse, sheep, and zebra.

When curating the dataset for CREMI (Funke et al., ) and Google Maps (Isola et al., ), we manually added a (white) border to all the images. By definition of 1-dim topology, a hole is completely surrounded by a boundary. Hence, we needed to add a border to obtain the correct number of holes/regions.

**Table 5: Dataset composition\cref@constructprefixpage\cref@result**
| Dataset | Topological Constraint (Betti Number) |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |  |
| Shapes | 2K | 2K | 2K | 2K | 2K | 2K | 2K | 2K | 2K | 2K |
| COCO (Animals) | 520 | 517 | 503 | 297 | 176 | 85 | 64 | 38 | 30 | 27 |
| Google Maps | 549 | 669 | 1099 | 1220 | 1343 | 1806 | 602 | 1470 | 1054 | 662 |
| CREMI | 2160 | 1992 | 3726 | 3505 | 1644 | 580 | 187 | 207 | 170 | 112 |

## Appendix B Implementation Details

page\cref@result
All ADM-T and TDN experiments were conducted on 1 NVIDIA RTX A6000 GPU, with a batch size of 16 and a learning rate of $2\times 10^{-5}$.
As mentioned in \crefmethod:ours, our diffusion model is parameterized to predict in noise space, and we use \crefeq:x0approx to get an estimate of the noiseless image. Although diffusion models can be parameterized to predict the noiseless state directly, we find from existing works (Hang et al., ; Wang et al., ) that their performance is poorer compared to predicting the noise. Hence we stick to the configuration of predicting the noise. This also allows us to load pretrained weights from OpenAI () for our experiments instead of training from scratch.

For training ADM-T and TDN, we use the PyTorch codebase from Dhariwal & Nichol ()(^4^44[https://github.com/openai/guided-diffusion](https://github.com/openai/guided-diffusion)) and use the LSUN Bedrooms pretrained model checkpoint (OpenAI, ) to fine-tune from. To compute the birth death pairs of each topological structure, we use the Cubical Ripser (Kaji et al., ) library. We will publicly release the code upon acceptance of the paper.

In \creffig:teaser, \creffig:qual-0dim, \creffig:qual-1dim,  \creffig:qual-1dim-supp, and \creffig:rings, the Stable Diffusion (Rombach et al., ) results are generated using the Diffusers(^5^55[https://huggingface.co/docs/diffusers/en/index](https://huggingface.co/docs/diffusers/en/index)) library with pretrained checkpoint from . For DALL·E 3, we generate images using the OpenAI API(^6^66[https://platform.openai.com/docs/api-reference/introduction](https://platform.openai.com/docs/api-reference/introduction)). For Attention Refocusing (AR) (Phung et al., ), we use their publicly available codebase(^7^77[https://github.com/Attention-Refocusing/attention-refocusing](https://github.com/Attention-Refocusing/attention-refocusing)) alongwith GPT-4 (Achiam et al., ) from the OpenAI API to generate the layout maps. For rendering images from masks via ControlNet (Zhang et al., ), we use the Diffusers library with pretrained checkpoint from . For \creffig:qual-1dim, however, we fine-tune ControlNet on the CREMI dataset (Funke et al., ) so as to generate appropriate results for the corresponding text prompt.

Figure: Figure 9: Accuracy results on low and high object counts (COCO dataset).\cref@constructprefixpage\cref@result
Refer to caption: extracted/5944606/figures/method_comparison_chart.png

Figure: Figure 10: Qualitative results for 1-dim topological constraint. Stable Diffusion, DALL·E 3 and AR take text prompts as input (purple box). Rows 1-2: Results equivalent to CREMI. Rows 3-4: Results equivalent to Google Maps. Number of holes within each image is noted in its bottom-right inset. \cref@constructprefixpage\cref@result
Refer to caption: x10.png

Figure: Figure 11: Results on standalone holes (holes not relative to the image frame/boundary). Number of donuts within each mask/image is noted in its bottom-right inset.\cref@constructprefixpage\cref@result
Refer to caption: x11.png

## Appendix C Constraint-wise Results

page\cref@result
In \creffig:quant-c, we plot the accuracy of the different methods on the COCO dataset. For smaller object counts, that is, $c<=5$, the performance of each method is better than $c>5$. At higher object counts, although the accuracy of each method reduces, TDN still significantly outperforms the baselines.

## Appendix D Additional 1-dim Qualitative Results

page\cref@result
In \creffig:qual-1dim-supp, we show qualitative results of pretrained Stable Diffusion (Rombach et al., ), DALL·E 3 (OpenAI, ) and Attention Refocusing (AR) (Phung et al., ) for the same 1-dim constraints shown in \creffig:qual-1dim of the main paper. As 1-dim topology is hard to describe in words, we tried a lot of variations for the text prompts, and show the results from the best ones in the figure.

As the pretrained Stable Diffusion, DALL·E 3, and AR are not trained on Electron Microscopy images of cell neurons, their inaccurate results are understandable. For generating roads, however, these methods do generate visually appropriate images but struggle to maintain the correct number of holes/regions. AR additionally tends to generate images that appear to be divided into separate, unconnected sections. This is due to the use of layout maps in the reverse process. All these methods are limited to generating images with 0-dim topology (i.e., distinct objects), and do not extend to 1-dim topology. This shortcoming motivates our work on TopoDiffusionNet.

## Appendix E Additional Experiments on Holes

page\cref@result
In the main paper, we show experiments on 1-dim topology using the CREMI (Funke et al., ), and Google Maps (Isola et al., ) datasets. In these datasets, it makes sense that the holes are with respect to the image frame/boundary and span the whole image. However, TDN can handle 1-dim holes in general, not just those with respect to the boundary. To demonstrate this, we conduct experiments on standalone holes (not relative to the boundary), and present the results in \creftab:rings and \creffig:rings. We generate a synthetic dataset of circular rings (similar to the Shapes dataset) to train TDN, and use the prompt ‘donuts’ in ControlNet to render the image. While generating $c$ donuts could also be achieved using the 0-dim topological constraint, this experiment highlights the 1-dim generalizability of TDN.

**Table 6: Standalone holes**
| Method | Accuracy $\uparrow$ | F1 $\uparrow$ |
| --- | --- | --- |
| ADM-T | 0.79 $\pm$ 0.12 | 0.81 $\pm$ 0.11 |
| TDN (Ours) | 0.95 $\pm$ 0.03 | 0.96 $\pm$ 0.02 |

## Appendix F Additional Ablation Study

page\cref@result

We conduct an additional ablation study apart from the ones presented in the main paper.

Ablation study of the encoding network. In TDN, we use the topological constraint $c$ as a condition. We do this by first obtaining an embedding of $c$ via an Encoding Network (\creffig:overview), and then passing it to all the residual blocks in the denoising model.

In the results reported in the main paper, the Encoding Network is composed of a few linear layers. We use ‘LL’ to denote this configuration.

Another way to configure the Encoding Network is to use the Transformer sinusoidal position embedding (Vaswani et al., ). Let ‘PE’ denote this configuration. It uses the same code as the network generating the sinusoidal timestep embedding.

We show results comparing LL and PE in \creftab:enc when using our proposed objective function $\mathcal{L}_{\text{top}}$. We find that both configurations have comparable performance, with LL slightly outperforming PE.

**Table 7: Ablation study on Encoding Network for TDN\cref@constructprefixpage\cref@result**
| Dataset | Encoding |  | Accuracy $\uparrow$ |  | Precision $\uparrow$ |  | F1 $\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Shapes | LL |  | 0.9478 $\pm$ 0.0420 |  | 0.9499 $\pm$ 0.0492 |  | 0.9488 $\pm$ 0.0370 |
| PE |  | 0.9011 $\pm$ 0.0730 |  | 0.9132 $\pm$ 0.0683 |  | 0.9033 $\pm$ 0.0385 |  |
| COCO | LL |  | 0.8557 $\pm$ 0.0805 |  | 0.8670 $\pm$ 0.0636 |  | 0.8613 $\pm$ 0.0970 |
| (Animals) | PE |  | 0.8395 $\pm$ 0.0952 |  | 0.8436 $\pm$ 0.1152 |  | 0.8411 $\pm$ 0.1014 |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Please refer to the weaknesses of the main points to clarify.

Some minor curiosities are:

1) Would it be possible to combine 0-dim and 1-dim constraints?
2) Does the computation of persistent homology have an impact on the computational performance of the method?
3) While the method provides control only on the number $c$, is it possible to tune homology parameters to promote also bigger or smaller components?
soundness: 3
strengths: The posed problem is interesting, and it is relevant for a wide community. Defining new ways to control generation and especially incorporating this knowledge into the diffusion process itself provide interesting insights. The paper is also informative and well-organized: The text is well-written, and it is pretty straightforward. The proposed methodology is clear, and the figures are quite insightful. Experiments consider diverse datasets and control results are convincing.
summary: The paper proposes TopoDiffusionNet, a method to control some topological properties of 2D image generation, such as the number of distinct objects and separated regions.

TopoDiffusionNet is trained to take as input a condition $c$, representing the number of the structures, and to output a mask with exactly $c$ separate objects or regions. The network is optimized with a loss based on persistent homology; after computing the persistent diagram, the features are separated into "preserve" (the $c$ features with higher persistence), and "noise" (all the others). The network is trained to promote the first while suppressing the others. The generated mask can be used as a condition of a 2D generative ControlNet.

The method is tested on four datasets (Shapes, COCO, CREMI, Google Maps), showing significantly more control than previous methods.
weaknesses: My main concern is that the paper does not offer much insight into the trade-off between control and quality. It is pretty natural to wonder how the control impacts the quality of the generation compared with other methods. The qualitative examples shown seem ok, but it is also evident that the quality is quite degraded compared to alternatives -- this might be a cause of the generative 2D ControlNet itself or that the control does not incorporate enough domain knowledge and provide a mask difficult to respect (e.g., the mask of cats and birds do not look realistic, and I guess it requires quite some work for the ControlNet generation to compensate) . The only FID score is reported in an ablation study on the Shape dataset, which is not particularly insightful. I think reporting (at least) the same metric for other datasets and comparing it with other methods would be important to assess how much the provided approach destroys the generative capability of the generative backbone.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: It would be very helpful if the authors could respond to my two points noted above---the second one in particular. At the very least, some simple ablation studies can be conducted to help justify the overall algorithm. For instance, what if the conditioning masks simply contain circles or other basic shapes for the desired objects (even in the case of animals)? Would the ControlNet still understand the object type from the text prompt and use the simplified mask as the cardinality/topology conditioning?
soundness: 3
strengths: Maintaining desired compositionality is a key challenge for diffusion models, and getting a model to output the desired quantity of discrete objects is often difficult. Framing this problem through the lens of persistent homology makes a lot of sense and ties nicely to the difficulty of dealing with noisy samples at intermediate timesteps. Additionally, being able to condition on first homology is also a nice feature, albeit it seems a bit more contrived (and indeed the main demonstrated use case is in generating aerial road images).
summary: This paper proposes a procedure for conditioning text-to-image diffusion models on topological constraints. In particular, an auxiliary diffusion model is trained that generates binary segmentation masks that conform to a specified zeroth and first homology ranks (i.e., the number of connected components and holes). This is done both by providing the Betti numbers as input conditions to the network and by imposing a loss on the homology of the predicted image. Because the predicted images at early timesteps are very noisy, an approximation of the denoised image is computed, and the persistence module of that image is what is used in the loss objective.
weaknesses: My main concerns with this paper have to do with the overall novelty as well as the overall justification for the approach.

While this paper is indeed the first to consider diffusion-based image generation through the lens of TDA and persistent homology, the actual machinery proposed to accomplish this is very similar to prior work that combines TDA with deep learning --- a key insight is differentiating through the persistence computation, but this is not unique to the diffusion setting. I do agree that the observation that the a persistence module can well represent the homology of a noisy $x_0$ approximation is interesting, but this brings me to my second concern, which is that I am not convinced by the results in this paper that training a diffusion model is critical to the success of this method.

One of the main reasons for the success of image diffusion models is their stability and performance in generating high-quality complex natural images. The masks produced by the diffusion model in this paper, however, are quire primitive and lack detail or high-frequency detail. I wonder if a much simpler generative model, e.g., a VAE, could be just as effective at producing these masks, that could then equally serve as input to a ControlNet. Moreover, I am not convinced that an ML model is necessary to generate these mattes in the first place. Given that the only conditions that the model accepts are the Betti numbers and an object class (nothing more about the actual layout can be specified), it seems to me that such masks could be procedurally generated to obey the desired topology. In fact, looking at Figure 6, it appears that the object class condition is superfluous for the mask model, given that it also appears in the ControlNet prompt. If we look, e.g., at the birds (bottom right), the contour of the second bird from the right is completely different from the input mask. This suggests that the ControlNet largely ignores any higher-frequency context in the mattes.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Empirically (not in theory), can your code generate more complex specific shapes by choosing the right topological invariants as input?
soundness: 3
strengths: I am not an expert in image generation, but I like the idea of adding topological constraints to image generation. It could be very interesting to be able to accurately control resulting images' topology.  I found the approach to be rather straightforward and thus easy to understand and implement. Considering I could not find a prior work that aims to achieve the same result, I find this approach novel.
summary: This paper proposes a generative method for images that can be constrained in its resulting topology - number of objects, as well as the connectivity between objects. The core idea is to use persistent homology which essentially constructs, for each image produced in a diffusion process, a 3D embedding is created by marking all pixels that exceed each z>0 - this enables the authors to produce a topological signature for the image (for different z's) which enables to ignore noisy structures and evaluate the topological invariants of the resulting clean image. The authors then devise a simple loss that uses this information and regularize the neural network to respect the topological constraints.
weaknesses: Again, I am not an expert in image generation, however it seems to me that in the end, the paper focuses mainly on number of objects and less so on more important topological invariants such as connectivity. There are many interesting tests one could cook up, e.g., choose Betti numbers that force a very specific topology (double annulus) and show the method can achieve that. These more interesting topological invariants are also not directly measured in the quantitative evaluations, as far as I understand. I am also uncertain how far can this type of topological control be effective for practical uses, beyond, again, the trivial case of controlling number of objects.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: In Figure 4, the lower part of \alpha_3 seems to be cut off by a straight line? Is it because it is a cube? If so, why doesn't it appear on the other three boundaries?

Although I know this is not the task of this paper, it would be nice if higher-dimensional topological control, such as 3D, could be discussed as future work.
soundness: 3
strengths: The authors claim that they are the first to combine topological and diffusion models on image generation. Although I cannot be sure of this, I think it is indeed a problem that needs to be solved urgently, and this paper provides a possible solution to it.

The combination of persistent homology and diffusion modeling is new to me, and although the combination way is simple, it appears to be effective.

This paper is well written, with rich comparative data and ablation experiments on corresponding parameters.
summary: This paper proposes a method to control the topology of images generated by the diffusion model. By combining the persistent homology theory, a topological mask is generated with noise, and it is fed to the diffusion model as a condition together with the time steps, achieving more accurate topological control of image generation.
weaknesses: Should authors explain in more detail how persistent homology is calculated? I think the part about how it is integrated into the diffusion model and how the loss between the two sets of P and Q defined is well explained, but the specific calculation of persistent homology seems a bit lacking in details.

The persistence diagram could be more explicit, for example by using different point sizes and colors to show which P sets are more important.

Meta-review: The authors propose a method to introduce topological constraints into the generation process of image diffusion models. The method splits the generation process in two parts. First, masks are generated with topology guidance based on persistence diagrams. Then, the image is generated via ControlNet conditioned on the generated mask. The paper clearly shows the benefits of this design in the experiments, when generating instances with specified topology.

The reviews for this paper were 3 vs. 1 leaning positive. The positively-leaning reviewers found the paper to be interesting and sufficiently evaluated. The negative leaning reviewer questions the necessity of including topology in the generation process and was concerned about result quality.

In this diverging situation I took a closer look at the paper and decided to follow the positive evaluation. I found this paper to be interesting and insightful, providing a potential solution to a problem that we see in many generative image diffusion models. The experimental evaluation convinces me that the presented approach leads to clear benefits. The concepts introduced might also spark additional research in other domains.

Decision: Accept (Poster)