Topology-Aware Segmentation Using Discrete Morse Theory
Authors: Xiaoling Hu
Yusu Wang
Li Fuxin
Dimitris Samaras
Chao Chen
Venue: ICLR 2021 Spotlight (2021)
Keywords: Topology
Morse theory
Image segmentation

Abstract: In the segmentation of fine-scale structures from natural and biomedical images, per-pixel accuracy is not the only metric of concern. Topological correctness, such as vessel connectivity and membrane closure, is crucial for downstream analysis tasks. In this paper, we propose a new approach to train deep image segmentation networks for better topological accuracy. In particular, leveraging the power of discrete Morse theory (DMT), we identify global structures, including 1D skeletons and 2D patches, which are important for topological accuracy. Trained with a novel loss based on these global structures, the network performance is significantly improved especially near topologically challenging locations (such as weak spots of connections and membranes). On diverse datasets, our method achieves superior performance on both the DICE score and topological metrics.

Published as a conference paper at ICLR 2021
TOPOLOGY-AWAN SEGMENTATION USING DISCRETE MORSE THEORY
Xiaoling Hu Stony Brook University
Yusu Wang
Li Fuxin
University of California, San Diego
Oregon State University
Dimitris Samaras Stony Brook University
Chao Chen Stony Brook University
ABSTRACT
In the segmentation of fine-scale structures from natural and biomedical images, per-pixel accuracy is not the only metric concern. Topological correctness, such as vessel connectivity and membrane closure, is crucial for downstream analysis tasks. this paper, we propose a new approach to train deep image segmentation networks for better topological accuracy. In particular, leveraging the power of discrete Morse theory (DMT), we identify global structures, including ID skeletons and 2D patches, which are important for topological accuracy. Trained with novel loss based on these global structures, the network performance significantly improved especially near topologically challenging locations (such as weak spots of connections and membranes). On diverse datasets, our method achieves superior performance on both the DICE score and topological metries.
INTRODUCTION
Segmenting objects while preserving their global structure is challenging yet important problem. Various methods have been proposed to encourage neural networks to preserve fine details of objects (Long et al., 2015 He al. 2017 Chen al. 2014 2018 2017). Despite their high per-pixel accuracy, most of them are still prone to structural errors, such as missing small object instances, breaking thin connections, and leaving holes in membranes. These structural errors can significantly damage downstream analysis. For example, in the segmentation of biomedical structures such as membranes and vessels, small pixel errors at a junction will induce significant structure error, leading to catastrophic functional mistakes. See Fig. for an illustration.
Topology is a very global characterization that needs a lot of observations to learn. Any training set is insufficient in teaching the network to correctly reason about topology, especially near challenging spots, c.g., blurred membrane locations or weak vessel connections. neural network tends to learn from the clean-cut cases and converge quickly. Meanwhile, apologically-challene remain mis-classified, causing structural/topollgiica errors. We note that this issue cannot be alleviated even with more annotated (yet still unbalanced) images.
locations
We propose a novel approach that identifies critical topological structures during training and teaches neural network to learn from these structures. Our method can produce segmentations with correct topology. i.c., having the same Betti number (i.c., number of connected components and handles/tunnels) as the ground truth. Underlying our method is the classic Morse theory (Milnor 1963), which captures singularities of the gradient vector field of the likelihood function. Intuitively speaking, we treat the likelihood as terrain function and Morse theory helps us capture terrain structures such as ridges and valleys. See Fig. for an illustration. These structures, composed of ID and 2D manifold pieces, reveal the topological information captured by the (potentially noisy) likelihood function.
We consider these Morse structures as topologically critical; they encompass all potential skeletons of the object. We propose new loss that identifies these structures and enforce higher penalty along them. This way, we effectively address the sampling bias issue and ensure that the networks predict correctly near these topologically difficult locations. Since the Morse structures identified based
"Email: Xiaoling Hu (xiaolhu Prccssttoybbroo.ccd

---

Published as a conference paper at ICLR 2021
(a)
(b)
(c)
(d)
(c)
Figure 1: Illustration of the importance of topological correctness in neuron image segmentation task and the effectiveness of the proposed DMT-loss. The goal of this task is to segment membranes which partition the image into regions corresponding to neurons. (a) input neuron image with challenging locations (blur regions) highlighted. (b) ground truth segmentation of the membranes (dark blue) and the result neuron regions. (c) likelihood map of a baseline method without topological guarantee (Ronneberger et al. 2015). (d) segmentation results of the baseline method. Small pixel- wise errors lead to broken membranes, resulting in the merging of many neurons into one. (e) The topologically critical structure captured by the proposed DMT-loss (based on the likelihood in (c)). (f) Our method produces the correct topology and the correct partitioning of neurons.
on the (potentially noisy) likelihood function, they can be both false negatives (a structure can be true structure but was missed in the segmentation) and false positives (a hallucination of the model and should be removed). Our loss ensures that both kinds of structural mistakes are corrected.
Several technical challenges need to be addressed. First, classical Morse theory was defined for smooth functions on continuous domains. Computing the Morse structures can be expensive and numerically unstable. Furthermore, the entire set of Morse structures may include an excessive amount of structures, large portion of which can be noisy, irrelevant ones. To address these challenges, use the discrete version of Morse theory by Forman [1998 2002). For efficiency purposes, we also an approximation algorithm to compute 2D Morse structures with almost lincar time. The idea is to compute zero dimensional Morse structures of the dual image, which boils down to a minimum spanning tree computation. Finally, 2000 Edelsbrunner & Harer 2010) to prune spurious Morse structures that are not relevant.
the theory of persistent homology (Edelsbrunner et al..
Our Miscrete-Morse-theorr based loss, called the DMT-loss, can be evaluated efficiently and can effectively train the neural network to achieve high performance in both topological accuracy and per-pixel accuracy. Our method outperforms state-of-the-ar methods in multiple apology-relevane metrics (e.g., ARI and VOI) on various 2D and 3D benchmarks. It has superior performance in the Betti number error, which an exact measurement of the topological fidelity of the segmentation.
Related work. Closely related to our method are recent works on prristentthomologyy losses (Hu et al.) 2019 Clough et al.) (019). These methods identify a set of critical points of the likelihood function, e.g., saddles and extrema, as topologically critical locations for the neural network to memorize. However, only identifying sparse set of critical points at every epoch is inefficient terms of training. Instead, our method identifies a much bigger set of critical locations at each epoch, i.c., 1D or 2D Morse skeletons (curves and patches). This is beneficial in both training efficiency and model performance. Extending the critical location sets from points to 1D curves and 2D patches makes it much more efficient in training. Compared with Topoloss (Hu al., (019), we observe a 3-time speedup in practice. Furthermore, by focusing on more critical locations early, our method is more likely escape poor local minima of the loss landscape. Thus it achieves better topological accuracy than TopoLoss. The shorter training time may also contribute to better stability of the SGD algorithm, and thus better test accuracy (Hardt al. 2016).
Another topology-awar loss (Mosinska et al. 2018) uses pretrained filters to detect broken connec- tions. However, this method cannot be generalized
unobserved geometry and higher dimensional topology (loops and voids). We also refer to other existing work on topological features and their applications (Adams et al. [017 Reininghaus et al. 2015) Kusano et al., 2016; Carriere et al.) 2017, Ni et al. 2017) WW et al. (017). Deep neural networks have also been proposed to learn from topological features directly extracted from data (Hofer et al., 2017 Carrière et al., 2019). sersistent-homoloyy-innn
objective functions have been proposed for graphics (Poulenard et al.) 2018) machine learning (Chen et al., 2019 Hofer et al.] 2019). Discrete Morse theory has been used to identify skeleton structures from images; e.g., (DelgadoFFriddice) et al., 2015 Robins ct al. 2011 Wang et al., 2015). The resulting ID Morse structure has been used to enhance neural network architecture: e.g., in (Dey et al. 2019 it is used to both pre- and post-process images, while
2

---

Published as a conference paper at ICLR 2021
Input Image
Likelihood
CNN
Ground
Loce
DMT-loss Computation:
Critical
Compute discrete Morse structure;
Approximate 2D Structure using
Lamt
Structures;
Pruning using persistent homology;
Compute DMT-loss by comparing likelihood and ground truth restricted by the Morse structures
Figure 2: Overview of our method. Topologically critical and error- prror-puune structures are highlighted.
in (Banerjee al. 2020), the 1D Morse skeleton is used a topological prior (part of input) for an encoder-decoder deep network for semantic segmentation of microscopie neuroanatomical data. Our work, in contrast, uses Morse structures (beyond ID) of the output to strengthen the global structural signal more explicitly in an end-to-end training of a network.
Many methods have leveraged the power of deep neural networks (Ronneberger al. 2015 Long
et al. 2015 Badrinarayanan et al.) 20177] Ding et al.) 2019 Kervadec et al. [2019] Karimi & Salcudean
2019 Mosinska al.) 2018) for fine-scale structure segmentation. One may also enforce connectivity
constraints when postprocessing the likelihood map (Han et al. [003 Le Guyader & Vese 2008
Sundaramoorthi & Yezzi, 2007, Ségonne, 2008 Wu et al.) [017]] Gao et 2013| Vicente al.
2008: Nowozin & Lampert. 2009 ZZng et al. 2008 Chen et 2011
et al. [tuuher
ct al. 2013 Oswald al. 2014 Estrada et al. 2014).. However, when the deep neural network itself is topology-agnostis the likelihood map may be fundamentally flawed and cannot be salvaged
topologically. Specific to neuron image segmentation, some methods (Funke et al. 2017 Turaga
eet al.) 2009; Januszewski et al. 2018 Uzunbas et al. 2016; Ye et al. 2019) directly find neuron regions instead of their boundary/memiranns, These methods cannot be generalized to other types of data such satellite images, retinal images, vessel images, etc.
2
METHOD
We propose a novel loss to train a topology-aware network end-to-end. It uses global structures captured by discrete Morse theory (DMT) to discover critical topological structures. In particular, through the language of 1- and 2-stable manifolds, DMT helps identify 1D skeletons 2D sheets (separating 3D regions) that may be critical for structural accuracy. These Morse structures are used to define a DMT-loss that is essentially the cross-entropy loss constrained to these topologically critical structures. As the training continues, the neural network learns to better predict around these critical structures, and eventually achieves better topological accuracy. Please refer to Fig. 2 for an overview of our method.
2.1
MORSE THEORY
Morse theory (Milnor 1963) identifies topologically critical structures from a likelihood map (Fig. ((a). In particular, it views the likelihood as a terrain function (Fig. ((b) and extracts its landscape features such as mountain ridges and their high high-dimensional counterparts. The broken connection in the likelihood map corresponds to local dip in the mountain ridge of the terrain in Fig. (3b and Fig, [(c] The bottom of this dip is captured by so-called saddle point (S Fig. 3(c) of the likelihood map. The mountain ridge connected to this bottom point captures the main part of missing pixels. Such "mountain ridges" can be captured by the so-called stable manifold w.r.t. the saddle point using the language of Morse theory. By finding the saddle points and the stable manifold of the saddle points on the likelihood map, we can ensure the model learns to "correctly" handle pixels near these structures. We note that analogous scenario (such as blood vessels) as well as 2D signals (such as membranes of cells) in 3D images they can also be captured by saddles (of different indices) and their stable manifolds.
also happen with such ID signals
3

---

Published as a conference paper at ICLR 2021
130
M2
Figure 3: From left to right: (a) Likelihood map. (b) Density map: the z-axis value is the probability of the likelihood map in the left figure. (c) Density map for the highlighted region in the middle figure. M1 and M2 are maxima (red dots), V is minimum (yellow), S is a saddle (green) with its stable manifolds flowing to it from M1 and M2.
In this paper, we focus on the application of segmenting 2D and 3D images. Specifically, suppose we have a smooth function / Rdd - to model the likelihood (density) map. Given any point a E R2, the negative gradient - ff(x) = - 01 31 ai....,
D2. ........ 1T indicates the steepest descending direction of f. A point x = (21, x2; .2...... ................................ ..)) is critical if the function gradient at this point vanishes (i.e., (xx) = 0). For a well-behaved function (more formally, called Morse function) defined on Rd critical point could be a minimum, a maximum, or d - types of saddle points. See Fig. 3(c) for an example. For d 2, there is only one saddle point type. For d = 3, there are two saddle point types, referred to as index-l and index-2 saddles. Formally, when taking the eigen-values of the Hessian matrix at critical point, its index is equal to the number of negative eigenvalues.
Intuitively, imagine we put a drop of water on the graph of f (i.e. the terrain in Fig, ((b) at the lift of I onto this terrain, then- - fz) indicates the direction along which the water will flow down. If we track the trajectory of this water drop as it flows down, this gives rise to a so-called integral line (a flow line). Such flow lines can only start and end critical points] where the gradient vanishes.
The stable manifold S(p) of a critical point p is defined as the collection of points whose flow line ends at p. For a 2D function f: I² + R. for a saddle q, its stable manifold S(g) starts from local maxima (mountain peaks in the terrain) and ends at g, tracing out the mountain ridges separating different valleys (Fig. 3(c).. The stable manifold Sp) of minimum p. on the other hand, corresponds to the entire valley around this minimum p. See the valley point V and its corresponding stable field in Fig. 3(c) For a 3D function f : R3 - R. the stable manifold w.r.t. an index-2 saddle connects mountain peaks to saddles, tracing 1D mountain ridges in the case of 2D function. The stable manifold w.r.t. an index-1 saddle consists of flow lines starting at index-2 saddles and ending at g. Their union, called the 2-stable manifold of f. consists of a collection of 2-manifold pieces.
These stable manifolds indicate important topological structures (graph-like or sheet-like) based on the likelihood of the current neural network. Using these structures, we will propose novel loss (Sec. 2.3) to improve the topological awareness of the model. In practice, for images, we will leverage the discrete version of Morse theory for both numerical stability and easier simplification.
Discrete Morse theory. Due to space limitations, we briefly explain discrete Morse theory, leaving technical details to Appendix A.2.I We view a d-dimensional image, = 2 or 3, as a d-dimensional cubical complex, meaning it consists of 0-, 1-, 2- and 3-dimensional cells corresponding to vertices (pixels/voxels), edges, squares, and cubes as its building blocks. Discrete Morse theory (DMT), originally introduced in (Forman 1998 2002). is a combinatorial version of Morse theory for general cell complexes. In this setting, the analog of a gradient vector is a pair of adjacent cells, called discrete gradient vectors. The analog of an integral line (flow line) is sequence of such cell-pairs (discrete gradient vectors), forming a so-called V-path. Critical points correspond to critical cells which do not participate in any discrete gradient vectors. A minimum, saddle and a maximum for 3D domain intuitively correspond to a critical vertex, a critical edge, a critical square and a critical cube respectively. A 1-stable manifold in 2D will correspond a V-path, i.e., a sequence of cells, connecting a critical square (a maximum) and critical edge (a saddle). See Fig. 3(c) for an illustration. In 3D, it will be a V-path connecting critical cube and a critical square.
index- saddle, index-2
Mooe precisely, flow lines only tend critical points in the limit and never reach them.
4

---

Published as a conference paper at ICLR 2021
Figure 4: From left to right: (a) Sample likelihood map, (b) Ground truth, (c) improperly pruned structures and (d) properly pruned structures.
2.2 SIMPLIFICATION AND COMPUTATION
In this section, we describe how we extract discrete Morse structures corresponding to the 1-stable and 2-stable manifolds in the continuous analog. First, we prune unnecessary Morse structures, based on the theory persistent homology (Edelsbrunner et al.) 2000] [ddelsrunner & Harer 20T0). Second, we approximate the 2-stable manifold structures using O-stable manifolds of the dual to achieve high efficiency in practice, because is rather involved to compute based on the original definition.
Persistence-basee based structure pruning. While Morse structures reveal important structural informa- tion, they can be sensitive to noise. Without proper pruning, there be an excessive amount of Morse structures, many of which are spurious and not relevant to the true signal. See Fig. 4(c) for an
example. Similar previous approaches e.g.. (Sousbie) 2011 Friidrccss al. 2015 Wang
et al. 2015), we will prune these structures using persistent homology.
Persistent homology is one of the most important developments in the field of topological data analysis in the past two decades (Edelsbrunner & Harer 20100; Edelsbrunner et al. 2000). Intuitively speaking, we grow the complex by starting from the empty set and gradually include more and more cells using decreasing threshold. Through this course, new topological features can be created upon adding critical cell, and sometimes a feature can be destroyed upon adding another critical cell. The persistence algorithm (Edelsbrunner et al., (000)) will pair up these critical cells; that is, its output is set of critical cell pairs, where each pair captures the birth and death of topological features during this evolution. The persistence of pair is defined as the difference of function values of the two critical cells, intuitively measuring how long the topological feature lives in term f.
Using persistence, we prune critical cells that are less topologically salient, and thus their corresponding Morse structures. Recall each 1- and 2-stable Morse structure is constituted by V-paths flowing into critical cell (corresponding persistence associated with this critical cell to determine the saliency of the corresponding Morse structure. If the persistence is below certain threshold e, we prune the corresponding Morse structure via an operation called Morse cancellation (more details are in the Appendix (.2.2 F See Fig. ((d) for example Morse structures after pruning. We denote by S (e) and (2) the remaining sets of and 2-stable manifolds after pruming. We'll use these Morse structures to define the loss (Sec. 2..
saddle in the continuous setting). We then use the
Computation. We need an efficient algorithm compute (e) and S2(e) from a given likelihood f. because this computation needs to be carried out at each epoch. It is significantly more involved to define and compute S2(e) in the discrete Morse setting Calgado-Friedricht et al. 2015). Furthermore, this also requires the computation of persistent homology up to 2-dimensions, which takes time T = 0n*) (where w 2.37 is the exponent in the matrix multiplication time, i.e., the time to multiply two n X matrices). To this end, we propose to approximate S2(e) by S2(e) (more details can be found in Appendix (.2.3) (..233 which intuitively comes from the "boundary" of the stable manifold for minima. Note that in the smooth case for function f R33 + R, the closure of 2-stable manifolds exactly corresponds to the 2D sheets on the boundary of stable manifolds of minima. [ This is both conceptually clear and also avoids the costly persistence computation. In particular, given minimum q with persistence greater than the pruning threshold c, the collections of V-paths ending at the minimum q form a spanning tree Tç In fact, consider all minima (9I..... ge) with persistence at least e. [T, form a maximum spanning forest among all edges with persistence value smaller than (Bauer et al. 2012 Dey et al. 2018). Hence it can be computed easily in O(n log n) time, where is the image size.
"Technically, not all spurious structures can be pruned/cancelled. But in practice, most of them can. "This however is always true for the discrete Morse setting.
5

---

Published as a conference paper at ICLR 2021
We then take all the edges incident to nodes from different trees. The dual of these edges, denoted as S2(c), serves the "boundaries" separating different spanning trees (representing stable manifolds to different minima with persistence 2 c). See Fig. Overall, the computation of S2(c) takes only (n log n) by a maximum spanning tree algorithm.
Tree
Local minimum
Saddl
point
Edee
As for S1 (c), we use simplified algorithm of (Dey et al. 2018 which can the image size. For 3D image, the time is On log + T*), where T O is the time to compute persistent homology, where w 2.37 is the exponent in matrix multiplication time.
Local Annmmum
Tree
compute (c) in (n log n) time for a 2D image, in which
Figure 5: Spanning tree illustration.
2.3 THE BASED Loss FUNCTION AND TRAINING DETAILS
Our loss has two terms, the cross-entropy term, Luce and the DMT-loss, Lamt: L(f.g) Gm (f+ 9). in which o is the likelihood, g is the ground truth, and is the weight of Lami Here we focus one single image, while the actual loss is aggregated over the whole training set.
(f:e)
The DMT-loss enforces the correctness of the topologically challenging locations discovered by our algorithm. These locations are pixels of the (approximation of) 1- and 2-stable manifolds Si(c) and (e( of the likelihood, f (Fig. I()). Denote by M a binary mask of the union of pixels of all Morse structures in S₁ (c) U (c) We want to enforce these locations to be correctly segmented. We use the cross-entropy between the likelihood map f and ground truth g restricted to the Morse structures, formally, Lame(J g) Laoe(S o MM; MM ), in which is the Hadamard product.
Different topoloicaa error types. Recall that the Morse structures are computed over the potentially noisy likelihood function of neural network, which can help identify two types of structural errors: (1) false negative: a true structure that incomplete in the segmentation, but can be visible in the Morse structures. This types of false negatives (broken connections, holes in membrane) can be re- stored as the additional cross-entropy loss near the Morse structures will force the network to increase its likelihood value on these structures. (2) false positive: phantom structures hallucinated by the network when they do not exist (spurious branches, membrane pieces). These errors can be eliminated the extra cross entropy loss on these structures will force the network to decrease the likelihood values along these structures. We provide a more thorough discussion in the Appendix A.1.2
Differentiability.. We note that the Morse structures recomputed every epoch. The structures, as well as their mask My. may change with f. However, the change is not continuous; the output of the discrete Morse algorithm is combinatorial solution that does not change continuously with f. Instead, it only changes at singularities, i.e., when the function values of f at different pixels/voxels are the same. In other words, for a general f. the likelihood function is real-valued, so is unlikely two pixels share the exact same value. In case that they are, the persistence homology algorithm by default will break the tie and choose one as critical. The mask M, remains constant within a small neighborhood of current f. Therefore, the gradient of I.dma exists and can be computed naturally.
Training details. Although our method is architetture-agnosiic, for 2D datasets, we select an architecture driven by 2D U-net (Ronneberger et al. 2015); for 3D datasets, select architecture inspired by a 3D U-Net (Çiçek et al.] (0116). Both U-Net and 3D U-Net were originally designed for neuron segmentation tasks, capturing the fine-structures of images. In practice, we first pretrain the network with only the cross-entropy loss, and then train the network with the combined loss.
EXPERIMENTS ON 2D AND 3D DATASETS
Experiments on 2D dataset. Six natural and biomedical 2D datasets are used: ISBT12 AArganda- Carreras et al.] 2015). ISBI13 Trgganaa-Careere et al. 2013), CREMI, CrackTree Zou al. 2012). Road Mnihh 2013) and DRIVE (Staal et al. 2004). More details about the datasets are included in Appendix A.3.T For all the experiments, hyperparameters for both the proposed method and other baselines, and report the mean performance over the validation set. This also holds for 3D experiments.
3-fold cross- vols-valioation to tune
Evaluation metrics. We use five different evaluation metries: Pixel-wise accuracy, DICE score, ARI, VOI and the most important one is Betti number error, which directly compares the topology

---

Published as a conference paper at ICLR 2021
Table 1: Quantitative results for different models on several 2D image datasets
Method
Accuracy
DICE
ARI
VOI
Betti Error
ISBI13
DIVE
0.9642 + 0,0018 0.9658 + 0.0020 0.6923 + 0.0134
2.790 = 0.025
3,875 + 0.326
U-Net
0.9631 + 0.0024 0.9649 + 0.0057 0.7031 + 0.0256
2.583 + 0.078
3.463 + 0.435
Mosin.
0.9578 + 0.0029 0.9623 + 0.0047 0.7483 = 0.0367
1.534 = 0.063
2.952 = 0.379
TopoLoss
0.9569 0.0031 0.9689 = 0.0026 0.8064 = 0.0112
1.436 0.008
1.253 0.172
DMT
0.9625 + 0.0027
0.9712 + 0.0047 0,8289 = 0.0189
1.176 = 0.052
1.102 = 0.203
CREMI
DIVE
0.9498 0.0029 0.9542 + 0.0037 0.6532 0.0247
2.513
4
U-Net
0,9468 + 0.0048 0.9523 +: 0.0049 0.6723 + 0.0312
0.105
3.016 0.253
Mosin.
0.9467 + 0.0058
0.9489 +: 0.0053 0.7853 + 0.0281
1.623
0.083
1.973 + 0.310
TopoLoss
0.9456 + 0.0053
0.9596 = 0.0029 0.8083 = 0.0104
1,462
= 0.028
1.133 = 0.224
DMT
0.9475 0.0031
0.9653 = 0.0019 0.8203 = 0.0147
1.089
= 0.061
0.982 0.179
CrackTree
DIVE
0.9854 + 0.0052
0.0530
0.8634 0.0376
1.570 +570=0000 0.078
U-Net
0.9821 0.0097
=.6491 0.8749 + 0.0421
1.625 0.104
1.785 = 0.303
Mosin.
0.9833 + 0.0067
0.6527 + 0.0010 0.8897 = 0.0201
1.113 = 0.057
1.045 : 0.214
LopoLoss
0.9826 0.0084 0.6732 + 0.0041 0.9291 + 0.0123
0.997
- 0.011
0.672 = 0.176
DMT
0.9842 + 0.0041
0.6811 + 0.0047 0.9307 + 0.0172
0.901
+ 0.081
0.518 + 0.189
Road
DIVE
0.9734 H 0.0077 0.6743 0.0051
0.0051 0.8201 = 0.0128
2.368
= 0.203
3.598 = 0.783
U-Net
0.9786 + 0.0052 0.6612 +: 0.0016 0.8189 + 0.0097
2.249
+ 0.175
3.439 + 0.621
Mosin.
0.9754 0.0043 0.6673 + 0.0044
0.8456 + 0.0174
1.457
= 0.096
2.781 = 0.237
TopoLoss
0.9728 + 0.0063 0.6903 + 0.0038 0.8671 = 0.0068
1.234
= 0.037
1.275 + 0.192
DMT
0.9744 + 0.0049 0.7056 + 0.0022
0.8819 + 0.0104
1.092 + 0.129
0.995 + 0.301
(number of handles/voids) between the segmentation and the ground truth. More details about the evaluation metrics are provided in Appendix A.3.2 The last three metrics are topology-awer aware.
Baselines. DIVE (Fakhry et al., 2016) is popular neural network that predicts the probability of a pixel being membrane (border) pixel or not. U-Net (Ronneberger et al.) 2015) is one of the most powerful image segmentation methods trained with cross-entropy loss. Mosin. (Mosinskal et al. 2018) uses a topology-aware loss based on the response of selected filters from pretrained CNN. Lopo (Hu et al. 2019) proposes novel topological loss to learn to segment with correct generate segmentations by thresholding the predicted likelihood maps at 0.5, and this also holds for 3I) experiments.
topology. For all methods,
Quantitative and qualitative results. T quantitative results for four 2D image datasets, ISBI13, CREMI, CrackTrree and Road. The results are highlighted when they are significantly better, and statistical significance is determined by t-tests. Results for ISBI12 and DRIVE are included in Appendix A.3.3 The DMT-loss outperforms others in both DICE score and topological accuracy (ARI, VOI and Betti Error). Please note that here the backbone of TopoLoss is the same as in (Hu) et al., 2019), heavily engineered network. The performance of TopoLoss will be worse if we implement it using the same U-Net backbone as DMT-Loss. Comparisons with the same backbones can be found in Appendix [A.34]]
Fig. G shows qualitative results. Our method correctly segments fine structures such as membranes, roads and vessels. Our loss is a weighted combination of the cross entropy and DMT-losses. When = O, the proposed method degrades to a standard U-Net. The performance improvement over all datasets (U-Net and DMT line in Table demonstrates that our DMT-loss is helping the deep neural nets to learn a better structural segmentation.
Experiments 3D datasets. We use three different biomedical 3D datasets: ISBI13, CREMI and 3Dircadb (Soler et al. 2010). ISBI13 and CREMI, which have been discussed above, are originally 3D datasets, be used in both 2D and 3D evaluations. We also evaluate method on the open dataset 3Dircadb, which contains 20 enhanced CT volumes with artery annotations.
Evaluation metrics. We similar evaluation metrics the 2D part. Note that, in term of 2D images, for ARI and VOI, we compute the numbers for each slice and then average the numbers for different slices the final performance; for 3D images, we compute the performance for the whole
7

---

Published as a conference paper at ICLR 2021
Qualitative results of the proposed method compared to other models. From left to right: sample images, ground truth, results for DIVE, U-Net, Mosin., TopoLoss and our proposed DMT.
Figure 6:
Table Quantitative results for different models on several 3D datasets Accuracy
Method
DICE
ARI
VOI
Betti Error
ISBI13
3D DIVE
0.9723 = 0.0021 0.9681 = 0.0043 0.8719 1007799:0000 0.0189 1.208 10088:00 0.149 2.375 = 0.419
3D U-Net
0.9746 + 0.0025 0.9701 = 0.0012 0.8956 + 0.0391 1.123 + 0.091 1.954 0.585
MALA
0.9701 = 0.0018 0.9699 = 0.0013 0.8945 + 0.0481 0.901 + 0.106 1.103
0.207
3D TopoLoss
0.9689 = 0.0031 0.9752 = 0.0045 0.9043 + 0.0283 0,792 = 0.086 0.972 0.245
DMT
0.9701 0.0026 0.9803 = 0.0019 0.9149 + 0.0217 0.634 + 0.086 0.812 0.134
CREMI
3D DIVE
0.9503 = 0.0061 0.9641 = 0.0011 0.8514 0.0387 1,219 = 0.103 2.674 0.473
3D U-Net
0.9547 + 0.0038 0.9618 0.0026 0.8322 + 0.0315 1,416 + 0.097 2.313 0.501
MALA 0.9472 + 0.0027 0.9583 = 0.0023 0.8713 + 0.0286 1.109 + 0.093 1.114
0.309
3D TopoLoss
0.9523 0.0043 0.9672 0.0010 0.8726 + 0.0194 1.044 + 0.128 1.076 0.206
DMT
0,9529 = 0.0031 0.9731 = 0.0045 0.9013 + 0.0202 0.891 + 0.099 0.726 0.187
3Dircadb
3D DIVE
0.9618 = 0.0054 0.6097 = 0.0034
4.571
0.505
3D U-Net
0.9632 + 0.0009 0.5898 + 0.0025
4.131 + 0.483
MALA 0.9546 0.0033 0.5719 0.0043
2.982
0.105
3D Topol Loss
0.9561 + 0.0019 0.6138 + 0.0029
2.245 0.255
DMT
0.9587 0.0023 0.6257 0.0021
1.415 0.305
volume. For 2D images, compute the 1D Betti number (number of holes) to obtain the Betti Error; while for 3D images, we compute the 2D Betti number (number of voids) to obtain the Betti Error.
Baselines. 3D DIVE (Zeng et al.) 2017). 3D U-Net (Cişek et al. 2016). 3D TopoLoss (Hu et al. 2019) are the 3D versions for DIVE, U-Net and TopoLoss. MALA (Funke et al. 2017) trains the U-Net using new structured loss function.
Quantitative and qualitative results. Table shows the quantitative results for three different 3D image datasets, ISBI13, CREMI and 3Dircadb. Our method outperforms existing methods in topological accuracy (in all three topology -wwar metrics), which demonstrates the effectiveness of the proposed method. More qualitative results for 3D cases included in Appendix A.3.5
The benefit of the proposed DMT--lsss. Instead of capturing isolated critical points in TopoLoss Hu
al. 2019), the proposed DMT-loss captures the whole V-path as critical structures. Taking the patch in Fig. 4(a) as an example, TopoLoss identifies 2= 80 isolated critical pixels for further training. whereas the critical structures captured by the DMT-loss contain 2 1000 critical pixels (Fig. 4(d). We compare the efficiency of DMT-loss and TopoLoss using the same backbone network, evaluated on the CREMI 2D dataset. Both methods start from a reasonable pre-trained likelihood map. TopoLoss achieves 1.113 (Betti Error), taking <3h to converge; while DMT-loss achieves 0.956 (Betti Error), taking s: 1.22h to converge (the standard U-Net takes 20055h instead). Aside from converging faster, the DMT-loss is also less likely to converge to low-quality local minima. We hypothesize that the loss landscape of the topological loss will have more local minima than that of the DMT-loss, even though the global minima of both landscapes may have the same quality.
8

---

Published as a conference paper at ICLR 2021
Figure 7: Illustration comparison between the proposed DMT-loss and a simple reweighted cross entropy loss. Please refer to Fig. To(b) the likelihood map of baseline method without topological guarantee. (a) an input neuron image. (b) The topologically critical structures from the likelihood. captured by the proposed discrete Morse algorithm. These structures will be used in the DMT-Loss (c) ground truth. (d) The FP/EN pixels identified by simple re-weighting cross entropy loss.
Ablation study for persistence threshold e. As illustrated in Fig. different persistence thresholds c will identify different critical structures. The ablation experiment is also conducted on the CREMI 2D dataset. When é = 0.2 (See Fig. 4. the proposed DMT-loss achieves the best performance 0.982 (Betti Error). When E = 0.1 and e 0.3, the performance drops to 1.206 and 2. 105 (both in Betti Error), respectively. This makes sense for the following reasons: 1) for e 0.1, the DMT-loss captures lots of unnecessary structures which mislead the neural networks; 2) for e == 0.3, the DMT- loss misses lots of important critical structures, making the performance drop significantly. The is chosen via cross-validation. The ablation study for balanced term ,3 is provided in Appendix AA.3.6
Comparison with other simpler choices. rich locations. To demonstrate the effectiveness of the proposed method, we also compare with two baselines: canny edge detection, and ridge detection, which achieve 2.971 and 2.507 in terms of Betti Error respectively, much worse than our results (Betti Error: 0.982). A Although our focus is the Betti error, we also report per-pixel accuracy for reference (See Table 5 in Appendix for details). From the results we observe that the baseline models could not solve topological errors, even though they achieve high per-pixel accuracy. Without a persistent-homoles generate too many noisy structures, and thus are not effective as DMT-loss.
The proposed method essentially highlights geometrically
based pruning, these baselines
Robustness of the proposed method. We run another ablation study on images corrupted with Gaussian noise. The experiment is also conducted on the CREMI 2D dataset. From the results (See Table Appendix for details), the DMT-loss is fairly robust and maintains good performance even with high noise levels. The reason is that the Morse structures are computed on the likelihood map, which is already robust to noise.
Comparison with reweighted cross entropy loss. We run additional ablation study to compare with baseline reweighting the FP and FN pixels in the cross-entropy loss (Reweighting CE). The weights of the FP/FN pixels are hyperparameters tuned via cross-validation. The reweighting CE strategy achieves 2.753 in terms of Betti Error (on CREMI 2D data), and the DMT-loss is better than this baseline. The reason is that the DMT-loss specifically penalizes FP and FN pixels which are topology-critical. Meanwhile, reweighting CE adds weights to FP/FN pixels without discrimination. majority of these misclassified pixels are not topology-critical They are near the boundary of the foreground. Please see Fig. for an illustration.
4 CONCLUSION
In this paper, we proposed new DMT-loss to train deep image segmentation neural networks for better topological accuracy. With the power of discrete Morse theory (DMT), we could identify ID skeletons and 2D patches which are important for topological accuracy. Trained with the new loss based on these global structures, the networks perform significantly better, especially near topologically challenging locations (such weak spots of connections and membranes).
Acknowledgements. The research of Xiaoling Hu and Chao Chen is partially supported by NSF 199-1909038. The research of Li Fuxin is partially supported by NSF IIS- 1232. The research of Yusu Wang is partially supported by NSF under grants CCF- 1733798, RI-1815697, and by NIH under grants R01EB022899, RFI RFIMH125317. Dimitris Samaras is partially supported by gift from Adobe, the Partner University Fund, the NSF IUCRC for Visual and Decision Informatics and the SUNY2020 Infrastructure Transportation Security Center.

---

Published as a conference paper at ICLR 2021
REFERENCES
Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier. Persistence images: A stable vector representation of persistent homology. The Journal of Machine Learning Research, 18(1):218-252, 2017.
Bjoemn Andres, Jörg H Kappes, Thorsten Beier, Ullrich Köthe, and Fred A Hamprecht. Probabilistic image segmentation with closedness constraints. In 2011 International Conference on Computer Vision, pp. 261 1-2618. IEEE, 2011.
I Arganda-Carreras HS Seung, A Vishwanathan, and D Berger. 3d segmentation of neurites in em images challenge-ishi 2013, 2013.
Ignacio Arganda-Carreras Srinivas C Turaga, Daniel R Berger. Dan Cireşan, Alessandro Giusti, Luca M Gambardella, Jürgen Schmidhuber, Dmitry Laptev, Sarvesh Dwivedi, Joachim M Buh- mann, et al. Crowdsourcing the creation of image segmentation algorithms for connectomics.
Frontiers in neuroanatomy, 9: 1224, 2015.
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutional encoder- decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12):2481-2495 2017.
Samik Banerjee, Lucas Magee, Dingkang Wang, Xu Li, Bing-Xing Huo, Jaikishan Jayakumar, Katherine Matho, Meng-Kuan Lin, Keerthi Ram, Mohanasankar Sivaprakasam, et al. Semantic segmentation of microscopic neuroanatomical data by combining topological priors with encoder- decoder deep networks. Nature Machine Intelligence, 2(10):855-59,, 2020.
U. Bauer, C. Lange, and M. Wardetzky. Optimal topological simplification of discrete functions surfaces. Discr. Comput. Geom., 47(2):347 377, 2012. ISSN 1432-0444.
Mathieu Carriere, Marco Cuturi, and Steve Oudot. Sliced wasserstein kernel for persistence diagrams. In Proceedings of the 34th International Conference on Machine Hharning-Volumee 70, pp. 664-673.
JMLR. org, 2017.
Mathieu Carrière, Frédéric Chazal, Yuichi Ike, Théo Lacombe, Martin Royer, and Yuhei Umeda. A general neural network architecture for persistence diagrams and graph classification. arXiv preprint arX4v:/904.0 194.00989 2019.
Chao Chen, Daniel Freedman, and Christoph H Lampert. Enforcing topological constraints in random field image segmentation. In CVPR 2011, pp. 2089 2096. IEEE, 2011.
Chao Chen, Xiuyan Ni, Qinxun Bai, and Yusu Wang. A topological regularizer for classifiers via persistent homology. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 25.3 257225772222 2019.
Liang-Chieh Chen, George Papandreou, lasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Semantic image segmentation with deep convolutional nets and fully connected erfs. arXiv preprint arXiv:1412.700 2014.
Liang-Chie Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprini ArXiv:1706.0883 2017.
Liang-Chich Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4):834 848, 2018.
Özgün Çiçek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and cosputer-assisted intervention. pp. 424-432. Springer, 2016.
James R. Clough, Ilkay Oksuz, Nicholas Byrne, Veronika A. Zimmer, Julia A. Schnabel, and Andrew P. King. topological loss function for deep-learning based image segmentation using persistent homology, 2019, arXiv preprint at 11X0v:1910.01877.
10

---

Published as a conference paper at ICLR 2021
O. Delgado- Friedd-ss,, V. Robins, and A. Sheppard. Skeletonization and partitioning of digital images using discrete morse theory. IEEE Trans. Pattern Anal. Machine Intelligence, 37(3):654-666 March 2015. ISSN 0162-8828.
Dey, J. Wang, and Wang. Road network reconstruction from satellite images with machine learning supported by topological methods. In Proc. 27th ACM SIGSPATIAL Inil. Conf. Adv. Geographic Information Systems (GIS). pp. 520-523, 2019.
Tamal K. Dey, Jiayuan Wang, and Yusu Wang. Graph reconstruction by discrete morse theory. In Proc. 34th Intl. Sympos. Comput. Geom. (SoCG), pp. 31:1-31:15, 2018.
Henghui Ding, Xudong Jiang, Ai Qun Liu, Nadia Magnenat Thalmann, and Gang Wang. Boundary- aware feature propagation for scene segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 6819-6829 2019.
Herbert Edelsbrunner and John Harer. Computational lopology: an introduction. American Mathe- matical Soc., 2010.
Herbert Edelsbrunner, David Letscher, and Afra Zomorodian. Topological persistence and simplifica- tion. In Proceedings s1 Anmual Symposium on Foundations of Computer Science, pp. 454 463. IEEE, 2000.
Rolando Estrada, Carlo Tomasi, Scott C Schmidler, and Sina Farsiu. Tree topology estimation. IEEE Iransactions on pallern analysis and machine intelligence, 1688))1688-17000 2014.
Ahmed Fakhry, Hanchuan Peng, and Shuiwang Ji. Deep models for brain em image segmentation: novel insights and improved performance. Bioinformatics, 32(15):2352-2333 2016.
R. Forman. Morse theory for cell complexes. Advances in mathematics, 134(1)900-145, 1998.
Robin Forman. A user's guide to discrete morse theory. Sém. Lothar. Combin, 48:35pp. 2002.
Jan Funke, Fabian David Tschopp, William Grisaitis, Arlo Sheridan, Chandan Singh, Stephan Saalfeld, and Srinivas Turaga. A deep structured learning approach towards automating connec- tome reconstruction from 3d electron micrographs. arXiv preprint wtXh::109.002974 2017.
Mingchen Gao, Chao Chen, Shaoting Zhang, Zhen Qian, Dimitris Metaxas, and Leon Axel. Segment- ing the papillary muscles and the trabeculae from high resolution cardiac ct through restoration of topological handles. In International Conference on Information Processing in Medical Imaging. pp. 184-19999 195. Springer, 2013.
Xiao Han, Chenyang Xu, and Jerry Prince. A topology preserving level set method for geometric deformable models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(6): 755-768, 2003.
Moritz Hardt, Ben Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International Conference on Machine Learning, pp. 1225-1234, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference computer vision, pp. 2961-2969, 2017.
Christoph Hofer, Roland Kwitt, Mare Niethammer, and Andreas Uhl. Deep learning with topological
signatures. In Advances in Neural Information Processing Systems, pp. 1634-1644, 2017.
Christoph Hofer, Roland Kwitt, Mandar Dixit, and Mare Niethammer. Comnectivity-optimiii representation learning via persistent homology. arXiv preprint arXiv: (0t.:::9900000000 2019.
Xiaoling Hu, Fuxin Li, Dimitris Samaras, and Chao Chen. Topslogy deep image segmen- tation. In Advances in Neural Information Processing Systems, pp. 5658-5669, 2019.
Michat Januszewski. Jörgen Komfeld, Peter H Li, Art Pope, Tim Blakely, Larry Lindsey, Jeremy Wlaitin-Shepare Mike Tyka, Winfried Denk, and Viren Jain. High precisirciiion automated reconstruc- tion of neurons with flood-filling networks. Nature methods, 15(8):605-610, 2018.
11

---

Published as a conference paper at ICLR 2021
Davood Karimi and Septimiu E Salcudean. Reducing the hausdorff distance in medical image segmentation with convolutional neural networks. arXiv preprint 1xXiv:104.10000 2019.
Hoel Kervadec, Jihene Bouchtiba, Christian Desrosiers, Eric Granger, Jose Dolz, and Ismail Ben Ayed. Boundary loss for highly unbalanced segmentation. In International Conference on Medical Imaging with Deep Learning, pp. 285-296, 2019.
Genki Kusano, Yasuaki Hiraoka, and Kenji Fukumizu. Persistence weighted gaussian kernel for topological data analysis. In International Conference on Machine Learning, pp. 2004-2013, 2016.
Carole Le Guyader and Luminita A Vese. Self-repelling snakes for topology- segmentation models. IEEE Transactions on Image Processing, 17(5):767-779, 2008.
Jonathan Long. Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3444 2015.
John Milnor. Morse theory(AAM-5I) volume 51, Princeton university press, 1963.
Volodymyr Mnih. Machine learning for aerial image labeling. University of Toronto (Canada), 2013.
Agata Mosinska, Pablo Marquez-Neila Mateusz Koziński, and Pascal Fua. Beyond the pixel-wise loss for topology-aware delineation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3136-3145, 2018.
Xiuyan Ni, Novi Quadrianto, Yusu Wang, and Chao Chen. Composing tree graphical models with persistent homology features for clustering mixed-type data. In Proceedings of the 34th International Conference on Machine Learning-Volie Volume 70, pp. 2622-2631. JMLR. org, 2017.
Sebastian Nowozin and Christoph H Lampert. Global connectivity potentials for random field models. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 318-825, IEEE, 2009.
Martin Ralf Oswald, Jan Stühmer, and Daniel Cremers. Generalized connectivity constraints for spatio-temporal 3d reconstruction. In European Conference on Computer Vision, pp. 32-46. Springer, 2014.
Adrien Poulenard, Primoz Skraba, and Maks Ovsjanikov. Topological function optimization for continuous shape matching. In Computer Graphics Forum, volume 37, pp. 13-25. Wiley Online Library, 2018.
Jan Reininghaus, Stefan Huber, Ulrich Bauer, and Roland Kwitt. A stable multi-scale kernel for topological machine learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4741 144814448 2015.
V. Robins, P. J. Wood, and A. P. Sheppard. Theory and algorithms for constructing discrete morse complexes from grayscale digital images. IEEE Trans. Pattern Anal. Machine Intelligence, 33(8): 1646-1658, Aug 2011. ISSN 0162-8828.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference Medical image computing and computer- asxisted intervention, pp. 231- Springer, 2015.
Florent Ségonne. Active contours under topology control-genus preserving level sets. International Journal of Computer Vision, 19(2):0771177,22) 2008
L Soler, A Hostettler, V Agnus, A Charnoz, Fasquel, Moreau, A Osswald, M Bouhadjar, and J Marescaux. 3d image reconstruction for comparison of algorithm database: a -specft anatomical and medical image database. IRCAD, Strasbourg, France, Tech. Rep, 2010.
T. Sousbie. The persistent cosmic web and its filamentary structure Theory and implementation. 4 414:350-383, June 2011.
12

---

Published as a conference paper at ICLR 2021
Joes Staal, Michael D Abråmoff, Meindert Niemeijer, Max A Viergever, and Bram Van Ginneken. Ridge-based vessel segmentation in color images of the retina. IEEE transactions on medical imaging. 23(4):501-509, 2004.
Jan Stuhmer, Peter Schroder, and Daniel Cremers. Tree shape priors with connectivity constraints using convex relaxation on general graphs. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2336-2343, 2013.
Ganesh Sundaramoorthi and Anthony Yezzi. Global regularizing flows with topology preservation for active contours and polygons. IEEE Trunsactions on Image Processing, (63)):833-812, 2007.
Srinivas C Turagaa Kevin L. Briggman, Moritz Helmstaedter, Winfried Denk, and H Sebastian Seung. Maximin affinity learning of image segmentation. arXiv preprint arXiv:091 1.5322, 2009.
Mustafa Gokhan Uzunbas, Chao Chen, and Dimitris Metaxas. An efficient conditional random field approach for automatic and interactive neuron segmentation. Medical image analysis, 27:31-44, 2016.
Sara Vicente, Vladimir Kolmogorov, and Carsten Rother. Graph cut based image segmentation with connectivity priors. 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8. IEEE, 2008.
S. Wang, Y. Wang, and Y. Li. Efficient map reconstruction and augmentation via topological methods. In Proc. 23rd ACM SIGSPATIAL, pp. 25. ACM, 2015.
Pengxiang Wu, Chao Chen, Yusu Wang, Shaoting Zhang, Changhe Yuan, Zhen Qian, Dimitris Metaxas, and Leon Axel. Optimal topological cycles and their application in cardiac trabeculae restoration. In International Conference on Information Processing in Medical Imaging, pp. 80-92. Springer, 2017.
Ze Ye, Cong Chen, Changhe Yuan, and Chao Chen. Diverse multiple prediction on neuron image reconstruction. In International Conference Medical Image Computing and Computer-Assisted Intervention, pp. 468. Springer, 2019.
Tao Zeng, Bian Wu, and Shuiwang Ji. Deepem3d: approaching human-level performance on 3d anisotropic em image segmentation. Bioinformatics, 33(16))2555-256, 2017.
Yun Zeng, Dimitris Samaras, Wei Chen, and Qunsheng Peng. Topology cuts: novel min-cut/max-
flow algorithm for topology preserving segmentation in n-d images. Computer vision and image understanding, 11))))8199,, 2008.
Afra Zomorodian and Gunnar Carlsson. Computing persistent homology. Discrete Computational Geometry, 33(2):249 2005.
Qin Zou, Yu Cao, Qingquan Li, Qingzhou Mao, and Song Wang. Cracktree: Automatic crack detection from pavement images. Pattern Recognition Letters, 33(3):227-238, 2012.
13

---

Published as a conference paper at ICLR 2021
A APPENDIX
In Sec. A. we will discuss different types of topological errors. In Sec. A.2 we will provide more details the method part. In Sec. A.3 more details and results for the experiments will be provided.
A.1 DIFFERENT TYPES OF TOPOLOGICAL ERRORS
A.1.1 ILLUSTRATION 3D TOPOLOGICAL ERRORS
In Sec.2.1 of the main paper, we have already introduced discrete Morse theory with a 2D example. Here, we would like illustrate 3D topological errors with 3D examples.
Fig. and Fig. 9 iillustrate two different types of topological errors for 3D data. Fig illustrates an index-1 topological error for 3D synthetic data. 3D EM/neuron has the same type of topological error as the synthetic data. Fig illlusratte index-2 2 topological error for 3D vessel data.
S
Figure 8: Illustration of an index-1 topological error (3D hole in the middle) for 3D synthetic data. The ground truth is complete sheet without the hole. We intentionally weaken the likelihood function in the middle. So the segmentation has hole in the middle. Left: 3D segmentation result. S is a saddle point of the likelihood function. Its Hessian has 1 negative and positive eigenvalues. The stable manifold the saddle point is a 2D plane going through the saddle point and cutting the segmentation into two thin slices. Right: the likelihood function visualized on the 2D stable manifold of S. Red arrows illustrate how different V-paths (streamlines of negative gradient) flow to the saddle S.
A.1.2 FALSE NEGATIVE AND FALSE POSITIVE ERRORS
In the main paper, have mentioned that the proposed DMT-loss can capture and fix two different types of topological errors: false negative and false positive. illustrate these two types in Fig. I0 The two highlighted red rectangles represent the two types of topological errors: 1) The red rectangle on the right represents a sample of false negative error: part of the membrane structure is missing, due blurred region the membrane. 2) The red rectangle the left represents a sample of false positive error. In this specific case, is caused by mitochondria which are not the boundary of neurons.
In summary, with the help of the proposed DMT-loss, we can identify both these two types of topological errors, and then force the network increase/decrease its likelihood value on these structures to correctly segment the images with with correct topology.
A.2 ADDITIONAL DETAILS ON THE METHOD
A.2.1 DISCRETE MORSE THEBRY
We view dD image, d = 2 or 3, d-dimensional cubical complex, meaning it consists of 0-, 1-, 2- and 3-dimensional cells corresponding to vertices, edges, squares, and voxels (cubes) as its building blocks.
Discrete Morse theory (DMT), originally introduced in (Forman) 1998 2002), is combinatorial version of Morse theory for general cell complexes. There many beautiful results established for
14

---

Published as a conference paper at ICLR 2021
S
S
Figure 9: Illustration of an index-2 topological errоr from a 3D vessel image. The vessel segmentation has a broken connection near bottom-left of the Left image. Left: part of the 3D segmentation result. A The saddle point S corresponds to a broken connection. The Hessian of the likelihood at the saddle point has 2 negative and positive eigenvalues. Right: One slice of the 3D likelihood map passing the saddle point. The saddle point (yellow) and its I-stable manifold (red) are also drawn.
(a)
(b)
(c)
(d)
Figure 10: Illustration of two different types of topological errors captured by DMT-loss. (a) an
input neuron image with challenging locations highlighted. (b) likelihood map of a baseline method without topological guarantee (Ronneberger et al.] (015). (c) ground truth. (d) The topologically critical structure from the likelihood, captured by the proposed discrete Morse algorithm. These structures will be used in the DMT-Loss.
DMT, analogous classical Morse theory. We will however only briefly introduce some relevant concepts for the present paper, and we will describe it in the setting cubical complexes (instead of simplicial complexes) as it is more suitable for images.
Let K be a cubical complex. Given a p-cell т, we denote by a < T if a (p 1>-dimensional face for A discrete gradient vector (also called V-pair for simplicity) is pair (r, or) where < Now suppose we are given a collection V-pairs M(K) over the cubical complex K. sequence
.ooti where the superscript in a stands for the dimension of this cell a. form a V-path (Ti, o.)) E M(K) for for any E [1, k:] and o; < Ti-1 for any [1, k + 1]. A V-path is acyelic if (7o, 0++1) $ M(K). This collection of V-pairs M(K) form a discrete gradient vector field 5 if (cond-i) each cell in M(K) can only appear in at most one pair in M(K); and (cond-ii) all V-paths in M(K) are acyclic. Given discrete gradient vector field M(K). simplex
of cells π : -8 -f ,ah," . p+1
Tk
K is critical if it is not in any V-pair in M(K).
Even though a discrete gradient vector V-pair), say (7, .) is combinatorial pair instead of real vector, it still indicates a "flow" from 7 to its face a. A V-path thus corresponds to flow path (integral line) in the smooth setting. However, to make a collection of V-pairs valid analog of gradient field, (cond-i) says that at each simplex there should only be one "flow" direction; while (cond-ii) is necessarily as flow lines traced by gradient can only go down function values and thus never come back (thus acyclic).
We will not introduce the concept of discrete Morse function, the discrete gradient vector field is sufficient define all relevation notations.
15

---

Published as a conference paper at ICLR 2021
A critical simplex has "vanishing gradient" it is not involved in any V-pair in M(K) (i.e, there is no flow at this simplex). Given 2D cubical complex K a a discrete gradient vector field M(K), we can view critical 0-, 1-, 2- and 3-cells minima, saddle points, and maxima, respectively. If K is 3D, then we can view critical 0-, 1- 2- and 3-cells as minima, index- saddle, index-2 saddle and maxima, respectively.
Hence, a 1 I-stable manifold in 2D will correspond to V-path connecting a critical square (a maximum) and critical edge (a saddle), while in 3D, it will be a V-path connecting a critical cube and a critical square.
Morse cancellation. A given discrete gradient field M(K) could be noisy, e.g. there shallow valleys where the mountain ridge around it should be ignored. Fortunately, the discrete Morse theory provides an elegant and purely combinatorial way cancel pairs of critical simplices (and thus reduce their stable manifolds). In particular, given M(K), a pair of critical simplices (8(++1)00)) is cancellable if there is a unique V-path π = 5 = ōn, Ti, ōi..... da, 7 y from 6 to The Morse cancellation operation simple reverse all V-pairs along this path by removing all V-pairs along these path, and adding (ő (dt-1, 7%) to M(K) for any i E [1, + 1). It is easy to check that after the cancellation neither ó nor y is critical.
A.2.2 PERSISTENCE PRUNING
We can extend this vertex- valuue function p to function p K + R. by setting p(or for each cell to be the maximum p-value of each vertex in a. How to obtain discrete gradient vector field from such function p: K + R? Following the approach developed in (Wang al., 2015 Dey al., 2018). we initialize a trivial discrete gradient vector field where all cells are initially critical. Let r 0 be
a threshold for simplification. We then perform persistence algorithm (Edelsbrunner et al. 2000)
induced by the super-level set filtration of p and pair up all cells in K, denoted by Pp(K).
Persistent homology is one the most important development in the field of topological data analysis in the past two decades (Edelsbrunner & Harer 2010, Edelsbrunner et al.) 2000 Zomorodian & Carlsson| 2005). We will not introduce it formally here. Imagine we grow the complex K by starting from the empty set and gradually include more and more cells in decreasing P values. (More formally, this is the so-called super-level set filtration of induced by p.) Through this course, new topological feature can be created upon adding simplex a, and sometiems feature can be destroyed upon adding a simplex т. Persistence algorithm (Edelsbrunner et al.] 2000) will pair up simplices; that is, its output is a of pairs of simplices PA(K) {(o,T) where each pair captures the birth and death of topological features during this evolution. The persistence of a pair, say p = (o, )). is defined as (prs(p) = p(o) p(T), measuring how long the topological feature captured by p lives in term of p. In this case, we also write pers(ar) )- = pers(T) = pers(p) the persistence of simplex (say a or 7) can be viewed as the importance of this simplex.
With this intuition of the persistence pairings, we next perform Morse-cancellation operation to all pairs of these cells (a. E (K) in increasing order their persistence (i) its persistence pers(o, 0, 7) < (i.e, this pair has low persistence and thus not important); and (ii) this pair (6,7) is cancellable.
Let (K) be the resulting discrete gradient field after simplifying all low- wwwppersiteence critical simplices. We then construct the 1-stable and 2-stable manifolds for the remaining (high persistence, and thus important) saddles (critical 1-cell and 2-cells) from (K(.. Let ()()) and Sy(e) be the resulting collection of I- and 2-stable manifolds respectively. In particular, see an illustration of V-path (highlighted in black) corresponding to 1 1-stable manifold of the green saddle in Fig. 3(c).
A.2.3 MORE DETAILS ON THE APPROXIMATION OF S₂ VIA Sz.
We approximate S2 by taking the boundary of the stable manifold of the minima (basins/valleys in the terrain). This is like watershed algorithm: growing the basins from all minima until they meet. The stable manifolds of the minima is approximated using spanning trees. This algorithm is inspired by the continuous analog for Morse functions.
16

---

The resolution for each image is Published as a conference paper at ICLR 2021
A.3 EXPERIMENTS
A.3.1
DATASETS
We conduct experiments six different 2D datasets. The first three datasets are neuron images (Electron Microscopy images). The task is to segment membranes and eventually partition the image into neuron regions.
CREMI contains 125 images. The resolution for each image is 1250x 1250.
ISBI12 Arganda-Careeras et al. 2015) contains 30 images. The resolution for each image is 512x5 12. ISBI13 Arganda-Carreras et al. 2013) contains 100 images. 1024x1024.
The next three are natural image datasets, and their structures are vital for their functionality.
CrackTree (Zou et al. 2012 contains 206 images of cracks in road. The resolution for each image 600x800.
Road MMih 2013) has 1108 images from the Massachusetts Roads Dataset. The resolution for each image is 1500x1500.
DRIVE (Staal et al. 2004) retinal vessel segmentation dataset with 20 images. The resolution for each image is 584x565.
A.3.2 EVALCATION METRICS
We use five different evaluation metrics to evaluate the proposed DMT-loss.
Pixel-wise accuracy: Pixel-wise accuracy is of the most common metrics which measures the percentage of correctly classified pixels.
DICE score: DICE score (also known DICE coefficient, DICE similarity index) the same the Fl score.
Adapted Rand Index (ARI): ARI is the maximal F-score of the foreground-restriited Rand index, measure of similarity between two clusters. On this version of the Rand index we exclude the zero component of the original labels (background pixels of the ground truth).
Variation of Information (VOI): VOI is measure of the distance between two clusterings. It is closely related mutual information; indeed, it simple linear expression involving the mutual information.
Betti number error: which directly compares the topology (number of handles) between the segmentation and the ground truth. We randomly sample patches over the segmentation and report the average absolute difference between their Betti numbers and the corresponding ground truth patches.
A.3.3
QUANTITATIVE RESULTS FOR MORE 2D DATASETS
Table. shows quantitative results for ISBI12 and DRIVE.
A.3.4 FAIRNESS COMPARISONS WITH SAME BACKBONE NETWORKS
We copy the numbers of TopoLoss from Hu et al. 2019), which LopoLLss+DIVE. And in this paper, we use U-Net as the backbone. Indeed, with U-Net, TopoLoss will be worse and the gap will be even bigger. The DIVE used in (Hu et al. 2019) is more expensive and better designed specifically for EM images. We choose U-Net this manuscript as it is lightweight and easy to generalize to many datasets. We also apply our nackbone-agnostee DMT-loss to the DIVE network (Fakhry et al.) 2016). All the experiments are conducted on CREMI 2D dataset. The quantitative results (Betti Error) are shown in the Table
A.3.5 QUALITATIVE RESULTS FOR 3D DATASETS
Fig. 11 shows qualitative results for ISBI13 dataset.
17

---

Published as a conference paper at ICLR 2021
Table 3: Quantitative results for different models several 2D medical datasets
Method
Accuracy
DICE
ARI
VOI
Betti Error
ISBII2
DIVE
0.9640 + 0.0042 0.9709 + 0.0029 0.9434 + 0.0087
1.235 - 0.025
3.187 + 0.307
U-Net
0.9678 0.0021 0.9699 + 0.0048 0.9338 + 0.0072
1.367 - 0.031
2.785 + 0.269
Mosin.
0.9532 + 0.0063
0.9716 + 0.0022 0.9312 + 0.0052
0.983
+ 0.035
1.238 + 0.251
TopoLoss
0.9626 0.0038 0.9755 = 0.0041 0.9444 = 0.0076
0.782 = 0.019
0.429 = 0.104
DMT
0.9593 + 0.0035
0.9796 = 0.0033 0.9527 = 0.0052
0.671 = 0.027
0.391 = 0.114
DRIVE
DIVE
0.9549 0.0023 0.7543 0.0008 0.8407 + 0.0257
1.936 0.127
3.276 0.642
U-Net
0,9452 + 0.0058 0.7491 + 0.0027 0.8343 + 0.0413
1.975
0.046
3.643 0.536
Mosin.
0.9543 0.0047 0.7218 + 0.0013 0.8870 + 0.0386
1.167 : 0.026
2.784 + 0.293
TopoLoss
0.9521 + 0.0042 0.7621 = 0.0036 0.9024 = 0.0113
1.083 = 0.006 1.076 = 0.265
DMT
0.9495 0.0036 0.7733 + 0.0039
0.9077 + 0.0021
0,876 = 0.038
0.873 = 0.402
Table 4: Comparison with same backbones
U-Net
DIVE
TopoLoss 1.451 0.216 1.113 0.224
DMT-Loss 0.982 + 0.179 0.956 + 0.142
A.3.6 THE ABLATION STUDY FOR BALANCED TERM B.
We conduct another ablation study for the balanced weight of parameter 3. Note that, the parameter 3 is dataset dependent. We conduct the ablation experiment on CREMI 2D dataset. When 8 = 3. the proposed DMT-loss achieves best performance 0.982 (Betti Error). When 3 = and 3 = 4, the performance drops to 1.074 and 1.181 (both in Betti Error), respectively. The parameter 3 is also chosen via cross-validation.
A.3.7 COMPARISON WITH OTHER SIMPLER CHOICES
Table shows shows the results comparing with Canny edge detection and ridge detection
Table 5: Comparison with other simpler choices
Method
Accuracy Betti Error
DMT
0.9475
0.982
Canny edge detection
0.9386
2.971
Ridge detection
0.9443
2.507
A.3.8 RESULTS WITH DIFFERENT NOISE LEVELS
In Table 6 10% means the percentage of corrupted pixels, and 8/20 means the sdv of the added Gaussian noise. For reference, we note that the performance of the standard U-Net is 3.016 (Betti Error).
18

---

Published as a conference paper at ICLR 2021
Figure 11: Segmentation results for ISBI13 dataset and 3 randomly selected neurons.
Table 6: Results with different noise levels
Method
Accuracy Betti Error
DMT
0.9475
0.982
Gaussian (10%, 8) 0.9393
1.086
Gaussian (20%, 8)
0.9272
1.391
Method DMT
Accuracy Betti Error
0.9475
0.982
Reweighted CE
0.9481
2.753
19


Reviews:
[{"review_id": "jR0bNjWb89_", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper focus on the segmentation of images in which the correct topology of the segmentation an important role plays. The key idea of the paper is the use of Discrete Morse Theory to identify those areas of the segmentation/likelihood map that are important for ensuring the correct topology of the segmentation. This is achieved by modulating the normal binary cross entropy loss function so that areas of Morse structures are focused on during the learning. \n\nWhile DMT  is not my area of core expertise, I believe that the proposed use Morse structures for the end-to-end training of a network is novel and a very interesting idea. While the basic idea sounds simple, the implementation is not that simple. In particular, the computation of Morse structures is slow and also sometimes unstable. The paper addresses all of these issues nicely (adopting a fast approximation and pruning step).\n\nThe paper is well written but still difficult to read for somebody who is not very familiar with DMT. The proposed approach has been well evaluated on a number of biomedical and non-biomedical datasets. This evaluation includes the comparison to other approaches with and without topology-aware segmentation.", "extra_scores": {}, "extra_text": {}}, {"review_id": "gnyGXID427W", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary and contributions\nThis paper tackles the problem of topological correctness of segmentation of fine structures.\nEspecially for the study of biological or medical processes this is often essential.\nThe authors propose to use discrete Morse theory (DMT) to help a deep neural network focus on areas that are currently topologically incorrectly segmented.\nThe result of DMT is similar to the classical watershed transform, yet provides a sound mathematical basis.\nMoreover the paper claims that commonly used metrics often do not value topological correctness enough and proposes to use metrics that are more in line with this goal.\n\n## Strengths\n- The paper combines theoretical advances in topology and Morse theory with current practical methods\n in deep learning. Thus it tries to motivate its approach with a sound mathematical basis.\n- The imbalances often present in datasets with fine structures, resulting in a sampling bias, are are alleviated using global features.\n- It is written in a clear and comprehensible way\n- The method is evaluated on multiple datasets and compared to a range of related work\n\n## Weaknesses:\n- Many details are missing to make this work reproducible\n- The method uses topology to create a mask for the loss, however the topology is not directly enforced in the loss\n- Even though most of the datasets are from challenges, the results are not compared to the current leader board\n\n\n## Clarity:\nFor the most parts the paper is written very clearly. The introductions to Morse theory are well understandable.\nThe related work part is a little dense, making it hard to extract really relevant work.\n\n\n## Reproducibility:\nIn the current form it is not possible to reproduce the results without considerable effort.\nThere are many parameters missing.\n- It is mentioned that epsilon is chosen using cross-validation, but not in which value this resulted, the same holds for beta.\n- The authors mention that a neural network architecture inspired by the U-Net is used, however all details about the specifics of the used networks are missing (dimensions, layers, feature maps, learning rate, used optimizer etc)\n- The authors mention required training time, however without specifying the used hardware or number of steps.\n- Will the code be published? Is existing code used to compute the Morse structures or did the authors implement it?\n\n\n## Additional feedback, comments, suggestions for improvement\nGeneral comments:\n- It is a nice aspect for the reader that the authors tried to distinguish significant from insignificant improvements over previous methods.\n\nRequests for clarification:\n- Could the authors please clarify how they use the word epoch? Usually it is used to describe one pass through the whole dataset, but it sounds like\nit is used here to refer to a single step.\n- Page 5, computation: Could the authors please clarify how the (costly) persistence computation can be avoided if it is needed, according to the next\nsentence, to distinguish between below and above the persistence threshold epsilon.\n- The result seems to be similar to an increased weight for FP,FN and TP pixels or a decreased weight for TN pixels.\nCould the authors please elaborate on how the effect of their method differs from this.\nHave the authors performed experiments with this as a baseline/part of an ablation study (with cross-validated weights)?\nThis would be very interesting.\n\nClaims that might be formulated a bit to strongly:\n- The authors correctly claim that unbalanced data is an issue and that just using more annotated data (with the same unbalance) is often not a simple\nremedy. This is often true, however, a number of approaches have been developed to alleviate this problem and more annotations with more hard cases\ncan be helpful for it.\n- The authors claim that \"by focusing on more critical locations early, our method is more likely to escape poor local minima of the loss landscape\".\nCan the authors please elaborate on the experimental basis for this claim?\n- The authors claim, citing related work (Hardt et al), that shorter training time might improve performance. However, just the same, there is related\nwork claiming the opposite ([1]). This seems to be an open problem and thus speculating about it should be founded in experimental data.\n- Fig 1, caption (\"baseline ...without topological guarantee\"): This gives the impression that the presented method provides topological guarantees.\nIn the introduction the authors mention that the method tackles the sampling bias, which sounds reasonable, but it does not provide guarantees. If it\ndoes provide guarantees, could the authors please elaborate on how these are strongly enforced?\n\nSuggestions:\n- The authors nicely cite a lot of related work. However, it would be interesting how they are different and how the work at hand tackles specific\nproblems that are left open by them. Or if they solve the same problem with a very different approach, which is also perfectly fine and still valuable\nfor the field.\n- The complexity given for 3d images for S_1 gives, to a cursory look, the slightly misleading impression that it is still n log n whereas it seems to\nbe more than quadratic due to the n^w in T.\n- The conclusion might benefit from a few more details regarding for example remaining issues.\n- I understand that there is a page limit, however the discussion of the results comes up a bit short.\n\nTypos:\n- Page 4, Sec 2.2: on the the original definition\n\n## note about review:\n- I cannot fully judge the formal correctness of the authors' description of DMT.\n\n[1] Hoffer et. al.: Train longer, generalize better: closing thegeneralization gap in large batch training of neural networks\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Gsx31zUTDBm", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces an elegant, yet not fully novel, idea: use a discrete morse complex to define a new loss for image and volume segmentation. The idea of using topology in the loss is similar to TopoLoss, but the use of larger critical structures makes this approach slightly outperform topoloss.\n\nThe paper is in a difficult spot: most of the paper (pages 2-6) provides an overview of Morse theory, the discrete version, and an algorithm to compute it on images. I don’t see any novelty there, as it seems to be all coming from the cited papers, except for the loss in section 2.3. Could you clarify if this is the case or if there is anything else that is novel? If indeed that is the only novelty the paper could be shortened by citing the corresponding papers instead of repeating well-known material  (or maybe moving it to an appendix to make it self-contained).\n\nThe comparison is extensive and while the method outperforms most of the baselines, the results are quite similar (2D) to TopoLoss (which uses a loss based on persistent homology, closely related to the one proposed here), with minor differences (the more noticeable in the betti error, which is ~10-15% better). In 3D the difference is even smaller and there is no clear winner between the two.\n\nI do not understand how the Morse structure used in the loss can be differentiable. The authors state explicitly that both the structure and the mask change not continuously. I don’t understand the logical step that leads to the conclusion that the loss is differentiable, could you clarify?\n\nOverall, the paper is a variant of Topoloss which has slightly better results. I am neutral on this paper as I am not familiar with the expectations for an ICLR paper. I don’t think the paper novelty is sufficient for acceptance, but there are measurable numerical improvements in the 2D results that might be useful in applications (no applications of the resulting segmentations are shown in the paper, unfortunately).", "extra_scores": {}, "extra_text": {}}, {"review_id": "8L-lUURRcw9", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This manuscript introduces a segmentation framework that combines discrete morse theory to encourage topological correctness of the segmentation. Based on the evaluation on both 2D and 3D datasets, the proposed method has the potential to improve segmentation in selective applications where the topology is of great interest. \n\nSome constructive feedbacks:\nWhile the Morse theory itself seems to be interesting, the way it is incorporated with deep learning can be better justified. The MST algorithm is not a functional, is non-deterministic, and changes with iterations, so the converging property of the objective function is not clearly studied.\n\nBoth hyperparameters are not chosen by inner cross-validation. Instead, the optimal results reported for the proposed method are based on a search via cross-validation, which is not the case for other baselines according to my reading.\n\nThe proposed method essentially highlights segmentation accuracy in geometrically rich regions. I wonder how essential it is to have the Morse theory compared to other simpler choices, e.g. edge detector, ridges/valleys estimator, or even just a gradient field.\n\nOften times the biggest drawbacks for conventional geometric methods are their inferior generalizability and vulnerability to noise. An experiment with respect to noise level can be very helpful\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel loss for segmentation tasks, which incorporates reasoning about topological accuracy of predicted segmentations vs. ground truth. All reviewers, after the rebuttal period, recommend acceptance, and I agree -- it's an interesting paper, offering a potentially useful and clearly novel building block for training segmentation models.
Decision: Accept (Spotlight)