Localized Persistent Homologies for more Effective Deep Learning
Authors: Doruk Oner
Adélie Garin
Mateusz Kozinski
Kathryn Hess
Pascal Fua
Venue: ICLR 2022 Submitted (2022)
Keywords: Delineation
Persistent Homology
Topology
Aerial Images
Microscopy scans

Abstract: Persistent Homologies have been successfully used to increase the performance of deep networks trained to detect curvilinear structures and to improve the topological quality of the results. However, existing methods are very global and ignore the location of topological features. In this paper, we introduce an approach that relies on a new filtration function to account for location during network training. We demonstrate experimentally on 2D images of roads and 3D image stacks of neural processes that networks trained in this manner are better at recovering the topology of the curvilinear structures they extract.

Under review as a conference paper at ICLR 2022
PERSISTENT HOMOLOGY WITH IMPROVED LOCALITY INFORMATION FOR MORE EFFECTIVE DELINEATION
Anonymous authors Paper under double-blind review
ABSTRACT
We present a new, more effective way to use Persistent Homology (PH), a method compare the topology of two data sets, for training deep networks to delin- cate road networks in acrial images and neuronal processes in mieroscopy scans. Its essence is in a novel filtration function, derived from fusion of two existing techniques: thresholding based filtration, previously used to train deep networks to segment medical images, and filtration with height functions, used before for comparison of 2D and 3D shapes. We experimentally demonstrate that deep net- works trained with our Persistent-Homollee Howeeeet-ooooooobbassaoaaee -baed loss yield reconstructions of road networks and neuronal processes that preserve the connectivity of the origi- nals better than existing topological and non-topological loss functions.
INTRODUCTION
In many image segmentation tasks, the topology of the resulting mask as important as, if not more than, its pixel-wise accuracy. For example, a model of an aortic valve that does not form a ring biologically implausible. Similarly, networks of curvilinear structures- they roads in aerial images, blood vessels in Computer Tomography (CT) scans, or dendrites and axons in Light Microscopy (LM) image stacks-should not feature breaks that disrupt connectivity or false connections between disjoint structures. Unfortunately, deep networks trained by minimizing pixel- wise loss functions, such as the cross cross-entropy or the mean square error, are subject to such mistakes. This is in part because it often takes very few mislabeled pixels to alter the topology significantly with little impact on the pixel-wise accuracy. In other words, it is possible for a network trained in this manner deliver both good pixel classification accuracy and an incorrect topology.
Specialized solutions to this problem have been proposed in the form of loss functions that compare the topology of the prediction to that of the annotation. They are effective for specific applica- tions but do not naturally generalize. For example, the perceptual loss of (Mosiñska et al., (018)) penalizes topological differences between the prediction and the ground truth, but cannot be guaran- teed to detect them all. Similarly, minimizing the MALIS loss for segmenting electron microscopy scans (Briggman
al., 2009)] Funke et al., 2018) yields better region boundaries but does not pe- nalize interruptions in loopy linear structures. This has been addressed by (Oner et al.) (02TT) for delineation of 2D road networks but the proposed solution not applicable to 3D image stacks.
Persistent Homology (PH) (Edelsbrunner & Harer 2008) is an elegant approach to describing and comparing topological structure of data, which is well-established in the field of topological data analysis (TDA), It offers the promise to address the connectivity problem in generic way, both for 2D and 3D images. Homology the study of topological features in an object, such its connected components (0-homology classes), loops (1-homology classes) and closed surfaces (2-homology classes). Persistent homology detects homology classes in objects filtered at different scales. A homology class which appears particular scale and disappear at larger one is represented by a scale interval called a persistence interval. The set of persistence intervals for all the homology classes characterizes the overall topology of the structure. It can be represented by a persistence di- agram. The similarity of these diagrams across two different structures can then be used to quantify their topological similarity. This has been successfully exploited to train deep networks for delin- eation (Hu et al. (2019). image segmentation (Hu et al. 2019 Clough et al. 2019 2020) and crowd counting (Abousamra al. 202T).
However, existing techniques do not unleash the full power of persistent homology because the persistent diagrams are global image descriptors that ignore the location of the topological features,

---

Under review as a conference paper at ICLR 2022
(b)
(d)
Figure 1: 2D and 3D delineation. (a) Acrial image and slice of microscopy stack. (b) A network trained
using standard homology- based loss yields road and neurite interruptions. (c) One trained using our localized loss more topologically accurate and produces predictions that closely resemble the ground truth (d).
which reduces their descriptive power. As shown in Fig. this can result in networks that still fail to enforce the proper topology. This is because, when training deep network, a persistence-biee based loss can be low even if the network predicts a structure that is quite different from the ground-truth To remedy this, we introduce new approach to computing persistence diagrams that takes location into account and increases their descriptive power, as shown in Fig. 3 Our main contribution is novel filtration technique that combines two filtrations commonly used in TDA: thresholding based filtrations and height functions. It results in loss that applies both to 2D and 3D images and significantly improves performance compared state-of of of-the-art topological methods.
RELATED WORK
Training a deep network that produces topologically correct segmentations has typically been done by designing loss functions that, when minimized, favor plausible topology. In this section, we briefly review first those that do not rely on Persistent Homology, and then those that do.
Losses designed to enforce topological correctness Several such losses have been proposed al- ready to go beyond pixel-wise classification accuracy by encoding more global properties. In (Li et al. 2020), the connectivity between neighboring pixel pairs is used as additional source of su- pervision. This approach has been shown to improve connectivity, but since disconnections or false connections are not penalized explicitly, there is no guarantee it captures all such errors. The per- ceptual loss of (Mosiñska et al. 2018) is based on the assumption that pre-trained neural network can capture differences of connectivity between the prediction and the ground-truie However, even though it has been shown experimentally to improve the topology of masks produced by deep net, there no guarantee that this assumption holds in general. Making the Rand index of segmentations produced by the network similar to that of ground truth ones (Briggman et al., 2009 Funke et al.) 2018 helps when modeling tree-like structures, both 2D and in 3D, but cannot prevent disconnec- tions in loopy structures. This shortcoming has been addressed by (Oner et al.) 2021 by detecting disconnections of 2D loopy structures as intercomnections background regions, but the proposed solution does not generalize 3D.
Losses that rely on Persistent Homology Persistent Homology (Edelsbrunner et al. 2000 Zomorodian & Carlsson 2004) is an established topological data deseriptor. Among its numer- ous applications is comparing topological structures of binary images. It has been used to enforce the correct Betti number on binary masks resulting from inference in Markov Random Fields (Chen et al. 2011). Recently, it has been demonstrated that persistence diagrams can be computed also for greyscale images and differentiated with respect to the pixel values (Hu et al. 2019 Clough et al.) 2019 Gabrielsson al. 2020; Leygonie et al. 2021 Carriere et al. 2021). Hence, they can be used as loss or prior terms for training deep networks.
this vein, (Clough et al. 2019) proposed loss term that enforces sequence of desired Betti numbers on the predicted segmentation. This approach was further extended to a loss function that tends to equalize the Betti number of the pre-
diction and the ground truth (Clough et al. 2020). (Hu et al. 2019) proposed instead to construct
2

---

Under review as a conference paper at ICLR 2022
input
filtered binary masks at different scales
persistence diagram
death
scale axis
bh
birth
Figure 2:
Filtration. When the distance map shown on the is filtered by thresholding. the loop emerges scale 6t and is filled scale d", This gives rise to the point (b", dd) ) in the persistence diagram shown on the right. Here, thresholding means retaining all pixels whose value lower than the threshold.
loss term around a difference of persistence diagrams of the prediction and the ground truth. Their persistence diagrams are obtained by thresholding the prediction and the ground truth. As we show in the next section, for binary ground truth this results in trivial persistence diagrams, that encode no more information than the ground truth Betti number. In consequence, both approaches can be terpreted as equalizing the Betti numbers of the prediction and the ground truth. (Wang et al., 2020) predicted and ground truth distance maps instead of
improved upon this technique by applying binary annotations and class affinity maps. As discussed in more detail in section 3 this makes the loss function more effective at detecting and penalizing topological errors. Unfortunately, even improved technique remains susceptible errors provoked by incorrectly matching the persistence diagrams of the prediction and the ground truth. By making the diagrams depend on location of the topological features in the image, method makes them more diverse and minimizes potential for erroneous matches.
It has also been proposed to detect disconnections in predicted 2D and 3D structures using Discrete Morse Theory (Hu et al. 2021). Topological features that are inconsistent with the ground truth are then penalized in the loss function. However, when the annotations lack spatial precision, which is often the case for neurite and road centerline annotation like the ones studied here, ground-truth inaccuracies may confuse the network. By contrast, technique allows for considerable misalign- ment between the prediction and the ground truth.
METHOD
We first introduce Persistent Homology and its application to characterizing two-dimensional images and three-dimensional image stacks. As PH provides global descriptors that ignore location of topological features, then introduce our approach to accounting for it.
3.1 PERSISTENT HOMOLOGY
In the interest of simplicity, we introduce PH for binary images and image stacks, where homology classes are limited to connected components, loops, and closed surfaces. We refer the interested reader to the review (Edelsbrunner et al. 2000) for a more general treatment. applicable to non- image and higher-dimensional data.
At the heart of persistent homology is detecting homology classes (connected components, loops, closed surfaces) many different scales. The ones that exist over wide scale range are called persistent and deemed more likely to represent true features, as opposed to sampling artifacts or noise. Here, scale has a very specific meaning. It refers to the parameter of filtration function F that is applied to an image X to produce topological objects called cubical complex. Cubical complexes resulting from filtering images and their properties are described for instance by (Garin al. 2020) and by (Bleile et al. 2021b. A reader not familiar with algebraic topology can think of them as binary masks. The masks obtained for different scales form a sequence inclusions, that is, for pair of scale parameters $ < $2, the mask F(X, $1) is entirely contained within the mask F(X, 82). The simplest example of a function for filtering grayscale images is thresholding, where the threshold acts as the scale, shown in Fig.
As the scale changes, homology classes in the filtered cubical complex emerge and disappear. To capture this, the scale range is sampled from small to large, the image is filtered at the selected scale values, homology classes in the resulting binary masks are detected algebraically (Edelsbrunner et al. 2000), and correspondence is established between the homology classes found consecutive
3

---

Under review as a conference paper at ICLR 2022
scales. For each class, this yields pair (b, d), where b is the scale at which the homology class appears and d the scale at which it disappears. We will refer to them as birth and death times and to the interval [6, d] as the persistence interval of the homology class. The set Px {(bb, d de where Hx is the set of all homology classes found in the filtered image X, is called the persistence diagram of X, and was first introduced by (Barannikov T994). In practice, we use the Gudhi library (Maria et al., (014) to compute persistence diagrams from images. Fig. 2 depicts the birth and death of a specific homology class.
To compare images X, and X2, one-to-one matching is performed between their persistence dia- grams, Px, and Pxg. with the cost of matching a homology 9 E Hx, homology h E Hx, set to Gg.h = (i/ - b0)3 + (d" - -d9)² and the cost of leaving an interval [4, d) unmatched is set to the distance between the point (6, d) and the diagonal in IR2. The optimal matching can be found using the Hungarian algorithm. Its cost that denote as C X2) quantifies the topological discrep- ancy between X, and X₂ by penalizing differences between corresponding homology classes and ones that only appear in either X X2.
3.2 TRAINING DEEP NETWORKS USING PH
Let f be a network that associates to an image X segmentation mask Y = f(X) such that for all pixels or voxels p E Y. 0 S Yip] S and let Y be the corresponding ground-truth mask. A natural idea then is to train f by minimizing
Lio = L(Y, Y) + +aC(Y, Y)
where L is the standard loss function, either the Mean Square Error, the Cross Entropy, and is a paraeee,rraeter, which we set to 0.01 in practice. This is possible because C' is sub-differentiabl with respect to its inputs when filtration achieved by thresholding, as shown before (Hu et al. 2019 Clough et al. 2019 Leygonie et al. 2021). However, when the ground truth Y is binary, as often is, all structures emerge at scale zero and disappear at scale one. Hence, as shown in Fig. 3] the persistence intervals all are [0, 1] and filtering it by thresholding is uninformative. An approach to handling this difficulty is to replace the binary ground truth by its distance transform that can be thresholded over a wide range of threshold values to create different binary masks (Wang et al. 2020). Unfortunately, computing the persistence diagram of a ground truth distance transtorm still yields persistence diagrams in which the topological features of the original, binary ground truth spread along the 'death' axis but not along the 'birth' one: The distance value at the structures themselves is zero and, as a result, all the loops of the ground truth mask appear as soon as the scale value becomes positive. As shown in Fig. Bb), this may lead to erroneous matches between the persistence diagrams, which encourages the deep network to produce wrong segmentations. Moreover, this approach ignores the location of homology classes within the image. This is sub- optimal, because the predicted topological features should not be too far from the ground truth ones.
3.3 FILTRATION THAT PARTLY LOCATES TOPOLOGICAL FEATURES
To remedy the above-mentioned drawbacks of traditional PH, our goal is therefore to spread the persistence diagrams along both dimensions while also accounting for where in the image the ho- mology classes are. To this end, we draw our inspiration from another filtration technique called the height function (Turner et al. 2014p. It was originally designed for dirce-dimensional meshes and can be applied to binary images by assigning to each pixel a height value that is the coordinate of its projection along a selected straight line. Filtration is carried out by forming binary masks made of pixels whose height is smaller than the scale parameter (Garin & Tauzin 2019). As the scale is increased, the binary image is revealed in scan-lines perpendicular the height axis, one scan-line at a time. The birth and death times are the heights of pixels responsible for the emergence and disappearing of homology classes. As a result, the persistence diagram contains partial information about the location of topological features. Moreover, both birth and death times of different homol- ogy classes are distributed across scales. Additionally, it has been shown that a binary image can be reconstructed from as few as four persistence diagrams obtained with height functions with well- chosen directions (Betthauser] 2018). A height function is only defined for binary images, but the abovementioned result inspired us to extend its definition by combining it with thresholding distance maps. Given a scale 8, the value of the filtered binary mask at coordinates p is taken to be
(2) where 1(-) evaluates to one if the condition in the bracket is satisfied and to zero otherwise. In essence, this amounts to thresholding the sum of the height function g and the pixel values. From
F(Y, 8) [p] 1(Ypp) + g(p) < <))
4

---

Under review as a conference paper at ICLR 2022
imput
filtered binary masks
pers. diag.
ground truth
B
A''
prediction
B
A'
(a) Filtration by thresholding binary ground truth and predicted class affinity maps. Here, filiration involves decreasing the threshold from I O, and retaining the pixels greater than the threshold. binary masks resulting from filtering the ground truth different scales are all the
Note, that same and that all points the ground truth persistence diagram (top-right) coincide. This results in erroneous matches between the predicted and ground truth homology classes. Minimizing loss function based such filtration magnify the error.
ground truth
Wi
prediction
D
(b) Filtration by thresholding distance maps distributes the topological features the ground truth along the vertical but not the horizontal axis. This still results in erroneous matching between the predicted and ground truth homology classes: Loop D' in the prediction emerges when the threshold is high enough make the road brake disappear. Hence, it remains unmatched and the E loop created by the false positive road is matched the ground truth loop D.
ground truth
E
prediction
A
D
(c) Our localized filtration of distance maps distributes the persistence diagram of the ground truth across the plane, promoting correct matches between predicted and ground truth homology classes.
Figure 3: Comparing filtration functions on synthetic data. The binary ground truth road annotation (top- left in each table part) contains four loops, marked with eyan dashed lines. We synthesized predicted class affinity map (bottom-left in each part) by extending one road to the left and interrupting another. In conse- quence, loop B and D from the ground truth joined into B' in the prediction, and A is split into A' and E'. For each filtration method, show binary masks resulting from filtration different scales, pairs of persis- tence diagrams, and their optimal matches.
the perspective TDA, such combination of two filtration functions can be seen as a line in the fibered barcode defined by (Carriete & Blumberg 2020)
5

---

Under review as a conference paper at ICLR 2022
Probebiity
33
C4
Change in loss function
Change in loss function
Probsbiity
63
-27
211
-27
21
Change funetion
Chatge funation
(b)
(c)
(d)
Figure 4: Sensitivity of the topological loss term to the number of injected errors Ground truth distance maps road networks. (h) Distance maps corrupted by introducing false roads and interruptions. We randomly injected one error at a time, obtaining corrupt distance maps with 30 errors. We repeated this simulation 10 times. (c.d) The distribution function of change in the loss term in response to injecting 30 errors. our filtration.
In (c). C' is evaluated using the filtration by thresholding distance maps, whereas in (d) The probability of decreasing the existing loss term by injecting additional errors is around 0.4, whereas for loss term drops to 0.2. We conclude that our loss term is more monotonic with respect to the error number.
In its simplest form, g is linear function of pixel coordinates, and the region highlighted for any extends along line perpendicular the height axis, as shown Fig. 3(... But other forms of g are also possible. We tested
linear functions g(p) = w wTp. p. where w is a two-vector
encoding the orientation
of the height axis and the slope of the height function;
scaled distance point q in the image, g(p) allp -0-1l22, where q and a hyper- paremeerrrmetess: www,
the square of the height function (p) = p WP,, where W w w, and w is the hyper parameter encoding the slope of the function and the orientation of the height axis;
The function introduces partial information of location of topological features into the persistence diagram. This is illustrated by Fig. 3 where different values of the scale parameter make homology classes appear in different parts of the image. But, because the scale parameter must be scalar, it only pinpoint location of topological features in 2D or 3D images along direction. This could be addressed by evaluating the loss function many times for many different orientations of the height axis, or more generally, for many different typer-parameteer of g. This approach is legit- imized by the theoretical result by (Betthauser 2018). 1). who proved that four well chosen filtration directions suffice to completely represent a binary image. The problem of combining number of different filtration functions is known in topological literature as multipersistence (Carlsson & Zomorodian] 2009). But current multipersistence techniques are not easily plugged into deep learning framework due to the lack of results their differentiabiiityy Moreover, filtering the data along multiple directions would considerably slow down training. Instead, we randomly draw the hyper-parameters of the height function at each training iteration. We show in the supplementary material that, in practice, the simple linear function performs best.
3.4 VALIDATION ON SYNTHETIC DATA
We motivated our filtration technique by the fact that it introduces partial localization of topological features into the persistence diagrams and better spreads the diagrams across the plane. We validated it on synthetic data to show that it correlates better with the number of errors injected into a distance map than the baseline loss based on thresholding distance maps. To that end, we took two crops of ground truth road graphs of the RTracer dataset (astani et al. 2018) and generated faulty synthetic eтror time. randomly selected between road disconnection
distance maps by injecting and a false interconnection with equal probability. We then evaluated the topological discrepancy C' equation using either filtration by thresholding distance maps, combined filtration. We

---

Under review as a conference paper at ICLR 2022
plotted the distribution function of the change in C' resulting from enror injection in Fig. When using the standard approach, injecting new errors is likely to decrease this loss term. Our approach results in twofold reduction of this probability.
4 EXPERIMENTS
We now describe the dataset we have tested our approach on, the baselines to which we compare our results, and the metrics we used to assess the topological correctness of the segmentations. We then demonstrate that our new loss improves the topological correctness segmentation masks. We provide additional qualitative results and an ablation study in the supplementary material.
4.1 DATASETS
We experimented on three datasets.
RTracer. A recently published dataset of high-resolution satellite images covering urban areas of forty cities in six different countries (Bastani et al., 20.8). The ground truth was obtained from OpenStreetMap. Like (Bastani et al., 2018 Li et al. 2019) Yang et al. 2019 Mosińska et al.) 2020), we used twenty five cities as the training set and the remaining fifteen as the test set.
Massachusetts. The Massachusetts dataset (Mnih 2013) features both urban and rural neigh- borhoods, with many different kinds of roads ranging from small paths to highways. For a fair comparison to (Hu et al. 2019), we split the data into three equal folds and performed a three-way cross validation.
Neurons. The dataset is a part of proprictary 3D, 2-photon microscopy scan a whole mouse brain. It contains 14 stacks of size 250 x 250 x 200 voxels and a spatial resolution of 1.0 0.3 0.3 uum. We used ten stacks for training and the remaining four for testing.
Brain. The dataset contains two 3D images of neurons in mouse brain. The axons and dendrites have been outlined manually while viewing the sample under a microscope and the image has been captured later. The sample deformed in the meantime, resulting in misalignment between the annotation and the image. We use twelve stacks of size 150 x 200 x 200 voxels and spatial resolution of pm for training and ten of them for testing.
4.2 METHODS TESTED
To test the impact of our proposed filtration functions, used the standard U-Net architecture (Ron- neberger et al., 2013). with four blocks, each with two sequences of convolution-Red Rel----eel- nor- malization. Max-pooling
x 2 windows followed each of the blocks. The initial feature size was set to 32 and grew to 512 in the smallest feature map in the network. We augmented the training data with vertical and horizontal flips and random rotations and used the ADAM algorithm (Kingma & Ba, 2015) with the learning rate set to le - 4. We then used different version of the Liot of Eq. we minimized to train the network. We tested the following as baselines:
UN. L is the Cross Entropy loss for pixel classification and there is no topological discrep- ancy loss, that is, a 0.0.
UNE L is the mean squared error of the truncated distance to the closest foreground pixel, with no topological discrepancy loss.
Homo-Pre. L is the cross Entropy loss and we compute C' by thresholding pixel classification maps., as in (Hu et al.) 2019 Clough et al. 2019: 2020).
Homo-Reg. L. is the mean squared error and we compute C by thresholding the truncated distance maps, as in Wang et al. (2020).
Homo-Ours. L is the mean squared error and compute C' using our proposed filtration function.
In the last three cases, we set a to 0.01 for all our experiments. Like Hu al. (2019), we compute the loss in windows sized 64 64 pixels, and limit the method to homology classes order 1, that is, loops. This has two advantages. First, by convention, loops are created by the borders of the window. making disconnections in dead-ending roads neurites detected broken loops. Second, detection of homology classes is computationally expensive, and the time grows cubically with the number of pixels. In current setup, computing the loss for single window takes 0.5 seconds. Similarly to (Hu et al.] 2019), we did not observe any performance gain due to using homology classes of order connectce components-inn addition loops.
7

---

Under review as a conference paper at ICLR 2022
For completeness, also compared approach to recent techniques relying on persistent ho- mologies Segmentation (Bastani et al. 2018), RoadTracer (Bastani et al. 2018). Seg-Path (Mosiñska et al. 2020), RCNNU-Ner (Yang a., 2019), DeepRoad [Mättyus et al. 2017), PolyMapper (Li 2019), DMT (Hu et al. 2021), ) and ConnLoss (Oner et al., 2021). Segmentation, RoadTracer, RCNNU-Net, and PolyMapper do explicitly enforce topology constraints, while the others do and are discussed in the related work section. The outputs of these methods were shared by the authors directly with us on the Internet, and we computed all the performance metrics.
et al.
4.3 PERFORMANCE METRICS
Comparing connectivity of segmentation masks is difficult, because the reconstructions rarely over- lap with the ground truth, and often deviate from it significantly. There seems to be consensus concerning the best evaluation technique; we found five ceneectiittyoriiente metrics in concur- rently published recent work. To provide exhaustive evaluation, used all of them.
APLS for Average Path Length Similarity. It is defined as an aggregation of relative length dif- ference of shortest paths between pairs of corresponding points in the ground truth and predicted maps Etten et al. (2018).
TLTS. It is statistics of lengths of shortest paths between corresponding pairs of end points randomly selected the predicted and ground-truth networks Wegner et al. (2033. We report the fraction of paths for which the relative length difference is within 5%.
JCT. is a junction score that considers the number of roads intersecting at each junction Bastani et al. (2018). It consists of road recall, averaged over the intersections of the ground-truth and road precision, averaged over the intersections of the prediction. We report the corresponding score.
Betri. The Betti error, as defined by (Hu et al. 2019), is an average absolute difference between the number of topological structures seen in the ground truth and predicted delincations. We take random patches sized 64 x 64 from predictions, compute the number of I-homology classes (loops) and compare the numbers computed for the prediction and the ground truth. We average this difference over 10 trials. practice, compute the error we use the code made publicly available by the authors.
CCQ We complement the connectivity-orientte with the most popular metric that measures spatial co-oceurrence of annotated and predicted road pixels, rather than connectivity. The Correctness, where the definition of a true positive has been relaxed from spatial coincidence of prediction and an- notation to co-occurrence within distance of 5 pixels Wiedemann et al. (1998). We report the Quality as our single-number metric.
Completeness and Quality are equivalent to precision, recall and pearsection-over-nnite
Table I: On the Massachusetts dataset, our loss function outperforms all PH-based loss functions. The results for our method are means and standard deviations over three independent training runs.
Connectivity-oriented
pixel-based
Method
APLS
TLTS
JCT
Betti
CCQ
UNer-CE
60.9 3.9
41.6 4.1 41.9
72.0 + 2.7
3.12 + 0.6
66.9 + 2.6
UNet-MSE
61.3 3.7
71.9 + 2.9
3.09 + 0.7
67.3 + 2.3
DMT
64.7 + 2.9
45.8 + 2.8
80.6 + 2.4
0.99 + 0.4
74.9 + 1.9
ComnLoss
73.4 + 3.6
53.2 - 444
81.4 + 1.9
1.29 1 0.5
75.8 + 2.2
Homo-Pre
62.5 + 1.9
42.1 + 1.9
74.2 1.7
1.28 + 0.3
69.3 + 1.9
Homo-Reg
65.0 2.2
45.6
76.9 + 1.9
1.09 + 0.2
71.8 + 2.1
Homo-Ours
68.7 + 1.2
50.6 2.3
79.2 + 2.6
0.90 + 0.3
74.9 + 1.8
4.4 COMPARATIVE RESULTS
As shown in Tabs I and on the Massachusetts and RTracer data sets, our method outperforms the other methods based on Persistent Homology, which demonstrates that our approach to filtering is truly effective. It also outperforms the other 2D tracing algorithms targeted at handling aerial images, RoadTracer, Seg- Path, DeepRoad, and PolyMapper, at the exception of ConnLoss that does marginally better. This is presumably because ConnLoss explicitly penalizes each disconnection of the prediction, whereas persistence diagram is lossy topological descriptor that may fail to
8

---

Under review as a conference paper at ICLR 2022
Table 2: Our loss function outperforms all PH-based loss functions on the RTracer dataset. Means and standard deviations over cities from the test set are reported.
Connectivity-
pixel-based
Method
APLS
TLTS
JCT
Betti
CCQ
UNel-CE
63.4 +: 1.6
37.5 1 1.9
78,0 +: 1.0
3.08 1 0.6
59.7 1: 2.2 59.5 + 1.9
UNet-MSE
66.3 + 1.9
40.0 2.0
77.5 + 1.3
2.99 = 0.5
Segmentation
62.5 1.5
33.0 1.6
78.2 1.5
3.04
54.4 + 1.0
RoadTracer
59.1 0.8
1.5
81.2
2.85
47.8 + 1.6
Seg- Peg-Path
1.
1.7
7
0.4
54.0
RCNNU-Net
1.6
18.1
1.4
3.25
62.8 1.5
DeepRoad
24.
6.4
51.4
1.0
43.6:200
PolyMapper
2.3
3.95
8..0
0.9
35.7 1.4
ConnLoss
75,4 1 1.6
1.4
82.6
1.30
68.4 = 0.9
Homo-Pre
67.3 1 1.7
42,3
78.7 0.9
1.32 182:0.
61.9 1 1.9
Homo-Reg
69.9 1 16
45.1
79.6
1.3
63.2 1 1.6
Homo- Ours
73.8 1.8
47.8
81.3 1.6
0.89 1
1.7
Table 3: Comparative results on the Neurons dataset. Our loss outperforms all the baselines. The results for method means and standard deviations over three independent training runs.
Comnecivity-oriennee
pixel-based
Method
APLS
TLTS
Betti
CCQ
UNer-CE
79.9 1.5
80.8 + 2.2
2.33 0.6
90.6 +:: 2.0 90.4 1.9
UNet-MSE
80.2 1.6
80.9 = 2.0
2.31 0.7
Homo-Pre
83.5 - 1.0
82.1 1.7
1,06 - 0.2
91.2 1.8
Homo-Reg
85.4 1.2
83.4 1.5
0.91 0.2
92.5 1.6
Homo- Ours
86.9 - 1.1
85.2 + 1.9
0.80 0.2
93.3 = 1.9
Table 4: Comparative results on the Brain dataset. Our loss outperforms all PH-based losses. The results for our method are means and standard deviations over three independent training runs.
Comneciivity-oriented
pixel-based
Method
APLS
TLTS
Betti
CCQ
UNer-CE
65.8 1.8
63.6 + 1.3
2.89 0.4
70.4 1.9 70.6 1.8
UNEt-MSE
66.0 1.6
63.9 1.4
2.92 0.5
Homo-Po Pre
67.6 1.5
1.
1.39 0.2
71.5
Homo- Reg
70.5 1.5
68.8
1.3
72,6 1.3
Homo-Qurs
73.4 + 1.4
70.1 1.1
1.06 + 0.2
73.2 + 1.2
penalize some errors. However, ConnLoss does not naturally extend to 3D data, whereas our method does. On the 3D Neurons data set, it outperforms the competing algorithms, as evidenced by the results shown in Tab. 3 We provide qualitative results in the supplementary material.
5 CONCLUSION
We proposed improved approach using Persistent Homology to train deep networks to delin- cate curvilinear structures. It outperforms current such approaches by introducing an element of location in the filtration process. Unlike other powerful approaches (Oner et al. (02T)) to enforcing topological constraints on the output of deep networks, it generalizes naturally to 3D, which opens the door to future research in space that is critical for biomedical applications.
To further increase performance, we will address the fact that our proposed loss function has sparse gradients that only depend values at pixels that are critical for emergence and disappearance of topological features. This limits robustness and our future work will focus on developing more sophisticated topological descriptors with more smooth gradients.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Shahira Abousamra, Minh Hoai, Dimitris Samaras, and Chao Chen. Localization in the crowd with topological constraints. In AAAI Conference on Artificial Intelligence, 2021.
S. Barannikov. The framed Morse complex and its invariants. Advances in Soviet Mathematics, 21: 93-115, 1994.
Bastani, S. Hc, M. Alizadeh, H. Balakrishnam, S. Madden, S. Chawla, S. Abbar, and D. Dewitt. Roadtracer: Automatic Extraction of Road Networks from Aerial Images. In Conference Computer Vision and Pattern Recognition, 2018.
I.. Betthauser. Topological reconstruction of grayscale images. PhD thesis, University of Florida, 2018.
B. Bleile, A. Garin, T. Heiss, K. Maggs, and V. Robins. The persistent homology of dual digital image constructions, 2021.
K. Briggman, W. Denk, S. Seung, M. Helmstaedter, and S. Turaga. Maximin Affinity Learning of Image Segmentation. In Advances in Neural Information Processing Systems, pp. 1865-1873, 2009.
G. Carlsson and A. Zomorodian. The theory of multidimensional persistence. Discret. Comput. Geom., 42(1))71-99) 2009. doi: 10. 0011077//00444400999917666
M. Carriere, F. Chazal, M. Glisse, Y. Ike, H. Kannan, and Y. Umeda. Optimizing persistent homol- ogy based functions. International Conference on Muchine Learning, volume 139 of Proceed- ings of Machine Learning Research, pp. 1294-1303, PMLR, 2021.
Mathieu Carriete and Andrew Blumberg. Multiparameter persistence image for topological machine learning. In Advances in Neural Information Processing Systems, volume 33, pp. 22432-22444. Curran Associates, Inc., 2020.
C. Chen, D. Freedman, and C. Lampert. Enforcing topological constraints in random field image segmentation. In Conference on Computer Vision and Pattern Recognition, pp. 2089-2096, 2011.
Clough, I. Oksuz, N. Byrne, Schnabel, and A. King. Explicit Topological Priors for Deep- Learning Based Image Segmentation Using Persistent Homology. In Information Processing in Medical Imaging, 2019.
J. Clough, N. Byrne, I. Oksuz, V.A. Zimmer, J.A. Schnabel, and A. King. A topological loss function for deep-learning based image segmentation using persistent homology. IEEE Transactions on Patlern Analysis and Machine Intelligence, 2020.
H. Edelsbrunner and Harer. Persistent homology a survey. Contemporary mathematics, 453: 257-282, 2008.
H. Edelsbrunner, D. Letscher, and Zomorodian. Topological persistence and simplification. In Proceedings 41st annual symposium on foundations of computer science, pp. 454.463. IEEE, 2000.
A. Van Etten, D. Lindenbaum, and T. Bacastow. Spacenet: A Remote Sensing Dataset and Challenge Series. arXiv Preprint, 2018.
Funke, F. D. Tschopp, Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Recon- struction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(7):1669-16) 2018.
W. Grisaitis, A. Sheridan, C. Singh, S. Saalfeld, and C. Turaga. Large
R. Gabrielsson, B. Nelson, A. Dwaraknath, and P. Skraba. A topology layer for machine learning. In Proceedings of the Twenty Third International Conference Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 1553-1563, PMLR, 26-28 Aug 2020.
10

---

Under review as a conference paper at ICLR 2022
A. Garin and G. Tauzin. A Topological "Reading" Lesson: Classification of MNIST using TDA. In International Conference on Machine Learning and Applications, pp. 1551-1556, 2019.
A. Garin, T. Heiss, K. A. R. Maggs, B. Bleile, and V. Robins. Duality in persistent homology of images. ArXiv, 2020.
X. Hu, Li, D. Samaras, and C. Chen. Topseog Deep Image Segmentation. In Ad- vances in Neural Information Processing Systems, pp. 5658-5669, 2019.
X. Hu, Y. Wang, L. Fuxin, D. Samaras, and C. Chen. Topology-aware segmentation using discrete morse theory. In International Conference on Learning Representations, 2021.
D. Kingma and Ba. Adam: A Method for Stochastic Optimisation. In International Conference on Learning Representations, 2015.
Leygonie, Oudot, and U. Tillmann. framework differential calculus on persistence bar- codes. Foundations of Computational Mathematics, 2021.
X. Li, Y. Wang. L. Zhang, S. Liu, J. Mei, and Y. Li. Eanplooy-Ennhaned Urban Road Extraction via Geographic 2020.
Network. IEEE Trans. Geosci. Remote. Sens., 58(12):8819-88))
Z. Li, J. Wegner, and A. Lucchi. Topological Map Extraction from Overhead Images. In Interna- tional Conference on Computer Vision, 2019.
C. Maria, J. D. Boissonnat, M. Glisse, and M. Yvinec. The gudhi library: Simplicial complexes and persistent homology. In International congress on mathematical software, pp- 167-174. Springer, 2014.
G. Máttyus, W. Luo, and R. Urtasun. Deeproadmapper: Extracting Road Topology from Acrial Images. In International Conference on Computer Vision, pp. 3458 4458-366,, 2017.
V. Mnih. Machine Learning for Aerial Image Labeling. PhD thesis, University of Toronto, 2013.
A. Mosińska, Marquez-Neila, M. Kozinski, and P. Fua. Beyond the Pixel-Wise Loss for Topology- Aware Delineation. In Conference on Computer Vision and Pattern Recognition, pp. 3136-3145, 2018.
A. Mosifíska, M. Kozinski, and P. Fua. Joint Segmentation and Path Classification of Curvilinear Structures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1255)1155,,, 2020.
D. Oner, M. Koziński, L. Citraro, N. C. Dadap, A. G. Konings, and P. Fua. Promoting Connectiv- ity of Network-Like Structures by Enforcing Region Separation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2021.
O. Ronneberger, P. Fischer, and Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Conference on Medical Image Computing and Computer Assisted Intervention, pp. 234 241, 2015.
K. Turner, S. Mukherjee, and D. Boyer. Persistent Homology Transform for Modeling Shapes and Surfaces. Information and Inference, 3(4)331,,34,, 12 2014.
Fan Wang, Huidong Liu, Dimitris Samaras, and Chao Chen. TopoGan: A Topology-Awan Genera- tive Adversarial Network. European Conference on Computer Vision, pp. 118-136, 2020.
J.D. Wegner, J.A. Montoya-Zegarra, and K. Schindler. A Higher-Order CRF Model for Road Net- work Extraction. In Conference on Computer Vision and Pattern Recognition, pp. 1698-1705, 2013.
C. Wiedemann, C. Heipke, H. Mayer, and O. Jamet. Empirical Evaluation of Automatically Ex- tracted Road Axes. In Empirical Evaluation Techniques in Computer Vision, pp. 172-187, 1998.
11

---

Under review as a conference paper at ICLR 2022
X. Yang, X. Li, Y. Ye, R. Y. K. Lau, X. Zhang, and X. Huang. Road Detection and Centerline Extraction via Deep Recurrent Convolutional Neural Network U-Net. JEEE Transactions on Geoscience and Remote Sensing, pp. 1-12, 2019.
A. Zomorodian and G. Carlsson. Computing persistent homology. In ACM Symposium on Compu- tational Geometry, pp. 347-356. ACM, 2004, doi: N0.144/9788179977779
12

---

Under review as a conference paper at ICLR 2022
A SUPPLEMENTARY MATERIAL
A.1 QUALITATIVE RESULTS
In this section, provide qualitative results three test datasets. For each method, display the thresholded predictions with their skeletons overlaid in red. In the case the 3D dataset, the images show maximum intensity projections.
A.2 ABLATION STUDY
To investigate the impact of hnyeer-paraeete choices on performance, we ran three ablation studies.
Weighting the PH Loss We have varied the coefficient a in equation] [ while keeping the other parameters fixed. We report the results in Tab. 5) The best results are achieved for 0.01, and the performance decreases when Mean Square Loss is still important for overall performance, which is not a surprise as the gradient of our pessiteetthomology based loss sparse and concentrated at pixels critical for topological correctness.
ten times higher or lower. This suggests that the standard
Window size We changed the size of the window in which the persistent homology is computed. We report the results in Tab. 6 Our method performs best when using large windows that contain
Input
UNet-CE
UNet-MSE
DMT
ConnLoss
Homo-Pre Pre
Homo-Reg
Homo-Our
Figure 5: Comparative results the Massachusetts dataset.
13

---

Under review as a conference paper at ICLR 2022
significant portions of the structures of interest. We could not try larger ones because it would have increased the time needed to detect the homologies and slowed down the training too much.
We also evaluated the effect on performance of using different forms of function in equation that ties homology birth and death times to image coordinates, distributing the points in persistence diagram. We present the results Tab. The distance to random image point, or
Height Functions
Input
UNeI-MSE
Segmentation
RoadTracer
Seg-Path
RCNNU-Net
DeepRoad
PolyMapper
ConnLoss
Homo-Pre
Homo-Ro Regg
Homo-Ours
Figure 6: Comparative results the RTracer dataset.
14

---

Under review as a conference paper at ICLR 2022
Input
UNet-CE
UNeI-MSE
Homo-Pry Pre
Homo-Roe Reg
Homo Homo-Ours
Figure 7: Comparative results on the 3D Neurons dataset.
Table 5: Impact of changing the learning coefficient of localized PH loss the Massachusetts dataset. The window size is fixed to 64x64.
Conneciviiy--orieneed
pixel-based
APLS
TLTS
JCT
Betti
CCQ
le-3
64.9
46.0
77.1
1.21
72.3
le-2
68.7
50.6
79.2
0.90
74.9
le-1
67.1
48.9
77.8
0.94
74.6
le-0
64.8
45.8
76.2
1.10
72.0
Table 6: Impact of changing the window size when computing our localized loss the Mas- sachusetts dataset. The learning coefficient is fixed to le-2.
Connectivityy-orien
pixel-based
Window Size
APLS
TLTS
JCT
Betti
CCQ
8x8
62.1
41.9
73.0
2.84
67.2
16x16
62.7
42.4
74.5
2.09
68.8
32x32
65.4
45.7
77.1
1.17
72.5
64x64
68.7
50.6
79.2
0.90
74.9
use of quadratic instead of linear function image coordinates do not result in higher performance than the plain linear function.
15

---

Under review as a conference paper at ICLR 2022
Table 7: Performances of different height functions used for localized PH loss the Massachusells dataset. The learning coefficient is fixed to le-2 and window size to 64x64
Connectivity
pixel-based
Height Function
APLS
TLTS
JCT
Betti
CCQ
Distance point
67.8
49.4
77.9
1.01
73.6
Random Linear
68.7
50.6
79.2
0.90
74,9
Fixed Linear
67.5
48.7
76.5
1.15
73.0
Square
64.2
45.1
76.3
1.32
70.3
16


Reviews:
[{"review_id": "eZsR4fXxais", "reviewer": "Reviewer_Emfj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths and Weaknesses\n\n## Strengths\n- It is nice to see a well-motivated application of TDA in the context of image processing.\n- Overall, the flow of the work is clear and the paper is pleasant to read. \n- The experimental results reported seem solid, extensive, and may be a good contribution to the field of image segmentation in particular. \n\n## Weaknesses\n- The paper contains some caveats / inaccuracies related to TDA (see the Minor Comments section for details). In particular, the desire to filter an object with respect to two (or more) parameters $Y, g$ (here, distance + height) is present for a long-time in the TDA community, known as *multi-persistence* (which is way harder than single-parameter persistence and, roughly, untractable in practice for now). Simplifying this by instead looking at one (or many!) linear combination of the two maps $Y, g$ is already used in TDA, see for instance the work *Multiparameter Persistence Images for Topological Machine Learning* by Carrière and Blumberg (2020); if my understanding is correct, filtering with respect to $Y + g$ is very similar (if not the same) to picking one direction in the so-called *fibered barcode*. Even if this does not take away all the potential impact of the paper (and it is quite nice to think about **this** choice of $Y,g$ and that it seemingly works well in applications), I think this must be discussed and the position of the paper with respect to the TDA literature should be revised a bit. \n\n# Minor comments\n- \"Persistent homologies\" (abstract), I think it is not standard to use plural here. \"Persistent homology\" would fit better in my opinion.\n- (Ref) When introducing Persistent Homology (p1, third paragraph), I think it may be appropriate to cite (in addition) a founding paper of TDA or a reference book, such as the one(s) of G.Carlsonn & Zomorodian (e.g. *Computing Persistent Homology* (2005)), Edelsbrunner and Harer (2008), Chazal and Michel (2017), etc.\n- (Ref) About the possibility to  differentiate the map $X \\mapsto \\mathrm{Dgm}(X)$ (should $X$ be an image or something else), I think it may be worth citing the work *A topology layer for machine learning* by Bruel-Gabrielsson et al. (possibly along with other work, like *A framework for differential calculus on persistence barcodes* by Leygonie et al., and *Optimizing persistent homology based functions* by Carrière et al.).\n- (Typo) Caption of Figure 1, Something is wrong, a description of *(d)* is missing. \n- (Ref) Saying that \"Application of computer vision [of persistent homology] were scarce\" may be slightly exaggerated, see for instance the short survey *Topological Data Analysis in Computer Vision* by Bernstein et al. and the references therein. \n- (Typo) \"closed 3D surfaces\", I guess \"surfaces embedded in 3D\" may be more accurate. \n- (Ref) Citing Giotto (Tauzin et al.) for the \"matrix reduction algorithm\" to compute PH is not very accurate as well. This algorithm was introduced much earlier (see references at the beginning of https://en.wikipedia.org/wiki/Topological_data_analysis#Computation ). Of course, it is worth mentioning the numerical computations were done using Giotto though. \n- (Important) p4, \"The cost of leaving a homology unmatched\" is **not** constant (at least in standard definition of the distances between PDs). It is the distance between the point and its orthogonal projection on the diagonal, as drawn for instance on Figure 3. If you used a constant cost instead, this must be discussed and motivated. \n- Calling \"a homology $g \\in H_{X_1}$...\" a point in the persistence diagram is not accurate. Calling them \"A persistence interval\" as done few lines above is much more standard in my opinion.\n- Figure 4 and Section 3.4 are not fully convincing : though *(c)* is indeed non-monotonic, the trend seems similar (and perhaps more trials than just 5 would have smooth the curve. Plotting the standard deviation would be useful as well). I would not say that \"very noisy images can be seen as being very similar to error-free ones.\" ", "summary_of_the_paper": "Relying on recent progress regarding the (automatic) differentiation of « Persistent Homology » (a tool of Topological Data Analysis (TDA) to extract topological descriptors on top of complex data), this paper proposes a new filtration (essentially, a sum of the distance filtration and the height filtration) to favor proper topological reconstruction in the context of image processing (segmentation in particular). \nThe authors experimentally showcase their approach on different (2D and 3D) datasets and compare it with various competitors (from and outside TDA-based techniques) and obtain very competitive results.", "summary_of_the_review": "The paper is well-written and showcases an interesting application of TDA in the context of image processing supported by various numerical experiments. However, it contains some inaccuracies regarding TDA concepts/literature that must be corrected/discussed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "55aTE-J75cR", "reviewer": "Reviewer_jz5f", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper addresses an important problem of training network producing segmentation with desired topology. Traditionally, segmentation is done via pixel-wise losses, which are not able to properly capture topology, resulting in significant topological mistakes such as lack of connectivity or erroneous connections. Designing topologically-motivated losses, like in this paper, is an important problem for detecting objects like roads, vasculature, etc.\n\nThe paper builds upon an established framework of persistent homologies (PH) and extends it by adding positional embedding.\n\nThe results look convincing and the proposed modification seems simple and effective.\n\nThe topological loss based on PH is quite complex in the implementation. The paper would benefit from a more detailed review of the implementation including differentiation.\n\nThe author should discuss the relation and compare to other PH-based localization techniques, for example Hu et al (2019). In particular Hu et al used patch-based topology, which provides a degree of localization. How does this compare to the proposed method? DO the authors explicitly compare to the patch based approach?\n\n", "summary_of_the_paper": "The proposes a localized version of topological loss. Current methods perform matching in the space (i.e. persistence diagrams) that does not account for the location of topological features. The method proposes to add to the likelihood maps some sort of positional embedding, i.e. a function of image pixel coordinates chosen at random during training. The experiments demonstrate improved performance.", "summary_of_the_review": "The paper is addresses an important problem of topology-aware segmentation learning. The proposed framework extends previously existing PH diagram techniques by adding positional emebedding, this add locality to the framework. The experimental evaluation shows improvement upon global methods. Some lack of comparison with other localized methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "4tNYcC94M0K", "reviewer": "Reviewer_mHrP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While I find the topic of this paper highly fascinating, at present,\nthis paper is not yet ready for publication. There are three main issues\nI have with the current write-up:\n\n1. The work is marred by severe deficiencies in clarity/correctness.\n   I will provide more examples later but the text is using several\n   terms incorrectly and does not explain underlying concepts correctly.\n\n2. Missing delineation to existing work: while the paper discusses other\n   work that aims to combine image tasks with topological prior\n   information, it does not provide a sufficiently detailed delineation\n   to such work. This can also be seen in the related work section,\n   which misses the paper [*A Topological Loss Function for Deep-Learning\n   based Image Segmentation using Persistent\n   Homology*](https://arxiv.org/abs/1910.01877) by Clough et al.\n   (linking to the preprint here; the full paper appears in IEEE TPAMI).\n   While another paper of Clough et al. is discussed, the aforementioned\n   work should also be at briefly discussed and, ideally, be used as\n   a comparison partner.\n\n3. The experimental setup is preliminary and missing important details:\n   means and standard deviations are only provided for the proposed\n   method, whereas the numbers for other methods are presumably cited\n   from an additional publication. This does not enable a fair\n   comparison, given the fact that it is unclear whether the experiments\n   described in other papers followed the same training regimen.\n   Providing standard deviations is a critical facet that needs to be\n   changed prior to publication.\n\n   In addition, I find the selection of comparison metrics somewhat\n   questionable; in other publications, metrics such as accuracy, DICE\n   scores, ARI, etc. are being employed, whereas this paper reports\n   metrics that appear to be relatively non-standard to me. That being\n   said, I acknowledge that I am not familiar with all segmentation\n   literature; I am, however, very familiar with the segmentation\n   literature based on topological concepts, and these papers\n   consistently report different scores. I would therefore suggest, also\n   in the interest of making the paper more accessible, to include\n   additional scores and expand on existing ones (for instance, it would\n   be helpful to know what a 'good' score looks like, and whether larger\n   numbers are better or worse. All of this information can be gleaned\n   from the text at present but it could be more prominent).\n\n   Finally, I am confused by certain reported numbers and algorithms in\n   the experimental section:\n\n   - It seems that `ConnLoss` is performing better in all metrics.\n     I understand that it does not extend to 3D data, but if this is to\n     be the deciding property of the proposed algorithm, additional\n     experiments on 3D data sets are required. Performance on the\n     proprietary data set is hard to assess because on comparison\n     partner papers exist. While it is perfectly acceptable to use such\n     data sets, in the interest of a fair comparison, I would suggest to\n     also employ additional 3D data sets such as `ISBI13`, `CREMI`, or\n     `3Dircadb`, all of which are discussed in [the paper by Hu et\n     al.](https://arxiv.org/pdf/2103.09992.pdf).\n\n   - The 'Massachusetts' data set corresponds to the `ROAD` data set from\n    the aforementioned paper by Hu et al. However, Hu et al. report the\n    same Betti error as the one shown in this paper, albeit with\n    a standard deviation of `0.301`, which is comparable to the one\n    obtained by the proposed method. If the number are indeed cited from\n    this paper, additional information about variance needs to be shown\n     *and* the citation should be acknowledged in the caption of the\n    table, for instance, or in the experimental setup section. It needs\n    to be clear to readers which experiments where run by the authors\n    and which were cited from another paper. \n\n   - Why is the set of comparison partners changing for the different\n    experiments? The DMT method is applicable to 2D and 3D and [code\n    appears to be available](https://github.com/HuXiaoling/DMT_loss),\n    so it should be part of this comparison, in particular since the\n    paper claims that the proposed algorithm outperforms *all* PH-based\n    loss functions. (the same comments apply to the papers by Clough et\n    al.; for a fair comparison with TDA-based methods, they need to be\n    included).\n\n## Detailed comments\n\n- The title is slightly misleading: 'localisation' typically refers to\n  the process of finding geometrically useful representations of group\n  generators. Here, the filtration incorporates some geometrical\n  information but it does not make use of group generators. Using\n  'localised' here might be misconstrued at first glance. The title is\n  also misleading on a second level because the proposed loss does *not*\n  deal with deep learning in general, but with *image segmentation*\n  based on deep learning. The loss also does not improve the\n  effectiveness of deep learning methods in general. I urge the authors\n  to reconsider the title of this paper.\n\n- When discussing persistent homology, consider citing a survey or\n  a text book, such as the one by Edelsbrunner and Harer. The work by\n  Aktas et al. did *not* describe persistent homology for the first\n  time.\n\n- The abstract of the paper is too terse and needs to be slightly\n  expanded in order for it to be accessible by non-experts in the field.\n\n- Homology is typically used only in singular form or as a qualifier, as\n  in 'homology groups' when discussing TDA literature. The framework\n  itself is dubbed 'persistent homology' and a plural use is relatively\n  nonstandard.\n\n- The term 'homologies' is incorrectly used throughout the paper; use\n  the term 'simplicial chains' or 'topological features' instead. Or,\n  for the most precise moniker, you can use 'elements of a homology\n  group.'\n\n- Please check carefully when to use 'persistence' and when to use\n  'persistent.' The diagram is the 'persistence diagram' because it\n  contains information about the 'persistence' of features.\n\n- When claiming that '[...] our technique allows for considerable\n  misalignment [...],' provide a reference to the respective section in\n  the text.\n\n- The description of persistent homology in Section 3.1 needs to be\n  rewritten; use the term 'homology class' if need be instead of\n  'homologies.' A more technical description might be helpful here; I am\n  not sure whether non-expert readers are able to understand the gist of\n  the technique from the current write-up.\n\n- In Equation 2, consider adding a second set of parentheses; the idea\n  is that the sum of the two terms is $< s$, correct?\n\n- Figure 3 could be updated to make the difference between the proposed\n  filtration function and the 'default' one clearer.\n\n- When discussing experiments, please provide details on how the\n  hyperparameters were chosen.\n\n- The *Betti error* needs more details; the present description is\n  insufficient.\n\n## Minor comments\n\n- Citations are used inconsistently throughout the paper. Assuming that\n  you are using the `natbib` package, use `\\citep` for a parenthetical\n  citation and `\\citet` for a textual citation. Examples of places that\n  need such a fix can be found in the paragraph 'Losses that Rely on\n  Persistent Homologies' (which I would rename to the singular form, as\n  discussed above).\n\n", "summary_of_the_paper": "This paper presents a new approach for detecting topological structures\nin 2D and 3D imaging data. Based on the computational topology framework\npersistent homology, the paper develops a type of loss term that can be\nused to assess to what extent topology is being preserved when solving\nsegmentation or structural extraction tasks. A suite of experiments is\nused to assess the quality of the extracted structures.", "summary_of_the_review": "This paper deals with an interesting topic, viz. improving 2D and 3D\nimage tasks by incorporating more prior knowledge about topological\nstructures. While I consider this to be a highly relevant topic,\nI cannot endorse this paper for publication at present, mainly because\nof a lack of clarity, a lack of delineation to existing work, and issues\nin the experimental setup.\n\nAll of these will have to be rectified before I will be able to endorse\nthe paper for publication; I feel particularly strongly about the\nexperimental setup, which currently does not enable a fair comparison of\nthe methods.\n\nI realise that this is not the preferred outcome for the authors; at the\nsame time, I am confident that with additional rewriting and some\nadditional work on experiments, this paper can make a strong addition to\nthe ever-growing body of topology-based machine learning techniques.\n\n**Updated after rebuttal**: I thank the authors for their efforts; while some concerns\nwhere alleviated, I think the updates also showed that the current method does not\nsubstantially improve upon the state of the art in the 2D case, whereas for the 3D case,\nadditional comparison partners would be required. This is not the only way to improve\nthe paper, though: another potential direction for future improvement could be to invest\nin a more theoretical analysis of the proposed method. If either one of these directions\nwould be followed, it would serve to substantially improve the quality.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1Sz20cMRSnA", "reviewer": "Reviewer_ne7r", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed filtration is interesting, and the article is overall well written, but I think that the approach seems a bit incremental for now. There is no real theoretical back-up and the results, while being OKish, do not empirically justify by themselves the approach (to my opinion). Moreover, some parts of the writing could be improved.\n\nIn particular, I have the following comments:\n\n1. Section 3.1: \"the cost of leaving a homology unmatched set to a constant hyper-parameter\": this is not correct, the cost of leaving a point in a persistence diagram unmatched is usually its distance to the diagonal, otherwise the distance is not stable.\n\n2. Equation (2): are the values Y[p] and g(p) always comparable, i.e., on the same scale? If one is much larger than the other, it does not make a lot of sense to do a simple sum between the two. It would be more natural to use a linear combination of the two terms, with the combination coefficients chosen a priori or with cross-validation.\n\n3. Given that the authors aim at using two filtrations at the same time, it feels that persistence bimodules would also be an approach to discuss and compare to, especially since multiparameter persistence can now be computed efficiently, at least when there are no more than two filtrations.\n\n4. Figure 4 (c) and (d) do not have names on their axes.\n\n5. The wording chosen in the article is not great: localization has another specific meaning in Topological Data Analysis, it usually refers to the computation of representative cycles corresponding to homology classes. Hence, the title of the article is confusing and misleading for TDA practitioners.\n\n6. Since the persistent homology transform is known to enjoy some nice injectivity and inverse properties (https://arxiv.org/pdf/1810.10813.pdf), I wonder if such results could also be derived for the proposed loss.\n\n7. Section 4.2: \"alpha=0\": typo\n\n[Post rebuttal comment]: I updated my review after the author's response.", "summary_of_the_paper": "In this article, the authors propose a new topological loss for training neural networks on images representing curvilinear structures. In addition to the usual MSE or cross-entropy losses, they suggest to add a regularization term based on persistent homology, in order to preserve the topological properties in the images predicted by the network. However, the common filtrations proposed in the literature (pixel values, distance maps) for doing so have their own flaws and are not always suited for such images, so the authors propose a new filtration that combines those common filtration with filtrations based on heights, i.e., with the persistent homology transform. Using this new filtration, they then provide a few experiments on which their loss compares favorably to other topological and non-topological losses (used on the same fixed neural net architecture).", "summary_of_the_review": "The proposed approach is definitely promising, but I think that the work is too incremental for now to be accepted as is. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for segmentation of thin structures in 2D and 3D, based on persistent homology and using a new filtration. The method performs similarly to state-of-the-art methods on 2D datasets and outperforms some baselines in 3D.

After considering the authors' response and discussing, the reviewers have not arrived at a consensus. 

Pros include:
- Simple and reasonable approach
- Fairly strong experimental results

Some cons are:
- Missing theoretical contributions
- Experimental results on 2D datasets are not that strong, while on 3D datasets important baselines are missing
- At times unclear/unconventional presentation 

Overall, at this point I recommend rejection. The paper is promising, but since the main claim is good performance on 3D data, it is important to have a thorough empirical evaluation there, with the relevant baselines (as mentioned by the reviewers). I very much encourage the authors to polish the paper and submit to a different venue.
Decision: Reject