Topology-Aware Uncertainty for Image Segmentation
Authors: Saumya Gupta
Yikai Zhang
Xiaoling Hu
Prateek Prasanna
Chao Chen
Venue: NeurIPS 2023 poster (2023)
Keywords: Topological Representation
Discrete Morse Theory
Structural Uncertainty
Image Segmentation
TL;DR: We propose a novel Probabilistic DMT to model the inherent uncertainty within each structure (intra-structural uncertainty) by sampling its representations via a perturb-and-walk scheme.

Abstract: Segmentation of curvilinear structures such as vasculature and road networks is challenging due to relatively weak signals and complex geometry/topology. To facilitate and accelerate large scale annotation, one has to adopt semi-automatic approaches such as proofreading by experts. In this work, we focus on uncertainty estimation for such tasks, so that highly uncertain, and thus error-prone structures can be identified for human annotators to verify. Unlike most existing works, which provide pixel-wise uncertainty maps, we stipulate it is crucial to estimate uncertainty in the units of topological structures, e.g., small pieces of connections and branches. To achieve this, we leverage tools from topological data analysis, specifically discrete Morse theory (DMT), to first capture the structures, and then reason about their uncertainties. To model the uncertainty, we (1) propose a joint prediction model that estimates the uncertainty of a structure while taking the neighboring structures into consideration (inter-structural uncertainty); (2) propose a novel Probabilistic DMT to model the inherent uncertainty within each structure (intra-structural uncertainty) by sampling its representations via a perturb-and-walk scheme. On various 2D and 3D datasets, our method produces better structure-wise uncertainty maps compared to existing works. Code available at: https://github.com/Saumya-Gupta-26/struct-uncertainty

Topology-Aware Uncertainty for Image Segmentation
Saumya Guptal IM Yikai Zhang²
Xiaoling Hu Prateek Prasanna'
Chao Chen'
Stony Brook University, NY, USA M Stanley, NY, USA
Athinoula Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, MA, USA
Abstract
Segmentation of curvilinear structures such as vasculature and road networks is challenging due relatively weak signals and complex geometry/opology. To facilitate and accelerate large scale annotation, one has to adopt semi automatic approaches such as proofreading by experts. In this work, we focus on uncertainty estimation for such tasks, so that highly uncertain, and thus error-prone structures can be identified for human annotators verify. Unlike most existing works, which provide pixel-wise uncertainty maps, we stipulate it is crucial to estimate uncertainty the units of topological structures, e-g., small pieces of connections and branches. To achieve this, we leverage tools from topological data analysis, specifically discrete Morse theory (DMT), to first capture the structures, and then reason about their uncertainties. To model the uncertainty, we (1) propose a joint prediction model that estimates the uncertainty of structure while taking the neighboring structures into consideration (inter-structural uncertainty); (2) propose a novel Probabilistic DMT to model the inherent uncertainty within each structure (intra-structural uncertainty) by sampling its representations via perturb- and-walk scheme. On various 2D and 3D datasets, our method produces better structure-wise uncertainty maps compared existing works. Code available https://github. com Com/Sauyaa-Guptaaaa...oooa struct- uncertainty
1 Introduction
Curvilinear segmentation an essential initial step in various medical and non-medical applications, involving the precise extraction of fine-scale structures, such blood vessels, nerves, and other elongated objects [3]] For example, extraction of retinal vasculature is an essential precursor understanding disease progression and assessing therapeutic effects [5]. In civil engineering, road network and railway track segmentation can support urban planning and transportation system optimization (45) Despite the success of deep learning [ 6 25 37], automatic segmentation of thin structures remains challenging due to their relatively low visibility and complex topology. Existing segmentation methods often make topological errors such broken connections or missing branches.
As cost-efficient alternative, many applications, one employs semi-automatic techniques, e.g., iterative proofreading by human annotators 23 On the other hand, proofreading of complex fine- scale structures can be extremely time-consuming [53]. This necessitates a better strategy to direct the annotators' attention towards locations that are more error-prone. Following the classic active learning principle 18 55 61,, we may estimate the uncertainty [19 and concentrate on the locations
where neural network the least certain.
Despite many existing studies on segmentation uncertainty [12 51 60), most existing uncertainty estimation methods do not apply to curvilinear structure segmentation. Existing methods typically generate pixel-wise uncertainty maps. Such maps highlight pixels along the boundary of all structures,
"Email: saumgupta www.:pppaeeessstttooooo...o
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

a) Input image
GT
<) UNet
PHiSeg
Hu et
Ours
Figure 1: Motivating examples for structure-wis -wite uncertainty. In the segmentation result (c), orange highlights false positive structure, and pink highlights false negative. Methods (d)-(f) are uncertainty estimates of the prediction in (c). PHiSeg [4] assigns pixels along boundaries uncertain. Hu et al. [28] captures uncertainty at a structural level, but produces overconfident maps (assigns zero uncertainty to many structures). Ours produces better structure-wise uncertainty estimates: both the highlighted false positive/negative structures have high uncertainty.
regardless of their width or thickness (see Fig. dd). This offers limited information for human annotators; desirable uncertainty map should instead highlight the error- prror-prone "structures", e-g.. small vessels/branches or short stretches of roads that tend to be disconnected or missed.
In this paper, we propose new aopology-wwar uncertainty estimation method that highlights error- prone structures as whole (such as in Fig. (t). By highlighting structures with high uncertainty, our method empowers annotators accept or reject/correct structural proposals efficiently, thus streamlining the proofreading process. To capture the uncertainty of a given segmentation network's s prediction structural level, we require the realization of two key components: a) decompose the prediction into a set of constituent structures, including false positives and false negatives, and b) estimate uncertainties of all the structures. Furthermore, we need to consider two types of structural uncertainty, intra-structury and inter-structural. The intra-structural uncertainty of structure is due its intrinsic composition, e.g.. geometry, intensity, and the segmentation network's confidence. The inter-structural uncertainty is more contextual; it is due to interactions between neighboring structures. Our method explicitly models the two types of uncertainty.
Given a segmentation model, our method uses its likelihood map (Fig. (i) plus the input image to estimate structural uncertainties. First, we obtain structural decomposition of the prediction, i.c., collection of mne-pixel-wide picces/structures (see Fig. 2 (i). Each structure represents potential branch/connection according the segmentation model. We employ the principles of the classic discrete Morse theory (DMT) [14] 43
Intuitively, DMT treats the likelihood function as terrain function and extracts landscape features, e-g., mountain ridges or valleys, as structures. Note we are capturing all possible structures visible in the likelihood map, including the ones that do not appear in the segmentation due to low probability.
Next, we estimate uncertainties for all these structures. Existing uncertainty estimation approaches often sample multiple hypotheses and calculate the variance across them [17 34 69). However, this principle is not feasible in our problem; with N structures, the space of all their combinations is of exponential size (2-). Sampling from such a space very challenging. An alternative is to make independent uncertainty estimation on each structure. However, this is also suboptimal as it ignores inter-structural uncertainty. Fig. vi) shows three false positive structures. Treating all structures independently will incorrectly assign the horizontal structure with low uncertainty (see Fig. 2 vii)).
We propose joint inference model (i.e., graph neural network (59) jointly predict uncertainties on all the structures. This joint inference framework avoids explicit nnumeration/sampligg over the exponential size space of hypotheses. It also takes into account the inter-structural uncertainty. In Fig. 2 viii), our method correctly assigns high uncertainties to all three false positive structures. To supervise the training process, we use the attenuation loss proposed in (30 learn the uncertainty. As there is no 'uncertainty label', it is implicitly learned from the loss function.
An important contribution in this paper is novel Probabilistic DMT modeling the uncertainty, due to the geometry, topology, as well as the segmentation model's confidence. The classic DMT cannot capture the inherent uncertainty of structure; it produces single representation for the structure, i.e., a -one-pixe-wwdee skeleton. However, this skeleton is computed in deterministic manner. Such deterministic computation rigid and fails to capture possible variations of structure.
Instead, our Probabilistic DMT (Prob. DMT) represents each structure as sample skeletons drawn from an underlying generative process guided by the likelihood map (the skeleton resulting from the

---

skeletn
DMT
image
vij map
low confifence FP
Intra-structurl uncuriainnty
Inter-structural uncortainnty
Figure 2: (a) Intra-structural uncertainty: In the likelihood map (i), we highlight a false negative (FN) structure missed by the segmentation network. In (i), we show the skeletons representing structures from classic DMT. (iii), we highlight the GT (green) and the incorrect DMT skeleton (red) for the FN structure. In (iv), show skeleton samples by Prob. DMT (blue); (b) Inter-structural uncertainty (inter): a retinal image with very weak signal (v). The likelihood map (vi) shows three potential false positive (FP) structures, two vertical and one horizontal. Without inter-structural uncertainty (vii), the horizontal structure has high confidence. With inter-structural uncertainty (viii,, the horizontal structure gets higher uncertainty influenced by the two vertical structures.
original DMT being one of the samples). As illustrated in Fig. 2 jii), the original DMT generates skeleton that significantly deviates from the true structure due to the uncertainty inherent in the likelihood, contrast, Prob. DMT effectively captures potential variations (Fig. iv),, offering valuable insights into the impact of uncertainty on the structural composition. The greater the variation, the greater the intra-structural uncertainty. During training, we repeatedly sample skeletons for each structure, and feed the samples to the joint inference model for uncertainty prediction. Indeed, as shown Fig. 2iv), sampling multiple skeletons also leads to a better chance of uncovering the true structure. This observation inspires to use the uncertainty estimation method to re-calibrate the original segmentation model and achieve even better segmentation performance.
We note that the method in 28 (which we refer to as Hu et al.) also used DMT to decompose structures and estimate their uncertainty. However, this method used the classic DMT to deterministi- cally generate skeletons, and thus failed to model intra-structural uncertainty. Furthermore, instead of joint inference, the method sampled from all configurations; and to reduce the computational configuration space using a saliency measure called persis- tence 19) 64 (1]. The pruning was very coarse, thus resulting in suboptimal uncertainty estimation. As illustrated in Fig. [T), Hu et al. produces overconfident maps; most structures, including many false negatives and false positives, are assigned zero uncertainty. In contrast, our method produces much better uncertainty estimates (Fig. [(f), owing to the proper modeling of both intra-structure and inter-structural uncertainties.
burden, it pruned the exponential-sine
We summarize the main contributions of this paper as follows:
We propose a novel method to estimate the uncertainty a given segmentation network at structural level.
2. We propose Probabilistic DM', a probabilistic method to generate structural variations and to capture intra-structural uncertainty.
3. We propose joint prediction model on all the structures in order to capture inter-structural uncertainty.
Empirical evaluation shows that method achieves much better uncertainty estimates both 2D and 3D datasets, outperforming existing methods.
Related work
Hopology-guidee image segmentation. Several works focus maintaining the correct connectivity
or topology of thin structures. Topology-aware loss functions [46]
62 27
72
22 26 impose
per-pixel constraints to improve topological integrity. Discrete Morse theory has also been used to
improve the topological awareness of segmentation networks 29 topological tools to improve segmentation pixel level. which is weaker constraint compared to the structural level. In contrast, our method performs joint reasoning directly over the structures.
9
[1
56
7 3) These approaches
Uncertainty quantification. In recent years, there has been significant work uncertainty quan- tification (UQ) of deep neural networks I 20, 36). Here we review UQ techniques tailored for semantic segmentation. Pixel-wise uncertainty: Semantic segmentation is per-pixel classification
3

---

Input image
Likelihood
Seg. map
Segmentation
network
Luq
Probabilistic
Joint reasoning
DMT
Likelihood
Structural variations
Graph construction
Structure-wise uncertainty
Intra-structural uncertainty
Inter-structural uncertainty
External uncertainty quantification framework M
Figure 3: An overview of the proposed method Ma. The given segmentation network Fg has frozen weights. Probabilistic DMT decomposes the likelihood into structures, and samples skeleton representations of each. graph is then constructed over the structures to perform joint reasoning of their uncertainty. The training is supervised by comparing with the GT (via the loss Luq, red arrow).
task and naturally most UQ methods produce per-pixel uncertainty estimates. In 30 the authors propose a Bayesian framework using MC dropout [16] and learned loss attenuation to respectively capture model and data uncertainty. Recent methods have turned to generative models to generate multiple hypotheses, and the per-pixel variance across the hypotheses treated as uncertainty, Some works in this direction are an ensemble of M networks [34], single network with M heads (58), Prob.-UNet [32], and PHiSeg [4] Prob.-UNet integrates a conditional variational autoencoder [63 with UNet [57], generating multiple hypotheses via latent variable sampling. PHiSeg extends this by introducing latent variables at every UNet level, thereby producing more diverse samples. Structure- wise uncertainty: Methods such [41] 60) compute structure (volume) uncertainty averaging
the pixel-wise uncertainty estimates. The method closest to ours is Hu et al. (28] It is generative model derived from Prob.-UNet where the latent variable has meaning in topology (specifically, global persistence threshold). This threshold severely limits the structure space, overlooking several
false positive/negative structures. Thus they tend produce overconfident uncertainty estimates.
3 Method
Given a trained segmentation network, our goal is to capture the uncertainty of its prediction at a structural level. Note that we do not modify the network in any way; instead, we propose an external module that reasons the uncertainty of each structure in the segmentation. Fig. provides an overview of our method. Let F, denote the trained segmentation network, and M denote our proposed external uncertainty quantification framework. M3 takes as input the likelihood map of F and the input image. It generates a set of structures, and estimates uncertainty value for each of them. During training, Ma is trained by comparing with the ground truth (GT) annotation.
M3 consists of two primary modules to capture intra-structural and inter-structural uncertainty. The first module, Probabilistic DMT (Prob. DMT), generates structures based on the likelihood map. For each structure, it samples set of skeletons representing different variations. Details are provided in Sec. 3.T The second module jointly predicts the uncertainties of all the structures. At each training iteration, it takes one sample skeleton for each structure, plus the likelihood map and input image, as input. Details are described Sec. 3.2 Throughout the sections, we consider one data sample (æ. v) where r is an input image and is the segmentation GT. The likelihood map is f Fo(x).
3.1 Modelling the structural space
In this section, we first describe how DMT obtains the constituent structures of likelihood map. Then propose Prob. DMT formulation to capture intra-structural uncertainty.

---

Discrete Morse theory. Consider the likelihood map ff generated from the segmentation network Fo. We wish decompose into a set of structures, capturing not only the salient structures but also the faint ones. In the segmentation map, salient and faint structures broadly correspond to true positive and false negative structures. In Fig- 4b), b), we highlight the false negative (FN) structures. These structures are missed in the segmentation, but will be captured by DMT (Fig. d))..
DMT treats the likelihood map it into a Morse complex consisting of critical points, paths connecting them. patches between paths, and volumes enclosed by patches (for 3D images). Critical points are locations w with zero gradients (ww 0), i.e., minima, maxima, or saddle points. Paths, called V-paths, are routes connecting critical points via the non-critical ones. V-path connecting saddle point maxima called a stable manifold. These stable manifolds are the underlying terrain's s mountain ridges, and delineate structures of interest. Fig. [c.. we show the locations of saddles and maximas in the Morse complex, and Fig. d), we show the union all the stable manifolds connecting them. In this paper, we only focus the zero- and one- dimensional Morse structures, i.e., the union all stable manifolds and their associated saddle and maxima. We call the collection of such structures the Morse skeleton.
terrain function, decomposing
a) Likeelihood map
Seg. map
c) Critical points
d) Morse skeleton
Figure 4: Orange indicates FN structures; (c) shows sad- dle points (red) and maximas (blue). and omits minimas; (d) shows the union of the stable manifolds of the saddle points.
By default, DMT generates stable manifolds in completely de- terministic manner, failing to take into account the intra-structural uncertainty in the likelihood f. Therefore, these stable manifolds may fail to correctly delineate the true structure, as shown in Fig. [
Probabilistic DMT. To account for the inherent uncertainty, we explicitly model the structure as a collection of samples from an underlying generative process. The skeleton from the original DMT is just one possibility out of many. The method is achieved via ceturb-and-walle algorithm, in which we iteratively perturb the likelihood map, and regenerate the skeleton.
The rationale is that the likelihood map is weighted aggregation all possible skeleton representations. To inverse the aggregation and recover these skeletons is challenging. Instead, follow the classic perturb-and-maup princi- ple, which was used to efficiently sample from a complex discrete graphical model distribution [52) 24 (35]. We randomly perturb the likeli- hood function. For each perturbed likelihood, we compute a skeleton as sample. See Fig. 5 for an illustration. The sampled skeletons will reflect the uncertainty properly. For structure that is less salient in the likelihood map, the sam- ple skeletons will have large variations, generat- ing a large uncertainty. For salient structure in the likelihood map, the sample skeletons will be less variant, resulting in low uncertainty.
Likelihood f
DMT
Prob. DMT: Prob. DMT:2(
Gr
GT Figure 5: Structures (#1,42) sampled from the dis- tribution. Green arrow is path chosen using Q(c'); red arrow next step w/o considering Qa(c').
intermenizate
Assume a given likelihood function f and one of its structures, represented by V-path e connecting a maddle-maximum pair (ce, cm). We generate sample skeleton of the structure by first perturbing the likelihood with random noise. Next, we generate path connecting C₂ and cm- Recall in the original DMT, the skeleton is generated by following the mountain ridge. In other words, we start from the saddle point, and "walk" towards the maximum. At every step, we always walk to the neighboring pixel with the highest likelihood value. In Prob. DMT, we follow the same principle on the perturbed likelihood. However, the noisy perturbation of likelihood can cause the path to grow astray. Therefore, we additionally apply a distance-based regularizer to guide the walk towards the target C.n- We describe the process in detail below.
Let denote the structure obtained by following the V-path between (Ce, Cm) in the original DMT. In order to generate its sample skeleton é, we first draw a likelihood fn from distribution centered on
5

---

as fn ~ f + r. This process is independent of the perturbation model used, and we use Gaussian model in this work. As the variance of the Gaussian model is unknown, we use Bayesian probability theory to sample the variance from the Inverse Gamma distribution (its conjugate prior [49).
Once obtain fn. we regenerate the path between (C2, Cm). We take inspiration from random walk (38) as well as probability regularized walk [47] to generate the variant structure é from fr. Our walk algorithm continuously grows é starting from c2 and ending at Cm, one pixel at time, The algorithm considers both the terrain fn and the distance to the destination Cy to ensure path completeness. During the walk, given the current pixel location c, the next location c" is chosen as c" argmax (Q(c)) where, c E neighborhood [e and Q(c') = QQa(c") + (1 f) (c'), and, Qa(c)) -''la" We begin with c == c2 and continue in this manner e = d" till we reach Cm In Fig. 5 we show a deterministic structure obtained from DMT along with sample variations produced by our method. We demonstrate the intermediate steps in the algorithm: the red arrow denotes the next step without considering the distance regularizer Qd. while the green arrow denotes the next step using our formulation Q. Notice how only considering fn without Qa can prevent the path from reaching Cm- We thus require Qa to guide the path to completeness.
The structure è is a different realization of e, making each run of the Prob. DMT a stochastic one. We are thus able to explicitly model the structures as samples from a probability distribution. We also note that DMT is special case of Prob. DMT when 0 and 7 = 0. In practice, with some probability, we consider the original structure e from DMT over generating its variant é. Specifically, following a Bernoulli distribution, with small probability we retain e, while with probability 1 - we sample its variant using the perturb-and-wann separately and in parallel for every structure. The structures taken together form Morse skeleton. The output of Prob. DMT is effectively one sample skeleton from the space of Morse skeletons. We provide further information regarding Prob DMTT in the Appendix: Aouulines the pseudocode, B discusses the hyperparameters, and C reports its computational complexity.
algorithm outlined above. This process is done
3.2 Joint estimation of structural uncertainty
The Prob. DMT module gives us a set E of structures. Our final step is to jointly reason about the uncertainty of all of them. To achieve this, we use a regression network that takes as input each structure E E, and outputs whether it is true positive and the uncertainty of Fe in predicting it.
Details of the network. Structures interact with each other in the image space and are not independent. During uncertainty estimation, it is therefore crucial to consider their spatial context, i.c., inter- structural uncertainty. Hence, we use Graph Neural Networks (GNN) (59), specifically Graph Convolution Networks (GCN) 31) to jointly reason about the structures and capture the high-order spatial interactions. In the graph, each node represents structure, and edges between nodes exist when corresponding structures have non-zero overlaps (typically at endpoints). The input feature vector for each node is constructed as shown in Fig. 6 For every structure, we first concatenate [xf 1 J mm]. where x comes from the original input 2;; J from the likelihood map f (not fa); and m is a binary map indicating the presence of the structure. These xF, m are smaller crops/bounding boxes centered on the structure. After passing them through convolution blocks, we apply channel- wise pooling to obtain a fixed-length feature vector for training. We further concatenate the persistence value of the saddle point associated with the original DMT structure (aka stable manifold). Persistence value (from persistent homology [ is defined as the difference of function (likelihood) values of 2 critical cells maddle-maxima pair). It captures the importance of structure, thus making it valuable feature in our framework. Note that we do not use the perturbed fa from the Prob. DMT method when constructing the feature vector.
Training the network. We train the regression network using the attenuation loss proposed in [30] As there are no labels to learn uncertainty, it is implicitly learned during regression optimization. We fix Gaussian likelihood, and variance 8² is used measure of uncertainty. The network's 's head is split into two to predict p(e) of being true positive structure and its associated uncertainty 8 For numerical stability, we actually predict the log variance 8, log 8 The training loss is given in The structures that we obtain from Prob. DMT may not always fully overlap with the true GT
Eq. Ι structures, that is, some structures may only have partial overlap. We thus do not impose any hard
2 For neighborhood, we use 8-connectivity for 2D, and 26-connectivity for 3D in this work. If the path does not reach Cm. impose maximum limit on the update steps to prevent an infinite loop.

---

Persistence value
Convolution
Channel-wise max pooling
blocks
input feature
Figure 6: Construction of the input feature vector for each node (structure) in the GCN.
constraints in Eq. instead, ze is soft label, and is given by: āe - y o m) m), where y is the GT and @ is the Hadamard product. This value simply represents the proportion of the structure that overlaps with the GT, i.., the fraction of the structure that is a true positive.
1
(p(e)) 2e 1
LuQ(o) = EE
Σ VeEE
Se 2
(1)
2
exp (8))
In 30). 8 denoted the pixel-wise uncertainty of the framework's input. In our setting, the input to our framework is f = (T), and so 8² is modeled to capture the structure-wise uncertainty inherent in data and model Fg. Training 8 in this manner ensures that the network does not trivially predict high or low uncertainty, rather, predicts an uncertainty estimate that is dependent on the input.
3.3 Proposed module Ms
For Eq. to hold, we require Ma to be probabilistic network. We already show Sec. 3.T our formulation for Prob. DMT. Additionally, the regression network is also probabilistic as we use MC dropout [16].
skeletons from M
finol uncertainty heatmap
Inference procedure. We take T runs of Ms and com-
pute the uncertainty the mean 82 D T (62) We
similarly obtain p(e) from p(e). In Fig. 7 we illustrate the post-processing steps to obtain the structure-wise certainty heatmap. First, we obtain maps p Upe and
from Fb
overlaid map
82 = 2 having the same spatial resolution as the input æ. We then binarize p, and overlay it onto the segmentation map obtained from Fg. We do this because Prob. DMT gives us one-pixel wide skeleton structures but we need to recover the structure thickness. Next, we use shortest distance assign uncertainty values from 82 to the pixels in the overlaid map. The shortest distance uses paths only along the foreground pixels. In Fig. 7 we show how we obtain the final uncertainty heatmap from the skeleton heatmap. We also note that the overlaid map an additional output of our method: it is an improved segmentation map that can be used instead of the one obtained by Fg. We provide more details Appendix D
Figure 7: Post-processing procedure.
Experiments
Datasets. We evaluate method four datasets: DRIVE 665. ROSE 440) ROADS 44 and
PARSE 2022 Grand Challenge (39 70 The DRIVE dataset contains 2D retinal vasculature; ROSE 2D retinal OCTA (Optical Coherence Tomography Angiography) segmentation dataset, ROADS is a large non-medical dataset containing aerial images, and PARSE contains 3D CT scans of pulmonary arteries. We further describe the datasets and the data splits in Appendix
Baselines. We broadly split our comparison baselines into three types: a) Standard vessel seg- mentation methods: UNet [57]. Deep Nes (68), and CS2-Net [48]; b) Pixel-wise uncertainty
estimation methods: Pret.-UNet [32] and PHiSeg 4 c) Structure- wise uncertainty estimation method: Hu et al. (28). Implementation details are provided in Appendix F
Evaluation metries. To evaluate the quality uncertainty quantification, we use Expected Cal- ibration Error (ECE) [50] and Reliability Diagrams (RD)
g Furthermore, we also evaluate
the segmentation on metrics such as DICE (73), elDice 62. ART 22, VOI [42, Betti Number
error 27] and Betti Matching error (66). We include clDice, ARI, VOL, Betti Number and Betti

---

Input image
GT
UNet
PHiSeg
Hu et al.
Ours
GT
UNet
Hu et al.
Ours
Figure 8: Qualitative results compared to the uncertainty baselines. We show uncertainty estimates in the form heatmap. Green highlights false negatives and yellow highlights false positives. Row DRIVE; Row 2: ROSE; Row 3: ROADS; Row 4: PARSE (3D render).
Reliability Olagrem (DAVE)
Relibbiiyy Diagram IROSEI
Petiatiity Ulagram mrosOs
Seliabiity Diagram OPARSEI
1.0
Hur
0,8
Durs
Meal
04
0.5
Figure 9: Reliability diagrams samples from each dataset.
Matching as they are topology- based metrics and hence are sensitive to the performance on thin structures. Detailed definitions are present in Appendix
4.1
Results
Tab. shows the quantitative results against uncertainty methods, and Tab, shows the quantitative results on different backbone architectures. We show the respective qualitative results in Fig. X and
Fig. 10 In Fig. 9 we plot the Reliability Diagrams. We also perform the unpaired t-test 66 (95% confidence interval) determine the statistical significance. Each table reports the mean standard deviations for every metric, with statistically significant better performances in bold and numerically better (but not significant) performances in italics. For all the probabilistic methods, the average of five runs was used. For method, we generated the structure-wiss uncertainty estimates and the segmentation map by following the steps outlined in the 'Inference procedure' in Sec. 3.. Due to space constraints, results two metrics Betti Number and Betti Matching reported in Appendix We discuss the remaining performances below.
Performance of uncertainty estimation. Tab. shows that our method outperforms others on both ECE and segmentation metrics. Fig. displays RDs, with our method following the ideal line much closely compared to others. This is because we explicitly model the distribution of the structures, thereby quantifying the uncertainty of the segmentation network. In Fig. g we also see that our method generates better fidelity structure-wisc uncertainty maps compared to Hu et al. Our heatmaps

---

Input image
GT
UNet
UNet+Ours
Ours (uncertuinty)
CS²-Net CS²-Net-Ours Ours (uncertainty)
Figure 10: Qualitative results over different segmentation backbones. Green highlights false negatives.
Row 1: DRIVE; Row 2: ROSE; Row 3: ROADS; Row 4: PARSE.
Table I: Comparison against uncertainty baselines (all use UNet [57] as the backbone)
Dataset
Method
(%)
Dier
elDicet
ARI
VoL.
DRIVE
UNee 32
8.3316 0.043
0.7779 1 0,0219
0,7663 = 0.0492
0.7759
0.0532
0.3560 1: 0.0203
PHiScg 4
7,9316 0.0032
0.7851 0.0295
0.7712
0.0497
0.7767
0.0497
0.3527 + 0.0308
Hu al. 2X
8.0883 + 0.0036
0.7866
1 0.0141
0.7725 = 0.0342
0.7768
1 0.0403
0.3489
+ 0.0286
4.1633 0.0043
0.0372
0.7996
0.0301
Prob.-UNet
32
7,2795 0.0022
0.7378 + 0.0284
0.6485
0.0258
0,7219 + 0.0538
0.7769 : 0.0146
ROSE
PlliSeg 4
7.0875 0.0036
0.7415 1 0.0267
0.6552
0.0236
0.7309
0.0425
0.7638
1 0.0128
Hu al.
6.9243
0.0033
0.7429
+ 0.0132
0.6598
0.0172
0.7506
0.0302
0.7616
+ 0.0123
Ours
3.9904 1: 0.0041
0.7593 1 0.0171
0.6782
0.0119
0.7837
1: 0.0314
0.7403
1 0.0239
PARSE ROADS
Prob-UNNet
32
8,4318
0.0042
0.7194 - 0.0418
0,8058
0.0613
0.7350
0.0494
0.5602
= 0.0308
PHiSeg 4
7,9331 + 0.0038
0.7203 + 0.0366
0,8113
0.0521
0,7392
0.0416
0.5559 + 0.0295
Hu al.
7,8034 0.0029
0.7275
1 0.0361
0,8282
0.0493
0.7314
0.0391
0.5644
1 0.0239
4.1442 0.0031
0.7461 0.0364
0,8496
0.0455
0.7601
0.0349
0.5463
0.0218
Proh. UNel 32
9.9918 1. 0.0069
0.6002 1. 5.7751
0.6179
0.0804
0.6523 1 0.0654
0.8923
1 D.0417
PHiSeg 4
9,9280
0.0077
0.5910 + 3.0858
0,6080
0.0743
0.6512
0.0521
0.8839
+ 0.0297
Hlu al.
23
7.7891
1: 0.0075
0.6044
1 2.1583
0.6153
0.0724
0.6537
0.0363
0,8803
1 0.0318
Ours
4.0289
0.0073
0,6990 1 3 3.0826
0.6221
0.0613
0.6658
0.0461
0.8701
1 0.0332
assign non-zero uncertainty to several false positivessneeaativs in the backbone UNet's outputs. This is because reason about every structure while Hu et al. limits the structure space via pruning.
Performance over different backbones. Tab. 2 and Fig. 10 show that our method is backbone- agnostic. It consistently improves the segmentation quality and produces high fidelity uncertainty maps for each of the underlying networks. This validates the practical applicability of our method.
Performance of proofreading. One of the motivations of this work streamline the proofreading process. Structure-wise uncertainty can be used as guide, with a user having to simply accept/reject a structure. We conduct experiments on the ROSE dataset and simulate user interaction with our method and Hu et as.'s. The user is given each method's final segmentation map, and inspects structures decreasing order of uncertainty (till 0.5). Each uncertain structure is subjected to a yes/no decision, which is denoted as one 'click', The results are in Fig. 11 Our findings are consistent with the observation that Hu et al. assigns zero uncertainty to many structures; thus their margin of improvement is limited and saturates quickly.
Proofreading: ciDice #tlic
Ours
0.72
Hu al.
dDic
0.70
0.68
0.56
10.0
12.5
clicks
Figure 11: Proofreading.

---

Table 2: Comparison against different segmentation backbones
Dataset
Melhod
Dice
elDicet
ARTT
VOL.
UNet 57
0.0336
0.7586
0,0405
0.7530 = 0.0819
0.3697
0.0329
DRIVE
UNel [57
Ours
0.7976 1 0.0195
0.7974 1: 0.0372
0.7996
- 0.0301
0.3322 1: 0.0229
Deep Nes 68
0,5015 0.0250
0.1997
0.0431
0.7729
D.0457
0.3413
DeepVesselNet 18
+ Ouurs
0.8173 = 0.0190
0.8285 + 0.0361
0.8037
= 0.0361
0.3238 + 0.0192
CS C5*-Net 48
0.8189 = 0.0176
0.8123 = 0.0413
0.8204
= 0.0495
0.3417 = 0.0203
CS*-Net -Net [48 + -urs
0.8301 = 0.0172
0.8367
0.0305
0.8495
= 0.0301
0.3243
0.0258
UNet [5]
0:0197
0.6453
0.0165
0.7206
0.0126
UNet [57
Ours
0.7593 +: 0.0171
0.6782
1: 0.0119
0.7837
0.0314
0.7403
0.0239
ROSE
Deep VesselNet [8
0,7653
0.0101
0.6634
0.0192
0.7622
0.0302
0.7436
0.0163
DeepVesseNNet XS
+urs
0.7795
0.0205
0.6873
0.0195
0.7936
0.0282
0.7164 + 0.0226
CS -Net CNe-NNt
[8
0.7623 = 0.0285
0.6799 + 0.0127
0.7702
0.0522
0.7236 + 0.0157
CS -Net [48 + Ours
0,7886 1 0.0208
0.6968
0.0149
0.7981 1 0.0211
0.7072 0.0168
UNet [5]
0.7011 0.0426
0,7918
0.0679
0.7143
0.0526
0.5832
0.0345
ROADS
UNet [57] Ours
0.7461 + 0.0364
0.8496 1: 0.0455
0.7601
0.0349
0.5463 + 0.0218
Deep VesselNet
0.7518 0.0345
0.8248 0,0574
0.7921
0044
0.5641
0.0321
DespVesselNer [S
Ours
0.7673 = 0.0324
0,8513 + 0.0519
0.8139
0.0464
0.5357 + 0.0329
CS -Net 48
0.7539 = 0.0366
0.8341 = 0.0S11
0.8197
= 0.0426
0,5475 = 0.0468
CS*-Net 48 + Ours
0.7692 = 0.0372
0,8559
0.0528
0.8368
= 0.0419
0,5261 0.0411
UNet [7]
0,5905 3.0661
0.6104
0.0727
0,6509
0.0852
1,9738
0.0414
PARSE
UNet [57] Ours
0.6190 1 3.0826
0.6221 1 0.0613
0.6658
0.0461
0,8701
1. 0.0332
Deep NesselNes 68]
0.7208 3.0452
0.6801 0.0554
0.6923
0.0524
0,4907
0.0701
DespeestelNet XS
Ours
0.7376 = 3,7863
0.6983 = 0.0622
0,709%
0,0613
0,4711 = 0.0613
CS -Net A3
0.7630 + 3.9415
0.6918 + 0.0695
0.7138
= 0.0695
0.4273 + 0.0521
CS -Nt 48
+ Ours
0.7720 +: 2.8109
0.7113 + 0.0689
0.7343
0.0733
0.4078
0.0642
4.2 Ablation studies
To demonstrate the efficacy of the proposed method, we conduct ablation studies of the differ-
ent components in our pipeline, as well as check the effect of changing hyperparameter values.
We also include ablation studies on the
Table Ablation of different modules
dimensionality of the input feature vec-
tor, and size of the crops/bounding boxes
DMT
Reg. Nei
(C))
ciDice
(which we report in Appendix D]. All
DMT
GNN
6.3481 1. 0.0082
0.7729
0.0304
analyses are on the DRIVE dataset using
Pmob. DMT
MI.P
4.8202 + 0.0046
0.7745 + 0.0305
UNet [57] as the backbone.
Prob. DMT
GNN
4.1633 + 0.0043
0.7974 0.0372
Ablation of different modules. We conduct ablation studies on both parts of our framework: structure generation (DMT vs Prob. DMT), and regression network (GNN vs Multi-layer Perceptron (MLP)). The results are in Tab. B Prob. DMT results in a sharp improvement in ECE compared to the original DMT; this supports our assertion that Prob. DMT models intra-structural uncertainty. Similarly, using GNN over MLP results in improvement. The message-passing in GNNs accounts for inter-structural uncertainty, thus yielding higher fidelity uncertainty estimates.
Effect of hyperparameters. Our main hyperparameters are u, ox 3, with u used in the Bernoulli distribution, 7 in the path-generation algorithm, and (shape a, scale 3) as prior hyperparameters of the Inverse Gamma distribution. We achieve the best ECE when however, a reasonable range always yields improvement, thus demonstrating the robustness of the method. We provide results of testing different hyperparameter values in Appendix]] !
0.3, 7 0.2, a 2.0, 3 0.01,
Conclusion
In this work, we propose to quantify the structure-wise uncertainty of given segmentation network. Our framework explicitly models structures as samples from a probability distribution, thus helping to estimate intra-structural uncertainty. Furthermore, jointly reasoning over the structures, resulting in better fidelity uncertainty estimates. This structure- wise uncertainty quantification can streamline the proofreading process by reducing the time spent finding and correcting errors. Extensive experiments show the practical applicability of our method over different segmentation backbones and datasets. We further discuss the broader impact of our work and the limitations in Appendix appendix[annd [ppendix(Kjeespectivel
incorporate inter-structural uncertainty by
Acknowledgements. This research was partially supported by grants NSF 2144901, NCI R21CA258893, and NIH IR2ICA258493-01A1. The content solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
10

---

References
[1] Abdar, M., Pourpanah, F,, Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth,
P., Cao, X., Khosravi, A., Acharya, U.R., et al.: A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion (2021)
[2] Arganda- Carreras, I., Turaga, S.C., Berger, D.R., Cireşan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J., Laptev, D., Dwivedi, S., Buhmann, J.M., et al.: Crowdsourcing the creation of image segmentation algorithms for connectomics. Frontiers in neuroanatomy (2015)
[3] Banerjee, S., Magee, L., Wang, D., Li, X., Huo, B.X., Jayakumar, J., Matho, K., Lin, M.K., Ram, K., Sivaprakasam, M., et al.: Semantic segmentation of microscopic neuroanatomical data by combining topological priors with encoder-decoder deep networks. Nature machine intelligence (2020)
[4] Baumgartner, C.F., Tezcan, K.C., Chaitanya, K., Hötker, A.M., Muchlematter, U.J., Schawkat, K., Becker, A.S., Donati, O., Konukoglu, E.: Phiseg: Capturing uncertainty in medical image segmentation. In: MICCAI (2019)
[5] Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmen- tation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:1412.7000 (2014)
[6] Chen, L.C., Papandreou, G., Kokkinos, L,, Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. TPAMI (2017)
[7] Clough, J.R., Byrne, N., Oksuz, Zimmer, V.A., Schnabel, J.A., King, A.P.: topological loss function for deep-learning based image segmentation using persistent homology. TPAMI (2020)
[8] DeGroot, M.H., Fienberg, S.E.: The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician) (1983)
[9] Delgado-Frieddricss, O., Robins, V., Sheppard, A.: Skeletonization and partitioning of digital images using discrete morse theory. TPAMI (2014)
[10] Dey, T.K., Wang, J., Wang, Y.: Graph reconstruction by discrete morse theory. arXiv preprint arXiv:1803.05093 (2018)
[11] Dey, T.K., Wang, J., Wang, Y.: Road network reconstruction from satellite images with machine learning supported by topological methods. In: Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (2019)
[12] Eaton-Rosen, Z.,, Bragman, F., Bisdas, S., Ourselin, S., Cardoso, M.J.: Towards safe deep learn- ing: accurately quantifying biomarker uncertainty in neural network predictions. In: MICCAI (2018)
[13] Edelsbrunner, Letscher, Zomorodian: Topological persistence and simplification. Discrete Computational Geometry (2002)
[14] Forman, R.: A user's guide to discrete morse theory. Séminaire Lotharingien de Combinatoire jelectronic only] (2002)
[15] Fraz, M.M., Remagnino, P., Hoppe, A., Uyyanonvara, B., Rudnicka, A.R., Owen, C.G., Barman, S.A.: Blood vessel segmentation methodologies in retinal images-a survey. Computer methods and programs in biomedicine (2012)
[16] Gal, Y., Ghahramani, Z.: Bayesian convolutional neural networks with bernoulli approximate variational inference. arXiv preprint arXiv:1506.025 (2015)
[17] Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In: ICML (2016)
[18] Gal, Y., Islam, R., Ghahramani, Z.: Deep bayesian active learning with image data, In: ICMI. (2017)
11

---

[19] Gal, Y., et al.: Uncertainty in deep learning (2016)
[20] Gawlikowski, J., Tassi, C.R.N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al.: A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342 (2021)
[21] Gonzátez-Hidalgo, M.: A survey on curvilinear object segmentation multiple applications. Pattern Recognition (2016)
[22] Gupta, S., Hu, X., Kaan, J., Jin, M., Mpoy, M., Chung, K., Singh, G., Saltz, M., Kurc, T., Saltz, J., et al.: Learning topological interactions for multi-class medical image segmentation. In: ECCV (2022)
[23] Haehn, D., Kaynig, V., Tompkin, J., Lichtman, J.W., Pfister, H.: Guided proofreading of automatic segmentations for connectomics. In: CVPR (2018)
[24] Hazan, T., Jaakkola, T.: On the partition function and random maximum a-posteriori perturba-
tions. In: ICML (2012)
[25] He, K., Gkioxari, G., Dollár, Girshick, R.: Mask r-cnn. In: ICCV (2017)
[26] Hu, X.: Structure-aware image segmentation with homotopy warping. NeurIPS (2022)
[27] Hu, X., Li, F., Samaras, D., Chen, C.: preeeegy- deep image segmentation. In: NeurIPS (2019)
[28] Hu, X., Samaras, D., Chen, C.: Learning probabilistic topological representations using discrete morse theory. In: ICLR (2023)
[29] Hu, X., Wang. Y., Fuxin, L., Samaras, D., Chen, C.: Topology-aware segmentation using discrete morse theory. In: ICLR (2021)
[30] Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? In: NeurIPS (2017)
[31] Kipf, T.N. Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint 10Kiv:660900290 (2016)
[32] Kohl, S., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J.R., Maier-Hein, K., Eslami, S., Jimenez Rezende, D.,, Ronneherger, O.: probabilistic u-net for segmentation of ambiguous images. In: NeurIPS (2018)
[33] Kv, R., Prasad, K., Peralam Yegneswaran, P.:
Segmentation and classification approaches of clinically relevant curvilinear structures: A review. Journal of Medical Systems (2023)
[34] Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: NeurIPS (2017)
[35] Lazaro-Gredilla, M., Dedieu, A., George, D.: Perturb-and-max -ar--att Sampling and learning discrete energy-based models. In: NeurIPS (2021)
[36] Li, C., Hu, X., Chen, C.: Confidence estimation using unlabeled data. In: ICLR (2023)
[37] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)
[38] Lovász, L.: Random walks on graphs. Combinatorics, Paul erdos is cighty (1993)
[39] Luo, G., Wang, K., Liu, J., Li. S., Liang, X., X., Gan, S., Wang, W., Dong, S., Wang, W., et al.: Efficient automatic segmentation for multi-level pulmonary arteries: The parse challenge. arXiv preprint arXiv:2304.03700 (2023)
[40] Ma, Y., Hao, H., Xie, J., Fu, H., Zhang, J., Yang, J., Wang, Z.,, Liu, J., Zheng, Y., Zhao, Y.: Rose: retinal oct-angiography vessel segmentation dataset and new model. TMI (2020)
12

---

[41] McClure, P., Rho, N., Lee, J.A., Kaczmarzyk, J.R., Zheng, C.Y., Ghosh, S.S., Nielson, D.M., Thomas, A.G., Bandettini, P.. Pereira, F.: Knowing what you know in brain segmentation using bayesian deep neural networks. Frontiers in neuroinformatics (2019)
[42] Meilä, M.: Comparing clusterings- an information based distance. Journal of multivariate analysis (2007)
[43] Milnor, J.W., Spivak, M., Wells, R., Wells, R.: Morse theory (1963)
[44] Mnih, V: Machine learning for aerial image labeling. University of Toronto (Canada) (2013)
[45] Mnih, V., Hinton, G.E.: Learning to detect roads in high-resolution aerial images. In: ECCV (2010)
[46] Mosinska, A., Marquez-Neila, P., Koziński, M., Fua, P.: Beyond the pixel-wise loss for topology- aware delineation. In: CVPR (2018)
[47] Mou, L., Chen, L., Cheng, J., Gu, Z., Zhao, Y., Liu, J.: Dense dilated network with probability regularized walk for vessel detection. TMI (2019)
[48] Mou, I.,, Zhao, Y., Fu, H., Liu, Y., Cheng, J., Zheng, Y., Su, P., Yang, J., Chen, L., Frangi, A.F. et al.: Cs2-net: Deep learning segmentation of curvilinear structures medical imaging. MedIA (2021)
[49] Murphy, K.P.: Conjugate bayesian analysis of the gaussian distribution. def (2o2) (2007)
[50] Naeini, M.P., Cooper, G., Hauskrecht, M.: Obtaining well calibrated probabilities using bayesian binning. In: AAAI (2015)
[51] Nair, T., Precup, D., Arnold, D.L., Arbel, T.: Exploring uncertainty measures in deep networks for multiple selerosis lesion detection and segmentation. MedIA (2020)
[52] Papandreou, G., Yuille, A.L.: Perturb-and-man random fields: Using discrete optimization to learn and sample from energy models. In: ICCV (2011)
[53] Peng, H., Long, F., Zhao, T., Myers, E.: Proof-editing is the bottleneck of 3d neuron reconstruc- tion: the problem and solutions. Neuroinformaties (2011)
[54] Rand, W.M.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association (1971)
[55] Ren, P., Xiao, Y., Chang. X., Huang, P.Y., Li, Z., Gupta, B.B., Chen, X., Wang, X.: survey of deep active learning. ACM computing surveys (CSUR) (2021)
[56] Robins, V., Wood, P.J., Sheppard, A.P.: Theory and algorithms for constructing discrete morse complexes from grayscale digital images. TPAMI (2011)
[57] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: MICCAI (2015)
[58] Rupprecht, C., Laina, L., DiPietro, R.. Baust, M., Tombari, F., Navab, N., Hager, G.D.; Learning in an uncertain world: Representing ambiguity through multiple hypotheses. In: ICCV (2017)
[59] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE transactions on neural networks (2008)
[60] Seeböck, P., Orlando, J.I., Schlegl, T., Waldstein, S.M., Bogunović, H., Klimscha, S., Langs, G., Schuidt-Erfurth, U.: Exploiting epistemic uncertainty of anatomy segmentation for anomaly detection in retinal oct. TMI (2019)
[61] Settles, B.: Active learning literature survey (2009)
[62] Shit, S., Paetzold, J.C., Sekuboyina, A., Ezhov, I., Unger, A., Zhylka, A., Pluim, J.P., Bauer, U., Menze, B.H.: eldice-a novel tion. In: CVPR (2021)
loss function for tubular structure segmenta-
13

---

[63] Sohn, K., Lee. H., Yan, X.: Learning structured output representation using deep conditional generative models (2015)
[64] Sousbie, T.: The persistent cosmic web and its filamentary structure-i. theory and implementa- tion. Monthly Notices of the Royal Astronomical Society (2011)
[65] Staal, J., Abråmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge- badge-based vessel segmentation in color images of the retina. TMI (2004)
[66] Stucki, N., Paetzold, J.C., Shit, S., Menze, B., Bauer, U.: Topologically faithful image segmen- tation via induced matching persistence barcodes. In: ICML (2023)
[67] Student: The probable error mean. Biometrika (1908)
[68] Tettch, G., Efremov, V., Forkert, N.D., Schneider, M., Kirschke, J., Weber, B., Zimmer, C., Piraud, M., Menze, B.H.: Deepvesselnet: Vessel segmentation, centerline prediction, and bifurcation detection in 3-d angiographic volumes. Frontiers in Neuroscience (2020)
[69] Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T.: Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing (2019)
[70] Wang. K., Qiu, Z., Wang, W., Song, T., Cao, S., Zhao, Y., Liu, J., He, Y., Gan, S., Liang, X., Xu, M., Guo, Z.: Pulmonary artery segmentation challenge 2022 (Mar 2022). http:://doi.og/III www..oggg//...////. www.////////ooo.3333. 1336196, httpt::://oioorrg//eeeoo 0ttp////o12212/oooooo 6366199000.66666666000011909
[71] Wang, S., Wang, Y., Li, Y.: Efficient map reconstruction and augmentation via topological methods. In: Proceedings of the 23rd SIGSPATIAL. international conference on advances in geographic information systems (2015)
[72] Yang, J., Hu, X., Chen, C., Tsai, C.: A topological-attention convlstm network and its application to em images. In: MICCAI (2021)
[73] Zou, K.H., Warfield, S.K., Bharatha, A., Tempany, C.M., Kaus, M.R., Haker, S.J., Wells III, W.M., Jolesz, F.A., Kikinis, R.: Statistical validation of image segmentation quality based spatial overlap index I scientific reports. Academic radiology (2004)
14

---

Appendix
Appendix provides pseudocode our proposed Probabilistic DMT module.
Appendix provides detailed discussion on the hyperparameters used in this work.
Appendix
reports the computational complexity of the method.
Appendix
expands the 'Inference procedure' outlined in 3.3 of the main paper.
Appendix provides the details of the datasets used.
Appendix gives implementation details of our method and the baseline methods.
Appendix provides definitions of the evaluation metrics used.
Appendix provides results on topological metrics.
Appendix provides results of ablation studies mentioned in Sec. 4.2 of the main paper.
Appendix discusses the broader impact of this work.
Appendix K
discusses the limitations of this work.
A Probabilistic DMT
In Algo. [ we provide a pseudocode of our Probabilistic DMT module proposed in Sec. 3.1 We set mar_ste _at__stppae 50 in our implementation. The terminologies used are: f denotes the likelihood map from H&s centered on structure e; (Ca, Cm) are critical points between which the path is generated: c, denotes the saddle point and C. denotes the maxima. IG' denotes the Inverse Gamma distribution, and N denotes the Gaussian distribution.
Algorithm 1 Probabilistic DMT pseudocode
1: procedure PROB DMT(f C,, Cm)
2:
~ Bernoulli(u)
3:
if is True then è
4:
5: else
6: é DEETHRATE_PATH(] Cs. Cm)
7: end if
8: return è
9:
end procedure
10:
procedure GENERATE_PATICC, Co. Cm)
initialize initialize c Co
m has same spatial dimension
12:
initialize m|c +-
14:
initialize step +- O
o² TG ())
16:
fn ~ 1- +N(00))
while c + Cm and step < max_step do
18:
val +-
19:
for E Neighborhoood() do
20:
val[c'] + Th -e'la (1 7) fn(d'
21:
end for c argmax(val)
22:
Update current step
m|c 1 24:
step + step +
end while
26: return m
end procedure
15

---

B Discussion of hyperparameters
The main hyperparameters in this work are u, a. v. We describe the importance of each below:
This is the parameter for the Bernoulli distribution. In our Prob. DMT module, for every structure, we have a choice to either retain the structure as obtained from DMT, or, generate sample skeleton using the perturb-and-walk algorithm. We model this choice using the Bernoulli distribution. Essentially, in some runs we would like the original DMT structures to also interact with the others. Thus low value of works best. We found the best performance, that is, for every structure there is a 30% chance that it's DMT form is used and a 70% chance that sample variant is used. We find that 0.15 < S 0.3 have comparable performance.
0.3 to give
y: This hyperparameter is used in the weighted combination of distance Qa and likelihood fn to obtain Q(c'), which is used to determine the next pixel location. It maintains a tradcoff between the distance regularizer Qa and the perturbed likelihood fn- The higher the value of y, the greater the distance regularizer, and consequently the generated path will become closer to that of a straight line. This is not desirable, as a straight line would lose the original composition of the structure. Additionally, because of the perturbation in the likelihood, do not want the path to go astray. To ensure path completeness, we require Y to be non-zero. Through experiments, we obtain the best performance when 7 0.2.
a, 3: These are prior hyperparameters of the Inverse Gamma (IG) distribution. We perturb the likelihood using a Gaussian model. As the variance of the Gaussian model is unknown, we use Bayesian probability theory sample the variance from the IG distribution (its conjugate prior). And so, a is the shape parameter and /3 is the scale parameter of this IG distribution. Ideally would like small perturbation of the likelihood and not strong one. This is because a strong perturbation would corrupt wholly and we would not be able sample reasonable skeleton. At the same time, the perturbation should not be too small, otherwise we will not obtain significant variant. The mean of the IG distribution is Q-I (when > 3, >> 0), which on average is the value of the sampled variance for the Gaussian distribution. We achieve the best performance when a = 2.0 and B = 0.01. The resulting sampled variance for the Gaussian model thus generates reasonable perturbation.
3
C Computational complexity
We report the inference time for 5 runs a 256 256 input image patch as follows: Prob.-UNet: 0. 1.196 see; PHiSeg: 1.811 see; Hu et al.: 5.485 see; Ours: 7.433 sec.
The module which takes the most time is the DMT/ / DMT computation. Presently, this is the most optimized version as we have implemented it as an external module in C++. We will work towards porting the code to run on GPU to bring down the runtime even more.
Following 10], the computational complexity of DMT is On log n). where is the number of pixels in the image. Since Prob. DMT additionally computes structure variants, the complexity is O(n log n + m) where m approximately the number of foreground pixels, and typically m << n for curvilinear structure datasets. The linear term is added as we traverse each foreground pixel only once when generating the sample skeleton.
D Inference procedure
We illustrate the inference procedure in Fig. 12 There two outcomes of our framework Mg. namely, the trueture-wise uncertainty heatmap as well as an improved discrete segmentation map that can be used instead of the obtained by Fe.
For each structure e, obtain p(e) (the regression output) and 82 (the uncertainty) from Mg- We take runs of Mg and then for each structure e, we compute the mean across T runs as -= (82). and, p(e) = = E p(e).. Next, we consider only those structures for which p(e) 2 0.5, i.e., has a minimum probability of 50% of being true positive. This the threshold
52 ===1 1 Et
step in Fig.
12 We do this so to consider only the true positive, false positive, and false negative
16

---

Ireb.
Grodl
Ausi
Sivar ANGonaA
Answay
Figure Inference procedure. Stars (+) denote the final outcomes of our framework M
Fig. 12 (d)
Fig. 12 (f)
cesste gagna via Cesttoo distance
Fig. 12
Figure 13: Zoomed-in views of Fig. 12
structures in the final outcomes. We use these structures to create skeletal discrete segmentation map (see Fig. [2(c)) which has the same spatial resolution as Fig. a)).) As we want to recover the thickness of each structure, we overlay the two maps to get the final discrete segmentation map (see
Fig. 12 e)).
The uncertainty heatmap that we obtain from Ma is also skeletal (see Fig. d2)... We recover the structure thickness to get the final uncertainty heatmap (see Fig. 12 f)) We shortest distance do this. Shortest distance used to assign uncertainty values from Fig. 12 to the pixels in Fig. e).. The shortest distance This ensures that pixels within a structure are not assigned uncertainty values from other nearby structures. We provide a zoomed-in view in Fig. I3
paths only along the foreground pixels and not along the background ones.
We note that generating the Morse complex is computationally heavy, however, it needs to be computed only once across the T runs. As described in Sec. 3.1 the sampled structures are between (c,, Cm), and so the Morse complex is generated only in the first run.
E Dataset details
In this paper, we validate our results on four segmentation datasets: DRIVE ROADS [44), and PARSE 2022 Grand Challenge [39 77).
655
ROSE
17

---

DRIVE 665 The DRIVE dataset is 2D retinal vessel dataset with 40 images. Each image has resolution of 584 565. We use the dataset's predetermined split of 20 training images and 20 test images. For training, we keep aside four randomly-chosen samples as validation, and train on the remaining 16 samples.
ROSE 40) The ROSE dataset is a 2D retinal OCTA (Optical Coherence Tomography Angiography) segmentation dataset. We use ROSE-1 (SVC) in this work. has a predetermined split of 30 train and 9 test samples, with each sample having a resolution of 304 x 304. For training, we keep aside four randomly-chosen samples as validation, and train on the remaining 26 samples.
ROADS [44] The ROADS dataset is a large, non-medical dataset containing 1171 aerial images (1108 train, 14 validation, and 49 test), each of 1500 x 1500 resolution. It is challenging dataset due to obstruction from nearby trees, shadows, varying texture/color of roads, road class imbalance, and
PARSE [3] The PARSE dataset a CT dataset containing pulmonary artery segmentations. The dataset contains 100 volumes and their sizes vary from 512 x 512 x 228 to 512 512 x 376. As there is no predetermined train/test split, we use 4-fold cross-validation and report the average performance.
F Implementation details
We use the PyTorch framework, a single NVIDIA Tesla V100-SXM2 GPU (32G Memory), and a Dual Intel Xeon Silver 4216 (PU@22100 (16 cores) for all the experiments. The training hyperparameters for our method as well as the baselines are as tabulated in Tab. 4 Note that although PARSE is a 3D dataset, all the segmentation networks (backbones) Fe are 2D, that is, the networks are trained on 2D slices of the dataset. This was done to maintain a fair comparison across all baselines, as methods such as Prob.-UNet and PHiSeg only had 2D implementations available.
F.1 Baselines
We use the publicly available codes for the baselines:
UNet [57]
Prob.-UNet
321
http:://gittbb.cco//tteeannneee
PHiSeg
Hu et al. 28
http:::/gtthub..cmm http:////tttuunnn//iiiilllllauur
DeepVesselNet 68
C 48
httpgs//gghhubbccm/////Daa/////3
F.2
Our method
Our code available at /ttttps://githuub. http://www..oome/mmmmumuuuttaaaa Suppaaaaaaaaaattt-ttturra 1/66/struct-oo cnntctuuneerttttyny For reproducibility. provide the architecture details as follows. The 'Joint reasoning' module de- scribed in Sec. 3.2 of our framework is a Graph Neural Network (GNN) (59, specifically a Graph Convolution Network (GCN) (31. As per Fig. [6] the input feature vector for each graph node constructed by passing [x fo f0,,7 ml through the following architecture: C(3, 24) + ReLU > D(0.2) C(3, 32) ReLU + (0.2) + MaxPool + Concat (pers) where, C(a, b) denotes a convolution layer having kernel size a and number of output channels b, D(p) denotes a Dropout layer with probability pt MarPool denotes the adaptive maxpool I returning for each channel; and pers is a scalar value denoting the persistence of the structure. Furthermore, the bounding box size of [x", fr, m] is 32 x 32 centered at each structure for 2D (for 3D, it is 32 x 32 x 32).
1 output
The aforementioned layers generate the input feature vector for each graph node. They are then passed through the GNN which contains the following layers: GCN + ReLU + D(0.2)
http:://wppccooogoooooo/ss/aaaaa www..aaawpatiieeeooool...d. html
18

---

Table 4: Training configuration
Datasel
Model
Patch Size; Batch Size
Learning rale (L.R;; Oplimizer
UNot
128 x 128:8 S
Ic-3 with schedule Adan with weight decay 30-5
Deep Ves
128 x 128; 8
1e-3 with LR scheduler; Adam with weight decay 3e-5
DRIVE
CS Net
128 x 128; 8
1e.3 with LR scheduler; Adam with weight decay 3e.5 le-3; Adam with weight decay
Prob.-NNet
x 128:
PhiSeg
128 128;
4 with LR scheduler; Adam with weight decay 1e-5
Hu et al.
128 x 128; 8
le-3; Adam with weight decay
128:
1o-3; Adam with weight docay
UNet
128 x 128; 6
le.3 with LR scheduler: Adam with weight decay 3e:5
ROSE, ROADS
DespVesselNet
x 128;
le-3 scheduler; Adam weight decay
CS2-Net
x 128:
1c-3 with scheduler; Adam with weight decay 30-5
Prob-UNet
128 x 128; 6
1e-3; Adam with weight decay 0
PhiSeg
128:;
scheduler; Adam with weight decay
Hu et al.
128 x 128: 6
1e-3; Adam with weight docay
Ours
128 x 128; 6
1e-3; Adam with weight decay
UNot
128;
Ic-3
scheduler. Adam with weight decay
DeepVesselNet
128 x 128; 8
1e-3 with LR scheduler: Adam with weight decay 3e-5
PARSE
CS? -Net
128 x 128; 8
le with LR scheduler; Adam with weight decay 3e
Prob--UNet
128 128:
le-3; Adam with weight decay
PhiSeg
128 x 128: 8
lc-4 with LR schedulor: Adam with weight decay lo-5
Hu
128:
1e-3; Adam weight decay
Owrs
128 x 128.
1e-3; Adam with weight docay 0
GCN(64) + ReLU D(0.2) GCN(322 ReLU D(0.2), where GCN(a) denotes
GCNConv Shaving a number of output channels. The output from this sequence of layers is then fed to two separate GC)(() layers output the regression likelihood p(e) and the uncertainty 82. As per GNN fashion, the weights of the layers are shared across all the nodes.
G Evaluation metrics
We use both segmentation and uncertainty metrics to evaluate our method. We describe the metrics in detail below.
G.1 Uncertainty metrics
We use Reliability diagrams 8 and Expected calibration error (ECE) (50 to evaluate the quality of uncertainty. As both the metrics were originally designed for classification, we adapt from the classification task to semantic segmentation by treating each pixel as an independent sample. For both metrics, we first divide the probability interval [0, 1] into N equal-sized probability intervals (each size ) ). We use N = 20 bins in this work. We then calculate the accuracy and confidence of each bin.
Reliability diagrams (RD) Reliability diagrams are visual representation of model calibration
by plotting the expected accuracy as function of confidence (con fionfedecce = 1 uncertainty). Perfect calibration corresponds an identity function in the RD, i.e., the model is not over/under- confident. Consider the set of pixels/structures whose predicted probabilities fall into the bin B, The accuracy and confidence are given by:
acc(B:)
Σ
1 (P(x)=
=
B.
VrEB;
cB,)(B;) =
P(x)
B,
FrεB;
conv. http://www..oaaooooooeeoecceeecc
http:////.ccaaaaloo//////c/ocecr RecuceeLROnP/ateeau..mmmml
http:////.yoococooo////////////c hww.:hoottt.mmmmmhtttttm.m. Adam.html
19

---

Table 5: Comparison topological metrics Betti Number and Betti Matching for different datasets. All methods use UNet as the backbone. f8 5tt and 6ee denote Betti Number error in O-dim, 1-dim, and 2-dim respectively. μ0 and μu"" denote Betti Matching error in O-dim and 1-dim respectively. The statistically significant better performances are highlighted in bold
Dataset
Method
1
Pg
M1
UNet
166.3154 = 12.1065
9.3149 + 4.2062
205.8312 1 12.4496
28.9587
5.3766
DRIVE
Prob--Nect
146.6373 11.0831
7.8197 + 3.1980
191.2790 11.2324
24.7826
4.9684
PHiSeg
145,3777 = 12.4873
7.1542 + 3.6436
190.0528 + 10.3576
24.0893
+ 4.1123
Ilu et
140.8317 - 11.5502
2.2372
188.6573 10.2403
23.7263
Ours
127,4041 = 10.7344
4.6172 + 2.6586
161.4536 + 9.7017
20.6835
+ 3,4121
UNet
231.5081 Σ 15.5573
9.8826 1.6486
243.2775 1: 16.9274
14.8922
1.6793
ROSE
Prob-UNNet
229.7987
15.4307
9.0396 1.6315
240,3295 16.8371
14.0771 + 2.0375
PHiSeg
220.0644
= 14,6356
7.8644 + 2,0692
228. 4344 1: 17.8907
11.0377 + 1.9422
et
219,7530
7.6981 1.5677
226.2989 16.1992
Ours
203.5791
13.6467
5,05S3 + 1.4734
210.1763 + 15.1485
8,6489 = 1.5646
UNet
75.6666
8. 1.1079
25.5777 1. 7,4432
78.2291 1 9. 9.5055
30.5104
6.6921
PARSE ROADS
Proh-UNet
70.3564
7.5929
24,1852 + 7.0812
72.8129 9.1638
29,8630 6.1777
PHiSeg
68.7237 = 7.9177
24.1772 + 6,5982
70.2788 + 8.2474
28.9467 = 5.9164
et
+: 23.5863
62.4951 7.7601
5.8736
Ours
48.6735
5.9286
17,2683 + 4.6162
47.1429
+ 6,7905
23.1837 + 5,4451
UNet
673.7016 = 23.9541
79.5825 10.9693
18.4316 - 29432
Prob-UNet
620.1903 22.0012
51,4995 + 8.4096
16.7046 2.2499
PHiSeg
587.2137 = 22.6800
45.9331 + 8.7251
15.8529 = 3.0218
Hu et Ours
555,9788
+
13,9498
520.4991 22.4327
33,0532 + 7,8453
10,3831 2.1035
where, Y is the discrete segmentation ground truth (GT), and Y is the discrete segmentation map outputted by the model. In our method, Y is shown in Fig. (2)c). Additionally, f is the pixel- wise probability (likelihood) outputted by the model, whereas in our case, it is the saucture-wise uncertainty 8 (Fig d)) For method and Hu et al., the B, denotes structures, while in the other methods, it denotes pixels.
Expected calibration error (ECE) [50] RDs are only visual cue, and so we also use ECE: scalar to summarize the calibration performance. RDs do not take into account the number of pixels/structures in each bin. Thus, to account for such variations of the number of samples in bin, we use ECE. given by:
Bil n
ECF
Jace( (B:) con [B.)]
where n Σx [Bel is the total number of pixels/structures The difference between ace and conf for a given bin represents the calibration gap. When there is perfect calibration, ECE is zero. The definition of acc and con remains the same as defined for RDs.
G.2 Segmentation metrics
DICE (73 DICE score popular metric which measures the area/volumetric overlap between the predicted and ground truth discrete masks. It overcomes the class imbalance problem in the pixel-wise accuracy metric by considering only the foreground classes for measuring the overlap. The higher the DICF, the better the segmentation.
elDice 62 elDice is derived from DICE, however, clDice uses the skeleton of the predictions. This makes it sensitive to the performance of thin structures like vessels which is important in curvilinear segmentation. The higher the value, the better the segmentation.
ARI [44] The Rand index computes similarity between two clusterings. This raw score is "adjusted for chance" to get ARI (Adjusted Rand Index) (2). The ARI takes into account the fact that some agreement between the two clusterings can occur by chance, and it adjusts the Rand index to account for this possibility. The higher the value, the better the segmentation.
VOI 42 The VOI metric defined as the sum of the conditional entropies between two segmenta- tions. A lower VOI value indicates better segmentation.
20

---

Table 6: Additional ablation study results of method the DRIVE dataset using UNet the backbone. The best results (as reported in Tab. of the main paper) are in bold. In the table, Feature vector denotes the length of the input feature vector the GCN, while Bounding box denotes the size of the crop/bounding box centered on each structure. These hyperparameters are discussed in detail in Appendix B The denotes the concatenation of the scalar persistence value
Fenture vector
ECE (%H)
elDice?
16
4.9621 0.0048
0.7906 +: 0.0394
32 +
4.1633 # 0.0043
0,7974 = 0.0372
4.1667
Bounding box
ECE (%1]
elDiee?
16
0.0047
=
32 x 32
4,1633 0.0043
0.7974 = 0.0372
64 x 64
4.1689 + 0.0042
0.7921 + 0.0363
Betti Number error [2 This metric computes the difference between the Betti numbers of the prediction and the ground truth. We separately compute this metric for O-dimension and I-dimension structures.
Betti Matching error [66] This metric is similar to Betti Number, however, in Betti Matching, the spatial location of the O-dimension and I-dimension components is also taken into account. Hence this metric is stricter compared to Betti Number error.
H Quantitative results on topological metrics
In Tab. we provide results on two topologically important metrics, namely, Betti Number error 27 and Betti Matching eιror [66). Our method consistently improves the segmentation result in terms of topology. This consistent with our results in Tab. of the main paper where our method outperforms the baselines on other topology-basee metrics like elDice [62]. ARI (2 and VOI [42]. Note that for the 3D PARSE dataset, we were unable to provide Betti Matching error results as its official implementation handles only 2D inputs.
I Ablation study
We check the effect of the different hyperparameters in our work by conducting experiments on the DRIVE dataset using UNet the backbone. Our main hyperparameters
ECE vs. (a.B)
B= 1.0e-2
4.8
are 0,,,, 3,, 7, with u used in the Bernoulli distribution, 7 in the aath-generation algorithm, and (a, 3 prior hy-
2.0e-2
ECE (%)
perparameters of the Inverse Gamma distribution. We test different values and report the ECE (the lower the better)
4.4
in Fig. 14 For all the experiments, we set u = 0.3. We 0.01,
achieve the best ECE when 7 0.2, a 2.0, however, reasonable range always yields improvement (notice how non-zero y results in sharp improvement). This demonstrates the robustness of our proposed method.
1.3
ECE vs. Y
4.8
We also provide additional ablation studies in Tab. 6 We now include ablation studies on the dimensionality of the input feature vector, and size of the crops/bounding boxes, and report this in Tab, 6 We obtain the best results when the input feature vector size is 32 and the bounding box is 32 x 32, For lower values (16 and 16 x 16), the perfor- mance reduces, while for higher values (64 and 64 x 64) we did not observe any statistically significant improve- ment. Thus maintain the tradeoff between complexity and performance, we respectively use 32 and 32 x 32 for these hyperparameters.
4.7
4.6
ECE (%)
4.5
4.3
4.2
Figure 14: Effect of hyperparameters.
is the shape parameter; is the scale parameter.
21

---

Broader impact
In this work, we aim capture structure-wise uncertainty of a given network, where structure is defined to be coherent set of pixels user can intuitively understand, e.g., small vessels/branches, short stretches road etc. Fine-scale structures such as vessels, neurons, and membranes often consist of interconnected branches or structures that form a cohesive entity. Thus structure structur-wwise wise uncertainty maps can highlight uncertain instances or branches as whole, providing more accurate indication of regions where the segmentation may be inaccurate or uncertain. This is beneficial for proofreading or error-correction tasks as they can direct the focus of human annotators to uncertain structures that require further attention. This can save time and effort compared to pixel-wise uncertainty maps that highlight numerous pixels as uncertain, many of which do not require correction. Thus structure- wise uncertainty provide more interpretable estimates and is desirable approach for improving segmentation accuracy and supporting downstream analysis tasks. This can go a long way the benefit of proofreading is twofold: it improves segmentation quality, and it also helps expand the body of labeled data that can be further used to train automatic segmentation methods. Our work is thus useful tool in streamlining the process of scalable annotation. At the present stage, do not foresee any potential negative societal impacts.
K Limitations
Our method currently fits in the context of curvilinear segmentation. In general, large object segmentation could also benefit from structure-wist uncertainty (structures in this case would be smaller patches/volumes), Discrete Morse theory can be used in this setting, however, we would need to make use of topological features other than the stable manifold. In its present form, our proposed solution currently not applicable in a setting beyond curvilinear segmentation.
22


Reviews:
[{"review_id": "BV3RfvG0Fq", "reviewer": "Reviewer_qp6i", "summary": "\nThis work aims to contribute to proofreading by proposing uncertain structures in a topological sense. The work proposes a method to quantify a form of structure-wise uncertainty from segmentations, where the framework explicitly models structures as samples from a probability distribution. First, the structures are extracted via discrete Morse theory (DMT). Next, the uncertainty is modeled via a joint prediction model that estimates the uncertainty of a structure in consideration of the surrounding structures. Furthermore, the authors propose a novel probabilistic DMT concept to model intra-structure uncertainty. The method is then successfully experimentally validated. ", "questions": "This is a minor question regarding the utility of the method. The authors motivate by and mention that their contribution is a way to simplify and improve proofreading. Do you have practical experiments in a proofreading setting that you can share? Is there an experiment with human experts, e.g., readers in ophthalmology? ", "limitations": "The authors do not provide a dedicated limitations section. I would like to learn more about the limitations of their method in the context of stricter definitions in algebraic topology. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-  The work presents a real generalization of the work on DMT for segmentation [25]. E.g., if I understand it correctly, if one chooses perturbation 0, the result of the method will be exactly the DMT result. This is a nice property and a strong extension of prior work. \n\n- The motivation is clear and interesting. \n\n- The method is well described and formalized. ", "weaknesses": "\n1) **Experimentation:** The authors state to propose a \"topology-aware\" method. However, the authors do not evaluate their results on popular topology-related metrics. E.g., Betti number errors in dimensions 0,1, and 2 (for the 3D dataset). Evaluation of the performance with respect to these metrics will improve the Experimentation. Comparison to the recently published Betti matching error (1), which considers the spatial agreement of the topological structures, would further increase the interpretability of results. \n\n\n2) **Method** If I understand the method correctly, the DMT calculation at the end could be seen as a post-processing step or additional network to improve connectivity on the results; it has been shown for curvilinear structure segmentation that such an additional step improves segmentation performance. Clearly, such a concept makes this a multi-step procedure which has limitations compared to the cited method by Hu et al. \n\n3) The definition of \"topological structures\" is not very clear, and to me, it appears to not align with some definitions in algebraic topology, especially in dimension-1. Intuitively I would expect a \"topological structure\" to be (e.g., in dimension-1) a closed loop. This appears not to be the case here. If I misunderstand this, could the authors clarify how they represent the cycles, and how this is different from their representation of features in dimension-0 and provide more explanations? \n\n\n\n**References**: \n\n[] are references from manuscript \n\n(1) Stucki, N. , et al. \"Topologically Faithful Image Segmentation via Induced Matching of Persistence Barcodes.\" ICML (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback! Please find our responses to specific queries below.\n\n**Q1:** The work presents a real generalization of the work on DMT for segmentation [25].\n\n**A1:** We would like to clarify that the goal of this paper is very different from [25]. [25] and other topology-preserving segmentation methods focus only on a segmentation network. In contrast, our method assumes a given segmentation network, and focuses on estimating the uncertainty of the given segmentation output at a structural level. Topology-preserving segmentation is only one of the many applications of our work.\n\n**Q2:** Evaluate results on popular topology-related metrics, e.g., Betti Number errors and Betti Matching error. The comparison would further increase the interpretability of results.\n\n**A2:** Thank you for the suggestion. We now provide these results; please see 2. in the global response ‘Author Rebuttal by Authors’ above.\n\n**Q3:** If I understand the method correctly, the DMT calculation at the end could be seen as a post-processing step or additional network to improve connectivity on the results; it has been shown for curvilinear structure segmentation that such an additional step improves segmentation performance. Clearly, such a concept makes this a multi-step procedure which has limitations compared to the cited method by Hu et al.\n\n**A3:** As we clarify in Q1/A1, given a segmentation network, our goal is to capture the uncertainty of its prediction at a structural level. As the segmentation network is not part of our contribution (and instead is an input to our method), our work cannot be considered as a post-processing/multi-step approach. This is also reflective of real-world scenarios where segmentation networks are often black-boxes, with users being allowed to only access results of the network and not its internals. Thus using only the results of the given segmentation network, our method is able to generate structure-wise uncertainty estimates to streamline the proofreading process. \n\n**Q4:** The definition of \"topological structures\" is not very clear, and to me, it appears to not align with some definitions in algebraic topology, especially in dimension-1. Intuitively I would expect a \"topological structure\" to be (e.g., in dimension-1) a closed loop. This appears not to be the case here. If I misunderstand this, could the authors clarify how they represent the cycles, and how this is different from their representation of features in dimension-0 and provide more explanations?\n\n**A4:** This is an excellent question and we would like to clarify the relationship between Morse theory and the theory of persistent homology [4]. In L147, we describe Discrete Morse theory (DMT), and specifically in L159-161, we state that the “structures” are zero- and one-dimensional Morse structures. Morse structures are essentially critical points and special paths connecting them. Morse theory has a very strong relationship with the theory of persistent homology; given a Morse complex, one can exactly compute the persistent homology [5,6]. Discrete Morse theory has been used in the literature for the computation and simplification of persistent homology. \n\nRegarding the representation of cycles/loops (dim-1) in the sense of algebraic topology, while we do not explicitly model them, our method implicitly induces their topological correctness. We express in the paper that our method is topology-aware because of the strong relationship Morse complexes have with topology. When the prediction of the Morse structures is correct, the topology in all dimensions is guaranteed to be correct.\n\n**Q5:** Do you have practical experiments in a proofreading setting that you can share? Is there an experiment with human experts, e.g., readers in ophthalmology?\n\n**A5:** Yes, in L308 of the main paper, we had included results of proofreading experiments comparing our method with Hu et al.’s on the ROSE dataset. Our method was able to improve the segmentation result significantly with a relatively fewer number of clicks. We conducted these experiments with a group of researchers. As future work, we have plans to include clinicians to test our framework.\n\n**Q6:** The authors do not provide a dedicated limitations section. I would like to learn more about the limitations of their method in the context of stricter definitions in algebraic topology.\n\n**A6:** We included the limitations section in Section 14 of the supplementary. As answered in Q4/A4 above, our method uses Morse theory instead of persistent homology, and that there is a strong relationship between the two. As future work, we find that beyond curvilinear segmentation, general object segmentation can also benefit from structure-wise uncertainty (structures in this case would be smaller patches/volumes of the object). Discrete Morse theory can be used in this setting, however, we would need to make use of topological features other than the stable manifold. \n\nThank you very much for your review! We hope we were able to clarify your comments, and we would be happy to discuss further!"}, {"review_id": "d5uT7yNnhN", "reviewer": "Reviewer_oww6", "summary": "This paper proposes a novel method for the estimation of uncertainty of the structures in the segmentation results from existing methods/models, in order to facilitate the subsequent proofreading process. To this end, it models the intra-structure and inter-structure uncertainties in two modules. While the former considers the geometry, contrast and model’s confidence, the latter considers neighbours and thus context. The former is implemented using the Probabilistic discrete Morse theory (DMT), which samples the Morse skeletons using the inverse Gamma distribution. The latter uses graph neural network (GNN) for the prediction of the uncertainty of all the structures jointly through regression using the attenuation loss. The proposed method has been validated over three publicly accessible datasets and compared with several state-of-the-art. Relatively better results have been obtained in different metrics. A number of ablation studies have also been carried out on variants of the DMT and GNN and some hyperparameters. Some insights have been accumulated into how the proposed method behaves under different configurations.              ", "questions": "Please see the comments/concerns in weaknesses,\n\nMore detailed comments are as follows:\n1.\tL41: accept or reject/correct structural proposals efficiently: relative to what: prior knowledge or experience?\n2.\tL132, each structure: some elaborations would help: what that is: line segment, blob, junction, keypoint, or anything else.\n3.\tL140-14, At each training iteration, it takes one sample skeleton for each structure: Does this mean that the number of iterations will be determined by the number of the total structures and the number of sampled skeletons for each structure? How to count the number of the structures? any details?\n4.\tL170, a perturb-and-walk algorithm: refs?\n5.\tL192-194, At every step, we always walk to the neighboring pixel with the  highest likelihood value: This is a heuristic only, which may not hold in some cases.  \n6.\tL224, The output of Prob. DMT is effectively one sample skeleton: is the same number of sample skeletons as the number of runs? Can all the sample skeletons be directly used to calculate the uncertainties? How to count the number of structures?\n7.\tL227-228, network that takes as input each structure: This is confusing: takes as input each structure each time or all the structures together? If the former is the case, do you have to run the model many times? what is the motivation for this? Can they be combined together and run the model once to directly estimate the uncertainties?\n8.\tL237, are smaller crops/bounding boxes: what are the sizes of the boxes? Any details?\n9.\tSe 3.3: shortest distance: This is heuristic only, which may not hold in some cases.\n10.\tL299, others -> the others\n", "limitations": "The limitations of the proposed work are discussed in the supplementary materials.\n\nThe potential negative social impact are not highly related and thus are not discussed. ", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.\tThe topic of the paper is interesting and important for the subsequent validation of the results for image segmentation and find many applications in the read world such as medial image segmentation and analysis, object classification and recognition, industrial quality assurance, etc.\n2.\tA novel method has been proposed to post-process the segmentation results produced by some existing models for their proofreading and validation. It includes two main modules: intra-structure uncertainty and inter-structure uncertainty estimation. While the former is modelled using the probabilistic DMT, the latter is modelling using the graph neural network. The method is well motivated and supported by solid theory.    \n3.\tThe proposed method has been validated over three publicly accessible datasets and compared with several state-of-the-art. Relatively better results have been obtained in different metrics. The t-test has also been performed whether the improvement is significant.\n4.\tA number of ablation studies have been carried out on the variants of DMT and GNN and some hyperparameters in the process of sampling of Morse skeletons. Some insights have been accumulated into how the proposed method behaves under different conditions, that would instruct how it can be applied in the real world.\n5.\tThe proposed uncertainty estimation has been applied to re-calibrate the segmentation results obtained. The experimental results have shown that it does help identify the false positives and false negatives.      \n", "weaknesses": "1.\tWhile the paper targets the proofreading of the segmentation results by experts, its necessity could be emphasised: the automatic algorithms cannot guarantee  the correctness of the segmentation, especially for medical imaging where the anatomy and structures may vary from one subject to another, and their prior knowledge is not always available  This is also the process for the relevant researchers/experts to learn and accumulate insights about the variation of the structures of the subjects for individualised diagnosis, treatment and medicine.\n2.\tSome details, elaborations, clarifications and discussions are missing from place to place. For example, while the construction of the graph is described later, its main idea could be summarised in Introduction. “structure” is widely used throughout the paper but its count has ever been discussed. While the ground truth is again used to train the GNN, some discussions could be made about its special requirement: GT plays a crucial role in the validation and guiding the estimation of the uncertainty for false positives and false negatives and thus, may have special requirement about quality and reliability. It is not clear how to guarantee this, especially when the datasets are large and include subtle structures. On the other hand, is that possible to use the salient structures to guide the search for faint structures in the spirit of Canny edge detection?  “Persistence value” in L239-240: how to calculate it? where is it from? No details or references are given.    \n3.\tThe key steps for the re-calculation of the Morse skeletons in probabilistic DMT may require further investigation. The next point selection criteria should be stated explicitly: maximize the sum of the inverse distance and the likelihood from the candidates and discuss why this criterion is feasible, especially when the distance is not really comparable directly with the likelihood. For the definition of Q(c’), this definition may require further investigation: what is the rationale for this definition? while the first term is inverse distance, is that comparable to the likelihood in the second term? Is this optimal? Is there any other alternative? Is that possible to draw conclusion that the first term will always lie in the unit interval [0, 1]? Overall, this is still a heuristic, which may not hold in some cases. “This process is done separately and in parallel for every structure.” In L222-223: how many Morse skeletons were sampled for each structure and can they be directly used to calculate as uncertainty their means and variances? The final results may be affected by the number of runs and their combinations made.\n4.\tThe computational complexity and time have not been analysed and reported. Thus, it is not clear how efficient the proposed method is and how much time it requires to process a set of given images. \n5.\tFurther analysis of the experimental results would help. For example, while the proposed method is effective in improving the results of the existing methods in ECE, clDice, ARI and VOI, but it is not always in Dice. It is not clear why. Any further insights and explanations would really help.  \n6.\tMore ablation studies could be carried out on other parameters and components such as the dimensionality of input feature vector,  crops/bounding boxes on the structure, alternatives to the shortest distance for structure inference. \n7.\tThe claim that the proposed method may be applicable to non-medical applications: civil engineering, road network and railway track segmentation, has not been validated. \n8.\tIt is not clear how the prior knowledge can be used to guide the image segmentation, uncertainty estimation, and result re-calibration, rather than just based on some heuristics in skeleton-recalculation and structure inference, while such heuristics may not hold in some case. Also, it is not clear how such heuristics contribute to the final errors. Some further investigation would be encouraged.  \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback! We will revise our manuscript accordingly and address your questions below.\n\n**Q1:** Guarantee of GT quality/reliability?\n\n**A1:** We use public datasets which are considered reasonably good, though are not guaranteed to be flawless. Our method can aid in refining the datasets: structure-wise uncertainty can reveal potential annotation errors.\n\n**Q2:** Can salient structures guide search for faint structures like Canny edge detection (CED)?\n\n**A2:** Our work is primarily on uncertainty estimation, tested with human-involved proofreading. An alternative is a human-free automated system that utilizes uncertainty to accept/reject structures. Your analogy of CED suggests high-confidence (low uncertainty) structures could support inclusion of nearby faint (low confidence) ones. This is intriguing and merits further exploration.\n\n**Q3:** Details/ref of Persistence Value L239.\n\n**A3:** We apologize, the missing reference is an oversight on our part. Persistence value (from persistent homology [4]) is defined as the difference of function (likelihood) values of 2 critical cells (saddle-maxima pair). It captures the importance of a structure, thus making it a valuable feature in our framework.\n\n**Q4:** Rationale of $Q(c’)$? Is the first term (inverse distance) comparable to the second term (likelihood)? Is this optimal? Alternatives? Is the first term always in [0,1]? Overall it’s a heuristic which may not always hold.\n\n**A4:** We provide the rationale for $Q(c’)$, especially the inverse-distance term $Q_d$, in L196, L215. As we perform the walk algorithm on a perturbed input, it’s possible the path would go astray and not reach the destination. $Q_d$ acts as a regularizer to likelihood $f_n$, guiding the path from source $c_s$ to destination $c_m$, ensuring path completeness.\n\n$Q_d$ will always be in [0,1] as it’s the inverse of Euclidean distance. Given 2 different pixel locations, the minimum Euclidean distance will be at least 1, and so $Q_d$ will be at most 1. The range of $f_n, Q_d$ are [0,1], so they are comparable. Their combination is weighted by $\\gamma$ for the final $Q(c’)$ metric.\n\nFig.13 (supple.) shows ablation study on $\\gamma$ emphasizing $Q_d$’s importance. When $\\gamma = 0, Q_d$ is not used, leading to decrease in performance. Notice $\\gamma > 0$ results in sharp improvement, empirically showing $Q_d$ is essential in the next point selection criteria $Q(c’)$.\n\nYou are correct that $Q(c’)$ is heuristic, but we find it works well in practice. That said, our future goal is to explore a theoretically guaranteed algorithm which can estimate a distribution of Morse complexes from noisy observations.\n\n**Q5: a)** L222: How many skeletons are sampled per structure? Does this equal the no. of runs? Do runs and their combinations influence results? **b)** L140: Does no. of training iterations depend on no. of sampled skeletons? **c)** L227: Can structures be combined to run the model once for direct uncertainty estimation?\n\n**A5: a)** In L294, we mention we take 5 runs, i.e., we sample 5 Morse skeletons per structure. L261 ‘Inference procedure’ outlines how T runs of the framework generate uncertainty estimates. While results vary with run count and resulting combinations, we find that T > 5 did not result in statistically significant improvement. Note that T runs are typical for uncertainty-estimation techniques, including the probabilistic methods we compare against.\n\n**b)** No. of epochs during training is independent of T. Every epoch, we sample 1 skeleton per structure.\n\n**c)** During inference, we conduct T runs, sampling a different skeleton in each run. As you mentioned, the alternative is to pre-generate multiple skeletons at once, and feed them together to obtain uncertainty in one pass. However, we want a flexible framework. If N skeletons are pre-generated, the network will rigidly require N structures, limiting flexibility for users with specific time/memory needs. Our approach lets users tweak T without retraining, unlike the alternative where altering N mandates retraining.\n\n**Q6:** Sec 3.3 shortest distance is a heuristic which may not always hold. Alternatives?\n\n**A6:** In L269, we emphasize that the shortest distance is applied solely to foreground pixels, a stronger constraint than using the shortest distance by itself. This ensures that structures separated by background do not mistakenly assign uncertainties to one another.\n\n **Q7:** How can prior knowledge guide segmentation, uncertainty and re-calibration, rather than just based on heuristics in skeleton-recalculation and inference?\n\n**A7:**  We are unclear about what you mean by ‘using prior knowledge’. We assume you mean how annotators’ knowledge can be integrated via active learning. Presently, we focus on generating structure-wise uncertainty estimates that can highlight uncertain structures and solicit further input from annotators. How to better learn from user input is a good yet different question that deserves future study. We hope this answers your query, and would be happy to address more comments during the discussion period.\n\nAs for heuristics, we believe you are referring to $Q(c’)$. We explain it in Q4/A4, and would like to reiterate that despite a heuristic, we obtain good results in our experimentation.\n\n**Q8:** L41: accept/reject structural proposals: based on prior knowledge or experience?\n\n**A8:** This is subjective: in medical contexts, clinicians rely on expertise, while in non-medical contexts like ROADS [1], an average user can apply their judgment. Thus decisions depend on various factors like context, task, participants, etc.\n\n**Q9:** L132: What is a structure?\n\n**A9:** In L160, we describe structures as Morse structures: V-paths connecting saddle-maxima pairs. A structure is thus a piece of a larger curvilinear structure (Fig.4, 5 visualize some structures that we encounter).\n\nThank you so much for your review! We would be happy to discuss further!"}, {"review_id": "UsUSeAuert", "reviewer": "Reviewer_7bwv", "summary": "This paper proposed a topology-aware uncertainty estimation method to segment curvilinear objects. The main contribution focuses on the application of discrete Morse theory (DMT). On several public datasets, the proposed method achieves SOTA performance and the visual results demonstrate the connectivity of vessels or other objects can be enhanced.", "questions": "See the above weaknesses.", "limitations": "See the above weaknesses.", "rating": 5, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Clear and well-organized paper.\n2. The object connectivity is improved by the proposed sound framework.", "weaknesses": "1. This paper is mainly based on [24]. The technical contribution is a bit marginal here. Please clearly state the main differences and the take-home insights.\n2. Some non-deep methods are also good at achieving better topologies. See:\n[1] Liu, Siqi, et al. \"Rivulet: 3D neuron morphology tracing with iterative back-tracking.\" Neuroinformatics 14 (2016): 387-401.\nPlease include them for discussion and comparison.\n3. The title is a bit over-claimed. \"Topology-Aware Uncertainty for Curvilinear Object Segmentation\" might be more suitable.\n4. Using graph models to capture the topology information is not new. See:\n[2] Shin, Seung Yeon, et al. \"Deep vessel segmentation by learning graphical connectivity.\" Medical image analysis 58 (2019): 101556.\nMore discussions and comparison should be added.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback! We will revise our manuscript accordingly. Please find our responses to specific queries below.\n\n**Q1:** This paper is mainly based on [24]. The technical contribution is a bit marginal here. Please clearly state the main differences and the take-home insights.\n\n**A1:** While we tackle the same problem as [24] (i.e., structure-wise uncertainty estimation), our method is significantly different from theirs. We described the key differences in L87 of the main paper, and we elaborate below. We refer to the method in [24] as Hu et al.\n- Hu et al. uses classic DMT to deterministically generate skeletons, thus failing to model intra-structural uncertainty. As we show in Fig. 2a) and Fig. 5 of the main paper, DMT structures often differ from the true GT structure. This is a common problem because of the tortuous nature of the structures. The uncertainty formulation in Hu et. al. mainly relies on the persistence of a structure, thus failing to capture the uncertainty with respect to the structure composition itself. In the ablation study in Table 3, we show how our proposed Prob. DMT results in improvement over DMT. Also as stated in L219, DMT is just one specific instance of our Prob. DMT.\n- We propose a joint inference model to jointly predict uncertainties of all the structures. This joint inference framework avoids explicit enumeration/sampling over the exponential size space of hypotheses. This is in contrast to Hu et al.’s method whose main aim was to generate multiple segmentation hypotheses, thus suffering from the enumeration problem. To reduce this computational burden, they used a global persistence value to prune structures in each run. This pruning was coarse, and such a global thresholding/pruning is harsh in practice, leading to suboptimal uncertainty estimation.\n- Our approach incorporates inter-structural uncertainty using GNNs, recognizing that structures in image space interact with each other and are not isolated. During uncertainty estimation, it is therefore crucial to consider their spatial context, i.e., inter-structural uncertainty. In the Table 3 ablation study, we show how incorporating GNN for inter-structural uncertainty results in improvement.\n- Finally, from Fig. 1, 8, 14 and Tables 1, 6, it is evident that Hu et al.’s method tends to produce over-confident uncertainty estimates --- they assign zero uncertainty (100% confidence) to most structures. On the other hand, our method, accounting for both intra- and inter-structural uncertainties, produces higher fidelity uncertainty estimates.\n\nWe believe the above points strongly differentiate our work from [24], both in terms of methodology as well as performance.\n\n**Q2:** Some non-deep methods are also good at achieving better topologies. See: [1] Liu, Siqi, et al. \"Rivulet: 3D neuron morphology tracing with iterative back-tracking.\" Neuroinformatics 14 (2016): 387-401. Please include them for discussion and comparison.\n\n**A2:** Thank you for providing this citation. While Rivulet aims to enhance segmentation quality, our primary goal is structural-level uncertainty estimation, with segmentation improvement coming naturally. Hence directly comparing the two does not seem straight-forward. Furthermore, while Rivulet also generates centerline skeletons similar to DMT, DMT has the important property that it decomposes the likelihood into a set of constituent structures (each structure is a path between a saddle-maxima pair). This decomposition is crucial as we ultimately estimate the uncertainty for each of these structures. Rivulet does not provide any such decomposition and hence cannot be used as a substitute in our framework. \n\n**Q3:** The title is a bit over-claimed. \"Topology-Aware Uncertainty for Curvilinear Object Segmentation\" might be more suitable.\n\n**A3:** We understand the sentiment and will update the title to \"Topology-Aware Uncertainty for Curvilinear Structure Segmentation\" in the revised version.\n\n**Q4:** Using graph models to capture the topology information is not new. See: [2] Shin, Seung Yeon, et al. \"Deep vessel segmentation by learning graphical connectivity.\" Medical image analysis 58 (2019): 101556. More discussions and comparison should be added.\n\n**A4:** Thank you for providing this citation. Indeed, the referenced paper uses graph models for vessel segmentation, however, our method utilizes GNNs in a different way. In the referenced paper, the vertices of the graph are pixels sampled from vessel centerlines. In contrast, our method generates a structure-level graph, treating entire structures (collections of pixels) as vertices. Thus we directly model the structures in the graph, instead of a sampled subset of points. Moreover, our graph model generates uncertainty estimates, while the referenced paper focuses on classifying pixels as vessels. \n\nThank you so much for your review! We would be happy to discuss further!"}, {"review_id": "9XmFb4FM0y", "reviewer": "Reviewer_ns8Y", "summary": "This paper proposes a framework that utilizes a probabilistic approach to extract structure-wise uncertainty estimates. This is achieved by extending the DMT to a probabilistic setting that models each structure as a sample from a probability distribution, thus capturing the intra-structural uncertainty. The proposed method then incorporates inter-structural uncertainty through a regression network which jointly reasons over the structures, using a Graph Neural Network (GNN). Additionally, a specialized inference procedure and post-processing steps are used to generate a structure-wise uncertainty heatmap, which can improve segmentation and quantify uncertainty more effectively.", "questions": "Please add larger dataset and evaluate the proposed method. Also, explain more about the hyperparameters.", "limitations": "Please see the 'questions' part.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The authors demonstrate the versatility of their method by applying it to multiple segmentation network backbones and datasets. The proposed method was found to improve the quality of segmentation and produce high fidelity uncertainty maps for each network, making it backbone-agnostic.\n- The introduction of a probabilistic DMT and a GNN to reason about inter-structural uncertainty is a significant advancement in this field.\n- The proposed method for quantifying structure-wise uncertainty in segmentation networks has significant implications for medical image analysis. The ability to better understand and represent the uncertainty can potentially lead to improved segmentation quality, and could be crucial in medical applications where decision-making is based on these segmentation outputs.", "weaknesses": "- Small dataset size: DRIVE dataset 40 images. ROSE 39 images. PARSE 100 volumes. Larger dataset should be considered to evaluate the proposed method.\n- Limited Explanation of Hyperparameter: while the authors have considered a range of hyperparameters in main body and supplementary, the paper could benefit from a more detailed discussion of the influence of these hyperparameters on the method's performance and how the optimal values were determined.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback! Please find our responses to specific queries below.\n\n**Q1:** Please add a larger dataset to evaluate the proposed method.\n\n**A1:** As recommended, we conduct these experiments. Please see 1. in the global response ‘Author Rebuttal by Authors’ above.\n\n**Q2:** Limited Explanation of Hyperparameter: while the authors have considered a range of hyperparameters in main body and supplementary, the paper could benefit from a more detailed discussion of the influence of these hyperparameters on the method's performance and how the optimal values were determined.\n\n**A2:** This is a good question and we provide a detailed discussion as requested. \n\nThe main hyperparameters in this work are $u, \\gamma, \\alpha, \\beta$. We describe the importance of each below:\n- $u$ : This is the parameter for the Bernoulli distribution, and we introduce it in L221 of the main paper. In our Prob. DMT module, for every structure, we have a choice to either retain the structure as obtained from DMT, or, generate a sample skeleton using the perturb-and-walk algorithm. We model this choice using the Bernoulli distribution. Essentially, in some runs we would like the original DMT structures to also interact with the others. Thus a low value of $u$ works best. We found $u = 0.3$ to give the best performance, that is, for every structure there is a 30% chance that it’s DMT form is used and a 70% chance that a sample variant is used. We find that $0.15 \\leq u \\leq 0.3$ have comparable performance.\n- $\\gamma$ : This hyperparameter is used in the weighted combination of distance $Q_d$ and likelihood $f_n$ to obtain $Q(c’)$, which is used to determine the next pixel location. We introduce $\\gamma$ in L210 of the main paper. It maintains a tradeoff between the distance regularizer $Q_d$ and the perturbed likelihood $f_n$. The higher the value of $\\gamma$, the greater is the distance regularizer, and consequently the generated path will become closer to that of a straight line. This is not desirable, as a straight line would lose the original composition of the structure. Additionally, because of the perturbation in the likelihood, we do not want the path to go astray. And so, to ensure path completeness, we require $\\gamma$ to be non-zero. Through experiments, we obtain the best performance when $\\gamma = 0.2$. We provide ablation study results of different $\\gamma$ values in Section 12 of the supplementary.\n- $\\alpha, \\beta$ : These are prior hyperparameters of the Inverse Gamma (IG) distribution which we introduce in L204 of the main paper. We perturb the likelihood using a Gaussian model. As the variance of the Gaussian model is unknown, we use Bayesian probability theory to sample the variance from the IG distribution (its conjugate prior). And so, $\\alpha$ is the shape parameter and $\\beta$ is the scale parameter of this IG distribution. Ideally we would like a small perturbation of the likelihood and not a strong one. This is because a strong perturbation would corrupt wholly and we would not be able to sample a reasonable skeleton. At the same time, the perturbation should not be too small, otherwise we will not obtain a significant variant. The mean of the IG distribution is $\\frac{\\beta}{\\alpha - 1}$ (when $\\alpha > 1, \\beta > 0$), which on average is the value of the sampled variance for the Gaussian distribution. We achieve the best performance when $\\alpha = 2.0$ and $\\beta = 0.01$. The resulting sampled variance for the Gaussian model thus generates reasonable perturbation. We provide ablation study results of different $\\alpha, \\beta$ values in Section 12 of the supplementary. At the extreme ends of the graph plot in Fig. 13 of the supplementary, the sampled variance is either too low or too high, resulting in a decrease in performance.\n\nWe hope the above discussion is helpful. We will definitely add this to the revised version of the paper. \n\nThank you very much for your review! We hope we were able to clarify your comments, and we would be happy to discuss further!"}]

Author rebuttal:
We thank the reviewers for their time and insightful feedback. We are encouraged that all the reviewers appreciated the novelty of the contribution, and found our work to be methodologically sound and effective. 

We have uploaded a 1-page PDF where we add results of additional experiments as requested by the reviewers:

**1. Validate the proposed method on a large non-medical dataset (oww6, ns8Y)**

We conduct additional experiments on ROADS [1] --- a large, non-medical dataset containing 1171 aerial images (1108/14/49 train/val/test), each of 1500 x 1500 resolution. It is a challenging dataset due to obstruction from nearby trees, shadows, varying texture/color of roads, road class imbalance etc.

The quantitative and qualitative results are provided in Table 6 and Fig. 14 of the rebuttal PDF respectively. Table 6 shows that our method outperforms the other probabilistic methods on both ECE and segmentation metrics. Fig. 14 shows that our method generates better fidelity structure-wise uncertainty maps compared to Hu et al. Our heatmaps assign non-zero uncertainty to several false positives/negatives in the backbone UNet’s outputs. This is because we reason about every structure while Hu et al. limits the structure space via pruning.

**2. Evaluate on topology-related metrics: Betti Number error [2] and Betti Matching error [3] (qp6i)** 

We provide results on these metrics in Table 7 of the rebuttal PDF. Our method consistently improves the segmentation result in terms of topology. This is consistent with our results in Table 1 of the main paper where our method outperforms the other methods on topology-based metrics like clDice, ARI and VOI. Note that for the 3D PARSE dataset, we were unable to provide Betti Matching error results as its official implementation handles only 2D inputs.

**3. Conduct ablation study on other parameters like dimensionality of input feature vector and crops/bounding boxes (oww6)**

We now include ablation studies on the dimensionality of the input feature vector, and size of the crops/bounding boxes, and report this in Table 8 of the rebuttal PDF. We obtain the best results when the input feature vector size is 32 and the bounding box is 32 x 32. For lower values (16 and 16 x 16), the performance reduces, while for higher values (64 and 64 x 64) we did not observe any statistically significant improvement. Thus to maintain the tradeoff between complexity and performance, we respectively use 32 and 32 x 32 for these hyperparameters. 

**4. Please report computational complexity and time (oww6)**

We report the inference time for 5 runs on a 256 x 256 input image patch as follows: Prob.-UNet: 0.196 sec; PHiSeg: 1.811 sec; Hu et al.: 5.485 sec; Ours: 7.433 sec.

The module which takes the most time is the DMT / Prob.DMT computation. Presently, this is the most optimized version as we have implemented it as an external module in C++. We will work towards porting the code to run on GPU to bring down the runtime even more.

Following [7], the computational complexity of DMT is $O(n \log n)$, where $n$ is the number of pixels in the image. Since Prob. DMT additionally computes structure variants, the complexity is $O(n \log n + m)$ where $m$ is approximately the number of foreground pixels, and typically $m << n$ for curvilinear structure datasets. The linear term $m$ is added as we traverse each foreground pixel only once when generating the sample skeleton.

**5. While the proposed method is effective in improving the results on ECE, clDice, ARI and VOI, but not always on Dice. Please provide insights and explanations (oww6)**

This observation is accurate for datasets having curvilinear structures. This is because improvements in segmentation are obtained by recovering broken connections and false negative structures. As curvilinear structures are inherently thin (only a few pixels wide), the recovered connections and false negatives are also quite thin and hence do not affect the Dice score greatly. That being said, we would like to emphasize that we do achieve statistically significant improvement even in Dice for all the 2D datasets (DRIVE, ROSE, ROADS). In the case of the 3D PARSE dataset, we obtain numerically better results for Dice although it is not statistically significant.

**6. L237: What are the sizes of smaller crops/bounding boxes? (oww6)**

We provided these details in L77 of the supplementary. The size of the bounding box was 32 x 32 for 2D datasets, and 32 x 32 x 32 for the 3D dataset. In the Table 8 ablation study, we found this value to have a good tradeoff between computational complexity and performance.

**7. Refs for L170 perturb-and-walk algo? (oww6)**

We give our proposed algorithm the custom name of “perturb-and-walk”, and hence there is no reference. Nonetheless, its design is inspired by the random walk algorithm, for which we cite references in L206.

We further reply individually to each reviewer to address their specific questions. 

**References used throughout the rebuttal:**

[1] Volodymyr Mnih. Machine learning for aerial image labeling. University of Toronto (Canada), 2013

[2] Hu, Xiaoling, et al. "Topology-preserving deep image segmentation." NeurIPS, 2019

[3] Stucki, Nico, et al. "Topologically faithful image segmentation via induced matching of persistence barcodes." ICML, 2023

[4] Edelsbrunner, Letscher, and Zomorodian. "Topological persistence and simplification." Discrete & Computational Geometry 28 (2002)

[5] Robins, Vanessa, Peter John Wood, and Adrian P. Sheppard. "Theory and algorithms for constructing discrete Morse complexes from grayscale digital images." TPAMI (2011)

[6] Mischaikow, Konstantin, and Vidit Nanda. "Morse theory for filtrations and efficient computation of persistent homology." Discrete & Computational Geometry 50 (2013): 330-353

[7] Dey, Tamal K., Jiayuan Wang, and Yusu Wang. "Graph reconstruction by discrete Morse theory." arXiv preprint arXiv:1803.05093 (2018)

Meta-review: This paper proposes a method for estimating segmentation uncertainty for segmentation of topological structures in images, to be used to "proof-read" the segmentations in a postprocessing step. This is important, as topological structures can easily be broken by a few incorrectly segmented pixels. The paper was appreciated by the reviewers already before rebuttal, and the authors' rebuttal further responded to reviewer concerns.

In their final version, the authors should consider the reviewer comments carefully to improve the quality of their paper. The authors should also follow up on all promised improvements. The authors should also make sure to discuss the limitations of their work in the main paper, not just the supplements.
Decision: Accept (poster)