Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this article, the authors propose a new topological loss for training neural networks on images representing curvilinear structures. In addition to the usual MSE or cross-entropy losses, they suggest to add a regularization term based on persistent homology, in order to preserve the topological properties in the images predicted by the network. However, the common filtrations proposed in the literature (pixel values, distance maps) for doing so have their own flaws and are not always suited for such images, so the authors propose a new filtration that combines those common filtration with filtrations based on heights, i.e., with the persistent homology transform. Using this new filtration, they then provide a few experiments on which their loss compares favorably to other topological and non-topological losses (used on the same fixed neural net architecture).
main_review: The proposed filtration is interesting, and the article is overall well written, but I think that the approach seems a bit incremental for now. There is no real theoretical back-up and the results, while being OKish, do not empirically justify by themselves the approach (to my opinion). Moreover, some parts of the writing could be improved.

In particular, I have the following comments:

1. Section 3.1: "the cost of leaving a homology unmatched set to a constant hyper-parameter": this is not correct, the cost of leaving a point in a persistence diagram unmatched is usually its distance to the diagonal, otherwise the distance is not stable.

2. Equation (2): are the values Y[p] and g(p) always comparable, i.e., on the same scale? If one is much larger than the other, it does not make a lot of sense to do a simple sum between the two. It would be more natural to use a linear combination of the two terms, with the combination coefficients chosen a priori or with cross-validation.

3. Given that the authors aim at using two filtrations at the same time, it feels that persistence bimodules would also be an approach to discuss and compare to, especially since multiparameter persistence can now be computed efficiently, at least when there are no more than two filtrations.

4. Figure 4 (c) and (d) do not have names on their axes.

5. The wording chosen in the article is not great: localization has another specific meaning in Topological Data Analysis, it usually refers to the computation of representative cycles corresponding to homology classes. Hence, the title of the article is confusing and misleading for TDA practitioners.

6. Since the persistent homology transform is known to enjoy some nice injectivity and inverse properties (https://arxiv.org/pdf/1810.10813.pdf), I wonder if such results could also be derived for the proposed loss.

7. Section 4.2: "alpha=0": typo

[Post rebuttal comment]: I updated my review after the author's response.
summary_of_the_review: The proposed approach is definitely promising, but I think that the work is too incremental for now to be accepted as is.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a new approach for detecting topological structures
in 2D and 3D imaging data. Based on the computational topology framework
persistent homology, the paper develops a type of loss term that can be
used to assess to what extent topology is being preserved when solving
segmentation or structural extraction tasks. A suite of experiments is
used to assess the quality of the extracted structures.
main_review: While I find the topic of this paper highly fascinating, at present,
this paper is not yet ready for publication. There are three main issues
I have with the current write-up:

1. The work is marred by severe deficiencies in clarity/correctness.
   I will provide more examples later but the text is using several
   terms incorrectly and does not explain underlying concepts correctly.

2. Missing delineation to existing work: while the paper discusses other
   work that aims to combine image tasks with topological prior
   information, it does not provide a sufficiently detailed delineation
   to such work. This can also be seen in the related work section,
   which misses the paper [*A Topological Loss Function for Deep-Learning
   based Image Segmentation using Persistent
   Homology*](https://arxiv.org/abs/1910.01877) by Clough et al.
   (linking to the preprint here; the full paper appears in IEEE TPAMI).
   While another paper of Clough et al. is discussed, the aforementioned
   work should also be at briefly discussed and, ideally, be used as
   a comparison partner.

3. The experimental setup is preliminary and missing important details:
   means and standard deviations are only provided for the proposed
   method, whereas the numbers for other methods are presumably cited
   from an additional publication. This does not enable a fair
   comparison, given the fact that it is unclear whether the experiments
   described in other papers followed the same training regimen.
   Providing standard deviations is a critical facet that needs to be
   changed prior to publication.

   In addition, I find the selection of comparison metrics somewhat
   questionable; in other publications, metrics such as accuracy, DICE
   scores, ARI, etc. are being employed, whereas this paper reports
   metrics that appear to be relatively non-standard to me. That being
   said, I acknowledge that I am not familiar with all segmentation
   literature; I am, however, very familiar with the segmentation
   literature based on topological concepts, and these papers
   consistently report different scores. I would therefore suggest, also
   in the interest of making the paper more accessible, to include
   additional scores and expand on existing ones (for instance, it would
   be helpful to know what a 'good' score looks like, and whether larger
   numbers are better or worse. All of this information can be gleaned
   from the text at present but it could be more prominent).

   Finally, I am confused by certain reported numbers and algorithms in
   the experimental section:

   - It seems that `ConnLoss` is performing better in all metrics.
     I understand that it does not extend to 3D data, but if this is to
     be the deciding property of the proposed algorithm, additional
     experiments on 3D data sets are required. Performance on the
     proprietary data set is hard to assess because on comparison
     partner papers exist. While it is perfectly acceptable to use such
     data sets, in the interest of a fair comparison, I would suggest to
     also employ additional 3D data sets such as `ISBI13`, `CREMI`, or
     `3Dircadb`, all of which are discussed in [the paper by Hu et
     al.](https://arxiv.org/pdf/2103.09992.pdf).

   - The 'Massachusetts' data set corresponds to the `ROAD` data set from
    the aforementioned paper by Hu et al. However, Hu et al. report the
    same Betti error as the one shown in this paper, albeit with
    a standard deviation of `0.301`, which is comparable to the one
    obtained by the proposed method. If the number are indeed cited from
    this paper, additional information about variance needs to be shown
     *and* the citation should be acknowledged in the caption of the
    table, for instance, or in the experimental setup section. It needs
    to be clear to readers which experiments where run by the authors
    and which were cited from another paper. 

   - Why is the set of comparison partners changing for the different
    experiments? The DMT method is applicable to 2D and 3D and [code
    appears to be available](https://github.com/HuXiaoling/DMT_loss),
    so it should be part of this comparison, in particular since the
    paper claims that the proposed algorithm outperforms *all* PH-based
    loss functions. (the same comments apply to the papers by Clough et
    al.; for a fair comparison with TDA-based methods, they need to be
    included).

## Detailed comments

- The title is slightly misleading: 'localisation' typically refers to
  the process of finding geometrically useful representations of group
  generators. Here, the filtration incorporates some geometrical
  information but it does not make use of group generators. Using
  'localised' here might be misconstrued at first glance. The title is
  also misleading on a second level because the proposed loss does *not*
  deal with deep learning in general, but with *image segmentation*
  based on deep learning. The loss also does not improve the
  effectiveness of deep learning methods in general. I urge the authors
  to reconsider the title of this paper.

- When discussing persistent homology, consider citing a survey or
  a text book, such as the one by Edelsbrunner and Harer. The work by
  Aktas et al. did *not* describe persistent homology for the first
  time.

- The abstract of the paper is too terse and needs to be slightly
  expanded in order for it to be accessible by non-experts in the field.

- Homology is typically used only in singular form or as a qualifier, as
  in 'homology groups' when discussing TDA literature. The framework
  itself is dubbed 'persistent homology' and a plural use is relatively
  nonstandard.

- The term 'homologies' is incorrectly used throughout the paper; use
  the term 'simplicial chains' or 'topological features' instead. Or,
  for the most precise moniker, you can use 'elements of a homology
  group.'

- Please check carefully when to use 'persistence' and when to use
  'persistent.' The diagram is the 'persistence diagram' because it
  contains information about the 'persistence' of features.

- When claiming that '[...] our technique allows for considerable
  misalignment [...],' provide a reference to the respective section in
  the text.

- The description of persistent homology in Section 3.1 needs to be
  rewritten; use the term 'homology class' if need be instead of
  'homologies.' A more technical description might be helpful here; I am
  not sure whether non-expert readers are able to understand the gist of
  the technique from the current write-up.

- In Equation 2, consider adding a second set of parentheses; the idea
  is that the sum of the two terms is $< s$, correct?

- Figure 3 could be updated to make the difference between the proposed
  filtration function and the 'default' one clearer.

- When discussing experiments, please provide details on how the
  hyperparameters were chosen.

- The *Betti error* needs more details; the present description is
  insufficient.

## Minor comments

- Citations are used inconsistently throughout the paper. Assuming that
  you are using the `natbib` package, use `\citep` for a parenthetical
  citation and `\citet` for a textual citation. Examples of places that
  need such a fix can be found in the paragraph 'Losses that Rely on
  Persistent Homologies' (which I would rename to the singular form, as
  discussed above).


summary_of_the_review: This paper deals with an interesting topic, viz. improving 2D and 3D
image tasks by incorporating more prior knowledge about topological
structures. While I consider this to be a highly relevant topic,
I cannot endorse this paper for publication at present, mainly because
of a lack of clarity, a lack of delineation to existing work, and issues
in the experimental setup.

All of these will have to be rectified before I will be able to endorse
the paper for publication; I feel particularly strongly about the
experimental setup, which currently does not enable a fair comparison of
the methods.

I realise that this is not the preferred outcome for the authors; at the
same time, I am confident that with additional rewriting and some
additional work on experiments, this paper can make a strong addition to
the ever-growing body of topology-based machine learning techniques.

**Updated after rebuttal**: I thank the authors for their efforts; while some concerns
where alleviated, I think the updates also showed that the current method does not
substantially improve upon the state of the art in the 2D case, whereas for the 3D case,
additional comparison partners would be required. This is not the only way to improve
the paper, though: another potential direction for future improvement could be to invest
in a more theoretical analysis of the proposed method. If either one of these directions
would be followed, it would serve to substantially improve the quality.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The proposes a localized version of topological loss. Current methods perform matching in the space (i.e. persistence diagrams) that does not account for the location of topological features. The method proposes to add to the likelihood maps some sort of positional embedding, i.e. a function of image pixel coordinates chosen at random during training. The experiments demonstrate improved performance.
main_review: The paper addresses an important problem of training network producing segmentation with desired topology. Traditionally, segmentation is done via pixel-wise losses, which are not able to properly capture topology, resulting in significant topological mistakes such as lack of connectivity or erroneous connections. Designing topologically-motivated losses, like in this paper, is an important problem for detecting objects like roads, vasculature, etc.

The paper builds upon an established framework of persistent homologies (PH) and extends it by adding positional embedding.

The results look convincing and the proposed modification seems simple and effective.

The topological loss based on PH is quite complex in the implementation. The paper would benefit from a more detailed review of the implementation including differentiation.

The author should discuss the relation and compare to other PH-based localization techniques, for example Hu et al (2019). In particular Hu et al used patch-based topology, which provides a degree of localization. How does this compare to the proposed method? DO the authors explicitly compare to the patch based approach?


summary_of_the_review: The paper is addresses an important problem of topology-aware segmentation learning. The proposed framework extends previously existing PH diagram techniques by adding positional emebedding, this add locality to the framework. The experimental evaluation shows improvement upon global methods. Some lack of comparison with other localized methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Relying on recent progress regarding the (automatic) differentiation of « Persistent Homology » (a tool of Topological Data Analysis (TDA) to extract topological descriptors on top of complex data), this paper proposes a new filtration (essentially, a sum of the distance filtration and the height filtration) to favor proper topological reconstruction in the context of image processing (segmentation in particular). 
The authors experimentally showcase their approach on different (2D and 3D) datasets and compare it with various competitors (from and outside TDA-based techniques) and obtain very competitive results.
main_review: # Strengths and Weaknesses

## Strengths
- It is nice to see a well-motivated application of TDA in the context of image processing.
- Overall, the flow of the work is clear and the paper is pleasant to read. 
- The experimental results reported seem solid, extensive, and may be a good contribution to the field of image segmentation in particular. 

## Weaknesses
- The paper contains some caveats / inaccuracies related to TDA (see the Minor Comments section for details). In particular, the desire to filter an object with respect to two (or more) parameters $Y, g$ (here, distance + height) is present for a long-time in the TDA community, known as *multi-persistence* (which is way harder than single-parameter persistence and, roughly, untractable in practice for now). Simplifying this by instead looking at one (or many!) linear combination of the two maps $Y, g$ is already used in TDA, see for instance the work *Multiparameter Persistence Images for Topological Machine Learning* by Carrière and Blumberg (2020); if my understanding is correct, filtering with respect to $Y + g$ is very similar (if not the same) to picking one direction in the so-called *fibered barcode*. Even if this does not take away all the potential impact of the paper (and it is quite nice to think about **this** choice of $Y,g$ and that it seemingly works well in applications), I think this must be discussed and the position of the paper with respect to the TDA literature should be revised a bit. 

# Minor comments
- "Persistent homologies" (abstract), I think it is not standard to use plural here. "Persistent homology" would fit better in my opinion.
- (Ref) When introducing Persistent Homology (p1, third paragraph), I think it may be appropriate to cite (in addition) a founding paper of TDA or a reference book, such as the one(s) of G.Carlsonn & Zomorodian (e.g. *Computing Persistent Homology* (2005)), Edelsbrunner and Harer (2008), Chazal and Michel (2017), etc.
- (Ref) About the possibility to  differentiate the map $X \mapsto \mathrm{Dgm}(X)$ (should $X$ be an image or something else), I think it may be worth citing the work *A topology layer for machine learning* by Bruel-Gabrielsson et al. (possibly along with other work, like *A framework for differential calculus on persistence barcodes* by Leygonie et al., and *Optimizing persistent homology based functions* by Carrière et al.).
- (Typo) Caption of Figure 1, Something is wrong, a description of *(d)* is missing. 
- (Ref) Saying that "Application of computer vision [of persistent homology] were scarce" may be slightly exaggerated, see for instance the short survey *Topological Data Analysis in Computer Vision* by Bernstein et al. and the references therein. 
- (Typo) "closed 3D surfaces", I guess "surfaces embedded in 3D" may be more accurate. 
- (Ref) Citing Giotto (Tauzin et al.) for the "matrix reduction algorithm" to compute PH is not very accurate as well. This algorithm was introduced much earlier (see references at the beginning of https://en.wikipedia.org/wiki/Topological_data_analysis#Computation ). Of course, it is worth mentioning the numerical computations were done using Giotto though. 
- (Important) p4, "The cost of leaving a homology unmatched" is **not** constant (at least in standard definition of the distances between PDs). It is the distance between the point and its orthogonal projection on the diagonal, as drawn for instance on Figure 3. If you used a constant cost instead, this must be discussed and motivated. 
- Calling "a homology $g \in H_{X_1}$..." a point in the persistence diagram is not accurate. Calling them "A persistence interval" as done few lines above is much more standard in my opinion.
- Figure 4 and Section 3.4 are not fully convincing : though *(c)* is indeed non-monotonic, the trend seems similar (and perhaps more trials than just 5 would have smooth the curve. Plotting the standard deviation would be useful as well). I would not say that "very noisy images can be seen as being very similar to error-free ones." 
summary_of_the_review: The paper is well-written and showcases an interesting application of TDA in the context of image processing supported by various numerical experiments. However, it contains some inaccuracies regarding TDA concepts/literature that must be corrected/discussed.
review_title: null

Meta-review: The paper proposes a method for segmentation of thin structures in 2D and 3D, based on persistent homology and using a new filtration. The method performs similarly to state-of-the-art methods on 2D datasets and outperforms some baselines in 3D.

After considering the authors' response and discussing, the reviewers have not arrived at a consensus. 

Pros include:
- Simple and reasonable approach
- Fairly strong experimental results

Some cons are:
- Missing theoretical contributions
- Experimental results on 2D datasets are not that strong, while on 3D datasets important baselines are missing
- At times unclear/unconventional presentation 

Overall, at this point I recommend rejection. The paper is promising, but since the main claim is good performance on 3D data, it is important to have a thorough empirical evaluation there, with the relevant baselines (as mentioned by the reviewers). I very much encourage the authors to polish the paper and submit to a different venue.
Decision: Reject