Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose a new loss and evaluation metric based on persistence theory for matching different image segmentations. The proposed loss can be quite different from the usual Wasserstein distance between persistence diagrams, and is rather based on the algebraic foundations of persistence theory, which turns out beneficial for practical purposes as the Wasserstein distances often induces matching errors. Finally the authors provide an exhaustive set of numerical experiments showcasing the efficiency of the new loss.
strength_and_weaknesses: Strengths:
---The proposed loss seems quite efficient from the numerical experiments
---It is rather original as topology-based segmentations are often based on standard distances between persistence diagrams

Weaknesses:
---I think the writing of the paper could be improved, as I find it quite difficult to follow for someone that is not aware of topological data analysis and its algebraic foundations.
---I think it would be interesting to compare against the simpler, more naive method of using the Wasserstein distances between L and min(L,G) and between min(L,G) and G. It seems that composing these two matchings would already provide better segmentations than the Wasserstein distance between L and G. Why are induced matchings better than this?
---Optimizing losses based on persistent homology usually requires heavy mathematics (http://proceedings.mlr.press/v139/carriere21a.html, https://link.springer.com/article/10.1007/s10208-021-09522-y), so I was wondering if the gradient of TopoMatch is well defined also, and if it enjoys some kind of convergence guarantees?
clarity,_quality,_novelty_and_reproducibility: Clarity could be improved with more toy examples. The proposed loss is quite original though.
summary_of_the_review: Overall, I think the paper is fine. The writing could be improved by adding some experiments from the supplementary inside the main body of the text and by simplifying a bit the theoretical section, but I think it is OK globally.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper provides a metric TopoMatch to evaluate the topological correctness of image segmentation masks. TopoMatch matching is based on induced matching and its advantage over Wasserstein matching is its consideration of spatial correspondence of topological features. Also, TopoMatch can be used as a loss to improve the segmentation results. This method is evaluated on six datasets. Although the segmentation accuracy is not always the best, its topological correctness performs best in all experiments.
strength_and_weaknesses: Strengths:

 +) This paper works on an interesting and important task, the topological correctness of segmented masks. 

 +) The proposed TopoMatch sounds reasonable according to its description and experimental results. 

Weakness:

 -) The writing of this paper could be greatly improved for a better understanding. For instance, more detailed explanations of figures are desired. In Fig.1, it says the same color means the matched cycles. How about white color? A further explanation of Fig.1 in the main text is missing, which takes time to get the information this figure conveys.  Also, the main content should be self-contained and understandable without the need of an appendix. However, in Section 3.3, the reader needs to read the appendix to understand #ker and #coker. 

-) More explanation of the physical meaning of TopoMatch is desired. What is p in the TopoMatch loss equation? 

-) The comparison image is defined as C = min(L, G) in the main text, while in the caption of Fig.7, it is defined as C = max(L, G). Why the TopoMatch matching is the composition of two induced matching? 

-) The contribution of this paper should be explicitly pointed out. It seems like the main contribution is the use of two-induced matching to define a new metric, which seems to be incremental. 

-) The experimental results show that the correction of the topology of segmentation hinders the improvement of segmentation accuracy. Also, once the dice score is over 0.9, the topology correctness, even the spatial correspondences, would not be a big issue, as demonstrated by the Betti number. Is it necessary to have such a complex term in the loss function? Or can we choose a good segmentation mode and just use the Betti number as a constraint? It seems like improving the segmentation accuracy is another good way to correct the topological issues of segmentation. So, in what situations, we do need the topology correctness term? 

-) Perhaps more descriptions on the meat of this paper while concisely introducing some background and making connections to the method proposed in this paper. 
clarity,_quality,_novelty_and_reproducibility: The clarity of this paper could be further improved, as mentioned above. The writing quality has room to improve, and the experimental results do not show impressive improvement or a good balance of segmentation accuracy and topology correctness. Also, the novelty is limited and the reproducibility of this work depends on the release of its source code. 
summary_of_the_review: Overall, the current shape of this paper makes it not easy to be understood. The experimental results do not convince me to choose it for my segmentation task.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper shows how induced matchings guarantee the spatially correct matching between barcodes in a segmentation setting. Furthermore, this paper proposes an efficient algorithm to compute TopoMatch for images. Also this paper shows that TopoMatch is an interpretable metric to evaluate the topological correctness of segmentations. Moreover, this paper demonstrates how induced matchings can be used to train segmentation networks and improve the topological correctness of the segmentations across all 6 baseline datasets while preserving volumetric segmentation performance.
strength_and_weaknesses: **Strengths**:

1. This paper achieves a spatially correct matching between the topological features (persistence barcodes) of label (ground truth) and prediction (output of the neural network).

2. This paper proposes an efficient algorithm to compute TopoMatch for images. This paper shows that TopoMatch is an interpretable metric to evaluate the topological correctness of segmentations. It is better than the Betti number as an evaluation metric.

3. Matching L & G directly has spatial matching issues; using induced matching via map C helps with spatially correct matching.

4. This paper uses both background and/or foreground class topology, depending on dataset.

**Weaknesses**:

1. Values in Table 1 are not correctly bold (worse performing is also bold).

2. The proposed method achieves worse pixel-wise performances compared with baselines.
clarity,_quality,_novelty_and_reproducibility: Clarity: This paper is well written and easy to follow.

Quality: This paper overcomes the issues of Hu et al.(2019) and achieves better topological results.

Novelty: This paper naturally extends Hu et al.(2019) and contributes to the community.

Reproducibility: I am optimistic to reproduce the main results of this paper as the paper provides lots of details.
summary_of_the_review: This paper is a non-trivial extension of Hu et al.(2019) and achieves better topological performances.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:
The paper suggests a new loss function and evaluation metric for matching image segmentations based on persistent homology. 

Strengths:
The loss is novel and interesting, and seems computationally advantageous.

Weaknesses:
- Clarity: The reviewers find the paper challenging to read.
- Experiments: Important baselines are missing.
- Suitability for ICLR: The reviewers are in doubt whether the work is right for the ICLR venue

The reviewers appreciate the novel approach, and with a bit of work to improve clarity and tailor the experiments to show why this approach is important for the ML crowd, this paper would be suitable for a top ML conference. But in its current state, the paper is unfortunately not ready.

justification_for_why_not_higher_score: While two reviewers recommend acceptance, their reviews are not overly positive, and the reviewers consistently point out the lack of clarity in particular.
Decision: Reject