Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: Overall, the unsupervised 3D segmentation task tackles one major pain point of the medical imaging field, where the professional annotation is hard and expensive to obtain. This work adopts 3D VAE to map 3D patches into a hyperbolic latent space, then apply clustering followed by the Hungarian algorithm to finish the segmentation task. Though with merits on the introduced hyperbolic space and the gyroplane conv, I have the following concerns and questions. 

Although I totally appreciate the exploration of the fully unsupervised method to do the 3D segmentation to tackle the lacking of annotation problem, my major concern is that a reliable medical imaging system cannot be obtained through this way and won't be clinically meaningful. Results in Table 1 and Table 3 shows that the 3D semi-supervised by training a vanilla 3D UNet only on 2% of the ground-truth data can perform much better than the proposed unsupervised method. I believe even this simple semi-supervised way is closer to the practical scenario, not to mention that a medical AI system needs to be as reliable as possible for doctors to use.

My questions:
1) What is the running time comparison between the between the Hyperbolic space and the Euclidean space?
2) Can the ablation studies done on the BraTS data? I would personally think this way can be more serious and convincing rather than on a synthesized toy dataset.
review_title: An interesting research topic in the medical field but limited clinical use in practice

final_score: rating: 7: Good paper, accept
review_content: The paper proposes to train a variational autoencoder with a hyperbolic hidden layer augmenting the ELBO loss with a hierarchical triplet loss between nested subsets of data suitable for spatial data structured hierarchically. This process results in representations that can be clustered and used for unsupervised segmentation.

# Strengths

Obtaining segmentation labels for 3D data is notoriously difficult and error-prone as even the experts are unable to stay focused for a very long time required to label voxels in a large MRI volume, as an example. Practical advances in unsupervised labeling are a welcome addition to application areas relying on 3D segmentation.

This paper is able to exploit the recently developed approach to hyperbolic convolutions in a variational autoencoder training with a simple addition of a hierarchical triplet loss. Notably, this simple idea leads to performance improvements over other existing unsupervised or self-supervised methods.

Demonstrations on synthetic data and a couple of real datasets, together with comparison to existing methods is a plus that helps to understand the value of the proposed approach.

# Weaknesses and comments

The limited technical novelty of the contribution, as most of the innovations are although recent but already published elsewhere.

The application component is not as rigorously evaluated as one would need to assess the real practical applicability of the approach. For example, in the case of the tumor, the size may bear a large meaning, and volume error in addition to the DICE coefficient would provide much-needed information. For electron microscopy, tomograms or even for the tumor to be treated with radiation therapy the Hausdorff distance may be an informative and potentially more appropriate metric of comparison. Basically, it is very difficult to assess experiments in any meaningful way besides - "this might be interesting".

Neither of the measurements provides error bars (all tables), so it is impossible to assess the uncertainty (variability of the approach) and for values that are not too far apart estimate if the prediction is even statistically different.

It would be good to discuss the training process, hyperparameter search, and computational complexity of the proposed method.

Could the encoder be trained directly using the triplet loss without invoking the VA at all?
review_title: An interesting result with a high practical relevance to neuroimaging

final_score: rating: 7: Good paper, accept
review_content: The paper considers learning hyperbolic representations for unsupervised 3D segmentation. Since the general task of producing annotations for 3D data can be expensive (e.g. for segmentation in dense voxel grids), this is an important problem. The paper proposes to learn hierarchical data structures (e.g. 3D biomedical images) with a hyperbolic variational autoencoder. The paper adapts different metric learning approaches, such as triplet loss and computing a Frechet mean on Riemannian manifolds for clustering. 

The paper is easy to read and interesting to the computer vision and biomedical imaging communities. In terms of machine learning theory, there is not much novelty since the paper combines different existing approaches into a single framework. However, the paper tackles a difficult problem which is the representation of hierarchical 3D data with a simple solution.

- I think that the self-supervised aspect of the approach in the hierarchical triplet loss paragraph is important. However, there does not seem to be much emphasis on it, and I did not fully understand how the positive and negative child patches are generated. Could you please clarify that part and maybe illustrate it in a figure?

- Since the approach is used in the unsupervised setup, how are hyperparameters tuned? There is a discussion about hyperparameter tuning in Section A.7 but the setup is not clear to me. Do you use annotations/labels for the validation set? What is the stopping criterion? How do you pick the best set of hyperparameters? This needs clarification.

- It is nice to see an ablation study of different components of the Euclidean vs hyperbolic approach in Table 2. However, it is known that hyperbolic representations perform better than Euclidean representations to representation trees in low-dimensional space. In high-dimensional space, Euclidean and hyperbolic representations generally perform similarly. 
If I understand correctly, the dimensionality of the wrapped normal distribution is 2 for the toy dataset. Could we have an ablation study of the impact of higher dimensions for both hyperbolic and Euclidean representations? This would be particularly interesting for real-world applications.
What is the dimensionality of the latent space in Section 4.2 and 4.3?

In conclusion, I think that the paper is interesting, but needs some clarifications.

Minor comment: representation dimensionality d and distance d use the same font.
review_title: Interesting paper but needs clarifications for experiments

final_score: rating: 3: Clear rejection
review_content: The authors of this manuscript propose an unsupervised learning framework for 3D segmentation of biomedical images. Specifically, the proposed method learns effective representations for 3D patches using variational autoencoder (VAE) with a hyperbolic latent space. Its main contribution lies at that it introduces a new unsupervised learning framework including hyperbolic convolutional VAE and hierarchical triplet loss. This work conducts experiments on toy dataset, the Brain Tumor Segmentation dataset, and cryo-EM data. The experiment demonstrates competitive performance of the proposed method.

Major Strengths of The Paper
1. The idea of hyperbolic 3D convolutional VAE is interesting.
2. The experiment shows that this idea performs favorably against the state-of-the-art.

Major Weaknesses of The paper 
1. The manuscript contains some grammatical errors, and the writing could also be improved. For example, there are some grammar mistakes like “our work is a first to introduce…”. The paper will be more clear if the authors could further polish the manuscript.
2. The proposed model is not novel enough. In my personal opinion, the proposed model generalizes the gyroplane layer and hyperbolic representations in previous literature to 3D biomedical images. The multi-scale sampling and triplet-loss are also commonly used mechanisms. It would be better if the authors could elaborate the novelty of the proposed model.
3. The biologically-inspired toy dataset does not look close to a biomedical dataset. Although we totally understand there should be differences between simulated dataset and real dataset, the toy dataset (i.e. composed of regular shapes) oversimplifies the biomedical image, and makes the experiment results less convincing. It will be better if the authors can simulate images close to real biomedical images, at least with irregular shapes or fuzzy boundaries. 
4. For the baseline selection, the authors used the model in paper “Unsupervised Segmentation of Hyperspectral Images Using 3D Convolutional Autoencoders,” which is published in IEEE Geoscience and Remote Sensing Letters. There should be a domain gap between remote sensing images and biomedical images. I personally think it may not be a fair comparison for the baseline methods. It will be better if the authors could select some of the state-of-the-art unsupervised 3D segmentation methods as baselines.
review_title: Learning Hyperbolic Representations for Unsupervised 3D Segmentation

Meta-review: This application paper applies hyperbolic convolutions in VAE learning to perform unsupervised 3D segmentation. 
Addition of these components enables performance improvements in the unsupervised segmentation task.
Overall, the paper is borderline and the reviewers mention the limited novelty of the approach, which largely uses components that have been developed before. Even though the paper presents an application of these methods to a relevant and significantly more challenging task than prior work, I recommend rejection from ICLR due to concerns about novelty.
Decision: Reject