Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
Authors: Joy Hsu
Jeffrey Gu
Gong Her Wu
Wah Chiu
Serena Yeung
Venue: NeurIPS 2021 Poster (2021)
Keywords: hyperbolic
hierarchy
representation learning
unsupervised segmentation
computer vision
biomedical

Abstract: We consider the task of representation learning for unsupervised segmentation of 3D voxel-grid biomedical images. We show that models that capture implicit hierarchical relationships between subvolumes are better suited for this task. To that end, we consider encoder-decoder architectures with a hyperbolic latent space, to explicitly capture hierarchical relationships present in subvolumes of the data. We propose utilizing a 3D hyperbolic variational autoencoder with a novel gyroplane convolutional layer to map from the embedding space back to 3D images. To capture these relationships, we introduce an essential self-supervised loss---in addition to the standard VAE loss---which infers approximate hierarchies and encourages implicitly related subvolumes to be mapped closer in the embedding space. We present experiments on synthetic datasets along with a dataset from the medical domain to validate our hypothesis.

Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
Joy Hsu"
Jeffrey Gu* ICME Stanford University j je effguustanford.co edu
Gong Her Wu
Department of Computer Science Stanford University
Department of Bioengineering
Stanford University wukon@stanf ord. edu
j jycjostand Ostanffffff ord. edu
Wah Chiu
Serena Yeung
Department of Bioengineering
Department of Biomedical Data Science Stanford University
Stanford University achc@stanforde edu
syyeungastanford.cmm edu
Abstract
We consider the task of representation learning for unsupervised segmentation 3D voxel-grid biomedical images. We show that models that capture implicit hierarchical relationships between subvolumes are better suited for this task. To that end, we consider encoder-decoder architectures with hyperbolic latent space, to explicitly capture hierarchical relationships present in subvolumes of the data. We propose utilizing 3D hyperbolic variational autoencoder with novel gyroplane convolutional layer to map from the embedding space back to 3D images. To capture these relationships. we introduce an essential self-supervised loss-in addition the standard VAE loss-which infers approximate hierarchies and encourages implicitly related subvolumes to be mapped closer in the embedding space. We present experiments on both synthetic data and biomedical data validate our hypothesis.
Introduction
Advances in biomedical imaging techniques such as cryogenic electron tomography (cryo-ET) and magnetic resonance imaging (MRI) have resulted in an ever-increasing amount of 3D biomedical image data. In these data domains, a growing body of work shows that, when provided with labels. machine learning models achieve good performance
many tasks [Ciçek et al. 2016 Milletari et al., 2017 Dou al. 2017 Falk al. 2019]. However, these labels, especially for segmentation, are very costly as they often have to be provided by experts in the appropriate field. Consequently, supervised learning and even semi- sumi--uperviied learning remain limited in this setting as (1) tasks and domains very diverse, making it intractable for experts provide labelled data for all problems; and (2) experts can only label objects they already know, restricting the potential for scientific discovery using machine learning methods. In this work, we tackle the task of unsupervised segmentation in 3D biomedical image data.
Our key insight is that 3D biomedical images have inherent hierarchical structure. For example, the cryo-ET domain, an image of a cell has hierarchy that at the highest level comprises the entire cell; at finer level comprises organelles such as the mitochondria and nucleus; and at an even
'These authors contributed equally.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

finer level comprises sub-structures such as the nucleolus of nucleus or protein machineries within organelles. Such types of hierarchies are present in many types of biomedical images (e.g., nested anatomical structures within MRI images). We hypothesize that in the unsupervised setting, models that are able to encode this internal hierarchical structure will provide better data representations for downstream tasks. To that end, we propose learning representations based on embedding subvolumes of 3D images in hyperbolic space.
In contrast to traditional Euclidean embeddings, hyperbolic embeddings better preserve hierarchical relationships present in the data. Hyperbolic representations have been proposed as continuous way to represent hierarchical data, due to their ability to embed trees with arbitrarily low error [Sarkar 2011]. A recent line of work utilizes hyperbolic representations to model hierarchical data across domains ranging from natural language word taxonomies [Nickel and Kiela, 2017 2018
and graphs [Nickel and Kiela 2017, Mathieu et al., 2019 Ovinnikov| 2019 Chami et al.. 2019]. to image classification |Mathieu et al.) [019]. In these settings, the objects, and in most cases their relationships, are explicitly encoded in the data. However, 3D biomedical images consist of subvolumes that represent parts of an implicit hierarchical structure. In our case, for any single 3D voxel-grid, we wish to embed and infer the implicit relationships between all of its subvolumes without any supervision.
To embed our 3D images in hyperbolic space, we use a 3D hyperbolic variational autoencoder (VAE). For the decoder, we propose a gyroplane convolutional layer which maps from the latent space back to 3D images while respecting hyperbolic geometry. In addition to the VAE loss, we propose an essential self-supervised loss to capture the hierarchical structure present in the data. More specifically, we consider reconstruction of implicit hierarchies as pretext task. Concretely, we add triplet loss which encourages a child subvolume to be mapped close to its parent subvolume in hyperbolic space. To capture hierarchical relationships of varying granularity, we train on subvolumes sampled at multiple scales. Finally, for specified scale, we cluster the subvolumes in latent space and produce a segmentation map.
We evaluate our model on datasets with different domains: synthetic datasets and a medical image dataset. We construct synthetic datasets where we generate structures at various scales and show that our model segments objects at multiple levels of hierarchy better than all prior unsupervised segmentation methods. We demonstrate performance gains ranging from 7% for the smallest objects to 32% for the largest objects. On the real-world medical image dataset (BraTS Brain Tumor Segmentation Challenge) [Menze et al.] [014] [Bakas et al., 2017 2018), we show that our method outperforms prior works by 19%, and even achieves comparable performance to semi-supervised methods although we do not use any labels.
2 Related Work
Segmentation on 3D voxel data Many diverse biomedical images, ranging from MRI and CT scans to fluorescence microscopy, come in the form of 3D voxel-grids. Since 3D voxel-grids are dense, computer vision tasks such as supervised segmentation are commonly performed using deep learning architectures with 3D convolutional layers [Chen et al., 2016 Dou al., 2017 Hesamian et al., 2019 Zheng et al. 2019]. However, due to the challenges of obtaining voxel- level annotations in 3D, there has been significant effort in finding semi-supervised approaches, including using labels only from several fully annotated 2D slices of an input volume [Çiçek et 20., using smaller set of segmentations with joint segmentation and registration [uu and Niethammer 2019), and using one segmented input in conjunction with other unlabelled data [Zhao et al. 2019]
Unsupervised approaches for 3D segmentation are useful not only for further reducing the manual annotation effort required, which is especially expensive for segmentation, but also for scientific discovery tasks where we would like to identify previously unknown structures for which annotations are impossible to produce. Moriya et al. (2018) extends to 3D data an iterative approach of feature learning followed by clustering Yang et al., 2016). Nalepa et al. (2020) uses a 3D convolutional autoencoder architecture and performs clustering of the latent representations. Another approach, [Dalca et al. 2018), uses a network pre-trained on manual segmentations from a separate dataset to perform unsupervised segmentation of 3D biomedical images. However, this limits applicability to areas where we already have dataset with manual annotations and makes it unsuitable for unbiased scientific discovery. Gur et al. (2019] and Kitrungrotsakul et al. (2019) develop unsupervised methods

---

for 3D segmentation of vessel structures, but these are specialized and do not generalize to the segmentation of other structures, [Uzunova al. [019] utilizes knowledge of background patches with no patholoogy, and Baur et al. 2018 uses deep autoencoding models for unsupervised anomaly
detection.
Another line of work performs unsupervised 2D segmentation, such as Ji et al. 2019) which proposes mutual information objective for clustering, and Caron of an encoder as pseudo-labels. While these methods can be applied to 2D slices of a 3D volume to perform 3D segmentation, they generally suffer limitations due to insufficient modeling the 3D spatial information. None of the aforementioned approaches explicitly learn hierarchical structure of the data, which is the main focus of our work.
[2018), which uses the clustered output
Hyperbolic representations A recent line of work employs hyperbolic space represent hierar- chical structures, with the intuition that tree structures can be naturally embedded into continuous hyperbolic space [Nickel and Kiela 2017]. These works utilize hyperbolic representations for variety of tasks, including MNIST classification [Mathieu et al., 2019, Nagano et al. 2019) Ovin- nikov 2 natural language processsing tasks such embedding word taxonomies and entailment [Nickel and Kiela 2017 Ganea et al. 2018 8) link prediction and node classification [Chami et al. 2019]. and game playing [Nagano et al. 2019]. In most of these works, hierarchical structure is explicitly encoded in data. In contrast, we seek to capture implicit hierarchical structure arising from composition within 3D images.
Several architectures have been proposed in order to learn hyperbolic representations, including hyperbolic VAEs [Mathieu et al. 2019 Nagano al. [009 Ovinnikov 2019). feed-forward and recurrent hyperbolic neural networks architectures [Ganea et al. 2018). and hyperbolic graph con- volutional networks [Chami et al., 2019]. We extend the hyperbolic VAE framework to the task of learning hyperbolic representations from subvolumes of complex 3D images, and use this to perform unsupervised segmentation.
Self-supervision Providing self-supervision by solving pretext tasks is one common approach for learning unsupervised visual representations. Pretext tasks leverage properties of the input data prior knowledge supervisory signals in order learn better representations. Examples of pretext tasks include finding the relative position of two patches sampled from an image [Doersch] et al. 2015). solving jigsaw puzzles (Noroozi and Favaro 2016), and predicting pixel movements of videos in subsequent frames [Pathak et al.] 2017]. contrast, we propose the pretext task of reconstructing implicit hierarchy in 3D voxel-grid images, to learn effective hyperbolic representations for downstream segmentation.
3 Methods
In this section, we describe our approach for learning hyperbolic representations of subvolumes (3D patches) from 3D voxel-grid data. We propose model that comprises 3D convolutional variational autoencoder (VAE) with hyperbolic representation space and gyroplane convolutional layer. We train our model with self-supervision through a novel hierarchical triplet loss and multi- patch sampling scheme. Then, we cluster the learned representations using hyperbolic k-means to produce 3D segmentations. In Section 3.1] we provide an overview of hyperbolic space. In Section 3.2] we describe our VAE framework with the proposed gyroplane convolutional layer and self-supervised hierarchical triplet loss. Finally, in Section 3.3 we discuss our approach of hyperbolic clustering for segmentation.
3.1 Hyperbolic formulation
Hyperbolic space We embed subvolumes of 3D voxel-grid data in hyperbolic space, a non- Euclidean space with constant negative curvature. negative curvature spaces, the area of disc increases exponentially with the radius. We can think of this growth as analogous to the exponential increase leaves at each level of a tree. Hence hyperbolic space can encode trees with arbitrarily low error [Sarkar] 2011 and can be considered as the continuous version of hierarchical structures. Unlike trees, hyperbolic space is smooth, permitting our use of deep learning on representations. For additional background geometry and hyperbolic space, see the Appendix.
3

---

Poincaré ball model of hyperbolic geometry In this work we use the Poincaré ball as our model of hyperbolic geometry. The Poincaré ball (of curvature -1) the open ball of radius 1 centered at the origin equipped with the metric tensor 9p = (A2)² where the conformal factor X |- and ge is the Euclidean metric tensor (i.e., the Euclidean dot product). Formally, this makes the Poincaré ball a Riemannian manifold. For an introduction to Riemannian manifolds, see the Appendix. The distance dp between points on the Poincaré ball is given by:
2
d2 v) = cosh -1 (1 (1
|
|
We use the exponential and logarithm maps to map from Euclidean space to the Poincaré ball and Vice versa. On the Poincaré ball, we note that the exponential and logarithm maps have the closed form expressions
ex) tanh
2 ||/|
log,(1) = tanh - (Il - 2 @ vll),
2 A
2 3 y
|| - 2 + y/1
where @ denotes Mobius addition, which was introduced by Ungar 2001 as a way to define vector operations on hyperbolic space.
Wrapped Normal Distribution The importance of the normal distribution in Euclidean space has led to many generalizations of the normal distribution to Riemannian manifolds. We use the wrapped normal distribution [Mathieu al., 2019, Nagano et al. 2019), which can be defined on an arbitrary Riemannian manifold as the push-forward measure obtained by mapping the normal distribution in Euclidean space along the manifold's exponential map. On the Poincaré ball, the probability density function of the wrapped normal with mean μ and covariance Σ is:
dp(p,2) Wz Σ) = (N)(a (inh (1,,2)
(4)
where the subscripts P. E indicate distributions over the Poincaré ball and Euclidean space, respec- tively. We use the sampling and reparametrization scheme of Mathieu al. (2019 in order to sample and train our VAE using the wrapped normal distribution.
3.2 Unsupervised hyperbolic representation learning
3D hyperbolic variational autoencoder We propose hyperbolic VAE that consists of 3D convolutional encoder and decoder to handle 3D input. Our 3D convolutional encoder maps sampled subvolumes of the input into hyperbolic space and produces the parameters of the variational posterior. Our 3D convolutional decoder then reconstructs the 3D subvolumes from sampled latent hyperbolic representations. To ensure that both the encoder and decoder respect the geometry the latent space, we follow Mathieu et al. [019) and apply an exponential map to the output of the encoder, and use our novel gyroplane convolutional layer as the first layer of the decoder. We define the prior and variational posterior to be the wrapped normal distribution, which encourages our hierarchical representations to spread out on the Poincaré ball. Figure | illustrates an overview of our VAE framework.
Our variational autoencoder takes as input patch of fixed size m X m X m. The model learns representations of subvolumes from the input (X in Figure I that can subsequently be used to perform voxel-level segmentation of the whole volume. To learn hierarchical structure in 3D images, we train the VAE on 3D patches generated using multi-scale sampling scheme that samples patches of size X r x r, where size r is randomly sampled and resized to m. Our method learns to embed each patch as part of hierarchy in hyperbolic space.
Gyroplane convolutional layer introduce gyroplane convolutional layer to effectively map from hyperbolic embedding space to Fuclidean space. This allows us to keep the advantages of convolutional layers, such as locality, weight sharing, and translation equivariance. Our model's encoder output (μ in Figure I has
For learning better hyperbolic representations of 3D images, we

---

Sampled anchor, positive child, and negative child from input volume
Hierarchieal triplet loss
Ppes
pos
Euclidean Hyperbolic
X
Hp
neg
Gyroplane convolutional
Joeg
Input volume
Sampled 3D patch
Wrapped normal distribution
layer
Exponential map
3D
3D Conv Docoder
Conv Encoder
Z
Softplus
Figure I: Our method learns hyperbolic representations of subvolumes of 3D voxel-grid data through a 3D hyperbolic VAE with a gyroplane convolutional layer. We enhance the VAE training objective with self-supervised hierarchical triplet loss that facilitates learning implicit hierarchical structure within the VAB's hyperbolic latent space.
product manifold structure, since it is a Cartesian product of vectors in hyperbolic space. To map this 3D image in Euclidean space, we generalize the usual Fuclidean convolutional layer by replacing the Euclidean affine transformation with an affine transformation the manifold.
One way to define affine transformation on the Poincaré ball is the gyroplane layer [Ganea et al. 2018). The derivation of the gyroplane layer is motivated by the fact we can express Euclidean affine where dæ Euclidean distance and Ha,p = {= E IR (a, z p) = 0J. Ha.p is called the decision hyperplane. Ganea et al. 2018 defines the gyroplane layer fa.p from this formulation by replacing each component with its hyperbolic equivalent:
transformation as: (a, ---) p) sgn (a, p)) [a)]|a (z, Ha.p)
[d
(z)(2) = sgn ((a, logglo (z))p) (a/p Ha.p)
(5)
where Ha.p is the hyperbolic decision boundary Ha,p = f= E (a, logp())) = 0}, and the distance to the hyperbolic decision boundary dp(22, Ha.p) is
< + 2, a) (1-11 + |3]|||]]]]
dp Ho.p) sinh -1
(6)
We can now define our gyroplane convolutional layer by generalizing the Euclidean affine transfor- mation using the gyroplane layer. For simplicity, suppose x is a 4D tensor containing elements of the Poincaré ball and our kernel size le x k, with an odd & value. Our gyroplane convolutional layer is defined as:
[k/2]
k/2J
1+1k/2)
Σ
Sap(Fa....))
Ur,stt
Q [2/2]
B=s- [//2]
The gyroplane convolutional layer can be extended in the same way Euclidean convolutional layers to incorporate even kernel size k, input and output channels, padding, stride, and dilation.
Self-supervised hierarchical triplet loss As model is trained subvolumes of the 3D input, we cannot easily obtain the implicit hierarchical structure of the whole volume. To encode this structure in model, we introduce self-supervision through the reconstruction of inferred hierarchy as a pretext task. This task encourages our learned representations on the Poincaré ball to reflect parent-child relationships of the input's implicit hierarchy.
Our self-supervision takes the form of a triplet loss that encourages hierarchically-relteed patches to have more similar representations. Since any two arbitrary patches may have some hierarchical relationship, we sample patches for our triplet loss capture hierarchy tractable way. To sample 3D patches for our triplet loss, we first sample anchor patch that acts parent patch (red
5

---

volume X in Figure D. We then sample the positive patch as smaller subpatch that resides within the anchor patch (pink volume pos in Figure The anchor and positive patches form parent-child relationship that we encourage to have closer representations in hyperbolic space, which has the interpretation as belonging the same branch of the hierarchy (Hi and Mppos in Figure T The exponentially growing surface area near the edge of hyperbolic space allow this natural parent-child structure to form. We then sample a negative patch as a spatially distant patch (blue volume neg) that does not overlap with the anchor patch. The triplet loss encourages the the negative patch's s representation to belong to a different branch of the hierarchy (Up and pneg).
Our choice of positive and negative patches is motivated by the compositional hierarchy of 3D volumes. The hierarchical triplet loss encourages the anchor patch and sub-patch (parent and positive child) to have similar representations, while encouraging the anchor patch and a distant patch (parent and negative child) to have dissimilar representations. We use this implicit hierarchy reconstruction as pretext task to encourage learning relationships between nested objects in 3D biomedical images. Our multi patch sampling scheme and triplet loss formulation allows representations to encode implicit structure in hyperbolic space.
Our hierarchical triplet loss can be formulated with any dissimilarity measure dl between the encoder outputs (see Figure of the anchor Mp- positive child Ppos and negative child Pueg- For our model, we take d to be the Poincaré ball distance dp and define our triplet loss with margin a as:
Ttippttt (Ppp Pposs (aeg) == max(0, dp /ppas) dp (Mp. Mneg) + +a)
(8)
This formulation can be extended to any metric space by taking the dissimilarity measure d to be the space's metric. In particular, for our ablations using an Euclidean latent space, we take the dissimilarity measure d to be the Euclidean distance.
We optimize loss function that can be decomposed as the standard evidence lower bound (ELBO) loss for variational autoencoders and our hicrarchical triplet loss that encourages the learning of structure in hyperbolic space. |athieu et al. [019 generalized the ELBO loss to Riemannian manifold latent spaces
Optimization
LELBO = / log (Po(x)=)p(z)
g (0(=|=)) S logp(w)
where d2(2)) /G(z)) dz is the measure induced the manifold by the Riemannian metric G(2) (see Appendix). Our total loss is then formulated as
Troral LELBO + [rriplet
(10)
where 3 is hyperparameter that controls the strength of the hierarchical triplet loss.
3.3 Segmentation by clustering representations
Hyperbolic clustering In 3D segmentation, we seek to assign each voxel v a segmentation label 8y E (1,...,n), n). where is the number of segmentation classes. We perform segmentation by clustering the representations of patches centered at each voxel. We first use our trained VAE encoder to generate latent representations Hly for each voxel v. We do this by taking patch of fixed size X P X p centered at upsampling or downsampling it to the encoder input size m X m x m, and then encoding the patch to retrieve puy We then cluster the μ., into n clusters, and produce segmentation by assigning each v the cluster label of He. We perform clustering through a k-means algorithm that respects hyperbolic geometry, which we derive by replacing the Euclidean centroid and distance computations of classical k- k-means with their counterparts in Riemannian geometry, the Fréchet mean and manifold distance. We calculate the Fréchet mean using the algorithm of Lou et al.) 2020)
4 Experiments
We evaluate our method quantitatively on both synthetic 3D datasets simulating biological image data as well as the real-world Brain Tumor Segmentation (BraTS) tumor segmentation dataset. Our cologically-inspired synthetic datasets allows quantitative evaluation of segmentation multiple levels of hierarchy, while the BraTS dataset is a well-known benchmark for 3D MRI segmentation.

---

Implementation details For all models, the encoder of our variational autoencoder is comprised of four 3D convolutional layers with kernel size 5 of increasing filter depth {16, 32, 64, 128]. The decoder has the same structure, except with decreasing filter depth and a gyroplane convolutional layer as the initial layer. We use / 1e3 as the weighting factor between LELBO and Treippet and = 0.2 as the triplet margin. In all experiments, we fix the representation dimension to be d = 2, and show latent dimension ablations in the Appendix. We train our model using the Adam optimizer Kingma and Ba 2014). For inference, we obtain the latent representations of 5 x 5 x 5 patches densely across the full volume, and then perform hyperbolic k-means clustering, where the number of clusters k is hyperparameter that controls the granularity of the segmentation. For quantitative evaluation, we then use the Hungarian algorithm [Kuhn 1955 to match each predicted segmentation class with corresponding ground truth label.
We utilize two anchor patch sampling schemes, one for input of smaller sizes and one for larger sizes. In both schemes, for a given 3DD volume, we sample i patch centers uniformly with patch size r, and upsampling or downsampling to size m X m x m. In the sampling scheme for smaller inputs, the patch size is sampled uniformly, whereas in the sampling scheme for larger inputs, is sampled log-uniformly. This scheme is motivated by the following observations: for larger patches, a small change in is less likely to correspond to significant semantic difference, and inherent structure causes the different levels of hierarchy to naturally follow a log scale. For training on the synthetic dataset, we sample 3D volume sizes uniformly, and for BraTS we sample using the log scale.
For every sample of an anchor patch size r X X r. we generate a positive child patch as a smaller patch of the anchor patch as follows: the positive child patch is subvolume within the anchor patch with size rchild X Tchild X rehild, where rchild [fffmin: P r rgap), and rgap a hyperparameter representing the gap in size between the anchor size and the child size. The negative child is a patch of size Tchild X Tchild X Tehild that does not overlap with the anchor patch.
4.1 Iiologically-inspired synthetic dataset
Since we want to evaluate segmentation performance at multiple levels of hierarchy and most 3D datasets do not have the necessary annotation, we first generate synthetic dataset. This dataset en- ables more thorough evaluation of the effectiveness of our model for unsupervised 3D segmentation. Our synthetic dataset is inspired by cryo-ET images of cells. Each volume in our synthetic dataset contains multiple levels of hierarchy with the objects at each level differentiated by texture, size, and shape. Figure 2 shows an example input volume with sampled slices shown. Our dataset consists of 120 total volumes, which we split into S0 training, 20 validation, and 20 test examples. Each synthetic volume has size 50 x 50 50. Additional information on the synthetic dataset generation process well as more difficult version of the dataset, where the boundaries of each shape perturbed, is described and benchmarked in Appendix A.S.
Figure 2: Sampled 2D slices from 3D volume in our biologically-inspinn synthetic dataset. The top row showcases the raw input data, and the bottom row showcases the ground truth segmentation.
To demonstrate segmentation performance on objects at different scales, we evaluate on the three levels of hierarchy defined above and use the average class DICE score compare segmentation performance. Since our model is unsupervised, we assign segmentation classes to ground truth labels using the Hungarian algorithm. See results in Table T and Table 2 We also show results on a more challenging irregular synthetic dataset in Appendix A.5.
Comparison with prior approaches Table shows quantitative comparison of our method with prior state-of-the-ane 3D unsupervised and 2D unsupervised (which we extend to 3D) models. In addition, we also compare our method to prior semi- semi-supervised work, as unsupervised 3D segmenta- tion is a relatively unexplored field, and we provide basclines with different levels of supervision for
7

---

Table 1: Comparison with prior approaches synthetic dataset
DICE Level
DICE L
DICE Level
SUPERVISION TYPE
CICEK ET AI.
2016
0.968
0.829
0.668
3D SEMI-SUPERVISHD
ZHAO ET
2019
0.989
0.655
0.357
3D SEMI-SUPERVISED
NALEPA ET 2020
0.530
0.276
0.112
3D UNSUPERVISED
JT AL. 2019
0.589
0,291
0.150
2D TO 3D UNSUPERVISED
MORTYA ET AL..
2018
0.628
0.311
0.141
3D UNSUPERVISED
OURS
0.952
0.541
0.216
3D UNSUPERVISED
Table 2: Ablation studies on synthetic dataset
LATENT SPACE
CONHIGURATION
DICE Level
DICE Level
DICE Level
EUCLIDEAN
BASE TRIPLET
0.784
0.322
0.109
0.761
0.342
0.153
HYPERBOLIC
BASE
0.832
0.352
0.135
GYROCONY
0.905
0.473
0,204
TRIPLET
0,945
0.534
0.222
GYROCONY & TRIPLET
0.952
0.540
0.216
additional reference. Çiçek et al. (2016) was trained with 2% of the ground truth slices in each of the xy, y/,, and æz planes, and Zhao et al. [019] was trained with one fully annotated atlas, which can both still be expensive given the large size of many 3D biomedical images. Ji et al. [2019] was implemented using the authors' original code and extrapolated to 3D by applying the method to each slice. For Falepa al. [2020) and Moriya al. [018]. re-implemented their methods as the original code was unavailable. Our model performs significantly better than all unsupervised prior work all levels of hierarchy. We also perform comparably to the semi-supervised approach of Zhao et al. 2019). despite not using any labels.
Ablation Table presents ablation studies on the hierarchical synthetic dataset comparing our contributions: Euclidean vs. hyperbolic representations, the addition of our gyroplane convolutional layer, and the addition of our hicrarchical triplet loss. The base Euclidean configuration the 3D convolutional VAE with Euclidean latent space, gyroplane convolutional layer, and trained with just the ELBO loss. The triplet Euclidean configuration adds the hierarchical triplet loss to the base Euclidean configuration. The base hyperbolic configuration is the same as the base Euclidean configuration except with hyperbolic latent space. The triplet configuration the hyperbolic analogue of the Euclidean triplet configuration, and gyroconv configurations have the addition of the gyroplane convolutional layer.
Hyperbolic representations outperform their Fuclidean counterparts in all experiments. We attribute this to the more efficient and better organization of hyperbolic representations, When we introduce our self-supervised triplet loss, performance improves significantly for our hyperbolic models. Per- formance for our Euclidean model does not improve as much, likely due to information loss in representing hierarchical input. Introducing the gyroplane convolutional layer shows clear improve- ment over the base hyperbolic model, which demonstrates the benefit having a convolutional layer that respects the geometry of the latent space. The combination of the triplet loss and gyroplane convolutional layer exhibits the most gain over the base hyperbolic model, with smaller gains over the model with just the added triplet loss. We show the importance of our hierarchical self-supervision for learning effective representations that capture implicit hierarchical structure.
4.2 Brain Tumor Segmentation challenge dataset
The BraTS 2019 dataset is public, well-established benchmark dataset containing MRI scans of brain tumors and voxel-level ground truth annotations of tumor segmentation masks [Menze] al. 2014 Bakas al. 2017 2018). The seans are of dimension 200 200 x 155 and have four modalities; use the FLAIR modality, which is the most commonly used onc-modality input. We

---

Table 3: Table shows comparison BraTS 2019 dataset. Figure shows qualitative example where top left image is a slice from a 3D test volume, and the three other images show segmentations with 2, 3, 4 numbers of clustering centroids respectively, illustrating multiple levels of hierarchy learned.
BRATS DATASET
DICE WT
ALGORITHM TYPE
SOTA JIANG ET AL.
2019
0.888
3D FULLY-SUP.
ZHAO ET AL.
2019
0.648
3D SEMI-SUP.
ÇIÇEK ET AL.
2016
0.760
3D SEMI-SUP.
JIET AL.
2019
0,211
2D-TO-3D UNSUP.
MORIYA ET AL.
2018
0.425
3D UNSUP.
NALEPA
2020
0.495
3D UNSUP.
OURS
0.684
3D UNSUP.
the same evaluation metric as in the BraTS challenge. and compare DICE score whole tumor (WT) segmentation, which detectable solely from FLAIR, as well as average and 95 percentile Hausdorff' distance for the competing unsupervised methods (see Table There 259 high grade glioma (HGG) labelled training examples, which we split into 180 train, 39 validation, and 40 test examples. We do not the official validation test sets because the ground truth annotations for these sets are not publicly available.
Ssows the DICE score comparison of our results against prior work. For fair comparison, all basclines trained with only the FLAIR modality. The only exception is the current state- of-the-art suply-supervised result (Jiang et al. 2019) in Table 3 which uses all 4 modalities. We show this for reference the reported number is trained on the full train and evaluated on the BraTS test set.
Table Comparison of our method against prior unsupervised work in Hausdorff distance. (Lower is better.)
upper bound;
AVERAGE
95%
MORIYA
2018
118.144
170.434
ET AL.
96.865
114.400
Our best model performs significantly better than all prior unsupervised methods, and in addition outperforms one 3D cemi-supervised model. This demonstrates the ability of our self- supervised hyperbolic representations to effectively capture the hierarchical structure in individual brain scans. We granular segmentation with three clusters for quantitative evaluation in order to capture the tumor, brain, and background, then use the Hungarian algorithm for assignment. In Table 4 demonstrate that model also outperforms all prior methods average and 95 percentile Hausdorff metrics. In addition, we also show qualitative results for our model (see Figure (3), which include byproduct segmentations from the same model with different numbers of clusters specified, showcasing additionally discovered features in the sean that could be clinically useful.
NALEPA ET AL.
[2020]
87.704
110.803
OURS
77.940
97.641
Conclusion
propose a method for learning hyperbolic representations of 3D voxel-grid images that captures the implicil hierarchical structure in biomedical data, and show that these representations are well suited for the task unsupervised 3D segmentation. We conduct our representation lcarning through a hyperbolic 3D convolutional VAE with novel gyroplane convolutional layer that respects hyperbolic geometry. We then enhance the VAE training objective with a self-supervised hicrarchical triplet loss that learns implicit hierarchical structure within the VAE's hyperbolic latent space as pretext task. We demonstrate that hyperbolic clustering of learned voxel-level representations can be used to achieve ofte--of-theaar unsupervised 3D segmentation synthetic hierarchical datasets and the real-world BraTS dataset.
Acknowledgments. We thank support from the Chan-Zuckerrerg Initiative (Neurodegennratiin Challenge Network Collaborative Pairs grant DAF2021-221728 J.H., J.G., G.W., W.C., and Y.) and National Institutes of Hcalth (grant number POINS092525 to W.C.).

---

References
Bakas, Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. S. Kirby, B. Freymann, K. Farahani, and C. Davatzikos. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data, 4:170117, 2017.
Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, R. T. Shinohara, C. Berger, S. M. Ha, M. Rozycki, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629, 2018.
C. Baur, B. Wiestler, S. Albarqouni, and N. Navab. Deep autoencoding models for unsupervised anomaly segmentation in brain mr images. In International MICCAI Bruinlesion Workshop, pages 161-169. Springer, 2018
M. Caron, P. Bojanowski, A. Joulin, and M. Douze. Deep clustering for unsupervised learning of visual features. In Pruceedings of the European Conferencee on Computer Vision (ECCV), pages 132-149, 2018.
I. Chami, R. Ying, Re, and J. Leskovic. Hyperbolic graph convolutional neural networks. Advances in Neural Information Processing Systems (NIPS 2019), 2019.
Chen, L. Yang. Y. Zhang. M. Alber, and D. Z. Chen. Combining fully convolutional and recurrent neural networks for 3d biomedical image segmentation. In Advances in neural information processing systems, pages 1046-3044, 2016.
Ö. Çiçek, A. Abdulkadir, S. Lienkamp. Brox, and O. Ronneberger. 3d u-net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assinne assistte intervention, pages 424 432. Springer. 2016.
A. V. Dalca, J. Guttag, and M. R. Sabuncu. Anatomical priors in convolutional networks for unsupervised biomedical segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9290-9299, 2018.
C. Doersch, A. Gupta, and A. A. Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision, pages 1422-1430, 2015.
Q Dou, I.. Yu, H. Chen, Y. Jin, X. Yang. J. Qin, and P.-A. Heng. 3d deeply supervised network for automated segmentation of volumetric medical images. Medical image analysis, 41:40-54, 2017.
T. Falk, D. Mai, R. Bensch, Ö. Çięek, A. Abdulkadir, Y. Marrakchi, A. Böhm, J. Deubner, Z. Jäckel, K. Seiwald, al. U-net: deep learning for cell counting, detection, and morphometry. Nature methods, 16( 16(1):67-70, 2019.
O.-E. Ganea, G. Becigneul, and T. Hoffmann. Hyperbolic neural networks. Advances in Neural Information Processing Systems 31 (NIPS 2018), 2018.
S. Gur, L. Wolf, L. Golgher, and P. Blinder. Unsupervised microvascular image segmentation using an active contours mimicking neural network. In Proceedings of the IEEE International Conference Computer Vision, pages 10722- 10731, 2019.
M. H. Hesamian, W. Jia, X. He, and P. Kennedy. Deep learning techniques for medical image segmentation: Achievements and challenges. Journal of digital imaging, 32(4):582- 596-568 2019.
X. Ji, J. F. Henriques, and A. Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings the Vision, pages 9865 99744, 2019.
the IEEE International Conference Computer
Z. Jiang, C. Ding, M. Liu, and D. Tao. Two-stage cascaded u-net: 1st place solution to brats challenge 2019 segmentation task. In International MICCAI Brainlesion Workshop, pages 231-241. Springer, 2019.
D. P. Kingma and J. Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
10

---

T. Kitrungrotsakul, X.-H. Han, Y. Iwamoto, L. Lin, A. H. Foruzan, W. Xiong, and Y.-W. Chen. Vesselnet: A deep convolutional neural network with multi pathways for robust hepatic vessel segmentation. Computerized Medical Imaging and Graphics, 75:74-83, 2019.
H. W. Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.
A. Lou, Katsman, Q. Jiang, S. Belongie, S.-N. Lim, and C. Sa. Differentiating through the fr), 'echet mean. arXiv preprint arXiv:2003.00333, 2020.
E. Mathieu, C. L. Lan, J. Maddison, and R. T. Y. W. Tee. Continuous hierarchical representations with poincaré variational auto-encoders, Advances in Neural Information Processing Systems 32 (NIPS 2019), 2019.
B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramen Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, Slotboom, R. Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions medical imaging, 34(10): 1903))999222000 2024, 2014.
Milletari, S.-A. Ahmadi, C. Kroll, A. Plate, V. Rozanski, J. Maiostre, J. Levin, O. Dietrich, B. Ertl- Wagner, K. Bötzel, et al. Hough-enn: deep learning for segmentation of deep brain regions in mri and ultrasound. Computer Vision and Image Understanding, 164:92-102, 2017.
T. Moriya, H. R. Roth, Nakamura, H. Oda, K. Nagara, M. Oda, and K. Mori. Unsupervised segmentation of 3d medical images based on clustering and deep representation learning. In Medical Imaging 2018: Biomedical Applications in Molecular, Structural, and Functional Imaging. volume 10578, page 1057820. International Society for Optics and Photonics, 2018.
Y. Nagano, S. Yamaguchi, Y. Fujita, and M. Koyama. wrapped normal distribution on hyperbolic space for gradient-baat learning. Proceedings of Machine Learning Research, 97, 2019.
Nalepa, M. Myller, Y. Imai, K. ichi Honda, Takeda, and M. Antoniak. Unsupervised segmentation of hyperspectral images using 3d convolutional autoencoders. IEEE Geoscience and Remote Sensing Letters, pages 1-5, 2020.
M. Nickel and D. Kiela. Poincaré embeddings for learning hierarchical representations. In Advances in neural information processing systems, pages 6338-6347, 2017.
M. Nickel and D. Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. arXiv preprint arXiv:1806.034/ 2018.
M. Noroozi and P. Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision, pages 69- 84. Springer, 2016.
I. Ovinnikov. Poincar' 'e wasserstein autoencoder. arXiv preprint arXiv: 1901:1191 2019.
D. Pathak, R. Girshick, P. Dollár, T. Darrell, and B. Hariharan. Learning features by watching objects move. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2701-2710, 2017.
R. Sarkar. Low distortion delaunay embedding of trees in hyperbolic plane. In International Symposium on Graph Drawing, pages 355-366. Springer, 2011.
A. A. Ungar. Hyperbolic trigonometry and its application in the poincaré ball model of hyperbolic geometry. Computers & Mathematics with Applications, 41(1-2):135-144 -47, 2001.
H. Uzunova, S. Schultz, H. Handels, and Ehrhardt. Unsupervised pathology detection in medical images using conditional variational autoencoders. International journal of computer assisted radiology and surgery, 14(3):551445 2019.
Z. Xu and M. Niethammer. Deepatlas: Joint semi-supervised learning of image registration and segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 420-429. Springer, 2019.

---

J. Yang. D. Parikh, and D. Batra. Joint unsupervised learning of deep representations and image clusters. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5147-5156, 2016.
A. Zhao, G. Balakrishnan, F. Durand, J. V. Guttag, and A. V. Dalca. Data augmentation using learned transformations for one-shot medical image segmentation. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 8543-8553, 2019.
H. Zheng, Y. Zhang. L. Yang, P. Liang, Z. Zhao, C. Wang, and D. Z. Chen. A new ensemble learning framework for 3d biomedical image segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 5909-5916, 2019.
12


Reviews:
[{"review_id": "zqZwLLKxAum", "reviewer": "Reviewer_c7iX", "summary": "The authors present a novel representation for learning hierarchical relations within samples (3D image pixels) in an embedding space. A component that that enables this mapping is a novel self-supervised loss function. Results on real data indicate advantageous properties. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "* Information on hierarchical representations are often ignore in multi-class classification / image segmentation tasks.\n* The concept of mapping into a hyperbolic space can be considered to be innovative, as can the loss function\n* Results on exemplary application data is promising. \n* A discussion of how a hierarchical organization of the data can be used explicitly in this framework would be helpful. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations: yes\n\nSocietal impact: does not apply", "review_text": "", "extra_scores": {"ethical_concerns": "none", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "uFDrJSOT-AN", "reviewer": "Reviewer_rQZz", "summary": "The authors employ hyperbolic embeddings to capture implicit hierarchical relationships in biomedical imaging. Novel components of the proposed method are (1) a self-supervised triplet loss and (2) a gyroplane convolutional layer. They validated their method on (1) synthetic toy dataset and (2) BraTS'19 benchmark, where the proposed method demonstrated superior/competitve performance compared to other unsupervised/semi-supervised/supervised methods.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this paper proposes an interesting combination of hyperbolic embeddings and self-supervised learning, and applies it to biomedical imaging which is in need of advanced unsupervsed learning methods. As such, I would consider the result of this paper worth reporting despite of limited technical novelty. However, I have a major reservation on experimental validation of the proposed method.\n\n[Originality]\n\nTechnical novelty of the proposed method is limited to (1) introducing self-supervised triplet loss (which is not novel by itself) and (2) generalizing the gyroplane layer (Ganea et al. 2018) to the gyroplane convolutional layer (novel but limited). Nevertheless, it is a novel exploration to adopt hyperbolic embeddings to capture implicit hierarchical structures in biomedical images.\n\n[Quality]\n\nThe quality of the paper is okay, but more thorough experimental validation is required. The synthetic toy dataset is only a proof-of-concept that doesn't tell much about the validity of the proposed method. It's an interesting start point for research, but much more emphasis should be put on real world validation. One thing is that although the toy dataset is said to be inspired by cellular organization, it is not that similar to the actual cell in terms of the scale. The superficial/conceptual resemblance isn't really meaningful/helpful. There exist a lot of publicly available electron microscopy (EM) images of cells, so it would be interesting to see what the proposed method learns from EM cellular images.\n\nSecond, the validation on the BraTS'19 dataset is too limited. Reproducing other methods is non-trivial. There must be a sanity check on whether the used code (either original or reimplementation) and experimental setup reproduce the original numbers reported by the original authors. And that's why the public benchmark and its leaderboard exist. It's a *fair* battleground for competiting methods. Why didn't the authors try to use all modalities (not just FLAIR) and evaluate their method on the test set? Also, qaulitative inspection is largely missing. People would find it interesting to see a thorough qualitative inspection of the proposed method on real world datasets, especially given that the method is unsupervised and claimed to be useful for novel discovery.  \n\n[Clarity]\n\nThe paper is written clearly in general, organized well, and adequately informative.\n\n[Significance]\n\nIt is significant to advance unsupervised learning methods for biomedical imaging. Self-supervised learning was facilitated by the application of hyperbolic embeddings to capture implicit hierarchical structures, which is another significant finding. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It doesn't seem like the authors adequately addressed the limitations of the proposed method.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "8cn2NOK91Sb", "reviewer": "Reviewer_e4nE", "summary": "In this paper, the authors proposed to learn hyperbolic representations using 3D hyperbolic VAE with a novel gyroplane convolutional layer before 3D conv encoder. Apart from standard VAE loss, a self supervised triplet loss is also added to attract related subvolumes and repel unrelated subvolumns in the embedding space. They demonstrated their results on a biologically-inspired synthetic dataset and a brain tumor segmentation dataset, and showed that their approach achieves state-of-the-art unsupervised 3D segmentation. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: while this work can be viewed as a combination of known methods, I feel that the work contains sufficient novelty. 1) the application of the hyperbolic latent to this particular task is novel, 2) modifications to [Ganea et al, 2018] to make the gyroplane layer convolutional, a necessary step for the VAE, and 3) the self-supervised pretext task and accompanying loss. I am sold on the intuition that the hyperbolic latent has some nice properties for these types of problems.\n\nQuality: Overall, I am impressed with the technical soundness and completeness of the work. However, I have a few questions: \n- Regarding the pretext task and triplett loss: The negative patch is a spatially distant patch, but can it be content-wise similar to the positive patch as well? In Figure 1 and Table 3, we can see that both the synthetic BraTS datasets have a higher degree of symmetry than natural images. For example in Figure 2, if a positive patch is at the top left corner, is it fair to name the patch at the bottom right corner as a negative?\n- The role of the upsampling and downsampling is unclear\n- Why use d=2? Unless I am reading the table in the appendix wrong, you’d benefit from a higher dimensional latent.\n\nClarity: I think the clarity of the work is excellent overall. Great job!\n\nSignificance: I believe the task addressed here is an important one, so from this perspective I think the work has significance. The interest to the NeurIPS community would be higher if results could be shown in another domain. Also, as a sanity-check it would be good to validate your reimplementation of Naelpa et al. [2020] and Moriya et al. [2018] by reproducing their reported results in the Appendix. It would also be preferable to report multiple runs, or better yet perform statistical tests. You may want to add [1] to your comparisons.\n\nOther comments:\n- You may want to add [2] to your related work\n- Interesting that gyroconv + triplet performs worse at DICE level 3… thoughts?\n- The broader impact, I believe, should be have been included in the main article\n\n[1] Uzunova, Hristina, et al. \"Unsupervised pathology detection in medical images using conditional variational autoencoders.\" International journal of computer assisted radiology and surgery 14.3 (2019): 451-461.\n\n[2] Baur, Christoph, et al. \"Deep autoencoding models for unsupervised anomaly segmentation in brain MR images.\" International MICCAI Brainlesion Workshop. Springer, Cham, 2018.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No, but it seems there is no potential negative social impact", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "0ZczqMtkRge", "reviewer": "Reviewer_oTyp", "summary": "This paper proposes a  VAE architecture with a hyperbolic latent space called 3D hyperbolic VAE. The proposed method is evaluated on synthetic datasets and a medical dataset, showing its effectiveness for the unsupervised 3D segmentation task.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality:**  The novelty is limited, as the proposed method is a combination of exsiting techniques. However, I appreciate that this paper tries to solve a difficult task, i.e., unsupervised segmentation of 3D voxel-grid biomedical images.\n\n**Quality:**  (1) I would suggest authors to use more evaluation criterions. In the 3D biomedical segmentation task, Jaccord score,  95% Hausdorff Distance (95HD), and  Average Surface Distance (ASD) are  also usually used. Providing results in terms of these criterions can better evaluate the proposed method.\n\n(2) Table 3 shows the comparision with previous state-of-the-art methods on BraTS 2019. But there are not many methods on the unsupervised segmentation task. I think authors should consider more methods for comparision, even some frameworks in the computer vision field for the unsupervised segmentation of 3D data (e.g., videos). Plus, more real medical datasets can be used for evaluation.\n\n**Clarity:**  The paper is clearly written and easy to follow.\n\nOverall, I am concern about the novelty of the paper, and authors may need to provide more experimental results in terms of different evaluation indicators and more real datasets. So, I vote for rejection, but the score might be increased based on the authors' response and the discussion.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is no evident limitation in the proposed method.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the unsupervised segmentation of 3D biomedical images. The authors propose to leverage hyperbolic representations as continuous models for modeling hierarchical structures. A VAE based on Gyroplane convolutional layers at the decoder level is introduced, and trained with a self-supervised loss in addition to the ELBO data fitting based on making distance between .

The paper initially received three weak accept and one weak reject recommendation. The main concerns pointed out by reviewers were about the lack of novelty of the proposed method, and the need to compare to stronger baselines and on broader datasets. After rebuttal, all reviewers stick on their initial ratings. 

The AC carefully read the submission and authors' feedback. Although the approach uses known tools (hyperbolic representation, self-supervised learning), the AC considers that its adaptation for unsupervised 3D biomedical image segmentation is meaningful. The paper is also well written and clear. The experiments are overall convincing, although the AC agrees with the reviewers' concerns related to the fact that the synthetic dataset could have been designed more thoroughly. 
The AC thus recommends acceptance, but highly encourages the authors to carefully take into account and reviewers' comments and there feedback to improve the final version of the paper. Especially, the different metrics should be included in the main paper, and the author should also verify that their reimplementation of the baselines reach the performances reported in the published papers. The submission has been discussed with the senior area chair who agrees with the recommendation.
Decision: Accept (Poster)