ShapeEmbed: a self-supervised learning framework for 2D contour quantification
Authors: Anna Foix Romero
Craig Russell
Alexander Krull
Virginie Uhlmann
Venue: NeurIPS 2025 poster ()
Keywords: shape quantification
shape descriptors
VAE
generative model
representation learning
microscopy
Primary area: machine_learning_for_sciences
TL;DR: ShapeEmbed is a self-supervised method using Euclidean distance matrices to encode the shape of objects in 2D images in a way that is robust to shape-preserving geometric transformations

Abstract: The shape of objects is an important source of visual information in a wide range of applications. One of the core challenges of shape quantification is to ensure that the extracted measurements remain invariant to transformations that preserve an object’s intrinsic geometry, such as changing its size, orientation, and position in the image. In this work, we introduce ShapeEmbed, a self-supervised representation learning framework designed to encode the contour of objects in 2D images, represented as a Euclidean distance matrix, into a shape descriptor that is invariant to translation, scaling, rotation, reflection, and point indexing. Our approach overcomes the limitations of traditional shape descriptors while improving upon existing state-of-the-art autoencoder-based approaches. We demonstrate that the descriptors learned by our framework outperform their competitors in shape classification tasks on natural and biological images. We envision our approach to be of particular relevance to biological imaging applications.

ShapeEmbed: a self-supervised learning framework for 2D contour quantification
Anna Foix Romero O Craig R Alexander Krull 2 and Virginie Uhlmann 1,
European Bioinformatics Institute, European Molecular Biology Laboratory, Cambridge, UK, {afoix, ctr26, uhlmann]@@bbi ac..uk
School of Computer Science, University of Birmingham, Birmingham, UK, a.f. ff.f.u krulk@bham.e.. ac. uk Department of Molecular Life Sciences, University of Zurich, Zurich, CH, virginie wwl.ann@mls .zh.ch
"Corresponding authors
Abstract
The shape of objects is an important source of visual information in a wide range of applications. One of the core challenges of shape quantification to ensure that the extracted measurements remain invariant to transformations that preserve an object's 's intrinsic geometry, such as changing its size, orientation, and position in the image. In this work, we introduce ShapeEmbed, a self-supervised repre- sentation learning framework designed to encode the contour of objects in 2D images, represented
Euclidean distance matrix, into shape descriptor that is invariant to translation, scaling, rotation, reflection, and point indexing. Our ap- proach overcomes the limitations of traditional shape descriptors while improving upon existing state-of-the the-hhr-tt autoenccder--assee approaches. We demonstrate that the descriptors learned by our framework outperform their competitors in shape classification tasks on natural and biological images. We envision our approach to be of particular relevance biological imaging applications.
Introduction
The outline of objects in 2D images carries essential information about their shape. In natural images, humans are often able to recognize objects purely based on their silhouette without relying texture color (Wagemans et al. 2008). Shape information unaltered by many geometric operations such as similarity transformations [Dryden and Mardia (0166] and is also unaffected by irrelevant and distracting imaging variables, such as lighting conditions or imaging setups. This is particularly relevant in biological imaging, where the shapes of living systems extracted from microscopy images serve as phenotypic fingerprints reveal cell identity, cell states, and response to chemical treatments across a wide range of imaging scales, settings, and modalities PPllchh and Heisenberg, 2009. Rangamani et al. 2013, Grosser et al. 2021 Zinchenko et al. 2023). All of these aspects make shape highly desirable abstraction from mxxel-intessity based images, enabling visualization, outlier detection, and unsupervised discovery of underlying patterns [Loo et al. 2007
Sailem et al. 2015).
The standard way of describing objects in 2D images is with binary segmentation masks, where pixels inside of an object's 's outline are set to and pixels outside to However, while such a
representation is readily produced by segmentation algorithms and allows for abstracting from lighting and imaging conditions, it is not invariant to transformations such as translation, rotation, reflection, and scaling. As such, the same object appearing twice in an image at different location orientation will yield segmentation masks that can only be recognized as equivalent after tedious processing.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To circumvent this and preserve invariance to similarity transformations, shape information is traditionally captured through statisties computed from the mask image, such as region properties (e.g., area and curvature) or Fourier descriptors [Pincus and Theriot 2007]. Such methods, however, are averaging and condensing information by design, thus providing an incomplete description from which it is impossible to fully reconstruct the original outline in all of its details.
Representation learning has recently gained attention as strategy utilizing autoencoders (Hinton and Salakhutdinoy 2006 Kingma and Welling 2014) to derive descriptors that can capture all intricacies of object shapes while producing descriptors that are invariant to irrelevant geometric transformations. The vast majority of the methods proposed so far [Chan et al., 2020 Ruan and Murphy] [019 Vadgama al. 2022] 2023] aim to encode segmentation masks by re relying on complex traiiin strategies to ensure that the resultng latent code representations are geometrically invariant.
Here, We introduce ShapeEmbed, novel approach to extract shape descriptors relying repre- sentation learning that leverages a simple architecture and training procedure to ensure invariance to translation, scaling, rotation, and reflection. Instead of directly encoding segmentation masks, we propose to encode instead a distance matrix (Dokmanic et al. 2015) representation of object outlines. The distance matrix contains all pairwise distances the points on the outline of an object and is inherently invariant to translation and rotation. It also fully describes the object contour and allows reconstructing it via information. On the other hand, distance matrices are not invariant to indexation. Even for simply connected outlines described as a sequence of ordered points, the choice of the origin and direction of travel of the sequence will result in different distance matrices. These different distance matrices will, however, be equivalent up to elementary permutations of rows and columns. Leveraging this property, we can implement invariance to indexation in the encoding step through specific architecture of the encoder and the inclusion of a new loss function, leading to a latent descriptor of shape that is robust to all thape-preserving geometric transformations.
scaling (MDS) [Cox and Cox 2000] without loss of
Distance matrices have been used for long time characterize shapes and to compute shape dissimilarities without alignment [u et al. 2012, Konukoglu et al. [0122] Govek et al. 2023). While the use of pairwise point distances these previous works similar to what we propose, we do not use the point distances directly but instead as an input to a representation learning model that maps outlines to points in a latent shape descriptor space with generative properties. Our approach has similarities with Alphafold [Jumper et al. 202T]. where distance matrices are used to describe the structure of proteins. However, as proteins are open linear structures with a clearly defined start and end point, the problem of indexation invariance encountered with closed outlines does not arise. We are thus, to the best of our knowledge, the first to overcome this issue and propose a framework to encode distance matrices of closed 2D contours with a VAE.
We evaluate our method by using a simple logistic regression classifier applied to the latent repre- sentation as a downstream shape classification task. We demonstrate that ShapeEmbed outperforms traditional bbad as well as learning- based methods on range of different problems, includ- ing computer vision benchmarks and biological imaging datasets. Further quantitative exploration of the structure of our latent space indicates that its structure captures meaningful aspects of the shape of objects in images. In summary, our main contributions are as follows:
1. We introduce, to the best of our knowledge, the first self-supervised representation learning model that learns shape descriptors from distance matrices. The descriptors are, by design, invariant scaling, translation, rotation, reflection, and re-indexing.
2. We are, to the best of our knowledge, the first to propose a solution to achieve indexation invariance in a VAE architecture for shape description based on padding operation in the encoder, operating jointly with a new loss function.
3. We show that our method outperforms the representation learning state-of-the-art and classical baselines on downstream shape classification tasks.
2 Related Work
We hereafter review relevant prior works on image-based shape quantification.
Statistics-based methods. Shape quantification relying on summary statisties aims to assemble a large enough collection of features, assuming that their ensemble provides sufficiently complete

---

description of the object's shape. The features themselves are handcrafted by design and most often consist of quantities such as area, perimeter, and curvature [van der Walt et al.] 2014]. Due to its simplicity and good empirical performance, this approach is overwhelmingly used in biological imaging [Bakal et al. 2007| [Barker et al., 2022]. Many summary statistics are inherently invariant to geometric transformations such as rotation and translation, but only partially capture shape information. As such, they are often unable to distinguish subtle shape differences.
Decomposition methods. Decomposition methods seek to approximate an object's shape by a set of basis elements. The shape descriptor then corresponds to the coefficients of that approximation, and the original outline can be reconstructed as a weighted sum of the basis elements. The most common example of decomposition based shape descriptors are the Elliptical Fourier Descriptors (EFD. [ersoon and Fu, 1977 Kuhl and Giardina 1982). EFD are inherently invariant to similarity transformations, but often perform poorly in classification tasks as discriminative information tends to be hidden in noisy higher-order approximation coefficients.
Learning baase methods. Following the success of autoencoders [Hinton and Salakhutdinov 2006
variational autoencoders (VAE, [Kingma and Welling, (2014), contrastive learning [Chen et al. 2020
He al. 2020], and vision transformers [Caron et al. [02T He et al. 2022] for representation learning, se self-supervised learning of shape descriptors directly from object masks appeared as a natural strategy to alleviate the shortcomings of classical methods. Methods have been proposed to encode images of 2D objects into a latent representation of the underlying object's shape [Chan al.] 2020 Zaritsky et 202T) but are often not invariant to translation, scaling, and rotation. To mitigate this issue, generic prealignment step can be carried out (Ruan and Murphy 2019). However, as shown in [Burgess et al. 2024). does not consistently produce good results.
A framework that employs invariant risk minimization to learn invariant shape descriptors was recently introduced in [Hossain et al.) 2024]. The approach focuses on capturing invariant features in latent shape spaces parameterized by deformable transformations. While being robust to environmental variations, this method does not explicitly focus on achieving invariance to geometric transformations in the resulting shape representations and is heavily tailored to medical imaging data, with limited applicability other types of images.
The points composing the contour of a 2D mask can be viewed as a 3D point cloud where all points lie on a plane, thus making recent works towards rotation-ivvariant point networks potentially relevant to the problem of 2D shape representation learning. Several rotation invariant architectures (Li et al.) 2021a ZZhan et al., 2022 Li et al., 2021b] have been proposed for classification, segmentation, and shape retrieval in a supervised fashion, but fewer consider the problem of learning shape representations without any labels [2Furuya et al. 2024]. Processing point clouds, which are by definition sets and therefore unordered, however, differs from processing contours, defined in our case as ordered sequences of points. Relying on ordered sequences, as we do in our approach, is critical to maintain information about point connectivity and straightforwardly reconstructing outlines for visualization, which is essential for biological imaging applications.
introduced a VAE model trained to produce a latent space that explicitly disentangles 2D geometric shape descriptor from the orientation of the input object. The decoder network takes the intnttatioonivvriinn shape descriptor together with the orientation as input and is thus able to reconstruct the original 2D contour. Both of these methods are superficially similar to ours in that they use a VAE and achieve rotation invariance. However, while [Vadgama et al. [022] 2023) explicitly estimate rotation using their encoder network, our method bypasses this step by using the already invvaiannn distance matrix representation as input to the encoder. As neither of these works evaluates their method on a downstream task and unfortunately do not provide a code repository, we were unable to include them in our results comparison.
Recently, (Vadgama et al. 2022 2023
Most closely related to our work is O2VAR [Burgess et al. 2024]. a VAE model that encodes segmentation masks into an orientation- invariant latent code representation. The key idea of this approach is to rely on an encoder with iration-equivariinn 2019) together with pooling to achieve invariance. In the O2VAE model, a realignment step is required during training orient the input with its reconstruction. While O2VAE uses an elaborate special encoder to achieve rotation invariance, our method is inherently use of distance matrix representation and only requires simple modifications to the VAE architecture to achieve indexation invariance.
convolutional layers (Weiler and Cesa
due to its
3

---

3 Proposed Approach
Custom recon. loss
Regularization loss
Not invariant
Inva. Encoder
Decoder
Segmentation mask
Outline points
Distance matrix
Recon. dist. Recon. outline
Latent code
matrix
points
Translation invariant Not:
Translation invariant + Indexation invariant
+
Rotation invariant
Indexation invariant
Rotation invariant
+ Reflection invariant
Scale invariant
Reflection invariant
Scale invariant
Figure 1: Overview of ShapeEmbed. ShapeEmbed converts the outline of an object from 2D segmentation mask into a normalized distance matrix representation that is translation, rotation, and scale invariant. Relying that adds indexation and reflection invariance. The resulting latent code forms powerful shape descriptor that can be used for downstream tasks such as classification, and allows for reconstructing the original outline, albeit arbitrarily indexed, rotated, translated, and reflected.
VAE model, it then encodes distance matrices into latent representation
ShapeEmbed extracts the outline of objects in 2D segmentation masks to construct distance matrix representation that is then used to train a VAE model to learn a latent representation of shape. Thanks to combination of the distance matrix properties and the VAE model design, the resulting latent codes are invariant to translation, rotation, reflection, scaling, and contour point indexation (Figure J In the following, we describe ShapeEmbed step-by-step and discuss how we achieve these different types of invariance in our framework.
3.1 From Segmentation Masks to Distance Matrices
ShapeEmbed operates with contours that simply connected and described by an ordered sequence of successive points. Starting with a 2D binary segmentation mask, we first extract a simply-connected pixel outline. We use the marching squares algorithm [Lorense 1987]. but other methods would be equally applicable. We then interpolate the pixels the outline with a parametric linear spline curve that we uniformly sample starting an arbitrary position on the outline and going counterclockwise to yield sequence of N successive points x, = (Iz, 31). N is hyperparameter that we set to 64 by default, and that can be adjusted depending on the number of pixels composing the outline. We then construct the corresponding N x N distance matrix D with entries di,j |x; Xýl. which is the Euclidean distance between points x, and x,. Distance matrices are naturally invariant to translation and rotation. To make them additionally invariant to scaling, we normalize them using the matrix norm, as formally demonstrated Supplementary Section A.
Despite the sequence being ordered to capture the succession of points along the object contour, our distance matrices are sensitive the choice of origin (starting point) in the sequence and direction of travel (clockwise or counterclockwise). both of which impact the ordering of the matrix entries. Upon changing the starting point and/or direction of travel, the matrix entries will be shifted diagonally (change of origin) as well as horizontally and vertically mirrored (change of direction of travel), as illustrated in Figure More precisely, for a given distance matrix D, we denote the equivalent distance matrices obtained by choosing point number & E {O..... N --11 1} as origin and E (13 as direction of travel as k This yields total of 2N different equivalent matrices representing the same point sequence. The matrix entries given by
IQQ...,N::
d d(so+k) mod N. (jo+k) mod N,
where di, are the entries of the original distance matrix D.
We propose minor modification to the encoder architecture in our VAE that makes it unable to distinguish between these re-indexations. Together with a modified loss function, our VAE is thus guaranteed to map all possible equivalent indexings of successive outline points to the same latent vector. Importantly, indexation invariance also grants our approach invariance to mirror reflection: assuming fixed choice of origin and direction of travel, mirror reflection of the outline will

---

indeed correspond to change of direction of travel, resulting in a distance matrix that is mirrored horizontally and vertically.
start Index
index
start Index
(a)
(b)
(c)
(d)
(c)
(f)
Figure Effect of indexation changes on the distance matrix. point outline (a) and its cor- responding distance matrix (b) obtained by traveling the sequence of successive outline points counterclockwise from given choice of origin (start index). Changing the direction travel equivalent to traveling through mirror-reflected version of the outline in the counterclockwise direction (c) and yields a distance matrix that is mirrored horizontally and vertically (d). different choice of origin (c) produces diagonally-shifted version (f) of the original distance matrix (b).
3.2 VAE Model with Custom Indexation Invariant Encoder
ShapeEmbed relies on a VAE model that encodes distance matrices into a latent code representation that is invariant shape-preserving transformations of the original outline.
Since distance matrices are 2D structures, they naturally lend themselves being processed by powerful and established convolutional backbones developed for image data [Bengio et al. [2013]. In our implementation, we thus use an encoder network based on the ResNet- architecture [He et al. 2016 that we mirror in the decoder path.
Remembering that our normalized distance matrices are naturally invariant to translation, rotation, and scaling but not to point indexation, we designed novel indexation invariant encoder architecture to ensure that our latent codes only carry information about intrinsic shape. As outlined in 1.1. different choices of origin in the outline point sequence result in distance matrices that are shifted diagonally. Conveniently, the convolutional layers ResNet- are shift equivariant, meaning that a shifted input will result in an identical but shifted output. Carefully considering boundary conditions, we propose use circular padding (i.e., padding by repeated tiling) in every convolutional layer, which directly corresponds to the modulo operation in As result, the convolutional layers are shifl-equivariant and produce equal but shifted outputs for all possible distance matrix indexations (starting points), By combining convolutions with subsequent global pooling operation, we could in principle achieve true ihattiinvariance. However, ResNet-18 reduces the tensor size in multiple steps using local pooling operations and strided convolutions. Strictly speaking, when convolutions such architectures, the result is therefore no longer truly shift equivariant or invariant [Rumberger et al. 2021]. In practice, however observe that architecture is sufficient help prevent the latent codes from capturing indexation changes, as demonstrated experimentally in Section 4
used within
Our final encoder backbone is therefore modified ResNet-18 where the standard convolutional and pooling operations have been replaced with layers that incorporate circular padding. To make our encoder additionally invariant to the direction of travel of the outline, we process each matrix twice using the backbone, once its original form and once horizontally and vertically mirrored. We then sum the two resulting output vectors to create architecture that is unable distinguish between matrix and its mirrored version, rendering it invariant to reflection.
3.3 Loss Function
Indexation Invariant Reconstruction Loss. Considering that our encoder is sufficiently invariant indexation, follows that next no information about the indexation the point sequence is present in the latent code. This is problem when computing the reconstruction loss: as the same latent code could have been created by any shifted and mirrored version impossible to know which version of the matrix should be reconstructed to match the input any of these alternative versions correct as they all generate the same point sequence. To account for this ambiguity, introduce novel reconstruction loss that equally rewards all equivalent versions. To
distance matrix,
5

---

compute it, we generate all 2N alternative versions D of the input distance matrix. We then define the reconstruction loss
min DD ke kE -11,0
MSE(D,, (S,(D) D
(2)
where is the decoded distance matrix (reconstruction), D is the true distance matrix (input), D an alternatively indexed version of D (see (I), and MSE(.,) is the mean squared error over all matrix entries. This approach ensures that the decoder learns to reconstruct a version of the input distance matrix that minimizes the reconstruction error regardless of indexation. It thus effectively removes the ambiguity without losing indexation invariance. By incorporating this loss into the training process, the model is encouraged focus the intrinsic geometric structure of the outlines rather than being sensitive to the arbitrary choice of indexation of the sequence of points composing them.
Distance Matrix Regularization Losses. We use several Euclidean distance matrix properties to regularize the learning process and encourage the decoder produce distance matrix-like output, leading to the formulation of three regularization terms.
First, as the distance from point to itself is null, all entries in the leading diagonal of the distance matrix should be zero, which translates to Lding (D) A Σ di,;. where d is the ith entry in the diagonal of D. Secondly, as the Euclidean distance is non- non-negativv, all entries should be greater than equal to zero, which translates to Lnan-neg
(D) No Σラ= Σラ min(d., 0). Third and finally, since the Euclidean distance is symmetric, the matrix should be symmetric too, which translates to Leym ()= = MSE (D.DT).
Overall Loss. Putting everything together, we use the following weighted sum as a loss to train our model:
LVAE Lree + 3L... + 7Lding + 8 &onon-nog + eLay::
where LKL is the classical Kullback-Leible divergence loss (Kingma and Welling 2014). Lie is our custom reconstruction loss ( and 8, y, 6, and are scalar hyperparameters. The hyperparameter 3 allows tuning the model to focus more on feature extraction and reconstruction (smaller (3) or on producing smooth latent space that can be used in generative context (larger (3) (Higgins et al., 2017]. We empirically set it to 10-10 by default, as this value was observed to balance accurate reconstructions and meaningful sampling in the latent space. The hyperparameters 7, 8, and e are all set by default to 10-5, which was empirically found through hyperparameter tuning.
3.4 Outline Reconstruction
Although we assess the latent representation learned by ShapeEmbed in downstream shape quan- tification tasks, it is useful to be able to reconstruct outlines from the latent codes for visualisation and quality control purposes. Outline points can be retrieved from distance matrix using the MDS algorithm [Cox and Cox 2000). However, despite the regularization terms presented in 3.3 the outputs of Shape are neither truly symmetric nor have leading diagonal composed of perfect zeros, and are therefore not true distance matrices. These deviations are fortunately typically negligible and within numerical error range, meaning that the leading diagonal values can be set to zero without significant loss of information. To enforce symmetry, we also take the average of
the matrix and its transpose as 1 (Ō + DT). This operation is guaranteed to produce a symmetric matrix, thus allowing us to apply MDS. The algorithm is initialized with random set of 2D points and iteratively updates them to minimize the difference between the entries of the distance matrix and the Euclidean distances between the points. MDS is guaranteed to converge, but not to the same solution every time.
However, the solutions it recovers are all equivalent up to rotation, translation, and reflection, meaning that the resulting outline will be arbitrarily rotated, translated, and reflected. Since the distance matrices inputted to the model are normalized for scale, the matrix norm must be carried over to the post-procsssing step and applied to the output distance matrix before MDS if one wants to recover the originally-sizee outline.

---

Experiments
In this section, we review the datasets and evaluation metrics we use in our experiments, provide the implementation details of our method, present and discuss the performance of ShapeLīmbed against relevant competitors, perform in in-depth ablation studies to inspect the importance of the various invariance properties granted by our model, and finally demonstrate the added value of ShapeEmbed to identify subtle phenotypes in biological images. ShapeEmbed is implemented in Python and is available at https:: / //gi thub. com/ hae//hlmm www.uhmmmmgggoouuiiteeooo Smaanggroup/Shapeemaeeeee under the MIT license. Further implementation details are provided in Supplementary Section B.
4.1 Datasets
MNIST. The MNIST benchmark dataset (GNU GPL, Deng 2012) consists of grayscale images of handwritten digits from 0 to 9, with approximately 7, 000 images per class, amounting to total of 70, 000 images.
MPEG-7. The MPEG-7 CE-Shape-1 Part B dataset (LGPL-3.0, mpe 2009]) I is a benchmark for shape matching and retrieval tasks. It consists of 1, 400 binary masks of objects belonging to 70 classes, with 20 images per class. Each class represents a distinct object category, such as different animals, tools, or symbols, designed cover a range of shape variability.
BBBC110. The Broad Bioimage Benchmark Collection 10 (BBBC010, no license, [Ljosa et al. 2012 is biological imaging dataset designed to test phenotypic profiling at the whole-organism level. It contains a total of 1, 407 individual binary masks of C. elegans nematodes divided into a live and dead class, each containing 768 and 639 individuals, respectively.
MEE. The Mouse Embryonic Fibroblast (MEF, MIT License, (Phillip et al. 2021 dataset is a challenging biological imaging dataset containing 300 images of multiple cells distributed across three classes: sirele-patterned triangle- peattleepatterned, and control (non-patterned) surfaces, with 100 images per class. Although the original dataset includes two color channels corresponding to an actin and a nuclei stain, we here only use the actin channel as it captures whole cells. Binary masks of individual cells are provided, leading to total of 26, 198 objects distributed into 3, 192 in the control, 6, 624 in the triangle, and 6, 565 in the circle class, respectively.
HeLa Kyoto. The HeLa Kyoto dataset (CellCognition project, CC-BY 4.0 License. (Held et al. 2010)) consists of fluorescence microscopy images of H2B-mCCerry-stain
HeLa Kyoto cell nuclei. labeling chromatin and capturing nuclear morphology during mitosis. Already segmented and cropped masks for individual nuclei are available under the "classifier data" section of the dataset. We consider 313 objects in 4 classes that are representative of nuclei at key phases of mitosis (category name and number of samples in parentheses): early anaphase (earlyana, 40), lateana (lateana, 83), metaphase (meta, 110), and prometaphase (prometa, 80).
Mouse Osteosarcoma Cells (MOC). The MOC dataset (MIT license, [Miolane et al. 20200 consists of fluorescence microscopy images of mouse osteosarcoma cells. In this dataset, cells have been exposed cytoskeletal perturbation through treatment with the single drugs jasplakinolide (Jasp) and cytochalasin D (Cytd). The 649 cells are divided in 3 classes: control class with 318 untreated cells, Cytd class with 175 cells, and Jasp class with 156 cells.
Additional details on experimental settings for each datasets, as well as example images and masks illustrating the two bioimaging datasets considered are provided in Supplementary Section C.
4.2 Baselines and Evaluation Strategy
We compare the performance of ShapeEmbed for shape classification against two classical shape anal- ysis baselines (Elliptical Fourier Descriptors [Persoon and Fu 1977] and Region Properties (van der Walt al. 2014). D. against two state-of-the-re representation learning models (the contrastive learning framework SimCLR [Chen al.] 2020) and the vision transformer Masked Autoencoder [He et al., 2022], and against our direct competitor (O2VAE [Burgess et al.) 2024)). ). Additional details on the implementation of these methods are provided in Supplementary Section D.
To quantitatively evaluate the quality of the different shape descriptors we consider, we rely on a downstream classification task. We train logistic regression classifier [Bisong] 2019 following
7

---

Table 1: Benchmark datasets results (FI-score + 0, higher is better).
METHOD
MNIST
MPEG-7
REGION PROP.
0,81 = 0.00
0.70 = 0.01
EFD
0.62 = 0.01
0.08 = 0.01
SIMCLR
0.59 + 0.01
0.13 0.02
MAE (VIT-B)
0.95
0.65 + 0.00
MAE (VIT-L)
0.84
0.63 + 0.04
MAE (VIT-H)
0.85
0.60 = 0.01
MAE (VIT-B, PRETRAINED)
0.92
0.42 0.01
MAE (VIT-L, PRETRAINED)
0.93
0.51 0.02
MAE (VIT-H, PRETRAINED)
0.94
0.57 = 0.02
O2VAE
0.86
0.13 = 0.02
SHAPEEMBED
0.96 0.01
0.75 0.02
Table Scaling and indexation invariance ablation results (Fl-score + a, higher better). "None" indicates indexation invariance and scale normalization.
METHOD
SMNIST
SMPEG
NONE
0.87 = 0.01
0,24 # 0.03
No INDEX. INV.
0.88 0.01
0.59 0.07
No NORM.
0.91 0.01
0.42 0.02
SHAPEEMBED
0.95+0.000 0.00
0.70 + 0.09
5-fold cross-validation strategy, and report the mean and standard deviation of the Fi-score performance metric over the 5 data folds. The Fi-score balances precision and recall and thus provides reliable measure of performance across the considered datasets Ye et al., [012). with higher F1 -core indicating better performance. For the benchmarking and biological imaging experiments, also report additional metrics (log loss, accuracy, precision, and recall) in Supplementary sections.
4.3 Benchmarking
We quantitatively evaluate the performance of region properties, EFD, SimCLR, MAE (trained from scratch and pretrained), O2VAE, and ShapeEmbed on the MNIST and MPEG-7 datasets. We highlight in Table superior performance of ShapeEmbed over the classical and self-supervised learning baselines, and against its primary competitor. We report additional metries for the same experiment in Supplementary Section E, which lead to the same conclusions. We hypothesize that subpar performance SimCLR due to the construction of positive pairs, created through image augmentation. To achieve true rotation, scaling, and positional invariance with contrastive learning, one would need to define an alternative way of creating positive pairs of masks that would comprchensively cover all the transformations experiments are not meant to push the state-of-the-ar in MNIST classification, but instead to evaluate the information content of the shape representation learned by the different methods we consider.
normalize for in ShapeEmbed. We stress that these
4.4 Ablation Studies
Scaling and Indexation Invariance. We evaluate the importance of the normalization step and of the various modifications implemented in our VAE achieve indexation invariance ShapeEmbed's ability to perform under varying object sizes and contour indexations. To assess the effect of our modified encoder and custom indexation invariant reconstruction loss, we created modified version of ShapeEmbed in which the circular padding mechanism is replaced by constant padding of 1 and where the indexation invariant reconstruction loss (2) is substituted with the standard MSE reconstruction loss. To evaluate the effect of normalization, we simply skipped it and retained the original, non-normalized distance matrices. In Table version of MNIST (sMNIST) and MPEG-7 (sMPEG-7) described in Supplementary Section C. We observe that removing indexation invariance and skipping the normalization step results in drop in performance both sMNIST and sMPEG-7, with an even more drastic effect on the latter. These
2 we report Fl FI-scores on a candomly-scaled

---

results illustrate that, when ShapeEmbed does not include scaling and indexation invariance, it captures features in the latent space that are irrelevant to intrinsic shape information and therefore interfere with downstream tasks.
Rotation and Translation Invariance. We test the robustness of our model to positional and orientation variations, which are frequently encountered in real-world data. Unlike scaling and indexation invariance, which are explicitly enforced in the model, rotation and translation invariance are inherent to the distance matrix representation we use in ShapeEmbed. Ablating the distance matrix representation thus results in encoding the image mask directly with a vanilla VAE. For the sake of completeness, we also include the performance of O2VAE as a reference, as it partially addresses rotation and translation invariance but still uses masks as input. The results reported in Table[]illustrate the positive impact of the distance matrix representation. On randomly translated and rotated versions of MNIST and MPEG-7 (rMNIST and rMPEG-7, respectively) as described in Supplementary Section C, Shaee scores higher than any of the considered alternatives. The gap in performance between ShapeEmbed and the other considered approaches highlights the difficulty of extracting relevant shape features in the absence of explicit translation and rotation invariance in dataset that exhibits great variability in object orientation and position, and demonstrates the value of the distance matrix representation. We further qualitatively explore the effect of rotation and scaling invariance on the learned representation in Supplementary Section F.
Further Ablation Studies. We experimentally explore two more ablation studies in Supplementary Section F: the added value of relying on the VAE latent codes as opposed using distance matrices directly as shape descriptors, and the effect of the distance matrix regularization terms in the loss.
4.5 Application to Biological Imaging
One of the main motivations for this work is its application to biological imaging. Shape, captured in 2D contours on microscopy images, is one of the most information-rich phenotypic characteristics and provides insights into range of biological phenomena. Shape analysis in biological images is particularly challenging as objects in these datasets typically appear unaligned. not centered, and may exhibit extensive size variations. Additionally, shape differences in biology often appear as subtle changes, the magnitude and nature of which are typically unknown a priori, making unbiased data exploration invaluable. While experiments usually aim uncover biological labels (e-g., cell type, cell state), living systems don't 't come with annotations and researchers only have access to experimental labels (e.g., treated untreated samples). Using these experimental labels as proxies for the underlying biological labels is inherently problematic due to individual variability: two samples treated identically may respond differently because of natural variations. Self-supervised approaches are especially valuable in approaching this problem as they enable the investigation of biological labels independently of experimental categories.
While scale invariance is traditionally considered to be essential for shape analysis [Dokmanic et al. 2015) and our framework is inherently scale-invariant, scale may be crucial feature in some applications. To account for this, we additionally consider a variant of ShapeEmbed that preserves scale information by saving the norm of the distance matrix before normalization and concatenating it to the latent code for downstream classification. The decision to consider scale as relevant feature be included as nuisance transformation to be removed entirely depends on the use case and biological question considered. We assess the value of ShapeEimbed on its own and with size information included (referred as Shapefimbed+S2) on biological imaging datasets at the organism (BBBC010, a well-charraterrized biological benchmark) and cellular (MEF, a harder, real-life example where the shape component is known to be essential) scales and report performance against the classical and self-supervised learning baselines, as well as against our main competitor, in Table E ShapeEmbed consistently outperforms other considered methods, and additional performance can be gained by adding back object size as an extra feature. As objects in the MEF dataset exhibit experimentally-induees shape variations, summary statistics. which include size-related metrics (such as the area), perform exceptionally well. This observation highlights the importance of offering a flexible way to handle size information that can adapt to the context. In Supplementary Section G. we report additional metrics for these experiments that lead to the same conclusions. We also qualitatively explore the latent space learned by ShapeEmbed on the BBBCO10 and MEF datasets, discuss the robustness of
size differences between classes in addition to true

---

Table 3: Rotation and translation invariance ablation results (F1-score - a, higher is better). The input to VAE and O2VAE are binary masks, while ShapeEmbed uses distance matrices as input.
METHOD
RMNIST
RMPEG
VAE
0.38 0.01
0.04 = 0.02
O2VAE
0.66 0.01
0.10 = 0.02
SHAPEEMBED
0.85 + 0.01
0.66 = 0.05
Table Biological imaging datasets results (Fl-score + or, higher is better).
METHOD
MEF
BBBC010
HELA KYOTO
MOC
REGION PROP.
0.72 0.01
0.82 + 0.00
0.57 0.08
0.61 0.07
EFD
0.33 0.04
0.55 0.04
0.22 0.11
0.38 0.02
SIMCLR
0.43 0.03
0.56 0.12
0.48 = 0.17
0.65
MAE (VIT-B)
0.54
0.60 0.12
0.47 0.13
0.57
MAE (VIT-L)
0.53 0.02
0.51 0.07
0.42 0.09
0.60
MAE (VIT-H)
0.55 0.02
0.72 0.06
0.51 +.09
MAE (VIT-B, PRETRAINED)
0.61
0.65 + 0.08
0.72 0.02
0.48
MAE (VIT-L, PRETRAINED)
0.00
0.77 = 0.03
0.76 0.03
0.66
MAE (VIT-H, PRETRAINED)
0.65 0.01
0.76 = 0.06
0.73 0.08
0.63 0.05
O2VAE
0.53 4.53
0.61 = 0.08
0.63 0.08
0.60 0.06
SHAPEEMBED
0.67 +
0.83 + 0.00
0.80 0.04
0.68 0.01
SHAPEEMRED4SZ
0.76 + 0.01
0.87 + 0.01
0.85 0.04
0.70 0.05
our method the quality of the input segmentation masks, and explore the generative properties of our model.
Conclusion
We introduced ShapeEmbed, an original self-supervised representation learning framework based on custom VAE that can, from segmentation masks, extract latent representation of shape that is agnostic to position, size, orientation, reflection, and contour point indexing. The key ideas behind method are the use of distance matrices, the implementation of simple but essential modifications to the encoder path of our VAE, and the use of novel loss terms. We experimentally demonstrated the superior performance of ShapeEmbed over existing methods for shape quantification over range of natural and biological images. We expect ShapeEmbed to be of valuable use for the unbiased exploration of shape variation in image datasets, and expect it be most impactful in biological imaging where the size, orientation, and position of objects are highly unpredictable and shape differences are subtle. Although ShapeEmbedd's 's requirement for simply-connected contours arguably restrictive, we have in practice not encountered any case where it cannot be satisfied either because the objects of interest don't have holes, or because they can be fully defined by a simply-connected midline relying on a ridge detector. Although designed for closed contours, ShapeEmbed can in principle be applied open curves as well, as we briefly discuss Supplementary Section H. While our current encoder achieves reflection invariance by processing each matrix as well as as a flipped version, this computational overload could be avoided in the future by using reflection equivariant convolutions [Cohen and Welling 2016). Our current implementation uses several regularization terms in the loss to promote the decoded outputs be valid Euclidean distance matrices, but lacks strict guarantees that the distance matrix constraints are satisfied. An exciting area for future research could be to instead parameterize distance matrix properties are guaranteed to be valid [Dokmanic et al.) 2015]. Finally, ShapeEmbed currently limited to 2D images, but it provides a strong basis for 3D extension incorporating concepts from the shape signatures literature OOsada et al. 2002 to be explored in future work.
predicted matrix such a way that
10

---

Acknowledgements
AFR acknowledges funding from the European Molecular Biology Laboratory. CTR acknowledges funding from the European Union's s Horizon Europe research and innovation programme under grant agreement 101057970 (AI4Life) and from the European Molecular Biology Laboratory. VU acknowledges funding from the European Molecular Biology Laboratory and from the University of Zurich. The authors thank the members the Uhimann group at EMBL-EBI, as well as Anna Kreshuk, Martin Weigert, and Albert Dominguez Mantes for helpful discussions on this work.
References
Johan Wagemans, Joeni De Winter, Hans Op de Beeck, Annemie Ploeger, Tom Beckers, and Peter Vanroose. Identification of everyday objects on the basis of silhouette and outline versions. Perception, 37(2):207-244, 2008.
Ian L. Dryden and Kanti V Mardia. Statistical shape analysis: with applications in R, volume 995.
John Wiley & Sons, 2016.
Ewa Paluch and Carl-Philipp Heisenberg. Biology and physics of cell shape changes in development. Current Biology, 19(17):R70-RR99) 2009.
Padmini Rangamani, Azi Lipshtat, Evren U Azeloglu, Rhodora Cristina Calizo, Mufeng Hu, Saba Ghassemi, James Hone, Suzanne Scarlata, Susana Neves, and Ravi Iyengar. Decoding informa- tion in cell shape. Cell, 154(6):1356-136) 2013.
Steffen Grosser, Jürgen Lippoldt, Linda Oswald, Matthias Merkel, Daniel M Sussman, Frédéric Renner, Pablo Gottheil, Erik W Morawetz, Thomas Fuhs, Xiaofan Xie, et al. Cell and nucleus shape as an indicator of tissue fluidity in carcinoma. Physical Review X, 11(1):011033 2021.
Valentyna Zinchenko, Johannes Hugger, Virginie Uhimann, Detlev Arendt, and Anna Kreshuk. Morphofeatures for unsupervised exploration of cell types, tissues, and organs in volume electron microscopy. Elife, 12:880918, 2023.
Lit-Hsin Loo, Lani F Wu, and Steven J Altschuler. Image-based multivariate profiling of drug responses from single cells. Nature methods, 4(5):445-453, 2007.
Heba Z Sailem, Julia E. Sero, and Chris Bakal. Visualizing cellular imaging data using phenoplot. Nature communications, 6(1):5825, 2015.
Zachary Pincus and JA Theriot. Comparison of quantitative methods for cell-shape analysis. Journal of microscopy, 227(2): (000)::100566...0 2007.
Geoffrey E Hinton and Ruslan Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 3 313(5786):504-500) 2006..
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR), Banff, AB, Canada, 2014. URL
1312.61266 6114
Caleb K Chan, Amalia Hadjitheodorou, Tony Y-C'ssi and Julie A Theriot. Quantitative comparison of principal component analysis and unsupervised deep learning using variational autoencoders for shape analysis of motile cells. bioRxiv, pages 2020-06, 2020.
Xiongtao Ruan and Robert F Murphy. Evaluation of methods for generative modeling of cell and nuclear shape. Bioinformatics, 35(14):2455-2485 2019.
Sharvaree Vadgama, Jakub Mikolaj Tomczak, and Erik J Bekkers. Kendall shape-vae: Learning shapes in generative framework. In NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations, New Orleans, LA, USA, 2022.
Sharvaree Vadgama, Jakub M Tomezak, and Erik Bekkers. Continuous kendall shape variational autoencoders. International Conference on Geometric Science of Information, pages 73-81, Saint-Malo, France, 2023. Springer.

---

Ivan Dokmanic, Reza Parhizkar, Juri Ranieri, and Martin Vetterli. Euclidean distance matrices: essential theory, algorithms, and upplications. IEEE Signal Processing Magazine, 32(6):12-30, 2015.
Trevor F Cox and Michael AA Cox. Multiddmeensional scaling. CRC press, 2000.
Rongxiang Hu, Wei Jia, Haibin Ling, and Deshuang Huang. Multiscale distance matrix for fast plant leaf recognition. IEEE transactions on image processing, 21(11):466774467,,22 2012.
Ender Konukoglu, Ben Glocker, Antonio Criminisi, and Kilian M Pohl. Wesd weighte spectral distance for measuring shape dissimilarity. IEEE transactions pattern analysis and machine intelligence, 35(9):228--2297, 2012.
Kiya W Govek, Patrick Nicodemus, Yuxuan Lin, Jake Crawford, Artur B Saturnino, Hannah Cui, Kristi Zoga. Michael P Hart, and Pablo G Camara. Cajal enables analysis and integration of single-cell morphological data using metric geometry. Nature Communications, 14(1):3672 2023.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583- 2021.
Stéfan van der Walt, Johannes L. Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D. Warner, Neil Yager, Emmanuelle Gouillart, Tony Yu, and the scikit-image contributors. scikit- image: image processing in Python. PeerJ, 2:e453, 6 2014. doi: 10.7717/pperj.455.
Chris Bakal, John Aach, George Church, and Norbert Perrimon. Quantitative morphological signa- tures define local signaling networks regulating cell morphology. science, 2007.
Charlie George Barker, Eirini Petsalaki, Girolamo Giudice, Julia Sero, Emmanuel Nsa Ekpenyong. Chris Bakal, and Evangelia Petsalaki. Identification of phenotype-specing networks from paired gene expression-cee cerllslonneelllll shape imaging data. Genome Research, 32(():750-76) 2022.
Eric Persoon and King-Sun Fu. Shape discrimination using fourier descriptors. IEEE Transactions on systems, man, and cybernetics, 7(3):100-179 1977.
Frank P Kuhl and Charles R Giardina. Elliptic fourier features of closed contour: Computer Graphics and Image Processing, 18(3):236-258 1982 doi: htts:://oi.oorg orr///0....o/..... /016/01 46666644(82)90033)
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PmLR, 2020.
Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9729 9738, 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the JEEE/CVF international conference on computer vision, pages 9650-9660, 2021.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
Assaf Zaritsky, Andrew R. Jamieson, Erik S. Welf, Andres Nevarez, Justin Cillay, Ugur Eskiocak, Brandi L. Cantarel, and Gaudenz Danuser. Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. Cell Systems, 12 (7):733-747.ed) 2021.
James Burgess, Jeffrey Nirschl, Maria-Clara Zanellati, Alejandro Lozano, Sarah Cohen, and Serena Yeung-Levy. Orientation-invariint autoencoders learn robust representations for shape profiling of cells and organelles. Nature Communications, 15(1):0022, 2024. doi: 10. 1038/
441467-024-45662-4
12

---

Tonmoy Hossain, Jing Ma, Jundong Li, and Miaomiao Zhang. Invariant shape representation learning for image classification. arXiv preprint arXiv:2411.1220. 2024.
Xianzhi Li, Ruihui Li, Guangyong Chen, Chi- Ming Fu, Daniel Cohen- and Pheng- Heng. A intation-invarian framework for deep point cloud analysis. IEEE transactions on visualization and computer graphics, 28(12) (8(12):503-445 4514, 2021a.
Zhiyuan Zhang, Binh-Son Hua, and Sai-Kit Yeung. Riconv++: Effective rotation invariant con- volutions for 3d point clouds deep learning. International Journal of Computer Vision, 130(5): 1228-1243, 2022.
Feiran Li, Kent Fujiwara, Fumio Okura, and Yasuyuki Matsushita. A closer look rotation-invarian deep point cloud analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16218-16227, 2021b.
Takahiko Furuya, Zhoujie Chen, Ryutarou Ohbuchi, and Zhenzhong Kuang. Self-supervised learning of rotation-invariant 3d point set features using transformer and its self-distillation. Computer Vision and Image Understanding, 244:104025, 2024.
Maurice Weiler and Gabriele Cesa. General -equiivrrii steerable cnns. In Advances in Neural Information Processing Systems (NewrIPS), volume 32, pages 5214-5224, Vancouver, BC, Canada, 2019, URL htttt:tt///axiiiia
08251
WE Lorense. A high resolution 3d surface construction algorithm. Proc. Siggraph 87, 1987.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1822 2013.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, Las Vegas, NV, USA, 2016.
Josef Lorenz Rumberger, Xiaoyan Yu, Peter Hirsch, Melanie Dohmen, Vanessa Emanuela Guarino, Ashkan Mokarian, Lisa Mais, Jan Funke, and Dagmar Kainmueller. How shift equivariance impacts metric learning for instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7128-7136. Montreal, Canada, 2021.
Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Mathew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. ICLR (Poster), 3, 2017.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142 2012.
MPEG-7 Core Experiment CE-Shape-1.
Benchmarking image database for shape recognition
techniques. https://www ehu, eus https//ccwwttoooooccccec www.cciiiindxx.e hwtphidde.ppppp//ppeeaaee wwtp:pppp//pPppp-p Corrr-Corre__xpiienttt
CE-Shape- 1_ Tees__Set... BelosstSett.Beeeeeeeeee Bettee_Beecmaarkinge iaaggg_mmmee_ee hwwp:/ninaanaanaeeeeeeeeeee www.daaasssooossapppeeeeeeeeeeee recape_recogmitionn
techniques 2009. Accessed: 2025-01-30.
Vebjorn Ljosa, Katherine L Sokolnicki, and Anne Carpenter. Annotated high- thgh-throughput mi- croscopy image sets for validation. Nature Methods, 9(7):637-637 July 2012. ISSN 1548- 7091, 1548-7105. doi: 10. 1008//meth.2088. http:/tp:///www nature.... com/ http//atiicce.eemm nmeth 2083
Jude M. Phillip, Kyu-Sang Han, Wei- Wei-Chiang Chen, Denis Wirtz, and Pei-Hsun Wu. A robust unsu- pervised machine-learning method to quantify the morphological heterogeneity of cells and nuclei. Nature Protocols, 16(2):754-774, 2 2021. ISSN 1750-2799. doi: 10. 10.1038841155--00-000000
M. Held, M. H. Schmitz, B. Fischer, T. Walter, B. Neumann, M. H. Olma, M. Peter, J. Ellenberg, and D. W. Gerlich. Cellcognition: time-resolved phenotype annotation in high-throughput live cell imaging. Nature Methods, 7(9)7777754, 2010. doi: 10. no.t/hhh 1486. URL https
//doi. //doi.org//i 10. 1038/nmethl 1486
13

---

Nina Miolane, Nicolas Guigui, Alice Le Brigant, Johan Mathe, Benjamin Hou, Yann Thanwerdas, Stefan Heyder, Olivier Peltre, Niklas Koep. Hadi Zaatiti, Hatem Hajri, Yann Cabanes, Thomas Gerald, Paul Chauchat, Christian Shewmake, Daniel Brooks, Bernhard Kainz, Claire Donnat, Susan Holmes, and Xavier Pennec. Geomstats: A python package for riemannian geometry in machine learning. Journal of Machine Learning Research, 21(22)):1-9 2020. URL.http 027 html
//jmrr
Ekaba Bisong. Logistic Regression, pages 250. Apress, Berkeley, CA, 2019. ISBN 978- 1-4842-4470-1 doi: 10.1007/978-1-44444477--- 20. URL https://doi org.
778-1-4842-4470-821
Nan Ye, Kian Ming A. Chai, Wee Sun Lee, and Hai Leong Chieu. Optimizing f-measures: a tale of two approaches. In Proceedings of the 29th International Conference on International Conference on Machine Learning. ICML' 12, page 1555-1562, Edinburgh, Scotland, 2012.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In Proceedings of the 33rd International Conference on Machine Learning (ICML), pages 2990-2999. PMLR, 2016.
Robert Osada, Thomas Funkhouser, Bernard Chazelle, and David Dobkin. Shape distributions. ACM Transactions Graphics (TOG), 27-4)3077-32,, 2002.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We describe the problem addressed, articulate the key aspects of our proposed method, and outline our main contributions the abstract and introduction. We also specify the intended scope of our method.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We state the domain of applicability of our method (simply-connected 2D contours described as sequence of successive points). We highlight in the paper title, the abstract, and in Sections and 2 that we are working on contours and therefore do not include intensity, texture, and other sources of information derived from images. We additionally also provide study of the robustness of our method in our Supplementary Material. Although we did not include Limitations section, we discuss the primary limitations of the method throughout the paper and our conclusion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specificatiin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
15

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The full set of assumptions of the method are described in Section 3 and formal proofs the invariance properties of the distance matrix representation are provided in Supplementary Section A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the data we use in the paper are publicly-available We also provide our code in GitHub repositories at htttps://gtthiii ShapeEmbed| (method implementation) and https: thub. com/ uh umammggroupp (data processing). We provide in our Supplementary Material details of how our method is implemented and our experiments are carried out (Supple- mentary Section B), how we used the datasets considered (Supplementary Section C), how we implemented our baselines (Supplementary Section D), and how we ran our further experiments (Supplementary Sections E, F, and G).
Convert
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution a specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general, releasing code and data is often good way accomplish this, but reproducibility detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing model checkpoint, other means that are appropriate to the research performed.
also be provided via
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
16

---

(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should descrihe the architecture clearly and fully.
(c) the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
All the data use in the paper are publicly-availant We also provide our code in GitHub repositories at htttt:://///// //gg::: com/ uhlmanngroup/ ShapeEmbed (method implemen- tation) and https://github com/ www.uaangoooo//aa// Sotpup//ooooeetiaaryaaa.e B www.rtBiinnaaaasaoaaass (data processing). We provide in our Supplementary Material details of how our method is imple- mented and our experiments are carried out (Supplementary Section B), how we used the datasets considered (Supplementary Section C), how we implemented our baselines (Sup- plementary Section D), and how we ran our further experiments (Supplementary Sections E, F, and G).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https ciaápss.ccpublcc//uuu hww.lliic/uu.ddssddddduuiiiiiis http///oodSmssssiooooooo sow.ubiiissiiooooiccccc.. for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the training and test details required to reproduce and understand the results sections in Supplementary Material in Sections B, C, D. E, F, and G.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All of our results include the mean and standard deviation of the metric considered over a 5-fold cross-validation to allow for a aattitically-mmaningee comparison of performance across methods, indicated in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide details of the hardware characteristics and of the resources used for our experiments in Supplementary Sections B and C.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp////earpppp.c//////////hts
18

---

Answer: [Yes]
Justification: the authors read the NeurIPS Code Ethics and confirm that the research complies with it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The work only contributes to providing a novel technique for specific image analysis task (specifically, shape quantification). We expect it to be of particular relevance to fundamental curiosity-driver biological research and have the potential to positively impact life science research in general, but cannot identify any risk of negative societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our method does not posses any identifiable risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
19

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The creators or the original datasets, resources and libraries used in our paper are credited with the appropriate references. The license of the datasets and codebases mentioned are indicated, and their terms of use are respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URI..
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwpprswittcooeeeoo thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code of the method we introduce in the paper released open-source under the MIT license and made publicly available, along with documentation, example data, and all supporting scripts to reproduce the results of the paper, in GitHub repository at /https:// /ttp::////tthtte com/ hataaaaannnooosSaaeeaeeeeeeeeeee
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
20

---

Justification: We don't perform any crowdsourcing and research with human subjects therefore does not apply for our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The work does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p///narrrpsscc/ooofeereaeee for what should or should not be described.
21

---

ShapeEmbed: a self-supervised learning framework for 2D contour quantification
Supplementary Material
Anna Foix Romere Craig Russel Alexander Krull and Virginie Uhlmann @
"urrpeaan Bioinformatics Institute, European Molecular Biology Laboratory, Cambridge, UK, {afoix, ctr26, uhlmann)@ebi. ac. uk
School of Computer Science, University of Birmingham, Birmingham, UK, a. a.f.f. sarul1@bham... ac uk Department of Molecular Life Sciences, University of Zurich, Zurich, CH, virginie. uhlmann@mls uzh. ch
"Corresponding authors
A Distance Matrix Properties
Translation and Rotation Invariance. Translation and rotation information is, by design, not captured by distance matrices. Any translation of two points x,, X2 does not affect the Euclidean distance and, consequently, leaves the distance matrix unchanged. This can be formally demonstrated by considering translation operation that shifts all points in a sequence by vector (tz, ty). The translation operation thus transforms all point (I, y) the original sequence into (z', (x
tz, + f. The Euclidean distance between any two points (x²,, v() and (z',.(z) in the translated sequence can straightforwardly be shown to be equal to the distance between (æ,, y:) and (æz, Uj) in the original sequence as the translation terms cancel out.
Similarly, the distance matrix remains unaffected by rotation. Considering a rotation operation that rotates the points in sequence by angle in the counterclockwise direction, the rotation operation transforms all point (x, y) in the original sequence to (x',0') (x cos y sin 0, sin + y cos 0). The Euclidean distance between any points (ar v/) and (i'), uf)) in the rotated sequence
be straightforwardly demonstrated to be equal the distance between (ar,, vi) and (rz, Mj) in the original contour as the trigonometric terms get reduced through the Pythagorean identity.
Scale Invariance. Distance matrices not automatically invariant to scaling. Scaling all points (z, y) a sequence by factor a will also scale all distances d,,j by the same factor, thus resulting in scaled distance matrix D' aD. However, distance matrices can easily be made invariant to scaling upon normalization by the Frobenius matrix norm given by
N
N
D P
(di.))2.
(I)
;=1==1
Preprint, Under review.

---

The Frobenius norm of a matrix scaled by a factor a is obtained as
||aD|| F = al a|l,,
(2)
meaning that scale invariance can be achieved upon normalisation by
D
D ||D||
(3)
B Additional Implementation Details
ShapeEmbed is implemented in Python using the PyTorch library (Paszke et al., 2019]. BSD- style license available at ltttp:///githutt is available at
http://orcooooo//mmma//aaa/aaa/a htwp///////annniinnaaaaa. LICENSE), and Experiments were conducted on a machine with an Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz and an NVIDIA A A100 80GB PCIe GPU. In all experiments, we used the ADAM optimizer [Kingma and Ba 2015 with learning rate of 10 for 350 epochs for the datasets we considered. We monitored training using TensorBoard ([Abadi et al./ 2015), Apache-2.0 license) and observed that all the models we trained converged appropriately within the considered number of epochs.
institutional cluster. Runtimes for each experiment are indicated in Section C The creation of distance matrices from segmentation masks was run CPUs (Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz and Intel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz) using a single core per run. All performance metrics were computed on CPUs. For the robustness experiments (Section G.3 segmentation with cellpose was run in GPU mode.
All of experiments were run single GPU
C Additional Details on the Considered Datasets
We provide example images and corresponding masks for the four bioimaging datasets we consider in Figures
[nd The BBBC010 dataset offers relatively simple and well-characterizzd biological benchmark, while the MEF dataset allows us to assess performance on a harder, real-life example where shape information known be essential. The MEF dataset is considered to be a faithful representative of a real bioimaging use-case, as testified by its use as reference in recent works proposing unsupervised frameworks for biological shape analysis (Phillip et al.. 202 Burgess et al. 2024). The HeLa Kyoto dataset is an interesting example of a small microscopy image dataset comprising limited number of highly similar objects, where morphological differences are subtle. Finally, the MOC dataset contains cells exhibiting broad spectrum of morphological variability rangin from smooth to spiky and irregular contours. For this reason, we used a larger distance matrix size (512 x 512) on this dataset to capture details of the cell contours that would otherwise be smoothed out with fewer points.
We provide detailed information on the different datasets used in our experiments in Table I In our experiments, we divided each dataset into an 80% / 20% split for training and testing, respectively, relying on stratified sampling (Särndal et al., 2003] to account for class imbalance. Our model architecture dynamically adjusts to arbitrary input distance matrix sizes that are powers of two by determining the number of upsampling steps required. The specific runtimes for each of the considered datasets were as follows: 49 hours for MNIST, between 4 and hours for MPEG-7, between 4 and 5 hours for BBBCO10, between 7 and 8 hours for MEF, between 22 and 35 minutes for Hela Kyoto, and between and hours for MOC. Taken together, the number of objects contained in each of these datasets and their size distribution (as indicated in Table TD. along with the reported runtimes, provide a sense of how ShapeEmbed scales with dataset size. The latent space dimension is configurable to balance representational capacity and computational efficiency. By default and in all of our experiments, the latent space is composed of 128 dimensions.
The MNIST and MPEG-7 datasets, in their original form, consist of objects that all have roughly the same size and that have been aligned and centered. For our ablation experiments, we constructed modified versions of the MNIST and MPEG-7 datasets that incorporate size variability through random object scaling (referred to as SMNIST and sMPEG-7 the main manuscript), as well as positional and rotational variability through random object translation and rotation (referred to as rMNIST and rMPEG-7 in the main manuscript). As a result, objects in these modified datasets neither appear centered nor aligned in the images and exhibit wide range of different sizes.

---

Table Summary of the considered datasets and associated experimental settings.
DATASET
IMAGE SIZE
NUMBER OF
MAXIMUM OUTLINE
MINIMUM OUTLINE
DISTANCE MATRIX
LATENT SPACE
OBJECTS
SIZE
SIZE
SIZE
SIZE
MNIST
28 X 28
70000
177
31
32
128
MPEG-7
128 X 128
1400
8049
153
64
128
MEF
128 x 128
1407
3201
63
64
128
BBBCO10
64 x 64
26198
409
135
64
128
HELA KYOTO
64
313
248
64
128
MOC
512 x 512
649
3302
59
512
128
D Baselines Implementation Details
D.1 Classical Baselines
We run all classical baselines using as input the same outline points from which build distance matrices for ShapeEmbed.
Region Properties. We extract 19 region properties features that pertain shape using the region- props functionality of the scikit-image library (https://scikitt
org/ MIT license), more specifically the skimage measure module. The 19 measurements we include are all the shape descriptors provided by scikit-image that have no obvious redundancy or localization information. These include the area, convex_ area, perimeter, fris_major... jo__ten,th axis_minor _engthh,, extent, eccentricity. solidity, feret_deeee diameter max, hu hu_ moments, bbox and amount to total to 19 features.
Elliptical Fourier Descriptors. We use the pyefd library lattps://pyedd.rr htctss//pyeedd.redtteedos.. en/latest/ MIT license) to compute Elliptical Fourier Descriptors (EFDs). The Fourier descriptors are extracted using pyefd. elliptic cetpiiiccfuuiieeeeeeeeeeeepe ffffrrirrrr _eti_ddrrrrttorr. where the number of harmonics in the decomposition is controlled by the order parameter. We the order to 30, resulting in 120 coefficients per object as each harmonic generates four coefficients (ana_nay bnb_nbnn cnc_nen, and dnd_ndn). In that way, the EFD feature vector has comparable number of dimensions the ShapeEmbed latent space. To ensure invariance to scale, rotation, and translation, the coefficients are normalized using pyefd. normalize_ed
readthedocs. io/
D.2 Self-Supervised Learning Baselines
We all self-supervised baselines using input the same binary masks from which extract outline points and build distance matrices for ShapeEmbed.
SimCLR. We created SimCLR model with ResNet18 backbone of 128 output dimensions relying on the original codebase (https: //gi thub. com/s https/stalllss//e /esil///////LR//////// MIT license). We trained for 200 epochs (as we did for ShapeEmbed) using the default configuration and set of transforms to create positive pairs.
Masked AutoEncoders (MAE). We benchmarked against the 3 "off-the-shelf" MAE vision trans- CC-BY-NC 4.0 license) configurations ("base"" "large", and "huge", referred to as ViT-b, VIT-I, and ViT-h respectively in our results). We resized the input masks to 224 x 224, the input size expected by MAE by default. We used batch size of 16 (as in the original MAE paper) and 200 epochs (as did for ShapcEmbed), In addition, also benchmarked our method against a version of MAE pretrained on ImageNet- IK by initializing the with the PyTorch checkpoints provided at https:///ggh httptttb.ccoo.ccccoooeeeeeeraare mmae ICC-BY-NC 4.0 license).
formers (https://github
O2VAE. We used the native implementation provided in [Burgess et al. 2024 hhttps://githul
com/ jomb0/ o2vae, MIT license), running the model with recommended hyperparameters to ensure consistency and fairness with the published setup. While 02VAE can incorporate both shape and texture information, we here used binary masks as inputs since we specifically focus on shape in comparison.

---

(a)
(c)
(d)
Figure 1: Sample images from the BBBCO10 dataset illustrating the (a) live and (c) dead experimental conditions, along with their corresponding instance segmentation masks (b and d).
E
Additional Benchmarking Results
In addition to the Fi-scores reported in the main manuscript in Section 4.3, we report in Tables the same results with additional significant digits and without rounding, along with the accuracy. precision, recall, and log-loss score as additional metrics to evaluate the performance of our method on the MNIST and MPEG-7 datasets, respectively. The log-loss score quantifies the quality of probabilistic predictions by penalizing incorrect predictions with high confidence [Bishop] 2006). Higher values of accuracy, precision, and recall, as well as lower log-loss score values, indicate better performance.
The relative performance observed between MAE and ShapeEmbed originates from differences the representations learned by each method and in the confidence of their predictions. MAE, trained to reconstruct masked portions input images, capture lixel-level variations in the binary masks. This can lead to embeddings that support highly confident, low-entropy predictions at the classification step and directly drive down log-loss by assigning high probabilities to the correct class. In contrast, ShapeEimbed encodes structural and geometric properties of the contour through the distance matrix representation, which yields robust and generalizable features that perform well across all reported metries without optimizing for specific one.

---

(c)
(d)
Figure 2: Sample images from the MEF dataset illustrating the (a) control (non-patter patteeraeeddd (c) pacleepatterned, and (c) ftangle-pattered experimental conditions, along with their corresponding instance segmentation masks (b, d, and f).
5

---

(b)
(c)
(d)
(c)
(I)
(h)
Figure Sample images from the HeLa Kyoto dataset illustrating nuclei from the (a) early anaphase, (b) late anaphase. (c) metaphase, (d) prometaphase classes, along with their corresponding instance segmentation masks (e, f, g and h)
(a)
Figure 4: Sample masks generated from the contour coordinates provided in the MOC dataset illustrating the (a) control (non-treated), (c) cytochalasin D-treated (Cytd), and (e) jasplakinolide- treated (Jasp) experimental conditions.

---

Table 2: Additional performance metrics on on the MNIST dataset.
METRIC
F1-SCORE
ACCURACY
PRECISION
RECAL
LOG-LOSS
REGIONS PROP.
0.809 0.003
0.801 + 0.003
01808 0.002
0.807 0.003
0.675 + 0.010
EFD
0.623 0.013
0,621 = 0.001
0,631 + 0.011
0.630 = 0.011
1.005 + 0.001
SIMCLR
0,593 0.011
0.598 + 0.011
0.334 :: 0.012
0.598 = 0.011
1.188 1 0.033
MAE (VIT-B)
0.953 0.026
0.953 = 0.003
0.953 0.003
0.952 = 0.004
0.062 + 0.020
MAE (VIT-L)
0.840 0.009
0.841 + 0.008
0.841 0.009
0.840 0.009
0.520 +: 0.021
MAE (VIT-H)
01848 0.007
0.921 + 0.004
0.919 +: 0.003
0.921 = 0.004
0.369 + 0.018
MAE (VIT-B, PRETRAINED)
0.921 0.009
0.921 + 0.007
0,930 0.010
0.912 = 0.006
0.211 + 0,009
MAE (VIT-L, PRETRAINED)
0.934 0.013
0.935 - 0.013
0.940 0.003
0.929 0.007
0.199 + 0.007
MAE (VIT-H, PRETRAINED)
0.942 0.012
0.941 + 010002
0.946 + 0.004
0.938 0.001
0.204 + 0.017
O2VAE
0.859 0.007
0.859 = 0.008
0.860 + 0.006
0.859 0.008
0.717 + 0.043
SHAPEEMBED
0.963 0.007
0.961 + 0.005
0.964 0.009
0.964 0.008
0.187 0.020
Table 3: Additional performance metrics on the MPEG-7 dataset,
METRIC
FI-SCORE
ACCURACY
PXECISION
RECALL
LOG-LOSS
REGION PROP:
0.701 0.014
0.724 +: 0.002
0.714 0. 0.003
0.688 0.027
1.257 + 0.113
EFD
0.079 + 0.008
0.077 + 0.003
0.076 + 0.006
0.076 = 0.008
2.909 + 0.015
SIMCLR
0,128 0.020
0.141 0.016
0.145 + 0.022
0.141 0.016
22.502 0.522
MAE (VIT-B)
0.616 0.001
4.024
0,660 0.016
0.675
1.071
MAE (VIT-L)
0.627 0.040
0.654 0.007
0.637 + 0.037
0.654 0.037
1.465 + 0.112
MAE (VIT-II)
0,600 = 0.010
0.133
0.000
0.601 = 0.045
1.767 1 0.079
MAE (VIT-B, PRETRAINED)
0.422 0.013
0.436
0.400
0.447 pm0.099
1.887 + 0.084
MAE (VIT-L, PRETRAINED)
0.512 = 0.019
4.000
0.518 + 0.030
0.016
1.723 + 0.082
MAE (VIT-H, PRETRAINED)
0,571 + 0.021
0.674
0,580 0.000
0.565
1.812 + 0.079
O2VAE
0.128 0.020
0.566 +: 0.008
0.538 +: 0.010
0.566 -.008
2.891 + 0.070
SHAPKEMBED
0.751 0.024
0.763 + 0.037
0.716 + 0.002
0.763 = 0.036
1.158 + 0.206
F Additional Ablation Results
F1 Effect of Rotation and Scaling Invariance on the Learned Representation
To complement Section 4.4 of the main manuscript and further qualitatively explore the effect of rotation and scaling invariance on the learned representation, we generated 2D projections of the latent space learned by the vanilla VAE and by ShapeEmbed relying on the t-SNE [van der Maaten and Hinton [008 dimensionality reduction technique. We display the t-SNE projections of the rMNIST latent space in Figure where individual data points are colored according to the class label of their original input image. We observe that the latent representation learned by the vanilla VAE randomly structured and does not allow resolving individual classes. The latent representation learned by ShapeEmbed, however, aggregates data points with similar class labels together, as one would expect the vanilla VAE to behave on the standard MNIST dataset composed of pre-aligned and centered objects. The t-SNE algorithm is used with a random seed of 42 and a perplexity of 5, which are commonly used default parameters.
F.2 Added Value of the Representation Learning Model
Our motivation for encoding distance matrices with VAE is that the ShapeEmbed model will distill these input structures into a compact and highly informative latent representation. To verify that the VAE encoding step genuinely results in better shape descriptors, we compare the classification performance obtained with ShapeEmbed latent codes against that obtained when relying on raw distance matrices directly in our classifier. The results presented in Tables 4
6 and
consistently demonstrate that the latent representation learned by ShapeEmbed allows for better shape discrimination over the MNIST, MPEG-7, BBBCO1O, and MEF datasets.
F.3 Distance Matrix Regularization and Outline Reconstruction
The distance matrix regularization loss terms introduced in Section 3.3 of the main manuscript play an important role in ensuring that the latent space learned by ShapeEmbedd captures the structure of the distance matrices while maintaining reconstruction fidelity. To assess the impact of these custom regularization loss terms on downstream classification performance, we perform ablation experiments in which remove each of the regularization terms one by one. Results the MNIST, MPEG-7,
7

---

2
(a) Vanilla VAE
ShapeEmbed
Figure 5: Projection (t-SNE) of the rMNIST latent space. (a) The latent representation of rMNIST learned by vanilla VAE does not exhibit any noticeable structure and class separation. contrast, (b) the latent representation of rMNIST learned by ShapeEmbed, which ignores orientation and position, recovers clusters of data points that match their underlying class.
Table Effect of the VAE encoding on classification performance. Mean and standard deviation of the F1-score over 5-fold cross- validation... Higher values indicate better performance.
DATASET
DISTANCE MATRICES
SHAPEEMBED
MNIST
0.912 = 0.011
0.963 + 0.007
MPEG-7
0.377 = 0.038
0.751 0.024
BBBC010
0.737 = 0.025
0.866 + 0.008
MEF
0.299 = 0.006
0.746 0.006
Table 5: Effect of the VAE encoding classification performance. Mean and standard deviation of the accuracy over 5-fold cross-validation. Higher values indicate better performance.
DATASET
DISTANCE MATRICES
SHAPEEMBED
MNIST
0.915 + 0.009
0.961 0.005
MPEG-7
0.378 = 0.029
0.763 + 0.037
BBBC010
0.737 = 0.025
0.831 + 0.003
MEF
0.343 = 0.007
0.670 + 0.009
Table 6: Effect of the VAE encoding on classification performance. Mean and standard deviation of the precision over 5-fold cros-validation. Higher values indicate better performance.
DATASET
DISTANCE MATRICES
SHAPPEEMBD
MNIST
0.911 + 0.008
0.964 + 0.009
MPEG-7
0.378 + 0.022
0.716 + 0.002
BBBC010
0.737 = 0.025
0.811 0.011
MEF
0.452 = 0.024
0.674 + 0.004
BBBCO10, and MEF datasets shown in Table 9 illustrate that performance is not heavily affected by the absence of distance matrix regularization terms, which is expected the classification task relies on the latent representations and does not exploit the outline reconstruction.
Reconstructions of a good enough quality may, however, be of strong interest in tasks that focus on generative modeling or inverse mapping from the latent space. This motivates the inclusion of regularization terms, demonstrated in Figure where we visually illustrate the impact of removing the distance matrix regularization loss terms on the quality of the reconstructed outlines in the MNIST and MPEG-7 datasets. Even though downstream classification performance is only negligibly affected

---

Table Effect of the VAE encoding on classification performance. Mean and standard deviation of the recall over 5-fold validatoi........ Higher values indicate better performance.
DATASET
DISTANCE MATRICES
SHAPEEMBED
MNIST
0.911 = 0.007
0.964 + 0. 0.008
MPEG-7
0.379 = 0.020
0.763 = 0.036
BBBCO10
0.734 = 0.026
0.812 + 0.009
MEF
0.343 + 0.007
0.670 0.009
Table Effect of the VAE encoding on classification performance. Mean and standard deviation of the log-loss over 5-fold vross-validation. Lower values indicate better performance.
DATASET
DISTANCE MATRICES
SHAPEEMBED
MNIST
0.371 = 0.018
0.187 + 0.020
MPEG-7
1.201 = 0.013
1.158 0. 0.206
BBBC010
2.889 + 0.686
0.640 + 0.016
MEF
1.202 + 0.066
0.801 + 0.019
Table Effect of the distance matrix regularization loss terms on classification performance. Mcan and standard deviation of the Fl-score over 5-fold cross-validation. Higher values indicate better
performance.
METHOD
MNIST
MPEG-7
BBBC010
MEF
No SYMMETRY
0,941 + 0.006
0.536 0.057
0.769 + 0.015
0.681 + 0.034
No DIAGONAL
0.952 + 0.005
0.604 + 0.067
0.801 0.022
0.733 - 0.044
NO NON-NEGATIVITY
0.923 + 0.007
0.581 0.037
0.789 0.011
0.701 0.052
SHAPEEMBED
0.963 + 0.007
0.751 = 0.024
0.831 = 0.003
0.760 = 0.008
by the ablation of these regularization terms, the reconstructions without regularization are noticeably degraded.
As ShapeEmbed relies on a VAE model, it allows for reconstructing outlines from their latent codes, but can in addition also generate outlines from vectors that have been sampled in the latent space. We illustrate in Figure examples dataset along with the mean outline of each of these classes. To generate the mean outlines, we compute the average latent representation of each class and decode it back into an outline in image space. Specifically, for each unique label in the dataset, we extract the latent vectors of all objects having that label and compute their mean. This averaged latent vector then sent through the model's decoder to reconstruct the mean outline. We observe that the mean outlines correctly match the intuition of the average shape of each class, despite the various sizes and orientations of individual objects within the classes.
randomly-picke
samples from different classes of the MPEG-7
Furthermore, we can also generate outlines by randomly sampling the latent space. To do so, we retrain our model and set the hyperparameter 3 to 10-5, thus enforcing smoother and more structured latent space that aligns with normal prior. We then generate random latent vectors by sampling from a normal distribution with mean of zero and a standard deviation of one. The sampled vectors are finally sent through the decoder to reconstruct their corresponding outlines. We illustrate in Figure & exxamlees of outlines generated in this manner from the latent space learned by Emepe on the MNIST dataset. We observe that the generated outlines form plausible shapes of the different digits present in MNIST.

---

(a)
(b)
(d)
Figure 6: Effect of distance matrix regularization outline reconstruction. For a randomly-picke sample the MPEG-7 and MNIST datasets, we show the original outline (a and d), the reconstructed outline without any of the distance matrix regularization terms (b and e), and the reconstructed outline with all distance matrix regularization terms (c and f).
G
Additional Biological Imaging Results
G.1 Additional Performance Metrics
In addition to the Fi-scores reported in the main manuscript in Section 4.3, we report in Ta- bles [0]
12 and 13 the same results with additional significant digits and without rounding, along with accuracy, precision, recall, and log-loss score as additional metrics evaluate the performance of our method on the BBBC010, MEF, HeLa Kyoto, and MOC datasets, respectively. As space allows, we here include additional significant digits the results we report. Higher values of accuracy. precision, and recall, as well as lower log-loss score values, indicate better performance. As in the main manuscript, we include results obtained with ShapeEmbed directly, as well as those obtained by adding size (i.e., the distance matrix norm) back motivated by the fact that, in biological imaging, size invariance may either be a crucial or entirely irrelevant feature depending conditions (such as varying magnifications) but undesired when size differences are biologically meaningful (such as varying growth rate).
extra feature. This flexibility is
the context: it is necessary when size differences arise from imaging
important to note that BBBCO10 is not pure shape dataset: it originally also includes intensity and texture information. We here do not use BBBCO10 to demonstrate that we provide the best results possible when attempting to classify this dataset, but instead show that we can learn, without any supervision, good representation of the biological shapes it contains in way that allows for
10

---

Randomly-picked samples
Mcan outline
Apple
Butterfly
Phone
Fork
Figure 7: Mean outline generation in the MPEG-7 dataset. We illustrate 3 mundomly-pickerr samples in the classes of the MPEG-7 dataset (first three columns), along with the mean outline obtained by decoding the vector corresponding to the average of each of the classes in the ShapeEmbed latent space (last column).
6
(a)
(b)
(c)
(d)
Figure 8: Novel outline generation in the MNIST dataset. We illustrate 4 outlines obtained by randomly sampling vectors from a normal distribution in the ShapeEmbed latent space that are subsequently reconstructed through the decoder path.
the unbiased exploration of the data. This motivates the comparison against shape-only baseline (Region Prop.) in Table TU] opposed to the method proposed in Wahlby et al. [2012), which reports higher classification accuracy on this dataset by leveraging intensity and texture features.
We observe that region properties obtain lower log-loss on the BBBCO10 dataset despite underper- forming according to all other metrics relative to the version of ShapeEmbed that includes object size. The log-loss assigns higher penalties for incorrect predictions that have high certainty. From this, we hypothesize that ShapeEimbed obtains a higher F1-score because the latent codes it produces result in more confident and more correct predictions, also overconfident errors. As consequence, SmapeEmbed obtains slightly higher log-loss when compared region properties.

---

Table 10: Additional performance metrics on the BBBC010 dataset.
METRIC
F1-SCORE
ACCURACY
PRECISION
RECAL
LOG-LOSS
REGION PROP.
0.821 0.002
0.824 +: 0.022
01826 0.021
0.824 0.022
0.412 + 0.027
EFD
0.517 = 0.038
0.579 : 0.031
0.574 + 0.039
0.579 = 0.031
0.682 + 0.009
SIMCLR
0.562 + 0.117
0.567 + 0.115
0.369 +: 0.119
0.567 = 0.115
0.762 + 0. 125
MAE (VIT-B)
0.597 0.119
0.628 - 0.105
0.633 1 0.107
0.625 00:000.0 0.105
0.716 + 0.156
MAE (VTT-L)
0.514 0.072
0.720 0.580
0.723 0.058
0.721 0.060
0.632 +: 0.082
MAE (VIT-H)
0.718 = 0.059
0.657 + 0.081
0.671 +: 0.090
0.657 = 0.081
0.649 + 0.083
MAE (VIT-B, PRETRAINED)
0.619 = 0.081
0.052 : 0.017
0.654 1: 0.005
0.057 0.009
0.711 + 0.075
MAE (VIT-L, PRETRAINED)
0.771 0.031
0.773 0.002
0.790 0.006
0.650 +: 0.045
MAE (VIT-H, PRETRAINED)
0.762 0.059
0.768 + 01008
0.765 + 0.022
0.764 + 0.007
0.699 + 0.017
O2VAE
0,005 0.080
0.602 = 0.071
0.630 + 0.084
0.624 0.712
0.661 + 0.020
SHAPEEMBED
0.831 0.003
0,831 + 0.003
0.811 0.011
0.812 0.009
0.640 0.016
SHAPREMEED + SIZR
0.871 0.008
0.872 + 0.015
0.866 + 0.009
0.866 0.009
0.509 + 0.136
Table 11: Additional performance metrics on the MEF dataset.
METRIC
F1-SCORE
ACCURACY
PRECISION
RECALL
LOG-LOSS
REGION PROP.
0.722 + 0.006
0.722 + 0.006
0.732 + 0.007
0.723 = 0.008
0.649 + 0,011
EFD
0.326 0.041
0.403 + 0.002
0.324 + 0.005
0.353 0.008
1.051 + 0.001
SIMCLR
0.434 0.031
0,414 + 0.029
0.451 0.032
0.444 0.029
1.11 1 0.020
MAE (VIT-B)
0.537 0,030
0,537 + 0.031
0.539 + 0.030
0.546 = 0.029
0.895 + 0.021
MAE (VTT-L.)
0.532 0.019
0.535 + 0.019
0.534 + 0.020
0.535 0.018
0.885 + 0.028
MAE (VIT-H)
0.549 0.023
0.352 0.024
0.549 0.023
0.830 + 0.034
MAE (VIT-B, PRETRAINED)
0,611 = 0.013
0.619
0.609 +: 0.005
0.626 = 0.001
0.702 + 0.008
MAE (VIT-L, PRETRAINED)
0,613 = 0.024
0.651
0.000
0.015
0.688 1 0.033
MAE (VIT-II, PRETRAINED)
0.651 0.009
0.07
0.656 0.002
0.653 0.006
0.609 + 0,033
O2VAH
0.527 0.023
0.527 + 0.024
0.528 + 0.023
0.527 0.024
1.191 + 0.108
SHAPEEMBED
0.670 = 0.009
0.670
0.674 + 0.004
0.670 0.009
0.801 + 0.019
SHAPEEMBED + SIZE
0.760 = 0.008
0.755 0.006
0.751 + 0.006
0.753 = 0.005
0.640 + 0.016
Table 12: Additional performance metrics on the Hel.a Kyoto dataset.
METRIC
FI-SCORE
ACCURACY
PRECISION
RECALL
LOG-LOSS
REGION PROP.
0.569 + 0.076
0.578 + 0.066
0.554 + 0.050
0.594 + 0.003
0.782 + 0.095
EFD
0.221 0.110
0.255 0.125
0.223 1 0.103
0.221 1 0.100
2.003 : 0.036
SIMCLR
0.482 0.117
0.509 0.013
0.470 0.015
0.494 + 0.009
1.069 = 0.044
MAE (VIT-B)
0.472
0.488
0.001
0.471
0.002
0.477 + 0.003
1.002 0.101
MAE (VIT-L)
0.422 0.009
0.435 0.008
0.422 0.008
0.419 : 0.009
1.053 = 0.099
MAE (VIT-H)
0.444 +.04
0.449 + 0.007
0.446 = 0.001
0.444 + 0.001
1.042 = 0.008
MAE (VIT-B, PRETRAINED)
0.721 1 0.002
0.728 1 0.005
0.721 1 0.002
0.728 1 0.005
0.659 - 0.011
MAE (VIT-L, PRETRAINED)
0.761 + 0.003
0.770 = 0.011
0.761 = 0.003
U.765 1 0.001
0.699 = 0.017
MAE (VIT-H, PRETRAINED)
0.732 1 0.001
0.741 0.017
0.732 1 0.001
0.733 1 0.004
0.653 1 0.005
O2VAE
0.630 0.082
0.639
0.025
0.630 1 0.082
0,635 1 0.062
0.699 1 0.018
SHAPEEMRED
0.801 0.041
0.811 0.015
0.801
0.041
0.810 + 0.361
0.639 0.026
SHAPKEMBKD SIZE
0.854 1 0.044
0.854 0.010
0.854 0.044
0.853 + 0.024
0.601 0.013
Table 13: Additional performance metrics on the MOC dataset.
METRIC
F1-SCORE
ACCURACY
PRECISION
RECALL
LOG-LOSS
REGION PROP.
0.604 0.074
0.606
0.011
0.599 1 0.034
0.609 1 0.074
0.816 1 0.007
EFD
0.382 0.021
0.386
0.106
0.377 + 0.021
0.387 + 0.008
1.093 = 0.014
SIMCLR
0.650 + 0.014
0.653
0.022
0.647
0.014
0.653 + 0.011
0.721 0.065
MAE (VIT-B)
0.571 0.003
0.581
0.007
0.566 0.003
0.576 + 0.002
0.788 0.100
MAE (VIT-L)
0.598 + 0.001
0.601 + 0.002
0.596 0.003
0.600 + 0.001
0.746 = 0.054
MAE (VIT-H)
0.513 1 0.001
0.531 1: 0.006
0.504 0.006
0.522 1 0.001
0.965 = O 00066
MAE (VIT-B, PRETRAINED)
0.481 + 0.007
0.499 = 0.004
0.472 0.007
0.472 + 0.007
1.003 = 0.034
MAE (VIT-L, PRETRAINED)
0.664 1 0.002
0.672 0.002
0.660 1 0.005
0.668 1 0.002
0.622 0.022
MAE (VIT-H, PRETRAINED)
0.631 0.051
0.638
0.003
0.628
0.634 0.021
0.686 1 0.076
02VAE
0.595 0.026
0.599 0.003
0.593
0.026
0.590 + 0.016
0.816 0.066
SHAPEEMBED
0.682 1 0.012
0.699 0.004
0.673 0.012
0.683 1. 0.009
0.649 0.008
SHAPEEMBED SIZE
0.711 + 0.002
+.09+
0.707 + 0.011
0.631 = 0.001
12

---

G.2 Qualitative Exploration of the Latent Space
Further to quantitative classification results, we also qualitatively explore the latent space learned by ShapeEmbed on the BBBC010 and MEF datasets through the 2D t-SNE projection displayed in Figures 9 TO and obtained with the same parameters as Figure
In Figure 9 individual data points are colored according to the class label of their original input image in the BBBCO10 dataset, which is either dead or alive. In BBBCO10, labels have been derived from experimental conditions (whether the sample has been treated by lethal substance not), When dead, C. elegans nematodes straighten to look like a rod, while they swim sinusoidally and curve when alive. ShapeEmbed is not only able to identify these distinct shape populations without supervision, but upon inspection of the structure of the latent space, it also reveals instances where experimental labels do not align with biological reality. Several of the "misclassified" data points indeed correspond to mislabeled nematodes that are either alive despite having been treated or dead despite being untreated. This interesting finding illustrates that classification performance alone is not a good indicator of the ability to distinguish between biological states in the BBBC010 dataset, as some nematodes labeled live appear to be dead and vice versa.
dead
alive
Figure 9: Projection (t-SNE) of the BBBCO10 latent space learned by ShapeEmbed. Data points are grouped according to their corresponding classes, namely dead (straight rods) and live (curved worms) nematodes. A closer inspection of data points that seem be misplaced reveals that their associated class label does not reflect their actual shape.
In Figure I0] individual data points are colored according the class label of their original input image in the MEF dataset, which is either "circle", "triangle". or "control". The MEF dataset contains images of cells that were cultured on fibronectin micropattern surfaces to enforce cell shape constraints, as described in Hale et al. (20TT). This real use case illustrates the challenge of untangling individual biological variability from experimental variability and has been the dataset of choice in recent papers proposing unsupervised frameworks for biological shape analysis (Phillip) al. 2021 Burgess et al. 2024). ShapeEmbed is capable of separating the cells grown on circle- and mniangle-pateennn cultured on patterned surface, adopt diverse shapes that overlap with the two other classes. This confirmed by closer inspection of the masks: cells cultured on an unpatterned surface may naturally exhibit the whole range of shapes observed in cells cultured on patterned surfaces. This observation illustrates that relying in biology often provides richer information than relying on the output of a classifier, and is therefore more likely to provide insights into the underlying mechanisms at play.
surfaces, but in addition reveals that cells in the control class, which are not
structured latent space to investigate the distributions of shape phenotypes
These two practical examples highlight the value of ShapeEmbed as method explore and discover shape variations in fully unsupervised manner to investigate, untangle, and understand complex shape variations in biological experiments. Having methods that allow such an unbiased exploration of the distribution of biological shapes can be valuable in many settings, from assessing the efficacy of drug treatments to analyzing biopsies, where cell type identification must ideally be carried out without prior knowledge potentially biased manual annotation.
13

---

Circle
Control
Triangle
Figure 10: Projection (1-SNE) of the MEF latent space learned by ShapeEmbed. Data points grouped according to the shape of the micropatterned Fibronectin-coated surface on which cells have been grown how they have been micropatterned. Cells in the control class have been seeded on surface devoid of any micropattern. The control cells appear to adopt heterogeneous shapes that cover the range morphologies exhibited by cells grown on both the triangle- and circle-patterned surfaces.
G.3 Robustness to Segmentation Quality
The results presented in the main manuscript rely on clean segmentation masks as ShapeEmbed aims to learn representation of shape information. Although image segmentation has received compar- atively much more attention in the microscopy image analysis literature than shape representation learning [Lucas et al., 2021]. it remains a challenging problem. To reassure about the usefulness of ShapeEmbed real-world applications where clean segmentation masks may be difficult to obtain, we demonstrate in the following that ShapeEmbed is able to accurately encode shape information even when poor image quality results poor segmentation quality.
To carry out these experiments, we synthetically degraded the MEF dataset with various levels of image noise, simulating combination of Poisson shot noise and Gaussian readout noise as encountered in microscopy image acquisition. Given ground truth image 1, we first normalize it to the (0, 1] range and denote the normalized image Inom- To account for the shot noise, we then scale this image by factor E {1. 10, 100, 1000} and sample each pixel value from a Poisson distribution using À- = p- Inorm- yielding Ip. To account for the readout noise,
add ixxel-independent Gaussian noise with standard deviation y, resulting in the degraded image JPG. To reverse the intensity scaling and make a comparison to I meaningful, we divide TPG by p. resulting the final degraded image naisy- = TpG/p. /p. To evaluate the level of degradation of the image, we compute the peak tignal-tonoiis. ratio (PSNR) between Inoisy and the normalized ground truth image Inom- This procedure allows us to control the noise severity via the scaling factor p and construct four progressively degraded datasets corresponding to = 10000, 100, 10, 1, which we refer to as "low", "mid-low", "mid-high", and "high" noise levels, respectively. In this setup, changing the scaling factor corresponds to recording a microscopy image with different exposure or laser power to control the amount of available light.
We then segment individual object instances in the degraded MEF dataset with high, mid-high, mid-low, and low noise levels using the state-of-the-art microscopy image segmentation deep learning algorithm cellpose [Stringer et al., 2021] keeping all parameters at their default value. From the instance segmentations obtained with cellpose, we create binary masks for each individual objects and extract object contours using the find_contoury function from the scikit-image library
14

---

(d)
(h)
Sample images from the "circle" class of the MEF dataset illustrating the various noise levels (top row) and the resulting instance segmentations (bottom row) obtained with cellpose. High (a and e), mid-high (b and f),, mid-low (c and g), and low (d and h) noise. Examples of original, noise-free images from the MEF dataset can be found in Figure
Figure 11:
MIT license). Each contour is resampled fixed number of points (64 for the MEF dataset) using the splprep and splev spline interpolation functions of the scipy library (Virtamen et al. 2020]. BSD-3-Clause license). We finally compute Euclidean distance matrices from the resampled contour points input them ShapeEmbed.
(https://scikitt-iii httsssiiiiiiimmme..toe
We provide illustrative images of the different noise levels and of the resulting segmentation results in Figure 11. and a summary of the dataset we generate in Table 14
Table 14: Summary of the noise degraded versions of the MEF dataset.
DATASET
POISSON
GAUSSIAN STANDARD DEVIATION
PSNR
NUMBER OF OBJECTS
IOU
SCALING P
HIGH
0.9
0.345
2071
0.082
MID-HIGH
10
0.9
17.249
17026
0.649
MID-L.WW
100
0.9
29.536
19299
0.698
Low
1000
0.9
39.860
19762
0.702
NONE (GT)
NONE
NONE
NONE
16381
We measure the performance of ShapeEmbed on a downstream classification task as described in Section 4.2 of the main manuscript and report the FI-score metric. In [2] we report the and standard deviation the classification results on the four considered noise conditions (high, mid-high, mid-low, and low), well on the original, non-degraded dataset. We observe that even though performance increases as the noise level decreases as we would expect, ShapeEmbed performs well (Fl-score< 0.7) as soon segmentation quality is decent, which already happens the mid-low noise level. Interestingly, ShapeEmbed achieves excellent performance even in the highest level of noise as, in this condition, the images are so bad (PSNR 0) that only a very small subset of objects (2071 out of 16381, amounting to 12%) are segmented and systematic class-specific segmentation emrors make straightforward to distinguish different classes resulting in a comparatively simpler classification problem. The reported performance in high noise conditions is therefore more reflecting of segmentation artefacts than of Smaed'' performance. We nevertheless decided include for the sake of completeness.
15

---

0.76
0.74
F1 Score
0.68
none
low (39.860)
mid-low (29.536)
mid hid-hiihh (17.249)
high (0.345)
(PSNR)
Figure 12: Fl-score across varying levels of noise levels degrading the MEF dataset. The x-axis reports qualitative noise levels corresponding to increasing Poisson scaling factors (none to high), while the y-axis reports the mean Fl-score across the MEF dataset degraded with varying noise levels. Red dots correspond to the mean Fl-score, with black lines indicating the standard deviation.
G.4 Generative Properties
ShapeEmbed's generative properties are particularly interesting for bioimaging applications, as they make it possible qualitatively explore the shape distribution present the dataset by calculating statistics or carrying out sampling in the latent space. We illustrate in Figure 13 three examples of contours aandomly-picke
among the data points of the two classes of the BBBCO10 dataset, along with the mean outline of each of these classes (generated as in F.3 While the first three outlines illustrated in each row of Figure 13 are the contours of objects that were present in the original BBBCO10 dataset, the mean outline does not correspond to a data point and is obtained thanks to the generative properties and decoding capabilities of our model. We observe that mean outlines accurately capture the shape signatures expected from each of the classes: straight for the dead class, and curved for the live class.
Randomly-picken samples
Mean outline
Alive
Dead
Figure 13: Mean outline generation in the BBBC010 dataset. We illustrate data points randomly picked among the two classes BBBC010 dataset (first three columns), along with the mean outline obtained by decoding the vector corresponding to the centroid of each of the classes in the ShapeEmbed latent space (last column).
We further depict Figure 14 examples outlines generated by randomly sampling the latent space learned by SmapeEEmeed on BBBC010. The vectors were sampled and decoded into outlines as in F3 observe that the newly-generated outlines, which do not correspond to any "real" data point, are similar to the shapes present in the original BBBCO10 dataset and look like plausible C. elegans
16

---

nematode outlines. This kind of generative capability can be of strong interest for the shape-aware generation of synthetic data as well as for hypothesis testing.
(a)
(b)
(c)
(d)
Figure 14: Novel outline generation in the BBBC010 dataset. We illustrate 4 outlines obtained by randomly sampling vectors from a normal distribution in the ShapeEmbed latent space that are subsequently reconstructed through the decoder path.
H Application to open curves
While ShapeEmbed built for closed contours, it can in principle be applied to open curves as well, as the distance matrix representation does not require closed contour. In fact, applying the approach to open contours simplifies the problem, as it leaves only two possible starting points to index the contour. As result, the circular padding implemented in the encoder is longer required and the indexation- -nixaationiivariant loss can be simplified to only consider two possible indexations instead of 2NV for a contour composed of N points.
17

---

References
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, seig--erformance deep learning library. In Advances Neural Information Processing Systems 32, pages 8024-8035.
Curran Associates, Inc., 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICIR), San Diego, CA, USA, 2015. URL
aataa/////111/ www.oogg////aaeeeeeo arxia. www.xiivorr///ooo
6980
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL httph:ttp///// http::://wwww.. www.settssoooo...oo org/ Software available from tensorftow.org.
Jude M. Phillip. Kyu-Sang Han, Wei-Chiang Chen, Denis Wirtz, and Pei-Hsun Wu. A robust unsu- pervised machine-learning method quantify the morphological heterogeneity of cells and nuclei. Nature Protocols, 16(2):754-774, 2 2021. ISSN 1750-2799. doi: 10.1038//41159--00-000000
James Burgess. Jeffrey J Nirschl, Maria-Clare Zanellati, Alejandro Lozano, Sarah Cohen, and Serena Ycung-Levy. Onientation-invaaiinn autoencoders learn robust representations for shape profiling of cells and organelles. Nature Communications, 15(1):1022, 2024. doi: 10.1038/
44144--424443362.4
Carl-Erik Särndal, Bengt Swensson, and Jan Wretman. Model assisted survey sampling. Springer Science & Business Media, 2003.
Christopher M. Bishop. Pottern Recognition and Machine Learning- Springer, New York, NY, 2006. ISBN 9780387310732 URL https://tww: wwwwwwwwpwgg...co www.aangrrr.ooe//oooo htt.m//g//boo////708377777117770 /780987770777773
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579-2605. 2008.
Carolina Wählby, Lee Kamentsky, Zihan Liu, Tammy Riklin-Raviv, Annie L Conery, Eyleen O'rourke, Katherine L Sokolnicki, Orane Visvikis, Vebjom Ljosa, Javier E Irazoqui, al. An image analysis toolbox for tighthhruughput c. elegans assays. Nature methods, 9())714-716. 2012.
Christopher M Hale, Wei-Chiang Chen, Shyam Khatau, Brian R Daniels, Jerry SH Lee, and Denis Wirtz. Smrt analysis of mtoc and nuclear positioning reveals the role of eb1 and liel in single-cell polarization. Journal of cell science, 124(24):427744285. 2011.
Alice M Lucas, Pearl V Ryder, Bin Li, Beth A Cimini, Kevin W Eliceiri, and Anne E Carpenter. Open-source deep-lcarning software for bioimage segmentation. Molecular Biology of the Cell, 32 (9):823-829, 2021.
Carsen Stringer, Tim Wang, Michalis Michaelos, and Marius Pachitariu. Cellpose: a generalist algorithm for cellular segmentation. Nature methods, 18(1):100- 106, 2021.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland. Tyler Reddy, David Cournapeau,
Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Ker,, Eric Larson, C Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-279 2020. doi: 00.1038/4415922 192--199------6666
18


Reviews:
[{"review_id": "ODufXGSYel", "reviewer": "Reviewer_Kna5", "summary": "This paper introduces ShapeEmbed, a self-supervised framework for learning transformation-invariant 2D shape representations from binary masks. The key idea is to represent an object's shape via its Euclidean distance matrix over uniformly sampled contour points, which is inherently invariant to translation, rotation, and reflection. To achieve scale and contour-index invariance, the authors normalize the matrix and modify a convolutional variational autoencoder (VAE) to include circular padding and reflection-averaged encoding. Furthermore, they define a symmetric reconstruction loss that accounts for all cyclic permutations and mirrored versions of the input, ensuring that the learned latent shape descriptors are fully invariant. The framework is validated across synthetic and real-world datasets, including biological microscopy images, showing strong performance in downstream shape classification and generative applications.", "questions": "1. Section 3.2 states that the encoder outputs a shifted latent representation, whereas Section 3.3 claims that the encoder is invariant to indexation. However, a shifted output and indexation invariance are not necessarily equivalent concepts. Could the authors clarify this apparent discrepancy and elaborate on how indexation invariance is formally ensured?\n\n2. The proposed framework seems to rely on high-quality segmentation masks. However, in real-world scenarios, segmentation masks often contain noise. How robust is your method to such noisy inputs? Have you evaluated its sensitivity to imperfect masks?\n\n3. In Section 3.3, you introduce several regularization terms to encourage the output distance matrix to be non-negative, symmetric, and have zero diagonal. However, these soft constraints do not strictly guarantee that the reconstructed matrix satisfies all distance matrix properties. Have you considered constructing a distance matrix that is analytically guaranteed to meet these constraints, possibly through a parameterization rooted in mathematical structure (e.g., analogous to how quaternions can be used to ensure valid rotation matrices, or how eigen-decomposition can yield symmetric matrices)?", "limitations": "Yes, the authors clearly acknowledge the limitation to simply-connected 2D shapes and suggest potential 3D extensions and nested shape modeling as future work.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**. \n\nThe paper addresses an important and well-defined problem of learning transformation-invariant shape representations in a self-supervised manner. The key contribution lies in leveraging the distance matrix as an input modality, combined with architectural and loss-based mechanisms to enforce invariance to translation, rotation, scaling, reflection, and contour reindexing. The methodology is elegant, mathematically sound, and requires minimal pre-processing. Compared to prior work like O2VAE and SimCLR-based mask encoders, ShapeEmbed demonstrates superior downstream performance on a range of datasets, especially in challenging biological settings where shape is semantically meaningful.\n\n**Weaknesses**\n\nOn the downside, while the paper is thorough in its formulation and evaluation, the scope of the method is limited to simply connected 2D shapes with single contours. It cannot directly handle shapes with holes, disconnected components, or 3D geometries, which limits applicability to more general vision problems. Although experiments cover a reasonable range of datasets and ablations, some aspects—such as the dependence on contour sampling resolution, alternative normalization schemes, or quantitative contour reconstruction fidelity—could be further explored to strengthen the generality and rigor of the evaluation. The reliance on Euclidean distance matrices also may pose computational challenges when scaling to high-resolution contours.", "was_revised": true, "final_justification": "The authors have provided clear and satisfactory responses to my main concerns. Their clarification on the encoder’s shift-invariance resolves the confusion around indexation, and the revised explanation is both technically accurate and well-formulated. Their robustness experiments—though only in the supplement—demonstrate practical applicability under noisy masks. I also appreciate their thoughtful discussion on enforcing valid distance matrices and their openness to future improvements. These clarifications strengthen the case for the method’s reliability and broader potential. I am raising my overall score to 5 accordingly.", "rebuttal": "## Limitation of the method to simply-connected 2D shapes with single contours\nThe reviewer points out that ShapeEmbed cannot directly handle shapes with holes or disconnected components. Although it is entirely true that shapes with holes are not natively handled by ShapeEmbed, the external outline of an object with holes can be sufficient to describe its shape with no major impact on performance, as is the case with many of the classes in the MNIST dataset (0, 4, 6, 8, and 9). While ShapeEmbed does currently take a single distance matrix as a shape descriptor, it could be modified in the future to allow its input tensor to include additional channels for additional shape components in the same way font glyph descriptors embed several paths to draw letters with holes or disconnected components.\nRegarding 3D geometries, we are currently working on a follow-up model adapted to 3D surfaces, as further elaborated in our response to reviewer pLNK.\n\n## Shifted output and indexation invariance in ShapeEmbed’s encoder.\nThe reviewer asks about the properties of the encoder and writes that “a shifted output and indexation invariance are not necessarily equivalent concepts”. We agree with this. The reviewer’s comment helped us identify that the way we explained the properties of our encoder may cause confusion.\nTo elaborate further on this point: choosing a different indexation (i.e. using a different location along the contour as starting point) corresponds to diagonally shifting the input matrix. This effect can be observed in Figure 2 by comparing panels a,b with panels e,f. In Section 2.3, we write that convolutional layers are **equivariant with respect to shifting**, that is, a shifted input (caused, for instance, by choosing a different starting point along the contour) will result in an identical but shifted output. However, the encoder as a whole does not only contain convolutional layers, but also pooling operations. If we now consider a combination of convolutional layers and subsequent global pooling operations, the resulting network will be perfectly **shift invariant**, meaning all shifted versions of an input will result in identical outputs irrespective of how they have been shifted. As we discuss lines 192 to 196, our architecture does not however ensure perfect shift invariance since it uses non-global pooling and strided convolutions. In practice, we find it to provide a sufficient level of invariance as elaborated in Section 4. We propose to modify the following text in Section 4, line 190 as follows to clarify:\n\n*As a result, the convolutional layers are shift-equivariant and produce equal but shifted outputs for all possible distance matrix indexations (starting points). By combining convolutions with a subsequent global pooling operation, we could in principle achieve true shift-invariance. However, ResNet-18 reduces the tensor size in multiple steps using local pooling operations and strided convolutions. Strictly speaking, …*\n\n## Robustness of the model to noise.\nThe reviewer asks for an evaluation of ShapeEmbed’s robustness to noisy segmentation masks. Our Supplementary Material Section G, entitled “Additional Biological Imaging Results”, contains a subsection G3 entitled “Robustness to Segmentation Quality” that is of relevance to address this question. There, we demonstrate ShapeEmbed’s ability to accurately encode shape information even with poor quality, noisy images, resulting in low quality segmentation masks that mimic challenging real-world applications. We synthetically degraded the MEF dataset considered in our manuscript with various levels of image noise, simulating a combination of Poisson noise and Gaussian readout noise as commonly encountered in microscopy image acquisition. Supplementary figure 9 shows examples of images and resulting segmentation masks after degradation with noise. Supplementary table 12 presents a summary of the image statistics of the noise-degraded versions of the MEF dataset. Supplementary Figure 10 reports ShapeEmbed’s performance, measured on the downstream classification task described in the main manuscript (Section 4.2), with the F1-score reported as a metric. These experiments illustrate how noisy data would affect ShapeEmbed’s latent representation quality and are currently referred to in the main manuscript (Section 4.5). Although we could not include these experiments in the main manuscript due to space constraints, we would be open to referencing them more explicitly if deemed useful.\n\n## Ensuring the validity of predicted distance matrices.\nWe very much appreciate the reviewer's suggestion to ensure the prediction of valid distance matrices not via extra loss functions but via analytical guarantees (as outlined for instance in[1]). We see multiple avenues toward achieving this. Possibilities include ideas outlined in[2] , or predicting point coordinates of an arbitrarily shifted and rotated contour, which can then be used to compute the distance matrix to be used in our loss function. While we attempted to implement this early on, we encountered difficulties in training our model with such a hard constraint. In contrast, we empirically observed that training our model with the regularizers we are now using (which are indeed not hard constraints) was comparatively much simpler and sufficient to yield excellent performance. We have continued to explore this area of research and have achieved encouraging results since the submission of our manuscript, but we still consider it to require further exploration and to be beyond the scope of ShapeEmbed as presented here. We would however like to mention this as a possible area of future work in our conclusion section as follows:\n\n*Our current implementation uses several regularization terms in the loss to promote the decoded outputs to be valid Euclidean distance matrices, but lacks strict guarantees that the distance matrix constraints are satisfied. An exciting area for future research could be to instead parameterize the predicted matrix in such a way that distance matrix properties are guaranteed to be valid [1] .*\n\n\n[1]: Dokmanic, Ivan, et al. \"Euclidean distance matrices: essential theory, algorithms, and applications.\" IEEE Signal Processing Magazine 32.6 (2015): 12-30.\n\n[2]: Hoffmann, Moritz, and Frank Noé. \"Generating valid Euclidean distance matrices.\" arXiv preprint arXiv:1910.03131 (2019)."}, {"review_id": "cZ1O8Tf8ti", "reviewer": "Reviewer_VYfD", "summary": "This paper investigates the problem of accurately describing object contours in 2D images and proposes a novel method called ShapeEmbed. Based on a learning-based framework, ShapeEmbed learns a shape representation that is invariant to translation, scaling, rotation, reflection, and point indexing, and applies it to downstream tasks. Experiments are conducted to validate the effectiveness of the proposed method.", "questions": "Please refer to the weaknesses section, and in particular, highlight the originality of your work—rather than presenting it as an incremental contribution (i.e., a mere combination of existing modules). Emphasize whether your approach addresses a previously unsolved challenge.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is clearly written and easy to understand.\n\n2. This paper focuses on a core challenge in object shape modeling, and correspondingly proposes a shape descriptor that is invariant to translation, scaling, rotation, reflection, and point indexing.\n\n3. The effectiveness of the proposed shape descriptor is validated through experiments.\n\nWeaknesses:\n\n1. The novelty of the paper is not clearly articulated. The core challenges in shape representation—such as achieving invariance to translation, scaling, and rotation—have been long-standing problems addressed by many prior works. The use of a distance matrix to achieve these invariances is not particularly surprising, and similar approaches have been explored previously (as the authors themselves acknowledge). Then, what's the novelty and contributions? \nIf the key contribution lies in addressing the indexation invariance problem for closed contours, this should be clearly emphasized in the Introduction.\n\n2. Has the issue of indexation invariance truly not been addressed in prior work? It is unclear whether the authors have conducted a sufficiently comprehensive review of related literature on this aspect.\n\n3. There is a notable performance gap between the baselines on MNIST and MPEG-7. The authors should provide a clearer comparison of these two datasets, highlighting their differences in structure and task complexity during dataset description.\n\n4. I'm not an expert in shape representation, but the number of datasets used in the experiments seems relatively limited. Are there additional benchmark datasets or comparison methods that could further support the generality and robustness of the proposed approach?", "was_revised": true, "final_justification": "The authors do not address my concerns w.r.t. novelty and contribution. I will keep my original score.", "rebuttal": "## Clarification of the novelty and contributions.\nAs correctly stated by the reviewer and acknowledged in our manuscript, we are not the first to propose a distance matrix representation for shapes. However, to the best of our knowledge, we are the first to utilise distance matrices representing closed contours to derive a learned latent representation of shape enabling us to achieve superior accuracy in downstream classification tasks and opening the door to generative shape modelling (see Supplementary Figures 6 and 12). As the reviewer correctly states, one of our key contributions is to propose a novel solution to the problem of indexation invariance. In addition to the discussion we currently have on lines 58-67 of our manuscript, we propose to also state this more explicitly in the final version of our contribution list as follows:\n1. We introduce, **to the best of our knowledge, the first self-supervised** representation learning model that learns shape descriptors from distance matrices. The descriptors are, by design, invariant to scaling, translation, rotation, reflection, and re-indexing.\n2. We are, **to the best of our knowledge, the first** to propose a solution to achieve indexation invariance in a VAE architecture for shape description based on a padding operation in the encoder, operating jointly with a new loss function.\n3. We show that our method outperforms the representation learning state-of-the-art and classical baselines on downstream shape classification tasks. \n\n\n## Prior work on indexation invariance.\nAfter extensive literature research, we were unable to identify previous methods using distance matrices to derive a learned latent representation of shape, or methods that address indexation invariance with self-supervised learning models in the context of shape quantification. We would appreciate any hint to potentially missed prior work the reviewer is aware of, which we would be keen to include in the Related Work section.\n\n## Performance difference between MNIST and MPEG.7.\nThe reviewer points out the performance gap between the MNIST and MPEG-7 datasets and asks for clarifications as to what may be causing it. The information we provide in our Supplementary Material Section C entitled “Additional Details on the Considered Datasets” motivating the choice of the datasets we focus on and summarizing their properties (Supplementary Table 1) is of relevance to address this point. As indicated in Supplementary Table 1, MNIST is a very large (~70’000 images) and relatively easy dataset (thanks to its ~7’000 data points per class) that, even though not designed for shape analysis, serves as an excellent established deep learning model benchmark to assess the performance of ShapeEmbed against other existing methods. In contrast, MPEG-7 is a benchmark dataset designed for shape analysis, but not for deep learning models: it is comparatively much smaller than MNIST with only 1400 objects split between 70 classes with 20 objects each. In contrast to MNIST, MPEG-7 therefore allows us to assess the performance of ShapeEmbed and its competitors on limited, complex data composed of many categories for which only few instances are available (as is also often the case in biological imaging). We hypothesize that these differences in the properties of the two datasets are the reason for the much lower performance observed across all considered methods on MPEG-7. The small number of images per class in MPEG-7 makes this dataset particularly challenging for learning-based methods, while the classical summary statistics achieves respectable results. We consider it one of the strengths of ShapeEmbed that it is able to excel under such conditions.\n\n\n## Additional datasets.\nIn addition to the 4 datasets we already have included in our manuscript (two computer vision datasets, two biological imaging datasets), we propose to include two additional biological imaging datasets to further demonstrate the broad usability of ShapeEmbed. More precisely, the following text will be added to the Supplementary Material in Section C “Additional Details on the Considered Datasets” and the results will be included in Section G “Additional Biological Imaging Results” in the camera-ready version of the manuscript:\n\n*The HeLa Kyoto dataset (CellCognition project, 10.1038/nmeth.1486) consists of fluorescence microscopy images of H2B-mCherry-stained HeLa Kyoto cell nuclei, labeling chromatin and capturing nuclear morphology during mitosis. Already segmented and cropped masks for individual nuclei are available under the “classifier data” section of the dataset. We consider 4 classes that are representative of nuclei at key phases of mitosis (category name and number of samples in parentheses): early anaphase (earlyana, 96 nuclei), interphase (inter, 216 nuclei), metaphase (meta, 169 nuclei), and prometaphase (prometa, 168 nuclei).*\n\n*The MOC dataset (10.5281/zenodo.6554587) consists of fluorescence microscopy images of mouse osteosarcoma cells. In this dataset, cells have been exposed to cytoskeletal perturbation through treatment with the single drugs jasplakinolide (Jasp) and cytochalasin D (Cytd). The cells are divided in 3 classes: a control class with untreated cells (318 cells), a Cytd class (175 cells), and Jasp class (156 cells).*\n\n*In Table X (table number to be adjusted when incorporated in the document), we report the F1-scores obtained when classifying the HeLa Kyoto and MOC datasets with the competing methods considered in the main manuscript (Section 4.3) and ShapeEmbed.*\n\n|  | MOC | Hela Kyoto |\n| --- | --- | --- |\n| Region properties | 0.61 ±0.07 | 0.57 ±0.08 |\n| MAE ViT-B (not pretrained) | 0.57 ±0.02 | 0.47 ±0.13 |\n| MAE ViT-L (not pretrained) | 0.60 ±0.07 | 0.42 ±0.09 |\n| MAE ViT-H (not pretrained) | 0.51 ±0.09 | 0.44 ±0.21 |\n| SimCLR | 0.65 ±0.14 | 0.48 ±0.17 |\n| O2VAE | 0.60 ±0.06 | 0.63 ±0.08 |\n| ShapeEmbed | 0.68 ± 0.01 | 0.80 ± 0.04 |\n| ShapeEmbed + size | 0.70 ± 0.05 | 0.85 ± 0.04 |\n\n**Table X**. *Additional biological dataset results (F1-score ± σ, higher is better), for both classical methods and SotA learning.* methods."}, {"review_id": "8FGcAWcapF", "reviewer": "Reviewer_REU4", "summary": "The paper is about self-supervised learning for 2D contours. The authors aim to inject translation, scaling, rotation, reflection invariance to the representation. So this can work for the specific 2D contour task.", "questions": "No.", "limitations": "See the previous section. While the paper showed some interesting results, its generalization capabilities raise concerns – particularly regarding large-scale datasets and broader problem applicability", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "I think the paper delivered some interesting and strong results. To make the representation invariant to several transformations, the authors first used the distance matrices as the inputs and then carefully modified conv-based networks and loss functions. In the end, the representation is invariant to translation, scale, rotation and reflection. \n\nHowever I have the following concerns,\n1. The core idea is basically incorporating inductive bias into the design. While this enables native handling of the target problem, it may compromise generalization capabilities. This trade-off explains the results in Table 4: performance improves when size information is included, yet other tables demonstrate that such information isn't universally required.\n2. The experiments are all about small datasets such as minist.", "was_revised": true, "final_justification": "Thanks for the response. Some of my concerns are addressed. I keep my current rating because I still believe the application and the dataset is somewhat limited.", "rebuttal": "## Inductive bias and generalization capability.\nThe reviewer’s comment is as follows: \n\n*“The core idea is basically incorporating inductive bias into the design. While this enables native handling of the target problem, it may compromise generalization capabilities. This trade-off explains the results in Table 4: performance improves when size information is included, yet other tables demonstrate that such information isn't universally required.”*\n\nThe explicit incorporation of rotation invariance, scaling invariance, translation invariance, reflection invariance, and indexation invariance (often referred to as parameterization invariance in the shape analysis literature) is indeed at the core of our method. While these invariances may be considered to be inductive biases, they are essential in characterizing shape as classically defined to be the information that remains regardless of these geometric transformations [1]. Generally speaking, building in the correct inductive biases for a given problem increases generalizability, prevents overfitting, and reduces the amount of required training data [2]. \n\nWe however recognize that invariance to scaling, although included in the classical definition of shape, is not always necessary and can even be counter productive depending on the application. In biological imaging specifically, the size of objects (e.g., cells, organisms) often carries important information. To find a middle ground that both acknowledges the classical definition of shape and remains flexible in situation where size matters, we therefore chose to leave the option to reintroduce scale information by concatenating it to the latent vector for downstream tasks when desired, as demonstrated in our experiments by the “ShapeEmbed” and “ShapeEmbed + size” versions. We believe that the suitability of a scale-invariant model genuinely depends on the application and is therefore a choice better left to the user. To clarify these points, we propose to expand the discussion in Section 4, line 359 in the final version of the paper:\n\n*While scale invariance is traditionally considered to be essential for shape analysis [1] and our framework is inherently scale-invariant, scale may be a crucial feature in some applications. To account for this, we additionally consider a variant of ShapeEmbed that preserves scale information by saving the norm of the distance matrix before normalization and concatenating it to the latent code for downstream classification. The decision to consider scale as a relevant feature to be included or as a nuisance transformation to be removed entirely depends on the use case and biological question considered.*\n\n[1]: Klingenberg, Christian Peter. \"Walking on Kendall’s shape space: understanding shape spaces and their coordinate systems.\" Evolutionary Biology 47.4 (2020): 334-352.\n\n[2]: Zhang, Chiyuan, et al. \"Identity Crisis: Memorization and Generalization Under Extreme Overparameterization.\" International Conference on Learning Representations. (2019) \n\n\n## Size of the datasets.\nThe reviewer states that all the experiments are performed on small datasets and specifically raises concerns about ShapeEmbed’s ability to scale to larger datasets. As we are not completely sure what the reviewer specifically means by “small” datasets, we address two possible interpretations of this concern. We would also like to point out that we have now run new experiments on additional biological imaging datasets in our reply to reviewer VYfD.\nShould the concern be with ShapeEmbed’s ability to handle datasets that are composed of many images, our experiments are performed on datasets of various sizes as indicated in Table 1 of Supplementary section C “Additional Details on the Considered Datasets”. In our two computer vision benchmarks, the MPEG-7 dataset is representative of a small dataset (1400 objects) while MNIST is representative of a large dataset (70’000 objects). Similarly for the biological imaging datasets, MEF is representative of a small dataset (1’407 objects) while BBBC010 is representative of a large dataset (26’198 objects).\n\nIf the concern instead has to do with the size of the original images (from which contours are extracted), we recall that ShapeEmbed takes distance matrices built from points composing the contour of single objects as input. As the points of the contour are sampled from a continuous spline interpolation of the original pixels forming the outline of the object, the number of points we draw is a free parameter that is independent from the resolution of the original image. For instance, we draw 32 points to describe the contour of objects in the 28x28 images of the MNIST dataset, and 64 points to describe the contour of objects in the 128x128 images  of the MPEG-7 dataset. The number of points used to describe the contour is a free parameter that is set empirically depending on the size of the objects in the dataset considered and the observed performance (as having too few points obviously results in poorly described shapes and therefore a less performant model), but we do not observe it to significantly impact training time. For more information, we point the reviewer to our answer to reviewer iSBr, we further demonstrate the impact of the choice of the number of contour points (and therefore the size of the resulting distance matrix) on training time and number of parameters in the model."}, {"review_id": "eLh7c09d2n", "reviewer": "Reviewer_iSBr", "summary": "The paper introduces ShapeEmbed, a self-supervised learning (SSL) framework that is invariant to scaling, translation, rotation, reflection, and re-indexing of shape contours. The input is the cross-distance matrix between all points of the input contour. The model is based on a VAE architecture (which includes circular padding in its convolution layers), which, together with a novel loss function, allows for invariance to indexation. The method seems novel, and the experiments (along with ones in the supplementary) provide strong support for the authors’ claims.", "questions": "This is based on the Weaknesses section, questions are ordered by importance: \n1. As mentioned in the Weaknesses section, my main concern is that the method is not scalable. What is the trade-off between the number of contour points and training time (for at least some of the datasets)? \n\n2. What is the number of trainable parameters of the model? ResNet18 usually has ~11M parameters. Does that mean ShapeEmbed (Enc. + Dec.) has ~22M? If so, this might explain the long training time. Can a simple CNN architecture obtain comparable results in a shorter training time? \n\n3. Computation: using 2N alternative versions, each matrix is entered twice, original and mirrored. - This might also explain the long training time and, what I imagine, high GPU RAM consumption. Is that notion correct? If so, can it be mitigated somehow?\n\n4. Comparison with SSL methods (MAE and SimCLR) might not be fair - SimCLR training is known to be painfully long, did it properly converge? MAE, on the other hand, has 86M/304M/632M params. For B/L/H, respectively. \na) Training it on small datasets such as MEF/MPEG-7 (~1400 samples) might be ill-fitted. \nb) The patch-based approach might not be adequate for small images (28x28 and 64x64 for MNIST and BBBC010). Have you considered comparing ShapeEmbed to the pretrained versions of SimCLR and/or MAE? \n\n5. MNIST and MPEG are considered “easy” datasets. While on the harder ones (MEF, BBBC010) ShapeEmbed (w/o Sz) is not much \nbetter than Region Prop. in L368-L370 you mention that “summary statistics, [...],  perform exceptionally well”. Can you elaborate on what the added benefits of ShapeEmbed?\n\n6. Since the reconstruction seems optional, why not use a standard AE?", "limitations": "The limitations appear as part of the \"NeurIPS Paper Checklist\".", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "# Strengths:\n1. The paper is well written. The introduction is clear, and the problem formulation and motivation are well defined. The paper is clearly placed w.r.t. the related literature. \n2. The authors break down different invariances and thoroughly explain how ShapeEmbed obtains them:\n* Distance Matrix - translation, rotation and scale invariance.\n* Process mirrored version of the distance matrix - invariance to the direction of travel.\n* Circular padding - indexation invariance.\n3. Experiments - the experiments are divided into three:\n* Scaling and Indexation Invariance.\n* Rotation and Translation Invariance.\n* Application to Biological Imaging.\nwith the addition of an ablation study. \n\nThe results suggest the proposed method indeed encodes the aforementioned invariances in its architecture and training procedure. This is evident when ShapeEmbed is compared to standard baselines and the main competitors. \n\n4. Supplementary Material (SUPMAT) - There is excellent information in the SUPMAT. Additional experiments, ablations and visualization. I recommend that the authors consider including figure 5 (mean outline reconstruction) and figure 7 (t-sne for the BBBC010 dataset). I understand that this might not be possible due to space limitations. \n \n# Weaknesses\n## Major Weaknesses and Questions: \n1. Scalability and runtime - my main concern is that the method is not scalable. As mentioned in SUPMAT Section C. L48-50, it takes ~49 hours to train the model on MNIST with 64 points per image contour (or 32 according to Table 1. in the SUPMAT?) on a A100 GPU. I would like to see the trade-off between the number of contour points and training time to performance for at least some of the datasets.  \n\n2. What is the number of trainable parameters of the model? ResNet18 usually has ~11M parameters. Does that mean ShapeEmbed (Enc. + Dec.) has ~22M? If so, this might explain the long training time. Can a simple CNN architecture obtain comparable results in a shorter training time? \n\n3. Indexation Invariant Reconstruction Loss: by using 2N alternative versions, each matrix is entered twice, original and mirrored. - This might also explain the long training time and, what I imagine, high GPU RAM consumption. \n\n4. Comparison with SSL methods (MAE and SimCLR) might not be fair - SimCLR training is known to be painfully long, did it properly converge? MAE, on the other hand, has 86M/304M/632M params. for B/L/H, respectively. Training it on \nsmall datasets such as MEF/MPEG-7 (~1400 samples) might be ill-fitted. Additionally, the patch-based approach might not be adequate for small images (28x28 and 64x64 for MNIST and BBBC010). Have you considered comparing ShapeEmbed to the pretrained versions of SimCLR and/or MAE? \n\n## Minor Weaknesses\n1. Limited scope - the paper tackles a problem that might be considered `limited' - invariance to geometric properties of well-defined shape contours. That being said, I can see how the method can be applied in a more extensive scope and on top of foundation models such as Segment Anything Model (SAM). \n2. MNIST and MPEG are considered “easy” datasets, while on the harder ones (MEF, BBBC010) ShapeEmbed (w/o Sz) is not much \nbetter than Region Prop. in L368-L370 you mention that “summary statistics, [...],  perform exceptionally well”. Can you elaborate on what the added benefits of ShapeEmbed?\n\n3. Since the reconstruction seems optional, why not use a standard AE? \n\n## Typos:\nL251 - but not to to the same", "was_revised": true, "final_justification": "The authors have addressed my concerns in their detailed rebuttal. They have performed additional experiments w.r.t the proposed method's scalability and the fairness of their comparison with the competing method. After reviewing the rebuttal, as well as the rest of the reviews and associated rebuttals, I have decided to increase my overall score.", "rebuttal": "## Scalability of the method and impact of the number of contour points on training time. \nTo illustrate the impact of the number of contours points on training time, we report below the time required to complete one epoch for each of the considered dataset and with increasing number of contour points in the input distance matrices. The reported times are in seconds and calculated as the average time for a single epoch of training over 400 epochs. In parenthesis, we also indicate the total time for the full 400 epoch training. We ran these experiments on a GPU cluster with NVIDIA A100 GPUs. While the number of contour points indeed has an impact on training time, it is dominated by dataset size as one epoch requires processing the entire dataset once. More specifically, an increase in the number of points by a factor of 8 (32 points to 256) only leads to an increase by a factor of 1.77 (175s to 311s) in training time per epoch.\n\n||32|64|128|256|\n|---|---|---|---|---|\n|MNIST (70’000 images)|174.5862s (~19h24)|180.5762s (~20h06)|270.49s (~30h03)|310.96s (~35h04)|\n|MEFS(1’407 images)|68.4477s (~7h36)|70.1835s (~7h48)|74.2795s (~8h15)|86.1598s (~9h34)|\n|BBBC010(26’198 images)|6.1271s (~41min)|6.1887s(~41min)|6.6136s (~44min)|7.5953s (~51min)|\n|MPEG-7(1’400 images)|4.99s (~33min)|5.17s (~34min)|5.37s (~36min)|6.32s (~42min)|\n\n## Number of trainable parameters and comparison to a simple CNN\nWe report below the number of parameters in ShapeEmbed based on the input size. To obtain these values, we used the ‘summary’ functionality of the ‘torchinfo’ library. Variations in the number of contours points, and therefore in the input size, have a minimal impact on the total number of parameters.  \n\n|Input size|Total number of parameters|\n|---| --- |\n|32x32|14153889|\n|64x64|14161841|\n|128x128|14163769|\n|256x256|14164221|\n\nThe reviewer also asks whether a simple CNN could obtain comparable results in a shorter training time. We are not sure if “simple” refers to an architecture that would not be based on distance matrices (instead on images), or to a smaller network architecture (instead of a ResNet backbone). An architecture that would not be based on distance matrices would not benefit from built-in invariance properties, and would thus be substantially disadvantaged as compared to our method, as seen by the performance of O2VAE. A lighter version of our encoder and decoder architectures could certainly be conceived, but would require extensive ablation experiments to validate it over a well-established backbone like ResNet. We therefore believe this to be a valuable exploration for future work, but to go beyond the scope of our current paper. \n\n## Efficiency of the indexation invariance implementation\nIt is true that the encoder processes each matrix twice to achieve reflection invariance. However, our loss function only compares the output of the decoder to 2N versions of our target matrix, which are computed on the fly, meaning that the decoder doesn’t have to run 2N times. In the future, we could make the encoder invariant to reflection without requiring it to be executed twice by using reflection equivariant convolutions [1]. We propose to mention this in our conclusion section as follows:\n*\"While our current encoder achieves reflection invariance by processing each matrix as well as as a flipped version, this computational overload could be avoided in the future by using reflection equivariant convolutions [1].\"*\n\n[1]: Cohen, Taco, and Max Welling. \"Group equivariant convolutional networks.\" International conference on machine learning. PMLR, 2016.\n\n## Fairness of the comparison with standard SSL methods\nThe MAE and SimCLR models we trained converged for all datasets in our original experiments. We confirmed this by re-running the experiments to answer this comment. We unfortunately cannot provide figures of the loss curves in the rebuttal, but could add them to our Supplementary Material.\n\nThe reviewer also asks for a comparison of ShapeEmbed to pretrained versions of SimCLR and/or MAE. Below, we report the F1-score of the downstream classification tasks of each of the datasets (including two additional biological imaging datasets, see our response to reviewer VYfD) using pretrained MAE models. To do these experiments, we initialized our MAE model using the pytorch checkpoints provided in the original Github repository from Facebook Research. We did not include results with a pretrained SimCLR model as the implementation we use (link to the repo in the original manuscript) neither provides pretrained weights nor a straightforward way to initialise the model for inference. As the other pretrained SimCLR models we could find in platforms like Hugging Face would require major changes to fit in our pipeline, we were unable to experiment with them during the short time of the rebuttal. \n\nPerformance increases across all MAE modalities when using pretrained models. We believe that this can be attributed to the large number of images the pretrained models have already been exposed to, which help compensate for the relatively small amount of data available in all datasets except MNIST. ShapeEmbed, trained from scratch, however continues to outperform these competing methods. It further suggests that the invariances built into ShapeEmbed allows it to learn meaningful representations even with limited data. We thank the reviewer for suggesting this experiment and will include these results in Table 1 in the final version.\n\n**Benchmark datasets**\n||MNIST|MPEG-7|\n|---|---|---|\n|MAE ViT-B pretrained|0.92±0.01|0.42±0.01|\n|MAE ViT-L pretrained|0.93±0.01|0.51±0.02|\n|MAE ViT-H pretrained|0.94±0.01|0.57±0.02|\n|ShapeEmbed|0.96±0.01|0.75±0.02|\n\n**Biological imaging datasets (previous)** \n||MEF|BBBC010|\n|---|---|---|\n|MAE ViT-B pretrained|0.61±0.01|0.65±0.08|\n|MAE ViT-L pretrained|0.64±0.02|0.77±0.03|\n|MAE ViT-H pretrained|0.65±0.01|0.76±0.06|\n|ShapeEmbed|0.67±0.01|0.837±0.01|\n|ShapeEmbed+size|0.76±0.01|0.87±0.01|\n\n**Biological imaging datasets (new)** \n||MOC|HeLa Kyoto|\n|---|---|---|\n|MAE ViT-B pretrained|0.48±0.15|0.72±0.02|\n|MAE ViT-L pretrained|0.66±0.03|0.76±0.03|\n|MAE ViT-H pretrained|0.63±0.05|0.73±0.08|\n|ShapeEmbed|0.68±0.01|0.80±0.04|\n|ShapeEmbed+size|0.70±0.05|0.85±0.04|\n\n## Limited scope of the method\nThe reviewer points out that the problem tackled by ShapeEmbed may be “considered limited”. While we agree that shape quantification has not gotten as much attention as other problems such as image segmentation, we believe it to be an important challenge in particular in biological imaging. We therefore consider that it is relevant to propose innovative methods focusing on the quantification of segmented images, especially with biological imaging in mind.\n\nWe fully agree with the reviewer that our method can be combined with upstream segmentation methods such as SAM, and have ourselves combined it with cellpose (see Supplementary Section G.3).\n\n## Added benefit of ShapeEmbed\nWe respectfully disagree with the characterization of MPEG-7 as an “easy” dataset, in that its small size (1400 images for 70 classes) makes it highly challenging for DL methods which typically require many more images to generalize well. Most DL models struggle to learn meaningful representations on MPEG-7, as demonstrated in our experiments. In contrast, ShapeEmbed performs despite the large number of classes and the small number of images per class in this dataset, which speaks to its efficiency and suitability in low-data regimes.\n\nThe reviewer also requests an explanation of why ShapeEmbed without size does not exhibit much better performance than summary statistics (region properties) on the biological imaging datasets. In the context of biological imaging, size can be relevant to the phenomenon observed. Several of the classical summary statistics (e.g., area and perimeter) explicitly encode size while ShapeEmbed encodes “shape” as traditionally-defined, without size information [1]. As a result, ShapeEmbed without size is disadvantaged in experiments where size is an essential feature for the downstream classification task. Region properties are however inherently limited in capturing shape variability as they reduce contours to a small set of global statistics and fail to capture localized variations. This is illustrated by the fact that, when size is explicitly added back to the ShapeEmbed latent codes, our method not only becomes competitive but surpasses region properties. Unlike region properties, ShapeEmbed in addition learns a continuous and structured latent space where shapes that cluster together are semantically and morphologically similar. This geometric interpretability is hard to build into region properties and similar handcrafted approaches. Furthermore, our method is generative and thus enables visual investigation of the latent space as well as synthesis of novel shapes, as illustrated in Supplementary Section G.1, Figures 5 and 6. As a result, ShapeEmbed is not only a classification tool but also enables exploratory shape analysis.\n\nWe also point to our reply to reviewer VYfD, where we further discuss MNIST and MPEG-7 and provide results on two additional biological imaging datasets.\n\n[1] I.L. Dryden & K.V. Mardia (1998). Statistical Shape Analysis. John Wiley & Sons. ISBN 978-0-471-95816-1\n\n## Use of a standard AE instead of a VAE\nOur method could indeed in principle use a standard AE architecture. However, the VAE provides a dense continuous latent space enabling interpolation and averaging as illustrated in Supplementary Figures 5 and 11. This capability also makes it possible to generate shapes de novo according to the learned distribution, as shown in Supplementary Figures 6 and 12. While we did not quantitatively explore these capabilities in depth in our manuscript as it focuses on the construction of the model itself, they are of strong interest to biological imaging and are therefore an essential feature of our design."}, {"review_id": "i0f9CkU2zh", "reviewer": "Reviewer_pLNK", "summary": "This manuscript presents ShapeEmbed, a self-supervised framework for learning transformation-invariant shape descriptors from segmentation masks. The use of pairwise distance matrices, architectural symmetry, and indexation-aware loss design is well-motivated and results in both strong discriminative performance and generative capability. Extensive experiments support the method’s utility across multiple datasets. The approach is timely and well-suited for bioimaging applications.", "questions": "--This paper discuss the case of closed curve, how the proposed descriptor describe the open curve?\n-- When there are two many points, the scale of the distance matrix will be very large, how about the  efficiency of the proposed algorithm?\n--Given the same 2D contour, when we sampled it with different number of points, for example, one for 100 and the other for 1000, then we will achieved two different distance matrix and different latent features, how about the similarity of these two latent feature? That is, the proposed representation is also invariant to the dense of the sampled points or not?\n--Can this method can be generalized to the case of 3D space curve or not?", "limitations": "--When there are a large number of the sampled points, the efficiency  of the proposed method may very poor. \n--The invariance claims (translation, rotation, scaling, reflection, indexation) are central to the novelty of ShapeEmbed. While most are intuitively justified via the properties of distance matrices, the reflection and indexation invariance depend on architectural choices and loss design, which require clearer theoretical treatment.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "---Strengths: This paper presents a technically sound and well-motivated framework for learning transformation-invariant shape representations using distance matrices and a customized VAE architecture.  The approach is novel, particularly in how it handles reflection and indexation invariance—two aspects often neglected in prior work.  The experimental evaluation is thorough, covering multiple natural and biological datasets, and is supported by rich ablation studies, visualizations, and generative sampling.  The method is clearly relevant for biological imaging and other applications requiring shape-based analysis.  The manuscript is clearly written, well-structured, and supported by detailed supplementary materials, which enhance the reproducibility and transparency.\n\n---Weaknesses: Despite the comprehensive classification experiments, the paper falls short in demonstrating the broader applicability of the learned shape representations. Additionally, on biological datasets like MEF and BBBC010, the Region Prop. method performs competitively, suggesting that much of the discriminative power may stem from simple size or area cues. The paper would benefit from a deeper analysis of whether ShapeEmbed captures richer, non-size-based shape information, particularly in biologically relevant contexts.", "was_revised": false, "final_justification": "", "rebuttal": "## Applicability of shape representations and importance of size and area cues.\nThe reviewer mentions that ShapeEmbed “falls short in demonstrating the broader applicability of the learned shape representations” and that  “much of the discriminative power may stem from simple size or area cues”. We are discussing these points in detail in our response to reviewer iSBr (“Added benefit of ShapeEmbed” and “Limited scope of the method”) and summarize the key elements here. Size indeed plays an important role in many biological imaging problems, which is why Region Properties perform competitively. That said, even in situations where size information is crucial, region properties only capture limited information about shape and are therefore outperformed by the scale-invariant version of ShapeEmbed on all the biological imaging datasets we considered except MEF, as illustrated by the new experiments we present in our reply to reviewer VYfD (7% better on MOC and 23% better on HeLa Kyoto). Importantly, adding size information back to the ShapeEmbed latent codes only yields an additional improvement of 2% and 5%, respectively. We believe all-in-all our results highlight the power of learned shape descriptors beyond simple size and area cues in biological imaging data, as well as in more classical imaging data (see Table 1 in our manuscript).\n\n## Application of the method to open curves.\nWe did not explicitly investigate open curves in our manuscript as our target application is biological imaging, where objects are most often described by their closed contour. That said, ShapeEmbed can be applied to open curves as well. In fact, open curves are simpler to deal with than closed ones as they only have two possible starting points: the sole remaining ambiguity is the direction in which the points are traversed. As a result, the circular padding we implemented in the encoder would no longer be required and our indexation-invariant loss would be simplified to only consider 2 possible indexations instead of 2N for a contour composed of N points. We propose to include this point in a new section of our Supplementary Material as follows:\n\n***Application to open curves**. While ShapeEmbed is built for closed contours, it can in principle be applied to open curves as well, as the distance matrix representation does not require a closed contour. In fact, applying the approach to open contours simplifies the problem, as it leaves only two possible starting points to index the contour. As a result, the circular padding we implemented in the encoder is no longer required and the indexation-invariant loss can be simplified to only consider two possible indexations instead of 2N for a contour composed of N points.*\n\n## Impact of the number of points on the efficiency of the method.\nIt is true that the number of points used to create the distance matrix will influence the amount of required memory as well as the required training time. We invite the reviewer to read our response to reviewer iSBr for a detailed experimental analysis of this topic.\n\n## Invariance with respect to the number of sampled contour points.\nAs described in Section 3.1 of our manuscript, we draw points from an object’s contour to create a distance matrix by first interpolating the pixels originally composing the object outline, and then uniformly sampling a given number of points from that interpolation. As pointed out by the reviewer, it is therefore possible to sample the same 2D contour with different numbers of points. We can expect each contour to possess an optimal sampling threshold that defines the minimum number of points required to accurately represent it. Insufficient sampling below this threshold results in an inadequate contour description, while oversampling beyond this threshold does not increase description accuracy. This minimum number of points correlates directly with geometric complexity: smooth contours have lower sampling requirements, whereas complex, irregular contours necessitate denser sampling. The notion of invariance to sampling pointed out by the reviewer would really only apply to oversampled contours, as undersampled contours would result in different shapes and would therefore, by design, not have the same representation in our latent space. As it is highly challenging to know a priori what the minimum number of points for a given contour is, it is extremely hard to formulate a model that would implement invariance to sampling only when appropriate. In our current architecture, we therefore implemented the number of points used to describe the contour as a hyper parameter that must be fixed prior to training. \n\nIn the future, it could be possible to build models with variable point numbers to have a framework ready to implement sampling invariance. Supporting variable numbers of points would also allow for the training of inhomogeneous datasets. One further area of interest for future work is the design of a flexible method that could add or remove points by encoding and decoding a matrix allowing us to increase or decrease the level of detail.\n\n## Application of the method to 3D curves.\nWe are confident that our method can be adapted to curves in 3D space as the distance matrix representation as well as the MDS reconstruction algorithm also apply in higher dimensions [1]. In preliminary follow-up work to this manuscript, we successfully applied our method to synthetic curves in 3D space. We however believe that a more exciting future direction is to adapt our method to 3D surfaces described as point clouds. The main obstacle towards that goal is that, unlike points on a contour, points on a surface do not have a clear order. This in turn requires a new approach to achieving indexation invariance, as considering all shifted versions of the distance matrix is no longer sufficient in such a setting.\n\n[1]: Dokmanic, Ivan, et al. \"Euclidean distance matrices: essential theory, algorithms, and applications.\" IEEE Signal Processing Magazine 32.6 (2015): 12-30."}]

Author rebuttal:

Meta-review: This paper introduces a self-supervised method for learning transformation-invariant 2D shape representations from binary masks. Reviewers highlighted several strengths: (a) the novelty of combining distance matrices with a latent space learned through a VAE, enforcing invariance to translation, rotation, scaling, reflection, and contour reindexing, and (b) strong experimental results across multiple datasets, particularly in challenging biological settings. Major weaknesses identified were: (a) the method is limited to 2D shapes with single contours and does not currently extend to 3D geometries, (b) the evaluated datasets are somewhat limited, and (c) concerns about scalability and runtime.

The paper received three accepts, one borderline accept, and one borderline reject (VYfD). Four of the five reviewers agreed that the strengths outweighed the weaknesses and that the authors’ responses in the rebuttal and during the author-reviewer discussion adequately addressed their concerns. During the AC-reviewer discussion, the negative reviewer maintained that combining distance matrices and a VAE for self-supervised learning of 2D shape representations “may not meet the novelty threshold” and that “the methodological innovation remains relatively limited.” In contrast, the other reviewers emphasized that the method is novel, noting the absence of prior work encoding contour representations that are invariant to translation, rotation, scaling, reflection, and indexing.

The AC concurs with this latter assessment and further notes that, according to the NeurIPS 2025 reviewer guidelines, originality can be established through “a novel combination of existing techniques,” provided that "the rationale for the combination is well articulated". The AC finds that the submission meets the novelty requirement, and recommends acceptance. The authors are strongly encouraged to integrate the additional experiments, results, and clarifications from the rebuttal and discussion into the final version of the paper!
Decision: Accept (poster)