Title: ShapeEmbed: a self-supervised learning framework for 2D contour quantification
Venue: NeurIPS 2025
Accepted: yes
Keywords: shape quantification

shape descriptors

VAE

generative model

representation learning

microscopy
Abstract: The shape of objects is an important source of visual information in a wide range of applications. One of the core challenges of shape quantification is to ensure that the extracted measurements remain invariant to transformations that preserve an object’s intrinsic geometry, such as changing its size, orientation, and position in the image. In this work, we introduce ShapeEmbed, a self-supervised representation learning framework designed to encode the contour of objects in 2D images, represented as a Euclidean distance matrix, into a shape descriptor that is invariant to translation, scaling, rotation, reflection, and point indexing. Our approach overcomes the limitations of traditional shape descriptors while improving upon existing state-of-the-art autoencoder-based approaches. We demonstrate that the descriptors learned by our framework outperform their competitors in shape classification tasks on natural and biological images. We envision our approach to be of particular relevance to biological imaging applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript presents ShapeEmbed, a self-supervised framework for learning transformation-invariant shape descriptors from segmentation masks. The use of pairwise distance matrices, architectural symmetry, and indexation-aware loss design is well-motivated and results in both strong discriminative performance and generative capability. Extensive experiments support the method’s utility across multiple datasets. The approach is timely and well-suited for bioimaging applications.\n\nSTRENGTHS AND WEAKNESSES:\n---Strengths: This paper presents a technically sound and well-motivated framework for learning transformation-invariant shape representations using distance matrices and a customized VAE architecture.  The approach is novel, particularly in how it handles reflection and indexation invariance—two aspects often neglected in prior work.  The experimental evaluation is thorough, covering multiple natural and biological datasets, and is supported by rich ablation studies, visualizations, and generative sampling.  The method is clearly relevant for biological imaging and other applications requiring shape-based analysis.  The manuscript is clearly written, well-structured, and supported by detailed supplementary materials, which enhance the reproducibility and transparency.\n\n---Weaknesses: Despite the comprehensive classification experiments, the paper falls short in demonstrating the broader applicability of the learned shape representations. Additionally, on biological datasets like MEF and BBBC010, the Region Prop. method performs competitively, suggesting that much of the discriminative power may stem from simple size or area cues. The paper would benefit from a deeper analysis of whether ShapeEmbed captures richer, non-size-based shape information, particularly in biologically relevant contexts.\n\nQUESTIONS:\n--This paper discuss the case of closed curve, how the proposed descriptor describe the open curve?\n-- When there are two many points, the scale of the distance matrix will be very large, how about the  efficiency of the proposed algorithm?\n--Given the same 2D contour, when we sampled it with different number of points, for example, one for 100 and the other for 1000, then we will achieved two different distance matrix and different latent features, how about the similarity of these two latent feature? That is, the proposed representation is also invariant to the dense of the sampled points or not?\n--Can this method can be generalized to the case of 3D space curve or not?\n\nLIMITATIONS:\n--When there are a large number of the sampled points, the efficiency  of the proposed method may very poor. \n--The invariance claims (translation, rotation, scaling, reflection, indexation) are central to the novelty of ShapeEmbed. While most are intuitively justified via the properties of distance matrices, the reflection and indexation invariance depend on architectural choices and loss design, which require clearer theoretical treatment.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces ShapeEmbed, a self-supervised learning (SSL) framework that is invariant to scaling, translation, rotation, reflection, and re-indexing of shape contours. The input is the cross-distance matrix between all points of the input contour. The model is based on a VAE architecture (which includes circular padding in its convolution layers), which, together with a novel loss function, allows for invariance to indexation. The method seems novel, and the experiments (along with ones in the supplementary) provide strong support for the authors’ claims.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n1. The paper is well written. The introduction is clear, and the problem formulation and motivation are well defined. The paper is clearly placed w.r.t. the related literature. \n2. The authors break down different invariances and thoroughly explain how ShapeEmbed obtains them:\n* Distance Matrix - translation, rotation and scale invariance.\n* Process mirrored version of the distance matrix - invariance to the direction of travel.\n* Circular padding - indexation invariance.\n3. Experiments - the experiments are divided into three:\n* Scaling and Indexation Invariance.\n* Rotation and Translation Invariance.\n* Application to Biological Imaging.\nwith the addition of an ablation study. \n\nThe results suggest the proposed method indeed encodes the aforementioned invariances in its architecture and training procedure. This is evident when ShapeEmbed is compared to standard baselines and the main competitors. \n\n4. Supplementary Material (SUPMAT) - There is excellent information in the SUPMAT. Additional experiments, ablations and visualization. I recommend that the authors consider including figure 5 (mean outline reconstruction) and figure 7 (t-sne for the BBBC010 dataset). I understand that this might not be possible due to space limitations. \n \n# Weaknesses\n## Major Weaknesses and Questions: \n1. Scalability and runtime - my main concern is that the method is not scalable. As mentioned in SUPMAT Section C. L48-50, it takes ~49 hours to train the model on MNIST with 64 points per image contour (or 32 according to Table 1. in the SUPMAT?) on a A100 GPU. I would like to see the trade-off between the number of contour points and training time to performance for at least some of the datasets.  \n\n2. What is the number of trainable parameters of the model? ResNet18 usually has ~11M parameters. Does that mean ShapeEmbed (Enc. + Dec.) has ~22M? If so, this might explain the long training time. Can a simple CNN architecture obtain comparable results in a shorter training time? \n\n3. Indexation Invariant Reconstruction Loss: by using 2N alternative versions, each matrix is entered twice, original and mirrored. - This might also explain the long training time and, what I imagine, high GPU RAM consumption. \n\n4. Comparison with SSL methods (MAE and SimCLR) might not be fair - SimCLR training is known to be painfully long, did it properly converge? MAE, on the other hand, has 86M/304M/632M params. for B/L/H, respectively. Training it on \nsmall datasets such as MEF/MPEG-7 (~1400 samples) might be ill-fitted. Additionally, the patch-based approach might not be adequate for small images (28x28 and 64x64 for MNIST and BBBC010). Have you considered comparing ShapeEmbed to the pretrained versions of SimCLR and/or MAE? \n\n## Minor Weaknesses\n1. Limited scope - the paper tackles a problem that might be considered `limited' - invariance to geometric properties of well-defined shape contours. That being said, I can see how the method can be applied in a more extensive scope and on top of foundation models such as Segment Anything Model (SAM). \n2. MNIST and MPEG are considered “easy” datasets, while on the harder ones (MEF, BBBC010) ShapeEmbed (w/o Sz) is not much \nbetter than Region Prop. in L368-L370 you mention that “summary statistics, [...],  perform exceptionally well”. Can you elaborate on what the added benefits of ShapeEmbed?\n\n3. Since the reconstruction seems optional, why not use a standard AE? \n\n## Typos:\nL251 - but not to to the same\n\nQUESTIONS:\nThis is based on the Weaknesses section, questions are ordered by importance: \n1. As mentioned in the Weaknesses section, my main concern is that the method is not scalable. What is the trade-off between the number of contour points and training time (for at least some of the datasets)? \n\n2. What is the number of trainable parameters of the model? ResNet18 usually has ~11M parameters. Does that mean ShapeEmbed (Enc. + Dec.) has ~22M? If so, this might explain the long training time. Can a simple CNN architecture obtain comparable results in a shorter training time? \n\n3. Computation: using 2N alternative versions, each matrix is entered twice, original and mirrored. - This might also explain the long training time and, what I imagine, high GPU RAM consumption. Is that notion correct? If so, can it be mitigated somehow?\n\n4. Comparison with SSL methods (MAE and SimCLR) might not be fair - SimCLR training is known to be painfully long, did it properly converge? MAE, on the other hand, has 86M/304M/632M params. For B/L/H, respectively. \na) Training it on small datasets such as MEF/MPEG-7 (~1400 samples) might be ill-fitted. \nb) The patch-based approach might not be adequate for small images (28x28 and 64x64 for MNIST and BBBC010). Have you considered comparing ShapeEmbed to the pretrained versions of SimCLR and/or MAE? \n\n5. MNIST and MPEG are considered “easy” datasets. While on the harder ones (MEF, BBBC010) ShapeEmbed (w/o Sz) is not much \nbetter than Region Prop. in L368-L370 you mention that “summary statistics, [...],  perform exceptionally well”. Can you elaborate on what the added benefits of ShapeEmbed?\n\n6. Since the reconstruction seems optional, why not use a standard AE?\n\nLIMITATIONS:\nThe limitations appear as part of the \"NeurIPS Paper Checklist\".", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

comments: SUMMARY:
The paper is about self-supervised learning for 2D contours. The authors aim to inject translation, scaling, rotation, reflection invariance to the representation. So this can work for the specific 2D contour task.

STRENGTHS AND WEAKNESSES:
I think the paper delivered some interesting and strong results. To make the representation invariant to several transformations, the authors first used the distance matrices as the inputs and then carefully modified conv-based networks and loss functions. In the end, the representation is invariant to translation, scale, rotation and reflection. 

However I have the following concerns,
1. The core idea is basically incorporating inductive bias into the design. While this enables native handling of the target problem, it may compromise generalization capabilities. This trade-off explains the results in Table 4: performance improves when size information is included, yet other tables demonstrate that such information isn't universally required.
2. The experiments are all about small datasets such as minist.

QUESTIONS:
No.

LIMITATIONS:
See the previous section. While the paper showed some interesting results, its generalization capabilities raise concerns – particularly regarding large-scale datasets and broader problem applicability
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the problem of accurately describing object contours in 2D images and proposes a novel method called ShapeEmbed. Based on a learning-based framework, ShapeEmbed learns a shape representation that is invariant to translation, scaling, rotation, reflection, and point indexing, and applies it to downstream tasks. Experiments are conducted to validate the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is clearly written and easy to understand.\n\n2. This paper focuses on a core challenge in object shape modeling, and correspondingly proposes a shape descriptor that is invariant to translation, scaling, rotation, reflection, and point indexing.\n\n3. The effectiveness of the proposed shape descriptor is validated through experiments.\n\nWeaknesses:\n\n1. The novelty of the paper is not clearly articulated. The core challenges in shape representation—such as achieving invariance to translation, scaling, and rotation—have been long-standing problems addressed by many prior works. The use of a distance matrix to achieve these invariances is not particularly surprising, and similar approaches have been explored previously (as the authors themselves acknowledge). Then, what's the novelty and contributions? \nIf the key contribution lies in addressing the indexation invariance problem for closed contours, this should be clearly emphasized in the Introduction.\n\n2. Has the issue of indexation invariance truly not been addressed in prior work? It is unclear whether the authors have conducted a sufficiently comprehensive review of related literature on this aspect.\n\n3. There is a notable performance gap between the baselines on MNIST and MPEG-7. The authors should provide a clearer comparison of these two datasets, highlighting their differences in structure and task complexity during dataset description.\n\n4. I'm not an expert in shape representation, but the number of datasets used in the experiments seems relatively limited. Are there additional benchmark datasets or comparison methods that could further support the generality and robustness of the proposed approach?\n\nQUESTIONS:\nPlease refer to the weaknesses section, and in particular, highlight the originality of your work—rather than presenting it as an incremental contribution (i.e., a mere combination of existing modules). Emphasize whether your approach addresses a previously unsolved challenge.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ShapeEmbed, a self-supervised framework for learning transformation-invariant 2D shape representations from binary masks. The key idea is to represent an object's shape via its Euclidean distance matrix over uniformly sampled contour points, which is inherently invariant to translation, rotation, and reflection. To achieve scale and contour-index invariance, the authors normalize the matrix and modify a convolutional variational autoencoder (VAE) to include circular padding and reflection-averaged encoding. Furthermore, they define a symmetric reconstruction loss that accounts for all cyclic permutations and mirrored versions of the input, ensuring that the learned latent shape descriptors are fully invariant. The framework is validated across synthetic and real-world datasets, including biological microscopy images, showing strong performance in downstream shape classification and generative applications.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**. \n\nThe paper addresses an important and well-defined problem of learning transformation-invariant shape representations in a self-supervised manner. The key contribution lies in leveraging the distance matrix as an input modality, combined with architectural and loss-based mechanisms to enforce invariance to translation, rotation, scaling, reflection, and contour reindexing. The methodology is elegant, mathematically sound, and requires minimal pre-processing. Compared to prior work like O2VAE and SimCLR-based mask encoders, ShapeEmbed demonstrates superior downstream performance on a range of datasets, especially in challenging biological settings where shape is semantically meaningful.\n\n**Weaknesses**\n\nOn the downside, while the paper is thorough in its formulation and evaluation, the scope of the method is limited to simply connected 2D shapes with single contours. It cannot directly handle shapes with holes, disconnected components, or 3D geometries, which limits applicability to more general vision problems. Although experiments cover a reasonable range of datasets and ablations, some aspects—such as the dependence on contour sampling resolution, alternative normalization schemes, or quantitative contour reconstruction fidelity—could be further explored to strengthen the generality and rigor of the evaluation. The reliance on Euclidean distance matrices also may pose computational challenges when scaling to high-resolution contours.\n\nQUESTIONS:\n1. Section 3.2 states that the encoder outputs a shifted latent representation, whereas Section 3.3 claims that the encoder is invariant to indexation. However, a shifted output and indexation invariance are not necessarily equivalent concepts. Could the authors clarify this apparent discrepancy and elaborate on how indexation invariance is formally ensured?\n\n2. The proposed framework seems to rely on high-quality segmentation masks. However, in real-world scenarios, segmentation masks often contain noise. How robust is your method to such noisy inputs? Have you evaluated its sensitivity to imperfect masks?\n\n3. In Section 3.3, you introduce several regularization terms to encourage the output distance matrix to be non-negative, symmetric, and have zero diagonal. However, these soft constraints do not strictly guarantee that the reconstructed matrix satisfies all distance matrix properties. Have you considered constructing a distance matrix that is analytically guaranteed to meet these constraints, possibly through a parameterization rooted in mathematical structure (e.g., analogous to how quaternions can be used to ensure valid rotation matrices, or how eigen-decomposition can yield symmetric matrices)?\n\nLIMITATIONS:\nYes, the authors clearly acknowledge the limitation to simply-connected 2D shapes and suggest potential 3D extensions and nested shape modeling as future work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}