Title: T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: self-supervised learning

minimum spanning tree

dimensional collapse

dimension estimation

topological data analysis
Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations without labeled data, often by enforcing invariance to input transformations such as rotations or blurring.
Recent studies have highlighted two pivotal properties for effective representations: (i) avoiding dimensional collapse-where the learned features occupy only a low-dimensional subspace, and (ii) enhancing uniformity of the induced distribution.
In this work, we introduce T-REGS, a simple regularization framework for SSL based on the length of the Minimum Spanning Tree (MST) over the learned representation.
We provide theoretical analysis demonstrating that T-REGS simultaneously mitigates dimensional collapse and promotes distribution uniformity on arbitrary compact Riemannian manifolds.
Several experiments on synthetic data and on classical SSL benchmarks validate the effectiveness of our approach at enhancing representation quality.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces T-REGS, a framework based on Minimum Spanning Tree (MST), to regularize SSL methods to prevent dimensional collapse and promote uniformity. Theoretically, under the assumption of strict sphere constraint, the authors observe that when $n \\leq d+1$ ($n$ is the point number and $d$ is the dimension), the optimal MST length is achieved when the embeddings form a simplex. For large samples, the authors first relate the MST length to the  R\\'{e}nyi entropy and then propose that uniform distribution maximizes the entropy. In the experiments, T-REG demonstrates the ability the spread the initial point cloud uniformly on the sphere. Additionally, T-REGS yields competitive results across different datasets, though it fails to outperform the baseline.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper provides a detailed theoretical analysis to gain insights of MST and guide the design of T-REG.\n2. The authors conduct experiments to demonstrate the effects of T-REG (Figure 2).\n3. The idea of this paper is novel and the writing is generally easy to follow.\n\nWeaknesses:\n1. The theoretical analysis is based on strict sphere constraints, which cannot be completely ensured in reality. Since regularization is only able to gradually enforce the sphere constraints, a relaxed assumption is more suitable for the setting.\n2. There is still a gap between the performance of T-REGS and the baselines, making the effectiveness of the method not convincing enough. More trials regarding the proper choices of the parameters are needed.\n3. It is observed in Table 2 that higher values of $\\frac{\\beta}{\\gamma}$ and $\\frac{\\gamma}{\\lambda}$ lead to better performance. Intuitively, this would mean that $\\mathcal{L}_E$ and $\\mathcal{L}_S$ are not that necessary. To justify the regularization terms, you should first show that no regularization term has poorer performance than your method, and that extremely large $\\frac{\\beta}{\\gamma}$ and $\\frac{\\gamma}{\\lambda}$ also harms the classification accuracy.\n\nQUESTIONS:\n1. Are there any results of combining T-REGS with existing SSL methods, both contrastive and non-contrastive? This would further demonstrate the effectiveness of T-REGS.\n2. Can T-REGS be generalized to supervised learning?\n3. What is the extra cost of MST computation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces T-REGS, a novel regularization framework for joint-embedding self-supervised learning (JE-SSL). The core idea is to regularize the learned embeddings by maximizing the length of the Minimum Spanning Tree (MST) constructed over the embedding vectors within a batch, while simultaneously applying a soft constraint to keep the embeddings on the unit hypersphere.\n\nThe authors claim that this approach simultaneously addresses two critical issues in representation learning:\n\nDimensional Collapse: It encourages the embeddings to occupy the full dimensionality of the representation space.\nSample Uniformity: It promotes an even distribution of embeddings on the hypersphere.\nThe paper provides a solid theoretical analysis, connecting the MST length maximization to statistical dimension estimation and Rényi entropy maximization. This analysis shows that for small sample sizes, the optimal configuration of points under the T-REG loss forms a regular simplex, and asymptotically for large samples, it encourages a uniform distribution. The method's effectiveness is validated through experiments on synthetic data, which clearly illustrate its properties, and on standard SSL benchmarks (CIFAR-10/100, ImageNet-100/1k), where it achieves competitive performance compared to state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n1.Novelty and Elegance of the Core Idea: The central contribution—using the length of the Minimum Spanning Tree as a regularizer—is highly novel and conceptually elegant. It draws a clever and principled connection between a classic algorithm from graph theory, concepts from statistical dimension estimation, and the modern challenges of self-supervised representation learning. This is a significant conceptual contribution that could inspire new research directions.\n2.Strong Theoretical Grounding: The paper is not just a heuristic proposal; it is backed by rigorous theoretical analysis (Section 4.1).\nProposition 4.1 provides a clear and intuitive result for the small-batch regime (n ≤ d + 1), showing that the loss is maximized when embeddings form a regular simplex. This directly proves that the regularizer prevents dimensional collapse in this setting.\nTheorem 4.2 and Proposition 4.3 establish an asymptotic link between maximizing the MST length and maximizing the Rényi entropy of the embedding distribution. Since maximum entropy on a compact set corresponds to the uniform distribution, this provides a strong justification for the claim that T-REGS promotes sample uniformity.\n3.Simplicity and Practicality: Compared to some competing methods that aim for similar goals, T-REGS is relatively simple. For instance, the Optimal Transport-based regularizer proposed by Fang et al. can be computationally expensive and numerically unstable. T-REGS, while requiring an MST computation, avoids complex operations like SVD and is conceptually straightforward to implement using existing fast, parallelized MST algorithms.\n\nWeaknesses \n\n1.Empirical Performance: While the results are competitive, T-REGS does not consistently set a new state-of-the-art on the reported benchmarks. For example, on CIFAR-100, it lags behind several W2-regularized methods, and on ImageNet-1k, it is slightly outperformed by methods like INTL and Zero-CL. While SOTA performance is not a strict requirement for a conceptually novel paper, a stronger showing would make the practical case for adoption more compelling. The authors acknowledge this limitation in the conclusion [305-306].\n2.Computational Overhead of MST: The paper states that fast GPU-based MST implementations exist, but it does not provide a practical analysis of the computational overhead. For a batch of size B, constructing the complete graph has a complexity of O(B²), and standard MST algorithms like Kruskal's or Prim's are at least O(B²). This could become a bottleneck for very large batch sizes, which are common in SSL. A comparison of wall-clock training time per step against baselines like VICReg (which computes a covariance matrix, O(B*D²)) or BarlowTwins would be a valuable addition to assess the practical scalability.\n\nQUESTIONS:\nrefer to weaknesses.\n\nLIMITATIONS:\nrefer to weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo avoid dimensional collapse and enhance uniformity in self-supervised learning, the authors proposed the Minimum Spanning Tree (MST) over the learned representation. Moreover, the authors provided theoretical analysis.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n(1) The authors proposed the Minimum Spanning Tree (MST) to avoid dimensional collapse and enhance uniformity in self-supervised learning, which is a novel approach.\n\n(2) The authors provided a great deal of theoretical analysis for the proposed Minimum Spanning Tree (MST).\n\n\n**Weaknesses:**\n\n(1) The authors did not provide a relatively clear explanation of how the Minimum Spanning Tree (MST) is combined with Self-Supervised learning. The picture provided by the authors in Figure 1 is common knowledge. Instead of providing a common-sense picture, the authors should have provided a picture in the introduction to help readers intuitively understand how the Minimum Spanning Tree (MST) is combined with Self-Supervised learning.\n\n(2) I didn’t understand how the Minimum Spanning Tree solves the problems of avoiding dimensional collapse and enhancing uniformity in self-supervised learning.\n\nI admit that I haven’t fully understood the author’s paper. If the author can address my two main concerns and some detailed issues, I will raise the score.\n\nQUESTIONS:\n(1) In lines 66-70, the author introduced the concept of $\\alpha$-minimum spanning tree, but the author didn’t combine it well with the context of self-supervised learning. I’m very curious about how to construct a connected acyclic graph given the embeddings of self-supervised learning.\n\n(2) I want to know how the Minimum Spanning Tree solves the problems of avoiding dimensional collapse and enhancing uniformity in self-supervised learning.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper proposes T-REGS as a regularization technique to SSL method which encourages uniformity of the representation. T-REGS maximizes the length of the minimum spanning tree, which connects to entropy maximization.

STRENGTHS AND WEAKNESSES:
**Strength**

The theoretical foundation seems to be sound.

**Weakness**

The writing of the paper can be greatly improved. E.g. proofs can be moved to appendix. In the main paper, instead of showing proofs, explain why these theoretical results are important for its application here. 

Comparing to other existing regularization techniques, T-REGS doesn't seem to result in better performance. It's unclear what's the benefit of adopting T-REGS compared to other methods.

QUESTIONS:
1. Could you elaborate how exactly the length of minimum spanning tree is computed? Which algorithm is used for finding the minimum spanning tree found given a batch of features and what's the complexity of the algorithm? Could you compare the complexity with other regularization techniques?

2. It's unclear to me why T-REGS only uses MSE as the representation learning loss. Why would it not be able to act as a generic regularization for all SSL methods?

3. Do you have any ablations for the value of alpha?

4. In many popular SSL framework, the features are normalized to unit sphere. It has also been shown such normalization is beneficial. Why does the paper propose to regularize it via the soft sphere-constraint instead of a hard constraint? Could you provide empirical evidence suggesting this way better?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03