Title: Bridging Arbitrary and Tree Metrics via Differentiable Gromov Hyperbolicity
Venue: NeurIPS 2025
Accepted: yes
Keywords: tree metric learning

delta-hyperbolicity

geometrical machine learning

differentiation
Abstract: Trees and the associated shortest-path tree metrics provide a powerful framework for representing hierarchical and combinatorial structures in data. Given an arbitrary metric space, its deviation from a tree metric can be quantified by Gromov’s $\delta$-hyperbolicity. Nonetheless, designing algorithms that bridge an arbitrary metric to its closest tree metric is still a vivid subject of interest, as most common approaches are either heuristical and lack guarantees, or perform moderately well. In this work, we introduce a novel differentiable optimization framework, coined DeltaZero, that solves this problem. Our method leverages a smooth surrogate for Gromov’s $\delta$-hyperbolicity which enables a gradient-based optimization, with a tractable complexity. The corresponding optimization procedure is derived from a problem with better worst case guarantees than existing bounds, and is justified statistically. Experiments on synthetic and real-world datasets demonstrate that our method consistently achieves state-of-the-art distortion.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a tree metric learning method based on Gromov hyperbolicity. The authors begin by formulating the problem as minimum distortion tree metric approximation under the $\\ell_\\infty$ norm. Then, they introduce a smooth relaxation using the log-sum-exp function, along with a batched version of Gromov hyperbolicity for differentiable and efficient computation. The method is evaluated on five unweighted graph datasets and two non-graph metric datasets, with results reported in terms of $\\ell_\\infty$ error.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength\n\nTree and tree metric learning is a specific form of hierarchical representation learning, which plays an important role in many machine learning tasks, especially given the prevalence of hierarchical structure in real-world data. The problem addressed in this paper is relevant to both the representation learning and hyperbolic geometry communities. The paper is clearly written and easy to follow. A key strength is the theoretical guarantee provided in Theorem 3.3, which shows that the proposed batched formulation asymptotically approximates Gromov hyperbolicity. I believe the theoretical contribution is solid and should be appreciated.\n \n## Weakness\n- In the formulation of the Minimum Distortion Tree Metric Approximation problem (line 35), the choice of using the $\\|\\|\\cdot||\\_\\infty$  norm is not clearly justified. Many hyperbolic representation learning methods measure distortion using average distortion rather than worst-case. Since the proposed differentiable approximation of Gromov hyperbolicity is built upon the $\\|\\|\\cdot||\\_\\infty$ norm, it would be important to explain why alternative norms (e.g., average distortion) are not suitable or less appropriate for this setting.\n- The Gromov tree embedding referenced in Theorem 3.1 is not defined. Specifically, the construction of the mapping Φ is not described, which makes it difficult to follow the basis of the theorem.\n- The optimization problem in Equation 3 and the Minimum Distortion Tree Metric Approximation problem (line 35) are based on the $\\|\\|\\cdot||\\_\\infty$  norm, which motivates the use of a smooth relaxation due to the max and min operations. However, the final optimization objective in Equation 6 switches to using $\\|\\|\\cdot||\\_2\\^2$, and the paper does not clarify this shift. If the initial motivation was centered around $\\|\\|\\cdot||\\_\\infty$, the rationale for optimizing under a different norm should be made explicit.\n- In Table 2, the $\\ell_1$ average error is reported, but this metric is not formally defined in the paper. Moreover, based on this metric, the NJ method performs best and significantly outperforms the proposed method. Although NJ does not guarantee low worst-case distortion, the paper would benefit from a discussion or example where minimizing worst-case distortion (as targeted by the proposed method) is practically important.\n- While the proposed approach focuses on tree metric learning, it is based on hyperbolicity, and the paper does not engage with related work in hyperbolic representation learning. Some prior methods also construct trees using hyperbolic space, and some prior hyperbolic representation learning methods can be trained in a differentiable manner. A discussion of these approaches and a comparison with relevant baselines would strengthen the paper and better position its contributions within the broader literature.\n\nQUESTIONS:\n- Mean Average Precision (MAP) is a widely used evaluation metric in representation learning, especially for measuring tree and hyperbolic embedding quality. Could you report the MAP performance of your method compared to baselines?\n- What are the δ-hyperbolicity or Gromov hyperbolicity values of the datasets used in your experiments? This would help contextualize how suitable these datasets are for tree-based representations.\n- In Table 1, the LayeringTree method has missing entries (marked as “–”) for the ZEISEL and IBD datasets. Could you explain why these results are not reported?\n- Are there any assumptions made about how data points are mapped to the tree? For instance, are points always assumed to be the leaves of the tree, or can they also include internal nodes?\n- Is the resulting tree and its associated tree metric unique, given a distance matrix, or is the solution dependent on initialization or optimization variability?\n- Beyond distance approximation, do you see your method being applicable to standard graph representation tasks such as node classification or link prediction? If so, how might your approach contribute to improved performance in those settings?\n\nLIMITATIONS:\nThe author discuss the scalability issue for the proposed method in Section 5.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is motivated by the problem of finding a tree metric space embedding of a given arbitrary finite metric space that preserves the original pairwise distances. To this end, the authors propose an algorithm that consists of two major steps: 1) finding a surrogate finite metric space that best preserves the original distance and meanwhile has as close to a tree structure as possible; 2) embedding this surrogate into an actual tree using the existing Gromov’s tree embedding. The main contribution is in the first step. \"preserving the original distances\" is enforced by $l^\\infty$ norm (in implementation $l^2$ norm is used for smoothness) and closeness to a tree is measured by Gromov hyperbolicity, which is not smooth by the min and max function in the definition. The authors formulate this problem as a constrained optimization problem with an objective as a balanced sum of these two loss components, where the Gromov hyperbolicity is replaced by a smooth version using the log-sum-exp operator and minibatch to alleviate the computation burden. Then the loss is differentiable and the optimization can be solved using projected gradient descent. The experiments show the benefit of this method in hierarchical clustering, and the quality of the embedding itself.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper is well motivated. The embedding of an arbitrary metric into a tree is an interesting problem that is important in revealing the hierarchical structures of data, enabling fast tree algorithms, etc.\n2. It reads very well, and I love that the authors care to give intuitive explanations to definitions, algorithms.\n3. The smoothed version of Gromov hyperbolicity is novel.\n4. Theoretical bounds are given for each approximation step of the Gromov hyperbolicity. The authors also provide critical analysis of the bounds, e.g. the restrictiveness of the assumptions. \n5. Experiments show significantly improved accuracy of hierarchical clustering in synthetic data, and superior quality of the tree embedding of the proposed method over baselines. Sensitivity analysis is provided for better understanding of the parameters.\n\n\nWeaknesses:\n1. Although the optimization scheme may provide better worst-case bounds that give theoretical guarantees, in real-world applications the average-case bound is more relevant, and as shown in Table 2, the proposed method is not doing well.\n2. Even after applying the batched approximation to reduce the computation complexity, the proposed algorithm takes ten to a hundred times longer than the baselines (Table 3). This hinders the application in large-scale data.\n3. It would be better if the authors could show the applications such as hierarchical clustering in real datasets. But the experiments are sufficient in their current form.\n\nQUESTIONS:\n1. In the batch sampling, the author mentioned the K subsets need to be independent, can you clarify what the independence here means? In Algorithm 1, the sampling seems just to be random, how is the independence enforced?\n2. Can the authors explain what the blue vertices in the optimized tree of DeltaZero denote? Based on my understanding, the method preserves the number of points as the original input. I wonder why it would give rise to additional nodes.\n3. Can the authors provide more discussion on why as all equivalent norms, minimizing the $l^2$ norm would yield better $l^\\infty$ distortion but worse $l^1$ distortion? Also, how does using $l^2$ norm in implementations affect Theorem 3.1, if not, why don't you use $l^2$ norm in Equation (3) from the start?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the minimum-distortion tree metric approximation problem: given a finite arbitrary metric space $(X, d_X)$, the goal is to approximate it via a tree metric $d_T$ (with embedding $\\Phi : X \\to T$) that minimizes the worst-case distortion\n\\[\n\\max_{x,y} \\bigl| d_X(x,y) - d_T\\bigl(\\Phi(x), \\Phi(y)\\bigr)\\bigr|.\n\\]\nThis is motivated by the fact that tree metrics capture hierarchical structure, and any deviation from a tree metric can be measured by Gromov’s $\\delta$-hyperbolicity (which is zero if and only if the metric is tree-like).\n\nIn this paper, the authors propose DELTAZERO, a differentiable optimization approach that ensures theoretical distortion guarantees over prior methods (for example, Neighbor-Joining, TreeRep, LayeringTree). It is a two-step process that jointly adjusts the metric to remain close to the original while enforcing small hyperbolicity (the trade-off between tree-likeness and fidelity to the original metric is governed by a regularization parameter $\\mu$). In the first step, they introduce a surrogate metric space $\\left(X', d_{X'}\\right)$ and solve the optimization problem:\n\\[\n\\min_{d_{X'} \\le d_X} \\quad \\mu \\,\\|d_X - d_{X'}\\|_\\infty \\;+\\; \\delta\\left( d_{X'} \\right).\n\\]\nTo make this solvable by gradient-based methods, the authors replace the max/min definitions of $\\delta$ with smooth log-sum-exp approximations and further approximate global hyperbolicity via batch sampling of point subsets. In practice, they optimize the distance matrix $D$ (subject to metric constraints) by projected gradient descent: each step uses ADAM, followed by projection onto the space of valid metrics via Floyd–Warshall. Finally, in the last step, the optimized metric $D^*$ is converted to a tree metric (for example, via a single-linkage hierarchical clustering procedure).\n\nThe paper is generally well-written and organized. Formal definitions are provided, which helps orient the reader. However, the content is mathematically heavy, and a reader may need a background in hyperbolic metrics to follow all the details.\n\nThe paper addresses a relevant problem important in several fields such as hierarchical clustering, phylogenetics, and network modeling. The approach of linking Gromov hyperbolicity to differentiable optimization is new and indicates high potential significance. Provided the experimental results hold up, the contribution has clear practical value.\n\nThe submission contains some original ideas as well as effective use of established techniques, combining them to achieve the results.\n\nSTRENGTHS AND WEAKNESSES:\nThe submission seems to be technically sound overall.  The definitions and background are clearly stated, and proofs are deferred to appendices, indicating mathematical care. However, I have a few questions regarding some of the claims made.\n\n\nStrengths:\n\n1) The paper introduces a fresh perspective by combining Gromov hyperbolicity with differentiable optimization. Formulating the tree-approximation as a smooth optimization is innovative. The use of a log-sum-exp surrogate for hyperbolicity to make it differentiable for enabling gradient descent is an elegant idea. Unlike previous heuristics, this approach provides a framework that directly ties the distortion objective to a geometric measure (hyperbolicity). \n\n\n2) The experimental results (as provided by the author(s) substantiate the claims. DELTAZERO consistently outperforms the standard algorithms currently in practice.\n\n\n3) It does have some diverse and important future work potential.\n\n\nWeakness:\n\n1) Proposition 3.4 states that $\\delta(D)$ is piecewise-affine but non-convex, so the smoothed surrogate remains non-convex. As a result, gradient descent may get stuck in local minima. The paper does not discuss initialization or how sensitive the results are to local optima. It uses ADAM and early stopping, but there is no guarantee the global optimum of the metric loss is found. \n\n\n2) The method introduces several hyperparameters (batch size $m$, number of batches $K$, smoothing $\\lambda$, tradeoff $\\mu$, learning rate, etc.) Tuning these is nontrivial. The paper uses grid-search to find the best settings for each dataset. It is unclear how performance goes down if these are not optimally tuned. This reliance on manual hyperparameter search may lead to some limitations. Additionally, the condition in Theorem 3.1 for improved distortion ($\\mu\\geq 1/(2\\log(n-2))$) may not be satisfied in practice, and its role is not deeply discussed.\n\n\n3) While the method is compared against several baselines, the discussion of limitations of existing methods is brief. For instance, it is claimed that NJ, TREEREP, etc., “lack theoretical guarantees”. which is true for worst-case distortion but somewhat overstated in practical terms. Also, some baselines (like HCCROOTEDTREEFIT) depend heavily on root choice, but no analysis is given on how robust DELTAZERO is to analogous choices (e.g. the choice of basepoint $w$ in the final embedding). The “classical GROMOV” method mentioned is not fully explained in the main text (only in an appendix), making it unclear whether comparisons are fully fair.\n\n\n4) The paper states that the surrogate metric is “eventually embedded into an actual tree” and mentions single-linkage clustering. However, details on the final embedding step (and its complexity) are omitted.This part invokes a variant of Gromov’s algorithm, but this area is only briefly sketched.\n\n\n5) Though the current best known bound for computing Gromov hyperbolicity (which is $n^{3.69}$) is beaten, still each projected gradient step requires an $O(n^3)$ Floyd–Warshall projection to enforce triangle inequalities. While mini-batching reduces the cost of hyperbolicity estimation, “scalability to very large graphs” is still constrained by the cubic projection cost.\n\n\n6) In some cases the improvements are small. For example, on the CORA graph DELTAZERO’s worst-case distortion (7.59) is only slightly better than the second-best (7.76 by LayeringTree), a 2.3% gain. This suggests that when the input metric is already very hyperbolic (small diameter relative to $n$), room for improvement might be limited. Also, In grid-like graph cases DELTAZERO is outperformed by LAYERINGTREE. The variability across datasets hints that in some regimes  the benefit may not be substantial.\n\nQUESTIONS:\n1) Please explain the choice of parameters.\n\n\n2) How do you ensure the $\\mu\\geq 1/(2\\log(n-2))$ criterion holds?\n\n\n3) How do you tackle the non-convexity of Gromov hyperbolicity $\\delta_D$?\n\nLIMITATIONS:\nThe submission does not specify the scheme of choosing optimal parameters, and how the non-convexity is tackled is not clear. Also, it doesn’t claim to perform well in general settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DELTAZERO, a novel differentiable optimization framework for approximating arbitrary metric spaces with tree metrics by leveraging a smooth surrogate for Gromov's $\\delta$-hyperbolicity. The method is evaluated on both synthetic and real-world datasets, demonstrating state-of-the-art performance in minimizing distortion.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The method addresses a fundamental problem in hierarchical representation learning.\n\n2. The work provides theoretical foundations, including Proposition 3.2 bounding the approximation error, and Theorem 3.1 showing improved distortion guarantees.\n\n---\n\n**Weaknesses**\n\n1. The method feels like it's combining existing techniques (projected gradient descent + Floyd-Warshall) rather than introducing something fundamentally new.\n\n2. The $O(Km^4 + n^3)$ runtime seems pretty heavy - especially that $n^3$ part. For real-world use, we'd need to know how this scales with bigger graphs. Some scalability tests would really help show if this is practical.\n\n3. Runtime performance is concerning - the method takes over 5600 seconds to process just 3005 nodes (Table 3 in Appendix D.1). This makes it impractical for real-world applications where we typically need to handle much larger graphs.\n\n4. The paper doesn't discuss how quickly the method converges. It would be helpful to see how the solution quality improves with more iterations (T). Some analysis here would give users better intuition about tuning this parameter.\n\nQUESTIONS:\n1. Why use $\\ell_\\infty$ norm instead of Frobenius norm $||D_X-D||_F^2$ like metric nearness problems [1,2]?\n\n2. Are there any other tree embedding method can be combined instead of Gromov’s method?\n\n3. Is it possible to directly use the four-point conditions as the optimization constraints?\n\n[1] \"The metric nearness problem.\" SIAM Journal on Matrix Analysis and Applications, 2008.\n\n[2] \"Metric nearness made practical.\" AAAI, 2023.\n\nLIMITATIONS:\nThe primary limitations concern scalability and computational efficiency.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-10"}