Title: Fréchet Geodesic Boosting
Venue: NeurIPS 2025
Accepted: yes
Keywords: geodesic metric space

gradient boosting

non-Euclidean output

Wasserstein space
Abstract: Gradient boosting has become a cornerstone of machine learning, enabling base learners such as decision trees to achieve exceptional predictive performance. While existing algorithms primarily handle scalar or Euclidean outputs, increasingly prevalent complex-structured data, such as distributions, networks, and manifold-valued outputs, present challenges for traditional methods. Such non-Euclidean data lack algebraic structures such as addition, subtraction, or scalar multiplication required by standard gradient boosting frameworks. To address these challenges, we introduce Fréchet geodesic boosting (FGBoost), a novel approach tailored for outputs residing in geodesic metric spaces. FGBoost leverages geodesics as proxies for residuals and constructs ensembles in a way that respects the intrinsic geometry of the output space. Through theoretical analysis, extensive simulations, and real-world applications, we demonstrate the strong performance and adaptability of FGBoost, showcasing its potential for modeling complex data.


Peer reviews:
comments: SUMMARY:
The paper Fréchet Geodesic Boosting generalizes gradient boosting to Riemannian manifolds by introducing a novel algorithm Fréchet Geodesic Boosting (FGBoost). Traditional boosting methods operate in Euclidean spaces, where residuals are computed as vector differences. However, in non-Euclidean settings, subtraction is not well-defined. FGBoosting replaces residuals with tangent vectors derived from the Fréchet mean, allowing the model to minimize a loss defined over geodesic distances.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper clearly identifies the limitations of classical boosting in curve spaces and provides a nice solution.
2. This is a novel step in extending boosting to settings with non-Euclidean geometry.

Weaknesses:
1. It would be better a further theoretical discussion. The paper doesn't provide formal convergence guarantees.
2. The algorithm relies on having nice exponential and logs maps, which can be a limitation on certain manifolds.
3. Learning tangent vectors using regressors may introduce approximation errors, especially in curved regions where tangent space linearization is less accurate.
FGBoost is a powerful method for learning on Riemannian manifolds and significantly extends boosting to curved domains but the results present computational challenges, limited theoretical analysis, and dependence on manifold-specific operations.

QUESTIONS:
1. What assumptions are required for the manifold to ensure the algorithm is well defined? Like, completeness, injectivity radius, etc.
2. How does the algorithm ensure consistency across tangent spaces at different base points during updates?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose Fréchet Geodesic Boosting (FGBoost), a boosting method that works with outputs residing in unique geodesic spaces.\nSome brief theoretical analysis is conducted, establishing, Under the additional assumption of the geodesic space being a Hadamard space, that the underlying optimization is well-posed and a uniform convergence bound (showing that the minimizer of the empirical risk functional converges to that of the minimizer of the populational risk functional).\nA significant amount of numerical experiments are also presented, showcasing the superiority of the proposed FGBoost algorithm throughout.\nThe authors also generalize SHAP explanations to their models (a standard tool in practical applications of existing boosting models) in the supplementary material.\n\nSTRENGTHS AND WEAKNESSES:\nThe problem of boosting with non-Euclidean outputs is a very relevant one, and the focus on geodesic spaces is quite sensible.\nI find the work to be fairly easy to read, interesting and with convincing results.\nI am leaning towards acceptance, and just have a couple of points in which I would appreciate some clarification on.\n\n- On the step-wise fitting procedure: in lines 182-186, it is said that $\\hat{f}_k$ is chosen as the minimizer\n  $$ \\hat{f}\\_k = \\arg \\min\\_f \\frac{1}{n} \\sum\\_{i=1}^n d^2 ( Y\\_i, T\\_{\\hat{F}\\_{k-1}(\\mathbf{X}_i) \\oplus \\\\{\\nu \\odot f(\\mathbf{X}\\_i)\\\\}}(Y\\_0) ). $$\n  However, it is not 100% clear how this argmin is solved in practice; this is not a simple MSE minimization problem, which e.g. CART solves for trees; the outputs of $f$ gets added to the current model and then passes through the transport map $T$, to only then be piped through a (possibly non-MSE?) squared metric.\n  Overall it looks like quite a fairly tricky optimization problem, somewhat similar to those posed in the combinatorial boosting literature (e.g. [ExactBoost](https://proceedings.mlr.press/v151/csillag22a.html) and [IPBoost](https://proceedings.mlr.press/v119/pfetsch20a.html); in fact, it seems particularly close to Equation (4) in ExactBoost, which requires meta-heuristic approaches to be solved).\n  On the other hand, the authors say that they use tree fitting for FGBoost, so it feels as though I am missing something.\n\n- In the theoretical analysis, I felt the lack of a proof of convergence: FGBoost is trained in a greedy fashion, and so is (in principle) a descent algorithm. In Section 4 it is proven that the risk functional is strongly convex (and thus has a unique minimizer), but this need not imply that FGBoost will converge to it. While not critical, this would be a nice result to have. Is there some particular difficulty in proving such a result for FGBoost?\n\n- Other loss functions: one of the great boons of gradient boosting is that one can easily change the loss function being fitted without having to tweak the base learners. FGBoost, in comparison, seems hard-wired to an MSE-like objective based on the space's metric. Is this fundamental to the approach?\n\n- Minor clarifications on the proof of Proposition 2:\n    - It's not immediately clear why the first display equation (line 573-574 upper bound on $d_{\\mathcal{G}}^2 (\\gamma, \\gamma_t)$) holds; I presume it is due to the Hadamard space assumption?\n    - Why must the intermediate points $\\gamma_t$ (line 573) depend continuously on the endpoints $\\gamma_0, \\gamma_1$?\n    - Also, this proof invokes the diameter of $\\mathcal{G}(\\mathcal{M})$ (as does Theorem 2), which is stated to be finite because $\\mathcal{M}$ is bounded; however, I could not find this assumption stated elsewhere. (The assumption is fine, just needs to be stated more explicitly in the main text.)\n\nQUESTIONS:\nI'd appreciate some clarifications on the points raised in the 'Strengths&Weaknesses' text:\n- How is each iteration's argmin solved?\n- Any blockers on a convergence proof?\n- Is working with the MSE-like objective fundamental to your approach?\n- And the clarifications on the proof of Proposition 2.\n\nLIMITATIONS:\nLimitations were appropriately discussed. (Except perhaps for a minor boundedness assumption that could be more explicitly stated, but which seems fairly mild.)", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Fréchet Geodesic Boosting (FGBoost), a novel method that extends the gradient boosting framework to regression settings with Euclidean predictors and non-Euclidean outputs residing in geodesic metric spaces. The key idea is to use geodesics—the shortest paths between points in these spaces—as a proxy for residuals, which are not well-defined in non-Euclidean domains. This viewpoint allows a smooth extension of the gradient boosting framework to non-Euclidean settings. The author(s) establish theoretical guarantees for FGBoost, including the existence and uniqueness of the solution and the consistency of the risk minimizer, under the assumption that the output space is a Hadamard space. The author(s) also provide SHAP values for interpretability of FGBoost. The method's improved performance against competitors is demonstrated through three simulation studies and three real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality:** \n\nI believe this paper is of high quality. The presentation of the method and its theoretical and experimental justifications are strong. The evaluation across three different non-Euclidean output types, plus several real-world datasets, makes the demonstrated performance advantages convincing.\n\n**Clarity:** \n\nThe paper is very clear and well-written. The main contributions are highlighted and previous work in regression in non-Euclidean settings is presented. The development of FGBoost is smooth and well-motivated, drawing a clear link that FGBoost is a generalization of standard gradient boosting methods for geodesic metric spaces. Both theoretical and experimental justifications are convincing, and the three clear examples are helpful. I also appreciate the discussions on scalability and limitations in the Appendix, though it would help with transparency if a reference to this was added in the main text.\n\n**Significance + Originality:**\n\nI’m not very familiar with the related literature, but it seems that the work is novel in tackling general regression problems with Euclidean predictors and non-Euclidean outputs using a boosting framework. However, the paper references Matsubara (2024), which has a very similar motivation and core insight, leveraging Wasserstein gradients in the space of distributions. This could diminish the novelty a bit, as a key idea of using a geometric proxy for residuals is captured in that prior work. Still, this paper's contribution is distinct in its generality beyond distributions and its theoretical guarantees.\n\nI think a couple points may limit the method's scope of impact. \n\n- While placing FGBoost in the general context of geodesic metric spaces is a strength, for actual usage, it seems a practitioner would have to derive the geodesic transport map if their problem is not one of the three examples provided. This may limit the adoptability of the method, and more guidance here could be helpful.\n- It is also not clear how other metrics might impact FGBoost, especially those that result in non-Hadamard spaces where the theoretical guarantees do not hold. An experiment exploring this could possibly strengthen the paper by defining the practical boundaries of the algorithm.\n\n[1] Takuo Matsubara. Wasserstein gradient boosting: A framework for distribution-valued supervised learning. In *Advances in Neural Information Processing Systems*, 2024.\n\nQUESTIONS:\nI alluded to some of these questions in my strengths and weaknesses discussion:\n\n- The theoretical justification of the methodology seems grounded in Hadamard spaces. The paper does well in explaining the three settings presented. However, I wonder how robust the results are to metrics that are not those listed. Perhaps an experiment with a metric that does not lead to a Hadamard space could be helpful to see how robust this approach could be. A response here would clarify the practical boundaries of the method.\n\n- The general framework for geodesic metric spaces is great, but it seems a practitioner would have to derive the geodesic transport map themselves if their problem isn't one of the three examples in the paper. This might limit how widely the method is adopted. Perhaps more guidance on the principles for constructing these maps could be helpful for new users. Addressing this could make the work much more accessible.\n\n- The implementation of tree-based FGBoost relies on finding the Fréchet mean of residual geodesics in each leaf. In the analysis, the chosen metric for the space of geodesics, $d_\\mathcal{G}$ conveniently decouples the metric to align with the empirical loss. Does this decoupling on the geodesic space have any tradeoffs among computability/convenience and geometry of geodesics space? May not be necessary, but a comment or two could help clarify here.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new gradient boosting framework which adapts for non-Euclidean output spaces. The new framework starts from a Frechet mean of regression labels, and adds geodesics in each boosting round to the current prediction of each sample. FGBoost uses decision trees as base learners to partition the data into regions. Optimal geodesics to add are calculated for samples in each region as the Frechet mean of the residual geodesics of these samples. When the output space is a Hadamard space, there exists a consistent esitmator to the global unique minimizer of the boosting loss.\n\nNumerical experiments are conducted in both simulation datasets and real-world applications, covering probability and graph outputs. FGBoost shows superior performance over existing regression models for non-Euclidean outputs.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\n1. As a powerful out-of-the-shelf machine learning algorithm, gradient boosting has been applied to many regression/classification/ranking problems. It is definitely quite valuable to extend gradient boosting to more diverse output spaces. In this case, a geodesic space.\n2. Gradient boosting (with decision trees) has been always taken as black-box machine learning tools without prior knowledge applied in the algorithm. However, the results in Appendix H show the potential advantage of explicit geometry awareness through regression in the graph space, over simple vectorized numerial representations of graph with existing gradient boosting algorithm. I personally think this is interesting if the advantage can be more widely and thoroughly verified.\n3. The approach lays on solid theoretical foundations. The adaption from traditional gradient boosting is natural and clear.\n4. The paper is clearly written.\n\nWeknesses:\n1. Some details are missing, i.e., how the trees are constructed in FGBoost.\n2. It would be better if FGBoost could be evaluated over more real-world applications, as well as comparing with traditional gradient boosting approaches.\n3. The requirement of geodesic transport map may limit the application of the algorithm. For example, the geodesic transport map of distributions requires the calculation of quantiles, which is not trivial for most of the probability distributions.\n4. Calculation of Frechet mean is more expensive than calculation of mean of scalar values in traditional gradient boosting. Thus FGBoost  could be more expensive. It would be good to think how to implement FGBoost efficiently. Especially when facing different metric spaces, the definition of distance can be very different. It can be hard to propose a unified library or framework which can handle all metric spaces efficiently.\n\nQUESTIONS:\n1. Why are all the SHAP values shown in Figure 2 positive? Are the absolute values plotted? Or just the SHAP values of features with highest positive values are plotted?\n2. How the trees are constructed in FGBoost?\n3. In the experiment comparing with XGBoost, XGBoost is trained to fit each value of the Laplacian matrix. It would be interesting to see how the performance changes with traditional gradient boosting when fitting multiple values together, given the recent progress of multiple output gradient boosting decision trees (e.g., https://proceedings.neurips.cc/paper_files/paper/2022/file/a36c3dbe676fa8445715a31a90c66ab3-Paper-Conference.pdf).\n\nLIMITATIONS:\n1. It would be better to further address the limitations due to geodesic transport map.\n2. It would be better to discuss the potential of efficient implementation of FGBoost, given that existing gradient boosting algorithms are know for its highly optimized implementations for fast training.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-10"}