Title: Riemannian Consistency Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: Consistency Model

Flow Matching

Generative Model

Riemannian Manifold
Abstract: Consistency models are a class of generative models that enable few-step generation for diffusion and flow matching models. While consistency models have achieved promising results on Euclidean domains like images, their applications to Riemannian manifolds remain challenging due to the curved geometry. In this work, we propose the Riemannian Consistency Model (RCM), which, for the first time, enables few-step consistency modeling while respecting the intrinsic manifold constraint imposed by the Riemannian geometry. Leveraging the covariant derivative and exponential-map-based parameterization, we derive the closed-form solutions for both discrete- and continuous-time training objectives for RCM. We then demonstrate theoretical equivalence between the two variants of RCM: Riemannian consistency distillation (RCD) that relies on a teacher model to approximate the marginal vector field, and Riemannian consistency training (RCT) that utilizes the conditional vector field for training. We further propose a simplified training objective that eliminates the need for the complicated differential calculation. Finally, we provide a unique kinematics perspective for interpreting the RCM objective, offering new theoretical angles. Through extensive experiments, we manifest the superior generative quality of RCM in few-step generation on various non-Euclidean manifolds, including flat-tori, spheres, and the 3D rotation group SO(3), spanning a variety of crucial real-world applications such as RNA and protein generation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a new way to train few-step generative models on Riemannian manifolds, taking inspiration from and generalising existing literature on consistency models. They show a simplified training objective that allows for simpler and faster computation, and, for small ($N = 2$), equal amounts of steps, they show that their method outperforms typical Riemannian flow matching.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper presents a relatively simple and intuitive method to train a few-step generative model on a Riemannian manifold, which is a relatively unexplored topic.\n2. The proposed method seems to improve significantly on its usual flow matching counterpart.\n3. The authors illustrate with understandable examples their approach in this work, namely in section 3.3, and they present differential geometry in rather clear terms.\n\nWeaknesses:\n\n1. I believe that the need for few-step generative models on Riemannian geometries could have been better motivated in the introduction.\n2. There are no theoretical guarantees relating to the learnt distribution of the model, although I do not believe it is of primary importance.\n3. I find the argument at the end of section 3.1 (“The proof above immediately…”) particularly hand-wavy, and do not get how equation 8 is a natural consequence.\n4. There is not much new conceptual content introduced in this work—which would have been fine if the experiments were more extensive.\n5. The experiments are not extensive enough, as they show only low-dimensional cases (sphere, flat torus and $\\mathrm{SO}(3)$).\n6. The experiments include no comparable baselines, such as [1] (cited in section 5, which is perhaps the most important one to include), or Euclidean few-step generators [2, 3]. It is also quite unclear which consistency method has been used as a baseline for “Euclidean consistency models”: is it the authors’ framework on an Euclidean geometry, or another method, which only works on Euclidean geometry?\n\nI believe that I would be happy to reconsider my assessment if given more empirical evidence: more datasets (in higher dimensions), more comparable baselines.\n\nQUESTIONS:\n1. Do you have any theoretical results in terms of the learnt distribution? It seems that the authors have not delved too deep in Probability Theory over Riemannian manifolds—which is fine, but I think it would consolidate the paper theoretically rather substantially.\n2. Have you tried any other datasets, such as those in [4]?\n3. Have you tried to run [1] on the datasets you use?\n4. To compare the different methods, it would have been particularly convenient to also obtain the NLL on the test sets. Is it possible to evaluate the NLL using your method?\n5. I see no errors on your numbers. Have you re-run your experiments to obtain confidence intervals? They should be relatively cheap, considering the dimensionality of the considered datasets.\n\nLIMITATIONS:\nIt is unclear whether the method could be scaled up to higher dimensions.\n\nReferences:\n---\n\n[1] Peter Sorrenson, Felix Draxler, Armand Rousselot, Sander Hummerich, Ullrich Köthe (2023). Learning distributions on manifolds with free-form flows.\n\n[2] Linqi Zhou, Stefano Ermon, Jiaming Song (2025). Inductive Moment Matching\n\n[3] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, Stefano Ermon (2023). Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion.\n\n[4] Ricky T. Q. Chen, Yaron Lipman (2023). Flow matching on General Geometries.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Riemannian Consistency Models (RCM) by extending the traditional consistency model to Riemannian manifolds. In the process, they derive a closed-form continuous-time objective tailored to manifold-specific operations. By applying the marginalization trick, the authors prove the mathematical equivalence of Riemannian consistency training and distillation. Additionally, a simpler objective is proposed that avoids computing differentials of the exponential map, yet maintains performance, with a kinematic interpretation offering intuitive insight into the method.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n> The paper is very well-written, easy to follow, with sufficient discussion on similar approaches extending diffusion/flow models in non-Euclidean data supports. Supplementary notes on rudimentary notions of geometry are appreciable.\n\n**Weakness**\n>- While the motivation to adopt consistency models, amenable to data supported on a Riemannian manifold, is clear, the paper lacks empirical validation. For example, what is the extent of improvement in inference time (or convergence) over both Riemannian diffusion models [1] and/or vanilla consistency models [2].  \n>- Given that the calculations in a non-Euclidean space are far more computationally demanding compared to their Euclidean counterparts, the paper must show that the advantages of adopting this approach outweigh the added processing cost. This is missing in the current article.\n>- Computational infrastructure under which experiments were carried out, code repositories, and specific execution details seem to be missing in the current article. \n\n\n> [1] Huang, Chin-Wei, Milad Aghajohari, Joey Bose, Prakash Panangaden, and Aaron C. Courville. Riemannian diffusion models. Advances in Neural Information Processing Systems 35 (2022): 2750-2761\n>\n> [2] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. 2023. Consistency models. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), Vol. 202. JMLR.org, Article 1335, 32211–32252.\n\nQUESTIONS:\n- _[Line 86-87]_ Kindly define the weights $w_t$. Do they follow a similar specification to that in [1] or [3]?\n- _[Line 127]_ As pertaining will always include an inherent additional error due to generalization to domains, what should be a sufficient condition for $\\dot{x}$ such that RCM converges to the target distribution? \n- _[Line 245]_ Is there any rationale behind sticking to the linear scheduler? \n- _[Line 252]_ The choice of KDE to obtain density estimates is questionable, since in higher dimensions (even as low as 10-15), choosing suitable bandwidths becomes difficult. At best, carefully chosen kernels can recover error bounds adapted to intrinsic dimensions under the manifold hypothesis (see [4]), theoretically. In practice, there is hardly any improvement.   \n- There remains the issue of training instability that can occur when consistency models are trained from scratch in a naïve manner—specifically, by directly optimizing the objective as in (4) while closely adhering to $\\Delta t \\approx 0$. The authors of [3] call it the Curse of Consistency. In particular, the model tends to converge slowly after a certain point, perhaps due to accumulated consistency errors from each interval. Does the proposed model experience anything similar, since it follows a simple training regime akin to [2] (following _Line 87_,  $\\Delta t$ is quite small for large $N$)? \n- Heuristically speaking, deep generative models often can be observed to work well enough, being agnostic to the fact that underlying data distributions are supported on intrinsic structures embedded into the ambient space. They silently adopt the intrinsic dimensions in their error rates (e.g., GANs [5], WAE [6], etc. However, to my knowledge, error bounds on diffusion model-based generations are yet to be shown to follow the same.). In this context, does RCM really generalize the Euclidean case _[Line 139]_ or rather extend the framework onto manifolds? Given the additional costs due to hyperbolic/Riemannian operations, is it really meaningful to propose a new model altogether? \n- Also, kindly respond to the points raised as 'Weakness'.\n\n> [3] Zhengyang Geng, Ashwini Pokle, Weijian Luo, Justin Lin, and J Zico Kolter. 2025. Consistency Models Made Easy. In The Thirteenth International Conference on Learning Representations. \n> \n> [4] Jisu Kim, Jaehyeok Shin, Alessandro Rinaldo, and Larry Wasserman. Uniform convergence rate of the kernel density estimator adaptive to intrinsic volume dimension. In International Conference on Machine Learning, pp. 3398-3407. PMLR, 2019.\n>\n> [5] Jian Huang, Yuling Jiao, Zhen Li, Shiao Liu, Yang Wang, and Yunfei Yang. An error analysis of generative adversarial networks for learning distributions. Journal of machine learning research 23, no. 116 (2022): 1-43.\n>\n> [6] Saptarshi Chakraborty and Peter Bartlett. A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data. In The Twelfth International Conference on Learning Representations. 2024.\n\nLIMITATIONS:\nWhile the article does not have a dedicated section explaining limitations, the apparent ones mostly relate to empirical validation.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper proposes the Riemannian Consistency Model (RCM), allowing few-step generative modeling on Riemannian manifolds (e.g., spheres, tori, SO(3)). It adapts consistency models to respect manifold geometry using exponential maps and covariant derivatives, and includes both theoretical and practical improvements. RCM outperforms previous Euclidean and flow matching baselines in experiments.

STRENGTHS AND WEAKNESSES:
Strengths:
- Extends consistency models to non-Euclidean manifolds and provides the simplified training objective
- Strong theoretical analysis and proofs to support the proposed approach
- Results on multiple manifolds and datasets with respect to the selected baselines

Weaknesses:
- The main experimental limitations of the RCM paper, compared to RFM, are a narrower focus on simple manifolds, lack of high-dimensional and general (mesh/bounded) manifold experiments after the literature [1].
- Missing baselines: Diffusion Model, Riemannian Diffusion Model, Riemannian Score-Model, etc.
- Missing ablation: (NFE vs. Performance) plot could give a better overview of the effectiveness of the proposed approach. 

[1] Chen, Ricky TQ, and Yaron Lipman. "Riemannian flow matching on general geometries." arXiv e-prints (2023): arXiv-2302.

QUESTIONS:
Please see the weaknesses.

LIMITATIONS:
Yes, although authors mention experiments on larger datasets are their limitations, however, baseline [1] do report performance on these datasets. I would advise authors to include at least one such dataset to showcase the scalability and others can fall under this limitation.
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Riemannian Consistency Model (RCM), a framework to extend consistency models to general Riemannian manifolds for fast, few-step generation. The authors present a principled approach using the exponential map and covariant derivative to correctly enforce the consistency objective while respecting the manifold's geometry. The work derives closed-form objectives for both distillation-based (RCD) and training-from-scratch (RCT) versions of the model and proves their theoretical equivalence. A key practical contribution is a simplified objective that is easier to implement without sacrificing performance. Through extensive experiments on spheres, tori, and the SO(3) rotation group, the paper demonstrates that RCM significantly outperforms naive baselines and few-step flow matching, highlighting the importance of a geometry-aware approach for generative modeling on non-Euclidean data.\n\nSTRENGTHS AND WEAKNESSES:\n## **Strengths**\n\n- As far as I am aware, this work appears to be the first to develop a comprehensive and theoretically sound framework for consistency models on general Riemannian manifolds. This is a significant step forward, as it enables fast, high-quality generative modeling for a wide range of important scientific data (e.g., protein structures, robotics, geoscience) that naturally reside on non-Euclidean domains. \n- The paper is technically very strong; both the the continuous-time RCM objective which seems a highly non-trivial result, but also the equivalence between the distillation (RCD) and training (RCT) variants  to further solidify the theoretical foundation. Also the kinematic perspective is really great, and the author are able to present rather intricate formalisms in a physical-intuitive manner making the paper quite accessible.  \n-  The authors demonstrate strong practical utility by not only presenting the full, complex objective but also proposing a simplified version (sRCM). This simplified loss eliminates the need to compute the differentials of the exponential map, which can be a significant implementation issue, yet it achieves excellent empirical results. \n\n\n## **Weaknesses** \n- The appendix shows that the method requires non-standard techniques like \"Tangent Warmup\" and \"Tangent Clipping\" to ensure stable training. The need for these specialized heuristics suggests that the proposed loss functions, while theoretically derived, may be inherently difficult to optimize and sensitive to the magnitude of the geometric correction terms. Could the authors reflect on how hard it is to work with the method in practice? \n-  The practical application of RCM is challenging. First, applying the model to a new manifold requires deriving the specific forms of the exponential map, logarithm map, and Christoffel symbols for the covariant derivative. Can the author illustrate more for what manifolds this is doable?  \n- The empirical evaluation, while covering several manifolds, is limited in two key ways. First, the experiments are almost exclusively focused on a 2-step generation scenario, with no analysis of the quality vs. computation trade-off at other few-step counts (e.g., 4, 8, 16 steps), which would give significant insight in the general trend in comparison. Second, the datasets used are of a relatively small scale and low dimensionality. This makes it difficult to assess the method's scalability and performance characteristics on larger, more complex problems.\n- The paper seems to not analyze the runtime complexity of the proposed method. The calculation of the covariant derivative term at each training step is computationally more expensive than the simple time derivative used in Euclidean models. This introduces an additional, unquantified computational overhead that may become significant for high-dimensional manifolds.\n\nQUESTIONS:\n- Your appendix mentions the use of \"Tangent Warmup\" and \"Tangent Clipping\" for stable training. Could you reflect on the practical difficulty of working with the proposed objectives? What is the primary source of the instability that necessitates these heuristics, and does the simplified sRCM objective require them to the same extent as the full RCM?\n- The paper successfully applies RCM to tori, spheres, and SO(3). For which other classes of manifolds are the required geometric quantities (exponential/logarithm maps, Christoffel symbols) readily available or straightforward to derive? A brief discussion on the scope of manifolds where RCM is practically applicable would be insightful.\n- The experiments focus almost exclusively on a 2-step generation scenario. How does the model's performance and the quality of the generated samples evolve as the number of inference steps increases to 4, 8, or 16? An analysis of this quality-vs-computation trade-off is crucial for understanding the model's behavior.\n- Could you provide an analysis of the runtime complexity introduced by the covariant derivative calculation during training? How does this computational overhead compare to the Euclidean consistency models, and how is it expected to scale with the dimensionality of the manifold?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper presents an extension of Consistency Models to the case of non-Euclidean Riemannian manifolds. The ability to accelerate diffusion/flow matching on such manifolds is impactful for a number of scientific applications ranging from protein generation to astrophysics and had until this paper not received sufficient attention. The paper develops the key results necessary to derive both consistency distillation and consistency training on Riemannian manifolds, introduces simplified objectives that bypass some of complexities of geometric calculus, and performs adequate evaluations of the different variants of the proposed models on established and appropriate benchmark problems.

STRENGTHS AND WEAKNESSES:
**Strengths**:
- The paper is very well written, provides appropriate background and survey of the related literature, and includes useful insights to help the reader understand the different aspects of the consistency loss on Riemanian manifolds. 
- The derivations are mathematically sound (as far as I can tell), providing the necessary key results to build consistency models on Riemannian manifolds.
- The evaluations are appropriate and demonstrate 1. the gain of using a proper riemannian distillation strategy over a naive implementation of a Euclidean consistency model in the ambient Euclidean space, 2. the speed up made possible by consistency modeling compared to standard flow matching.

This is a well rounded paper, I do not have significant weaknesses to point out.

QUESTIONS:
- I noticed a missing stop gradient in Eq 2

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-07