Title: Riemannian Proximal Sampler for High-accuracy Sampling on Manifolds
Venue: NeurIPS 2025
Accepted: yes
Keywords: High-accuracy sampling

Riemannian manifolds

Proximal Sampler

Heat-kernels
Abstract: We introduce the \textit{Riemannian Proximal Sampler}, a method for sampling from densities defined on Riemannian manifolds. The performance of this sampler critically depends on two key oracles: the \textit{Manifold Brownian Increments (MBI)} oracle and the \textit{Riemannian Heat-kernel (RHK)} oracle. We establish high-accuracy sampling guarantees for the Riemannian Proximal Sampler, showing that generating samples with \(\varepsilon\)-accuracy requires \(\mathcal{O}(\log(1/\varepsilon))\) iterations in Kullback-Leibler divergence assuming access to exact oracles and \(\mathcal{O}(\log^2(1/\varepsilon))\) iterations in the total variation metric assuming access to sufficiently accurate inexact oracles. Furthermore, we present practical implementations of these oracles by leveraging heat-kernel truncation and Varadhan’s asymptotics. In the latter case, we interpret the Riemannian Proximal Sampler as a discretization of the entropy-regularized Riemannian Proximal Point Method on the associated Wasserstein space. We provide preliminary numerical results that illustrate the effectiveness of the proposed methodology.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers sampling from distributions of the form $\\exp(-f)$ over spherical manifolds. Under the high-accuracy regime, the authors provide a Riemannian proximal sampler that produces a sample whose law is $\\epsilon$-close to the target distribution in TV distance within an iteration complexity polylogarithmic in $\\epsilon$. It is well-known that the major difficulty in manifold Langevin-based sampling is the intractability of simulating manifold Brownian motion, as the kernel generally lacks a closed-form expression. The authors overcome the issue by truncating the heat kernel and show that an ideal proximal sampler achieves polylog complexity with an inexact proximal operator and the truncated manifold Brownian motion. They further instantiate the algorithm using rejection sampling and show that the implementation preserves the same iteration complexity.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nOverall, this paper is well-written and easy to follow. It should be the first to show that an ideal algorithm can achieve linear convergence by incorporating the inexact proximal operator on a Riemannian manifold. The authors demonstrate strong mathematical maturity and solid knowledge of both differential geometry and numerical methods. The theoretical analysis appears to be rigorous, and the proofs are generally convincing.\n\n**Weaknesses**\n\nHowever, I remain dubious about the novelty and the broader significance of this work. First, the results seem to be a relatively easy and straightforward extension of proximal sampling techniques from the Euclidean setting. The adaption is perhaps expected, and might limit the originality of the results.\n\nA more serious concern arises from a practical perspective. The implementation of the Riemannian proximal sampler relies crucially on rejection sampling to correct the additional bias introduced by the inexact oracles. But, it is folklore that rejection sampling can be computationally intractable in real-world and high-dimensional applications as it might require an exponential number of proposals in dimension. This brings serious issues and may cause the \"practical implementations\" to be totally infeasible in practice. It might be possible that the inexact method can be made more practical by considering other common techniques like MH used in high-accucary sampling, but the authors do not consider this or offer an explanation. Moreover, it is okay to be without an empirical section for a purely theoretical paper. However, the total lack of experiments is problematic when the practicality of the algorithm is not fully justified by theoretical results alone. Therefore, I would recommend rejection unless these concerns are addressed.\n\nQUESTIONS:\nFuther questios are already mentioned in the weakness part.\n\nLIMITATIONS:\nThe paper is majorly theoretical and does not have negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article generalizes an existing Euclidean proximal sampler to Riemannian manifolds, where the performance of the sampled, measured by its convergence rate, is established. The proposed sampler is based on two key oracles MBI and RHK, aiming to sampling the exact Brownian motion on manifolds. The practical implementations of the oracles are discussed, and analyzed. Preliminary numerical results, show the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: The article is mostly well written and easy to follow. The main results in Theorem 6 and 8 are easy to understand, though they seem to be highly technical (and novel) to prove. The nature of the results is of theoretical interest, notably Corollary 10 gives a concert case to show the implication of the main results. \n\nWeakness: The idea of Varadhan’s asymptotics mentioned in Section 6 is not very intuitive. A better explanation is needed (see questions below). Some notations should be unified to better understand the results. Due to the lack of space, it is a pity that all the numerical results are given in Appendix.\n\nQUESTIONS:\n-\tWhy in Theorem 8, there is no distinction between positive and negative curvature situations, as in Theorem 6?\n-\tAlgorithm 2 and 3, when it is possible to find a suitable t, i.e. to ensure the existence of t and C_RHK or C_MBI such that … it is unclear whether C_RHK and C_MBI are negative. \n-\tThe idea of Varadnan asymptotics makes sense when eta -> 0, according to Fact 5. However, the rates of inexact oracles rely on large eta in Assumption 1, therefore it is conceptually contradictory to use Varadnan asymptotics in a large eta regime. A clearer explanation about this is needed to ensure that these results are correct. \n\n-\tClarify the space of g in Definition 4. For all g in which space?\n-\tThe notations of pi^Y|X and pi^X|Y in Algorithm 1 are not consistent with Section 4. Are they the same as pi_eta^Y|X and pi_eta^X|Y in Assumption 1?\n-\tClarify the notation tilde O (difference to O) used in several places (e.g. thereom 8).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a Riemannian extension of the proximal sampler [1, 2]. Instead of the Gaussian Oracle and Restricted Gaussian Oracle used in the standard proximal sampler, the paper introduces the Manifold Brownian Increment (MBI) oracle and the Riemannian Heat Kernel (RHK) oracle (Section 3), and derives convergence guarantees (Section 4). Unlike in the Euclidean case, these oracles may not be exact. Therefore, the paper proposes approximate implementations of these oracles (Section 5), which still guarantee convergence (Section 4).\n\n\n[1] Yin Tat Lee, Ruoqi Shen, Kevin Tian. Structured Logconcave Sampling with a Restricted Gaussian Oracle. Proceedings of Thirty Fourth Conference on Learning Theory, 2021.\n\n[2] Yongxin Chen, Sinho Chewi, Adil Salim, Andre Wibisono, Improved analysis for a proximal algorithm for sampling. Proceedings of Thirty Fifth Conference on Learning Theory, 2022.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n* To the best of my knowledge, this is the first paper to introduce a Riemannian version of the proximal sampler. In particular, the method works under a more general setting (e.g., less smoothness, LSI) than previous Riemannian samplers [3]. The fact that the proposed method does not require $L$-smoothness of the potential is favorable, especially since verifying such a condition on Riemannian manifolds is less straightforward (and is, in fact, conjecturally related to the curvature of the manifold [4]).\n\n* While the main proof follows a similar structure to the Euclidean case (Theorem 6), extending the canonical assumptions in the Euclidean space to the Riemannian setting appears to be nontrivial and involve additional technical challenges (e.g., Proposition 31).\n\n* The fact that the proposed approximate algorithm satisfies the required assumptions (Section 5.2) and still guarantees convergence is a favorable result. \n\n### Weaknesses \n\n* Minor comment: In Section 2.1, Line 133, it would be helpful to specify the metric being used. Assigning a different metric (e.g., the Bures–Wasserstein metric) on the same space can result in a positively curved geometry.\n\n* The analysis relies on the LSI condition of the distribution on the manifold. However, it is unclear how strict or mild this condition is in the context of Riemannian manifolds. It would have been helpful to illustrate how strict or mild this assumption is by providing examples or criteria that can build the intuition (see also the below question section).\n\n[3] Xiang Cheng, Jingzhao Zhang, Suvrit Sra. Efficient Sampling on Riemannian Manifolds via Langevin MCMC. Proceedings of the 36th International Conference on Neural Information Processing System, 2022.\n\n[4] Foivos Alimisis, Antonio Orvieto, Gary Becigneul, Aurelien Lucchi. A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimization. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics. 2020.\n\nQUESTIONS:\n* I am not sure whether Lines 601–602 are correct. Does the function satisfy Assumption 3.3 of [5]? At a glance, $\\lambda_*$ appears to be possibly negative. If so, Theorem 3.4 of [5] would not apply. It is unclear to me whether $f$ actually satisfies the LSI.\n\n* Related to the previous question: as mentioned in the paper, there exists no globally supported non-constant strongly log-concave density on compact manifolds. More generally, it is conjectured that the convexity and smoothness of the function are closely related to the curvature of the manifold (see the paragraph before Section 5 in [4]). In this regard, I suspect that the existence of densities satisfying the LSI is not always guaranteed and may require some curvature condition on the manifold. Do the authors have any thoughts on this? Theorems 28 and 31 provide some partial answers, but I wonder whether there are any other known examples.\n\n* In Theorems 28 or 31, if $\\kappa < 0$, then $\\alpha < 0$. In this case, Theorem 6 appears to diverge, even if the step size is controlled, since the denominator becomes less than 1. Is this divergence inevitable?\n\n[5] Mufan Bill Li, Murat A. Erdogdu. Riemannian Langevin Algorithm for Solving Semidefinite Programs. Bernoulli 29(4), 2023.\n\nLIMITATIONS:\nThe authors expressed limitations of the work. I have stated additional limitations that I found in the weaknesses section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nHigh-accuracy sampling on manifold is proposed by extending the work in Lee et al. [2021]. This algorithm is firstly discussed assuming the exact oracles are available, and then discussed under the inexact case.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: The paper provides a high accuracy sampler on manifold, which is the first attempt towards this direction. The convergence of the sampler is firstly established under the exact MBI and RHK oracles, and then discussed based on the inexact oracles via heat kernel truncation. \n\nWeaknesses: \n* LSI somehow corresponds the  KL condition in optimization. In optimization, there is a well-established framework for the convergence analysis of the algorithms under the KL condition, even for Riemannian optimization. Similarly, after assuming LSI,  it seems the analysis follows a well-established route similar to the Euclidean case, even though there is a key property that needs to be carefully checked.\n * The paper also considers analysis under the inexact oracle via kernel truncation, it seems there are no experiments to validate the effectiveness of this scheme. Overall, despite the theoretically high accurate result, I guess how to do the sampling efficiently to satisfy the two oracles approximately is still very challenging for the manifold case.\n\nQUESTIONS:\n* What exactly $d$ in Line 292 refers to?\n* In Proposition~9, the truncation level is given in order for the inexact sample to achieve certain accuracy. I am wondering whether this should also relies on how fast the reject sampling converges?\n* In Line 144, what does the ``minimal solution'' exactly mean?\n* Overall, what properties should $f$ satisfy in order to satisfy the LSI and PI properties?\n* In Line 584, should it be the  Riemannian Langevin Monte Carlo Algorithm?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}