Title: Fast Non-Log-Concave Sampling under Nonconvex Equality and Inequality Constraints with Landing
Venue: NeurIPS 2025
Accepted: yes
Keywords: constrained sampling

nonconvex manifold

overdamped Langevin dynamics
Abstract: Sampling from constrained statistical distributions is a fundamental task in various fields including Bayesian statistics, computational chemistry, and statistical physics.
This article considers the cases where the constrained distribution is described by an unconstrained density, as well as additional equality and/or inequality constraints, which often make the constraint set nonconvex. Existing methods for nonconvex constraint set $\Sigma \subset \mathbb{R}^d$ defined by equality or inequality constraints commonly rely on costly projection steps. Moreover, they cannot handle equality and inequality constraints simultaneously as each method only specialized in one case. In addition, rigorous and quantitative convergence guarantee is often lacking.
In this paper, we introduce Overdamped Langevin with LAnding (OLLA), a new framework that can design overdamped Langevin dynamics accommodating both equality and inequality constraints. The proposed dynamics also deterministically corrects trajectories along the normal direction of the constraint surface, thus obviating the need for explicit projections. We show that, under suitable regularity conditions on the target density 
and $\Sigma$, OLLA converges exponentially fast in $W_2$ distance to the constrained target density $\rho_\Sigma(x) \propto \exp(-f(x))d\sigma_\Sigma$.
Lastly, through experiments, we demonstrate the efficiency of OLLA compared to projection-based constrained Langevin algorithms and their slack variable variants, highlighting its favorable computational cost and reasonable empirical mixing.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Overdamped Langevin with Landing (OLLA), a novel projection-free sampling algorithm for distributions defined by complex, and potentially non-convex, equality and inequality constraints. By decomposing Langevin dynamics into tangential and normal components and enforcing constraint satisfaction via an exponential \"landing\" drift, OLLA provably converges exponentially fast in 2-Wasserstein distance under a log-Sobolev inequality (LSI) assumption. The authors also present a practical Euler–Maruyama discretization (OLLA-H) using a Hutchinson estimator for curvature corrections, achieving only $O(N)$ gradient costs per step.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* The authors provide a unified framework to handle arbitrary smooth equality and inequality constraints without explicit projections.\n* The authors provide rigorous non-asymptotic exponential convergence guarantees in the 2-Wasserstein distance, leveraging LSI constants.\n* Practical discretization via Hutchinson trace estimation reduces per-step cost to $O(N)$ gradient evaluations.\n* The paper is generally well-structured.\n\n**Weaknesses**\n* The theoretical convergence rate has a similar mathematical form to that of unconstrained Langevin algorithms under good conditions, depending on an LSI constant. Intuitively, one might expect that constraining the sampling domain would improve the LSI constant and lead to faster convergence, especially as the domain diameter shrinks [1-3]. The paper does not provide such an analysis or an estimate of the LSI constant in terms of the domain geometry. It is unclear how difficult such an analysis would be. Could authors comment on that, and if possible, offer a theoretical advantage in convergence rate?\n* The experimental evaluation is confined to synthetic tasks. As it is a paper with a strong theoretical focus, this is understandable.\n* Typos: line 210: \"Metropolis-Hasting correction\" -> \"Metropolis-Hastings correction\"\n\n**References**\n\n[1] Reflection Couplings and Contraction Rates for Diffusions, Probability Theory and Related Fields, 2016\n\n[2] Sampling from a Log-Concave Distribution with Projected Langevin Monte Carlo, Discrete & Computational Geometry, 2018\n\n[3] Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning, COLT, 2021\n\nQUESTIONS:\nDo the analysis and convergence guarantees extend naturally to high dimensions? If not, could the authors discuss the challenges in making this dependence explicit?\n\nLIMITATIONS:\nThere are no limitations or potential negative societal impacts of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes OLLA, a projection-free Langevin dynamics method for sampling under nonconvex equality/inequality constraints. By combining tangential noise with normal drift (\"landing\"), OLLA enforces constraints without projections or slack variables. Theoretical analysis shows exponential convergence in Wasserstein distance, while a practical variant (OLLA-H) uses trace estimation for scalability. Experiments demonstrate superior efficiency over baselines in high dimensions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* This paper introduces a novel SDE design with a landing term for constraint enforcement, combined with rigorous non-asymptotic convergence analysis for three constraint scenarios.\n* This paper addresses a critical gap in constrained sampling, especially for nonconvex sets where projections fail, which also has practical impact for Bayesian inference, generative modeling, and other ML applications with constraints.\n* This is the first method to unify equality/inequality constraints in Langevin dynamics without projections.\n\nWeaknesses:\n* Critical assumptions are deferred to the appendix, making it harder to evaluate the method’s applicability upfront. This obscures potential limitations, such as sensitivity to constraint degeneracy or nonsmoothness, which should be highlighted in the main text.\n* Hyperparameters $(\\alpha, \\epsilon)$ are sensitive but lack principled selection guidelines.\n* Limited to synthetic experiments; real-world benchmarks would strengthen the impact.\n\nQUESTIONS:\n* How should practitioners choose $(\\alpha, \\epsilon)$ for real-world problems? Are there heuristics or adaptive schemes?\n* How does OLLA behave when constraints violate LICQ (e.g., degenerate gradients)?\n* The paper notes degraded performance for large $m$. Is there a theoretical or empirical threshold beyond which OLLA becomes impractical?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper studies the problem of sampling from a density satisfying log-Sobolev inequality with nonlinear equality and inequality constraines, even when the feasible set is nonconvex. The authors propose a novel framework, OLLA, which modifies Langevin dynamics to enforce constraints via a landing mechanism. By operating on the tangent space of the constraint manifold, OLLA avoids costly projections. The method achieves exponential convergence under some assumptions and is efficiently discretized using Hutchinson trace estimation. Empirical results demonstrate that OLLA matches or exceeds the performance of existing constrained sampling algorithms in both accuracy and computational efficiency, particularly in high-dimensional settings.

STRENGTHS AND WEAKNESSES:
Strengths:

This paper propose a novel sampling framework combined with projection free landing mechanism in optimization and shows exponential convergence guarantees in 2-Wasserstein distance under mild LICQ and log-Sobolev assumptions. To make the algorithm more efficient they further modify the algorithm by using Hutchinson trace estimation, yielding low per-step cost even in high dimensions. 

Weakness:

The stated assumptions are unclear and may not hold in practical scenarios, and the algorithm is highly sensitive to its hyperparameters, yet the authors provide no guidance on how to select them.

QUESTIONS:
How should one select the landing rate and step size?

Since there is no theoretical analsysis for OLLA-H, how is the Hutchinson trace estimator affect the convergence rate?

Could you clarify the assumptions underlying the main theorems and illustrate concrete settings in which they hold?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper makes use of the landing technique -- incorporting constraints directly into the guiding stochastic differential equation (SDE) -- from non-convex optimization to handle both equality and inequality constraints in sampling. The proposed algorithm, Overdamped Langevin with LAnding (OLLA), avoids projection to the feasible set, and hence runs efficiently and is proven to converge exponentially fast. With Hutchinson trace estimation, to approximate the Hessian in the mean curvature term, OLLA could scale to high dimensions.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n\n* The paper integrates the both equality and inequality constraints into Langevin dynamics and avoids projection.\n* The proposed OLLA algorithm comes with theoretic convergence guarantee.\n* OLLA-H algorithm scales to high dimensions and has SOTA performance compared with similar samplers with constraints.\n\n# Weaknesses:\n\n* It is a little disappointing that OLLA-H has comparable performance with CGHMC in the included tests. Hence it remains doubt whether practitioners choose one over the other.\n* In all the simulation tests, the highest dimensionality is 700. The paper could be strengthened by including an example of dimension bigger than 1000.\n* In all the simulation tests, the target distributions are relatively simple -- either uniform or Gaussian, though the constraints are complicated and versatile. This may not be very realistic --  in practice both distribution and constraints could be complicated. It would be more convincing to test OLLA(-H) on some more practical distributions with certain difficulty to sample.\n\nQUESTIONS:\nHere are a few more questions and suggestions:\n\n* In the submanifold $\\Sigma$, the inequality constraint is $g(x)\\leq 0$. However, in the set of active inequality constraints, why we have $g_i(x)\\geq 0$?\n* line 97: $g_{I_x}$ $\\to$ $g_{|I_x|}$.\n* In Section 3, the constants $\\kappa$, $\\delta$ and tabular neighborhood were given very abruptly and there lacked some background.\n* line 123, $h(X_t)$ $\\to$ $h_i(X_t)$.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}