Title: Wasserstein Convergence of Critically Damped Langevin Diffusions
Venue: NeurIPS 2025
Accepted: yes
Keywords: Wasserstein convergence; score-based generative models; kinetic Langevin diffusion
Abstract: Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications and benefit from strong theoretical guarantees. Recently, methods inspired by statistical mechanics, in particular, Hamiltonian dynamics, have introduced Critically-damped Langevin Diffusions (CLDs), which define diffusion processes on extended spaces by coupling the data with auxiliary variables. These approaches, along with their associated score-matching and sampling procedures, have been shown to outperform standard diffusion-based samplers numerically. In this paper, we analyze a generalized dynamic that extends classical CLDs by introducing an additional hyperparameter controlling the noise applied to the data coordinate, thereby better exploiting the extended space. We further derive a novel upper bound on the sampling error of CLD-based generative models in the Wasserstein metric.  This additional hyperparameter influences the smoothness of sample paths, and our discretization error analysis provides practical guidance for its tuning, leading to improved sampling performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies Critically Damped Langevin Diffusions (CLDs), a second-order extension of score-based generative models, and provides the first W2 convergence guarantees under weak assumptions. \n\nThe authors first prove that vanilla CLD samplers converge in W2 with an error bound decomposed into mixing, discretization ($O(\\sqrt{h})$), and score-approximation terms. \n\nThe authors then further introduce a new hyperparameter $\\epsilon$ controlling noise on the position coordinate and prove an improved $O(h)$ discretization rate under stronger log-concavity assumptions, and give practical tuning guidance. \n\nExperiments on the 50-dimensional Funnel distribution validate that small $\\epsilon$ yields lower W2 error and variance than no $\\epsilon$.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The theoretical results are nontrivial and fill a gap in the convergence theory of CLD-based samplers. *The reviewer acknowledges here unfamiliarity with the latest advances in Langevin diffusion analysis and did not check all derivations.*\n2. This is the first paper to derive W2 convergence rates for CLDs (previous work focused on KL divergence). The regularized variant in Section 4 is a novel extension with provable benefits.\n3. The main text is generally well organized.\n\n**Weaknesses:**\n1. There is no analysis or experiment on warm-up (burn-in) strategies or initialization sensitivity.\n2. It would be great to include an explicit pseudocode block in the main text to summarize the algorithm for analysis.\n3. Experimental validation is limited to synthetic tasks. As this work mainly focuses on theoretical advances, the reviewer thinks it is acceptable.\n\nQUESTIONS:\nCan the additional noise hyperparameter $\\epsilon$ in Section 4 be interpreted as a form of warm-up or preconditioning?\n\nLIMITATIONS:\nThere are no societal impact considerations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies Wasserstein convergence of score-based generative diffusion models where the forward process is governed by a critically damped Langevin diffusion. A modified dynamic is proposed that incorporates an additional hyperparameter that controls the noise applied to the data. Some numerical experiments are also provided.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n(1) The paper is a theoretical paper. The analysis seems to be rigorous. \n\n(2) The Wasserstein convergence guarantees for critically damped Langevin diffusions on the Euclidean space seem to be new.\n\nWeaknesses:\n\n(1) The theoretical convergence guarantees for critically damped Langevin diffusion models have already been studied in the literature, including Chen et al. (2023) that is cited in the paper. There, the authors claimed that critically damped Langevin diffusion models cannot improve upon the theoretical guarantees for using the classical score-based diffusion model based on overdamped Langevin diffusion. By reading this paper, it is not clear what is the advantage using critically damped Langevin diffusions and whether acceleration can be achieved.\n\n(2) The paper mentions that Chen et al. (2023) and Conforti et al. (2025) analyzed the convergence of SGMs in the extended space using kinetic Langevin-dynamics. Their results use other measures, such as KL divergence in Conforti et al. (2025). KL sometimes can imply Wasserstein, for example on a compact domain, or when the target distribution is strongly-log-concave, which is assumed in H4 in the paper.  The paper should highlight why for the settings considered in this paper,  Wasserstein convergence cannot be obtained using existing results or at least using the existing approaches in the literature.\n\n(3) I find the assumption H4 very strong, because it is not just strong-log-concavity, but at every $t$.\n\nQUESTIONS:\n(1) I am still a bit confused with Theorem 3.1. and Theorem 4.1. In Theorem 3.1., you obtained Wasserstein convergence guarantees without H4, the strongly-log-concave assumption. But then in Theorem 4.1. you need H4. It would be helpful if you can explain a bit why for Theorem 4.1. you need H4, but not for Theorem 3.1.\n\n(2) Right before Theorem 4.1., you mentioned your results match existing results for SGMs in the literature. But if so, what is the advantage using your proposed modified dynamics compared to SGMs?\n\nLIMITATIONS:\nIt seems the author(s) did not discuss limitations in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nScore-based generative models rely on a forward diffusion process that reduces the data distribution into an easy-to-sample target distribution (often a Gaussian). Therefore, a wide variety of diffusion processes can be used in principle. This paper focuses on the study of the Wasserstein convergence of the critically-damped Langevin diffusion, which often showcases a smoother sampling trajectory when compared to the more standard choices. \n\nThis paper studies the Wasserstein convergence of diffusion models that seek to invert the critically-damped Langevin diffusion, and provides a convergence result (Theorem 3.1) that characterizes Wasserstein distance on the extended phase space $\\pi\\_{\\text{data}} \\otimes \\pi\\_v$. Building upon this result, this paper posits that the convergence property of this forward process can be improved by a slight modification of the diffusion coefficient, and illustrates empirically via a synthetic dataset that the introduction of this parameter leads to better Wasserstein convergence.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, I think this paper is well-written with interesting theoretical results in the study of forward processes (often outlooked in the context of diffusion modeling). \n\n## Strengths\n- The paper presents a theoretical foundation to the study of critically-damped Langevin diffusion as a candidate for forward processes, which will be of general interest to researchers of diffusion models. The $W_2$ convergence presents a new result. \n- The paper is clearly written, with useful references to related work that draws parallel between KL convergence and Wasserstein convergence. \n- While this paper does not provide an explanation about why CLD can outperform standard Ornstein-Uhlenbeck process, its suggestion of a modified forward process is nevertheless useful in itself, and provides further guidelines to further study of forward processes backed by convergence theory. \n## Weaknesses\nI think this paper's strengths outweigh its weaknesses, however, I find the theoretical insights provided by this paper somewhat limited in applicability because of the following reasons. \n- Although this paper presents a novel result, it does not ultimately answer the question of the better numerical performance of kinetic Langevin sampler compared to the standard Ornstein-Uhlenbeck process: instead, the paper opts to provide an alternative forward process that better exploits the added flexibility introduced by auxiliary variables. \n- Theorem 3.1 seems useful, but one can argue that convergence in the extended state space  $\\pi\\_{\\text{data}} \\otimes \\pi\\_v$ is not a point of interest: the moment variables are introduced as auxiliary variables for the forward process. \n- The numerical illustration seems somewhat contrived. The setting is largely synthetic, and the paper admits that comparing across different choices of $\\epsilon$s can be problematic. \n- The absence of a practical dataset is usually acceptable for a theory paper, however, it still constitutes a weakness for this paper, especially when the paper proposes a brand new way of parametrizing forward processes. When comparing across different $\\epsilon$s can be seen as subjective, it will be compelling to see the introduction of the regularization parameter make a different in a practical setting.\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the convergence behavior of critically-damped Langevin diffusions and derives upper bounds on the sampling error in terms of the 2-Wasserstein distance between the generated and target data distributions. The main contribution is Theorem 3.1, which, under assumptions H1–H3, establishes a bound on the Wasserstein distance comprising three additive components: discretization error, approximation error, and a convergence term related to the initial distribution. This analysis contrasts with prior results that primarily focused on KL divergence. Furthermore, Theorem 4.1 strengthens the bound by introducing an additional assumption of strong log-concavity of the intermediate distributions $p_t$ for all  $0 \\leq t \\leq T$, leading to improved convergence guarantees.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper addresses a relevant theoretical topic concerning the convergence of Langevin diffusions.\n\n2. The convergence guarantee is provided in terms of the Wasserstein distance, which could be useful for other theoretical studies of Langevin-based generative modeling.\n\n**Weaknesses:**\n\n1. The paper is theoretically heavy and difficult to follow for a general machine learning audience. The authors do not provide simplified examples or intuitive remarks to help readers understand the main results. I highly recommend that the authors include intuitive examples or interpretations of their theorems to help readers better appreciate the significance and applicability of the results. In the current form, it is difficult to assess the non-triviality and practical value of the theoretical contributions.\n\n2. While the authors offer a brief justification on page 5 regarding the mildness of their assumptions, it would be much clearer if they included concrete examples of data distributions that satisfy these assumptions. In particular, Assumption H3 appears non-standard and seems difficult to verify in practice. For instance, assuming the data distribution is Gaussian, how would one find a valid constant $M$ to apply the theoretical result?\n\n3. The constant $h$ in Theorems 3.1 and 4.1 is vaguely defined. While looking into the proofs, I noticed that certain conditions on $h$ are required, but these are not mentioned explicitly in the theorem statements. Moreover, it appears that the constant \n$c_1$ may depend on the choice of $h$, which raises concerns about whether the bound might hold trivially. The authors should provide a more detailed discussion of the role of $h$ in the bounds to clarify the implications and non-triviality of their results.\n\n4. Following the above point, I would like to ask the authors to show what Theorem 3.1 or 4.1 mean when applied to a standard distribution such as the unit-variance Gaussian or uniform distribution. Can they compute the corresponding constants $h$ and $M$ in these cases to illustrate the implications of the bound in concrete settings?\n\n5. Finally, it would be nice if the authors elaborated on the comparison between their Wasserstein-2 bounds and the existing KL divergence bounds. Is it possible to relate the two using standard inequalities between divergence measures? A discussion of such a relationship would enhance the understanding of the novelty and potential advantages of the authors' analysis.\n\nQUESTIONS:\nPlease see my previous response.\n\nLIMITATIONS:\nPlease see my previous response.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}