Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics
Authors: Omar Chehab
Anna Korba
Austin J Stromme
Adrien Vacher
Venue: ICLR 2025 Poster (2025)
Keywords: Sampling
Langevin
Annealing
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Geometric tempering is a popular approach to sampling from challenging multi-modal probability distributions by instead sampling from a sequence of distributions which interpolate, using the geometric mean, between an easier proposal distribution and the target distribution. In this paper, we theoretically investigate the soundness of this approach when the sampling algorithm is Langevin dynamics, proving both upper and lower bounds. Our upper bounds are the first analysis in the literature under functional inequalities. They assert the convergence of tempered Langevin in continuous and discrete-time, and their minimization leads to closed-form optimal tempering schedules for some pairs of proposal and target distributions. Our lower bounds demonstrate a simple case where the geometric tempering takes exponential time, and further reveal that the geometric tempering can suffer from poor functional inequalities and slow convergence, even when the target distribution is well-conditioned. Overall, our results indicate that the geometric tempering may not help, and can even be harmful for convergence.

Published as a conference paper at ICLR 2025
PROVABLE CONVERGENCE AND LIMITATIONS OF GEOMETRIC TEMPERING FOR LANGEVIN DYNAMICS
Omar Chehab, Anna Korba, Austin Stromme & Adrien Vacher
Department of Statistics CREST, ENSAE, IP Paris
France
{emir. chehab, anna. korba, aust in. stromme, adrien. vacher
ABSTRACT
Geometric tempering a popular approach to sampling from challenging multi- modal probability distributions by instead sampling from a sequence of distribu- tions which interpolate, using the geometric mean, between an easier proposal distribution and the target distribution. In this paper, theoretically investigate the soundness of this approach when the sampling algorithm is Langevin dynam- ies, proving both upper and lower bounds. Our upper bounds are the first analysis in the literature under functional inequalities. They assert the convergence of tem- pered Langevin in continuous and discrete-time, and their minimization leads closed-form optimal tempering schedules for some pairs of proposal and target distributions. Our lower bounds demonstrate simple case where the geometric tempering takes exponential time, and further reveal that the geometric tempering can suffer from poor functional inequalities and slow convergence, even when the target distribution is well-conditioned. Overall, our results indicate that geometric tempering may not help, and can even be harmful for convergence.
INTRODUCTION
Sampling from a target distribution N whose density is known up to a normalizing constant is chal- lenging problem in statistics and machine learning, and is currently the subject of intense interest due to applications in Bayesian statisties (Dai al. 2020) and energy-based models in deep learn- ing (Song et al. 2021a), among other areas. In these settings, the normalizing constant of the target distribution π is typically intractable, and Markov Chain Monte Carlo (MCMC) algorithms (Roberts) and Rosenthal 2004 Robert and Casella 2004 are commonly used to generate Markov chains in the ambient space, whose law eventually approximates the target distribution.
Among MCMC algorithms, the Unadjusted Langevin Algorithm (ULA), which corresponds time discretization of a Langevin diffusion process, has attracted considerable attention due to its simplicity, theoretical grounding, and utility in practice (Roberts and Tweedie 1996; Wibisono, 2018 Duurmus et al.) 2019 (Song and Ermon (019). For example, ULA can be proven to converge smooth and strongly log-concave (Durmus and Moulines 2016 Durmus et al. 2019). However, many cases in practice require to sample from distributions which not log-concave, and indeed potentially even multi-modal (Parisi] 1981 Zhang et al. 2020). In such settings, the convergence of ULA is governed by functional inequalities which ef- fectively quantify the convexity, or lack thereof, of the target distribution (Vempala and Wibisono 2019). Nonetheless, truly multi-modal target distributions generally have poor functional inequali- ties, thus leading to weak convergence guarantees for ULA. This phenomenon is not merely a the- oretical artifact, and is well-known amongst practitioners that when sampling from multi-modal distributions, algorithms based on ULA can get stuck in local modes and suffer from slow conver- gence (Deng et al. 2020).
quickly when the target distribution
Tempering or annealing is a popular technique (Neal 1998 Gelman and Meng 1998 Syed et al. 2022) to overcome the deficiencies of ULA and other MCMC methods in the Rather than sample directly from the target distribution म, tempering samples from a sequence of distributions that interpolate between easier, unimodal proposal distribution and the more chal-
setting.

---

Published as a conference paper at ICLR 2025
lenging и. Intuitively, tempering may help escape local modes and explore the entire target distribu- tion (Syed et al. 2022). Many possible interpolating paths for tempering exist, but to be practically useful the path must be implementable with the chosen MCMC scheme, and should improve con- vergence when compared the latter run directly against the target π
One of the most popular choices for the tempering path is the geometric tempering (Gelman and Meng 1998 Neal 1998), where the intermediate distributions are defined to be the geometric aver- ages M4 X 1 " with [0, 1] the (inverse) temperature parameter. At "hot" temperature, is close to 0 and the distribution yl is close to the proposal which can be chosen with large variance for better exploration, and when the temperature is gradually "cooled" to X 1, μx recovers the original target π. In practice, the geometric tempering does not require access to the normalizing constants, and has accordingly been widely applied to various MCMC methods (Dai et al. 2020 Chopin and Papaspillopoulos]] (020). However, theoretical understanding of the geometric tempering is limited. This includes identifying situations where it improves upon standard sampling procedures as well as offering guidance for the selection of the temperature schedule, crucial question in practice where
a number of heuristics have been proposed (Chopin and Papaspiliopoulos 2020
Jasra et al. 2011
Chopin et al. 2024 Kiwaki 2015).
Contributions. In this paper, we develop theoretical understanding of geometric tempering com- bined with Langevin dynamics: we refer to this tempered Langevin dynamics. We make three main contributions:
1. We provide in Theorems and 3 the first convergence result for tempered Langevin dy- namics in Kullback-Leibler divergence (KL). Our results here are for general tempering schedules and depend on the functional inequalities (og-Sooolev) of the intermediate dis- tributions along the tempering sequence. In Proposition O we derive the optimal tempering schedule for our continuous time upper bound in the strongly log-concave setting.
2. To go beyond the strongly-log concave setting, we must understand the behavior of func- tional inequalities along the geometric tempering. Our result here, Theorem is negative, and shows that, surprisingly, even when the proposal and target have favorable functional inequalities, the geometric tempering can exponentially worsen these inequalities.
3. Although the poor functional inequalities in Theorem + are a worrying sign for the geo- metric tempering, they do not yet rule out fast convergence since functional inequalities only govern worst-case convergence. We thus analyze a simple bi-modal example where we show in Theorem 8 that the geometric path takes exponential time to converge in total variation (TV), and then show Theorem that,, surprisingly, similar results even hold for uni-modal target with favorable functional inequalities.
In sum, our results establish sufficient conditions for the tempered Langevin dynamics to converge in KL divergence. We find some limited situations where these improve upon the rates of standard Langevin dynamics, but we also find that the geometric tempering can worsen functional inequalities and suffer from slow convergence, both in the setting of multi-modal target and even for uni-modal targets with reasonable functional inequalities.
Organization. This paper is organized as follows. In the remainder of this section we discuss lated work and describe our notation. In section we discuss background about ULA and geometric tempering. In section 3. we state our upper bounds in KL on the convergence of the geometric tem- pering with Langevin dynamics under functional inequalities, both in continuous and discrete time. In section 4 we show that the geometric path can have poor functional inequalities even when the proposal and target do not; yet, in the strongly-concave case, we derive explicit results from our upper bounds and highlight situations where tempering may be beneficial. In section 5 we describe two examples where geometric tempering with ULA provably has slow convergence in TV. Proofs and additional numerical validations are collected in the Appendix.
Related work. Given the empirical success of ULA with sequence of tempered target distribu- tions is desirable to obtain theoretical guarantees on the convergence of the scheme, and especially to understand when and why certain sequences of moving targets can guide misguide the sampling process towards the final target. Closely related to our setting, TTang et al. (2024) also study the con- vergence of tempered I Langevin dynamics for the geometric sequence. However, they focus on the
2

---

Published as a conference paper at ICLR 2025
simulated annealing setting. In this case the geometric sequence is taken as T² ttht degenerates to target distribution which is a Dirac located at the global maximum of the log density of π as À goes to infinity, so that sampling actually becomes optimization task (Kirkpatrick et al. 1983 Ceryy 1985). Using an explicit temperature schedule, they measure convergence in probability. For an- other sequence of intermediate targets obtained by convolving the target and proposal distributions, and using an explicit schedule, [Lee et al. (2022) prove fast convergence in TV. Finally, for general sequence of intermediate targets, Guo al. (2024) very recently obtained rate of convergence that depends on Wasserstein metric derivative along the path of distributions: their result suggests that an optimal path can be obtained as a Wasserstein geodesic between the initial and target distributions. A modification of tempered Langevin dynamics, called Sequential Monte Carlo, that importantly includes resampling of particles, has been shown to achieve fast convergence in TV (Schweizer) 2012 Paulin et al. 2018, Mathews and Schmidler] [0244 Lee and Santana-Gijzen 2024). These results apply to different sampling processes and rely on strong assumptions, effectively modeling far-away modes by disjoint sets (Schweizer 2012 Mathews and Schmidler (024), using a specific path that interpolates between a uniform and target distribution by increasing the number of compo- nents that follow the target's law (Paulin et al., 2018), assuming uniformly bounded consecutive distributions along the path (Lee and Santana-Gijzes (024). In contrast, our results are specific to the geometric sequence, but we obtain upper and lower bounds on convergence that explicitly depend on the time.
The tempered iterates defined by the geometric path are also at the basis of simulated or parallel tempering schemes (Geyer 1991. Marinari and Parisi 1992; Hukushima and Nemoto 1996; Syed al. 202I). which are MCMC algorithms where the temperature is a random variable instead of monotonic function of time. Both schemes produce samples at all temperatures and involve swap- ping particles between hotter and colder temperatures. In that setting, some works have investigated the spectral gap of these methods, which is related to the rate of convergence in TV distance (Madras) and Zheng 2003; Woodard et al.] 2009a]b). b). When that rate is polynomially (resp. exponentially) decreasing in the problem size, convergence is said to be fast (resp. 'torpid"), These rates studied for arbitrary target distributions, using lower and upper bounds. Namely, WWoodadd et al. (2009a b) show that for target distributions which have modes with different weights or shapes, convergence can be slow, and that symmetric modes are required for fast convergence. This generalizes previous findings that for specific targets with two symmetric and equally weighted modes, convergence is fast (Madras and Zheng 2003), whereas for another specific target with three asymmetric modes, convergence slow for any schedule (Bhatnagar and Randall 2015). Ge et al. (2018) also prove fast convergence when the target is a Gaussian mixture. More recently. Chen et al. (2019) study the Poincaré constant in parallel tempering, which governs the rate of convergence in x" divergence: they show it improves upon standard Langevin, and relate this improvement to the exchange rate of particles between different temperatures. Our work differs in several important ways: we study a different sampling algorithm (Unadjusted Langevin algorithm), obtain upper and lower bounds directly in time rather than on the spectral gap, and prove explicit rates of convergence, as well as lower bounds, for simple choices of proposal and target distributions.
Notation. C. (IRd) denotes the set of infinitely differentiable functions with compact support. P(R4) denotes the set of probability measures p on Rd. For p E P(R4), we denote that p is absolutely continuous w.r.t q using p < 9 and we use dp/dq to denote the Radon-Nikodym derivative. The set of probability measures which are absolutely continuous with respect to the Lebesgue measure is written Pac (Rdd). The Total Variation (TV) distance is defined as TV(p. q) = Sueeemumm suPACR" (P(A)-(( ()), where the supremum runs over all Borel sets. For any p P(R))) T22 (2) is
the space of functions f: Rd + R such that J' [[ < co. We denote by || (1= and (, LP())
respectively the norm and the inner product of the Hilbert space L (p) For p & g, the Kullback-
Leibler (KL) divergence is defined as KL(p, q) = l'lo dg dp, the diviergge as x² q) =
J ( - 1)2dd dq and the Fisher divergenc as FD(p.q) = || log (4:) ]|( and too otherwise. For a measurable function f: Rd + R we define the variance Var,(f) := EpI(f - Ep(f1) for p E P(RR)). We write the standard Gaussian on IR with mean a and variance o² as Ma o²), and the uniform measure on Borel set A C Id with finite and positive Lebesgue measure unif A- Given a closed set A C d we define d(x, A) == ynf || - yil for all x E Rdd, Finally, we define the error function by erf(2) == 3/ So e dx.
3

---

Published as a conference paper at ICLR 2025
2 BACKGROUND
In this section, we provide some background on functional inequalities, Langevin dynamics, and geometric tempering.
Functional inequalities. Let 9 E (R)( We say that satisfies the Poincaré inequality with constant Cp 2 if for all f E C (RR),
()) S Cpl CpIVV/I
and let Cp(q) be the best constant in Eq. [ or too if it does not exist. We say that satisfies the log-Sobel See-ooolle inequality with constant Cr.s if for all f E C (RA),
ent, = Ey 8 In
f2
)
= 2C VV
(2)
Egg (f2)
and let CLs(4) be the best constant in Eq. 2 or +o0 does not exist. Note that log-Sobolev implies Poincaré with the same constant (Bakry et al. 2014 that CLs(g) 2 Cp(q). If e and the potential V is -ty-Strongly convex, then satisfies Eq. 2 with constant <x,
-. However, the latter
more general, including for instance distributions g whose potentials are bounded perturbations of a strongly convex potential (Bakry et al. 2014 Cattiaux and Guillin 2022).
Langevin dynamics. Let
Pac (RR.). The Unadjusted Langevin Algorithm (ULA) Parisi Besag 1994) consists in sampling a target distribution a using noisy gradient ascent
1981
Xx+1 = Xx + hV log (Xx)) V2hcR, ek ((,,1a))
(3)
with step size h > 0 at iteration k E N. Setting time as t= = hk and taking the continuous limit obtained as - 0, Eq. 3 defines continuous process known as the Langevin diffusion. The convergence of the law of the Langevin diffusion to the equilibrium measure is then governed by the Poincaré and log-Sobolev inequalities. In particular, we denote the law of the Langevin diffusion at time t by Pt. then Pt converges to π in KL with exponential rate determined by the log-Sobol Sog-oeo constant of π (Vempala and Wibisono
2019 Theorem 2), namely
(4) However, for multi-modal distributions such Gaussian mixtures, the log-Sobolev constant can grow exponentially with the distance between modes (Chen et al. 202T). Nevertheless, ULA re- mains popular choice, due to its computational simplicity: simulating Eq. Jonly requires access to the score log ग which does not depend on the target's normalizing constant.
KL(Pe, T) S (C.s(t))''KL((( x).
Langevin with moving targets. Many heuristics broadly known annealing or tempering consist in using ULA to sample a path, or sequence of distributions (/4)ER, instead of the single target
и. The hope that this sequence of intermediate distributions will improve the convergence of the
ULA sampler: Different tempering algorithms sample the path sequentially in time (Dai et al. 2020 Neal 1996 Zhang al. 2020).
Neal 1998 Rubin 1987), back-and-forth in time (Lee al. 202
or at all times jointly (Marinari and Parisi
1992
Geyer
1991] This paper deals with the first case,
i...
X = X2 + h V log Mu (Xx) + / 2hack, 6 ~ (O,
(5)
where the target now updated ("moved")) at each iteration. This generic sampling method has been used in digendiimnsional spaces (Wu et al. 2020 Thin et al.) 2022 Geffner and Domke (003) Song and Ermon 2019) and has achieved of-te-of-the-art results for sampling images, where is sometimes known by the names Annealed Langevin Dynamics (Song and Ermon (019) or the "cor- rector" sampler (Song et al., 2021b). is therefore of interest to find moving targets whose geometry is well-suited to ULA's convergence properties. A number of paths (He)LER, can be used to guide the process toward the final target æ. Many of them interpolate between a proposal distribution v that is easy to sample and the target distribution for example by taking their convolution (Song and Ermon 2019 (Sng et al. 20216 [Albrrgo et al., (023), their geometric mean (Neal) 1998). or
following the gradient flow of loss from proposal to target (Tieleman 2008 Carbone al. 2024
4

---

Published as a conference paper at ICLR 2025
Marion et al.) 2024). The path obtained by convolving the two distributions is the default choice for sampling from so-called "diffusion models", yet the scores V log μ4, along that path are not an- alytically tractable in setting when the density of is known up to normalization constant,
and estimating them is the subject of current research (Huang et al. 2024 He et al. 2024 Grenioux
et al. 2024 Saremi al. 2024).
Geometric tempering. The path obtained by taking the geometric mean of the proposal and tar- get distributions has distinguished itself in the sampling literature (Neal 1998. Gelman and Meng 1998). It is written as
pe(x) = -c)
E
(6)
where cx, normalizing factor, (R.((R) is a proposal distribution and X(-) RR, + [0, 1] is an increasing function called the tempering schedule. It has recently been shown that the ge- ometric path can be identified to a time-discretized gradient flow of the Kullback- gence to the target, with respect to the Fisher-Ran Rao distance (Chopin et al. 2024,] Domingo-Enrich and Pooladian 2023). Its main advantage is its computational tractability, since the score of X log v + x V log is known in closed-form This path is default in some sampling libraries (Cabezas et al.] (023) and remains a popular choice in recent sampling litera- ture (Thin et al. 2021 Geffner and Domke, 2023 Dai et al. 2020) and applications (Bradley and Nakkiran 2024 Ramesh et al. 2022] Saharia et al. 2022; Dieleman 2022). A special case of the geometric path is especially popular, choosing to be "uniform", or more formally, equal to the Lebesgue measure: (zx) = c. (a > this choice is commonly used in practice to sample from un-normalized distributions parameterized by deep neural network (Wu et al.] 20020] Grathwohl
diver-
Vlog [4g =
et al. 2020; Nijkamp et al., 2019; Ye et al.] (017)) or for global optimization (Marinari and Parisi] T²o and tempering
define the probability measure schedule 7/2 == λr=Ao and rewrite (x)) (2) A as special case of Eq. 6 with the tempering 1-Xo
1992
Note that when À O,
schedule 7/- When is not chosen to be "uniform" it is often chosen as simple distribution, such
as a Gaussian (Cabezas et al. 2023 Zhang et al. 2021 Thin et al. 202I).
3 CONVERGENCE RATE FOR TEMPERED LANGEVIN DYNAMICS
Throughout, we take as given proposal and target distributions v and æ, as well as a temperature schedule A: R, + [0, 1J. which we assume satisfy the following conditions.
Assumption (Regularity of proposal, target, and tempering) The proposal v and the target no have densities with respect to the Lebesgue measure, which we write & e and evV. The tempering schedule (Ne)22 is such that X: R,+ + [0, 1] and de is non-decreasing in t and weakly differentiable.
In addition to this basic regularity, we also make the following quantitative assumptions on the negative log densities.
Assumption 2 (Lipschitz gradients and dissipativity) The negative log densities Vo, V2 have Lips- chitz continuous gradients on all of Rd, with Lipschitz constants Lv, Lax respectively. In addition, V. and Vx satisfy the dissipativity inequalities
(VV.(I), x)) 2 au||xr|1 bu,
(VV+()), 2 am ||/11/112 bu:
with constants a, ax. by, by > O.
The Lipschitz assumption is used both for our discrete time results as well as guaranteeing exis- tence and uniqueness of the continuous time dynamics; see Appendix A. for more discussion on this latter point. The dissipativity condition is common in the sampling literature (Conforti 2024 Erdogdu et al. 2022) as it implies finite log-Sobolev constant under Lipschitz gradients (Catti- aux et at. 2010). Our only quantitative use of the dissipativity condition will be to get control on the second moments of the tempering dynamics, but it is also convenient ensure the Sobolee constants remain finite along the tempering path. Indeed, dissipativity of both V,, and Vn implies dissipativity of μe from Eq. 6 so Cr.s(u:) xc. Our results will crucially rely on the size of the inverse of these log- Sog-Soeoeev constants, so we define the notation
04 = CLS(M)) 0,
2 0.
(8)
5

---

Published as a conference paper at ICLR 2025
As simple example, notice that if vn, V are ay, O, strongly convex, then 04 2 (1 t)a, ta In section 4.1, we investigate how a depends on the log- Sogoolee constants of v and x, and here proceed with the statement of our convergence results.
Given the proposal y and target R, as well as tempering schedule (Ae)120. we are interested in the tempered Langevin dynamics, where the moving target is the geometrie path μt, as defined in Eq. 6 In continuous time, this is the stochastic differential equation
dX. - -(1 ((1 + VvV + V2WW,
with initialization Xo Pu for some Pu P(R)), and where W, is standard d-dimensional Brownian motion. We denote by Pt the density of Xt given by the continuous time dynamics Eq. 9 Our main upper bound in continuous time follows.
Theorem (Continuous time) Suppose Assumption 7 landZhold and let (ae)e20 be the inverse log- Sobolev constants in Eq. [ assumed to be integrable. Let pt be the law of Eq. Wwith initialization Po and denote by de the weak time derivative of the tempering schedule. Then, for all 2 0:
KL(Pt, R) S exp -2 (a,ds) KL Mo) + A(1 (01--a)- + A S A exp -2 f aodo ds
(10)
where A 2(Lx + Ly)( ( (((d+bbtbb)) + Epo (1=111)))?)) depends on the constants from Assumption 2 the second moment of the initial distribution po, and linearly on the dimension
auhar
The proof of Theorem can be found in Appendix A.2 To the best of our knowledge, Theorem
is the first convergence analysis of Tempered Langevin dynamics, namely Eq. 9 in the literature. The first two terms of the upper bound deal with the start and end of the tempering: the first one measures the convergence rate to the first tempered distribution μi and the second one measures how far the current tempered distribution is from the target. The first term is set to zero when the schedule starts with X; 0 and the sampling process Eq. is initialized with the proposal Po v. The second term is null when X = 1 at the time when convergence is evaluated. The last term in the upper bound involves the tempering speed Å, as well the geometry of the moving targets 14, via their inverse log-Sobolev constants a₁ between inverse temperatures Åg and X and, in particular, null when the annealing schedule is constant. Note that, in Appendix A.3] we translate Theorem to give sufficient conditions for convergence to the target at precision KL π.) < e.
Remark 2 (Recovering standard upper bound for Langevin dynamics without tempering) Notice that Eq. recovers standard Langevin dynamics when we set = for all I E Tk.. In this case, only the first term in the upper bound is non-zero, and the bound recovers the standard continuous- time upper bound for Langevin dynamics, as recalled in Eq. 4
Next, we analyze the Muler-Maruyams discretization of the continuous time Tempered Langevin Dynamics Eq. 9 Namely, we take Xo Po and then, for a tempering sequence (At) and a sequence of step-sizes (h+)=20. we follow the iteration
XR+1 Xx h441((1 Ma+1)VV, AR+IVV,)+ + /2h++1ER41
(11)
where 6 is independent standard Gaussian noise. In other words, at each iteration k we take Langevin step of size h&41 towards Ms+1 == Cu-1vl-At4+
Theorem (Discrete time) Suppose Assumptions 7 Z hold, and let (ax)x21 be the inverse log- Sobolev constants as in Eq. g Define the smoothness constant of the tempered path be L := (1 Top + > To,, and let pk be the law of Eq. 5 with initialization po- Then, so long
ax A Ov
h& S min(
4L 2(L. L2 1)
for all k, we have
KL m) S exp
Σ
anyh))
KL Ho) + +A'(1= -xa)
j=1
k +
k
k
k
cxp
exp
Σ
agy)))
+
h a exp (- Σ ayhi) (12)
1=1
j=i
i=1
j=it1
6

---

Published as a conference paper at ICLR 2025
+bx)/++1)) 3(d+b,-bz depends the constants Aa,A1 from Assumption the second moment of the initial distribution po- and linearly the dimensional
where A' = ( max [Epo[[==]]]
A,,
The proof of Theorem 3 can be found in Appendix The first three terms the discrete- equivalent of those obtained in the continuous-time setup of Theorem [ The novelty the fourth term which is the bias from the discretization: it becomes null as the step sizes hą tend to zero Otherwise, it involves the geometry
the intermediate target distributions via their smoothness their inverse log-Sobolev constants oyp- Again, when there is no tem-
constants Li, additionally pering, i.e, setting X = 1, since the second and third term cancel, we recover the known upper bound on the convergence of ULA (Vempala and Wibisono, 2019. Th. 2) up a multiplicative constant. Similarly to the continuous case. we derive in Appendix B.2 sufficient conditions for converging to the target with a given precision (T,P2E, T) <.
Understanding the upper bounds in this section mainly involves two key points: the geometry of the moving targets their inverse tog-Sobolev constants dtg and the tempering schedule X(-). These will be the focus of the next sections.
4 ANALYSIS AND OPTIMIZATION OF THE UPPER-BOUNDS
In this section we explore the continuous-time upper bound from Theorem We first present in section 4.1 a simple example where the log-Sobolev constants of the intermediate distributions along the geometric mean path can be exponentially worse than those of the target and proposal. Motivated by this result, we then conduct a detailed study of the optimal tempering schedule in section 4.2 in the setting where both and π are strongly log-concave.
4.1 GEOMETRIC TEMPERING CAN EXPONENTIALLY WORSEN FUNCTIONAL INEQUALITIES
Because of the fundamental role that log- Tog-Soolee constants play governing the convergence of Langevin dynamics without tempering. it is natural that our upper bounds for tempered Langevin dynamics depend on the log-Sobolev constants of the intermediate distributions. Nonetheless, the size of these log-Soboler constants is crucial for ensuring rapid convergence of tempered Langevin dynamics. We are thus led to ask a fascinating yet, to the best of our knowledge, new question: how do the log-Sobolev constants along the geometric tempering path depend the proposal and target distributions?
Developing complete answer this question is in- teresting direction for future work, but here we provide simple example which shows that, in general, the ge- ometric path can actually make functional inequalities exponentially worse. As is commonly done in prac- tice, we take the proposal distribution to be Gaus- sian (Cabezas et al., 2023) ZZhang et al. 2021 Thin al. 2021). For the target, we take parameter m and put
Bi-modality in geometric tempering
Proposal
Target
π = (1 e01--mm/))
- Umm
(13)
where un is the smoothed uniform distribution on Im [-m, 2m] with density proportional to
Figure 1: Bi-modal intermediate distribu- tion = 0.45 with v, n as in Eq. 13 for
e +d(a Note that without the small mixing with the smoothed uniform distribution in Eq. [3] the target would be strongly log-concave, and the geometric path would remain well-conditioned. The next result shows that while this small mixing does not greatly worsen the log-Sobolev constant of it does cause the geometric tempering have exponentially worse log-Sobolev constant, in fact even Poincaré constant.
10.
Theorem 4 Let (/(0, 1) and be defined in Eq. 13 for (T)(T) & 81m yet for all E [], 1}.
10. Then CLs(V) and
[
Cr(ux) 2 4. 10
m 21-
m²

---

Published as a conference paper at ICLR 2025
Since CLs()) 2 Cp(-), the above holds, in particular, with Cis(UX) the left-hand side.
The proof of Theorem can found in Appendix D.3 This example is plotted in Figure | the intuition is that while both distributions unimodal and thus well-conditioned, the small mix- ing with the uniform measure creates multi mudti-mo in the geometric tempering. The proof relies on careful analysis to characterize this multi-modality, and then uses a test function which simply linearly interpolates between the modes to witness the exponentially large Poincaré constant.
Theorem 4 rules out the possibilityy that the geometrie tempering generically improves, preserves, functional inequalities. Since our upper bounds depend on the log-Sobolev constants of the geometric tempering, they also can suffer from such exponentially poor conditioning. But since log-Sobolev inequalities only govern worst-case convergence, it is still at least priori possible that geometric tempering nonetheless achieves fast convergence. We investigate this question further in section 5 and here instead continue exploring our upper bounds.
4.2 OPTIMAL TEMPERING IN THE STRONGLY LOG-CONCAVE CASE
Other than the log-Sobolev constants at,, the main input to our upper bounds is the tempering sched- ule A. and in this section we use our upper bounds to investigate the optimal schedule, an impor- tant question in practice (Grosse et al., 2013, Song and Ermon 2020). Both to make our analysis tractable and because of the degeneracy in the non log-concave case identified in the previous sec- tion, we here restrict to the strongly log-cone concave setting. Specifically, we assume that both the proposal and the target ı are log-concave with strong-concavity parameters On,, 0yy 0. In this case, 0₁ 2 (1 - X1)a, + A,a
We start by reformulating the continuous time result that we obtained in Theorem in the case where both the proposal v and target are log-concave.
Corollary 5 Suppose v and π satisfy Assumptions and are On and an -tronnlyy log-concave respectively so that as 2 Ae + (1 - Xe)au, and that the process is initialized at the proposal distribution Po v. Then for all 0. we have
:-())=1-2 %'' Xoas exp
KL(P:.T] T) S AGo(A),
exp ( ( duddsdss
(14)
where A is as in Theorem 7 Suppose additionally that ax 2 au. Then Gx(A) is minimized by the vanilla Langevin scheme X(t) 1.
The proof of Corollary is in Appendix C.I and relies on integration by parts. This new expression allows us to optimize on the schedule À independently of the unknown quantity A. As one could ex- pect, the corollary above states that when the target is already better conditioned than the proposal v, so O' 2 Ou, there is no need to temper and the optimal tempering scheme is given by vanilla Langevin À = 1. In the next proposition, with respect to v, then one should indeed use a custom tempering scheme other than Langevin.
show on the contrary that if is too poorly conditioned
Proposition 6 Suppose On < ay. Then the functional G((A) minimized for the scheme
<au av 1 + av
x(s) minn
(15)
ax 2 + 20y"
In particular, the optimal schedule does not depend on the horizon f, and when Oa 2 av/2, vanilla Langevin X(1)= = 1 optimal.
Proposition 6 is proven in Appendix C.2 Hence when the target is sufficiently peakier than the proposal Orr 2 a./ tempering does not improve convergence beyond that of vanilla Langevin. Conversely, when the target is flat enough ag az/2, then tempered Langevin with the schedule in Eq. [5] does improve convergence over vanilla Langevin. While the optimal schedule is provided analytically in Eq. 15] it cannot be computed exactly in practice given that the constant ar that describes the geometry of the target distribution unknown, On the other hand, in the limit of a flat and log-concave target Or 0, the optimal schedule tends X(s) = 2+ous which can be implemented in practice.
A natural question follows: can other schedules, which may not be optimal but are feasible to implement in general, actually improve convergence beyond that of vanilla Langevin? A simple
8

---

Published as a conference paper at ICLR 2025
example is the linear schedule. Here, obtain explicit convergence rate that does not improve on standard Langevin at large times, but that faster than standard Langevin at small times for small
ag.
Assume a > ax. Let t > O until which the continoous process 9 is run. Then, with the linear tempering scheme ((s) & defined on [0,1], G2 in the upper-bound of Corollary 5 Bwitts
Proposition 7
Gx(A) - Al(a, - ax) erfex (amy -
O+ ) - -av-toa)) erffx V
t
Ov
Ov Okm
where erfex is the complementary scaled error function crfex(x) = cn (1 -1- orf(x)) In particular, as 1 XG, we have Gx(A)
1
20,2"
The proof of Proposition appears in Ap- pendix
Opeimal bound
Varilla upserbound
Finally, compare in Figure 2 the value of the upper-bound at different time horizons t using the optimal tempering scheme given in Eq. [5 the linear tempering scheme A(s) = and vanilla Langevin A(s) = 1, for ar, 0.01 and G2 = 1.0. We observe that the optimal schedule always yields a lower value of G (as it should) and that it provides a clear advantage over vanilla Langevin at short horizons f. How- ever, as the horizon grows, this edge is even- tually lost. Similarly, we observe that the lin- tempering schedule improves over vanilla Langevin at short time horizons yet is eventually beaten as the horizon grows.
04-
02-
0.0
25
30
75
100
125
150
175
200
Figure 2: Value of the upper-bound G uling the optimal tempering, tempering,, linear tempering, and standard Langevin.
EXPONENTIALLY SLOW CONVERGENCE FOR TEMPERED LANGEVIN DYNAMICS
The upper bounds in Theorem show that, so long as the log-Sobolev constants remain well-behaved along the geometric tempering, Langevin with moving target can successfully sample from the target distribution. While this assumption holds in favorable cases where the proposal and target are both strongly log-concave, it may fail even when both distributions are well-conditioned yet not log- concave, as demonstrated in Theorem 4 Nevertheless, such poor functional inequalities do not necessarily rule out fast convergence of the geometric tempering because functional inequalities only govern mixing in the worst case, and so it is still possible priori that our upper bounds are loose in such cases. The purpose of this section is to develop rigorous lower bounds for the geometric tempering in two simple examples where the log-Sobolev constants of the intermediate distributions are poor.
Setup. Throughout this section, let the proposal distribution be the standard Gaussian (O, 1). as is common for the geometric tempering (Cabezas et al., 2023 Dai et al. 2020 Zhang al. 2021 Thin et al. 2021). We work in the setting of discrete temperature schedule but with continuous time inner Langevin iterations. In particular, we take as fixed a discrete temperature schedule Ay= = 0 < X₂
< A A 1 and sequence of times T ...... Tk that the inner Langevin iteration is run for. We then put pt, == v and define inductively p to be the law at time I of Langevin initialized at h-1 and run towards μıg- Our goal is then to study the convergence of the final output pK towards as a function of the temperature and time sequences, as well as the target ा.
Bi-modal target. Since annealing is designed to overcome multi-modality and associated poor mixing in the target, we begin by studying a toy model of multi-modality. Specifically, we take a parameter >> 0, and consider
1 N(,, 1) =-2 N(m,
v v==,((0,1) 1),
= 2
(16)

---

Published as a conference paper at ICLR 2025
It can be checked that 9, as well some of the intermediate distributions in the geometric tempering, have log-Sobolev constant exponential in the mode separation m. However, as we mentioned above, it remains possible priori that the geometric tempering avoids this exponentially poor conditioning by following non-worst case distributions. The following result shows that this is not the case, and indeed that the total time spent on Langevin dynamics must be exponential in ensure convergence.
Theorem Suppose v, a are as Eq. folor m 2 11. Then
TV T)
16 e-m m /64
Tk.
20
The proof of Theorem 8 can be found in Appendix D.2 Several remarks are in order. First, we em- phasize that this result has no dependence whatsoever on the tempering schedule. Second, because of classical inequalities between divergences (Polyanskiy and Wu 2022 Section 7.6), the same re-
sult also holds with V KL(pT, pa), v xxx (Ax) on the left-hand side. And finally, we remark that other works have studied related tempering algorithms with uniform, rather than Gaussian, proposal, and shown that in this case tempered dynamics can successfully sample from mixtures of Gaussians, so long as each mixture component has the same variance (Woodard et al.] 2009b] Ge et al. 2018). Theorem X shows that, in contrast, tempered Langevin dynamies with Gaussian pro- posal can fail for mixtures of Gaussians with identical variance, and therefore additionally suggests an unfavorable property of Gaussian proposals compared with uniform.
Well-conditioned target. The previous result establishes a simple case where tempering is a nat- ural approach, yet the geometric tempering suffers from exponentially poor performance. Since the target distribution in that case has poor functional inequalities, a simpler sanity check for the geometric tempering is to establish its convergence when the target is actually well-conditioned. We now analyze an example similar to that of Theorem 4 where the target has decent log-Sobolev inequalities, and show that, nonetheless, the geometric tempering suffers from exponentially poor convergence, at least until the end of the tempering scheme. For some > 0, put
1 : 2 Nm,. - uu
1
v==
(17)
where, as in section 4.1 9m is the probability distribution with density proportional to e ed
for Im = [-m, 2m.. Similar to Theorem although ग has a reasonable log-Sobolev constant, the geometric tempering is partially bimodal. The next result shows that the geometric tempering suffers from exponentially slow convergence, at least until the tempering is almost at
Theorem Suppose v, π are as in Eq. for m 2 4. Then CLS (T) S 324m². yet for all [K]
1 TV T) - V Σ T
for 6ū := 6m - (1-Xx)mm/10
The proof of Thhhore Iean be found in Appendix D33
This result demonstrates a simple case where geometric tempering suffers from exponentially slower convergence than Langevin dynamics, and to the best of our knowledge the first result of this kind for the geometric tempering in the literature.
As for Theorem 8 the statement holds with / (L(Pr,Ma) / x//x2/x 2 me) the left-hand side. We finally remark that Theorems ( and andWare are both consequences of more general TV lower bound, Theorem 20 in Appendix D.I which may be of further interest.
CONCLUSION
We provided the first convergence analysis of tempered Langevin dynamics, Eq. 9 in the literature, both for continuous (Theorem D. and discrete (Theorem time. Our bounds are naturally driven by the log-Sobolev constants of the geometric tempering path, and we identified in Theorem 4 a surpris- ing degeneracy of the geometric tempering where it can make these constants exponentially worse
10

---

Published as a conference paper at ICLR 2025
than that of the target. Restricting to the strongly log- log-concave setting, rigorously established the optimal tempering schedule for our upper bounds in Proposition and identified a regime where it is strictly distinct from vanilla Langevin. Finally, we developed rigorous lower bounds proving exponentially slow convergence for a bimodal target in Theorem g and even demonstrated a novel failure of the geometric tempering for uni-modal target in Theorem 9 where it has exponentially worse performance than vanilla Langevin. Interesting questions for future work include developing more complete understanding of the log-Sobolev constants along the geometric tempering, par- ticularly for the uniform proposal, as well as identifying alternative paths for Langevin which have more favorable properties. In this connection, we mention some recent works which modify the geometric path so that mode weights are kept constant along the path (Bhatnagar and Randall 2015 Tawn al. 2018); would be interesting to extend our analysis to these algorithms.
Acknowledgements A. Vacher, O. Chehab, and A. Korba were supported by funding from the French ANR JCJC WOS granted to A. Korba.
REFERENCES
C. Dai, J. Heng, P.E. Jacob, and N. Whiteley. An Invitation to Sequential Monte Carlo Samplers. Journal of the American Stafistical Association, 117:1587-1600, 2020.
Y. Song, S. Sohl-Dickstein, D.P. Kingma, A. Kumar, S. Ermon, and Poole. -rore-Based Genera- tive Modeling through Stochastic Differential Equations. In International Conference on Learn- ing Representations, 2021a
G.O. Roberts and J.S. Rosenthal. General state space Markov chains and MCMC algorithms. Prob- ability Surveys, 1, 2004.
C.P. Robert and G. Casella. Monte Carlo statistical methods. Springer Verlag, 2004.
G.O. Roberts and R.L. Tweedie. Exponential convergence of Langevin distributions and their dis- crete approximations. Bernoulli, pages 341-363, 1996.
A. Wibisono. Sampling as optimization in the space of measures: The Langevin dynamics composite optimization problem. In Conference on Learning Theory, pages 2093-3027. PMLR, 2018.
A. Durmus, S. Majewski, and B. Miasojedow. Analysis of Langevin Monte Carlo via convex opti- mization. The Journal of Machine Learning Research, 20(1):2666 1):2666-2711 2019.
Y. Song and S. Ermon. Generative Modeling by Estimating Gradients of the Data Distribution. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
A. Durmus and E. Moulines. High-dimensional bayesian inference via the unadjusted langevin algorithm. Bernoulli, 2016.
G. Parisi. Correlation Functions and Computer Simulations. Nucl. Phys. B, 180:378, 1981.
R. Zhang, C. Li, J. Zhang, C. Chen, and A. Gordon Wilson. Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning. In International Conference on Learning Representations, 2020.
S. Vempala and A. Wibisono. Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices. Advances Neural Information Processing Systems, 32, 2019.
W. Deng, G. Lin, and F. Liang. A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions. In Advances in Neural Information Processing Systems, 2020.
R.M. Neal. Annealed importance sampling. Statistics and Computing. 11:125-139, 1998.
A. Gelman and X.-L. Meng. Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling Path Sampling. Statistical Science, 13:163-185, 1998.

---

Published as a conference paper at ICLR 2025
S. Syed, A. Bouchard-Cöte G. Deligiannidis, and A. Doucet. Non-Reversibe Parallel Tempering: A Scalable Highly Parallel MCMC Scheme. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(2):221-350, 2022.
N. Chopin and O. Papaspiliopoolos. An Introduction Sequential Monte Carlo. Springer Series in Statistics. Springer International Publishing, 2020. ISBN 9783030478452.
A. Jasra, D. A. Stephens, A. Doucet, and T. Tsagaris. Inference for Lévy-Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo. Scandinavian Journal of Statistics, 38(1):1-22, 2011.
N. Chopin, F. Crucinio, and A. Korba. A connection between Tempering and Entropic Mirror Descent. International Conference on Machine Learning, 2024.
Kiwaki, Variational Optimization of Annealing Schedules. ArXiv, abs/ 502.05532, 2015.
W. Tang, Wu, and X. Zhou. Discrete-time simulated annealing: A convergence analysis via the Eyring Kramers law. Numerical Algebra, Control and Optimization, 2024. ISSN 2155-3289.
S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi. Optimization by Simulated Annealing. Science, 220: 671-680, 1983.
V. Cerný. Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45:41-51, 1985.
H. Lee, J. Lu, and Y. Tan. Convergence for score-based generative modeling with polynomial complexity. In Advances in Neural Information Processing Systems, 2022.
W. Guo, M. Tao, and Y. Chen. Provable Benefit of Annealed Langevin Monte Carlo for Non-log- concave Sampling. ArXiv, abs/2407.16936, 2024.
N. Schweizer. Non-asymptotic Error Bounds for Sequential MCMC Methods in Multimodal Set- tings. ArXiv, abs/ 20/110056733. 2012.
D. Paulin, A. Jasra, and A. Thiery. Error Bounds for Sequential Monte Carlo Samplers for Multi- modal Distributions. ArXiv, abs/1509.08775, 2018.
J. Mathews and S.C. Schmidler. Finite sample complexity of sequential Monte Carlo estimators on multimodal target distributions. The Annals of Applied Probability, 34(1B): 199-1223, 2024.
H. Lee and M. Santana-Gijzen. Convergence Bounds for Sequential Monte Carlo on Multimodal Distributions using Soft Decomposition, 2024.
C. J. Geyer. Markov Chain Monte Carlo Maximum likelihood. In E. M. Keramides, editor, Comput- ing Science and Statisties: Proceedings of the 23rd Symposium the Interface, pages 156-163, 1991.
E. Marinari and G. Parisi. Simulated tempering: new Monte Carlo scheme. EPL, 19:451-458, 1992.
K. Hukushima and K. Nemoto. Exchange Monte Carlo Method and Application to Spin Glass Simulations. Journal of the Physical Society of Japan, 6546):1604-1688, 1996.
Syed, V. Romaniello. T. Campbell, and Bouchard-Côté. Parallel tempering on optimized paths. In International Conference on Machine Learning, pages 10033-10042. PMLR, 2021.
N. Madras and Z. Zheng. On the Swapping Algorithm. Random Struct. Algorithms, 22:66-97, 01 2003.
D. Woodard, S. Schmidler, and M. Huber. Sufficient Conditions for Torpid Mixing of Parallel and Simulated Tempering. Electronic Journal of Probability, 14(none):780 804, 2009a.
D. Woodard, S. Schmidler, and M. Huber. Conditions for Rapid Mixing of Parallel and Simulated Tempering on Multimodal Distributions. The Annals of Applied Probability, 19, 06 2009b.
12

---

Published as a conference paper at ICLR 2025
N. Bhatnagar and D. Randall. Simulated Tempering and Swapping Mean-Ficla Models. Journal of Statistical Physics, 164:495-5300 2015.
R. Ge, H. Lee, and A. Risteski. Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition. ArXiv, abs/1 htss/1812.0093,, 2018.
Y. Chen, J. Chen, Dong, J. Peng, and Z. Wang. Accelerating Nonconvex Learning via Replica Exchange Langevin diffusion. In International Conference on Learning Representations, 2019.
D. Bakry, Gentil, M. Ledoux, et al. Analysis and Geometry of Markov Diffusion Operators, volume 103. Springer, 2014.
P. Cattiaux and A. Guillin. Functional inequalities for perturbed measures with applications to log- concave measures and to some Bayesian problems. Bernoulli, 28(4):2294-232 -3221, 2022.
J. Besag. "Comments on "Representations of knowledge in complex systems" by U. Grenander and MI Miller. Journal of the Royal Statistical Society, Series B., 56:591-592, 1994.
H.-. Chen, S. Chewi, and Niles-Weed. Dimension-free Log-SSoollee inequalities for mixture distributions. Journal of Functional Analysis, 281((1):109236, 2021.
D. B. Rubin. Comment on "The calculation of posterior distributions by data augmentation", by M.
A. Tanner and W. H. Wong, Journal of the American Statistical Association, 1987.
Y. T. Lee, R. Shen, and K. Tian. Structured logconcave sampling with a Restricted Gaussian Oracle. In Conference on Learning Theory, volume 134 of Proceedings of Machine Learning Research, pages 2993-3050. PMLR, 2021.
R. M. Neal. Sampling from multimodal distributions using tempered transitions. Statistics and Computing, 6:353-366, 1996.
H. Wu, J. Köhler, and F. Noe. Stochastic Normalizing Flows. In Advances in Neural Information Processing Systems, volume 33, pages 5933-5944. Curran Associates, Inc., 2020.
A. Thin, N. Kotelevskii, A. Doucet, A. Durmus, E. Moulines, and M. Panov. Monte Carlo Variational Eecoodnss In International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 1 10247-10257. PMLR, 18-24 Jul 2021.
T. Geffner and J. Domke. Langevin Diffusion Variational Inference. In International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 576-593. PMLR, 25-27 Apr 2023.
Song, C. Meng, and S. Ermon. Denoising Diffusion Implicit Models. International Conference on Learning Representations, 2021b.
M.S. Albergo, N.M. Boffi, and E. Vanden-Eijnden Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. ArXiv, abs/2303.08 08797, 2023.
T. Tieleman. Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. In International Conference Machine Learning, 2008
D. Carbone, M. Hua, Coste, and E. Vanden -ijnden. Efficient Training of Energy- Based Models Using Jarzynski Equality. Advances in Neural Information Processing Systems, 36, 2024.
P. Marion, A. Korba, P. Bartlett, M. Blondel, V. De Bortoli, A. Doucet, F. Llinares-Lóper C. Pa- quette, and Q. Berthet. Implicit Diffusion: Efficient Optimization through Stochastic Sampling. ICML 2024 Workshop on Differentiable Almost Everything: Differentiable Relaxations, Algo- rithms, Operators, and Simulators, 2024.
X. Huang, H. Dong, Y. Hao, Y. Ma, and T. Zhang. Reverse Diffusion Monte Carlo. In International Conference on Learning Representations, 2024.
Y. He, K. Rojas, and M. Tao. Zeroth-Order Sampling Methods for Mon-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion. ArXiv, abs/2402. 17886 2024.
13

---

Published as a conference paper at ICLR 2025
L. Grenioux, M. Noble, M. Gabrié, and A.O. Durmus. Stochastic Localization via Iterative Posterior Sampling. In International Conference on Machine Learning, 2024.
Saremi, J. Park, and Bach. Chain Log-Concave Markov Chains. In International Con- ference Learning Representations, 2024.
C. Domingo-Eninn Enrichh and A.-A. Pooladian. An Explicit Expansion of the Kullback-Leibler Diver- gence along its Fisher-Rae Rao Gradient Flow. Transactions on Machine Learning Research, 2023.
A. Cabezas, Lao, and R. Louf. BlackJAX: A sampling library for JAX, 2023. URL http:
7/github. www/black//aa/cccoccooca jax- maaxx-devs/blackjjaa blackjax
A. Bradley and P. Nakkiran. Classifier-Free Guidance is Predictor-Corrector. ArXiv, 0as/2408.09000 2024.
A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical Text-Conditional Image Generation with CLIP Latents. ArXiv, abs/2204.06123, 2022.
C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, J. Ho, D. Fleet, M. Norouzi. Photorealistic Text-to-lmage Diffusion Models with Deep Language Understanding. In Advances in Neural Information Pro- cessing Systems, volume 35, pages 36479-36494. Curran Associates, Inc., 2022.
Dieleman.
Guidance: a cheat code for diffusion models, 2022.
W. Grathwohl, K.-C. Wang. J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky. Your Classifier is Secretly an Energy Based Model and You Should Treat Like One. In International Conference Learning Representations, 2020.
E. Nijkamp, M. Hill, S.-C. Zhu, and Y. N. Wu. Learning Non-Convergent Non-Persistent Short- Run MCMC Toward Energy-Bane Based Model. Advances in Neural Information Processing Systems, 2019.
N. Ye, Z. Zhu, and R. Mantiuk. Langevin Dynamics with Continuous Tempering for Training Deep Neural Networks. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
G. Zhang, K. Hsu. J. Li, C. Finn, and R. Baker Grosse. Differentiable Annealed Importance Sam- pling and the Perils of Gradient Noise. In Advances in Neural Information Processing Systems, 2021.
G. Conforti. Weak semiconvexity estimates for Schrödinger potentials and logarithmic Sobolev inequality for Schrödinger bridges. Probability Theory and Related Fields, 2024.
M. A. Erdogdu, R. Hosseinzadch, and Zhang. Convergence of Langevin Monte Carlo in chi- squared and Rényi divergence. In International Conference Artificial Intelligence and Statis- fics, pages 8151-8175. PMLR, 2022.
Cattiaux, A. Guillin, and L.-M. Wu. A note on Talagrand's ' transportation inequality and logarith- mic Sobolev inequality. Probability theory and related fields, 148:285-304, 2010.
R. Grosse, C. Maddison, and R. Salakhutdinov. Annealing Between Distributions by Averaging Moments. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
Y. Song and S. Ermon. Improved Techniques for Training Score-Based Generative Models. In Advances in Neural Information Processing Systems, volume 33, pages 12438-12449 Curran Associates, Inc., 2020.
Y. Polyanskiy and Y. Wu. Information Theory: From Coding to Learning. Cambridge University Press, 2022.
N.G. Tawn, G.O. Roberts, and J.S. Rosenthal. Weight-preserving simulated tempering. Statistics and Computing, 30:27-41, 2018.
14

---

Published as a conference paper at ICLR 2025
K. Itó. On Stochastic Differential Equations, volume 4. American Mathematical Society New York, 1951.
S. Kotz, LLI Gikhman, and A.V. Skorokhod. The Theory of Stochastic Processes II. Classics in Mathematics. Springer Berlin Heidelberg, 2004.
V.I. Bogachev, N. V. Krylov, M. Röckner, and V Shaposhnikov. Kokkeoooo Equa- tions, volume 207. American Mathematical Society, 2022.
C. Le Bris and P.-L. Lions. Existence and Uniqueness of Solutions Fokker-Planck Type Equations with Irregular Coefficients. Communications in Partial Differential Equations, 3377):1222-1317, 2008.
S. Mischler. Lectures Notes: An introduction to evolution PDEs. Chapter 0: On the Gronwall Lemma. CEREMADE, Université Paris-Dauphine, September 2019.
A. Schlichting. Poincaré and Log- Soboleev Inequalities for Mixtures. Entropy, 21(1):89, 2019.
R. Holley and D. W. Stroock. Logarithmic Sobolev inequalities and stochastic Ising models. Journal of Statistical Physics, 46:1159-1194, 1987.
A. Maurais and Y. Marzouk. Sampling in unit time with kernel Fisher-Rao flow. Conference Machine Learning, 2024.
International
O. Chehab, A. Hyvarinen, and A. Risteski. Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Koise-Contrastive Estimation, and beyond. Advances in Neural Information Processing Systems, 36, 2024.
15

---

Published as a conference paper at ICLR 2025
APPENDIX
The paper and appendix are organized as follows.
Introduction
2
Background
Convergence rate for tempered Langevin dynamics
Analysis and optimization of the upper-bounds
4.1
Geometric tempering can exponentially worsen functional inequalities
4.2
Optimal tempering in the strongly log- log-concave case
5
Exponentially slow convergence for tempered Langevin dynamics
6
Conclusion
10
Convergence of continuous-time tempered Langevin dynamics
17
A.I
poseepsss of continuous tempered Langevin dynamics
17
Proof of Theorem
17
A.3
Continuous-time convergence in precision form
19
A.4
Proofs of technical lemmas from Appendix|A. A.2
19
Convergence of discrete-time time tempered Langevin dynamics
21
Proof of Theorem 3
21
B.2
Discrete-time convergence in precision form
................................
23
B.3
Technical lemma from Appendix|B. B.1
24
Optimization of continuous-time tempered Langevin Dynamics
25
Proof of Corollary
25
Proof of Proposition
26
Proof of Proposition
32
I Lower bounds
D.1 A general lower bound
33
D.2 Lower bounds for bimodal target
35
D.3 Lower bounds for unimodal target
36
D.4 Upper bounds on the log Sobolev constant of the unimodal target
Additional numerical illustrations
42
16

---

Published as a conference paper at ICLR 2025
A CONVERGENCE OF CONTINUOUS- TIME TEMPERED LANGEVIN DYNAMICS
In this section, we give the proof our continuous-time upper bound, Theorem [ In Appendix A. discuss the well-posedness of the continuous time dynamics. The proof of Theorem is then given in Appendix AA.2] In the next section, Appendix A.3] we derive from Theorem | for convergence to given precision. In Appendix A.4 we give the proofs of the lemmas used in the proof of Theorem
guarnnteess
A.1 WELL-POSEDNESS OF CONTINUOUS TEMPERED LANGEVIN DYNAMICS
Notice that Equation the drift b(t,z) V log (x) is time-dependent and the diffusion coefficient is constant. Still, existence and uniqueness of solutions can be guaranteed under mild assumptions. For instance, if V log and Voog are L, and L--Lisschitz respectively, the drift is also Lipschitz with bounded constant L S max Lv, Lx) (since the dynamic X is bounded between and 1), hence satisfy the usual linear growth condition in the second argument. this case, there exists unique, continuous strong solution (x))>0 adapted to the filtration generated by the Brownian motion which satisfies Equation (9) (It
et al.) (2004). Strong uniqueness also means that if two processes satisfy this equation with the same initial conditions, their trajectories almost surely indistinguishable. The law (Pe)e2o of Equation (9) satisfies a Bokkerr-Plande equation (Bogachev al. 2022) that can be written
Öp: V.
Pt 11
pt log
(18)
at
Generally, proving the existence and uniqueness of solutions for FPEs is more difficult than for SDEs, but one can also get under mild assumptions existence and uniqueness for solutions of Equa- tion (88), under the same Lipschitzness assumptions on the score of proposal and target distributions, and using that the tempering dynamics are bounded in [0, 1] (Bris and Lions 2008 Proposition 2).
A.2 PROOF OF THEOREM 1
In this section, we will prove Theorem on the convergence rate of tempered Langevin dynamics. The proof is in two steps: first, we compute how the process tracks the moving target μr, then we compute how well the moving target hh tracks the final target æ.
Step 1: contraction of KK(Pt, H4) over time.
Using the chain rule, we have
d
KL mr)
/ log
Pt
Opt
Pt 0j4
b.
di
at
14
Ot
The first term a involves the Langevin dynamics given by the Bokker-Planck equation Eq 18 Hence, using integration by parts and the inverse log-Sobole constants defined in Eq. g we obtain:
a /
log
Pt 14
V
(p.V log
Pt me )
pt v log
Pt
14g
FD(p:, /4)) -2,, (L.
(19)
The second term b is specific to the tempering scheme: it involves the tempering dynamics jus which zero when there is no tempering and are here determined by the tempering rule μ4 = cx, v² de a where Cx, 1/ - is normalizing factor, that we will denote c in this proof to alleviate notation. We can compute the (log) tempering dynamics
0 Olg 14 Øt
(^e log " v
+ +lgg v- - log / v ul = 2e log
a --- ot
Øt
v
J -
Å log π 'o f log x vl vl-de
-1 i 2010g log π Ey,, log T
so that the second term becomes
/
① log Mr 2e Epp. log π
log =1
Pt Øt
Ep.
v/
To control this term prove the following Lemma in Appendix A.4
17

---

Published as a conference paper at ICLR 2025
Lemma 10 (Discrepancy between the laws of the sampling process and tempering path) We have
Byg
log v
n
Ep.
log
"
2 + Lw) ((d + by + bm) + Exo [11x13]).
A
Let A be the right-hand side in Lemma 10
namely
2(d + by + bx) + p 111=1131.
A = 2 + Lu)
(20)
dy A ax
Then we obtain
b 6 A.
(21)
Combining Eq. 19aand and Eq. 21 we obtain
d KL(P,, S -2a, KL pe) AÅ,
To conclude this step, apply Grönwall's lemma (Mischler 2019 Lemma 1.1) to yield
-20,dds (L(p),pt) S exp -2222 KL(p,, + L in exp (f." -2ar,d ds.
Step compare KL(pe, me) and (L(pe, R²) This measures how well the moving target μr tracks the final target x. We write
KL σ) KL(pt, + KL TT
KL (b)
= -T((pt, + Eyp log
Me
-KI((t;, + - X ER
log
log
We make the following observation on the log normalizing constants, proved Appendix A.4 Lemma (Bounding the normalizing constant the law of the tempering path) We have
0 S log e min (x. KT, Xx) (T,T
Using Lemma to control the normalizing constant cz, we obtain
KL T) < (L pc) + (1 Xe) Ae) log ( + Ex log - )
Finally, we control this last term using similar estimates as in Lemma 10 the proof is again deferred to Appendix12 A.4 Lemma (Discrepancy between the laws of the sampling process and target) We have
2( Tw)) (2)
(2 + by + bm) Epo 111=:11]] a, az
E
log
E.p.
log b
Applying this result we obtain
KL(Pt, m)) S KL(Pt, + A.)
Step putting it together.
We find
KL(pi, m) < (wi) + (uz) + (uz)
(22)
(u1) = exp -2 //' ands
KT.(po,
(u2) = (1 - -x)AA
(ua) /'" in exp
-2 ando
badooaas )ds
18

---

Published as a conference paper at ICLR 2025
A.3 CONTINUOUS TIME CONVERGENCE IN PRECISION FORM
As in Vempala and Wibisono 2019), we can derive from Theorem 1 sufficient conditions for con- vergence to precision by setting each of the three terms in the upper-bound to be less than e/3.
Corollary 13 Suppose the assumptions Theorem Nolld To achieve convergence with preciston
e > 0 such that KL(pt, π) < sufficient conditions on the time and schedule X(-) are
3LL(p Ho)
1
6A
1 >> max
log
log
(23)
20min
Ommin
d 3A
(24)
A 6 + 2x/2:
(25)
where Qmin = mills00 0s smallest log-Sobolev constant among the path of tempered distri- butions.
Proof of Corollary 13
We upper-bound the first term by (u1) 2texmin KL Po) that (KL(oo)))o
sufficient condition for (u1) < is
log
20tmin
A sufficient condition (u2) < e/3 A, E]1 ft 1]. If is bijective, this yields t x-1
Finally, we upper-bound the third term:
I// ' exp
-2 ff aodv ds
(us) A
(26)
A (
1/2
-2 2 andu
ds (" i (-2 -2 f andv jds) ds)
in exp
(27)
2/2
fr apdo
1//2
2 f"r )pds)
A exp
(28)
2/2
1/2
< A [expp- -lamm (Ne/2 Ao) + (de X2/2))
(29)
A eexp(=tamin + (de 24/2))
(30)
so that sufficient conditions for (wz) 3 can be obtained by setting each of the terms smaller than E. yielding
log GA and A A4/2 < e/6.
A.4 PROOFS TECHNICAL LEMMAS FROM APPENDIX A.2
In this section, we give the deferred proofs from Appendix A.2 We first state and prove two observa- tions that will be useful, before turning to the deferred proofs. The following proposition translates the dissipativity assumptions into a condition on the tails of the proposal and target.
Proposition 14 (Tails of the proposal and target) For all x E Rdd, we have
by
(1e0112 + b
Vx(x)
inf (v) < 2L, (| + yERd
V=(2)
inf V+(v) S 2L
DERS
ax
Proof of Proposition
14
Observe that because
(VV2(r), 2 ar||=/1 by.
the function V, must be minimized by some point zo E B(0, v Au Therefore, for all x e Rd.
(x) inf Vv(2) Vz( Vo( s L, 11 zoll2 < 2L, (1=x||² bv) a,
MER
The proof for V. is identical, omit it.
19

---

Published as a conference paper at ICLR 2025
The next observation is main quantitative use of the dissipativity assumption, and crucial for our control on the terms arising from the tempering dynamics in the continuous time convergence proof. Lemma 15 (Bounded second moment along the dynamics) For all Z 0,
pp [||==]]]]] & max (Epullla>)1) + by + bx
A
Proof of Lemma 15 we find
Using the Fokker-Plancee equation Eq. 18 and integration by parts several times,
BpE || ||x||] 1x/l/2v. V (pe V log Pt
=2 / (xr, Vlogp: V log pt)
2d -2 8 (L X,)V,, d vVV)).
Applying the dissipativity assumption, we obtain
B, ||1 - 2(d + (1 d.)6 + debx) 2((1 Ar)a, + Aca ) [||=111] 2( + b, + b,) 2(a, A o. 1p,
Therefore, as soon pp || exceeds the level (d + by + bx)/ A a,),, ), it must start decreasing.
The result follows.
We now give the proofs of the Lemmas from Appendix A.2
Proof of Lemma 10
Use Proposition 14 to obtain
E,4 log
π
- Ex, log
Ey, Vv Val + E...
V7 - -V=]
- Ep, [V. - inf Vo(y)] + Ep, [V=
inf Vz(1)]
DERd
VER
< 2n, |||=||] + 2L Ep. |||=||2] + 2L by
bx
+
2L
av
ax
Let M be the quantity appearing on the right- side of Lemma 15
Then we obtain
E log
π y Ep. log
2L [|=|||) - 2L [M +
bx
+ 2L,
by
ar
av
To handle E,, ||1,1 we apply the dissipativity assumption to yield
E,, |||0||| < (1 Ar)a + Xea
1
Eyi ((1 (([1-Ae) V.V + AeV., x) + + de)oe tebs
Integration by parts gives
d+ (1 - de)ber + debr (1 dz)a + dea
d + bu + bx
Ey ||
av A Oz
Hence,
Em
log 1/
Ep,
log π
bx <2L
+ max [1::111
+ by + +b
ax
Ov A
by
+ d d bw +bx
+ 2L,
ar
av A ax
Using max(x, y) s x + when v, 2 and
by by
+ by by
<
ar
dx
Ou ax
conclude.
20

---

Published as a conference paper at ICLR 2025
Proof of Lemma 17
We have
log = log
- log
ui--da m
The lower bound is immediate from Hölder's inequality. For the upper bound, write
(
A
log "1 Ae (L
log C. = log E,, using Jensen's 's inequality. Similarly, we can write
- A
[(*) 77
1-
log ct = log Ex and therefore conclude that
S (1- Ez
log V
-1- No) (z,,)))
loger S min (Nr (1- (L)
Proof of Lemma
12
Denoting by Vi (resp.
V the infimum of V (resp. Vo); we have
Ex
log
Ep.
log
pc(V= V (V. V2)) + ddT(V - VC) -V= - V.))
Using Proposition
14 obtain the upper-bound
π - 2 Ep. |1 + bx + 2L E || by
Ex
log
Ep.
log
an
Let M be the quantity appearing on the right-hand side of Lemmal 15 Then
E
log
pi
log .
< 2
M
bx
+ 2 E. ||
b
0g
On the other hand, a direct application of the dissipativity condition Assumption 2 implies
d bx
d + b, + by a2 A az
E. [| S an
E. Krr, VV+(2))
bx]
at
Therefore
E
log π
- Ep,
log
< 2L max EEpo [1x]]]]]]
d bu + D
+ bn
On A ax
ax
2L d + by + bu 2L. by
Or A at
aw
Again, bounding max(z, y) S for 2 and using
by
bx S + by b
O ax Ou A aT
conclude.
B CONVERGENCE OF DISCRETE-TIME TEMPERED LANGEVIN DYNAMICS
In this section we give proof of our discrete-time upper bound, Theorem 3 In Appendix B. outline the proof of Theorem 3 In Appendix B.2 we derive from Theorem sufficient conditions
precision (pk, π) < e. And in Appendix B.3 we give the deferred proof from Appendix B.
B. PROOF OF THEOREM 3
We next prove Theorem Broadly speaking, the proof is in two steps: first, compute how the process Pk tracks the moving target Pa. then we compute how well the moving target μ4 tracks the final target
21

---

Published as a conference paper at ICLR 2025
Step 1: contraction of (p&, pe)) Using Vempala and Wibisono (2019 Lemma 3), under As-
sumptions
and and h2 I
0g
we obtain
423
KL (R) KL(ps-1, pa )e ache 6
(31)
Next, can compute the (log) tempering dynamics of Mi
Ev Au Ax
log u lo (Ax Ax-1) log
log
y -
π
1 No- T
Ab-1 y2-
V
(Ax (R-1)
log
ul -1-dE
(Ax AR-1) log
V/
log Bux
(
Ak A
π
(Ax AR-1)
1og
1/
Exs
log
π
71
where we used Jensen's incquality at the last step. Thus
KL(pk-1, (4)
(L(P Mk- (1L ( 11--1
-KL( Mx-1) / Pk-1 log
p4-1
P4
KL (x-1) + (Ax - >x-1)
Eps
log 1/
Epg
π log
- w7
Using Proposition 14 we can upper bound the difference of expectations via Buc
log n V Ep--1 log
- 2L, [x |||==11++++++
by ( 2L 111=1111=
D
a,
ax
The next LeMma, proved in Appendix B.3 controls the second moment of Pk. Lemma 16 (Bounded second moment along the dynamics) Suppose hx I min (1,
a. a. 2T
Then
Ep.. [|[x]]]]]] S max (E [1+=1111 2(3(b, + bz) + d) ax A ap A
For the second moment of μı we use the following bound from the proof of Lemma I0
d + bu bx a, A AO
Ep, |||
To combine these bounds, let us define
A' = 2(4x + { max (Epull==1?), 2(3(b, + b,,)2++ d)
+ 3(d + by + bm)
(32)
an A
ax A
Plugging into Eq. we obtain
(T (Pk, (1x) & KL (4-1) ayhn + (Ax - A achu + 6h
Now put di = Σj= 0zhi. Unrolling the recursion, we obtain
KL(Pk, S se KL no) +
Σ
(A((A: ---1je -ach + +627dL) ) -
Vits
(33)
1=1
Step 2: compare (L mk) and KI,, Here we use Lemma
11 to control the log-
normalizing constants and obtain
KL(Pk, x)
KI(pe, + KL(Pk, x)
KI pk)
pk
= KK(pp, m + Eps
log
π
log 1/
= (K) + - >x4
+ log Ck
< (L)(e,me) + - -Ax)
log P
+ KL(TT,)) v)).
22

---

Published as a conference paper at ICLR 2025
To control the term in parentheses, apply Proposition 14 to yield Pp |||
6x
(.)
Pa log π KT(T, - 2
+ 2T
E
ax
ag
Using Lemma
15 and the following bound from the proof of Lemma 12
+ b + by
E 1
S ay A ax
we find
+ KL( v) A', log V1
E pk
for A' in Eq. 32
Step 3: putting it together. Combining the results of steps and 2, we can finally write KT(pt; x) s (v1) (u2) (uz) + (v4)
(34)
(v1) = exp Σ (باr (I
(12) = (1 - -xx)AA
k
(vs) A' >> (A: exp (- anhh
(w)) = > dL exp Σ agh1)
j=it1
B.2 DISCRETE-TINE TIME CONVERGENCE IN PRECISION FORM
Corollary 17 To achieve convergence with precision conditions are
0 such that KL(pk, < sufficient
h min
Omin
Omnnn
Or A Ov
(35)
40min
96 d 4(Lx + L2)2
2(Lx L )2
(KL(
2 hommio
8A'
h: > max
log
log
(36)
h1amin
c
A > 4A'
(37)
1 < 24A'
A - [A/22
(38)
where Ommin min.s0 Og and Lmax maxx50 Le denote the smallest (resp. largest) log-Sobolev (resp. smoothness) constant among the path of tempered distributions.
Proof of Corollary 77 Again, as in Vempala and Wibisono (2019), we can derive sufficient condi- tions for convergence with given precision KL(Ps, π) < by setting each of the four terms in the opper-bound inferior to e/4.
First, upper-bound the fourth term:
khaammn
(wa) < 6h2 d max
(k-i)hammn = 2
(39)
max
1
horwn
4
2 max
< 6/t d.? max
6h 2 max 3/1min
(40)
hotmmt
O'min
where in the last inequality, similarly to Vempala and Wibisono
2019 we use that e > g
for 0 < = homin < which holds assuming that h < 40minn Thus, a sufficient condition for
(1)) < is 3212 C.
23

---

Published as a conference paper at ICLR 2025
Next, upper-bound the first term by (u1) -khamin (pu Hu) so that sufficient condition for (wi) < c/4 k
1 log (KKI((p..Ho)
hommin
A sufficient condition (U2) < <</4 is d E]1 IA 1..
Finally, we upper-bound the third term:
(Us) == A' Σ (A: - A1-1) exp ( (- 21hon
ag)
(41)
im1
jm
1k/2
A'(
Σ
(As 11-11 exp
- h Σ
a1)
+ Σ (A+
(
1 exp (-h
Σ
as))
(42)
i=1
j=i
k=/1/
k exp (-h Σ as)
12/2]
S A'
Σ
(>: A +
Σ (Ai-\i-1))
xi-1))
(43)
i= 1//
1=1
1A/2
S A' exp( sep(--hiee h-kaaii// (2)(//) X1) + (A+ -11x/2)))
(44)
S A' exp((hhammi/ /2) + A'(x (1/221)
(45)
so that sufficient conditions for (u3) < & can be obtained by setting each of the two terms smaller than g. yielding & funin log SA'' and A AL1/21 < AA.
B.3 TECHNICAL LEMMA FROM APPENDIX B.
In this section, we prove Lemma 16
Proof of Lemma 16 We recall that the particles follow the recursion
Xx+1 = Xx h4 VV (X+) + v 222
where zy follows a standard normal distribution on RRd and X Pk- Hence, we have
E XR+1| |
[[[] X he VV (Xx,) + 21
(46)
[E[] X₂ hog VVA (Xt) [ + 2dhx
(47)
= E[[ 1= 2h x VV,, (Xx)) + + (xx + 2dh
(48)
The dissipativity assumption yields E VV,, (xx) 2 (ax A a, |[|| X | - (b, + b,) and as in the proof of Proposition 14 we have VV, (X+) < 2( + T,,,, (2 |
by- so that uu
BlllX 12] E|| & ||3| 2/v ((a, a |11 3| (br + bz)) 2dhx
by + b
+ 2 L,)² (E1IIX*11))
au A og
Letting L == Lo + Lu. a = au A a,r and = by + by we obtain
Lb
11 / ||3] - 2hx (a (a-1x 121l))B11 [a (b + d)
12
= = /| [1] 2/ (a - ---b)) + 2hv (b+ d + hu
Now by induction, assume that X 1- < ma 11), 2(3b/2+d) a h > 0 hence if E[|] ||-]
).
By construction on hgs
b+d+hy 2h, 12) then E||| Xx+1 || < [| and in par-
ticular El| X|||X++111 [| < max(Ep, ||| 2(3b/2+d) ). Now, if EI 1| awl
L+d// then if L2)
24

---

Published as a conference paper at ICLR 2025
1 > (a h2 we have
b + d - h2 b/
B|| X|||X++111] -
2h1(a - -bg. + 2h + d h1 b/aa)
(49)
12
b + - h2L2b/a
(50)
2(3b/2 + d) a
(51)
by construction on h2. Conversely, 1 - 2hx(a - h2 then
L26
X 1| S 2h ( + h4
a
S 2(3b/22 + d).
C OPTIMIZATION OF CONTINUOUS-TIME TEMPERED LANGEVIN DYNAMICS
C.1 PROOF OF COROLLARY 5
We will rewrite and bound the terms (w1). ). (u2) and (ua) in the upper bound of Theorem provided
in Eq. 3q.34 so that the role of the tempering schedule X()) is more explicit, at the expense of a looser upper-bound.
Rewriting (ua). We begin with the third term, using integration by parts to write
(u3) A (' 2, exp
-2 (22 a,dv) ds
-2 f a,dv - L' Asas exp
-2 (22 a, )ds)
= =A(>= do exp
exp
-2
Recalling (w2). Recall that
(uz) = A(1 - Nτ)
Rewriting (uz). We recall that (w1) exp( exp(-22/0 So a,d KL Mo). Assuming that the process initialized at the proposal distribution Po V/ can the tempering rule be -- As A to rewrite the KL term
(L, do KL T) log co-
By positivity of log co. can write log co No (I( v). In particular
KL(U, no) S Aa(KL(U, + KL(T,P))
= =>0 s0((115to) - V -Va(x) - VV)))((x)+ /IVoL)) -V, - -VV(x)) - vz))d(())
< A.A.
Upper bounding the sum. We can now combine upper bounds on (ur), (w2) and (ua) and sim- plify the result, using that the terms A exp( -2 fo a, and AA, cancel out, to finally yield
G((A):= = 1- -2 L teaa exp (-2 f. andv) ds.. ds.
KL(pr, S A. G.()),
(52)
25

---

Published as a conference paper at ICLR 2025
Optimal schedule when ax dv
Now, let D. exp (-2 Sa andu). We have
- - A.c.D.ds.
Applying integration by parts, we find
Gx(A) 1 (3. + / d. =1 de to (' o Dd
Since A 2 0 and Φ 2 e - 2(t-s)an (using the assumption On 2 ay), it follows that
G((A) 2 1 d Age 2to, + S// ie 2(t 2(t-s)o-ds.
Reversing the integration by parts, obtain
G((A) 2 /."" Xazze 2(c-s)a ds 2 1 21-2 2/// axe 2(e- 2(t
where the last inequality uses A S 1. Recognizing the right-hand side as G, evaluated at schedule
À = 1, we conclude.
C.2 PROOF PROPOSITION 6
Sketch of proof. We first make the change of variable ((s) = exp J a,du)) so that the problem
can be re-written
Ovu
2 ds
Ov 29 2
sup @Ef. G22 - ax
where It is the set of strictly positive functions Φ : [0,4] + R with weak second derivative and such that ag 2 & S a and o < <² and (tt) 1. Unlike the previous objective G(-), note that our new problem has now a concave objective yet has non-convex (and non closed) constraints. Hence we make convex relaxation and solve instead
1
&dds T
sup DEJ, 02 OlT
where Jr the of non-negative functions in W ((0,(((, such that ag 2 < 4 and (t) 1. After showing that an optimal solution & indeed exists, we explicitly describe it and show that belongs I a.k.a. it verifies the original constraints. In order to come up with an explicit solution 4, we make a disjunction of cases on whether the constraint a, 1 & is saturated: if always saturated, then & is exponential if not, then it must be linear maximal neighborhood around the non saturated point. We then prove that this maximal neighborhood unique, pathwise connected and can only be of the form [0, zo] with To < t so that is linear on [0, zo. and is then exponential on [xo.t].
Proof of Prop. 6 tempering schedules is equivalent solving
We first notice that minimizing G, defined in Corollary 5 over the set of admissible
sup (' X,ay exp (-2 dndd)dss, XEA J
where A, is the set of functions from [0,4] to [0, 1] with non-negative weak derivative. Proposition 18 Suppose ax a, and let I, be the of positive functions & with weak second derivative, such that 4; = 1, a. +b < & I a, de and क 2² is non non-positive measure. Then
sup /'' "eass exp ( (-2 (nndu) ) sup AeA:
_(ay (av -u-"" $3
0ly 03)
(53)
oeI, Ox ax 2
2
26

---

Published as a conference paper at ICLR 2025
Proof. Let À E A₁ and denote 4, (s) = exp( ( S a.du). The function & is strictly positive and such that 4 1. Its first and second derivatives read
b. = ag ag
.. = åg 2 + $2
x.
In particular, since À E [0, 1] and 0g (1 \a)ar >g we have 02 (0n, and by positiveness of d it holds
a + v 1 a.
Similarly, since À is a non- non-negative measure, (a ay ) is a non positiis measure and so is 6.22. This implies in particular
④ 6² àΦ² 0.
Hence we verified that 4. Furthermore, reads
S.' Asas exp ( (22 f-" andu ds
(av as)as exp -2 f anduy ds
Ott 0
O2 o $,
$3
av
ar 0
Oly 1
[ (2ds)
av
a
Oly (1 $ f (s())ss)
a₂ 1l
In particular, we have
sup ( Asas exp (-2 andu) ds v sup Aede
av i2ds
tET, ay 0g
2
Conversely, let & E lt, defining a, 0 log + b/4 and X that X E A, Furthermore, the previous computations show that
aa-as we have by construction
l (2ds) S/" "xos exp (-2 andd )do. do.
Ogy 2
Og Ox
In partieular, we recover the reverse inequality
1 Cv
d2ds
Ow
0
S sup I Axcreepp (-2 (nddu) )o. do. C xeλ,
sup ax 2
TEI, Ogy
2
Hence the problem can be re-written as the minimization of the convex functional & + So $3 + av 3 under the constraint E Jt. As can be noted, the constraint $6 क² < is non-convex and 2 the constraints & 0, & has a weak second derivative are non-closed. Hence we are going to relax all of these constraints and solve instead
/ dd + 2
av 22
inf 4EJ,
(54)
where Jj ([,)([0,8) is the set of positive functions such that 4, 1, a < + almost everywhere. Note that we also dropped the constraint & a. for convenience of the proof. Nevertheless, we show that at the optimum, the solution of the problem above verifies all the initial constraints.
Proposition 19 Assume Our < Og- Then Eq. expression:
54 admits a minimizer + which has the following
1. If or 22, then T. (()(())))
27

---

Published as a conference paper at ICLR 2025
2. If ou t2,
< ax S then
o0 -) -1)
2a ca c
- ov
-t)
8 $ 1 2
Φ caels-1)
au
$ dr dv
3. of 0r S
av 42 then 4,
du 24
8 (1
tay 24
Proof. We aim to solve the minimization problem defined in Eq. 54
Existence of minimum. Since the objective has compact sub-level sets and is continuous for W ((O, d) norm, we can extract a minimizing sequence (Φn), in Jr that converges in W122 Because W metrizes pointwise convergence, obtain that the
towards some & in W ((o,t)) tl). infimum verifies two of the constraints: 4 = and 1b 2 0 almost everywhere.
Then, since Un E Jr, for so, such that so, 50 + [0, we have S ax S &n, and then:
an (2() S
ax
(2n) S
S0000 to
(dn) (1n)eoth (Ln)s
h
50
Letting go infinity and h to zero, obtain, for almost all so [0,1]
0g 4 S + 20
hence @ E Jr, showing & is a minimizer of Eq. 54
Explicit expression of the minimum. We now derive an explicit expression for & based on whether or not there exists a point zo where the constraint
Ogg ① < Φ
saturated.
Case 1:
the inequality is saturated everywhere. In the case where the inequality constraint above is saturated for almost everywhere, we obtain that the minimizer of the form 4, K cxp(a,s) and using the fact that o; 1, we recover that + = exp((s - (<)(((s))))))
Case the inequality is not saturated saturated everywhere. Now let us assume that there exists vn where the inequality is not saturated by the minimizer
a, (x)) < ((xo).
We can make a disjunction of cases with respect to
If .o We will show that is linear on j0, t]. We first define a neighborhood of an of size that satisfies certain inequality. By continuity of & and the definition of the derivative, there exists an e > such that for all 8 <
Og (t) (t)) l(t - 8)
(55)
5
Let us now define the neighborhood size to be the largest e > 0 such that - 6 2 and such that the inequality above holds for all 8
Next, define candidate & that is linear in this neighborhood, and equal to the minimizer + outside the neighhorhood. Let
((x) ((x) if c,
t1 (:) +() -0) + ( - c, t].
By construction, & verifies the constraints, i.e. & Jr. Indeed, taking 8 in Eq. 55 and using the monotonicity of क. get successively for almost every [0, t):
(()) Φ(t c) ((x).
ag x(x) < a, (()
e
28

---

Published as a conference paper at ICLR 2025
Hence, & is suitable candidate for solving Problem Eq. 54
Finally, we show that the candidate we built & achieves smaller objective than minimizer and must therefore equal to a minimizer E Jr. By optimality.
[. (s) + Ohe $(0)² & J. (s)dss + av (())
2
and since & and & coincide [0, - e]. holds
[ (s)dss (0(1) 5(1 e))
e
The same inequality in the other direction is obtained by applying Jensen's inequality
[ (s)dss 2 { ((s)ds)²
(1( + =))²
e
We are therefore in the equality case of Jensen with strongly convex and smooth function, so must hold for almost everywhere in the neighborhood [t e, t] that
((t) ( -c
((s)
and in particular O is linear in the neighborhood [t t] and is of the form ((x) ar + b.
Finally, we will show that the neighborhood covers the entire interval [0, or in other words, that t. Suppose it is not the case and < t. Then, by continuity of db at the neighborhood border
t
a₂ (() ((t) Φ(t -)
Combined with the fact that (t) this implies On which is contradiction with the inequality Eq. [53 that characterizes the neighborhood, and in particular yields a ag. Hence, we must have = and in particular, d is linear on the whole interval [0, t].
If ao Ej0,[[ As before we can define and e... as the largest e > 0 such that xi + t < (resp. a, ( + 5) for all 5 c (resp. (r(zo)-b(Zz
To - 0) and ((u++6)
a₂ (20)) for all < c_). Again, we obtain that D must be linear on [zo. Do + c+] of the form ((x) = a x + by and linear (2o e-, mo] of the form (xx) a_ x b_. Furthermore, since & is differentiable
Co, its derivative must read
((x h) (
( o h)
(200) lim h-
lim h-+0
which implies a4 a == and by continuity of at To. we also recover b4 = b... In particular,
& is linear across the whole interval [ro e-, vo + e+ We have now four possibilities: either a) and xi + < c) xig - e__
To - < and Zg + c = I, b) 2ig - e
and xo + = d) Zo c_ 0 and Zo + c < The case a) simply corresponds to the case where +b is linear across the whole interval. The case b) implics by continuity that = a ((x)) = a, D( + ++)
which implies that = 0. This cannot hold as we initially assumed that a (<0)) < ((x)) = which would result in violation of the positivity constraint. The case e) yields the contradiction a, (2,)()))) and a > a ((a)). Finally, let us deal with case d). We prove that in this case, there exists no other point outside [0. + <4] such that the constraint o, < Φ is not saturated in particular this implies that क linear on (0, Fo + c+] and of the form exp(ax.)) on (ro + c4, ft.
Let us assume that there exists 1 outside of [0, + c.] such that
a² (x)) < ((x1).
Just as before, this implies the existence of é. such that + is linear across [0, æ + E..] and such that
$ (x) + ē4) ((x))
a, 2 ē+) ) <
E
29

---

Published as a conference paper at ICLR 2025
By linearity, if we take such that 2o + x1 ét, the right hand side is also given by ((x0t)))((z0) and since + is strictly increasing, the left-hand side can be lower-bounded by ax (Er + which implies
① (50 + +e) (
a (x0 + +c)
<
for all such that To + < æ1 + é.. Since 21 + + > Io + E+, this contradicts the definiton of which was chosen as the largest e such that for all c S c+.
$ Φ + c) (
0x ( 6)
In particular, the constraint a=$ < is always saturated outside of (0, ++]l
If 2g = 0 As before, we obtain that is either linear across the whole space [0, t). either there exists Tp such that & is linear across [0, mol and of the form K cxp(a, [ t and such that Ox (( (2r)).
Summary of the minimizer candidates We can now summarize the candidates we obtained for the minimizer function Φ. It has the three following possible forms:
Form 1: Φ is an exponential function of the form K exp(ag<) on the entire interval [0, t].
Form 2: Φ is a linear function on the entire interval [0,t].
Form 3: O is linear function then exponential function on the entire interval.
Specifically, there exists xo EjO, o{ such that is linear on [0, zo! and of the form = on [zo. E| and such that ag ((zo) = (z0))
We shall determine which candidate & among the three is the true minimizer, by evaluating the objective function Eq. for each and picking the candidate which achieves the lowest value.
implies that the function & is entirely determined and given by (bx) exp(ax (x t)). In this case, the value of the objective is given by
Form 1. Since (t)
L 4 (s)ds or (0) =
1 2a, (x 1)
20,t
ar
2a, t (a, an).
Jo
20
2
2
Form 2. The function + is of the form ((x) ar + b. Since +(t) 1, the coefficient b is given by 1 1-aad at. Furthermore, the constraint 4 < ④ is equivalent solving 2 Our and the constraint 2 0 becomes a S 1/t. Hence,
case where 1/1 < On,, there is linear admissible potential. In the case where, 0r 1/t, need to solve the following one dimensional quadratic problem
inf la 1/12a20,
+
av at)?
2
The objective can be rewritten as 0(a) a² fav) - aata, + qv. The minimum of 0 is given by
a, 2 to
(56)
Note that we always have a, S 1/t, hence if ax S a, the minimum is attained for a a, and is given by
0(a") 0₂ 10 Ovy 4 2ta, 2
lo + tax,
O₂ 2 + ta,
If Or >> a. the minimum is attained for = an and is given by
8(az) t + av - (1-azt)² ta² + Or a + (tax)
200 ag
= taz aa, + C.aay )+ + av
2
2
2
2
2
30

---

Published as a conference paper at ICLR 2025
There exists 0 < Fo < t such that the function d is of the form or + b over [0, and then is given by ea (w-t) over [st, 1] and which is such that (xv)) o, $ (2o). The continuity of & implies that az9 c²x(20-() and the previous equation gives (ara + b) b)ar, Both these equations uniquely determine a and b when To is fixed as
Form 3.
e" (50 (#0-2), b b= ea (1 (1 axTT))
and in particular the positivity constraint equivalent xo I 1 note that the constraint a, S indeed respected over [0, f/. Hence, we must solve the following one dimensional problem
ar
inf
Too 2 (0, t) +
On 2 2
20 (20 (x0-1)) )) au 20, (zo (0000)((1))))))))))))) (1 0g.
Let us remark that 20 = 0 being an admissible candidate, the potential (xx) = e (x-t) is indeed an admissible candidate and in particular, the value of the problem above is always lower than the one obtained with Form 1. Denoting f the objective function, let us compute the derivative of f- For all To. we have
fz(0 af aZ + 2 a2 + t1 -x)a7) 2000001))
-ppas - Tp(a je2(
= 2 ar 20 -a, - x0ax) +1
(1) z0ax))
=ne o (20k7 a, xn(()))
The term (2a, a= x)ax)) cancels for xo =
2 hence if ar 2 2, the derivative
a, f' is always non-negative for all 2 0 hence the optimal choice of zo is given by xg 0. Let us remark that this case implies on > a, where a, is defined in Eq. [56] and and for which the optimal Form 2. is given by ((r) = a, + (1 lxz) which an admissibile candidate for Form 3. This proves that whenever 0z 2 the optimal potential is given by ((z) = c=-(s-t) with = t. and then increases. In particular, since
If 0x 2 then f decreases between 0 and 1 aa a, -- 0s the minimum attained for xo = min(t, 1 2.. In the case where S - which is equivalent to a S a, with a, defined Eq, 566] the obtained potential linear hence is necessarily sub-optimal with respect to Form 2. which yields as an optimal potential
(xx) = a. + (1 (1-10) a,t).
Let us now place ourselves in the case > 1 2. In the case where > 2 which equivalent to an > a., the optimal potential yielded by Form 2. (if any) is given by (xx) a + (1 axt) which is admissible for Form 3. by taking To t hence, Form 3. is necessarily sub-optimal.
As conclusion, we have obtained the following expressions for 4:
If ag > 2r, we have ((x) = (x-t),
2. If au < ar S Sx, then given by t2
(xx) age (=t 2 2)
+
2x. 2 ar( -2- ov
if 5 2
o
aq au
(xx) (x-(x)) t) if >
a.
3. If Ox < ta then (xx) 2-tar x. +
ta 24
Note that in any case, t is in fact continuously differentiable and verifies almost everywhere H < d² as well as < Φ hence it is a solution of the problem
[ (3())ds 2 (0)
inf 020 (()== a, D5
31

---

Published as a conference paper at ICLR 2025
The optimal @ is linked the optimal tempering scheme À as A 2--0z av
with as 0g log( +b)
Hence obtain
X min (1, (a, (a 2)
ap(app +1
C.3 PROOF OF PROPOSITION 7
Reversing the integration by parts we have
Gr(A)= = / exp (-2 of dodd)ds = S/¹ 2 exp (-2 (-2
ff aodu)ds.
Hence
G((A) = 1 o exp
-2
Opdv)ds
1
/ exp
-
(- - av) + ou)du)
t
1
/
O5 av 1
8 J
exp
] (t² - =²²)= - -2(t - s)au ds
2
1
/ exp
Ov a0 az s2 20,s t1ar + a+) )s)
7
Recall that for generic a, b, 0, we have
f exp((aui + be - c)
vi
exp(b> /4u - erf( ()((2()) + constant.
We may apply this formula since
ap. it yields the equality claim:
1 V exp
to
) )
Vo (Vav
at Viay - -erf erf (- Viax
Gi(A) V/2//2 Or
aty 0g
Va as
Now, let consider the behavior of (. as 1. + co. Here, we Taylor expand to obtain
Vtay
)
Viag
erf
erf
V au
Cq
Vau
axp
a, a+
to
to
cxp
exp
tπ
ax
an
ta
ta
=
+
exp
exp
13/2
a₂
a
a₂ at7
Therefore
(ta, a 2z
+ +0 1
(ta az
GA(A)
exp
exp
200
21a,
Ory 0x
O2 az
This yields the claimed asymptotic result.
These results are numerically validated in Figure
D LOWER BOUNDS
In this section, we give the proofs of Theorems 4 bound in total variation for the setup we consider in Theorems g and 9 We apply this result to the bimodal example in section D.2 We then study the unimodal examples from Theorems 4 and in section D33 The bound on the log-Sobolev constant for the uni-modal example is deferred to
8 and 9 In section D.I we prove general lower
section D.4 Throughout,
the notation introduced in section 5
32

---

Published as a conference paper at ICLR 2025
15
10
0.0
Nb. of iterations
Predicted rate (linear schedule)
Tempered ULA (Linear schedule)
Figure 3: Numerical validation of the rate of convergence predicted by Proposition 7 Dashed lines prediction from Proposition aand and full lines are from simulations of the process using 10 000 particles. The proposal and target are two-dimensional Gaussians, with zero mean and covariance matrices that have constant diagonal, equal one for the proposal and 10 for the target. As expected, the predicted rate from Proposition 7 (in dashed red) matches the approximated upper bound from Theorem (in dashed yellow) well as particle- baseed simulations (full lines), for large values of time.
D.1 A GENERAL LOWER BOUND
In this section state and prove the following general lower bound for the geometric path; in the following sections it is applied to obtain Theorems 8
and
Theorem 20 (Total variation lower bounds for the geometric path) Suppose v, x E Pac (R) have
lig-cipsc densities and there is interval [a and X [0, 1J. B > 0 such that:
1. Bounded scores on 1: both [V((=),, [V) B for x E I.
2. Small mass on I: for all X [0, A=], have (x(T)= $ 5.
Then for each k E [K] such that Av Aa.
TV T) 2 ([b, 00)) =(1) - ((0)))((bs)) o0), -
B b-a
Vo
(E m) (1)
This result will be typically be applied in conjuction with the following lemma, which allows us to control the x² terms arising above when we have pointwise comparisons of one distribution to another.
Lemma 21 Suppose that there exists a C C>C such that in the support n, v(x) S CT (x)). Then
2268 ux) S ad -.
Proof of Lemma 21 Take E [K]. Write
(pt)2 -1 1= I
(po)²
x² m4))
14k-1 1.
14
Mk-1 14
Now, letting = Cx for each E (....., ..k...,,,, kJ, we have
pk-1
Ck-1
A du m de -d < -d--Au--
Ck-1
14 Ck
Ck
Hence
(p/)2
(p m4)
Ck-1
Cb
Pk-1
(p-11)
122
Ck-1
1.
Cb
14-1
33

---

Published as a conference paper at ICLR 2025
But since (- (k-1) is non-increasing under Langevin dynamies towards Mk-1- find that
x (pb. pk) S Crd-dde-
Cx-1
/
(Po
1
Ck
pk-1
Proceeding recursively, we obtain
(p) mbs) 1 A
CD
A
cu
1 S CA*
Ck
Ck
where at the last step we use the fact that for all E [0, 1]. Jensen's inequality implies that
1 =
/ -2
(2) S.
Proof. For convenience, let us put
(x(x) == log
A [0,
Define
x <
(xx) = b-a
(x
E 1
b.
We write
x(V) - p4 (v) = =(1))= v(()) + Σ
P PT._, (v) - (v).
1=1
Observe
M(²) 2 mlb, o0)).
On the other hand,
v(v) s v((a, v(I) v(bb, 20)) S 5 v(b,
We thus find
(1)) P7, (vb) 2 ob) (((b, - 8 8
i-1 PT.
(xb) pr. (1b)
To reduce to a lower bound in total variation, we note that
(ه) p4. (+) ((aa, p (lb, 00)) S TV(pf 1) (I)
We finally obtain the lower bound
TV +) 2 (bb, oo)) - x()) - ((1,,00) + Σ P (1b) (1.)
(v))
(57)
To conclude from Eq. 57 we study the above summands. Notice that by definition, PT;-1 (1)) pb()), so that these terms are the difference of the expectation of sb at the start and end of Langevin dynamics run towards μ4, Hence,
P (v) - S/a Th
A- PI
P-_1(w) (v)
(w. pt) 1e
Now, pf evolves by the weighted Fokker-Planck operator
d.ri L A (VVr,, Vpt):
34

---

Published as a conference paper at ICLR 2025
so by self-adjointness of Lye. obtain
% Th
('mh (w) LumPt) rawm)
0, p.)
Lo Th
(C w.Pl) Le
Notice that since vi is linear where it is non-constant, have
[- V2) IEI
(p))()) - (VV. (z), Vu(())
0
x x&I. I.
Hence
(Cpc 1. (l) p()) = b { Vr (x)
-1
$ a [ (=()d)u(())
B
Now, we use Cauchy-Schii Schwarz to observe that
[ p. (z)ddun))) S (u(1 ( 2/2
(x2 Ma) + 1)
s vol mx) 1)1/2
volx (wh mx) 1) 1)1
where the step follows because x²(,, (1x)) non-increasing along Langevin dynamics towards Mk. Hence
- (v) =- --" (Cne (Cm.v.pt)u p()
BT. Vo(x (p) MK) + 1)2/2.
PT. k-1 (w)
Plugging this inequality into Eq. 57 yields the result.
D.2 LOWER BOUNDS FOR BIMODAL TARGET
In this section, we apply the general lower bound from Theorem 20 to prove Theorem g and
cordingly throughout take v and π to be as defined there. We begin by collecting some useful facts about the geometric path between and n; the proof is included after that of Theorem g
Proposition 22 (Useful facts about the bi-modal example.) Let u, be as in Theorem for 2 11. Then
1. for all E [0, 1). the normalizing constant cx [1,2].
2. for each [0,
(1lm/4,33///m S 6e -33
3. for all k [K] and E [0, Th).
x2 (pt px) < 1.
Proof of Theorem 8 We will apply Theorem 20 with / - [m/ 3m/ /4, An easy calculation shows
that we may take 2m for the first assumption, and Proposition 22 shows that d = and
8 = 6e -n? ==//32 suffice for the second assumption. We thus obtain
TV T) 2 ((3m44 oo)) -133m/4 oc)) 12c m² /32 /32 4V6
-12
Σ
Tixx +1)11/22) m n/4
35

---

Published as a conference paper at ICLR 2025
Applying Proposition 22 control the x² terms yields
TV m) 2 π ((3m/4, v(33m/4 oc)) 12e 1-e--mmmm mm2 16
Ti
mm2/32
Note that by Mill's ratio, we can bound using m 2 that
v(33/44, 00))
9m /32 S e
mmmmm m2/33/
On the other hand,
x [(3mm44,00)
7 7(mm/2 o0)) T (1m// 0, 3m/ 4))
2 T (1m/2 2 oc) - I 7(m/4 3m/4]
1
x ([m/ 4,3m/44)
2
1
6e m /32
2
where at the last step we applied Proposition 22 one more time. Plugging in terms yields
TV
19e
/3 19mmm
16
T e- //2
1=1
Now, we use our assumption that m 2 11 to observe that
19c m2 //2
20
and finally conclude.
Proof of Proposition 22 Proof of Fact any X E [0, 1].
Note that Cauchy-Schwarz immediately implies that for
= / 1ーー 1.
On the other hand, using the fact that 2 2, we have that
/ "22222~~
cA
The claim about the normalizing constant follows.
Proof of Fact 2. We use the previous part to observe
3m/4
vm(mm/433//44 < 2
v (x) (x)dr
m/4
< 2v [m/4, 3m -
T[m/44 3m/4])^
2v( 2m(m/4 4, 3m/4)).
where the last equality follows by symmetry. We then observe that by Mills' ratio,
3m/ 4)) v(mm/4, oc)) S 3e 3m-mm// /3/
The result follows.
Proof of Fact The result follows immediately from Lemma 21 once we observe that v S 2T.
D.3 LOWER ROUNDS FOR UNIMODAL TARGET
In this section, we prove Theorem as well as Theorem distributions with different mixture weights, this section we generalize slightly consider, for some a E (V2log2// 2/oggmm m. 1), the target
9
Since these results concern the same
2m 2 (2) -ee d'm" mm2 2/ Um:
=
(58)
By taking a = 1/V2, we recover the setting of Theorem 4 and by taking a = /2 2og recover the setting of Theorem Note that when dealing with various numerical constants, will frequently use the assumption m 2 10 without comment. For both results, the following facts will be useful.
2/m we
36

---

Published as a conference paper at ICLR 2025
Proposition 23 (Useful facts about the unimodal target) Let (0, 1) and ıT be as in Eq. 58 for m 2 10. Then
1. For all E [0, 1].
pa(1m(1 - a)/2 m(1 a)]) I 5am cxe -daammammm /22 a)) mm/m//
2. 1f a 2 then for all X E [0, 1].
exe exe-Au-m²p 22
(l(l-oom - a)/2)) 2
10m
3. For all X such that 2X 2/m, we have
px(mm(1 - a))2, 2 22 - A11--1)
4. For all X E [0,
Alemmaam2/22 /2 + - 21(2-))m²) S Sax S 10me 1n
2m²m=??
We also use the following.
Lemma 24 (Log-Sobolec constants of the unimodal target.) Ler π be as in Eq. 58 for m 2 10. Then
8
CLS(TT) S 1- 2e aa- m²
with the convention that the right-hand side is 324m² when = /2log2/m
The proof of Lemma 24 requires a digression into log-Sobole inequalities for mixtures and restric- tions, so is deferred the next section, Appendix D.4
We first prove Theorems and then return to proving Proposition 23
Proof of Theorem 4 The fact that CLS(P) = is the Gaussian log-Soboles inequality, and the bound (L.(mT= S 6m follows immediately from Lemma 24 upon plugging in a = 1/v2.
For the lower bound Cr(ux), take À [22 1] and let / [m(1 a)/2, m(1 a)]; recall that a E [0, 1) is a free parameter controlling the mixture weights in Eq. 58
Put
x < m(1 a)/2,
(x)) =
(1-a)m (x m(1 - a)/2)
x xEl T
m(1 a).
Then
4
V (((x)))))) = m²²1 a)² M4 (T( (T).
On the other hand,
Var,u (w) - (1))
2 21(000 m1 a)/2) - m1)o + +)((T))
=ma((-00, ())
a)/2)))
-24x(41)) a)/2)) (1)
2 2 a)) 1a0i-aaa(oo00))0 a)/21)) - 31x
=x(((000 m(1 a) (1m1)) a)/2, 31x
37

---

Published as a conference paper at ICLR 2025
We obtain
Var (v)
Cp(ux) 2
| 1 (HAx)
m² a) (x(T)
(1x( - a2), 3m² a)²
(59)
4
4
We'll now apply Proposition 23 with = 1/v2. With this choice of a and for E [2. 1]. the
sufficient conditions are all verified, so that
(
m(1 -ll0mm/)))))((((((((() htt/://m1100///00000 00))
100am a (1 A)m (1-a)
Applying Proposition
23 once more, this time to control cx we obtain
(x( ~>((~>>((0)))) m(1 - a))2)))) a))2, 2 400am3
(e- -o00mmamm=p
1 3(11-))nn AA)m2)
To complete the proof of Theorem 4 we finally plug in a = 1/v2, and observe that for A E 1
holds that e ->a" m² /2 < A(1 Njm²,, so that
o))2 px px((-00, m(1 (L010)"\((ommmmmo0/000_010 (m21))200
2
1
e (1-A)m²
800m
Plugging into Eq. 59 we find
Cp((x) 2 m² - 1/V2)2
(1-A)m² A)m²/100 -3) 3).
4
800m
Observe that 1
12 - - 1//2/2 < 1, so that
3m2
Cr(px) 48 800m
(1-A)m²1100
2
e2 (1-x)m²/1000 mm..
4
4 10
Proof of Theorem 9 The equality Cv.((v) 1 is the Gaussian log-Sobolev inequality. For the log- Sobolev constant of æ. the result is immediate from Lemma 24 upon plugging in /2 2/m.
For the lower bound total variation, we will apply Theorem 20 using the interval = [m(1 a)/2, m(1 a)]: recall that E [0, 1) is a free parameter controlling the mixture weights in Eq. 58 that we will eventually set to a = /2 log 2/m.
In this case, it clear that we may bound the scores with B m. The main issue is then to get control on μx (I), and for this we apply Proposition 23 to obtain
(x < 50am e -(1-1)(1-0))
So for X, : Ag. we may take
= 50am 2 -(1-(_-)))
Theorem 20 then implies
TV x) 2 - =)) -a)/2, m(1 -n(1-aa)) a))) v[mm[ a), 20)) - 8
I-a Vō- Tr62 mi) + 1)112).
2
Next, we observe that 4m 2 v(x) for all ir E R, so that
1
>
Um (x) 2 v(x).
2
8m
38

---

Published as a conference paper at ICLR 2025
Hence may Lemma 21 with C 8m to obtain
TV x) 2 x[m(1 a), 00)) T ((m(1 a2/2, m(1 (0(1-aa)) -a]l v[[m[ o0)
8/m 1
k
vo. ΣΣ Ti
Let us specialize to /2 2/m. In this case, Mills' ratio implies v([m(1 a), oo)) S e (1-a)mmm
2m
For the other two terms, let us adopt the notation Im =
1 e 2 mm) and then write
2
([m(1 - a), o0)) - π [[m(11 2)/2.m(( mmm1)
2m
( (2)((()) Um (x))dx
ml(1-a) 1 (2mm(()) 2²th um (xm(()) (2))
m1)-a)
m(1-a)/2
1
2m
(m(l-a)
1 yam
f um un(v)do
gm(x)dd
Im Ax)dr +
2
m(1-a)
m(l1))
1
2m
1
(((1a)
9m
Im (x)dr
2
m(1-a)
m(1 aa/2
mm(+aa)
m+1
gm(x)dx
2
(m(2) 2
m(1-a)
mm-1
We finally obtain
TV T)
1
22m2/5 6 1-10m Vo- T
For this choice of a, note that
8 < 3 -1 (1-Ae)m²? )mm 1/00
By the restriction on m we know 1/4 - e 2m2 15 2 1 1/5, yielding the result.
Proof of Prop. 23
For case of notation, let us write 9m = -e ((--m)? Also, put (z)
/2*
Cun e Im)) for Cam normalizing constant and observe that
e dm). dx = 3m + 2nT =
1
Cam
< Cum
4m
3m
Therefore,
m=2m²12
m² Um (x) 5
gm(())
Yor E [m(1 a), ma(1 a)),
(60)
3m
m
and
Jm (m( S E -22m²/2 < 4me 4m'e-a'm m222²22
Um (x)
E [- a)] U [m(1 1 a), 2m).
(61)
Proof of Fact
We use Eq. 6l observe that
#x(mm(1 (000(110//2nmg)) - a)]) =
y u
mm(1-))
m a))
S (5mm) c >a'm²
-
m(1 an(1-a)/2
I 5am2 - >03 /2 (1-X)m²(1-00)
39

---

Published as a conference paper at ICLR 2025
Proof of Fact 2. We compute
(m(1-a)/2
mall-o0,m(1-00 aa)/1]
cxv-->a a dx
-00
m o0//
2¹ 7 dx
C2
mm(1
22
d'm² 12
1-1 pm(1-a)/2
4m
1T
m(1-a)/2
Aa2m²12
mm
a3/2
2 (1
C2
e
4v2mm
ml)12//2
2a²m²/2
mLL-a)/22
e -1==a
dx
4V22m
-m(1-a)/2
222m²/2
e 2= dx
CA
4/2mm
2e
222
Cx
4
Aa2m?/2
CxC
10m
Proof of Fact 3. We calculate,
f
100
[[[m(1-a)/2,0)
crel -xel-A A dr
mm(1-a)/2
6) 2m
211-11) }((x- dx
ス
m(1-o)/2
2T
2) -)(111)m 2
2m
1 (a--Am) dx
m(1 n) V2T
where we used the fact that under the hypothesis, we have m(1 a)/2 & Am 1. Proof of Fact 4. We compute
1
20
(=-aum222 12 x
m2/
21
CX
(2 (1-)22
4m
m//
e
222
m/2
2 -l dr
4 22mm
-mm/2
Am?mn? /2
m/2
3/2² dx
4m/m
J-m/2
2c -1/2 -1/2c->an?m mm?mm??
20rm
e Xa?mn /2
10m
40

---

Published as a conference paper at ICLR 2025
On the other hand, we use Eq. 60000dd and Eq. 61 to bound
1
CA
m a)
m(l+a)
2m
v u
ul-dpp
m(1-a)
mil+a)
m
x (m[[+a)
2m
m a)
5 2
1² -= n +
T
m
m((-o)
2m
A
m((l-a) e =2(-1 dx
< 2
3
(2T)(1-A)/2
-m
2
mm(1+a)
per v2-1
00
=1(1- - m)² dx
1²-A +
e
21T
mm(1-a
2m
$ 4e Aa-m2//2
2 AA(1-A)m²
c (m=-2m)2 dr
2
c- -2²
dx
2n
2
< 4e Aa'm m2 + 2e 1(1mA)mm t 2m²
I 4(e- da'm² + 1111X\m²²
D.4 UPPER BOUNDS ON THE LOG-SOBOLEV CONSTANT OF THE UNIMODAL TARGET
In this section prove Lemma 24 on the log-Sobolev constant of the unimodal target defined in Eq. 17 We use the following result which controls the log-Sobolev constants of mixtures, and appears (Schlichting, 2019, Corollary 2).
Theorem 25 (Upper bounds log-Sobolev constant of mixtures (Schlichting 2019) Suppose Qo Q1 E (T(R)) are such that Qo & Q1. For p E [0, consider the mixture Qp pQo + - (1-p)Q Then
CLs(Op) S max (11+11- Pppp)) (1 p1p(1 + (2o (1)))cs((1))
for
log p- pogg-1lo(1-m)
p 0. 5 u
Ap
2p-1
2
p
With this result in hand, we just need to get control on the the log-Soboler constant of um- This is accomplished in the following lemma. Lemma 26 (Log-Sobolev constant of Wm) We have
CLs(Um) 16m
Proof of Lemma 26 We will apply the Holley-Stroock perturbation principle to a custom compari- distribution. Indeed, for a parameter 0 be chosen later, let
[x= 2m)²
2m +
m (x) = ar (a
2 + 8r
a E [-m m 2m + r]
(x + m)
< r,
where we take a,,, 8, to make @ continuously differentiable; in particular
3mr
6, = 8
2r + 3m
Note that O, is a, strongly convex, so that CLS(E (0)(=-())))) +1)+ a.
1.
On the other hand
3m
8,
2 2 < or
d(x, Imm ) 2= carl
2 + r) + 8,
2
41

---

Published as a conference paper at ICLR 2025
Hence
1 osc( d( _d(( (m) or
3m
,2
Hence by the Holley-Stroock perturbation principle Holley and Stroock (1987) we know that
3m
r²
1
3m 2
r) 2 + r²
CLs(Um) I exp (ar( + +r)² CLs(E (o((e-Tr) = ar Taking 1/m, we find
exp (a.
(2m)2 + 2m²) = (2 + 3m exp ( +
1
CLs(Um) - (2 + 3m²) exp exp
2m I 16m²
3m
With the result in hand, we can prove Lemma
Proof of Lemma 24 We first compute (2 1), um). For this, let write Um
Crsm (x,dm, and note that
I e j (x,5m) 2 3m & 4m.
Cum
Hence
(NV um) +1 (x-m))+ + dx
2m
00
um)
2m
e (x-mm) +3 +
2m
2m
(ddem))++ dx
-00
π
2m
xcc
é
(8- ((-m))*+
dx
π
2m
2m
m
2m 2m
(o-m dx
dx +
-00
mm
2m
/c
dx
2m
2m
m
c {(= dx + 2m f e (x-mm) dr
2m
noc
(2-m)) dr
π
2m
2m 5
(V2T + V + 2 < 5m.
Using Lemma 26 and plugging into Theorem 25 with = 1 aa?m212 we find
CLs(R) S max { (1 Ap 80m² + pdp)} S 162m Ap
The result follows.
E ADDITIONAL NUMERICAL ILLUSTRATIONS
The geometric path is often illustrated in setup where the initialization is chosen in the middle of a two symmetric modes: see for example Cabezas et al. (2023) Sampling Book, Tempered SMC) or Maurais and Marzouk (2024 Fig. 1) or Chehab al. (2024 Fig. 1). In this very specific setting, the geometric path conveys sense that it evolves particle positions. In more general setting, we can observe in Figure 4 that once the closer mode is reached, the path seems evolve the particle
weights, which is problematic for Langevin dynamics.
42

---

Published as a conference paper at ICLR 2025
0.006
-2
-2
2
2
-2
Iter:: 0
Iter:: 4000
Iter:: 9000
Iter:: 9860
Iter.:: 9999
20
-20
-20
20
-20
20
-20
20
-20
20
-20
20
Iter. 0
Iter. 4000
Iter. 9000
Iter. 9990
Iter. 9999
-100
-100
-100
-100
-100
0 -100
Figure 4: Geometric path from a Gaussian to a Gaussian mixture. We observe that that this path displaces mass "horizontally" to the nearest modes (left columns), and then "vertically" to the re- maining modes (right columns). Intuitively, this second part problematic for Langevin sampler.
43


Reviews:
[{"review_id": "qjMFBrHHjS", "reviewer": "Reviewer_Exbp", "summary": "This work studies the convergence guarantee of geometric tempering for the Langevin diffusion and its time-discretization the Langevin algorithm. The authors prove a convergence rate under a general tempering schedule, demonstrating dependency on the isoperimetry of the intermediate probability measures, in particular their log-Sobolev constant. While this constant can be suitably controlled when both measures are strongly log-concave, the authors show that even when both proposal and target densities are unimodal, intermediate measures can suffer from a poor log-Sobolev constant that scales exponentially with the distance between the modes of proposal and target measures.", "questions": "* I believe for $KL(p_0, \\mu_0)$ to disappear in Corollary 5, one needs to set $\\lambda_0 = 0$. In that case, it would not be possible to choose $\\lambda_t = 1$ for all $t > 0$ in a continuous manner.\n\n* If all $\\lambda_i$s are very close to 1 in Theorem 9, we are effectively running vanilla Langevin. In that case, why should we have exponential convergence time?\n\n* The vanilla Langevin analysis only requires the log-Sobolev inequality and smoothness for discretization. Why do we additionally need dissipativity of proposal and target measures here?\n\n    * In fact, the Langevin algorithm is known to convergence under extremely mild conditions, namely a weak Poincaré inequality (which holds for all locally bounded potentials, although without explicit control on the constant) and smoothness of the gradients, see e.g. [1] and references therein. Are there major challenges for obtaining convergence guarantees under (weak) Poincaré inequalities for the tempered Langevin algorithm? \n\n* Is there a sense in which one can choose optimal proposal distributions $\\nu$ when we only know some information about $\\pi$?\n\n* I believe a summation over $i$ is missing in Equation (12).\n\n* Some typos:\n    * Line 152 missing absolute continuity before “... and $+\\infty$ otherwise”.\n    * Line 175: potential -> potentials\n    * Line 119, 233, 253, 467: missing parentheses in citation\n    * Line 238: satisfy -> satisfies\n    * Line 336, 339, 383: section … -> Section …\n    * Line 420: are unknown -> is unknown\n    * Line 425: where we obtain\n    * A typo in Line 439 makes the sentence unreadable.\n\n\n\n\n---\n[1] A. Mousavi-Hosseini, T. Farghly, Y. He, K. Balasubramanian, M. A. Erdogdu. \"Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincaré Inequality\". COLT 2023.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "While I am not an expert in annealing or tempering algorithms for sampling, it seems that this is the first paper that proves the convergence of geometric tempering for the Langevin diffusion using functional inequalities, which is interesting. The negative results also provide a good example of why tempering may not work in practice, despite both proposal and target measures having suitable isoperimetry.", "weaknesses": "* The related work section could be better structured. For example, breaking into multiple paragraphs and adding paragraph titles could help with readability and following the discussion.\n* The lower bound examples hold in dimension 1, and show exponentially bad dependence on the distance between modes. While these bounds are interesting, it is not very intuitive to me why it would be natural for the distance between modes to grow in fixed dimensions. On the other hand, in a high-dimensional setting, it is more intuitive that $m$ grows with square root of dimension. Could it be straightforward to (perhaps only intuitively) extend the lower bounds to high-dimensional settings?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XIEjAH4JNF", "reviewer": "Reviewer_8HQm", "summary": "This paper presents a theoretical analysis of geometric tempering when applied to Langevin dynamics, a popular sampling method in machine learning and statistics. Geometric tempering is a technique that attempts to improve sampling from complex multi-modal distributions by sampling from a sequence of intermediate distributions that interpolate between an easy-to-sample proposal distribution and the target distribution. The authors provide the first convergence analysis under functional inequalities, proving both upper and lower bounds for tempered Langevin dynamics in continuous and discrete time. They also derive optimal tempering schedules for certain pairs of proposal and target distributions.", "questions": "Please see question abobe", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Perhaps surprisingly, the paper's findings are largely negative regarding the effectiveness of geometric tempering. The authors demonstrate that geometric tempering can actually worsen functional inequalities exponentially, even when both the proposal and target distributions have favorable properties. Through theoretical analysis, they show a simple bimodal case where geometric tempering takes exponential time to converge. More strikingly, they prove that similar poor convergence results can occur even with unimodal target distributions that have good functional inequalities. These results suggest that geometric tempering may not only fail to help with convergence but could actually be harmful in some cases, challenging the conventional wisdom about its utility.", "weaknesses": "In this paper they consider targets of the form $\\nu^{1 - \\lambda} \\pi^{\\lambda}$, where $\\nu$ is called the proposal. In many other prior works, the targets are of the form $\\pi^{\\lambda}$, which corresponds to $\\nu$ being an improper uniform distribution. This seems to be the main source of the largely negative results provided in this paper. Could the authors clarify the reason for considering target the above form?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KeaMzd75Lk", "reviewer": "Reviewer_pomC", "summary": "This paper offers a thorough study of geometric tempering combined with a Langevin MCMC scheme. In particular, a general theory is given which characterizes the error induced by said dynamics for arbitrary tempering schemes. Negative results are then given for the efficacy of tempering schemes (over the naive Langevin dynamics) both in terms of the intermediate distributions' log-Sobolev constants, as well as the worst-case convergence rate, although some regimes where the tempering is beneficial are highlighted.", "questions": "The following suggestions relate to minor areas of the paper:\n\nIn Figure 2, should we not be scaling the $y$-axis logarithmically for a more reasonable demonstration?\n\nThe comment after (3) is strange. Probably, you mean to take $kh = t$ for a fixed choice of $t \\in \\mathbb R$, and then some schedule $h = t/K$ for a set of integers $K$, rather than what is written.\n\nThere is a spacing issue in Line 190~191.\n\nLine 240: Lebesgue -> Lebesgue measure.\n\nLine 425: where obtain-> where we obtain\n\nIt is a bit strange to cite Durmus 2019 for the Langevin rate in the str. convex + smooth setting, compared to earlier work such as [1].\n\nDurmus, Alain, and Eric Moulines. \"High-dimensional Bayesian inference via the unadjusted Langevin algorithm.\" (2019): 2854-2882.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The main positive result on geometric tempering (Theorem 3) seems quite thorough (in that it comprises every reasonable regime of interest) and about as good as one could hope for in this context.\n\nThe negative example is very intuitive and is a worthwhile inclusion into the paper. It offers a good characterization about why one might be skeptical about the occasional poor performance of these schemes in practice, and gives good intuition about the heart of the problem (the appearance of multimodality).\n\nThe inclusion of more concrete lower bounds is also insightful.", "weaknesses": "It would be more helpful if the paper offered more positive examples of instances where the tempering can improve over vanilla Langevin by at least polynomial factors; in particular, a comparative bound would be helpful in Propositions 6, 7.\n\nIt would also be good if the paper could explore further the areas where tempering has a provable benefit over Langevin, especially in cases of multimodality where the algorithm would likely be used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F4NLY75sF9", "reviewer": "Reviewer_mTXb", "summary": "This work analyzes the convergence rate of Langevin dynamics with geometric tempering (LD-GT), a modification of the Langevin dynamics which attempts to follow the geometric path between a proposal distribution $\\nu$ (e.g. a standard Gaussian) and the target distribution $\\pi$. \nMore precisely, LD-GT with tempering schedule $(\\lambda_k)_k \\subset [0,1]$ is\n\n$$X_0 \\sim \\nu, ~~~~ X_{k+1} = X_k + h \\nabla \\log \\mu_k(X_k) + \\sqrt{2 h} \\epsilon_k,$$\n\nwhere $\\mu_k \\propto \\nu^{1-\\lambda_k} \\pi^{\\lambda_k}$ is the geometric path (and $\\epsilon_k$ are independent Gaussians).\n\n(Per the authors' account of the literature,) LD-GT was proposed since the 1990s, and one of the motivations is the intuition that sampling progressively from the path $\\mu_k$ is easier than sampling directly from the target $\\pi$, especially if $\\pi$ is multi-modal. \n\nThis work's contributions are two-fold:\n- Precise convergence guarantees for LD-GT under certain common assumptions on the proposal $\\nu$ and the target $\\pi$: Poincare inequality (PI), log-Sobolev inequality (LSI), strong log-concavity.\\\nTo this aim, a key sub-question is to estimate the PI or LSI constant of the path $(\\mu_k)_k$. Besides showing how to best utilize well-known upper bounds, the authors also identify cases where these constants are surprisingly poor, leading to the next item.\n- This work provides evidence that the original intuition motivating LD-GT is wrong, by exhibiting cases where LD-GT must converge very slowly (regardless of the choice of schedule). Remarkably this can even happen for well-conditioned and uni-modal targets $\\pi$, for which vanilla LD can be expected to converge fast.", "questions": "The theoretical results in this work suggest that geometric tempering may not help the convergence of Langevin dynamics. Yet tempering is a strategy that is used in practice (per the authors' presentation of the literature). In practice, is tempering observed to lead to improved performance compared to vanilla Langevin dynamics? If yes, is there any intuitive reason why?\n\nMinor questions:\n- Would the conclusions, and the analysis techniques, of this paper still apply if one takes $\\nu$ to be the Lebesgue measure instead of a probability measure?\n- In practice, is the proposal $\\nu$ always taken to be a Gaussian? Is it sometimes taken to be the Lebesgue measure? Or multi-modal?\n- Is the $\\frac{1}{2\\alpha_\\pi t}$ rate in Proposition 7 (line 433) classical? If so please give a reference.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper addresses a natural question on the convergence of a sampling algorithm. The positive results (first item in \"Summary\") are of theoretical interest, as they address the technically difficult question of optimizing the upper bounds w.r.t the temperature schedule. The negative findings are surprising: namely, adding geometric tempering may actually slow down Langevin dynamics. This new insight is significant for both theory and practice.\n\nThe presentation is very clear and \"flows\" very nicely. All the technical claims are correct are far as I checked.", "weaknesses": "No substantial weaknesses, but the negative results of this paper naturally lead to a question which is not addressed nor mentioned in this paper, see \"Questions\" below.\n\nMinor comments on the presentation:\n- use citep instead of citet on lines 119, 233, 254, 468\n- correct typos and/or grammar on lines 271, 326, 420, 496, 1389, 1494, 1838\n- justify the fact that chi^2, KL > TV rather than say it \"of course\" holds (line 493)\n- line 988 contains the proof of Corollary 13, not 17\n- add details on the argument on line 1824 (I could not reconstruct it using Cauchy-Schwarz, only Jensen)\n- use different markers for each curve in Figure 2\n- consider using a log scale or showing less iterations in Figure 3\n- consider including the example of section 4.1 in Figure 4, in addition to Figure 1 (which shows only $\\lambda \\in \\{0, 0.45, 1\\}$)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)