Title: On the Mode-Seeking Properties of Langevin Dynamics
Venue: NeurIPS 2024
Accepted: no
Keywords: Langevin dynamics

convergence analysis

mixture distribution

mode-seeking
Abstract: The Langevin Dynamics framework, which aims to generate samples from the score function of a probability distribution, is widely used for analyzing and interpreting score-based generative modeling. While the convergence behavior of Langevin Dynamics under unimodal distributions has been extensively studied in the literature, in practice the data distribution could consist of multiple distinct modes. In this work, we investigate Langevin Dynamics in producing samples from multimodal distributions and theoretically study its mode-seeking properties. We prove that under a variety of sub-Gaussian mixtures, Langevin Dynamics is unlikely to find all mixture components within a sub-exponential number of steps in the data dimension. To reduce the mode-seeking tendencies of Langevin Dynamics, we propose \emph{Chained Langevin Dynamics}, which divides the data vector into patches of constant size and generates every patch sequentially conditioned on the previous patches. We perform a theoretical analysis of Chained Langevin Dynamics by reducing it to sampling from a constant-dimensional distribution. We present the results of several numerical experiments on synthetic and real image datasets, supporting our theoretical results on the iteration complexities of sample generation from mixture distributions using the chained and vanilla Langevin Dynamics.

Abstract

The Langevin Dynamics framework, which aims to generate samples from the score function of a probability distribution, is widely used for analyzing and interpreting score-based generative modeling. While the convergence behavior of Langevin Dynamics under unimodal distributions has been extensively studied in the literature, in practice the data distribution could consist of multiple distinct modes. In this work, we investigate Langevin Dynamics in producing samples from multimodal distributions and theoretically study its mode-seeking properties. We prove that under a variety of sub-Gaussian mixtures, Langevin Dynamics is unlikely to find all mixture components within a sub-exponential number of steps in the data dimension. To reduce the mode-seeking tendencies of Langevin Dynamics, we propose Chained Langevin Dynamics , which divides the data vector into patches of constant size and generates every patch sequentially conditioned on the previous patches. We perform a theoretical analysis of Chained Langevin Dynamics by reducing it to sampling from a constant-dimensional distribution. We present the results of several numerical experiments on synthetic and real image datasets, supporting our theoretical results on the iteration complexities of sample generation from mixture distributions using the chained and vanilla Langevin Dynamics.

1 Introduction

A central task in unsupervised learning involves learning the underlying probability distribution of training data and efficiently generating new samples from the distribution. Score-based generative modeling (SGM) (Song et al., 2020c) has achieved state-of-the-art performance in various learning tasks including image generation (Song and Ermon, 2019, 2020; Ho et al., 2020; Song et al., 2020a; Ramesh et al., 2022; Rombach et al., 2022), audio synthesis (Chen et al., 2020; Kong et al., 2020), and video generation (Ho et al., 2022; Blattmann et al., 2023). In addition to the successful empirical results, the convergence analysis of SGM has attracted significant attention in the recent literature (Lee et al., 2022, 2023; Chen et al., 2023; Li et al., 2023, 2024).

Stochastic gradient Langevin dynamics (SGLD) (Welling and Teh, 2011), as a fundamental method- ology to implement and interpret SGM, can produce samples from the (Stein) score function of a

probability density, i.e., the gradient of the log probability density function with respect to data. It has been widely recognized that a pitfall of SGLD is its slow mixing rate (Wooddard et al., 2009;

Raginsky et al., 2017; Lee et al., 2018). Specifically, Song and Ermon (2019) shows that under a multi-modal data distribution, the samples from Langevin dynamics may have an incorrect relative

density across the modes. Based on this finding, Song and Ermon (2019) proposes anneal Langevin 33

dynamics , which injects different levels of Gaussian noise into the data distribution and samples with 34

SGLD on the perturbed distribution. While outputting the correct relative density across modes can 35

be challenging for SGLD, a natural question is whether SGLD would be able to find all the modes of 36

37

a multi-modal distribution.

In this work, we study this question by analyzing the mode-seeking properties of SGLD. The notion 38 of mode-seekingness (Bishop, 2006; Ke et al., 2021; Li and Farnia, 2023) refers to the property that a 39 generative model captures only a subset of the modes of a multi-modal distribution. We note that 40 a similar problem, known as metastability, has been studied in the context of Langevin diffusion, 41 a continuous-time version of SGLD described by stochastic differential equation (SDE) (Bovier 42 et al., 2002, 2004; Gayrard et al., 2005). Specifically, Bovier et al. (2002) gave a sharp bound on 43 the mean hitting time of Langevin diffusion and proved that it may require exponential (in the space 44 dimensionality d ) time for transition between modes. Regarding discrete SGLD, Lee et al. (2018) 45 constructed a probability distribution whose density is close to a mixture of two well-separated 46 isotropic Gaussians, and proved that SGLD could not find one of the two modes within an exponential 47 number of steps. However, further exploration of mode-seeking tendencies of SGLD and its variants 48 such as annealed Langevin dynamics for general distributions is still lacking in the literature. 49

50

51

52

53

54

55

56

57

In this work, we theoretically formulate and demonstrate the potential mode-seeking tendency of SGLD. We begin by analyzing the convergence under a variety of Gaussian mixture probability distributions, under which SGLD could fail to visit all the mixture components within sub-exponential steps (in the data dimension). Subsequently, we generalize this result to mixture distributions with sub-Gaussian modes. This generalization extends our earlier result on Gaussian mixtures to a significantly larger family of mixture models, as the sub-Gaussian family includes any distribution over an ℓ 2 -norm-bounded support set. Furthermore, we extend our theoretical results to anneal Langevin dynamics with bounded noise scales.

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

To reduce SGLD's large iteration complexity shown under a high-dimensional input vector, we propose Chained Langevin Dynamics (Chained-LD) . Since SGLD could suffer from the curse of dimensionality, we decompose the sample x ∈ R d into d/Q patches x (1) , · · · , x ( d/Q ) , each of constant size Q , and sequentially generate every patch x ( q ) for all q ∈ [ d/Q ] statistically conditioned on previous patches, i.e., P ( x ( q ) | x (0) , · · · x ( q -1) ) . The combination of all patches generated from the conditional distribution faithfully follows the probability density P ( x ) , while learning each patch requires less cost due to the reduced dimension. We also provide a theoretical analysis of Chained-LD by reducing the convergence of a d -dimensional sample to the convergence of each patch.

Finally, we present the results of several numerical experiments to validate our theoretical findings. For synthetic experiments, we consider moderately high-dimensional Gaussian mixture models, where the vanilla and annealed Langevin dynamics could not find all the components within a million steps, while Chained-LD could capture all the components with correct frequencies in O (10 4 ) steps. For experiments on real image datasets, we consider a mixture of two modes by using the original images from MNIST/Fashion-MNIST training dataset (black background and white digits/objects) as the first mode and constructing the second mode by i.i.d. flipping the images (white background and black digits/objects) with probability 0.5. Following from Song and Ermon (2019), we trained a Noise Conditional Score Network (NCSN) to estimate the score function. Our numerical results indicate that vanilla Langevin dynamics can fail to capture the two modes, as also observed by Song and Ermon (2019). On the other hand, Chained-LD was capable of finding both modes regardless of initialization. We summarize the contributions of this work as follows:

Theoretically studying the mode-seeking properties of vanilla and annealed Langevin dynamics,

Proposing Chained Langevin Dynamics (Chained-LD), which decomposes the sample into patches and sequentially generates each patch conditioned on previous patches,

Providing a theoretical analysis of the convergence behavior of Chained-LD,

Numerically comparing the mode-seeking properties of vanilla, annealed, and chained Langevin dynamics.

Notations: We use [ n ] to denote the set { 1 , 2 , · · · , n } . Also, in the paper, ∥·∥ refers to the ℓ 2 norm. We use 0 n and 1 n to denote a 0-vector and 1-vector of length n . We use I n to denote the identity matrix of size n × n . In the text, TV stands for the total variation distance.

2 Related Works

Langevin Dynamics: The convergence guarantees for Langevin diffusion, a continuous version of 88 Langevin dynamics, are classical results extensively studied in the literature (Bhattacharya, 1978; 89

Roberts and Tweedie, 1996; Bakry and Émery, 1983; Bakry et al., 2008). Langevin dynamics, also 90 known as Langevin Monte Carlo, is a discretization of Langevin diffusion typically modeled as a 91 Markov Chain Monte Carlo (Welling and Teh, 2011). For unimodal distributions, e.g., the probability 92 density function that is log-concave or satisfies log-Sobolev inequality, the convergence of Langevin 93 dynamics is provably fast (Dalalyan, 2017; Durmus and Moulines, 2017; Vempala and Wibisono, 94 2019). However, for multimodal distributions, the non-asymptotic convergence analysis is much more 95 challenging (Cheng et al., 2018). Raginsky et al. (2017) gave an upper bound on the convergence time 96 of Langevin dynamics for arbitrary non-log-concave distributions with certain regularity assumptions, 97 which, however, could be exponentially large without imposing more restrictive assumptions. Lee 98 et al. (2018) studied the special case of a mixture of Gaussians of equal variance and provided 99 heuristic analysis of sampling from general non-log-concave distributions. 100

101

102

103

104

105

106

107

108

109

Mode-Seekingness of Langevin Dynamics: The investigation of the mode-seekingness of generative models starts with different generative adversarial network (GAN) (Goodfellow et al., 2014) model formulations and divergence measures, from both the practical (Goodfellow, 2016; Poole et al., 2016) and theoretical (Shannon et al., 2020; Li and Farnia, 2023) perspectives. In the context of Langevin dynamics, mode-seekingness is closely related to a lower bound on the transition time between two modes, e.g., two local maximums. Bovier et al. (2002, 2004); Gayrard et al. (2005) studied the mean hitting time of the continuous Langevin diffusion. Lee et al. (2018) proved the existence of a mixture of two Gaussian distributions whose covariance matrices differ by a constant factor, Langevin dynamics cannot find both modes in polynomial time.

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

137

Score-based Generative Modeling: Since Song et al. (2020b) proposed sliced score matching which can train deep models to learn the score functions of implicit probability distributions on highdimensional data, score-based generative modeling (SGM) has been going through a spurt of growth. Annealed Langevin dynamics (Song and Ermon, 2019) estimates the noise score of the probability density perturbed by Gaussian noise and utilizes stochastic gradient Langevin dynamics to generate samples from a sequence of decreasing noise scales. Song and Ermon (2020) conducted a heuristic analysis of the effect of noise levels on the performance of annealed Langevin dynamics. Denoising diffusion probabilistic model (DDPM) (Ho et al., 2020) incorporates a step-by-step introduction of random noise into data, followed by learning to reverse this diffusion process in order to generate desired data samples from the noise. Song et al. (2020c) unified anneal Langevin dynamics and DDPM via a stochastic differential equation. A recent line of work focuses on the non-asymptotic convergence guarantees for SGM with an imperfect score estimation under various assumptions on the data distribution (Block et al., 2020; De Bortoli et al., 2021; Lee et al., 2022; Chen et al., 2023; Benton et al., 2023; Li et al., 2023, 2024).

3 Preliminaries

3.1 Langevin Dynamics

Generative modeling aims to produce samples such that their distribution is close to the underlying true distribution P . For a continuously differentiable probability density P ( x ) on R d , its score function is defined as the gradient of the log probability density function (PDF) ∇ x log P ( x ) . Langevin diffusion is a stochastic process defined by the stochastic differential equation (SDE)

where w t is the Wiener process on R d . To generate samples from Langevin diffusion, Welling and Teh (2011) proposed stochastic gradient Langevin dynamics (SGLD), a discretization of the SDE for T iterations. Each iteration of SGLD is defined as

where δ t is the step size and ϵ t ∼ N ( 0 d , I d ) is Gaussian noise. It has been widely recognized 133 that Langevin diffusion could take exponential time to mix without additional assumptions on the 134 probability density (Bovier et al., 2002, 2004; Gayrard et al., 2005; Raginsky et al., 2017; Lee et al., 135 2018). To combat the slow mixing, Song and Ermon (2019) proposed annealed Langevin dynamics 136 2

by perturbing the probability density with Gaussian noise of variance

σ

, i.e.,

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

and running SGLD on the perturbed data distribution P σ t ( x ) with gradually decreasing noise levels { σ t } t ∈ [ T ] , i.e.,

where δ t is the step size and ϵ t ∼ N ( 0 d , I d ) is Gaussian noise. When the noise level σ is vanishingly small, the perturbed distribution is close to the true distribution, i.e., P σ ( x ) ≈ P ( x ) . Since we do not have direct access to the (perturbed) score function, Song and Ermon (2019) proposed the Noise Conditional Score Network (NCSN) s θ ( x , σ ) to jointly estimate the scores of all perturbed data distributions, i.e.,

To train the NCSN, Song and Ermon (2019) adopted denoising score matching, which minimizes the following loss

Assuming the NCSN has enough capacity, s θ ∗ ( x , σ ) minimizes the loss L ( θ ; { σ t } t ∈ [ T ] ) if and only if s θ ∗ ( x , σ t ) = ∇ x log P σ t ( x ) almost surely for all t ∈ [ T ] .

3.2 Multi-Modal Distributions

Our work focuses on multi-modal distributions. We use P = ∑ i ∈ [ k ] w i P ( i ) to represent a mixture of k modes, where each mode P ( i ) is a probability density with frequency w i such that w i > 0 for all i ∈ [ k ] and ∑ i ∈ [ k ] w i = 1 . In our theoretical analysis, we consider Gaussian mixtures and sub-Gaussian mixtures, i.e., every component P ( i ) is a Gaussian or sub-Gaussian distribution. A probability distribution p ( z ) of dimension d is defined as a sub-Gaussian distribution with parameter ν 2 if, given the mean vector µ := E z ∼ p [ z ] , the moment generating function (MGF) of p satisfies the following inequality for every vector α ∈ R d :

We remark that sub-Gaussian distributions include a wide variety of distributions such as Gaussian distributions and any distribution within a bounded ℓ 2 -norm distance from the mean µ . From equation 2 we note that the perturbed distribution is the convolution of the original distribution and a Gaussian random variable, i.e., for random variables z ∼ p and t ∼ N ( 0 d , I d ) , their sum z + t ∼ p σ follows the perturbed distribution with noise level σ . Therefore, a perturbed (sub)Gaussian distribution remains (sub)Gaussian. We formalize this property in Proposition 1 and defer the proof to Appendix A for completeness.

Proposition 1. Suppose the perturbed distribution of a d -dimensional probability distribution p with noise level σ is p σ , then the mean of the perturbed distribution is the same as the original distribution, i.e., E z ∼ p σ [ z ] = E z ∼ p [ z ] . If p = N ( µ , Σ ) is a Gaussian distribution, p σ = N ( µ , Σ + σ 2 I d ) is also a Gaussian distribution. If p is a sub-Gaussian distribution with parameter ν 2 , p σ is a sub-Gaussian distribution with parameter ( ν 2 + σ 2 ) .

4 Theoretical Analysis of the Mode-Seeking Properties of Langevin Dynamics

In this section, we theoretically investigate the mode-seeking properties of vanilla and annealed Langevin dynamics. We begin with analyzing Langevin dynamics in Gaussian mixtures.

4.1 Langevin Dynamics in Gaussian Mixtures

Assumption 1. Consider a data distribution P := ∑ k i =0 w i P ( i ) as a mixture of Gaussian distributions, where 1 ≤ k = o ( d ) and w i > 0 is a positive constant such that ∑ k i =0 w i = 1 . Suppose that P ( i ) = N ( µ i , ν 2 i I d ) is a Gaussian distribution over R d for all i ∈ { 0 } ∪ [ k ] such that for all i ∈ [ k ] , ν i < ν 0 and ∥ µ i -µ 0 ∥ 2 ≤ ν 2 0 -ν 2 i 2 ( log ( ν 2 i ν 2 0 ) -ν 2 i 2 ν 2 0 + ν 2 0 2 ν 2 i ) d . Denote ν max := max i ∈ [ k ] ν i .

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

Regarding the first requirement ν i < ν 0 , we first note that the probability density p ( z ) of a Gaussian distribution N ( µ , ν 2 I d ) decays exponentially in terms of ∥ z -µ ∥ 2 ν 2 . When a state z is sufficiently far from all modes (i.e., ∥ z ∥ ≫ ∥ µ i ∥ ), the Gaussian distribution with the largest variance (i.e., P (0) in Assumption 1) dominates all other modes because ∥ z -µ 0 ∥ 2 ν 2 0 ≈ ∥ z ∥ 2 ν 2 0 ≫ ∥ z ∥ 2 ν 2 i ≈ ∥ z -µ i ∥ 2 ν 2 i . We call such mode P (0) the universal mode . Therefore, if z is initialized far from all modes, it can only converge to the universal mode because the gradient information of other modes is masked. Once z enters the universal mode P (0) , if the step size δ t of Langevin dynamics is small (i.e., δ t ≤ ν 2 0 ), it would take exponential steps to escape the local mode P (0) ; while if the step size is large (i.e., δ t > ν 2 0 ), the state z would again be far from all modes and thus the universal mode P (0) dominates all other modes. Hence, z can only visit the universal mode unless the stochastic noise ϵ t miraculously leads it to the region of another mode. In addition, it can be verified that log ( ν 2 i ν 2 0 ) -ν 2 i 2 ν 2 0 + ν 2 0 2 ν 2 i is a positive constant for ν i < ν 0 , thus the second requirement of Assumption 1 essentially represents ∥ µ i -µ 0 ∥ 2 ≤ O ( d ) . We formalize the intuition in Theorem 1 and defer the proof to Appendix A.1.

Theorem 1. Consider a data distribution P satisfying Assumption 1. We follow Langevin dynamics for T = exp( O ( d )) steps. Suppose the sample is initialized in P (0) , then with probability at least 1 -T · exp( -Ω( d )) , we have ∥ x t -µ i ∥ 2 > ν 2 0 + ν 2 max 2 d for all t ∈ { 0 } ∪ [ T ] and i ∈ [ k ] .

We note that ∥ x t -µ i ∥ 2 > ν 2 0 + ν 2 max 2 d is a strong notion of mode-seekingness, since the probability density of mode P ( i ) = N ( µ i , ν 2 i I d ) concentrates around the ℓ 2 -norm ball { z : ∥ z -µ i ∥ 2 ≤ ν 2 i d } . This notion can also easily be translated into a lower bound in terms of other distance measures such as total variation distance and Wasserstein 2-distance. Moreover, in Theorem 2 we extend the result to annealed Langevin dynamics with bounded noise level, and the proof is deferred to Appendix A.2.

Theorem 2. Consider a data distribution P satisfying Assumption 1. We follow annealed Langevin dynamics for T = exp( O ( d )) steps with noise levels c σ ≥ σ 0 ≥ · · · ≥ σ T ≥ 0 for constant c σ > 0 . In addition, assume for all i ∈ [ k ] , ∥ µ i -µ 0 ∥ 2 ≤ ν 2 0 -ν 2 i 2 ( log ( ν 2 i + c 2 σ ν 2 0 + c 2 σ ) -ν 2 i + c 2 σ 2 ν 2 0 + c 2 σ + ν 2 0 + c 2 σ 2 ν 2 i + c 2 σ ) d . Suppose that the sample is initialized in P (0) σ 0 , then with probability at least 1 -T · exp( -Ω( d )) , we have ∥ x t -µ i ∥ 2 > ν 2 0 + ν 2 max +2 σ 2 t 2 d for all t ∈ { 0 } ∪ [ T ] and i ∈ [ k ] .

4.2 Langevin Dynamics in Sub-Gaussian Mixtures

We further generalize our results to sub-Gaussian mixtures. We impose the following assumptions on the mixture. It is worth noting that these assumptions automatically hold for Gaussian mixtures.

Assumption 2. Consider a data distribution P := ∑ k i =0 w i P ( i ) as a mixture of sub-Gaussian distributions, where 1 ≤ k = o ( d ) and w i > 0 is a positive constant such that ∑ k i =0 w i = 1 Suppose that P (0) = N ( µ 0 , ν 2 I d ) is Gaussian and for all i ∈ [ k ] , P ( i ) satisfies

. 0 i. P ( i ) is a sub-Gaussian distribution of mean µ i with parameter ν 2 i , ii. P ( i ) is differentiable and ∇ P ( i ) ( µ i ) = 0 d , iii. the score function of P ( i ) is L i -Lipschitz such that L i ≤ c L ν 2 i for some constant c L > 0 , iv. ν 2 0 > max { 1 , 4( c 2 L + c ν c L ) c ν (1 -c ν ) } ν 2 max 1 -c ν for constant c ν ∈ (0 , 1) , where ν max := max i ∈ [ k ] ν i , v. ∥ µ i -µ 0 ∥ 2 ≤ (1 -c ν ) ν 2 0 -ν 2 i 2(1 -c ν ) ( log c ν ν 2 i ( c 2 L + c ν c L ) ν 2 0 -ν 2 i 2(1 -c ν ) ν 2 0 + (1 -c ν ) ν 2 0 2 ν 2 i ) d .

We validate the feasibility of Assumption 2.v. in Lemma 9 in the Appendix. With Assumption 2, we show the mode-seeking tendency of Langevin dynamics under sub-Gaussian distributions in Theorem 3 and defer the proof to Appendix A.3.

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

Algorithm 1 Chained Langevin Dynamics (Chained-LD)

Require: Patch size Q , dimension d , conditional score function estimator s θ , number of iterations T , noise levels { σ t } t ∈ [ TQ/d ] , step size { δ t } t ∈ [ TQ/d ] .

1: Initialize x 0 , and divide x 0 into d/Q patches x (1) 0 , · · · x ( d/Q ) 0 of equal size Q

2: for q ← 1 to d/Q do

3: for t ← 1 to TQ/d do

5: end for

6: x ( q ) 0 ← x ( q ) TQ/d

7: end for

8: return x TQ/d

Theorem 3. Consider a data distribution P satisfying Assumption 2. We follow Langevin dynamics for T = exp( O ( d )) steps. Suppose the sample is initialized in P (0) , then with probability at least 1 -T · exp( -O ( d )) , we have ∥ x t -µ i ∥ 2 > ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d for all t ∈ { 0 } ∪ [ T ] and i ∈ [ k ] .

Finally, we slightly modify Assumption 2 and extend our results to annealed Langevin dynamics under sub-Gaussian mixtures in Theorem 4. The details of Assumption 3 and the proof of Theorem 4 are deferred to Appendix A.4.

Theorem 4. Consider a data distribution P satisfying Assumption 3. We follow annealed Langevin dynamics for T = exp( O ( d )) steps with noise levels c σ ≥ σ 0 ≥ ··· ≥ σ T ≥ 0 . Suppose the sample is initialized in P (0) σ 0 , then with probability at least 1 -T · exp( -O ( d )) , we have ∥ x t -µ i ∥ 2 > ( ν 2 0 + σ 2 t 2 + ν 2 max + σ 2 t 2(1 -c ν ) ) d for all t ∈ { 0 } ∪ [ T ] and i ∈ [ k ] .

5 Chained Langevin Dynamics

To reduce the mode-seeking tendencies of vanilla and annealed Langevin dynamics, we propose Chained Langevin Dynamics (Chained-LD) in Algorithm 1. While vanilla and annealed Langevin dynamics apply gradient updates to all coordinates of the sample in every step, we decompose the sample into patches of constant size and generate each patch sequentially to alleviate the exponential dependency on the dimensionality. More precisely, we divide a sample x into d/Q patches x (1) , · · · x ( d/Q ) of some constant size Q , and apply annealed Langevin dynamics to sample each patch x ( q ) (for q ∈ [ d/Q ] ) from the conditional distribution P ( x ( q ) | x (1) , · · · x ( q -1) ) .

An ideal conditional score function estimator s θ could jointly estimate the scores of all perturbed conditional patch distribution, i.e., ∀ σ ∈ { σ t } t ∈ [ TQ/d ] , q ∈ [ d/Q ] ,

Following from Song and Ermon (2019), we use the denoising score matching to train the estimator. For a given σ , the denoising score matching objective is

Then, combining the objectives gives the following loss 239

As shown in Vincent (2011), an estimator s θ with enough capacity minimizes the loss L if and only if 240 s θ outputs the scores of all perturbed conditional patch distribution almost surely. Ideally, if a sampler 241

242

243

244

245

246

247

248

249

250

251

252

6 Numerical Results

In this section, we empirically evaluated the mode-seeking tendencies of vanilla, annealed, and 253 chained Langevin dynamics. We performed numerical experiments on synthetic Gaussian mixture 254 models and real image datasets including MNIST (LeCun, 1998) and Fashion-MNIST (Xiao et al., 255 2017). Details on the experiment setup are deferred to Appendix B. 256

Mode 0: 100.0%

Mode 0: 19.8%

Mode 1: 40.3%

Samples

1500

IIx-μol12

Samples

Mode 2:39.9%

1500

IXx-μol12

1000

1000

from initial

500

from target

500

0

0

distribution

distribution

1500

1500

1000》

1000》

0

500

500V2

0

500

500↓21

1xμ112

1000

1500

0

1x-μ112

1000

1500

0

#Iterations

T = 1000

T = 10000

T = 100000

T = 1000000

Mode 0: 100.0%

Mode 0: 100.0%

Mode 0: 100.0%

Mode 0: 100.0%

Vanilla

1000

1500

-μol12

08

1000

1500

-μol12

1500

1000

llor-

1000

1500

500

500

500

500

0

0

0

0

Dynamics

1500

1500

1500

1500

1000

11xμ112

500

0001

15000

4

0

Xxμ112

500

1000

1500

0

500V21

1x=μ1l2

500

1000

1500

4

0

X-μ112

500

1000

15000

Mode 0: 100.0%

Mode 0: 100.0%

Mode 0: 100.0%

Mode 0: 100.0%

Annealed

1500

-μol12

1500

1500

l/or -

1500

-μol12

1000

1000

1000

1000

500

500

500

500

0

0

0

0

Dynamics

1500

1000

1500

1500

1500

Xxμ112

500

1000

15000

4

0

11xμ2112

500

1000

1500

500V21

11xμ11²

500

1000

15000

500↓21

[1xμ11²

500

1000

15000

500V2

Mode 0: 73.9%

Mode 0: 29.9%

Mode 0: 23.0%

Mode 0: 20.9%

Chained

Mode 2:13.3%

Mode 1: 12.8%

1500

l/0r-

Mode 1: 36.8%

Mode 2:33.3%

1500

Mode 2: 38.3%

Mode 1: 38.7%

1500

Mode 2: 39.6%

Mode 1: 39.5%

1500

llor-x

1000

1000

1000

1000

500

500

500

500

0

0

0

0

Dynamics

1500

1500

1500

1500

1000

1000

1000

x=μ112

500

1000

15000

4

500

1000

1500

0

500↓21

500

1000

15000

5002

11xμ11²

500

1000

15000

500↓2

perfectly generates every patch, combining all patches gives a sample from the original distribution since P ( x ) = ∏ q ∈ [ d/Q ] P ( x ( q ) | x (1) , · · · x ( q -1) ) . In Theorem 5 we give a linear reduction from producing samples of dimension d using Chained-LD to learning the distribution of a Q -dimensional variable for constant Q . The proof of Theorem 5 is deferred to Appendix A.5.

Theorem 5. Consider a sampler algorithm taking the first q -1 patches x (1) , · · · , x ( q -1) as input and outputing a sample of the next patch x ( q ) with probability ˆ P ( x ( q ) | x (1) , · · · , x ( q -1) ) for all q ∈ [ d/Q ] . Suppose that for every q ∈ [ d/Q ] and any given previous patches x (1) , · · · , x ( q -1) , the sampler algorithm can achieve

in τ ( ε, d ) iterations for some ε > 0 . Then, equipped with the sampler algorithm, the Chained-LD algorithm in d Q · τ ( ε, d ) iterations can achieve

257

258

259

260

261

262

263

264

265

Samples from initial distribution

Samples from target distribution

#Iterations

T = 3000

T = 10000

T = 30000

T = 100000

Vanilla

Dynamics

Annealed

Dynamics

Chained

Dynamics

Synthetic Gaussian mixture model: We define the data distribution P as a mixture of three Gaussian components in dimension d = 100 , where mode 0 defined as P (0) = N ( 0 d , 3 I d ) is the universal mode with the largest variance, and mode 1 and mode 2 are respectively defined as P (1) = N ( 1 d , I d ) and P (2) = N ( -1 d , I d ) . The frequencies of the three modes are 0.2, 0.4 and 0.4, i.e.,

As shown in Figure 1, vanilla and annealed Langevin dynamics cannot find mode 1 or 2 within 10 6 iterations if the sample is initialized in mode 0, while chained Langevin dynamics can find the other two modes in 1000 steps and correctly recover their frequencies as gradually increasing the number of iterations. In Appendix B.1 we present additional experiments on samples initialized in mode 1 or 2, which also verify the mode-seeking tendencies of vanilla and annealed Langevin dynamics.

Image datasets: We construct the distribution as a mixture of two modes by using the original images 266 from MNIST/Fashion-MNIST training dataset (black background and white digits/objects) as the 267 first mode and constructing the second mode by i.i.d. randomly flipping an image (white background 268 and black digits/objects) with probability 0.5. Regarding the neural network architecture of the score 269 function estimator, for vanilla and annealed Langevin dynamics we use U-Net (Ronneberger et al., 270 2015) following from Song and Ermon (2019). For chained Langevin dynamics, we proposed to use 271 Recurrent Neural Network (RNN) architectures. We note that for a sequence of inputs, the output of 272 RNN from the previous step is fed as input to the current step. Therefore, in the scenario of chained 273 Langevin dynamics, the hidden state of RNN contains information about the previous patches and 274 allows the network to estimate the conditional score function ∇ x ( q ) log P ( x ( q ) | x (1) , · · · x ( q -1) ) . 275

More implementation details are deferred to Appendix B.2. 276

Samples from initial distribution

Samples from target distribution

#Iterations

T = 3000

T = 10000

T = 30000

T = 100000

Vanilla

Dynamics

Annealed

Dynamics

Chained

Dynamics

The numerical results on image datasets are shown in Figures 2 and 3. Vanilla Langevin dynamics 277 fails to generate reasonable samples, as also observed in Song and Ermon (2019). When the sample 278 is initialized as original images from the datasets, annealed Langevin dynamics tends to generate 279 samples from the same mode, while chained Langevin dynamics can generate samples from both 280 modes. Additional experiments are deferred to Appendix B.2. 281

282

283

284

285

286

287

288

289

290

7 Conclusion

In this work, we theoretically and numerically studied the mode-seeking properties of vanilla and annealed Langevin dynamics sampling methods under a multi-modal distribution. We characterized Gaussian and sub-Gaussian mixture models under which Langevin dynamics are unlikely to find all the components within a sub-exponential number of iterations. To reduce the mode-seeking tendency of vanilla Langevin dynamics, we proposed Chained Langevin Dynamics (Chained-LD) and analyzed its convergence behavior. Studying the connections between Chained-LD and denoising diffusion models will be an interesting topic for future exploration.

Limitations

Our RNN-based implementation of Chained-LD is currently limited to image data generation tasks. 291 An interesting future direction is to extend the application of Chained-LD to other domains such as 292 audio and text data. Another future direction could be to study the convergence of Chained-LD under 293 an imperfect score estimation which we did not address in our analysis. 294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

References

Bakry, D., Barthe, F., Cattiaux, P., and Guillin, A. (2008). A simple proof of the poincaré inequality for a large class of probability measures. Electronic Communications in Probability [electronic only] , 13:60-66.

Bakry, D. and Émery, M. (1983). Diffusions hypercontractives. Seminaire de Probabilites XIX , page 177.

Benton, J., De Bortoli, V., Doucet, A., and Deligiannidis, G. (2023). Linear convergence bounds for diffusion models via stochastic localization. arXiv preprint arXiv:2308.03686 .

Bhattacharya, R. (1978). Criteria for recurrence and existence of invariant measures for multidimensional diffusions. The Annals of Probability , pages 541-553.

Bishop, C. M. (2006). Pattern recognition and machine learning. Springer google schola , 2:645-678.

Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., and Kreis, K. (2023). Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22563-22575.

Block, A., Mroueh, Y., and Rakhlin, A. (2020). Generative modeling with denoising auto-encoders and langevin sampling. arXiv preprint arXiv:2002.00107 .

Bovier, A., Eckhoff, M., Gayrard, V., and Klein, M. (2002). Metastability and low lying spectra in reversible markov chains. Communications in mathematical physics , 228:219-255.

Bovier, A., Eckhoff, M., Gayrard, V., and Klein, M. (2004). Metastability in reversible diffusion processes i: Sharp asymptotics for capacities and exit times. Journal of the European Mathematical Society , 6(4):399-424.

Chen, N., Zhang, Y., Zen, H., Weiss, R. J., Norouzi, M., and Chan, W. (2020). Wavegrad: Estimating gradients for waveform generation. In International Conference on Learning Representations .

Chen, S., Chewi, S., Li, J., Li, Y ., Salim, A., and Zhang, A. R. (2023). Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In International Conference on Learning Representations .

Cheng, X., Chatterji, N. S., Abbasi-Yadkori, Y., Bartlett, P. L., and Jordan, M. I. (2018). Sharp convergence rates for langevin dynamics in the nonconvex setting. arXiv preprint arXiv:1805.01648 .

Dalalyan, A. S. (2017). Theoretical guarantees for approximate sampling from smooth and logconcave densities. Journal of the Royal Statistical Society Series B: Statistical Methodology , 79(3):651-676.

De Bortoli, V., Thornton, J., Heng, J., and Doucet, A. (2021). Diffusion schrödinger bridge with applications to score-based generative modeling. Advances in Neural Information Processing Systems , 34:17695-17709.

Durmus, A. and Moulines, É. (2017). Nonasymptotic convergence analysis for the unadjusted langevin algorithm. The Annals of Applied Probability , 27(3):1551-1587.

Gayrard, V., Bovier, A., and Klein, M. (2005). Metastability in reversible diffusion processes ii: Precise asymptotics for small eigenvalues. Journal of the European Mathematical Society , 7(1):69-99.

Goodfellow, I. (2016). Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 .

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems , 27.

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. (2022). Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 .

Ho, J., Jain, A., and Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851.

Ke, L., Choudhury, S., Barnes, M., Sun, W., Lee, G., and Srinivasa, S. (2021). Imitation learning as f-divergence minimization. In Algorithmic Foundations of Robotics XIV: Proceedings of the Fourteenth Workshop on the Algorithmic Foundations of Robotics 14 , pages 313-329. Springer.

Kong, Z., Ping, W., Huang, J., Zhao, K., and Catanzaro, B. (2020). Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations .

Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection. Annals of statistics , pages 1302-1338.

LeCun, Y. (1998). The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/ .

Lee, H., Lu, J., and Tan, Y . (2022). Convergence for score-based generative modeling with polynomial complexity. Advances in Neural Information Processing Systems , 35:22870-22882.

Lee, H., Lu, J., and Tan, Y. (2023). Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory , pages 946-985. PMLR.

Lee, H., Risteski, A., and Ge, R. (2018). Beyond log-concavity: Provable guarantees for sampling multi-modal distributions using simulated tempering langevin monte carlo. Advances in neural information processing systems , 31.

Li, C. T. and Farnia, F. (2023). Mode-seeking divergences: theory and applications to gans. In International Conference on Artificial Intelligence and Statistics , pages 8321-8350. PMLR.

Li, G., Huang, Y., Efimov, T., Wei, Y., Chi, Y., and Chen, Y. (2024). Accelerating convergence of score-based diffusion models, provably. arXiv preprint arXiv:2403.03852 .

Li, G., Wei, Y ., Chen, Y ., and Chi, Y . (2023). Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfth International Conference on Learning Representations .

Lin, G., Milan, A., Shen, C., and Reid, I. (2017). Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1925-1934.

Poole, B., Alemi, A. A., Sohl-Dickstein, J., and Angelova, A. (2016). Improved generator objectives for gans. arXiv preprint arXiv:1612.02780 .

Raginsky, M., Rakhlin, A., and Telgarsky, M. (2017). Non-convex learning via stochastic gradient langevin dynamics: a nonasymptotic analysis. In Conference on Learning Theory , pages 16741703. PMLR.

Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 , 1(2):3.

Roberts, G. O. and Tweedie, R. L. (1996). Exponential convergence of langevin distributions and their discrete approximations. Bernoulli , pages 341-363.

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695.

Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18 , pages 234-241. Springer.

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

Sak, H., Senior, A., and Beaufays, F. (2014). Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128 .

Shannon, M., Poole, B., Mariooryad, S., Bagby, T., Battenberg, E., Kao, D., Stanton, D., and Skerry-Ryan, R. (2020). Non-saturating gan training as divergence minimization. arXiv preprint arXiv:2010.08029 .

Song, J., Meng, C., and Ermon, S. (2020a). Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 .

Song, Y. and Ermon, S. (2019). Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems , 32.

Song, Y. and Ermon, S. (2020). Improved techniques for training score-based generative models. Advances in neural information processing systems , 33:12438-12448.

Song, Y., Garg, S., Shi, J., and Ermon, S. (2020b). Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence , pages 574-584. PMLR.

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. (2020c). Scorebased generative modeling through stochastic differential equations. In International Conference on Learning Representations .

Vempala, S. and Wibisono, A. (2019). Rapid convergence of the unadjusted langevin algorithm: Isoperimetry suffices. Advances in neural information processing systems , 32.

Vincent, P. (2011). A connection between score matching and denoising autoencoders. Neural computation , 23(7):1661-1674.

Welling, M. and Teh, Y. W. (2011). Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11) , pages 681-688. Citeseer.

Wooddard, D. B., Schmidler, S. C., and Huber, M. (2009). Conditions for rapid mixing of parallel and simulated tempering on multimodal distributions. The Annals of Applied Probability , 19(2):617640.

Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 .

413

414

415

416

417

418

419

420

421

422

423

424

A Theoretical Analysis on the Mode-Seeking Tendency of Langevin Dynamics

We begin by introducing some well-established lemmas used in our proof. We first provide the proof of Proposition 1 for completeness:

Proof of Proposition 1. By the definition in equation 2, we have

For random variables t ∼ p and y ∼ N ( 0 d , I d ) , their sum z = t + y ∼ p σ follows the perturbed distribution with noise level σ . Therefore,

If t ∼ p = N ( µ , Σ ) follows a Gaussian distribution, we have z = t + y ∼ p σ = N ( µ , Σ + σ 2 I d ) . If p is a sub-Gaussian distribution with parameter ν 2 , we have z = t + y ∼ p σ is a sub-Gaussian distribution with parameter ( ν 2 + σ 2 ) . Hence we obtain Proposition 1.

We use the following lemma on the tail bound for multivariate Gaussian random variables.

Lemma 1 (Lemma 1, Laurent and Massart (2000)) . Suppose that a random variable z ∼ N ( 0 d , I d ) . Then for any λ > 0 ,

We also use a tail bound for one-dimensional Gaussian random variables and provide the proof here 425 for completeness. 426

Lemma 2. Suppose a random variable Z ∼ N (0 , 1) . Then for any t > 0 , 427

Proof of Lemma 2. Since z t ≥ 1 for all z ∈ [ t, ∞ ) , we have 428

429

430

431

432

433

434

435

436

437

Since the Gaussian distribution is symmetric, we have P ( Z ≥ t ) = P ( Z ≤ -t ) . Hence we obtain the desired bound.

A.1 Proof of Theorem 1: Langevin Dynamics under Gaussian Mixtures

Without loss of generality, we assume that µ 0 = 0 d for simplicity. Let r and n respectively denote the rank and nullity of the vector space { µ i } i ∈ [ k ] , then we have r + n = d and 0 ≤ r ≤ k = o ( d ) . Denote R ∈ R d × r an orthonormal basis of the vector space { µ i } i ∈ [ k ] , and denote N ∈ R d × n an orthonormal basis of the null space of { µ i } i ∈ [ k ] . Now consider decomposing the sample x t by

where r t ∈ R r , n t ∈ R n . Then we have

Similarly, we decompose the noise ϵ t into

where ϵ ( r ) t ∈ R r , ϵ ( n ) t ∈ R n . Then we have 438

Since a linear combination of a Gaussian random variable still follows Gaussian distribution, by 439 ϵ t ∼ N ( 0 d , I d ) , R T R = I r , and N T N = I n we obtain 440

By the definition of Langevin dynamics in equation 1, the two components of x t follow from the 441 update rule: 442

It is worth noting that since N T µ i = 0 n . To show ∥ x t -µ i ∥ 2 > ν 2 0 + ν 2 max 2 d , it suffices to prove 443

We start by proving that the initialization of the state x 0 has a large norm on the null space with high probability in the following proposition.

Proposition 2. Suppose that a sample x 0 is initialized in the distribution P (0) , i.e., x 0 ∼ P (0) , then for any constant ν max < ν 0 , with probability at least 1 -exp( -Ω( d )) , we have ∥ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d .

Proof of Proposition 2. Since x 0 ∼ P (0) = N ( 0 d , ν 2 0 I d ) and N T N = I n , we know n 0 = N T x 0 ∼ N ( 0 n , ν 2 0 I n ) . Therefore, by Lemma 1 we can bound

where the second last step follows from the assumption d -n = r = o ( d ) . Hence we complete the proof of Proposition 2.

Then, with the assumption that the initialization satisfies ∥ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d , the following proposition shows that ∥ n t ∥ remains large with high probability.

Proposition 3. Consider a data distribution P satisfies the constraints specified in Theorem 1. We follow the Langevin dynamics for T = exp( O ( d )) steps. Suppose that the initial sample satisfies ∥ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d , then with probability at least 1 -T · exp( -Ω( d )) , we have that ∥ n t ∥ 2 > ν 2 0 + ν 2 max 2 d for all t ∈ { 0 } ∪ [ T ] .

Proof of Proposition 3. To establish a lower bound on ∥ n t ∥ , we consider different cases of the step size δ t . Intuitively, when δ t is large enough, n t will be too noisy due to the introduction of random noise √ δ t ϵ ( n ) t in equation 5. While for small δ t , the update of n t is bounded and thus we can iteratively analyze n t . We first handle the case of large δ t in the following lemma.

Lemma 3. If δ t > ν 2 0 , with probability at least 1 -exp( -Ω( d )) , for n t satisfying equation 5, we have ∥ n t ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d regardless of the previous state x t -1 .

Proof of Lemma 3. Denote v := n t -1 + δ t 2 N T ∇ x log P ( x t -1 ) for simplicity. Note that v is fixed for any given x t -1 . We decompose ϵ ( n ) t into a vector aligning with v and another vector orthogonal

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

to v . Consider an orthonormal matrix M ∈ R n × ( n -1) such that M T v = 0 n -1 and M T M = I n -1 . 466 By denoting u := ϵ ( n ) t -MM T ϵ ( n ) t we have M T u = 0 n -1 , thus we obtain 467

Since ϵ ( n ) t ∼ N ( 0 n , I n ) and M T M = I n -1 , we obtain M T ϵ ( n ) t ∼ N ( 0 n -1 , I n -1 ) . Therefore, by 468 Lemma 1 we can bound 469

where the second last step follows from the assumption d -n = r = o ( d ) . Hence we complete the 470 proof of Lemma 3. 471

472

473

474

We then consider the case when δ t ≤ ν 2 0 . Let r := R T x and n := N T x , then x = Rr + Nn . We first show that when ∥ n ∥ 2 ≥ ν 2 0 + ν 2 max 2 d , P ( i ) ( x ) is exponentially smaller than P (0) ( x ) for all i ∈ [ k ] in the following lemma.

475

476

Lemma 4. Given that ∥ n ∥ 2 ≥ ν 2 0 + ν 2 max 2 d and ∥ µ i ∥ 2 ≤ ν 2 0 -ν 2 i 2 ( log ( ν 2 i ν 2 0 ) -ν 2 i 2 ν 2 0 + ν 2 0 2 ν 2 i ) d for all i ∈ [ k ] , we have P ( i ) ( x ) P (0) ( x ) ≤ exp( -Ω( d )) for all i ∈ [ k ] .

Proof of Lemma 4. For all i ∈ [ k ] , define ρ i ( x ) := P ( i ) ( x ) P (0) ( x ) , then 477

where the last step follows from the definition that R ∈ R d × r an orthonormal basis of the vector space 478 { µ i } i ∈ [ k ] and N T N = I n . Since ν 2 0 > ν 2 i , the quadratic term ∥ r ∥ 2 2 ν 2 0 - ∥ r -R T µ i ∥ 2 2 ν 2 i is maximized at 479

r = ν 2 0 R T µ i ν 2 0 -ν 2 i . Therefore, 480

Hence, for ∥ n ∥ 2 ≥ ν 2 0 + ν 2 max 2 d and ∥ µ i ∥ 2 ≤ ν 2 0 -ν 2 i 2 ( log ( ν 2 i ν 2 0 ) -ν 2 i 2 ν 2 0 + ν 2 0 2 ν 2 i ) d , we have 481

Notice that for function f ( z ) = log z -z 2 + 1 2 z , we have f (1) = 0 and d d z f ( z ) = 1 z -1 2 -1 2 z 2 = 482 -1 2 ( 1 z -1 ) 2 < 0 when z ∈ (0 , 1) . Thus, log ( ν 2 i ν 2 0 ) -ν 2 i 2 ν 2 0 + ν 2 0 2 ν 2 i is a positive constant for ν i < ν 0 , 483 i.e., ρ i ( x ) = exp( -Ω( d )) . Therefore we finish the proof of Lemma 4. 484

485

486

487

488

Lemma 4 implies that when ∥ n ∥ is large, the Gaussian mode P (0) dominates other modes P ( i ) . To bound ∥ n t ∥ , we first consider a simpler case that ∥ n t -1 ∥ is large. Intuitively, the following lemma proves that when the previous state n t -1 is far from a mode, a single step of Langevin dynamics with bounded step size is not enough to find the mode.

489

490

491

Lemma 5. Suppose δ t ≤ ν 2 0 and ∥ n t -1 ∥ 2 > 36 ν 2 0 d , then for n t following from equation 5, we have ∥ n t ∥ 2 ≥ ν 2 0 d with probability at least 1 -exp( -Ω( d )) .

Proof of Lemma 5. From the recursion of n t in equation 5 we have

By Lemma 4, we have P ( i ) ( x j -1 ) P (0) ( x j -1 ) ≤ exp( -Ω( d )) for all i ∈ [ k ] , therefore 492

̸

On the other hand, from ϵ ( n ) t ∼ N ( 0 n , I n ) we know ⟨ n t -1 , ϵ ( n ) t ⟩ ∥ n t -1 ∥ ∼ N (0 , 1) for any fixed n t -1 = 0 n , 493 hence by Lemma 2 we have 494

Combining equation 6, equation 7 and equation 8 gives that 495

with probability at least 1 -8 √ 2 πd exp ( -d 32 ) = 1 -exp( -Ω( d )) . This proves Lemma 5. 496

We then proceed to bound ∥ n t ∥ iteratively for ∥ n t -1 ∥ 2 ≤ 36 ν 2 0 d . Recall that equation 5 gives 497

We notice that the difficulty of solving n t exhibits in the dependence of log P ( x t -1 ) on r t -1 . Since 498 P = ∑ k i =0 w i P ( i ) = ∑ k i =0 w i N ( µ i , ν 2 i I d ) , we can rewrite the score function as 499

Now, instead of directly working with n t , we consider a surrogate recursion ˆ n t such that ˆ n 0 = n 0 500 and for all t ≥ 1 , 501

The advantage of the surrogate recursion is that ˆ n t is independent of r , thus we can obtain the closed-form solution to ˆ n t . Before we proceed to bound ˆ n t , we first show that ˆ n t is sufficiently close to the original recursion n t in the following lemma.

502

503

504

505

506

Proof of Lemma 6. Upon comparing equation 5 and equation 10, by equation 9 we have that for all 507 j ∈ [ t ] , 508

By Lemma 4, we have P ( i ) ( x j -1 ) P (0) ( x j -1 ) ≤ exp( -Ω( d )) for all i ∈ [ k ] , hence we obtain a recursive bound 509

Finally, by ˆ n 0 = n 0 , we have 510

Hence we obtain Lemma 6. 511

512

513

We then proceed to analyze ˆ n t , The following lemma gives us the closed-form solution of ˆ n t . We slightly abuse the notations here, e.g., ∏ c 2 i = c 1 ( 1 -δ i 2 ν 2 0 ) = 1 and ∑ c 2 j = c 1 δ j = 0 for c 1 > c 2 .

514

515

Lemma 7. For all t ≥ 0 , ˆ n t ∼ N ( ∏ t i =1 ( 1 -δ i 2 ν 2 0 ) n 0 , ∑ t j =1 ∏ t i = j +1 ( 1 -δ i 2 ν 2 0 ) 2 δ j I n ) , where the mean and covariance satisfy ∏ t i =1 ( 1 -δ i 2 ν 2 0 ) 2 + 1 ν 2 0 ∑ t j =1 ∏ t i = j +1 ( 1 -δ i 2 ν 2 0 ) 2 δ j ≥ 1 .

Proof of Lemma 7. We prove the two properties by induction. When t = 0 , they are trivial. Suppose 516 they hold for t -1 , then for the distribution of ˆ n t , we have 517

For the second property, 518

Hence we finish the proof of Lemma 7. 519

520

521

522

Armed with Lemma 7, we are now ready to establish the lower bound on ∥ ˆ n t ∥ . For simplicity, denote α := ∏ t i =1 ( 1 -δ i 2 ν 2 0 ) 2 and β := 1 ν 2 0 ∑ t j =1 ∏ t i = j +1 ( 1 -δ i 2 ν 2 0 ) 2 δ j . By Lemma 7 we know ˆ n t ∼ N ( α n 0 , βν 2 0 I n ) , so we can write ˆ n t = α n 0 + √ βν 0 ϵ , where ϵ ∼ N ( 0 n , I n ) .

523

524

Lemma 8. Given that ∥ ˆ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d , we have ∥ ˆ n t ∥ 2 ≥ 5 ν 2 0 +3 ν 2 max 8 d with probability at least 1 -exp( -Ω( d )) .

Proof of Lemma 8. By ˆ n t = α n 0 + √ βν 0 ϵ we have 525

By Lemma 1 we can bound 526

where the second last step follows from the assumption d -n = r = o ( d ) . Since ϵ ∼ N ( 0 n , I n ) , 527 we know ⟨ n 0 , ϵ ⟩ ∥ n 0 ∥ ∼ N (0 , 1) . Therefore by Lemma 2, 528

Conditioned on ∥ ˆ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d , ∥ ϵ ∥ 2 > 3 ν 2 0 + ν 2 max 4 ν 2 0 d and 1 ∥ n 0 ∥ ⟨ n 0 , ϵ ⟩ > -ν 2 0 -ν 2 max 4 ν 0 √ 3 ν 2 0 + ν 2 max d , 529

since Lemma 7 gives α 2 + β ≥ 1 we have 530

531

532

533

534

535

536

537

538

539

540

541

542

543

Hence by union bound, we complete the proof of Lemma 8.

Upon having all the above lemmas, we are now ready to establish Proposition 3 by induction. Suppose the theorem holds for all T values of 1 , · · · , T -1 . We consider the following 3 cases:

If there exists some t ∈ [ T ] such that δ t > ν 2 0 , by Lemma 3 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ 3 ν 2 0 + ν 2 max 4 d , thus the problem reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 for all t ∈ [ T ] . If there exists some t ∈ [ T ] such that ∥ n t -1 ∥ 2 > 36 ν 2 0 d , by Lemma 5 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ ν 2 0 d > 3 ν 2 0 + ν 2 max 4 d , thus the problem similarly reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 and ∥ n t -1 ∥ 2 ≤ 36 ν 2 0 d for all t ∈ [ T ] . Conditioned on ∥ n t -1 ∥ 2 > ν 2 0 + ν 2 max 2 d for all t ∈ [ T ] , by Lemma 6 we have that for T = exp( O ( d )) ,

By Lemma 8 we have that with probability at least 1 -exp( -Ω( d )) ,

Combining the two inequalities implies the desired bound 544

Hence by induction we obtain ∥ n t ∥ 2 > ν 2 0 + ν 2 max 2 d for all t ∈ [ T ] with probability at least

Therefore we complete the proof of Proposition 3.

Finally, combining Propositions 2 and 3 finishes the proof of Theorem 1.

A.2 Proof of Theorem 2: Annealed Langevin Dynamics under Gaussian Mixtures

To establish Theorem 2, we first note from Proposition 1 that perturbing a Gaussian distribution N ( µ , ν 2 I d ) with noise level σ results in a Gaussian distribution N ( µ , ( ν 2 + σ 2 ) I d ) . Therefore, for a Gaussian mixture P = ∑ k i =0 w i P ( i ) = ∑ k i =0 w i N ( µ i , ν 2 i I d ) , the perturbed distribution of noise level σ is

Similar to the proof of Theorem 1, we decompose

where R ∈ R d × r an orthonormal basis of the vector space { µ i } i ∈ [ k ] and N ∈ R d × n an orthonormal basis of the null space of { µ i } i ∈ [ k ] . Now, we prove Theorem 2 by applying the techniques developed in Appendix A.1 via substituting ν 2 with ν 2 + σ 2 t at time step t .

First, by Proposition 2, suppose that the sample is initialized in the distribution P (0) σ 0 , then with probability at least 1 -exp( -Ω( d )) , we have

Then, with the assumption that the initialization satisfies ∥ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max +4 σ 2 0 4 d , the following proposition similar to Proposition 3 shows that ∥ n t ∥ remains large with high probability.

Proposition 4. Consider a data distribution P satisfies the constraints specified in Theorem 2. We follow annealed Langevin dynamics for T = exp( O ( d )) steps with noise level c σ ≥ σ 0 ≥ σ 1 ≥ σ 2 ≥ · · · ≥ σ T ≥ 0 for some constant c σ > 0 . Suppose that the initial sample satisfies ∥ n 0 ∥ 2 ≥ 3 ν 2 0 + ν 2 max +4 σ 2 0 4 d , then with probability at least 1 -T · exp( -Ω( d )) , we have that ∥ n t ∥ 2 > ν 2 0 + ν 2 max +2 σ 2 t 2 d for all t ∈ { 0 } ∪ [ T ] .

Proof of Proposition 4. We prove Proposition 4 by induction. Suppose the theorem holds for all T values of 1 , · · · , T -1 . We consider the following 3 cases:

If there exists some t ∈ [ T ] such that δ t > ν 2 0 + σ 2 t , by Lemma 3 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ 3( ν 2 0 + σ 2 t )+( ν 2 max + σ 2 t ) 4 d = 3 ν 2 0 + ν 2 max +4 σ 2 t 4 d , thus the problem reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 + σ 2 t for all t ∈ [ T ] . If there exists some t ∈ [ T ] such that ∥ n t -1 ∥ 2 > 36( ν 2 0 + σ 2 t -1 ) d ≥ 36( ν 2 0 + σ 2 t ) d , by Lemma 5 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ ( ν 2 0 + σ 2 t ) d > 3 ν 2 0 + ν 2 max +4 σ 2 t 4 d , thus the problem similarly reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

Suppose δ t ≤ ν 2 0 + σ 2 t and ∥ n t -1 ∥ 2 ≤ 36( ν 2 0 + σ 2 t -1 ) d for all t ∈ [ T ] . Consider a surrogate sequence ˆ n such that ˆ n = n and for all t ≥ 1 ,

577 t 0 0 578

Since ν 0 > ν i and c σ ≥ σ t for all t ∈ { 0 } ∪ [ T ] , we have ν 2 i + c 2 σ ν 2 0 + c 2 σ ≥ ν 2 i + σ 2 t ν 2 0 + σ 2 t . Notice that for 579 function f ( z ) = log z -z 2 + 1 2 z , we have d d z f ( z ) = 1 z -1 2 -1 2 z 2 = -1 2 ( 1 z -1 ) 2 ≤ 0 . 580 Thus, by the assumption 581

582

583 584

585

we have that for all t ∈ [ T ] ,

Conditioned on ∥ n t -1 ∥ 2 > ν 2 0 + ν 2 max +2 σ 2 t -1 2 d for all t ∈ [ T ] , by Lemma 6 we have that for T = exp( O ( d )) ,

By Lemma 8 we have that with probability at least 1 -exp( -Ω( d )) ,

Combining the two inequalities implies the desired bound 586

Hence by induction we obtain ∥ n t ∥ 2 > ν 2 0 + ν 2 max +2 σ 2 t 2 d for all t ∈ { 0 }∪ [ T ] with probability 587 at least 588

Therefore we complete the proof of Proposition 4.

Finally, combining equation 11 and Proposition 4 finishes the proof of Theorem 2.

A.3 Proof of Theorem 3: Langevin Dynamics under Sub-Gaussian Mixtures

The proof framework is similar to the proof of Theorem 1. To begin with, we validate Assumption 2.v. in the following lemma:

Lemma9. For constants ν 0 , ν i , c ν , c L satisfying Assumptions 2.iii. and 2.iv., we have (1 -c ν ) ν 2 0 -ν 2 i 2(1 -c ν ) > 0 and log c ν ν 2 i ( c 2 L + c ν c L ) ν 2 0 -ν 2 i 2(1 -c ν ) ν 2 0 + (1 -c ν ) ν 2 0 2 ν 2 i > 0 are both positive constants.

Proof of Lemma 9. From Assumption 2.iv. that ν 2 0 > ν 2 max 1 -c ν ≥ ν 2 i 1 -c ν , we easily obtain (1 -c ν ) ν 2 0 -ν 2 i 2(1 -c ν ) > 0 is a positive constant. For the second property, let f ( z ) := log c ν ν 2 i ( c 2 L + c ν c L ) z -ν 2 i 2(1 -c ν ) z + (1 -c ν ) z 2 ν 2 i . For any z > ν 2 i 1 -c ν , the derivative of f ( z ) satisfies

589

590

591

592

593

594

595

596

597

598

Therefore, when 4( c 2 L + c ν c L ) c ν (1 -c ν ) ≤ 1 , we have 599

When 4( c 2 L + c ν c L ) c ν (1 -c ν ) > 1 , we have 600

Thus we obtain Lemma 9. 601

Without loss of generality, we assume µ 0 = 0 d . Similar to the proof of Theorem 1, we decompose 602

where R ∈ R d × r an orthonormal basis of the vector space { µ i } i ∈ [ k ] and N ∈ R d × n an orthonormal basis of the null space of { µ i } i ∈ [ k ] . To show ∥ x t -µ i ∥ 2 > ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d , it suffices to prove ∥ n t ∥ 2 > ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d . By Proposition 2, if x 0 is initialized in the distribution P (0) , i.e., x 0 ∼ P (0) , since ν 2 0 > 1 1 -c ν ν 2 max , with probability at least 1 -exp( -Ω( d )) we have

Then, conditioned on ∥ n 0 ∥ 2 ≥ ( 3 ν 2 0 4 + ν 2 max 4(1 -c ν ) ) d , the following proposition shows that ∥ n t ∥ remains large with high probability.

Proposition 5. Consider a distribution P satisfying Assumption 2. We follow the Langevin dynamics for T = exp( O ( d )) steps. Suppose that the initial sample satisfies ∥ n 0 ∥ 2 ≥ ( 3 ν 2 0 4 + ν 2 max 4(1 -c ν ) ) d , then with probability at least 1 -T · exp( -Ω( d )) , we have that ∥ n t ∥ 2 > ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d for all t ∈ { 0 } ∪ [ T ] .

Proof of Proposition 5. Firstly, by Lemma 3, if δ t > ν 2 0 , since ν 2 0 > ν 2 max 1 -c ν , we similarly have that ∥ n t ∥ 2 ≥ ( 3 ν 2 0 4 + ν 2 max 4(1 -c ν ) ) d with probability at least 1 -exp( -Ω( d )) regardless of the previous state x t -1 . We then consider the case when δ t ≤ ν 2 0 . Intuitively, we aim to prove that the score function is close to -x ν 2 0 when ∥ n ∥ 2 ≥ ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d . Towards this goal, we first show that P (0) ( x ) is exponentially larger than P ( i ) ( x ) for all i ∈ [ k ] in the following lemma:

Lemma 10. Suppose P satisfies Assumption 2. Then for any ∥ n ∥ 2 ≥ ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d , we have P ( i ) ( x ) P (0) ( x ) ≤ exp( -Ω( d )) and ∥ ∇ x P ( i ) ( x ) ∥ P ( x ) ≤ exp( -Ω( d )) for all i ∈ [ k ] .

Proof of Lemma 10. We first give an upper bound on the sub-Gaussian probability density. For any vector v ∈ R d , by considering some vector m ∈ R d , from Markov's inequality and the definition in equation 4 we can bound

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

Upon optimizing the last term at m = v -µ i ν 2 i , we obtain 623

Denote B := { z : ( v -µ i ) T ( v -z ) ≤ 0 } . To bound P z ∼ P ( i ) ( z ∈ B ) , we first note that 624

where equation 14 follows from Assumption 2.ii. that ∇ log P ( i ) ( µ i ) = 0 d and Assumption 2.iii. 625 that the score function ∇ log P ( i ) is L i -Lipschitz. Therefore we obtain 626

By observing that g : B → { z : ( v -µ i ) T ( v -z ) ≥ 0 } with g ( z ) = 2 v -z is a bijection such that 627 ∥ v -z ∥ = ∥ v -g ( z ) ∥ for any z ∈ B , we have 628

Hence, by combining equation 13, equation 15, and equation 16, we obtain 629

By Assumption 2.iii. that L i ≤ c L ν 2 i we obtain the following bound on the probability density: 630

Then we can bound the ratio of P ( i ) and P (0) . For all i ∈ [ k ] , define ρ i ( x ) := P ( i ) ( x ) P (0) ( x ) , then we have 631

where the last step follows from the definition that R ∈ R d × r an orthogonal basis of the vector space 632 { µ i } i ∈ [ k ] and N T N = I n . Since ν 2 i < (1 -c ν ) ν 2 0 , the quadratic term ∥ r ∥ 2 2 ν 2 0 -(1 -c ν ) ∥ r -R T µ i ∥ 2 2 ν 2 i is 633 maximized at r = (1 -c ν ) ν 2 0 R T µ i (1 -c ν ) ν 2 0 -ν 2 i . Therefore, we obtain 634

Hence, for ∥ µ i -µ 0 ∥ 2 ≤ (1 -c ν ) ν 2 0 -ν 2 i 2(1 -c ν ) ( log c ν ν 2 i ( c 2 L + c ν c L ) ν 2 0 -ν 2 i 2(1 -c ν ) ν 2 0 + (1 -c ν ) ν 2 0 2 ν 2 i ) d and ∥ n ∥ 2 ≥ 635 ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d , we have 636

From Lemma 9, we obtain ρ i ( x ) ≤ exp( -Ω( d )) . 637

To show ∥ ∇ x P ( i ) ( x ) ∥ P ( x ) ≤ exp( -Ω( d )) , from Assumptions 2.ii. and 2.iii. we have 638

Therefore, we can bound ∥ ∇ x P ( i ) ( x ) ∥ P ( x ) ≤ c L ν 2 i ρ i ( x ) ∥ x -µ i ∥ . When ∥ x -µ i ∥ = exp( o ( d )) is 639 small, by ρ i ( x ) ≤ exp( -Ω( d )) we directly have ∥ ∇ x P ( i ) ( x ) ∥ P ( x ) ≤ exp( -Ω( d )) . When ∥ x -µ i ∥ = 640 exp(Ω( d )) is exceedingly large, from equation 17 we have 641

Since ν 2 0 > ν 2 i 1 -c ν , when ∥ x -µ i ∥ = exp(Ω( d )) ≫∥ µ i ∥ we have 642

643

644

645

646

647

Therefore ∥ ∇ x P ( i ) ( x ) ∥ P ( x ) ≤ exp( -Ω( d )) . Thus we complete the proof of Lemma 10.

Similar to Lemma 5, the following lemma proves that when the previous state n t -1 is far from a mode, a single step of Langevin dynamics with bounded step size is not enough to find the mode.

Lemma 11. Suppose δ t ≤ ν 2 0 and ∥ n t -1 ∥ 2 > 36 ν 2 0 d , then we have ∥ n t ∥ 2 ≥ ν 2 0 d with probability at least 1 -exp( -Ω( d )) .

Proof of Lemma 11. For simplicity, denote v := n t -1 + δ t 2 N T ∇ x log P ( x t -1 ) . Since P = 648 ∑ k i =0 w i P ( i ) and P (0) = N ( µ 0 , ν 2 0 I d ) , the score function can be written as 649

For ∥ n t -1 ∥ 2 > 36 ν 2 0 d by Lemma 10 we have ∥ ∇ x P ( i ) ( x t -1 ) ∥ P ( x t -1 ) ≤ exp( -Ω( d )) . Since δ t ≤ ν 2 0 , we 650 can bound the norm of v by 651

̸

On the other hand, from ϵ ( n ) t ∼ N ( 0 n , I n ) we know ⟨ v , ϵ ( n ) t ⟩ ∥ v ∥ ∼ N (0 , 1) for any fixed v = 0 n , hence 652 by Lemma 2 we have 653

Combining the above inequalities gives 654

with probability at least 1 -8 √ 2 πd exp ( -d 32 ) = 1 -exp( -Ω( d )) . This proves Lemma 11. 655

When ∥ n t -1 ∥ 2 ≤ 36 ν 2 0 d , similar to Theorem 1, we consider a surrogate recursion ˆ n t such that 656 ˆ n 0 = n 0 and for all t ≥ 1 , 657

658

The following Lemma shows that ˆ n t is sufficiently close to the original recursion n t .

659

660

Lemma 12. For any t ≥ 1 , given that for all j ∈ [ t ] , δ j ≤ ν 2 0 and ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d ≤ ∥ n j -1 ∥ 2 ≤ 36 ν 2 0 d , if µ i satisfies Assumption 2.v. for all i ∈ [ k ] , we have ∥ ˆ n t -n t ∥ ≤ t exp(Ω( d )) √ d .

Proof of Lemma 12. By equation 18 we have that for all j ∈ [ t ] , 661

By Lemma 10, we have P ( i ) ( x j -1 ) P (0) ( x j -1 ) ≤ exp( -Ω( d )) and ∥ ∇ x P ( i ) ( x j -1 ) ∥ P ( x j -1 ) ≤ exp( -Ω( d )) for all 662 i ∈ [ k ] , hence from ∥ n j -1 ∥ ≤ 6 ν 0 √ d we obtain a recursive bound 663

Finally, by ˆ n 0 = n 0 , we have

Hence we obtain Lemma 12.

Armed with the above lemmas, we are now ready to establish Proposition 5 by induction. Please note that we also apply some lemmas from the proof of Theorem 1 by substituting ν 2 max with ν 2 max 1 -c ν . Suppose the theorem holds for all T values of 1 , · · · , T -1 . We consider the following 3 cases:

If there exists some t ∈ [ T ] such that δ t > ν 2 0 , by Lemma 3 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ ( 3 ν 2 0 4 + ν 2 max 4(1 -c ν ) ) d , thus the problem reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 for all t ∈ [ T ] . If there exists some t ∈ [ T ] such that ∥ n t -1 ∥ 2 > 36 ν 2 0 d , by Lemma 11 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ ν 2 0 d > ( 3 ν 2 0 4 + ν 2 max 4(1 -c ν ) ) d , thus the problem similarly reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 and ∥ n t -1 ∥ 2 ≤ 36 ν 2 0 d for all t ∈ [ T ] . Conditioned on ∥ n t -1 ∥ 2 > ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d for all t ∈ [ T ] , by Lemma 12 we have that for T = exp( O ( d )) ,

By Lemma 8 we have that with probability at least 1 -exp( -Ω( d )) ,

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

Combining the two inequalities implies the desired bound 679

Hence by induction we obtain ∥ n t ∥ 2 > ( ν 2 0 2 + ν 2 max 2(1 -c ν ) ) d for all t ∈ [ T ] with probability at least

Therefore we complete the proof of Proposition 5.

Finally, combining equation 12 and Proposition 5 finishes the proof of Theorem 3.

A.4 Proof of Theorem 4: Annealed Langevin Dynamics under Sub-Gaussian Mixtures

Assumption 3. Consider a data distribution P := ∑ k i =0 w i P ( i ) as a mixture of sub-Gaussian distributions, where 1 ≤ k = o ( d ) and w i > 0 is a positive constant such that ∑ k i =0 w i = 1 . Suppose that P (0) = N ( µ 0 , ν 2 0 I d ) is Gaussian and for all i ∈ [ k ] , P ( i ) satisfies

P ( i ) is a sub-Gaussian distribution of mean µ i with parameter ν 2 i ,

P ( i ) is differentiable and ∇ P ( i ) σ t ( µ i ) = 0 d for all t ∈ { 0 } ∪ [ T ] ,

for all t ∈ { 0 } ∪ [ T ] , the score function of P ( i ) σ t is L i,t -Lipschitz such that L i,t ≤ c L ν 2 i + σ 2 t for some constant c L > 0 ,

iv. ν 2 0 > max { 1 , 4( c 2 L + c ν c L ) c ν (1 -c ν ) } ν 2 max + c 2 σ 1 -c ν -c 2 σ for constant c ν ∈ (0 , 1) , where ν max := max i ∈ [ k ] ν i ,

The feasibility of Assumption 3.v. can be validated by substituting ν 2 in Lemma 9 with ν 2 + c 2 σ . To establish Theorem 4, we first note from Proposition 1 that for a sub-Gaussian mixture P = ∑ k i =0 w i P ( i ) , the perturbed distribution of noise level σ is P σ = ∑ k i =0 w i P ( i ) σ , where P (0) = N ( µ 0 , ( ν 2 i + σ 2 ) I d ) and P ( i ) is a sub-Gaussian distribution with mean µ i and sub-Gaussian parameter ( ν 2 i + σ 2 ) . Similar to the proof of Theorem 1, we decompose

where R ∈ R d × r an orthonormal basis of the vector space { µ i } i ∈ [ k ] and N ∈ R d × n an orthonormal basis of the null space of { µ i } i ∈ [ k ] . Now, we prove Theorem 4 by applying the techniques developed in Appendix A.1 and A.3 via substituting ν 2 and ν 2 1 -c ν with ν 2 + σ 2 t 1 -c ν at time step t . Note that for all t ∈ { 0 } ∪ [ T ] , Assumption 3.iv. implies ν 2 0 + σ 2 t > max { 1 , 4( c 2 L + c ν c L ) c ν (1 -c ν ) } ν 2 max + σ 2 t 1 -c ν because c σ ≥ σ t .

First, by Proposition 2, suppose that the sample is initialized in the distribution P (0) σ 0 , then with probability at least 1 -exp( -Ω( d )) , we have

Then, with the assumption that the initialization satisfies ∥ n 0 ∥ 2 ≥ ( 3( ν 2 0 + σ 2 0 ) 4 + ν 2 max + σ 2 0 4(1 -c ν ) ) d , the following proposition similar to Proposition 5 shows that ∥ n t ∥ remains large with high probability. Proposition 6. Consider a distribution P satisfying Assumption 3. We follow annealed Langevin dynamics for T = exp( O ( d )) steps with noise level c σ ≥ σ 0 ≥ σ 1 ≥ · · · ≥ σ T ≥ 0 for some constant c σ > 0 . Suppose that the initial sample satisfies ∥ n 0 ∥ 2 ≥ ( 3( ν 2 0 + σ 2 0 ) 4 + ν 2 max + σ 2 0 4(1 -c ν ) ) d , then with probability at least 1 -T · exp( -Ω( d )) , we have that ∥ n t ∥ 2 > ( ν 2 0 + σ 2 t 2 + ν 2 max + σ 2 t 2(1 -c ν ) ) d for all t ∈ { 0 } ∪ [ T ] .

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

Proof of Proposition 6. We prove Proposition 6 by induction. Suppose the theorem holds for all T values of 1 , · · · , T -1 . We consider the following 3 cases:

If there exists some t ∈ [ T ] such that δ t > ν 2 0 + σ 2 t , by Lemma 3 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ ( 3( ν 2 0 + σ 2 t ) 4 + ν 2 max + σ 2 t 4(1 -c ν ) ) d , thus the problem reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 + σ 2 t for all t ∈ [ T ] . If there exists some t ∈ [ T ] such that ∥ n t -1 ∥ 2 > 36( ν 2 0 + σ 2 t -1 ) d ≥ 36( ν 2 0 + σ 2 t ) d , by Lemma 11 we know that with probability at least 1 -exp( -Ω( d )) , we have ∥ n t ∥ 2 ≥ ( ν 2 0 + σ 2 t ) d > ( 3( ν 2 0 + σ 2 t ) 4 + ν 2 max + σ 2 t 4(1 -c ν ) ) d , thus the problem similarly reduces to the two sub-arrays n 0 , · · · , n t -1 and n t , · · · , n T , which can be solved by induction.

Suppose δ t ≤ ν 2 0 + σ 2 t and ∥ n t -1 ∥ 2 ≤ 36( ν 2 0 + σ 2 t -1 ) d for all t ∈ [ T ] . Consider a surrogate sequence ˆ n t such that ˆ n 0 = n 0 and for all t ≥ 1 ,

Since ν 0 > ν i and c σ ≥ σ t for all t ∈ { 0 } ∪ [ T ] , we have ν 2 i + c 2 σ ν 2 0 + c 2 σ > ν 2 i + σ 2 t ν 2 0 + σ 2 t . Notice that for 726 function f ( z ) = log z -z 2 + 1 2 z , we have d d z f ( z ) = 1 z -1 2 -1 2 z 2 = -1 2 ( 1 z -1 ) 2 ≤ 0 . 727 Thus, by Assumption 3.v. we have that for all t ∈ [ T ] , 728

Conditioned on ∥ n t -1 ∥ 2 > ( ν 2 0 + σ 2 t -1 2 + ν 2 max + σ 2 t -1 2(1 -c ν ) ) d for all t ∈ [ T ] , by Lemma 12 we 729 have that for T = exp( O ( d )) , 730

731

732

733

734

By Lemma 8 we have that with probability at least 1 -exp( -Ω( d )) ,

Combining the two inequalities implies the desired bound

Hence by induction we obtain ∥ n t ∥ 2 > ( ν 2 0 + σ 2 T 2 + ν 2 max + σ 2 T 2(1 -c ν ) ) d for all t ∈ [ T ] with probability at least

Therefore we complete the proof of Proposition 6. 735

Finally, combining equation 20 and Proposition 6 finishes the proof of Theorem 4. 736

A.5 Proof of Theorem 5: Convergence Analysis of Chained Langevin Dynamics 737

For simplicity, denote x [ q ] = { x (1) , · · · , x ( q ) } . By the definition of total variation distance, for all 738 q ∈ [ d/Q ] we have 739

Upon summing up the above inequality for all q ∈ [ d/Q ] , we obtain 740

Thus we finish the proof of Theorem 5. 741

742

743

744

745

746

747

748

749

750

751

752

753

B Additional Experiments

Algorithm Setup: Our choices of algorithm hyperparameters are based on Song and Ermon (2019). We consider L = 10 different standard deviations such that { λ i } i ∈ [ L ] is a geometric sequence with λ 1 = 1 and λ 10 = 0 . 01 . For annealed Langevin dynamics with T iterations, we choose the noise levels { σ t } t ∈ [ T ] by repeating every element of { λ i } i ∈ [ L ] for T/L times and we set the step size as δ t = 2 × 10 -5 · σ 2 t /σ 2 T for every t ∈ [ T ] . For vanilla Langevin dynamics with T iterations, we use the same step size as annealed Langevin dynamics. For chained Langevin dynamics with T iterations, the patch size Q is chosen depending on different tasks. For every patch of chained Langevin dynamics, we choose the noise levels { σ t } t ∈ [ TQ/d ] by repeating every element of { λ i } i ∈ [ L ] for TQ/dL times and we set the step size as δ t = 2 × 10 -5 · σ 2 t /σ 2 TQ/d for every t ∈ [ TQ/d ] .

B.1 Synthetic Gaussian Mixture Model

We choose the data distribution P as a mixture of three Gaussian components in dimension d = 100 :

Since the distribution is given, we assume that the sampling algorithms have access to the ground-truth 754 score function. We set the batch size as 1000 and patch size Q = 10 for chained Langevin dynamics. 755 We use T ∈ { 10 3 , 10 4 , 10 5 , 10 6 } iterations for vanilla, annealed, and chained Langevin dynamics. 756 The initial samples are i.i.d. chosen from P (0) , P (1) , or P (2) , and the results are presented in Figures 757

Mode 1: 100.0%

Mode 0: 19.8%

Mode 1: 40.3%

Samples

1500

x-μol12

Samples

Mode 2:39.9%

1500

1IIx-μol12

1000

1000

from initial

500

from target

500

0

0

distribution

1500

distribution

1500

1000%

1000》

500

50021

0

500

5002

Xx-μ112

1000

1500

0

11xμ2112

1000

1500

0

#Iterations

T = 1000

T = 10000

T = 100000

T = 1000000

Mode 0: 89.8%

Mode 0: 100.0%

Mode 0:100.0%

Mode 0: 100.0%

Vanilla

Mode 1: 10.2%

1500

llor-

1500

lor

1500

llor -

1500

1000

1000

1000

1000

500

500

500

500

Dynamics

0

0

0

0

1500

1500

1500

1500

1xμ112

500

1000

15000

0

1xμ112

500

0001

1500

0

500↓21

Xμ112

500

1000

15000

5002

xμ1

500

1000

15000

500V2

Mode 0: 6.1%

Mode 0: 14.4%

Mode 0: 71.2%

Mode 0: 100.0%

Mode 1: 93.9%

Mode 1: 85.6%

Mode 1: 28.8%

Annealed

1500

1500

1500

1500

1000

1000

1000

1000

500

500

500

500

0

0

0

0

Dynamics

1500

1500

1500

1500

1Xμ112

500

1000

15000

1xμ112

500

1000

1500

500V21

X=μ2112

500

1000

15000

500↓21

0

Xμ11²

500

1000

15000

500V2

Mode 0: 37.3%

Mode 0: 27.8%

Mode 0: 20.7%

Mode 0: 22.4%

Mode 1: 48.1%

Mode 1: 37.7%

Mode 1: 39.1%

Mode 1: 39.9%

Chained

Mode 2:14.6%

1500

X-μoll2

Mode 2:34.5%

1500

X-μol12

Mode 2:40.2%

1500

IIx-μol12

Mode 2: 37.7%

1500

1000

1000

1000

1000

500

500

500

500

0

0

0

Dynamics

1500

1500

1500

1500

500

1000

15000

500V2l

1x=μ112

500

0001

15000

500Vz21

Xμ112

500

0001

15000

500V21

x=μ112

500

1000

15000

500↓2

1xμ112

1, 4, and 5 respectively. The two subfigures above the dashed line illustrate the samples from the 758 initial distribution and target distribution, and the subfigures below the dashed line are the samples 759 generated by different algorithms. A sample x is clustered in mode 1 if it satisfies ∥ x -µ 1 ∥ 2 ≤ 5 d 760 and ∥ x -µ 1 ∥ 2 ≤ ∥ x -µ 2 ∥ 2 ; in mode 2 if ∥ x -µ 2 ∥ 2 ≤ 5 d and ∥ x -µ 1 ∥ 2 > ∥ x -µ 2 ∥ 2 ; and in 761 mode 0 otherwise. The experiments were run on an Intel Xeon CPU with 2.90GHz. 762

763

764

765

766

767

768

769

B.2 Image Datasets

Our implementation and hyperparameter selection are based on Song and Ermon (2019). During training, we i.i.d. randomly flip an image with probability 0.5 to construct the two modes (i.e., original and flipped images). All models are optimized by Adam with learning rate 0.001 and batch size 128 for a total of 200000 training steps, and we use the model at the last iteration to generate the samples. We perform experiments on MNIST (LeCun, 1998) (CC BY-SA 3.0 License) and Fashion-MNIST (Xiao et al., 2017) (MIT License) datasets and we set the patch size as Q = 14 .

For the score networks of vanilla and annealed Langevin dynamics, following from Song and Ermon 770 (2019), we use the 4-cascaded RefineNet (Lin et al., 2017), a modern variant of U-Net (Ronneberger 771 et al., 2015) with residual design. For the score networks of chained Langevin dynamics, we use the 772 official PyTorch implementation of an LSTM network (Sak et al., 2014) followed by a linear layer. 773 For MNIST and Fashion-MNIST datasets, we set the input size of the LSTM as Q = 14 , the number 774 of features in the hidden state as 1024, and the number of recurrent layers as 2. The inputs of LSTM 775 include inputting tensor, hidden state, and cell state, and the outputs of LSTM include the next hidden 776 state and cell state, which can be fed to the next input. To estimate the noisy score function, we first 777 input the noise level σ (repeated for Q times to match the input size of LSTM) and all-0 hidden and 778

779

780

781

782

783

784

785

786

787

788

789

Mode 2: 100.0%

Mode 0: 19.8%

Mode 1: 40.3%

Samples

1500

llor-x

Samples

Mode 2:39.9%

1500

IIx-μolI2

1000

1000

from initial

500

from target

500

0

0

distribution

distribution

1500

1500

1000%

0001

0

1x-μ112

500

1000

1500

0

5002

+

0

I1x-μ112

500

1000

1500

0

500V21

#Iterations

T = 1000

T = 10000

T = 100000

T = 1000000

Mode 0: 88.8%

Mode 0: 100.0%

Mode 0: 100.0%

Mode 0: 100.0%

Vanilla

Mode 2: 11.2%

1500

1500

zllor

1500

zllor

1500

llor

1000

1000

1000

1000

500

500

500

500

1

0

0

0

0

Dynamics

1500

1500

1500

1500

1000

1000》

1000》

0

11xμ112

500

1000

15000

500μ2l

4

0

1xμ112

500

1000

1500

0

500μzl

Xxμ112

500

1000

1500

。

4

0

1|x-μ2112

500

1000

15000

500μ2l

Mode 0: 7.3%

Mode 0: 18.3%

Mode 0: 70.9%

Mode 0: 100.0%

Annealed

Mode 2: 92.7%

1500

llor -

Mode 2: 81.7%

1500

llor-

Mode 2: 29.1%

1500

-μol12

1500

1000

1000

1000

1000

500

500

500

500

0

0

Dynamics

1500

1500

1500

1500

x=μ112

500

1000

15000

500V2

1xμ11²

500

1000

1500

0

4

I1xμ1l2

500

1000

15000

5002

0

Xx=μ11²

500

1000

15000

500V2

Mode 0: 35.8%

Mode 0: 31.4%

Mode 0: 22.4%

Mode 0: 19.6%

Mode 1: 13.7%

Mode 1: 34.2%

Mode 1: 40.9%

Mode 1: 40.8%

Chained

Mode 2: 50.5%

1000

1500

ll0r-

Mode 2: 34.4%

1000

1500

（-μol1²

Mode 2:36.7%

1500

1000

llor

Mode 2: 39.6%

1000

1500

llor

500

500

500

500

0

0

Dynamics

1500

1000

1500

1000%

1500

1500

0

500

1000

15000

5002

0

500

1000

15000

50021

4

V

500

1000

15000

5002

0

500

1000

15000

500V2

1xμ112

1Xxμ112

Xμ112

Xxμ112

cell states to obtain an initialization of the hidden and cell states. Then, we divide a sample into d/Q patches and input the sequence of patches to the LSTM. For every output hidden state corresponding to one patch, we apply a linear layer of size 1024 × Q to estimate the noisy score function of the patch.

To generate samples, we use T ∈ { 3000 , 10000 , 30000 , 100000 } iterations for vanilla, annealed, and chained Langevin dynamics. The initial samples are chosen as either original or flipped images from the dataset, and the results for MNIST and Fashion-MNIST datasets are presented in Figures 2, 6, 3, and 7 respectively. The two subfigures above the dashed line illustrate the samples from the initial distribution and target distribution, and the subfigures below the dashed line are the samples generated by different algorithms. High-quality figures generated by annealed and chained Langevin dynamics for T = 100000 iterations are presented in Figures 8 and 9.

All experiments were run with one RTX3090 GPU. It is worth noting that the training and inference 790 time of chained Langevin dynamics using LSTM is considerably faster than vanilla/annealed Langevin 791 dynamics using RefineNet. For a course of 200000 training steps on MNIST/Fashion-MNIST, due 792 to the different network architectures, LSTM takes around 2.3 hours while RefineNet takes around 793 9.2 hours. Concerning image generation, chained Langevin dynamics is significantly faster than 794 vanilla/annealed Langevin dynamics since every iteration of chained Langevin dynamics only updates 795 a patch of constant size, while every iteration of vanilla/annealed Langevin dynamics requires 796 computing all coordinates of the sample. One iteration of chained Langevin dynamics using LSTM 797 takes around 1.97 ms, while one iteration of vanilla/annealed Langevin dynamics using RefineNet 798 takes around 43.7 ms. 799

800

Samples from initial distribution

Samples from target distribution

4

&

41

82

#Iterations

T = 3000

T = 10000

T = 30000

T = 100000

Vanilla

Dynamics

Annealed

Dynamics

Chained

Dynamics

C Boarder Impacts

This paper presents work whose goal is to advance the field of machine learning. No potential societal 801 consequence of this work needs to be highlighted here. 802

Samples from initial distribution

Samples from target distribution

#Iterations

T = 3000

T = 10000

T = 30000

T = 100000

Vanilla

Dynamics

Annealed

Dynamics

Chained

Dynamics

Initialization

Original Images

Flipped Images

一

L

Annealed

C

7

y

4

一

7

Dynamics

4

12

4

h

3

2

0

5

7

2

2

S

h

3

3

女

1

9

2

2

6

3

B

6

7

3

7

3

9

↑

2

9

h

5

?

Chained

h

2

4

b

力

D

Dynamics

7

S

2

2.

名

9

子

9

2

2

5

2

5

a

C

4

3

。

7

O

7

7

C

S

3

7

6

4

9

2

2

3

7

8

Initialization

Original Images

Flipped Images

Annealed

Dynamics

Chained

Dynamics

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We list the paper's contributions about the mode-seeking tendencies of vanilla, annealed, and chained Langevin Dynamics in the abstract and at the end of Section 1. The scope of this work is also discussed in Section 1.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of this work are discussed in Section 7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The assumptions and proof of every theorem are clearly stated. For Theorem 1, the assumptions are listed in Assumption 1 and the proof is in Appendix A.1; for Theorem 2, the assumptions are listed in Assumption 1 and the proof is in Appendix A.2; for Theorem 3, the assumptions are listed in Assumption 2 and the proof is in Appendix A.3; for Theorem 4, the assumptions are listed in Assumption 3 and the proof is in Appendix A.4.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide all the information about our numerical experiments in Section 6 and Appendix B that enable readers to reproduce our results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We submitted our code in supplementary materials for the readers to reproduce our numerical results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All details of the numerical experiments are listed in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For the results of synthetic data, we report 1000 samples for every experiment. For the results of image datasets, we report 49 or 100 samples for every experiment.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

961

962

963

964

965

966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

991

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Information on the computational resources of the numerical experiments is provided in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We read the guidelines and wrote the paper according to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: As discussed in Appendix C, this work has potential societal consequences,

1010

none of which needs to be specially highlighted.

Guidelines: 1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045

1046

1047

1048

1049

1050

1051

1052

1053

1054

1055

1056

1057

1058

1059

1060

1061

1062

1063

1064

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The models used in this work do not have a high risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All assets are properly cited in our work.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

1065

1066

1067

1068

1069

1070

1071

1072

1073

1074

1075

1076

1077

1078

1079

1080

1081

1082

1083

1084

1085

1086

1087

1088

1089

1090

1091

1092

1093

1094

1095

1096

1097

1098

1099

1100

1101

1102

1103

1104

1105

1106

1107

1108

1109

1110

1111

1112

1113

1114

1115

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Crowdsourcing or research with human subjects is not involved in this work.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Crowdsourcing or research with human subjects is not involved in our paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

1116

1117

1118

1119

1120

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the Langevin process for sampling from a target distribution $\\pi$. This process is known to be slow-converging for multimodal targets: in practice, it has been observed that the process gets \"stuck\" in some modes of the target, and do not \"reach\" other modes of the target. The authors provide theoretical results for this behavior. In Theorem 1, they prove that by evolving a particle with the Langevin process during exponential time (in the dimension), the particle will still be far away (in probability) from some modes. They also prove, in Theorem 2, that this negative result holds even when using the popular heuristic of \"annealing\" the Langevin process using intermediate distributions, obtained by adding different levels of Gaussian noise to the target samples. \n\nInstead, the authors propose running an alternative sampling process which they call \"Chained Langevin dynamics\". This consists in running \"annealed\" Langevin processes for each component of the target distribution, that is, for each $\\pi(x_i | x_{-i})$. The authors estimate the score of each of these conditional targets using a score-matching loss, and empirically demonstrate the ability of their process to reach the different target modes in a limited time. Theoretically, they prove their process approximates the target (in TV divergence) in linear time (in the dimension).\n\nSTRENGTHS:\nThe authors provide an interesting perspective on popular sampling processes, Langevin and its annealed counterpart. The paper is clearly written and the results are an interesting contribution to the sampling community.\n\nWEAKNESSES:\nSee questions.\n\nQUESTIONS:\n**Q1**: Theorem 5 is stated at the distribution-level (the result is for $p_t$), while Theorems 1 and 2 are stated at the particle-level (the result is for $x_t \\sim p_t$ ). The authors hint that Theorems 1 and 2 could also be formulated at the distribution-level: \"this notion can also easily be translated into a lower bound in terms of other distance measures such as total variation distance and Wasserstein 2-distance\". If this is an easy result to add, could the authors add it to their paper? Lower-bounds are notoriously hard(er than upper bounds) on the convergence. Namely, stating Theorem 2 as a lower bound on Annealed Langevin Dynamics would be an important contribution. \n\n**Q2**: Theorem 2 is surprising to me. It seems to convey that Langevin Dynamics are *worse* when annealing, in the sense that the particle $x_t$ is *further* away by the additive constant $2 \\sigma_t^2$ from the non-dominant modes $\\mu_i$. This seems to be in contradiction with theoretical and empirical results that show that annealing helps reach all the modes of the target distribution, as conveyed in Figure 1 of [1] or Figure 3 of [2] or even Figure [3] of the authors' submission where the Annealed Langevin process produces more accurate samples than the Vanilla Langevin process. Can the authors explain why that is?\n\n**Q3**: When introducing the Chained Langevin Dynamics, the authors seem to motivate this process by saying that by sampling one component at a time from $\\pi(x_i | x_{-i})$, this reduces the complexity in the dimension. Could the authors elaborate on this? My understand is that if each of the one-dimensional, conditional targets $\\pi(x_i | x_{-i})$ were heavily multimodal, then Annealed Langevin dynamics would still struggle to cover all the modes, although it would struggle less than in higher dimensions in light of Theorem 2 taking $d >> 1$.\n\n[1] Dynamical Regimes of Diffusion Models. Biroli et al. Arxiv, 2024. \n\n[2] Generative Modeling by Estimating Gradients of the Data Distribution. Song et al. NeurIPS 2019.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

comments: SUMMARY:
The authors study Langevin dynamics (as well as its annealed counterpart) for gaussian mixtures and sub-gaussian mixtures. In Sec. 4, they prove that Langevin remains stuck in the "dominant mode" for an at least exponential time, a claim that is often made in the ML literature but which is never formally proved. In Sec. 5, they provide a sequential method to get rid of this dependence.

STRENGTHS:
It is healthy to finally have a paper that explicitly prove the claims made in the ML literature and that were known in practice for a long time.  Furthermore, it shows that, unlike what was believed, annealed Langevin also fails.

WEAKNESSES:
I do not understand why it is sensible to say that initially, $p_0$ should follow $P_0$, one of the component of the mixture, isn't it a rather strong assumption?

QUESTIONS:
Do similar results hold when $p_0$ is initialized at $\mathcal{N}(0, Id)$ and this component is not in the target mixture?

LIMITATIONS:
It seems like the assumption $p_0 \sim P_0$ is not enough justified. Also, I would have like an insight of the proof of the Theorems in Sec. 4.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA new algorithm is proposed, called Chained Langevin Dynamics, to improve on the mode-seeking properties of Langevin Dynamics, after annleade Langevin Dynamics had been proposed but did not give significant improvements.\nResults about the mode-seeking properties of the three algorithms are obtained.\nThe results of numerical experiments on synthetic and real image datasets are also shown.\n\nSTRENGTHS:\nVery inspiring idea on how to improve mode-search for multimodal distributions.\nVery clear presentation of premises and of the old and new algorithms.\nThe new algorithm looks very powerful.\n\nWEAKNESSES:\nNo evident connection has been established between experiments and mathematical results.\nThe description/comment of experiments could have been more accurate (see Questions).\n\nQUESTIONS:\nIn structured data, does the order of patches matter?\n\nHow large is the selected size Q of the patches in the examples? Why was it selected that way? Smaller patches help convergence due to the reduced dimension. What happens for too small patches? Does the algorithm still work?\n\n\"Regarding the neural network architecture of the score function estimator, for vanilla and annealed Langevin dynamics we use U-Net (Ronneberger et al., 2015) following from Song and Ermon (2019). For chained Langevin dynamics, we proposed to use Recurrent Neural Network (RNN) architectures.\" The change in the set up seems a little arbitrary. What did it happen by using U-Net for chained Langevin dynamics or, viceversa, using the RNN for vanilla and annealed Langevin dynamics?\n\nMore on the general concepts of the paper, the authors might find interesting that the principle of chained Langevin dynamics seems based on \"nucleation\" of different modes and spreading of the information of the randomly selected mode through the conditional probability to the entire image, much like the mechanism described in this old paper that allowed reconstruction of images from very little sampling \nStatistical-Physics-Based Reconstruction in Compressed Sensing\nF. Krzakala, M. Mézard, F. Sausset, Y. F. Sun, and L. Zdeborová\nPhys. Rev. X 2, 021005 – Published 11 May 2012\n\nLIMITATIONS:\nThere is a section about limitations in the text.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies Langevin-based algorithms for sampling from multimodal distributions, motivated by generative modeling. The main content of the paper are lower bounds on the convergence of both Langevin and annealed Langevin for mixtures of Gaussian and sub-Gaussian distributions, as well as a proposed modification of the annealed Langevin dynamics to operate on coordinate patches one-at-a-time.\n\nSTRENGTHS:\n- Sampling from multimodal distributions is an importnat problem both theoretically and practically.\n\n- The Chained Langevin Dynamics algorithm that is proposed appears to be novel.\n\n- The empirical results are promising, albeit in a rather contrived setting.\n\nWEAKNESSES:\n- The lower bounds hold only for the distance between the sample and the mean, rather than any standard notion of distance or divergence between probability measures. Moreover, I do not expect that these bounds imply such a quantity is large.\n\n- Related to the above point, it is difficult to appreciate the significance of the lower bound since the lower bound does not depend on the separation between the means. In particular, it seems the lower bounds only show that the iterate remains roughly on the order of the larger variance which is, for example, not surprising in the case where the variances are all of the same order.\n\n- The hidden constants in the $\\Omega$ notation are important but difficult to find (as they are suppressed in the main text and some of the appendix). In particular, there should be dependence on the mixture weights but this can't be seen from their result.\n\n- It is unclear if the upper bound in Theorem 5 can be instantiated for their algorithm (see question below).\n\nQUESTIONS:\n- In light of [1], there is arguably \"no mystery\" in terms of convergence of the reversed Langevin dynamics in diffusion models: as long as the score function can be accurately estimated, the reversed dynamics will converge to the target. At a high level, why do you then study annealed Langevin in the setting where the scores are known exactly?\n\n- I suggest you stop using the $\\Omega$ notation and make the hidden constants more clear as they are very important for understanding your result.\n\n- With regard to Theorem 5, a remark that explains, even if conjecturally, how the run-time of the Langevin algorithm might scale in the dimension and how the bound in Theorem 5 does, or does not, imply that your method could be successful would be greatly helpful.\n\n- On page 2 you write \"Regarding discrete SGLD, Lee et al. (2018) constructed a probability distribution whose density is close to a mixture of two well-separated isotropic Gaussians, and proved that SGLD could not find one of the two modes within an exponential number of steps.\" However, I was not able to find this lower bound in Lee et al. (2018). Which result specifically are you referring to?\n\n[1] Chen, S., Chewi, S., Li, J., Li, Y., Salim, A., and Zhang, A. R. (2023). Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In International Conference on Learning Representations\n\nLIMITATIONS:\nThe limitations of the work have been adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-20"}