Title: Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions
Venue: NeurIPS 2023
Accepted: yes
Keywords: Sampling

MCMC

Conditional Mixing

Non-log-concave Distributions
Abstract: MCMC algorithms offer empirically efficient tools for sampling from a target distribution $\pi(x) \propto \exp(-V(x))$. However, on the theory side, MCMC algorithms suffer from slow mixing rate when $\pi(x)$ is non-log-concave. Our work examines this gap and shows that when Poincar\'e-style inequality holds on a subset $\mathcal{X}$ of the state space, the conditional distribution of MCMC iterates over $\mathcal{X}$ mixes fast to the true conditional distribution. This fast mixing guarantee can hold in cases when global mixing is provably slow. We formalize the statement and quantify the conditional mixing rate. We further show that conditional mixing can have interesting implications for sampling from mixtures of Gaussians, parameter estimation for Gaussian mixture models, and Gibbs-sampling with well-connected local minima.

Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions

Abstract

MCMC algorithms offer empirically efficient tools for sampling from a target distribution π ( x ) ∝ exp( -V ( x )) . However, on the theory side, MCMC algorithms suffer from slow mixing rate when π ( x ) is non-log-concave. Our work examines this gap and shows that when Poincaré-style inequality holds on a subset X of the state space, the conditional distribution of MCMC iterates over X mixes fast to the true conditional distribution. This fast mixing guarantee can hold in cases when global mixing is provably slow. We formalize the statement and quantify the conditional mixing rate. We further show that conditional mixing can have interesting implications for sampling from mixtures of Gaussians, parameter estimation for Gaussian mixture models and Gibbs-sampling with well-connected local minima.

1 Introduction

Sampling from a given target distribution of the form π ( x ) ∝ e -V ( x ) plays a central role in many machine learning problems, such as Bayesian inference, optimization, and generative modeling [9, 14, 15]. The Langevin MCMC algorithm in particular has received a lot of recent attention; it makes use of the first-order gradient information ∇ V ( x ) , and can be viewed as the sampling analog of gradient descent.

Langevin MCMC has been shown to converge quickly when π ( x ) is log-concave [6, 10]. More recently, similar guarantees have been established for p ( x ) satisfying weaker conditions such as log-Sobolev inequality (LSI) [23], for instance, π ( x ) ∝ e -V ( x ) can have a good LSI constant when V ( x ) is a small perturbation of a convex function. However, few guarantees exist for general nonlog-concave distributions. One simple example is when p ( x ) is a well-separated mixture of two Gaussian distributions; in this case, one verifies that Langevin MCMC mixes at a rate proportional to the inverse of the exponential of the separation distance.

Many important modern machine-learning problems are highly non-convex, one prominent class being functions arising from neural networks. Though finding the global minimum of a non-convex function can be difficult, gradient descent can often be shown to converge rather quickly to a local optimum [3]. This raises the important question:

What is the sampling analog of a local minimum? And how can we sample efficiently from such a minimum?

Bohan Wang ∗

USTC

Yusong Zhu ∗

In [3], authors provide a partial answer to this question by adopting the view of Langevin Diffusion as the gradient flow of KL divergence in probability space . Under this perspective, the gradient of KL divergence is given by the Fisher Information (FI). Authors of [3] show that LMC achieves ϵ error in FI in O (1 /ϵ 2 ) steps.

However, one crucial question remains unanswered: how does the local optimality of FI help us sample from non-convex distributions? Intuitively, small FI is useful for characterizing local convergence when π is multi-modal. Authors of [3] suggested this connection, but it remains unclear how local mixing is defined and how small FI can quantitatively lead to good local mixing . Thus motivated, one of the goals of this paper is to provide a useful interpretation of the FI bound.

To this end, we propose a rigorous quantitative measure of "local mixing". We also provide a more general notion of "stationary point" for the sampling problem. Under these definitions, we show that LMC can achieve ϵ error in our proposed "measure of local mixing", in polynomial time. Finally, we consider discrete-time analog of the aforementioned ideas, and prove local convergence for random walk on a hypercube. Below is a detailed description of our theoretical contributions.

1.1 Main Contributions

We define a notion of conditional convergence : Let X ⊆ Ω denote a subset of the state space. We study the convergence of π t |X to π |X , where π t denotes distributions of LMC iterates and π denotes the target distribution. This definition of convergence is much weaker than the standard global convergence, but in exchange, LMC can achieve fast conditional convergence in settings where global convergence is known to be exponentially slow.

We define local Logarithmic Sobolev Inequality and show how to combine it with existing results on the convergence of Fisher information to derive the conditional convergence of LMC.

When local Logarithmic Sobolev Inequality does not hold, we define local Poincaré Inequality and Poincaré Fisher information (which is an analogy of Fisher information). We show the convergence of Poincaré Fisher information assuming strong dissipativity and show the conditional convergence of LMC when local Poincaré Inequality is present.

To showcase the applications of our results, we respectively study sampling from Gaussian mixture model with the same covariance and sampling from the power posterior distribution of symmetric two-component Gaussian mixtures. The global isometric constants of these examples are exponential in dimension and may have slow global convergence. We show that the local isoperimetric constants of these examples are polynomial in dimension, and prove fast conditional convergence for these examples.

In Theorem 1, we consider an application of our result to Gibbs sampling on discrete state space. We show that fast conditional mixing happens when the spectral gap is not small. We further show in Theorem 2 that a subset has large spectral gap if it contains only one local minimum and is well connected.

2 Related Work

When the target distribution π is strongly log-concave, the entropy Ent π [ µ π ] is known to be strongly convex with respect to µ , and thus Langevin Dynamics converges exponentially fast [2]. Such a result is later extended to LMC [8, 4, 9, 7, 21]. Several works further loosen the assumption by using isoperimetric inequalities such as Logarithmic Sobolev Inequality and Poincaré Inequality instead of strong log-concavity [24, 5, 25, 11]. However, there are few existing works on general non-log-concave sampling. Recently, Balasubramanian [3] defines convergence in relative Fisher information as a kind of "weak convergence" for sampling and proves a polynomial guarantee in general non-log-concave case only assuming global Lipschitz condition. However, this paper doesn't give any rigorous statistical interpretation of this weak convergence; Majka et al. [18] and Erdogdu et al. [11] study the cases when the target distribution is non-log-concave but has some good tail growth or curvature; Ma et al. [17] analyze the situation when the target distribution is non-log-concave inside the region but log-concave outside. Although these works give strict proofs of polynomial time guarantee in their setting, their results only hold for a small branch of non-log-concave distributions. It is still hardly possible to obtain a polynomial guarantee in general

non-log-concave cases. Multimodality, as a special case of non-log-concavity, has attracted lots of attention due to its prevalence in applied science. Many modified versions of MCMC were proposed to try to tackle the sampling of these distributions, such as Darting MC [1], Wormhole HMC [16], and etc. However, these algorithms require explicit knowledge of the location of the modes.

3 Conditional Mixing for MCMC

In this section, we provide a formal definition of "conditional mixing". Specifically, let µ t denote the distribution of a Markov chain { Z t } t at time t . We assume that the Markov chain dynamics is reversible with a unique stationary distribution π . Existing analyses mostly focus on understanding the rate of convergence measured by d ( µ t , π ) where d is some probability distance.

However, unless the stationary distribution π satisfies certain restrictive properties (e.g., logconcavity), the rate of convergence can be exponentially slow in the problem dimension or the distribution moments even for simple distributions such as the mixture of Gaussians. For this reason, we consider a weaker notion of convergence below.

Definition 1 (Conditional mixing) . Given a distribution µ t supported on the state space Ω . We say µ t converges conditioned on set X ⊆ Ω with respect to the divergence d if

where we have the conditional distribution

For now we can think of the distance d as the total variation distance. Later we will discuss stronger divergence such as KL-divergence or Chi-squared divergence.

The focus of our work is on identifying several sufficient conditions for fast convergence, and quantitatively bounding the convergence rate under these conditions. We focus on two MCMC algorithms: the Langevin Monte Carlo in continuous space, and the Gibbs sampling algorithm in discrete space. We further discuss the implications of conditional convergence for the two algorithms.

4 Conditional Mixing for Langevin Monte Carlo

In this section, we study the conditional mixing of the Langevin Monte Carlo (LMC) algorithm. This section is organized as follows: in Subsection 4.1, we first introduce the Langevin Monte Carlo algorithm, Langevin Dynamics, functional inequalities, and the Fisher information; in Subsection 4.2, we provide our main results characterizing the conditional convergence of LMC; finally, in Subsection 4.3, we showcase two applications of our main results.

4.1 Preliminaries

Langevin Monte Carlo. We are interested in the convergence of Langevin Monte Carlo (LMC), which is a standard algorithm employed to sample from a target probability density π ∝ e -V : R d → R , where V is called the potential function . The pseudocode of LMC is given in Algorithm 1.

Algorithm 1 Langevin Monte Carlo

Input: Initial parameter z , potential function V , step size h , number of iteration T

1: Initialization z 0 ← z

2: For t = 0 → T :

3: Generate Gaussian random vector ξ t ∼ N (0 , I d ) √

4: Update z ( t +1) h ← z th -h ∇ V ( z th ) + 2 hξ t

5: EndFor

LMC can be viewed as a time-discretization of Langevin Dynamics (LD), described by the following stochastic differential equation:

We can interpolate LMC following the similar manner of LD as

One can easily observe that { Z kh } ∞ k =0 in Eq. (2) has the same joint distribution as { z kh } ∞ k =0 in Algorithm 1, and thus Eq. (2) is a continuous-time interpolation of Algorithm 1.

Poincaré Inequality & Logarithmic Sobolev Inequalities. The convergence of LMC does not necessarily hold for all potential functions, and quantitative bounds require specific conditions over the potential function V . Poincare Inequality (PI) and Logarithmic Sobolev Inequality (LSI) are two commonly used conditions in the analysis of LMC convergence. We present these below:

Definition 2 (Poincaré Inequality) . A probability measure π on R d satisfies the Poincaré inequality with constant ρ > 0 (abbreviated as PI( ρ ) ), if for all functions f : R d → R ,

Definition 3 (Logarithmic Sobolev Inequality) . Given a function f : R d → R + and a probability measure π on R d , define Ent π ( f ) ≜ ∫ f log f d π ( x ) -∫ f d π ( x ) ( log ∫ f d π ( x ) ) . We say that a distribution π on R d satisfies the Logarithmic Sobolev Inequality (abbreviated as LSI( α ) ) with some constant α , if for all functions f : R d → R + ,

Both PI and LSI can imply the convergence of LMC when the step size h is small. Specifically, denote π th as the distribution of z th in LMC (Algorithm 1) and ˜ π t as the distribution of Z t in Langevin Dynamics (Eq.(1)). We further denote π t as the distribution of Z t interpolating LMC. The left-hand-side of Eq.(PI) with f = ˜ π t π is then the derivative (w.r.t. time) of the entropy of f , i.e., Ent π [ ˜ π t π ] , and thus we directly establish the exponential convergence of Var p [ ˜ π t π ] . The convergence of LMC can then be induced by bounding the discretization error between LMC and Langevin Dynamics. The methodology is similar when we have LSI.

Fisher information. It is well-known that if V is either strongly convex or convex with bounded support, then it obeys both PI and LSI, and the convergence of LMC follows immediately according to the arguments above. However, real-world sampling problems usually have non-convex potential functions, and for these problems we may no longer have either PI or LSI. If we revisit the above methodology to establish the convergence of LMC under LSI, we find that we still have

and thus lim T →∞ min t ∈ [0 ,T ] ∫ ∥ ∥ ∇ ˜ π t π ( x ) ∥ ∥ 2 d π ( x ) = 0 . Following this methodology, [3] uses the considers a notion of convergence which is defined using Fisher Information FI( µ || π ) ≜ ∫ ∥∇ ln µ/π ∥ 2 d µ , which they use to analyze LMC convergence. We present the result of [3] for completeness:

Proposition 1 (Theorem 2, [3]) . Assume ∇ V is L -lipschitz. Then, for any step size h ∈ (0 , 1 6 L ) ,

4.2 Rates of convergence

4.2.1 Conditional mixing under local Logarithmic Sobolev Inequality

We first show that if the target distribution π obeys a local Logarithmic Sobolev Inequality (defined below), then convergence of Fisher information implies conditional convergence.

Definition 4 (Local Logarithmic Sobolev Inequality) . We say that a distribution π on R d satisfies the local Logarithmic Sobolev Inequality over a subset S ⊂ R d with some constant α (abbreviated as LSI S ( α ) ), if π | S satisfies LSI ( α ) .

Definition 4 characterizes the local property of one distribution. It is considerably weaker than global LSI (i.e., Definition 3) when S is convex, and recovers LSI when S = R d . The intuition is that when the distribution is multi-modal, it is quite possible that LSI does not hold (or hold but with an exponentially small constant). In contrast, we can reasonably expect local LSI hold within each mode. In Subsection 4.3, we will show that a Gaussian mixture model with the same covariance satisfies Local Logarithmic Sobolev Inequality. We show in the following lemma that, whenever we have an estimation on the Fisher information between µ and π , we can turn it to an estimation of the entropy between µ | S and π | S given LSI S ( α ) .

Lemma 1. Let S ⊂ X and assume that π obeys LSI S ( α ) . For any distribution µ such that FI( µ || π ) ≤ ε , we have that either µ ( S ) ≤ √ ε √ α , or Ent π | S [ µ | S π | S ] ≤ √ ε √ α .

We defer the proof of Lemma 1 to Appendix A.1. Intuitively, Lemma 1 states that if the distribution π satisfies local LSI over the region S and has small global Fisher information with respect to a distribution µ , one can expect that either the probability mass of µ over S is minor, or µ is close to π both conditional over S . In other words, µ is close to π conditional over S whenever µ has a moderate mass over S . Note that Proposition 1 has already guaranteed global FI of LMC with little assumption, which together with Lemma 1 leads to the following corollary:

Corollary 1. Assume ∇ V is L -lipschitz and S ⊂ X satisfies that π obeys LSI S ( α ) . Define ¯ π Th = ∫ Th 0 π t d t Th . Choosing step size h = 1 √ T , we have that either ¯ π Th ( S ) ≤ O ( √ d 4 √ Ent ( π 0 π ) √ α 4 √ T ) , or

Corollary 1 is one of our key results. Intuitively, it shows that LMC can achieve local mixing when only local LSI is assumed. As we discussed under Definition 4, we can reasonably expect that local LSI holds within each mode when the distribution is multi-modal, in which case Corollary 1 can be further applied to show local mixing with polynomial rate. Note here the dependence of the local mixing rate is polynomial over the dimension d , meaning that considering local mixing helps to get rid of the exponential dependence over the dimension d in the global mixing rate when studying multi-modal distributions.

4.2.2 Conditional Convergence under local Poincaré Inequality

We have established the conditional convergence of LMC under local Logarithmic Sobolev Inequality above. However, there are cases where even local LSI fails to hold. To tackle these cases and inspired by Poincaré Inequality, we introduce a weaker notion than local LSI called local Poincaré Inequality:

Definition 5 (Local Poincaré Inequality) . Adistribution π on R d satisfies the local Poincaré Inequality over a subset S ⊂ R d with constant ρ (abbreviated as PI S ( ρ ) ), if π | S satisfies PI( ρ ) .

As Poincaré Inequality is weaker than Logarithmic Sobolev Inequality, one can easily see that local Poincaré Inequality is also weaker than local Logarithmic Sobolev Inequality. Based on local Poincaré Inequality, we have the following Poincaré version of Lemma 1, which converts an estimation of Poincaré Fisher information to an estimation of chi-squared divergence of the conditional distributions.

Lemma 2. Define Poincaré Fisher information as PFI( µ || π ) ≜ ∫ ∥∇ ( µ/π ) ∥ 2 d µ . Let S ⊂ X and assume that π obeys PI S ( ρ ) . For any distribution µ such that PFI( µ || π ) ≤ ε , we have that either µ ( S ) 2 Var π | S [ µ | S π | S ] ≤ επ ( S ) ρ .

To derive the conditional convergence of LMC under local Poincaré inequality, we further need to bound Poincaré Fisher information of LMC. Such a result, however, does not exist in the literature to our best knowledge. In analogy to the analysis in [3], we develop the following lemma to control the Fisher information of LD (recall that ˜ π t is the distribution of Langevin Dynamics in time t ).

Proposition 2. Denote ¯ ˜ π Th = ∫ Th 0 ˜ π t d t Th . Then, we have the following estimation of the PFI between ¯ ˜ π Th and π .

The proof can be directly derived by taking the derivative of Var π [ ˜ π t π ] with respect to t . We defer the formal proof to Appendix A.2. Combining Lemma 2, Proposition 2, and recent advances in discrete error analysis of LD [19], we obtain the following result showing conditional convergence under local Poincaré inequality.

Corollary 2. Assume ∇ V and ∇ 2 V is L -lipschtiz, V satisfies strong dissipativity, i.e., ⟨∇ V ( x ) , x ⟩ ≥ m ∥ x ∥ 2 -b , and S ⊂ X satisfies that π | S obeys PI( α ) . Initialize z 0 ∼ N (0 , σ 2 I ) and select h = ˜ Θ ( Var π [ ˜ π 0 π ] 1 3 T 2 3 ρ 1 3 d 2 3 ) . Recall that ¯ π Th = ∫ Th 0 π t d t Th . Then either ¯ π Th ( S ) = O ( d 1 6 Var π [ ˜ π 0 π ] 1 6 ρ 1 6 T 1 12 ) , or

Corollary 2 shows that conditional mixing can be established when local PI holds, which is a weaker assumption than local LSI. However, the assumption of Corollary 2 is stronger than its LSI counterpart, and the mixing rate depends on Var π [ ˜ π 0 π ] instead of Ent ( ˜ π 0 π ) , which can be considerably large. These drawbacks are mainly due to that the dynamics of chi-square distance are much harder than that of KL divergence, even when global PI holds [12].

Extension to Rényi divergence. In some prior works, e.g. [5, 23], Rényi divergence is considered a natural 'measure' of convergence, due to the analytical simplicity of the resulting expressions. One may wonder if our methodology above can be extended to establish a local mixing rate of Rényi divergence. Unfortunately, there are technical difficulties which for now we have not find a way to tackle or bypass, which we list in Appendix A.3.

4.3 Applications

In this subsection, we apply Corollary 1 and Corollary 2 to two concrete examples to demonstrate their usage in obtaining local mixing rates. We start by analyzing the example of sampling from Gaussian Mixture Models with uniform covariance and then turn to the example of sampling from the power posterior distribution of symmetric two-component Gaussian mixtures.

4.3.1 Case Study: Sampling from Gaussian Mixture Model with the uniform covariance

Target distribution and potential function. We are interested in the gaussian mixture model formed by gaussians with the same variance but different means. Specifically, the target distribution is defined as π = ∑ n i =1 w i p i ∈ ∆( R d ) , where w i > 0 , ∑ n i =1 w i = 1 , p i ∼ N ( µ i , Σ ) and Σ ≻ σ 2 I d . The potential function is then defined as V ( x ) = -log( ∑ n i =1 w i p i ( x )) .

̸

Partition of Space. We divide the space according to the sub-level set. Specifically, we define S i ≜ { x : p i ( x ) ≥ p j ( x ) , ∀ j = i } . One can easily verify that ∪ n i =1 S i = R d . Furthermore, S i is convex since by the definition of p i , we have

̸

As Σ is positive definite and symmetric, we can decompose Σ into UU , where U is also positive definite and symmetric. We then obtain

̸

and thus U -1 S i is one region of the Voronoi diagrams generated by { U -1 µ i } n i =1 , which is convex. As S i can be obtained by performing linear transformation to U -1 S i , we obtain that S i is also convex.

Verification of local Logarithmic Sobolev Inequality. We prove the local Logarithmic Sobolev Inequality of π over each partition S i as follows.

Lemma 3 is proved by first showing p i | S i obeys LSI( σ -2 ) through Bakry-Émery criterion due to the convexity of S i and p i is strongly convex, and then applying Holley-Stroock perturbation principle by viewing π as a perturbation of p i over S i . The concrete proof is deferred to Appendix B.1.

Verification of Lipschitz gradient. By direct calculation, we derive the following bound on the Lipschitz constant of ∇ V .

As a conclusion, we obtain the following guarantee of running LMC over this example.

Corollary 3 indicates that running LMC over gaussian mixture distribution (with the same covariance) can ensure local mixing with cost polynomial over the dimension and independent of the distance between the means. By contrast, it is a folk-tale that even the global mixing of gaussian mixture with 2 components has a global mixing rate exponential over the distance, which suggests local mixing analysis provide a more concrete analysis of what is going on over each component.

4.3.2 Sampling from the power posterior distribution of symmetric Gaussian mixtures

Target distribution and potential function. The symmetric Gaussian mixture model is given as

Here φ ( x ; θ 0 , I d ) denotes the multivariate Gaussian distribution with location parameter θ 0 ∈ R d and covariance matrix I d . Without loss of generality, we assume θ 0 ∈ span { e 1 } , i.e., all coordinates of θ 0 except the first is zero, and θ 0 = ∥ θ 0 ∥ e 1 . The power posterior distribution, or the target distribution, is defined as π n,β/n ( θ | { X i } n i =1 ) := ∏ n i =1 ( f θ ( X i )) β/n λ ( θ ) ∫ ∏ n i =1 ( f u ( X i )) β/n λ ( u ) du . We set λ ≡ 1 for simplicity. When no ambiguity is possible, we abbreviate π n,β/n ( θ | { X i } n i =1 ) as π ( θ ) . The potential function is then given as V ( θ ) := β n ∑ n i =1 log ( 1 2 φ ( θ -X i ) + 1 2 φ ( θ + X i ) ) +log λ ( θ ) .

Partition of Space. With an accuracy budget ε , we define R 1 = [0 , A ] ×B (0 , M ) , R 2 = [ -A, 0] × B (0 , M ) , and R 3 = ( R 1 ∪ R 2 ) c , where A and M are ε -dependent hyperparameter specified latter.

Verification of local Poincaré Inequality over R 1 and R 2 . We have the following characterization for the local Poincaré Inequality over R 1 and R 2 :

Lemma 5. If A,M ≥ ∥ θ 0 ∥ +1 and n ≥ ˜ Θ(( A + M ) 2 d log(1 /δ )) , then with probability at least 1 -δ with respect to the sampling of { X i } n i =1 , we have that π obeys PI R 1 (Θ(1 / ( A 4 M 2 ))) and PI R 2 (Θ(1 / ( A 4 M 2 ))) .

The lemma is derived by first considering the distribution ¯ π corresponding to the potential function ¯ V = E V and proving local Poincaré Inequality of ¯ π | R 1 (or ¯ π | R 2 ), then bounding the difference between ¯ V and V through concentration inequality, and finally completing the proof by applying Holley-Stroock perturbation principle to pass local Poincaré Inequality from ¯ π | R 1 to π | R 1 . A concrete proof is deferred to Appendix B.2.

Verification of strong dissipativity. Similar to local Logarithmic Sobolev Inequality, we can show strong dissipativity of V holds in high probability.

Lemma 6. If n > ˜ Θ(( d + ∥ θ 0 ∥ 2 ) log(1 /δ )) , then with probability at least 1 -δ over the sampling of { X i } n i =1 , ∥∇ 2 V ( θ ) ∥ ≤ O (1+ ∥ θ 0 ∥ 2 ) , ⟨∇ V ( θ ) , θ ⟩ ≥ Ω( ∥ θ ∥ 2 ) -O ( ∥ θ 0 ∥ 2 +1 ) , and ∥∇ 3 V ( θ ) ∥ ≤ O ( d ) .

Bounding the probability of R 3 . Using strong dissipativity, we can bound π t ( R 3 ) as follows.

Lemma 7. If n > ˜ Θ(( d + ∥ θ 0 ∥ 2 ) log(1 /δ )) , then with probability at least 1 -δ over the sampling of { X i } n i =1 , we have π t ( R 3 ) ≤ 32 e 16 β ( ∥ θ 0 ∥ 2 +1 ) +6 d -min { A,M } 2 2 β .

All in all, we obtain the following characterization of running LMC over this example.

Corollary 4. Initialize z 0 ∼ N (0 , σ 2 I ) for σ 2 ≤ 1 1+ L and select h = ˜ Θ ( 1 T 2 3 ) . Set A = M = Θ( d +log(1 /ε )) . If T = Ω( d 2 /ε 12 ) and n > ˜ Θ(( d + ∥ θ 0 ∥ 2 ) log(1 /δ )) , then with probability at least 1 -δ over the sampling of { X i } n i =1 , either ¯ π Th ( R i ) ≤ ε or D TV [¯ π Th | R i || π | R i ] ≤ ε for i ∈ { 1 , 2 } , and ¯ π Th ( R 3 ) ≤ ε .

As a comparison, such a problem was also studied by [20], where they constructed a specific sampling algorithm with prior knowledge of this problem to achieve global convergence. In contrast, we analyze LMC, which does not require any prior knowledge of the problem, and derive the conditional convergence of LMC.

5 Conditional Mixing for Gibbs Sampling on Finite States

In previous sections, we showed that conditional mixing can happen for LMC on a continuous state space. We now show that similar results hold for MCMC algorithms on a finite state space. For simplicity, we consider an energy function f : { 0 , 1 } d → { 0 , 1 , 2 , ..., M } =: [ M ] defined on the vertices of a hypercube. Denote its corresponding Gibbs measure π ( x ) ∝ e -f ( x ) .

We consider vertices of the hypercube as a d-regular graph where for any x, y ∈ { 0 , 1 } d , an edge exists x ∼ y if and only if they differ by one coordinate, d Hamming ( x, y ) = 1 . Then a lazy Gibbs sampler has the following transition matrix on this finite graph:

Note that the process is lazy because, p ( x, x ) ≥ 1 / 2 for any x . This is assumed for simplicity of analysis to avoid almost periodic behaviors. This assumption does not make the analysis less general, as a lazy self loop with probability 1 / 2 only changes the mixing time by a multiplicative absolute constant (see Corollary 9.5 [22]).

To prove conditional convergence, we need an analogue of conditional Poincaré Inequality. The story for the discrete state space can be more convoluted as the transition matrix, in addition to the stationary distribution, plays a role here. Many of the analyses below are inspired by [13].

First, given a finite number of subsets {X i } i ≤ m , we define the conditional probability π i = π |X i supported on X i , and hence for w i = π ( X i ) ,

We also need to design a conditioned transition kernel so that ∀ x, y ∈ X i ,

where X c i denotes the complement of X i , and hence the conditioned kernel simply rejects all outgoing transitions. Then we can easily tell that p i is reversible with a unique stationary distribution π i . We are now ready to give an analogue of Corollary 1.

Theorem 1. Given a sequence of subsets {X i } i ≤ m . If for every i , P i defined in (3) as a distribution on X i has spectral gap at least α , then we have that either for some t , the distribution µ t has small probability on X i , µ t ( X i ) ≤ π ( X i ) T -1 / 4 , or the conditional mixing happens with respect to Chi-squared divergence,

The proof builds upon the convergence of the Dirichlet form and can be found in Appendix C. The above theorem suggests that if the spectral gap of the Gibbs sampling algorithm is lower bounded on a local subset, then after a polynomial number of iterations, we get either µ t has very small probability on this set, or conditioned on the set, the distribution is close to the stationary distribution.

One thing less clear, in finite-state Gibbs sampling as compared to the LMC, is when would the spectral gap for a Gibbs distribution be large. For the Langevin Monte Carlo algorithm, classical results show that the spectral gap cannot be too small if the stationary distribution is locally near log-concave. Below we provide an analogue of this observation for discrete state space.

5.1 Spectral Gap for Gibbs Sampling

We first define a graph G = ( V, E ) , where V = { 0 , 1 } d , and ( x, y ) ∈ E if and only if d Hamming ( x, y ) = 1 . Then we define quasi-concavity in this case. Note that Gibbs sampling in each iteration can only move to a neighbor or stay at the current vertex. Then we introduce the counterpart of quasi-convexity on a function defined on vertices of graphs.

Definition 6. Let G = ( V ′ , E ′ ) be a subgraph, V ′ ⊆ V, E ′ = ( x, y ) ∈ E | x ∈ V ′ , y ∈ V ′ . We say a function f : V → [ M ] is quasi-convex with a radius D on a subgraph G = ( V ′ , E ′ ) , if there exists a local minimum v ∗ ∈ V ′ such that for any v ∈ V ′ , any shortest path v → v 1 → ... → v ∗ from v → v ∗ is of length at most D , and f is non-increasing along the path.

Before providing a guarantee for the spectral gap, we give two examples of quasi-convexity.

Example 5.1. If g : ❘ + → ❘ is a quasi-convex function defined on positive reals. Then for a given x ∗ ∈ V ′ ⊆ V any function f ( x ) = g ( a · d ( x, x ∗ ) + b ) is a quasi-convex function on the graph where a, b are reals and d ( x, y ) is the shortest path length from x to y .

Example 5.2. If on a subset V ′ ⊆ V , f ( x ) = c T x for some c ∈ ❘ d , x ∈ { 0 , 1 } d (i.e. f is a linear function), then f is quasi-convex on the graph.

The next theorem states that for such a function, the spectral gap is polynomial in problem dimension d and the diameter of the region D .

Theorem 2. If a function f is quasi-convex with a radius D on a subset X i ⊆ { 0 , 1 } d , then the conditional transition of Gibbs sampling defined in (3) has a spectral gap lower bounded by 1 16 d 2 D 2 .

The proof studies the conductance of the Markov chain and can be found in Appendix D. The above theorem suggests that although Gibbs sampling can mix very slowly, on any subset with a well connected local minimum, the conditional mixing to the stationary distribution can be fast. This concludes our analysis and we move on to verify our statements with experiments in the next section.

6 Experiments

6.1 Observations on Gaussian Mixture

In this section, we conduct experiments to verify the theoretical results and compare global mixing versus conditional mixing for Gaussian mixture models. We take three Gaussian mixtures: ν 1 = 0 . 9 N 1 ( -10 , 1) + 0 . 1 N 1 (10 , 1) , ν 2 = 0 . 15 N 1 ( -5 , 1) + 0 . 15 N 1 ( -2 . 5 , 1) + 0 . 3 N 1 (0 , 1) + 0 . 2 N 1 (2 . 5 , 1) + 0 . 2 N 1 (5 , 1) , and ν 3 = 0 . 4 N 2 (( -5 , -5) , I 2 ) + 0 . 4 N 2 ((5 , 5) , I 2 ) + 0 . 1 N 2 (( -5 , 5) , I 2 ) + 0 . 1 N 2 ((5 , -5) , I 2 ) as our target distributions. We use Algorithm 1 as our sampling algorithm, and set step size h = 10 -2 . The initial distributions are both uniform in a large enough range. We plot the sampling distribution after T = 500 , 5000 , 500 rounds respectively in Figure 1a, 1b, and 1c, and plot the conditional and global KL divergence in Figure 1d, 1e, and 1f.

We make the following observations: (1) The global KL divergences of the sampling distributions of ν 1 and ν 3 decrease fast at first. Then they maintain at a constant level and never converge to 0. It is reasonable since ν 1 , ν 3 have very bad LSI constants (exponential in the distance between means). Thus, by classic Langevin Danymics analysis results[23], global KL divergence would have an exponentially slow convergence rate. (2) Both the global and conditional divergences of the sampling distribution of ν 2 converge very fast. This is because dense Gaussian mixtures like ν 2 have good LSI constant. The KL values are noisy due to the limited calculation precision of KL divergence. (3) The conditional divergence of the sampling distribution of ν 2 converges faster than the global divergence because the LSI constant bound of ν 2 is much worse than the conclusion in Corollary 1, which means conditional convergence is faster than global convergence. (4) The conditional KL divergences of the sampling distributions of ν 1 and ν 3 converge to 0 very fast (the flat part is due to the limit of calculation precision), which could be seen as a verification of our theoretical result. (5) The sampling distributions of ν 1 and ν 3 after T iterations contain all of the Gaussian components in target distributions, the only difference between them is weight. Since learning the right weight and component will directly lead to global KL convergence, this observation could be seen as an example of the gap between global convergence and conditional convergence.

(a) sample distribution(

ν

1

)

(b) sample distribution(

ν

2

)

(c) sample distribution(

ν

3

)

(d) KL divergence(

ν

1

)

(e) KL divergence(

ν

2

)

(f) KL divergence(

ν

3

)

sampling pdf

0.12

sampling pdf

0.3

target pdf

0.10

target pdf

5

0.08

>0.2

70.06

0

0.1

0.04

0.02

5

0.020

-10

0

10

20

0.00

-10

5

0

5

10

-5

0

5

X

X

X

4

global

(. dlld)7x)60l

2

conditioned on x<0

conditioned on x>0

((.d||d)>)60|

0

conditioned on -1.25 < × < 1.25

conditioned on-3.75<×<-1.25

(. dld)7y)60|

1

conditioned onx<0,y<0

conditioned on x>0,y >0

conditioned on x<0, y > 0

conditioned on 1.25 < x < 3.75

0

conditioned on x>0,y <0

conditioned on x>3.75

0

-2

4

-2

0

100

200

300

400

500

0

1000

2000300040005000

0

100

200

300

400

500

iteration

iteration

iteration

6.2 Observations on LMC with restarts

In the LMC analysis, We study the evolution of a distribution p 0 that usually is an absolutely continuous distribution. However, in practice, a more common implementation is as below: one first randomly generates an initial point x 0 , runs the sampling algorithm for T iterations, and then collects samples along a single trajectory. A gap between theory and practice here is that we always assume continuity and smoothness conditions on the initial distribution in theory, while in practice, we only generate a limited number of initial points (sometimes only one point) to run the sampling algorithm.

For log-concave sampling, this gap is usually negligible since the ergodicity of Langevin Dynamics guarantees that we could always capture the features of the target distribution. Thus, it's reasonable that there are plenty of works about the discretization error on the time scale, while we hardly pay attention to the approximation error of initial distribution. When it comes to non-log-concave sampling, this gap may become crucial. We conduct several experiments in Appendix E to verify this conjecture and show that LMC with restarts could empirically help to eliminate the gap and improve the convergence speed.

7 Conclusions

Our work examines sampling problems where the global mixing of an MCMC algorithm is slow. We show that in such cases, fast conditional mixing can be achieved on subsets where the target distribution has benign local structures. We make the above statements rigorous and provide polynomial-time guarantees for conditional mixing. We give several examples, such as the mixture of Gaussian and the power posterior to show that the benign local structure often exists despite the global mixing rate is exponentially slow.

Much remains to be done. Theoretically, whether faster convergence rates can be achieved or a lower bound exists remain unknown. Instantiating our analyses to more MCMC algorithms may also lead to new observations. More importantly, the implication of being able to sample efficiently from local distributions requires more careful analysis. This may lead to a new theoretical guarantee for problems with symmetry (such as permutation symmetry, sign symmetry, rotation invariance, etc) where all local minima are equally good and sampling from any mode suffices.

Acknowledgments and Disclosure of Funding

Jingzhao Zhang acknowledges support from Tsinghua University Initiative Scientific Research Program.

References

S. Ahn, Y. Chen, and M. Welling. Distributed and adaptive darting monte carlo through regenerations. In International Conference on Artificial Intelligence and Statistics , 2013.

D. Bakry and M. Émery. Diffusions hypercontractives. In Séminaire de Probabilités XIX 1983/84: Proceedings , pages 177-206. Springer, 2006.

K. Balasubramanian, S. Chewi, M. A. Erdogdu, A. Salim, and S. Zhang. Towards a theory of non-log-concave sampling: first-order stationarity guarantees for langevin monte carlo. In Conference on Learning Theory , pages 2896-2923. PMLR, 2022.

X. Cheng and P. Bartlett. Convergence of langevin mcmc in kl-divergence. In Algorithmic Learning Theory , pages 186-211. PMLR, 2018.

S. Chewi, M. A. Erdogdu, M. B. Li, R. Shen, and M. Zhang. Analysis of langevin monte carlo from poincaré to log-sobolev, 2021.

A. S. Dalalyan. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society. Series B (Statistical Methodology) , pages 651-676, 2017.

A. S. Dalalyan and A. Karagulyan. User-friendly guarantees for the langevin monte carlo with inaccurate gradient. Stochastic Processes and their Applications , 129(12):5278-5311, 2019.

A. S. Dalalyan and A. B. Tsybakov. Sparse regression learning by aggregation and langevin monte-carlo. Journal of Computer and System Sciences , 78(5):1423-1443, 2012.

A. Durmus, S. Majewski, and B. Miasojedow. Analysis of langevin monte carlo via convex optimization. The Journal of Machine Learning Research , 20(1):2666-2711, 2019.

A. Durmus and E. Moulines. Nonasymptotic convergence analysis for the unadjusted langevin algorithm. 2017.

M. A. Erdogdu and R. Hosseinzadeh. On the convergence of langevin monte carlo: The interplay between tail growth and smoothness, 2020.

M. A. Erdogdu, R. Hosseinzadeh, and S. Zhang. Convergence of langevin monte carlo in chi-squared and rényi divergence. In International Conference on Artificial Intelligence and Statistics , pages 8151-8175. PMLR, 2022.

Y. Guan and S. M. Krone. Small-world mcmc and convergence to multi-modal distributions: From slow mixing to fast mixing. 2007.

A. T. Kalai and S. Vempala. Simulated annealing for convex optimization. Mathematics of Operations Research , 31(2):253-266, 2006.

D. Kingma, T. Salimans, B. Poole, and J. Ho. Variational diffusion models. Advances in neural information processing systems , 34:21696-21707, 2021.

S. Lan, J. Streets, and B. Shahbaba. Wormhole hamiltonian monte carlo, 2014.

Y.-A. Ma, Y. Chen, C. Jin, N. Flammarion, and M. I. Jordan. Sampling can be faster than optimization. Proceedings of the National Academy of Sciences , 116(42):20881-20885, sep 2019.

M. B. Majka, A. Mijatovi´ c, and L. Szpruch. Non-asymptotic bounds for sampling algorithms without log-concavity, 2019.

W. Mou, N. Flammarion, M. J. Wainwright, and P. L. Bartlett. Improved bounds for discretization of langevin diffusions: Near-optimal rates without convexity. Bernoulli , 28(3):1577-1601, 2022.

W. Mou, N. Ho, M. J. Wainwright, P. L. Bartlett, and M. I. Jordan. Sampling for bayesian mixture models: Mcmc with polynomial-time mixing. arXiv preprint arXiv:1912.05153 , 2019.

A. Mousavi-Hosseini, T. Farghly, Y. He, K. Balasubramanian, and M. A. Erdogdu. Towards a complete analysis of langevin monte carlo: Beyond poincar \ 'e inequality. arXiv preprint arXiv:2303.03589 , 2023.

Y. Peres and P. Sousi. Mixing times are hitting times of large sets. Journal of Theoretical Probability , 28:488-519, 2015.

S. Vempala and A. Wibisono. Rapid convergence of the unadjusted langevin algorithm: Isoperimetry suffices. Advances in neural information processing systems , 32, 2019.

S. S. Vempala. Recent progress and open problems in algorithmic convex geometry. In IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS 2010) . Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2010.

A. Wibisono. Proximal langevin algorithm: Rapid convergence under isoperimetry. arXiv preprint arXiv:1911.01469 , 2019.

A Proofs of results in Section 4.2

A.1 Local LSI: Proof of Lemma 1

Proof. Since

if µ ( S ) ≤ √ ε α , the proof is finished. Otherwise, we have

The proof is completed.

A.2 Local PI: Proof of Proposition 2

To begin with, we provide the proof of Lemma 2.

Proof of Lemma 2. Since

The proof is completed by noticing that

We then prove Proposition 2.

Proof of Propsition 2. By applying the definition of LD, we have

Taking integration of both sides of the above equation then gives

The proof is then completed by noting that according to Hölder's inequality,

The proof is then completed.

We then restate the main result from [19], which provides a tight characterization of the discretization error between LMC and LD.

Proposition 3 (Theorem 2, [19]) . Assume ∇ V and ∇ 2 V is L -lipschtiz. Initialize z 0 according to Gaussian and select h = O (1 /L 1 ) . Then for t ∈ [0 , T h ] , KL(˜ π t || π t ) = O ( h 2 d 2 T ) .

It should be noticed that in the original version of [Theorem 2, [19]], the result is only stated for interger time, i.e., t = kh where k ∈ { 0 , · · · , T } . However, their proof also applies to non-integer time which leads to the above proposition.

We are now ready to prove Corollary 2.

Proof of Corollary 2. Combing Proposition 2 and Lemma 2, we have that

Using Pinsker's inequality, we further obtain

On the other hand, according to Proposition 3 and Pinsker's inequality, we obtain that for t ∈ [0 , T h ] ,

and thus

Due to the convexity of L 1 norm, we then obtain

Let A be a positive constant. According to Eq. (4), we have either ¯ ˜ π Th ( S ) ≤ A , or D TV [ ¯ ˜ π Th | S || π | S ] = O ( √ Var π [ ˜ π 0 π ] A √ Thρ ) . In the former case, combined with Eq. (5), we have that

In the latter case, we have that

and thus

The proof is completed by selecting h = Var π [ ˜ π 0 π ] 1 3 T 2 3 ρ 1 3 d 2 3 , and A = d 1 6 Var π [ ˜ π 0 π ] 1 6 ρ 1 6 T 1 12 .

A.3 Difficulty of extension to Rényi divergence

Given two distribution π and µ , Rényi divergence of q between them, i.e., R q ( µ ∥ π ) is defined as follows:

For simplicity, here we consider Langevin Dynamics defined as Eq. (1) with the distribution of time t denoted as ˜ π t . The measure of convergence is then defined as R q ( π t ∥ π ) , the derivative of which, according to (Lemma 6, [23]) is given as

where

If we would like to follow the routine of Lemma 1, we need to firstly lower bound G q ( π t || π ) F q ( π t || π ) by G q (( π t | S ) || ( π | S )) F q (( π t | S ) || ( π | S )) and secondly transfer G q (( π t | S ) || ( π | S )) F q (( π t | S ) || ( π | S )) back to R q (( π t | S ) ∥ ( π | S )) . The second step can be accomplished following the same routine as (Lemma 9,[24]) using local PI. However, we are still unaware of how to implement the first step. This is because although we have

which is similar to the analysis of local LSI, we also have

In other words, when restricting to S , both the denominator and the numerator of G q ( π t || π ) F q ( π t || π ) will get smaller, which makes G q ( π t || π ) F q ( π t || π ) no longer lower bounded by G q (( π t | S ) || ( π | S )) F q (( π t | S ) || ( π | S )) . We leave how to resolve this challenge as an interesting future work.

B Proof of Applications

B.1 Proof of gaussian-mixture

To start with, we recall Bakry-Émery criterion and Holley-Stroock perturbation principle.

Lemma 8 (Bakry-Émery criterion) . Let Ω ⊂ R n be convex and let H : Ω → R be a Hamiltonian with Gibbs measure µ ( x ) ∝ e -H ( x ) 1 Ω ( x ) and assume that ∇ 2 H ( x ) ≥ κ > 0 for all x ∈ supp ( µ ) . Then µ satisfies LSI( κ ) .

Lemma 9 (Holley-Stroock perturbation principle) . If p ∈ ∆(Ω) satisfies LSI( π t ) , and ψ : Ω → R satisfies m ≤ ψ ≤ M , where m,M > 0 . Then, q ∈ ∆(Ω) ∝ ψp satisfies LSI( m M π t ) .

We are now ready to prove Lemma 3.

Proof of Lemma 3. Since S i is convex, by applying Lemma 8, we obtain p i | S i is LSI( σ 2 ) . Let c = min i w i .

Meanwhile, we have over S j

̸

̸

Therefore, by Holley-Stroock perturbation principle, we have that p | S j satisfies LSI( 1 c σ -2 ) .

Proof of Lemma 4. Through direct calculation, we obtain

Then, for any a ∈ R d with ∥ a ∥ = 1 , we have

and

The proof is completed.

B.2 Proof of sampling from the power posterior distribution of symmetric Gaussian mixtures

To begin with, define the expected potential function ¯ V as

We further define the corresponding distribution as ¯ π ∝ e -¯ V .

The following lemmas will be needed in the proof.

Lemma 10 (Theorem 2, [20]) . If A,M ≥ ∥∥ θ 0 ∥∥ +1 , we have ¯ π | R 1 satisfies PI(Θ(1 /A M )) . Lemma 11 (Lemma 4, [20]) . If A,M ≥ ∥ θ 0 ∥ +1 , we have with probability at least 1 -δ ,

4 2

Lemma 12. Suppose V obeys strong dissipativity, i.e., ∇ V is L -lipschitz and ⟨∇ V ( x ) , x ⟩ ≥ m ∥ x ∥ 2 -b . Then, running LMC with h < 1 16 L and h ≤ 8 m 4 b +32 d , we have

Proof. Define f ( x ) = e 1 4 m ∥ x ∥ 2 . Assume wlog that ∇ V (0) = 0 . Then,

Let E k denote expectation wrt ξ k . Then

Using the fact that χ 2 variable is sub-exponential,

On the other hand, notice that ⟨ ξ k , x k ⟩ ∼ N (0 , ∥ x k ∥ 2 ) is a 1-dimensional gaussian random variable. It can be shown that

Combining the above,

Let E denote expectation wrt all randomness. Then

Suppose that x k is drawn from the invariant distribution under LMC. Then we know that E f ( x k ) = E f ( x k +1 ) . In this case,

Moving things around gives

We are now ready to prove the lemmas in the main text.

Proof of Lemma 5. Based on Lemma 11, if n ≥ ˜ Θ(( A + M ) 2 d log(1 /δ )) , we have with probability at least 1 -δ ,

As a result, we have ¯ π | R 1 π | R 1 ( θ ) = Θ(1) , and by Lemma 9 and Lemma 10, the proof is completed.

Proof of Lemma 6. To begin with, set ξ ∼ N (0 , I d ) , we have

We obtain

following a standard empirical process argument due to n ≥ ˜ Θ(( A + M ) 2 d log(1 /δ )) (see Lemma 6, [20] as an example).

Meanwhile, denote τ i = 1 if X i is sampled from N ( θ 0 , I d ) and τ i = -1 else-wise. We have

and thus ( β - ∥ θ 0 ∥ 2 β -β n ∑ n i =1 ∥ X i -τ i ∥ θ 0 ∥ e 1 ∥ 2 ) I d ≤ ∇ 2 V ( θ ) ≤ β I d . As X i -τ i ∥ θ 0 ∥ e 1 ∼ N (0 , I d ) , by concentration inequality of chi-square variable and since n ≥ ˜ Θ(( A + M ) 2 d log(1 /δ )) , we have with probability at least 1 -δ ,

The claim for ∥∇ 3 V ∥ can be calculated following the similar routine. The proof is completed.

Proof of Lemma 7. The claim directly follows by applying Markov's inequality to Lemma 12.

C Proof of Theorem 1

Proof. We consider the variational form of spectral gap. We first define the Dirichlet form and the variance

Then by the fact that P is lazy, and hence P = 1 2 ( I + ˆ P ) for some reversible transition ˆ P , we have

By nonnegativity E P ( ϕ, ϕ ) ≥ 0 , , we have

By the variational form of spectral gap,

Therefore, applying the Markov chain generated by P i we have that for any i ≤ m,

We then note that

̸

We note by (PI) that for any x, y ∈ X i , x = y, P ( x, y ) = P i ( x, y ) Therefore, the second term is zero. Hence,

Combining with (7) and (8) we get,

The claim then follows by discussing if for all t , µ t ( X i ) ≤ π ( X i ) T -1 / 4 , then

D Proof for Theorem 2

Proof. Given the subgraph V , we have that conditioned transition kernel is

We analyze the conductance on graph V ′ induced by p ′ ,

For any partition V 1 , V 2 of V ′ , without loss of generality assume the local min of f is in V 2 , v ∗ ∈ V 2 . For simplicity, for any x ∈ V 1 , we denote τ ( x ) be the last element in its shortest path to v ∗ . We note that

Denote τ ( V 1 ) be the set of such elements in V 1 . Then we have

Further denote τ ( x, r ) for some r ∈ { 0 , 1 , ..., d } as the set of elements at distance r from τ ( x ) along the descending shortest paths. We provide an illustrative figure in 2. Then we have that

We note by reversibility that

Further by the fact that the function along the path is descending, we get that

Therefore, when π ( V 1 ) ≤ 1 / 2 we have

Therefore,

Hence we have when π ( V 1 ) ≥ 1 / 2 we have

The theorem then follows by Cheeger's inequality.

E Additional Experiments

We still use π 3 in 6.1 as our target distribution. We initially set n = 1 , 10 , 2000 particles to run the LMC sampling algorithm respectively. We collect the locations of these particles after T = 1000 iterations as valid samples. The target distribution is shown in Figure 3a; empirical distributions using n = 1 , 10 , 2000 particles are shown in Figure 3b, 3c, and 3d.

We highlight two observations from the experiments. First when we use only one particle, it is always trapped in one Gaussian component and we never get samples from other modes. It has very bad global convergence, but still gets good conditional convergence: The convergence result conditioned

(a) target distribution

(b) 1 particle

(c) 10 particles

(d) 2000 particles

5

5

5

5

0

>

0

0

0

5

-5

0

5

-5

0

0

5

0

5

X

X

X

X

on the 1st quadrant is good, and the sample after T iterations has no probability distributed on the other 3 quadrants. Second , when the number of particles grows, the sample after T iterations has non-negligible probabilities distributed on all of the 4 quadrants, which means we could capture all of the Gaussian components. The results indicate that although restarting LMC may not directly lead to global convergence, it may help us capture the features in multi-modal distributions, which further implies we may empirically eliminate the "small probability mass" condition in Corollary 1 and mitigate the gap between sampling distribution and target distribution.

A broader implication could be on ensemble and stochastic averaging in neural network training, where ensemble follows the restart procedure where as stochastic averaging is closer to sampling from a single trajectory. Our theory and experiment suggest that the two can have very different distributions in the end.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new concept of conditional convergence of an algorithm (i.e. convergence of the distribution restricted to e.g. a local mode), and presents extension of recent results using e.g. Poincare Inequality to bound this new measure.\n\nSTRENGTHS:\nI think this paper tackles an important problem: sampling from multi-modal distributions is likely to get stuck in a local optima and so understanding how well it samples from that mode is sensible (particularly when thinking about how a sampling based approach compares to an optimisation one).\n\nThe idea and definition of conditional convergence is new and interesting -- and one could see a new body of research around this type of topic.\n\nThe theory is based on a single, simple results (Lemma 1) which can then be used to generalise existing results on convergence to conditional convergence.\n\nWEAKNESSES:\nThe main weakness is that the current result, e.g. Corollary 1, are weaker than one would hope. In particular \n\nCorollary 1 seems to have a time-dependent step-size (so to understand pi_T we need a smaller step size as T increases). This is different from how algorithms are implemented.\n\nThe conclusion after corollary 1 does not make sense to me. “Either the probability mass … over S converges to 0, or…” : But whilst LMC can converge slowly, it still converges to pi (if we ignore the discretisation error). Thus surely for any set S that is not a null-set of pi we have pi_T(S) will be non-zero in the limit as T goes to infinity?\n\nMore generally, this corollary is quite the result you want. You want something that says “if I have a partition of the the state-space in sets S_1,S_2,\\ldots, and observe that the proportion of time LMC stays in S_i is greater than … then the LMC output restricted to S_i will be close to pi restricted to S_i” or similar. \n\nSo I struggle to really understand the practical importance of the result.\n\nSeparately, the presentation could be improved in places: \n\nE.g. missing “the”s for *the* local Poincare inequality, and *the* Poincare Figure information (and e.g. “fisher”->”Fisher”, gaussian to Gaussian etc.); also “Gibbs sampling on discrete state *spaces*” etc.\n\nWhen describing Langevin Monte Carlo, there are two different approaches depending on whether you use a Metropolis correction. The paper could be clearer about what its results relate to. Form the definition in Algorithm 1 it looks like you are considering the unadjusted Langevin algorithm (ULA).\n\nIf you are using ULA, then there is a discretisation error — i.e. the stationary distribution for ULA is different to that of the Langevin diffusion/the distribution you want to sample from. I think the paper could be up-front about this, and discuss how this impacts the results in the paper. I also wonder if informal statements such as “The convergence of LMC” (top p.4) are really referring to the convergence of the Langevin diffusion and not ULA. The paper does comment on “PI .. implies the convergence of LMC when the step size h is small”, but this could be more precise. I guess part of the confusion is that you are commenting on converging, but it is not completely clear what you are converging to. In general the ULA will not converge to pi for a fixed (albeit small) step-size as the discretisation error will change the stationary distribution.\n\nThe argument under Fisher information seems to ignore the discretisation error — i.e. be just for the Langevin diffusion, but is says for LMC. (Though this is corrected in Proposition 1 which is for LMC). This part could have been improved with some references to the results you are summarising.\n\n[It may be that a cleaner presentation would be to have had a subsection commenting on the approximation error between ULA and the Langevin diffusion, and then presenting the results just for the Langevin diffusion?]\n\nQUESTIONS:\nCan you more convincingly explain the practical importance of Corollary 1? \n\nAs part of this, it feels like the rate of convergence matters (as we have convergence of LMC in general, just that it can be very slow). The slowness of LMC is, I believe, linked to e.g. constants for convergence depending on features of the target (more separated modes implies slower convergence) -- so it feels like you want stronger results than Corollary 1 where you get a handle on the constants. I.e. are the results in Corollary 1 uniform over targets?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies MCMC algorithms like the Langevin dynamics and Gibbs sampler on non-log-concave distributions. Many natural distributions are non-log-concave and multimodal, for example, mixtures of Gaussians and the posterior distribution of Gaussian mixtures. While classical results show that MCMC algorithms suffer from slow mixing on such multimodal distributions, the paper shows that when isoperimetric inequalities such as Poincare or log-Sobolev hold on a subset X of the state space, the conditional distribution of the MCMC iterate over X mixes fast to the conditional distribution of the target distribution on X. Thus, the paper shows that while MCMC algorithms converge to the true global distribution slowly, it can still converge very fast locally. For example, on a mixture of two isomorphic Gaussians, the Langevin dynamics (LMC) converges to the true conditional distribution around each mode but might put the wrong weight on the two Gaussian components. i.e. the true distribution is mu = 1/2 N(-u, sigma^2) +1/2 N (u, sigma^2) but the distribution of the LMC might be 1/3 N(-u, sigma^2) + 2/3 N(u,sigma^2). To show these results, the paper uses that for any target distribution mu, including non-log-concave ones, the LMC quickly converges to a distribution nu whose Fisher information to mu is small [Balasubramanian, Chewi, Erdogdu, Salim, Zhang—PMLR’22], then uses isoperimetric inequalities for the conditional distribution of mu on subsets of the state space to show that if the conditional distribution of mu on S satisfies isoperimetric inequality, then either nu puts small mass on S or the conditional distribution of nu on S is close to that of mu on S in Kullback-Leiber or chi-square distance (see Lemma 1, Corollary 1 for the case when the conditional distributions satisfy log-Sobolev inequalities, and Lemma 2, Theorem 2 and Corollary 2 for the case when the conditional distributions satisfy the weaker Poincare inequalities). Note that this is essentially the best statement one can hope for: for example, if mu is a mixture of two Gaussians whose centers are very far apart, and the LMC is initialized at the center of the first Gaussian, then the LMC will put almost all mass on the first Gaussian component and almost 0 mass on the second Gaussian component and one cannot have any reasonable guarantee about the conditional distribution of the LMC iterate on regions around the mode of the second component. In Theorem 2, the paper proves an analogous result for Gibbs sampler on discrete state space such as the hypercube.  For a distribution mu supported on the hypercube {0,1}^d, the Gibbs sampler operates by picking one random coordinate and flipping the value at that coordinate from 0 to 1 or 1 to 0 with probabilities chosen so that mu is the stationary distribution (see Section 5). The paper shows that if the vertices of the hypercube can be partitioned into subsets X_1,.., X_m such that on each subset, the Markov chain induced by the Gibbs sampler has a large spectral gap, then either the distribution nu produced by the Gibbs sampler puts small mass on X_i or the conditional of nu on X_ is close to that of mu on X_i.\n\nSTRENGTHS:\nThe paper shows an interesting result using relatively simple techniques. While fast convergence of LMC iterates in Fisher information for general non-log-concave distribution and local isoperimetric inequalities are known and used in previous works (see [Balasubramanian, Chewi, Erdogdu, Salim, Zhang—PMLR’22] and [Mou, Ho, Wainwright, Bartlett, Jordan’2019]), the paper cleverly combines these two ingredients together to show local mixing of familiar MCMC algorithms likThe paper shows an interesting result using relatively simple techniques. While fast convergence of LMC iterates in Fisher information for general non-log-concave distribution and local isoperimetric inequalities are known and used in previous works (see [Balasubramanian, Chewi, Erdogdu, Salim, Zhang—PMLR’22] and [Mou, Ho, Wainwright, Bartlett, Jordan’2019]), the paper cleverly combines these two ingredients together to show local mixing of familiar MCMC algorithms like the LMC and Gibbs sampler on multimodal distributions. The main results appear to be novel and correct. I verify the proof of Lemma 1, Corollary 1, and Lemma 3.e the LMC and Gibbs sampler on multimodal distributions.\n\nWEAKNESSES:\nThe result on mixtures of Gaussians requires the assumption that the Gaussian components share the same covariance. It’s unclear if this assumption is natural, and the author(s) don’t give any justification for this assumption.\nThe paper has a few typos. Details below.\n- Proof of Lemma 1, supplement, appendix A: The claim is either $\\mu(S) \\leq sqrt{\\epsilon}/sqrt{\\alpha}$ or $Ent_{\\pi|S}(\\mu|S || \\nu|S )\\leq sqrt{\\epsilon}/sqrt{\\alpha}$ but the proof instead show that either $\\mu(S) \\leq \\sqrt{\\epsilon}$ or $Ent_{\\pi|S}(\\mu|S || \\nu|S )\\leq sqrt{\\epsilon}/\\alpha.$ The fix is simple, in line 416, it should be $\\mu(S)\\leq \\sqrt{\\epsilon}/\\sqrt{\\alpha}$ and in line the following displayed equation, the rhs should be $\\epsilon/(\\sqrt{\\epsilon}/\\sqrt{\\alpha}) = \\sqrt{\\epsilon \\alpha}.$\n- In proof of Lemma 3, supplement, appendix C.1: c is undefined, though I believe c = min_i w_i so that c\\leq w_j and cp_j \\leq w_j p_j holds in the first line of the proof. The proof, Lemma 3 only proves that p |S_j satisfies LSI with constant 1/min_i w_i sigma^{-2},  but lemma 3 claims that the LSI constant is max_i w_i/min_i w_i sigma^{-2}. However, 1 \\leq m max_i w_i where m is the number of components/parts in the partition of the state space, so the bound the proof achieves is only worse than the claimed bound by a factor of m, which doesn’t significantly affect the result.\n-In the statement of Lemma 2, in both the main paper and supplement, pi satisfies PFI -> pi satisfies PI.\n-In section 4.3.1 of the main paper, line 210, P is undefined, but I believe the author(s) mean U.\n-In Lemma 5, lines 239 and 240 of the main paper, the author claims that the target distribution satisfies local LSI, but its proof (Lemma 13, line 467 of the supplement) states that the target distribution only satisfies local Poincare inequalities.\n\nQUESTIONS:\n-For mixtures of Gaussians, is the assumption that Gaussian components have the same covariance matrix necessary? Is this assumption natural?\n-The quasi-concave condition for the Gibbs sampler to have a large spectral gap appears rather unnatural. The author(s) could consider investigating natural multimodal discrete distributions such as the Ising/Curie-Weiss model at low temperatures (see [Levin-Luczak-Peres—Probability and Related Fields’2010]).\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

comments: SUMMARY:
This work studies the convergence of MCMC algorithms for sampling from non-log-concave distributions. This is much less well-understood than the log-concave setting. The authors introduce the notion of conditional mixing, this occurs when the markov chain is close to the true (conditional) distribution when conditioned on being in some specific part of the space. They give sufficient conditions under which conditional mixing occurs. They give applications to sampling from mixture distributions of gaussians and related problems.

STRENGTHS:
The authors propose a new framework to go beyond log-concave sampling, an important problem in the sampling literature and give evidence for its utility. They show that in some cases conditional mixing can appear very quickly, whereas global mixing provably takes much longer.

The paper is overall well-written.

WEAKNESSES:
I think non-expert readers could benefit from a high-level technical overview.

QUESTIONS:
Regarding sampling from a Gaussian mixture distribution: Could you mention where your techniques break down when allowing differing covariances for the mixture distributions?

LIMITATIONS:
Maybe I missed this, but I do not think limitations were mentioned.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper, following the framework of Balasubramanian et al., shows that for target distributions that are non-log-concave, isoperimetric inequalities on subsets of the state space will yield fast mixing for the conditional distributions of MCMC on that space. This adds formal justification for the observed phenomena of metastability and “local convergence”. Additionally, some analogous conditional convergence results are also reported for a random walk on a hypercube.\n\nSTRENGTHS:\nThe definition and usage of these local isoperimetric conditions is entirely novel, as well as the local mixing results. In particular, the mixing rates seem to offer very good insight into the phenomenon of “metastability”, i.e. that particular modes may be well-explored while the global structure is not correct. \n\nThe paper is well illustrated with examples such as the Gaussian mixture and the posterior sampling example, with transparent computation of the constants. In particular, it is easy to see when the conditions of the paper hold, and to compute the resulting rate estimates.\n\nThe hypercube sampling result is novel, although it is difficult to assess this in the context of existing literature since the assumptions/results differ greatly.\n\nExperimental evidence is also provided to quantify the phenomena of local convergence.\n\nWEAKNESSES:\nThe rate of local mixing in both local LSI and local Poincare cases are quite bad. See my remark in the “Questions” section.\n\nThe discrete state space case is difficult to assess compared to other results for discrete space MCMC. If this is to be a main result in the paper, I would recommend a more detailed survey of the literature on discrete space MCMC with relevant comments be included in the Related Work.\n\nThe experiments are not surprising and seem to illustrate the same phenomena as seen in earlier works on multimodal sampling, but I would argue that they are still fairly useful illustrations in the context of this paper. In my opinion, the experiments section could be shortened.\n\nThis paper could benefit from some proofreading to catch grammatical mistakes, of which I was able to find quite a few.\n\nTo summarize, I feel that this paper makes unique contributions to the theory and intuition of MCMC algorithms, and the issues with it are relatively minor. Therefore I am happy to recommend acceptance.\n\nQUESTIONS:\nThe local mixing seems to be at a significantly worse rate than the global mixing under e.g. LSI, which we would expect to be something like $d/\\epsilon$ at least for the LSI case. On the other hand, we obtain something more like $d^2/\\epsilon^4$. Is there a principled reason for this difference and could the rate potentially be improved, or is the analysis expected to be somewhat sharp.\n\nIn some prior works, e.g. [6, 23], Renyi divergence is considered as a natural “measure” of convergence, due to the analytical simplicity of the resulting expressions. Would analogous results hold for this case? Furthermore, is there any barrier to considering e.g. Latala-Oleszkiewicz or other isoperimetries in the analysis? I would imagine so given that the “Fisher information” would now need to correspond to an inequality of LO-type.\n\nHow does the discrete sampling result compare to existing MCMC results in that field? See my comment in “Weaknesses”.\n\nTypos:\nFisher in Fisher information should always be capitalized\n\nL. 74 space after Balasubramanian\n\nL. 81 distribution -> distributions\n\nL. 82. Multimode -> multimodality\n\nL. 106 “the Langevin Monte Carlo algorithm”, function inequalities -> functional inequalities\n\nThe definition of a Lipschitz condition should be given explicitly.\n\nSome in-line equations are a bit difficult to read and should be presented as displays, e.g. L. 231 or L. 252.\n\nL. 286 “D” -> “Appendix D”\n\nLIMITATIONS:\nNone beyond those raised in earlier sections.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}