SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Partition of Space. We divide the space according to the sub-level set. Specifically, we define  $S_{i} \triangleq \{x : p_{i}(x) \geq p_{j}(x), \forall j \neq i\}$ . One can easily verify that  $\cup_{i=1}^{n} S_{i} = \mathbb{R}^{d}$ . Furthermore,  $S_{i}$  is convex since by the definition of  $p_{i}$ , we have
$$
S _ {i} = \{x: (x - \mu_ {i}) ^ {\top} \pmb {\Sigma} ^ {- 1} (x - \mu_ {i}) \leq (x - \mu_ {j}) ^ {\top} \pmb {\Sigma} ^ {- 1} (x - \mu_ {j}), \forall j \neq i \}.
$$
As  $\Sigma$  is positive definite and symmetric, we can decompose  $\Sigma$  into  $UU$ , where  $U$  is also positive definite and symmetric. We then obtain
$$
U ^ {- 1} S _ {i} = \left\{x: \left(x - U ^ {- 1} \mu_ {i}\right) ^ {\top} \left(x - U ^ {- 1} \mu_ {i}\right) \leq \left(x - U ^ {- 1} \mu_ {j}\right) ^ {\top} \left(x - U ^ {- 1} \mu_ {j}\right), \forall j \neq i \right\},
$$
and thus  $U^{-1}S_{i}$  is one region of the Voronoi diagrams generated by  $\{U^{-1}\mu_i\}_{i = 1}^n$ , which is convex. As  $S_{i}$  can be obtained by performing linear transformation to  $U^{-1}S_{i}$ , we obtain that  $S_{i}$  is also convex.
Verification of local Logarithmic Sobolev Inequality. We prove the local Logarithmic Sobolev Inequality of  $\pi$  over each partition  $S_{i}$  as follows.
Lemma 3. For all  $i\in \{1,\dots ,n\}$ $\pi |S_{i}$  obeys LSI  $(\frac{\sigma^{-2}\min_{i\in[n]}w_i}{\max_{i\in[n]}w_i})$
Lemma 3 is proved by first showing  $p_i|S_i$  obeys LSI(  $\sigma^{-2}$  ) through Bakry-Émery criterion due to the convexity of  $S_i$  and  $p_i$  is strongly convex, and then applying Holley-Stroock perturbation principle by viewing  $\pi$  as a perturbation of  $p_i$  over  $S_i$ . The concrete proof is deferred to Appendix B.1.
Verification of Lipschitz gradient. By direct calculation, we derive the following bound on the Lipschitz constant of  $\nabla V$ .
Lemma 4.  $\forall x\in \mathbb{R}^d$ $\| \nabla^2 V(x)\| \leq \frac{\max_{i,j}\|\mu_i - \mu_j\|^2}{\sigma^4} +\sigma^{-2}.$
As a conclusion, we obtain the following guarantee of running LMC over this example.
Corollary 3. Let the stepsize of LMC be  $h = \Theta\left(\frac{1}{\sqrt{T}}\right)$ . Assume  $T > \Theta\left(\frac{d^2}{\varepsilon^4}\frac{\max_{i\in[n]}w_i^2}{\sigma^{-4}\min_{i\in[n]}w_i^2}\right)$ . Then, for every  $i\in [n]$ , either  $\bar{\pi}_{Th}(S_i)\leq \varepsilon$ , or  $\mathrm{Ent}_{\pi |S_i}\left[\frac{\bar{\pi}_{Th}|S_i}{\pi|S_i}\right]\leq \varepsilon$ .
Corollary 3 indicates that running LMC over gaussian mixture distribution (with the same covariance) can ensure local mixing with cost polynomial over the dimension and independent of the distance between the means. By contrast, it is a folk-tale that even the global mixing of gaussian mixture with 2 components has a global mixing rate exponential over the distance, which suggests local mixing analysis provide a more concrete analysis of what is going on over each component.
4.3.2 Sampling from the power posterior distribution of symmetric Gaussian mixtures
Target distribution and potential function. The symmetric Gaussian mixture model is given as
$$
f _ {\theta_ {0}} (x) \triangleq \frac {1}{2} \varphi (x; \theta_ {0}, \mathbb {I} _ {d}) + \frac {1}{2} \varphi (x; - \theta_ {0}, \mathbb {I} _ {d}).
$$
Here  $\varphi(x; \theta_0, \mathbb{I}_d)$  denotes the multivariate Gaussian distribution with location parameter  $\theta_0 \in \mathbb{R}^d$  and covariance matrix  $\mathbb{I}_d$ . Without loss of generality, we assume  $\theta_0 \in \operatorname{span}\{e_1\}$ , i.e., all coordinates of  $\theta_0$  except the first is zero, and  $\theta_0 = \| \theta_0 \| e_1$ . The power posterior distribution, or the target distribution, is defined as  $\pi_{n,\beta/n}(\theta \mid \{X_i\}_{i=1}^n) \coloneqq \frac{\prod_{i=1}^n (f_\theta(X_i))^{\beta/n} \lambda(\theta)}{\int \prod_{i=1}^n (f_u(X_i))^{\beta/n} \lambda(u) du}$ . We set  $\lambda \equiv 1$  for simplicity. When no ambiguity is possible, we abbreviate  $\pi_{n,\beta/n}(\theta \mid \{X_i\}_{i=1}^n)$  as  $\pi(\theta)$ . The potential function is then given as  $V(\theta) \coloneqq \frac{\beta}{n} \sum_{i=1}^n \log \left( \frac{1}{2} \varphi(\theta - X_i) + \frac{1}{2} \varphi(\theta + X_i) \right) + \log \lambda(\theta)$ .
Partition of Space. With an accuracy budget  $\varepsilon$ , we define  $R_{1} = [0, A] \times \mathcal{B}(0, M)$ ,  $R_{2} = [-A, 0] \times \mathcal{B}(0, M)$ , and  $R_{3} = (R_{1} \cup R_{2})^{c}$ , where  $A$  and  $M$  are  $\varepsilon$ -dependent hyperparameter specified latter.
Verification of local Poincaré Inequality over  $R_{1}$  and  $R_{2}$ . We have the following characterization for the local Poincaré Inequality over  $R_{1}$  and  $R_{2}$ :
Lemma 5. If  $A, M \geq \| \theta_0\| + 1$  and  $n \geq \tilde{\Theta}((A + M)^2 d\log(1/\delta))$ , then with probability at least  $1 - \delta$  with respect to the sampling of  $\{X_i\}_{i=1}^n$ , we have that  $\pi$  obeys  $\mathrm{PI}_{R_1}(\Theta(1/(A^4M^2)))$  and  $\mathrm{PI}_{R_2}(\Theta(1/(A^4M^2)))$ .
The lemma is derived by first considering the distribution  $\bar{\pi}$  corresponding to the potential function  $\bar{V} = \mathbb{E}V$  and proving local Poincaré Inequality of  $\bar{\pi}|R_1$  (or  $\bar{\pi}|R_2$ ), then bounding the difference between  $\bar{V}$  and  $V$  through concentration inequality, and finally completing the proof by applying Holley-Stroock perturbation principle to pass local Poincaré Inequality from  $\bar{\pi}|R_1$  to  $\pi|R_1$ . A concrete proof is deferred to Appendix B.2.
Verification of strong dissipativity. Similar to local Logarithmic Sobolev Inequality, we can show strong dissipativity of  $V$  holds in high probability.
Lemma 6. If  $n > \tilde{\Theta}((d + \| \theta_0\|^2)\log(1 / \delta))$ , then with probability at least  $1 - \delta$  over the sampling of  $\{X_i\}_{i=1}^n$ ,  $\|\nabla^2 V(\theta)\| \leq \mathcal{O}(1 + \| \theta_0\|^2)$ ,  $\langle \nabla V(\theta), \theta \rangle \geq \Omega(\|\theta\|^2) - \mathcal{O}\left(\|\theta_0\|^2 + 1\right)$ , and  $\|\nabla^3 V(\theta)\| \leq \mathcal{O}(d)$ .
Bounding the probability of  $R_{3}$ . Using strong dissipativity, we can bound  $\pi_t(R_3)$  as follows.
Lemma 7. If  $n > \tilde{\Theta}((d + \| \theta_0 \|^2) \log(1 / \delta))$ , then with probability at least  $1 - \delta$  over the sampling of  $\{X_i\}_{i=1}^n$ , we have  $\pi_t(R_3) \leq 32e^{\frac{16\beta(\|\theta_0\|^2 + 1) + 6d - \min\{A,M\}^2}{2\beta}}$ .
All in all, we obtain the following characterization of running LMC over this example.
Corollary 4. Initialize  $z_0 \sim \mathcal{N}(0, \sigma^2\mathbb{I})$  for  $\sigma^2 \leq \frac{1}{1 + L}$  and select  $h = \tilde{\Theta}\left(\frac{1}{T^{\frac{3}{2}}}\right)$ . Set  $A = M = \Theta(d + \log(1 / \varepsilon))$ . If  $T = \Omega(d^2 / \varepsilon^{12})$  and  $n > \tilde{\Theta}((d + \| \theta_0\|^2)\log(1 / \delta))$ , then with probability at least  $1 - \delta$  over the sampling of  $\{X_i\}_{i=1}^n$ , either  $\bar{\pi}_{Th}(R_i) \leq \varepsilon$  or  $D_{TV}[\bar{\pi}_{Th}|R_i||\pi|R_i] \leq \varepsilon$  for  $i \in \{1,2\}$  and  $\bar{\pi}_{Th}(R_3) \leq \varepsilon$ .
As a comparison, such a problem was also studied by [20], where they constructed a specific sampling algorithm with prior knowledge of this problem to achieve global convergence. In contrast, we analyze LMC, which does not require any prior knowledge of the problem, and derive the conditional convergence of LMC.
5 Conditional Mixing for Gibbs Sampling on Finite States
In previous sections, we showed that conditional mixing can happen for LMC on a continuous state space. We now show that similar results hold for MCMC algorithms on a finite state space. For simplicity, we consider an energy function  $f: \{0,1\}^d \to \{0,1,2,\dots,M\} =: [M]$  defined on the vertices of a hypercube. Denote its corresponding Gibbs measure  $\pi(x) \propto e^{-f(x)}$ .
We consider vertices of the hypercube as a d-regular graph where for any  $x, y \in \{0,1\}^d$ , an edge exists  $x \sim y$  if and only if they differ by one coordinate,  $d_{\mathrm{Hamming}}(x,y) = 1$ . Then a lazy Gibbs sampler has the following transition matrix on this finite graph:
$$
p (x, y) = \left\{ \begin{array}{l l} \frac {1}{2 d} \frac {\pi (y)}{\pi (y) + \pi (x)}, & y \sim x, \\ 1 - \frac {1}{2 d} \sum_ {x ^ {\prime}, \text {s . t .} x ^ {\prime} \sim x} \frac {\pi (x ^ {\prime})}{\pi (x ^ {\prime}) + \pi (x)}, & y = x, \\ 0, & \text {o t h e r w i s e}. \end{array} \right.
$$
Note that the process is lazy because,  $p(x,x) \geq 1 / 2$  for any  $x$ . This is assumed for simplicity of analysis to avoid almost periodic behaviors. This assumption does not make the analysis less general, as a lazy self loop with probability  $1 / 2$  only changes the mixing time by a multiplicative absolute constant (see Corollary 9.5 [22]).
To prove conditional convergence, we need an analogue of conditional Poincaré Inequality. The story for the discrete state space can be more convoluted as the transition matrix, in addition to the stationary distribution, plays a role here. Many of the analyses below are inspired by [13].
First, given a finite number of subsets  $\{\mathcal{X}_i\}_{i\leq m}$ , we define the conditional probability  $\pi_i = \pi |\mathcal{X}_i$  supported on  $\mathcal{X}_i$ , and hence for  $w_{i} = \pi (\mathcal{X}_{i})$ ,
$$
\pi (x) = \sum_ {i = 1} ^ {m} w _ {i} \pi_ {i} (x) \mathbb {1} \left\{x \in \mathcal {X} _ {i} \right\}.
$$
We also need to design a conditioned transition kernel so that  $\forall x, y \in \mathcal{X}_i$ ,
$$
p _ {i} (x, y) = p (x, y) + \mathbb {1} \{x = y \} P \left(x, \mathcal {X} _ {i} ^ {c}\right), \tag {3}
$$
where  $\mathcal{X}_i^c$  denotes the complement of  $\mathcal{X}_i$ , and hence the conditioned kernel simply rejects all outgoing transitions. Then we can easily tell that  $p_i$  is reversible with a unique stationary distribution  $\pi_i$ . We are now ready to give an analogue of Corollary 1.
Theorem 1. Given a sequence of subsets  $\{\mathcal{X}_i\}_{i\leq m}$ . If for every  $i$ ,  $P_{i}$  defined in (3) as a distribution on  $\mathcal{X}_i$  has spectral gap at least  $\alpha$ , then we have that either for some  $t$ , the distribution  $\mu_t$  has small probability on  $\mathcal{X}_i$ ,  $\mu_t(\mathcal{X}_i)\leq \pi (\mathcal{X}_i)T^{-1 / 4}$ , or the conditional mixing happens with respect to Chi-squared divergence,
$$
\frac {1}{T} \sum_ {t} \mathrm {V a r} _ {\pi_ {i}} \left[ \frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}} \right] \leq \frac {1}{\alpha \sqrt {T}} \mathrm {V a r} _ {\pi} \left[ \frac {\mu_ {0}}{\pi} \right].
$$
The proof builds upon the convergence of the Dirichlet form and can be found in Appendix C. The above theorem suggests that if the spectral gap of the Gibbs sampling algorithm is lower bounded on a local subset, then after a polynomial number of iterations, we get either  $\mu_t$  has very small probability on this set, or conditioned on the set, the distribution is close to the stationary distribution.
One thing less clear, in finite-state Gibbs sampling as compared to the LMC, is when would the spectral gap for a Gibbs distribution be large. For the Langevin Monte Carlo algorithm, classical results show that the spectral gap cannot be too small if the stationary distribution is locally near log-concave. Below we provide an analogue of this observation for discrete state space.
5.1 Spectral Gap for Gibbs Sampling
We first define a graph  $\mathcal{G} = (V,E)$ , where  $V = \{0,1\}^d$ , and  $(x,y) \in E$  if and only if  $d_{\mathrm{Hamming}}(x,y) = 1$ . Then we define quasi-concavity in this case. Note that Gibbs sampling in each iteration can only move to a neighbor or stay at the current vertex. Then we introduce the counterpart of quasi-convexity on a function defined on vertices of graphs.
Definition 6. Let  $\mathcal{G} = (V', E')$  be a subgraph,  $V' \subseteq V$ ,  $E' = (x, y) \in E | x \in V'$ ,  $y \in V'$ . We say a function  $f: V \to [M]$  is quasi-convex with a radius  $D$  on a subgraph  $\mathcal{G} = (V', E')$ , if there exists a local minimum  $v* \in V'$  such that for any  $v \in V'$ , any shortest path  $v \to v_1 \to \ldots \to v^*$  from  $v \to v^*$  is of length at most  $D$ , and  $f$  is non-increasing along the path.
Before providing a guarantee for the spectral gap, we give two examples of quasi-convexity.
Example 5.1. If  $g: \mathbb{R}^+ \to \mathbb{R}$  is a quasi-convex function defined on positive reals. Then for a given  $x^* \in V' \subseteq V$  any function  $f(x) = g(a \cdot d(x, x^*) + b)$  is a quasi-convex function on the graph where  $a, b$  are reals and  $d(x, y)$  is the shortest path length from  $x$  to  $y$ .
Example 5.2. If on a subset  $V' \subseteq V$ ,  $f(x) = c^T x$  for some  $c \in \mathbb{R}^d$ ,  $x \in \{0,1\}^d$  (i.e.  $f$  is a linear function), then  $f$  is quasi-convex on the graph.
The next theorem states that for such a function, the spectral gap is polynomial in problem dimension  $d$  and the diameter of the region  $D$ .
Theorem 2. If a function  $f$  is quasi-convex with a radius  $D$  on a subset  $\mathcal{X}_i \subseteq \{0,1\}^d$ , then the conditional transition of Gibbs sampling defined in (3) has a spectral gap lower bounded by  $\frac{1}{16d^2D^2}$ .
The proof studies the conductance of the Markov chain and can be found in Appendix D. The above theorem suggests that although Gibbs sampling can mix very slowly, on any subset with a well connected local minimum, the conditional mixing to the stationary distribution can be fast. This concludes our analysis and we move on to verify our statements with experiments in the next section.
6.1 Observations on Gaussian Mixture
In this section, we conduct experiments to verify the theoretical results and compare global mixing versus conditional mixing for Gaussian mixture models. We take three Gaussian mixtures:  $\nu_{1} = 0.9\mathbb{N}_{1}(-10,1) + 0.1\mathbb{N}_{1}(10,1)$ ,  $\nu_{2} = 0.15\mathbb{N}_{1}(-5,1) + 0.15\mathbb{N}_{1}(-2.5,1) + 0.3\mathbb{N}_{1}(0,1) + 0.2\mathbb{N}_{1}(2.5,1) + 0.2\mathbb{N}_{1}(5,1)$ , and  $\nu_{3} = 0.4\mathbb{N}_{2}((-5,-5),I_{2}) + 0.4\mathbb{N}_{2}((5,5),I_{2}) + 0.1\mathbb{N}_{2}((-5,5),I_{2}) + 0.1\mathbb{N}_{2}((5,-5),I_{2})$  as our target distributions. We use Algorithm 1 as our sampling algorithm, and set step size  $h = 10^{-2}$ . The initial distributions are both uniform in a large enough range. We plot the sampling distribution after  $T = 500,5000,500$  rounds respectively in Figure 1a, 1b, and 1c, and plot the conditional and global KL divergence in Figure 1d, 1e, and 1f.
We make the following observations: (1) The global KL divergences of the sampling distributions of  $\nu_{1}$  and  $\nu_{3}$  decrease fast at first. Then they maintain at a constant level and never converge to 0. It is reasonable since  $\nu_{1},\nu_{3}$  have very bad LSI constants (exponential in the distance between means). Thus, by classic Langevin Danymics analysis results[23], global KL divergence would have an exponentially slow convergence rate. (2) Both the global and conditional divergences of the sampling distribution of  $\nu_{2}$  converge very fast. This is because dense Gaussian mixtures like  $\nu_{2}$  have good LSI constant. The KL values are noisy due to the limited calculation precision of KL divergence. (3) The conditional divergence of the sampling distribution of  $\nu_{2}$  converges faster than the global divergence because the LSI constant bound of  $\nu_{2}$  is much worse than the conclusion in Corollary 1, which means conditional convergence is faster than global convergence. (4) The conditional KL divergences of the sampling distributions of  $\nu_{1}$  and  $\nu_{3}$  converge to 0 very fast (the flat part is due to the limit of calculation precision), which could be seen as a verification of our theoretical result. (5) The sampling distributions of  $\nu_{1}$  and  $\nu_{3}$  after  $T$  iterations contain all of the Gaussian components in target distributions, the only difference between them is weight. Since learning the right weight and component will directly lead to global KL convergence, this observation could be seen as an example of the gap between global convergence and conditional convergence.
(a) sample distribution  $(\nu_{1})$
(d) KL divergence  $(\nu_{1})$
Figure 1: we plot the sampling distributions after  $T$  iterations and the KL divergences w.r.t  $t$
(b) sample distribution  $(\nu_{2})$
(e) KL divergence  $(\nu_{2})$
(c) sample distribution  $(\nu_{3})$
(f) KL divergence  $(\nu_{3})$
6.2 Observations on LMC with restarts
In the LMC analysis, We study the evolution of a distribution  $p_0$  that usually is an absolutely continuous distribution. However, in practice, a more common implementation is as below: one first randomly generates an initial point  $x_0$ , runs the sampling algorithm for  $T$  iterations, and then collects samples along a single trajectory. A gap between theory and practice here is that we always assume continuity and smoothness conditions on the initial distribution in theory, while in practice, we only generate a limited number of initial points (sometimes only one point) to run the sampling algorithm.
For log-concave sampling, this gap is usually negligible since the ergodicity of Langevin Dynamics guarantees that we could always capture the features of the target distribution. Thus, it's reasonable that there are plenty of works about the discretization error on the time scale, while we hardly pay attention to the approximation error of initial distribution. When it comes to non-log-concave sampling, this gap may become crucial. We conduct several experiments in Appendix E to verify this conjecture and show that LMC with restarts could empirically help to eliminate the gap and improve the convergence speed.
Our work examines sampling problems where the global mixing of an MCMC algorithm is slow. We show that in such cases, fast conditional mixing can be achieved on subsets where the target distribution has benign local structures. We make the above statements rigorous and provide polynomial-time guarantees for conditional mixing. We give several examples, such as the mixture of Gaussian and the power posterior to show that the benign local structure often exists despite the global mixing rate is exponentially slow.
Much remains to be done. Theoretically, whether faster convergence rates can be achieved or a lower bound exists remain unknown. Instantiating our analyses to more MCMC algorithms may also lead to new observations. More importantly, the implication of being able to sample efficiently from local distributions requires more careful analysis. This may lead to a new theoretical guarantee for problems with symmetry (such as permutation symmetry, sign symmetry, rotation invariance, etc) where all local minima are equally good and sampling from any mode suffices.
Jingzhao Zhang acknowledges support from Tsinghua University Initiative Scientific Research Program.
[1] S. Ahn, Y. Chen, and M. Welling. Distributed and adaptive darting monte carlo through regenerations. In International Conference on Artificial Intelligence and Statistics, 2013.
[2] D. Bakry and M. Émery. Diffusions hypercontractives. In Séminaire de Probabilités XIX 1983/84: Proceedings, pages 177-206. Springer, 2006.
[3] K. Balasubramanian, S. Chewi, M. A. Erdogdu, A. Salim, and S. Zhang. Towards a theory of non-log-concave sampling: first-order stationarity guarantees for Langevin monte carlo. In Conference on Learning Theory, pages 2896-2923. PMLR, 2022.
[4] X. Cheng and P. Bartlett. Convergence of Langevin mcmc in kl-divergence. In Algorithmic Learning Theory, pages 186-211. PMLR, 2018.
[5] S. Chewi, M. A. Erdogdu, M. B. Li, R. Shen, and M. Zhang. Analysis of Langevin monte carlo from poincaré to log-sobolev, 2021.
[6] A. S. Dalalyan. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society. Series B (Statistical Methodology), pages 651-676, 2017.
[7] A. S. Dalalyan and A. Karagulyan. User-friendly guarantees for the langevin monte carlo with inaccurate gradient. Stochastic Processes and their Applications, 129(12):5278-5311, 2019.
[8] A. S. Dalalyan and A. B. Tsybakov. Sparse regression learning by aggregation and Langevin monte-carlo. Journal of Computer and System Sciences, 78(5):1423-1443, 2012.
[9] A. Durmus, S. Majewski, and B. Miasojedow. Analysis of Langevin monte carlo via convex optimization. The Journal of Machine Learning Research, 20(1):2666-2711, 2019.
[10] A. Durmus and E. Moulines. Nonasymptotic convergence analysis for the unadjusted Langevin algorithm. 2017.
[11] M. A. Erdogdu and R. Hosseinzadeh. On the convergence of Langevin monte carlo: The interplay between tail growth and smoothness, 2020.
[12] M. A. Erdogdu, R. Hosseinzadeh, and S. Zhang. Convergence of Langevin monte carlo in chi-squared and rényi divergence. In International Conference on Artificial Intelligence and Statistics, pages 8151-8175. PMLR, 2022.
[13] Y. Guan and S. M. Krone. Small-world mcmc and convergence to multi-modal distributions: From slow mixing to fast mixing. 2007.
[14] A. T. Kalai and S. Vempala. Simulated annealing for convex optimization. Mathematics of Operations Research, 31(2):253-266, 2006.
[15] D. Kingma, T. Salimans, B. Poole, and J. Ho. Variational diffusion models. Advances in neural information processing systems, 34:21696-21707, 2021.
[16] S. Lan, J. Streets, and B. Shahbaba. Wormhole hamiltonian monte carlo, 2014.
[17] Y.-A. Ma, Y. Chen, C. Jin, N. Flammarion, and M. I. Jordan. Sampling can be faster than optimization. Proceedings of the National Academy of Sciences, 116(42):20881-20885, sep 2019.
[18] M. B. Majka, A. Mijatović, and L. Szpruch. Non-asymptotic bounds for sampling algorithms without log-concavity, 2019.
[19] W. Mou, N. Flammarion, M. J. Wainwright, and P. L. Bartlett. Improved bounds for discretization of Langevin diffusions: Near-optimal rates without convexity. Bernoulli, 28(3):1577-1601, 2022.
[20] W. Mou, N. Ho, M. J. Wainwright, P. L. Bartlett, and M. I. Jordan. Sampling for bayesian mixture models: Mcmc with polynomial-time mixing. arXiv preprint arXiv:1912.05153, 2019.
[21] A. Mousavi-Hosseini, T. Farghly, Y. He, K. Balasubramanian, and M. A. Erdogdu. Towards a complete analysis of langevin monte carlo: Beyond poincar\`e inequality. arXiv preprint arXiv:2303.03589, 2023.
[22] Y. Peres and P. Sousi. Mixing times are hitting times of large sets. Journal of Theoretical Probability, 28:488-519, 2015.
[23] S. Vempala and A. Wibisono. Rapid convergence of the unadjusted Langevin algorithm: Isoperimetry suffices. Advances in neural information processing systems, 32, 2019.
[24] S. S. Vempala. Recent progress and open problems in algorithmic convex geometry. In IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS 2010). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2010.
[25] A. Wibisono. Proximal Langevin algorithm: Rapid convergence under isoperimetry. arXiv preprint arXiv:1911.01469, 2019.
A Proofs of results in Section 4.2
A.1 Local LSI: Proof of Lemma 1
$$
\mu (S) \int_ {S} \left\| \nabla \frac {\mu | _ {S}}{\pi | _ {S}} (x) \right\| ^ {2} \frac {\pi | _ {\mathcal {X} _ {j}} (x) ^ {2}}{\mu | _ {\mathcal {X} _ {j}} (x)} \mathrm {d} x = \int_ {S} \left\| \nabla \frac {\mu}{\pi} (x) \right\| ^ {2} \frac {\pi (x) ^ {2}}{\mu (x)} \mathrm {d} x \leq \int_ {\mathcal {X}} \left\| \nabla \frac {\mu}{\pi} (x) \right\| ^ {2} \frac {\pi (x) ^ {2}}{\mu (x)} \mathrm {d} x,
$$
if  $\mu (S)\leq \sqrt{\frac{\varepsilon}{\alpha}}$  , the proof is finished. Otherwise, we have
$$
\alpha \operatorname {E n t} _ {\pi | _ {S}} \left[ \frac {\mu | _ {S}}{\pi | _ {S}} \right] \leq \int_ {S} \left\| \nabla \frac {\mu | _ {S}}{\pi | _ {S}} (x) \right\| ^ {2} \frac {\pi | _ {\mathcal {X} _ {j}} (x) ^ {2}}{\mu | _ {\mathcal {X} _ {j}} (x)} \mathrm {d} x \leq \sqrt {\varepsilon \alpha}.
$$
A.2 Local PI: Proof of Proposition 2
To begin with, we provide the proof of Lemma 2.
Proof of Lemma 2. Since