SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {\mu (S) ^ {2}}{\pi (S)} \int_ {S} \left\| \nabla \frac {\mu | _ {S}}{\pi | _ {S}} (x) \right\| ^ {2} \pi | _ {S} (x) \mathrm {d} x = \int_ {S} \left\| \nabla \frac {\mu}{\pi} (x) \right\| ^ {2} \pi (x) \mathrm {d} x \leq \int_ {\mathcal {X}} \left\| \nabla \frac {\mu}{\pi} (x) \right\| ^ {2} \pi (x) \mathrm {d} x.
$$
The proof is completed by noticing that
$$
\int_ {S} \left\| \nabla \frac {\mu | _ {S}}{\pi | _ {S}} (x) \right\| ^ {2} \pi | _ {S} (x) d x = \operatorname {P F I} (\mu | _ {S} | | \pi | _ {S}) \geq \rho \operatorname {V a r} _ {\pi | _ {S}} \left[ \frac {\mu | _ {S}}{\pi | _ {S}} \right].
$$
We then prove Proposition 2.
Proof of Proposition 2. By applying the definition of LD, we have
$$
\frac {\mathrm {d V a r} _ {\pi} \left[ \frac {\tilde {\pi} _ {t}}{\pi} \right]}{\mathrm {d} t} = - 2 \operatorname {P F I} \left(\tilde {\pi} _ {t} \| \pi\right).
$$
Taking integration of both sides of the above equation then gives
$$
\operatorname {V a r} _ {\pi} \left[ \frac {\tilde {\pi} _ {0}}{\pi} \right] \geq \operatorname {V a r} _ {\pi} \left[ \frac {\tilde {\pi} _ {0}}{\pi} \right] - \operatorname {V a r} _ {\pi} \left[ \frac {\tilde {\pi} _ {T}}{\pi} \right] = 2 \int_ {0} ^ {T h} \operatorname {P F I} (\tilde {\pi} _ {t} \| \pi)   \mathrm {d} t.
$$
The proof is then completed by noting that according to Hölder's inequality,
$$
\int_ {\mathcal {X}} \left\| \nabla \frac {\int_ {0} ^ {T h} \tilde {\pi} _ {t} \mathrm {d} t}{\pi} (x) \right\| ^ {2} \pi (x) \mathrm {d} x \leq \int_ {0} ^ {T h} \int_ {\mathcal {X}} \left\| \nabla \frac {\tilde {\pi} _ {t}}{\pi} (x) \right\| ^ {2} \pi (x) \mathrm {d} x \mathrm {d} t = \int_ {0} ^ {T h} \operatorname {P F I} (\tilde {\pi} _ {t} \| \pi) \mathrm {d} t.
$$
We then restate the main result from [19], which provides a tight characterization of the discretization error between LMC and LD.
Proposition 3 (Theorem 2, [19]). Assume  $\nabla V$  and  $\nabla^2 V$  is  $L$ -lipschitz. Initialize  $z_0$  according to Gaussian and select  $h = \mathcal{O}(1 / L_1)$ . Then for  $t \in [0, Th]$ ,  $\mathrm{KL}(\tilde{\pi}_t||\pi_t) = \mathcal{O}(h^2 d^2 T)$ .
It should be noticed that in the original version of [Theorem 2, [19]], the result is only stated for interger time, i.e.,  $t = kh$  where  $k \in \{0, \dots, T\}$ . However, their proof also applies to non-integer time which leads to the above proposition.
We are now ready to prove Corollary 2.
Proof of Corollary 2. Combing Proposition 2 and Lemma 2, we have that
$$
\bar {\tilde {\pi}} _ {T h} (S) ^ {2} \mathrm {V a r} _ {\pi | _ {S}} \left[ \frac {\bar {\tilde {\pi}} _ {T h} | _ {S}}{\pi | _ {S}} \right] \leq \frac {\pi (S) \mathrm {V a r} _ {\pi} [ \tilde {\pi} _ {0} ]}{2 T h \rho}.
$$
Using Pinsker's inequality, we further obtain
$$
2 \bar {\tilde {\pi}} _ {T h} (S) ^ {2} D _ {\mathrm {T V}} \left[ \bar {\tilde {\pi}} _ {T h} | _ {S} | | \pi | _ {S} \right] ^ {2} \leq \frac {\pi (S) \operatorname {V a r} _ {\pi} \left[ \frac {\tilde {\pi} _ {0}}{\pi} \right]}{2 T h \rho}. \tag {4}
$$
On the other hand, according to Proposition 3 and Pinsker's inequality, we obtain that for  $t \in [0, Th]$ ,
$$
D _ {\mathrm {T V}} \left[ \tilde {\pi} _ {t} | | \pi_ {t} \right] ^ {2} \leq \mathcal {O} \left(h ^ {2} d ^ {2} T\right),
$$
$$
D _ {\mathrm {T V}} \left[ \tilde {\pi} _ {t} | | \pi_ {t} \right] \leq \mathcal {O} (h d \sqrt {T}).
$$
Due to the convexity of  $L_{1}$  norm, we then obtain
$$
\begin{array}{l} D _ {\mathrm {T V}} \left[ \bar {\tilde {\pi}} _ {T h} | | \bar {\pi} _ {T h} \right] = \sup _ {A} \left| \frac {1}{T h} \int_ {0} ^ {T h} \tilde {\pi} _ {t} (A) - \frac {1}{T h} \int_ {0} ^ {T h} \pi_ {t} (A) \mathrm {d} t \right| \\ \leq \frac {1}{T h} \int_ {0} ^ {T h} \sup  _ {A} | \tilde {\pi} _ {t} (A) - \pi_ {t} (A) | d t \\ \leq \frac {1}{T h} \int_ {0} ^ {T h} D _ {\mathrm {T V}} \left[ \tilde {\pi} _ {t} \right| | \pi_ {t} ] \mathrm {d} t = \mathcal {O} (h d \sqrt {T}). \tag {5} \\ \end{array}
$$
Let  $A$  be a positive constant. According to Eq. (4), we have either  $\tilde{\bar{\pi}}_{Th}(S) \leq A$ , or  $D_{\mathrm{TV}}[\tilde{\bar{\pi}}_{Th}|_S||\pi|_S] = \mathcal{O}\left(\frac{\sqrt{\mathrm{Var}_{\pi}[\tilde{\pi}_0]}}{A\sqrt{Th\rho}}\right)$ . In the former case, combined with Eq. (5), we have that
$$
\bar {\pi} _ {T h} (S) \leq A + \mathcal {O} (h d \sqrt {T}).
$$
In the latter case, we have that
$$
D _ {\mathrm {T V}} \left[ \bar {\tilde {\pi}} _ {T h} | _ {S} | | \bar {\pi} _ {T h} | _ {S} \right] \leq \mathcal {O} \left(\frac {h d \sqrt {T}}{A}\right),
$$
$$
D _ {\mathrm {T V}} \left[ \pi | | \bar {\pi} _ {T h} \right] \leq \mathcal {O} \left(\frac {\sqrt {\operatorname {V a r} _ {\pi} \left[ \frac {\bar {\pi} _ {0}}{\pi} \right]}}{A \sqrt {T h \rho}}\right) + \mathcal {O} \left(\frac {h d \sqrt {T}}{A}\right).
$$
The proof is completed by selecting  $h = \frac{\operatorname{Var}_{\pi}[\bar{\pi}_0]^\frac{1}{3}}{T^{\frac{2}{3}}\rho^{\frac{1}{3}}d^{\frac{2}{3}}}$ , and  $A = \frac{d^{\frac{1}{6}}\operatorname{Var}_{\pi}[\bar{\pi}_0]^\frac{1}{6}}{\rho^{\frac{1}{6}}T^{\frac{1}{12}}}$ .
A.3 Difficulty of extension to Rényi divergence
Given two distribution  $\pi$  and  $\mu$ , Rényi divergence of  $q$  between them, i.e.,  $\mathcal{R}_q(\mu \| \pi)$  is defined as follows:
$$
\mathcal {R} _ {q} (\mu \| \pi) \triangleq \frac {1}{q - 1} \ln \mathbb {E} _ {x \sim \pi} \left(\frac {\mu}{\pi} (x)\right) ^ {q}.
$$
For simplicity, here we consider Langevin Dynamics defined as Eq. (1) with the distribution of time  $t$  denoted as  $\tilde{\pi}_t$ . The measure of convergence is then defined as  $\mathcal{R}_q(\pi_t\| \pi)$ , the derivative of which, according to (Lemma 6, [23]) is given as
$$
\frac {d}{d t} \mathcal {R} _ {q} (\pi_ {t} | | \pi) = - q \frac {G _ {q} (\pi_ {t} | | \pi)}{F _ {q} (\pi_ {t} | | \pi)},
$$
$$
\begin{array}{l} G _ {q} \left(\pi_ {t} \| \pi\right) = \mathbb {E} _ {\pi} \left[ \left(\frac {\pi_ {t}}{\pi}\right) ^ {q} \left\| \nabla \log \frac {\pi_ {t}}{\pi} \right\| ^ {2} \right] = \mathbb {E} _ {\pi} \left[ \left(\frac {\pi_ {t}}{\pi}\right) ^ {q - 2} \left\| \nabla \frac {\pi_ {t}}{\pi} \right\| ^ {2} \right] = \frac {4}{q ^ {2}} \mathbb {E} _ {\pi} \left[ \left\| \nabla \left(\frac {\pi_ {t}}{\pi}\right) ^ {\frac {q}{2}} \right\| ^ {2} \right], \\ F _ {q} \left(\pi_ {t} | | \pi\right) = \mathbb {E} _ {\pi} \left[ \left(\frac {\pi_ {t}}{\pi}\right) ^ {q} \right]. \\ \end{array}
$$
If we would like to follow the routine of Lemma 1, we need to firstly lower bound  $\frac{G_q(\pi_t||\pi)}{F_q(\pi_t||\pi)}$  by  $\frac{G_q((\pi_t|S)||(\pi|S))}{F_q((\pi_t|S)||(\pi|S))}$  and secondly transfer  $\frac{G_q((\pi_t|S)||(\pi|S))}{F_q((\pi_t|S)||(\pi|S))}$  back to  $\mathcal{R}_q((\pi_t|S)||(\pi|S))$ . The second step can be accomplished following the same routine as (Lemma 9,[24]) using local PI. However, we are still unaware of how to implement the first step. This is because although we have
$$
G _ {q} \left(\pi_ {t} | | \pi\right) = \frac {4}{q ^ {2}} \mathbb {E} _ {\pi} \left[ \left\| \nabla \left(\frac {\pi_ {t}}{\pi}\right) ^ {\frac {q}{2}} \right\| ^ {2} \right] \geq \frac {4}{q ^ {2}} \mathbb {E} _ {\pi | S} \left[ \left\| \nabla \left(\frac {\pi_ {t} | S}{\pi | S}\right) ^ {\frac {q}{2}} \right\| ^ {2} \right] \times \frac {\pi_ {t} (S) ^ {q}}{\pi (S) ^ {q - 1}},
$$
which is similar to the analysis of local LSI, we also have
$$
F _ {q} \left(\pi_ {t} \| \pi\right) = \mathbb {E} _ {\pi} \left[ \left(\frac {\pi_ {t}}{\pi}\right) ^ {q} \right] \geq \mathbb {E} _ {\pi} \left[ \left(\frac {\pi_ {t}}{\pi}\right) ^ {q} \right] \times \frac {\pi_ {t} (S) ^ {q}}{\pi (S) ^ {q - 1}}.
$$
In other words, when restricting to  $S$ , both the denominator and the numerator of  $\frac{G_q(\pi_t||\pi)}{F_q(\pi_t||\pi)}$  will get smaller, which makes  $\frac{G_q(\pi_t||\pi)}{F_q(\pi_t||\pi)}$  no longer lower bounded by  $\frac{G_q((\pi_t|S)||(\pi|S))}{F_q((\pi_t|S)||(\pi|S))}$ . We leave how to resolve this challenge as an interesting future work.
B Proof of Applications
B.1 Proof of gaussian-mixture
To start with, we recall Bakry-Émery criterion and Holley-Stroock perturbation principle.
Lemma 8 (Bakry-Émery criterion). Let  $\Omega \subset \mathbb{R}^n$  be convex and let  $H: \Omega \to \mathbb{R}$  be a Hamiltonian with Gibbs measure  $\mu(x) \propto e^{-H(x)} 1_{\Omega}(x)$  and assume that  $\nabla^2 H(x) \geq \kappa > 0$  for all  $x \in \operatorname{supp}(\mu)$ . Then  $\mu$  satisfies  $\mathrm{LSI}(\kappa)$ .
Lemma 9 (Holley-Stroock perturbation principle). If  $p \in \Delta(\Omega)$  satisfies LSI  $(\pi_t)$ , and  $\psi : \Omega \to \mathbb{R}$  satisfies  $m \leq \psi \leq M$ , where  $m, M > 0$ . Then,  $q \in \Delta(\Omega)$  satisfies LSI  $\left( \frac{m}{M} \pi_t \right)$ .
Proof of Lemma 3. Since  $S_{i}$  is convex, by applying Lemma 8, we obtain  $p_i|_{S_i}$  is  $\mathrm{LSI}(\sigma^2)$ . Let  $c = \min_{i} w_{i}$ .
Meanwhile, we have over  $S_{j}$
$$
c p _ {j} \leq w _ {j} p _ {j} \leq p = \sum_ {i \neq j} w _ {i} p _ {i} + w _ {j} p _ {j} \leq \sum_ {i \neq j} w _ {i} p _ {j} + w _ {j} p _ {j} = p _ {j}.
$$
Therefore, by Holley-Stroock perturbation principle, we have that  $p|_{S_j}$  satisfies  $\mathrm{LSI}\left(\frac{1}{c}\sigma^{-2}\right)$ .
Proof of Lemma 4. Through direct calculation, we obtain
$$
\nabla^ {2} V = \Sigma^ {- 1} - \frac {1}{2} \Sigma^ {- 1} \frac {\sum_ {i , j} w _ {i} w _ {j} p _ {i} (x) p _ {j} (x) (\mu_ {i} - \mu_ {j}) (\mu_ {i} - \mu_ {j}) ^ {\top}}{(\sum_ {i = 1} ^ {n} w _ {i} p _ {i} (x)) ^ {2}} \Sigma^ {- 1}.
$$
Then, for any  $\mathbf{a} \in \mathbb{R}^d$  with  $\| \mathbf{a} \| = 1$ , we have
$$
\begin{array}{l} \boldsymbol {a} ^ {\top} \nabla^ {2} V \boldsymbol {a} = \boldsymbol {a} ^ {\top} \Sigma^ {- 1} \boldsymbol {a} - \frac {1}{2} \frac {\sum_ {i , j} w _ {i} w _ {j} p _ {i} (x) p _ {j} (x) | (\mu_ {i} - \mu_ {j}) ^ {\top} \Sigma^ {- 1} \boldsymbol {a} | ^ {2}}{(\sum_ {i = 1} ^ {n} w _ {i} p _ {i} (x)) ^ {2}} \\ \leq \boldsymbol {a} ^ {\top} \Sigma^ {- 1} \boldsymbol {a} \leq \frac {1}{\sigma^ {- 2}}, \\ \end{array}
$$
$$
\begin{array}{l} \boldsymbol {a} ^ {\top} \nabla^ {2} V \boldsymbol {a} \leq \boldsymbol {a} ^ {\top} \Sigma^ {- 1} \boldsymbol {a} - \frac {1}{2} \frac {\sum_ {i , j} w _ {i} w _ {j} p _ {i} (x) p _ {j} (x) | (\mu_ {i} - \mu_ {j}) ^ {\top} \Sigma^ {- 1} \boldsymbol {a} | ^ {2}}{(\sum_ {i = 1} ^ {n} w _ {i} p _ {i} (x)) ^ {2}} \\ \geq - \frac {1}{2} \frac {\sum_ {i , j} w _ {i} w _ {j} p _ {i} (x) p _ {j} (x) | (\mu_ {i} - \mu_ {j}) ^ {\top} \Sigma^ {- 1} \boldsymbol {a} | ^ {2}}{(\sum_ {i = 1} ^ {n} w _ {i} p _ {i} (x)) ^ {2}} \geq - \frac {\max _ {i , j} \| \mu_ {i} - \mu_ {j} \| ^ {2}}{\sigma^ {- 4}}. \\ \end{array}
$$
B.2 Proof of sampling from the power posterior distribution of symmetric Gaussian mixtures
To begin with, define the expected potential function  $\bar{V}$  as
$$
\bar {V} (\theta) := \beta \mathbb {E} _ {X} \log \left(\frac {1}{2} \varphi (\theta - X) + \frac {1}{2} \varphi (\theta + X)\right) + \log \lambda (\theta).
$$
We further define the corresponding distribution as  $\bar{\pi}\propto e^{-\bar{V}}$
The following lemmas will be needed in the proof.
Lemma 10 (Theorem 2, [20]). If  $A, M \geq \|\|\theta_0\|\| + 1$ , we have  $\bar{\pi}|_{R_1}$  satisfies  $\mathrm{PI}(\Theta(1 / A^4M^2))$ .
Lemma 11 (Lemma 4, [20]). If  $A,M\geq \|\theta_0\| + 1$ , we have with probability at least  $1 - \delta$
$$
\sup  _ {\theta \in [ 0, A ] \times \mathcal {B} (0, M)} | V (\theta) - \bar {V} (\theta) | \leq \mathcal {O} ((1 + A + M) \sqrt {\frac {d}{n} \log \frac {n (A + M + d)}{\delta}}).
$$
Lemma 12. Suppose  $V$  obeys strong dissipativity, i.e.,  $\nabla V$  is  $L$ -lipschitz and  $\langle \nabla V(x), x \rangle \geq m \| x \|^2 - b$ . Then, running LMC with  $h < \frac{1}{16L}$  and  $h \leq \frac{8m}{4b + 32d}$ , we have
$$
\mathbb {E} _ {x _ {k} \sim \pi_ {k}} e ^ {\frac {1}{4 m} \| x _ {k} \| ^ {2}} \leq 3 2 \cdot \exp \left(\frac {1}{4 m} (8 b + 6 4 d)\right).
$$
Proof. Define  $f(x) = e^{\frac{1}{4m}\| x\|^2}$ . Assume wlog that  $\nabla V(0) = 0$ .
$$
\begin{array}{l} f \left(x _ {k + 1}\right) = \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} + \frac {1}{2 m} \left\langle - h \nabla V \left(x _ {k}\right) + \sqrt {2 h} \xi_ {k}, x _ {k} \right\rangle + \frac {1}{4 m} \left\| - h \nabla V \left(x _ {k}\right) + \sqrt {2 h} \xi_ {k} \right\| ^ {2}\right) \\ \leq \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{2 m} \left(\| x _ {k} \| ^ {2} - b\right) + \frac {\sqrt {2 h}}{2 m} \left\langle \xi_ {k}, x _ {k} \right\rangle + \frac {1}{2 m} \left\| h ^ {2} \nabla V (x _ {k}) \right\| ^ {2} + \frac {h}{m} \| \xi_ {k} \| ^ {2}\right) \\ \leq \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{4 m} \left(\| x _ {k} \| ^ {2} - 2 b\right) + \frac {\sqrt {2 h}}{2 m} \langle \xi_ {k}, x _ {k} \rangle + \frac {h}{m} \| \xi_ {k} \| ^ {2}\right). \\ \end{array}
$$
Let  $\mathbb{E}_k$  denote expectation wrt  $\xi_{k}$  . Then
$$
\begin{array}{l} \mathbb {E} _ {k} \left[ f \left(x _ {k + 1}\right) \right] \leq \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{4 m} \left(\| x _ {k} \| ^ {2} - 2 b\right)\right) \cdot \mathbb {E} _ {k} \left[ \frac {\sqrt {2 h}}{2 m} \left\langle \xi_ {k}, x _ {k} \right\rangle + \frac {h}{m} \| \xi_ {k} \| ^ {2} \right] \\ \leq \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{4 m} \left(\| x _ {k} \| ^ {2} - 2 b\right)\right) \cdot \mathbb {E} _ {k} \left[ \frac {\sqrt {2 h}}{m} \langle \xi_ {k}, x _ {k} \rangle \right] ^ {1 / 2} \cdot \mathbb {E} _ {k} \left[ \frac {2 h}{m} \| \xi_ {k} \| ^ {2} \right] ^ {1 / 2} \\ \end{array}
$$
Using the fact that  $\chi^2$  variable is sub-exponential,
$$
\mathbb {E} _ {k} \left[ \exp \left(\frac {2 h}{m} \| \xi_ {k} \| ^ {2}\right) \right] \leq \exp \left(\frac {4 h d}{m}\right)
$$
On the other hand, notice that  $\langle \xi_k,x_k\rangle \sim \mathcal{N}(0,\| x_k\|^2)$  is a 1-dimensional gaussian random variable. It can be shown that
$$
\mathbb {E} _ {k} \left[ \exp \left(\frac {\sqrt {2 h}}{m} \langle \xi_ {k}, x _ {k} \rangle\right) \right] \leq \exp \left(\frac {2 h}{m ^ {2}} \| x _ {k} \| ^ {2}\right). \tag {6}
$$
Combining the above,
$$
\begin{array}{l} \mathbb {E} _ {k} \left[ f \left(x _ {k + 1}\right) \right] \leq \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{4 m} \left(\| x _ {k} \| ^ {2} - 2 b\right) + \frac {2 h d}{m} + \frac {h}{m ^ {2}} \| x _ {k} \| ^ {2}\right) \\ \leq \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{8 m} \left(\| x _ {k} \| ^ {2} - 4 b - 3 2 d\right)\right) \\ \end{array}
$$
Let  $\mathbb{E}$  denote expectation wrt all randomness. Then
$$
\begin{array}{l} \mathbb {E} [ f (x _ {k + 1}) ] \\ \leq \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{8 m} \left(\| x _ {k} \| ^ {2} - 4 b - 3 2 d\right)\right) \right] \\ = \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{8 m} \left(\| x _ {k} \| ^ {2} - 4 b - 3 2 d\right)\right) 1 \left\{\| x _ {k} \| ^ {2} \geq 8 b + 6 4 d \right\} \right] \\ + \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{8 m} \left(\| x _ {k} \| ^ {2} - 4 b - 3 2 d\right)\right) 1 \left\{\| x _ {k} \| ^ {2} \leq 8 b + 6 4 d \right\} \right] \\ \leq \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} - \frac {h}{1 6 m} (4 b + 3 2 d)\right) 1 \left\{\| x _ {k} \| ^ {2} \geq 8 b + 6 4 d \right\} \right] \\ + \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2} + \frac {h}{8 m} (4 b + 3 2 d)\right) 1 \left\{\| x _ {k} \| ^ {2} \leq 8 b + 6 4 d \right\} \right] \\ \leq \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) 1 \left\{\| x _ {k} \| ^ {2} \geq 8 b + 6 4 d \right\} \cdot \left(1 - \frac {h}{3 2 m} (4 b + 3 2 d)\right) \right] \\ + \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) 1 \left\{\| x _ {k} \| ^ {2} \leq 8 b + 6 4 d \right\} \cdot \left(1 + \frac {h}{4 m} (4 b + 3 2 d)\right) \right] \\ = \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) \right] - \frac {h}{3 2 m} (4 b + 3 2 d) \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) 1 \left\{\| x _ {k} \| ^ {2} \geq 8 b + 6 4 d \right\} \right] \\ + \frac {h}{4 m} (4 b + 3 2 d) \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) 1 \left\{\| x _ {k} \| ^ {2} \leq 8 b + 6 4 d \right\} \right] \\ \leq \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) \right] - \frac {h}{3 2 m} (4 b + 3 2 d) \mathbb {E} \left[ \exp \left(\frac {1}{4 m} \| x _ {k} \| ^ {2}\right) \right] \\ + \frac {h}{m} (4 b + 3 2 d) \cdot \exp \left(\frac {1}{4 m} (8 b + 6 4 d)\right) \\ = \mathbb {E} [ f (x _ {k}) ] - \frac {h}{3 2 m} (4 b + 3 2 d) \mathbb {E} [ f (x _ {k}) ] + \frac {h}{m} (4 b + 3 2 d) \cdot \exp \left(\frac {1}{4 m} (8 b + 6 4 d)\right). \\ \end{array}
$$
Suppose that  $x_{k}$  is drawn from the invariant distribution under LMC. Then we know that  $\mathbb{E}f(x_k) = \mathbb{E}f(x_{k + 1})$ . In this case,
$$
0 \leq - \frac {h}{3 2 m} (4 b + 3 2 d) \mathbb {E} [ f (x _ {k}) ] + \frac {h}{m} (4 b + 3 2 d) \cdot \exp \left(\frac {1}{4 m} (8 b + 6 4 d)\right).
$$
Moving things around gives
$$
\mathbb {E} f (x _ {k}) \leq 3 2 \cdot \exp \left(\frac {1}{4 m} (8 b + 6 4 d)\right).
$$
We are now ready to prove the lemmas in the main text.
Proof of Lemma 5. Based on Lemma 11, if  $n \geq \tilde{\Theta}((A + M)^2 d\log(1 / \delta))$ , we have with probability at least  $1 - \delta$
$$
\sup  _ {\theta \in R _ {1}} | V (\theta) - \bar {V} (\theta) | \leq \mathcal {O} (1).
$$
As a result, we have  $\frac{\bar{\pi}|R_1}{\pi|R_1}(\theta) = \Theta(1)$ , and by Lemma 9 and Lemma 10, the proof is completed.
Proof of Lemma 6. To begin with, set  $\xi \sim \mathcal{N}(0,\mathbb{I}_d)$ , we have
$$
\begin{array}{l} \left\langle \nabla \bar {V} (\theta), \theta \right\rangle = \beta \| \theta \| ^ {2} + \beta \mathbb {E} \left(\frac {- \varphi \left(\| \theta_ {0} \| e _ {1} + \xi - \theta\right) + \varphi \left(\| \theta_ {0} \| e _ {1} + \xi + \theta\right)}{\varphi \left(\| \theta_ {0} \| e _ {1} + \xi - \theta\right) + \varphi \left(\| \theta_ {0} \| e _ {1} + \xi + \theta\right)} \theta^ {\top} \left(\| \theta_ {0} \| e _ {1} + \xi\right)\right) \\ \geq \frac {\beta}{2} \| \theta \| ^ {2} - \beta (\| \theta_ {0} \| ^ {2} + 1). \\ \end{array}
$$
$$
\langle \nabla V (\theta), \theta \rangle \geq \frac {\beta}{2} \| \theta \| ^ {2} - 2 \beta \left(\| \theta_ {0} \| ^ {2} + 1\right)
$$
following a standard empirical process argument due to  $n \geq \tilde{\Theta}((A + M)^2 d\log(1/\delta))$  (see Lemma 6, [20] as an example).
Meanwhile, denote  $\tau_{i} = 1$  if  $X_{i}$  is sampled from  $\mathcal{N}(\theta_0,\mathbb{I}_d)$  and  $\tau_{i} = -1$  else-wise. We have
$$
\begin{array}{l} \nabla^ {2} V (\theta) = \beta \mathbb {I} _ {d} - \frac {\| \theta_ {0} \| ^ {2} \beta}{n} \sum_ {i = 1} ^ {n} \left(\frac {4 \varphi (X _ {i} - \theta) \varphi (X _ {i} + \theta)}{(\varphi (X _ {i} - \theta) + \varphi (X + \theta)) ^ {2}}\right) e _ {1} e _ {1} ^ {\top} \\ - \frac {\beta}{n} \sum_ {i = 1} ^ {n} \left(\frac {4 \varphi (X _ {i} - \theta) \varphi (X _ {i} + \theta)}{(\varphi (X _ {i} - \theta) + \varphi (X _ {i} + \theta)) ^ {2}} (X _ {i} - \tau_ {i} \| \theta_ {0} \| e _ {1}) (X _ {i} - \tau_ {i} \| \theta_ {0} \| e _ {1}) ^ {\top}\right), \\ \end{array}
$$
and thus  $(\beta - \| \theta_0\|^2\beta - \frac{\beta}{n}\sum_{i=1}^{n}\|X_i - \tau_i\|\theta_0\|e_1\|^2)\mathbb{I}_d \leq \nabla^2 V(\theta) \leq \beta \mathbb{I}_d$ . As  $X_i - \tau_i\|\theta_0\|e_1 \sim \mathcal{N}(0, \mathbb{I}_d)$ , by concentration inequality of chi-square variable and since  $n \geq \tilde{\Theta}((A + M)^2 d\log(1/\delta))$ , we have with probability at least  $1 - \delta$ ,
$$
\| \nabla^ {2} V (\theta) \| \leq 2 \beta (1 + \| \theta_ {0} \| ^ {2}).
$$
The claim for  $\| \nabla^3 V\|$  can be calculated following the similar routine. The proof is completed.
Proof of Lemma 7. The claim directly follows by applying Markov's inequality to Lemma 12.
Proof. We consider the variational form of spectral gap. We first define the Dirichlet form and the variance
$$
\mathcal {E} _ {P} (\phi , \phi) = \langle \phi , (I - P) \phi \rangle_ {\pi} = \sum_ {x, y} \pi (x) \phi (x) (\mathbb {I} (x, y) - P (x, y)) \phi (y),
$$
$$
\operatorname {V a r} _ {\pi} [ \phi ] = \sum_ {x} \pi (x) (\phi (x) - \mathbb {E} _ {\pi} [ \phi ]) ^ {2}
$$
Then by the fact that  $P$  is lazy, and hence  $P = \frac{1}{2} (I + \hat{P})$  for some reversible transition  $\hat{P}$ , we have
$$
\operatorname {V a r} _ {\pi} [ P \phi ] \leq \operatorname {V a r} _ {\pi} [ \phi ] - \mathcal {E} _ {P} (\phi , \phi),
$$
By nonnegativity  $\mathcal{E}_P(\phi ,\phi)\geq 0$  , we have
$$
\sum_ {t \leq T} \mathcal {E} _ {P} \left(\frac {\mu_ {t}}{\pi}, \frac {\mu_ {t}}{\pi}\right) \leq \operatorname {V a r} _ {\pi} \left[ \frac {\mu_ {0}}{\pi} \right]. \tag {7}
$$
By the variational form of spectral gap,
$$
\alpha \leq \inf  _ {\phi \text {n o n - c o n s t a n t}} \frac {\mathcal {E} _ {P} (\phi , \phi)}{\operatorname {V a r} _ {\pi} [ \phi ]}.
$$
Therefore, applying the Markov chain generated by  $P_{i}$  we have that for any  $i\leq m$
$$
\sum_ {t} \operatorname {V a r} _ {\pi_ {i}} \left[ \frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}} \right] \leq \frac {1}{\alpha} \sum_ {t} \mathcal {E} _ {P _ {i}} \left(\frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}}, \frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}}\right) \tag {8}
$$
We then note that
$$
\begin{array}{l} \mathcal {E} _ {P} \left(\frac {\mu_ {t}}{\pi}, \frac {\mu_ {t}}{\pi}\right) = \frac {1}{2} \sum_ {x, y} \pi (x) P (x, y) \left(\frac {\mu_ {t}}{\pi} (x) - \frac {\mu_ {t}}{\pi} (y)\right) ^ {2} \\ \geq \frac {1}{2} \sum_ {x, y \in \mathcal {X} _ {i}} \pi (x) P (x, y) \left(\frac {\mu_ {t}}{\pi} (x) - \frac {\mu_ {t}}{\pi} (y)\right) ^ {2} \\ = \frac {1}{2} \sum_ {x, y \in \mathcal {X} _ {i}} \pi (x) P _ {i} (x, y) \left(\frac {\mu_ {t}}{\pi} (x) - \frac {\mu_ {t}}{\pi} (y)\right) ^ {2} \\ + \frac {1}{2} \sum_ {x, y \in \mathcal {X} _ {i}} \pi (x) (P (x, y) - P _ {i} (x, y)) \left(\frac {\mu_ {t}}{\pi} (x) - \frac {\mu_ {t}}{\pi} (y)\right) ^ {2} \\ \end{array}
$$
We note by (PI) that for any  $x, y \in \mathcal{X}_i, x \neq y$ ,  $P(x, y) = P_i(x, y)$ . Therefore, the second term is zero. Hence,
$$
\mathcal {E} _ {P} \left(\frac {\mu_ {t}}{\pi}, \frac {\mu_ {t}}{\pi}\right) \geq \frac {\pi (\mathcal {X} _ {i}) ^ {2}}{2 \mu_ {t} (\mathcal {X} _ {i}) ^ {2}} \mathcal {E} _ {P _ {i}} \left(\frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}}, \frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}}\right)
$$
Combining with (7) and (8) we get,
$$
\frac {1}{T} \sum_ {t} \operatorname {V a r} _ {\pi_ {i}} \left[ \frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}} \right] / \mu_ {t} (\mathcal {X} _ {i}) ^ {2} \leq \frac {2}{\pi (\mathcal {X} _ {i}) ^ {2} \alpha T} \operatorname {V a r} _ {\pi} \left[ \frac {\mu_ {0}}{\pi} \right] \tag {9}
$$
The claim then follows by discussing if for all  $t, \mu_t(\mathcal{X}_i) \leq \pi(\mathcal{X}_i)T^{-1/4}$ , then
$$
\frac {1}{T} \sum_ {t} \operatorname {V a r} _ {\pi_ {i}} \left[ \frac {\mu_ {t} | \mathcal {X} _ {i}}{\pi | \mathcal {X} _ {i}} \right] \leq \frac {2}{\alpha T ^ {1 / 2}} \operatorname {V a r} _ {\pi} \left[ \frac {\mu_ {0}}{\pi}. \right] \tag {10}
$$
D Proof for Theorem 2
Proof. Given the subgraph  $V$ , we have that conditioned transition kernel is
$$
p ^ {\prime} (x, y) = p (x, y) + \mathbb {1} \left\{x = y \right\} P (x, (V ^ {\prime}) ^ {c}).
$$
We analyze the conductance on graph  $V^{\prime}$  induced by  $p^\prime$
$$
\Phi = \min  _ {V _ {1} \subset V ^ {\prime}} \frac {\sum_ {x \in V _ {1}} \pi (x) p ^ {\prime} (x , V _ {1} ^ {c})}{\min  \left\{\pi (V _ {1}) , 1 - \pi (V _ {1}) \right\}}
$$
For any partition  $V_{1}, V_{2}$  of  $V'$ , without loss of generality assume the local min of  $f$  is in  $V_{2}$ ,  $v^{*} \in V_{2}$ . For simplicity, for any  $x \in V_{1}$ , we denote  $\tau(x)$  be the last element in its shortest path to  $v^{*}$ . We note that
$$
d (x, \tau (x)) \leq D, \pi (x) \leq \pi (\tau (x)).
$$
Denote  $\tau(V_1)$  be the set of such elements in  $V_1$ . Then we have