SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {\partial \mu_ {t}}{\partial t} + \nabla \cdot (v _ {t} \mu_ {t}) = 0, \quad v _ {t} = - (\nabla^ {2} \phi) ^ {- 1} \nabla_ {W _ {2}} \mathcal {F} (\mu_ {t}). \tag {27}
$$
The WMF is rather different from the MWGF in (18). In particular, the WMF in (27) describes the evolution of  $\mu_t$  according to the Wasserstein tangent vector  $-(\nabla^2\phi)^{-1}\nabla_{W_2}\mathcal{F}(\mu_t)$ , where  $\mathcal{F}$  is a functional minimised at  $\pi$ , while the MWGF in (18) describes the evolution of the mirrored  $\eta_t := (\nabla \phi)_{\#}\mu_t$  according to the tangent vector  $-\nabla_{W_2}\mathcal{F}(\eta_t)$ , where  $\mathcal{F}$  is now a functional minimised at  $\nu = (\nabla \phi)_{\#}\pi$ . We refer to [19] for a detailed discussion of the mirror Langevin diffusion from this perspective; and [26] for a more general formulation of the WMF.
B.1.1 Continuous Time Results
We now study the properties of the dynamics in (23) in continuous time, starting with the dissipation of  $\mathrm{KL}(\mu_t|\pi)$  and  $\chi^2 (\mu_t|\pi)$  along the MLD. These results are a natural extension of existing results in the unconstrained case to our setting.
Proposition 1. The dissipation of  $\mathrm{KL}(\cdot ||\pi)$  along the MWGF in (21) is given by
$$
\frac {\mathrm {d} \operatorname {K L} \left(\mu_ {t} \| \pi\right)}{\mathrm {d} t} = - \left| \left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right) \right| \right| _ {L ^ {2} \left(\mu_ {t}\right)} ^ {2}. \tag {28}
$$
Proof. Using differential calculus in the Wasserstein space and the chain rule, we have that
$$
\frac {\mathrm {d} \operatorname {K L} \left(\eta_ {t} \| \nu\right)}{\mathrm {d} t} = \left\langle w _ {t}, \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \right\rangle_ {L ^ {2} \left(\eta_ {t}\right)} = - \left| \left| \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \right| \right| _ {L ^ {2} \left(\eta_ {t}\right)} ^ {2}. \tag {29}
$$
By [43, Theorem 2], we have that  $\mathrm{KL}(\eta ||\nu) = \mathrm{KL}(\mu ||\pi)$ . To deal with the term on the RHS, first note that, using the formula for the change of variable in a probability density, we have that
$$
\eta (y) = \mu (\nabla \phi^ {*} (y)) | \det  \nabla^ {2} \phi (\nabla \phi^ {*} (y)) | ^ {- 1}, \quad \nu (y) = \pi (\nabla \phi^ {*} (y)) | \det  \nabla^ {2} \phi (\nabla \phi^ {*} (y)) | ^ {- 1}. \tag {30}
$$
Thus, in particular, if  $y = \nabla \phi (x)$  , then we have that
$$
\log \left(\frac {\eta}{\nu}\right) (y) = \log \left(\frac {\mu}{\pi}\right) (\nabla \phi^ {*} (y)) = \log \left(\frac {\mu}{\pi}\right) (x). \tag {31}
$$
Using this, and the fact that  $\eta = (\nabla \phi)_{\#}\mu$  , we can now compute
$$
\begin{array}{l} \left| \left| \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \right| \right| _ {L ^ {2} \left(\eta_ {t}\right)} ^ {2} = \int \left| \left| \nabla_ {y} \log \left(\frac {\eta_ {t}}{\nu}\right) (y) \right| \right| ^ {2} d \eta_ {t} (y) (32) \\ = \int \left| \left| \nabla_ {y} \log \left(\frac {\eta_ {t}}{\nu}\right) (y) \right| \right| ^ {2} \mathrm {d} \left(\left(\nabla \phi\right) _ {\#} \mu_ {t}\right) (y) (33) \\ = \int \left| \left| \nabla_ {y} \log \left(\frac {\eta_ {t}}{\nu}\right) (\nabla \phi (x)) \right| \right| ^ {2} \mathrm {d} \mu_ {t} (x) (34) \\ = \int \left| \left| \nabla_ {y} \log \left(\frac {\mu_ {t}}{\pi}\right) (x) \right| \right| ^ {2} \mathrm {d} \mu_ {t} (x) (35) \\ = \int \left| \left| [ \nabla^ {2} \phi (x) ] ^ {- 1} \nabla_ {x} \log \left(\frac {\mu_ {t}}{\pi}\right) (y) \right| \right| ^ {2} d \mu_ {t} (y) (36) \\ = \left| \left| [ \nabla^ {2} \phi (y) ] ^ {- 1} \nabla_ {y} \log \left(\frac {\mu_ {t}}{\pi}\right) (y) \right| \right| _ {L ^ {2} (\mu_ {t})} ^ {2}. (37) \\ \end{array}
$$
The conclusion now follows.
Proposition 2. The dissipation of  $\chi^2 (\cdot ||\pi)$  along the MWGF in (21) is given by
$$
\frac {\mathrm {d} \chi^ {2} \left(\mu_ {t} \| \pi\right)}{\mathrm {d} t} = - 2 \left| \left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \frac {\mu_ {t}}{\pi} \right| \right| _ {L ^ {2} (\pi)} ^ {2}. \tag {38}
$$
Proof. The proof is rather similar to the previous one. In this case, using the fact that  $\nabla_{W_2}\mathcal{X}^2 (\eta ||\nu) = 2\nabla \frac{\eta}{\nu}$ , we have
$$
\frac {\mathrm {d} \chi^ {2} (\eta_ {t} | | \nu)}{\mathrm {d} t} = \left\langle - \nabla \log \left(\frac {\eta_ {t}}{\nu}\right), 2 \nabla \frac {\eta_ {t}}{\nu} \right\rangle_ {L ^ {2} (\eta_ {t})} = - 2 \left| \left| \nabla \frac {\eta_ {t}}{\nu} \right| \right| _ {L ^ {2} (\nu)} ^ {2}.
$$
We next observe, using now the fact that  $\frac{\eta}{\nu}(y) = \frac{\mu}{\pi}(x)$  which follows from (30), that
$$
\begin{array}{l} \chi^ {2} \left(\eta_ {t} \| \nu\right) = \int \left(\frac {\eta_ {t}}{\nu} (y) - 1\right) ^ {2} \mathrm {d} \nu (y) = \int \left(\frac {\eta_ {t}}{\nu} (y) - 1\right) ^ {2} \mathrm {d} ((\nabla \phi) _ {\#} \pi) (y) (39) \\ = \int \left(\frac {\eta_ {t}}{\nu} (\nabla \phi (x)) - 1\right) ^ {2} \mathrm {d} \pi (x) = \int \left(\frac {\mu_ {t}}{\pi} (x) - 1\right) ^ {2} \mathrm {d} \pi (x) = \chi^ {2} (\mu_ {t} | | \pi). (40) \\ \end{array}
$$
In addition, based on a very similar argument, we have that
$$
\begin{array}{l} \left| \left| \nabla \left(\frac {\eta_ {t}}{\nu}\right) \right| \right| _ {L ^ {2} (\nu)} ^ {2} = \int \left| \left| \nabla_ {y} \left(\frac {\eta_ {t}}{\nu}\right) (y) \right| \right| ^ {2} \mathrm {d} \nu (y) = \int \left| \left| \nabla_ {y} \left(\frac {\eta_ {t}}{\nu}\right) (y) \right| \right| ^ {2} \mathrm {d} ((\nabla \phi) _ {\#} \pi) (y) (41) \\ = \int \left| \left| \nabla_ {y} \left(\frac {\eta_ {t}}{\nu}\right) (\nabla \phi (x)) \right| \right| ^ {2} \mathrm {d} \pi (x) = \int \left| \left| \nabla_ {y} \left(\frac {\mu_ {t}}{\pi}\right) (x) \right| \right| ^ {2} \mathrm {d} \pi (x) (42) \\ = \int \left| \left| [ \nabla^ {2} \phi (y) ] ^ {- 1} \nabla_ {y} \left(\frac {\mu_ {t}}{\pi}\right) (y) \right| \right| ^ {2} \mathrm {d} \pi (y) = \left| \left| [ \nabla^ {2} \phi (y) ] ^ {- 1} \nabla_ {y} \left(\frac {\mu_ {t}}{\pi}\right) (y) \right| \right| _ {L ^ {2} (\pi)} ^ {2}. (43) \\ \end{array}
$$
Since the RHS of both (28) and (38) are non-positive, these results show that  $\mathrm{KL}(\mu_t|\pi)$  and  $\chi^2 (\mu_t|\pi)$  decrease along the MLD. As an immediate corollary, we have the following continuous time convergence rate for the time-average of  $||[\nabla^{2}\phi ]^{-1}\nabla \log (\frac{\mu_{t}}{\pi})||_{L^{2}(\mu_{t})}^{2}$  and  $||[\nabla^{2}\phi ]^{-1}\nabla (\frac{\mu_{t}}{\pi})||_{L^{2}(\pi)}^{2}$ .
Proposition 3. For any  $t \geq 0$ , it holds that
$$
\left. \right. \min  _ {0 \leq s \leq t} \left|\left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right)\right|\right| _ {L ^ {2} \left(\mu_ {t}\right)} ^ {2} \leq \frac {1}{t} \int_ {0} ^ {t} \left|\left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {s}}{\pi}\right)\right|\right| _ {L ^ {2} \left(\mu_ {s}\right)} ^ {2} d s \leq \frac {\mathrm {K L} (\mu_ {0} \| \pi)}{t}. \tag {44}
$$
Proposition 4. For any  $t \geq 0$ , it holds that
$$
\left. \right. \min  _ {0 \leq s \leq t} \left|\left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \left(\frac {\mu_ {t}}{\pi}\right)\right|\right| _ {L ^ {2} (\pi)} ^ {2} \leq \frac {1}{t} \int_ {0} ^ {t} \left|\left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \left(\frac {\mu_ {s}}{\pi}\right)\right|\right| _ {L ^ {2} (\pi)} ^ {2} d s \leq \frac {\chi^ {2} (\mu_ {0} \| \pi)}{t}. \tag {45}
$$
Proof. These results follow straightforwardly from integration of (28) or (38) and the non-negativity of the KL or  $\chi^2$  divergence.
To obtain stronger convergence results, we will require additional assumptions on the target measure. In the first case, we can assume that the target satisfies what we will refer to as a mirrored logarithmic Sobolev inequality (LSI).
Proposition 5. Assume that  $\pi$  satisfies a mirrored LSI with constant  $\lambda >0$ . In particular, for all  $\mu \in \mathcal{P}_2(\mathcal{X})$ , there exists  $\lambda >0$  such that
$$
\operatorname {K L} (\mu | | \pi) \leq \frac {1}{2 \lambda} \left| \left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu}{\pi}\right) \right| \right| _ {L ^ {2} (\mu)} ^ {2}. \tag {46}
$$
Then the KL divergence converges exponentially along the MWGF in (21):
$$
\mathrm {K L} \left(\mu_ {t} \| \pi\right) \leq e ^ {- 2 \lambda t} \mathrm {K L} \left(\mu_ {0} \| \pi\right). \tag {47}
$$
Proof. Substituting the mirrored LSI (46) into (28), we have that
$$
\frac {\mathrm {d} \mathrm {K L} \left(\mu_ {t} \| \pi\right)}{\mathrm {d} t} = - \left| \left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right) \right| \right| _ {L ^ {2} (\mu_ {t})} ^ {2} \leq - 2 \lambda \mathrm {K L} \left(\mu_ {t} \| \pi\right). \tag {48}
$$
The conclusion now follows straightforwardly from Gronwall's inequality [39].
Alternatively, under the assumption that the target satisfies a mirrored Poincaré inequality (PI), we can obtain exponential convergence in a number of metrics.
Proposition 6. Assume that  $\pi$  satisfies a mirrored PI with constant  $\kappa >0$ . In particular, for all locally Lipschitz  $g\in L^{2}(\pi)$ , there exists  $\kappa >0$  such that
$$
\operatorname {V a r} _ {\pi} [ g ] \leq \frac {1}{\kappa} \left| \left| \left[ \nabla^ {2} \phi \right] ^ {- 1} \nabla g \right| \right| _ {L ^ {2} (\pi)} ^ {2} \tag {49}
$$
Then the total variation distance, Hellinger distance, KL divergence, and the  $\chi^2$  divergence all converge exponentially along the MWGF in (21):
$$
\max  \left(2 \| \mu_ {t} - \pi \| _ {\mathrm {T V}} ^ {2}, H ^ {2} (\mu_ {t}, \pi), \frac {\lambda}{2} W _ {2} ^ {2} (\mu_ {t}, \pi), \operatorname {K L} (\mu_ {t} \| \pi), \chi^ {2} (\mu_ {t} \| \pi)\right) \leq e ^ {- 2 \kappa t} \chi^ {2} (\mu_ {0} \| \pi). \tag {50}
$$
Proof. The proof proceeds similarly to above. In particular, this time substituting the mirrored PI (49) with  $g = \frac{\mu_t}{\pi} - 1$  into (38), we have
$$
\frac {\mathrm {d} \chi^ {2} (\mu_ {t} | | \pi)}{\mathrm {d} t} = - 2 \left| \left| [ \nabla^ {2} \phi ] ^ {- 1} \nabla \frac {\mu_ {t}}{\pi} \right| \right| _ {L ^ {2} (\pi)} ^ {2} \leq - 2 \kappa \chi^ {2} (\mu_ {t} | | \pi).
$$
The result for the  $\chi^2$  divergence follows via an application of Gronwall's inequality. The remaining bounds follow from standard comparison inequalities [e.g., 93, Sec. 2.4].
Remark 2. The assumption that  $\pi$  satisfies the mirrored LSI in (46) or the mirrored PI in (49) is precisely equivalent to the assumption that the mirrored target  $\nu = (\nabla \phi)_{\#}\pi$  satisfies the classical LSI or the classical PI, respectively. The LSI holds, for example, for all strongly log-concave distributions, with constant equal to the reciprocal of the strong convexity constant of the potential [4]. The class of measures satisfying the PI is even larger, including all strongly log-concave measures and, more generally, all log-concave measures [10, 17, 47].
Remark 3. The mirrored LSI in (46) and the mirrored PI in (49) are distinct from the so-called mirror LSI [44, Assumption 2] and mirror PI [19, Definition 1]. These assumptions are used in [44, Proposition 1] and [19, Theorem 1], respectively, to establish exponential convergence of the mirror Langevin diffusion [e.g., 44, Equation 2], which we recall is distinct from the mirrored Langevin diffusion in (23); see Remark 1.
Remark 4. In Propositions 5 - 6 above, and also in Proposition 7 below, our assumptions on the target  $\pi$  are, in some sense, better viewed as assumptions on the mirrored target  $\nu = (\nabla \phi)_{\#} \pi$ . Indeed, by construction, the imposed assumptions are equivalent to the assumptions that the mirrored target  $\nu$  satisfies a LSI (Proposition 5, 7), or that the mirrored target  $\nu$  satisfies a PI (Proposition 6).
In some sense, it would be preferable to establish such results under more direct assumptions on the target  $\pi$  itself. One possibility is to assume that the target  $\pi$  is strongly log-concave. In this case, [43, Proof of Theorem 3] guarantees the existence of a mirror map such that the mirrored target  $\nu = (\nabla \phi)_{\#} \pi$  is also strongly log-concave. Thus, in particular, the mirrored target satisfies a LSI and a PI or, equivalently, the target  $\pi$  satisfies a mirrored LSI and a mirrored PI. Thus, Propositions 5 - 7 all still hold.
B.1.2 Discrete Time Results
In practice, of course, it is necessary to discretise the dynamics in (23) in time. Applying a standard Euler-Maruyama discretisation to (23), or equivalently a forward-flow discretisation to (21) [e.g., 99], one arrives at the mirrored Langevin algorithm (MLA) from [43], viz
$$
y _ {t + 1} = y _ {t} - h \nabla W (y _ {t}) + \sqrt {2 h} \xi_ {t}, \quad x _ {t + 1} = \nabla \phi^ {*} (y _ {t + 1}) \tag {51}
$$
where  $h > 0$  is the step size or learning rate and where  $(\xi_{t})_{t\in \mathbb{N}}$  are a sequence of i.i.d. standard normal random variables.
Proposition 7. Suppose that  $\pi$  satisfies the mirrored LSI with constant  $\lambda >0$ . In addition, suppose that  $\pi$  is mirror- $L$ -smooth. Then, if  $0 < \gamma \leq \frac{\lambda}{4L^2}$ , the iterates  $(x_{t})_{t\geq 0}$  in (51) satisfy
$$
\mathrm {K L} \left(\mu_ {t} \| \pi\right) \leq e ^ {- \lambda \gamma t} \mathrm {K L} \left(\mu_ {0} \| \pi\right) + \frac {8 \gamma d L ^ {2}}{\lambda}. \tag {52}
$$
Proof. By [43, Theorem 2], and (32) - (37) in the proof of Proposition 1, if  $\pi$  satisfies a mirrored LSI with constant  $\lambda > 0$ , then  $\nu = (\nabla \phi)_{\#} \pi$  satisfies a LSI with the same constant  $\lambda$ . In addition, by definition, if  $\pi$  is mirror- $L$ -smooth, then  $\nu$  is  $L$ -smooth. Thus, it follows from [94, Theorem 1] that
$$
\mathrm {K L} \left(\eta_ {t} | | \nu\right) \leq e ^ {- \lambda \gamma t} \mathrm {K L} \left(\eta_ {0} | | \nu\right) + \frac {8 \gamma d L ^ {2}}{\lambda}. \tag {53}
$$
Finally, using the invariance of the KL divergence under the mirror map  $\nabla \phi$  [43, Theorem 2], the result follows.
Remark 5. Using a similar approach, one can also obtain bounds on  $\chi^2 (\mu_t||\pi)$  under a mirrored LSI [35, Theorem 4], and for the Renyi divergence  $\mathcal{R}_q(\mu_t||\pi)$  under a mirrored LSI [94, Theorem 2] or a mirrored PI [94, Theorem 3].
Remark 6. One could also consider a forward Euler discretisation of the dynamics in (21), which leads to a mirrored version of the so-called deterministic Langevin Monte Carlo algorithm [40].
B.2 Mirrored Stein Variational Gradient Descent
We now turn our attention to the MSVGD dynamics given in Sec. 3.2.1, which we recall are defined by the continuity equation
$$
\frac {\partial \eta_ {t}}{\partial t} - \nabla \cdot \left(P _ {\eta_ {t}, k _ {\phi}} \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \eta_ {t}\right) = 0, \quad \mu_ {t} = \left(\nabla \phi^ {*}\right) _ {\#} \eta_ {t}, \tag {54}
$$
where, as previously,  $k_{\phi}$  is the kernel defined according to  $k_{\phi}(y,y^{\prime}) = k(\nabla \phi^{*}(y),\nabla \phi^{*}(y^{\prime}))$  , given some base kernel  $k:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$
B.2.1 Continuous Time Results
We now study the convergence properties of the continuous-time MSVGD dynamics. We note that a similar set of results can also found in [82, App. H]. We provide them here to highlight the analogues with the convergence results obtained for the MLD in Sec. B.1.
Proposition 8. The dissipation of the KL along the mirrored SVGD gradient flow is given by
$$
\frac {\mathrm {d} \operatorname {K L} \left(\mu_ {t} \mid \mid \pi\right)}{\mathrm {d} t} = - \left| \left| S _ {\mu_ {t}, k} \left[ \nabla^ {2} \phi \right] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2}. \tag {55}
$$
Proof. The proof is similar to the proof of Proposition 1. In particular, using differential calculus on the Wasserstein space and the chain rule, we have
$$
\frac {\mathrm {d} \operatorname {K L} \left(\eta_ {t} \mid \mid \nu\right)}{\mathrm {d} t} = \left\langle w _ {t}, \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \right\rangle_ {L ^ {2} \left(\eta_ {t}\right)} = - \left| \left| S _ {\eta_ {t}, k _ {\phi}} \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \right| \right| _ {\mathcal {H} _ {k _ {\phi}} ^ {d}} ^ {2}. \tag {56}
$$
By Theorem 2 in [43], we have that  $\mathrm{KL}(\eta ||\nu) = \mathrm{KL}(\mu ||\pi)$ . In addition, using (31), we can compute
$$
\begin{array}{l} \left| \left| S _ {\eta_ {t}, k _ {\phi}} \nabla \log \left(\frac {\eta_ {t}}{\nu}\right) \right| \right| _ {\mathcal {H} _ {k _ {\phi}} ^ {d}} ^ {2} (57) \\ = \iint k \left(\nabla \phi^ {*} (y), \nabla \phi^ {*} \left(y ^ {\prime}\right)\right) \left\langle \nabla_ {y} \log \left(\frac {\eta_ {t}}{\nu}\right) (y), \nabla_ {y ^ {\prime}} \log \left(\frac {\eta_ {t}}{\nu}\right) \left(y ^ {\prime}\right) \right\rangle d \eta_ {t} (y) d \eta_ {t} \left(y ^ {\prime}\right) (58) \\ = \iint k (x, x ^ {\prime}) \left\langle \nabla_ {y} \log \left(\frac {\mu_ {t}}{\pi}\right) (x), \nabla_ {y ^ {\prime}} \log \left(\frac {\mu_ {t}}{\pi}\right) \left(x ^ {\prime}\right) \right\rangle \mathrm {d} \mu_ {t} (x) \mathrm {d} \mu_ {t} \left(x ^ {\prime}\right) (59) \\ = \iint k (x, x ^ {\prime}) \left\langle [ \nabla^ {2} \phi (x) ] ^ {- 1} \nabla_ {x} \log \left(\frac {\mu_ {t}}{\pi}\right) (x), [ \nabla^ {2} \phi (x ^ {\prime}) ] ^ {- 1} \nabla_ {x ^ {\prime}} \log \left(\frac {\mu_ {t}}{\pi}\right) (x ^ {\prime}) \right\rangle d \mu_ {t} (x) d \mu_ {t} (x ^ {\prime}) \\ = \left| \left| S _ {\mu_ {t}, k} \left[ \nabla^ {2} \phi \right] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2}. (60) \\ \end{array}
$$
2We say that  $\pi \propto e^{-V}$  is mirror-  $L$  -smooth if  $\nu \propto e^{-W}$  is  $L$  -smooth. This holds, in particular, if  $W = -V\circ \nabla \phi^{*} + \log \operatorname *{det}\nabla^{2}\phi^{*}$  has Hessian bounded by  $L$
Remark 7. The quantity on the RHS of (55) is referred to in [86] as the mirrored Stein Fisher information of  $\mu$  with respect to  $\pi$ , with mirror map  $\nabla \phi$ . This quantity represents the Stein Fisher information [32] between  $(\nabla \phi)_{\#} \mu$  and  $(\nabla \phi)_{\#} \pi$ , given the kernel  $k_{\phi}(\cdot, \cdot) = k(\nabla \phi^{*}(\cdot), \nabla \phi^{*}(\cdot))$ .
Similar to before, a straightforward consequence of this result is a continuous time convergence rate for the average of the mirrored Stein discrepancy along the continuous-time MSVGD dynamics.
Proposition 9. For all  $t \geq 0$ , it holds that
$$
\min  _ {0 \leq s \leq t} \left| \left| S _ {\mu_ {s}, k} [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {s}}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2} \leq \frac {1}{t} \int_ {0} ^ {t} \left| \left| S _ {\mu_ {s}, k} [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {s}}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2} d s \leq \frac {\mathrm {K L} (\mu_ {0} \| \pi)}{t}. \tag {61}
$$
Proof. We integrate (55), and use the non-negativity of the KL divergence.
To obtain stronger convergence guarantees, we must once again assume some additional properties on the target. Here, a natural choice is the mirrored Stein LSI, the analogue of the Stein LSI introduced in [32] in the mirrored setting; see also [82, Definition 2].
Proposition 10. Assume that  $\pi$  satisfies the mirrored Stein LSI with constant  $\lambda > 0$ . In particular, for all  $\mu \in \mathcal{P}_2(\mathcal{X})$ , there exists  $\lambda > 0$  such that
$$
\operatorname {K L} (\mu | | \pi) \leq \frac {1}{2 \lambda} \left| \left| S _ {\mu , k} [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2}. \tag {62}
$$
Then the KL divergence converges exponentially along the continuous-time MSVGD dynamics:
$$
\mathrm {K L} \left(\mu_ {t} | | \pi\right) \leq e ^ {- 2 \lambda t} \mathrm {K L} \left(\mu_ {0} | | \pi\right). \tag {63}
$$
Proof. The result is a straightforward consequence of (55) and (62). In particular, substituting (62) into (55), we have
$$
\frac {\mathrm {d} \operatorname {K L} \left(\mu_ {t} \| \pi\right)}{\mathrm {d} t} = - \left| \left| S _ {\mu_ {t}, k} [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2} \leq 2 \lambda \operatorname {K L} \left(\mu_ {t} \| \pi\right). \tag {64}
$$
We conclude by applying Gronwall's inequality.
Remark 8. The assumption that  $\pi$  satisfies the mirrored Stein LSI is equivalent to the assumption that  $\nu = (\nabla \phi)_{\#}\pi$  satisfies the classical Stein LSI with kernel  $k_{\phi}$ . The Stein LSI is rather less well known than the classical LSI, and holds for a much smaller class of targets. In particular, this condition fails to hold if the kernel  $k$  is too regular with respect to the target  $\pi$ , e.g., if  $\pi$  has exponential tails and the derivatives of the kernel  $k$  and the potential  $V$  grow at most polynomially [32, 52]. It remains an open problem to establish conditions for which this conditions hold.
B.2.2 Discrete Time Results
In discrete time, [86] recently established a descent lemma and a complexity bound for MSVGD in the population limit. For completeness, we recall one of these results here.
Theorem 1 (Corollary 1 in [86]). Suppose that the following assumptions hold:
(1) The mirror function  $\phi : \mathcal{X} \to (-\infty, +\infty)$  is strongly  $K$ -convex.
(2) There exist  $B_{1}, B_{2} > 0$  such that for all  $x \in \mathcal{X}$ ,  $k(x, x') \leq B_{1}^{2}$  and  $\partial_{x_i} \partial_{x_j} k(x, x')|_{x = x'} \leq B_2^2$  for all  $i \in [d]$ ,  $j \in [d]$ .
(3) There exist  $L_0, L_1 \geq 0$  such that  $||\nabla^2 W(y)|| \leq L_0 + L_1||\nabla W(y)||$ .
(4) There exists  $p \geq 1$ ,  $y_0 \in \mathbb{R}^d$ , and  $s > 0$  such that  $\int \exp [s||y - y_0||^p] \, \mathrm{d}\nu(y) < \infty$ .
(5) There exists  $p \geq 1$ ,  $C_p > 0$  such that  $||\nabla W(y)|| \leq C_p (||y||^p + 1)$  for any  $y \in \mathbb{R}^d$ .
Then, provided  $\gamma$  satisfies [86, Equation 21], then
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left| \left| S _ {\mu_ {t}, k} [ \nabla^ {2} \phi ] ^ {- 1} \nabla \log \left(\frac {\mu_ {t}}{\pi}\right) \right| \right| _ {\mathcal {H} _ {k} ^ {d}} ^ {2} \leq \frac {2 \mathrm {K L} (\mu_ {0} \| \pi)}{T \gamma}. \tag {65}
$$
Remark 9. The results in [86] are obtained by extending the proofs used to establish convergence of SVGD in [85] to the mirrored setting. In this case, essentially all of the same arguments can be applied (in the dual space), provided suitable assumptions are imposed on the mirrored target  $\nu = (\nabla \phi)_{\#} \pi$ . The disadvantage of this approach is that the required assumptions are specified in terms of the mirrored target  $\nu$ , rather than the target  $\pi$ .
Following a similar approach to [43, Theorem 3], let us show how to recover this result under a standard assumption on  $\pi$  itself. Suppose we replace [86, Assumption 4] by the assumption that  $\pi$  is strongly log-concave. Under this assumption, [43, Proof of Theorem 3] guarantees the existence of a mirror map such that  $\nu$  is also strongly log-concave. By the Bakry-Emery criterion [4], it follows that the  $T_{2}$  inequality is satisfied by the dual target  $\nu$ , and thus so too is the  $T_{1}$  inequality. Finally, the  $T_{1}$  inequality is necessary and sufficient for [86, Assumption 4] by [95, Theorem 22.10]. Thus, [86, Corollary 1] holds whenever the target  $\pi$  is strongly log-concave.
B.3 Mirrored Laplacian Adjusted Wasserstein Gradient Descent
We now consider the MWGF of the chi-squared divergence; see also [20, Sec. 3] or [2, Theorem 11.2.1] in the unconstrained case. Following similar steps to [20], we will then obtained a mirrored version of the Laplacian Adjusted Wasserstein Gradient Descent (LAWGD) algorithm.
B.3.1 MWGF of the Chi-Squared Divergence
Suppose that  $\mathcal{F}(\eta) = \chi^2 (\eta ||\nu)$  with  $\nabla_{W_2}\mathcal{F}(\eta) = 2\nabla \left(\frac{\eta}{\nu}\right)$ . Setting  $w_{t} = -\nabla_{W_{2}}\mathcal{F}(\eta_{t})$  in (4), we are interested in the following MWGF