SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$\theta_{l1},\ldots ,\theta_{lK} = \mathrm{Gram - }$  Schmidt  $(\theta_{l1},\dots ,\theta_{lK})$
$$
\text {R e t u r n .} \left(\frac {1}{L K} \sum_ {l = 1} ^ {L} \sum_ {k = 1} ^ {K} \mathrm {W} _ {p} ^ {p} \left(\theta_ {l k} \sharp \mu , \theta_ {l k} \sharp \nu\right)\right) ^ {\frac {1}{p}}
$$
Algorithm 4 Max-K sliced Wasserstein distance
Input. Probability measures  $\mu, \nu$ , learning rate  $\eta$ , the dimension  $d$ , the order  $p$ , the number of iterations  $T > 1$ , and the number of orthogonal projections  $K > 1$ .
Initialize  $\hat{\theta}_{01},\dots,\hat{\theta}_{0K}$  to be orthogonal.
for  $k = 1$  to  $K$  do
$$
\hat {\theta} _ {t k} = \theta_ {t k} + \eta \cdot \nabla_ {\hat {\theta} _ {t - 1 k}} \mathrm {W} _ {p} (\hat {\theta} _ {t - 1 k} \sharp \mu , \hat {\theta} _ {t - 1 k} \sharp \nu)
$$
$$
\hat {\theta} _ {t 1}, \dots , \hat {\theta} _ {t K} = \operatorname {G r a m - S c h m i d t} (\hat {\theta} _ {t 1}, \dots , \hat {\theta} _ {t K})
$$
$$
\text {R e t u r n .} \left(\frac {1}{K} \sum_ {k = 1} ^ {K} \mathrm {W} _ {p} ^ {p} (\hat {\theta} _ {T k} \sharp \mu , \hat {\theta} _ {T k} \sharp \nu)\right) ^ {\frac {1}{p}}
$$
A.2 Von Mises-Fisher Distribution
We first start with the definition of von Mises-Fisher (vMF) distribution.
Algorithm 5 Sampling from vMF distribution
Input. location  $\epsilon$ , concentration  $\kappa$ , dimension  $d$ , unit vector  $e_1 = (1,0,\dots,0)$
Draw  $v\sim \mathcal{U}(\mathbb{S}^{d - 2})$
$$
b \leftarrow \frac {- 2 \kappa + \sqrt {4 \kappa^ {2} + (d - 1) ^ {2}}}{d - 1}, a \leftarrow \frac {(d - 1) + 2 \kappa + \sqrt {4 \kappa^ {2} + (d - 1) ^ {2}}}{4}, m \leftarrow \frac {4 a b}{(1 + b)} - (d - 1) \log (d - 1)
$$
$$
\mathrm {D r a w} \psi \sim \operatorname {B e t a} \left(\frac {1}{2} (d - 1), \frac {1}{2} (d - 1)\right)
$$
$$
\omega \leftarrow h (\psi , \kappa) = \frac {1 - (1 + b) \psi}{1 - (1 - b) \psi}
$$
$$
t \leftarrow \frac {2 a b}{1 - (1 - b) \psi}
$$
$$
\text {D r a w} u \sim \mathcal {U} ([ 0, 1 ])
$$
$$
\text {u n t i l} (d - 1) \log (t) - t + m \geq \log (u)
$$
$$
h _ {1} \leftarrow (\omega , \sqrt {1 - \omega^ {2}} v ^ {\top}) ^ {\top}
$$
$$
\epsilon^ {\prime} \leftarrow e _ {1} - \epsilon
$$
$$
u = \frac {\epsilon^ {\prime}}{| | \epsilon^ {\prime} | | _ {2}}
$$
$$
U = I - 2 u u ^ {\top}
$$
Output.  $Uh_{1}$
Definition 3. The von Mises-Fisher distribution (vMF)[23] is a probability distribution on the unit hypersphere  $\mathbb{S}^{d - 1}$  with the density function be:
$$
f (x | \epsilon , \kappa) := C _ {d} (\kappa) \exp \left(\kappa \epsilon^ {\top} x\right), \tag {9}
$$
where  $\epsilon \in \mathbb{S}^{d - 1}$  is the location vector;  $\kappa \geq 0$  is the concentration parameter, and  $C_d(\kappa) \coloneqq \frac{\kappa^{d / 2 - 1}}{(2\pi)^{d / 2}I_{d / 2 - 1}(\kappa)}$  is the normalization constant. Here,  $I_v$  is the modified Bessel function of the first kind at order  $v$  [56].
The vMF distribution is a continuous distribution, its mass concentrates around the mean  $\epsilon$ , and its density decreases when  $x$  goes away from  $\epsilon$ . When  $\kappa \to 0$ , vMF converges in distribution to  $\mathcal{U}(\mathbb{S}^{d-1})$ , and when  $\kappa \to \infty$ , vMF converges in distribution to the Dirac distribution centered at  $\epsilon$  [55].
Sampling: We review the sampling process in Algorithm 5 [13, 45]. The sampling process of vMF distribution is based on the rejection sampling procedure. It is worth noting that the sampling algorithm is doing reparameterization implicitly. However, we only use the algorithm to obtain random samples without estimating stochastic gradients.
A.3 Algorithms for Computing Markovian Sliced Wasserstein Distances
We first start with the general computation of MSW in Algorithm 6. For the orthogonal-based transition in oMSW, we use  $\theta_{lt} \sim \mathcal{U}(S_{\theta_{lt-1}}^{d-1})$  by first sampling  $\theta_{lt}' \sim \mathcal{U}(\mathbb{S}^{d-1})$  then set  $\theta_{lt} = \theta_{lt} - \frac{\langle\theta_{lt}',\theta_{lt}\rangle}{\langle\theta_{lt}',\theta_{lt}'\rangle}\theta_{lt}'$  then normalize  $\theta_{lt} = \frac{\theta_{lt}}{||\theta_{lt}||_2}$ . For deterministic input-aware transition, iMSW, we set  $\theta_{lt} = \theta_{lt-1} + \eta\nabla_{\theta_{lt-1}}W_p(\theta_{lt-1}\sharp \mu,\theta_{lt-1}\sharp \nu)$  then normalize  $\theta_{lt} = \frac{\theta_{lt}}{||\theta_{lt}||_2}$ . For probabilistic input-aware transition, viMSW,  $\theta_{lt} \sim \mathrm{vMF}(\theta_t|\epsilon = \mathrm{Proj}_{\mathbb{S}^{d-1}}\theta_{lt}'',\kappa)$  with  $\theta_{lt}' = \theta_{lt-1} + \eta\nabla_{\theta_{lt-1}}W_p(\theta_{lt-1}\sharp \mu,\theta_{lt-1}\sharp \nu)$ .
Algorithm 6 Markovian sliced Wasserstein distance
Input. Probability measures  $\mu, \nu$ , the dimension  $d$ , the order  $p$ , the number of projections  $L$ , and the number of timesteps  $T$ .  
for  $l = 1$  to  $L$  do  
Draw  $\theta_{l0} \sim \sigma(\theta_0)$   
for  $t = 1$  to  $T - 1$  do  
Draw  $\theta_{lt} \sim \sigma_t(\theta_t|\theta_{lt - 1})$   
end for  
end for  
Return.  $\left(\frac{1}{LT}\sum_{l = 1}^{L}\sum_{t = 1}^{T}\mathbf{W}_p^p (\theta_{lt}\sharp \mu ,\theta_{lt}\sharp \nu)\right)^{\frac{1}{p}}$
A.4 Burned Thinned Markovian Sliced Wasserstein Distance
We continue the discussion on burned thinned MSW in Section 3.3. We first start with the Monte Carlo estimation of burned thinned MSW.
Monte Carlo Estimation: We samples  $\theta_{11},\ldots ,\theta_{L1}\sim \sigma_1(\theta_1)$  for  $L\geq 1$  , then we samples  $\theta_{lt}\sim \sigma_t(\theta_t|\theta_{lt - 1})$  for  $t = 1,\dots ,T$  and  $l = 1,\dots ,L$  . We then obtain samples  $\theta_{lt}^{\prime}$  by filtering out  $t <   M$  and  $t\% N\neq 0$  from the set  $\{\theta_{lt}\}$  for  $l = 1,\dots ,L$  and  $t = 1,\dots ,T$  . The Monte Carlo approximation of the burned-thinned Markovian sliced Wasserstein distance is:
$$
\widehat {\mathrm {M S W}} _ {p, T, N, M} (\mu , \nu) = \left(\frac {N}{L (T - M)} \sum_ {l = 1} ^ {L} \sum_ {t = 1} ^ {(T - M) / N} W _ {p} ^ {p} \left(\theta_ {l t} ^ {\prime} \sharp \mu , \theta_ {l t} ^ {\prime} \sharp \nu\right)\right) ^ {\frac {1}{p}}. \tag {10}
$$
Theoretical properties. We first state the following assumption: A2. Given  $T > M \geq 0$ ,  $N \geq 1$ , the prior distribution  $\sigma_{1}(\theta_{1})$  and the transition distribution  $\sigma_{t}(\theta_{t}|\theta_{t-1})$  are chosen such that there exists marginals  $\sigma_{t}(\theta_{t}) = \int_{t^{-}} \sigma(\theta_{1},\dots,\theta_{t}) dt^{-}$  with  $t \geq M$  and  $t\%N = 0$ ,  $t^{-} = \{t' = 1,\dots,T|t' \neq t\}$ .
The assumption A2 can be easily obtained by using vMF transition, e.g., in probabilistic input-aware transition. From this assumption, we can derive theoretical properties of burned-thinned MSW including topological properties and statistical complexity.
Proposition 4. For any  $p \geq 1, T \geq 1, M \geq 0, N \geq 1$ , and dimension  $d \geq 1$ , if A2 holds, the burned thinned Markovian sliced Wasserstein distance  $MSW_{p,T,N,M}(\cdot, \cdot)$  is a valid metric on the space of probability measures  $\mathcal{P}_p(\mathbb{R}^d)$ , namely, it satisfies the (i) non-negativity, (ii) symmetry, (iii) triangle inequality, and (iv) identity.
The proof of Proposition 4 follows directly the proof of Theorem 1 in Appendix B.1.
Proposition 5 (Weak Convergence). For any  $p \geq 1$ ,  $T \geq 1$ ,  $M \geq 0$ ,  $N \geq 1$ , and dimension  $d \geq 1$ , if A2 holds, the convergence of probability measures in  $\mathcal{P}_p(\mathbb{R}^d)$  under the burned thinned Markovian sliced Wasserstein distance  $MSW_{p,T,N,M}(\cdot ,\cdot)$  implies weak convergence of probability measures and vice versa.
The proof of Proposition 5 follows directly the proof of Theorem 2 in Appendix B.2.
Proposition 6. For any  $p \geq 1$  and dimension  $d \geq 1$ , for any  $T \geq 1$ ,  $M \geq 0$ ,  $N \geq 1$  and  $\mu, \nu \in \mathcal{P}_p(\mathbb{R}^d)$ ,  $MSW_{p,T,N,M}(\mu, \nu) \leq Max-SW_p(\mu, \nu) \leq W_p(\mu, \nu)$ .
The proof of Proposition 6 follows directly the proof of Proposition 1 in Appendix B.3.
Proposition 7 (Sample Complexity). Let  $X_{1}, X_{2}, \ldots, X_{n}$  be i.i.d. samples from the probability measure  $\mu$  being supported on compact set of  $\mathbb{R}^d$ . We denote the empirical measure  $\mu_{n} = \frac{1}{n} \sum_{i=1}^{n} \delta_{X_{i}}$ . Then, for any  $p \geq 1$  and  $T \geq 1$ ,  $M \geq 0$ ,  $N \geq 1$ , there exists a universal constant  $C > 0$  such that
$$
\mathbb {E} \left[ M S W _ {p, T, N, M} \left(\mu_ {n}, \mu\right) \right] \leq C \sqrt {(d + 1) \log n / n},
$$
The proof of Proposition 7 follows directly the proof of Proposition 2 in Appendix B.4.
Proposition 8 (Monte Carlo error). For any  $p \geq 1$ ,  $T \geq 1$ ,  $M \geq 0$ ,  $N \geq 1$ , dimension  $d \geq 1$ , and  $\mu, \nu \in \mathcal{P}_p(\mathbb{R}^d)$ , we have:
$$
\begin{array}{l} \mathbb {E} \left| \widehat {M S W} _ {p, T, N, M} ^ {p} (\mu , \nu) - M S W _ {p, T, N, M} ^ {p} (\mu , \nu) \right| \\ \leq \frac {N}{\sqrt {L} T (T - M)} \operatorname {V a r} \left[ \sum_ {t = 1} ^ {(T - M) / N} W _ {p} ^ {p} \left(\theta_ {t} ^ {\prime} \sharp \mu , \theta_ {t} ^ {\prime} \sharp \nu\right) \right] ^ {\frac {1}{2}}, \\ \end{array}
$$
where the variance is with respect to  $\sigma (\theta_1^{\prime},\dots ,\theta_{(T - M) / N}^{\prime})$
The proof of Proposition 8 follows directly the proof of Proposition 3 in Appendix B.5.
A.5 Discussions on Related Works
K-SW is autoregressive decomposition. In MSW, we assume that the joint distribution over projecting directions has the first-order Markov structure:  $\sigma (\theta_1,\dots ,\theta_T) = \sigma_1(\theta_1)\prod_{t = 2}^T\sigma_t(\theta_t|\theta_{t - 1})$ . However, we can consider the full autoregressive decomposition  $\sigma (\theta_1,\ldots ,\theta_T) = \sigma_1(\theta_1)\prod_{t = 2}^T\sigma_t(\theta_t|\theta_1,\ldots ,\theta_{t - 1})$ . Let  $T = K$  in K-SW, hence the transition distribution that is used in K-SW is:  $\sigma_t(\theta_t|\theta_1,\dots ,\theta_{t - 1}) = \mathrm{Gram - Schmidschmidt}_{\theta_1,\dots ,\theta_{t - 1}}\sharp \mathcal{U}(\mathbb{S}^{d - 1})$ , where Gram-Schmidt $_{\theta_1,\dots ,\theta_{t - 1}}(\theta_t)$  denotes the Gram-Schmidt process update that applies on  $\theta_t$ .
Generalization of Max-K-SW. Similar to Max-SW, we can derive a Markovian-based K-sliced Wasserstein distance that generalizes the idea of the projected gradient ascent update in Max-K-SW. However, the distance considers the transition on the Stiefel manifold instead of the unit hypersphere, hence, it will be more computationally expensive. Moreover, orthogonality might not be a good constraint. Therefore, the generalization of Max-K-SW might not have many advantages.
Beyond the projected sub-gradient ascent update. In the input-aware transition for MSW, we utilize the projected sub-gradient update as the transition function to create a new projecting direction. Therefore, we could other optimization techniques such as momentum, adaptive stepsize, and so on to create the transition function. We will leave the investigation about this direction to future work.
Applications to other sliced Wasserstein variants. The Markovian approach can be applied to other variants of sliced Wasserstein distances e.g., generalized sliced Wasserstein [26], augmented sliced Wasserstein distance [10], projected robust Wasserstein (PRW) [47, 32, 22] ( $k > 1$  dimensional projection), convolution sliced Wasserstein [43], sliced partial optimal transport [6, 2], and so on.
Markovian sliced Wasserstein distances in other applications. We can apply MSW to the setting in [31] which is an implementation technique that utilizes both RAM and GPUs' memory for training sliced Wasserstein generative models. MSW can also replace sliced Wasserstein distance in pooling in [38]. Similarly, MSW can be used in applications that exist sliced Wasserstein distance e.g., clustering [28], Bayesian inference [39, 61], domain adaptation [60], and so on.
(i), (ii): the MSW is an expectation of the one-dimensional Wasserstein distance hence the non-negativity and symmetry properties of the MSW follow directly by the non-negativity and symmetry of the Wasserstein distance.
(iii) From the definition of MSW in Definition 1, given three probability measures  $\mu_1,\mu_2,\mu_3\in$ $\mathcal{P}_p(\mathbb{R}^d)$  we have:
$$
\begin{array}{l} \mathrm {M S W} _ {p, T} \left(\mu_ {1}, \mu_ {3}\right) = \left(\mathbb {E} _ {\left(\theta_ {1: T}\right) \sim \sigma \left(\theta_ {1: T}\right)} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} W _ {p} ^ {p} \left(\theta_ {t} \sharp \mu_ {1}, \theta_ {t} \sharp \mu_ {3}\right) \right]\right) ^ {\frac {1}{p}} \\ \leq \left(\mathbb {E} _ {\left(\theta_ {1: T}\right) \sim \sigma \left(\theta_ {1: T}\right)} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} \left(W _ {p} \left(\theta_ {t} \sharp \mu_ {1}, \theta_ {t} \sharp \mu_ {2}\right) + W _ {p} \left(\theta_ {t} \sharp \mu_ {2}, \theta_ {t} \sharp \mu_ {3}\right)\right) ^ {p} \right]\right) ^ {\frac {1}{p}} \\ \leq \left(\mathbb {E} _ {\left(\theta_ {1: T}\right) \sim \sigma \left(\theta_ {1: T}\right)} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} W _ {p} ^ {p} \left(\theta_ {t} \sharp \mu_ {1}, \theta_ {t} \sharp \mu_ {2}\right) \right]\right) ^ {\frac {1}{p}} \\ \left. + \left(\mathbb {E} _ {\left(\theta_ {1: T}\right) \sim \sigma \left(\theta_ {1: T}\right)} \left[ \frac {1}{T} \sum_ {t = 1} ^ {T} W _ {p} ^ {p} \left(\theta_ {t} \sharp \mu_ {2}, \theta_ {t} \sharp \mu_ {3}\right) \right]\right)\right) ^ {\frac {1}{p}} \\ = \mathrm {M S W} _ {p, T} \left(\mu_ {1}, \mu_ {2}\right) + \mathrm {M S W} _ {p, T} \left(\mu_ {2}, \mu_ {3}\right), \\ \end{array}
$$
where the first inequality is due to the triangle inequality of Wasserstein distance and the second inequality is due to the Minkowski inequality. We complete the triangle inequality proof.
(iv) We need to show that  $\mathrm{MSW}_{p,T}(\mu, \nu) = 0$  if and only if  $\mu = \nu$ . First, from the definition of MSW, we obtain directly  $\mu = \nu$  implies  $\mathrm{MSW}_{p,T}(\mu, \nu) = 0$ . For the reverse direction, we use the same proof technique in [8]. If  $\mathrm{MSW}_{p,T}(\mu, \nu) = 0$ , we have  $\int_{\mathbb{S}(d-1) \otimes T} \frac{1}{T} \sum_{t=1}^{T} \mathrm{W}_p(\theta_t \sharp \mu, \theta_t \sharp \nu) \, \mathrm{d}\sigma(\theta_{1:T}) = 0$ . If A1 holds, namely, the prior distribution  $\sigma_1(\theta_1)$  is supported on all the unit-hypersphere or exists a transition distribution  $\sigma_t(\theta_t | \theta_{t-1})$  is supported on all the unit-hypersphere, we have  $W_p(\theta \sharp \mu, \theta \sharp \nu) = 0$  for all  $\theta \in \mathbb{S}^{d-1}$  where  $\sigma$  denotes the prior or the transition distribution that satisfies the assumption A1. From the identity property of the Wasserstein distance, we obtain  $\theta \sharp \mu = \theta \sharp \nu$  for  $\sigma$ -a.e  $\theta \in \mathbb{S}^{d-1}$ . Therefore, for any  $t \in \mathbb{R}$  and  $\theta \in \mathbb{S}^{d-1}$ , we have:
Our goal is to show that for any sequence of probability measures  $(\mu_k)_{k \in \mathbb{N}}$  and  $\mu$  in  $\mathcal{P}_p(\mathbb{R}^d)$ ,  $\lim_{k \to +\infty} \mathrm{MSW}_{p,T}(\mu_k, \mu) = 0$  if and only if for any continuous and bounded function  $f: \mathbb{R}^d \to \mathbb{R}$ ,  $\lim_{k \to +\infty} \int f \, \mathrm{d}\mu_k = \int f \, \mathrm{d}\mu$ . The proof follows the techniques in [41]. We first state the following lemma.
Lemma 1. For any  $T \geq 1$ , and dimension  $d \geq 1$ , if  $\mathbf{A}1$  holds and a sequence of probability measures  $(\mu_k)_{k \in \mathbb{N}}$  satisfies  $\lim_{k \to +\infty} MSW_{1,T}(\mu_k, \mu) = 0$  with  $\mu$  in  $\mathcal{P}(\mathbb{R}^d)$ , there exists an increasing function  $\phi: \mathbb{N} \to \mathbb{N}$  such that the subsequence  $(\mu_{\phi(k)})_{k \in \mathbb{N}}$  converges weakly to  $\mu$ .
Proof. We are given that  $\lim_{k\to +\infty}\mathbf{MSW}_{1,T}(\mu_k,\mu) = 0$ , therefore  $\lim_{k\to \infty}\int_{\mathbb{S}(d - 1)\otimes T}\frac{1}{T}\sum_{t = 1}^{T}\mathbf{W}_1(\theta_t\sharp \mu_k,\theta_t\sharp \mu)\mathrm{d}\sigma (\theta_{1:T}) = 0$ . If  $\mathbf{A1}$  holds, namely, the prior
distribution  $\sigma_{1}(\theta_{1})$  is supported on all the unit-hypersphere or exists a transition distribution  $\sigma_{t}(\theta_{t}|\theta_{t - 1})$  is supported on all the unit-hypersphere, we have
$$
\lim  _ {k \rightarrow \infty} \int_ {\mathbb {S} ^ {d - 1}} \mathrm {W} _ {1} \left(\theta \sharp \mu_ {k}, \theta \sharp \mu\right) \mathrm {d} \boldsymbol {\sigma} (\theta) = 0,
$$
where  $\sigma$  denotes the prior or the transition distribution that satisfies the assumption A1. From Theorem 2.2.5 in [3], there exists an increasing function  $\phi : \mathbb{N} \to \mathbb{N}$  such that  $\lim_{k \to \infty} \mathrm{W}_1(\theta \sharp \mu_{\phi(k)}, \theta \sharp \nu) = 0$  for  $\sigma$ -a.e  $\theta \in \mathbb{S}^{d-1}$ . Since the Wasserstein distance of implies weak convergence [58],  $\left(\theta \sharp \mu_{\phi(k)}\right)_{k \in \mathbb{N}}$  converges weakly to  $\theta \sharp \mu$  for  $\sigma$ -a.e  $\theta \in \mathbb{S}^{d-1}$ .
Let  $\Phi_{\mu} = \int_{\mathbb{R}^d}\mathrm{e}^{\mathrm{i}\langle v,w\rangle}\mathrm{d}\mu (w)$  be the characteristic function of  $\mu \in \mathcal{P}(\mathbb{R}^d)$ , we have the weak convergence implies the convergence of characteristic function (Theorem 4.3 [24]):  $\lim_{k\to \infty}\Phi_{\theta^{\sharp}\mu_{\phi (k)}}(s) = \Phi_{\theta^{\sharp}\mu}(s),\quad \forall s\in \mathbb{R},$  for  $\sigma$ -a.e  $\theta \in \mathbb{S}^{d - 1}$ . Therefore,  $\lim_{k\to \infty}\Phi_{\mu_{\phi (k)}}(z) = \Phi_{\mu}(z)$ , for almost most every  $z\in \mathbb{R}^d$ .
For any  $\gamma > 0$  and a continuous function  $f: \mathbb{R}^d \to \mathbb{R}$  with compact support, we denote  $f_{\gamma}(x) = f * g_{\gamma}(x) = (2\pi \gamma^2)^{-d/2} \int_{\mathbb{R}^d} f(x - z) \exp \left(-\|z\|^2 / (2\gamma^2)\right) \, \mathrm{d}z$  where  $g_{\gamma}$  is the density function of  $\mathcal{N}(0, \gamma I_d)$ . We have:
$$
\begin{array}{l} \int_ {\mathbb {R} ^ {d}} f _ {\gamma} (z) \mathrm {d} \mu_ {\phi (k)} (z) = \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) g _ {\gamma} (z - w) \mathrm {d} w \mathrm {d} \mu_ {\phi (k)} (z) \\ = \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \exp \left(- | | z - w | | ^ {2} / (2 \gamma^ {2})\right) \mathrm {d} w \mathrm {d} \mu_ {\phi (k)} (z) \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \int_ {\mathbb {R} ^ {d}} e ^ {i \langle z - w, x \rangle} g _ {1 / \gamma} (x) d x d w d \mu_ {\phi (k)} (z) \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \int_ {\mathbb {R} ^ {d}} e ^ {- \mathrm {i} \langle w, x \rangle} e ^ {\mathrm {i} \langle z, x \rangle} g _ {1 / \gamma} (x) \mathrm {d} x \mathrm {d} w \mathrm {d} \mu_ {\phi (k)} (z) \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) e ^ {- \mathrm {i} \langle w, x \rangle} g _ {1 / \gamma} (x) \int_ {\mathbb {R} ^ {d}} e ^ {\mathrm {i} \langle z, x \rangle} \mathrm {d} \mu_ {\phi (k)} (z) \mathrm {d} x \mathrm {d} w \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \mathrm {e} ^ {- \mathrm {i} \langle w, x \rangle} g _ {1 / \gamma} (x) \Phi_ {\mu_ {\phi (k)}} (x) \mathrm {d} x \mathrm {d} w \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \mathcal {F} [ f ] (x) g _ {1 / \gamma} (x) \Phi_ {\mu_ {\phi (k)}} (x) \mathrm {d} x, \\ \end{array}
$$
where the third equality is due to the fact that  $\int_{\mathbb{R}^d}e^{\mathrm{i}\langle z - w,x\rangle}g_{1 / \gamma}(x)\mathrm{d}x = \exp (-||z - w||^2 /(2\gamma^2))$  and  $\mathcal{F}[f](w) = \int_{\mathbb{R}^{d'}}f(x)e^{-i\langle w,x\rangle}dx$  denotes the Fourier transform of the bounded function  $f$ . Similarly, we have
$$
\begin{array}{l} \int_ {\mathbb {R} ^ {d}} f _ {\gamma} (z) \mathrm {d} \mu (z) = \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) g _ {\gamma} (z - w) \mathrm {d} w \mathrm {d} \mu (z) \\ = \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \exp \left(- | | z - w | | ^ {2} / \left(2 \gamma^ {2}\right)\right) \mathrm {d} w \mathrm {d} \mu (z) \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \int_ {\mathbb {R} ^ {d}} e ^ {i \langle z - w, x \rangle} g _ {1 / \gamma} (x) \mathrm {d} x \mathrm {d} w \mathrm {d} \mu (z) \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \int_ {\mathbb {R} ^ {d}} e ^ {- \mathrm {i} \langle w, x \rangle} e ^ {\mathrm {i} \langle z, x \rangle} g _ {1 / \gamma} (x) \mathrm {d} x \mathrm {d} w \mathrm {d} \mu (z) \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) e ^ {- \mathrm {i} \langle w, x \rangle} g _ {1 / \gamma} (x) \int_ {\mathbb {R} ^ {d}} e ^ {\mathrm {i} \langle z, x \rangle} \mathrm {d} \mu (z) \mathrm {d} x \mathrm {d} w \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \int_ {\mathbb {R} ^ {d}} f (w) \mathrm {e} ^ {- \mathrm {i} \langle w, x \rangle} g _ {1 / \gamma} (x) \Phi_ {\mu} (x) \mathrm {d} x \mathrm {d} w \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \mathcal {F} [ f ] (x) g _ {1 / \gamma} (x) \Phi_ {\mu} (x) \mathrm {d} x. \\ \end{array}
$$
Since  $f$  is assumed to have compact support,  $\mathcal{F}[f]$  exists and is bounded by  $\int_{\mathbb{R}^d} |f(w)| \, \mathrm{d}w < +\infty$ . Hence, for any  $k \in \mathbb{R}$  and  $x \in \mathbb{R}^d$ , we have  $\left| \mathcal{F}[f](x) g_{1/\gamma}(x) \Phi_{\mu_{\phi(k)}}(x) \right| \leq$
$g_{1 / \gamma}(x)\int_{\mathbb{R}^d}|f(w)|\mathrm{d}w$  and  $\left|\mathcal{F}[f](x)g_{1 / \gamma}(x)\Phi_\mu (x)\right|\leq g_{1 / \gamma}(x)\int_{\mathbb{R}^d}|f(w)|\mathrm{d}w.$  Using the proved result of  $\lim_{k\to \infty}\Phi_{\mu_{\phi (k)}}(z) = \Phi_\mu (z)$  and Lebesgue's Dominated Convergence Therefore, we obtain
$$
\begin{array}{l} \lim _ {k \to \infty} \int_ {\mathbb {R} ^ {d}} f _ {\gamma} (z) \mathrm {d} \mu_ {\phi (k)} (z) = \lim _ {k \to \infty} \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \mathcal {F} [ f ] (x) g _ {1 / \gamma} (x) \Phi_ {\mu_ {\phi (k)}} (x) \mathrm {d} x \\ = \left(2 \pi \gamma^ {2}\right) ^ {- d / 2} \int_ {\mathbb {R} ^ {d}} \mathcal {F} [ f ] (x) g _ {1 / \gamma} (x) \Phi_ {\mu_ {\phi (k)}} (x) \mathrm {d} x \\ = \int_ {\mathbb {R} ^ {d}} f _ {\gamma} (z) \mathrm {d} \mu (z). \\ \end{array}
$$
Moreover, we have:
$$
\begin{array}{l} \lim  _ {\gamma \rightarrow 0} \lim  _ {k \rightarrow + \infty} \sup  _ {k \rightarrow + \infty} \left| \int_ {\mathbb {R} ^ {d}} f (z) \mathrm {d} \mu_ {\phi (k)} (z) - \int_ {\mathbb {R} ^ {d}} f (z) \mathrm {d} \mu (z) \right| \\ \leq \lim  _ {\gamma \rightarrow 0} \lim  _ {k \rightarrow + \infty} \left[ 2 \sup  _ {z \in \mathbb {R} ^ {d}} | f (z) - f _ {\gamma} (z) | + \left| \int_ {\mathbb {R} ^ {d}} f _ {\gamma} (z) \mathrm {d} \mu_ {\phi (k)} (z) - \int_ {\mathbb {R} ^ {d}} f _ {\gamma} (z) \mathrm {d} \mu (z) \right|\right] \\ = \lim  _ {\gamma \to 0} 2 \sup  _ {z \in \mathbb {R} ^ {d}} | f (z) - f _ {\gamma} (z) | = 0, \\ \end{array}
$$
which implies  $\left(\mu_{\phi (k)}\right)_{k\in \mathbb{N}}$  converges weakly to  $\mu$