SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By Gronwall's inequality, we obtain that
$$
D _ {p} ^ {p} \left(\pi^ {0}\right) (t) \leq D _ {p} ^ {p} (0) \exp \left(C _ {V, R} e ^ {C T} t\right).
$$
Now since  $\pi^0\in \Gamma (\nu_1,\nu_2)$  and  $\mu_{i,t} = (X(t,\cdot ,\nu_i))_{\#}\nu_i$  , the mapping
$$
\Xi_ {t}: (\boldsymbol {x} _ {1}, \boldsymbol {x} _ {2}) \in \mathbb {R} ^ {2 d} \mapsto (X (t, \boldsymbol {x} _ {1}, \nu_ {1}), X (t, \boldsymbol {x} _ {2}, \nu_ {2})) \in \mathbb {R} ^ {2 d}
$$
satisfies that  $(\Xi_{t})_{\#}\pi^{0}\in \Gamma (\mu_{1,t},\mu_{2,t})$  . As a consequence we have that
$$
\begin{array}{l} \mathcal {W} _ {p} ^ {p} \left(\mu_ {1, t}, \mu_ {2, t}\right) = \inf  _ {\pi \in \Gamma \left(\mu_ {1, t}, \mu_ {2, t}\right)} \int_ {\mathbb {R} ^ {2 d}} \| \boldsymbol {x} _ {1} - \boldsymbol {x} _ {2} \| ^ {p} \pi \left(\mathrm {d} \boldsymbol {x} _ {1} \mathrm {d} \boldsymbol {x} _ {2}\right) \\ \leq \inf  _ {\pi^ {0} \in \Gamma (\nu_ {1}, \nu_ {2})} D _ {p} ^ {p} (\pi^ {0}) (t) \\ \leq \exp \left(C _ {V, R} e ^ {C T} T\right) \inf  _ {\pi^ {0} \in \Gamma (\nu_ {1}, \nu_ {2})} D _ {p} ^ {p} \left(\pi^ {0}\right) (0) \\ = \exp \left(C _ {V, R} e ^ {C T} T\right) \cdot \mathcal {W} _ {p} ^ {p} (\nu_ {1}, \nu_ {2}). \\ \end{array}
$$
I Proofs for Section 3.1
Proof of Theorem 3.1. For any  $\theta = (\pmb {\mu},\Sigma)\in \Theta$  , define  $\widetilde{E} (\pmb {\mu},\Sigma)\coloneqq E(\rho)$  . Then we have
$$
\widetilde {E} (\boldsymbol {\mu}, \Sigma) = \mathrm {K L} (\rho \| \rho^ {*}) = \frac {1}{2} \left(\operatorname {t r} (Q ^ {- 1} \Sigma) - \log \det  (Q ^ {- 1} \Sigma) - d + (\boldsymbol {\mu} - \boldsymbol {b}) ^ {\top} Q ^ {- 1} (\boldsymbol {\mu} - \boldsymbol {b})\right),
$$
where  $\rho$  is the density of  $\mathcal{N}(\pmb {\mu},\Sigma)$  and  $\rho^{*}$  is the density of  $\mathcal{N}(\pmb {b},Q)$
Now we consider the gradient flow on the submanifold  $\Theta$
$$
\dot {\theta} _ {t} = - G _ {\theta_ {t}} ^ {- 1} (\nabla_ {\theta_ {t}} \widetilde {E} (\theta_ {t})).
$$
For clarity note that here
$$
\nabla_ {\theta_ {t}} \widetilde {E} (\theta_ {t}) := \left(\nabla_ {\boldsymbol {\mu} _ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}), \nabla_ {\Sigma_ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t})\right),
$$
where  $\nabla_{\Sigma}\widetilde{E} (\pmb {\mu},\Sigma)$  denotes the standard matrix derivative (not the covariant derivative or affine connection in the contexts of Riemannian geometry). We calculate that
$$
\nabla_ {\boldsymbol {\mu}} \widetilde {E} (\boldsymbol {\mu}, \Sigma) = Q ^ {- 1} (\boldsymbol {\mu} - \boldsymbol {b}), \quad \nabla_ {\Sigma} \widetilde {E} (\boldsymbol {\mu}, \Sigma) = \frac {1}{2} (Q ^ {- 1} - \Sigma^ {- 1}).
$$
Thus, the gradient flow on  $\Theta$  is equivalent to
$$
\begin{array}{l} \Leftrightarrow \left\{ \begin{array}{l} \dot {\boldsymbol {\mu}} _ {t} = - \left(2 \nabla_ {\Sigma_ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) \Sigma_ {t} \boldsymbol {\mu} _ {t} + (1 + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t}) \nabla_ {\boldsymbol {\mu} _ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t})\right) \\ \dot {\Sigma} _ {t} = - \Sigma_ {t} \left(2 \Sigma_ {t} \nabla_ {\Sigma_ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) + \boldsymbol {\mu} _ {t} \nabla_ {\boldsymbol {\mu} _ {t}} ^ {\top} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t})\right) \\ \qquad - \left(2 \nabla_ {\Sigma_ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) \Sigma_ {t} + \nabla_ {\boldsymbol {\mu} _ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) \boldsymbol {\mu} _ {t} ^ {\top}\right) \Sigma_ {t} \end{array} \right. \\ \Leftrightarrow \left\{ \begin{array}{l} \dot {\boldsymbol {\mu}} _ {t} = (I - Q ^ {- 1} \Sigma_ {t}) \boldsymbol {\mu} _ {t} - (1 + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t}) Q ^ {- 1} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) \\ \dot {\Sigma} _ {t} = 2 \Sigma_ {t} - \Sigma_ {t} \left(\Sigma_ {t} + \boldsymbol {\mu} _ {t} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) ^ {\top}\right) Q ^ {- 1} - Q ^ {- 1} \left(\Sigma_ {t} + (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) \boldsymbol {\mu} _ {t} ^ {\top}\right) \Sigma_ {t}. \end{array} \right. \tag {75} \\ \end{array}
$$
Note that it is trivial to check that the functions on the right-hand-side of (3.1) are locally Lipschitz with respect to  $\pmb{\mu}_t$  and  $\Sigma_t$  (continuously differentiable hence locally Lipschitz). By Picard-Lindelöf theorem, this ODE system given  $\pmb{\mu}_0 \in \mathbb{R}^d$ ,  $\Sigma_0 \in \mathrm{Sym}^+(d, \mathbb{R})$  has a unique solution on  $t \in [0, \epsilon)$  for some  $\epsilon > 0$ . Let
$$
s := \sup  \left\{t \in \mathbb {R} ^ {+} \cup \{\infty \}: (7 5) \text {h a s a (u n i q u e) s o l u t i o n o n} [ 0, s) \right\}.
$$
For convenience we define the curve on  $\Theta = \mathbb{R}^d\times \mathrm{Sym}^+ (d,\mathbb{R})$  by
$$
\gamma : [ 0, s) \rightarrow \mathbb {R} ^ {d} \times \operatorname {S y m} (d, \mathbb {R}), \quad \gamma (t) := (\boldsymbol {\mu} _ {t}, \Sigma_ {t}).
$$
Next we consider the density flow given by
$$
\dot {\rho} _ {t} = - G _ {\rho_ {t}} ^ {- 1} \frac {\delta E (\rho_ {t})}{\delta \rho_ {t}} = \nabla \cdot \left(\rho_ {t} (\cdot) \int K (\cdot , \boldsymbol {y}) \left(\nabla \rho_ {t} (\boldsymbol {y}) + \rho_ {t} (\boldsymbol {y}) \nabla V (\boldsymbol {y})\right) d \boldsymbol {y}\right). \tag {76}
$$
By Theorem C.2 we know that there is a unique solution  $\rho_t$  in  $\mathcal{P}(\mathbb{R}^d)$  for  $t\in [0,\infty)$ . We claim that
Claim.  $\rho_t$  is a Gaussian density for  $t\in [0,s)$
In fact, if we let  $\widetilde{\rho}_t\coloneqq \phi (\gamma (t))$  , where  $\phi$  is the immersion
$$
\begin{array}{l} \begin{array}{c c c c} \phi : & \Theta & \to & \mathcal {P} (\mathbb {R} ^ {d}) \end{array} \\ \theta \quad \mapsto \quad \rho (\cdot , \theta). \\ \end{array}
$$
By uniqueness of the solution it suffices to prove that (76) holds for  $\rho_t = \widetilde{\rho}_t$ . This can be checked by direct calculation of course. But there is also a more elegant way to show it. We consider the following commutative diagram.
$$
\begin{array}{l} T _ {\theta} \Theta \xrightarrow {\mathrm {d} \phi_ {\theta}} T _ {\tilde {\rho}} \mathcal {P} (\mathbb {R} ^ {d}) \\ G _ {\theta} \Bigg \downarrow \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \left\downarrow G _ {\tilde {\rho}} \left. \right. \\ T _ {\theta} ^ {*} \Theta \xrightarrow {\psi_ {\theta}} T _ {\tilde {\rho}} ^ {*} \mathcal {P} (\mathbb {R} ^ {d}) \\ \end{array}
$$
Here  $\mathrm{d}\phi_{\theta}$  is the pushforward of the immersion  $\phi$  at point  $\theta$  and  $\psi_{\theta}$  is the inverse of the pullback map  $\phi^{*}:T_{\tilde{\rho}}^{*}\mathcal{P}(\mathbb{R}^{d})\to T_{\theta}^{*}\Theta$  restricted on  $\operatorname {Im}G_{\widetilde{\rho}_t}\circ \mathrm{d}\phi_\theta (\simeq T_\theta^*\Theta)$ . The diagram is commutative due to the fact that  $G_{\theta}$  is the canonical isomorphism on the submanifold  $\Theta$  induced from  $\mathcal{P}(\mathbb{R}^d)$ .
Now we show that  $\frac{\delta E}{\delta \widetilde{\rho}_t} \in \operatorname{Im} G_{\widetilde{\rho}_t^\circ} \mathrm{d}\phi_\theta$ . In the proof of Theorem B.3, we have shown that  $\psi_\theta$  maps  $(\pmb{b}, \frac{1}{2} S)$  to some  $\Phi$  such that  $\nabla \Phi(\pmb{x}) = S(\pmb{x} - \pmb{\mu}) + \pmb{b}$ , where  $\pmb{\mu}$  is obtained from  $G_\theta^{-1}(\pmb{b}, \frac{1}{2} S)$ . Thus,  $\operatorname{Im} \psi_\theta$  contains all functions (or more precisely the equivalent classes of functions that differ by a constant) with the form
$$
\Phi (\boldsymbol {x}) = \frac {1}{2} (\boldsymbol {x} - \boldsymbol {\mu}) ^ {\top} S (\boldsymbol {x} - \boldsymbol {\mu}) + \boldsymbol {b x} + C, \quad S \in \operatorname {S y m} (d, \mathbb {R}), \boldsymbol {b} \in \mathbb {R} ^ {d}, C \text {i s a n y c o n s t a n t}.
$$
In other words,  $\operatorname{Im}\psi_{\theta}$  contains all quadratic forms on  $\mathbb{R}^d$ . Note that we can derive that
$$
\frac {\delta E (\widetilde {\rho} _ {t})}{\delta \widetilde {\rho} _ {t}} = \log \widetilde {\rho} _ {t} + V
$$
is exactly a quadratic form. Thus,
$$
\frac {\delta E}{\delta \widetilde {\rho} _ {t}} \in \operatorname {I m} \psi_ {\theta_ {t}} = \operatorname {I m} \psi_ {\theta_ {t}} \circ G _ {\theta_ {t}} = \operatorname {I m} G _ {\widetilde {\rho} _ {t}} \circ \mathrm {d} \phi_ {\theta}.
$$
Next since  $\mathrm{d}\phi_{\theta_t}$  maps the tangent vector  $\frac{\partial}{\partial\theta_t} \in T_{\theta_t}\Theta$  to  $\frac{\delta}{\delta\widetilde{\rho_t}} \in T_{\widetilde{\rho_t}}\mathcal{P}(\mathbb{R}^d)$ , we have that
$$
\phi^ {*} \frac {\delta E}{\delta \widetilde {\rho} _ {t}} = \nabla_ {\theta_ {t}} \widetilde {E}.
$$
Combining this with the fact that  $\frac{\delta E}{\delta \widetilde{\rho}_t} \in \operatorname{Im} G_{\widetilde{\rho}_t} \circ \mathrm{d}\phi_\theta$ , we get
$$
\frac {\delta E}{\delta \widetilde {\rho} _ {t}} = \psi_ {\theta} (\nabla_ {\theta_ {t}} \widetilde {E}).
$$
$$
G _ {\tilde {\rho} _ {t}} ^ {- 1} \frac {\delta E}{\delta \tilde {\rho} _ {t}} = G _ {\tilde {\rho} _ {t}} ^ {- 1} \psi_ {\theta} (\nabla_ {\theta_ {t}} \widetilde {E}) = \mathrm {d} \phi_ {\theta_ {t}} G _ {\theta_ {t}} ^ {- 1} (\nabla_ {\theta_ {t}} \widetilde {E}).
$$
And we conclude
$$
\dot {\widetilde {\rho}} _ {t} = \mathrm {d} \phi_ {\theta_ {t}} \dot {\theta} _ {t} = - \mathrm {d} \phi_ {\theta_ {t}} G _ {\theta_ {t}} ^ {- 1} (\nabla_ {\theta_ {t}} \widetilde {E}) = - G _ {\widetilde {\rho} _ {t}} ^ {- 1} \frac {\delta E}{\delta \widetilde {\rho} _ {t}}.
$$
The claim is proven.
Now back to the original problem. Suppose  $s < \infty$ . Since we know that the mean field PDE (76) has a unique solution on  $[0, \infty)$ , in particular, it exists on  $[0, s]$ . Note that the weak limit of Gaussian distributions is Gaussian and since  $\rho_s \in \mathcal{P}(\mathbb{R}^d)$  it does not degenerate. By definition of  $\phi$  we have that  $\phi^{-1}(\rho_s) \in \Theta$ . By letting  $(\pmb{\mu}_s, \Sigma_s) = \gamma(s) \coloneqq \phi^{-1}(\rho_s)$ , we obtain the solution of (75) on  $[0, s]$ . Again by Picard-Lindelöf theorem there exists a small neighborhood  $[s, s + \epsilon')$  such that (75) has a unique solution. This together with the solution on  $[0, s]$  contradicts the definition of  $s$ . Therefore, we conclude that  $s = \infty$ . (75) has a unique global solution corresponding to the mean and covariance matrix of  $\rho_t$ .
Next, we prove that  $\rho_{t}$  converges weakly to  $\rho^{*}$  as  $t\to \infty$ . We calculate the quantity  $\widetilde{E} (\pmb {\mu}_t,\Sigma_t)$ . By Jacobi's formula in matrix calculus (Theorem 8.1 in [59]), we have
$$
\partial_ {t} \det  \Sigma_ {t} = \det  \Sigma_ {t} \operatorname {t r} \left(\Sigma_ {t} ^ {- 1} \dot {\Sigma} _ {t}\right).
$$
Thus, we derive that
$$
\begin{array}{l} \dot {\tilde {E}} \left(\boldsymbol {\mu} _ {t}, \Sigma_ {t}\right) = \frac {1}{2} \operatorname {t r} \left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \dot {\Sigma} _ {t}\right) + \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) ^ {\top} Q ^ {- 1} \dot {\boldsymbol {\mu}} _ {t} \\ = - \operatorname {t r} \left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) ^ {2} \Sigma_ {t} ^ {2}\right) - 2 \operatorname {t r} \left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \Sigma_ {t} \boldsymbol {\mu} _ {t} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) ^ {\top} Q ^ {- 1}\right) \\ - \left(1 + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t}\right) \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) ^ {\top} Q ^ {- 2} \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) \\ = - \operatorname {t r} \left(\left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \Sigma_ {t} + Q ^ {- 1} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) \boldsymbol {\mu} _ {t} ^ {\top}\right) ^ {\top} \left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \Sigma_ {t} + Q ^ {- 1} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) \boldsymbol {\mu} _ {t} ^ {\top}\right)\right) \\ - \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) ^ {\top} Q ^ {- 2} \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) \leq 0. \\ \end{array}
$$
Noticing that
$$
0 \leq - \int_ {0} ^ {t} \dot {\tilde {E}} (\boldsymbol {\mu} _ {s}, \Sigma_ {s}) d s = \tilde {E} (\boldsymbol {\mu} _ {0}, \Sigma_ {0}) - \tilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) <   \infty ,
$$
we obtain that  $\hat{\tilde{E}} (\pmb {\mu}_t,\Sigma_t)\to 0$  as  $t\rightarrow \infty$  , which is equivalent to  $\pmb {\mu}_t\rightarrow \pmb{b}$  and  $\Sigma_t\rightarrow Q$  by checking the expression above. Thus, we have shown that  $\rho_{t}$  converges weakly to  $\rho^{*}$  which is the density function of  $\mathcal{N}(\boldsymbol {b},Q)$
Finally, we show the convergence rates of  $\pmb{\mu}_t$  and  $\Sigma_t$ . Since we have already proven that  $\rho_t \to \rho^*$ , it implies that
$$
\boldsymbol {\mu} _ {t} - \boldsymbol {b} = o (1), \quad \Sigma_ {t} - Q = o (1), \quad \Sigma_ {t} ^ {- 1} - Q ^ {- 1} = o (1).
$$
If we set  $\pmb{\eta}_t = Q^{-1}(\pmb{\mu}_t - \pmb{b})$  and  $S_{t} = (Q^{-1} - \Sigma_{t}^{-1})\Sigma_{t}$ , then
$$
\begin{array}{l} - \dot {\tilde {E}} (\pmb {\mu} _ {t}, \Sigma_ {t}) \\ = \operatorname {t r} \left(\left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \Sigma_ {t} + Q ^ {- 1} \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) \boldsymbol {\mu} _ {t} ^ {\top}\right) ^ {\top} \left(\left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \Sigma_ {t} + Q ^ {- 1} \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) \boldsymbol {\mu} _ {t} ^ {\top}\right)\right) \\ + \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) ^ {\top} Q ^ {- 2} \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) \\ = \operatorname {t r} \left(\left(S _ {t} + \boldsymbol {\eta} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right) ^ {\top} \left(S _ {t} + \boldsymbol {\eta} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right)\right) + \boldsymbol {\eta} _ {t} ^ {\top} \boldsymbol {\eta} _ {t} \\ = \left[ \operatorname {v e c} ^ {\top} (S _ {t}) \quad \boldsymbol {\eta} _ {t} ^ {\top} \right] \left[ \begin{array}{c c} I _ {d ^ {2}} & \boldsymbol {\mu} _ {t} \otimes I _ {d} \\ \boldsymbol {\mu} _ {t} ^ {\top} \otimes I _ {d} & (1 + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t}) I _ {d} \end{array} \right] \left[ \begin{array}{c} \operatorname {v e c} (S _ {t}) \\ \boldsymbol {\eta} _ {t} \end{array} \right] \\ = \left[ \operatorname {v e c} ^ {\top} (S _ {t}) \quad \boldsymbol {\eta} _ {t} ^ {\top} \right] \left[ \begin{array}{c c} I _ {d ^ {2}} & \boldsymbol {b} _ {t} \otimes I _ {d} \\ \boldsymbol {b} _ {t} ^ {\top} \otimes I _ {d} & (1 + \boldsymbol {b} _ {t} ^ {\top} \boldsymbol {b} _ {t}) I _ {d} \end{array} \right] \left[ \begin{array}{c} \operatorname {v e c} (S _ {t}) \\ \boldsymbol {\eta} _ {t} \end{array} \right] + o (\| S _ {t} \| + \| \boldsymbol {\eta} _ {t} \|). \\ \end{array}
$$
On the other hand,  $\widetilde{E} (\pmb{\mu}_t,\Sigma_t)$  can be written as
$$
\begin{array}{l} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) = \frac {1}{2} \left(\operatorname {t r} (Q ^ {- 1} \Sigma) - \log \det  (Q ^ {- 1} \Sigma) - d + (\boldsymbol {\mu} - \boldsymbol {b}) ^ {\top} Q ^ {- 1} (\boldsymbol {\mu} - \boldsymbol {b})\right) \\ = \frac {1}{2} \left(\operatorname {t r} \left(S _ {t}\right) - \log \det  \left(I _ {d} + S _ {t}\right) + \boldsymbol {\eta} _ {t} ^ {\top} Q \boldsymbol {\eta} _ {t}\right) \\ = \frac {1}{4} \operatorname {t r} \left(S _ {t} ^ {\top} S _ {t}\right) + \frac {1}{2} \boldsymbol {\eta} _ {t} ^ {\top} Q \boldsymbol {\eta} _ {t} + o \left(\left\| S _ {t} \right\| ^ {2}\right) \\ = \frac {1}{4} \left[ \operatorname {v e c} ^ {\top} \left(S _ {t}\right) \quad \boldsymbol {\eta} _ {t} ^ {\top} \right] \left[ \begin{array}{c c} I _ {d ^ {2}} & \\ & 2 Q \end{array} \right] \left[ \begin{array}{c} \operatorname {v e c} \left(S _ {t}\right) \\ \boldsymbol {\eta} _ {t} \end{array} \right] + o (\| S _ {t} \| ^ {2}) \\ \end{array}
$$
Now we prove that  $\forall \epsilon > 0$  there exists  $T > 0$  such that  $-\dot{\widetilde{E}}(\pmb{\mu}_t, \Sigma_t) \geq 4(\gamma - \epsilon)\widetilde{E}(\pmb{\mu}_t, \Sigma_t)$  for  $t \geq T$ . It suffices to show that
$$
\left[ \begin{array}{c c} I _ {d ^ {2}} & \boldsymbol {b} _ {t} \otimes I _ {d} \\ \boldsymbol {b} _ {t} ^ {\top} \otimes I _ {d} & (1 + \boldsymbol {b} _ {t} ^ {\top} \boldsymbol {b} _ {t}) I _ {d} \end{array} \right] \succeq \gamma \left[ \begin{array}{c c} I _ {d ^ {2}} & \\ & 2 Q \end{array} \right],
$$
$$
\left[ \begin{array}{c c} I _ {d ^ {2}} & \frac {1}{\sqrt {2}} \boldsymbol {b} \otimes Q ^ {- 1 / 2} \\ \frac {1}{\sqrt {2}} \boldsymbol {b} ^ {\top} \otimes Q ^ {- 1 / 2} & \frac {1}{2} (1 + \boldsymbol {b} ^ {\top} \boldsymbol {b}) Q ^ {- 1} \end{array} \right] \succeq \gamma I _ {d ^ {2} + d}.
$$
This is true because by definition  $\gamma$  is the smallest eigenvalue of the matrix.
By Gronwall's inequality, we know  $\widetilde{E} (\pmb {\mu}_t,\Sigma_t) = \mathcal{O}(e^{-4(\gamma -\epsilon)t})$  . Thus, we conclude
$$
\| \boldsymbol {\mu} _ {t} - \boldsymbol {b} \| = \mathcal {O} (e ^ {- 2 (\gamma - \epsilon) t}), \quad \| \Sigma_ {t} - Q \| = \mathcal {O} (e ^ {- 2 (\gamma - \epsilon) t}), \quad \forall \epsilon > 0.
$$
Finally, we provide a lower bound on  $\gamma$ . Note that for any  $u > 0$  if  $\pmb{b} \neq \mathbf{0}$  we have
$$
\left[ \begin{array}{c c} \frac {1}{1 + u} I _ {d ^ {2}} & \frac {1}{\sqrt {2}} \boldsymbol {b} \otimes Q ^ {- 1 / 2} \\ \frac {1}{\sqrt {2}} \boldsymbol {b} ^ {\top} \otimes Q ^ {- 1 / 2} & \frac {1 + u}{2} \boldsymbol {b} ^ {\top} \boldsymbol {b} Q ^ {- 1} \end{array} \right] \succeq 0.
$$
$$
\left[ \begin{array}{c c} I _ {d ^ {2}} & \frac {1}{\sqrt {2}} \boldsymbol {b} \otimes Q ^ {- 1 / 2} \\ \frac {1}{\sqrt {2}} \boldsymbol {b} ^ {\top} \otimes Q ^ {- 1 / 2} & \frac {1}{2} (1 + \boldsymbol {b} ^ {\top} \boldsymbol {b}) Q ^ {- 1} \end{array} \right] \succeq \left[ \begin{array}{c c} \frac {u}{1 + u} I _ {d ^ {2}} & \\ & \frac {1}{2} (1 - u \boldsymbol {b} ^ {\top} \boldsymbol {b}) Q ^ {- 1} \end{array} \right] =: \Omega_ {u}.
$$
Since  $\lambda_{\mathrm{max}}$  is the largest eigenvalue of  $Q$ , we know the smallest eigenvalue of  $\Omega_u$  is given by
$$
\min  \left\{\frac {u}{1 + u}, \frac {1 - u \boldsymbol {b} ^ {\top} \boldsymbol {b}}{2 \lambda_ {\max }} \right\}, \text {w h e r e} u > 0.
$$
We find  $u$  such that this quantity is maximized and get
$$
\begin{array}{l} \gamma \geq \max _ {u > 0} \min \left\{\frac {u}{1 + u}, \frac {1 - u \boldsymbol {b} ^ {\top} \boldsymbol {b}}{2 \lambda_ {\max}} \right\} = \frac {2}{1 + \boldsymbol {b} ^ {\top} \boldsymbol {b} + 2 \lambda_ {\max} + \sqrt {(1 + \boldsymbol {b} ^ {\top} \boldsymbol {b} + 2 \lambda_ {\max}) ^ {2} - 8 \lambda_ {\max}}} \\ > \frac {1}{1 + \boldsymbol {b} ^ {\top} \boldsymbol {b} + 2 \lambda_ {\max}}. \\ \end{array}
$$
If  $b = 0$ , then the smallest eigenvalue is given by
$$
\gamma = \min  \left\{1, \frac {1}{2 \lambda_ {\max }} \right\} > \frac {1}{1 + 2 \lambda_ {\max }}.
$$
Proof of Theorem 3.2. Equation (5) is a direct corollary of Theorem 3.1.
By Theorem C.2 there is a unique global solution. Thus, we only need to check that the  $\Sigma_t$  given by (6) and (7) satisfy the algebraic Riccati equation (5).
$$
\Sigma_ {t} ^ {- 1} = e ^ {- 2 t} \Sigma_ {0} ^ {- 1} + (1 - e ^ {- 2 t}) Q ^ {- 1}, \tag {77}
$$
we take the derivative with respect to  $t$  and get
$$
- \Sigma_ {t} ^ {- 1} \dot {\Sigma} _ {t} \Sigma_ {t} ^ {- 1} = 2 e ^ {- 2 t} \left(Q ^ {- 1} - \Sigma_ {0} ^ {- 1}\right). \tag {78}
$$
Substituting (78) into (77), we get
$$
2 \Sigma_ {t} ^ {- 1} = 2 Q ^ {- 1} + \Sigma_ {t} ^ {- 1} \dot {\Sigma} _ {t} \Sigma_ {t} ^ {- 1}.
$$
Multiplying by  $\Sigma_t^2$  and using the fact that  $\Sigma_t$  and  $Q$  commute, we can see that the algebraic Riccati equation (5) holds.
$$
\Sigma_ {t} = I + \frac {\eta (1 - e ^ {- 2 t})}{1 + \eta e ^ {- 2 t}} \pmb {v} \pmb {v} ^ {\top},
$$
we apply the Sherman-Morrison formula:
$$
\left(\Sigma_ {t}\right) ^ {- 1} = I - \frac {\frac {\eta \left(1 - e ^ {- 2 t}\right)}{1 + \eta e ^ {- 2 t}}}{1 + \frac {\eta \left(1 - e ^ {- 2 t}\right)}{1 + \eta e ^ {- 2 t}}} \boldsymbol {v} \boldsymbol {v} ^ {\top} = I - \frac {\eta \left(1 - e ^ {- 2 t}\right)}{1 + \eta} \boldsymbol {v} \boldsymbol {v} ^ {\top}.
$$
$$
\begin{array}{l} 2 \Sigma_ {t} ^ {- 1} \left(Q ^ {- 1} - \Sigma_ {t} ^ {- 1}\right) \Sigma_ {t} = 2 \left(I - \frac {\eta (1 - e ^ {- 2 t})}{1 + \eta} \boldsymbol {v} \boldsymbol {v} ^ {\top}\right) \left(- \frac {\eta e ^ {- 2 t}}{1 + \eta} \boldsymbol {v} \boldsymbol {v} ^ {\top}\right) \left(I + \frac {\eta (1 - e ^ {- 2 t})}{1 + \eta e ^ {- 2 t}} \boldsymbol {v} \boldsymbol {v} ^ {\top}\right) \\ = - \frac {2 \eta e ^ {- 2 t}}{1 + \eta} \pmb {v} \pmb {v} ^ {\top} = \partial_ {t} \big (\Sigma_ {t} ^ {- 1} \big). \\ \end{array}
$$
$$
\left(\Sigma_ {t} ^ {- 1} - Q ^ {- 1}\right) \Sigma_ {t} = \frac {\eta e ^ {- 2 t}}{1 + \eta} \boldsymbol {v} \boldsymbol {v} ^ {\top} \left(I + \frac {\eta (1 - e ^ {- 2 t})}{1 + \eta e ^ {- 2 t}} \boldsymbol {v} \boldsymbol {v} ^ {\top}\right) = \frac {\eta e ^ {- 2 t}}{1 + \eta e ^ {- 2 t}} \boldsymbol {v} \boldsymbol {v} ^ {\top}.
$$
$$
2 \operatorname {t r} \bigl (\bigl (\Sigma_ {t} ^ {- 1} - Q ^ {- 1} \bigr) \Sigma_ {t} \bigr) = \frac {2 \eta e ^ {- 2 t}}{1 + \eta e ^ {- 2 t}} \operatorname {t r} (\pmb {v} \pmb {v} ^ {\top}) = \frac {\eta e ^ {- 2 t}}{1 + \eta e ^ {- 2 t}} \operatorname {t r} (\pmb {v} ^ {\top} \pmb {v}) = \frac {\eta e ^ {- 2 t}}{1 + \eta e ^ {- 2 t}}.
$$
$$
\det  (\Sigma_ {t}) = 1 + \frac {\eta (1 - e ^ {- 2 t})}{1 + \eta e ^ {- 2 t}} \boldsymbol {v} ^ {\top} \boldsymbol {v} = \frac {1 + \eta}{1 + \eta e ^ {- 2 t}}.
$$
$$
\partial_ {t} \log \det (\Sigma_ {t}) = - \frac {- 2 \eta e ^ {- 2 t}}{1 + \eta e ^ {- 2 t}} = \frac {2 \eta e ^ {- 2 t}}{1 + \eta e ^ {- 2 t}}.
$$
$$
\rho_ {t} = (2 \pi) ^ {- d / 2} \left(\det (\Sigma_ {t})\right) ^ {- 1 / 2} \exp \left(- \frac {1}{2} \boldsymbol {x} ^ {\top} \Sigma_ {t} ^ {- 1} \boldsymbol {x}\right)
$$
is a solution with the initial condition  $\Sigma_0 = I_d$ . The theorem follows by the uniqueness of the solution of the mean field PDE (Theorem C.2).
Proof of Theorem 3.4. Similar to the proof of Theorem 3.1, we have
$$
\dot {\boldsymbol {\mu}} _ {t} = - \nabla_ {\boldsymbol {\mu} _ {t}} \widetilde {E} (\boldsymbol {\mu} _ {t}, \Sigma_ {t}) = - Q ^ {- 1} (\boldsymbol {\mu} _ {t} - \boldsymbol {b})
$$
$$
\begin{array}{l} \dot {\Sigma} _ {t} = - 2 \Sigma_ {t} ^ {2} \left((1 - \nu) \Sigma_ {t} + \nu I\right) ^ {- 1} \nabla_ {\Sigma_ {t}} \widetilde {E} (\Sigma_ {t}) - 2 \nabla_ {\Sigma_ {t}} \widetilde {E} (\Sigma_ {t}) \Sigma_ {t} ^ {2} \left((1 - \nu) \Sigma_ {t} + \nu I\right) ^ {- 1} \\ \Leftrightarrow \dot {\Sigma} _ {t} = 2 ((1 - \nu) \Sigma_ {t} + \nu I) ^ {- 1} \Sigma_ {t} - ((1 - \nu) \Sigma_ {t} + \nu I) ^ {- 1} \Sigma_ {t} ^ {2} Q ^ {- 1} - Q ^ {- 1} ((1 - \nu) \Sigma_ {t} + \nu I) ^ {- 1} \Sigma_ {t} ^ {2}. \\ \end{array}
$$
Following the arguments similar to the proof of Theorem 3.1, we can show
(8) has a unique global solution,
-  $\rho_t$  is the density of  $\mathcal{N}(\pmb{\mu}_t, \Sigma_t)$  given by (8),
-  $\dot{\tilde{E}}(\theta_t) \leq 0$  and  $\dot{\tilde{E}}(\pmb{\mu}_t, \Sigma_t) \to 0$  as  $t \to \infty$ ,
-  $\rho_{t}$  converges weakly to  $\rho^{*}$ .
Finally suppose  $\Sigma_0Q = Q\Sigma_0$ , then  $\Sigma_t$  also commutes with  $Q$  since 0 is a solution of the ODE satisfied by  $\Sigma_tQ - Q\Sigma_t$  and the solution is unique. Thus, we can diagonalize  $\Sigma_t$  and  $Q$  simultaneously. Then there exists orthogonal matrix  $P$  such that
$$
\Sigma_ {t} = P ^ {\top} \operatorname {d i a g} \left\{\sigma_ {1} ^ {(t)}, \dots , \sigma_ {d} ^ {(t)} \right\} P, \quad Q = P ^ {\top} \operatorname {d i a g} \left\{\lambda_ {1}, \dots , \lambda_ {d} \right\} P.
$$
And (9) reduces to