SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\dot {\sigma} _ {i} ^ {(t)} = \frac {2 \sigma_ {i} ^ {(t)} (\lambda_ {i} - \sigma_ {i} ^ {(t)})}{\lambda_ {i} ((1 - \nu) \sigma_ {i} ^ {(t)} + \nu)}.
$$
Solving this ODE, we get
$$
\frac {(\sigma_ {i} ^ {(t)} - \lambda_ {i}) ^ {(1 - \nu) \lambda_ {i} + \nu}}{(\sigma_ {i} ^ {(t)}) ^ {\nu}} = \frac {(\sigma_ {i} ^ {(0)} - \lambda_ {i}) ^ {(1 - \nu) \lambda_ {i} + \nu}}{(\sigma_ {i} ^ {(0)}) ^ {\nu}} e ^ {- 2 t}.
$$
Thus, we have  $\sigma_i^{(t)}\to \lambda_i$  as  $t\rightarrow \infty$  and
$$
\left| \sigma_ {i} ^ {(t)} - \lambda_ {i} \right| = \mathcal {O} \left(e ^ {- 2 t / ((1 - \nu) \lambda_ {i} + \nu)}\right).
$$
In particular, we conclude  $\| \Sigma_t - Q\| = \mathcal{O}\left(e^{-2t / ((1 - \nu)\lambda +\nu)}\right)$  where  $\lambda$  is the largest eigenvalue of  $Q$ .
Proof of Theorem 3.5. First, we define the Hamiltonian on the centered Gaussian submanifold by
$$
\mathcal {H} \left(\Sigma_ {t}, S _ {t}\right) := \frac {1}{2} \operatorname {t r} \left(S _ {t} \left(G _ {\Sigma_ {t}} ^ {- 1} S _ {t}\right)\right) + E \left(\Sigma_ {t}\right).
$$
By Corollary B.5 we have
$$
G _ {\Sigma} ^ {- 1} S = 2 \left(\Sigma^ {2} S + S \Sigma^ {2}\right).
$$
Thus, the Hamiltonian is reduced to
$$
\mathcal {H} \left(\Sigma_ {t}, S _ {t}\right) = 2 \operatorname {t r} \left(\Sigma_ {t} ^ {2} S _ {t} ^ {2}\right) + \frac {1}{2} \left(\operatorname {t r} \left(Q ^ {- 1} \Sigma_ {t}\right) - \log \det  \left(Q ^ {- 1} \Sigma_ {t}\right) - d\right).
$$
$$
\nabla_ {\Sigma_ {t}} \mathcal {H} (\Sigma_ {t}, S _ {t}) = 2 \left(\Sigma_ {t} S _ {t} ^ {2} + S _ {t} ^ {2} \Sigma_ {t}\right) + \frac {1}{2} (Q ^ {- 1} - \Sigma_ {t} ^ {- 1}), \quad \nabla_ {S _ {t}} \mathcal {H} (\Sigma_ {t}, S _ {t}) = 2 \left(\Sigma_ {t} ^ {2} S _ {t} + S _ {t} \Sigma_ {t} ^ {2}\right),
$$
and thus the Hamiltonian or AIG flow on the Gaussian submanifold is given by (12).
Next we show  $\Sigma_t$  is well-defined and remains positive definite. We check that  $\mathcal{H}_t \coloneqq \mathcal{H}(\Sigma_t, S_t)$  is decreasing with respect to  $t$ .
$$
\begin{array}{l} \frac {\mathrm {d} \mathcal {H} _ {t}}{\mathrm {d} t} = \operatorname {t r} \left(\nabla_ {S _ {t}} \mathcal {H} _ {t} \dot {S} _ {t} + \nabla_ {\Sigma_ {t}} \mathcal {H} _ {t} \dot {\Sigma} _ {t}\right) \\ = \operatorname {t r} \left(\nabla_ {S _ {t}} \mathcal {H} _ {t} \left(- \alpha_ {t} S _ {t} - \nabla_ {\Sigma_ {t}} \mathcal {H} _ {t}\right) + \nabla_ {\Sigma_ {t}} \mathcal {H} _ {t} \nabla_ {S _ {t}} \mathcal {H} _ {t}\right) \\ = - 4 \alpha_ {t} \operatorname {t r} \left(\Sigma_ {t} ^ {2} S _ {t} ^ {2}\right) \leq 0. \\ \end{array}
$$
Let  $\sigma_{t}$  be the smallest eigenvalue of  $\Sigma_t$ . Then
$$
\log \det  (\Sigma_ {t} Q ^ {- 1}) = \log \det  \Sigma_ {t} - \log \det  Q \geq d \log \sigma_ {t} - \log \det  Q.
$$
$$
\begin{array}{l} - \frac {d}{2} (\log \sigma_ {t} + 1) + \frac {1}{2} \log \det  Q \leq - \frac {1}{2} \left(\log \det  \left(\Sigma_ {t} Q ^ {- 1}\right) + d\right) \\ \leq E (\Sigma_ {t}) \leq H _ {t} \leq H _ {0}, \\ \end{array}
$$
$$
\sigma_ {t} \geq \exp (\log \det Q / d - 2 H _ {0} / d - 1).
$$
This means that the smallest eigenvalue of  $\Sigma_t$  has a positive lower bound. Thus,  $\Sigma_t \in \mathrm{Sym}^+(d, \mathbb{R})$  for any  $t \geq 0$ .
Finally we show that the AIG flow on the centered Gaussian submanifold coincides with the one on the density manifold. Similar to the proof of Theorem 3.1, we have a commutative graph
$$
\begin{array}{l} \begin{array}{c}T_{\Sigma}\Theta_{0}\xrightarrow{\mathrm{d}\phi_{\Sigma}} T_{\rho}\mathcal{P}(\mathbb{R}^{d})\\ G_{\Sigma}\Big{\downarrow}\qquad \qquad \qquad \Big{\downarrow}G_{\rho} \end{array} \\ T _ {\Sigma} ^ {*} \Theta_ {0} \xrightarrow {\psi_ {\Sigma}} T _ {\rho} ^ {*} \mathcal {P} (\mathbb {R} ^ {d}) \\ \end{array}
$$
Here  $\psi_{\Sigma}: S \to \Phi(\boldsymbol{x}) = \boldsymbol{x}^{\top} S \boldsymbol{x} + C$ , i.e., it maps a symmetric matrix to the quadratic function  $\Phi(\boldsymbol{x})$  (or more precisely the equivalent classes of quadratic functions that differ by a constant). Now the only things we need to show are that  $\frac{\delta \mathcal{H}}{\delta \rho_t} \in \operatorname{Im} \psi_{\Sigma}$  and that  $\frac{\delta \mathcal{H}}{\delta \Phi_t} \in \operatorname{Im} \mathrm{d}\phi_{\Sigma}$ .
$\frac{\delta\mathcal{H}}{\delta\Phi_t} = G_\rho^{-1}\Phi_t \in \mathrm{Im}\, \mathrm{d}\phi_\Sigma$  is trivially true from the commutative graph. Now since  $\frac{\delta E}{\delta\rho_t} = \log \rho_t + V \in \mathrm{Im}\, \psi_\Sigma$  ( $\rho_t$  is centered Gaussian density and check the definition of  $\psi_\Sigma$ ), it suffices to prove
$$
\frac {\delta}{\delta \rho_ {t}} \int \Phi G _ {\rho} ^ {- 1} \Phi \mathrm {d} \pmb {x} \in \mathrm {I m} \psi_ {\Sigma}.
$$
$$
\begin{array}{l} \frac {\delta}{\delta \rho_ {t}} \int \Phi_ {t} G _ {\rho} ^ {- 1} \Phi_ {t} d \mathbf {x} = - \frac {\delta}{\delta \rho_ {t}} \int \Phi_ {t} \nabla \cdot \left(\rho_ {t} (\cdot) \int K (\cdot , \mathbf {y}) \rho_ {t} (\mathbf {y}) \nabla \Phi_ {t} (\mathbf {y}) d \mathbf {y}\right) \\ = \frac {\delta}{\delta \rho_ {t}} \int \nabla \Phi_ {t} \cdot \left(\rho_ {t} (\cdot) \int K (\cdot , \mathbf {y}) \rho_ {t} (\mathbf {y}) \nabla \Phi (\mathbf {y}) d \mathbf {y}\right) \\ = \nabla \Phi_ {t} \cdot \int K (\cdot , \boldsymbol {y}) \rho_ {t} (\boldsymbol {y}) \nabla \Phi (\boldsymbol {y}) d \boldsymbol {y} \\ = \left(S _ {t} \boldsymbol {x}\right) ^ {\top} \left(S _ {t} \Sigma_ {t} \boldsymbol {x}\right) \\ = \frac {1}{2} \boldsymbol {x} ^ {\top} \left(S _ {t} ^ {2} \Sigma_ {t} + \Sigma_ {t} S _ {t} ^ {2}\right) \boldsymbol {x} \in \operatorname {I m} \psi_ {\Sigma}. \\ \end{array}
$$
We remark that the fact the Stein AIG flow remains Gaussian is highly non-trivial. In fact it requires  $\frac{\delta\mathcal{H}}{\delta\rho_t}$  to lie in the cotangent space of the Gaussian submanifold. One sufficient condition is that the variational derivatives of both the kinetic and potential energies lie in this same space, and the former could be interpreted as: The Gaussian submanifold is totally geodesic under the given metric, meaning that any geodesic flow with an initial position and velocity chosen from the Gaussian submanifold remains Gaussian. Fortunately both the Wasserstein metric and the Stein metric satisfy this property.
J Proofs for Section 3.2
Proof of Theorem 3.6. From the proof of Theorem 3.1 we know that (16) has a unique solution that is continuous in  $t$  and bounded for any  $t \in [0, T]$  with  $T < \infty$ . Thus, the linear system (15) also has a unique solution.
Now we check that the sample mean  $\pmb{\mu}_t$  and covariance matrix  $C_t$  satisfy (16). We simplify the right-han-side of (13) using  $\pmb{\mu}_t$  and  $C_t$ .
$$
\begin{array}{l} R H S = \boldsymbol {x} _ {i} ^ {(t)} - \frac {1}{N} \sum_ {j = 1} ^ {N} \left(\left(\boldsymbol {x} _ {i} ^ {(t)}\right) ^ {\top} \boldsymbol {x} _ {j} ^ {(t)} + 1\right) Q ^ {- 1} \left(\boldsymbol {x} _ {j} ^ {(t)} - \boldsymbol {b}\right) \\ = \boldsymbol {x} _ {i} ^ {(t)} - \frac {1}{N} \sum_ {j = 1} ^ {N} Q ^ {- 1} \left(\boldsymbol {x} _ {j} ^ {(t)} - \boldsymbol {b}\right) \left(\left(\boldsymbol {x} _ {i} ^ {(t)}\right) ^ {\top} \boldsymbol {x} _ {j} ^ {(t)} + 1\right) \\ = \boldsymbol {x} _ {i} ^ {(t)} - Q ^ {- 1} \frac {1}{N} \sum_ {j = 1} ^ {N} \boldsymbol {x} _ {j} ^ {(t)} \left(\left(\boldsymbol {x} _ {j} ^ {(t)}\right) ^ {\top} \boldsymbol {x} _ {i} ^ {(t)} + 1\right) + Q ^ {- 1} \boldsymbol {b} \frac {1}{N} \sum_ {j = 1} ^ {N} \left(\left(\boldsymbol {x} _ {j} ^ {(t)}\right) ^ {\top} \boldsymbol {x} _ {i} ^ {(t)} + 1\right) \\ = \left(I - Q ^ {- 1} \left(C _ {t} + \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right) + Q ^ {- 1} \boldsymbol {b} \boldsymbol {\mu} _ {t} ^ {\top}\right) \boldsymbol {x} _ {i} ^ {(t)} - Q ^ {- 1} \boldsymbol {\mu} _ {t} + Q ^ {- 1} \boldsymbol {b}. \tag {79} \\ \end{array}
$$
Let  $X_{t} = (\pmb{x}_{1}^{(t)},\dots ,\pmb{x}_{N}^{(t)})^{\top}$ . Then we have that
$$
\boldsymbol {\mu} _ {t} = \frac {X _ {t} ^ {\top} \mathbf {1}}{N}, \quad C _ {t} = \frac {X _ {t} ^ {\top} X _ {t}}{N} - \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}.
$$
Then (79) can be written in the matrix form as
$$
\dot {X} _ {t} = X _ {t} \left(I - \left(C _ {t} + \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right) Q ^ {- 1} + \boldsymbol {\mu} _ {t} \boldsymbol {b} ^ {\top} Q ^ {- 1}\right) - \mathbf {1} \boldsymbol {\mu} _ {t} ^ {\top} Q ^ {- 1} + \mathbf {1} \boldsymbol {b} ^ {\top} Q ^ {- 1}. \tag {80}
$$
Multiplying by  $\mathbf{1}^{\top} / N$  on the left, we get
$$
\dot {\boldsymbol {\mu}} _ {t} ^ {\top} = \boldsymbol {\mu} _ {t} ^ {\top} - \boldsymbol {\mu} _ {t} ^ {\top} \left(C _ {t} + \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right) Q ^ {- 1} + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t} \boldsymbol {b} ^ {\top} Q ^ {- 1} - \left(\boldsymbol {\mu} _ {t} - \boldsymbol {b}\right) ^ {\top} Q ^ {- 1}.
$$
$$
\dot {\boldsymbol {\mu}} _ {t} = (I - Q ^ {- 1} C _ {t}) \boldsymbol {\mu} _ {t} - (1 + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t}) Q ^ {- 1} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}).
$$
Note that  $\dot{C}_t = (\dot{X}_t^\top X_t + X_t^\top \dot{X}_t) / N - \pmb {\mu}_t\dot{\pmb{\mu}}_t - \dot{\pmb{\mu}}_t\pmb {\mu}_t^\top$  . Substituting (80) we obtain
$$
\dot {C} _ {t} = 2 C _ {t} - C _ {t} \left(C _ {t} + \boldsymbol {\mu} _ {t} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) ^ {\top}\right) Q ^ {- 1} - Q ^ {- 1} \left(C _ {t} + (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) \boldsymbol {\mu} _ {t} ^ {\top}\right) C _ {t}.
$$
Next we show that (14) and (15) satisfies (13).
$$
\begin{array}{l} R H S = (I - Q ^ {- 1} (C _ {t} + \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}) + Q ^ {- 1} \boldsymbol {b} \boldsymbol {\mu} _ {t} ^ {\top}) \boldsymbol {x} _ {i} ^ {(t)} - Q ^ {- 1} \boldsymbol {\mu} _ {t} + Q ^ {- 1} \boldsymbol {b} \\ = \left(I - Q ^ {- 1} \left(C _ {t} + \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right) + Q ^ {- 1} \boldsymbol {b} \boldsymbol {\mu} _ {t} ^ {\top}\right) \left(\boldsymbol {x} _ {i} ^ {(t)} - \boldsymbol {\mu} _ {t}\right) + \\ (I - Q ^ {- 1} C _ {t}) \boldsymbol {\mu} _ {t} - (1 + \boldsymbol {\mu} _ {t} ^ {\top} \boldsymbol {\mu} _ {t}) Q ^ {- 1} (\boldsymbol {\mu} _ {t} - \boldsymbol {b}) \\ = \left(I - Q ^ {- 1} \left(C _ {t} + \boldsymbol {\mu} _ {t} \boldsymbol {\mu} _ {t} ^ {\top}\right) + Q ^ {- 1} \boldsymbol {b} \boldsymbol {\mu} _ {t} ^ {\top}\right) A _ {t} \left(\boldsymbol {x} _ {i} ^ {(0)} - \boldsymbol {\mu} _ {0}\right) + \dot {\boldsymbol {\mu}} _ {t} \\ = \dot {A} _ {t} \left(\boldsymbol {x} _ {i} ^ {(0)} - \boldsymbol {\mu} _ {0}\right) + \dot {\boldsymbol {\mu}} _ {t} = \dot {\boldsymbol {x}} _ {i} ^ {(t)}. \\ \end{array}
$$
By Theorem C.3 (13) has a unique solution. Thus, the solution of (13) is given by (14)-(16).
The proof of Theorem 3.7 is quite long and tedious so we defer it to Appendix L.
Proof of Theorem 3.8. If  $C_0$  is non-singular, this is a direct corollary of Theorem 3.6 and Theorem 3.2. To also accommodate for the singular case, we provide a direct proof by calculation. Since  $C_0Q = QC_0$ , we know that  $C_0$  and  $Q$  are simultaneously diagonalizable. There exists some orthogonal matrix  $P_0$  such that we have the spectral decompositions
$$
C _ {0} = P _ {0} ^ {\top} D _ {0} P _ {0}, \quad Q = P _ {0} ^ {\top} Q _ {0} P _ {0},
$$
where  $D_0 = \mathrm{diag}(\lambda_1^{(0)},\dots ,\lambda_d^{(0)})$  and  $Q_{0} = \mathrm{diag}(q_{1},\dots ,q_{d})$  are diagonal matrices. Let  $D_{t}\coloneqq P_{0}C_{t}P_{0}^{\top}$ . (18) can be rewritten as
$$
\pmb {x} _ {i} ^ {(t)} = P _ {0} ^ {\top} \operatorname {d i a g} \big ((e ^ {- 2 t} + (1 - e ^ {- 2 t}) \lambda_ {1} ^ {(0)} / q _ {1}) ^ {- 1 / 2}, \dots , (e ^ {- 2 t} + (1 - e ^ {- 2 t}) \lambda_ {d} ^ {(0)} / q _ {d}) ^ {- 1 / 2} \big) P _ {0} \pmb {x} _ {i} ^ {(0)}.
$$
Thus, by taking the derivative with respect to  $t$ , we obtain
$$
\begin{array}{l} \dot {\boldsymbol {x}} _ {i} ^ {(t)} = e ^ {- 2 t} P _ {0} ^ {\top} \operatorname {d i a g} \left(\frac {1 - \lambda_ {1} ^ {(0)} / q _ {1}}{(e ^ {- 2 t} + (1 - e ^ {- 2 t}) \lambda_ {1} ^ {(0)} / q _ {1}) ^ {3 / 2}}, \dots , \frac {1 - \lambda_ {d} ^ {(0)} / q _ {d}}{(e ^ {- 2 t} + (1 - e ^ {- 2 t}) \lambda_ {d} ^ {(0)} / q _ {d}) ^ {3 / 2}}\right) P _ {0} \boldsymbol {x} _ {i} ^ {(0)} \\ = U \big (e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0} \big) ^ {- 3 / 2} \pmb {x} _ {i} ^ {(0)}, \\ \end{array}
$$
$$
U = e ^ {- 2 t} P _ {0} ^ {\top} \operatorname {d i a g} \left(1 - \lambda_ {1} ^ {(0)} / q _ {1}, \dots , 1 - \lambda_ {d} ^ {(0)} / q _ {d}\right) P _ {0} = e ^ {- 2 t} \left(I - Q ^ {- 1} C _ {0}\right).
$$
On the other hand, we check that
$$
\begin{array}{l} \boldsymbol {x} _ {i} ^ {(t)} - \frac {1}{N} \sum_ {j = 1} ^ {N} \left(\left(\boldsymbol {x} _ {i} ^ {(t)}\right) ^ {\top} \boldsymbol {x} _ {j} ^ {(t)} + 1\right) Q ^ {- 1} \boldsymbol {x} _ {j} ^ {(t)} \\ = \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {i} ^ {(0)} - \frac {1}{N} \sum_ {j = 1} ^ {N} \left(\left(\boldsymbol {x} _ {i} ^ {(0)}\right) ^ {\top} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1} \boldsymbol {x} _ {j} ^ {(0)} + 1\right). \\ Q ^ {- 1} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {j} ^ {(0)} \\ = \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {i} ^ {(0)} - \frac {1}{N} \sum_ {j = 1} ^ {N} Q ^ {- 1} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {j} ^ {(0)} \\ - \frac {1}{N} \sum_ {j = 1} ^ {N} \left(\boldsymbol {x} _ {j} ^ {(0)}\right) ^ {\top} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1} \boldsymbol {x} _ {i} ^ {(0)}. \\ Q ^ {- 1} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {j} ^ {(0)} \\ = \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {i} ^ {(0)} \\ - \frac {1}{N} \sum_ {j = 1} ^ {N} Q ^ {- 1} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {j} ^ {(0)} \left(\boldsymbol {x} _ {j} ^ {(0)}\right) ^ {\top} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1} \boldsymbol {x} _ {i} ^ {(0)} \\ = \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} \boldsymbol {x} _ {i} ^ {(0)} \\ - Q ^ {- 1} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1 / 2} C _ {0} \left(e ^ {- 2 t} I + (1 - e ^ {- 2 t}) Q ^ {- 1} C _ {0}\right) ^ {- 1} \boldsymbol {x} _ {i} ^ {(0)} \\ = e ^ {- 2 t} \left(I - Q ^ {- 1} C _ {0}\right) \left(e ^ {- 2 t} I + \left(1 - e ^ {- 2 t}\right) Q ^ {- 1} C _ {0}\right) ^ {- 3 / 2} \boldsymbol {x} _ {i} ^ {(0)} \\ = U \left(e ^ {- 2 t} I + \left(1 - e ^ {- 2 t}\right) Q ^ {- 1} C _ {0}\right) ^ {- 3 / 2} \boldsymbol {x} _ {i} ^ {(0)}. \\ \end{array}
$$
Thus, we conclude that (18) is a solution of (13). By Theorem C.3, the solution for (13) is unique and hence the theorem follows.
Proof of Theorem 3.9. For any particle system of the R-SVGF, we can derive that
$$
\begin{array}{l} \dot {X} _ {t} = \left((1 - \nu) \left(\frac {X _ {t} X _ {t} ^ {\top}}{N} + \frac {\mathbf {1 1} ^ {\top}}{N}\right) + \nu I _ {d}\right) ^ {- 1} \left(X _ {t} - \frac {1}{N} \left(X _ {t} X _ {t} ^ {\top} + \mathbf {1 1} ^ {\top}\right) X _ {t} Q ^ {- 1}\right) \\ = \left((1 - \nu) \left(\frac {X _ {t} X _ {t} ^ {\top}}{N} + \frac {\mathbf {1 1} ^ {\top}}{N}\right) + \nu I _ {d}\right) ^ {- 1} X _ {t} (I - C _ {t} Q ^ {- 1}). \\ \end{array}
$$
By Sherman-Morrison formula we have
$$
\left(\nu I _ {d} + (1 - \nu) \frac {\mathbf {1 1} ^ {\top}}{N}\right) ^ {- 1} = \frac {1}{\nu} I _ {d} - \frac {1 - \nu}{\nu} \frac {\mathbf {1 1} ^ {\top}}{N}.
$$
By Woodbury matrix identity we derive
$$
\begin{array}{l} \left((1 - \nu) \left(\frac {X _ {t} X _ {t} ^ {\top}}{N} + \frac {\mathbf {1 1} ^ {\top}}{N}\right) + \nu I _ {d}\right) ^ {- 1} \\ = \left(\nu I _ {d} + (1 - \nu) \frac {\mathbf {1 1} ^ {\top}}{N}\right) ^ {- 1} - \left(\nu I _ {d} + (1 - \nu) \frac {\mathbf {1 1} ^ {\top}}{N}\right) ^ {- 1} X _ {t}. \\ \frac {1}{N} \left(\frac {1}{1 - \nu} I _ {d} + X _ {t} ^ {\top} \left(\nu I _ {d} + (1 - \nu) \frac {\mathbf {1 1} ^ {\top}}{N}\right) ^ {- 1} X _ {t}\right) ^ {- 1} X _ {t} ^ {\top} \left(\nu I _ {d} + (1 - \nu) \frac {\mathbf {1 1} ^ {\top}}{N}\right) ^ {- 1} \\ = \left(\frac {1}{\nu} I _ {d} - \frac {1 - \nu}{\nu} \frac {\mathbf {1 1} ^ {\top}}{N}\right) - \frac {1}{N \nu^ {2}} X _ {t} \left(\frac {1}{1 - \nu} I _ {d} + \frac {1}{\nu} C _ {t}\right) ^ {- 1} X _ {t} ^ {\top}. \\ \end{array}
$$
Substituting this into (81), we have
$$
\begin{array}{l} \dot {X} _ {t} = \left(\frac {1}{\nu} I _ {d} - \frac {1 - \nu}{\nu} \frac {\mathbf {1 1} ^ {\top}}{N}\right) X _ {t} (I - C _ {t} Q ^ {- 1}) - \frac {1}{N \nu^ {2}} X _ {t} \left(\frac {1}{1 - \nu} I _ {d} + \frac {1}{\nu} C _ {t}\right) ^ {- 1} X _ {t} ^ {\top} X _ {t} (I - C _ {t} Q ^ {- 1}) \\ = \frac {1}{\nu} X _ {t} - \frac {1}{\nu} X _ {t} C _ {t} Q ^ {- 1} - \frac {1}{\nu} X _ {t} \left(\frac {1}{1 - \nu} I _ {d} + \frac {1}{v} C _ {t}\right) ^ {- 1} \left(\frac {1}{\nu} C _ {t} - \frac {1}{\nu} C _ {t} ^ {2} Q ^ {- 1}\right) \\ = \frac {1}{\nu} X _ {t} \left(I _ {d} - \left(\frac {1}{1 - \nu} I _ {d} + \frac {1}{\nu} C _ {t}\right) ^ {- 1} \frac {1}{\nu} C _ {t}\right) (I - C _ {t} Q ^ {- 1}) \\ = X _ {t} \left(\nu I _ {d} + (1 - \nu) C _ {t}\right) ^ {- 1} \left(I - C _ {t} Q ^ {- 1}\right) \tag {81} \\ \end{array}
$$
Multiplying by  $X_{t}^{\top}$  on the left we get
$$
\frac {X _ {t} ^ {\top} \dot {X} _ {t}}{N} = (\nu I _ {d} + (1 - \nu) C _ {t}) ^ {- 1} C _ {t} (I - C _ {t} Q ^ {- 1}).
$$
Thus, the derivative of covariance matrix  $C_t$  is given by
$$
\begin{array}{l} \dot {C} _ {t} = \frac {X _ {t} ^ {\top} \dot {X} _ {t}}{N} + \frac {\dot {X} _ {t} ^ {\top} X _ {t}}{N} \\ = (\nu I _ {d} + (1 - \nu) C _ {t}) ^ {- 1} C _ {t} (I - C _ {t} Q ^ {- 1}) + (I - Q ^ {- 1} C _ {t}) (\nu I _ {d} + (1 - \nu) C _ {t}) ^ {- 1} C _ {t} \\ = 2 (\nu I _ {d} + (1 - \nu) C _ {t}) ^ {- 1} C _ {t} - (\nu I _ {d} + (1 - \nu) C _ {t}) ^ {- 1} C _ {t} ^ {2} Q ^ {- 1} - Q ^ {- 1} (\nu I _ {d} + (1 - \nu) C _ {t}) ^ {- 1} C _ {t} ^ {2}. \\ \end{array}
$$
Next we show that (19) and (20) satisfies (81).
$$
R H S = X _ {0} A _ {t} ^ {\top} \left(\nu I _ {d} + (1 - \nu) C _ {t}\right) ^ {- 1} \left(I - C _ {t} Q ^ {- 1}\right) = X _ {0} \dot {A} _ {t} ^ {\top} = \dot {X} _ {t}.
$$
Similar to the proof of Theorem 3.4, it could be shown that the R-SVGF also has a unique solution and the proof is complete.
Next we show Theorem 3.10.
Lemma J.1. The covariance matrix  $C_t$  ( $t = 1,2,\dots$ ) of the discrete-time finite particle system satisfies the following equation
$$
C _ {t + 1} = \left(I + \epsilon \left(I - Q ^ {- 1} C _ {t}\right)\right) C _ {t} \left(I + \epsilon \left(I - Q ^ {- 1} C _ {t}\right)\right) ^ {\top}.
$$
Proof. Let  $X = \left(\pmb{x}_1,\dots ,\pmb{x}_N\right)^\top$  . Then (21) can be written as
$$
X _ {t + 1} = X _ {t} + \epsilon X _ {t} \left(I - C _ {t} Q ^ {- 1}\right).
$$
$$
C _ {t + 1} = \frac {X _ {t} ^ {\top} X _ {t}}{N} = \left(I + \epsilon \left(I - Q ^ {- 1} C _ {t}\right)\right) C _ {t} \left(I + \epsilon \left(I - Q ^ {- 1} C _ {t}\right)\right) ^ {\top}.
$$
Figure 6: Plot of the function  $f_{\epsilon}(x) = (1 + \epsilon (1 - x))^{2}x$ .
Proof of Theorem 3.10. First since  $C_0Q = QC_0$  they are simultaneously diagonalizable. By Theorem 3.8 any  $C_t$  and  $Q$  are simultaneously diagonalizable. Thus, without loss of generality we assume all  $C_t$  and  $Q$  are diagonal matrices. Then by Lemma J.1 we know
$$
Q ^ {- 1} C _ {t + 1} = (I + \epsilon (I - Q ^ {- 1} C _ {t})) ^ {2} Q ^ {- 1} C _ {t}.
$$
If we define  $f_{\epsilon}(x) = (1 + \epsilon (1 - x))^{2}x$  then every entry (i.e., eigenvalue) of  $Q^{-1}C_t$  follows the  $f_{\epsilon}$ -iteration trajectory of the corresponding entry of  $Q^{-1}C_0$ .
The fixed points of  $f_{\epsilon}(x) = (1 + \epsilon (1 - x))^{2}x$  are  $\{0, 1, 2 / \epsilon + 1\}$ . If  $0 < \epsilon < 1$  then  $|f'(0)| > 1$ ,  $|f'(2 / \epsilon + 1)| > 1$ , and  $|f'(1)| < 1$ . By Proposition 1.9 of [26], 1 is an attracting fixed point while 0 and  $2 / \epsilon + 1$  are repelling fixed points. By definition there exists an interval around 1 such that for all initial points in that interval, the trajectory of any eigenvalue of  $Q^{-1}C_t$  converges to 1. We now quantify that result further.
Note that  $f_{\epsilon}^{\prime}(x) = (1 + \epsilon - \epsilon x)(1 + \epsilon - 3\epsilon x)$  is an upward-opening parabola whose zeros are  $1/3 + 1/(3\epsilon)$  and  $1 + \epsilon$ . Thus,  $f_{\epsilon}^{\prime}$  is monotone decreasing on  $[0, 1/3 + 1/(3\epsilon)]$ . In particular, both equations  $f_{\epsilon}^{\prime}(x) = 1$  and  $f_{\epsilon}^{\prime}(x) = 1 - \epsilon$  have two distinct roots, the smaller ones lying on  $(0, 1/3 + 1/(3\epsilon))$ . Define  $w_{\epsilon}$  to be the smaller root of  $f_{\epsilon}^{\prime}(x) = 1$  and  $u_{\epsilon}$ , the smaller root of  $f_{\epsilon}^{\prime}(x) = 1 - \epsilon$ . Since  $f_{\epsilon}^{\prime}$  is monotone decreasing and  $f_{\epsilon}^{\prime}(1) = 1 - 2\epsilon$ , we have  $0 < w_{\epsilon} < u_{\epsilon} < 1$ .
Thus, for any  $x \in [w_{\epsilon}, 1/3 + 1/(3\epsilon)]$  we have  $0 \leq f_{\epsilon}'(x) \leq 1$  and  $0 \leq f_{\epsilon}(x) \leq f_{\epsilon}(1/3 + 1/(3\epsilon)) < 1/3 + 1/(3\epsilon)$  (here we have used the condition  $\epsilon < 0.5$ ). On the other hand, since  $f_{\epsilon}'(0) \geq 1$  and  $0$  and  $1$  are two fixed points, it holds that  $f_{\epsilon}(x) \geq x$  for any  $x \in [0,1]$ , which implies that for any  $x \in [w_{\epsilon}, 1/3 + 1/(3\epsilon)]$  we have  $f_{\epsilon}(x) \geq f_{\epsilon}(w_{\epsilon}) \geq w_{\epsilon}$ . Hence we know that  $f_{\epsilon}'$  is a contraction map and  $f_{\epsilon}([w_{\epsilon}, 1/3 + 1/(3\epsilon)]) \subset [w_{\epsilon}, 1/3 + 1/(3\epsilon)]$ , which implies that there is a unique fixed point (i.e., 1) such that the  $f_{\epsilon}$ -iteration trajectory converges to it.
Next we prove that for any  $x \in (0, 1 + 1 / \epsilon)$  the trajectory falls into the interval  $[w_{\epsilon}, 1 / 3 + 1 / (3\epsilon)]$  after finite iterations. If  $x \in (1 / 3 + 1 / (3\epsilon), 1 + 1 / \epsilon)$  then after one iteration we get  $f_{\epsilon}(x) \in (0, f_{\epsilon}(1 / 3 + 1 / (3\epsilon))) \subset (0, 1 / 3 + 1 / (3\epsilon))$ . We claim that if  $x \in (0, w_{\epsilon})$  then after  $t_0 := \left\lceil \frac{\log w_{\epsilon} / x}{2\log(1 + \epsilon(1 - w_{\epsilon}))} \right\rceil$
we have  $f_{\epsilon}^{(t_0)}(x) \in [w_{\epsilon}, 1]$ . Firstly  $f_{\epsilon}^{(t)}(x) \leq 1$  for any  $t$ . It suffices to prove  $f_{\epsilon}^{(t_0)}(x) \geq w_{\epsilon}$ . Suppose  $f_{\epsilon}^{(t_0)}(x) < w_{\epsilon}$ . Then for any  $0 \leq t \leq t_0$  we have  $f_{\epsilon}^{(t)}(x) < w_{\epsilon}$ . By definition of  $t_0$  we know
$$
f _ {\epsilon} ^ {(t _ {0})} (x) > \left(1 + \epsilon \left(1 - w _ {\epsilon}\right)\right) ^ {2 t _ {0}} x \geq w _ {\epsilon}.
$$
This is a contradiction. Thus, the claim holds and in conclusion for any  $x \in (0,1 + 1 / \epsilon)$  the  $f_{\epsilon}$ -iteration trajectory converges to the fixed point 1.
Finally we consider the case when  $x \in [u_{\epsilon}, 1/3 + 1/(3\epsilon)]$ . Since  $f_{\epsilon}'$  is monotone decreasing here we have  $f_{\epsilon}'(x) \in [0, 1 - \epsilon]$ . And by similar arguments we know  $f_{\epsilon}([u_{\epsilon}, 1/3 + 1/(3\epsilon)]) \subset [u_{\epsilon}, 1/3 + 1/(3\epsilon)]$ . Thus, we conclude
$$
\left| f _ {\epsilon} ^ {(t)} (x) - 1 \right| \leq (1 - \epsilon) \left| f _ {\epsilon} ^ {(t - 1)} (x) - 1 \right| \leq \dots \leq (1 - \epsilon) ^ {t} | x - 1 | \leq e ^ {- \epsilon t} | x - 1 |.
$$
K Proofs for Section 4
The following result was derived in [43]. We provide the proof below for completeness.
Lemma K.1. Let  $\rho^{*}$  be a probability measure and  $\rho_{\theta}$  be a Gaussian measure with parameters  $\theta = (\mu, \Sigma)$ . Then we have the following expressions: