SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Arguably the most important stochastic process is the Wiener process  $W$  which formalizes the physical concept of the Brownian motion, i.e., the random motion of a small particle inside a fluid caused by thermal fluctuations, see [68, Chapter 3] for a heuristic approach to the topic. For stochastic processes  $X$  that are sufficiently well-behaved, one can define a stochastic integral  $\int_0^t X_s \, \mathrm{d}W_s$ , called Itô integral of  $X$  with respect to  $W$ , cf. [66, Chapter IV.2; 57, Chapter 3.1; 63, Chapter 3.2]. Notably, the result, is again a stochastic process so that one can implicitly define stochastic processes via an Itô integral equation of the form
$$
X _ {t} = X _ {0} + \int_ {0} ^ {t} f (X _ {s}, s) \mathrm {d} s + \int_ {0} ^ {t} \sigma (X _ {s}, s) \mathrm {d} W _ {s} \tag {12}
$$
often denoted as an Itô stochastic differential equation (SDE)
$$
\mathrm {d} X _ {t} = f \left(X _ {t}, t\right) \mathrm {d} t + \sigma \left(X _ {t}, t\right) \mathrm {d} W _ {t}, \quad X _ {0} = \xi . \tag {13}
$$
Here,  $f: \mathbb{R}^n \times [0,T] \to \mathbb{R}^n$  is called the drift coefficient,  $\sigma: \mathbb{R}^n \times [0,T] \to \mathbb{R}^{n \times m}$  is called the diffusion coefficient, and the initial value  $\xi$  is a random variable that is independent of the Wiener process  $W$ . Further information about different notions of solutions to an SDE, i.e., strong and weak solutions, as well as a comprehensive consideration of their existence and uniqueness can be found in [66, Chapter 9; 57, Chapter 5; 63, Chapter 5]. Additionally, for a treatment of SDEs from a numerical perspective, we recommend [64].
In this work, we consider time series data as realizations of a stochastic process, i.e., as samples from a function-valued random variable as described in Section 3.1. To perform variational inference in such a setting, we utilize SDEs to construct function-valued distributions. Specifically, we consider non-autonomous linear Itô SDEs with multiplicative noise that evolve on a homogeneous space of a (quadratic) matrix Lie group. To ensure the existence and uniqueness of strong solutions, the drift and diffusion coefficients that appear in the considered SDEs satisfy the Lipschitz and growth conditions of [57, Theorem 5.2.1]
A remark on notation. As outlined in Section 3.4, we aim to learn the parameters  $\phi$  of an approximate posterior. Thus, the SDEs we consider are not specified by an initial random variable  $X_0 = \xi$ , but by an initial distribution  $\mathcal{P}^{X_0}$  such that  $X_0 \sim \mathcal{P}^{X_0}$ . Therefore, we utilize the notation
$$
\mathrm {d} X _ {t} = f \left(X _ {t}, t\right) \mathrm {d} t + \sigma \left(X _ {t}, t\right) \mathrm {d} W _ {t}, \quad X _ {0} \sim \mathcal {P} ^ {X _ {0}}. \tag {14}
$$
A (strong) solution to such an SDE is then understood as a stochastic process  $X$  that  $(i)$  is adapted (see [66, Definition 4.2]),  $(ii)$  has continuous sample paths  $t \mapsto X(\omega, \cdot)$ ,  $(iii)$  almost surely satisfies the integral equation in Eq. (12), and  $(iv)$  there is a random variable  $\xi \sim \mathcal{P}^{X_0}$  that is independent of the Wiener process  $W$  such that  $X_0 = \xi$  holds almost surely. In accordance with [57, Theorem 5.2.1], we demand that and the initial distribution  $\xi$  has finite second moment, but, of course, distinct choices of  $X_0 = \xi$  might define distinct strong solutions  $X$ .
E.2 SDEs in quadratic matrix Lie groups
Definition 1 (cf. [65]). Let  $\mathbb{R}^{n\times n}$  be the vector space of real  $n\times n$  matrices. We denote the general linear group, i.e., the group of invertible real  $n\times n$  matrices equipped with the matrix product, as
$$
\operatorname {G L} (n) = \left\{\mathbf {A} \in \mathbb {R} ^ {n \times n} \mid \det  \mathbf {A} \neq 0 \right\}. \tag {15}
$$
$\operatorname{GL}(n)$  is a Lie group with Lie algebra
$$
\mathfrak {g l} (n) = \left(\mathbb {R} ^ {n \times n}, [ \cdot , \cdot ]\right), \tag {16}
$$
i.e.,  $\mathbb{R}^{n\times n}$  together with the commutator bracket  $(\mathbf{A},\mathbf{B})\mapsto [\mathbf{A},\mathbf{B}] = \mathbf{AB} - \mathbf{BA}$ .
In the context of this work, we specifically consider quadratic matrix Lie groups.
Definition 2 (cf. [62]). Given some fixed matrix  $\mathbf{P} \in \mathbb{R}^{n \times n}$ , the quadratic matrix Lie group  $\mathcal{G}$  is defined as
$$
\mathcal {G} = \mathcal {G} (\mathbf {P}) = \left\{\mathbf {A} \in \operatorname {G L} (n): \mathbf {A} ^ {\top} \mathbf {P} \mathbf {A} = \mathbf {P} \right\}. \tag {17}
$$
The Lie algebra  $\mathfrak{g}$  of  $\mathcal{G}$  then consists of the  $n\times n$  matrices  $\mathbf{V}$  that satisfy  $\mathbf{V}^{\top}\mathbf{P} + \mathbf{P}\mathbf{V} = \mathbf{0}$ .
Moreover, we consider non-autonomous linear Ito SDEs in quadratic matrix Lie groups  $\mathcal{G}$ . The following lemma (Lemma 3) provides sufficient conditions to ensure that an SDE solution evolves in  $\mathcal{G}$ . This will pave the way to introduce a class of SDEs on homogeneous spaces in Appendix E.3 via the Lie group action.
Lemma 3 (cf. [6, Chapter 4.1], [12]). Let  $G$  be a stochastic process that solves the matrix Itô SDE
$$
\mathrm {d} G _ {t} = \left(\mathbf {V} _ {0} (t) \mathrm {d} t + \sum_ {i = 1} ^ {m} \mathrm {d} w _ {t} ^ {i} \mathbf {V} _ {i}\right) G _ {t}, \quad G _ {0} = \mathbf {I} _ {\mathrm {n}}, \tag {18}
$$
where  $\mathbf{V}_0:[0,T]\to \mathbb{R}^{n\times n}$ ,  $\mathbf{V}_1,\ldots ,\mathbf{V}_m\in \mathbb{R}^{n\times n}$  and  $w^1,\dots,w^m$  are independent scalar-valued Wiener processes. The process  $G$  stays in the quadratic matrix Lie group  $\mathcal{G}(\mathbf{P})$  defined by  $\mathbf{P}\in \mathbb{R}^{n\times n}$  if
(i)  $\mathbf{V}_0(t) = \mathbf{K}(t) + \frac{1}{2}\sum_{i=1}^{m}\mathbf{V}_i^2$ , with  $\mathbf{K}:[0,T]\to \mathfrak{g}$ , and
(ii)  $\mathbf{V}_1,\ldots ,\mathbf{V}_m\in \mathfrak{g}$
Remark. Although the matrices  $\mathbf{V}_1, \ldots, \mathbf{V}_m \in \mathfrak{g}$  can be chosen arbitrarily, it is natural to choose an orthogonal basis of the Lie algebra  $\mathfrak{g}$ . Then  $m = \dim \mathfrak{g}$  and each direction in the vector space  $\mathfrak{g}$  provides an independent contribution to the overall noise in Eq. (18).
Proof. The process  $G$  stays in the quadratic matrix Lie group  $\mathcal{G}(\mathbf{P})$  if  $G_{t}^{\top}\mathbf{P}G_{t}$  is constant, i.e., if  $\mathrm{d}(G_t^\top \mathbf{P}G_t) = \mathbf{0}$ . The latter is to be understood in the Itô sense, i.e.,
$$
\begin{array}{l} \mathbf {0} = \mathrm {d} (G _ {t} ^ {\top} \mathbf {P} G _ {t}) = \mathrm {d} G _ {t} ^ {\top} \mathbf {P} G _ {t} + G _ {t} \mathbf {P} \mathrm {d} G _ {t} + \mathrm {d} G _ {t} ^ {\top} \mathbf {P} \mathrm {d} G _ {t} \\ = G _ {t} ^ {\top} \left(\left(\mathbf {V} _ {0} ^ {\top} (t) \mathbf {P} + \mathbf {P V} _ {0} (t)\right) \mathrm {d} t + \sum_ {i = 1} ^ {m} \mathrm {d} w _ {t} ^ {i} \left(\mathbf {V} _ {i} ^ {\top} \mathbf {P} + \mathbf {P V} _ {i}\right) + \sum_ {i = 1} ^ {m} \mathrm {d} t \mathbf {V} _ {i} ^ {\top} \mathbf {P V} _ {i}\right) G _ {t}. \tag {19} \\ \end{array}
$$
Here,  $\mathrm{d}w_{t}^{i}\mathrm{d}w_{t}^{j} = \delta_{ij}\mathrm{d}t$ , as  $w^{i}$  and  $w^{j}$  are independent scalar Wiener processes. Eq. (19) implies the following two conditions:
$$
\mathbf {V} _ {i} ^ {\top} \mathbf {P} + \mathbf {P} \mathbf {V} _ {i} = \mathbf {0} \text {f o r} i = 1, \dots , m \quad \text {a n d} \quad \mathbf {V} _ {0} ^ {\top} (t) \mathbf {P} + \mathbf {P} \mathbf {V} _ {0} (t) = - \sum_ {i = 1} ^ {m} \mathbf {V} _ {i} ^ {\top} \mathbf {P} \mathbf {V} _ {i}. \tag {20}
$$
As the following calculation shows, these conditions are satisfied if  $\mathbf{V}_1,\ldots ,\mathbf{V}_m\in \mathfrak{g}$  and  $\mathbf{V}_0(t) = \mathbf{K}(t) + \frac{1}{2}\sum_{i = 1}^{m}\mathbf{V}_i^2$  where  $\mathbf{K}:[0,T]\to \mathfrak{g}$ .
$$
\begin{array}{l} \mathbf {V} _ {0} ^ {\top} (t) \mathbf {P} + \mathbf {P} \mathbf {V} _ {0} (t) = \underbrace {\mathbf {K} ^ {\top} (t) \mathbf {P} + \mathbf {P} \mathbf {K} (t)} _ {= \mathbf {0}} + \frac {1}{2} \left(\sum_ {i = 1} ^ {m} \mathbf {V} _ {i} ^ {2}\right) ^ {\top} \mathbf {P} + \frac {1}{2} \mathbf {P} \left(\sum_ {i = 1} ^ {m} \mathbf {V} _ {i} ^ {2}\right) (21) \\ = \frac {1}{2} \sum_ {i = 1} ^ {m} \left(\mathbf {V} _ {i} ^ {\top} \mathbf {V} _ {i} ^ {\top} \mathbf {P} + \mathbf {P V} _ {i} \mathbf {V} _ {i}\right) = - \frac {1}{2} \sum_ {i = 1} ^ {m} \left(\mathbf {V} _ {i} ^ {\top} \mathbf {P V} _ {i} + \mathbf {V} _ {i} ^ {\top} \mathbf {P V} _ {i}\right) (22) \\ = - \sum_ {i = 1} ^ {m} \mathbf {V} _ {i} ^ {\top} \mathbf {P} \mathbf {V} _ {i}. (23) \\ \end{array}
$$
Numerical solution. Algorithm 1 lists our choice of a numerical solver, which is the one-step geometric Euler-Maruyama scheme from [36] (see also [40, Section 3.1]).
This scheme consists of three parts. First, we divide  $[0,T]$  into subintervals  $[t_j,t_{j + 1}]$ , starting with  $t_0 = 0$  until  $t_{j + 1} = T$ . Second, we solve Eq. (18) pathwise in  $\mathcal{G}$  over successive intervals  $[t_j,t_{j + 1}]$  by applying a McKean-Gangolli Injection [12] in combination with a numerical solver. Third, the (approximate) path solution is eventually computed as  $G_{j + 1} = G_{j}\exp (\Omega_{j + 1})$ .
Algorithm 1 Geometric Euler-Maruyama Algorithm (g-EM) [36].
1:  $t_0\gets 0$    
2:  $G_0\gets \mathbf{I}_{\mathrm{n}}$    
3: repeat over successive subintervals  $[t_j,t_{j + 1}],j\geq 0$    
4:  $\delta_j\gets t_{j + 1} - t_j$    
5:  $\delta w_j^i\sim \mathcal{N}(0,\delta_j)$  for  $i = 1,\ldots ,m$    
6:  $\Omega_{j + 1}\leftarrow \left(\mathbf{V}_0(t_j) - \frac{1}{2}\sum_{i = 1}^{m}\mathbf{V}_i^2\right)\delta_j + \sum_{i = 1}^{m}\delta w_j^i\mathbf{V}_i$ $\triangleright \Omega_{j + 1}\in \mathfrak{g}$    
7:  $G_{j + 1}\gets G_j\exp (\Omega_{j + 1})$  Matrix exp. ensures that  $G_{j + 1}\in \mathcal{G}$    
8: until  $t_{j + 1} = T$
Remark. We remark that in the original derivation of the algorithm in [36], the group acts from the right (as in Algorithm 1). However, the same derivation can be done for a left group action which is what we use in our implementation and the presentation within the main manuscript. Furthermore, under condition (i) from Lemma 3, we see that line 6 reduces to
$$
\Omega_ {j + 1} \leftarrow \mathbf {K} (t _ {j}) \delta_ {j} + \sum_ {i = 1} ^ {m} \delta w _ {j} ^ {i} \mathbf {V} _ {i}.
$$
E.3 SDEs in homogeneous spaces of quadratic matrix Lie groups
In the following, we use our results of Appendix E.2 to define stochastic processes in a homogeneous space  $\mathbb{M}$  of a quadratic matrix Lie group  $\mathcal{G}$ . The main idea is, given a starting point  $Z_0\in \mathbb{M}$ , to translate a path  $G:[0,T]\to \mathcal{G}$  into a path  $Z = G\cdot Z_{0}$  in  $\mathbb{M}$  via the Lie group action. Notably, this construction can be done at the level of stochastic processes, i.e., a stochastic process  $G:[0,T]\times \Omega \rightarrow \mathcal{G}$  induces a stochastic process  $Z = G\cdot Z_{0}:[0,T]\times \Omega \rightarrow \mathbb{M}$  called a one-point motion, see [31, Chapter 2]. In particular, if  $Z_{0}$  follows some distribution  $\mathcal{P}$  in  $\mathbb{M}$  and  $G$  solves an SDE of the form as in Eq. (18), then  $Z$  solves  $\mathrm{d}Z_{t} = \mathrm{d}G_{t}\cdot Z_{0}$ , i.e.,
$$
\mathrm {d} Z _ {t} = \left(\mathbf {V} _ {0} (t) \mathrm {d} t + \sum_ {i = 1} ^ {m} \mathrm {d} w _ {t} ^ {i} \mathbf {V} _ {i}\right) \cdot Z _ {t}, \quad Z _ {0} \sim \mathcal {P} ^ {Z _ {0}}. \tag {24}
$$
Notably, Eq. (3) guarantees that a solution to the SDE in Eq. (18) remains in the group  $\mathcal{G}$ , and consequently a solution to Eq. (24) remains in  $\mathbb{M}$ . As we only consider Lie group actions that are given by matrix-vector multiplication, we substitute the notation  $G\cdot Z$  by the more concise  $GZ$  from now on.
Remark. For subsequent results, it is convenient to represent Eq. (24) in the form of
$$
\mathrm {d} Z _ {t} = \mathbf {V} _ {0} (t) Z _ {t} \mathrm {d} t + \left[ \mathbf {V} _ {1} Z _ {t}, \mathbf {V} _ {2} Z _ {t}, \dots , \mathbf {V} _ {m} Z _ {t} \right] \mathrm {d} W _ {t}, \quad Z _ {0} \sim \mathcal {P} ^ {Z _ {0}}, \tag {25}
$$
i.e., with diffusion coefficient  $\sigma : (\mathbf{z}, t) \mapsto [\mathbf{V}_1\mathbf{z}, \dots, \mathbf{V}_m\mathbf{z}] \in \mathbb{R}^{n \times m}$  and an  $m$ -dimensional Wiener process  $W = [w^1, w^2, \dots, w^m]^\top$ . Notably, the matrix  $\sigma(\mathbf{z}, t)\sigma(\mathbf{z}, t)^\top$  is given by
$$
\sigma (\mathbf {z}, t) \sigma (\mathbf {z}, t) ^ {\top} = \sum_ {i = 1} ^ {m} \mathbf {V} _ {i} \mathbf {z} \mathbf {z} ^ {\top} \mathbf {V} _ {i} ^ {\top}. \tag {26}
$$
Numerical solution. As  $Z = GZ_{0}$ , a numerical solution to Eq. (24) is easily obtained from the numerical approximation of  $G$ , i.e., the numerical solution to Eq. (18) via Algorithm 1. In fact, it suffices to replace line 2 by  $Z_{0} \sim \mathcal{P}^{Z_{0}}$  and line 7 by  $Z_{j+1} \gets \exp(\Omega_{j+1})Z_{j}$ . Notably, we do not need to compute the matrices  $G_{j}$  but only the iterates  $\Omega_{j}$ .
E.4 SDEs on the unit  $n$ -sphere
Next, we consider the homogeneous space  $(\mathbb{S}^{n - 1},\mathrm{SO}(n))$ , i.e., the unit  $n$ -sphere  $\mathbb{S}^{n - 1}$  together with the action of the group  $\mathrm{SO}(n)$  of rotations.
Notably, the orthogonal group  $\mathrm{O}(n)$  is the quadratic matrix Lie group  $\mathcal{G}(\mathbf{P})$  defined by  $\mathbf{P} = \mathbf{I}_{\mathrm{n}}$ , i.e.,  $\mathrm{O}(n) = \left\{\mathbf{A} \in \mathbb{R}^{n \times n} \mid \mathbf{A}^{\top}\mathbf{A} = \mathbf{I}_{\mathrm{n}}\right\}$  and the special orthogonal group  $\mathrm{SO}(n)$  is its connected component that contains the identity  $\mathbf{I}_{\mathrm{n}}$ . Thus, Lemma 3 ensures that a solution to Eq. (24) remains in  $\mathbb{S}^{n-1}$  if  $\mathbf{K} : [0, T] \to \mathfrak{so}(n)$  and  $\mathbf{V}_1, \ldots, \mathbf{V}_m \in \mathfrak{so}(n)$ .
In the variational setting of the main manuscript, we consider stochastic processes that solve such an SDE for the choice of  $\{\mathbf{V}_i\}_{i=1}^m = \left\{\alpha (\mathbf{e}_k \mathbf{e}_l^\top - \mathbf{e}_l \mathbf{e}_k^\top)\right\}_{1 \leq k < l \leq n}$ , where  $\alpha \neq 0$  and  $\mathbf{e}_k$  denotes the  $k$ -th standard basis vector of  $\mathbb{R}^n$ . With that choice, Eq. (24) becomes
$$
\mathrm {d} Z _ {t} = \left(\mathbf {K} (t) - \alpha^ {2} \frac {n - 1}{2} \mathbf {I} _ {\mathrm {n}}\right) Z _ {t} \mathrm {d} t + \alpha \mathrm {d} U _ {t} Z _ {t}, \quad Z _ {0} \sim \mathcal {P} ^ {Z _ {0}}. \tag {27}
$$
The diffusion term in Eq. (27) consists of a matrix-valued noise  $\mathrm{d}U_{t}$  where  $u_{ii} = 0$  and  $u_{ij} = -u_{ji}$  are independent Wiener processes (for  $i < j$ ). The following lemma (Lemma 4) converts the diffusion term to the standard form of only a vector-valued Wiener process. This facilitates (in Appendix E.5) deriving the KL-divergence between two solutions of such a SDE from existing results.
Lemma 4. Let  $W$  be an  $n$ -dimensional Wiener process and let  $\mathcal{P}^{Z_0}$  be a distribution on  $\mathbb{R}^n$  with finite second moment. Further, let  $\mathbf{K} : [0,T] \to \mathfrak{so}(n)$  be continuous. The solutions to the SDE in Eq. (27) and
$$
\mathrm {d} Z _ {t} = \left(\mathbf {K} (t) - \alpha^ {2} \frac {n - 1}{2} \mathbf {I} _ {\mathrm {n}}\right) Z _ {t} \mathrm {d} t + \alpha \left(\mathbf {I} _ {\mathrm {n}} - Z _ {t} Z _ {t} ^ {\top}\right) \mathrm {d} W _ {t}, \quad Z _ {0} \sim \mathcal {P} ^ {Z _ {0}} \tag {28}
$$
have the same probability distribution. Furthermore, for a given initial value  $Z_0 \stackrel{a.s.}{=} \xi \sim \mathcal{P}^{Z_0}$ , a sample path solution of each equation also is a sample path solution of the other one.
Proof. W.l.o.g. assume  $\alpha = 1$ . The SDEs from Eq. (27) and Eq. (28) have the same drift coefficient but a different diffusion term. Recall that Eq. (27) can be written as in Eq. (25), and denote the diffusion coefficients of Eq. (25) and Eq. (28) by  $\sigma_1: (\mathbf{z}, t) \mapsto [\mathbf{V}_1\mathbf{z}, \dots, \mathbf{V}_m\mathbf{z}]$  and  $\sigma_2: (\mathbf{z}, t) \mapsto \mathbf{I}_n - \mathbf{z}\mathbf{z}^\top$ , respectively. The lemma then follows from [61, Theorem 2.1] which states that in the present setting, the solution to Eq. (27) and Eq. (28) have the same distribution and equal sample paths if  $\sigma_1\sigma_1^\top = \sigma_2\sigma_2^\top$ , i.e., if for every  $\mathbf{z} \in \mathbb{R}^n$ ,
$$
\sum_ {i = 1} ^ {m} \mathbf {V} _ {i} \mathbf {z} \mathbf {z} ^ {\top} \mathbf {V} _ {i} ^ {\top} = \left(\mathbf {I} _ {\mathrm {n}} - \mathbf {z} \mathbf {z} ^ {\top}\right) \left(\mathbf {I} _ {\mathrm {n}} - \mathbf {z} \mathbf {z} ^ {\top}\right) ^ {\top}. \tag {29}
$$
To prove this equality, first, note that  $\mathbf{I}_{\mathrm{n}} - \mathbf{z}\mathbf{z}^{\top}$  is the projection onto the orthogonal complement of  $\mathbf{z}$  and so
$$
\left(\mathbf {I} _ {\mathrm {n}} - \mathbf {z} \mathbf {z} ^ {\top}\right) \left(\mathbf {I} _ {\mathrm {n}} - \mathbf {z} \mathbf {z} ^ {\top}\right) ^ {\top} = \mathbf {I} _ {\mathrm {n}} - \mathbf {z} \mathbf {z} ^ {\top}.
$$
To calculate the left-hand side, we insert our choice of  $\{\mathbf{V}_i\}_{i=1}^m = \left\{\alpha(\mathbf{e}_k \mathbf{e}_l^\top - \mathbf{e}_l \mathbf{e}_k^\top)\right\}_{1 \leq k < l \leq n}$ , which yields
$$
\begin{array}{l} \sum_ {i = 1} ^ {m} \mathbf {V} _ {i} \mathbf {z} \mathbf {z} ^ {\top} \mathbf {V} _ {i} ^ {\top} = - \sum_ {k = 1} ^ {n} \sum_ {l = 1} ^ {k - 1} \left(\mathbf {e} _ {k} \mathbf {e} _ {l} ^ {\top} - \mathbf {e} _ {l} \mathbf {e} _ {k} ^ {\top}\right) \mathbf {z} \mathbf {z} ^ {\top} \left(\mathbf {e} _ {k} \mathbf {e} _ {l} ^ {\top} - \mathbf {e} _ {l} \mathbf {e} _ {k} ^ {\top}\right) \\ = \underbrace {\sum_ {k = 1} ^ {n} \mathbf {e} _ {k} \mathbf {e} _ {k} ^ {\top}} _ {\mathbf {I} _ {\mathrm {n}}} \underbrace {\sum_ {l = 1} ^ {n} \langle \mathbf {e} _ {l} , \mathbf {z} \rangle \langle \mathbf {e} _ {l} , \mathbf {z} \rangle} _ {\| \mathbf {z} \| ^ {2} = 1} - \underbrace {\sum_ {k = 1} ^ {n} \langle \mathbf {e} _ {k} , \mathbf {z} \rangle \mathbf {e} _ {k}} _ {\mathbf {z}} \underbrace {\sum_ {l = 1} ^ {n} \langle \mathbf {e} _ {l} , \mathbf {z} \rangle \mathbf {e} _ {l} ^ {\top}} _ {\mathbf {z} ^ {\top}} \\ \mathbf {\Sigma} = \mathbf {I _ {n}} - \mathbf {z z} ^ {\top} \mathbf {\Sigma}. \\ \end{array}
$$
We want to point out that for vanishing drift  $\mathbf{K} = \mathbf{0}$ , the SDE in Eq. (28) becomes the well-known Stroock equation [23; 55] for a spherical Wiener process, i.e., a diffusion process on the sphere  $\mathbb{S}^{n-1}$  that is invariant under rotations. In the variational setting of the main manuscript, we use this particular process to define an uninformative prior.
To illustrate that the condition  $\mathbf{K} = \mathbf{0}$  defines a process  $Z$  that is invariant under rotations  $\mathbf{R} \in \mathrm{SO}(n)$ , we consider the process  $\mathbf{R}Z$ , cf. [23, p. 220f.]. For brevity, we introduce  $f: \mathbf{z} \mapsto -\alpha^2 (n - 1) / 2\mathbf{z}$  and  $\sigma : \mathbf{z} \mapsto \alpha (\mathbf{I}_n - \mathbf{z}\mathbf{z}^\top)$  so that Eq. (28) becomes  $\mathrm{d}Z_t = f(Z_t) + \sigma (Z_t)dW_t$ . Notably, the identities  $\mathbf{R}f(Z_t) = f(\mathbf{R}Z_t)$  and  $\mathbf{R}\sigma (Z_t)\mathbf{R}^\top = \sigma (\mathbf{R}Z_t)$  hold. Thus, the process  $\mathbf{R}Z_t$  satisfies
$$
\begin{array}{l} \mathrm {d} (\mathbf {R} Z _ {t}) = \mathbf {R} \mathrm {d} Z _ {t} \\ = \mathbf {R} f (Z _ {t}) \mathrm {d} t + \mathbf {R} \sigma (Z _ {t}) \mathrm {d} W _ {t} \\ = \mathbf {R} f (Z _ {t}) \mathrm {d} t + \mathbf {R} \sigma (Z _ {t}) \mathbf {R} ^ {\top} \mathbf {R} \mathrm {d} W _ {t} \\ = f \left(\mathbf {R} Z _ {t}\right) \mathrm {d} t + \sigma \left(\mathbf {R} Z _ {t}\right) \mathrm {d} \left(\mathbf {R} W _ {t}\right) \\ = f \left(\mathbf {R} Z _ {t}\right) \mathrm {d} t + \sigma \left(\mathbf {R} Z _ {t}\right) \mathrm {d} \tilde {W} _ {t}. \\ \end{array}
$$
As  $\tilde{W}_t = \mathbf{R}W_t$  is again a Wiener process, the processes  $Z_{t}$  and  $\mathbf{R}Z_{t}$  follow the same SDE. The invariance of  $Z$  w.r.t. rotations is also apparent from the SDE in Eq. (27) as  $U_{t}$  is already invariant under rotations, i.e., at each  $t\in [0,T]$  the random variables  $\tilde{U}_t = \mathbf{R}^\top U_t\mathbf{R}$  and  $U_{t}$  are equal in distribution.
E.5 Kullback-Leibler (KL) divergences
First, in Lemma 5, we state the general form of the KL divergence between path distributions of stochastic processes in  $\mathbb{R}^n$ .
Lemma 5. Let  $X$  and  $Y$  be stochastic processes with (path) distributions  $\mathbb{P}^X$  and  $\mathbb{P}^Y$ , respectively, and let  $W$  be a vector-valued Wiener process; all with state space  $\mathbb{R}^n$ . Further, let  $\mathcal{P}^{X_0}$  and  $\mathcal{P}^{Y_0}$  be distributions on  $\mathbb{R}^n$  with finite second moments. If  $X$  and  $Y$  solve<sup>10</sup> the following Itô SDEs on  $[0,T]$ ,
$$
\mathrm {d} X _ {t} = f \left(X _ {t}, t\right) \mathrm {d} t + \sigma \left(X _ {t}, t\right) \mathrm {d} W _ {t}, \quad X _ {0} \sim \mathcal {P} ^ {X _ {0}}, \tag {30}
$$
$$
\mathrm {d} Y _ {t} = g \left(Y _ {t}, t\right) \mathrm {d} t + \sigma \left(Y _ {t}, t\right) \mathrm {d} W _ {t}, \quad Y _ {0} \sim \mathcal {P} ^ {Y _ {0}}, \tag {31}
$$
then the KL divergence between  $\mathbb{P}^Y$  and  $\mathbb{P}^X$  is given by
$$
D _ {\mathrm {K L}} \left(\mathbb {P} ^ {Y} \| \mathbb {P} ^ {X}\right) =
$$
$$
D _ {\mathrm {K L}} \left(\mathcal {P} ^ {Y _ {0}} \| \mathcal {P} ^ {X _ {0}}\right) + \frac {1}{2} \int_ {0} ^ {T} \int_ {\mathbb {R} ^ {n}} q _ {Y _ {t}} (\mathbf {z}) [ f (\mathbf {z}, t) - g (\mathbf {z}, t) ] ^ {\top} \boldsymbol {\Sigma} ^ {+} (\mathbf {z}, t) [ f (\mathbf {z}, t) - g (\mathbf {z}, t) ] \mathrm {d} \mathbf {z} \mathrm {d} t. \tag {32}
$$
Here,  $f$  and  $g$  denote vector-valued drift coefficients, and  $\sigma$  denotes the matrix-valued diffusion coefficient. Furthermore,  $q_{Y_t}$  denotes the marginal density of  $Y_t$  (i.e., the process  $Y$  at time  $t$ ) and  $\pmb{\Sigma}^{+}$  denotes the pseudo inverse of  $\pmb{\Sigma} = \sigma \pmb{\sigma}^{\top}$ .
<sup>10</sup>We further assume that  $f, g$  and  $\sigma$  satisfy the assumptions of [57, Theorem 5.2.1] such that a unique solution exists (for a given  $X_0, Y_0 \stackrel{\mathrm{a.s.}}{=} \xi$ ).
For a proof via Girsanov's theorem, we refer the reader to [69, Section A] and [67, Lemma 2.1]. For a heuristic derivation via the limit of an Euler discretization, we refer to [43, Section 4].
The following corollary specifies Lemma 5 to a particular class of SDEs that is compatible with the SDEs introduced in Appendix E.3.
Corollary 6. Let  $\mathbf{V}_0^X, \mathbf{V}_0^Y : [0,T] \to \mathbb{R}^{n \times n}$  be continuous maps and let  $\mathbf{V}_1, \ldots, \mathbf{V}_m \in \mathbb{R}^{n \times n}$ . Further, let  $w^1, \ldots, w^m$  be independent scalar-valued Wiener processes and let  $\mathcal{P}^{X_0}$  and  $\mathcal{P}^{Y_0}$  be distributions on  $\mathbb{R}^n$  with finite second moments. If  $X$  and  $Y$  are stochastic processes on  $[0,T]$  with (path) distributions  $\mathbb{P}^X$  and  $\mathbb{P}^Y$ , resp., that solve
$$
\mathrm {d} X _ {t} = \left(\mathbf {V} _ {0} ^ {X} (t) \mathrm {d} t + \sum_ {i = 1} ^ {m} \mathrm {d} w _ {t} ^ {i} \mathbf {V} _ {i}\right) X _ {t}, \quad X _ {0} \sim \mathcal {P} ^ {X _ {0}}, \tag {33}
$$
$$
\mathrm {d} Y _ {t} = \left(\mathbf {V} _ {0} ^ {Y} (t) \mathrm {d} t + \sum_ {i = 1} ^ {m} \mathrm {d} w _ {t} ^ {i} \mathbf {V} _ {i}\right) Y _ {t}, \quad Y _ {0} \sim \mathcal {P} ^ {Y _ {0}}, \tag {34}
$$
then, the KL divergence between  $\mathbb{P}^Y$  and  $\mathbb{P}^X$  is given by
$$
D _ {\mathrm {K L}} \left(\mathbb {P} ^ {Y} \| \mathbb {P} ^ {X}\right) = D _ {\mathrm {K L}} \left(\mathcal {P} ^ {Y _ {0}} \| \mathcal {P} ^ {X _ {0}}\right) + \frac {1}{2} \int_ {0} ^ {T} \int_ {\mathbb {R} ^ {n}} q _ {Y _ {t}} (\mathbf {z}) \left[ \mathbf {V} _ {0} ^ {\Delta} (t) \mathbf {z} \right] ^ {\top} \boldsymbol {\Sigma} ^ {+} (\mathbf {z}) \left[ \mathbf {V} _ {0} ^ {\Delta} (t) \mathbf {z} \right] \mathrm {d} \mathbf {z} \mathrm {d} t, \tag {35}
$$