SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Condition 1. For a given  $k$ , define  $\mathbb{Q}_{(k)}$  to be the path measure corresponding to the (infinite) system
$$
\left\{ \begin{array}{l} \mathrm {d} X _ {t} ^ {i} = b ^ {i} \left(X _ {t}, t\right) \mathrm {d} t + \mathrm {d} W _ {t} ^ {i}, \quad i \neq k \\ \mathrm {d} X _ {t} ^ {i} = \mathrm {d} W _ {t} ^ {k}, \quad i = k. \end{array} \right. \tag {22}
$$
We say that  $\mathbb{Q}$  satisfies the finite local entropy condition if  $\mathrm{KL}\left[\mathbb{Q}\parallel \mathbb{Q}_{(k)}\right] < \infty ,\forall k$
Define  $\mathcal{F}_t^{(i)} = \sigma (X_0^i,X_s^j,0\leq s\leq t,j\neq i)$
Assumption 1.
$$
\int_ {0} ^ {T} b ^ {i} \left(X _ {t}, t\right) ^ {2} \mathrm {d} t + \sum_ {j \neq i} \mathbb {E} \left[ \int_ {0} ^ {T} \left(b ^ {j} \left(X _ {t}, t\right) - \mathbb {E} \left[ b ^ {j} \left(X _ {t}, t\right) \mid \mathcal {F} _ {t} ^ {(i)} \right]\right) ^ {2} \mathrm {d} t \right] <   \infty , \mathbb {Q} _ {(i)} a. s. \tag {23}
$$
Notice that if Assumption 1 is true, then Condition 1 holds (Föllmer (1986), Thm. 2.23)
Theorem 3. If KL  $\left[\mathbb{Q}\parallel \mathbb{Q}_{(k)}\right] < \infty$  then KL  $\left[\hat{\mathbb{Q}}\parallel \hat{\mathbb{Q}}_{(k)}\right] < \infty$
Proof. The proof can be obtained by adapting the result of Lemma 3.6 of Föllmer & Wakolbinger (1986).  $\square$
This Theorem states that if the forward FDP path measure  $\mathbb{Q}$  satisfies the finite local entropy condition, then also the reverse FDP path measure  $\hat{\mathbb{Q}}$  satisfies the finite local entropy condition.
Theorem 4. Let  $\mathbb{Q}$  be a finite entropy measure. Then:
$$
\left\{ \begin{array}{l l} \mathrm {d} X _ {t} ^ {k} = b ^ {k} \left(X _ {t}, t\right) \mathrm {d} t + \mathrm {d} W _ {t} ^ {k}, & \text {u n d e r} \quad \mathbb {Q} \\ \mathrm {d} \hat {X} _ {t} ^ {k} = \hat {b} ^ {k} \left(\hat {X} _ {t}, t\right) \mathrm {d} t + \mathrm {d} \hat {W} _ {t} ^ {k}, & \text {u n d e r} \quad \hat {\mathbb {Q}} \end{array} \right. \tag {24}
$$
$$
\frac {\partial \log \left(\rho_ {t} ^ {(d)} \left(x ^ {k} \mid x ^ {j} , j \neq k\right)\right)}{\partial x ^ {k}} = \hat {b} ^ {k} (x, T - t) + b ^ {k} (x, t) \tag {25}
$$
Proof. For the proof, we refer to Theorem 3.14 of Föllmer & Wakolbinger (1986).
A.2 Millet Formulation
Let  $L^2 (R) = \{x\in H:\sum r^i (x^i)^2 <  \infty \}$ . For simplicity, we overload the notation of the letter  $K$ , and use it for generic constants, that might be different on a case by case basis.
Assumption 2.
$$
\forall x \in L ^ {2} (R), \sup  _ {t} \left\{\sum r ^ {i} \left(b ^ {i} (x, t)\right) ^ {2} \right\} + \sum \left(r ^ {i}\right) ^ {2} \leq K \left(1 + \sum r ^ {i} \left(x ^ {i}\right) ^ {2}\right)
$$
$$
\forall x, y \in L ^ {2} (R), \sup  _ {t} \{\sum r ^ {i} (b ^ {i} (x, t) - b ^ {i} (y, t)) ^ {2} \} \leq K \sum r ^ {i} (x ^ {i} - y ^ {i}) ^ {2}
$$
This assumption is simply the translation of H1 from Millet et al. (1989) to our notation.
Assumption 3. There exists an increasing sequence of finite subsets  $J(n), n \in \mathbb{N}, \cup_{n} J(n) = \mathbb{N}$  such that  $\forall n \in \mathbb{N}, M > 0$  there exists a constant  $K(M, n)$  such that the following holds:
$$
\sup  _ {t} \left(\sup  _ {i \in J (n)} \left(\left(\sup  _ {x} | b ^ {i} (x, t) |: \sup  _ {j \in J (n)} | x ^ {j} | \leq M\right) + \sum_ {j} r ^ {j}\right)\right) \leq K (M, n).
$$
Again, this assumption is simply the translation of H5 from Millet et al. (1989) to our notation.
Assumption 4. Either i):
$$
\forall x, y \in L ^ {2} (R), \sup  _ {t} \left\{\sum r ^ {i} \left(b ^ {i} (x, t) - b ^ {i} (y, t)\right) ^ {2} \right\} \leq K \sum \left(r ^ {i}\right) ^ {2} \left(x ^ {i} - y ^ {i}\right) ^ {2},
$$
or ii):  $\forall i, b^{i}(x)$  is a function of  $x$  for at most  $M$  coordinates and
$$
\forall x, y \in L ^ {2} (R), \sup  _ {t} \{\sum (r ^ {i}) ^ {2} (b ^ {i} (x, t) - b ^ {i} (y, t)) ^ {2} \} \leq K \sum (r ^ {i}) ^ {2} (x ^ {i} - y ^ {i}) ^ {2}.
$$
This corresponds to satisfying either H3 or jointly H2 and H4 of Millet et al. (1989). For simplicity, we can combine together the different assumptions into
Assumption 5. Let Assumption 2, Assumption 3, and Assumption 4 hold.
Finally, we state required assumptions about the density:
Assumption 6. Suppose that the initial condition is  $X_0 \in L^2(R)$ .
- Assume that the conditional law of  $x^i$  given  $x^j, j \neq i$  has density  $\rho_t^{(d)}(x^i \mid x^j, j \neq i)$  w.r.t Lebesgue measure on  $\mathbb{R}$ .
- Assume that  $\int_{t_0}^1\int_{D_J}|r^i\frac{\mathrm{d}}{\mathrm{d}x^i} (\rho_t^{(d)}(x^i\mid x^j,j\neq i))|\mathrm{d}x^i\rho_t(\mathrm{d}x^{j\neq i})\mathrm{d}t < \infty$  for fixed subset  $J\subset \mathbb{N},t_0 > 0$  and  $D_{J} = \{(\prod_{j\in J}K_{j})\times (\prod_{j\notin J}\mathbb{R}),K_{j}$  compact in  $\mathbb{R}\} \cap L^2 (R)$ .
We reported in our notation the content of Theorem 4.3 of Millet et al. (1989). This can be used to prove the existence of the backward process.
A.3 Proof of Theorem 1
If  $R = I$ , then we assume Assumption 1. Consequently,  $\mathbb{Q}$  is a finite entropy measure. Then Theorem 4 holds, from which the desired result. If, instead  $R \neq I$ , then we require Assumption 5, Assumption 6. Application of Thm 4.3 of Millet et al. (1989) allows to prove the validity of Theorem 1 also in this case.
A.3.1 Proof of Corollary 1
Assumption 5 is required directly. We need to show that with the considered restrictions Assumption 6 is valid.
Since  $\sum_{i}r^{i} < \infty$ , then  $\sum_{i}(r^{i})^{2} = K_{a} < \infty$ . Moreover,  $(b^{i}(x^{i},t))^{2} < K_{b}^{2}(x^{i})^{2}$ . Then,  $\forall x\in L^2 (R)$ , the following holds  $\sup_t\{\sum r^i (b^i (x,t))^2\} +\sum (r^i)^2\leq \sum r^i K_b^2 (x^i)^2 +K_a\leq$ $\max (K_a,K_b^2)\left(1 + \sum r^i (x^i)^2\right)$ . Similarly,  $\forall x,y\in L^{2}(R)$  we have  $\sup_t\{\sum r^i (b^i (x,t) - b^i (y,t))^2\} \leq \sum r^i K_b^2 (x^i -y^i)^2$ . Thus Assumption 2 is satisfied.
Since  $b^{i}(x,t)$  is bounded and independent on  $t$ , Assumption 3 is satisfied, as explicitly discussed in Millet et al. (1989).
Finally, since  $b^{i}(x)$  is a function of  $x$  for  $M = 1$  coordinate, and  $\sup_{t} \{ \sum (r^{i})^{2} (b^{i}(x,t) - b^{i}(y,t))^{2} \} \leq \sum (r^{i})^{2} K_{b}^{2} (x^{i} - y^{i})^{2}$ , Assumption 4 is satisfied.
Then, combined together Assumption 5 holds.
A.4 Girsanov Regularity
Condition 2. Assume that  $\gamma_{\theta}(x,t)$  is an  $\hat{\mathcal{F}}$  measurable process and that either:
$$
\mathbb {E} _ {\hat {\mathbb {Q}}} \left[ \exp \left(\frac {1}{2} \int_ {0} ^ {T} \left\| \gamma_ {\boldsymbol {\theta}} (\hat {X} _ {t}, t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t\right) \right] = \mathbb {E} _ {\mathbb {Q}} \left[ \exp \left(\frac {1}{2} \int_ {0} ^ {T} \| \gamma_ {\boldsymbol {\theta}} (X _ {t}, t) \| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t\right) \right] <   \infty , \tag {26}
$$
$$
\exists \delta > 0: \mathbb {E} _ {\hat {\mathbb {Q}}} \left[ \exp \left(\frac {1}{2} \left\| \gamma_ {\boldsymbol {\theta}} (\hat {X} _ {\delta}, \delta) \right\| _ {R ^ {\frac {1}{2}} H} \mathrm {d} t\right) \right] <   \infty . \tag {27}
$$
This is equivalent to the regularity condition in eq. 10.23 of Da Prato & Zabczyk (2014) or Proposition 10.17 in Da Prato & Zabczyk (2014).
A.5 Proof of KL divergence expression
We leverage Equation (7) to express the Kullback-Leibler divergence as:
$$
\mathrm {K L} \left[ \hat {\mathbb {Q}} \| \hat {\mathbb {P}} ^ {\chi_ {T}} \right] = \mathbb {E} _ {\hat {\mathbb {Q}}} \left[ \log \frac {\mathrm {d} \hat {\mathbb {Q}} _ {0}}{\mathrm {d} \hat {\mathbb {P}} _ {0}} + \log \frac {\mathrm {d} \rho_ {T}}{\mathrm {d} \chi_ {T}} \right] = \mathbb {E} _ {\hat {\mathbb {Q}}} \left[ \log \frac {\mathrm {d} \hat {\mathbb {Q}} _ {0}}{\mathrm {d} \hat {\mathbb {P}} _ {0}} \right] + \mathrm {K L} [ \rho_ {T} \| \chi_ {T} ] =
$$
$$
\mathbb {E} _ {\hat {\mathbb {Q}}} \left[ - \int_ {0} ^ {T} \langle \gamma_ {\boldsymbol {\theta}} (\hat {X} _ {t}, t), \mathrm {d} \hat {W} _ {t} \rangle_ {R ^ {\frac {1}{2}} H} + \frac {1}{2} \int_ {0} ^ {T} \left\| \gamma_ {\boldsymbol {\theta}} (\hat {X} _ {t}, t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \right] + \operatorname {K L} [ \rho_ {T} \| \chi_ {T} ] =
$$
$$
\frac {1}{2} \mathbb {E} _ {\hat {\mathbb {Q}}} \left[ \int_ {0} ^ {T} \left\| \gamma_ {\boldsymbol {\theta}} (\hat {X} _ {t}, t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \right] + \operatorname {K L} [ \rho_ {T} \| \chi_ {T} ] = \frac {1}{2} \mathbb {E} _ {\mathbb {Q}} \left[ \int_ {0} ^ {T} \| \gamma_ {\boldsymbol {\theta}} (X _ {t}, t) \| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \right] + \operatorname {K L} [ \rho_ {T} \| \chi_ {T} ].
$$
Moreover, since
$$
\mathrm {K L} \left[ \hat {\mathbb {Q}} \parallel \hat {\mathbb {P}} ^ {\chi_ {T}} \right] = \mathbb {E} _ {\mathbb {Q}} \left[ \log \frac {\mathrm {d} \hat {\mathbb {Q}} _ {T}}{\mathrm {d} \hat {\mathbb {P}} _ {T} ^ {\chi_ {T}}} + \log \frac {\mathrm {d} \rho_ {0}}{\mathrm {d} \chi_ {0}} \right] \geq \mathrm {K L} \left[ \rho_ {0} \parallel \chi_ {0} \right],
$$
we can combine the two results and obtain Equation (8)
A.6 Conditional score matching
In this subsection we prove the equality in Equation (13):
$$
\begin{array}{l} \mathbb {E} _ {\mathbb {Q}} \left[ \int_ {0} ^ {T} \| \gamma_ {\boldsymbol {\theta}} (X _ {t}, t) \| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \right] = \int_ {0} ^ {T} \int_ {H} \| \gamma_ {\boldsymbol {\theta}} (x, t) \| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \mathrm {d} \rho_ {t} (x) = \\ \int_ {0} ^ {T} \int_ {H} \left\| D _ {x} \log \rho_ {T - t} ^ {(d)} (x) - s _ {\boldsymbol {\theta}} (x, T - t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \mathrm {d} \rho_ {t} (x) = \\ \int_ {0} ^ {T} \int_ {H \times H} \left\| D _ {x} \log \rho_ {t} ^ {(d)} (x) - s _ {\boldsymbol {\theta}} (x, t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \mathrm {d} \rho_ {t} (x, x _ {0}) = \\ \int_ {0} ^ {T} \int_ {H \times H} \left\| D _ {x} \log \rho_ {t} ^ {(d)} (x) - D _ {x} \log \rho_ {t} ^ {(d)} (x \mid x _ {0}) + D _ {x} \log \rho_ {t} ^ {(d)} (x \mid x _ {0}) - s _ {\boldsymbol {\theta}} (x, t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} \mathrm {d} t \mathrm {d} \rho_ {t} (x, x _ {0}) = \\ \int_ {0} ^ {T} \int_ {H \times H} \left\| D _ {x} \log \rho_ {t} ^ {(d)} (x) - D _ {x} \log \rho_ {t} ^ {(d)} (x \mid x _ {0}) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} + \left\| D _ {x} \log \rho_ {t} ^ {(d)} (x \mid x _ {0}) - s _ {\boldsymbol {\theta}} (x, t) \right\| _ {R ^ {\frac {1}{2}} H} ^ {2} + \\ 2 \left\langle D _ {x} \log \rho_ {t} ^ {(d)} (x) - D _ {x} \log \rho_ {t} ^ {(d)} (x \mid x _ {0}), D _ {x} \log \rho_ {t} ^ {(d)} (x \mid x _ {0}) - s _ {\boldsymbol {\theta}} (x, t) \right\rangle \mathrm {d} t \mathrm {d} \rho_ {t} (x, x _ {0}). \\ \end{array}
$$
To simplify the equality, we need to notice that:
$$
\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) \mathrm {d} x ^ {i} = \mathrm {d} \rho_ {t} (x ^ {i} | x ^ {j \neq i}) = \int_ {x _ {0}} \mathrm {d} \rho_ {t} (x _ {0} | x) \mathrm {d} \rho_ {t} (x ^ {i} | x ^ {j \neq i}) = \int_ {x _ {0}} \mathrm {d} \rho_ {t} (x ^ {i}, x _ {0} | x ^ {j \neq i}) =
$$
$$
\int_ {x _ {0}} \mathrm {d} \rho_ {t} (x ^ {i} | x _ {0}, x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} | x ^ {j \neq i}) = \mathrm {d} x ^ {i} \int_ {x _ {0}} \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0}, x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} | x ^ {j \neq i}).
$$
Then, computing
$$
\int_ {x _ {0}} \frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \log \rho^ {(d)} (x ^ {i} | x ^ {j \neq i}, x _ {0}) \mathrm {d} \rho_ {t} (x, x _ {0}) = \int_ {x _ {0}} \frac {\frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \rho^ {(d)} (x ^ {i} | x ^ {j \neq i} , x _ {0})}{\rho^ {(d)} (x ^ {i} | x ^ {j \neq i} , x _ {0})} \mathrm {d} \rho_ {t} (x, x _ {0}) =
$$
$$
\int_ {x _ {0}} \frac {\frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \rho^ {(d)} (x ^ {i} | x ^ {j \neq i} , x _ {0})}{\rho^ {(d)} (x ^ {i} | x ^ {j \neq i} , x _ {0})} \mathrm {d} \rho_ {t} (x ^ {i} | x ^ {j \neq i}, x _ {0}) \mathrm {d} \rho_ {t} (x _ {0}, x ^ {j \neq i}) = \int_ {x _ {0}} \frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \rho^ {(d)} (x ^ {i} | x ^ {j \neq i}, x _ {0}) \mathrm {d} x ^ {i} \mathrm {d} \rho_ {t} (x _ {0}, x ^ {j \neq i}) =
$$
$$
\int_ {x _ {0}} \frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \rho^ {(d)} \left(x ^ {i} \mid x ^ {j \neq i}, x _ {0}\right) \mathrm {d} x ^ {i} \mathrm {d} \rho_ {t} \left(x _ {0} \mid x ^ {j \neq i}\right) \mathrm {d} \rho_ {t} \left(x ^ {j \neq i}\right) = \frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \left(\int_ {x _ {0}} \rho^ {(d)} \left(x ^ {i} \mid x ^ {j \neq i}, x _ {0}\right) \mathrm {d} \rho_ {t} \left(x _ {0} \mid x ^ {j \neq i}\right)\right) \mathrm {d} x ^ {i} \mathrm {d} \rho_ {t} \left(x ^ {j \neq i}\right) =
$$
$$
\frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) \mathrm {d} x ^ {i} \mathrm {d} \rho_ {t} (x ^ {j \neq i}) = \frac {\mathrm {d} \log \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})}{\mathrm {d} x ^ {i}} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) \mathrm {d} x ^ {i} \mathrm {d} \rho_ {t} (x ^ {j \neq i}) = \frac {\mathrm {d} \log \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \ne i})}{\mathrm {d} x ^ {i}} \mathrm {d} \rho_ {t} (x)
$$
Consequently:
$$
\int_ {H \times H} \left\langle D _ {x} \log \rho_ {t} ^ {(d)} (x) - D _ {x} \log \rho_ {t} ^ {(d)} (x | x _ {0}), s _ {\pmb {\theta}} (x, t) \right\rangle \mathrm {d} \rho_ {t} (x, x _ {0}) = 0.
$$
Combining together and rearranging the terms, we get the desired Equation (13).
A.7 Explicit expression of score function
As mentioned in the text, we consider the case  $f = 0$ . In this case, there exists a weak solution to Equation (1) as:
$$
X _ {t} = \exp (t \mathcal {A}) X _ {0} + \int_ {0} ^ {t} \exp ((t - s) \mathcal {A}) \mathrm {d} W _ {s}. \tag {28}
$$
Consequently, the true score function has expression:
$$
\begin{array}{l} \frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \log \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) = \frac {\frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \frac {\frac {\mathrm {d}}{\mathrm {d} x ^ {i}} \int_ {x _ {0}} \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0} , x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} | x ^ {j \neq i})}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- \int_ {x _ {0}} (s ^ {i}) ^ {- 1} \left(x ^ {i} - \exp \bigl (t b ^ {i} \bigr) x _ {0} ^ {i}\right) \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0} , x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} | x ^ {j \neq i})}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- (s ^ {i}) ^ {- 1} \left(x ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) - \int_ {x _ {0}} \exp (t b ^ {i}) x _ {0} ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0} , x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} | x ^ {j \neq i})\right)}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- (s ^ {i}) ^ {- 1} \left(x ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) - \int_ {x _ {0}} \exp \big (t b ^ {i} \big) x _ {0} ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0} , x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} | x ^ {j \neq i})\right)}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- (s ^ {i}) ^ {- 1} \left(x ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) - \int_ {x _ {0} ^ {i}} \exp (t b ^ {i}) x _ {0} ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0} ^ {i} , x ^ {j \neq i}) \mathrm {d} \rho_ {t} (x _ {0} ^ {i} | x ^ {j \neq i})\right)}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- (s ^ {i}) ^ {- 1} \left(x ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) - \int_ {x _ {0} ^ {i}} \exp \big (t b ^ {i} \big) x _ {0} ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x _ {0} ^ {i} , x ^ {j \neq i}) \rho^ {(d)} (x _ {0} ^ {i} | x ^ {j \neq i}) \mathrm {d} x _ {0} ^ {i}\right)}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- (s ^ {i}) ^ {- 1} \left(x ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) - \int_ {x _ {0} ^ {i}} \exp (t b ^ {i}) x _ {0} ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} , x _ {0} ^ {i} | x ^ {j \neq i}) \mathrm {d} x _ {0} ^ {i}\right)}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ \frac {- (s ^ {i}) ^ {- 1} \left(x ^ {i} \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}) - \int_ {x _ {0} ^ {i}} \exp (t b ^ {i}) x _ {0} ^ {i} \rho_ {t} ^ {(d)} (x _ {0} ^ {i} | x) \mathrm {d} x _ {0} ^ {i}\right) \rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i}))}{\rho_ {t} ^ {(d)} (x ^ {i} | x ^ {j \neq i})} = \\ - \left(s ^ {i}\right) ^ {- 1} \left(x ^ {i} - \int_ {x _ {0} ^ {i}} \exp \left(t b ^ {i}\right) x _ {0} ^ {i} \rho_ {t} ^ {(d)} \left(x _ {0} ^ {i} | x\right) \mathrm {d} x _ {0} ^ {i}\right) \\ \end{array}
$$
where  $s^i = r^i \frac{\exp(2b^i t) - 1}{2b^i}$ . This is exactly the desired Equation (11). Similar calculations allow to prove  $D_x \log \rho_t^{(d)}(x \mid x_0) = -\mathcal{S}(t)^{-1} (x - \exp(t \mathcal{A}) x_0)$ .
B Fokker Planck equation
In this Section we discuss the infinite dimensional generalization of the classical Fokker Planck equation. We can associate to Eq. (1) the differential operator:
$$
\mathcal {L} _ {0} u (x, t) = D _ {t} u (x, t) + \underbrace {\frac {1}{2} \operatorname {T r} \left\{R D _ {x} ^ {2} u (x , t) \right\} + \langle \mathcal {A} x + f (x , t) , D _ {x} u (x , t) \rangle} _ {\mathcal {L} u (x, t)}, \quad x \in H, t \in [ 0, T ], \tag {29}
$$
where  $D_{t}$  is the time derivative,  $D_{x},D_{x}^{2}$  are first and second order Fréchet derivatives in space. The domain of the operator  $\mathcal{L}_0$  is  $D(\mathcal{L}_0)$ , the linear span of real parts of functions  $u_{\phi ,h} = \phi (t)\exp (i\langle x,h(t)\rangle),x\in H,t\in [0,T]$  where  $\phi \in C^1 ([0,T]),\phi (T) = 0,h\in C^1 ([0,T];D(\mathcal{A}^\dagger))$  where  $\dagger$  indicates adjoint. Provided appropriate conditions are satisfied, see for example Bogachev et al. (2009, 2011), the time varying measure  $\rho_t(\mathrm{d}x)\mathrm{d}t$  exists, is unique, and solves the Fokker-Planck equation  $\mathcal{L}_0^\dagger \rho_t = 0$
C Uncertainty principle
We here clarify that Hilbert spaces of square integrable functions that are not, in general, simultaneously homogeneous and separable. For example, while  $\mathbb{R}$  is homogeneous, the set of square integrable functions over  $\mathbb{R}$  is not separable, since the basis is the uncountable set  $\cos(2\pi \nu p)$ ,  $\sin(2\pi \nu p)$ ,  $\nu \in \mathbb{R}$ . Then, FDP requirements are not met, as we need a countable basis. Moreover, we would need in
general an infinite number of samples (grid over the whole  $\mathbb{R}$ ) to reconstruct the functions. Conversely, a set like the interval  $I = [0,1] \subset \mathbb{R}$  has countable basis  $\cos(2\pi tp), \sin(2\pi tp), t \in \mathbb{Z}$  (and thus is separable) and, considering  $x$  to be band-limited, a sampling grid with finite cardinality would allow to reconstruct of the function. However,  $I$  is not homogeneous as no isometry group exists. Consequently, Theorem 2 is not applicable. To fix the issue, one could naively think of extending any function defined over  $I$  to the whole  $\mathbb{R}$  by considering  $\bar{x}[p] = x[p], p \in I$  and  $\bar{x}[p] = 0, p \notin I$ . Obviously, if  $x \in L_2(I)$  then  $\bar{x} \in L_2(\mathbb{R})$ . However, since  $\bar{x}$  has finite support, it cannot be bandlimited, making such an approach not a viable solution. In classical signal processing literature, the problem is usually referred to as the uncertainty principle (Slepian, 1983).
D A complete example
We present an example in which we cast Equation (20) for square integrable functions over the interval  $I = [0,1]$ ,  $L^2(I)$ . In this case, one natural selection for the basis is the Fourier basis  $e^k = \{\ldots, \exp(-j2\pi 2p), \exp(-j2\pi p), 1, \exp(j2\pi p), \exp(j2\pi 2p), \ldots\}$ . Assume the operator  $\mathcal{A}$  to be a pseudo-differential operator, such that  $\langle \mathcal{A}x, e^k \rangle = b^k x^k$ . Also, assume that  $b^k, r^k$  are selected such that conditions of Corollary 1 are met, and consequently the backward process exists. Since we are working with samples collected on the grid  $x \left[ \frac{i}{N} \right]$  we first map the samples to the frequency domain, and then build a Fourier-like representation with a finite set of sinusoids. We then define the mapping  $\mathfrak{F}(z^i)^k \stackrel{\mathrm{def}}{=} \sum_{i=0}^{N-1} z^i \exp\left(-j2\pi k \frac{i}{N}\right)$  and its inverse  $\mathfrak{I}(z^i)^k \stackrel{\mathrm{def}}{=} N^{-1} \sum_{i=0}^{N-1} z^i \exp\left(j2\pi k \frac{i}{N}\right)$ . This suggests to consider the following expression for the interpolating functions:
$$
\xi^ {i} = \frac {1}{N} \sum_ {k = 0} ^ {N - 1} e ^ {k} \exp \left(- j 2 \pi k \frac {i}{N}\right) = \frac {1}{N} \sum_ {k = 0} ^ {N - 1} \exp \left(j 2 \pi k \left(p - \frac {i}{N}\right)\right).
$$
Those functions are indeed nothing but a frequency truncated version of the sinc function, which is the classical reconstruction function of the sampling theorem on 1-D signals. Moreover  $\langle \xi^i,\zeta^k\rangle = \delta (i - k)$ . We are now ready to show  $i$  ) the expression of the forward process,  $ii$  ) the expression of the parametric score function  $s_\theta$  and  $\gamma_{\theta}$  , iii) the computation of the ELBO and finally  $iv$  ) the expression for the backward process.
The forward process defined in Equation (20) has expression:
$$
\mathrm {d} X _ {t} [ ^ {k} / N ] = \Im \left(b ^ {l} \mathfrak {F} \left(X _ {t} [ ^ {i} / N ]\right) ^ {l}\right) ^ {k} \mathrm {d} t + \mathrm {d} W _ {t} [ ^ {k} / N ], \quad k = 1, \dots , | Z |, \tag {30}
$$
where  $\mathrm{d}W_{t}[^{k} / N]\simeq \mathfrak{F}(\mathrm{d}W_{t}^{i})^{k}$ . Simple calculations show that  $X_{t}[^{k} / N]$  is equivalent in distribution to
$$
X _ {t} \left[ ^ {k} / N \right] = \Im \left(\exp (b ^ {l} t) \mathfrak {F} \left(X _ {0} [ ^ {i} / N ]\right) ^ {l} + \sqrt {s ^ {l}} \epsilon^ {l}\right) ^ {k}, \tag {31}
$$
where  $s^l = \langle S(t), e^l \rangle = r^l \frac{\exp(2b^l t) - 1}{2b^l}$  and  $\epsilon^l \sim \mathcal{N}(0,1)$ , allowing simulation of the forward process in a single step. The parametric score function can be approximated as:
$$
\begin{array}{l} s _ {\theta} \left(\sum_ {i} X _ {t} [ i / N ] \xi^ {i}, t\right) [ i / N ] = \tag {32} \\ - \Im \left(\frac {\mathfrak {F} (X _ {t} [ ^ {i} / N ]) ^ {k} - \exp (b ^ {k} t) \mathfrak {F} (n (g (X _ {t} [ ^ {l} / N ]) , t , \pmb {\theta}) [ ^ {l} / N ])}{s ^ {k}}\right) ^ {i}. \\ \end{array}
$$
Similarly:
$$
\begin{array}{l} \tilde {\gamma} _ {\theta} \left(\sum_ {i} X _ {t} [ i / N ] \xi^ {i}, \sum_ {i} X _ {0} [ i / N ] \xi^ {i}, t\right) [ i / N ] = \tag {33} \\ \left. - \Im \left(\frac {\exp (b ^ {k} t)}{s ^ {k}} \left(\mathfrak {F} \left(n (g (X _ {t} [ l / N ]), t, \boldsymbol {\theta}) [ l / N ] - X _ {0} [ l / N ]\right) ^ {k}\right)\right) ^ {i}. \right. \\ \end{array}
$$
<sup>3</sup>We stress that although we should consider a real Hilbert space, we select the complex exponential to avoid cluttering the notation. It is possible to select  $\{\cos(2\pi p), \sin(2\pi p), \cos(2\pi 2p), \sin(2\pi 2p), \dots \}$  as a basis, and redoing the calculations in this Section we can obtain a functionally equivalent scheme as the one with the real basis.
Combining Equation (31) and Equation (33) we can fully characterize the training objective defined in Equation (19). Then, it is possible to optimize the value of the parameters  $\theta$  with any gradient-based optimizer.
Finally, the backward process approximation is expressed as:
$$
\mathrm {d} \hat {X} _ {t} [ ^ {k} / N ] = - \Im \left(b ^ {l} \mathfrak {F} (\hat {X} _ {t} [ ^ {i} / N ]) ^ {l}\right) ^ {k} + \Im \left(r ^ {l} \mathfrak {F} \left(s _ {\theta} \left(\sum_ {i} \hat {X} _ {t} [ ^ {i} / N ] \xi^ {i}, T - t\right) [ ^ {i} / N ]\right) ^ {l}\right) \mathrm {d} t + \mathrm {d} W _ {t} [ ^ {k} / N ] \tag {34}
$$
$$
k = 1, \ldots , | Z |,
$$
from which new samples can be generated.
D.1 Proofs
We start by proving Equation (30). Starting from the drift term of Equation (20), we have the following chain of equalities: