SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
d F \left(x _ {t}, t\right) = \left(\frac {d F}{d t} + J _ {F} \mu + \frac {g (t) ^ {2}}{2} \triangle F\right) d t + g (t) J _ {F} d B _ {t}.
$$
Lastly, if  $x_{t}$  is instead defined with a reverse noise,
$$
d x _ {t} = \mu (x _ {t}, t) d t + g (t) d \bar {B} _ {t},
$$
then the multi-valued Ito's lemma is modified as follows:
$$
d F \left(x _ {t}, t\right) = \left(\frac {d F}{d t} + J _ {F} \mu - \frac {g (t) ^ {2}}{2} \triangle F\right) d t + g (t) J _ {F} d \bar {B} _ {t}. \tag {13}
$$
Lastly, we present the Fokker-Planck equation which states that the probability distribution that corresponds to diffusion processes satisfy a certain partial differential equation:
Lemma A.3 (Fokker-Planck equation). Let  $x_{t}$  be defined by
where  $x_{t}, \mu(x, t) \in \mathbb{R}^{d}$  and  $B_{t}$  is a Brownian motion in  $\mathbb{R}^{d}$ . Denote by  $p(x, t)$  the density at point  $x$  on time  $t$ . Then,
$$
\frac {\partial}{\partial t} p (x, t) = - \nabla \cdot (\mu (x, t) p (x, t)) + \frac {g (t) ^ {2}}{2} \triangle p (x, t) = - p \nabla \cdot \mu - \mu \nabla \cdot p + \frac {g (t) ^ {2}}{2} \triangle p.
$$
Preliminaries on analytic functions
Definition A.4. A function  $f\colon \mathbb{R}^d\to \mathbb{R}$  is analytic on  $\mathbb{R}^d$  if for any  $x_0,x\in \mathbb{R}^d$ , the Taylor series of  $f$  around  $x_0$ , evaluated at  $x$ , converges to  $f(x)$ . We say that  $F = (f_{1},\ldots ,f_{n})\colon \mathbb{R}^{d}\to \mathbb{R}^{d}$  is an analytic function if  $f_{i}$  is analytic for all  $i\in \{1,\dots ,n\}$ .
Lemma A.5. If  $F, G \colon \mathbb{R}^d \to \mathbb{R}^d$  are two analytic functions and if  $F = G$  for all  $x \in U$  where  $U \subseteq \mathbb{R}^d$ ,  $U \neq 0$ , is an open set, then  $F = G$  on all  $\mathbb{R}^d$ .
This is a well known result and a proof sketch was given in Section 3.
The heat equation. The following is a Folklore lemma on the uniqueness of the solutions to the heat equation:
Lemma A.6. Let  $p$  and  $p'$  be two continuous functions on  $\mathbb{R}^d \times [t_0, 1]$  that satisfy the heat equation
$$
\frac {\partial p}{\partial t} = \frac {g (t) ^ {2}}{2} \triangle p. \tag {14}
$$
Further, assume that  $p(\cdot ,t_0) = p'(\cdot ,t_0)$ . Then,  $p = p'$  for all  $t\in [t_0,1]$ .
A.3 Main proof
In what appears below we denote
$$
s (x, t) := \frac {h (x , t) - x}{\sigma_ {t} ^ {2}}. \tag {15}
$$
We start by claiming that if  $h$  satisfies Property 1, then  $s$  satisfies the PDE Eq. (11): (proof in Section A.4)
Lemma A.7. Let  $h$  satisfy Property 1 and define  $s$  according to Eq. (15). Then,  $s$  satisfies Eq. (11).
Next, we claim that the score function of any diffusion process satisfies the PDE Eq. (11): (proof in Section A.5)
Lemma A.8. Let  $s$  be the score function of some diffusion process that is defined by Eq. (4). Then,  $s$  satisfies the PDE Eq. (11).
To complete the first part of the proof, denote by  $p(\cdot, t)$  the probability distribution such that  $s(x, t) = \nabla \log p(x, t)$ , whose existence follows from Property 2. We would like to argue that  $\{p(\cdot, t)\}_{t \in (0,1]}$  corresponds the probability density of the diffusion
$$
d x _ {t} = g (t) d B _ {t}. \tag {16}
$$
It suffices to show that for any  $t_0 > 0$ ,  $\{p(\cdot, t)\}_{t \in (t_0, 1]}$  corresponds to the same diffusion. To show the latter, let  $t_0 \in (0, 1)$  and consider the diffusion process according to Eq. (16) with the initial condition that  $x_{t_0} \sim p(\cdot, t_0)$ . Denote its score function by  $s'$  and notice that it satisfies the PDE Eq. (11) and the initial condition  $s'(x, t_0) = \nabla_x \log p(x, t_0) = s(x, t_0)$ , where the first equality follows from the definition of a score function and the second from the construction of  $p(x, t_0)$ . Further, recall that  $s(x, t)$  satisfies the same PDE Eq. (11) by Lemma A.4. Next we will show that  $s = s'$  for all  $t \in [t_0, 1]$ , and this will follow from the following lemma: (proof in Section A.6)
Lemma A.9. Let  $s$  and  $s'$  be two solutions for the PDE (11) on the domain  $\mathbb{R}^d \times [t_0, 1]$  that satisfy the same initial condition at  $t_0$ :  $s(x, t_0) = s'(x, t_0)$  for all  $x$ . Further, assume that for all  $t \in [t_0, 1]$  there exist probability densities  $p(\cdot, t)$  and  $p'(\cdot, t)$  such that  $s(x, t) = \nabla_x \log p(x, t)$  and  $s'(x, t) = \nabla_x \log p'(x, t)$  for all  $x$ . Then,  $s = s'$  on all of  $\mathbb{R}^d \times [t_0, 1]$ .
Then, by uniqueness of the PDE one obtains that  $s = s'$  for all  $t \in [t_0, 1]$ . Hence,  $s$  is the score of a diffusion for all  $t \geq t_0$  and this holds for any  $t_0 > 0$ , hence this holds for any  $t > 0$ . This concludes the proof of the first part of the theorem.
For the second part, let  $s^*$  denote some score function of a diffusion process that satisfies Eq. (4). Assume that for some  $t_0 > 0$  and some open subset  $U \subseteq \mathbb{R}^d$ ,  $s = s^*$ , namely  $s(x,t_0) = s^*(x,t_0)$  for all  $t_0 > 0$  and all  $x \in U$ . First, we would like to argue that if  $s(x,t)$  is the score function of some diffusion process that satisfies Eq. (4), then for any  $t_0 > 0$  it holds that  $s(x,t_0)$  is an analytic function (proof in Section A.7)
Lemma A.10. Let  $x_{t}$  obey the SDE Eq. (4) with the initial condition  $x_0 \sim \mu_0$ . Let  $t > 0$  and let  $s(x,t)$  denote the score function of  $x_{t}$ , namely,  $s(x,t) = \nabla_{x}\log p(x,t)$  where  $p(x,t)$  is the density of  $x_{t}$ . Assume that  $\mu_0$  is a bounded-support distribution. Then,  $s(x,t)$  is an analytic function.
Since both  $s$  and  $s^*$  are scores of diffusion processes, then  $s(x, t_0)$  and  $s^*(x, t_0)$  are analytic functions. Using the fact that  $s = s^*$  on  $U \times \{t_0\}$  and using Lemma A.5 we derive that  $s(x, t_0) = s^*(x, t_0)$  for all  $x$ . Let  $p$  and  $p^*$  denote the densities that correspond to the score functions  $s$  and  $s^*$  and by definition of a score function, we obtain that for all  $x$ ,
$$
\nabla \log p (x, t _ {0}) = s (x, t _ {0}) = s ^ {*} (x, t _ {0}) = \nabla \log p ^ {*} (x, t _ {0}),
$$
which implies, by integration, that
$$
\log p (x, t _ {0}) = \log p ^ {*} (x, t _ {0}) + c
$$
for some constant  $c\in \mathbb{R}$ . However,  $c = 0$ . Indeed,
$$
1 = \int p (x, t _ {0}) d x = \int e ^ {\log p (x, t _ {0})} d x = \int e ^ {\log p ^ {*} (x, t _ {0}) + c} d x = \int p ^ {*} (x, t _ {0}) e ^ {c} d x = e ^ {c},
$$
which implies that  $c = 0$  as required. As a consequence, the following lemma implies that  $p(x,0) = p^{*}(x,0)$  for all  $x$  (proof in Section A.8):
Lemma A.11. Let  $x_{t}$  and  $y_{t}$  be stochastic processes that follow Eq. (4) with initial conditions  $x_0 \sim \mu_0$  and  $y_0 \sim \mu_0'$  and assume that  $\mu_0$  and  $\mu_0'$  are bounded-support. Assume that for some  $t_0 > 0$ ,  $x_{t_0}$  and  $y_{t_0}$  have the same distribution. Then,  $\mu_0 = \mu_0'$ .
Without loss of generality, one can replace 0 with any  $\tilde{t} \in (0, t_0)$ , to obtain that  $p(x, \tilde{t}) = p^*(x, \tilde{t})$  for any  $\tilde{t} \in [0, t_0]$ . Now,  $p(x, t_0)$  is analytic, from Lemma A.5, hence it is continuous. Consequently, Lemma A.6 implies that  $p = p^*$  in  $\mathbb{R}^d \times [t_0, 1]$ . This concludes that  $p = p^*$  in all the domain, which implies that  $s = \nabla \log p = \nabla \log p^* = s^*$ , as required.
A.4 Proof of Lemma A.7
We use Ito's lemma, and in particular Eq. (13), to get a PDE for the function  $h(x_{t}, t)$  where  $x_{t}$  satisfies the stochastic process
$$
d x _ {t} = - g (t) ^ {2} s (x _ {t}, t) d t + g (t) d \overline {{B}} _ {t}.
$$
Ito's formula yields that
$$
d h (x _ {t}, t) = \left(\frac {\partial h}{\partial t} - g (t) ^ {2} J _ {F} s - \frac {g (t) ^ {2}}{2} \triangle h\right) d t + \sigma J _ {h} d \bar {B} _ {t}.
$$
Since  $(h, s)$  satisfies Property 1 and using Lemma 3.1,  $h$  is a reverse martingale which implies that the term that multiplies  $dt$  has to equal zero. In particular, we have that
$$
\frac {\partial h}{\partial t} - g (t) ^ {2} J _ {h} s - \frac {g (t) ^ {2}}{2} \triangle h = 0. \tag {17}
$$
By Eq. (15),
$$
s = \frac {h - x}{\sigma_ {t} ^ {2}}.
$$
$$
h = x + \sigma_ {t} ^ {2} s.
$$
Substituting this in Eq. (17) and using the relation  $d\sigma_t^2 /dt = g(t)^2$  that follows from Eq. (15), one obtains that
$$
\begin{array}{l} 0 = \frac {\partial}{\partial t} \left(x + \sigma_ {t} ^ {2} s\right) - g (t) ^ {2} J _ {x + \sigma_ {t} ^ {2} s} s - \frac {g (t) ^ {2}}{2} \triangle \left(x + \sigma_ {t} ^ {2} s\right) \\ = g (t) ^ {2} s + \sigma_ {t} ^ {2} \frac {\partial s}{\partial t} - g (t) ^ {2} \left(I + \sigma_ {t} ^ {2} J _ {s}\right) s - \frac {g (t) ^ {2} \sigma_ {t} ^ {2}}{2} \triangle s \\ = \sigma_ {t} ^ {2} \frac {\partial s}{\partial t} - g (t) ^ {2} \sigma_ {t} ^ {2} J _ {s} s - \frac {g (t) ^ {2} \sigma_ {t} ^ {2}}{2} \triangle s. \\ \end{array}
$$
Dividing by  $\sigma_t^2$ , we get that
$$
\frac {\partial s}{\partial t} - g (t) ^ {2} J _ {s} s - \frac {g (t) ^ {2}}{2} \triangle s = 0,
$$
which is what we wanted to prove.
A.5 Proof of Lemma A.8
We present as a consequence of the Fokker-Plank equation (Lemma A.3) a PDE for the log density  $\log p$ :
Lemma A.12. Let  $x_{t}$  be defined by
$$
d x _ {t} = \mu (x _ {t}, t) d t + g (t) d B _ {t}.
$$
$$
\frac {\partial \log p}{\partial t} = - \nabla \cdot \mu - \mu \nabla \cdot \log p + \frac {g (t) ^ {2} \| \nabla \log p \| ^ {2}}{2} + \frac {g (t) ^ {2} \triangle \log p}{2}
$$
Proof. We would like to replace the partial derivatives of  $p$  that appear in Lemma A.3 with the partial derivatives of  $\log p$ . Using the formula
$$
\frac {\partial \log p}{\partial t} = \frac {1}{p} \frac {\partial p}{\partial t},
$$
one obtains that
$$
\frac {\partial p}{\partial t} = p \frac {\partial \log p}{\partial t}.
$$
$$
\frac {\partial p}{\partial x _ {i}} = p \frac {\partial \log p}{\partial x _ {i}} \tag {18}
$$
which also implies that
$$
\nabla p = p \nabla \log p, \quad \nabla \cdot p = p \nabla \cdot \log p.
$$
Differentiating Eq. (18) again with respect to  $x_{i}$  and applying Eq. (18) once more, one obtains that
$$
\frac {\partial^ {2} p}{\partial x _ {i} ^ {2}} = \frac {\partial}{\partial x _ {i}} \left(p \frac {\partial \log p}{\partial x _ {i}}\right) = \frac {\partial p}{\partial x _ {i}} \frac {\partial \log p}{\partial x _ {i}} + p \frac {\partial^ {2} \log p}{\partial x _ {i} ^ {2}} = p \left(\left(\frac {\partial \log p}{\partial x _ {i}}\right) ^ {2} + \frac {\partial^ {2} \log p}{\partial x _ {i} ^ {2}}\right).
$$
Summing over  $i$ , one obtains that
$$
\triangle p = p \sum_ {i = 1} ^ {n} \left(\left(\frac {\partial \log p}{\partial x _ {i}}\right) ^ {2} + \frac {\partial^ {2} \log p}{\partial x _ {i} ^ {2}}\right) = p \| \nabla \log p \| ^ {2} + p \triangle \log p. \tag {19}
$$
Substituting the partials derivatives of  $p$  inside the Fokker-Planck equation in Lemma A.3, one obtains that
$$
p \frac {\partial \log p}{\partial t} = - p \nabla \cdot \mu - \mu (p \nabla \cdot \log p) + \frac {g (t) ^ {2}}{2} \left(p \| \nabla \log p \| ^ {2} + p \triangle \log p\right).
$$
Dividing by  $p$ , one gets that
$$
\frac {\partial \log p}{\partial t} = - \nabla \cdot \mu - \mu \nabla \cdot \log p + \frac {g (t) ^ {2} \| \nabla \log p \| ^ {2}}{2} + \frac {g (t) ^ {2} \triangle \log p}{2}.
$$
We are ready to prove Lemma A.8: Substituting  $\mu = 0$  in Lemma A.12, one obtains that
$$
\frac {\partial \log p}{\partial t} = \frac {g (t) ^ {2} \| \nabla \log p \| ^ {2}}{2} + \frac {g (t) ^ {2} \triangle \log p}{2}.
$$
Taking the gradient with respect to  $x$ , one obtains that
$$
\nabla \frac {\partial \log p}{\partial t} = \frac {g (t) ^ {2} \nabla \| \nabla \log p \| ^ {2}}{2} + \frac {g (t) ^ {2} \nabla \triangle \log p}{2}. \tag {20}
$$
Since  $\partial/\partial x_{i}$  commutes with  $\partial/\partial t$ , it holds that
$$
\nabla \frac {\partial \log p}{\partial t} = \frac {\partial}{\partial t} \nabla \log p = \frac {\partial s}{\partial t}, \tag {21}
$$
recalling that by definition  $s = \nabla \log p$ . Further,
$$
\frac {\partial}{\partial x _ {i}} \| \nabla \log p \| ^ {2} = \sum_ {j = 1} ^ {n} \frac {\partial}{\partial x _ {i}} \left(\frac {\partial \log p}{\partial x _ {j}}\right) ^ {2} = 2 \sum_ {j = 1} ^ {n} \frac {\partial^ {2} \log p}{\partial x _ {i} \partial x _ {j}} \frac {\partial \log p}{\partial x _ {j}} = 2 (H _ {\log p} \nabla \log p) _ {i},
$$
where for any function  $f\colon \mathbb{R}^d\to \mathbb{R}$ ,  $H_{f}$  is the Hessian function of  $f$  that is defined by
$$
(H _ {f}) _ {i j} = \frac {\partial^ {2} f}{\partial x _ {i} \partial x _ {j}}
$$
$$
\nabla \| \nabla \log p \| ^ {2} = 2 H _ {\log p} \nabla \log p.
$$
Further, notice that
$$
H _ {f} = J _ {\nabla f},
$$
$$
\nabla \| \nabla \log p \| ^ {2} = 2 J _ {\nabla \log p} \nabla \log p = 2 J _ {s} s. \tag {22}
$$
Lastly, we get that by the commutative property of partial derivatives,
$$
\nabla \triangle \log p = \triangle \nabla \log p = \triangle s. \tag {23}
$$
Substituting Eq. (21), Eq. (22) and Eq. (23) in Eq. (20), one obtains that
$$
\frac {\partial s}{\partial t} = g (t) ^ {2} J _ {s} s + \frac {g (t) ^ {2} \triangle s}{2},
$$
A.6 Proof of Lemma A.9
We will prove that  $p$  and  $p'$  satisfy the same PDE (which is the heat equation). Recall that  $s$  and  $s'$  satisfy
$$
\frac {\partial s}{\partial t} = g (t) ^ {2} \left(J _ {s} s + \frac {1}{2} \triangle s\right) = \frac {g (t) ^ {2}}{2} (\nabla \| s \| ^ {2} + \triangle s)
$$
By substituting  $s = \nabla \log p$ ,
$$
\frac {\partial \nabla \log p}{\partial t} = \frac {g (t) ^ {2}}{2} \left(\nabla \| \nabla \log p \| ^ {2} + \triangle \nabla \log p\right).
$$
By exchanging the order of derivatives, we obtain that
$$
\nabla \frac {\partial \log p}{\partial t} = \nabla \frac {g (t) ^ {2}}{2} \left(\| \nabla \log p \| ^ {2} + \triangle \log p\right).
$$
By integrating, this implies that
$$
\frac {\partial \log p}{\partial t} = \frac {g (t) ^ {2}}{2} \left(\| \nabla \log p \| ^ {2} + \triangle \log p\right) + c (t),
$$
where  $c(t)$  depends only on  $t$ . Eq. (19) shows that
$$
\triangle \log p = \frac {\triangle p}{p} - \| \nabla \log p \| ^ {2}.
$$
By substituting this in the equation above, we obtain that
$$
\frac {\partial \log p}{\partial t} = \frac {g (t) ^ {2}}{2} \frac {\triangle p}{p} + c (t).
$$
By multiplying both sides with  $p$ , we get that
$$
\frac {\partial p}{\partial t} = p \frac {\partial \log p}{\partial t} = \frac {g (t) ^ {2}}{2} \triangle p + c (t). \tag {24}
$$
Since  $p$  is a probability distribution,
$$
\int_ {\mathbb {R} ^ {d}} p (x, t) d x = 1,
$$
$$
\int {\frac {\partial p (x , t)}{\partial t}} d x = \frac {\partial}{\partial t} \int_ {\mathbb {R} ^ {d}} p (x, t) d x = \frac {\partial 1}{\partial t} = 0.
$$
Integrating over Eq. (24) we obtain that
$$
0 = \int \frac {g (t) ^ {2}}{2} \triangle p + c (t) d x = 0 + \int c (t) d x,
$$
where the last equation holds since the integral of a Laplacian of probability density integrates to 0. It follows that  $c(t) = 0$  which implies that
$$
\frac {\partial p}{\partial t} = \frac {g (t) ^ {2}}{2} \triangle p, \tag {25}
$$
and the same PDE holds where  $p'$  replaces  $p$ , and this follows without loss of generality. Further, since  $\log p$  and  $\log p'$  are differentiable, it holds that  $p(\cdot, t)$  and  $p'(\cdot, t)$  are continuous for all fixed  $t$ . This implies that  $p$  and  $p'$  are continuous as functions of  $x$  and  $t$  since they both satisfy the heat equation Eq. (14). Consequently, Lemma A.6 implies that  $p = p'$  on  $\mathbb{R}^d \times [t_0, 1]$ . Finally,  $s = \nabla \log p = \nabla \log p' = s'$ , as required.
A.7 Proof of Lemma A.10
First, recall that since  $x_{t}$  satisfies Eq. (4) with the initial condition  $x_0 \sim \mu_0$ , then  $x_{t} \sim \mu_{0} + \mathcal{N}(0,\sigma_{t}^{2}I)$ , namely,  $x_{t}$  is the addition of a random variable drawn from  $\mu_0$  and an independent Gaussian  $\mathcal{N}(0,\sigma_t^2 I)$ . Therefore, the density of  $x_{t}$ , which we denote by  $p(x,t)$ , equals
$$
p (x, a) = \mathbb {E} _ {a \sim \mu_ {0}} \left[ \frac {1}{\sqrt {2 \pi} \sigma_ {t}} \exp \left(- \frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) \right].
$$
Using the equation
$$
\nabla_ {x} \log f (x) = \frac {\nabla_ {x} f (x)}{f (x)},
$$
we get that
$$
s (x, a) = \nabla_ {x} \log p (x, a) = \frac {\nabla_ {x} p (x , a)}{p (x , a)} = \frac {\mathbb {E} _ {a \sim \mu_ {0}} \left[ \frac {1}{\sqrt {2 \pi} \sigma_ {t}} \frac {x - a}{\sigma_ {t} ^ {2}} \exp \left(- \frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) \right]}{\mathbb {E} _ {a \sim \mu_ {0}} \left[ \frac {1}{\sqrt {2 \pi} \sigma_ {t}} \exp \left(- \frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) \right]} \tag {26}
$$
By using the fact that the Taylor formula for  $e^x$  equals
$$
e ^ {x} = \sum_ {i = 0} ^ {\infty} \frac {e ^ {i}}{i !},
$$
we obtain that the right hand side of Eq. (26) equals
$$
\frac {\mathbb {E} _ {a \sim \mu_ {0}} \left[ \frac {1}{\sqrt {2 \pi} \sigma_ {t}} \frac {x - a}{\sigma_ {t} ^ {2}} \sum_ {i = 0} ^ {\infty} \frac {(- 1) ^ {i}}{i !} \left(\frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) ^ {i} \right]}{\mathbb {E} _ {a \sim \mu_ {0}} \left[ \frac {1}{\sqrt {2 \pi} \sigma_ {t}} \sum_ {i = 0} ^ {\infty} \frac {(- 1) ^ {i}}{i !} \left(\frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) ^ {i} \right]} = \frac {\mathbb {E} _ {a \sim \mu_ {0}} \left[ \frac {x - a}{\sigma_ {t} ^ {2}} \sum_ {i = 0} ^ {\infty} \frac {(- 1) ^ {i}}{i !} \left(\frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) ^ {i} \right]}{\mathbb {E} _ {a \sim \mu_ {0}} (\sum_ {i = 0} ^ {\infty} \frac {(- 1) ^ {i}}{i !} \left(\frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) ^ {i} ]} \tag {27}
$$
We will use the following property of analytic functions: if  $f$  and  $g$  are analytic functions over  $\mathbb{R}^d$  and  $g(x) \neq 0$  for all  $x$  then  $f / g$  is analytic over  $\mathbb{R}^d$ . Since the denominator at the right-hand side of Eq. (27) is nonzero, it suffices to prove that the numerator and the denominator are analytic. We will prove for the denominator and the proof for the numerator is nearly identical. By assumption of this lemma, the support of  $\mu_0$  is bounded, hence there is some  $M > 0$  such that  $\| x \| \leq M$  for any  $x$  in the support. Then,
$$
\left| \frac {(- 1) ^ {i}}{i !} \left(\frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) ^ {i} \right| \leq \frac {1}{i !} \left(\frac {x ^ {2} + a ^ {2}}{\sigma_ {t} ^ {2}}\right) ^ {i} = \frac {M ^ {2 i}}{\sigma_ {t} ^ {2 i} i !}.
$$
This bound is independent on  $a$ , and summing these absolute values of coefficients for  $i \in \mathbb{N}$ , one obtains a convergent series. Hence we can replace the summation and the expectation in the denominator at the right hand side of Eq. (27) to get that it equals
$$
\sum_ {i = 0} ^ {\infty} \frac {(- 1) ^ {i}}{i !} \mathbb {E} _ {a \sim \mu_ {0}} \left[ \left(\frac {\| x - a \| ^ {2}}{2 \sigma_ {t} ^ {2}}\right) ^ {i} \right]. \tag {28}
$$
This is the Taylor series around 0 of the above-described denominator it converges to the value of the denominator at any  $x$ . While this Taylor series is taken around 0, we note the Taylor series around any other point  $x_0 \in \mathbb{R}^n$  converges as well. This can be shown by shifting the coordinate system by a constant vector such that  $x_0$  shifts to 0 and applying the same proof. One deduces that the Taylor series for the denominator around any point  $x_0$  converges on all  $\mathbb{R}^d$ , which implies that the denominator in the right hand side of Eq. (27) is analytic. The numerator is analytic as well by the same argument. Therefore the ratio, which equals  $s(x, t)$ , is analytic as well as required.
A.8 Proof of Lemma A.11
Let  $t > 0$ , denote by  $\mu_t$  and  $\mu_t'$  the distributions of  $x_t$  and  $x_t'$ , respectively, and by  $p(x,t)$  and  $p'(x,t)$  the densities of these variables. Then,  $\mu_t = \mu_0 + \mathcal{N}(0,\sigma_t^2 I)$ , namely,  $\mu_t$  is obtained by adding an independent sample from  $\mu_0$  with an independent  $\mathcal{N}(0,\sigma_t^2 I)$  variable, and similarly for  $\mu_t'$ . Hence, the density  $p(x,t)$  is the convolution of the densities  $p(x,0)$  with the density of a Gaussian  $\mathcal{N}(0,\sigma_t^2 I)$ .
Denote by  $\hat{p}(y,t)$  the Fourier transform of the density  $p(x,t)$  with respect to  $x$  (while keeping  $t$  fixed) and similarly define  $\hat{p}'$  as the Fourier transform of  $p'$ . Denote by  $g$  and by  $\hat{g}$  the density of  $\mathcal{N}(0,\sigma_t^2 I)$  and its Fourier transform, respectively. Denote the convolution of two functions by the operator  $*$ . Then,
$$
p (x, t) = p (x, 0) * g (x), \quad p ^ {\prime} (x, t) = p ^ {\prime} (x, 0) * g (x).
$$
Since the Fourier transform turns convolutions into multiplications, one obtains that
$$
\hat {p} (y, t) = \hat {p} (y, 0) \hat {g} (y), \quad \hat {p} ^ {\prime} (y, t) = \hat {p} ^ {\prime} (y, 0) \hat {g} (y).
$$
Since  $p(x,t) = p'(x,t)$  we obtain that  $\hat{p} (y,t) = \hat{p} '(y,t)$ . Consequently,
$$
\hat {p} (y, 0) \hat {g} (y) = \hat {p} ^ {\prime} (y, 0) \hat {g} (y)
$$
Since the Fourier transform of a Gaussian is nonzero, we can divide by  $\hat{g} (y)$  to get that
$$
\hat {p} (y, 0) = \hat {p} ^ {\prime} (y, 0).
$$
This implies that the Fourier transform of  $p(x,0)$  equals that of  $p^{\prime}(x,0)$  hence  $p(x,0) = p^{\prime}(x,0)$  for all  $x$ , as required.
B Other proofs
B.1 Differentiating the loss function
Denote our parameter space as  $\Theta \subseteq \mathbb{R}^m$ . In order to differentiate  $L_{t,t',x}^{\mathrm{SM}}(\theta)$  with respect to  $\theta \in \Theta$ , we make the following calculations below, and we notice that  $\mathbb{E}_{\theta}$  is used to denote an expectation with respect to the distribution of  $x_{[t',t]}$  according to Eq. (7) with  $s = s_{\theta}$  and the initial condition  $x_{t} = x$ . In other words, the expectation is over  $x_{[t',t]}$  that is taken with respect to the sampler that is parameterized by  $\theta$  with the initial condition  $x_{t} = x$ . We denote by  $p_{\theta}(x_{[t',t]} \mid x_{t} = x)$  the corresponding density of  $x_{[t',t]}$ . For any function  $f = (f_{1},\ldots ,f_{n})\colon \Theta \to \mathbb{R}^{n}$ , denote by  $\nabla_{\theta}f$  the Jacobian matrix of  $f$ , where
$$
(\nabla_ {\theta} f) _ {i, j} = \frac {\partial f _ {i}}{\partial \theta_ {j}}.
$$
For notational consistency, if  $f$  is a single-valued function, namely, if  $n = 1$ , then  $\nabla_{\theta}f$  is a column vector. We begin with the following:
$$
\begin{array}{l} \nabla_ {\theta} \mathbb {E} _ {\theta} \left[ h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) \right] = \nabla_ {\theta} \int_ {\mathbb {R} ^ {d}} h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) p _ {\theta} \left(x _ {[ t ^ {\prime}, t ]} \mid x _ {t} = x\right) d x _ {t ^ {\prime}} \\ = \int_ {\mathbb {R} ^ {d}} \nabla_ {\theta} h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) p _ {\theta} \left(x _ {[ t ^ {\prime}, t ]} \mid x _ {t} = x\right) d x _ {t ^ {\prime}} + \int_ {\mathbb {R} ^ {d}} h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) \nabla_ {\theta} p _ {\theta} \left(x _ {[ t ^ {\prime}, t ]} \mid x _ {t} = x\right) d x _ {t ^ {\prime}} \\ = \mathbb {E} _ {\theta} \left[ \nabla_ {\theta} h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) \right] + \mathbb {E} _ {\theta} \left[ h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) \frac {\nabla_ {\theta} p _ {\theta} \left(x _ {[ t ^ {\prime} , t ]} \mid x _ {t} = x\right)}{p _ {\theta} \left(x _ {[ t ^ {\prime} , t ]} \mid x _ {t} = x\right)} \right] \\ = \mathbb {E} _ {\theta} \left[ \nabla_ {\theta} h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) \right] + \mathbb {E} _ {\theta} \left[ h _ {\theta} \left(x _ {t ^ {\prime}}, t ^ {\prime}\right) \nabla_ {\theta} \log \left(p _ {\theta} \left(x _ {[ t ^ {\prime}, t ]} \mid x _ {t} = x\right)\right) \right] \\ \end{array}
$$
Differentiating the whole loss, we get the following: