SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This appendix is organized as follows. In Section A, we present proofs for all of our propositions. Section A.1 presents a rigorous definition of our forward process using a more specific notation. This is then used in Section A.2.1 to prove the time reversal for our jump diffusions. We also present an intuitive proof of the time reversal using notation from the main text in Section A.2.2. In Section A.3 we prove Proposition 2 using the notation from the main text. We prove Proposition 3 in Section A.4 and we analyse the optimum of our objective directly without using stochastic process theory in Section A.5. In Section B we give more details on our objective and in Section C we detail how we apply diffusion guidance to our model. We give the full details for our experiments in Section D and finally, in Section E, we discuss the broader impacts of our work.
A.1 Notation and Setup
We here introduce a more rigorous notation for defining our trans-dimensional notation that will be used in a rigorous proof for the time-reversal of our jump diffusion. First, while it makes sense from a methodological and experimental point of view to present our setting as a transdimensional one, we slightly change the point of view in order to derive our theoretical results. We extend the space  $\mathbb{R}^d$  to  $\hat{\mathbb{R}}^d = \mathbb{R}^d \cup \{\infty\}$  using the one-point compactification of the space. We refer to [43] for details on this space. The point  $\infty$  will be understood as a mask. For instance, let  $x_1, x_2, x_3 \in \mathbb{R}^d$ . Then  $X = (x_1, x_2, x_3) \in (\hat{\mathbb{R}}^d)^N$  with  $N = 3$  corresponds to a vector for which all components are observed whereas  $X' = (x_1, \infty, x_3) \in (\hat{\mathbb{R}}^d)^N$  corresponds to a vector for which only the components on the first and third dimension are observed. The second dimension is masked in that case. Doing so, we will consider diffusion models on the space  $\mathsf{X} = (\hat{\mathbb{R}}^d)^N$  with  $d, N \in \mathbb{N}$ . In the case of a video diffusion model,  $N$  can be seen as the max number of frames. We will always consider that this space is equipped with its Borelian sigma-field  $\mathcal{X}$  and all probability measures will be defined on  $\mathcal{X}$ .
We denote  $\dim : \mathsf{X} \to \{0,1\}^N$  which is given for any  $X = \{x_{i}\}_{i=1}^{N} \in \mathsf{X}$  by
$$
\dim (X) = \{\delta_ {\mathbb {R} ^ {d}} (x _ {i}) \} _ {i = 1} ^ {N}.
$$
In other words,  $\dim(X)$  is a binary vector identifying the "dimension" of the vector  $X$ , i.e. which frames are observed. Going back to our example  $X = (x_{1}, x_{2}, x_{3}) \in (\hat{\mathbb{R}}^{d})^{N}$  and  $X' = (x_{1}, \infty, x_{3}) \in (\hat{\mathbb{R}}^{d})^{N}$ , we have that  $\dim(X) = \{1, 1, 1\}$  and  $\dim(X') = \{1, 0, 1\}$ . For any vector  $u \in \{0, 1\}^{N}$  we denote  $|u| = \sum_{i=1}^{N} u_{i}$ , i.e. the active dimensions of  $u$  (or equivalently the non-masked frames). For any  $X \in X$  and  $D \in \{0, 1\}^{N}$ , we denote  $X_{D} = \{X_{i}'\}_{i=1}^{N}$  with  $X_{i}' = X_{i}$  if  $D_{i} = 1$  and  $X_{i}' = \infty$  if  $D_{i} = 0$ .
We denote  $\mathrm{C}_b^k(\mathbb{R}^d, \mathbb{R})$  the set of functions which are  $k$  differentiable and bounded. Similarly, we denote  $\mathrm{C}_b^k(\mathbb{R}^d, \mathbb{R})$  the set of functions which are  $k$  differentiable and compactly supported. The set  $\mathrm{C}_0^k(\mathbb{R}^d, \mathbb{R})$  denotes the functions which are  $k$  differentiable and vanish when  $\| x \| \to +\infty$ . We note that  $f \in \mathrm{C}(\hat{\mathbb{R}}^d)$ , if  $f \in \mathrm{C}(\mathbb{R}^d)$  and  $f - f(\infty) \in \mathrm{C}_0(\mathbb{R}^d)$  and that  $f \in \mathrm{C}^k(\hat{\mathbb{R}}^d)$  for any  $k \in \mathbb{N}$  if the restriction of  $f$  to  $\mathbb{R}^d$  is in  $\mathrm{C}^k(\mathbb{R}^d)$  and  $f \in \mathrm{C}(\hat{\mathbb{R}}^d)$ .
A.1.1 Transdimensional infinitesimal generator
To introduce rigorously the transdimensional diffusion model defined in Section 3.1, we will introduce its infinitesimal generator. The infinitesimal generator of a stochastic process can be roughly defined as its "probabilistic derivative". More precisely, assume that a stochastic process  $(\mathbf{X}_t)_{t\geq 0}$  admits a transition semigroup  $(\mathrm{P}_t)_{t\geq 0}$ , i.e. for any  $t\geq 0$ ,  $\mathsf{A}\in \mathcal{X}$  and  $X\in \mathsf{X}$  we have  $\mathbb{P}(\mathbf{X}_t\in \mathsf{A}\mid \mathbf{X}_0 = x) = \mathrm{P}_t(x,\mathsf{A})$ , then the infinitesimal generator is defined as  $\mathcal{A}(f) = \lim_{t\to 0}(\mathrm{P}_t(f) - f) / t$ , for every  $f$  for which this quantity is well-defined.
Here, we start by introducing the infinitesimal generator of interest and give some intuition about its form. Then, we prove a time-reversal formula for this infinitesimal generator.
We consider  $b:\mathbb{R}^d\to \mathbb{R}^d$ $\alpha :\{0,1\}^{NM}\to \mathbb{R}_{+}$  . For any  $f\in \mathrm{C}^2 (\mathsf{X})$  and  $X\in \mathsf{X}$  we define
$$
\begin{array}{l} \mathcal {A} (f) (X) = \sum_ {i = 1} ^ {N} \left\{\langle b \left(X _ {i}\right), \nabla_ {x _ {i}} f (X) \rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X) \right\} \delta_ {\mathbb {R} ^ {d}} \left(X _ {i}\right) \tag {5} \\ - \sum_ {\mathsf {D} _ {1} \subset \mathsf {D} _ {0} ^ {\Delta_ {0}}} \dots \sum_ {\mathsf {D} _ {M} \subset \mathsf {D} _ {M - 1} ^ {\Delta_ {M - 1}}} \alpha (\mathsf {D} _ {0}, \ldots , \mathsf {D} _ {M}) \sum_ {i = 0} ^ {M - 1} (f (X) - f (X _ {\mathsf {D} _ {i + 1}})) \delta_ {\mathsf {D} _ {i}} (\dim (X)), \\ \end{array}
$$
where  $M \in \mathbb{N}$ ,  $\mathsf{D}_0 = \{1\}^N$ ,  $\{\Delta_j\}_{j=0}^{M-1} \in \mathbb{N}^M$  such that  $\sum_{j=0}^{M-1} \Delta_j < N$  and for any  $j \in \{0, \ldots, M-1\}$ ,  $\mathsf{D}_j^{\Delta_j}$  is the subset of  $\{0, 1\}^{1, \ldots, N}$  such that  $\mathsf{D}_{j+1} \in \mathsf{D}_j^{\Delta_j}$  if and only if  $\mathsf{D}_j \cdot \mathsf{D}_{j+1} = \mathsf{D}_{j+1}$ , where  $\cdot$  is the pointwise multiplication operator, and  $|\mathsf{D}_j| = |\mathsf{D}_{j+1}| + \Delta_j$ . The condition  $\mathsf{D}_j \cdot \mathsf{D}_{j+1} = \mathsf{D}_{j+1}$  means that the non-masked dimensions in  $\mathsf{D}_{j+1}$  are also non-masked dimensions in  $\mathsf{D}_j$ . The condition  $|\mathsf{D}_j| = |\mathsf{D}_{j+1}| + \Delta_j$  means that in order to go from  $\mathsf{D}_j$  to  $\mathsf{D}_{j+1}$ , one needs to mask exactly  $\Delta_j$  dimensions.
Therefore, a sequence  $\{\Delta_j\}_{j=0}^{M-1} \in \mathbb{N}^M$  such that  $\sum_{j=0}^{M-1} \Delta_j < N$  can be interpreted as a sequence of drops in dimension. At the core level, we have that  $|\mathsf{D}_M| = N - \sum_{j=0}^{M-1} \Delta_j$ . For instance if  $|\mathsf{D}_M| = 1$ , we have that at the end of the process, only one dimension is considered.
We choose  $\alpha$  such that  $\sum_{\mathsf{D}_1\subset \mathsf{D}_0^{\Delta_0}}\dots \sum_{\mathsf{D}_M\subset \mathsf{D}_{M - 1}^{\Delta_{M - 1}}}\alpha (\mathsf{D}_0,\ldots ,\mathsf{D}_M) = 1.$  Therefore,  $\alpha (\mathsf{D}_0,\ldots ,\mathsf{D}_M)$  corresponds to the probability to choose the dimension path  $\mathsf{D}_0\to \dots \to \mathsf{D}_M$
The part  $X \mapsto \langle b(X_i), \nabla_{x_i} f(X) \rangle + \frac{1}{2} \Delta_{x_i} f(X)$  is more classical and corresponds to the continuous part of the diffusion process. We refer to [44] for a thorough introduction on infinitesimal generators. For simplicity, we omit the schedule coefficients in (5).
A.1.2 Justification of the form of the infinitesimal generator
For any dimension path  $\mathsf{P} = \mathsf{D}_0\to \dots \to \mathsf{D}_M$  (recall that  $\mathsf{D}_0 = \{1\} ^N$ ), we define the jump kernel  $\mathbb{J}^{\mathrm{P}}$  as follows. For any  $x\in \mathsf{X}$ , we have  $\mathbb{J}^{\mathrm{P}}(X,\mathrm{d}Y) = \sum_{i = 0}^{M - 1}\delta_{\mathsf{D}_i}(\dim (X))\delta_{X_{\mathsf{D}_{i + 1}}}(\mathrm{d}Y)$ . This operator corresponds to the deletion operator introduced in Section 3.1. Hence, for any dimension path  $\mathsf{P} = \mathsf{D}_0\to \dots \to \mathsf{D}_M$ , we can define the associated infinitesimal generator: for any  $f\in \mathrm{C}^2 (\mathsf{X})$  and  $X\in \mathsf{X}$  we define
$$
\mathcal {A} ^ {\mathsf {P}} (f) (X) = \sum_ {i = 1} ^ {N} \{\langle b (x _ {i}), \nabla_ {x _ {i}} f (X) \rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X) \} \delta_ {\mathbb {R} ^ {d}} (X _ {i}) + \int_ {\mathsf {X}} (f (Y) - f (X)) \mathbb {J} ^ {\mathsf {P}} (X, \mathrm {d} Y).
$$
We can define the following jump kernel
$$
\mathbb {J} = \sum_ {\mathsf {D} _ {1} \subset \mathsf {D} _ {0} ^ {\Delta_ {0}}} \dots \sum_ {\mathsf {D} _ {M} \subset \mathsf {D} _ {M - 1} ^ {\Delta_ {M - 1}}} \alpha (\mathsf {D} _ {0}, \ldots , \mathsf {D} _ {M}) \mathbb {J} ^ {\mathsf {P}}.
$$
This corresponds to averaging the jump kernel over the different possible dimension paths. We have that for any  $f \in \mathrm{C}^2(\mathsf{X})$  and  $\bar{X} \in \mathsf{X}$
$$
\mathcal {A} (f) (X) = \sum_ {i = 1} ^ {N} \left\{\langle b \left(x _ {i}\right), \nabla_ {x _ {i}} f (X) \rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X) \right\} \delta_ {\mathbb {R} ^ {d}} \left(X _ {i}\right) + \int_ {\mathrm {X}} \left(f (Y) - f (X)\right) \mathbb {J} (X, \mathrm {d} Y). \tag {6}
$$
In other words,  $\mathcal{A} = \sum_{\mathsf{D}_1\subset \mathsf{D}_0^{\Delta_0}}\dots \sum_{\mathsf{D}_M\subset \mathsf{D}_{M - 1}^{\Delta_{M - 1}}}\alpha (\mathsf{D}_0,\ldots ,\mathsf{D}_M)\mathcal{A}^{\mathsf{P}}$
In what follows, we assume that there exists a Markov process  $(\mathbf{X}_t)_{t\geq 0}$  with infinitesimal generator  $\mathcal{A}$ . In order to sample from  $(\mathbf{X}_t)_{t\geq 0}$ , one choice is to first sample the dimension path  $\mathsf{P}$  according to the probability  $\alpha$ . Second sample from the Markov process associated with the infinitesimal generator  $\mathcal{A}^{\mathsf{P}}$ . We can approximately sample from this process using the Lie-Trotter-Kato formula [44, Corollary 6.7, p.33].
Denote  $(\mathrm{P}_t)_{t\geq 0}$  the semigroup associated with  $\mathcal{A}^{\mathrm{P}}$ $(\mathrm{Q}_t)_{t\geq 0}$  the semigroup associated with the continuous part of  $\mathcal{A}^{\mathrm{P}}$  and  $(\mathrm{J}_t)_{t\geq 0}$  the semigroup associated with the jump part of  $\mathcal{A}^{\mathrm{P}}$ . More precisely, we have that,  $(\mathrm{Q}_t)_{t\geq 0}$  is associated with  $\mathcal{A}_{\mathrm{cont}}$  such that for any  $f\in \mathrm{C}^2 (\mathsf{X})$  and  $X\in \mathsf{X}$
$$
\mathcal {A} _ {\mathrm {c o n t}} (f) (X) = \sum_ {i = 1} ^ {N} \{\langle b (X _ {i}), \nabla_ {x _ {i}} f (X) \rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X) \}.
$$
In addition, we have that,  $(\mathrm{Q}_t)_{t\geq 0}$  is associated with  $\mathcal{A}_{\mathrm{jump}}^{\mathrm{P}}$  such that for any  $f\in \mathrm{C}^2 (\mathsf{X})$  and  $X\in \mathsf{X}$
$$
\mathcal {A} _ {\mathrm {j u m p}} ^ {\mathsf {P}} (f) (X) = \int_ {\mathsf {X}} (f (Y) - f (X)) \mathbb {J} ^ {\mathsf {P}} (X, \mathrm {d} Y).
$$
First, note that  $\mathcal{A}_{\mathrm{cont}}$  corresponds to the infinitesimal generator of a classical diffusion on the components which are not set to  $\infty$ . Hence, we can approximately sample from  $(\mathrm{Q}_t)_{t\geq 0}$  by sampling according to the Euler-Maruyama discretization of the associated diffusion, i.e. by setting
$$
\mathbf {X} _ {t} \approx \mathbf {X} _ {0} + t b (\mathbf {X} _ {0}) + \sqrt {t} Z, \tag {7}
$$
where  $Z$  is a Gaussian random variable.
Similarly, in order to sample from  $(\mathrm{J}_t)_{t\geq 0}$ , one should sample from the jump process defined as follows. On the interval  $[0,\tau)$ , we have  $\mathbf{\bar{X}}_t = \mathbf{X}_0$ . At time  $\tau$ , we define  $\mathbf{X}_1\sim \mathbb{J}(\mathbf{X}_0,\cdot)$  and repeat the procedure. In this case  $\tau$  is defined as an exponential random variable with parameter 1. For  $t > 0$  small enough the probability that  $t > \tau$  is of order  $t$ . Therefore, we sample from J, i.e. the deletion kernel, with probability  $t$ . Combining this approximation and (7), we get approximate samplers for  $(\mathrm{Q}_t)_{t\geq 0}$  and  $(\mathrm{J}_t)_{t\geq 0}$ . Under mild assumptions, the Lie-Trotter-Kato formula ensures that for any  $t\geq 0$
$$
\mathrm {P} _ {t} = \lim  _ {n \rightarrow + \infty} \left(\mathrm {Q} _ {t / n} \mathrm {J} _ {t / n}\right) ^ {n}.
$$
This justifies sampling according to Algorithm 1 (in the case of the forward process).
For the proof of Proposition 1, we first provide a rigorous proof using the notation introduced in A.1. We then follow this with a second proof that aims to be more intuitive using the notation used in the main paper.
A.2.1 Time-reversal for the transdimensional infinitesimal generator and Proof of Proposition 1
We are now going to derive the formula for the time-reversal of the transdimensional infinitesimal generator  $\mathcal{A}$ , see (5). This corresponds to a rigorous proof of Proposition 1. We refer to Section A.2.2 for a more intuitive, albeit less-rigorous, proof. We start by introducing the kernel  $\mathbb{K}^{\mathrm{P}}$  given for any dimension path  $\mathsf{D}_0\to \dots \to \mathsf{D}_M$ , for any  $i\in \{0,\ldots ,M - 1\}$ ,  $Y\in \mathsf{D}_{i + 1}$  and  $\mathsf{A}\in \mathcal{X}$  by
$$
\mathbb {K} ^ {\mathbf {P}} (Y, \mathsf {A}) = \sum_ {i = 0} ^ {M - 1} \delta_ {\mathsf {D} _ {i + 1}} (\dim (Y)) \int_ {\mathsf {A} \cap \mathsf {D} _ {i}} \frac {p _ {t} ((X _ {\mathsf {D} _ {i} \setminus \mathsf {D} _ {i + 1}} , Y _ {\mathsf {D} _ {i + 1}}) | \dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i}) \mathbb {P} (\dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i})}{p _ {t} (Y _ {\mathsf {D} _ {i + 1}} | \dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i + 1}) \mathbb {P} (\dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i + 1})} d \mathsf {X} _ {\mathsf {D} _ {i} \setminus \mathsf {D} _ {i + 1}}.
$$
Note that this kernel is the same as the one considered in Proposition 1. It is well-defined under the following assumption.
Assumption 1. For any  $t > 0$  and  $\mathsf{D} \subset \{0,1\}^N$ , we have that  $\mathbf{X}_t$  conditioned to  $\dim(\mathbf{X}_t) = \mathsf{D}$  admits a density w.r.t. the  $|\mathsf{D}|$ -dimensional Lebesgue measure, denoted  $p_t(\cdot |\dim(\mathbf{X}_t) = \mathsf{D})$ .
The following result will be key to establish the time-reversal formula.
Lemma 1. Assume A1. Let  $\mathsf{A},\mathsf{B}\in \mathcal{X}$ . Let  $\mathsf{P}$  be a dimension path  $\mathsf{D}_0\to \dots \to \mathsf{D}_M$  with  $M\in \mathbb{N}$ . Then, we have
$$
\mathbb {E} [ \mathbf {1} _ {\mathsf {A}} (\mathbf {X} _ {t}) \mathbb {J} ^ {\mathsf {P}} (\mathbf {X} _ {t}, \mathsf {B}) ] = \mathbb {E} [ \mathbf {1} _ {\mathsf {B}} (\mathbf {X} _ {t}) \mathbb {K} ^ {\mathsf {P}} (\mathbf {X} _ {t}, \mathsf {A}) ].
$$
Proof. Let  $\mathsf{A}, \mathsf{B} \in \mathcal{X}$ . We have
$$
\begin{array}{l} \mathbb {E} \left[ \mathbf {1} _ {\mathsf {A}} \left(\mathbf {X} _ {t}\right) \mathbb {J} ^ {\mathsf {P}} \left(\mathbf {X} _ {t}, \mathsf {B}\right) \right] = \sum_ {i = 0} ^ {M - 1} \mathbb {E} \left[ \mathbf {1} _ {\mathsf {A}} \left(\mathbf {X} _ {t}\right) \delta_ {\mathsf {D} _ {i}} \left(\dim \left(\mathbf {X} _ {t}\right)\right) \mathbf {1} _ {\mathsf {B}} \left(\left(\mathbf {X} _ {t}\right) _ {\mathsf {D} _ {i + 1}}\right) \right] \\ = \sum_ {i = 0} ^ {M - 1} \int_ {\mathsf {A} \cap \mathsf {D} _ {i}} p _ {t} (X _ {\mathsf {D} _ {i}} | \dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i}) \mathbb {P} (\dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i}) \mathbf {1} _ {\mathsf {B}} (X _ {\mathsf {D} _ {i + 1}}) \mathrm {d} X _ {\mathsf {D} _ {i}} \\ = \sum_ {i = 0} ^ {M - 1} \int_ {\mathsf {A} \cap \mathsf {D} _ {i}} p _ {t} \left(X _ {\mathsf {D} _ {i}} \mid \dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i}\right) \mathbb {P} \left(\dim (\mathbf {X} _ {t}) = \mathsf {D} _ {i}\right) \mathbf {1} _ {\mathsf {B}} \left(X _ {\mathsf {D} _ {i + 1}}\right) \mathrm {d} X _ {\mathsf {D} _ {i + 1}} \mathrm {d} X _ {\mathsf {D} _ {i} \backslash \mathsf {D} _ {i + 1}} \\ = \sum_ {i = 0} ^ {M - 1} \int_ {\mathsf {B} \cap \mathsf {D} _ {i + 1}} \mathbf {1} _ {\mathsf {B}} \left(X _ {\mathsf {D} _ {i + 1}}\right) \\ \times \left(\int_ {\mathbf {A} \cap \mathbf {D} _ {i}} \mathbf {1} _ {\mathbf {A}} \left(X _ {\mathbf {D} _ {i}}\right) p _ {t} \left(X _ {\mathbf {D} _ {i}} \mid \dim (\mathbf {X} _ {t}) = \mathbf {D} _ {i}\right) \mathbb {P} \left(\dim (\mathbf {X} _ {t}) = \mathbf {D} _ {i}\right) \mathrm {d} X _ {\mathbf {D} _ {i} \backslash \mathbf {D} _ {i + 1}}\right) \mathrm {d} X _ {\mathbf {D} _ {i + 1}} \\ = \sum_ {i = 0} ^ {M - 1} \int_ {\mathsf {B} \cap \mathsf {D} _ {i + 1}} \mathbf {1} _ {\mathsf {B}} \left(X _ {\mathsf {D} _ {i + 1}}\right) \\ \times \mathbb {K} ^ {\mathrm {P}} \left(X _ {\mathrm {D} _ {i + 1}}, \mathrm {A}\right) p _ {t} \left(X _ {\mathrm {D} _ {i + 1}} \mid \dim (\mathbf {X} _ {t}) = \mathrm {D} _ {i + 1}\right) \mathbb {P} \left(\dim (\mathbf {X} _ {t}) = \mathrm {D} _ {i + 1}\right) \mathrm {d} X _ {\mathrm {D} _ {i + 1}} \\ = \sum_ {i = 0} ^ {M - 1} \mathbb {E} \left[ \delta_ {\mathsf {D} _ {i + 1}} \left(\dim (\mathbf {X} _ {t})\right) \mathbb {K} ^ {\mathsf {P}} \left(\mathbf {X} _ {t}, \mathsf {A}\right) \mathbf {1} _ {\mathsf {B}} \left(\mathbf {X} _ {t}\right) \right], \\ \end{array}
$$
Lemma 1 shows that  $\mathbb{K}^{\mathrm{P}}$  verifies the flux equation associated with  $\mathbb{J}^{\mathrm{P}}$ . The flux equation is the discrete state-space equivalent of the classical time-reversal formula for continuous state-space. We refer to [45] for a rigorous treatment of time-reversal with jumps under entropic conditions.
We are also going to consider the following assumption which ensures that the integration by part formula is valid.
Assumption 2. For any  $t > 0$  and  $i \in \{1, \dots, N\}$ ,  $\mathbf{X}_t$  admits a smooth density w.r.t. the  $N$ -dimensional Lebesgue measure denoted  $p_t$  and we have that for any  $f, h \in \mathrm{C}_b^2((\mathbb{R}^d)^N)$  for any  $u \in [0, t]$  and  $i \in \{1, \dots, N\}$
$$
\begin{array}{l} \mathbb {E} \left[ \delta_ {\mathbb {R} ^ {d}} \left(\left(\mathbf {X} _ {u}\right) _ {i}\right) \left\langle \nabla_ {x _ {i}} f \left(\mathbf {X} _ {u}\right), \nabla_ {x _ {i}} h \left(\mathbf {X} _ {u}\right) \right\rangle \right] \\ = - \mathbb {E} [ \delta_ {\mathbb {R} ^ {d}} ((\mathbf {X} _ {u}) _ {i}) h (\mathbf {X} _ {u}) (\Delta_ {x _ {i}} f (\mathbf {X} _ {u}) + \langle \nabla_ {x _ {i}} \log p _ {u} (\mathbf {X} _ {u}), \nabla_ {x _ {i}} f (\mathbf {X} _ {u}) \rangle) ]. \\ \end{array}
$$
The second assumption ensures that we can apply the backward Kolmogorov evolution equation.
Assumption 3. For any  $g \in C^2(\mathsf{X})$  and  $t > 0$ , we have that for any  $u \in [0, t]$  and  $X \in \mathsf{X}$ ,  $\partial_u g(u, X) + \mathcal{A}(g)(u, X) = 0$ , where for any  $u \in [0, t]$  and  $X \in \mathsf{X}$ ,  $g(u, X) = \mathbb{E}[g(\mathbf{X}_t) \mid \mathbf{X}_u = X]$ .
We refer to [19] for conditions under A2 and A3 are valid in the setting of diffusion processes.
Proposition 4. Assume A1, A2 and A3. Assume that there exists a Markov process  $(\mathbf{X}_t)_{t\geq 0}$  solution of the martingale problem associated with (6). Let  $T > 0$  and consider  $(\mathbf{Y}_t)_{t\in [0,T]} = (\mathbf{X}_{T - t})_{t\in [0,T]}$ . Then  $(\mathbf{Y}_t)_{t\in [0,T]}$  is solution to the martingale problem associated with  $\mathcal{R}$ , where for any  $f\in \mathrm{C}^2 (\mathsf{X})$ ,  $t\in (0,T)$  and  $x\in \mathsf{X}$  we have
$$
\begin{array}{l} \mathcal {R} (f) (t, X) = \sum_ {i = 1} ^ {N} \left\{- \langle b (X _ {i}) + \nabla_ {x _ {i}} \log p _ {t} (X), \nabla_ {x _ {i}} f (X) \rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X) \right\} \delta_ {\mathbb {R} ^ {d}} (X _ {i}) \\ + \int_ {\mathsf {X}} (f (Y) - f (X)) \mathbb {K} (X, \mathrm {d} Y). \\ \end{array}
$$
Proof. Let  $f, g \in \mathrm{C}^2(\mathsf{X})$ . In what follows, we show that for any  $s, t \in [0, T]$  with  $t \geq s$
$$
\mathbb {E} \left[ \left(f \left(\mathbf {Y} _ {t}\right) - f \left(\mathbf {Y} _ {s}\right)\right) g \left(\mathbf {Y} _ {s}\right) \right] = \mathbb {E} \left[ g \left(\mathbf {Y} _ {s}\right) \int_ {s} ^ {t} \mathcal {R} (f) \left(u, \mathbf {Y} _ {u}\right) \mathrm {d} u \right].
$$
More precisely, we show that for any  $s,t\in [0,T]$  with  $t\geq s$
$$
\mathbb {E} [ (f (\mathbf {X} _ {t}) - f (\mathbf {X} _ {s})) g (\mathbf {X} _ {t}) ] = \mathbb {E} [ - g (\mathbf {X} _ {t}) \int_ {s} ^ {t} \mathcal {R} (f) (u, \mathbf {X} _ {u}) \mathrm {d} u ].
$$
Let  $s, t \in [0, T]$ , with  $t \geq s$ . Next, we denote for any  $u \in [0, t]$  and  $X \in \mathsf{X}$ ,  $g(u, X) = \mathbb{E}[g(\mathbf{X}_t) \mid \mathbf{X}_u = X]$ . Using A3, we have that for any  $u \in [0, t]$  and  $X \in \mathsf{X}$ ,  $\partial_u g(u, X) + \mathcal{A}(g)(u, X) = 0$ , i.e.  $g$  satisfies the backward Kolmogorov equation. For any  $u \in [0, t]$  and  $X \in \mathsf{X}$ , we have
$$
\begin{array}{l} \mathcal {A} (f g) (u, X) = \partial_ {u} g (u, X) f (X) + \sum_ {i = 1} ^ {N} (\langle b (X _ {i}), \nabla_ {x _ {i}} g (u, X) \rangle + \frac {1}{2} \Delta_ {x _ {i}} g (u, X _ {i})) f (X) \delta_ {\mathbb {R} ^ {d}} (X _ {i}) \\ + \sum_ {i = 1} ^ {N} \left(\left\langle b \left(X _ {i}\right), \nabla_ {x _ {i}} f (X) \right\rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X)\right) g (u, X) \delta_ {\mathbb {R} ^ {d}} \left(X _ {i}\right) \\ + \sum_ {i = 1} ^ {N} \delta_ {\mathbb {R} ^ {d}} (X _ {i}) \langle \nabla_ {x _ {i}} f (X), \nabla_ {x _ {i}} g (u, X) \rangle + \mathbb {J} (X, f g) \\ = \partial_ {u} g (u, X) f (X) + \mathcal {A} (g) (u, X) f (X) + \mathbb {J} (X, f g) - \mathbb {J} (X, g) f (X) \\ + \sum_ {i = 1} ^ {N} \left(\left\langle b \left(X _ {i}\right), \nabla_ {x _ {i}} f (X) \right\rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X)\right) g (u, X) \delta_ {\mathbb {R} ^ {d}} \left(X _ {i}\right) \\ + \sum_ {i = 1} ^ {N} \delta_ {\mathbb {R} ^ {d}} \left(X _ {i}\right) \langle \nabla_ {x _ {i}} f (X), \nabla_ {x _ {i}} g (u, X) \rangle \\ = \sum_ {i = 1} ^ {N} \left(\left\langle b \left(X _ {i}\right), \nabla_ {x _ {i}} f (X) \right\rangle + \frac {1}{2} \Delta_ {x _ {i}} f (X)\right) g (u, X) \delta_ {\mathbb {R} ^ {d}} \left(X _ {i}\right) \\ + \sum_ {i = 1} ^ {N} \delta_ {\mathbb {R} ^ {d}} (X _ {i}) \langle \nabla_ {x _ {i}} f (X), \nabla_ {x _ {i}} g (u, X) \rangle + \mathbb {J} (X, f g) - \mathbb {J} (X, g) f (X). \tag {8} \\ \end{array}
$$
Using A2, we have that for any  $u \in [0, t]$  and  $i \in \{1, \dots, N\}$
$$
\begin{array}{l} \mathbb {E} \left[ \delta_ {\mathbb {R} ^ {d}} \left(\left(\mathbf {X} _ {u}\right) _ {i}\right) \left\langle \nabla_ {x _ {i}} f \left(\mathbf {X} _ {u}\right), \nabla_ {x _ {i}} g (u, \mathbf {X} _ {u}) \right\rangle \right] \\ = - \mathbb {E} \left[ \delta_ {\mathbb {R} ^ {d}} \left(\left(\mathbf {X} _ {u}\right) _ {i}\right) g (u, \mathbf {X} _ {u}) \left(\Delta_ {x _ {i}} f (\mathbf {X} _ {u}) + \left\langle \nabla_ {x _ {i}} \log p _ {u} \left(\mathbf {X} _ {u}\right), \nabla_ {x _ {i}} f (\mathbf {X} _ {u}) \right\rangle\right) \right]. \tag {9} \\ \end{array}
$$
In addition, we have that for any  $X \in \mathsf{X}$  and  $u \in [0, t]$ ,  $\mathbb{J}(X, fg) - \mathbb{J}(X, g)f(X) = \int_{\mathsf{X}} g(u, Y)(f(Y) - f(X))\mathbb{J}(X, \mathrm{d}Y)$ . Using Lemma 1, we get
$$
\mathbb {E} [ \mathbb {J} (\mathbf {X} _ {u}, f g) - \mathbb {J} (\mathbf {X} _ {u}, f) g (u, \mathbf {X} _ {u}) ] = - \mathbb {E} [ g (u, \mathbf {X} _ {u}) \mathbb {K} (\mathbf {X} _ {u}, f) ]. \tag {10}
$$
Therefore, using (8), (9) and (10), we have
$$
\mathbb {E} \left[ \mathcal {A} (f g) (u, \mathbf {X} _ {u}) \right] = \mathbb {E} \left[ - \mathcal {R} (f) (u, \mathbf {X} _ {u}) g (u, \mathbf {X} _ {u}) \right].
$$