Latent Abstractions in Generative Diffusion Models

## Abstract

Abstract In this work we study how diffusion-based generative models produce high-dimensional data, such as an image, by implicitly relying on a manifestation of a low-dimensional set of latent abstractions, that guide the generative process. We present a novel theoretical framework that extends Nonlinear Filtering ( \oldtextsc nlf ) , and that offers a unique perspective on \oldtextsc sde -based generative models.
The development of our theory relies on a novel formulation of the joint (state and measurement) dynamics, and an information-theoretic measure of the influence of the system state on the measurement process.
According to our theory, diffusion models can be cast as a system of \oldtextsc sde , describing a non-linear filter in which the evolution of unobservable latent abstractions steers the dynamics of an observable measurement process (corresponding to the generative pathways). In addition, we present an empirical study to validate our theory and previous empirical results on the emergence of latent abstractions at different stages of the generative process.

## 1 Introduction

Generative models have become a cornerstone of modern machine learning, offering powerful methods for synthesizing high-quality data across various domains such as image and video synthesis (Dhariwal & Nichol, ; Ho et al., ; He et al., ), natural language processing (Li et al., ; He et al., ; Gulrajani & Hashimoto, ; Lou et al., ), audio generation (Kong et al., ; Liu et al., ), and molecular structures and general 3D shapes (Trippe et al., ; Hoogeboom et al., ; Luo & Hu, ; Zeng et al., ), to name a few. These models transform an initial distribution, which is simple to sample from, into one that approximates the data distribution. Among these, diffusion-based models designed through the lenses of Stochastic Differential Equations (\oldtextscsdes) (Song et al., ; Ho et al., ; Albergo et al., ) have gained popularity due to their ability to generate realistic and diverse data samples through a series of stochastic transformations.

In such models, the data generation process, as described by a substantial body of empirical research (Chen et al., ; Linhardt et al., ; Tang et al., ), appears to develop according to distinct stages: high-level semantics emerge first, followed by the incorporation of low-level details, culminating in a refinement (denoising) phase. Despite ample evidence, a comprehensive theoretical framework for modeling these dynamics remains underexplored.
Indeed, despite recent work on \oldtextscsde-based generative models (Berner et al., ; Richter & Berner, ; Ye et al., ; Raginsky, ) shed new lights on such models, they fall short of explicitly investigating the emergence of abstract representations in the generative process. We address this gap by establishing a new framework for elucidating how generative models construct and leverage latent abstractions, approached through the paradigm of \oldtextscnlf (Bain & Crisan, ; Van Handel, ; Kutschireiter et al., ).

\oldtextscnlf is used across diverse engineering domains (Bain & Crisan, ), as it provides robust methodologies for the estimation and prediction of a system’s state amidst uncertainty and noise.
\oldtextscnlf enables the inference of dynamic latent variables that define the system state based on observed data, offering a Bayesian interpretation of state evolution and the ability to incorporate stochastic system dynamics.
The problem we consider is the following: an unobservable random variable $X$ is measured through a noisy continuous-time process $Y_{t}$, wherein the influence of $X$ on the noisy process is described by an observation function $H$, with the noise component modeled as a Brownian motion term. The goal is to estimate the a-posteriori measure $\pi_{t}$ of the variable $X$ given the entire historical trajectory of the measurement process $Y_{t}$.

In this work, we establish a connection between \oldtextscsde-based generative models and \oldtextscnlf by observing that they can be interpreted as simulations of \oldtextscnlf dynamics.
In our framework, the latent abstraction, which corresponds to certain real-world properties within the scope of classical nonlinear filtering and remains unaffected in a causal manner by the posterior process $\pi_{t}$, is implicitly simulated and iteratively refined. We explore the connection between latent abstractions and the a-posteriori process, through the concept of filtrations – broadly defined as collections of progressively increasing information sets – and offer a rigorous theory to study the emergence and influence of latent abstractions throughout the data generation process.
Our theoretical contributions unfold as follows.

In § 2 we show how to reformulate classical \oldtextscnlf results such that the measurement process is the only available information, and derive the corresponding dynamics of both the latent abstraction and the measurement process. These results are summarized in Theorem 2 and Theorem 3.

Given the new dynamics, in Theorem 4 we show how to estimate the a-posteriori measure of the \oldtextscnlf model, and present a novel derivation to compute the mutual information between the measurement process and random variables derived from a transformation of the latent abstractions in Theorem 5. Finally, we show in Theorem 6, that the a-posteriori measure is a sufficient statistics for any random variable derived from the latent abstractions, when only having access to the measurement process.

Building on these general results, in § 3 we present a novel perspective on continuous-time score-based diffusion models, which is summarised in Equation 10. We propose to view such generative models as \oldtextscnlf simulators that progress in two stages: first our model updates the a-posteriori measure representing a sufficient statistics of the latent abstractions, second, it uses a projection of the a-posteriori measure to update the measurement process. Such intuitive understanding is the result of several fundamental steps.
In Theorem 7 and Theorem 8, we show that the common view of score-based diffusion models by which they evolve according to forward (noising) and backward (generative) dynamics is compatible with the \oldtextscnlf formulation, in which there is no need to distinguish between such phases. In other words, the \oldtextscnlf perspective of Equation 10 is a valid generative model.
In Appendix H, we provide additional results (see Lemma 1), focusing on the specific case of linear diffusion models, which are the most popular instance of score-based generative models in use today.
In § 4, we summarize the main intuitions behind our \oldtextscnlf framework.

Our results explain, by means of a theoretically sound framework, the emergence of latent abstractions that has been observed by a large body of empirical work (Bisk et al., ; Bender & Koller, ; Li et al., ; Park et al., ; Kwon et al., ; Chen et al., ; Linhardt et al., ; Tang et al., ; Xiang et al., ; Haas et al., ). The closest research to our findings is discussed in (Sclocchi et al., ), albeit from a different mathematical perspective.
To root our theoretical results in additional empirical evidence, we conclude our work in § 5 with a series of experiments on score-based generative models (Song et al., ), where we 1) validate existing probing techniques to measure the emergence of latent abstractions, 2) compute the mutual information as derived in our framework, and show that it is a suitable approach to measure the relation between the generative process and latent abstractions, 3) introduce a new measurement protocol to further confirm the connections between our theory, and how practical diffusion-based generative models operate.

## 2 Nonlinear Filtering

Consider two random variables $Y_{t}$ and $X$, corresponding to a stochastic measurement process ($Y_{t}$) of some underlying latent abstraction ($X$). We construct our universe sample space $\Omega$ as the combination of the space of continuous functions in the interval $[0,T]$ ($T\in\mathbb{R}^{+}$) and of a complete separable metric space $\mathcal{S}$, i.e., $\Omega=\mathcal{C}([0,T],\mathbb{R}^{N})\times\mathcal{S}$. On this space, we consider the joint canonical process $Z_{t}(\omega)=[Y_{t},X]=[\omega_{t}^{y},\omega^{x}]$ for all $\omega\in\Omega$, with $\omega=[\omega^{y},\omega^{x}]$. In this work we indicate with $\sigma(\cdot)$ sigma-algebras. Consider the growing filtration naturally induced by the canonical process $\mathcal{F}^{Y,X}_{t}=\sigma(Y_{0\leq s\leq t},X)$ (a short-hand for $\sigma(\sigma(Y_{0\leq s\leq t})\cup\sigma(X))$), and define $\mathcal{F}=\mathcal{F}^{Y,X}_{T}$.We build the probability triplet $(\Omega,\mathcal{F},\mathrm{P})$, where the probability measure $\mathrm{P}$ is selected such that the process $\{Z_{0\leq t\leq T},\mathcal{F}^{Y,X}_{0\leq t\leq T}\}$ has the following \oldtextscsde representation

$$ $Y_{t}=Y_{0}+\int_{0}^{t}H(Y_{s},X,s)\differential s+W_{t},$ (1) $$

where $\{W_{0\leq t\leq T},\mathcal{F}^{Y,X}_{0\leq t\leq T}\}$ is a Brownian motion with initial value 0
and $H:\Omega\times[0,T]\rightarrow\mathbb{R}^{N}$
is an observation process.
All standard technical assumptions are available in Appendix A.

Next, we provide the necessary background on \oldtextscnlf, to pave the way for understanding its connection with the generative models of interest.
The most important building block of the \oldtextscnlf literature is represented by the conditional probability measure $\mathrm{P}[X\in A\,|\,\mathcal{F}^{Y}_{t}]$ (notice the reduced filtration $\mathcal{F}^{Y}_{t}\subset\mathcal{F}^{Y,X}_{t}$), which summarizes, a-posteriori, the distribution of $X$ given observations of the measurement process until time $t$, that is, $Y_{0\leq s\leq t}$.

###### Theorem 1 .

[Thm 2.1 (Bain & Crisan, )]
Consider the probability triplet $(\Omega,\mathcal{F},\mathrm{P})$, the metric space $\mathcal{S}$ and its Borel sigma-algebra $\mathcal{B}(\mathcal{S})$.
There exists a (probability measure valued $\mathcal{P}(\mathcal{S})$) process $\{{{\pi}}_{0\leq t\leq T},\mathcal{F}^{Y}_{0\leq t\leq T}\}$, with a progressively measurable modification, such that for all $A\in\mathcal{B}(\mathcal{S})$, the conditional probability measure $\mathrm{P}[X\in A\,|\,\mathcal{F}^{Y}_{t}]$ is well defined and is equal to ${{\pi}_{t}}(A)$.

The conditional probability measure is extremely important, as the fundamental goal of nonlinear filtering is the solution of the following problem. Here, we introduce the quantity $\phi$, which is a random variable derived from the latent abstractions $X$.

###### Problem 1 .

For any fixed $\phi:\mathcal{S}\to\mathbb{R}$ bounded and measurable, given knowledge of the measurement process $Y_{0\leq s\leq t}$, compute $\mathbb{E}_{\mathrm{P}}[\phi(X)\,|\,\mathcal{F}^{Y}_{t}]$. This amounts to computing

$$ $\langle{{\pi}_{t}},\phi\rangle=\int_{\mathcal{S}}\phi(x)\differential{{\pi}_{t }}(x).$ (2) $$

In simple terms, 1 involves studying the existence of the a-posteriori measure and the implementation of efficient algorithms for its update, using the flowing stream of incoming information $Y_{t}$. We first focus our attention on the existence of an analytic expression for the value of the a-posteriori expected measure ${{\pi}_{t}}$.
Then, we quantify the interaction dynamics between observable measurements and $\phi$, through the lenses of mutual information $\mathcal{I}(Y_{0\leq s\leq t};\phi)$, which is an extension of the problems considered in (Newton, ; Duncan, ; ; Mitter & Newton, ).

###### Theorem 1 .

###### Problem 1 .

### 2.1 Technical preliminaries

We set the stage of our work by revisiting the measurement process $Y_{t}$, and express it in a way that does not require access to unobservable information.
Indeed, while $Y_{t}$ is naturally adapted w.r.t. its own filtration $\mathcal{F}^{Y}_{t}$, and consequently to any other growing filtration $\mathcal{R}_{t}$ such $\mathcal{F}^{Y,X}_{t}\supseteq\mathcal{R}_{t}\supseteq\mathcal{F}^{Y}_{t}$, the representation in Equation 1 is in general not adapted, letting aside degenerate cases.

Let’s consider the family of growing filtrations $\mathcal{R}_{t}=\sigma(\mathcal{R}_{0}\cup\sigma(Y_{0\leq s\leq t}-Y_{0}))$, where $\sigma(Y_{0})\subseteq\mathcal{R}_{0}\subseteq\sigma(X,Y_{0})$. Intuitively $\mathcal{R}_{0}$ allows to modulate between the two extreme cases of knowing only the initial conditions of the \oldtextscsde, that is $Y_{0}$, to the case of complete knowledge of the whole latent abstraction $X$, and anything in between. As shown hereafter, the original process $Y_{t}$ associated to the space $(\Omega,\mathcal{F},\mathrm{P})$ which solves Equation 1, also solves Equation 4, that is adapted on the reduced filtration $\mathcal{R}_{t}$. This allows us to reason about the partial observation of the latent abstraction ($\mathcal{R}_{0}$ vs $\sigma(X,Y_{0})$), without incurring in the problem of the measurement process $Y_{t}$ being statistically dependent of the whole latent abstraction $X$.

Armed with such representation, we study under which change of measure the process $Y_{t}-Y_{0}$ behaves as a Brownian motion (Theorem 3). This serves the purpose of simplifying the calculation of the expected value of $\phi$ given $Y_{t}$, as described in 1. Indeed, if $Y_{t}-Y_{0}$ is a Brownian motion independent of $\phi$, its knowledge does not influence our best guess for $\phi$, i.e. the conditional expected value. Moreover, our alternative representation is instrumental for the efficient and simple computation of the mutual information $\mathcal{I}(Y_{0\leq s\leq t};\phi)$, where the different measures involved in the Radon-Nikodym derivatives will be compared against the same reference Brownian measures.

The first step to define our representation is provided by the following

###### Theorem 2 .

[Proof].
Consider the the probability triplet $(\Omega,\mathcal{F},\mathrm{P})$, the process in Equation 1 defined on it, and the growing filtration $\mathcal{R}_{t}=\sigma(\mathcal{R}_{0}\cup\sigma(Y_{0\leq s\leq t}-Y_{0}))$. Define a new stochastic process

$$ ${W}^{\mathcal{R}}_{t}\stackrel{{\scriptstyle\text{def}}}{{=}}Y_{t}-Y_{0}-\int_ {0}^{t}\mathbb{E}_{\mathrm{P}}(H(Y_{s},X,s)\,|\,\mathcal{R}_{s})\differential s.$ (3) $$

Then, $\{{W}^{\mathcal{R}}_{0\leq t\leq T},\mathcal{R}_{0\leq t\leq T}\}$ is a Brownian motion.
Notice that if $\mathcal{R}_{t}=\mathcal{F}^{Y,X}_{t}$, then ${W}^{\mathcal{R}}_{t}=W_{t}$.

Following Theorem 2, the process $\{Y_{0\leq t\leq T},\mathcal{R}_{0\leq t\leq T}\}$ has \oldtextscsde representation

$$ $Y_{t}=Y_{0}+\int_{0}^{t}\mathbb{E}_{\mathrm{P}}(H(Y_{s},X,s)\,|\,\mathcal{R}_{ s})\differential s+{W}^{\mathcal{R}}_{t}.$ (4) $$

Next, we derive the change of measure necessary for the process $\tilde{W}_{t}\stackrel{{\scriptstyle\text{def}}}{{=}}Y_{t}-Y_{0}$ to be a Brownian motion w.r.t to the filtration $\mathcal{R}_{t}$. To do this, we apply the Girsanov theorem (Øksendal, ) to $\tilde{W}_{t}$ which, in general, admits a $\mathcal{R}$ – adapted representation $\int_{0}^{t}\mathbb{E}_{\mathrm{P}}(H(Y_{s},X,s)\,|\,\mathcal{R}_{s})
\differential s+{W}^{\mathcal{R}}_{t}$.

###### Theorem 3 .

[Proof].
Define the new probability space $(\Omega,\mathcal{R}_{T},{\mathrm{Q}^{\mathcal{R}}})$ via the measure ${\mathrm{Q}^{\mathcal{R}}}(A)=\mathbb{E}_{\mathrm{P}}\left[\mathbf{1}(A)({\psi
^{\mathcal{R}}_{T}})^{-1}\right]$, for $A\in\mathcal{R}_{T}$, where

$$ $\psi^{\mathcal{R}}_{t}\stackrel{{\scriptstyle\text{def}}}{{=}}\exp(\int_{0}^{t }\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\,\mathcal{R}_{s}]\differential Y_{s}- \frac{1}{2}\int_{0}^{t}\norm{\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\,\mathcal {R}_{s}]}^{2}\differential s),$ (5) $$

and

$$ ${\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}=\mathbb{E}_{\mathrm{P}}\left [\mathbf{1}(A)\mathbb{E}_{\mathrm{P}}[({\psi^{\mathcal{R}}_{T}})^{-1}\,|\, \mathcal{R}_{t}]\right]=\mathbb{E}_{\mathrm{P}}\left[\mathbf{1}(A)({\psi^{ \mathcal{R}}_{t}})^{-1}\right].$ $$

Then, the stochastic process $\{\tilde{W}_{0\leq t\leq T},\mathcal{R}_{0\leq t\leq T}\}$ is a Brownian motion on the space $(\Omega,\mathcal{R}_{T},{\mathrm{Q}^{\mathcal{R}}})$.

A direct consequence of Theorem 3 is that the process $\tilde{W}_{t}$ is independent of any $\mathcal{R}_{0}$ measurable random variable under the measure ${\mathrm{Q}^{\mathcal{R}}}$. Moreover, it holds that for all $\mathcal{R}^{\prime}_{t}\subseteq\mathcal{R}_{t}$,
${\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}^{\prime}_{t}}=\mathrm{Q}^{
\mathcal{R}^{\prime}}\,|\,_{\mathcal{R}^{\prime}_{t}}$.

###### Theorem 2 .

###### Theorem 3 .

### 2.2 A-Posteriori Measure and Mutual Information

As we did in § 2 for the process $\pi_{t}$, here we introduce a new process $\pi^{\mathcal{R}}_{t}$ which represents the conditional law of $X$ given the filtration $\mathcal{R}_{t}=\sigma(\mathcal{R}_{0}\cup\sigma(Y_{0\leq s\leq t}-Y_{0}))$. More precisely, for all $A\in\mathcal{B}(\mathcal{S})$, the conditional probability measure $\mathrm{P}[X\in A\,|\,\mathcal{R}_{t}]$ is well defined and is equal to $\pi^{\mathcal{R}}_{t}(A)$. Moreover, for any $\phi:\mathcal{S}\to\mathbb{R}$ bounded and measurable, $\mathbb{E}_{\mathrm{P}}[\phi(X)\,|\,\mathcal{R}_{t}]=\langle\pi^{\mathcal{R}}_
{t},\phi\rangle$. Notice that if $\mathcal{R}=\mathcal{F}^{Y}$ then $\pi^{\mathcal{R}}$ reduces to $\pi$.

Armed with Theorem 3, we are ready to derive the expression for the a-posteriori measure $\pi^{\mathcal{R}}_{t}$ and the mutual information between observable measurements and the unavailable information about the latent abstractions, that materialize in the random variable $\phi$.

###### Theorem 4 .

[Proof].
The measure-valued process $\pi^{\mathcal{R}}_{t}$ solves in weak sense (see Appendix D for a precise definition), the following \oldtextscsde

$$ $\displaystyle\pi^{\mathcal{R}}_{t}=\pi^{\mathcal{R}}_{0}+\int_{0}^{t}\pi^{ \mathcal{R}}_{s}\left(H(Y_{s},\cdot,s)-\langle\pi^{\mathcal{R}}_{s},H(Y_{s}, \cdot,s)\rangle\right)\left(\differential Y_{s}-\langle\pi^{\mathcal{R}}_{s},H (Y_{s},\cdot,s)\rangle\differential s\right),$ (6) $$

where the initial condition $\pi_{0}$ satisfies $\pi^{\mathcal{R}}_{0}(A)=\mathrm{P}[X\in A\,|\,\mathcal{R}_{0}]$ for all $A\in\mathcal{B}(\mathcal{S})$.

When $\mathcal{R}=\mathcal{F}^{Y}$, Equation 6 is the well-know Kushner-Stratonovitch (or Fujisaki-Kallianpur-Kunita) equation (see e.g. Bain & Crisan ()).
A proof for uniqueness of the solution of Equation 6 can be approached by considering the strategies in (Fotsa-Mbogne & Pardoux, ), but is outside the scope of this work. The (recursive) expression in Equation 6 is particularly useful for engineering purposes since, in general, it is usually not known in which variables $\phi(X)$, representing latent abstractions, we could be interested in. Keeping track of the whole distribution $\pi^{\mathcal{R}}_{t}$ at time $t$ is the most cost-effective solution, as we will show later.

Our next goal is to quantify the interaction dynamics between observable measurements and latent abstractions that materialize through the variable $\phi(X)$ (from now on we write only $\phi$ for the sake of brevity): in Theorem 5 we derive the mutual information $\mathcal{I}(Y_{0\leq s\leq t};\phi)$.

###### Theorem 5 .

[Proof]
The mutual information between observable measurements $Y_{0\leq s\leq t}$ and $\phi$ is defined as:

$$ $\mathcal{I}(Y_{0\leq s\leq t};\phi)\stackrel{{\scriptstyle\text{def}}}{{=}} \int\log\frac{\differential\mathrm{P}_{\#Y_{0\leq s\leq t},\phi}}{ \differential\mathrm{P}_{\#Y_{0\leq s\leq t}}\differential\mathrm{P}_{\#\phi}} \differential\mathrm{P}_{\#Y_{0\leq s\leq t},\phi}.$ (7) $$

It holds that such quantity is equal to $\mathbb{E}_{\mathrm{P}}\left[\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{
R}_{t}}}{\differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\differential
\mathrm{P}\,|\,_{\sigma(\phi)}}\right]$, which can be simplified as follows:

$$ $\mathcal{I}(Y_{0};\phi)+\frac{1}{2}\mathbb{E}_{\mathrm{P}}\left[\int_{0}^{t} \norm{\mathbb{E}_{\mathrm{P}}[H(X,Y_{s},s)\,|\,\mathcal{F}^{Y}_{s}]-\mathbb{E} _{\mathrm{P}}[H(X,Y_{s},s)\,|\,\mathcal{R}_{s}]}^{2}\differential s\right].$ (8) $$

The mutual information computed by Equation 8 is composed by two elements: first, the mutual information between the initial measurements $Y_{0}$ and $\phi$, which is typically zero by construction. The second term quantifies how much the best prediction of the observation function $H$ is influenced by the extra knowledge of $\phi$, in addition to the measurement history $Y_{0\leq s\leq t}$.
By adhering to the premise that the conditional expectation of a stochastic variable constitutes the optimal estimator given the conditioning information, the integral on the r.h.s quantifies the expected square difference between predictions, having access to measurements only ($\mathbb{E}_{\mathrm{P}}[\cdot\,|\,\mathcal{F}^{Y}_{t}]$) and those incorporating additional information ($\mathbb{E}_{\mathrm{P}}[\cdot\,|\,\mathcal{R}_{t}]$).

Even though a precise characterization for general observation functions and and variables $\phi$ is typically out of reach, a qualitative analysis is possible. First, the mutual information between $\phi$ and the measurements depends on i) how much the amplitude of $H$ is impacted by knowledge of $\phi$ and ii) the number of elements of $H$ which are impacted (informally, how much localized vs global is the impact of $\phi$). Second, it is possible to define a hierarchical interpretation about the emergence of the various latent factors: a variable with a local impact can “appear”, in an information theoretic sense, only if the impact of other global variables is resolved, otherwise the remaining uncertainty of the global variables makes knowledge of the local variable irrelevant. In classical diffusion models, this is empirically known (Chen et al., ; Linhardt et al., ; Tang et al., ), and corresponds to the phenomenon where semantics emerges before details (global vs local details in our language).

Now, consider any $\mathcal{F}^{Y}_{t}$ measurable random variable $\tilde{Y}_{t}$, defined as a mapping to a generic measurable space $(\Psi,\mathcal{B}(\Psi))$, which means it can also be seen as a process.
The data processing inequality states that the mutual information between such $\tilde{Y}$ and $\phi$ will be smaller than the mutual information between the original measurement process and $\phi$. However, it can be shown that all the relevant information about the random variable $\phi$ contained in $\mathcal{F}^{Y}_{t}$ is equivalently contained in the filtering process at time instant $t$, that is $\pi_{t}$. This is not trivial, since $\pi_{t}$ is a $\mathcal{F}^{Y}_{t}$-measurable quantity, i.e., $\sigma(\pi_{t})\subset\mathcal{F}^{Y}_{t}$. In other words, we show that $\pi_{t}$ is a sufficient statistic for any $\sigma(X)$ measurable random variable when starting from the measurement process.

###### Theorem 6 .

[Proof]
For any $\mathcal{F}^{Y}_{t}$ measurable random variable $\tilde{Y}_{t}:\Omega\rightarrow\Psi$, the following inequality holds:

$$ $\mathcal{I}(\tilde{Y};\phi)\leq\mathcal{I}(Y_{0\leq s\leq t};\phi).$ (9) $$

For a given $t\geq 0$, the measurement process $Y_{0\leq s\leq t}$ and $X$ are conditionally-independent given $\pi_{t}$. This implies that $\mathrm{P}(A\,|\,\sigma(\pi_{t}))=\mathrm{P}(A\,|\,\mathcal{F}^{Y}_{t}),\quad
\forall A\in\sigma(X)$. Then
$\mathcal{I}(Y_{0\leq s\leq t};\phi)=\mathcal{I}(\pi_{t};\phi)$ (i.e. Equation 9 is attained with equality).

While $\pi_{t}$ contains all the relevant information about $\phi$, the same cannot be said about the conditional expectation, i.e. the particular case $\tilde{Y}=\langle\pi_{t},\phi\rangle$. Indeed, from Equation 2, $\langle\pi_{t},\phi\rangle$ is obtained as a transformation of $\pi_{t}$ and thus can be interpreted as a $\mathcal{F}^{Y}_{t}$ measurable quantity subject to the constraint of Equation 9.
As a particular case, the quantity $\langle{{\pi}_{t}},H\rangle$, of central importance in the construction of generative models § 3, carries in general less information about $\phi$ than the un-projected $\pi_{t}$.

###### Theorem 4 .

###### Theorem 5 .

###### Theorem 6 .

## 3 Generative Modelling

We are interested in generative models for a given $\sigma(X)$-measurable random variable $V$.
An intuitive illustration of how data generation works according to our framework is as follows. Consider, for example, the image domain, and the availability of a rendering engine that takes as an input a computer program describing a scene (coordinates of objects, textures, light sources, auxiliary labels, etc …) and that produces an output image of the scene. In a similar vein, a generative model learns how to use latent variables (which are not explicitly provided in input, but rather implicitly learned through training) to generate an image.
For such model to work, one valid strategy is to consider an \oldtextscsde in the form of Equation 1 where the following holds(^1^11 From a strict technical point of view, 1 might be incompatible with other assumptions in Appendix A, or proving compatibility could require particular effort. Such details are discussed in Appendix G.).

###### Assumption 1 .

The stochastic process $Y_{t}$ satisfies $Y_{T}=V,\quad\mathrm{P}-a.s.$

Then, we could numerically simulate the dynamics of Equation 1 until time $T$. Indeed, starting from initial conditions $Y_{0}$, we could obtain $Y_{T}$ that, under 1, is precisely $V$. Unfortunately, such a simple idea requires explicit access to $X$, as it is evident from Equation 1. In mathematical terms, Equation 1 is adapted to the filtration $\mathcal{F}^{Y,X}_{t}$. However, we have shown how to reduce the available information to account only for historical values of $Y_{t}$. Then, we can combine the result in Theorem 4 with Theorem 2 and re-interpret Equation 4, which is a valid generative model, as

$$ $\begin{cases}\pi_{t}=\pi_{0}+\int_{0}^{t}\pi_{s}\left(H-\langle{{\pi}_{s}},H \rangle\right)\left(\differential Y_{s}-\langle{{\pi}_{s}},H\rangle \differential s\right),\\ Y_{t}=Y_{0}+\int_{0}^{t}\langle{{\pi}_{s}},H\rangle\differential s+{W}^{ \mathcal{F}^{Y}}_{t},\end{cases}$ (10) $$

where $H$ denotes $H(Y_{s},\cdot,s)$.
Explicit simulation of Equation 10 only requires knowledge of the whole history of the measurement process: provided 1 holds, it allows generation of a sample of the random variable $V$.

Although the discussion in this work includes a large class of observation functions, we focus on the particular case of generative diffusion models (Song et al., ). Typically, such models are presented through the lenses of a forward noising process and backward (in time) \oldtextscsdes, following the intuition of Anderson (). Next, according to the framework we introduce in this work, we reinterpret such models under the perspective of enlargement of filtrations.

Consider the reversed process $\hat{Y}_{t}\stackrel{{\scriptstyle\text{def}}}{{=}}Y_{T-t}$ defined on $(\Omega,\mathcal{F},\mathrm{P})$ and the corresponding filtration $\mathcal{F}_{t}^{\hat{Y}}\stackrel{{\scriptstyle\text{def}}}{{=}}\sigma(\hat{Y
}_{0\leq s\leq t})$. The measure $\mathrm{P}$ is selected such that the process $\hat{Y}_{t}$ has $\mathcal{F}_{t}^{\hat{Y}}$–adapted expression

$$ $\hat{Y}_{t}=V+\int\limits_{0}^{t}F(\hat{Y}_{s},s)\differential s+\hat{W}_{t},$ (11) $$

where $\{\hat{W}_{t},\mathcal{F}_{t}^{\hat{Y}}\}$ is a Brownian motion. Then, 1 is valid since $Y_{T}=\hat{Y}_{0}=V$.
Note that Equation 11, albeit with a different notation, is reminiscent of the forward \oldtextscsde that is typically used as the starting point to illustrate score-based generative models (Song et al., ). In particular, $F(\cdot)$ corresponds to the drift term of such a diffusion \oldtextscsde.

Equation 11 is equivalent to $Y_{t}=V+\int\limits_{t}^{T}F(Y_{s},T-s)\differential s+\hat{W}_{T-t}$, which is an expression for the process $Y_{t}$, which is adapted to $\mathcal{F}^{\hat{Y}}$. This constitutes the first step to derive an equivalent backward (generative) process according to the traditional framework of score-based diffusion models. Note that such an equivalent representation is not useful for simulation purposes: the goals of the next steps is to transform it such that it is adapted to $\mathcal{F}^{Y}$. Indeed, using simple algebra, it holds that

$$ $Y_{t}=Y_{0}-\int\limits_{0}^{t}F(Y_{s},T-s)\differential s+\left(-Y_{0}+V+\int \limits_{0}^{T}F(Y_{s},T-s)\differential s+\hat{W}_{T-t}\right),$ $$

where the last term in the parentheses is equal to $-\hat{W}_{T}+\hat{W}_{T-t}$.

Note that $\mathcal{F}_{t}^{Y}=\sigma(\hat{Y}_{T-t\leq s\leq T})$. Since $\sigma(\hat{Y}_{T-t\leq s\leq T})=\sigma(\hat{W}_{T-t\leq s\leq T})\cup\sigma(
\hat{Y}_{T-t})$, we can apply the result in (Pardoux, ) (Thm 2.2) to claim the following: $-\hat{W}_{T}+\hat{W}_{T-t}-\int_{0}^{t}\nabla\log\hat{p}(Y_{s},T-s)\differential
s$ is a Brownian motion adapted to $\mathcal{F}^{Y}_{t}$, where this time $\mathrm{P}(\hat{Y}_{t}\in\differential y)=\hat{p}(y,t)\differential y$. Then (Pardoux, )

###### Theorem 7 .

Consider the stochastic process $Y_{t}$ which solves Equation 11. The same stochastic process also admits a $\mathcal{F}^{Y}_{t}$–adapted representation

$$ $Y_{t}=Y_{0}+\int_{0}^{t}\underbrace{-F(Y_{s},T-s)+\nabla\log\hat{p}(Y_{s},T-s) }_{\text{In \lx@cref{creftypecap~refnum}{theo:connection},we call this }F^{ \prime}(Y_{s},s)}\differential s+W_{t}.$ (12) $$

Equation 12 corresponds to the backward diffusion process from (Song et al., ) and, because it is adapted to the filtration $\mathcal{F}^{Y}$, it represents a valid, and easy to simulate, measurement process.

By now, it is clear how to go from an $\mathcal{F}^{Y,X}$–adapted filtration to a $\mathcal{F}^{Y}$–adapted one. We also showed that a $\mathcal{F}^{Y}$–adapted filtration can be linked to the reverse, $\mathcal{F}^{\hat{Y}}$–adapted process induced by a forward diffusion \oldtextscsde. What remains to be discussed is the connection that exists between the $\mathcal{F}^{Y}$–adapted filtration, and its enlarged version $\mathcal{F}^{Y,X}$. In other words, we have shown that a forward, diffusion \oldtextscsde admits a backward process which is compatible with our generative model that simulates a \oldtextscnlf process having access only to measurements, but we need to make sure that such process admits a formulation that is compatible the standard \oldtextscnlf framework in which latent abstractions are available.

To do this, we can leverage existing results about Markovian bridges (Rogers & Williams, ; Ye et al., ) (and further work (Aksamit et al., ; Ouwehand, ; Grigorian & Jarrow, ; Çetin & Danilova, ) on filtration enlargement). This requires assumptions about the existence and well-behavedness of densities $p(y,t)$ of the \oldtextscsde process, defined by the logarithm of the Radon-Nikodym derivative of the instantaneous measure $\mathrm{P}(Y_{t}\in\differential y)$ w.r.t. the Lebesgue measure in $\mathbb{R}^{N}$, $\mathrm{P}(Y_{t}\in\differential y)=p(y,t)\differential y$(^2^22Similarly to what discussed in footnote 1, the analysis of the existence of the process adapted to $\mathcal{F}^{Y}_{t}$ is considered in the time interval $[0,T)$ (Haussmann & Pardoux, ). See also Appendix G.).

###### Theorem 8 .

Suppose that on $(\Omega,\mathcal{F},\mathrm{P})$ the Markov stochastic process $Y_{t}$ satisfies

$$ $Y_{t}=Y_{0}+\int_{0}^{t}F^{\prime}(Y_{s},s)\differential s+W_{t},$ (13) $$

where $\{W_{0\leq t\leq T},\mathcal{F}^{Y}_{0\leq t\leq T}\}$ is a Brownian motion and $F$ satisfies the requirements for existence and well definition of the stochastic integral (Shreve, ). Moreover, let 1 hold. Then, the same process admits $\mathcal{R}_{t}=\sigma(Y_{0\leq s\leq t},Y_{T})$–adapted representation

$$ $Y_{t}=Y_{0}+\int_{0}^{t}F^{\prime}(Y_{s},s)+\nabla_{Y_{s}}\log p(Y_{T}\,|\,Y_{ s})\differential s+\beta_{t},$ (14) $$

where $p(Y_{T}\,|\,Y_{s})$ is the density w.r.t the Lebesgue measure of the probability $\mathrm{P}(Y_{T}\,|\,\sigma(Y_{s}))$, and $\{\beta_{0\leq t\leq T},\mathcal{R}_{0\leq t\leq T}\}$ is a Brownian motion.

The connection between time reversal of diffusion processes and enlarged filtrations is finalized with the result of Al-Hussaini & Elliott (), Thm. 3.3, where it is proved how the $\beta_{t}$ term of Equation 14 is a Brownian motion, using the techniques of time reversals of \oldtextscsdes.

Since $\hat{p}(y,T-t)={p}(y,t)$, the enlarged filtration version of Equation 12 reads

$$ $Y_{t}=Y_{0}+\int_{0}^{t}\underbrace{-F(Y_{s},T-s)+\nabla_{Y_{s}}\log p(Y_{s}\, |\,Y_{T})\differential s}_{\text{Equivalent to }H(Y_{t},X,t)=-F(Y_{s},T-s)+ \nabla_{Y_{s}}\log p(Y_{s}\,|\,g(X))}+W_{t}.$ (15) $$

Note that the dependence of $Y_{t}$ on the latent abstractions $X$ is implicitly defined by conditioning the score term $\nabla_{Y_{s}}\log p(Y_{s}\,|\,Y_{T})$ by $Y_{T}$, which is the “rendering” of $X$ into the observable data domain.

Clearly, Equation 15 can be reverted to the starting generative Equation 12 by mimicking the results which allowed us to go from Equation 1 to Equation 4, by noticing that $\mathbb{E}_{\mathrm{P}}[\nabla_{Y_{s}}\log p(Y_{T}\,|\,Y_{s})\,|\,\mathcal{F}^
{Y}_{t}]=0$ (informally, this is obtained since $\int\nabla_{y_{s}}\log p(y_{t}\,|\,y_{s})p(y_{t}\,|\,y_{s})\differential y_{t}
=\int\nabla_{y_{s}}p(y_{t}\,|\,y_{s})\differential y_{t}=0$).

It is also important to notice that we can derive the expression for the mutual information between the measurement process and a sample from the data distribution, as follows

$$ $\mathcal{I}(Y_{0\leq s\leq t};V)=\mathcal{I}(Y_{0};V)+\frac{1}{2}\mathbb{E}_{ \mathrm{P}}\left[\int_{0}^{t}\norm{\nabla_{Y_{s}}\log p(Y_{s})-\nabla_{Y_{s}} \log p(Y_{s}\,|\,Y_{T})}^{2}\differential s\right].$ (16) $$

Mutual information is tightly related to the classical loss function of generative diffusion models.

Furthermore, by casting the result of Equation 8 according to the forms of Equations 12 and 15, we obtain the simple and elegant expression

$$ $\mathcal{I}(Y_{0\leq s\leq t};V)=\mathcal{I}(Y_{0};V)+\frac{1}{2}\mathbb{E}_{ \mathrm{P}}\left[\int_{0}^{t}\norm{\nabla_{Y_{s}}\log p(Y_{T}\,|\,Y_{s})}^{2} \differential s\right].$ (17) $$

In Appendix H, we present a specialization of our framework for the particular case of linear diffusion models, recovering the expressions for the variance-preserving and variance-exploding \oldtextscsdes that are the foundations of score-based generative models (Song et al., ).

###### Assumption 1 .

###### Theorem 7 .

###### Theorem 8 .

## 4 An informal summary of the results

We shall now take a step back from the rigour of this work, and provide an intuitive summary of our results, using Figure 1 as a reference.

Figure: Figure 1: Graphical intuition for our results: nonlinear filtering (left) and generative modelling (right).

We begin with an illustration of \oldtextscnlf, shown on the left of the figure. We consider an observable latent abstraction $X$ and the measurement process $Y_{t}$, which for ease of illustration we consider evolving in discrete time, i.e. $Y_{0},Y_{1},\dots$, and whose joint evolution is described by Equation 1. Such interaction is shown in blue: $Y_{3}$ depends on its immediate past $Y_{2}$ and the latent abstraction $X$.

The a-posteriori measure process $\pi_{t}$ is updated in an iterative fashion, by integrating the flux of information. We show this in green: $\pi_{1}$ is obtained by updating $\pi_{0}$ with $Y_{1}-Y_{0}$ (the equivalent of $\differential Y_{t}$). This evolution is described by Kushner’s equation, which has been derived informally from the result of Equation 6.
The a-posteriori process is a sufficient statistic for the latent abstraction $X$: for example, $\pi_{3}$ contains the same information about $\phi$ than the whole $Y_{0},\dots,Y_{3}$ (red boxes). Instead, in general, a projected statistic $\langle\pi_{t},{\phi}\rangle$ contains less information than the whole measurement process (this is shown in orange, for time instant 2). The mutual information between all these variables is proven in Theorem 6, whereas the actual value of $\mathcal{I}(Y_{0\leq s\leq t};\phi)$ is shown in Theorem 5.

Next, we focus on generative modelling. As by our definition, any stochastic process satisfying 1 ($Y_{3}=V$, in the figure) can be used for generative purposes. Since the latent abstraction is by definition not available, it is not possible to simulate directly the dynamics using Equation 1 (dashed lines from $X$ to $Y_{t}$). Instead, we derive a version of the process adapted to the history of $Y_{t}$ alone, together with the update of the projection $\langle\pi_{t},H\rangle$, which amounts to simulating Equation 10.

The update of the upper part of Equation 10, which is a particular case of Equation 6, can be interpreted as the composition of two steps: 1) (green) the update of the a-posteriori measure given new available measurements, and, 2) (orange) the projection of the whole $\pi_{t}$ into the statistic of interest.
The update of the measurement process, i.e. the lower part of Equation 10, is color-coded in blue. This is in stark contrast to the \oldtextscnlf case, as the update of e.g. $Y_{3}=V$ does not depend directly on $X$.
The system in Equation 10 and its simulation describes the emergence of latent world representations in \oldtextscsde-based generative models:

The theory developed in this work guarantees that the mutual information between measurements and any statistics $\phi$, grows as described by Theorem 5.
Our framework offers a new perspective, according to which, the dynamics of \oldtextscsde-based generative models (Song et al., ) implicitly mimic the two steps procedure described in the box above. We claim that this is the reason why it is possible to dissect the parametric drift of such generative models and find a representation of the abstract state distribution ${{\pi}_{t}}$, encoded into their activations. Next, we set to root our theoretical findings in experimental evidence.

## 5 Empirical Evidence

We complement existing empirical studies (Park et al., ; Kwon et al., ; Chen et al., ; Linhardt et al., ; Tang et al., ; Xiang et al., ; Haas et al., ; Sclocchi et al., ) that first measured the interactions between the generative process of diffusion models and latent abstractions, by focusing on a particular dataset that allows for a fine grained assessment of the influence of latent factors.

Dataset.
We use the Shapes3D (Kim & Mnih, ) dataset, which is a collection of $64\times 64$ ray-tracing generated images, depicting simple 3D-scenes, with an object (a sphere, cube, …) placed in a space, described by several attributes (color, size, orientation). Attributes have been derived from the computer program that the ray-tracing software executed to generate the scene: these are transformed into labels associated to each image. In our experiments, such labels are the materialization of the latent abstractions $X$ we consider in this work (see § J.1 for details).

Measurement Protocols.
For our experiments, we use the base \oldtextscncspp model described by  Song et al. (): specifically, our denoising score network corresponds to a \oldtextscu-net (Ronneberger et al., ). We train the unconditional version of this model from scratch, using score-matching objective. Detailed hyper-parameters and training settings are provided in § J.2. Next, we summarize three techniques to measure the emergence of latent abstractions through the lenses of the labels associated to each image in our dataset.
For all such techniques, we use a specific “measurement” subset of our dataset, which we partition in 246 training, 150 validation, and 371 test examples. We use a multi-label stratification algorithm (Sechidis et al., ; Szymański & Kajdanowicz, ) to guarantee a balanced distribution of labels across all dataset splits.

Figure: Figure 2: Effect of the variance-exploding schedule on an image corrupted by noise with intensity $\tau$.
Refer to caption: x1.png

Linear probing. Each image in the measurement subset is perturbed with noise, using a variance-exploding schedule (Song et al., ), with noise levels increasing from 0 to 1 in steps of 0.1, as shown in Figure 2.
We extract several feature maps from all the linear and convolutional layers of the denoising score network, for each perturbed image, resulting in a total of 162 feature map sets for each noise level. This process yields 11 different datasets per layer, which we use to train a linear classifier (our probe) for each of these datasets, using the training subset. In these experiments, we use a batch size of 64 and adjust the learning rate based on the noise level (see § J.3). Classifier performance is optimized by selecting models based on their log-probability accuracy observed on the validation subset. The final evaluation of each classifier is conducted on the test subset. Classification accuracy, measured by the model log likelihood, is a proxy of latent abstraction emergence (Chen et al., ).

Mutual information estimation. We estimate mutual information between the labels and the outputs of the diffusion model across varying diffusion times, using Equation 39 (which is a specialized version of our theory for linear diffusion models, see Appendix H) and adopt the same methodology discussed by Franzese et al. () to learn conditional and unconditional score functions, and to approximate the mutual information. The training process uses a randomized conditioning scheme: 33% of training instances are conditioned on all labels, 33% on a single label, and the remaining 33% are trained unconditionally. See § J.4 for additional details.

Forking. We propose a new technique to measure at which stage of the generative process, image features described by our labels emerge. Given an initial noise sample, we proceed with numerical integration of the backward \oldtextscsde (Song et al., ) up to time $\tau$. At this point, we fork $k$ replicas of the backward process, and continue the $k$ generative pathways independently until numerical integration concludes. We use a simple classifier (a pre-trained ResNet50 (He et al., ) with an additional linear layer trained from scratch) to verify that labels are coherent across the $k$ forks. Coherency is measured using the entropy of the label distribution output by our simple classifier on each latent factor for all the $k$ forks. Intuitively: if we fork the process at time $\tau=0.6$, and the $k$ forks all end up displaying a cube in the image (entropy equals 0), this implies that the object shape is a latent abstraction that has already emerged by time $\tau$. Conversely, lack of coherence implies that such a latent factor has not yet influenced the generative process. Details of the classifier training and sampling procedure are provided in § J.5.

Figure: Figure 3: Mutual information, Entropy across forked generative pathways, and Probing results as functions of $\tau$.
Refer to caption: x2.png

Results.
We present our results in Figure 3. We note that some attributes like floor hue, wall hue and shape emerge earlier than others, which corroborates the hierarchical nature of latent abstractions, a phenomenon that is related to the spatial extent of each attribute in pixel space. This is evident from the results of linear probing, where we evaluate the performance of linear probes trained on features maps extracted from the denoiser network, and from the mutual information measurement strategy and the measured entropy of the predicted labels across forked generative pathways.
Entropy decreases with $\tau$, which marks the moment in which the generative process proceeds along $k$ forks. When generative pathways converge to a unique scene with identical predicted labels (entropy reaches zero), this means that the model has committed to a specific set of latent factors. This coincides with the same noise level corresponding to high accuracy for the linear probe, and high-values of mutual information. Further ablation experiments are presented in § J.6.

## 6 Conclusion

Despite their tremendous success in many practical applications, a deep understanding of how \oldtextscsde-based generative models operate remained elusive.
A particularly intriguing aspect of several empirical work was to uncover the capacity of generative models to create entirely new data by combining latent factors learned from examples. To the best of our knowledge, there exist no theoretical framework that attempted at describing such phenomenon.

In this work, we closed this gap, and presented a novel theory — that builds on the framework of \oldtextscnlf— to describe the implicit dynamics allowing
\oldtextscsde-based generative models to tap into latent abstractions and guide the generative process. Our theory, that required advancing the standard \oldtextscnlf formulation, culminates in a new system of joint \oldtextscsdes that fully describe the iterative process of data generation. Furthermore, we derived an information-theoretic measure to study the influence of latent abstractions, which provides a concrete understanding of the joint dynamics.

To root our theory into concrete examples, we collected experimental evidence by means of novel (and established) measurement strategies, that corroborates our understanding of diffusion models. Latent abstractions emerge according to an implicitly learned hierarchy, and can appear early on in the data generation process, much earlier than what is visible in the data domain.
Our theory is especially useful as it allows analyses and measurements of generative pathways, opening up opportunities for a variety of applications, including image editing, and improved conditional generation.

## 7 Acknowledgments

G.F. and P.M were partially funded by project MUSECOM2
- AI-enabled MUltimodal SEmantic COMmunications and COMputing, in the Machine Learning-based Communication Systems, towards Wireless AI (WAI), Call 2022,
ChistERA. M.M. acknowledges the financial support of the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (AdG ELISA project, Grant agreement No. 101054746).

## Appendix A Assumptions

###### Assumption 2 .

Whenever we mention a filtration, we assume as usual that it is augmented with the $\mathrm{P}-$ null sets, i.e. if the set $N$ is such that $\mathrm{P}(N)=0$, then all $A\subseteq N$ should be in the filtration.

###### Assumption 3 .

$$ $\mathbb{E}_{\mathrm{P}}[\int_{0}^{t}\norm{H(Y_{s},X,s)}\differential s]<\infty.$ (18) $$

###### Assumption 4 .

$$ $\mathrm{P}(\int_{0}^{t}\norm{\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\,\mathcal {F}^{Y}_{s}]}^{2}\differential s<\infty)=1.$ (19) $$

Eq 2.5 Fundamentals of Stochastic Filtering. Necessary for validity of Equation 3.

###### Assumption 5 .

$$ $\mathbb{E}_{\mathrm{P}}[\int_{0}^{t}\norm{H(Y_{s},X,s)}^{2}\differential s]<\infty.$ (20) $$

Note: this assumption implies 3 and 4. Despite it is more restrictive, it turns out that it is often easier to check.

Eq 3.19 Fundamentals of Stochastic Filtering. Necessary for validity of Theorem 3.

###### Assumption 6 .

$$ $\mathbb{E}_{\mathrm{P}}[\exp{\frac{1}{2}\int_{0}^{t}\norm{H(Y_{s},X,s)}^{2} \differential s}]<\infty,$ (21) $$

and

$$ $\mathbb{E}_{\mathrm{P}}[\exp{\frac{1}{2}\int_{0}^{t}\norm{\mathbb{E}_{\mathrm{ P}}[H(Y_{s},X,s)\,|\,\mathcal{R}_{s}]}^{2}\differential s}]<\infty,$ (22) $$

Note: 6, as well as 5, are trivially verified when $H$ is bounded.

###### Assumption 2 .

###### Assumption 3 .

###### Assumption 4 .

###### Assumption 5 .

###### Assumption 6 .

## Appendix B Proof of Theorem 2

We start by combining Equation 3 and Equation 1

$$ $\displaystyle W^{\mathcal{R}}_{t}$ $\displaystyle=Y_{0}+\int_{0}^{t}H(Y_{s},X,s)\differential s+W_{t}-Y_{0}-\int_{ 0}^{t}\mathbb{E}_{\mathrm{P}}(H(Y_{s},X,s)\,|\,\mathcal{R}_{s})\differential s$ $\displaystyle=\int_{0}^{t}H(Y_{s},X,s)\differential s+W_{t}-\int_{0}^{t} \mathbb{E}_{\mathrm{P}}(H(Y_{s},X,s)\,|\,\mathcal{R}_{s})\differential s.$ $$

We begin by showing that it is a martingale. For any $0\leq\tau\leq t$ it holds

$$ $\displaystyle\mathbb{E}_{\mathrm{P}}[W^{\mathcal{R}}_{t}\,|\,\mathcal{R}_{\tau}]$ $\displaystyle=\mathbb{E}_{\mathrm{P}}[\int_{0}^{t}H(Y_{s},X,s)\differential s \,|\,\mathcal{R}_{\tau}]+\mathbb{E}_{\mathrm{P}}[W_{t}\,|\,\mathcal{R}_{\tau}]$ $\displaystyle\qquad-\mathbb{E}_{\mathrm{P}}[\int_{0}^{t}\mathbb{E}_{\mathrm{P} }(H(s,Y_{s},X)\,|\,\mathcal{R}_{s})\differential s\,|\,\mathcal{R}_{\tau}]$ $\displaystyle=\int_{0}^{t}\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\,\mathcal{R} _{\tau}]\differential s+\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[W_{t} \,|\,\mathcal{F}^{Y,X}_{\tau}]\,|\,\mathcal{R}_{\tau}]$ $\displaystyle\qquad-\int_{0}^{\tau}\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\, \mathcal{R}_{s}]\differential s-\int_{\tau}^{t}\mathbb{E}_{\mathrm{P}}[H(Y_{s} ,X,s)\,|\,\mathcal{R}_{\tau}]\differential s$ $\displaystyle=\int_{0}^{\tau}\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\,\mathcal {R}_{\tau}]\differential s+\mathbb{E}_{\mathrm{P}}[W_{\tau}\,|\,\mathcal{R}_{ \tau}]+W^{\mathcal{R}}_{\tau}+Y_{0}-Y_{\tau}$ $\displaystyle=\mathbb{E}_{\mathrm{P}}[\int_{0}^{\tau}H(Y_{s},X,s)\differential s +W_{\tau}+Y_{0}-Y_{\tau}\,|\,\mathcal{R}_{\tau}]+W^{\mathcal{R}}_{\tau}=W^{ \mathcal{R}}_{\tau}.$ $$

Moreover, it is easy to check that the cross-variation of $W^{\mathcal{R}}_{t}$ is the same as the one of $W_{t}$. Then, we can conclude the proof by Levy’s characterization of Brownian motion ($W^{\mathcal{R}}_{0}=0$).

## Appendix C Proof of Theorem 3

First, by combining the definition of $\psi^{\mathcal{R}}_{t}$ and the fact that $\differential Y_{t}=\mathbb{E}_{\mathrm{P}}[H(Y_{t},X,t)\,|\,\mathcal{R}_{t}]+
\differential W^{\mathcal{R}}_{t}$ we obtain

$$ $(\psi^{\mathcal{R}}_{t})^{-1}=\exp(-\int_{0}^{t}\mathbb{E}_{\mathrm{P}}[H(Y_{s },X,s)\,|\,\mathcal{R}_{s}]\differential W^{\mathcal{R}}_{s}-\frac{1}{2}\int_{ 0}^{t}\norm{\mathbb{E}_{\mathrm{P}}[H(Y_{s},X,s)\,|\,\mathcal{R}_{s}]}^{2} \differential s).$ (23) $$

Notice that by 6 (which is actually the usual Novikov’s condition), the local martingale $(\psi^{\mathcal{R}}_{t})^{-1}$ is a real-valued martingale starting from $(\psi^{\mathcal{R}}_{0})^{-1}=1$. Then, we can apply Girsanov theorem and conclude that $\differential{\mathrm{Q}^{\mathcal{R}}}=\psi^{\mathcal{R}}_{T}\differential
\mathrm{P}$ is a probability measure under which the process $\{\tilde{W}_{0\leq t\leq T},\mathcal{R}_{0\leq t\leq T}\}$, with

$$ $\tilde{W_{t}}=W^{\mathcal{R}}_{t}+\int_{0}^{t}\mathbb{E}_{\mathrm{P}}[H(Y_{t}, X,s)\,|\,\mathcal{R}_{t}]\differential s,$ $$

is a Brownian motion on the space $(\Omega,\mathcal{R}_{T},{\mathrm{Q}^{\mathcal{R}}})$.

## Appendix D Proof of Theorem 4

First, let us give a precise meaning to being a weak solution of Equation 6. We say that $\pi^{\mathcal{R}}_{t}$ solves (6) in a weak sense in, for any for any $\phi:\mathcal{S}\to\mathbb{R}$ bounded and measurable, it holds

$$ $\displaystyle\langle\pi^{\mathcal{R}}_{t},\phi\rangle$ $\displaystyle=\langle\pi^{\mathcal{R}}_{0},\phi\rangle$ (24) $\displaystyle\quad+\int_{0}^{t}\left(\langle\pi^{\mathcal{R}}_{s},H(Y_{s}, \cdot,s)\phi\rangle-\langle\pi^{\mathcal{R}}_{s},\phi\rangle\langle\pi^{ \mathcal{R}}_{s},H(Y_{s},\cdot,s)\rangle\right)\left(\differential Y_{s}- \langle\pi^{\mathcal{R}}_{s},H(Y_{s},\cdot,s)\rangle\differential s\right).$ $$

Let us recall that, on $(\Omega,\mathcal{F},\mathrm{P})$, the process $Y_{t}$ has the SDE representation (1), where $\{W_{0\leq t\leq T},\mathcal{F}^{Y,X}_{0\leq t\leq T}\}$ is a Brownian motion. Moreover, by Theorem 3 with $\mathcal{R}_{t}=\mathcal{F}^{Y,X}_{t}$, it holds that $\{(Y-Y_{0})_{0\leq t\leq T},\mathcal{F}^{Y,X}_{0\leq t\leq T}\}$ is a Brownian motion on the space $(\Omega,\mathcal{F},{\mathrm{Q}}^{\mathcal{F}^{Y,X}})$, where $\differential{\mathrm{Q}}^{\mathcal{F}^{Y,X}}=(\psi^{\mathcal{F}^{Y,X}}_{T})^{
-1}\differential\mathrm{P}$ and

$$ $\psi^{\mathcal{F}^{Y,X}}_{t}=\exp(\int_{0}^{t}H(Y_{s},X,s)\differential Y_{s}- \frac{1}{2}\int_{0}^{t}\norm{H(Y_{s},X,s)}^{2}\differential s).$ (25) $$

For notation simplicity, in this subsection $\psi_{t}^{\mathcal{F}^{Y,X}}$ and ${\mathrm{Q}}^{\mathcal{F}^{Y,X}}$ are simply indicated as $\pi_{t}$, $\psi_{t}$ and ${\mathrm{Q}}$ respectively.

Since we aim at showing that (24) holds, let us fix $\phi$ and let us start from $\mathbb{E}_{\mathrm{P}}[\phi(X)\,|\,\mathcal{R}_{t}]=\langle\pi^{\mathcal{R}}_
{t},\phi\rangle$.
Bayes Theorem provides us with the following

$$ $\langle\pi^{\mathcal{R}}_{t},\phi\rangle=\mathbb{E}_{\mathrm{P}}[\phi(X)\,|\, \mathcal{R}_{t}]=\frac{\mathbb{E}_{{\mathrm{Q}}}[\frac{\differential\mathrm{P} }{\differential{\mathrm{Q}}}\phi(X)\,|\,\mathcal{R}_{t}]}{\mathbb{E}_{{\mathrm {Q}}}[\frac{\differential\mathrm{P}}{\differential{\mathrm{Q}}}\,|\,\mathcal{R }_{t}]}=\frac{\mathbb{E}_{{\mathrm{Q}}}[\psi_{T}\phi(X)\,|\,\mathcal{R}_{t}]}{ \mathbb{E}_{{\mathrm{Q}}}[\psi_{T}\,|\,\mathcal{R}_{t}]}\stackrel{{ \scriptstyle\text{def}}}{{=}}\frac{\langle\hat{\pi}^{\mathcal{R}}_{t},\phi \rangle}{\langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle}.$ (26) $$

Starting from the numerator $\langle\hat{\pi}^{\mathcal{R}}_{t},\phi\rangle$, we involve the tower property of conditional expectation and the fact that $\psi_{t}$ is $\mathcal{F}^{Y,X}_{t}$ measurable to write

$$ $\displaystyle\langle\hat{\pi}^{\mathcal{R}}_{t},\phi\rangle$ $\displaystyle=\mathbb{E}_{{\mathrm{Q}}}[\psi_{T}\phi(X)\,|\,\mathcal{R}_{t}]= \mathbb{E}_{{\mathrm{Q}}}\left[\mathbb{E}_{{\mathrm{Q}}}\left[\psi_{T}\phi(X) \,|\,\mathcal{F}^{Y,X}_{t}\right]\,|\,\mathcal{R}_{t}\right]$ $\displaystyle=\mathbb{E}_{{\mathrm{Q}}}\left[\mathbb{E}_{{\mathrm{Q}}}\left[ \psi_{T}\,|\,\mathcal{F}^{Y,X}_{t}\right]\phi(X)\,|\,\mathcal{R}_{t}\right]= \mathbb{E}_{{\mathrm{Q}}}\left[\psi_{t}\phi(X)\,|\,\mathcal{R}_{t}\right].$ (27) $$

Recalling the definition of $\psi_{t}$ (see Equation 25), we have

$$ $\displaystyle\differential\psi_{t}=\psi_{t}H(Y_{t},X,t)\differential Y_{t},$ (28) $$

from which it follows

$$ $\psi_{t}=1+\int_{0}^{t}\psi_{s}H(Y_{s},X,s)\differential Y_{s}.$ (29) $$

We continue processing Equation 27, using Equation 29, as

$$ $\displaystyle\mathbb{E}_{{\mathrm{Q}}}\left[\psi_{t}\phi(X)\,|\,\mathcal{R}_{s }\right]$ $\displaystyle=\mathbb{E}_{{\mathrm{Q}}}\left[\left(1+\int_{0}^{t}\psi_{s}H(Y_{ s},X,s)\differential Y_{s}\right)\phi(X)\,|\,\mathcal{R}_{t}\right]$ $\displaystyle=\mathbb{E}_{{\mathrm{Q}}}\left[\phi(X)\,|\,\mathcal{R}_{t}\right ]+\mathbb{E}_{{\mathrm{Q}}}\left[\int_{0}^{t}\psi_{s}H(Y_{s},X,s)\phi(X) \differential Y_{s}\,|\,\mathcal{R}_{t}\right]$ $\displaystyle=\mathbb{E}_{{\mathrm{Q}}}\left[\phi(X)\,|\,\mathcal{R}_{t}\right ]+\int_{0}^{t}\mathbb{E}_{{\mathrm{Q}}}\left[\psi_{s}H(Y_{s},X,s)\phi(X)\,|\, \mathcal{R}_{s}\right]\differential Y_{s},$ $$

where to obtain the last equality we used Lemma 5.4 in Xiong (). We also recall that, under ${\mathrm{Q}}$, the process $(Y_{t}-Y_{0})$ is independent of $X$. Thus, since $\mathcal{R}_{t}=\sigma(\mathcal{R}_{0}\cup\sigma(Y_{0\leq s\leq t}-Y_{0}))$ and $\frac{\differential\mathrm{P}}{\differential{\mathrm{Q}}}\,|\,_{\mathcal{F}^{Y
,X}_{0}}=1$, we obtain $\mathbb{E}_{{\mathrm{Q}}}\left[\phi(X)\,|\,\mathcal{R}_{t}\right]=\mathbb{E}_{
\mathrm{P}}[\phi(X)\,|\,\mathcal{R}_{0}]$. Concluding and rearranging:

$$ $\langle\hat{\pi}^{\mathcal{R}}_{t},\phi\rangle=\langle\hat{\pi}^{\mathcal{R}}_ {0},\phi\rangle+\int_{0}^{t}\langle\hat{\pi}^{\mathcal{R}}_{s},\phi H(Y_{s}, \cdot,s)\rangle\differential Y_{s}.$ $$

Obviously by the same arguments $\langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle=\mathbb{E}_{{\mathrm{Q}}}[\frac{
\differential\mathrm{P}}{\differential{\mathrm{Q}}}\,|\,\mathcal{R}_{t}]=
\mathbb{E}_{{\mathrm{Q}}}\left[\psi_{t}\,|\,\mathcal{R}_{t}\right]$, and

$$ $\langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle=1+\int_{0}^{t}\langle\hat{\pi}^{ \mathcal{R}}_{s},H(Y_{s},\cdot,s)\rangle\differential Y_{s}.$ (30) $$

From now on, for simplicity we assume that all the processes involved in our computations are $1$-dimensional. The extension to the multidimensional case is trivial. First, let us notice that, by (30) and Itô’s lemma, it holds

$$ $\differential(\langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle^{-1})=-\frac{\langle \hat{\pi}^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle}{\langle\hat{\pi}^{ \mathcal{R}}_{t},1\rangle^{2}}\differential Y_{s}+\frac{\langle\hat{\pi}^{ \mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle^{2}}{\langle\hat{\pi}^{\mathcal{R}}_{ t},1\rangle^{3}}\differential t.$ (31) $$

Then, by the stochastic product rule,

$$ $\displaystyle\differential\langle\pi^{\mathcal{R}}_{t},\psi\rangle$ $\displaystyle=\differential\left(\langle\hat{\pi}^{\mathcal{R}}_{t},\phi \rangle\langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle^{-1}\right)$ $\displaystyle=\langle\hat{\pi}^{\mathcal{R}}_{t},\phi\rangle\differential( \langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle^{-1})+\langle\hat{\pi}^{\mathcal{R }}_{t},1\rangle^{-1}\differential\langle\hat{\pi}^{\mathcal{R}}_{t},\phi \rangle-\langle\hat{\pi}^{\mathcal{R}}_{t},\phi H(Y_{t},\cdot,t)\rangle\frac{ \langle\hat{\pi}^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle}{\langle\hat{\pi}^{ \mathcal{R}}_{t},1\rangle^{2}}\differential t$ $\displaystyle=-\langle\hat{\pi}^{\mathcal{R}}_{t},\phi\rangle\frac{\langle\hat {\pi}^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle}{\langle\hat{\pi}^{\mathcal{R} }_{t},1\rangle^{2}}\differential Y_{t}+\langle\hat{\pi}^{\mathcal{R}}_{t},\phi \rangle\frac{\langle\hat{\pi}^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle^{2}}{ \langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle^{3}}\differential t$ $\displaystyle\quad+\frac{\langle\hat{\pi}^{\mathcal{R}}_{t},\phi H(Y_{t},\cdot ,t)\rangle}{\langle\hat{\pi}^{\mathcal{R}}_{t},1\rangle}\differential Y_{t}- \langle\hat{\pi}^{\mathcal{R}}_{t},\phi H(Y_{t},\cdot,t)\rangle\frac{\langle \hat{\pi}^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle}{\langle\hat{\pi}^{ \mathcal{R}}_{t},1\rangle^{2}}\differential t.$ $$

Recalling (26) and rearranging the terms lead us to

$$ $\displaystyle\differential\langle\pi^{\mathcal{R}}_{t},\psi\rangle$ $\displaystyle=-\langle\pi^{\mathcal{R}}_{t},\phi\rangle\langle\pi^{\mathcal{R} }_{t},H(Y_{t},\cdot,t)\rangle\differential Y_{t}+\langle\pi^{\mathcal{R}}_{t}, \phi\rangle\langle\pi^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle^{2}\differential t$ $\displaystyle\quad+\langle\pi^{\mathcal{R}}_{t},\phi H(Y_{t},\cdot,t)\rangle \differential Y_{t}-\langle\pi^{\mathcal{R}}_{t},\phi H(Y_{t},\cdot,t)\rangle \langle\pi^{\mathcal{R}}_{t},H(Y_{t},\cdot,t)\rangle\differential t$ $\displaystyle=\left(\langle\pi^{\mathcal{R}}_{t},\phi H(Y_{t},\cdot,t)\rangle- \langle\pi^{\mathcal{R}}_{t},\phi\rangle\langle\pi^{\mathcal{R}}_{t},H(Y_{t}, \cdot,t)\rangle\right)\left(\differential Y_{t}-\langle\pi^{\mathcal{R}}_{t},H (Y_{t},\cdot,t)\rangle\differential t\right).$ $$

## Appendix E Proof of Theorem 5

The proof of this Theorem involves two separate parts. First, we should show the second equality in Equation 7, i.e. $\int\log\frac{\differential\mathrm{P}_{\#Y_{0\leq s\leq t},\phi}}{
\differential\mathrm{P}_{\#Y_{0\leq s\leq t}}\differential\mathrm{P}_{\#\phi}}
\differential\mathrm{P}_{\#Y_{0\leq s\leq t},\phi}=\mathbb{E}_{\mathrm{P}}
\left[\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{\differential
\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi
)}}\right]$. Then, we should prove that the r.h.s of Equation 7 is equal to Equation 8.

### E.1 Part 1

We overload in this Section the notation adopted in the rest of the paper for sake of simplicity in exposition. A random variable $X$ on a probability space $(\Omega,\mathcal{F},\mathrm{P})$ is defined as a measurable mapping $X:\Omega\rightarrow\Psi$, where the measure space $(\Psi,\mathcal{G})$ satisfies the usual assumptions. To be precise, $X$ is measurable w.r.t. $\mathcal{F}$ if $\forall E\in\mathcal{G},X^{-1}(E)\in\mathcal{F}$, where $X^{-1}(E)=\{\omega\in\Omega:X(\omega)\in E\}$. Equivalently, $\forall E\in\mathcal{G},\exists S\in\mathcal{F}:X(S)=E$.
Of all the possible sigma-algebras which allow measurability, the sigma algebra induced by the random variable, $\sigma(X)$, is the smallest one. It can be shown that $\sigma(X)=X^{-1}(\mathcal{G})=\{A=X^{-1}(B)|B\in\mathcal{G}\}$. We also denote by $\mathrm{P}_{\#}X\colon\mathcal{G}\to[0,1]$ the push-forward measure associated to $X$ (i.e. the law), which is defined by the relation $\mathrm{P}_{\#X}(E)=\mathrm{P}(X^{-1}(E))$ for any $E\in\mathcal{G}$. Moreover, for any $\mathcal{G}$-measurable $\phi$, the following integration rule holds

$$ $\int_{\Psi}\varphi(x)\differential\mathrm{P}_{\#X}(x)=\int_{\Omega}\varphi(X( \omega))\differential\mathrm{P}(\omega).$ (32) $$

Let us focus on $(\Omega,\sigma(X),\mathrm{P})$ and let us consider a new measure $\mathrm{Q}$ absolutely continuous w.r.t. $\mathrm{P}$. Radon-Nikodym theorem guarantees existence of a $\sigma(X)$-measurable function $Z\colon\Omega\to[0,+\infty)$ (the “derivative” $\frac{\differential\mathrm{Q}}{\differential\mathrm{P}}=Z$) such that $\mathrm{Q}(A)=\int_{A}Z\differential\mathrm{P}$, for all $A\in\sigma(X)$. Moreover, by Doob’s measurability criterion (see e.g. Lemma 1.13 in Kallenberg ()), there exists a $\mathcal{G}$-measurable map $f\colon\Psi\to[0,+\infty)$ such that $Z=f(X)$. Then, for any $E\in\mathcal{G}$,

$$ $\displaystyle\mathrm{Q}_{\#X}(E)$ $\displaystyle=\mathrm{Q}(X^{-1}(E))=\int_{X^{-1}(E)}f(X)\differential\mathrm{P }(\omega)=\int_{\Omega}\mathbf{1}_{X^{-1}(E)}(\omega)f(X(\omega))\differential \mathrm{P}(\omega)$ $\displaystyle=\int_{\Omega}\mathbf{1}_{E}(X(\omega))f(X(\omega))\differential \mathrm{P}(\omega)=\int_{\Psi}\mathbf{1}_{E}(x)f(x)\differential\mathrm{P}_{\# X}(x)=\int_{E}f(x)\differential\mathrm{P}_{\#X}(x).$ $$

In summary, we have that $\frac{\differential\mathrm{Q}_{\#X}}{\differential\mathrm{P}_{\#X}}=f$, with $f\colon\Psi\to[0,+\infty)$.

Finally, then,

$$ $\displaystyle\int_{\Psi}\log(\frac{\differential\mathrm{P}_{\#X}}{ \differential\mathrm{Q}_{\#X}})\differential\mathrm{P}_{\#X}=-\int_{\Psi}\log( f)\differential\mathrm{P}_{\#X}=-\int_{\Omega}\log(f(X))\differential\mathrm{P }=\int_{\Omega}\log\frac{\differential\mathrm{P}}{\differential\mathrm{Q}} \differential\mathrm{P}=\mathbb{E}_{\mathrm{P}}[\log\frac{\differential\mathrm {P}}{\differential\mathrm{Q}}].$ (33) $$

What discussed so far, allows to prove that $\int\log\frac{\differential\mathrm{P}_{\#Y_{0\leq s\leq t},\phi}}{
\differential\mathrm{P}_{\#Y_{0\leq s\leq t}}\differential\mathrm{P}_{\#\phi}}
\differential\mathrm{P}_{\#Y_{0\leq s\leq t},\phi}=\mathbb{E}_{\mathrm{P}}
\left[\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{\differential
\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi
)}}\right]$. Indeed:

- •
Consider on the space $(\Omega,\mathcal{R}_{t},\mathrm{P}\,|\,_{\mathcal{R}_{t}})$ the random variable $T=(Y_{0\leq s\leq t},\phi)$. By construction, $\sigma(T)=\mathcal{R}_{t}$.
- •
Suppose that $\mathrm{P}\,|\,_{\mathcal{R}_{t}}$ is absolutely continuous w.r.t $\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\times\mathrm{P}\,|\,_{\sigma(\phi)}$ (proved in the next subsection).
- •
Then the desired equality follows from Equation 33.

### E.2 Part 2

Before proceeding, remember that the following holds: for all $\mathcal{R}^{\prime}_{t}\subseteq\mathcal{R}_{t}$,
${\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}^{\prime}_{t}}=\mathrm{Q}^{
\mathcal{R}^{\prime}}\,|\,_{\mathcal{R}^{\prime}_{t}}$.

We restart from the r.h.s. of Equation 7. Thanks to the chain rule for Radon-Nykodim derivatives

$$ $\displaystyle\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{ \differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\, _{\sigma(\phi)}}$ $\displaystyle=\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}\frac{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}{\differential \mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi )}}$ $\displaystyle=\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}\frac{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{F}^{Y}_{t}}}{ \differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}}\frac{\differential{\mathrm {Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}{\differential{\mathrm{Q}^{\mathcal{ R}}}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}$ $\displaystyle=\log\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}\frac{ \differential\mathrm{Q}^{\mathcal{F}^{Y}}\,|\,_{\mathcal{F}^{Y}_{t}}}{ \differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}}\frac{\differential{\mathrm {Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}{\differential{\mathrm{Q}^{\mathcal{ R}}}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}$ $\displaystyle=\log\psi^{\mathcal{R}}_{t}(\psi^{\mathcal{F}^{Y}}_{t})^{-1}\frac {\differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{F}^{Y}_{t}}}{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{F}^{Y}_{t}} \differential\mathrm{P}\,|\,_{\sigma(\phi)}}$ $\displaystyle=\log\psi^{\mathcal{R}}_{t}-\log\psi^{\mathcal{F}^{Y}}_{t}+\log \frac{\differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{R}_{t}}}{ \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\mathcal{F}^{Y}_{t}} \differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\sigma(\phi)}},$ $$

where we used Theorem 3 to make $\psi^{\mathcal{R}}_{t}$ and $\psi^{\mathcal{F}^{Y}}_{t}$ appear, and the fact that $\differential{\mathrm{Q}^{\mathcal{R}}}\,|\,_{\sigma(\phi)}=\differential
\mathrm{P}\,|\,_{\sigma(\phi)}$.

Consequently

$$ $\displaystyle\mathbb{E}_{\mathrm{P}}\left[\log\frac{\differential\mathrm{P}\,| \,_{\mathcal{R}_{t}}}{\differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}} \differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right]=\mathbb{E}_{\mathrm{P}} \left[\log\psi^{\mathcal{R}}_{t}-\log\psi^{\mathcal{F}^{Y}}_{t}\right]+ \mathcal{I}(Y_{0};\phi)$ $\displaystyle=\mathbb{E}_{\mathrm{P}}\left[\int_{0}^{t}\mathbb{E}_{\mathrm{P}} [h(Y_{s},X,s)\,|\,\mathcal{R}_{s}]\differential W^{\mathcal{R}}_{s}+\frac{1}{2 }\int_{0}^{t}\norm{\mathbb{E}_{\mathrm{P}}[h(Y_{s},X,s)\,|\,\mathcal{R}_{s}]}^ {2}\differential s\right]$ $\displaystyle-\mathbb{E}_{\mathrm{P}}\left[\int_{0}^{t}\mathbb{E}_{\mathrm{P}} [h(Y_{s},X,s)\,|\,\mathcal{F}^{Y}_{s}]\differential W^{\mathcal{F}^{Y}}_{s}+ \frac{1}{2}\int_{0}^{t}\norm{\mathbb{E}_{\mathrm{P}}[h(Y_{s},X,s)\,|\,\mathcal {F}^{Y}_{s}]}^{2}\differential s\right]+\mathcal{I}(Y_{0};\phi)$ $\displaystyle=\frac{1}{2}\mathbb{E}_{\mathrm{P}}\left[\int_{0}^{t}\norm{ \mathbb{E}_{\mathrm{P}}[h(Y_{s},X,s)\,|\,\mathcal{R}_{s}]}^{2}-\norm{\mathbb{E }_{\mathrm{P}}[h(Y_{s},X,s)\,|\,\mathcal{F}^{Y}_{s}]}^{2}\differential s\right ]+\mathcal{I}(Y_{0};\phi).$ $$

Actually, the result in the main is in a slightly different form. To show equivalence, it is necessary to prove that

$$ $\displaystyle\mathbb{E}_{\mathrm{P}}\left[\norm{\mathbb{E}_{\mathrm{P}}[h(Y_{s },X,s)\,|\,\mathcal{F}^{Y}_{s}]}^{2}\right]$ $\displaystyle-2\mathbb{E}_{\mathrm{P}}\left[\mathbb{E}_{\mathrm{P}}[h(Y_{s},X, s)\,|\,\mathcal{F}^{Y}_{s}]\mathbb{E}_{\mathrm{P}}[h(Y_{s},X,s)\,|\,\mathcal{R }_{s}]\right]$ $\displaystyle=-\mathbb{E}_{\mathrm{P}}\left[\norm{\mathbb{E}_{\mathrm{P}}[h(Y_ {s},X,s)\,|\,\mathcal{F}^{Y}_{s}]}^{2}\right]$ $$

which is trivially true since $\mathbb{E}_{\mathrm{P}}\left[\cdot\,|\,\mathcal{F}^{Y}_{t}\right]=\mathbb{E}_{
\mathrm{P}}\left[\mathbb{E}_{\mathrm{P}}\left[\cdot\,|\,\mathcal{R}_{s}\right]
\,|\,\mathcal{F}^{Y}_{t}\right]$.

## Appendix F Proof of Theorem 6

### F.1 Proof of Equation 9

The inequality is proven considering that: i)

$$ $\mathcal{I}(Y_{0\leq s\leq t};\phi)=\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{ Y}_{t}}\times\mathrm{P}\,|\,_{\sigma(\phi)}}\left[\eta\left(\frac{ \differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{\differential\mathrm{P}\,|\,_{ \mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right)\right]$ $$

and

$$ $\mathcal{I}(\tilde{Y}_{t};\phi)=\mathbb{E}_{\mathrm{P}\,|\,_{\sigma(\tilde{Y}_ {t})}\times\mathrm{P}\,|\,_{\sigma(\phi)}}\left[\eta\left(\frac{\differential \mathrm{P}\,|\,_{\sigma(\tilde{Y}_{t},\phi)}}{\differential\mathrm{P}\,|\,_{ \sigma(\tilde{Y}_{t})}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right) \right]=\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\times\mathrm{P}\,|\, _{\sigma(\phi)}}\left[\eta\left(\frac{\differential\mathrm{P}\,|\,_{\sigma( \tilde{Y}_{t},\phi)}}{\differential\mathrm{P}\,|\,_{\sigma(\tilde{Y}_{t})} \differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right)\right],$ $$

with $\eta(x)=x\log x$, ii) that $\frac{\differential\mathrm{P}\,|\,_{\sigma(\tilde{Y}_{t},\phi)}}{\differential
\mathrm{P}\,|\,_{\sigma(\tilde{Y}_{t})}\differential\mathrm{P}\,|\,_{\sigma(
\phi)}}=\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\times\mathrm{P}\,|\,
_{\sigma(\phi)}}\left[\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{
\differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,
_{\sigma(\phi)}}\,|\,\sigma(\tilde{Y}_{t},\phi)\right]$ and iii) that Jensen’s inequality holds ($\eta$ is convex on its domain)

$$ $\displaystyle\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\times\mathrm{P} \,|\,_{\sigma(\phi)}}\left[\eta\left(\frac{\differential\mathrm{P}\,|\,_{ \sigma(\tilde{Y}_{t},\phi)}}{\differential\mathrm{P}\,|\,_{\sigma(\tilde{Y}_{t })}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right)\right]$ $\displaystyle\qquad=\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\times \mathrm{P}\,|\,_{\sigma(\phi)}}\left[\eta\left(\mathbb{E}_{\mathrm{P}\,|\,_{ \mathcal{F}^{Y}_{t}}\times\mathrm{P}\,|\,_{\sigma(\phi)}}\left[\frac{ \differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{\differential\mathrm{P}\,|\,_{ \mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}\,|\,\sigma( \tilde{Y}_{t},\phi)\right]\right)\right]$ $\displaystyle\qquad\quad\leq\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}} \times\mathrm{P}\,|\,_{\sigma(\phi)}}\left[\mathbb{E}_{\mathrm{P}\,|\,_{ \mathcal{F}^{Y}_{t}}\times\mathrm{P}\,|\,_{\sigma(\phi)}}\left[\eta\left(\frac {\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{\differential\mathrm{P}\,|\,_ {\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right)\,|\, \sigma(\tilde{Y}_{t},\phi)\right]\right]$ $\displaystyle\qquad=\mathbb{E}_{\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}}\times \mathrm{P}\,|\,_{\sigma(\phi)}}\left[\eta\left(\frac{\differential\mathrm{P}\, |\,_{\mathcal{R}_{t}}}{\differential\mathrm{P}\,|\,_{\mathcal{F}^{Y}_{t}} \differential\mathrm{P}\,|\,_{\sigma(\phi)}}\right)\right].$ $$

### F.2 Proof of Conditional Independence and Mutual Information Equality

Formally the condition of conditional independence given $\pi$ is satisfied if for any $a_{1},a_{2}$ positive random variables which are respectively $\sigma(X)$ and $\mathcal{F}^{Y}_{t}$ measurable, the following holds: $\mathbb{E}_{\mathrm{P}}[a_{1}a_{2}\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm{P}
}[a_{1}\,|\,\sigma(\pi_{t})]\mathbb{E}_{\mathrm{P}}[a_{2}\,|\,\sigma(\pi_{t})]$ (see for instance Van Putten & van Schuppen ()).

The sigma-algebra $\sigma(\pi_{t})$ is by definition the smallest one that makes $\pi_{t}$ measurable. Since $\pi_{t}$ is $\mathcal{F}^{Y}_{t}$ measurable, clearly $\sigma(\pi_{t})\subseteq\mathcal{F}^{Y}_{t}$. By the very definition of conditional expectation, $\mathbb{E}_{\mathrm{P}}[a_{1}\,|\,\mathcal{F}^{Y}_{t}]=\langle\pi_{t},a_{1}\rangle$, which is an $\sigma(\pi_{t})$ measurable quantity. Then $\mathbb{E}_{\mathrm{P}}[a_{1}a_{2}\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm{P}
}[\mathbb{E}_{\mathrm{P}}[a_{1}a_{2}\,|\,\mathcal{F}^{Y}_{t}]\,|\,\sigma(\pi_{
t})]=\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[a_{1}\,|\,\mathcal{F}^{Y}
_{t}]a_{2}\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}
}[\langle\pi_{t},a_{1}\rangle a_{2}\,|\,\sigma(\pi_{t})]=\langle\pi_{t},a_{1}
\rangle\mathbb{E}_{\mathrm{P}}[a_{2}\,|\,\sigma(\pi_{t})]$. Since $\langle\pi_{t},a_{1}\rangle=\mathbb{E}_{\mathrm{P}}[\langle\pi_{t},a_{1}
\rangle\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[a
_{1}\,|\,\mathcal{F}^{Y}_{t}]\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm{P}}[a_{
1}\,|\,\sigma(\pi_{t})]$, the proof of conditional independence is concluded.

In summary, $\sigma(X)$ and $\mathcal{F}^{Y}_{t}$ are conditionally independent given $\sigma(\pi_{t})$ ($\subset\mathcal{F}^{Y}_{t}$). This implies that $\mathrm{P}(A\,|\,\sigma(\pi_{t}))=\mathrm{P}(A\,|\,\mathcal{F}^{Y}_{t}),\quad
\forall A\in\sigma(X)$, or equivalently $\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm
{P}}[\mathbf{1}(A)\,|\,\mathcal{F}^{Y}_{t}]$. To prove this, it is sufficient to show that for any $B\in\mathcal{F}^{Y}_{t}$, $\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\,|\,\sigma(\pi_{
t})]\mathbf{1}(B)]=\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\mathbf{1}(B)]$. By standard properties of conditional expectation $\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\,|\,\sigma(\pi_{
t})]\mathbf{1}(B)]=\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[\mathbf{1}(
A)\,|\,\sigma(\pi_{t})]\mathbb{E}_{\mathrm{P}}[\mathbf{1}(B)\,|\,\sigma(\pi_{t
})]]$. Due to conditional independence $\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\,|\,\sigma(\pi_{t})]\mathbb{E}_{\mathrm{
P}}[\mathbf{1}(B)\,|\,\sigma(\pi_{t})]=\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)
\mathbf{1}(B)\,|\,\sigma(\pi_{t})]$. Then, $\mathbb{E}_{\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\,|\,\sigma(\pi_{
t})]\mathbb{E}_{\mathrm{P}}[\mathbf{1}(B)\,|\,\sigma(\pi_{t})]]=\mathbb{E}_{
\mathrm{P}}[\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\mathbf{1}(B)\,|\,\sigma(\pi_
{t})]]=\mathbb{E}_{\mathrm{P}}[\mathbf{1}(A)\mathbf{1}(B)]$.

The mutual information equality is then proved considering that $\frac{\differential\mathrm{P}\,|\,_{\mathcal{R}_{t}}}{\differential\mathrm{P}
\,|\,_{\mathcal{F}^{Y}_{t}}\differential\mathrm{P}\,|\,_{\sigma(\phi)}}=\frac{
\differential\mathrm{P}(\omega^{x}\,|\,{\mathcal{F}^{Y}_{t}})}{\differential
\mathrm{P}(\omega^{x})}$, since the conditional probabilities exist, and that $\mathrm{P}(\omega^{x}\,|\,{\mathcal{F}^{Y}_{t}})=\mathrm{P}(\omega^{x}\,|\,
\sigma(\pi_{t}))$.

## Appendix G A technical note

As anticipated in the main, 1 might be incompatible with the other technical assumptions in Appendix A.
The problem might arise for singularities in the drift term at time $t=T$, which are usually present in the construction of dynamics satisfying 1 like stochastic bridges. This mathematical subtlety can be more clearly interpreted by noticing that when 1 is satisfied the evolution of the posterior process $\pi_{t}$ at time $T$ can occupy a portion of the space of dimensionality lower than at any $T-\epsilon$, $\epsilon>0$. Or, we can notice that if 1 is satisfied, $\mathcal{I}(Y_{0\leq s\leq T};V)=\mathcal{I}(V;V)$ which can be equal to infinity depending on the actual structure of $\mathcal{S}$ and the mapping $V$. In many cases, a simple technical solution is to consider in the analysis only dynamics of the process in the time interval $[0,T)$(^3^33This is akin to the discussion of arbitrage strategies in finance when the initial filtration is augmented with knowledge of the future value at certain time instants, and the fact that while the new process adapted w.r.t the new filtration is also a martingale w.r.t. a given new measure for all $t\in[0,T)$, it fails to do so for $t=T$ (thus giving an arbitrage opportunity).).In the reduced time interval $[0,T)$, the technical assumptions are generally shown to be satisfied. For the practical purposes explored in this work this restriction makes no difference, and consequently neglect it for the rest of our discussion.

## Appendix H Linear Diffusion Models

Consider the particular case of linear generative diffusion models Song et al. (), which are widely adopted in the literature and by practitioners. We consider the particular case of Equation 11, where the function $F$ has linear expression

$$ $\hat{Y}_{t}=\hat{Y}_{0}-\alpha\int\limits_{0}^{t}\hat{Y}_{s}\differential s+ \hat{W}_{t},$ (34) $$

for a given $\alpha\geq 0$. We assume of course again that 1 holds, which implies that we should select $\hat{Y}_{0}=Y_{T}=V$. Now, $\alpha$ dictates the behavior of the \oldtextscsde, which can be cast to the so called Variance-Preserving and Variance Exploding schedules of diffusion models Song et al. (). In diffusion models jargon, Equation 34 is typically referred to as a noising process. Indeed, by analysing the evolution of Equation 34, $\hat{Y}_{t}$ evolves to a noisier and noisier version of $V$ as $t$ grows. In particular, it holds that

$$ $\hat{Y}_{t}=\exp(-\alpha t)V+\exp(-\alpha t)\int_{0}^{t}\exp(\alpha s) \differential\hat{W}_{s}.$ $$

The next result is a particular case of Theorem 7.

###### Lemma 1 .

Consider the stochastic process $Y_{t}$ which solves Equation 34. The same stochastic process also admits a $\mathcal{F}^{Y}_{t}$–adapted representation

$$ $Y_{t}=Y_{0}+\int_{0}^{t}\alpha Y_{s}+2\alpha\frac{\exp(-\alpha(T-s))\mathbb{E} _{\mathrm{P}}[V\,|\,\sigma(Y_{s})]-Y_{s}}{1-\exp(-2\alpha(T-s))}\differential s +W_{t},$ (35) $$

where $Y_{0}=\exp(-\alpha T)V+\sqrt{\frac{1-\exp(-2\alpha T)}{2\alpha}}\epsilon$, with $\epsilon$ a standard Gaussian random variable independent of $V$ and $W_{t}$.

As discussed in the main paper, we can now show that the same generative dynamics can be obtained under the \oldtextscnlf framework we present in this work, without the need to explicitly defining a backward and a forward process. In particular, we can directly select a observation function that corresponds to an Orstein-Uhlenbeck bridge (Mazzolo, ; Corlay, ), consequently satisfying 1, and obtain the generative dynamics of classical diffusion models. In particular we consider the following about $H$(^4^44Notice that with $H$ selected as in 7 the validity of the theory considered is restricted to the time interval $[0,T)$, see also Appendix G.):

###### Assumption 7 .

The function $H$ in Equation 1 is selected to be of the linear form

$$ $H(Y_{t},X,t)=m_{t}V-\frac{\differential\log m_{t}}{\differential t}Y_{t},$ (36) $$

with $m_{t}=\frac{\alpha}{\sinh{(\alpha(T-t))}}$, where $\alpha\geq 0$. When $\alpha=0$, $m_{t}=\frac{\differential\log m_{t}}{\differential t}=\frac{1}{T-t}$. Furthermore, $Y_{0}$ is selected as in Theorem 7. Under this assumption, $Y_{T}=V,\quad\mathrm{P}-a.s.$, i.e. 1 is satisfied [Proof].

In summary, the particular case of Equation 1 (which is $\mathcal{F}^{Y,X}$ adapted) under 7, can be transformed into a generative model leveraging Theorem 2, since 1 holds. When doing so, we obtain that the process $Y_{t}$ has $\mathcal{F}^{Y}$ adapted representation equal to

$$ $Y_{t}=Y_{0}+\int_{0}^{t}m_{s}\mathbb{E}_{\mathrm{P}}(V\,|\,\mathcal{F}^{Y}_{s} )\differential s-\int_{0}^{t}\frac{\differential\log m_{s}}{\differential s}Y_ {s}\differential s+{W}^{\mathcal{F}^{Y}}_{t},$ (37) $$

which is nothing but Equation 35 after some simple algebraic manipulation. The only relevant detail worth deeper exposition is the clarification about the actual computation of expectation of interest. If $\mathrm{P}$ is selected such that $\hat{Y}_{t}$ solves Equation 34, we have that

$$ $\mathbb{E}_{\mathrm{P}}(V\,|\,\mathcal{F}^{Y}_{t})=\mathbb{E}_{\mathrm{P}}(Y_{ T}\,|\,\sigma(Y_{0\leq s\leq t}))=\mathbb{E}_{\mathrm{P}}(\hat{Y}_{0}\,|\, \sigma(\hat{Y}_{T-t\leq s\leq T}))=\mathbb{E}_{\mathrm{P}}(\hat{Y}_{0}\,|\, \sigma(\hat{Y}_{T-t}))=\mathbb{E}_{\mathrm{P}}(V\,|\,\sigma(Y_{t})),$ (38) $$

where the second to last equality is due to the Markov nature of $\hat{Y}_{t}$.

Moreover, in this particular case we can express the mutual information $\mathcal{I}(Y_{0\leq s\leq t};\phi)=\mathcal{I}(Y_{t};\phi)$ ( where we removed the past of $Y$ since the following Markov chain holds $\phi\rightarrow\hat{Y}_{0}\rightarrow\hat{Y}_{t>0}$) can be expressed in the simpler form

$$ $\mathcal{I}(Y_{t};\phi)=\mathcal{I}(Y_{0};\phi)+\frac{1}{2}\mathbb{E}_{\mathrm {P}}\left[\int_{0}^{t}m^{2}_{s}\norm{\mathbb{E}_{\mathrm{P}}[V\,|\,\sigma(Y_{s })]-\mathbb{E}_{\mathrm{P}}[V\,|\,\sigma(Y_{s},\phi)]}^{2}\differential s\right]$ (39) $$

matching the result described in Franzese et al. (), obtained with the formalism of time reversal of \oldtextscsdes.

###### Lemma 1 .

###### Assumption 7 .

## Appendix I Discussion about 7

This is easily checked thanks to the following equality

$$ $Y_{t}=Y_{0}\frac{m_{0}}{m_{t}}+V\frac{m_{0}}{m_{T-t}}+\int_{0}^{t}\frac{m_{s}} {m_{t}}\differential W_{s}.$ (40) $$

To avoid cluttering the notation, we define $f_{t}=\frac{\differential\log m_{t}}{\differential t}$. To show that Equation 40 is true, it is sufficient to observe i) that initial conditions are met and ii) that the time differential of the process is the correct one. We proceed to show that indeed the second condition holds (the first one is trivially observed to be true).

$$ $\displaystyle\differential Y_{t}=-\alpha Y_{0}\frac{\cosh(\alpha(T-t))}{\sinh( \alpha T)}+\alpha r(X)\frac{\cosh(\alpha t)}{\sinh(\alpha T)}-\alpha\cosh( \alpha(T-t))\int_{0}^{t}\frac{1}{\sinh(\alpha(T-s))}\differential W_{s}+ \differential W_{t}$ $\displaystyle=-\alpha\frac{\cosh(\alpha(T-t))}{\sinh(\alpha(T-t))}\left(Y_{0} \frac{\sinh(\alpha(T-t))}{\sinh(\alpha T)}+\int_{0}^{t}\frac{\sinh(\alpha(T-t) )}{\sinh(\alpha(T-s))}\differential W_{s}\right)+\alpha r(X)\frac{\cosh(\alpha t )}{\sinh(\alpha T)}+\differential W_{t}$ $\displaystyle=-\alpha\coth(\alpha(T-t))\left(Y_{t}-r(X)\frac{\sinh(\alpha t)}{ \sinh(\alpha T)}\right)+\alpha r(X)\frac{\cosh(\alpha t)}{\sinh(\alpha T)}+ \differential W_{t}$ $\displaystyle=-f_{t}Y_{t}+\alpha r(X)\left(\frac{\coth(\alpha(T-t))\sinh( \alpha t)}{\sinh(\alpha T)}+\frac{\cosh(\alpha t)}{\sinh(\alpha T)}\right)+ \differential W_{t}$ $\displaystyle=-f_{t}Y_{t}+\alpha r(X)\left(\frac{\coth(\alpha(T-t))\sinh( \alpha t)+\cosh(\alpha t)}{\sinh(\alpha T)}\right)+\differential W_{t}$ $\displaystyle=-f_{t}Y_{t}+\alpha r(X)\left(\frac{\coth(\alpha(T-t))\sinh( \alpha t)+\cosh(\alpha t)}{\sinh(\alpha T)}\right)+\differential W_{t}$ $\displaystyle=-f_{t}Y_{t}+m_{t}r(X)+\differential W_{t}$ $$

where the result is obtained considering that

$$ $\displaystyle\frac{\coth(\alpha(T-t))\sinh(\alpha t)+\cosh(\alpha t)}{\sinh( \alpha T)}=\frac{\frac{e^{\alpha(T-t)}+e^{-\alpha(T-t)}}{e^{\alpha(T-t)}-e^{- \alpha(T-t)}}\left(e^{\alpha t}-e^{-\alpha t}\right)+\left(e^{\alpha t}+e^{- \alpha t}\right)}{e^{\alpha T}-e^{-\alpha T}}$ $\displaystyle\qquad=\frac{\frac{e^{\alpha T}+e^{-\alpha(T-2t)}-e^{\alpha(T-2t) }-e^{-\alpha T}}{e^{\alpha(T-t)}-e^{-\alpha(T-t)}}+\left(e^{\alpha t}+e^{- \alpha t}\right)}{e^{\alpha T}-e^{-\alpha T}}$ $\displaystyle\qquad=\frac{e^{\alpha T}+e^{-\alpha(T-2t)}-e^{\alpha(T-2t)}-e^{- \alpha T}+e^{\alpha T}-e^{-\alpha(T-2t)}+e^{\alpha(T-2t)}-e^{-\alpha T}}{\left (e^{\alpha(T-t)}-e^{-\alpha(T-t)}\right)\left(e^{\alpha T}-e^{-\alpha T}\right)}$ $\displaystyle\qquad=\frac{2}{e^{\alpha(T-t)}-e^{-\alpha(T-t)}}.$ $$

## Appendix J Experimental Details

### J.1 Dataset details

The Shapes3D dataset (Kim & Mnih, ) includes the following attributes and the number of classes for each, as shown in Table 1.

**Table 1: Attributes and class counts in the Shapes3D dataset.**
| Attribute | Number of Classes |
| --- | --- |
| Floor hue | 10 |
| Object hue | 10 |
| Orientation | 15 |
| Scale | 8 |
| Shape | 4 |
| Wall hue | 10 |

### J.2 Unconditional Diffusion Model Training

We train the unconditional denoising score network using the NCSN++ architecture (Song et al., ), which corresponds to a \oldtextscU-Net (Ronneberger et al., ). The model is trained from scratch using the score-matching objective. The training hyperparameters are summarized in Table 2.

**Table 2: Hyperparameters for unconditional diffusion model training.**
| Parameter | Value |
| --- | --- |
| Epochs | 100 |
| Batch size | 256 |
| Learning rate | $1\times 10^{-4}$ |
| Optimizer | AdamW (Loshchilov & Hutter, ) |
| $\beta_{1}$ | 0.95 |
| $\beta_{2}$ | 0.999 |
| Weight decay | $1\times 10^{-6}$ |
| Epsilon | $1\times 10^{-8}$ |
| Learning rate scheduler | Cosine annealing with warmup |
| Warmup steps | 500 |
| Gradient clipping | 1.0 |
| EMA decay | 0.9999 |
| Mixed precision | FP16 |
| Scheduler | Variance Exploding (Song et al., ) |
| $\sigma_{\text{min}}$ | 0.01 |
| $\sigma_{\text{max}}$ | 90 |
| Loss function | Denoising score matching (Song et al., ) |

### J.3 Linear Probing Experiment Details

In the linear probing experiments, we train a linear classifier on the feature maps extracted from the denoising score network at various noise levels $\tau$. The training details are provided in Table 3.

**Table 3: Hyperparameters for linear probing experiments.**
| Parameter | Value |
| --- | --- |
| Batch size | 64 |
| Loss function | Cross-Entropy Loss |
| Optimizer | Adam (Kingma & Ba, ) |
| Learning rate | $1\times 10^{-6}$ for $\tau=0.9$ or $\tau=0.99$<br>$1\times 10^{-4}$ for other $\tau$ values |
| Number of epochs | 30 |
| Inputs | Feature maps (used as-is in the linear layer)<br>Noisy images (scaled to $[-1,+1]$) |

### J.4 Mutual Information Estimation Experiment Details

For mutual information estimation, we train a conditional diffusion model using the same NCSN++ architecture as before. The conditioning is incorporated by adding a distinct class embedding for each label present in the input image, added to the input embedding along with the timestep embedding. The hyperparameters are the same as those used for the unconditional diffusion model (see Table 2).

To calculate the mutual information, we use Equation 39, estimating the integral using the midpoint rule with 999 points uniformly spaced in $[0,T]$.

Figure: Figure 4: Visualization of the forking experiment with $\texttt{num_forks}=4$ and one initial seed. The image at time $\tau=0.4$ is quite noisy. In the final generations after forking, the images exhibit coherence in the labels shape, wall hue, floor hue, and object hue. However, there is variation in orientation and scale.
Refer to caption: x3.png

### J.5 Forking Experiment Details

In the forking experiments, we use a ResNet50 (He et al., ) model with an additional linear layer, trained from scratch, to classify the generated images and assess label coherence across forks. The training details for the classifier are summarized in Table 4.

**Table 4: Hyperparameters for the classifier in forking experiments.**
| Parameter | Value |
| --- | --- |
| Image size | 224 (resized with bilinear interpolation) |
| Image scaling | $[-1,+1]$ |
| Dataset split | Training set: 72%<br>Validation set: 8%<br>Test set: 20% |
| Early stopping | Stop when validation accuracy exceeds 99%<br>Evaluated every 1000 steps |
| Number of epochs | 1 |
| Optimizer | Adam (Kingma & Ba, ) |
| Learning rate | $1\times 10^{-4}$ |

During the sampling process of the forking experiment, we use the settings summarized in Table 5.

**Table 5: Sampling settings for the forking experiments.**
| Parameter | Value |
| --- | --- |
| Stochastic predictor | Euler-Maruyama method with 1000 steps |
| Corrector | Langevin dynamics with 1 step |
| Signal-to-noise ratio (SNR) | 0.06 |
| Number of forks ($k$) | 100 |
| Number of seeds | 10 (independent initial noise samples) |

### J.6 Linear probing on raw data

In Figure 5, we evaluate the performance of linear probes trained on features maps extracted from the denoiser network, and show compare their log probability accuracy with a linear probe trained on the raw, noisy input and a random guesser. Throughout the generative process, linear probes obtain higher accuracy than the baselines: for large noise levels, a linear probe on raw input data fails, whereas the inner layers of the denoising network extract features that are sufficient to discern latent labels.

Figure: Figure 5: Log-probability accuracy of linear classifiers at $\tau$. ’Feature map’ classifiers are trained on network features; ’Noisy Image’ trained on noisy images; ’Random Guess’ is the baseline for random guessing.
Refer to caption: x4.png

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 6
presentation: 1
questions: See weaknesses.
soundness: 2
strengths: They rigorously define the connection between nonlinear filtering and diffusion.
summary: The authors prove the mathematical equivalence between diffusion models and nonlinear filtering, where a stochastic process Y is controlled by an unobserved variable X. The authors show that diffusion generative modeling is related to inference of X using Y. If this is the case then the internal representations of denoiser networks must reflect our belief in X. They have empirical results demonstrating when certain latent statistics emerge during generation.
weaknesses: The paper asks the reader to spend hours understanding their notations and results before describing the motivation in section 4. The paper is very challenging to read as is and I strongly recommend the authors clearly and succinctly describe their contributions before the technical details, which should also mostly be moved to the appendix. 

While a connection between two mathematical frameworks is challenging to derive, it is only interesting if it leads to a deeper understanding of either technique. The fundamental claim of the paper -- that denoising networks must internally represent latent abstractions -- is seemingly obvious given that these networks are trying to predict the denoised state of Y. I would appreciate it if the authors could clearly articulate what they've learned about diffusion models in the language people currently use to describe diffusion models (if this paper is about diffusion models, by the way, perhaps the whole notation should be in terms of a diffusion model, not NLF).

---

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: * Lines 228-233: could you expand on the claim regarding hierarchical emergence of latent factors? For instance, would an experiment focusing on local details, such as grain on texture, further validate this? If the mutual information of these details increased later in the process, it would support this hierarchical perspective.
* For the mutual information estimation process on line 476, could you clarify the final numerical approach used to approximate this measure during the generative process?
soundness: 3
strengths: * Well-motivated introduction, situating the work within existing literature and highlighting its contributions.
* The structure is logically organized, and the mathematical formulations are rigorous, enhancing the soundness of the theory.
* The proposed information-theoretic measure effectively captures changes in latent abstractions during generation, with experiments showing jumps in the measure that align with theoretical expectations.
summary: This paper introduces a theoretical framework that extends Nonlinear Filtering (NLF) to explain the emergence and influence of latent abstractions in diffusion-based generative models. Through the lens of stochastic differential equations (SDEs), the authors model the dynamics of observable and latent variables, showing how latent abstractions emerge hierarchically during the generative process. The study derives an information-theoretic measure to quantify the influence of these latent factors and validates the theory with experiments demonstrating distinct jumps in this measure, supporting the proposed model.
weaknesses: I have no significant weaknesses to report. Please see questions for minor points of clarification.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. The work by Scolocchi et al that is cited in the manuscript seems to contain many similar claims and the forward-backward experiments there seem similar to the forking experiments here, though the theoretical methods are different. Can the authors comment more on the differences and similarities and possible relations between the two works in the paper? 
2.  The caption of Figure 2 (noise with intensity $\tau$) seems to contradict ths figure? 
3. In line 405, do you mean "$\pi_3$ contains the same information... as the whole ...."? 
4. The results such as those shown in Figure 3 are given as a function of $\tau$, but I assume the meaning of which depends on the noising schedule chosen for the diffusion model. Wouldn't it give more insight to discuss them with respect to intrinsic quantities such as the signal-to-noise ration instead of the "time" $\tau$?
soundness: 3
strengths: The NLF viewpoint on diffusion model is to my knowledge new, and potentially interesting. The use of mutual information measurement is also helpful in quantifying the effect of noising/denoising on the latent variables (here labels). The experiments seem carefully carried out.
summary: This work offers a new way to theoretically understand the generative process of diffusion models using nonlinear filtering. In particular, the authors interpret the generative process in diffusion model as simulating the NLF dynamics, where an unobservable latent abstraction drives observable outcomes in the generative process. Second, the authors propose a concrete method to compute the mutual information between measurement processes and latent abstractions, measuring the extent to which a certain latent variable has been constructed at different stages of the generative process. Third, experiments have been conducted using the standard denoising diffusion model on the Shapes3D data with 6 attributes (floor hue, wall hue, object hue, scale, shape, orientations) as labels. Apart from mutual information, the authors also use the log likelihood (using a trained classifier network) and the entropy in a forking experiments, as proxies for latent variable abstraction.
weaknesses: The methodology is sound provided that the datasets can be understood as being dictated by a low-dimensional set of latent variables, and that it is possible to understand the generative process as inferring the latent variables in a Baysian way. The experiments are also conducted solely on such a synthetic dataset, with each sample having a predetermined set of labels. Either the including of a theoretical discussion on this underlying basic assumption, or devising experiments on more diverse datasets, preferably both, will make the relevance of the work more convincing.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: Please see the Weakness part.

Q1: Could this NLF framework be extended to the deterministic sampling process?
soundness: 3
strengths: 1.	The link between the NLF and diffusion models is really interesting.
2.	This work includes theoretical and empirical parts, which help us understand the role of latent abstractions.
summary: This work studies the role of latent abstractions in the stochastic general process of diffusion models by extending the nonlinear filtering (NLF) framework. More specifically, this work proves that the generative process can be viewed as a switch process between $Y_t$ and $\pi_t$, which corresponds to the image and latent abstractions space. Furthermore, the synthetic and real-world experiments support their theoretical results.
weaknesses: 1.	There are many theorems in the introduction part, which makes it hard to find the most important results of this work. It would be better to polish the presentation of this work. More specifically, I think it would be helpful to present Eq. (10) and Figure 1 and introduce the relationship between Eq. (10) and the classic stochastic process.
2.	It would be better to discuss the relationship between the experiment part and the theoretical part. It seems that the only connection is the mutual information estimation part.

Meta-review: The paper 'Latent Abstractions in Generative Diffusion Models' was reviewed by 4 reviewers who gave it an average score of 5.25 (3+6+6+6). The reviewers gave merit to the paper for the clear motivation, rigour in presentation, and interesting viewpoints. On the other hand, the reviewers were less enthusiastic about the rather technical presentation, lack of connection to practical benchmark problems/demonstrations, and disconnection from a presentation that would benefit the wider ICLR audience. Even if the majority of the reviewers end up giving this submission a 'borderline accept' score, the reviewer consensus is that the paper is not quite fit for ICLR in its current form.

Decision: Reject