## 7 Diffusion Probabilistic Models  
Diffusion Probabilistic Models (DPM) are a class of latent variable models of the form \(p_{\theta}(\mathbf{x}_{0}):=\int p_{\theta}(\mathbf{x}_{0:T})d\mathbf{x}_{1:T}\) where the data vector \(\mathbf{x}_{0}\sim q(\mathbf{x}_{0})\) is of the same dimension as latent vectors \(\mathbf{x}_{1},...,\mathbf{x}_{T}\) and \(p\) is parameterized by \(\theta\)[7]. The model chains the input variable vector \(\mathbf{x}_{0}\) with the \(T\) latent variable vectors and defines a forward and reverse process over the chain. The model architecture presumes a learned Gaussian transition from each vector in the chain to the next, with each transition parameterized by a neural network of input and output dimension equal to the input variable vector. (We assume that the reader is familiar with the semantics and notation that is the lingua franca of variational inference and diffusion processes so that this high-level overview may serve as a refresher. For a full treatment and development of variational inference and VAE, see [21], for Denoising Diffusion Probabilistic Models see [7], and for Conditional Score-Based Diffusion, see [5]).  
The approximate posterior \(q(\mathbf{x}_{1:T}|\mathbf{x}_{0})\), or forward process, is defined as a Markov chain that transforms input data vectors to a target distribution by gradually adding noise at each of \(T\) steps according to a learnable variance schedule \(\beta_{1},\ldots,\beta_{T}\):  
\[q(\mathbf{x}_{1:T}|\mathbf{x}_{0}):=\prod_{t=1}^{T}q(\mathbf{x}_{t}|\mathbf{x}_ {t-1}),\qquad\quad q(\mathbf{x}_{t}|\mathbf{x}_{t-1}):=\mathcal{N}(\mathbf{x} _{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I}) \tag{9}\]  
The reverse process is a joint probability \(p_{\theta}(\mathbf{x}_{0:T})\) defined as a Markov chain starting at \(p(\mathbf{x}_{T})=\mathcal{N}(\mathbf{x}_{T};\mathbf{0},\mathbf{I})\) that transforms its input across \(T\) learned Gaussian transitions:  
\[p_{\theta}(\mathbf{x}_{0:T}):=p(\mathbf{x}_{T})\prod_{t=1}^{T}p_{\theta}( \mathbf{x}_{t-1}|\mathbf{x}_{t}),\quad\quad p_{\theta}(\mathbf{x}_{t-1}| \mathbf{x}_{t}):=\mathcal{N}(\mathbf{x}_{t-1};\boldsymbol{\mu}_{\theta}( \mathbf{x}_{t},t),\Sigma_{\theta}(\mathbf{x}_{t},t)\mathbf{I}) \tag{10}\]  
Diffusion models admit closed form expressions for sampling \(\mathbf{x}_{t}\) at arbitrary step \(t\). Being differentiable they support training by stochastic gradient descent to optimize the variational lower bound \(L\) on negative log likelihood:  
\[\mathbb{E}[-\log p_{\theta}(\mathbf{x}_{0})]\leq\mathbb{E}_{q}\bigg{[}-\log \frac{p_{\theta}(\mathbf{x}_{0:T})}{q(\mathbf{x}_{1:T}|\mathbf{x}_{0})}\bigg{]} =\mathbb{E}_{q}\bigg{[}-\log p(\mathbf{x}_{T})-\sum_{t\geq 1}\log\frac{p_{ \theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})}{q(\mathbf{x}_{t}|\mathbf{x}_{t-1})} \bigg{]}:=L \tag{11}\]  
where \(\mathbb{E}\) is the expectation. Sampling of \(\mathbf{x}_{t}\) at arbitrary time step \(t\) is achieved as follows:  
\[q(\mathbf{x}_{t}|\mathbf{x}_{0})=\mathcal{N}(\mathbf{x}_{t};\sqrt{\tilde{ \mathbf{x}}}\mathbf{x}_{0},(1-\tilde{\mathbf{x}}_{t})\mathbf{I}) \tag{12}\]  
where \(\alpha_{t}:=1-\beta_{t}\) and \(\tilde{\alpha}_{t}:=\prod_{s=1}^{t}\alpha_{s}\).