# Denoising Diffusion Models on Model-Based Latent Space
## 2 Related Works
### Diffusion Models  
Diffusion models are a class of likelihood-based generative models which define the generative process as a mapping from a prior distribution \(\mathcal{N}(0,\text{Id})\) to the target unknown distribution \(\mathcal{P}\) of the real data. This is achieved by defining a forward degradation (or diffusion) process \(q\) which gradually corrupts a data sample \(x_{0}\) with additive Gaussian noise at each time step \(t\) with \(t\in(1,\dots,T)\) and a reverse degradation process which learns an approximation \(p_{\theta}\) parameterized by the weights \(\theta\) of the denoising process.  
The forward process \(q(x_{t}|x_{t-1})\) is modeled by a conditional Gaussian distribution which, at each step, is defined by the mean \(\mu_{t}=\mu_{t}(x_{t-1})\) and variance \(\beta_{t}\):  
\[q(x_{t}|x_{t-1})=\mathcal{N}(\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}\text{Id}) \tag{2}\]  
where \(0<\beta_{1}<\beta_{2}<\dots<\beta_{T}<1\) are fixed according to a predefined variance scheduler, which in the original work is defined as linearly increasing from \(\beta_{1}=10^{-4}\) to \(\beta_{T}=2\cdot 10^{-2}\) with \(T=1000\). The reverse process \(p_{\theta}(x_{t-1}|x_{t})\) is also a conditional Gaussian distribution  
\[p_{\theta}(x_{t-1}|x_{t})=\mathcal{N}(\mu_{\theta}(x_{t},t),\beta_{t}\text{Id}) \tag{3}\]  
with \(\theta\) usually being the parameters of a powerful denoising neural network and \(\mu_{\theta}\) being the mean of the learned inverse process. With the variance being fixed, only the mean of the posterior distribution has to be learned. Once the model is trained, sampling from the approximated distribution of real images can be performed effectively by starting with a realization of pure noise \(\epsilon\sim\mathcal{N}(0,\text{Id})\) and sampling \(x_{t-1}\) from \(p_{\theta}\) for \(t=T,T-1,\dots,1\) to recover the sample \(x_{0}\), which represents the initial generated state.  
A suitable objective function to learn the reverse process can be derived by a reparameterization of the mean \(\mu_{\theta}\), leading to the following (we refer to [1] for the complete derivation):  
\[\mathcal{L}_{\epsilon}(\theta)=||\epsilon-\epsilon_{\theta}(x_{t},t)||_{2}^{2} \tag{4}\]  
\[t\sim\mathcal{U}(0,T)\quad\epsilon\sim\mathcal{N}(0,\text{Id})\]  
where \(\epsilon_{\theta}(x_{t},t)\) is the neural network tasked with estimating the noise level on \(x_{t}\) at the uniformly drawn time step \(t\). Since the forward process is a fixed Markov chain, \(x_{t}\) can be directly computed at any noise level \(t\) without going through the intermediate levels. Hence, Equation (4) can be efficiently optimized at random time steps \(t\) by simply predicting the noise realization \(\epsilon\) added to \(x_{0}\). An alternative parameterization of the mean can lead to a different definition of the training objective as the learning of the uncorrupted image \(\hat{x}_{0}^{\theta}\) given \(x_{t}\) and \(t\):  
\[\mathcal{L}_{x_{0}}(\theta)=||x_{0}-\hat{x}_{0}^{\theta}(x_{t},t)||_{2}^{2} \tag{5}\]Although the two formulations are theoretically equivalent, the former is generally preferred because of its observed better quality of the generated images. In the remainder of this paper, we use both expressions, and in particular, we leverage a combination of them when introducing our formulation for the reverse process in Section 3.2.2.  
We close this section by remarking that, starting from [1], DDPMs have been leveraged in both unconditional generation, where the image space is randomly sampled, and conditional generation, where the generative process is guided by a prior, such as the image class in a multiclass scenarios or even complex natural language queries. For the remainder of this work, we only focus on the unconditional generation task.