# Portrait Reification with Generative Diffusion Models
## 2 Denoising Diffusion Models  
Denoising Diffusion Models (DDM) [14] are the new state-of-the-art technology in the field of deep generative modeling, challenging the role previously held by Generative Adversarial Networks [6]. The distinctive properties of this generative technique, shared by many recent applications like [17; 18; 19], comprise excellent generation quality, high sensibility and responsiveness to conditioning, good sampling diversity, stability of training, and satisfactory scalability. Roughly, a diffusion model trains a single network to denoise images with a parametric amount of noise; this network is then used to generate new samples by iteratively denoising a given \"noisy\" image, starting from pure random noise and progressively removing a decreasing amount of noise. Figure 5 provides a straightforward graphical representation. This process is traditionally called _reverse diffusion_ since it is meant to \"invert\" the _direct diffusion_ process consisting in iteratively adding noise. In the case of Implicit Diffusion models [15] that we used for our application, reverse diffusion is deterministic. The only trainable component of the reverse diffusion process is a _denoising network_\(e_{\theta}(x_{t},\alpha_{t})\), which takes as input a noisy image \(x_{t}\) and a noise variance \(\alpha_{t}\), and tries to guess the noise present in the image. This model is trained as a traditional denoising network, taking a sample \(x_{0}\) from the dataset, corrupting it with a given amount of random noise, and trying to identify the noise in the noisy image. The pseudocode for training is given in Algorithm 1. We recall that the meaning of the tilde operation \(x\sim P\) is to _sample_\(x\) according to the probability distribution \(P\). \(P_{DATA}\) is the distribution of data points. ```
1:Fix a noise scheduling \((\alpha_{T},\ldots,\alpha_{1})\)
2:repeat
3:\(x_{0}\sim P_{DATA}\)\(\triangleright\) take a sample
4:\(t\sim\) Uniform(1,..,T)\(\triangleright\) choose a timestep
5:\(\epsilon\sim\mathcal{N}(0;I)\)\(\triangleright\) create random gaussian noise
6:\(x_{t}=\sqrt{\alpha_{t}}x_{0}+\sqrt{1-\alpha_{t}}\epsilon\)\(\triangleright\) corrupt the sample with noise rate \(\alpha_{t}\)
7: Take gradient descent step on \(||\epsilon-e_{\theta}(x_{t},\alpha_{t})||^{2}\)\(\triangleright\) backpropagate the loss
8:until converged
```  
**Algorithm 1** Training  
Generative sampling is an iterative process. Starting from a purely noisy image \(x_{T}\), we progressively remove noise by calling the denoising network. The denoised version of the image at timestep \(t\) is obtained by inverting step 6 of the training pseudocode, where \(x_{t-1}\) plays the role of \(x_{0}\). The pseudocode for sampling is given in Algorithm 2. Figure 5: Direct and reverse diffusion. ```
1:\(x_{T}\sim\mathcal{N}(0,I)\)
2:for\(t=T,...,1\)do
3:\(e=e_{\theta}(x_{t},\alpha_{t})\)\(\triangleright\) predict noise
4:\(\hat{x}_{0}=\frac{1}{\sqrt{\alpha_{t}}}(x_{t}-\frac{1-\alpha_{t}}{\sqrt{1- \alpha_{t}}}e)\)\(\triangleright\) compute denoised result
5:\(x_{t-1}=\sqrt{\alpha_{t-1}}\hat{x}_{0}+\sqrt{1-\alpha_{t-1}}e\)\(\triangleright\) re-inject noise at rate \(\alpha_{t-1}\)
6:endfor
```  
**Algorithm 2** Sampling  
The usual architecture for the denoising network is that of a U-Net [20], structured with a downsample sequence of layers followed by an upsample sequence, with skip connections added between the layers of the same size. The noise variance \(\alpha_{t}\) is taken as an additional input, vectorized and stacked along the channel axis. To improve the sensibility of the network to this value, \(\alpha_{t}\) is frequently embedded using an ad hoc sinusoidal transformation, splitting it into a set of frequencies, in a way similar to positional encodings in transformers [21]. An important aspect in implementing diffusion models is the scheduling of the diffusion noise \(\{\alpha_{t}\}_{t=1}^{T}\) during reverse diffusion.