# Denoising Diffusion Models on Model-Based Latent Space
## 4 Experiments and Evaluation
### Image Generation
#### 4.2.1 Experimental Setup  
We assess the generative performances in a single-class unconditional scenario on two subsets of the LSUN dataset [34] which are extremely popular in this context: the _cat_ subset and the _church_ one. The first is a large dataset of over 1.6 million images of cats acquired from a wide variety of sources. This is a very complicated dataset due to the extreme variability in the appearance and composition of the images, caused in part by the nature of the subject. In contrast, the dataset of churches is smaller with only 125 thousand images, and it is considerably easier to handle because of the chromatic and geometric regularity of the buildings represented. All the experiments are conducted with an image resolution before compression of \(512\times 512\), which means a latent representation of \(64\times 64\times 6\) using the OPQ-[32-8-8] encoding.  
We fix a common setup for all the experiments: the denoiser model is defined as a U-Net model with four down/up sampling stages and attention only at the bottleneck level in order to reduce the memory footprint. In total, the model has around 50 million parameters, which is a small number if compared with what is commonly used. We base our experimental setup on the Pytorch implementation by [PERSON] of the original DDPM paper, with the addition of Min-SNR weighting [35], which penalizes the loss terms proportionally to the inverse of the magnitude of the noise added al timestep \(t\). In this way, the model is discouraged from focusing on minor details and self-conditioning [36], which conditions the denoising process based on the previous estimate of the recovered sample \(\tilde{x}_{0}\). All the evaluated models are trained for 1 million steps with a batch size of 64 on 2 Nvidia v100 GPUs, which takes around 10 days. This represents an extremely tight computational budget for this kind of application. In comparison, in the original DDPM, a U-Net configuration with either 114 or 256 million parameters (approximately 2-5 times  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Encoding**} & \multirow{2}{*}{**C.R**} & \multicolumn{2}{c}{**Baboon**} & \multicolumn{2}{c}{**Peppers**} & \multicolumn{2}{c}{**Lenna**} \\ \cline{3-8}  & & **PSNR** & **SSIM** & **PSNR** & **SSIM** & **PSNR** & **SSIM** \\ \hline VQ-[8-8-8] & 1:64 & 28.83 & 0.40 & 30.75 & 0.63 & 31.23 & 0.67 \\ PQ-[32-8-8] & 1:32 & 29.07 & 0.60 & 31.28 & 0.69 & 31.80 & 0.74 \\ OPQ-[32-8-8] & 1:32 & 29.12 & 0.62 & 31.56 & 0.  
larger than ours) is leveraged and trained for the equivalent of over 100 v100 days, which is over 10 times our computational budget. An additional difference is that, thanks to the efficiency of the proposed encoding, we are able to generate samples of size \(512\times 512\), which after encoding with the chosen setup (OPQ-(32-8-8)) corresponds to latent codes of size \(64\times 64\times 6\). Other existing methods, such as the original DDPM [1], typically use samples of size \(256\times 256\), i.e., \(\frac{1}{4}\) of the size of our decoded samples. The results, expressed in terms of FID score, are shown in Table 2.