# Denoising Diffusion Models on Model-Based Latent Space
## 4 Experiments and Evaluation  
### Image Compression  
In this section, we detail the implementation detail of the VQ-based image encoding algorithm introduced in Section 3.1 and provide the qualitative analysis that led to picking the exact configuration used to train the generative models.  
The size of the patch is fixed to \(s=8\) for all models, and the VQ training set is defined by extracting all the possible patches from the 18 high-quality images of the McM dataset [31], which are augmented to 36 by vertically flipping each one, leading to just over 8.6 million patches. Once a VQ method (among VQ,PQ/OPQ, and RQ) is picked, a distinct VQ encoder is trained for each channel in YCbCr space by leveraging the implementations of the FAISS library [32] for maximum efficiency. Each channel can be encoded with a different bitrate; hence, we use the notation {VQ}-{\(Y_{b}\)-\(Cr_{b}\)-\(Cb_{b}\)} to uniquely identify the used combination of VQ type and assigned bitrates.  
Each method is tested on three standard images of size \(512\times 512\), which have been used in the image compression literature for decades ([[https://sipi.usc.edu/database/](https://sipi.usc.edu/database/)]([https://sipi.usc.edu/database/](https://sipi.usc.edu/database/)), accessed on 24 October 2023) [33]. The performance is evaluated with two different quality measures. The Peak Signal-to-Noise Ratio (PSNR) is defined as  
\[PSNR(x_{1},x_{2})=20 log_{10}\Bigg{(}\frac{255}{\sqrt{MSE(x1,x2)}} \Bigg{)} \tag{22}\] \[MSE(x_{1},x_{2})=\frac{1}{\hbar w}\sum_{i=1}^{h}\sum_{j=1}^{w} \|x_{1}(i,j)-x_{2}(i,j)\|_{2}^{2}\]  
and takes values on all the real lines, where the smaller the number, the worse the quality of the image. The other is the Structural SIMilarity index (SSIM), which can be computed as  
\[SSIM(x_{1},x_{2})=\frac{(2\mu_{x_{1}}\mu_{x_{2}}+c_{1})(2\sigma_{x_{1}x_{2}}+ c_{2})}{(\mu_{x_{1}}^{2}+\mu_{x_{2}}^{2}+c_{1})(\sigma_{x_{1}}^{2}+\sigma_{x_{2}} ^{2}+c_{2})}\]  
where, for \(i=1,2\), \(\mu_{x_{i}}\) is the mean of \(x_{i}\), \(\sigma_{x_{i}}\) is its standard deviation, and \(\sigma_{x_{1}x_{2}}\) is the covariance between the two images. The constants \(c_{1}\) and \(c_{2}\) are constants placed in order to avoid numerical issues. The SSIM of an image takes values in the interval \([0,1]\), where a score of 0 implies an heavy dissimilarity between the images, while a score of 1 means the contrary. We also report the compression ratio, which, since the patch size is fixed, can be conveniently computed as  
\[\mathcal{R}(q)=\frac{3*8*s^{2}}{Y_{b}+Cr_{b}+Cb_{b}} \tag{23}\]  
where \(Y_{b}\), \(Cr_{b}\), and \(Cb_{b}\) represent the number of bits needed to represented the pixel value for the respective channel.  
The results reported in Table 1 are in line with what was expected: baseline VQ severely underperforms (especially in terms of SSIM), and increasing the bitrate for the luminance channel yields far better results, while still achieving a high compression ratio. The absolute best results are obtained with RQ, which, however, turned out to be unsuitable for our application because of the commutative propriety of the sum, also being computationally inefficient for the sequential nature of the encoding. In addition, basic PQ really does not make much sense, because it produces highly redundant codebooks. For the above reasons, we chose to proceed with OPQ-32-8-8. We also report the results obtained on the same benchmark by two configurations of the pretrained VAE used in [6]. It is imperative to acknowledge that although the metrics under evaluation may not explicitly highlight this phenomenon, deep encoding tends to yield markedly more visually appealing images. This phenomenon can likely be attributed to the fact that shallow encoding introduces high-frequency and blocky artifacts, which are very noticeable to the human visual system, while deep encoding produces high-level errors that are not obvious at a glance unless a side-by-side comparison is made.