Kati8/ICLR_2023_2025 huggingface dataset:
authors: Pablo Pernias
authors: Dominic Rampas
authors: Mats Leon Richter
authors: Christopher Pal
authors: Marc Aubreville
reviews: 
Comments: The paper proposes a new text-to-image diffusion model architecture in which a base diffusion model is conditioned on a highly compressed 2D latent space obtained from a second diffusion model. Concretely, the "main" diffusion model denoises a higher-resolution image (e.g., 256x256 latent or pixel space) but is being conditioned on 24x24 feature map of the image that is to be generated. The 24x24 feature map is obtained by another diffusion model that is trained on that feature space. The resulting model is faster to train and faster to sample from, since both training and sampling of the 24x24 diffusion model is cheap and the large diffusion model at higher resolution benefits from the additional conditioning of the first diffusion model.
Strengths: The model architecture seems novel and based on the evaluation it seems to be faster to sample from while also being faster to train than other baseline models.

The paper builds on top of the latent diffusion architecture and outperforms similarly sized LDMs (and even Stable Diffusion 1 and 2) based on quantitative metrics and human user studies. Importantly, it does so while being faster to train and faster to sample from.
The evaluation is well done and compares against several strong baselines, performs severfal human user studies, and also highlights some weaknesses of the current model compared to other models (e.g. fewer high-frequency details).

Furthermore, the model and code to reproduce will be released.
Weaknesses: The approach is only tested on latent diffusion models. While there is no reason to believe it wouldn't work on pixel diffusion models it would be nice to verify this.
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: This paper introduces a new latent representation for images that can serve as compact semantic guidance for the current denoising diffusion process. Specifically, the proposed Wurstchen framework employs three stages of decoupling text-conditional image generation from high-resolution spaces. This supports an efficient optimization, which significantly reduces computational requirements for large-scale training. This architecture also enables faster inference.
Strengths: + This paper is well-written and easy to follow.
+ The field of efficient training is less discussed than inference, which makes this draft more valuable.
+ The Wurstchen framework can reduce ~9X GPU training hours yet maintain competitive T2I performance.
+ They provide comprehensive qualitative examples in the supplementary. The released code and checkpoint can benefit generative AI research.
Weaknesses: I am satisfied with the current draft. As it targets robust latent visual representations, there should be a detailed analysis (e.g., the quality of the latent features / the distribution of the compression space). This can make its claim more convincing.
Recommendation: 8: accept, good paper
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 4 excellent
Clarity: 3 good