SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Authors: Dustin Podell
Zion English
Kyle Lacey
Andreas Blattmann
Tim Dockhorn
Jonas Müller
Joe Penna
Robin Rombach
Venue: ICLR 2024 spotlight (2024)
Keywords: Image Synthesis
Diffusion
Generative AI
Primary area: generative models

Abstract: We present Stable Diffusion XL (SDXL), a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone, achieved by significantly increasing the number of attention blocks and including a second text encoder. Further, we design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. To ensure highest quality results, we also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using a post-hoc image-to-image technique. We demonstrate that SDXL improves dramatically over previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-the-art image generators such as Midjourney.

Published as a conference paper at ICLR 2024
SDXL: IMPROVING LATENT DIFFUSION MODELS FOR HIGH-RESOLUTION IMAGE SYNTHESIS
Dustin Podell
Zion English
Kyle Lacey
Andreas Blattmann
Tim Dockhorn
Jonas Müller
Joe Penna
Robin Rombach
MORRINE
I NODES
ABSTRACT
We present Stable Diffission XL (SDXL), latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffussion, SDXI. leverages three times larger UNet backbone, achieved by significantly increasing the number of attention blocks and including second text encoder. Further, we design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. To ensure highest quality results, we also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using post-hoc image-to-iee to-image technique. We demonstrate that SDXL improves dramatically over previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-ther tfrtt--artt image generators such as Midjourney (Holz 2023).
INTRODUCTION
The last year has brought enormous leaps in deep generative modeling across various data domains, such natural language (Touvron et al. 2023), audio (Huang et al.) 2023), and visual media (Rom. bach et al.) 2021 Ramesh et al.] 2022; Saharia et al. 2022 Singer et al. 2022 Ho et al. 2022 Blattmann et al. 2023 Esser al. 2003). In this report, focus the latter and unveil SDXL. a drastically improved version of Stable Diffussion. Stable Diffusion latent text-to-image dif- fusion model (DM) which serves the foundation for array of recent advancements in, e.g.,

---

Published as a conference paper at ICLR 2024
3D classification (Shen al. 2023) controllable image editing (Zhang & Agrawala 2023 image
personalization Gal et al. 2022). synthetic data augmentation (Stöckl 2022), graphical user interface prototyping (Wei et al. 2023), ) etc. Remarkably, the scope of applications has been extraordinarily extensive, encompassing fields as diverse as music generation (Forsgren & Martiros, 2022) and reconstructing images from fMRI brain scans (Takagi & Nishimoto 2023).
User studies demonstrate that SDXL consistently surpasses all previous versions Stable Diffusion by significant margin (see Fig. [D. D. In this report, we present the design choices which lead this boost in performance encompassing i) a 3x larger UNet-backbone compared to previous Stable Diffusion models (Sec. 2.I). it) two simple yet effective additional conditioning techniques (Sec. (..2) which do not require any form of additional supervision, and iii) separate diffusion based refinement model which applies a noising-denoising process (Meng et al. 2021 to the latents produced by SDXI. to improve the visual quality of its samples (Sec. 2.5
major concern the field visual media creation is that while black-box box-models are often recognized state-of-the-art, the opacity of their architecture prevents faithfully assessing and validating their performance. This lack of transparency hampers reproducibility. stifles innovation, and prevents the community from building upon these models to further the progress of science and art. Moreover, these closed-source strategies make it challenging to assess the biases and limitations of these models in impartial and objective way, which is crucial for their responsible and ethical deployment. With SDXL we are releasing an open model that achieves competitive performance with black-box image generation models (see Fig. 11 & Fig. 12
2 IMPROVING Stable Diffusion
In this section we present our improvements for the Stable Diffusion architecture. These are modular, and can be used individually together to extend any model. Although the following strategies implemented as extensions to latent diffusion models (LDMs) (Rombach et al. 2021), most of them also applicable their pixel-space counterparts.
50
40
36.93
Strefted
Relined
MNC.
Fial
Betber
Wins 188]
30
20
Figure 1: Left: Comparing preferences between SDXL and Stable Diffusion 1.5 & 2.1. While SDXL already clearly outperforms Stable Diffusion 1.5 2.1, adding the additional refinement stage boosts performance. Right: Visualization of the two-stage pipeline: We generate initial latents of size 128 x 128 using SDXL. Afterwards. we utilize a specialized high-resolution refinement model and apply SDEdit (Meng et al. (021) on the latents generated in the first step, using the same prompt. SDXL and the refinement model use the same autoencoder.
2.1 ARCHITECTURE & SCALE
Starting with the seminal works Ho et al. (2020) (Sng et al. (2020b), which demonstrated that DMs are powerful generative models for image synthesis, the convolutional UNet (Ronneberger et al. 2015) architecture has been the dominant architecture for diffusion-based image synthesis. However, with the development of foundational DMs (Saharia et al. 2022 Ramesh et al. 2022 Rombach al. 2021), the underlying architecture has constantly evolved: from adding stlf-attention
and improved upscaling layers (Dhariwal & Nichol 202T). over cross-attenie for text-to -mage
synthesis (Rombach al. 2021). pure wansformer-based architectures (Peebles & Xie 2022
2

---

Published as a conference paper at ICLR 2024
Table 1: Comparison of SDXL and older Stable Diffusioo models.
Model
SDXL
SD 1.4/1.5
SD 2.0/2.1
# of UNet params Transformer blocks
2.6B
860M
865M
[0, 2, 10]
[1, 1] [1. 2. 4, 4]
[1, 1, 1, 1]
Channel mult.
[1. 2, 4]
[1, 2, 4, 4]
Text encoder Context dim. Pooled text emb.
CLIP ViT-L & OpenCLIP ViT-bigG
CLIP ViT-L
OpenCLIP ViT-H
2048
768
1024 N/A
OpenCLIP ViT-bigG
N/A
We follow this trend and, following Hoogeboom et al. (203)) shift the bulk of the transformer computation to lower-level features the UNet. particular, and in contrast to the original Stable Diffussion architecture, we use a heterogeneous distribution of transformer blocks within the UNet: For efficiency reasons, we omit the transformer block at the highest feature level, use and 10 blocks at the lower levels, and remove the lowest level (8x downsampling) in the UNet altogether see Tab. [ for a comparison between the architectures of Stable Diffusion 1.x & 2.x and SDXL. for text conditioning. Specifically,
We opt for we use OpenCLIP Vig- (Ihharco et al.) 202I) in combination with CLIP ViT-L (Radford et al. 2021), where we concatenate the penultimate text encoder outputs along the channel-axis (Balaji et al. 2022). Besides using cross-attention layers to condition the model on the text-input, we follow Nichol al. (2021) and additionally condition the model on the pooled text embedding from the OpenCLIP model. These changes result in a model size of 2.6B parameters in the UNet, see Tab. The text encoders have total size of 81 817M parameters.
powerful pre-trained text encoder that
2.2 MICRO-CONDITIONS
1200
1024
Conditioning the Model on Image Size A no- torious shortcoming of the LDM paradigm (Rom- bach et al.) 2021) is the fact that training model requires a minimal image size, due to its two- stage architecture. The two main approaches to tackle this problem are either to discard all train- ing images below certain minimal resolution (for example, Stable Diffusioo 1.4/1.5 discarded all images with any size below 512 pixels), or, alternatively, upscale images that are too small. However, depending on the desired image res- olution, the former method can lead to signifi- cant portions of the training data being discarded, what will likely lead to a loss in performance and hurt generalization. We visualize such ef- which SDXL was pretrained. For this particular choice of data, discarding all samples below our pretraining res- olution of 256² pixels would lead to a significant 39% of discarded data. The second method, on the other hand, usually introduces upscaling artifacts which may leak into the final model outputs, causing, for example, blurry samples.
512-
256
128-
800
Height
64
128
256
512
1024
600
200
200
400
600
800
1000
1200
Width
Height-vs Wiithh distribution of pre-training dataset. Without the proposed size- conditioning, 39% of the data would be discarded due to edge lengths smaller than 256 pixels as visualized by the dashed black lines. Color intensity in each visu- alized cell is proportional to the number of samples.
Figure
fects in Fig. 2 for the dataset
Instead, we propose to condition the UNet model on the original image resolution, which trivially available during training. In particular, we provide the original (i.e., before any rescaling) height and width of the images as an additional conditioning to the model Csize (heriggimll Worigina)) Each component is independently embedded using Fourier feature encoding, and these encodings are concatenated into a single vector that we feed into the model by adding it to the timestep embedding (Dhariwal & Nichol 2021
At inference time, a user can then set the desired apparent resolution of the image via this size- conditioning. Evidently (see Fig. the model has learned to associate the conditioning Caze with resolution-dependent image features, which can be leveraged to modify the appearance of an output corresponding given prompt. Note that for the visualization shown in Fig. 3 we visualize samples
3

---

Published as a conference paper at ICLR 2024
Ssize = (64, 64)
Csize (128, 128).
Size (256, 256).
Size = (612, 512).
O0
"A robot painted as grafiti on brick wall. sidewalk is in fromt of the wall, and grass is growing out of cracks in the concrete."
" Panda mad scientist mixing sparkling chemicals, artstation." "
Figure 3: The effects of varying the condiiiiiiio::::: We show draw samples with the same random seed from SDXL and vary the size-conditionig as depicted above each column. The image quality clearly increases when conditioning on larger image sizes. Samples from the 512" model, see Sec. 2.5] Note: For this visualization, we use the 512 x 512 pixel base model (see Sec. 2.5 since the effect of size conditioning is more clearly visible before 1024 1024 finetuning. Best viewed zoomed in.
generated by the 512 x 512 model (see Sec. 2.5 for details), since the effects of the size conditioning less clearly visible after the subsequent multi-aspect (ratio) finetuning which use for our final SDXL. model.
We quantitatively assess the effects of this simple but effective conditioning technique by training and evaluating three LDMs on class conditional ImageNet (Deng et al.) 2009) at spatial size 5122: For the first model (CIN-572- only) we discard all training examples with at least one edge smaller than 512 pixels what results in a train dataset of only 70k images. For CIN-nocond we use all training examples but without size conditioning. This additional conditioning is only used for CIN-size-cond. After training generate 5k samples with 50 DDIM steps (Song al. 2020a) and (classifier-fie free) guidance scale of 5 (Ho & Salimans|
Table 2: Conditioning on the original spatial
size of the training examples improves perfor- mance on class-conditional ImageNet Deng et al. 2009) 512" resolution.
model
FID-5k 4
IS-5k f
CIN-512-only
43.84
110.64
CIN-nocond
39.76
211.50
CIN-size-cond
36.53
215.34
2022) for every model and compute IS Salimanss al. (2016) and FID Heusel et al. (2017) (against the full validation set). For CIN- size-cond we generate samples always conditioned on Csize (512, 512). Tab. 2 summmaiieess the results and verifies that CIN-size-cond improves upon the baseline models in both metrics. We attribute the degraded performance of CIN-512-onis to bad generalization due to overfitting on the small training dataset while the effects of a mode of blurry samples in the sample distribution of CIN-nocond result in reduced FII) score. Note that, although we find these classical quantitative scores not to be suitable for evaluating the performance of foundational (text-to-image) DMs [Saharia et al. (2022); ; Ramesh et al.] (2022); RRombach et al. (2021) (see App. they remain reasonable metries on ImageNet as the neural backbones of FID and IS have been trained on ImageNet itself.
Conditioning the Model on Cropping Parameters The first two rows of Fig. iillustrate typical failure mode of previous SD models: Synthesized objects can be cropped, such as the cut-off head of the cat in the left examples for SD 1-5 and SD 2-1. An intuitive explanation for this behavior is the use of random cropping during training of the model: As collating a batch in DL frameworks such PyTorch (Paszke et al. 2019) requires tensors of the same size, typical processing pipeline
4

---

Published as a conference paper at ICLR 2024
"A propaganda poster depicting cat dressed as french emperor napoleon holding a piece of cheese."
"a close-up of fire spitring dragon, cinematic shot."
SD 1-5
PANCEECMPEZ
SD 2-1
SDXL
Figure 4: Comparison of the output of SDXL with previous versions of Stable Diffusion, For each prompt, show random samples the respective model 50 steps of the DDIM sampler [Snn al. (2220a) and efg-scale 8.0 Salimans (2022). Additional samples Fig.
to (i) resize an image such that the shortest size matches the desired target size, followed by (ii) randomly cropping the image along the longer axis. While random cropping is natural form of data augmentation, can leak into generated samples, causing the malicious effects shown above.
To fix this problem, we propose another simple yet effective conditioning method: we uniformly sample crop coordinates Gop and Oet (integers specifying the amount of pixels cropped from the top-left corner along the height and width axes, respectively) and feed them into the model as conditioning parameters via Fourier feature embeddings, similar to the size conditioning described above. The concatenated embedding Ccrop is then used as additional conditioning parameter. We emphasize that this technique not limited to LDMs and could be used for any DM. Note that crop- and size-conditioning can be readily combined. In such a case, we concatenate the feature embedding along the channel dimension, before adding it to the timestep embedding in the UNet. Alg. illlstraate how sample Cerop and Ssiee during training if such combination is applied.
During dataloading.
Given that in our experience large scale datasets are, on average, object-centric, we set (Aup, Geett (0, 0) during inference and thereby obtain object-centered samples from the trained model.
Algorithm 1
Size and crop micro conditioning
Require: Training dataset of images D
Require: Target image for training (Aegts Wagt)
Require: Resizing function R
Require: cropping lunction function
Require: Model train T
See Fig. for an illustration: By tuning (Cops Cleft). can successfully simulate the amount of cropping during inference. This is and has been used in various forms with AR (Jun et al., 2020) models, and recently with diffusion mod- els (Karras et al. 2022).
converged +- False
while not converged
D
Woriginal + width(())
form of
boriginal height(sx)
Czize +- (herigimal Moriginal)
R(s,s) b
image target
if horiginal S Weriginal then
While other methods such as "data bucket- ing" (NovelAI] (023) successfully tackle the same task, we still benefit from cropping- induced data augmentation, while making sure that it does not leak into the generation process actually use it to advantage to gain more control over the image synthesis process. Furthermore, it is easy to implement and can
Clen 4400 width(x:) aw)
sample Clent
Cop
else if horiginal Moriginal then
Copp 24(0, height(s))
sample Cop
Glnn
end
corop (cwp: Pln))
C(., 8, #crop)
crop image to b train model
converged c- T(x,, Grizer Ccrop)
end while

---

Published as a conference paper at ICLR 2024
ccrop = (0, 0)
ccrop = (0, 256).
ccrop (256, 0).
ccrop (512, 512),
"An astronau riding a pig, highly realistic dslr photo, cinematic shot."
capybara made of lego sitting in realistic, natural field."
Figure 5: Varying the crop conditioning discussed in Sec. 2.2 See Fig. 4 and Fig. for samples from SD 1.5 and SD 2.1 which provide no explicit control of this parameter and thus introduce cropping artifacts. Samples from the 512 model, see Sec. 2.5
be applied in online fashion during training, without additional data preprocessing.
2.3 MULTI-ASPER TRAINING
Real-world datasets include images of widely varying sizes and aspect-ratios (c.f. fig. 2) While common output resolutions for text-to-image models square images of 512 x 512 or 1024 x 1024 pixels, we argue that this is a rather unnatural choice, given the widespread distribution and use of landscape (e.g., 16:9) or portrait format screens.
Motivated by this observation, we finetune our model to handle multiple aspect-ratios simultaneously: We follow common practice (NovelAl) 2023) and partition the data into buckets of different aspect ratios, where we keep the pixel count as close to 1024" pixels as possibly, varying height and width accordingly in multiples of 64. A full list of all aspect ratios used for training is provided in App. H During optimization, a training batch is composed of images from the same bucket, and we alternate between bucket sizes for each training step. Additionally, the model receives the bucket size (or, target size) as a conditioning, represented as tuple of integers car (higt. Wegr) which are embedded into a Fourier space similarly to the size- and crop-conditionings described above.
In practice, apply multi-asped training as finetuning stage after pretraining the model at fixed aspect-ratio and resolution and combine it with the conditioning techniques introduced in Sec. 2.2. via concatenation along the channel axis. Fig. [7 in App. [providss pyt bython-cods for this operation. Note that crop-conditioning and multi-aspect training are complementary operations, and crop-conditioning then only works within the bucket boundaries (usually 64 pixels). For case of implementation, however, we opt to keep this control parameter for multi-aspect models. We note that this mechanism can be extended to joint malti-aspect, multi-resolution training by varying the total pixel density. In practice, this means that the batch size can be adjusted dynamically depending on the current resolution, to make the best of availalable VRAM.

---

Published as a conference paper at ICLR 2024
2.4 IMPROVED AUTOENCODER
Stable Diffussion is LDM, operating in pre- trained, learned (and fixed) latent space of an autoencoder (AE). While the bulk of the mantic composition is done by the LDM (Rom- bach et al. 2021), frequency details in generated images by im- proving the AE. To this end, we train the same AE architecture used for the original Stable Dif- fusion at batch-sizz size of 256 and additionally track the weights with an exponential moving average. The resulting AE outperforms the origi- nal model in all evaluated reconstruction metrics, see Tab. 3 A small ablation for the influence of these parameters is reported in App. D we find
Table 3: Autoencoder reconstruction performance on the COCO2017 Lin al. (2013) validation split, images size 256 256 pixels. Note: Stable Diffusion 2.x uses an improved version of Stable Diffiusion 1.x's autoen- coder, where the decoder was finetumed with a reduced weight the perceptual loss /Zhng et al. (2018), and used compute. Note that
improve local, high-
autoencoder
trained from scratch.
model
PNSR
SSIM
LPIPS
rFID
SDXL- SD-VAE 1xx
24.7
0.73
0.88
4.4
23.4
0.69
0.96
5.0
SD-VAE 2.x
24.5
0.71
0.92
4.7
that EMA is helpful all our settings, while effects the large batch size are mixed. We use this AE for all of our experiments.
2.5 PUTTING EVERYTHING TOGETHER
We train the final model, SDXL, a multi-stage procedure. SDXL uses the autoencoder from Sec. 2.4 and discrete-ting diffusion schedule (Ho et al. 2020; Sohi-Dickstein et al. 2015) with 1000 steps. First, we pretrain a base model (see Tab. [ on an internal dataset whose height- and width-distribution visualized in Fig. for 600 000 optimization steps a resolution 256
256 pixels and batch- size of 2048, using size- and crop-conditioning as described in Sec. 2.2 We continue training on 512 px for another 200 000 optimization steps, and finally utilize mulli-aspect training (Sec. 2.3) in combination with an onfset-noise (Guttenberg & CrossLabs 2023] Lin et al. (203)) level of 0.05 to
train the model on different aspect ratios (Sec. 2.3 App. HD of 1024 x 1024 pixel area.
Figure 10213 samples (with zoom-ins) from SDXL without (left) and with (right) the refiner model
(see Sec. 2.5) Prompt: "Epic long distance cityscape photo of New York City flooded by the ocean and
overgrown buildings and jungle ruins in rainforest, at sunset, cinematic shot, highly detailed, 8k, golden light". See Fig. 14 for additional samples.

---

Published as a conference paper at ICLR 2024
Refinement Stage Empirically, we find that the resulting model sometimes yields samples of low local quality, see 6 To improve sample quality, we train a separate LDM in the same latent space, which is specialized high hugh-quality high resolution data and employ noising-denoising process as introduced by SDDddi (Meng et al. 2021) on the samples from the base model, or, alternatively, finish the denoising process with the refiner. We follow (Balaji et al.] 2022) and specialize this refinement model on the first 200 (discrete) noise scales. During inference, we render latents from the base SDXL, and directly diffuse and denoise them in latent space with the refinement model (see Fig. I using the same text input. We note that this step is optional, but improves sample quality for detailed backgrounds and human faces, as demonstrated in Fig. and Fig. 14
To assess the performance of our model (with and without refinement stage), we conduct user study, and let users pick their favorite generation from the following four models: SDXL, SDXI, (with refiner), Stable Diffussion 1.5 and Stable Diffusion 2.1. The results demonstrate the SDXL with the refinement stage is the highest rated choice, and outperforms Stable Diffusion 1.5 2.1 by significant margin (win rates: SDXL w/ refinement: 48.44%, SDXL base: 36.93%, Stable Diffusion 1.5: 7.91%, Stable Diffiusion 2.1: 6.71%). See Fig. | which also provides an overview of the full pipeline. However, when using classical performance metrics such FID and CLIP scores the improvements of SDXL over previous methods are not reflected as shown in Fig. and discussed in App. This aligns with and further backs the findings of Kirstain al. 2023
Input Image
Input Prompt Oulputs
Beautifud watercolor painting.
'An elephana.
'A house.
"Cute animals.
Figure Adding multimodal control: Replacing the pooled text representations of CLIP Radford et al. (2021), which were used during training, with CLIP image features turns SDXL into a multimodal image generator for text-cone controled image editing, which can even transfer abstract concepts such as "image grid" (bottom row) from the input image to its output. In contrast previous work Balaji et al, (2022). this does require joint image-ted text- conditiond pretraining only 1000 finetuning steps single network layer.
2.6 MULTIMODAL CONTROL
Starting with SDEdit (Meng al. 2021). adding image guidance beyond plain text has been major
focus of numerous recent works (Zhang & Agrawala, [2023] Mou et al. 2023) Ruiz et al., 2023
Kawar et al. 2023 Hertz et al. 2022), both with and without further training of the base model. In
8

---

Published as a conference paper at ICLR 2024
this section, we describe simple and efficient approach to turning SDXL into model guided by both text prompts and imput images.
As described in Sec. 2.1 we modify the original Stable Diffusion architecture by considering not only the text embedding sequence, but also the pooled (global) text representation of the CLIP-G text encoder. By taking advantage of the fact that the pooled CLIP feature space is (globally) shared image-text feature space, we can replace this global text representation with a global image representation from the CLIP-G image encoder. To account for the slight discrepancy between image and text embeddings, we fine-tune the embedding layer that maps the CLIP embedding to the UNet's timestep embedding space (where they are added), and leave the remaining parameters frozen.
Fig. demonstrates SDXL's multimodal processing capabilities after this fine-tuning, where we prompt the model with both an input image and text input. For example, the model is able to extract the concept "grid" from an input image and transfer it to another output controlled by a text prompt (see bottom row, Fig. 7 We note that similar approach was implemented in (Balaji et al.) 2022), utilizing joint image and video training (from scratch) with high dropout rates on the image conditioning. In contrast, using a trained text-to-image model of SDXL, we can replace the pooled CLIP embeddings in zero-shot manner and achieve high quality by finetuning only the embedding layer for a few thousand steps. Note that we only modify the base model and leave the refiner as is.
3 CONCLUSION & FUTURE WORK
This report presents an analysis of improvements to the foundation model Stable Diffusion for text-to- image synthesis. While achieve significant improvements in synthesized image quality, prompt adherence and composition, we believe the model may be improved further in the following aspects:
Single stage:
Currently, generate the best samples from SDXL with two-stage approach using our refinement model. This results in having to load two large models into memory, hampering accessibility and sampling speed. Future work should investigate ways provide a single stage.
Text synthesis: While the scale and the larger text encoder (OpenCLIP ViT-bigG (Ilharco et al. 2021)) help to improve the text rendering capabilities over previous versions of Stable Diffusion, incorporating byte-level tokenizers (Xue et al. [0022] Liu et al. (2023) or simply scaling the model to larger sizes (Yu et al. 2022, Saharia et al. (022)) should further improve text synthesis.
During the exploration stage of this work, we briefly experimented with transformer- based architectures such as UVTT (Hoogeboom et al. 2023) and DiT (Peebles & Xie 2022), but found no immediate benefit. We remain, however, optimistic that a careful hyperparameter study will eventually enable scaling to much larger raanformer-domminan
Architecture:
architectures.
While our improvements over Stable Diffiasion are significant, they come at the price of increased inference cost (both in VRAM and sampling speed). Future work will thus focus on decreasing the compute needed for inference, and increased sampling speed, for example through
Distillation:
guidance- (Meng et al., 2023), knowledge- (Dockhorn et al. 2023 Kim et al. 2023 Li al. 2023)
and progressive distillation (Salimans & Ho 2022 Berthelot et al. 2023 Meng et al. 2023).
Finally, our model trained in the discrete-time formulation of (Ho et al., 2020), and requires offset-noise (Guttenberg & CrossLabs [023] Lin et al. (202) for aesthetically pleasing results. The EDM-framework of Karras et al. (2022) is a promising candidate for future model training, as its formulation in continuous time allows for increased sampling flexibility and does not require noise-schedule corrections.

---

Published as a conference paper at ICLR 2024
REFERENCES
Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miik Aittala, Timo Aila. Samuli Laine, Bryan Catanzaro, Tero Karras, and Ming- Yu Liu. eDiff-I: Texl-to-Image Diffusion Models with an Ensemble of Expert Denoisers, arXiv:2211.01332 2022.
David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap. Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbot, and Eric Gu. TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation. WxXiv22333.0424.. 2023.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your Latents: nigh-Reesolutinn Video Synthesis with Latent Diffusion 2023.
Models. arXiv:2304.068.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Prafulla Dhariwal and Alex Nichol. Diffusion Models Beat GANs arXiv:2105.05233, 2021.
Image Synthesis.
Tim Dockhorn, Robin Rombach, Andreas Blattmann, and Yaoliang Yu. Distilling the Knowledge in Diffusion Models. CVPR Workshop on Generative Models for Computer Vision, 2023.
Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog. and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models, 2023.
Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang. and William Yang Wang. Training free structured diffusion guidance for compositional text-to-image synthesis. arXiv: 12.05222. 2023.
Seth Forsgren and Hayk Martiros. Riffusion Stable diffusion for real-time music generation, 2022.
URL /tlLhtttpp//...ffuiiooooomm about
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv:2208.011 016188 2022.
Nicholas Guttenberg and CrossLabs. Diffusion with offset noise, 2023. URL https://ww/ http:/aaa..ooooo/oooooooo/aa..ee ffussiooowi www.ooowwwwooooooooooooooo-----e
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-On Or. Prompt- to-prompt image editing with cross attention control, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by Two Time-Scale Update Rule Converge Local Nash Equilibrium. arXiv:1706. 08500, 2017.
Jonathan Ho and Tim Salimans. Classifier-Free Diffusion Guidance. arXiv:2207.125983 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. arXiv preprint arXiv:2006. /239, 2020.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, and Tim Salimans. Imagen Video: High Definition Video Generation with Diffusion Models. 10.0v:2200002300. 2022.
David Holz. Midjourney, 2023.
home/
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans, simple diffusion: End-to-end diffusion for high resolution images. arXiv preprint arXiv:2301.11099: 2023.
Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu. Xiang Yin, and Zhou Zhao. Make-An-Audee An-Audio: Text-To-Audio Generation with Prompt- Enhanced Diffusion Models. arXiv:2301. 12661, 2023.
10

---

Published as a conference paper at ICLR 2024
Aapo Hyvärinen and Peter Dayan. Estimation of -onmoozmllzeed Statistical Models by Score Matching. Journal of Machine Learning Research, 6(4), 2005.
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. OpenCLIP, July 2021. UtLhttps::/ //tctps://oi... org.1i.rg1.... 5281/
zenodo. 5143773
Heewoo Jun, Rewon Child, Mark Chen, John Schulman, Aditya Ramesh, Alec Radford, and Ilya Sutskever. Distribution Augmentation for Generative Modeling. In International Conference on Machine Learning, pp. 5006-5019 PMLR, 2020.
Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the Design Space of Diffusion- Based Generative Models. arXivv2206.0000 003644 2022.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models, 2023.
Bo-Kyeong Kim, Hyoung-Kyu Song. Thibault Castells, and Shinkook Choi. On Architectural Compression of Text-to-Image Diffusion Models. arXiv:2305.15599. 2023.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. arX1v:2305.01569, 2023.
Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds. MxXiv:2306.00900 2023.
Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common Diffusion Noise Schedules and Sample Steps are Flawed. arXiv:2305.08891, 2023.
Tsung- Yi Lin, Michael Maire, Serge Belongie. Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015.
Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, and Noah Constant, Character-aware models improve visual text rendering, 2023.
Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun- Zhu, and Stefano Ermon. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. arXi::2108.01073 2021.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik P. Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models, 2023.
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models, 2023.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. arXiv:2/12. 10741, 2021.
NovelAI. Novelai improvements stable diffuusion, 2023. URL https:tts///:oo..oooo novel....ooo...o...oooooo... on--onn--nnn stable--aaiiite www.sfff--nnneeeeee 20w.omnb22200000
net/nove wtw.oeell--mmproeeeeee....
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performaace deep learning library, 2019.
11

---

Published as a conference paper at ICLR 2024
William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. arXiv:2212.09748. 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2 103.00020, 2021.
Aditya Ramesh. How dall- 2 works, 2022. URL http:://ddityrremsss..co///////p da Balle2/dallex html
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical Text- Conditional Image Generation with CLIP Latents. arXiv:2204.06125, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer High- Resolution Image Synthesis with Latent Diffusion Models. arXiv preprini arXi:2211.100552, 2021.
Olaf Ronneherger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv: (5t.:::5590949 2015.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, 2023.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho. David Fleet, and Mohammad Norouzi. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding, arXiv:2205.11487, 2022.
Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models. arXiv preprint arXiv:2202.00512: 2022.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved Techniques for Training GANs. arXiv:1606.03499, 2016.
Sitian Shen, Zilin Zhu, Linqian Fan, Harry Zhang, and Xinxiao Wu. DifICLIP: Leveraging Stable Diffusion for Language Grounded 3D Classification. 105.v 2023.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang. Oron Ashual. Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-A-Viee A-Video: Text-to-Vide Video Generation without Text- Video Data. arXiv:2209.14792 2022.
Jascha Sohk-DDckstein, Eric A Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. arXiv:1503.03585, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
arXiv:2010.00250., 2020a.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. arXiv:2011.13456 2020b.
Andreas Stöckl. Evaluating synthetic image dataset generated with stable diffusion. arXiv:2211.01777. 2022.
Yu Takagi and Shinji Nishimoto. High- Resolutsoo Image Reconstruction With Latent Diffusion Models From Human Brain Activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14453-14463 2023.
12

---

Published as a conference paper at ICLR 2024
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and Efficient Foundation Language Models. arXivv23002. 13971, 2023.
Jialiang Wei, Anne-Lise Courbis, Thomas Lambolais, Binbin Xu, Pierre Louis Bernard, and Gérard Dray. Boosting gui prototyping with diffusion models. arXiv preprint arXiv:2306.06235 062233336066,, 2023.
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. Byt5: Towards a token-free future with pre-trained byte-to-byte models, 2022.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Bureu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation, 2022.
Lvmin Zhang and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. arXiv:2302.05543, 2023.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric, 2018.
13


Reviews:
[{"review_id": "TUdTXVGvGW", "reviewer": "Reviewer_pUdr", "summary": "This paper presents SDXL, a latent diffusion model for text-to-image synthesis. SDXL uses a larger U-Net compared to previous Stable Diffusion models, and adds a refiner module to improve visual quality of image samples. During SDXL training, the U-Net is conditioned on image size, image cropping information, and receives training data in multiple aspect ratios. A new VAE is trained, with improved reconstruction performance compared to earlier SD versions.", "questions": "1. There could more explanations on why your current autoencoder provides better reconstruction performance: which factor contributes more, the EMA choice, or larger batch size? \n2. How well does the refinement model work when refining other real images, or images generated by previous SD versions? That is, instead of directly refining latents in VAE latent space, but refining the VAE encoded latents from other real / generated images. \n3. I wonder the relative impact, or the priority of various choices, that is, which plays a bigger role: larger UNet, or each of the conditioning mechanism. I believe these insights are what the research community is insterested in.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. SDXL demonstrates impressive text-to-image generation results. It can serve as a strong base model for a broad range of research and applications in downstream image synthesis tasks.\n2. Some training and architectural choices are backed with convincing ablation experiments (e.g. Table 2). This can offer some valuable insights for future training of image and video generation models.\n3. The refinement model could be an additional contribution, to refine local details of real / generated images.", "weaknesses": "1. For \"conditioning on cropping\" and \"multi-aspect training\", this paper lacks adequate quantitative experiments to demonstrate their effectiveness.\n2. Minor mistakes:\n(1) In Figure 3, the image size should be (256, 256) instead of (256, 236).\n(2) In Figure 3 and Figure 4, the quotation mark for text prompt is incorrect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ObwicDKojq", "reviewer": "Reviewer_n7kc", "summary": "This paper presents SDXL, an extended version of the existing StableDiffusion model, showcasing its training and capabilities. The authors detail the methodology used in image training, including the resolution, crop, top, and left condition techniques, as well as the innovative multi-aspect resolution training approach. Furthermore, the paper sheds light on the advancements in AutoEncoder technology and the implementation of a Refinement model. The overall generation process is structured in two stages, allowing for high quality images. Also authors introduce diverse multimodal controls. This comprehensive explanation highlights the significant enhancements and versatility that SDXL brings to image generation, setting a new standard in the field.", "questions": "- Refinement training : Could you kindly explain about details of refinement training? This training is only for  200 (discrete) noise scales which is [200, 0] ? \n\n- Could you give me some intuition and the effect of using a combination of language models (CLIP ViT-L & OpenCLIP ViT-bigG)? Is it crucial to use all of them?\n\n- I wonder the author's opinion about using TransFormer architecture to train the DMs. Could you share more information on \"no immediate benefit\"?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper not only introduces a variety of innovative training methods but also provides clear intuition and thorough analysis to support them. The novel conditioning techniques and multi-aspect resolution training approaches presented in the manuscript are both groundbreaking and well-substantiated with comprehensive evaluations. The authors have done an excellent job of providing numerical metrics and a wide array of visual comparisons to showcase the effectiveness of their proposed methods. The manuscript is well-written, with a coherent structure that facilitates easy understanding, making it a significant contribution to the field.", "weaknesses": "The manuscript appears to lack detailed explanations on how the autoencoder (AE) has been improved or advanced. There is also no clear description of the user study conducted, raising questions about its methodology and implementation. Is it conducted in a manner similar to what is described in the supplementary materials? Providing more information and context on these aspects would greatly enhance the comprehensibility and robustness of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iExMMBJ8ul", "reviewer": "Reviewer_gmj5", "summary": "This method proposes SDXL,  a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone, achieved by significantly increasing the number of attention blocks and including a second text encoder. Also, the authors propose multiple novel conditioning schemes and train SDXL on multiple aspect ratios. Competitive performance achieved with SoTA models such as Midjourney.", "questions": "1. As discussed in your future work section, can SD be combined into single-stage training and further boost the performance?\n2. Can similar training pipeline introduced in this paper be applied to 3D diffusion training, and what are the challenges?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Very sound elaboration and experiments. The proposed improvement is reasonable and effective.\n2. The method is open sourced.", "weaknesses": "1. Lacks some comparison with SoTA pixel-space diffusion model such as Simple Diffusion from Google.\n2.  Also, SDXL still relied on two-stage training where a high-quality encoder is required. I wonder whether the two stage can be combined and lead to further performance boost?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ek8PcFB3Wy", "reviewer": "Reviewer_rcq7", "summary": "This paper introduces the following techniques to improve Stable Diffusion and it achieves much better performance according to user study. The trained model is also promised to be released publicly:\n1. A more scalable model architecture: modifications on transformer blocks, more powerful text encoders.\n2. Enable conditioning on image size and cropping.\n3. Training with multiple aspect ratio.\n4. Better Autoencoder.\n5. Additional refinement stage enables more high-resolution details.\n6. Additional finetuning that conditions on pooled text embedding enables multi-modal control during inference.", "questions": "It would be great if the authors could respond to the weakness points mentioned above. Thanks!", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper is well-written with clear high-level intuitions/ideas, low-level implementation details, as well as visualizations that justify the improvements. It is easy to follow and understand even for the audience that only have very basic knowledge or practice about diffusion models.\n\n2. The problems the authors are trying to tackle are very realistic in practice and the their proposed solutions are simple and effective: \n- in order to fully utilize all the training data with varying image resolutions, they propose to add conditioning on image size during training. \n- in order to solve the commonly-seen cut-out artifacts during generation (as in Fig. 4), they propose to add conditioning on cropping parameters during training.\n- in order to improve high-resolution details, they propose to train another refinement model and use it in a SDEdit way during inference.\n- in order to enable multi-modal control during inference, they propose to finetune only the embedding layer of a text-to-image SDXL such that it can condition on pooled CLIP text embedding as well, which could be naively replaced with image CLIP embedding during inference.\n\n3. The authors promised to open-source the model (last paragraph of Sec. 1), plus all the implementation details in both the main text and supplementary materials including the pseudo code in Fig. 17. I believe this work could be a huge add-on to the image generation community and enable more future works that build upon it, just like Stable Diffusion.", "weaknesses": "1. Multiple techniques are proposed in this work, therefore a more detailed ablation study would help the audience better understand what’s the influence of each individual component, especially for the two additional conditioning (size and crop). \n2. Particularly, I’m very curious on which method would be more critical to address the aspect ratio issue, the proposed cropping conditioning or data bucketing from NovelAI? According to the description in Sec. 2.3, these two techniques are combined together and there’s no separate evaluation on each of them.\n3. The rightmost examples in Fig. 5 look confusing to me: does (512, 512) mean the whole images are cropped out and there should be nothing inside (the caption says images are from a 512^2 model)? In this case, why does the top example still show basically the same image as the leftmost one while the bottom example shows mainly the background?\n4. The training data of SDXL is not mentioned in the paper. Is it the same as Stable Diffusion, i.e., LAION? If not, would the comparison still be able to justify the effectiveness of proposed techniques since the training data is different?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)