D ream S ync: Aligning Text-to-Image Generation with Image Understanding Feedback (2025)

Abstract: Despite their widespread success, Text-to-Image models (T2I) still struggle to produce images that are both aesthetically pleasing and faithful to the user’s input text. We introduce DreamSync, a simple yet effective training algorithm that improves T2I models to be faithful to the text input. DreamSync utilizes large vision-language models (VLMs) to effectively identify the fine-grained discrepancies between generated images and the text inputs and enable T2I models to self-improve without labeled data. First, it prompts the model to generate several candidate images for a given input text. Then, it uses two VLMs to select the best generation: a Visual Question Answering model that measures the alignment of generated images to the text, and another that measures the generation’s aesthetic quality. After selection, we use LoRA to iteratively finetune the T2I model to guide its generation towards the selected best generations. DreamSync does not need any additional human annotation, model architecture changes, or reinforcement learning. Despite its simplicity, DreamSync improves both the semantic alignment and aesthetic appeal of two diffusion-based T2I models, evidenced by multiple benchmarks (+1.7% on TIFA, +2.9% on DSG1K, +3.4% on VILA aesthetic) and human evaluation shows that DreamSync improves text rendering compared to SDXL by 18.5% on DSG1K benchmark.

## introduction although we invite creative liberty when we commission art, we expect an artist to follow our instructions. despite the advances in text-to-image (t2i) generation models @xcite @xcite @xcite , it remains challenging to obtain images that meticulously conform to users' intentions @xcite ., * equal contribution. work done at google. † work partly done at usc. today, there are efforts to address these challenges. for example, it is possible to manipulate attention maps based on linguistic structure to improve attribute-object binding @xcite ; or train reward models using human feedback to better align generations with user intent @xcite . unfortunately, these methods either operate on a specific model architecture @xcite or require expensive labeled human data @xcite . worse, most of these methods sacrifice aesthetic appeal when optimizing for faithfulness, which we confirm in our experiments. we introduce dreamsync, a simple yet effective framework that improves t2i generation faithfulness while maintaining aesthetic appeal. our approach extends work on fine-tuning t2i models for alignment, but does not require any human feedback. the key insight behind dreamsync is in leveraging the advances in vision-language models (vlms), which can identify fine-grained discrepancies between the generated image and the user's input text @xcite . intuitively at a high level, our method can be thought of as a scalable version of reinforcement prompt: "a little girl wearing a bright yellow dress and a copper crown is riding a badger through a field of flowers." ## dreamsync our method improves alignment and aesthetics in four steps (see figure 1 ): sample, evaluate, filter, and finetune. the high-level idea is that t2i models are capable of generating interesting and varied samples. these examples are further judged by vlms to pass qualification as faithful and aesthetic candidates for further finetuning t2i models. we next dive into each component more formally. ## generating diverse prompts with llms the success of dreamsync is dependent on the diversity of the prompts used during self-training. we need a large amount of diverse training prompts to ensure that dreamsync models have good generalization ability, rather than overfitting to a few entities. to obtain such prompts and corresponding question-answer pairs without human-in-the-loop, a cityscape with skyscrapers and flowers growing on the sides of the buildings a dark gray cat wearing a multi colored scarf around its neck, sitting on a wall a colorful anime illustration of a woman wearing a silver necklace, standing in a field of flowers, with a rainbow in the background a double decker bus driving down a street lined with red-brick buildings and floral gardens a scarecrow made of ice sculptures of animals an intriguing photo of an old man sitting on a bench in the park, lit by the setting sun a futuristic cityscape with a tied lidded trash can in the foreground a metallic blue car parked in the middle of a field of flowers we utilize large language models (llm) to create our prompt source. we choose palm 2(anil et al., 2023) and proceed as follows: we showcase the diversity of palm 2 generated prompts in figure 3 using qualitative examples and quantitive statistics of our generated prompts in appendix a.2. we use this set of generated prompts for all our experiments. ## experiments in this section, we conduct a series of experiments, with different evaluation strategies, to measure the effectiveness of dreamsync. we explain our training setup in §5.1. we conduct a human study with over 1000 images in §5.2 to understand the performance of dreamsync compared to the most stateof-the-art open-sourced model, sdxl. then, we report the results with automatic evaluation metrics and compare dreamsync against other methods in §5.3. we also evaluate our models with reward models trained with human preferences ( §5.4). finally, we conduct analyses on the training process and the effect of choosing different vqa models in §5.5. ## discussion a key design choice behind dreamsync is to maintain simplicity and automation throughout each step of the pipeline. despite this feature, our experimental results show that dreamsync can improve both sd v1.4 and sdxl on tifa, dsg, and visual appeal. in the case of sd v1.4, this improvement holds true compared to four different baseline models (two training-free and two rl-based). for sdxl, even though the base model achieves sota among open-source models, dreamsync can still substantially improve alignment and aesthetics. the effectiveness of dreamsync's self-training methodology opens the door for a new paradigm of parameter-efficient finetuning. indeed, the dream-sync pipeline is easily generalizable. for the train-ing prompts, we can construct a set with complex and non-conventional examples compared to standard web-scraped data. on the filtering and finetuning side, our framework shows that vlms can provide effective feedback for t2i models. together, these steps do not require human annotations, yet they can tailor a generative model toward desirable criteria. ## conclusion we introduce dreamsync, a versatile framework to improve text-to-image (t2i) synthesis with feedback from image understanding models. our dual vlm feedback mechanism helps in both the alignment of images with textual input and the aesthetic quality of the generated images. through evaluations on two challenging t2i benchmarks (with over five thousand prompts), we demonstrate that dreamsync can improve both sd v1.4 and sdxl for both alignment and visual appeal. the benchmarks also show that dreamsync performs well in both in-distribution and out-of-distribution settings. furthermore, human ratings and a human preference prediction model largely agree with dream-sync's improvement on benchmark datasets. for future work, one direction is to ground the feedback mechanism to give fine-grained annotations (e.g., bounding boxes to point out where in the image the misalignment lies). another direc-tion is to tailor the prompts used at each iteration of dreamsync to target different improvements: backpropagating vlm feedback to the prompt acquisition pipelines for continual learning.