Title: Dynamic Prompt Evolution via Multi-Attribute Feedback for Text-to-Image Generation
Primary area: generative models
Keywords: Multi-Attribute Feedback, Zero-Order Optimization, Prompt Evolution

Abstract: Most existing text-to-image methods primarily focus on enhancing model comprehension or tuning prompt strength, while overlooking the intrinsic expressive limitations of the prompts themselves—particularly in aligning the generated content with user-specified attributes.  In this paper, we propose \textit{Dynamic Prompt Evolution via Multi-Attribute Feedback} (DPE-MAF), a framework that integrates heuristic algorithms with the prior knowledge of large language models (LLMs) to dynamically generate optimal prompts tailored to user-intended attributes for text-to-image generation.Specifically, we formulate the prompt optimization task as a zero-order optimization problem within the natural language space. To address this, we introduce a \textit{Diffusion-Heuristic Optimization} (DHO) module, which employs LLMs to expand an initial prompt into a candidate population and performs heuristic iterative search guided by multiple attributes, thereby dynamically steering the natural language optimization toward user-desired outputs.To further guide the iterative process, we propose a \textit{Dynamic Contrastive Guidance Update} (DCGU) mechanism, which refines the search by contrasting semantic features between high-quality and low-quality prompts. This contrastive feedback facilitates convergence toward the global optimum. Our approach significantly enhances the ability of diffusion-based generative models to produce semantically consistent and high-fidelity images in text-to-image tasks. Extensive experiments demonstrate that DPE-MAF effectively evolves prompts to improve the performance of various diffusion models, surpassing state-of-the-art prompt-based approaches in text-to-image generation.

TL;DR: An Optimization Method for Input Prompts of Diffusion Models