B eautiful P rompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis (2023)

Abstract: Recently, diffusion-based deep generative models (e.g., Stable Diffusion) have shown impressive results in text-to-image synthesis. However, current text-to-image models often require multiple passes of prompt engineering by humans in order to produce satisfactory results for real-world applications. We propose BeautifulPrompt, a deep generative model to produce high-quality prompts from very simple raw descriptions, which enables diffusion-based models to generate more beautiful images. In our work, we first fine-tuned the BeautifulPrompt model over low-quality and high-quality collecting prompt pairs. Then, to ensure that our generated prompts can generate more beautiful images, we further propose a Reinforcement Learning with Visual AI Feedback technique to fine-tune our model to maximize the reward values of the generated prompts, where the reward values are calculated based on the PickScore and the Aesthetic Scores. Our results demonstrate that learning from visual AI feedback promises the potential to improve the quality of generated prompts and images significantly. We further showcase the integration of BeautifulPrompt to a cloud-native AI platform to provide better text-to-image generation service in the cloud.

## introduction text-to-image synthesis (tis) is one of the most spectacularly developed and widely applied techniques in generative artificial intelligence (ai), aiming to create realistic images with texts as input. recently, with the advance of the modeling power of large models, tis is undergoing a revolution. large-scale tis models, such as dalle @xcite , dalle-2 @xcite , latent diffusion models @xcite and imagen @xcite , significantly improve the state-of-the-art performance and allow users without artistic expertise to create unprecedented images through personal imagination. yet, tis models require users to write text prompts before model inference (e.g., "a majestic sailing ship"). writing such prompts that meet the designer's or art worker's needs is full of uncertainty, like opening a surprise box @xcite . this is due to the quality of the training data, leading to the need for detailed descriptions to produce high-quality images. in real-world scenarios, non-experts often find it difficult to write these prompts, and need to do iterative modification through trials and errors to re-generate the images, leading to a significant loss of time and computing resources. prompt engineering is an emerging research field, aiming to explore how to provide prompts for deep generative models and improve the efficiency of direct interaction between humans and ai @xcite . for example, a user can give a task-oriented prompt and ask chat-gpt @xcite to generate texts according to the prompt. for tis, the user can write a simple prompt and then ask chatgpt to supplement the contents. however, directly using chatgpt to write prompts falls into the dilemma of generating irrelevant and plausible images. hence, the generated prompts can be better in quality if the underlying language model is optimized for the task. we can see that fine-tuning a language model such as @xcite @xcite for tis prompt generation will be a more worthwhile exploration. in this paper, we propose a new generative model that can write high-quality prompts for diffusionbased models, named beautifulprompt. for better user experience, it re-writes and optimizes the original, low-quality prompts into high-qualities ones to generate better images. it also provides a good source of inspiration for further manual prompt editing. specifically, we first collect a dataset for training beautifulprompt using an automated data collection pipeline based on existing ai models. the dataset is used for supervised fine-tuning. we further propose a reinforcement learning with visual ai feedback (rlvaif) technique to maximize the reward values of the generated prompts, which are determined by a couple of trained reward models based on visual signals. the gradient update process of rlvaif makes the generated prompts more compatible with human preferences without any manual labeling. a simple comparison of prompts and the resulting images are shown in figure 1 . in summary, the main contributions of this study are as follows: ## dataset creation in this section, we show the detailed data collection process for beautifulprompt training. collection of prompt pairs. the goal of this step is collecting pairs of high-quality and lowquality prompts with similar semantics. as shown in figure 2 , the original data source is diffu-siondb @xcite , which contains unpaired prompts only. heuristically, we split the prompts into low-quality and high-quality ones according to the length of the prompts, the certain tags contained in the prompts, etc. next, we i) use blip @xcite to caption the images associated with high-quality prompts and treat the results as the corresponding low-quality prompts, as the captions are shorter and lack details; ii) use chatgpt to summarize the high-quality prompts and treat the summaries as low-quality prompts; iii) use chatgpt to generate better prompts from low-quality prompts; the results are considered high-quality prompts. 3 through the above three approaches, we obtain a large number of prompt pairs; however, the quality of these prompt pairs cannot be guaranteed. hence, we need to do further data cleaning and filtering. post-processing. we first filter out the examples that are non-english and nsfw (not safe for work). next, we filter out examples of images generated from high-quality prompts with low aesthetic scores @xcite . for the prompt pairs generated by the mentioned approaches i) and ii), we use the aesthetic score model @xcite to score the images, as diffusiondb already contains the images corresponding to the high-quality prompts. for high-quality prompts generated by the mentioned approach iii), we use the reward model r aes in section 4.2 to compute the scores. we also consider prompts' consistency, calculate the text similarity @xcite between low-quality and high-quality prompts in a pair, and filter out examples with low similarity. more details can be found in the appendix b. statistics. we finally collect 143k prompt pairs as our training set. in addition, we randomly extract 2k entries from low-quality prompts as our testing set. for the training set, the average lengths of low-quality and high-quality prompts are 40.3 and 197.8, respectively, indicating that high-quality prompts contain more descriptions of details. more statistics can be found in ## the beautifulprompt model inspired by instructgpt @xcite and chatgpt, in this section, we introduce the beau-tifulprompt training scheme in detail, which contains three stages (supervised fine-tuning, reward modeling training and reinforcement learning), as shown in figure 3 . ## industrial application in this section, we briefly discuss how our model benefits users in real-world applications. currently, we have integrated beautifulprompt into a cloudnative ai platform (platform of artificial intelligence, alibaba cloud foot_4 ) to assist users (especially designers and art workers) to create and edit artistic images based on a variety of stable diffusionstyle models, together with other modules such as lora (hu et al., 2021) and controlnet (zhang et al., 2023). users can freely perform any types of image generation and editing operations through webui. during any operation, users can invoke a beautifulprompt helper plug-in to assist the design or art creation process. in addition, based on the query per second (qps) requirements and the system workload, our inference service can automatically scale to an adjustable number of machines on gpu clusters. ## conclusion we propose a deep generative model named beau-tifulprompt to create high-quality prompts, which can be feed to stable diffusion-style models to produce more beautiful images. specifically, we collect and release a new dataset for training prompt engineering models.