Title: Domain-Specific Text-to-Image Generation: Planning, Merging, and Replacing with Training-free LLMs
Primary area: generative models
Keywords: Diffusion Models, Domain-Specific Generation, Circuit Diagram Synthesis, Large Language Models (LLMs)

Abstract: Diffusion-based techniques, such as Stable Diffusion, exhibit remarkable capabilities in text-to-image synthesis and editing. However, general text-to-image diffusion methods frequently fail to accurately generate domain-specific components, such as particular electrical elements in schematic circuit diagram. Lacking domain-specific knowledge, rules, and sufficient data,  existing methods may struggle with resource-consumption model training. To address these limitations, we propose a novel, training-free framework for mastering domain-specific text-to-image generation, namely Planning, Merging, and Replacing (PMR).  Specifically, PMR precisely generates domain-specific elements and their configurations, enabling schematic circuit diagram generation without requiring model fine-tuning. 
Based on the establishment of a knowledge base, PMR employs large language models (LLMs) to plan inter-component connectivity according to the requirements provided by users. 
PMR further utilizes LLMs to spatially arrange symbolic blocks (representing components) and their connecting wires. Subsequently, PMR has a fine-grained positional control and generates symbolic blocks and wires at designated locations.  Extensive experiments demonstrate that PMR outperforms existing methods in domain-specific generation.
Our work opens a potentially new avenue of automated domain-specific text-to-image generation.