Title: PhotoAgent: Exploratory Visual Aesthetic Planning with Large Vision Models
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: image enhancement, image composition; edit

Abstract: With the recent fast development of generative models, instruction-based image editing shows a great potential in generating high quality image. However, the quality of editing highly depends a carefully designed instruction, placing the burden of task decomposition and sequencing entirely on the user. To achieve autonomous image editing, we present PhotoAgent, a system that rethinks the paradigm of autonomous image editing. PhotoAgent autonomously reasons about the necessary edits, generates a robust action plan, and executes adjustments through a closed-loop process, all without requiring detailed step-by-step prompt engineering from the user. Our approach integrates large language models for intentional reasoning and dynamic planning, alongside a vision-language model for precise localized editing. This combination allows PhotoAgent to interpret high-level user instructions, decompose them into executable sub-tasks, and refine the output iteratively based on visual feedback. Extensive experiments demonstrate that PhotoAgent significantly outperforms existing methods in both instruction faithfulness and visual quality across a diverse range of editing scenarios.