UltraEdit: Instruction-based Fine-Grained Image Editing at Scale
Authors: name: Haozhe Zhao; name: Xiaojian Ma; name: Liang Chen; name: Shuzheng Si; name: Rujie Wu; name: Kaikai An; name: Peiyu Yu; name: Minjia Zhang; name: Qing Li; name: Baobao Chang
Published: 2024-07-07 06:50:22

Abstract: This paper presents UltraEdit, a large-scale (approximately 4 million editing
samples), automatically generated dataset for instruction-based image editing.
Our key idea is to address the drawbacks in existing image editing datasets
like InstructPix2Pix and MagicBrush, and provide a systematic approach to
producing massive and high-quality image editing samples. UltraEdit offers
several distinct advantages: 1) It features a broader range of editing
instructions by leveraging the creativity of large language models (LLMs)
alongside in-context editing examples from human raters; 2) Its data sources
are based on real images, including photographs and artworks, which provide
greater diversity and reduced bias compared to datasets solely generated by
text-to-image models; 3) It also supports region-based editing, enhanced by
high-quality, automatically produced region annotations. Our experiments show
that canonical diffusion-based editing baselines trained on UltraEdit set new
records on MagicBrush and Emu-Edit benchmarks. Our analysis further confirms
the crucial role of real image anchors and region-based editing data. The
dataset, code, and models can be found in https://ultra-editing.github.io.