UltraEdit: Instruction-based Fine-Grained Image Editing at Scale
Authors: name: Haozhe Zhao, name: Xiaojian Ma, name: Liang Chen, name: Shuzheng Si, name: Rujie Wu, name: Kaikai An, name: Peiyu Yu, name: Minjia Zhang, name: Qing Li, name: Baobao Chang

This paper presents UltraEdit, a large-scale (approximately 4 million editing
samples), automatically generated dataset for instruction-based image editing.
Our key idea is to address the drawbacks in existing image editing datasets
like InstructPix2Pix and MagicBrush, and provide a systematic approach to
producing massive and high-quality image editing samples. UltraEdit offers
several distinct advantages: 1) It features a broader range of editing
instructions by leveraging the creativity of large language models (LLMs)
alongside in-context editing examples from human raters; 2) Its data sources
are based on real images, including photographs and artworks, which provide
greater diversity and reduced bias compared to datasets solely generated by
text-to-image models; 3) It also supports region-based editing, enhanced by
high-quality, automatically produced region annotations. Our experiments show
that canonical diffusion-based editing baselines trained on UltraEdit set new
records on MagicBrush and Emu-Edit benchmarks. Our analysis further confirms
the crucial role of real image anchors and region-based editing data. The
dataset, code, and models can be found in https://ultra-editing.github.io.