OmniEdit: Building Image Editing Generalist Models Through Specialist
  Supervision
Authors: name: Cong Wei; name: Zheyang Xiong; name: Weiming Ren; name: Xinrun Du; name: Ge Zhang; name: Wenhu Chen
Published: 2024-11-11 18:21:43

Abstract: Instruction-guided image editing methods have demonstrated significant
potential by training diffusion models on automatically synthesized or manually
annotated image editing pairs. However, these methods remain far from
practical, real-life applications. We identify three primary challenges
contributing to this gap. Firstly, existing models have limited editing skills
due to the biased synthesis process. Secondly, these methods are trained with
datasets with a high volume of noise and artifacts. This is due to the
application of simple filtering methods like CLIP-score. Thirdly, all these
datasets are restricted to a single low resolution and fixed aspect ratio,
limiting the versatility to handle real-world use cases. In this paper, we
present \omniedit, which is an omnipotent editor to handle seven different
image editing tasks with any aspect ratio seamlessly. Our contribution is in
four folds: (1) \omniedit is trained by utilizing the supervision from seven
different specialist models to ensure task coverage. (2) we utilize importance
sampling based on the scores provided by large multimodal models (like GPT-4o)
instead of CLIP-score to improve the data quality. (3) we propose a new editing
architecture called EditNet to greatly boost the editing success rate, (4) we
provide images with different aspect ratios to ensure that our model can handle
any image in the wild. We have curated a test set containing images of
different aspect ratios, accompanied by diverse instructions to cover different
tasks. Both automatic evaluation and human evaluations demonstrate that
\omniedit can significantly outperform all the existing models. Our code,
dataset and model will be available at
https://tiger-ai-lab.github.io/OmniEdit/