DreamMix: Decoupling Object Attributes for Enhanced Editability in
  Customized Image Inpainting
Authors: name: Yicheng Yang; name: Pengxiang Li; name: Lu Zhang; name: Liqian Ma; name: Ping Hu; name: Siyu Du; name: Yunzhi Zhuge; name: Xu Jia; name: Huchuan Lu
Published: 2024-11-26 08:44:47

Abstract: Subject-driven image inpainting has emerged as a popular task in image
editing alongside recent advancements in diffusion models. Previous methods
primarily focus on identity preservation but struggle to maintain the
editability of inserted objects. In response, this paper introduces DreamMix, a
diffusion-based generative model adept at inserting target objects into given
scenes at user-specified locations while concurrently enabling arbitrary
text-driven modifications to their attributes. In particular, we leverage
advanced foundational inpainting models and introduce a disentangled
local-global inpainting framework to balance precise local object insertion
with effective global visual coherence. Additionally, we propose an Attribute
Decoupling Mechanism (ADM) and a Textual Attribute Substitution (TAS) module to
improve the diversity and discriminative capability of the text-based attribute
guidance, respectively. Extensive experiments demonstrate that DreamMix
effectively balances identity preservation and attribute editability across
various application scenarios, including object insertion, attribute editing,
and small object inpainting. Our code is publicly available at
https://github.com/mycfhs/DreamMix.