Editable Image Elements for Controllable Synthesis
Authors: name: Jiteng Mu; name: Michaël Gharbi; name: Richard Zhang; name: Eli Shechtman; name: Nuno Vasconcelos; name: Xiaolong Wang; name: Taesung Park
Published: 2024-04-24 17:59:11

Abstract: Diffusion models have made significant advances in text-guided synthesis
tasks. However, editing user-provided images remains challenging, as the high
dimensional noise input space of diffusion models is not naturally suited for
image inversion or spatial editing. In this work, we propose an image
representation that promotes spatial editing of input images using a diffusion
model. Concretely, we learn to encode an input into "image elements" that can
faithfully reconstruct an input image. These elements can be intuitively edited
by a user, and are decoded by a diffusion model into realistic images. We show
the effectiveness of our representation on various image editing tasks, such as
object resizing, rearrangement, dragging, de-occlusion, removal, variation, and
image composition. Project page:
https://jitengmu.github.io/Editable_Image_Elements/