Image Sculpting: Precise Object Editing with 3D Geometry Control
Authors: name: Jiraphon Yenphraphai; name: Xichen Pan; name: Sainan Liu; name: Daniele Panozzo; name: Saining Xie
Published: 2024-01-02 18:59:35

Abstract: We present Image Sculpting, a new framework for editing 2D images by
incorporating tools from 3D geometry and graphics. This approach differs
markedly from existing methods, which are confined to 2D spaces and typically
rely on textual instructions, leading to ambiguity and limited control. Image
Sculpting converts 2D objects into 3D, enabling direct interaction with their
3D geometry. Post-editing, these objects are re-rendered into 2D, merging into
the original image to produce high-fidelity results through a coarse-to-fine
enhancement process. The framework supports precise, quantifiable, and
physically-plausible editing options such as pose editing, rotation,
translation, 3D composition, carving, and serial addition. It marks an initial
step towards combining the creative freedom of generative models with the
precision of graphics pipelines.