Customize your NeRF: Adaptive Source Driven 3D Scene Editing via
  Local-Global Iterative Training
Authors: name: Runze He; name: Shaofei Huang; name: Xuecheng Nie; name: Tianrui Hui; name: Luoqi Liu; name: Jiao Dai; name: Jizhong Han; name: Guanbin Li; name: Si Liu
Published: 2023-12-04 06:25:06

Abstract: In this paper, we target the adaptive source driven 3D scene editing task by
proposing a CustomNeRF model that unifies a text description or a reference
image as the editing prompt. However, obtaining desired editing results
conformed with the editing prompt is nontrivial since there exist two
significant challenges, including accurate editing of only foreground regions
and multi-view consistency given a single-view reference image. To tackle the
first challenge, we propose a Local-Global Iterative Editing (LGIE) training
scheme that alternates between foreground region editing and full-image
editing, aimed at foreground-only manipulation while preserving the background.
For the second challenge, we also design a class-guided regularization that
exploits class priors within the generation model to alleviate the
inconsistency problem among different views in image-driven editing. Extensive
experiments show that our CustomNeRF produces precise editing results under
various real scenes for both text- and image-driven settings.