DATENeRF: Depth-Aware Text-based Editing of NeRFs
Authors: name: Sara Rojas; name: Julien Philip; name: Kai Zhang; name: Sai Bi; name: Fujun Luan; name: Bernard Ghanem; name: Kalyan Sunkavall
Published: 2024-04-06 06:48:16

Abstract: Recent advancements in diffusion models have shown remarkable proficiency in
editing 2D images based on text prompts. However, extending these techniques to
edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual
2D frames can result in inconsistencies across multiple views. Our crucial
insight is that a NeRF scene's geometry can serve as a bridge to integrate
these 2D edits. Utilizing this geometry, we employ a depth-conditioned
ControlNet to enhance the coherence of each 2D image modification. Moreover, we
introduce an inpainting approach that leverages the depth information of NeRF
scenes to distribute 2D edits across different images, ensuring robustness
against errors and resampling challenges. Our results reveal that this
methodology achieves more consistent, lifelike, and detailed edits than
existing leading methods for text-driven NeRF scene editing.