DreamCatalyst: Fast and High-Quality 3D Editing via Controlling
  Editability and Identity Preservation
Authors: name: Jiwook Kim; name: Seonho Lee; name: Jaeyo Shin; name: Jiho Choi; name: Hyunjung Shim
Published: 2024-07-16 05:26:14

Abstract: Score distillation sampling (SDS) has emerged as an effective framework in
text-driven 3D editing tasks due to its inherent 3D consistency. However,
existing SDS-based 3D editing methods suffer from extensive training time and
lead to low-quality results, primarily because these methods deviate from the
sampling dynamics of diffusion models. In this paper, we propose DreamCatalyst,
a novel framework that interprets SDS-based editing as a diffusion reverse
process. Our objective function considers the sampling dynamics, thereby making
the optimization process of DreamCatalyst an approximation of the diffusion
reverse process in editing tasks. DreamCatalyst aims to reduce training time
and improve editing quality. DreamCatalyst presents two modes: (1) a faster
mode, which edits the NeRF scene in only about 25 minutes, and (2) a
high-quality mode, which produces superior results in less than 70 minutes.
Specifically, our high-quality mode outperforms current state-of-the-art NeRF
editing methods both in terms of speed and quality. See more extensive results
on our project page: https://dream-catalyst.github.io.