DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservation
Authors: Jiwook Kim
Seonho Lee
Jaeyo Shin
Jiho Choi
Hyunjung Shim
Venue: ICLR 2025 Poster (2025)
Keywords: diffusion models
3D editing
score distillation sampling
NeRF
3D Gaussian Splatting
Primary area: generative models
TL;DR: We achieve fast and high-quality 3D editing in both NeRF and 3D Gaussian Splatting via our training objective.

Abstract: Score distillation sampling (SDS) has emerged as an effective framework in text-driven 3D editing tasks, leveraging diffusion models for 3D-consistent editing. However, existing SDS-based 3D editing methods suffer from long training times and produce low-quality results. We identify that the root cause of this performance degradation is their conflict with the sampling dynamics of diffusion models. Addressing this conflict allows us to treat SDS as a diffusion reverse process for 3D editing via sampling from data space. In contrast, existing methods naively distill the score function using diffusion models.  From these insights, we propose DreamCatalyst, a novel framework that considers these sampling dynamics in the SDS framework. Specifically, we devise the optimization process of our DreamCatalyst to approximate the diffusion reverse process in editing tasks, thereby aligning with diffusion sampling dynamics. As a result, DreamCatalyst successfully reduces training time and improves editing quality. Our method offers two modes: (1) a fast mode that edits Neural Radiance Fields (NeRF) scenes approximately 23 times faster than current state-of-the-art NeRF editing methods, and (2) a high-quality mode that produces superior results about 8 times faster than these methods. Notably, our high-quality mode outperforms current state-of-the-art NeRF editing methods in terms of both speed and quality. DreamCatalyst also surpasses the state-of-the-art 3D Gaussian Splatting (3DGS) editing methods, establishing itself as an effective and model-agnostic 3D editing solution.

Published as a conference paper at ICLR 2025
DREAMCATALYST: FAST AND HIGH-QUALITY 3D EDITING VIA CONTROLLING EDITABILITY AND IDEN- TITY PRESERVATION
Jiwook Kim", Seonho Lee", Jaeyo Shin, Jiho Choi Hyunjung Shim Graduate School of Artificial Intelligence, KAIST, Republic of Korea (to991 (eana1,,,gaaceeyrr.n. glanceyes, jaeyo shin, jihochoi,
ac. kr
Time
Batman"
".. Holk"
"a photo mun"
Time
Time
Joher"
Storm Trouper
photo a face"
"
".... Totkion Eyf
phuto of a bear statue"
polar beur"
grirzly beer""
punddo"
campuite"
"o photo of farmi"
Examples of 3D editing obtained by DreamCatake based on the given text prompt. DreamCatalyst not only aligns with the prompt high-quality but also effectively preserves the identity of scenes, achieving these edits a faster rate.
Figure I:
DreamCatalyst edits 3D scenes
ABSTRACT
Score distillation sampling (SDS) has emerged as an effective framework in text- driven 3D editing tasks, leveraging diffusion models for 3D-consistent editing. However, existing SDS-based based 3D editing methods suffer from long training times and produce low-quality results. We identify that the root cause of this perfor- mance degradation is their conflict with the sampling dynamics of diffusson mod- els. Addressing this conflict allows cess for 3D editing via sampling from data space. In contrast, existing methods naively distill the score function using diffusion models. From these insights, we propose DreamCatalyst, novel framework that considers these sampling dynam- ics in the SDS framework. Specifically, we devise the optimization process of our DreamCatalyst to approximate the diffusion reverse process in editing tasks, thereby aligning with diffusion sampling dynamics. As result, DreamCatalyst successfully reduces training time and improves editing quality. Our method of- fers two modes: (1) a fast mode that edits Neural Radiance Fields (NeRF) scenes
treat SDS
diffusion reverse pro-
EEqull contribution
See more extensive results project page:
www.aaatttalaa.aeeee .wt.st..gttute thub..

---

Published as a conference paper at ICLR 2025
approximately 23 times faster than current state-of-the NeRF editing methods, and (2) a high-quality mode that produces superior results about 8 times faster than these methods. Notably, our high-quality mode outperforms current state-of- the-art NeRF editing methods in terms of both speed and quality. DreamCatalyst also surpasses the state-of-the-ar 3D Gaussian Splatring (3DGS) editing methods, establishing itself as an effective and model-agnostic 3D editing solution.
INTRODUCTION
Neural Radiance Field (NeRF) (Mildenhall et al., 2021) and 3D Gaussian Splatting (3DGS) (Kerbl al., 2023) are widely used in recent text-driven 31) generation and editing. These tasks face challenges in data collection due to the need for images from diverse views of a 3D scene. Poole et al. (2022) address this issue by leveraging the rich priors of a large web-scale pretrained diffusion model (Rombach et al., 2022), proposing Score Distillation Sampling (SDS). It enables training parameterized models, especially NeRFs and 3DGS, without additional data collection.
While 3D scene generation has garnered substantial interest (Zhu et al., 2023; Tang et al., 2023; Wang et al., 2024), comparatively fewer studies have focused on 3D scene editing. The text-driven 3D editing task modifies a source scene to align with a target text prompt. Unlike 3D generation, 3D editing must consider not only alignment with the target text prompt but also identity preservation of the source scene. Posterior Distillation Sampling (PDS) (Koo et al., 2023) achieves this by consid- cring text-aligned editability and identity preservation through minimizing the proposed stochastic latent matching loss (Wu & De la Torre, 2023; Huberman et al., 2024).
However, we observe that PDS suffers from slow 3D editing and inferior editing quality due to its theoretical foundation in stochastic latent matching. First, the formulation of stochastic latent match- ing heavily prioritizes identity preservation over editability at low noise perturbation as Fig. 2a. This results in insufficient editing outcomes since imperceptible details are primarily generated (Choi et al., 2022) at low noise perturbation. Second, the stochastic latent matching loss conflicts with the diffusion reverse process for editing. In recent SDS-based 3D generation studies (Zhu et al., 2023; Huang et al., 2023; Lee et al., 2024), SDS is regarded as a reverse process of diffusion models by utilizing decreasing timestep sampling imitate the sampling procedure. This enables fast and high-quality 3D generation by sampling 3D contents with diffusion models. This new perspective allows us to design 3D generation via sampling rather than merely learning from score functions. However, directly imposing an approximated reverse process on PDS makes identity preservation challenging. In the early stages of editing, large noise perturbations hinder identity preservation of the source scene (Meng et al., 2021), making it challenging to balance between identity preservation and editability in editing tasks. For these reasons, the conflict between the diffusion reverse pro- cess and the stochastic latent matching loss leads to slower and inferior editing (see Appendix C.I). To avoid this conflict, PDS inevitably employs random timestep sampling, which differs from the diffusion reverse process. In short, editability is hampered at small timesteps, while the approxi- mated diffusion sampling process at large timesteps reduces identity preservation, as demonstrated in Fig. 2a. The divergence of the identity preservation coefficient& low noise perturbation leads to the conflict with editability. Additionally, the information loss of source features at large timesteps further conflicts with the diffusion reverse process. Therefore, the stochastic latent matching loss has several drawbacks in balancing between identity preservation and editability.
Even if well-designed balancing mechanism is available, there are limitations to quality improve- ment with reweighting formulation alone. It is because identity preservation and editability are trade-offs (Koo al., 2023). Modifying the model architecture is a conventional solution over- come this issue (Cao et al., 2023). Especially in SDS-based methods, many studies (Koo et al., 2023; Wang et al., 2024) finetune diffusion models to overcome the trade-offs. Specifically, several attempts have been made improve quality with Low-Rank Adaptation (LoRA) (Hu et al., 2021) 1) or Dreambooth (Ruiz et al., 2023). However, LoRA and Dreambooth require additional computation and training for the network. Since it leads to longer training times and additional memory costs, they are not suitable for achieving our main goal.
To address these issues, we propose (1) novel objective function to rebalance the weights between identity preservation and editability with respect to the level of noise perturbation. Additionally, present (2) an improved model architecture for high-quality results. For rebalancing, we in-
2

---

Published as a conference paper at ICLR 2025
troduce a general formulation of PDS by introducing new perspective of Delta Denoising Score (DDS) (Hertz et al., 2023), which is implicitly incorporated in PDS for editability. Our interpreta- tion indicates that the objective of SDS-based editing is equivalent to the single-step of denoising and renoising in the inversion-based reverse process of SDEdit (Meng et al., 2021). Moreover, we suggest two conditions for designing the objective function rebalance the weights. By integrating these conditions, we present a specialized formulation that considers the diffusion timestep roles and approximated diffusion reverse process to boost editing speed and improve quality.
Unlike the stochastic residual loss of PDS, our loss function provides more weight to identity preser- vation when timesteps are high. Meanwhile, it reduces the emphasis on identity preservation when timesteps are low. Accordingly, our loss function reweights identity preservation and editability by considering the role of diffusion timesteps for the editing task. This strategy provides two advan- tages. First, our loss ensures superior editing performance as fine details are synthesized. Second, the proposed loss reduces the conflict between SDSSbased editing and approximated diffusion re- verse process (Huang et al., 2023). This results in enhanced speed and editing quality. When the approximated diffusion reverse process is adopted to traditional methods (Hertz et al., 2023; Koo et al., 2023), significant degradation of source information occurs due strong perturbation at the early stages of training. However, our loss function mitigates the degradation of source information by reweighting, demonstrated in Fig. 2b.
Toward an improved model architecture, we introduce leveraging FreeU (Si et al., 2023) in SDS. It allows overcome the extensive computation and memory costs of LoRA and Dreambooth. FreeU does not require any time consumption and additional memory while improving quality. Moreover, FreeU enhances the editability by suppressing the high-frequency features, while preserv- ing the identity by amplifying the low-frequency features. These characteristics of FreeU harmonize with our formulation. It is because our formulation ensures identity preservation while FreeU en- hances editability without compromising identity preservation, as shown in Fig, 1.
We evaluate our method through qualitative and quantitative comparisons and user studies with baseline methods. Our results demonstrate that the proposed method outperforms the baselines in both editing speed and quality. Moreover, our method shows outstanding results on both NeRF and 3DGS. These results indicate DreamCatalyst is model-agnostic 3D editing framework, rather than a method specifically tailored to a particular 3D representation (Chen et al., 2024b). To the best of our knowledge, DreamCatalyst is the first to achieve tttee-oftut of-tthe-art art results and provide extensive experiments on both NeRF and 3DGS editing. In summary, our key contributions are as follows:
We suggest a general formulation for 3D editing by introducing a new interpretation of DDS as a reverse process of SDEdit. We present two conditions for designing objective function and specialized formulation, which enable fast and high-quality 3D editing via controlling identity preservation and editability both NeRF and 3DGS.
We first introduce using FreeU for 3D editing to enhance editability to overcome the trade- offs inherent in reweighting terms of the formulation for editing objectives.
2 PRELIMINARIES
2.1 DIFFUSION MODELS
A diffusion model (Song& & Ermon, 2019; Ho et al., 2020) consists of a forward process that grad- ually perturbs data point xp with noise and a reverse process that progressively denoises the noisy data. The forward process is defined as 2, = Vājzn +I- ăjc, E (0 I), where A/O0, Gaussian distribution, í [0, T] denotes the timestep, and æ; represents perturbed æn t. as, and a, := Ii O₂ are predefined noise scheduling coefficients. In contrast, the reverse process utilizes the score function, which is predicted with neural network, and a sampler. The score func- tion is equivalent to the denoising network ca, parameterized by 6, of diffusion model. It trained via denoising score matching as ming L(xv) = Er,elll |Il cø( t) e [1ž].
Denoising Diffusion Implicit Model (DDIM). In the context of DDIM (Song et al., 2020a), the reverse process can be expressed with Tweedie's formula (Efron, 2011) as
Lt-1 v Va + /I āt-1ē,
(1)
3

---

Published as a conference paper at ICLR 2025
T PDS
V PDS
10
Magnitudes
2
Magnitudes
300
1000
0
200
400
800
800
1,000
Timesteps
Timesteps (t)
(a) PDS and y PD8 magnitudes over timesteps
(b) 4- and V magnitudes over timesteps
Figure 2: Comparison of coefficients between PDS and our method across different timesteps. We plot the weighting functions of PDS and DreamCatalyst in (a) and (b), respectively. &DDS and PPS indicate the coefficient of identity preservation and editability of PDS. d and ur* are the coefficient of identity preservation and editability of ours, respectively. V and V3 are for extra special cases.
and ē V a 1 E mB
where Rujt = (a, -la.-v1-a-eet (x,e(,))
V1 04-1
Here, Z is predicted zo with æ,, and is noise term consisting of deterministic term and stochastic term (N, I). The deterministic sampling is achieved when 113, 0, as the stochas- ticity of noise term can be manipulated with the stochastic property controlling a hyperparameter y and a coefficient 3, v(1 a4-1)/(1 - a.) -a.)v- a1/a1-1-
2.2 SCORE DISTILLATION SAMPLING
In this section, review previous SDS methods that utilize pre-traines diffusion models as priors. Unlike the diffusion models that perform sampling in image space, FreemFusion (Poole et al., 2022) proposes the SDS framework that conducts sampling in the parameter space. SDS optimizes param- eterized models, such as differentiable image generators like NeRF and 3DGS, using the diffusion training objective. In 3D generation, SDS perturbs rendered images is a NeRF or 3DGS model, v are parameters of the model g, and c is a camera parameter. It is then distilled from pre-trained diffusion model with rich 2D priors to train NeRF or 3DGS. The training objective is defined as follows:
g( e) with noise e, where
min LSDS (gg = g(v, e)) = B.,elll o (F,, - 112).
(3)
where the predicted noise with classifier free guidance (CFG) is (x v. ca (x Va, + (Ea(zz, y, t) Eo (Ze, ye t)) (Ho & Salimans, 2022), Wy indicates the scale of text sext-ggiiance, is text prompt, and ye null-text. Partieularly, SDS omits the U-Net Jacobian term for computation
efficiency V LsDs (æo g( e)) = E,, (( (at, v, <) Omo
Even though SDS is effective framework for 3D generation, SDS suffers from editing tasks since SDS does not reflect identity preservation.
Delta Denoising Score (DDS). The editing task consists of key aspects: (1) preserving the source content's identity and (2) aligning with the target text prompt. Since the objective of SDS is designed for the generation task, it struggles to preserve the source identity. To address this, DDS is proposed to preserve the source identity by modifying the SDS loss function, as defined below:
L g(v,, c)) = Ill eğ (x= "0 ,t) E (x="" " 1) 1121.
(4)
where 2 and Th are the perturbed target data a and source data a at timestep t with random noise e, and /e and g are the target prompt and source prompt, respectively. The estimated noise of a:!e becomes pivot for maintaining the source identity.
4

---

Published as a conference paper at ICLR 2025
Freeli
UNeC
3D Scene
Source Image
Perturted Source Image
Predicted Nolsee:
Bazman
Freei)
Image
Image
Figure 3: Overall architecture. DreamCatalyst approximates inversion-based SDEdit with DDS loss and identity regularizer Riden- Furthermore, DreamCatalyst utilizes FreeU to enhance 3D edit- ing quality without additional computational cost and memory usage.
Posterior Distillation Sampling. We focus SDS-based editing, direct 31) editing method that achieves 31D-consistent results. contrast to Instruct-Ner22eRR (IN2N) (Haque et al., 2023), which edits source scenes in 2D space. While DDS shows remarkable editing in 2D images, suffers from quality degradation in 3D editing. This degradation occurs because 3D editing requires stronger identity preservation than 2D editing. However, DDS optimizes NeRF by minimizing the residual between the SDS loss of the source and target without any additional regularization for identity preservation. The absence regularization leads to deviation from the source. To address this, PDS introduces a stochastic latent matching loss to add an explicit identity preservation term DDS loss. The stochastic latent 2t, which contains the structural details of æo. calculated as (Vā,-1(L
ze (xo. y) = (xI-1 (11, f))/o(, o.. where o; == i-a, (1 ac) and μa (Z ai Dooe + Vor(1 ā.-1) // āz). Therefore, the stochastic latent matching loss follows:
L g( c)) Er,([l =,(",,", "/ 2x , "0
(5)
By ignoring the U-Net Jacobian term as done in SDS, the gradient of stochastic latent matching loss Lpps is written as
Oara Orp
V CPDS = [E,ee(= ()t)) ) zi (xe )))
(6)
Oaz tet
=E,,e P(D (()( 2 +
V PDS (L)
(a 8
J.
Oth
Identity preservation
gradient of Caus
g (t) and (t) are the defined coefficients as a function of the timestep t, each representing the coefficient of identity preservation and that of editability, respectively. The gradient of stochas- tic latent matching is equivalent to Equation 7, which means the PDS loss implicitly involves the explicit identity preservation term and DDS gradient term for editing.
DREAMCATALYST
3.1
MOTIVATION
We aim to design an objective function, like PDS, possessing two important properties. (1) It should include an explicit term for strong identity preservation. (2) It should align with the roles of diffusion timesteps and reduce the conflict with the approximated diffusion reverse sampling. To achieve this goal, identity preservation has to be stressed in large noise perturbation and does not diverge in small levels of perturbation by reweighting each term of Equation 7. However, the nature of the formula- tion of stochastic latent matching implicitly includes an identity preservation term and the gradient of the DDS loss, making it incapable directly adjusting the coefficients. Therefore, we provide a new interpretation of DDS and introduce general formulation of PDS through this perspective

---

Published as a conference paper at ICLR 2025
reweigh the terms. Furthermore, we propose a specialized formulation that aligns with the diffusion timestep roles and supports the diffusion reverse process. The specialized formulation has mainly two advantages: (1) our formulation leads to fine-detailed 3D edited results by considering diffusion timestep roles and (2) it immensely reduces training time with the diffusion reverse process.
3.2 GENERAL FORMULATION OF PDS
In this section, unveil the relationship between the reverse SDEdit process and DDS (see Ap- pendix A for preliminaries on SDEdit). The key insight of DreamCatalyst is that the objective of DDS equivalent the single-step DDIM-based SDEdit sampling. The reverse SDEdit process en- ables stochastic editing by solving the stochastic differential equations (SDEs) (Song et al., 2020b) with random sampled noise as 1t-1 = v + vI âE NN I). However, recent editing studies (Tumanyan et al., 2023; Cao et al., 2023) utilize the DDIM inversion to preserve the source identity. By combining the SDEdit and DDIM inversion for identity preservation, the DDIM-basee based SDEdit sampling is defined as
カ-1 V V + Vi-á--1ć
(8)
where x indicates the rendered image edit. Especially when we define r/8, 0 for deterministic sampling, the noise is as é = Eo (2 12 ynm 1). In this case, DDIM inversion-based perturbed image is defined as xt varah 1st + v1 ā, (t), which is the result of applying single forward step. We can rewrite the Equation with Equation 2 and the forward step as follows:
1st
/
VI
at
C (0 1 ,,tt)) + /1 ā
(9)
Vāg
Vac
VT-
=Vār-1 Vā,
y sC t) Eg (x
(x . ()) +
(10)
Vāx
Although the single-step denoising process of SDEdit is clear in the diffusion process with Equa- tion 8, inspired by DreamSampler (Kim et al., 2024), we can interpret the process as an optimization problem as follows:
V V V ā
(11)
vI vI-ai αt || (x)
Φ = arg min
1gt
1st 1| arg min 20
1ot stt,t) Ex (a " 1|2 (12)
Var
Equation 12 indicates that the DDS objective is equivalent to the objective of the optimization prob- lem, when æ g11 c). Thus, solving the DDS objective ensures equivalence the single-step process of SDEdit. By extension, optimizing the DDS objective with decreasing timestep sampling corresponds to the reverse SDEdit process. We notice that the proposed inversion is proximal inver- sion. Conventional DDIM-inversion calculates é with a multi-step inversion for pivoting. However, this multi-step inversion method requires extensive calculations for each multi-view image in 3D editing. To mitigate the computational burden, our method samples a single-step é for each level of noise perturbation, which enables the proximal inversion due to different é with respect to (. More- over, we stress that DreamSampler assumes 2 x despite introducing a similar observation. This assumption differs from the formulation presented in DDS (Hertz et al., 2023). Therefore, under this assumption, the general DDS objective presented in Equation cannot interpreted as an optimization problem. In contrast, we provide more general interpretation of DDS that is not limited by this assumption, offering a broader perspective compared to DreamSampler.
Since we express the DDS objective as the optimization problem in Equation 12, we can ap- ply additional regularization terms for enforcing identity preservation. These regularization terms allow supplementary guidance during the reverse SDEdit process, addressing the lack of identity preservation in DDS. Therefore, we propose the general formulation of PDS by adding the identity preservation regularizer Riten the DDS objective. That is,
(x) 1gt = =(( c)) = E., [ + M (tD)s where Riden =1 x
(13)
2
and (t) and (t) are weighting functions of the general formulation. Herein, the regularizer Riaen preserves the identity and DDS loss Lovs takes charge of editing. By understanding the explicit role of each term in loss function, we can easily control the formulation of PDS with Equation 13. With this controllable, generalized formulation of PDS, we then develop a specialized formulation that accounts for the sampling dynamics of diffusion models in the following section.

---

Published as a conference paper at ICLR 2025
3.3 DIFFUSION-PRIENDII
EDITING
Since our generalized PDS formula allows for explicit control of each term at every timestep, we can fully leverage the controllability align the 3D editing process with the sampling dynamics of diffusion models. Specifically, we propose a specialized formulation of Equation 13, which considers the roles of diffusion timestep and their aligmment with the reverse SDEdit process. The design choice of formulation in DreamCatalys aims to satisfy two conditions: (1) strong identity preservation large timesteps and (2) reducing identity preservation in small timesteps. The first condition, strong identity preservation in large timesteps, reduces the information loss of source features in high levels noise perturbation. This condition enables the preservation of the source features in the early diffusion reverse process. The second condition, weak identity preservation in small timesteps, leads to synthesizing fine details for 3D editing as the role of diffusion. The proposed specialized formulation of DreamCatalyst, which satisfies two conditions, is as follows:
DDraanCallly (z)) = =(b,, = E (t) + VV (t)
(14)
T5 (1(1) = xe"T", VV' = 8 + V L/T,
(15)
and X. 8, are hyperparameters. respectively. We set X 0.075, 5 = 0.2, and γ = 0.8 for all experiments. As shown in Fig. 2b, the formulation of DreamCatalyst fulfills the two conditions, thereby our objective function is suitable for editing tasks and diffusion reverse process. The pri- mary difference between DreamCatalyst and PDS lies in the formulation of (() and (t). This modification reduces editing time in two key ways: (I) the reweighting scheme enables the use of the approximated diffusion reverse process, familiar to the diffusion sampling procedure; (2) DreamCatalyst avoids inefficient distillation. In PDS, distillation at small timesteps leads to cessive identity preservation, which disrupts editing. In contrast, our weighting facilitates efficient distillation without interrupting the editing process. These factors significantly reduce the number of optimization steps, thereby decreasing the overall editing time.
The SDEdit process with minimizing DDreanCaalya requires diffusion reverse process-likely timestep sampling. To achieve this, we adopt decreasing timestep sampling, which uniformly samples timestep T + 1. While the non-increasing timestep sampling (Huang et al., 2023) is also a good option, we employ decreasing timestep sampling for fulfilling Equation 12 in all timesteps. Our specialized objective function with decreasing timestep sampling enables the SDEdit process with a parameterized model, especially NeRF and 3DGS in this paper. The overall framework of DreamCatalyst is illustrated in Fig. 3. Initially, L is sampled with decreasing timestep sampling- Subsequently, both the rendered and source images are perturbed according to l. Equation 14 is then computed with each perturbed image. Finally, the 3D model is optimized by the loss function. We omit the U-Net Jacobian term as previous works to calculate the gradient of DDreamCatalyat.
We demonstrate that fulfilling the two conditions enables effective 3D editing as Fig. 1. Dream- Catalyst not only achieves high-quality 3D scene edits across various scenarios but also performs editing faster than existing methods. Besides, we present two more special cases and show that the coefficients of specialized formulation remain robust long as the two conditions Fig. 2b. For an intuitive comparison, we fix th and vary only Vr", setting the two additional cases V (1) and (t) - 0.21/T. In section 4, we demonstrate that these special cases, which satisfy the two conditions, also surpass the existing sfate-of-the-art baselines. Thus, fulfilling two conditions enhances editing speed and quality.
satisfied in
3.4 ENHANCING EDITABILITY WITH FREEU
For an improved architecture, we introduce utilizing FreeU in 3D editing to enhance editability with- out additional memory usage and computational costs. FreeU suppresses high-frequency features by scaling up the backbone features, which contain a large amount of low-frequency information (Si et al., 2023). The amplifying backbone features stresses low-frequency features, thereby relatively reducing the impact of high-frequency features. Consequently, suppressing high-frequency features increases editability as sharp characteristics of high-- freeteerreuecyy features are smoothed as the edge fea- tures are weakened. Moreover, identity preservation, corresponding to the low-frequency domain, is maintained by amplifying backbone features. In conclusion, FreeU enhances editability without compromising identity preservation. In addition, adopting FreeU instead of Dreambooth and LoRA removes additional module computations. Consequently, integrating FreeU reduces editing time without incurring extra computational overhead.
7

---

Published as a conference paper at ICLR 2025
"a photo of a person" + clown"
"a photo of face"
skull"
"a photo of a plant" + tulip above the flowerpot with soil"
"a photo of a bear statue"
asiatic black bear"
(a) Source
(b) IN2N
(c) PDS
(d) Ours
Figure 4: Qualitative comparison with baseline methods NeRF scenes. We provide visual editing results NeRF scenes for each baseline method and DreamCatalyst. DreamCatalyst demon- strates more photorealistic editability while preserving the identity of source scenes such struc- tures and backgrounds.
EXPERIMENTS
We conduct experiments real scenes using datasets from IN2N and PDS. The types of scenes include sitting person, a full-body person, face, objects, and outdoor scenes. We evaluate our method and baselines eight scenes with 40 pairs of source and target prompts. For compar- isons, we evaluate our method against state-of-the-ard baselines on NeRF scenes: IN2N and PDS, as well as on 3DGS scenes: PDS, GaussianEditor (Chen et al., 2024b) and DGE (Chen et al., 2024a). Furthermore, we compare two modes of DreamCatalyst: (1) high-quality mode, and (2) a fast mode, which requires fewer training iterations than the high-quality mode. Additionally, we present an ablation study addressing how FreeU affects DreamCatalyst. Unless explicitly stated as using V (t) or (t). all results presented from our method (in both NeRF and 3DGS scenes) are based on the use of the default V (t) coefficient.
4.1 QUALITATIVE EVALUATION
In Fig. 4, we present qualitative comparison with the baseline methods on NeRF scenes, focusing on both identity preservation and editing effectiveness. DreamCatalyst consistently generates more detailed and photorealistic results compared to the baseline methods (e.g.. tulips generated by other methods appear blurred and lack fine details, compromising the edit). Furthermore, while other methods result in blurry and over-saturated backgrounds, DreamCatalyst preserves the identity of the source scenes by maintaining background details and overall structure. Although PDS aligns well with the target text prompt in terms of subject editing, it struggles with identity preservation.
8

---

Published as a conference paper at ICLR 2025
Quantitative comparison on NeRF Ours outperforms the baseline methods on NeRF editing. Bold represents the best result, and underline indicates the second-best result.
Table
10
scenes.
Image Siillrrtty
CLIP-Direc (f)
CLIP-Img
Aesthetic (1)
Method
(t)
(min, 4)
0.722
PDS
0.161
0.687
5.437
580
Ours (list)
0.746
5.557
Ours
0.180
0,746
$.688
~0
Oursifast)
Ours (w/ V:
0,178
0.745
5.659
~070
IN2N
Ours (w/ 455)
0.178
0.749
5.659
PDS
-5
165
cic
Cit
025
030
CLIP Directional Similarity
Table 2:
Quantitative comparison on 3DGS scenes. Bold represents the best result in each category, and underline indicates the second-best result among the model-specific methods.
Figure 5: Scatter plot comparing our method against baseline methods on NeRF scenes. The plot shows NeRF editing per- formance on CLIP directional similarity and CLIP image similarity for baseline methods: IN2N, PDS, and our method, including the
CLIP-Direc
CLIP-Ing
Aesthetic
Category
Method
(t)
Model-Agmottic
PD35
0.108
0.627
4,77
Ouns
0.171
0.724
5,336
fast training mode (fast). Trend lines are
GE (DDS)
0.150
0.593
4,860
(IN2N)
fitted using linear regression. Shaded areas
Speel-Specific
DGE
0.154
0.758
5.517
around the trend lines indicate the 95% con-
GE (Qurs)
0.152
0.773
5.588
GE (fursefast)
0.143
0.777
5.541
fidence intervals.
Specifically, the backgrounds in PDS results are particularly prone to over- saturaattation or unrealistic color shifts. This highlights a key limitation of PDS, as it tends to prioritize the edit at the expense of maintaining the original scene's identity. In contrast, DreamCatalyst achieves better balance editability and identity preservation, resulting in superior overall performance. Qualitative compar- isons on 3DGS scenes are provieded Appendix C.3.
4.2 QUANTITATIVE EVALUATION
We evaluate DreamCatalyst and baseline methods using three key metries: CLIP directional similar- ity (Patashnik et al., 2021), CLIP image similarity, and aesthetic score (Schuhmann, 2022). CLIP di- rectional similarity measures image-text alignment, CL.PP image similarity assesses identity preser- vation, and the aesthetic score reflects the visual quality of the edits. As shown in Tab. 1 and Fig. 5, DreamCatalyst achieves the highest scores across all metrics on NeRF editing. This is particularly notable, other baseline methods, such as PDS, tend to excel in one area-such as high CLIP directional similarity or aesthetic score- but struggle in others, particularly in maintaining identity preservation. DreamCatalyst, however, strikes a balance across all three metrics, generating edits that are both photorealistic and faithful to the source scene.
Additionally, measure the editing time for each method. For fair comparison, all methods evaluated at the same resolution. Dream Catalyst with a fast mode is approximately 23 times faster than PDS, and the high-quality mode about eight times faster than PDS. Despite IN2N performing edits in 2D space, which requires less time than direct 3D editing methods, DreamCatalyst is still 1.85x faster than IN2N, even in the high-quality mode. Moreover, as shown in Tab. 1, the results from our method using V and V consistently outperform the baselines across all metrics.
In the 3DGS setting, we first compare DreamCatalyst with baseline model-agnose editing method, which are distinct from 3DGS-specifis approaches. Further details of model-agnostie and model- specific methods are in Appendix B.2. As demonstrated in Tab. 2, DreamCatalyst achieves the highest scores across most metrics on 3DGS editing. This indicates that our approach is an effec- tive model-agnostic editing method applicable to both NeRF and 3DGS scenes. Additionally, inte- grating DreamCatalyst with GaussianEdiior (GE) yields state-of-the-arr results, demonstrating that DreamCatalyst enhances the performance of 3DGS Seeecfscc methods. Furthermore, the fast mode of GE with DreamCatlayst not only outperforms the baselines but also achieves training speedup of 1.25x compared to the vanilla GE. We emphasize that as better 3DGS baseline architectures using score distillation emerge, our approach be applied to further improve their performance.

---

Published as a conference paper at ICLR 2025
Table User studies. We conduct user stud- ies to measure human preference across three criteria. Our method is more preferred than other baselines. Bold indicates the best re- sult.
Table 4: Ablation on FreeU. Quantitative ablation
study the effects of FreeU. Using FreeU with setting of 1.1 achieves a balanced trade-off be-
tween editability and identity preservation in the gen- erated results. Bold indicates the best result.
Prompt Aligament (f)
Overall Quality
Identity Preservation
CLIF-Diree (t)
CLIP-Ing
Aesthetic (?)
Method
1.0 (w/o FreeU)
0.171
0,744
5.564
IN2N
19.13%
20.08%
20.05%
1.1 (ours)
0.180
0.746
5.688
PDS
58%
19.2155
20.21%
1.3
0.183
0.710
5.624
58.29%
60.71%
59.74%
(a) Source
(b) 1.1 (ours)
(c) = 1.3 (over-editing)
(d) w/o FreeU
Figure 6: Qualitative ablation FreeU. The first row shows the ablation of FreeU with the text prompt "Turn him into Batman." " The use of FreeU enhances editability, as demonstrated by the Batman logo (green bounding box) and instances of over-editing (red bounding box).
4.3
USER STUDY
We conduct user studies, as in Tab. 3, because the metrics assessing 2D images are insufficient for evaluating 3D scenes. Participants were asked select the best video from the baselines and DreamCatalyst based on 15 text prompts, evaluated across three criteria: (1) prompt alignment, (2) overall quality, and (3) identity preservation. To gather human preference data, utilized Amazon Mechanical Turk to survey 100 participants. As result, DreanCatalyst is preferred over the baselines by a large margin across all criteria, receiving nearly three times as many selections compared to other models. Further details about the user studies are provided in Appendix E.3.
4.4 ABLATIONS
We demonstrate the effectiveness FreeU in our method with qualitative and quantitative compar- isons. FreeU modifies the scale of upsampling features in U-Net decoder using parameter b. In this framework, increasing the value of b leads to the suppression of high-frequency components in image. We hypothesize that this characteristic of FreeU facilitates easier editing. However, if b is set too high, the editing process becomes excessively easy. This hypothesis is supported by the results shown Fig. 6 (b) and (c), where the use of FreeU with b 1.3 results in excessive editing. This over-editing is not confined to the primary subject but extends to the background as well. While increasing b FreeU can enhance the editing process, excessive suppression of high- frequency components
lead to overly smooth results and unintended editing artifacts. As shown Tab. 4, despite CLIP directional similarity increases as b increases, the values of the other metrics are decreased in b 1.3. Thus, FreeU with setting of b 1.1 achieves balanced results.
CONCLUSION
We propose general formulation for 3D editing by unveiling the relationship between the reverse SDEdit process and DDS. Based on this formulation, we introduce DreamCatalyst, which considers reverse the dynamics of diffusion process to edit 3D scenes with SDS-bascr approach SDEdit process. Moreover, we suggest using FreeU in Score Distillation to overcome the trade-offs between editability and identity preservation inherent in the formulation. Consequently, DreamCata- lyst achieves fast and high-quality 3D editing on both NeRF and 3DGS scenes. Through comparative analysis and user studies, we demonstrate that DreamCatalyst surpasses state-of-the-arr methods in both performance and editing speed.
10

---

Published as a conference paper at ICLR 2025
Acknowledgements
This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (NRF-2022R A2C30111141,,4 RS-2023-00219019)
and the Ministry Education (No. R5-2024-00348336,, Institute of Information & communica- tions Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2019-II) 90075, Artificial Intelligence Graduate School Frogram(KASST, No. 2021-0-02068, Artificial Intelligence Innovation Hub, and No. RS-2024-00457882 National Al Research Lab Project), and KEIT grant funded by the Korean government (MOTIE) (No. 2022-0-00680, No. 2022-0-01045).
Ethics Statement
There have been several advancements in text-driven 3D scene editing, yet few methods (Koo al., 2023) focus on altering the formulation of score distillation sampling to preserve the identity of the original scene. Our method enhances control over the degree of identity preservation, allowing for more careful and targeted edits that prevent excessive deviation from the source scene. This controllability particularly useful in applications where the integrity of the original scene must be maintained.
However, the ability to precisely manipulate 3D) content brings with significant ethical responsibil- ities. We emphasize the need for robust ethical guidelines and regulations to prevent the misuse of this technology that can violate human rights. The potential for misuse underscores the importance of ongoing discussions about the ethical frameworks.
Reproducibility Statement
Please refer to our official repository: DreanCatalyst. This repository provides the source code, setup, and datasets used in the experiments.
pectss:/ggithie cwm/kaistttteoco.cc wtrkkisst--a cvml
REFERENCES
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-18402, 2023.
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Mas- actrl: Tuning-free free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 22560-22570, 2023.
Minghao Chen, Iro Laina, and Andrea Vedaldi. Dge: Direct gaussian 3d editing by consistent multi-view editing. arXiv preprint arXiv:2404.1892, 2024a.
Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21476-21485, 2024b.
Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1472-11181, 2022.
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Associa- tion, 106(496):1602-1614, 2011.
Rinon Gal, Or Patashnik, Haggai Maron, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. Stylegan-nada: Clip-guided domain adaptation of image generators. ACM Transactions on Graphics (TOG), 41(4):1-13, 2022.
Yuan-Chen Guo, Ying-Tue Tian Liu, Ruizhi Shao, Christian Laforte, Vikram Voleti, Guan Luo, Chia- Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Sang-Hai Zhang. threestudio: A unified

---

Published as a conference paper at ICLR 2025
framework for 3d content generation. http://////...a..oco/cceeeeeeear threestudio, 2023.
Ayaan Haque, Matthew Tancik, Alexei A Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerff: Editing 3d scenes with instructions. In Proceedings of the IEEE/CVE In- lernational Conference on Computer Vision, pp. 19740-19750, 2023.
Amir Hertz, Kfrr Aberman, and Daniel Cohen-Or. Delta denoising score. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2328-2331 2237, 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021.
Jonathan Ho and Tim Salimans. arXiv:2207. 125988 2022.
Classifier-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusson probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.0968 2021.
Yukun Huang. Jianan Wang, Yukai Shi. Boshi Tang, Xianbiao Qi, and Lei Zhang. Dreamtime: An improved optimization strategy for diffusion-guudde 3d generation. In The Twelfth International Conference on Learning Representations, 2023.
Inbar Huberman, Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: In- version and manipulations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12469-12478, 2024.
Oren Katzir, Or Patashnik, Daniel Cohen-Or, and Dani Lischinski. Noise-free score distillation. arXiv preprint arXiv:2310. 17590, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL
inria www.ff/gwtp////aaae wttp:///d...aaii.nnnaaatt..taaaa
Jeongsol Kim, Geon Yeong Park, and Jong Chul Ye. Dreamsampler: Unifying diffusion sampling and score distillation for image manipulation. arXiv preprint arXiv:2403.114155 2024.
Subin Kim, Kyungmin Lee, June Suk Choi, Jongheon Jeong, Kihyuk Sohn, and Jinwoo Shin. Col- laborative score distillation for consistent visual synthesis. arXiv preprint arXiv:2307.04787, 2023.
Juil Koo, Chanho Park, and Minhyuk Sung. arXiv:2311.13831, 3, 2023.
Posterior distillation sampling.
arXiv preprint
Kyungmin Lee, Kihyuk Sohn, and Jinwoo Shin. Dreamflow: High-quality text-to-3d generation by approximating probability flow. arXiv preprint 2xXv:22433.14966 2024.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.0100 01007, 2021.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-11)) 106, 2021.
OpenAl. Gpt-4 technical report. arxiv 2303.08774. View Article, 2(5), 2023.
Francesco Palandra, Andrea Sanchietti, Daniele Baieri, and Emanuele Rodolà. Gsedit: Efficient text-guided editing of 3d objects via gaussian splatting. arXiv preprint S00i::2003.055::5 2024.
12

---

Published as a conference paper at ICLR 2025
Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text- driven manipulation of stylegan imagery. Proceedings of the IEEE/CVF international confer- 2021.
on computer vision, pp.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprini arXiv:2209. 14988, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 22500- 22510, 2023.
Christoph Schuhmann. Laion-aestheties. LAION. AI, Aug, 2022.
Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. arXiv preprint arXiv:2309. 1497, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint DrXiv:2011.13456 2020b.
Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, et al. Nerfstudio: A modular frame- work for neural radiance field development. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1-12, 2023.
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. In The Twelfth International Conference on Learning Representations, 2023.
Narck Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to- to-image translation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 1921-1930, 2023.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High- figh-fidelitty and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36, 2024.
Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero- shot image editing and guidance. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 7388-7387, 2023.
Junzhe Zhu, Peiye Zhuang, and Sanmi Koyejo. Hifa: High-fidelity text-to-3d generation with ad- vanced diffusion guidance. arXiv preprini arXiv:2305. 18766, 2023.
13

---

Published as a conference paper at ICLR 2025
A PRELIMINARIES
SDEdit. SDEdit (Meng et al., 2021) is proposed to edit the images with the trained score network (i.e., denoising network Eo in diffusion models) by solving the stochastic differential equations (SDEs) (Song et al., 2020b), SDEdit moves the data point sequentially by leveraging the estimated score function V p(x,) 2 ( 1) with the reverse Variance Preserving (VP) SDE:
Ft-1 1 + -V- ā- (O,
(16)
where À is a scaling factor. Iteratively shifting the data point with the reverse VP-SDE synthesizes the edited image.
B IMPLEMENTATION DETAILS
B. NERF EDITING
We utilize Nerfstudio (Tancik et al., 2023) for experiments. We train the nerfacto model from Nerfstudio for initialization with source scenes. For fair comparison, we set different training steps for each method: 3,000 iterations for our method, 1,000 iterations for our method with the fast mode, 15,000 iterations for IN2N, and 30,000 iterations for PDS, following the original settings of their respective baselines. Based on previous researches (Poole et al., 2022; Katzir et al., 2023) indicating that higher classifier guidance may lead to over-saturated and poor edited results, we select a weight of 7.5 for classifier-free guidance in our method. During editing, we train the model using the Adam optimizer and an exponential decay learning rate scheduler. Specifically, since our method requires fewer iterations than PDS, we set smaller warm-up steps of learning rate schedulers: 100 for proposal networks and fields, and 300 for camera optimizers. We use a DDIM scheduler with 500 inference steps.
We employ InstruetPix2Pix (IP2P) (Brooks et al., 2023) as a pretrained diffusion model, which takes the source image input to incorporate prior of the source. For this reason, method does not require additional training or fine-tuning, in contrast to PDS, which necessitates fine-tuning diffusion models with DreamBooth (Ruiz al., 2023) in advance. Additionally, we do not apply the refinement stage on PDS for an impartial comparison. All experiments are conducted on a single NVIDIA A6000 GPU.
B.2 3DGS EDITING
For a fair comparison in 3DGS editing, we classify the baseline methods into two categories: model-agnostic and model-specific approaches. Model-specific methods, such as Gaussian Editor (GE) (Chen et al., 2024b) and DGE (Chen et al., 2024a), are designed for 3DGS scene editing and are not directly applicable to NeRF editing. Specifically, GE introduces semantic tracing to enable continuous tracking of Gaussian semantic labels, dynamically constraining the editing regions. Ad- ditionally, it implements hierarchical Gaussian splatting, which categorizes Gaussians into different generations mitigate the effects of stochastic generative guidance. DGE performs 3DGS editing by applying key-view edits in 2D space and propagating the edited features across other views using epipolar constraints. This method involves iterative dataset updates, where the 3D scene is edited based on the 2D-edited frames. However, the techniques proposed in DGE are designed for explicit 3D models and cannot be applied to NeRF scenes. In contrast, model-agnostic approaches like PDS and DreamCatalyst can be applied across different 3D editing frameworks including NeRF and 3DGS. These methods modify score distillation sampling formulations or diffusion architectures, making them independent of the specific 3D model used.
We utilize two frameworks for a fair comparison in 3DGS editing within each category: Nerfs- tudio for model-ag methods and Threestudio (Guo et al., 2023) for model-specife methods. This selection is based on the fact that PDS is based on Nerfstudio, while GE and DGE employ Threestudio. For model-agnostic methods, we train the splat facto model from Nerfstudio for initialization with source scenes, following the setup in the PDS paper. We set 3,000 training steps for our method and 30,000 steps for PDS, adhering to the PDS baseline settings. For model-specific methods, we choose 1,500 steps for our method with GE high-quality mode and 1,200 steps in fast mode. Additionally, we set 500 steps and 1,500 steps for DGE and GE respectively, following
14

---

Published as a conference paper at ICLR 2025
the original settings of their baselines. As detailed in Appendix B. for NeRF Editing, same small warm-up steps for the learning rate schedulers for Gaussian parameters Nerfstudio, including centers, spherical harmonics, opacity, scaling, and rotation. These parameters, along with camera parameters, are trained for a maximum of 3,000 steps. In the comparison of model-specific methods using Threestudio, we apply the same learning rate scheduler settings for Gaussian pa- rameters. Additionally, to prevent over-densification in GE, we limit densification phases to 1 1,200 iterations the high-quality mode of our method, ensuring that excessive cloning or splitting does not occur at the end of the editing process. All other parameters follow the original GE settings, and the experiments are conducted on a single NVIDIA A6000 GPU.
the
B.3 TEXT-GUIDANCE IN DREAMCATALYST
In DreamCatalyst, we employed InstructPix2Pix (IP2P), which is prevalently used in NeRF and 3D Gaussian Splatting editing (Kim al., 2023; Palandra et al., 2024), for instructive editing. The guidance of IP2P is composed of image and text conditioning. DreamCatalyst sets wy = 0 for Eg (arte y 1) as Collaborative Score Distillation (CSD) (Kim et al., 2023), because contents in target and source prompts are often intersected. This setting prevents interruption in guidance toward the intersected contents. The image and text-guided noise prediction is calculated as follows:
G (xx lst t) co(x, Ve t) + wy (Es(
(at y x .t) Ey (xx Ye xxttt))))))))))),
twi (x) tet V0 t) cs (a Ye. to. t)), c V "0 .t) cσ(x;²" Vo t) + wi (== ya, "hh ,t) co(a ye, Ho t)), where W/ is a scale of umage-guidance and z and ih are embeddings of source image and null-image, respectively.
(17)
(18)
C ADDITIONAL QUALITATIVE EVALUATION
C.I COMPARISON WITH PDS USING DIFFUSION REVERSE PROCESS
As shown in Fig. 8, we compare method with PDS when using an approximated diffusion reverse process with decreasing timestep scheduling in NeRF scenes. PDS, originally designed with random timestep sampling, performs poorly in this setting, leading significant loss of fine details and identity features.
For instance, in the first row of Fig. 8, the model generates highly distorted figure with blurred outlines and missing structural detail in PDS. However, our method preserves the subject's physical characteristics. The same behavior observed in the second and third rows, where the objects generated by PDS appear more synthetic, lacking texture and natural form. Specifically, in the case of the third row, PDS loses the facial structure of the source image and produces an unrealistic outcomes.
This comparison underscores limitation of applying the approximated diffusion reverse process with decreasing timestep scheduling in PDS. While this approach may offer faster edits, it comes the expense of sacrificing essential identity features and fine-grained details. On the other hand, method effectively leverages the diffusion reverse process while maintaining the identity of the source scene.
C.2 COMPARISON IN NERF SCENES
We compare the results of our method the figures provided in PDS (Koo et al., 2023). 7 incorporate teaser results of Batman and tulip examples from PDS. Additionally, we include a comparison of editing face into skull using results obtained from the official PDS project page (https://posterior eestillation-ammplinn waanonnaaaangiiiiiiiii... github.io). As a results, DreamCatalyst demonstrates more realistic editing results while preserving the background details, outperforming the representative results of PDS.
C.3 COMPARISON IN 3DGS SCENES
Fig. 9 presents additional qualitative results for methods specifically designed for 3DGS, including Gaussian Editor (GE) and DGE, Since our method is adaptable various 3D editing frameworks
15

---

Published as a conference paper at ICLR 2025
"a photo of a man"
batman"
"a photo of a plant" - "... tulip above the flowerpot with soil'
"a photo of a face" - "... mustache"
(a) Source
(b) PDS
(c) Ours
Figure 7: Qualitative comparison with the original PDS paper. We compare our results with those presented in the original PDS paper. The top two rows show results from the teaser figure in the original PDS paper, and the last row from a figure on the official project page of PDS http://psttrooo--issilltttooooss samplingggithubioo........
through score distillation sampling, we also evaluate GE's performance when integrated with our approach.
Both GE and DGE rely on 2D segmentation masks for partial editing, which are then projected into 3D. However, unprojecting 2D segmentation masks onto 3D scenes often results in inconsistencies, as the masks may not accurately align with the 3D structure. Additionally, the segmentation text prompt may not fully correspond to the intended editing target, leading to unintended areas being included in the editing region. For instance, in the first and second rows of Fig. 9, GE introduces unintended edits irrelevant regions based on the provided prompts, such as altering the clothing in the Einstein image the arms of the clown. Moreover, in the third row, both baselines fail to gener- ate the tulip on the plant as specified by the target prompt. In contrast, the Gaussian Editor combined with our method demonstrates a better trade-off between editing quality and identity preservation.
Further qualitative comparisons between model-agnostic methods in 3DGS scenes are shown in Fig. 10. We compare method with 3D editing results from PDS without any refinement stage to ensure a fair comparison. In the first, second, and fourth rows of Fig. 10, PDS struggles to maintain the structural integrity of the source and generates blurry, ovtr-saturated results. Additionally, in the third row, PDS fails to edit the man into a Hulk as required by the target prompt. In contrast, DreamCatalyst produces more accurate, detailed, and visually superior edited results.
C.4 MORE QUALITATIVE ABLATION ON FREEU
As seen in Fig. 11, different values of result in varying levels of editability and identity preser- vation. When 1.0, the model often struggles to fully align with the prompts, leading to incomplete or subtle edits. For example, in the second row of Fig. 11, the editing with b = 1.0 fails apply the snow effect across the scene, Similarly, in the fourth row, the model fails render the elf-like green hair, suggesting that low 6 hinders the model's 's ability to achieve complete editing. Conversely, when is increased 1.3, over-editing becomes more pronounced, often resulting in unnatural outcomes or edits being applied to unintended areas. In the first row of Fig. 1, we observe that the Joker's s face is reflected in the background wall, an unintended consequence of over-editing. Likewise, in the third row, the sunflower appears both in the pot and the background, further demon- strating that b = 1.3 leads to excessive and unrealistic editing. Our proposed setting, b = 1.1, achieves optimal balance between identity preservation and editability. Thus, using FreeU with an adequate value of b produces editing results that are visually coherent while aligning with the text prompt without introducing artifacts over-editing seen in higher values of b.
16

---

Published as a conference paper at ICLR 2025
"a photo of a man" T Storm Trooper"
"a photo of a bamboo"
orange tree with oranges"
"a photo of a face" "...
"... skull"
(a) Source
PDS (decreasing timestep)
(c) Ours
Figure 8: Qualitative comparison with PDS using diffusion reverse process with decreasing timestep scheduling in NeRF scenes. When applying the PDS method with decreasing timestep scheduling, the results lose fine details and fail preserve key identity features.
Furthermore, we integrate FreeU with PDS qualitatively evaluate the effect of FreeU, As shown in 13, PDS with FreeU tends to produce over-edited results and more background distortions compared the original PDS. This indicates that the effect of FreeU is sensitive the editing capabilities of the baseline method. In contrast, DreamCatalyst without FreeU already produces more realistic and visually appealing edits compared PDS with FreeU. Thus, DreamCatalyst effectively balances editability and identity preservation by combining modified loss weighting and FreeU.
D ADDITIONAL QUANTITATIVE EVALUATION
D.1 COMPARISON IN CONVERGENCE SPEED
In this section, we highlight DreanCatalyst's faster convergence speed compared to other methods. 12 quantitatively demonstrates that DreamCatalyst converges significantly faster than the baseline methods. For this evaluation, utilized CLIP Directional similarity editing convergence behavior, since CLIP image similarity and Aesthetic score do not adequately capture the editing convergence behavior. 14 presents qualitative results highlighting the editing convergence. These results indicate that DreamCatalyst achieves substantially faster convergence compared to PDS and IN2N.
metric to reflect the
17

---

Published as a conference paper at ICLR 2025
"a photo of a face" -
Einstein"
"a photo of a person" clown" "
"a photo of a plant" - ".. tulip above the flowerpot with soil"
(a) Source
(b) GE (IN2N)
(c) DGE
(d) GE (Ours)
Figure 9: Qualitative comparison with model-specifie baseline methods 3DGS scenes.
D.2 QUANTITATIVE ABLATION ON FREEU
We conduct experiments applying FreeU to both PDS and DreamCatalyst, in Table 5, We set the FreeU hyperparameter = 1.1 for PDS, consistent with its configuration in DreamCatalyst. The results show slight increase in the CLIP-Directional Similarity. from 0.161 to 0.162, indicated enhanced editability with the use of FreeU. However, both the CLIP Image Similarity and Aesthetic Score decreased-from 0.687 0.668 and from 5.437 5.413, respectively. This decline can be attributed to PDS underweighting identity preservation at large timesteps as in 2a, leading to insufficient preservation of identity features. The improved editability from FreeU exacerbates this issue, resulting in a loss of original identity and the generation of unrealistic image outputs, as detailed in Appendix C.4 and visualized in 13.
However, integrating FreeU into our method led to significant improvements in both the CLIP- Directional Similarity (from 0.171 0. 180) and the Aesthetic Score (from 5.564 to 5.688), high- lighting enhanced editability and visual quality. DreamCatalyst achieves an effective balance be- tween editability and identity preservation by combining modified loss weighting with FreeU. These results indicate that while FreeU can enhance editability metrics, its effectiveness depends on the underlying method's ability to preserve identity features and produce realistic images. Therefore, combining modified loss weighting with FreeU is essential for achieving superior results in DreamCatalyst.
Table 5: Ablation FreeU for PDS. Quantitative ablation the effects of FreeU for PDS and ours.
Model
FreeU
CLIP-Direc (t)
CLIPP-Im (t)
Aesthetic (t)
PDS
X
0.161
0.687
5.437
PDS
/
0.162
0.668
5.413
Ours
X
0.171
0.744
5.564
Ours
V
0.180
0.746
5.688
18

---

Published as a conference paper at ICLR 2025
"a photo of a bear statue" + polar bear"
"a photo of a person" - Super Mario
aa photo of a man"
hulk"
"a photo of a person" + Leonardo Dicaprio"
(a) Source
(b) PDS
(c) Ours
Figure 10: Qualitative comparison with model-agnostic baseline methods on 3DGS scenes.
E EXPERIMENTAL DETAILS
E. EVALUATION METRICS
qualitative assessment of DreamCatalyst was conducted using three evaluation methods: CLIP image similarity, CLIP directional similarity, and aesthetic score. The CLIP image similarity met- ric (Hessel et al., 2021) quantifies the visual similarity between original and edited images based on CLIP embeddings, ensuring visual consistency between them. CLIP directional similarity met- ric (Gal et al., 2022) quantifies the alignment of the changes between two text captions (ground- truth and edited) with the changes in two images (ground-tree -rrtt and edited). Lastly, the aesthetic score metric quantifies the overall quality of the edited image, with the LAION Aesthetic Predic- tor (Schuhmann, 2022).
19

---

Published as a conference paper at ICLR 2025
"a photo of a man" ー Joker"
"a photo of a campsite"
just snowed"
"a photo of a plant" A "... sunflower above the flowerpot with soil"
"a photo of face" - ".. Tolkien Elf"
(a) Source
(b) b = 1.0 (w/o FreeU) (c) = 1.1 (ours)
(d) b 1.3
Figure 11: More qualitative ablation FreeU for method in NeRF editing. The figure illustrates the effects of varying b values on editability and identity preservation. Lower values struggle to fully apply the edits, while higher b values lead over-editing.
E.2 EVALUATION TEXT PROMPTS
Since IN2N (Haque et al., 2023), GE (Chen et al., 2024b), DGE (Chen et al., 2024a), and our method are based on IP2P (Brooks et al., 2023), these are specialized processing instruction-style text prompts. In contrast, PDS utilizes the Stable Diffusion model (Rombach et al., 2022), which is de- signed to understand description-style text prompts. Therefore, we conduct experiments using pairs of corresponding description-sty -tylle and instruction-style prompts curated from prior works (Haque ct al., 2023; Koo ct al., 2023) or generated by GPT-4 (OpenAI, 2023). We then conduct experiments text prompts (e.g., "a photo of Batman"), while IN2N, GE, DGE,
on PDS using description-styl and our method utilize instruction-style prompts (e.g., "Turn him into a Batman"). The detailed list of text prompts used for the evaluation can be found Tab. 6
E.3 USER STUDY DETAILS
We recruited 100 participants using Amazon Mechanical Turk to ensure a fair comparison. To maintain consistent evaluations of the 3D scenes, we followed the same camera trajectory for each scene and recorded rendered videos of both the source and edited NeRF scenes. To ensure data quality, included vigilance tasks, asking participants to identify trivially incorrect edited results (e.g., edited scenes from different source scenes) as in Fig. 15, Only participants who passed these tasks were included in the analysis.
20

---

Published as a conference paper at ICLR 2025
0.175
0.150
CLIP Directional Similarity
0.125
0.100
0.050
0.025
Ours
IN2N
PDS
0,000
10
3000
10000
30000
Iterations
Figure 12: Comparison of CLIP Directional Similarity across different iterations for ours, IN2N, and PDS models.
"a photo of a face ".. skull"
"a photo of a man" - Leonardo Dicaprio"
"a photo of bear statue"
asiatic black bear"
(a) Source
(b) PDS w/o FreeU
PDS w/ FreeU
(d) Ours w/o FreeU
Figure 13: More qualitative ablation on FreeU for PDS. We evaluate the effect of adding FreeU to PDS. Each column shows: (a) Source image, (b) Original PDS result (without FreeU), (c) PDS
with FreeU, (d) Our method without FreeU.
Participants were shown rendered vidco of source scene along with the corresponding source prompt. They were then asked to evaluate the edited scenes generated by different baselines, in- cluding IN2N (Haque et al., 2023), PDS (Koo al., 2023), and proposed method shown in
21

---

Published as a conference paper at ICLR 2025
Ours
IN2N
PDS
"a photo of campsite"
just snowed"
Ours
IN2N
PDS
"a photo of a plant" "... rulip above the flowerpot with soil"
Ours
IN2N
PDS
"a photo of a face" ".. the Tolkien Elf"
source
1,000 iter
2,000 iter
3,000 iter
10,000 iter
30,000 iter
Figure 14: Qualitative comparison of different models across editing iterations for different editing prompts.
Fig. 16. Each baseline was edited based on specific target prompt, and the baseline options were randomly shuffled for each evaluation.
Participants instructed select the best result for each criterion, based the following:
1. Prompt alignment (Editability): When editing the video, which video best aligns with the text prompt?
2. Identity preservation: When editing the source video, which edited video best preserves the background and identity of the source video?
3. Overall quality: When editing the video, which video shows the best editing quality?
22

---

Published as a conference paper at ICLR 2025
Scene
Source Prompt
Target Prompt (Description)
Target Prompt (Instruction)
Face
"a photo of face"
"a photo of the Tolkien EIr"
"Turn him into the Tolkien Elr" "Turn him into Emma Watson" "Turn him into Elon Musk"
Face
"a photo a face"
"a photo of the Emma Watson"
Face
"a photo a face"
"a photo of Elon Musk"
Face
"a photo of a face"
"a photo of an Einstein"
"Turn him into an Einstein"
Face
"a photo of a face"
"a photo of face with mustache"
"Give him a mustache" "Turn him into Leonardo Dicaprio"
Face
"a photo of face"
"a photo of Leonardo Dicaprio"
Face
"a photo a face"
"a photo a skull"
"Turn face into skull"
Bear
"a photo of bear
"a photo a grizzly bear"
"Turn the bear statue into a grizzly bear"
statue"
Bear
"a photo of bear
"a photo of panda"
"Turn the bear statue into panda"
statue"
Bear
"a photo of bear statuc"
"a photo an asiatic black bear"
"Turn the bear statue into an asiatic black bear"
Bear
"a photo of bear statue"
"a photo of a polar bear"
"Turn the bear statue into polar bear"
Bear
"a photo of bear statue"
"a photo a Bengal tiger"
"Turn the bear statue into Bengal liger"
Bear
"a photo of bear statue"
"a photo of a skull bear"
"Turn the bear statue into a skull bear" "Turn him into a Super Mario" "Turn him into clown"
Person
photo of person"
photo of Super Mario"
Person
"a photo of person" "
"a photo of clown"
Person
"a photo of person"
"a photo of a person reading a book"
"Make him reading a book"
Person
"a photo of person"
"a photo of person wearing
"Wear him sunglass"
sunglass"
Person
"a photo of person"
"a photo of Storm Trooper"
"Turn him into Storm Trooper" "Tum him into Elon Musk" "Turn him into Iron Man" "Turn him into Jack Sparrow"
Person
"a photo of person"
"a photo of Elon Musk" "a photo of Iron Man" "a photo of Jack Sparrow" "a photo bronze statue"
Person
"a photo of person"
Person
photo of person"
Person
"a photo of person"
"Make him into a bronze statue"
Plant
"a photo of plant"
"a photo of a rose above the
"Turn only the plant above the flowerpot into rose and keep soil"
flowerpot with soil" "a photo of a sunflower above the
Plant
"a photo of plant"
"Turn only the plant above the flowerpot into a sunflower and keep soil"
flowerpot with soil""
Plant
"a photo of plant"
"a photo of a tulpp above the flowerpot with soil" "a photo of an orange tree with oranges"
"Turn only the plant above the flowerpot into tulip and keep soil" "Tum the tree into orange with oranges"
Bamboo
photo of bamboo"
Campsite
"a photo of a campsite"
"a photo a campsite just snowed"
"Make it look like just snowed"
Campsite
"a photo of campsite" photo of farm"
"a photo of campsite sunset" "a photo a farm autumn"
"Make it sunset"
Farm
"Make it autumn"
Yuseung Yuseung
"a photo a man"
"a photo of Batman"
"Tum him into Batman"
"a photo of a man"
"a photo of Marvel's Spider-Man" "a photo of clown" photo a Joker"
"Tur him into Marvel's
Spider-Mar' "Turn him into clown" "Turn him into Joker" "Turn him into a Hulk" "Turn him into Thanos" "Turn him into I Leonardo Dicaprio"
Yuseung
"a photo of man"
Yuseung
"a photo a man"
Yuseung
"a photo of a man"
"a photo of a Hulk"
Yuseung
"a photo of a man"
"a photo of Thanos"
Yuseung
"a photo of man"
"a photo of Leonardo Dicaprio"
Yuseung
"a photo of man"
"a photo of Darth Vader"
"Tum him into a Darth Vader"
Yuseung
"a photo a man"
"a photo of Storm Trooper"
"Tum him into Storm Trooper"
Yuseung
"a photo of a man"
"a photo of a Deadpool"
"Tum him into Deadpool"
Yuseung
"a photo of a man"
"a photo of a baldman"
"Make him bald"
Table 6: Detailed evaluation text prompts. The description-style prompts are used for PDS, which employs the Stable Diffusion model (Rombach et al., 2022), while the instruction-style prompts are applied to IN2N, GE, DGE, and our method, all of which utilize the InstructPix2Pi,, model (Brooks al., 2023).
23

---

Published as a conference paper at ICLR 2025
(Original Video) Source
(Edited Video) Option A
(Edited Video) Option
[Edited Video) Optio
Questions
When which
Prompt: tullp flowerpot prompt?
Option
Option
Option
When editing source video, which video preserves the background structure the source video?
Option
Option
O Option
0. When editing the video, o which video shows the best editing quality?
Option A
Option
Option
Figure 15: Vigilance test for user study. The vigilance test was used to ensure participant atten- tiveness during the user study. Participants were asked to identify trivially incorrect edited videos, such as those with irrelevant or mismatched prompts and edits.
[Original
Video) Option
[Edited
Questions
editing
aligns prompt?
Prompt:
Option
Uption
O
Option
2. When editing the source videa, which edited video best preserves the background and structure of the source video?
Option
Option
Option
3. When ediingg the video, which video shows the best editing quality?
Option
Uption
O Option C
Figure 16: User study, Participants evaluated edited videos based on three criteria: prompt align- ment, identity preservation, and overall quality. Each video randomly shuffled for each evalua- tion to ensure unbiased selections. The target text prompts were highlighted in the prompt alignment questions.
F MORE VISUAL RESULTS
F.1 RESULTS FROM DIFFRENT MODES IN DREAMCATALYST
In Fig. 17, we present additional visual results demonstrating the effectiveness of DreamCatalyst in both its fast and high-quality modes. Across a variety of scenes, DreamCatalyst preserves both editability and identity consistency. In the first row, for the target prompt of turning a campsite scene into a sunset, both modes produce convincing results with natural lighting, though minor color variations are visible between the fast and high-quality modes. Similarly, in the second row, when editing a bear statue into panda, the fast mode accurately captures the desired editing, while the high-quality mode refines the textures for more photorealistic details. For more complex cases, such as the third, fourth, and fifth rows, the high-quality mode captures finer details, while the fast mode still maintains overall structural accuracy. These examples highlight the ability of our method to achieve high visual fidelity while balancing computational efficiency across different modes, making it adaptable to varying time constraints and quality requirements.
24

---

Published as a conference paper at ICLR 2025
"a photo of campsite"
sunser"
"a photo of a bear statue" - "... panda" "
"a photo of a plant" - "... tulip above the flowerpot with soil'
"a photo of a face"
"... Einstein"
"a photo of a man" + ".. hulk"
(a) Source
(b) Ours (fast)
(c) Ours
Figure 17: Visual results from DreamCatalyst in different modes. The fast mode of our method produces visually comparable results to the high-quality mode, while reducing editing time to ap- proximately 35% of the high-quality mode.
25

---

Published as a conference paper at ICLR 2025
"a photo of a man"
clown"
"a photo of a face" ".. skull"
(a) Source Figure 18: More visual results on weighting functions of LDns for NeRF editing,
(b) V (ours)
(c) V
(d) V
(d) Ours (with detailed text prompt)
(a) Source
(b) 2D editing with IP2P
(c) Ours
Figure 19: Limitations. The limitations of IP2P reflected in the 3D editing process. In cases where IP2P struggles with 2D edits for a simple text prompt as in (b), this failure extends to the 3D editing seen in result with IN2N. The simple text prompt like "Turn the plant into rose" fails to produce the desired outcomes as in (c). However, when using a more detailed prompt like "Turn only the plant above the flowerpot into a rose and keep soil," our method successfully generates rose on top of the branch as shown in (d).
F.2 RESULTS FROM DIFFERENT WRIGHTING FUNCTIONS
As defined in Equation 14. 1 is weighting function applied LDDS based on timesteps. As demonstrated in Fig. 2b, modifying the weighting function d to either V Vi increases editabil- ity at lower timesteps. However, these variations do not significantly impact the results compared to T For instance, in both the first and second rows of Fig. 18, there is no major difference among the outputs from different weighting functions, leading to only subtle changes the background.
Our findings confirm that fulfilling the two conditions mentioned in Sec. 3.1 enables effective 3D editing regardless of the specific choice weighting function. However, we observe that inordinate editability in small timesteps rarely induces trivial color saturations backgrounds-ce
row 18. We hypothesize that the excessive editability during the final stages induces these color saturation artifacts. To prevent these color saturations, editability in small timesteps. Thus, we utilize TP" as the default weighting function for LDDS. as supported by the quantitative results in Tab. 1. We leave the exploration of more optimal design choices for future work.
design (t)(t) to drastically decrease
G ADDITIONAL DISCUSSIONS
G.1 LIMITATIONS
DreamCatalyst uses IP2P (Brooks et al., 2023) as a pretrained diffusion model for distilling 2D prior knowledge in the 3D editing process. However, the limitations of IP2P carry over into 3D
26

---

Published as a conference paper at ICLR 2025
editing tasks. As shown in Fig. 19, when IP2P struggles with specific 2D edits, these limitations extend to our 3D editing results. IP2P does not effectively edit certain prompts in 2D, the 3D editing may also fail to align with target text prompt. As IP2P other 2D diffusion models improve in accuracy and prompt alignment, 3D editing capabilities of our method are expected to similarly improve, offering more reliable and precise outputs.
27


Reviews:
[{"review_id": "fGyaP3cw4k", "reviewer": "Reviewer_wbDz", "summary": "This work presents DreamCatalyst, a variation of score distillation loss for the purpose of editing 3D scenes. This variation on SDS contains two terms: one based on DDS that controls the editing capabilities of the loss and one that is a regularization term intended to preserve the identity of the scene. The formulation in DreamCatalyst produces better quality edits and reduces edit time as compared to existing methods. The method is evaluated both qualitatively through many figures and quantitatively showing automated metrics as well as a perceptual user study.", "questions": "Why is this method able to work with the approximated diffusion reverse process while standard PDS is not (Fig. 7). Is it just due to the coefficients phi and psi and in the case of PDS, these coefficients don't allow sufficient editability at small timesteps whereas DreamCatalyst’s do?\n\nMinor questions:\n- L:349-350 – “uniformly samples timestep t = T → 1” My understanding is that $t$ starts at $T$, ends at $1$, and then at an arbitrary iteration $i$, $t = T - i$. If this is the case, why is this uniform sampling? Maybe I am missing something here.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Strengths:\n- This work has promising results as the method shows impressive ability to edit only the regions indicated with the text prompt.\n- This work proposes a reinterpretation of PDS loss that better aligns to the diffusion reverse sampling process.\n- The proposed approach improves over existing techniques in both speed and edit quality.\n- This work applies FreeU to the optimization to get better quality edits without sacrificing identity preservation.", "weaknesses": "Weaknesses:\n- Given the similarity of DreamCatalyst to PDS, this work could benefit from a more clear / detailed discussion of the differences between these two approaches. Specifically, since the PDS loss in eq 14 in the PDS paper seems the same as eq 16 in this paper, my understanding is the main difference between these two is the hyperparameters that control the timestep dependent coefficients phi and psi for identity preservation and editability respectively. If this is the main difference, then it should be made more clear.\n- Since the increased speed is a key contribution of this work, more space should be devoted to explaining how this approach actually does so, as it is not clear to me in the current state. It seems to be due to the timestep sampling and approximated diffusion reverse process. However, exactly why it is faster was not clear. Additionally, a helpful experiment to highlight the speed would be to show DreamCatalyst vs IN2N VS PDS on 1k, 3k, 15k, and 30k iterations so that we can see what the quality looks like for these other methods when DreamCatalyst converges.\n- FreeU seems like an important component to increasing edit quality, but currently there don’t seem to be any experiments showing how important it is. While there is an ablation for the FreeU hyperparameter, an experiment comparing PDS and DreamCatalyst both with and without FreeU to see how much of an impact it makes would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LoTmV0neEo", "reviewer": "Reviewer_Hf3Z", "summary": "In this paper, the author proposes DreamCatalyst, a method for editing 3D scene using improved Posterior Distillation Sampling loss. Based on the analysis of PDS loss, the authors proved that the coefficients of ID-preserving loss and the DDS loss can be independently selected under DDIM inversion. They also proposed several rules for setting these coefficients under different time steps. As a result of these advances, DreamCatalyst out-performs previous 3D editing methods in both speed and quality.", "questions": "1. Why does DreamCatalyst rely that heavily on FreeU? In fig.6, with $b=1$, the model performs poorly compared with the teaser figure of the PDS paper. Is there a reasonable explanation for this?\n2. Just curious, is the determination of the functional forms of $\\Psi$ and $\\Phi$ based on better theoretical analysis or qualitative constraints? If it's qualitative analysis, what impact do other function families or parameters that meet the proposed conditions have on the results?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written and easy to follow.\n2. The analysis of PDS loss is interesting.\n3. Experiments show that the proposed method achieves good 3D editing results with faster speed.", "weaknesses": "1. The method proposed in the paper is actually just a supplement to PDS. The theoretical analysis merely shows that the weights of the two losses can be adjusted, a fact that was already discovered and utilized in previous methods like Fantasia3D and ProlificDreamer.\n2. Some of the cases used in the experiments are already present in the original PDS paper. The results from the original paper should be used for these cases. However, the PDS results provided by the authors show a significant discrepancy from the original paper. I suggest that the authors compare their results with those in the original PDS paper. I will adjust my review based on these comparisons.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zOSwN3UCiJ", "reviewer": "Reviewer_p9uk", "summary": "The paper presents an innovative way to achieve 3D-consistent image editing on a 3D representation (NeRF/3DGS). It builds upon several foundational works (SDS, DDS, and PDS) to generate the edited scene with better quality and speed simultaneously. Novelty of the paper, written in section 3.3, is a (diffusion) time-step dependent weighting of the two primary loss terms (equation 17).", "questions": "*4D editing of scenes is an active area of interest - could the authors comment on if/how this work could be adapted to such use-cases?\n\n*Could you clarify what is novel about lines 349-350 r.e. “we adopt decreasing timestep sampling.” Isn’t this standard diffusion sampling (more noise to less noise)?\n\n*Could the authors elaborate more on where exactly the time savings are achieved? Is it because fewer optimization steps are required to train the NeRF/3DGS after the author’s proposed modified loss/FreeU?\n\n*Elaborate more on how equation 18 was obtained?\n\n*Fig. 6 shows a qualitative ablation on FreeU. Could a quantitative evaluation be presented as well? How much of the improvements are due to FreeU vs. better loss weighting?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "*The strength of this paper is the evaluation - the quantitative and qualitative evaluation includes the most recent works and demonstrate this method is the top-performer for this task. Table 1 shows that this paper’s family of models is faster and more semantically-aligned than existing work. Fig. 5 shows that this method generally achieves more favorable CLIP scores while being more efficient. Both NeRFs and 3DGS are evaluated.", "weaknesses": "*Technical novelty may be a bit limited. FreeU makes all Stable Diffusion models better. The core contribution appears to be a smart scheme to dynamically balance weights of an existing loss function.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)