Rethinking Score Distilling Sampling for 3D Editing and Generation
Authors: Xingyu Miao
Haoran Duan
Yang Long
Jungong Han
Venue: ICML 2025 poster (2025)
Keywords: Score Distilling Sampling; 3D Editing and Generation
Primary area: applications->computer_vision

Abstract: Score Distillation Sampling (SDS) has emerged as a prominent method for text-to-3D generation by leveraging the strengths of 2D diffusion models. However, SDS is limited to generation tasks and lacks the capability to edit existing 3D assets. Conversely, variants of SDS that introduce editing capabilities often can not generate new 3D assets effectively. In this work, we observe that the processes of generation and editing within SDS and its variants have unified underlying gradient terms. Building on this insight, we propose Unified Distillation Sampling (UDS), a method that seamlessly integrates both the generation and editing of 3D assets. Essentially, UDS refines the gradient terms used in vanilla SDS methods, unifying them to support both tasks. Extensive experiments demonstrate that UDS not only outperforms baseline methods in generating 3D assets with richer details but also excels in editing tasks, thereby bridging the gap between 3D generation and editing.

Rethinking Score Distilling Sampling for 3D Editing and Generation
Xingyu Miao! Haoran Duan
Yang Long' Jungong Han
Abstract
bilities in numerous applications. However, the inherent complexity of 3D data (Miao al., 2025) presents chal- lenges for applying diffusion models in the 3D domain. Since most diffusion models are trained on 2D images, their effectiveness in generating 3D data is limited (Liu et al., 2024). Nevertheless, the extensive knowledge and gen- erative priors derived from 2D image generation models provide valuable insights and potential applications for 3D data generation.
Score Distillation Sampling (SDS) has emerged as a prominent method for text-to-3D generation by leveraging the strengths of 2D diffusion mod- els. However, SDS is limited to generation tasks and lacks the capability to edit existing 3D as- sets. Conversely, variants of SDS that introduce editing capabilities often can not generate new 3D assets effectively. In this work, we observe that the processes of generation and editing within SDS and its variants have unified underlying gra- dient terms. Building on this insight, we propose Unified Distillation Sampling (UDS), a method that seamlessly integrates both the generation and editing of 3D assets. Essentially, UDS refines the gradient terms used in vanilla SDS methods, unifying them to support both tasks. Extensive experiments demonstrate that UDS not only out- performs baseline methods in generating 3D as- sets with richer details but also excels editing tasks, thereby bridging the gap between 3D gen- eration and editing. The code is available on: https: /ttt::///gttbbb com/ coom//iingy0e UDS.
When working with 3D content, achieving realistic effects in generation and modification is crucial. Traditionally, these operations rely on specialized software and manual execution by experts. DreamFusion (Poole et al., 2022) introduced a breakthrough technique called Score Distil- lation Sampling (SDS) to overcome these limitations. By leveraging the generative priors of text-to- text-to-image diffusion models, SDS can generate 3D assets from text using Neural Radiance Fields (NeRF) (Mildenhall et al., 2021; Liu et al., 2020; Mildenhall et al., 2022) or 3D Gaussian Splatting (3D GS) (Kerbl et al., 2023). This approach surpasses the limitations of traditional 3D generation models and opens new possibilities for editing and generating 3D data.
However, previous studies have identified averaging ef- fect problem with SDS (Liang et al., 2023; Wang et al., 2024b; Wu et al., 2024; Lin et al., 2023; Wang et al., 2023). Specifically, the pseudo ground truths generated from differ- ent noise sources at the same viewing angle differ, and the update directions of these pseudo ground truths are applied to single 3D model simultaneously. This results in the final output being overly smooth and lacking in detail. Addition- ally, SDS primarily focuses on generation and lacks editing capabilities. To address this issue, Delta Denoising Score (DDS) (Hertz et al., 2023) extended SDS to include editing capabilities but did not fully resolve the averaging effect problem. Although DDS performs well on 2D images, its effectiveness in 3D scenes is unsatisfactory. Posterior Distil- lation Sampling (PDS) (Koo et al., 2024) further investigates the reasons for DDS's 's poor performance in 3D scenes. The findings indicate that the lack of identity recognition in the gradient optimization term of DDS makes it difficult to re- tain information from the original scene, leading to editing failures.
1. Introduction
In recent years, diffusion models (Hua et al., 2025) have achieved significant advancements across various fields (Sa- haria et al., 2022; Cao et al., 2024; Podell et al., 2023; Luo et al., 2023; Song & Ermon, 2019; Song et al., 2020b; Ho et al., 2020; Balaji et al., 2022). In particular, conditional diffusion models have been instrumental in enhancing con- ditional text generation, editing of 2D) images (Hertz al., 2022; Huberman-Spiegelglae et al., 2024; Lee et al., 2023), and audio processing (Ghosal et al., 2023; Huang et al., 2023). These models have demonstrated remarkable capa-
'Deaarmeent of Computer Science, Durham University, UK Departmment of Automation, Tsinghua University, China. Corre- spondence to: Yang Long <yang.longe durham.ac.uk>- Haoran Duan <haoranduan @tsinghua.cdu.cn>.
Proceedings of the nnnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
In this work, we aim to develop a unified method for both 3D

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
"a photo of a man" "
Thanos" "
Joker" "
Captain America"
Spider Man
"a photo of a plant"
with roses" "
"a photo of a face"
Einstein" "
"A girl with facial painting, long hair; photorealstiic IDDR.
"A woman, head, black hair; photorealistic, 8K, HIDR."
"Zombie JOKER, head, photoroalistic, HIDR."
"A portrait of IRONMAN, white hair, hoad, photorealistic, 8K, IIDB."
"A DSLR photo A Cream Cheese Donut.
"The warm, gooey brownie was flecked wish melting chocolate."
"A pincopple."
"A DSLR photo of Stylish Air Jordan shoe, best quality, 8K. IID."
"A highly-deutled sandcaetle.."
"A DSLR photo of chow chow puppy."
"An crupting volconn."
"Viking ave, fantasy, wespon blnnder.
Figure 1. Example of text-guided 3D editing and text-to-3D content generated from seratch by our UDS. We achieve superior 3D editing and 3D generation results with photorealistic quality in short training time. Please zoom in for details.
2

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
editing and generation. Specifically, we examine DDS and PDS-two representative 3D editing approaches-ante and dis- entangle their reconstruction term from the guidance term. This separation facilitates a more systematic analysis and comparison of various SDS variants. Then, we examine the factors contributing to their successes and failures, identi- fying notable similarities with the gradient terms used in recent DDIM- based SDS variants. Inspired by this insight, we propose Unified Distillation Sampling (UDS), provide general method for both 3D editing and generation tasks. Our UDS first approximates clear latent zo representation, which serves as the reconstruction term. This reconstruc- tion term is then combined with classifier-free guidance to supply the necessary gradient terms. UDS shows lower gra- dient variability and improved stability relative to previous methods, enabling the generation of superior edited results. In summary, contributions are:
more, Consistent3D (Wu et al., 2024) approaches SDS from the perspective of ordinary differential equations (ODE), developing technique called Consistency Distillation Sam- pling (CSD) to mitigate over-smoothing and inconsistency. Similarly, LucidDreamer (Liang et al., 2023) explores the loss function of SDS and introduces Interval Score Match- ing (ISM), a method conceptually similar to CSD but utiliz- ing reversible diffusion trajectories of DDIM (Song et al.. 2020a; Zhuo et al., 2024).
In terms of editing, Haque et al. (Hertz et al., 2023) in- troduced Iterative Dataset Updating (IDU), a text-driven method for editing NeRF using Instruct-Pix2Pix (Brooks et al., 2023) or editing 3D GS (Wang et al., 2024a; Qu et al., 2025; Xu et al., 2024). This approach progressively substi- tutes original reference images with edited versions during NeRF reconstruction, gradually morphing the scene towards the edited state through adjustments in reconstruction loss or attention weights (Duan et al., 2023). Meanwhile, Mirzae et al. refined this process in CerructtNeRF2NNeee (IN2N) (Haque et al., 2023) by targeting specific local areas for edit- ing. Nevertheless, this iterative image replacement method struggles with edits requiring significant shifts across differ- ent views, such as complex geometric changes or the addi- tion of new objects in undefined areas, and thus is mainly effective for appearance modifications. More recent ap- proaches have abandoned the iterative IDU method in favor of directly applying SDS combined with segmentation tech- niques for NeRF editing (Li et al., 2023; Park et al., 2023; Zhuang et al., 2023). Hertz et al. (Hertz et al., 2023) in- troduced Delta Denoising Score (DDS), an editable variant of SDS designed to reduce the noise gradient direction in SDS to better maintain the details of the original image, However, DDS does not sufficiently ensure the retention of identity information. While this limitation is less apparent in image editing, it becomes problematic in NeRe-based 3D scene editing, as NeRF-based or 3D GS methods tend to be sensitive to gradients, which may lead to significant deviations from the original content. Conversely, Koo et al. (Koo et al., 2024) propose Posterior Distillation Sampling (PDS), focusing on enhancing editability and maintaining identity in text-aligned editing by minimizing the random latent matching loss they introduced. However, PDS fails to adequately disentangle the identity preservation term from the classifier term and inherits the high CFG weight of SDS (i.e., CFG=100), leading to over-saturation and lack of diversity in the results. In contrast, we conduct an in-depth analysis of the gradient term in PDS and successfully dis- entangle the identity preservation term from the classifier term.
We investigate text-to-3D generation and editing meth- ods based on score sampling, identifying significant commonalities in their gradient optimization processes. We show that while these gradient terms serve differ- ent functions across various tasks, their forms remain consistent (Section 4) This consistency makes it possible to establish common methods across different tasks.
We propose a novel Unified Distillation Sampling (UDS) that enables unified processing of text-guided 3D editing and text-to-3D generation (Section 4.2). UDS achieves improved editing and generation results by utilizing a single gradient formula to generate more stable gradients.
Extensive experiments demonstrate the effectiveness of our UDS method across multiple applications, includ- ing the editing and generation of NeRF, the generation of 3D GS, and the editing of SVG. These results val- idate the effectiveness of UDS in achieving unified processing for both 3D editing and generation tasks.
2. Related Work
For generation tasks, Score Distillation Sampling (SDS)
was initially introduced in DreamFusion (Poolc et al., 2022)
to directly optimize 3D representations using pre-trained 2D
text-to-image diffusion models (Katzir et al., 2024). Score Jacobian Chaining (Wang et al., 2023) presents an alterna- tive approach that achieves results comparable to SDS but is based on different mathematical principles. ProlificDreamer (Wang et al., 2024b) conducts a thorough examination of the SDS objective function and introduces a sartcle-based vari- ational framework known as Variational Score Distillation (VSD), aimed at resolving issues of oversaturation, over- smoothing, and limited diversity inherent in SDS. Further-
3. Background
Diffusion Models The diffusion model comprises a forward process that progressively perturbs the initial data wo with
3

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
noise E and reverse process that incrementally denoises the noisy data. The forward process is defined as follows:
conceptualized image editing as a distribution-matching optimization problem. They treat the perturbation noise distributions of the original and edited images two sepa- rate distributions that need to be aligned. First, they extract information from a pre-trained diffusion model and use text conditions to guide the image toward a specific region within the noise distribution. Then, they determine the update di- rection by estimating the score difference between the target and source distributions and perform the update edit. This approach addresses the issue of unclear and blurry images caused by the noise gradients generated by SDS. This im- plicit editing operation does not require the use of masks, which are defined as:
Fy = Va + 0ge, 6 ~ (N(,
(1)
where æt is the noisy latent representation of zo at timestep i, {āg ) (with ag = 1 and öT = 0) denotes a set of time steps indexing a strictly monotonically decreasing noise schedule, and AO 1) represents the Gaussian distribution. In the reverse process, a diffusion model denoising network Cp. parameterized by o, and a sampler prediction score function are utilized. Typically, the network is trained using denoising score matching:
(in = Er,e (Ic)( <12]
VoLoDs = Er.e, [u((] (ca(utt) (0" g0 cs(la"" ((a "" (i)
Ou (6)
du
Score Distillation Sampling (SDS) As discussed in Sec- tion 2, SDS (Poole et al., 2022) is pioneering method for text-to-3D generation. It achieves this by secking modes for the conditional posterior prior in the DDPM (Ho et al., 2020) latent space. Specifically, noise is added to rendered images z := g(0, c). where g-)) represents a NeRF or 3D GS model, 9 denotes the parameters of the NeRF or 3D GS model g-). and is the camera parameter. The method then distills knowledge from a pre-trained diffusion model with rich 2D priors to train the NeRF or 3D GS model. The optimization objective is:
where æt and æद" represent the latent noise of x and zy at timestep t and they share the same noise e,. Although DDS extends SDS to allow editing, DDS actually lacks the identity item, so it is difficult to preserve the source identity. This loss of identity information causes DDS to commonly fail in 3D editing.
What makes Posterior Distillation Sampling (PDS) work? The DDS demonstrates promising editability for 2D content. However, it falls short for 3D editing, which demands stronger identity preservation than 2D. To address this problem, the PDS aims to achieve both conformity to the text and preservation of the source's 's identity using the stochastic generative process of DDPM. Specifically, PDS introduces stochastic latents, ensuring that the latent noise of the reference and target in the latent space match. This can be expressed as:
Cing (8) := Er, (t) [|,,( t, -
(3)
where wt) time-dependent weighting function, c is the standard Gaussian noise serving as the ground truth de- noising direction of æt at timestep I, and 6a(a I, y) is the predicted denoising score given the condition y. Ignoring the UNet Jacobian (Poole et al., 2022), the gradient of the SDS loss is:
(7) where 2-(-) is the stochastic latents at timestep t. The stochastic latent (-) is including the structural details of To and is calculated as:
Lans g()) E.,. ||=F(a]f ./)) xt (a0" y s
VoLsDs = Bi,e,c [w(t) (Es(II,t,,) v) c) (g(0.c) 00
(4)
For simplicity, we denote 6n == c (x 1, y) - e. The SDS employs Classifier Free Guidance, thus 8Rz in Equation (4) can be further expanded as:
21 y) = I1-1 Pg (2). y)
(8)
SSDS m2 = Co (2x, t, - c + (co( co(2x,,, (5)
1-ā,-1 Br, and the posterior mean presents where the o, := T-a+ ( 3) 0(t)ău + (t) Here, a(t) and (t)) are coefficients about timestep t. By ignoring the UNet Jacobian term as SDS, the gradient of LPDS is represented as:
where w is the weight of Classifier- Free Guidance (Ho & Salimans, 2022). The SDS loss can thus be divided into two components: the reconstruction term grecon and the classifier- free guidance term 8els
Volma Bes (=0) (ar (y (aST """)
Oz
80
(9)
-Be coll) (all - Masily
(eolor
. (ot ())
4. Methodology
Here, co(t) and (t) are coefficients defined with respect to the timestep t. We can observe that Equation (9) can be di- vided into two terms: one representing identity preservation, and the other corresponding to the equivalence for LDns-
4.1. Revisiting SDS Variant for Editing
What makes Delta Denoising Score (DDS) fail? Inspired by (Poole et al., 2022), Hertz et al. (Hertz et al., 2023)
4

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
Since the wy is unknown in the diffusion process, PDS us- ing Tweedie's formula (Chung et al., 2022) that leverages The expectation of the posterior distribution p(Z0o to approximate 2g. It can be expressed as:
Algorithm 1 Unified Distillation Sampling
1: Initialization: gradience scale w, time interval
2: while 0 is not converged do
3: Sample: Eo = s c)2 c e~ N(O,I), 4(1,
4: if Editing then
Zo R To = [[ = (ar v1 (eg( v)), (10)
5: for i = [sre, tg]] do
3t Vāna0 + VI 11aat āje
where co f, u) is the prediction of the condition diffusion model. The main idea of PDS is to edit from the stochastic latent, but in fact, as shown in the PDS optimization term after decomposition by Equation (10), PDS adds an addi-
Predict
y)) and c L, 0))
8: Approx. (1) via Eq. (10) or Eq. (15)
oz. # + ((Es((E,,t))) E 0))
10:
end for
tional identity information term. This is also the key to the success of PDS.
11: VoLUDS = (() - -
12: update 2. with VoLUDS
Disentangle DDS and PDS We can link Equation (10) and Equation (6) to Equation (5), we can find that DDS and PDS can be expressed as:
13:
else
14:
I; = Vá + vI -I-ārr āre
15:
Predict
v)) and cg(2;, t, 0))
16:
Ti-c = Va + ++VI-a 06-c6
&DDS =t=
== (1) t) - (x) t) + w
(11)
17:
Predict Egl 10,t--e,00 0))
18:
Approx. Tyj and Ty via Eq. (10) or Eq. (15)
19:
VOLLDS (() to ") + ((cs(z,,t,))
(.(,,,,,0)) update To with VoLUDS
60 =
+
E ut ()) (x +w
). (12)
20:
Ideeiity procrvction
21: end if
22: end while
According to Equation (11) and Equation (12), we can ob- serve that in the editing task, the gradient optimization term can actually be decomposed into two parts: one is the recon- struction term, and the other is the classifier- free guidance term. PDS operates primarily on stochastic latent. To sim- plify the description, we do not specify the coefficients in Equation (12). From the previous analysis, we know that DDS does not work well in editing, mainly because of the lack of identity-presse preserving terms. On the contrary, PDS effectively edits by introducing dientity-preservingg terms, although its complexity complicates the analysis. Link to generation task Inspired by recent work (Yu et al., 2023), we know that the weight w determines which term is dominant. In generation tasks, if the classifier-free guidance term is dominant, the generation proceeds as expected; the same applies to editing tasks. This observation also explains why DDS and PDS set w to 100. In addition, combined with the latest DDIM-based generation methods (Liang et al., 2023), we further discovered that these methods can be summarized as:
4.2. Unified Distillation Sampling (UDS)
Based on the above analysis, in order to meet the require- ments of editing tasks, the preservation of identity terms is particularly important. Therefore, we first combine the identity preservation term and the strategy without classifier guidance to effectively control the editing process, which can be expressed as:
gedit mr
2 10 + w (80ls
Jols
(14)
Note that Equation (14) and Equation (12) are very sim- ilar, but Equation (12) is simplified representation we derived and is not the true gradient term of PDS. In addition, Do is unknown during the generation process, thus we can leverage Equation (10) to approximate zo. However, this approach approximates Fy through single-step denoising. According some recent work, we know that this approx- imation is inaccurate and may affect the preservation of identity information in editing tasks. Fortunately, we can use the deterministic sampling process of DDIM to obtain No, which is expressed as:
6z E. (wt, 1,0) (0 c, c, +w (bel))
(13)
green
where is defined time step interval and w is typical value (e.g., 7.5). We can find that Equation (13), Equa- tion (11), and Equation (12) are very similar. This finding inspired us to explore whether a more unified form can be developed to accommodate both editing and generation tasks.
Dt VI aiE a1
Rt-1 = Vor-1
Vor
(15)
+ / - 11-1E t,)),
by using Equation (15), we can get Dy in an iteration. While for generation tasks, we simply modify the Equation (14)
5

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
Input Scene
Instruct-NeR NcRF2NoRF
DDS
PDS
Ours
"A photo of bamboo - "a photo of an apple tree"
"A photo of face
"a photo of clown
photo of plant"
"a photo of tulips"
"A photo of man
photo of Botmon"
Figure 2. Comparison with baseline methods in text-guided 3D editing. We present visual editing results for both UDS and the baseline methods. Experiments show that our UDS effectively edits 3D content to closely align with the input text prompts, while maintaining high level of photorealism. Notably, DDS (Hertz et al., 2023) and PDS (Koo et al., 2024) set CFG is 100, while our is 7.5.
as:
whereas UDS exhibits more streamlined formulation. The critical distinction is in the application of Tweedie's for- mula for 2o approximation: PDS employs conditional noise estimates while UDS utilizes unconditional counterparts. This fundamental difference explains UDS's capability to achieve comparable editing performance to PDS under small weights (e.g. 7.5) CFG settings. Essentially, UDS real- locates the weights between the reconstruction term and classifier- free guidance term through simplified formula- tion, resulting more intuitive gradient computation and significantly enhanced interpretability of the algorithm.
82eh 4
woels
(16)
where e is defined time step interval. Compared to Equa- tion (13), we only modify the reconstruction term greeon by replacing the predicted noise with the approximated &n- Additional, insight from recent work (Yu et al., 2023; McAl- lister et al., 2024), we also can add a negative classifier-free guidance term to improve generation quality. The Equa- tion (16) can be further rewrite as:
ggen 21
$b + wr gels-
w(fels
gels
(17)
For the generation task, ISM adopts noise predictions that are highly correlated with the data at two time intervals as reconstruction terms. These reconstruction terms reflect the (18) temporal change rate noise predictions through gradients, providing directional guidance for the denoising process. In contrast, our approach directly utilizes the approximate xn to replace the reconstruction term by constructing it from the differences between Zo predictions at two time intervals. Specifically, we use the difference between the predicted x from the previous time step and the predicted æ1 from the current time step as the guiding signal for the reconstruc- tion term. Since x zf-e retains more detailed information in early predictions, while =1f more stable but may lose details in later predictions, this difference not only reflects the temporal change rate but also naturally introduces prior information for preserving details. In this way, we can provide directional guidance for the denoising process sim-
Finally, we can express our UUDS as: =x
UDS
Aão + w Aols
Da
The gradient of our Luns can be defined as follows:
VoLuDS = Er,e. ( + @A gols 0g(0.c)
2, 80
(19)
The algorithm of our UDS is shown in the Algorithm I.
4.3. Discuss with PDS and ISM
For the editing task, we derive the UDS in Appendix A.4. It can be observed that our final formulation is quite similar to that of PDS. When disregarding the predicted unconditional noises ( t, 0) and (x,", (), PDS contains two time-dependent complex coefficients in its gradient terms,
6

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
Dream (808) (-50 mins)
Fantasia3D (~1
PromificDreamer (VSD) (-8h)
LucidDreamer (18)) (~1 h 20 mins)
Ours (-52 mins)
DSLR photo of the Imperial State Crown of England." "
"A DSLR photo of Schnauzer wearing pirate hat
Figure 3. Comparison with baseline methods in text-to-3D generation. Experiments demonstrate that approach can generate 3D content that closely aligns with input text prompts, exhibiting high fidelity intricate details. The running time methods measured on single 3090 GPU. Notably, we tried to reproduce PremificDreamer and LucidDreamer, but failed to achieve the results in the Dreamer paper. Therefore, we directly use the visualization results the LucidDreamer paper for these two methods.
Methods
CLIP Score
User Preference Rate (%b
processing instruction-styy text prompts, while DDS, PDS, and our method use the Stable Diffusion model (Saharia et al., 2022), which is designed to interpret description-style text prompts, we generate corresponding pairs description and instruction-style prompts for evaluation. We conducted experiments using description-style prompts (e.g.. "a photo of Batman") for DDS, PDS, and our method, while IN2N used instruction-style prompts (e.g., "Turn him into Bat- man'), We implemented NeRF- based methods using Three- studio (Guo et al., 2023) and 3D GS-based methods using LucidDreamer (Liang et al., 2023). Additionally, we com- pared several recent significant baseline methods, including DreamFusion (Poole al., 2022), Fantasia3D (Chen et al., 2023), ProlificDreamer (Wang et al., 2024b), and Lucid- Dreamer (Liang et al., 2023). For NeRF-based methods, we evaluated 15 prompts from Magic3D (Lin et al., 2023) and 415 prompts from Dream (Poole et al., 2022). For 3D based methods, where initialization significantly im- pacts the final results, we selected reproducible prompts from recent works for evaluation.
IN2N (Haque al., 2023)
0.2334
23,76
DDS (Hertz al., 2023)
0.2030
4.52
PDS (Koo et al., 2024)
0.2395
30.20
Ours
0.2498
41.52
Table The quantitative comparison of 3D editing performance hetween method and others. Our approach quantitatively out- performs the baseline methods. Bold text indicates the best result.
Methods
CLIP Score
User Preference Rate (9)
DreamFusion (Poole al., 2022)
0.2838
8.34
Fantasia3D (Chon 2023)
0.2813
13.30
ProlitieDreamer (Wing et al,, 20/24))
0:2719
Ours
0.2962
42.14
LacidDreumentSSSS (Liung al., 2023)
0.2384
10.24
LacidDreemen lang et 2023)
0,2897
41.39
Ours
Table 2. The quantitative comparison of 3D generation between our method and others. Our approach quantitatively outperforms the baseline methods, Bold text indicates the best result.
ilar to ISM, while effectively preserving and transferring detailed information during denoising.
5.2. Text Guided 3D Editing
5. Experiments
Results. Figure presents the qualitative comparisons of
3D editing with baseline methods. The editing results of the
5.1. Experiment Setup
IN2N method are generally darker in color, and there instances where editing is unsuccessful. For example, in row
For 3D editing, we implemented all methods using NeRFstu- dio and evaluated our approach, along with several baselines, on 8 scenes from real-world datasets provided by IN2N, us- ing 37 pairs of source and target text prompts. We compared our method against three baseline approaches: IN2N (Haque et al., 2023), DDS (Hertz et al., 2023), and PDS (Koo et al., 2024). Since IN2N based on IP2P, which specializes in
1, the attempt to transform bamboo into an apple tree failed to exhibit any characteristics of an apple tree. Regarding DDS, all results display over-smoothing and over-saturation, and row 1, the editing fails completely: entirely loses the identity the input scene, focusing solely on matching the input text. Although PDS achieves satisfactory results,

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
DDIM laverse
Tweedie's Formula
DreamFusion
Fantasia3D
SDS CFG-100
Ours CFG-7.5
Figure 4, Ablation for different approximate methods.
Figure 5. Ablation for SDS (Poole al., 2022) and UDS with different generation frameworks.
also demonstrates instances of over-editing, such as in row I and row 2. In row 1, the Apple company's logo is inserted and the background is altered; in the second row, the clown's hair shape does not align with that of the original input. In contrast, our method makes appropriate modifications while best preserving the original identity information of the input scene. For instance, the clown's hair shape in the face remains unchanged.
Wo Inverse
Figure 6. Ablation for generation task. Add noise by DDIM inverse strategy.
To quantitatively evaluate our editing results, we measured the CLIP score (Radford et al., 2021), which assesses the similarity between the edited 2D renderings and the target text prompts in the CLIP embedding space. As shown in Ta- ble 1, our method outperforms the baselines in quantitative metrics. This is further confirmed by the qualitative results in Figure Figure 2. where other baseline methods struggle to produce clear textures, especially in scenes with complex de- tails. To further assess the perceptual quality of the editing results, conducted user study comparing our method with the baselines. Following the setting used in PDS, users are shown the input 3D scene videos. the editing prompts, and the edited 3D) scene videos produced by our method and the baselines. They are then asked to choose the most appropriate edited 3D scene video. As shown Table 1, our editing results significantly outperform the basclines in human evaluation, receiving 41.25% of the selections compared to 30.20% for PDS, which was the second best, Ablation study. For approximate the æo we can employ single denoise Tweedic's formula and multi-steps DDIM inverse processes. In the Figure 4, using a multi-step DDIM inversion process approximation better preserves identity information, such as facial features. In contrast, a single- step approximation with Tweedie's 's formula more accurately reflects the input text. However, the multi-step DDIM inver- sion process increases the optimization time.
crown generated by our approach more closer to the text input, exhibiting a more precise geometric structure and realistie colors. Compared the Schnauzers produced by other methods, the Schnauzer generated by ours features hair textures and overall body shape that are closer to reality, with clearer and more detailed features.
As with the editing task, we use CLIP scores to quantita- tively evaluate the generation results. Table 2, our method outperforms the baseline in quantitative metrics. For fair comparison, we evaluate NeRF and 3D GS separately: we implement our method in the DreamerFusion framework to evaluate NeRF-basee methods, and we implement method in the LucimDr framework to evaluate 3D GS- based methods. To further evaluate the perceptual quality of the generated results, we also conduct user study to com- pare our approach with baselines. In Table 2, our editing results are significantly better than the baselines human evaluation, whether the NeRF-based method or the 3D GS-based method.
Ablation study. To verify the effectiveness of our method, we implemented UDS in DreamFusion and Fantasia3D frameworks respectively. In Figure 5, our method generates clearer and more realistic details compared to SDS. Addi- tionally, in Figure 6, we perform ablations on DDIM reverse process noise addition. Our results show that while this approach improves the quality of generated results, also increases time and resource costs.
5.3. Text-to-3D Generation
Results. Figure present the qualitative comparisons of text-to-3D generation with baseline methods. We all use the stable diffusion 2.1 for distillation and all experiments are conducted on a 3090 GPU for fair comparison. Our method achieves high-fidelite and geometrically consistent results while requiring less time and fewer resources. The
6. Conclusion
In this work, we aim to explore unifying the SDS variant method into comprehensive one applicable to both editing
8

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
and generation tasks. We investigate two SDS-based 3D scene editing, DDS and PDS, analyzing them identify their successes and limitations. We observe remarkable similarities between these methods and the gradient terms used in recent DDIM-based SDS variants, although they play different roles in each task. Upon the SDS-based 3D editing method, we introduce UDS, an SDS variant method capable of both editing and generation objectives. Extensive experiments demonstrate the effectiveness of our method.
ers. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(11):12747-127733 2023.
Ghosal, D., Majumder, N., Mehrish, A., and Poria, S. Text- to-audio generation using instruction sennte Ilm and latent diffusion model. arXiv preprint arXiv:2304.137.1 2023.
Guo, Y.-C., Liu, Y.-T., Shao, R., Laforte, C., Voleti, V., Luo, G., Chen, C.-H., Zou, Z.-X., Wang, C., Cao, Y.- P., and Zhang, S.-H. threestudio: A unified framework for 3d content generation. https: /tt::://ittttt com/ threest peadllo-poojec/ttheeee
Acknowledgements
2023.
This work was supported by the Royal Society In- ternational Exchanges Scheme-Towarde Collaborative Cloud-Edge Deep Learning Deployment under Grant IEC/NSFCC/23522; National Edge AI Hub for Real Data: Edge Intelligence for -yber-disturbances and Data Quality EP/Y028813// 131; and UK Medical Research Council (MRC) Innovation Fellowship under Grant MR/S003916/2.
Haque, A., Tancik, M., Efros, A. A., Holynski, A., and Kanazawa, A. Instruct-nerfZnerf. Editing 3d scenes with instructions. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 19740-19750, 2023.
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-Or, D. Prompt-to-prompt image editing with cross attention cont)ol.(2022). URL https:://axxi. org/abs/2208.016266 2022.
Impact Statement
This paper presents work whose goal advance the field of 3D Generation and Editing. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Hertz, A., Aberman, K., and Cohen-Or, D. Delta denoising score. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2328-2337, 2023.
References
Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Zhang, Q., Kreis, K., Aittala, M.. Aila, T., Laine, S., et al. ediff- i: Text-to-image diffusion models with an ensemble of expert denoisers, arXiv preprint arXiv:2211.01324, 2022.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Brooks, T., Holynski, A., and Efros, A. A. Instructpix2pix: Learning to follow image editing instructions. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-18402. 2023.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106 106.09685. 2021.
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P.-A., and Li, S. Z. A survey generative diffusion models. IEFE Transactions on Knowledge and Data Engineering, 2024.
Hua, L., Liu, F., Su, J., Miao, X., Ouyang, Z., Wang, Z., Hu, R., Wen, Z., Zhai, B., Long, Y., et al. Attention in diffu- sion model: A survey, arXiv preprint arXiv:2504.03738, 2025.
Chen, R., Chen, Y., Jiao, N., and Jia, K. Fantasia3d: Disen- tangling geometry and appearance for high-quality text- to-3d content creation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22246-22256, October 2023.
Huang, R., Huang, J., Yang, D., Ren, Liu, L., Li, M., Ye, Z., Liu, J., Yin, X., and Zhao, Z. Make-an-audio: Text-to- audio generation with prompt-enhanced diffusion models. In International Conference on Machine Learning, pp. 13916-13932. PMLR, 2023.
Chung, H., Kim, J., Mccann, M. T., Klasky, M. L., and Ye, J. C. Diffusion posterior sampling for general noisy in- verse problems. arXiv preprint 22X. 14687, 2022.
Suberman-SSiieelglaa I., Kulikov, V., and Michaeli, T. An edit friendly ddpm noise space: Inversion and manipu- lations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12469 12478, 2024.
Duan, H., Long, Y., Wang, S., Zhang, H., Willcocks, C. G., and Shao, L. Dynamic unary convolution in transform-

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
Jain, A., Xie, A., and Abbeel, P. Vectorfusion: Text-to-svg by abstracting pixel-based diffusion models. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1911-1920, 2023.
Miao, X., Duan, H., Bai, Y., Shah, T., Song, J., Long, Y., Ranjan, R., and Shao, L. Laser: Efficient language-guica guuddee segmentation in neural radiance fields. IEEE Transactions on Pattern Analysis & Machine Intelligence, (01):1-13, 2025.
Katzir, O., Patashnik, O., Cohen-On D., and Lischinski, D. Noise-free score distillation. In ICIR, 2024.
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, T., Ramamoorthi, R., and Ng. R. Nerf: Representing scenes as neural radiance fields for view synthesis. Communica- tions of the ACM, 65(1 (5(1)99--166 2021.
Kerbl, B., Kopanas, G., Leimkühler, T., and Drettakis, G. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL https://rrepo sampppo-smm... Anria.fr: Eungraph/ httpp///ddddaasssaaaaaaaaaaaa
Mildenhall, B., Hedman, P., Martin-Brualin R., Srinivasan, P. P., and Barron, J. T. Nerf in the dark: High dynamic range view synthesis from noisy raw images. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16190-16199, 2022.
Koo, J., Park, C., and Sung, M. Posterior distillation sam- pling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13352- 13361, 2024.
Park, J., Kwon, G., and Ye, J. C. Ed-nerf: Efficient text- guided editing of 3d scene using latent space nerf. arXiv preprint arXiv:2310.0277
Lee, Y., Kim, K., Kim, H., and Sung, M. Syncdiffusion: Coherent montage via synchronized joint diffusions. Ad- vances in Neural Information Processing Systems, 36: 2023.
2023.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., and Rombach, R. Sdxl: Im- proving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.0/1 000.0//2... 2023.
Li, Y., Dou, Y., Shi, Y., Lei, Y., Chen, X., Zhang, Y., Zhou, P., and Focaldreamer, B. N. Textdriven 3d editing via focal-fusion assembly, arXiv preprint arXiv:2308. 10608, 3(5), 2023.
Poole, B., Jain, A., Barron, T., and Mildenhall, B. Dream- fusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Liang Y., Yang, X., Lin, J., Li, H., Xu, X., and Chen, Y. Luciddreamer: Towards high-fidelity text-to-3d gen- eration via interval score matching. arXiv preprint arXiv:2311.11288. 2023.
Qu, Y., Chen, D., Li, X., Li, X., Zhang, S., Cao, L., and Ji, R. Drag your gaussian: Effective drag-based editing with distillation for 3d gaussian splatting. arXiv preprint arXiv:2501.1867. 2025.
Lin, C.-H., Gao, J., Tang, L.,, Takikawa, T., Zeng, X., Huang, X., Kreis, K., Fidler, S., Liu, M.-Y., and Lin, T.- Y. Magic3d: High-resolution text-to-3d content creation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision, 2021. URL https: / arxiv org/ abs 2t..//1131..0 00020.
Liu, J., Huang, X., Huang, T., Chen, L., Hou, Y., Tang, S., Liu, Z. Ouyang, W., Zuo, W., Jiang, J., et al. A comprehensive survey on 3d content generation. arXiv preprint ar Ar1i::2402.01 166, 2024.
Saharia, C., Chan, W., Saxena, S., Li, L., Whang. Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image dif- fusion models with deep language understanding. Ad- vances in neural information processing systems, 35: 36479-36494, 2022.
Liu, L., Gu, J., Zaw Lin, K., Chua, T.-S., and Theobalt, C. Neural sparse voxel fields. Advances Neural Informa- tion Processing Systems, 33:15651: 15551116636,, 2020.
Song, J., Meng. C., and Ermon, S. Denoising diffusion implicit models. arXiv:2010.02502, October 2020a. URL https: http://axvvvv..//e httpp/////xxxi..o///bbbbb/e/// wttp://ggggap/a////eoas 02502.
Luo, S., Tan, Y., Huang, L., Li, J., and Zhao, H. La- tent consistency models: Synthesizing higo-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
McAllister, D., Ge, S., Huang. J.-B., Jacobs, D. W., Efros, A. A., Holynski, A., and Kanazawa, A. Rethinking score distillation as a bridge between image distributions. arXiv preprint 4006 094/7, 2024.
Song, Y., Sohl-Dickstein, J., Kingma. D. P., Kumar, A., Er- mon, S., and Poole, B. basre generative modeling
10

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
through stochastic differential equations. arXiv preprini arXiv:2011.13456, 2020b.
Wang, H., Du, X., Li, J., Yeh, R. A.. and Shakhnarovich, G. Score jacobian chaining: Lifting pretrained 2d dif- fusion models for 3d generation. In Proceedings of the IEEF/CVF Conference Computer Vision and Patlern Recognition, pp. 12619-12629, 2023.
Wang, J., Fang, J., Zhang, X., Xie, L.., and Tian, Q. Gaus- sianeditor: Editing 3d gaussians delicately with text in- structions. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 20902- 20911, 2024a.
Wang, Z., Lu, C., Wang. Y., Bao, Li, C., Su. H., and Zhu, Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36, 2024b.
Wu, Z., Zhou, P., Yi, X., Yuan, X., and Zhang. H. Consis- tent3d: Towards consistent high-fidelity text-to-3d gener- ation with deterministic sampling prior. arXiv preprint arXiv:2401.090500 2024.
Xu, Y., Zhu, L.. and Yang. Y. Gg-editor: Locally editing 3d avatars with multimodal large language model guid- ance. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 10910-10919, 2024.
Yu, X., Guo, Y.-C., Li, Y., Liang. D., Zhang. S.-H., and Qi, X. Text-to-3d with classifier score distillation. arXiv preprint ar arXiv:2310. 1../3/1.9999//5. 19415, 2023.
Zhang. R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 586-595, 2018.
Zhuang. J., Wang, C.,, Lin, I.., Liu, L.., and Li, G. Dreamed- itor: Text-driven 3d scene editing with neural fields. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-10, 2023.
Zhuo, W. Ma, F., Fan, H., and Yang, Y. Vividdreamer: invariant score distillation for hyper-realistic text-to-3d 3d generation. In European Conference on Computer Vision, pp. 122-139. Springer, 2024.

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
A. Appendix
SDS (CPG-100)
VSD (CRG-7.5)
ISM (CFG-7.5)
rybucaa - (0
A.I. Connection with other generation methods
We explore the connection between the proposed Unified Distillation Sampling (UDS) method and several other meth- ods by implementing 2D example (Figure 7). We under- stand that the role of the classifier-free guidance term is to continuously guide the sample to align with the text condi- tion, while the role of the reconstruction term is to restore the sample to its initial state.
In SDS, the cosine similarity the reconstruction term is low the beginning of training and exhibits large fluctu- ations during training, while the classifier-free guidance term also fluctuates throughout the training process. In high-dimensional mixed Gaussian distribution, if the opti- mization target is e, the maximum likelihood solution will appear at the origin. However, in reality, most of the prob- ability density should be concentrated on a sphere with a radius of vd centered the origin. This indicates that in the early stages of training, when the sample lacks semantic information, the reconstruction term can effectively restore the sample. As training progresses and the sample gradually acquires semantic meaning, the reconstruction term begins to fluctuate due to mode-seeking behavior. The instability of the reconstruction term causes the classifier-free guid- ance term to also fluctuate, ultimately leading to issues such as oversmoothing. ISM, VSD, and our UDS methods not only avoid the loss of details and oversmoothing that may be caused by the reconstruction term by replacing in the reconstruction term but also allow for a smaller weight (e.g., 7.5) to be set for the classifier-free guidance term to prevent oversaturation.
"A DSLR photoo of dolphins"
Figure 7. Connection with other methods. The cosine similarity between each item and the updated loss the SDS (Poole al.. 2022). VSD (Wang et al., 2024b), ISM (Liang al., 2023), and our UDS during the training process. The higher similarity, the greater its weight in the loss.
DDS (+) ,
POS
4 2
(1) -6") ) + +n
DDPM add moie
Single unconditionl denclsing
Single corditiona denoiving
Editing graficn dincction
Figure 8. The workflow of 3D editing DDS (Hertz et al., 2023), PDS (Koo al., 2024) and our UDS.
results, our method is closest to VSD, as evidenced by the similarity curve.
Unlike SDS, the cosine similarity of the reconstruction term in ISM, VSD, and UDS is negative and fluctuates signifi- cantly the beginning of training. As training progresses, the cosine similarity of the reconstruction term gradually ap- proaches zero. We believe this is because, at the beginning of training, the rendered image is in an out-of-domain state, and the differences between different time steps are substan- tial. For VSD, the reconstruction term continuously aligns the distribution of pre-trained samples with the distribution of samples trained by LoRA (Hu et al., 2021), enhancing the detail performance of the samples. For UDS and ISM, the reconstruction term aligns the sample with its state at the previous timestep, which not only improves detail preserva- tion but also ensures the coherence of the generation process. Additionally, compared with ISM, the reconstruction terms of UDS are aligned in an approximate latent space instead of directly on the noise. Consequently, the cosine similarity of the reconstruction terms in UDS approaches zero faster and exhibits smaller fluctuations. However, this also results in more significant fluctuations in the classifier-free guidance terms of UDS compared to ISM. Based on the generation
A.2. Workflows
As illustrated in the Figure 8, we present schematic work- flows of our UDS method alongside two baseline ap- proaches, DDS (Hertz et al., 2023) and PDS (Koo et al., 2024). Specifically, our method computes gradients in clean latent space distribution that is closer to the data distri- bution. In contrast, DDS calculates gradients on a pure noise distribution, while PDS performs gradient computations on a mixed distribution.
A.3. More experiments
1. TEXT GUIDED 3D EDITING
Additional visualization results. We present more edit- ing results in Figure 15.
Abltionn for different wy, As presented in the main text, the PDS can preserve identity recognition because in-

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
Generation
(b) Editing
004
Wm donior
X
X200
X400
X650
Figure I0. Ablation for different We the inverse process of DDIM to denoise and obtain xy at different time steps for editing.
iraniing Steps
Griinn Mtbis
Input
Output
Figure 9. The normalized gradient norm for generated and edited tasks for 3D.
troduces an edentity-preseervin term of Zo in the gradient, while our method directly combines this term with the classifier term as the gradient. Further obser- vation of PDS reveals that it actually consists of the identity- preserving term and the noisy gradient term DDS. When the coefficient ignored, combining the identity term with the unconditional noise term is equivalent adding the classifier-free guidance term to the latent space at certain time step t during the noise addition process of the diffu- sion model. We hypothesize whether it is possible to use the inverse process of DDIM to edit at any timestep Ia ob- tained. We conducted an ablation experiment, as shown in Figure 10, and the results show that editing can be success- fully performed any time step. In other words, editing can be successful as long as it is not combination of pure noise. Although no obvious changes are seen in Figure 10, what effect will the noise have on editing when zn is com- bined with noise? As shown in Figure 11, we used PDS to conduct an experiment in which the reference prompt is consistent with the target prompt, and the weight of CFG O, that is, the editing is completely dominated by To and unconditional noise. In theory, the resulting image should not have any changes, but as shown in Figure 11, the man's man's face and watch have obviously changed. This shows that random noise affects the restoration of the image, causing some colors and textures to be abnormal. This does have big impact in the editing task, but it will affect the generated results in the generation task.
Figure 11. Ablation reference and target consistent prompt. Even if the weight of the CPG is 0 the details of the image will change.
A.3.2. TEXT-TO-3D GENERATION
Additional visualization results. We present more gener- ation results Figure 14.
Ablation for different z;. We also investigate how the different 2, affect the generation process. As shown in Fig- ure 12, we observe that at higher timesteps, the colors of the generated 3D assets exhibit some abnormalities. For example, the sesame seeds and chopped green onions on the bagel gradually take on blue. As discussed Ap- pendix 1. this issue is caused by the higher timesteps introducing more random Gaussian noise, which may lead to such anomalies.
The gradient analysis. As shown on the left side of Fig- ure 9, we present the normalized gradient norms generated by the SDS, VSD, ISM, and our proposed UDS methods. is evident that UDS, ISM. and VSD with LoRA have similar gradient norm ranges. For UDS, ISM, and VSD, all set with CFG=7.5, UDS shows more stable gradient variations, while ISM and VSD exhibit more noticeable fluctuations. However, the fluctuations in VSD are smaller than those in ISM, as VSD uses LoRA to learn the distribution of 3D assets, whereas ISM experiences larger fluctuations due to additional accumulated errors introduced by noise dur-
The gradient analysis. As shown the right side of Figure we show the normalized gradient norms generated by DDS, PDS, and our proposed UDS method. can be seen that UDS and PDS are similar in the range of gradient norms. In contrast, the gradient change of the UDS method is more stable, while the gradient fluctuations of DDS and PDS are more obvious. This mainly because DDS and PDS need to set CFG=100 to achieve convergence, which may lead to instability in the training process.
2

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
User Preference
Methods
CLIP Score
LPIPS
Rate (%%
SDS (Poole 2022)
0.2507
0.4381
26.51
DDS (Hertz al., 2023)
0,2370
0.5236
19.64
PDS (Koo al., 2024)
0.2432
0,3803
26.21
Ours
0.2576
0.3489
27.64
Table 3. The quantitative comparison of SVG editing performance between our method and others. Bold text indicates the hest result in each column.
Xo
K20e
Figure 12. Ablation for different Ty- We use the inverse process of DDIM denoise and obtain 2; at different time steps for generation.
Imput
SDS
DDS
PDS
Ours
ing the reverse DDIM process, leading to instability in the reconstruction term, as discussed in Figure 7. While the differences in 2D results are minimal, these fluctuations cause inconsistencies in the local details of the generated 3D assets. Additionally, SDS with CFG=100 shows very large fluctuations. Although it eventually converges to stable range, the range is quite large, leading to instability in the optimization process and poor 3D asset quality.
"A cat as 3D rendered"
"A dog as 3D rendered"
"A baby penguin"
"A baby penguin wearing crown"
A.3.3. TEXT GUIDED SVG EDITING
We also conduct experiments on some SVG images from VectorFusion (Jain al., 2023).
"A drawing of cat "at" "A drawing of dog"
Results. Figure 13 presents detailed qualitative com- parison of text-guided SVG editing across various baseline methods. For consistency, we utilize the stable diffusion 1.5 model for distillation in all approaches, ensuring that the comparisons are performed under fair conditions. All experiments were conducted on an NVIDIA 3090 GPU to maintain fairnsss in terms of computational resources. As shown in Figure 13, while all methods successfully modify the input SVG according to the target text prompts, our UDS and PDS methods demonstrate superior performance in pre- serving the structural semantics the original SVG. This is particularly evident in maintaining the overall color scheme and structure of the input SVG, which is most prominent in the third row of Figure 13. The ability to retain these visual and semantic features distinguishes our methods from the baseline approaches. This qualitative advantage is further corroborated by the quantitative results. As indicated in Table 3, our method outperforms the baseline approaches by a significant margin in the LPIPS metric (Zhang et al., 2018), which is specifically designed to measure the percep- tual similarity and fidelity to the input SVG. This suggests that method maintains higher level of detail and con- sistency with the original SVG compared to other methods. Despite the higher fidelity, our CLIP score remains com- petitive, showing that our approach balances the trade-off between preserving the integrity of the input and effectively implementing the changes dictated by the text prompt.
A
"A pumpkin" "A banana"
Figure 13. Comparison with baseline methods in SVG editing. We present visual editing results for other methods and ours. Our method preserves obvious element information such ture and color.
addition, we also conduct a user study, the results which are summarized in Table 3. This user study followed the same setup the one we employed for 3D editing, ensur- ing a consistent evaluation framework. The results of the user study show that, in terms of subjective human evalua- tion, our UDS method performs comparably to the baseline methods SDS and PDS. This parity in human preference highlights the effectiveness and reliability of our approach, both from a quantitative and qualitative perspective.
A.4. Derivation of Unified Distillation Sampling
To derive the Unified Distillation Sampling (UDS) com- prehensively, we begin by expressing the gradient of UDS as:
3

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
multiple candidate 3D scenes edited using different meth- ods. We collected responses from total of 102 participants. Each participant randomly performed 50 to 100 trials, and their selection data were recorded for subsequent analysis. To ensure the diversity and fairness of the evaluation results, the order of presentation of candidate content was randomly arranged in 50 100 trials for each participant.
VeLins Bi (u(t) (ag"
26
+
(6
8a(0.0)
(20)
The term 2.
20 + (8
8ck sor
can be decomposed as:
[a]' aee (uFF,,0)) Ja (177" -VI-- Gie (102", (.))
(21)
A.6. Failure Case
In generation tasks, success depends on the initialization; the initialization is poor, the generation is likely fail. In editing tasks, when there a large gap between the prompt and the original scene, failure may occur. For example, when changing the prompt from "A photo of a plant" to "a photo of balls", the result often does not generate balls on the branches, but rather places them at random positions in the scene.
FGB (ar tgt , .)) cΦ (=x th .t)
(22)
Subsequently, this expression simplifies to:
1 18 (ui)"
(E
(1.)),))) .6,0) c0(a7. ..)))
(23)
Va
+6 (a / co (2x ,t)
(24)
Here, the latent noisy æ; is defined as:
E; = / 0, + VI- āGE
(25)
We introduce the following notations for simplification:
2src = EGp (2) src 11a 1)
E (ax 1.
(26)
v1 ā
det = Es
1st ye t)
E
(a
tet
0)
(27)
Thus, the gradient of the UDS loss function can be succinetly expressed as:
VoLuns Ere,o (u(t) (x
2g (2
)) (28)
0g10.0)
A.5. User Study Details
conducted a user study to evaluate the performance of different methods based on human preferences. In the generation task, we showed participants side-by-side com- parison of 3D assets generated by each method. In each trial, participants received text prompt and rotated video of multiple candidate 3D assets generated using different meth- ods. In the editing task, we showed the bideeb effect of each method editing a 3D scene. In each trial, participants received text prompt, reference scene, and videos of

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
"A DSLR photo of A Rugged, vuttgeeieppive hibing boots with weathered leather fintsh. best quality, 8K, HD.
"A baby bunny siting on top of stack of pancakes.
"A sliond loaf of fresh bread.
Punther De Ville car.
"A forbidden casale high up muuntains.
"a DSLR photo of cat wearing armor."
plate pited high chocolate chip cookies.
on adorabie enttage thatched roof."
deficious croissant."
duarian,
"A Gundam model, with detailed panel lines and decals."
"Marble of
Figure 14. More results generation by UDS. Please in for details.

---

Rethinking Score Distilling Sampling for 3D Editing and Generation
"a photo of a face
skull" n
woman
"... old man "
tolkien elf"
Emma Stone"
"a photo of a landscape" "
autumn "
desert" "
wwntter"
"a photo of a campsite" "
"... with volcano"
"a photo of a man"
Deadpool"
Hulk"
Figure 15. More results edited by UDS. Please zoom in for details.
6


Reviews:
[{"review_id": "isW5K06ZZn", "reviewer": "Reviewer_uKsu", "summary": "This paper points out that previous SDS and its variants have only performed well in generation or editing tasks. To address this, a unified approach integrating 3D asset generation and editing is proposed. The authors observe that the generation and editing processes in SDS and its variants share a common fundamental gradient term, leading to the introduction of UDS, which enables a unified approach to handling both 3D editing and generation tasks. Extensive experiments validate its effectiveness across various applications, demonstrating the capability of UDS in unifying 3D editing and generation tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments and valuable suggestions, which have greatly improved the quality of our paper. Below, we address your concerns and clarify potential misunderstandings:\n\n>  **References**\n\nWe will include the **Gaussianeditor** and **Drag Your Gaussian** discussions in the revised version.\n\n>  **Weaknesses**\n\nThank you for your careful review of our symbols. We will correct these errors in the revised version as follows:\n\n- $\\epsilon_\\theta$ -> $\\epsilon_\\phi$.\n- $\\underbrace{w(\\boldsymbol{\\epsilon} _ \\theta (\\boldsymbol{x}_t,t,y) - \\boldsymbol{\\epsilon} _ \\theta (\\boldsymbol{x}_t,t,\\emptyset))} _ {\\delta _ {\\boldsymbol{x}_t}^\\text{cls}}$ -> $w\\underbrace{(\\boldsymbol{\\epsilon} _ \\theta (\\boldsymbol{x}_t,t,y) - \\boldsymbol{\\epsilon} _ \\theta (\\boldsymbol{x}_t,t,\\emptyset))} _ {\\delta _ {\\boldsymbol{x}_t}^\\text{cls}}$\n- $\\delta_{\\boldsymbol{x}t}^\\text{recon}$ -> $\\delta_{\\boldsymbol{x}_t}^\\text{recon}$\n\n>  **Questions**\n\n**1.** We provide the training times for the generation and editing tasks using the Tweedie formula and multi-step DDIM Inversion. Please note that we found that in the generation task, the DDIM interval does not significantly affect the quality of the results, and the differences in the 3D assets produced by DDIM and the Tweedie formula are minimal. Therefore, to speed up the generation process, we directly use the Tweedie formula for the generation task. For the editing task, we found that DDIM affects the editing outcome; a shorter DDIM interval retains more of the original information, but it also increases the editing time. (We also conducted corresponding ablation experiments in Figure 4 in the main text.)\n\n**Table 1. Training time of DDIM (different time intervals) and Tweedie in the generation task. Note that all experiments are performed on a 3090.**\n\n| Method | Time |\n| ----------- | ----------- |\n| Tweedie | ~52 mins |\n| DDIM (400 steps) | ~1 hour 21 mins |\n| DDIM (200 steps) | ~1 hour 30 mins |\n| DDIM (100 steps) | ~1 hour 49 mins  |\n| DDIM (50 steps) | ~3 hour 17 mins |\n\n**Table 2. Training time of DDIM (different time intervals) and Tweedie in the edit task. Note that all experiments are performed on a A6000.**\n\n\n| Method | Time |\n| ----------- | ----------- |\n| Tweedie | ~5 hour 5 mins |\n| DDIM (400 steps) | ~8 hour 20 mins |\n| DDIM (200 steps) | ~10 hour 37 mins |\n| DDIM (100 steps) | ~17 hour 15 mins  |\n| DDIM (50 steps) | ~49 hour 30 mins |\n\n**2.** The time interval $c$ is inspired by recent approaches that use DDIM-based noise addition, such as ISM. In ISM, the gradient is guided by an $\\epsilon$ that is highly correlated with the data at two time steps; in contrast, our UDS guides the gradient using a latent embedding $x_0$ that is closer to the data at two time steps. We provide an ablation study on the time interval $c$ here ([Figure 2. Ablation on different time interval](https://anonymous.4open.science/r/15488/2.pdf)).", "claims_and_evidence": "The authors argue that DDS fails in 3D editing while PDS remains effective due to PDS providing an explicit identity preservation term. This claim is reasonable, as it represents the most significant difference between PDS and DDS. The authors support this argument with theoretical analysis and mathematical formulations.\n\nFurthermore, the authors propose a simplified version of PDS by retaining the identity preservation term and the classifier-free guidance (CFG) term while removing the \\delta^{recon}_{x_t} term to better control the editing process. This claim is also justified, as demonstrated through comparative experiments with PDS. Specifically, in the example of transforming “a photo of a bamboo” into “a photo of an apple tree,” PDS produces an unreasonable result by incorporating the Apple company logo, whereas the proposed method achieves a more appropriate editing outcome.", "methods_and_evaluation_criteria": "The experimental setup, comparisons with other methods, and evaluation criteria in this paper are reasonable and effective.", "theoretical_claims": "I have not verified all of the correctness of the theoretical claims and proofs.", "experimental_designs_or_analyses": "The experimental design and analysis in this paper are reasonable and effective. However, since this work is similar to ISM and introduces the hyperparameter time interval c in the generation task, I believe an ablation study on this hyperparameter is necessary.", "relation_to_broader_scientific_literature": "Previous works have often focused on either a single 3D generation task or a 3D editing task. For example, DreamFusion specializes in 3D generation, while PDS focuses on 3D editing, lacking a unified framework. Given the similarities between generation and editing tasks, this paper proposes UDS to integrate both into a unified approach.", "essential_references_not_discussed": "This paper focuses on 3D editing tasks but lacks citations of relevant literature on 3D editing for 3D Gaussian Splatting (3DGS), such as GaussianEditor [1] and DYG [2].\n\n[1] Gaussianeditor: Editing 3d gaussians delicately with text instructions\n[2] Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting", "other_strengths_and_weaknesses": "In equation (4), the symbol θ is used to denote the optimizable parameters of the 3D scene, but it is also used to represent the diffusion model parameter ε_θ. Referring to equation (2), where φ is used for the diffusion model parameter, I believe this is likely a typographical error. The same issue appears in multiple equations throughout the paper.\n\nIn equation (5), delta^{cls}_{x_t} includes the CFG weight w. However, referring to equations (11), (12), and (13), where w is multiplied by delta^{cls}, I suspect that delta^{cls}_{x_t} in equation (5) should not include the CFG weight w.\n\nIn line 206, the t in delta^{recon}_{xt} should be a subscript, but it appears to be a typographical error.", "other_comments_or_suggestions": "Reference to \"Other Strengths And Weaknesses\".", "questions_for_authors": "This paper compares the effects of the Tweedie formula and multi-step DDIM Inversion through ablation experiments. Since multi-step DDIM Inversion incurs significant time overhead, I would like to understand the exact difference in runtime between the two methods.\n\nI would like to gain a deeper understanding of the design rationale behind equation (17). Specifically, why was the time interval c introduced, and how does its value impact the results?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5cunGLO6Pl", "reviewer": "Reviewer_tQkB", "summary": "This paper focuses on score-sampling based text-to-3D generation and editing. The authors first analyze the variants of SDS (e.g., DDS and PDS) and identify significant commonalities in their gradient optimization processes. The authors further introduce a unified distillation sampling (UDS) that enables unified processing of text-to-3D generation and editing. Experiments on 3D editing and generation validate the superiority of the proposed UDS.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments and valuable suggestions, which have greatly improved the quality of our paper. Below, we address your concerns and clarify potential misunderstandings:\n\n>  **Methods And Evaluation Criteria**\n\nMost previous works have focused on a single 3D generation or editing task. Despite the natural and widely recognized formal similarities between editing and generation, no work has explored the two jointly. Our work attempts to investigate both tasks simultaneously and successfully uses the same framework to complete different tasks, which provides a theoretical and experimental basis for subsequent research based on SDS methods. In the generation task, most previous methods use $\\epsilon$-guided reconstruction terms for optimization, while our approach uses latent embeddings $x_0$ that are closer to the real data. Since $x_0$ has less noise, this makes the generation process more stable and the generation quality is improved. For the editing task, we prove through derivation that compared to PDS, we omit the implicit identity constraint term in the gradient. This enables editing to be performed in the regular CFG setting and accelerates convergence, thereby improving the editing effect.\n\n>  **Theoretical**\n\nThe term on the right corresponds to the gradient expression used in DDS, specifically,\n$\\boldsymbol{\\epsilon} _ \\theta (\\boldsymbol{x} _ t^{\\text{tgt}}, y^{\\text{tgt}}, t) - \\boldsymbol{\\epsilon} _ \\theta (\\boldsymbol{x} _ t^{\\text{src}}, y^{\\text{src}}, t)$.\nThe estimated noise of $\\boldsymbol{x}_t^{\\text{src}}$serves as a reference for preserving the identity of the source. Although DDS achieves strong editing results in 2D image applications, it suffers from quality degradation when applied to 3D editing. This degradation occurs because 3D editing requires stronger identity preservation than 2D editing. In DDS, the optimization process minimizes the difference between the SDS losses of the source and target, but it does not introduce any additional constraint to explicitly preserve identity. As a result, the lack of regularization allows the optimization to drift away from the source identity. In other words, the gradient term of DDS is not independent of the identity preserving term, but lacks an explicit identity preserving constraint.\n\n>  **Experimental**\n\nWe provide additional quantitative ablation experiments. Due to rebuttal period constraints, for the 3D GS-based generation tasks, we conducted ablation on only 20 out of the 43 prompts available in 3D GS. For the NeRF-based generation tasks, ablation experiments were performed on 15 prompts from Magic3D. For the NeRF editing tasks, we conducted ablation on 10 different prompts in the ‘man’ scene.\n\n**Table1. Ablation for different approximate methods. (NeRF-based editing)**\n\n| Method | CLIP Score |\n| ----------- | ----------- |\n| DDIM Inverse (200 steps) | 0.2595 |\n| Tweedie's Formula | 0.2544 |\n\n**Table2. Ablation for SDS and UDS with different generation frameworks. (NeRF-based generation)**\n\n| Method | CLIP Score |\n| ----------- | ----------- |\n| DreamFusion (SDS CFG=100) | 0.2616 |\n| DreamFusion (UDS CFG=7.5) | 0.2874 |\n| Fantasia3D (SDS CFG=100) | 0.2730 |\n| Fantasia3D (UDS CFG=7.5) | 0.2959 |\n\n**Table3. Ablation for generation task. Add noise by DDIM inverse strategy. (3D GS-based generation)**\n\n\n| Method | CLIP Score |\n| ----------- | ----------- |\n| W/o DDIM Inverse | 0.2863 |\n| W/ DDIM Inverse | 0.2977 |\n\n\n>  **References**\n\nThank you very much for recognizing our work. We will discuss NFSD, ODS, and ISD in the revised version.", "claims_and_evidence": "All claims are supported by theoretical analysis and experiments.", "methods_and_evaluation_criteria": "The proposed UDS seamlessly integrates 3D generation and editing. However, the benefits and potential applications of unifying them are unclear. How does the introduced UDS improve generation/editing quality compared to previous approaches?", "theoretical_claims": "I am wondering why the rightmost term is independent to the identity preservation term in Equation (9)?", "experimental_designs_or_analyses": "Each ablation experiment is presented with qualitative results using only one example. It would be more convincing to provide some quantitative results or more visual examples.", "relation_to_broader_scientific_literature": "SDS is one of the most important techniques in text-to-3D generation. Based on SDS, several variants are proposed for image editing (DDS) and text-to-3D editing. This paper analyzes recent variants of SDS (i.e., DDS and PDS) and introduces a unified UDS for text-to-3D generation and editing.", "essential_references_not_discussed": "Some most recent SDS/DDS variants for text-to-3D generation and editing are missing, which are highly related to this work and should be compared. For instance, NFSD [a], ODS [b] and ISD [c]  are introduced for text-driven 3D generation or editing. \n\n[a] Noise-Free Score distillation, ICLR'24\n\n[b] GG-Editor: Locally Editing 3D Avatars with Multimodal Large Language Model Guidance, MM'24\n\n[c] VividDreamer: Invariant Score Distillation for Hyper-Realistic Text-to-3D Generation, ECCV'24", "other_strengths_and_weaknesses": "- The overall paper is well organized and written.\n\n- Comprehensive experiments are provided covering text-to-3D generation and editing as well as SVG editing.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wwtihsxYBu", "reviewer": "Reviewer_u3Tp", "summary": "This paper tackles the problem of 3D asset generation and editing using SDS. The paper identify limitations in existing SDS variants especially for edting. The core contribution is a new method called Unified Distillation Sampling UDS. The key ideas behind UDS are: gradient term unification. A few other improvements like integration with classifier free guidance are also proposed.\nThe paper demonstrates its effectiveness through experiments on both 3D generation and editing tasks. It compares UDS against several baselines using quantitative metrics and qualitative visual results. Experiments are conducted on NeRF and 3D Gaussian Splatting representations, as well as on SVG editing.\n\nI will structure this review using a strengths and weaknesses template to better organize the points.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments and valuable suggestions, which have greatly improved the quality of our paper. Below, we address your concerns and clarify potential misunderstandings:\n\n>  **Weaknesses**\n\n**1.** We present a comparison of the optimization time of our method with that of PDS and DDS in the editing task. Please note that when using the same training steps, our method performs similarly to PDS and DDS; however, we found that the UDS method converges faster, requiring only one-third of the steps needed by PDS and DDS.\n\n\n| Method | Time |\n| ----------- | ----------- |\n| PDS | ~17 hour 20 mins |\n| DDS | ~17 hour 20 mins |\n| Tweedie (Our) Full steps | ~17 hour 20 mins |\n| Tweedie (Our) 1/3 steps | ~5 hour 42 mins |\n| DDIM (400 steps) (Our) 1/3 steps | ~8 hour 20 mins |\n| DDIM (200 steps) (Our) 1/3 steps | ~10 hour 37 mins |\n| DDIM (100 steps) (Our) 1/3 steps | ~17 hour 15 mins  |\n| DDIM (50 steps) (Our) 1/3 steps | ~49 hour 30 mins |\n\n**2.** We compare several feed-forward methods: the early ShapE, 3DTopia, and the latest TRELLIS. We use the 43 prompts mentioned in our paper for generation. Because it is difficult to find a metric that works well for these two kind of methods, we render only the front view and calculate the CLIP score for evaluation.\n\n| Method | CLIP Score |\n| ----------- | ----------- |\n| ShapE [1] | 0.2134 |\n| 3DTopia [2] | 0.2248 |\n| TRELLIS [3] |  0.2697 |\n| Ours (3D GS) | 0.2531 |\n\n[1] Shap-E: Generating Conditional 3D Implicit Functions\n\n[2] 3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors\n\n[3] Structured 3D Latents for Scalable and Versatile 3D Generation\n\n**3.** Thank you for your suggestion, we will include the following content in the revised version. \n\nWe conducted a user study to evaluate the performance of different methods based on human preferences. In the generation task, we showed participants a side-by-side comparison of 3D assets generated by each method. In each trial, participants received a text prompt and a rotated video of multiple candidate 3D assets generated using different methods. In the editing task, we showed the side-by-side effect of each method editing a 3D scene. In each trial, participants received a text prompt, a reference scene, and videos of multiple candidate 3D scenes edited using different methods. [Figure 4](https://anonymous.4open.science/r/15488/4.pdf) is a screenshot of the user study page. We collected responses from a total of 102 participants. Each participant randomly performed 50 to 100 trials, and their selection data were recorded for subsequent analysis. To ensure the diversity and fairness of the evaluation results, the order of presentation of candidate content was randomly arranged in 50 to 100 trials for each participant.\n\n>  **Questions**\n\n**1.** In generation tasks, success depends on the initialization; if the initialization is poor, the generation is likely to fail. In editing tasks, when there is a large gap between the prompt and the original scene, failure may occur. For example, when changing the prompt from \"A photo of a plant\" to \"a photo of balls\", the result often does not generate balls on the branches, but rather places them at random positions in the scene. We will include the above content in the revised version. \n\n**2.** We thank you for raising this question. In fact, SDS-based methods all rely on pre-trained 2D diffusion models, but our UDS makes improvements in gradient construction. Specifically, we use latent embedding $x_0$ to replace $\\epsilon$ of previous SDS-based methods. Compared with $\\epsilon$, $x_0$ is closer to the data and has less noise, thereby weakening the propagation of 2D model deviations in 3D results. In other words, the unified gradient signal we obtain is more stable, which has been reflected in the experiments that the generated and edited results are richer in details and less distorted.\n\n**3.** We provide ablation study on the time interval $c$ here ([Figure 2. Ablation on different time interval](https://anonymous.4open.science/r/15488/2.pdf)). We provide ablation study on CFG $w$ here ([Figure 3. Ablation on different CFG](https://anonymous.4open.science/r/15488/3.pdf)).", "claims_and_evidence": "see others", "methods_and_evaluation_criteria": "see others", "theoretical_claims": "didn't check carefully", "experimental_designs_or_analyses": "see others", "relation_to_broader_scientific_literature": "yes, related.", "essential_references_not_discussed": "no", "other_strengths_and_weaknesses": "1: The main strength is the proposed unification of 3D generation and editing within a single framework. This is a good contribution, as prior work often treated these as separate problems. The analysis of the gradient terms is  well-motivated.\n\n2: The paper demonstrates that UDS outperforms existing methods in editing tasks, particularly in preserving the identity and details of the original 3D asset while incorporating the desired edits. The visual results are compelling, and the quantitative results support this claim. The authors evaluate UDS on a wide range of tasks (NeRF generation/editing, 3D GS generation, SVG editing) and compare it against a comprehensive set of baselines. They include both quantitative metrics and user studies, providing a thorough evaluation.\n\n3: The paper provides a solid theoretical justification for UDS, deriving it from an analysis of existing methods. The connection to DDIM inversion for the reconstruction term is a clever idea.\n\nWeaknesses:\n\n1: While UDS achieves better results, the paper acknowledges that using DDIM inversion for the reconstruction term increases computational cost compared to simpler approximations. A more detailed discussion of the trade-off between quality and efficiency would be beneficial. It is unclear how much longer UDS takes compared to, e.g. PDS or DDS. Though in text-3D experiments USD seems to be fast when compared to other baselines. \n\n2: Performance gain seems to be not very significant in both editing and generation tasks.  For generation, can the performance be compared to feed-forward models like trellis (Structured 3D Latents for Scalable and Versatile 3D Generation) . trellis might be very hard to compete with (and its recently published) so maybe testing a few weaker feed-forward 3D gen models would also be beneficial. \n\n3: The user study gives little information. More detail, such as the number of participants, the number of samples presented to the participants, and the interface used, should be included in the appendix.", "other_comments_or_suggestions": "no", "questions_for_authors": "Are there specific types of edits or generation tasks where UDS might struggle? Are there failure cases to be aware of?\n\nUDS, like other SDS-based methods, relies on the quality and biases of the pre-trained 2D diffusion model. The paper doesn't address how the limitations of the 2D model might propagate to the 3D results. Will UDS be better than SDS in terms of this? \n\nHow robust is the method to variations in w, the time step interval (c), and other parameters? Any results or hyper-parameter analysis?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fgDPRr0mal", "reviewer": "Reviewer_jT1C", "summary": "This paper introduces a novel variant of SDS, called UDS, which is designed to handle both editing and generation tasks. Via qualitative and quantitative experiments, the authors show that UDS surpasses previous baselines.\n\n## update after rebuttal\n\nAs I noted in my official review, I remain unconvinced regarding the paper's contribution.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments and valuable suggestions, which have greatly improved the quality of our paper. Below, we address your concerns and clarify potential misunderstandings:\n\n>  **Experimental**\n\n**1.** As shown in the [Figure 1](https://anonymous.4open.science/r/15488/1.pdf), we provide a quantitative comparison of our NeRF-based results, namely UDS+DreamFusion, with RichDreamer and Dreamcraft3D.\n\n**2.** Thank you for your suggestion, we will include the following content in the revised version. \n\nWe conducted a user study to evaluate the performance of different methods based on human preferences. In the generation task, we showed participants a side-by-side comparison of 3D assets generated by each method. In each trial, participants received a text prompt and a rotated video of multiple candidate 3D assets generated using different methods. In the editing task, we showed the side-by-side effect of each method editing a 3D scene. In each trial, participants received a text prompt, a reference scene, and videos of multiple candidate 3D scenes edited using different methods. [Figure 4](https://anonymous.4open.science/r/15488/4.pdf) is a screenshot of the user study page. We collected responses from a total of 102 participants. Each participant randomly performed 50 to 100 trials, and their selection data were recorded for subsequent analysis. To ensure the diversity and fairness of the evaluation results, the order of presentation of candidate content was randomly arranged in 50 to 100 trials for each participant.\n\n**3.1** In our work, all methods use SD 2.1 for the generation task and SD 1.5 for the editing task.\n\n**3.2** We follow LucidDreamer to use Pointe to initialize 3D GS.\n\n**3.3** We have conducted relevant ablation experiments on these two estimation methods (Tweedie formula and DDIM multi-step denoising) in the main text.\n\n**3.4** Regarding the prompts, as described in the main text, for NeRF-based methods, we use 15 prompts from Magic3D and 415 prompts from DreamFusion (these prompts can be easily found on their official website or threestudio). For 3D GS-based methods, we will add a prompt list in the revised version. We have already shown about 30 prompts in the main text.\n\n**4.** Due to ICML policy, we can't seem to provide videos. As shown in [Figure 5](https://anonymous.4open.science/r/15488/5.pdf), we provide more comparisons with multiple views of LucidDreamer.  For fair comparison, we use the same hyperparameters.\n\n>  **References**\n\nWe will add citations to the revised version, e.g. DPS [1].\n\n[1] Diffusion Posterior Sampling for General Noisy Inverse Problems. ICLR (2023)\n\n>  **Weaknesses**\n\n**1.** In generation tasks, success depends on the initialization; if the initialization is poor, the generation is likely to fail. In editing tasks, when there is a large gap between the prompt and the original scene, failure may occur. For example, when changing the prompt from \"A photo of a plant\" to \"a photo of balls\", the result often does not generate balls on the branches, but rather places them at random positions in the scene. We will include the above content in the revised version. \n\n**2.** Please refer to **Experimental 4**.\n\n**3.** Our work focuses on exploring a natural and unified formulation of generation and editing tasks rather than on solving the consistency issue. We believe that the inconsistencies in the generated hands can be easily addressed by incorporating multi-view information and geometric constraints. However, this is not the primary focus of our work.\n\n>  **Other Comments**\n\n\nPlease note that Equation 12 actually contains some important coefficients of PDS, which we simplify in the main text for easier understanding, while Equation 14 is our proposed method.", "claims_and_evidence": "The main claim of this paper is that UDS enables unified processing of editing and generation tasks. However, the gradients for editing and generation correspond to two distinct variants of SDS (see Equations (14) and (16)). This distinction is even more apparent in Algorithm 1, where an explicit if-else branch separates the two tasks, with only basic time-step sampling and noise sampling shared. The definition of \"unify\" is somewhat ambiguous. Moreover, the observed performance improvements seem to stem from a more accurate estimation of $x_0$ using DDIM and the introduction of a negative classifier-free guidance term, rather than from unification itself. The authors might consider reframing their contribution as a refinement of the SDS loss, which can be applied to both the editing and generation variants of SDS.\n\nAnother claim is that the gradient of UDS is more stable. The authors argue that DDS and PDS rely on a large classifier-free guidance (CFG) scale, leading to instability, but they do not explain why ISM and VSD, which use a small CFG, also exhibit instability. Additionally, the paper lacks a clear justification for why a large CFG may cause instability and how such instability affects the final output. Providing further analysis or empirical evidence would strengthen this claim.", "methods_and_evaluation_criteria": "The method appears effective based on the provided results.", "theoretical_claims": "I have reviewed the proofs, and to the best of my knowledge, they are correct.", "experimental_designs_or_analyses": "The experiments are not convincing.\n\n1. **Lack of Comparisons with SOTA Methods.** The paper does not compare UDS against SOTA SDS-based methods such as RichDreamer[S1] and Dreamcraft3D[S2].\n2. **Insufficient Details in the User Study.** The user study lacks methodological details. conducted to evaluate the overall quality of the generated content lacks detailed methodology. Key aspects such as participant demographics, selection criteria, and statistical significance testing are not provided.\n3. **Missing Experimental Details.** Several crucial implementation details are absent, including:\n    1. The version of Stable Diffusion used.\n    2. The initialization method for 3DGS.\n    3. Whether Equation (10) or Equation (15) was used to estimate $x_0$ in UDS\n    4. A list of prompts chosen for NeRF-based and 3DGS-based methods.\n4. **Limited Comparisons in Figure 3.** Figure 3 only compares LucidDreamer to UDS using two cases, which is insufficient given the similarities between the two methods. A supplementary file with additional comparisons—especially in video format—would provide a more comprehensive evaluation.\n\n[S1] RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D \n\n[S2] Dreamcraft3d: Hierarchical 3d generation with bootstrapped diffusion prior", "relation_to_broader_scientific_literature": "The paper builds on prior work in text-to-3D generation, including DreamFusion, Magic3D, and LucidDreamer, while also relating to differentiable 3D representations such as NeRF and 3D Gaussian Splatting.", "essential_references_not_discussed": "In Section 4.2, Line 255, the paper states: *According to some recent work, we know that this approximation is inaccurate and may affect the preservation of identity information in editing tasks\", lack of citation*. However, no citation is provided to support this claim.", "other_strengths_and_weaknesses": "1. Lack of Discussion on Limitations and Failure Cases\n2. Static images are inadequate. Including rotated video of each object would better illustrate the method’s effectiveness.\n3. Notable inconsistencies exists in the generated hands (Figure 2, Last Row)", "other_comments_or_suggestions": "Section 4.2, Line 250 states *Equation (12) is a simplified representation*. But it appears that Equation(14) is the simplified version, while Equation(12) represents the original gradient term of PDS.", "questions_for_authors": "See above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Based on thorough and well-motivated analyses of SDS, DDS, and PDS, the authors formulated the unified approach toward text-to-3D generation and editing. Experiment and analysis are well-excuted and clearly conveyed. A major part of reviewers agreed that this work deserves acceptance after author rebuttal. The theoretical unficiation of DDS and PDS, leading to the proposed UDS is appealing and well-motivated. AC agreed with most of the reviewers' positive opinions, evaluating that they outweigh the minor weaknesses.
Decision: Accept (poster)