Diffusion Feature Field for Text-based 3D Editing with Gaussian Splatting
Authors: Eunseo Koh
Sangeek Hyun
MinKyu Lee
Jiwoo Chung
Kangmin Seo
Jae-Pil Heo
Venue: NeurIPS 2025 poster ()
Keywords: Text-based 3DGS editing
3DGS generation
Primary area: applications
TL;DR: We propose DFFSplat, a novel method integrating 3D-consistent diffusion features and a dual-encoder to resolve view inconsistency (Janus problem) in text-based 3D editing, achieving state-of-the-art structural and semantic consistency.

Abstract: Recent advances in text-based image editing have motivated the extension of these techniques into the 3D domain. 
However, existing methods typically apply 2D diffusion models independently to multiple viewpoints, resulting in significant artifacts, most notably the Janus problem, due to inconsistencies across edited views. 
To address this, we propose a novel approach termed DFFSplat, which integrates a 3D-consistent diffusion feature field into the editing pipeline.
By rendering and injecting these 3D-consistent structural features into intermediate layers of a 2D diffusion model, our method effectively enforces geometric alignment and semantic coherence across views.
However, averaging 3D features during the feature field learning process can lead to the loss of fine texture details.
To overcome this, we introduce a dual-encoder architecture to disentangle view-independent structural information from view-dependent appearance details. 
By encoding only the disentangled structure into the 3D field and injecting it during 2D editing, our method produces semantically and multi-view coherent edited images while maintaining high text fidelity.
Additionally, we employ a time-invariance objective to ensure consistency across diffusion timesteps, enhancing the stability of learned representations.
Experimental results demonstrate that our method achieves state-of-the-art performance in terms of text-fidelity, and better preserves structural and semantic consistency compared to existing approaches.

Diffusion Feature Field for Text-based 3D Editing with Gaussian Splatting
Eunseo Koh Jiwoo Chung
Sangeek Hyun Kangmin Seo
MinKyu Lee Jac-Pil Heo*
Sungkyunkwan University
[colorrain, hsi1032, bluelati98, wldn0202, skmskku, jaepilheo Bacelleoo@sskkk...kk @skku.edu
Abstract
Recent advances in text-based image editing have motivated the extension of these techniques into the 3D domain. However, existing methods typically apply 2D diffusion models independently to multiple viewpoints, resulting in significant artifacts, most notably the Janus problem, due to inconsistencies across edited views. To address this, we propose novel approach termed DFFSplat, which integrates 3D-consistent diffusion feature field into the editing pipeline. By rendering and injecting these 3D-consistent structural features into intermediate layers of a 2D diffusion model, our method effectively enforces geometric alignment and semantic coherence across views. However, averaging 3D features during the feature field learning process can lead to the loss of fine texture details. To overcome this, introduce dual-encoder architecture to disentangle view-independent structural information from view-dependent appearance details. By encoding only the disentangled structure into the 3D field and injecting it during 2D editing phase, our method produces semantically and multi-view coherent edited images while maintaining high quality editing. Additionally, employ time-invariance objective to ensure consistency across diffusion timesteps, enhancing the stability learned representations. Experimental results demonstrate that our method achieves state-of-the-arr performance in terms of CLIP similarity, and better preserves structural and semantic consistency compared to existing approaches.
Introduction
Recent advances in text-based image generation models [28, 29, 27, 22] have enabled their widespread application, particularly in text-based image editing tasks [35, 10, 1. 14]. Driven by increasing demand for intuitive and efficient editing of 3D content, recent efforts have focused on extending these text-based editing techniques from the 2D domain into 3D representations. Consequently, numerous rext-based 3D editing [9, 13. 30, 43, 4, 37, 3] approaches have been proposed, significantly broadening their applicability across diverse fields, including film production, game development.
The primary objective of text-based 3D editing is to modify a given source representation according to user-provided textual instructions. To edit a source 3D representation based the given textual instruction, existing approaches typically follow a two-step process. First, editing methods applied to multi-view images rendered from the source 3D representation. Subsequently, the 3D representation optimized to align with these edited 2D reference views.
*Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Baseline
Ours
Baseline
Ours
Source 3DGS
3D Diffusion Feature Field
Viewti
Viewa4
Viewe2
ViewA3
"Turn a stone bear into a panda" "
Figure Diffusion-based 2D editing exhibits multi-view artifacts such as Janus problem, visible in the model's intermediate features, that lead to inconsistent edits across viewpoints (referred to as Baseline). For example, independent editing of 2D multi-view images causes the panda's 's face incorrectly appear the bear's back. By enforcing reature-leved multi-view and semantic consistency through our 3D diffusion feature field, these errors are resolved, yielding coherent edits across all views (referred to Ours).
In 2D editing process, achieving multi-view consistency across edited images is essential. Multi-view consistency ensures that the edited scenes remain visually coherent across different viewpoints; inconsistent multi-view images inevitably introduce artifacts in the reconstructed 3D scene. However, existing text-based 3D editing methods utilize 2D diffusion models based editing approaches [1] that independently process each view, making it challenging maintain multi-view consistency.
Here, we focus on the notable artifacts in 3D editing and generation known the Janus problem [12, 24, 36]. Due to the inherent frontal-view bias in 2D diffusion models, frontal-view semantics attributes are often incorrectly synthesized onto multiple non-frontal viewpoints. This issue leads the semantic information to be misaligned in the multi- view images of source 3D during the 2D editing process. Interestingly, we observe that multi-view artifacts, like panda's face its back, already emerge in the diffusion model's s intermediate features, highlighting that enforcing feature-level consistency is crucial to prevent such errors in the final 3D edits, as shown Figure 1.
To this end, we propose DFFSplat, which augments the 2D editing phase with a multi-view consistent diffusion feature. Specifically, aim to build unified 3D feature field of the source scene that is shared across multiple views, and inject the rendered feature during the 2D diffusion editing [1]. Our key insight is that guiding the diffusion process with the rendered features already coherent in 3D) encourages both geometric alignment across views and semantic fidelity to the source content.
However, 3D features are fused across multiple viewpoints, they converge tation and may lose per-view texture details. Accordingly. we observe that directly injecting these averaged features results in blurred edits. From this design, hypothesize that the information lost due to its view-dependent nature during 3D embedding primarily related to appearance, whereas the preserved hew-indepenneet information retained in the feature field relates to the semantic structure.
averaged represen-
To address the issue of blurry artifacts, we propose isolating only the view-independent semantic information into the Gaussian Splatting [15] representation, while preserving the remaining view- dependent features directly from the editing process guided by the textual instruction. Given the strong correlation between structural layouts and semantic maps, anticipate that such an isolation strategy will effectively maintain per-view semantic consistency and simultancously enhance the visual quality. Finally, fuse edited appearance features with rendered 3D-consistent semantic features and inject them into selected intermediate layers of the diffusion model.
Experimentally, our method achieves the state-of-the CLIP similarity score. More importantly, as shown Figure 1, it effectively mitigates semantic distortions such as the Janus problem, preserving source identities across views. Additionally, we quantify semantic consistency by measuring the preservation of feature correspondences between image pairs before and after 3D) editing, where our method achieves the highest correspondence score among existing approaches.

---

2
Related work
Diffusion-based text-based editing Recently, with the rapid progress of text-based image gen- eration, text-based image editing has also advanced significantly. DiffusionCLIP [17] integrates CLIP-based optimization to refine edits while preserving identity. Prompt-to-Prompt (P2P) [10] introduces a method that leverages source attention maps guide cross-attention manipulation, enabling precise and straccure-preserving modifications through prompt changes. Plug-and-Play (PnP) [35] presents framework that injects spatial features from source image to achieve con- trolled image-to-iman translation guided by text prompts while preserving the source layout. Recent approaches like InstructPix2Pix [I] fine-tune models on synthetic paired datasets generated using Prompt-to-Prompt [10], allowing diverse and complex edits through single forward pass.
Building text-guided 2D editing techniques, most existing 3D editing approaches operate by applying edits independently to per-view renderings of the source scene, which often leads to multi- view inconsistencies. In contrast, we learn seometry-aligness 3D consistent feature field and inject its rendered features during the editing process, thereby enforcing consistency across viewpoints.
Text-based 3D scene editing Many existing text-based 3D scene editing methods build upon text- based 2D image editing techniques. Instruct-NeRF2NeRR [9] applies iterative view-by-view editing using a 2D diffusion model, followed by NeRF [21] optimization. Subsequent studies [32, 5, 40] have proposed various approaches to improve multi-view consistency and enhance the quality of 3D scene editing.
More recently, with the emergence of 3D Gaussian Splatting (3DGS) as a new 3D representation characterized by its remarkably fast rendering speed, several methods have been introduced to enable text-based editing 3DGS. GaussianEditor [4] proposes 3D Gaussian splatting editing framework enabling fast and controllable object manipulation through semantic tracing and hierarchical repre- sentations. Subsequent works such Direct Gaussian Editor (DGE) [3] further improved multi-vien consistency through geometry-awarr edits, however, it is limited to adjacent views. The works of GaussCtrl [38] and VcEdit [37] propose novel methods enforce multi-view consistency, however not considering the semantic consistency across views. Our DFFSplat framework addresses these limitations by explicitly encoding structural consistency within learned 3D diffusion feature field, significantly improving semantic and multi-view coherence.
3 Preliminary
3.1 Representation of feature field in 3DGS
Recent active research on 3D scene understanding [18, 6, 16, 19, 20, 31, 39, 33, 8] aims to embed semantic features into 3DGS [25, 42, 26] by assigning learnable semantic vectors to each Gaussian and training them to match ground-truth semantic maps. The rendering process of semantic features can formulated analogously to that of RGB images I E R3xHxW where H, W represent the height and width of the image size, as follows:
C= C=Eçar To T. iEg
(1)
ieg
where the ordered set of Gaussians contributing to a given pixel. C and F are the rendered RGB color and semantic feature vector, and c; and fr are color and semantic features of the i- th Gaussian, respectively. ag the product of learnable opacity parameter and the Gaussian's projected density at the pixel, and T, is the accumulated transmittance up to i-th Gaussian, defined as T [[ az). Unlike RGB values which are view-dependent and thus require spherical harmonic (SH) coefficients conditioned on viewing direction, semantic features f are inherently view-independent. This eliminates the need for additional SH coefficients during the rendering.
In addition, recent practices [25] utilize low-dimensional semantic latent compressed by an au- toencoder as ground-truth semantic features for reducing memory consumption. This process can be formulated as below:
= E(F), == D(Z),
(2)
3

---

1. Extract diffusion feature from multiple views
"Turn stone bear Into a panda"
XT
render
tume
Source GS
Diffisson features
Source GS
Edited GS
Train autnencoder
3. Truin diffusion feature field
nender
Fapp
Zapp
Page
Zpp
Diffusion feature
Zate
Eate
Se-ge Fquatte
Levce
(a) Training process of autoencoder and diffusion feature field
(b) Editing process of GS
Figure Overview of the proposed method. (a) To obtain 3D-consistent diffusion features, we train 3D diffusion feature field. However, rendering these features for editing averages out per-view details and sacrifices fine textures. To prevent this, we introduce dual-encoder scheme that decomposes structural and appearance features at the autoencoder level. Then, we train the feature field exclusively on these structural features, which have multi-view consistent geometry. (b) For 3D editing, we edit the multi-view source images by injecting the rendered structural features from our learned feature field. By keeping the appearance feature from images being edited, the results remain 3D consistent while faithfully reflecting the user-given edit text.
where E(-), (.) are encoder and decoder, and Z, F. are their compressed semantic features, uncompressed and reconstructed counterparts, respectively. The training objective for autoencoder minimizes the distance between reconstructed and ground-truth features, while the 3DGS objective minimizes the distance between rendered and compressed features.
3D Gaussian Splatting editing via diffusion feature field
In this paper, focus addressing the inconsistencies frequently encountered in existing text-based 3D editing approaches, such as the Janus problem, where independently edited multi-view images fail to maintain consistent geometry and semanties.
To overcome these issues, we propose novel editing framework that constructs 3D-consistent diffusion feature field, which is then rendered and injected into the editing process to ensure multi- view and semantic consistency in the edited images. Furthermore, we observe that directly using the learned 3D features in editing causes blurriness in the edited images, as its per-view appearance information are averaged while building 3D feature field. To resolve this, we introduce a dual- encoder architecture that separates structural features from appearance information, and only embeds multi-view consistent structural information within the 3D feature field. Finally, we elaborate how our 3D-consistent structural features integrated into the 2D diffusion editing pipeline and the subsequent 3D reconstruction process.
4.1 3D-coniistent diffusion feature field for semantic consistent editing
Diffusion features inherently encode structural semantics within their intermediate representa- tions [35], enabling explicit control over the layout of generated images during editing. Therefore, by constructing diffusion features that are coherent across multiple views (i.e., 3D-consistent), directly injecting these coherent features into a per-view 2D diffusion editing model can effectively produce geometrically view-consistent edited images.
To achieve this, we first construct a diffusion feature field by aligning and fusing per-view diffusion features extracted from multi-vies images of the source 3D scene. Once trained, this feature field provides 3D-consistent diffusion features, which we render and inject at key layers within the diffusion model. This explicitly encourages geometric consistency across multi-view images and semantic fidelity to the original content from source throughout the editing process. This strategy proves effective because diffusion features extracted from multi-view images inherently possess degree of semantic coherence. Moreover, this 3D diffusion feature field also facilitates correction of inconsistent or semantically biased features that may arise in individual views. Such inconsistencies

---

are typically (I) fiew-dependen and (2) relatively sparse across the entire set of viewpoints. Therefore, by explicitly averaging structural information across multiple views, the 3D feature field can correct biased views by utilizing structural information from unbiased views. This capability of cross-view averaging offered by the 3D feature field effectively resolves these biases.
Following prior work [25, 42, 26]. we first train the 3D diffusion feature field on the source 3DGS, as detailed in Sec. 3.1. However, diffusion features differ notably from typical vision model features due to the presence of a timestep t. Thus, we construct a timestep-conditioned encoder Ebase which has inputs as diffusion feature of specific timestep F and its corresponding timestep t. This process can be formulated as below:
Tuase E (F',
where 7 is compressed latent of Ft by the encoder E However, training 3D feature field based on this naive encoder has several issues as noted in below section, so we introduce a dual-encoder scheme.
4.2 Dual-encoder scheme for decomposing structure-apppearanee features
By rendering features from our 3D diffusion feature field, we inherently obtain 3D-consistent features. However, aggregating information across multiple viewpoints causes per-view appearance details to become averaged out. Consequently, we observe that directly injecting these averaged features during editing results in blurred outputs with insufficient fine textures. This averaging issue is further exacerbated by the inherently time-variant nature of diffusion features during the denoising process, making it challenging accurately represent these features within conventional 3D feature field. Consequently, directly injecting these averaged features during editing results in blurred outputs lacking sufficient fine textures. Although such representations might be acceptable in 3D recognition tasks [25, 42], which only need view-independent semantic information, they significantly limit editing applications, where preserving visual detail and fidelity to user-provided instruction crucial.
To address this limitation, we explicitly separate appearance information (fine, per-view textures, and time-variant from structural information (geometric, first train the diffusion feature field of 3DGS exclusively on structural information to ensure robust geometric consistency across views. During editing, fuse the rendered features from this field with appearance information specifically generated to align with user- asoviddd prompts. This combined representation enables our method produce comantic-consistett edits, accurately reflecting user inputs while effectively preserving viewpoint-specific appearance details.
and time-invariant). We
To this end, propose dual-encoder scheme that separates appearance and structural features at the encoder level. Specifically, the encoder is divided into two distinct branches: a structure encoder Eat and an appearance encoder Eapp. Following conventional autoencoder setups, the outputs from these two encoders are combined and subsequently processed through a shared decoder for reconstruction. This process can be formally described as follows:
Zamp (Fpp(F",t)) t), Z (F", F
D(Z Zin))
where Z and app are structural and appearance features encoded by corresponding encoders, respectively.
Then, in training phase, we explicitly differentiate the roles of the two encoders by imposing an additional constraint features to learn on the latent representation of the structure encoder. Specifically, we leverage the self-similarity maps derived from DINOv2 [2, 23] features, which have demonstrated effectiveness in capturing structural information [34]. These self-similarity map serve an auxiliary objective by minimizing their difference from the self silf-similarity maps generated from the structure encoder's 's latent output. Due the inherent information- bottllnnnb characteristic of autoencoders, enforcing structural constraints within the structure encoder naturally encourages the complementary appearance encoder to capture per-view, appearance- speccfcc details. This structure similarity loss is formulated as follows:
Lur | (T)) 5(Z ||| where S(K),) cos-sim( K,, Kj);
where t is sampling from ~ (T/n, (T/n,2T//:) 2T. /n........ and n is the number of timestep interval, cos-sim(...) is the cosine similarity between two feature maps and S(K) e ha hw is self-similarity map of given feature map K e gexhw where
......,)) T) where T is the number of diffusion timesteps
5

---

8,J E {l...., hw). By leveraging structure information extracted from the rendered image I of the source 3D scene using a DINOv2 extractor Fdino Eur effectively captures semantic information. To match the spatial resolution with DINOv2 features, we interpolate the DINOv2 features to the same resolution as the structure encoder's 's latent output.
We further enforce that the structure encoder's s latent outputs at different timesteps remain consistent, ensuring they can be represented by single, static 3D scene. These constraints can be formulated as below:
sim
1
Σ LIVEZ
Zar
Z
Zulll2
n(n - 1)
Lar is a structure similarity loss that imposes the latent of structure encoder Zur to have structural information mostly, and Si is time-similarity loss that encourages the structure encoder's s latent representations to be invariant across diffusion timesteps.
Training 3D diffusion feature field As shown in Figure 2-(a), we train our 3D diffusion feature field using the dual-encoder scheme described in Sec. 3.1. Concretely, we optimize an autoencoder with three loss terms: (1) standard reconstruction loss to recover per-view diffusion features, (2) structural similarity loss Car based on DINOv2 self-similarity maps, and (3) a similaiilrrrtt loss Lisim to enforce invariance across diffusion timesteps. In addition, we assemble the structure encoder's s latent outputs Z over all timesteps t [0, 7]] for each source view, average them to form static 3D target feature, and render this target via our 3DGS parameterization. We then train the 3DGS parameters f by minimizing the feature-map discrepancy between the rendered feature map F and the time-averaged target Zar. thereby grounding our 3D field in multi-view, time-invariant structural information. This process can be formulated as below:
Lae Laecon + Aur + XgLr-sim Lrecon 1 EiE(o.Ty d(F", FF),
Lieat d(F F, 7rar): where 7.EI E Zir.
(8)
where -( is a distance function (e.g., L2 distance), and Lae and Lreat are objectives of autoencoder and feature of 3DGS, respectively, and λ₁ 22 are the strengths of regularization for structure and time-invariance.
4.3 3D Gaussian Splatting editing via view-consistency guidance
In this section, we describe how the trained diffusion feature field of 3DGS and the dual-encoder mechanism are leveraged to effectively obtain multi-view edited images. In particular, given an instruction prompt from the user, our primary goal during editing to preserve the appearance information corresponding to the edit, while ensuring semantic consistency with the source scene.
To achieve this, we first encode the features obtained during the editing process using the appearance encoder Espp. Then, to enforce semantic consistency with the source, we simultaneously render view-consistent semantic features from the 3DGS semantic feature field, thereby providing essential source semantic information. These rendered 3D-consistent semantic features are subsequently fused with the encoded appearance features and then passed through the decoder to obtain reconstructed diffusion features. Finally, by injecting these diffusion features into the original editing pipeline (i.e., by replacing the model's native features with our diffusion features at selected intermediate layers), we produce multi-view edited images that faithfully reflect the user's prompt while accurately preserving the source semantics (Figure 2-(b)). This process is formulated as below:
Finjected = aapp (F*, (F",t))t F),
where Fir injeeeed is modified diffusion feature during editing, which contains 3D-consistent semantic information and appearance of editing condition.
Because this injection effectively supplies a structure-guided conditioning, we augment the original guidance [11] in InstructPi2Pix [I] with additional "view-consistency guidance" term. This combined guidance ensures that edits not only follow the user's 's text instructions but also remain geometrically coherent across all views. This process is defined as below:
éo 2t: C1, C8, er) = co(21, o, o, Φ) + 81 (20 (2e, C12 Ø1 Φ) co(z,, Φ.)
(10)
88 (co(2,, Cr, Cs, Φ) c0(=t: cr, 0, の)) ST (2, co cs,

---

Source image
Gaussian
GaussCtrl
VcEdit
DGE
Ours
Into pouda
"Ture
"Tum Jin
"Malke the dlgg look conge
Figure 3: Qualitative comparison of edited 3D scene. While existing methods often suffer from multi-view inconsistencies due to the inherent frontal-view bias of diffusion models, causing severe artifacts in the edited 3DGS outputs, our approach effectively preserves the original source structure and achieves faithful edits aligned with the input text prompt.
where ey and ég is diffusion model and its guided prediction, and 2, is input noised latent, and e and cş and er are the conditions about image and structure and text, and 8(.) are a strength of guidance. Underlined notations denote modification, the "view-consistency guidance" additionally use.
5 Experiments
5.1
Implementation details
We conduct the experiments on 5 scenes from IN2N [9] and DreamEditor [43], including complex 360 scenes. To validate the effectiveness of the proposed methods, we compare ours with previous 3DGS editing methods [4, 38, 37, 3]. For a fair comparison, we use InstructPix2Pix as an image editor, which is the most popular editor used by prior methods [4, 3]. During editing, utilize classifier-fre guidance with scales of 7.5 for textual conditions and 1.5 for image conditions, unless specified otherwise, For each scene, edit 20 views of the source scene and use the edited images as the reconstruction target to update 3DGS. For training details of 3DGS, we iteratively update the parameters of 3DGS for total of 1000 iterations, while image editing is performed every 500 iterations. We use Adam optimizer to update the 3DGS parameters and LPIPS [41] and L1 loss as reconstruction objectives, following IN2N [9]. For training our dual-eneoder, we set À 1.0 and X2 = 0.05. We further elaborate on the additional implementation and details in supplementary.
Evaluation metric Following the previous baselines [9, 37, 3], we employ CLIP similarity and CLIP directional similarity [17, 7] as evaluation metrics for quantitative comparison. CLIP similarity is computed as the cosine similarity between text and image embeddings, and clip directional similarity measures the difference in embedding directions from the target embedding to the source embedding between the image and text modalities. Both metrics evaluate how well the edited 3DGS reflects target prompt by comparing image and text embeddings, but clip directional similarity is known for its robustness to mode collapse. We use every image available in the training dataset for evaluation, although only use 20 views of images for editing.
5.2 Comparisons with previous works
Quantitative comparison with previous works We first compare CLIP and its directional simi- larity to measure the text fidelity of the edited scene. As reported in Table 1, the proposed method achieves significantly higher scores both CL.IP and clip directional similarity. Furthermore, ours reports the lowest editing time, similar to prior arts [3].

---

Table 1: Quantitative comparison. CLIP and clip dir. denotes CLIP and clip directional similarities, respec- tively.
Table 2: Ablation study of proposed com- ponents and 3D feature field, showing com- plementary gains text-image alignment.
Method
CLIP
clip Dir.
Avg. Time
Method
CLIP
CLIP Dir.
GaussianEditor [4]
24.40
16.67
~mmin
Ours (Full method)
25.11
18.15
GaussCtrl [38]
22.87
13.17
~mmn
Lisim
25.09
18.09
VeEdit [37]
23.18
14.03
~/2min
Eupp
24.80
17.21
DGE [3]
24.66
17.21
4-min
Ours (Full method)
25.11
18.15
Ours
25.11
18.15
4mm
3D feature field
25.00
17.82
Source image
Ours
wio E
w/o 3D feat. field
Distortion of Feature (MSE)
Kil Kage
with
hed
3.5
Bold
"Turn him into spider
3.0
with mose
1.5
1.0
200
400
600
800
1000
Diffusion Timestep (O)
Figure 4: Ablation study. (Left) Qualitative comparison of ablated models. (Right) Measuring distortion between the reconstructed diffusion feature (F²) and the modified diffusion feature (f across ablated models. Lower distortion indicates better preservation of source representation. The presence an appearance encoder (Fapp) reduces the distortion introduced by the 3D diffusion feature field (" ("feat. field" in the legend), demonstrating dual-encoder effectively preserves information.
inject
Qualitative comparison with previous works Qualitative results shown in Figure 3. The pro- posed method produces much fewer artifacts the rendered images from edited scene. In particular, observe the edited results from composition and semantics of the source object. It proves the effectiveness of learning the underlying structure of the source 3D scene via our 3D diffusion feature field.
achieve prominent semantic consistency, preserving the
For example, the "Turn stone bear into a panda" instruction (1st row), competing methods (e.g., VcEdit, DGE) fail to correct the diffusion model's s frontal-view bias, resulting in multi-view inconsistencies that synthesize visible artifacts the edited 3DGS. contrast, ours successfully edit the 3D scene without such artifacts, demonstrating the coherence of edited images across all views. Moreover, the "Turn him into Batman" prompt (2td row), our approach better preserves the source geometry than DGE, demonstrating stronger semantic fidelity the original 3D scene. Finally, as shown in the last row, unlike other methods that exhibit strong frontal bias, our approach does not display such bias even the backside of the object.
5.3 Analysis on the proposed components
Ablation study We perform an ablation study on our key proposed components. specifically the dual-encoder scheme with structural loss (Fapp and Lar) and the time-invariance objective sim) Note that a naïve decomposition of encoders without the structural loss does not effectively separate learned representations: thus, we ablate these components together.
As reported Table 2, the dual-encoder scheme substantially impacts performance, improving the clip directional similarity. This emphasizes the importance of explicitly decomposing structural and appearance information to achieve detailed, text-consistent 3D edits. Visual examples comparing our final model its ablations provided on the left side of Figure 4. Without the structure decomposition, the edited results lose per-view details, becoming overly blurred due to averaged features during editing.
We also ablate the time-invariance objective Li and quantitatively confirm that it boosts both the CLIP similarity and directional scores. Additionally, we evaluate the significance of learning the 3D

---

Table 3: Difference in correspondence match- ing map between multi-view source and edited images from complex 360 scenes. Our method demonstrates superior editing performance un- der sparse-view conditions compared to others.
Source
Ours
Editor
DGE
Method
Bear
Dog
GaussianEditor [4]
5,08
8,01
GaussCtrl [38]
10.91
8,15
VeEdit [37]
6.02
6,59
DGE [3]
4.14
7,36
"Make the dog look like corgi"
Ours
3,93
5.53
Figure 5: Correspondence matching comparison between two edited views of the dog scene.
diffusion feature field itself. That is, we directly inject the original 2D feature into the editing process, instead of rendering the 3D feature field. As shown Table 2, omitting this field ("-3D feature field" row) significantly decreases text fidelity, as the edited images lose multi-view consistency and appear more blurred during 3D reconstruction process. Qualitative comparisons in Figure 4 further illustrate the necessity of the learned 3D field for achieving detailed textures.
Finally, measure the distortion caused by feature injection in the reconstructed diffusion fea- tures (Finees)) across our ablated models. Note that, we give null text an editing condition for only validating the distortion caused by feature injection, not the editing capability. The lower distortion indicates the injected feature F does not lose its fine-details, it can recover these details the reconstructed feature Ft inject' As illustrated in Figure
conducted an ablation study the pres- absence of the dual-encoder scheme (specifically, the appearance encoder, Eapp) and the 3D diffusion feature field. The presence of Eupp reduces the distortion caused by the 3D diffusion feature field, demonstrating that our dual-encoder effectively preserves information. Additionally, in the case without Egpp but with the 3D diffusion feature field, the distortion increases as the timestep (noise-level) increases, indicating that the 3D diffusion feature field struggles accurately retain time-invariant noise information.
Evaluating multi-view and semantic consistency via correspondence matching Previous experi- ments mainly evaluated fidelity to the input text prompts. Here, we further assess how well the edited results preserve the original structure and resolve known editing artifacts, such as the Janus problem. To achieve this, compute pixel-level correspondence matching maps among multi-view images rendered from both the original and edited 3D scenes, and then measure the difference between these maps. A smaller difference indicates better preservation of the source scene's 's geometry and improved multi-view consistency due to accurate pixel-level matching across views.
Quantitative results for correspondence matching differences are presented in Table 3. As shown, our proposed method achieves significantly lower distortion correspondence matching compared to baselines. Qualitative visualizations of the correspondence maps are provided in Figure 5, where baseline methods show noticeable artifacts (e.g.. generating faces on the back of animal), indicating failures in preserving source structure. In contrast, approach consistently preserves the original structure, resulting in highly accurate correspondence maps.
Analyze the impacts of 3D feature field via correspondence matching To further analyze the individual impacts of DINO regularization (Cer) and the 3D feature field, we conducted the correspondence matching experiment by selectively removing each component. Specifically, compared scenarios without DINO regularization and without the 3D feature field, respectively. Since the appearance encoder cannot be trained effectively without DINO regularization, we performed the comparative experiments excluding the appearance encoder. In Table 4, we observe that excluding 3D feature field results significantly larger difference map compared to excluding DINO regularization. This clearly indicates that the 3D feature field substantially contributes to improving multi-view consistency and semantic consistency, rather than DINO regularization.
the correspondence matching

---

Table Correspondence matching score without DINO regularization or 3D feature field.
Method
Bear scene
Dog scene
w/o Eapp
2.96
5.28
w/o Eupp + w/o DINO regularization
3.02
5.21
w/o Eapp w/o feature field
3.15
6.10
"Make the bear look like grizzly bear"
Gaussian Editor
Ours
Iteration
Iteration 1000
Figure 6: Edited 3DGS results across iterations. While our baseline, GaussianEditor, suffers from the
Janus problem, our method effectively mitigates
Visualization of the edited 3DGS over optimization iterations We additionally visualize the evolution of the edited 3DGS over the course of optimization iterations. As shown in Figure our baseline method (GaussianEditor [4]) suffers from the Janus problem, where undesired facial attributes emerge on the backside of the bear at very carly iterations and gradually intensify as optimization progresses. In contrast, proposed method effectively suppresses such artifacts throughout the optimization process, maintaining consistent and faithful editing results across all viewpoints.
6 Limitation
Our proposed editing framework is inherently constrained by the editing capability of Instruct- Pix2Pix; therefore, it cannot handle 3DGS editing cases that go beyond the editing scope defined by InstructPix2PPxx This dependency fundamentally limits the diversity and complexity of the edits that can be achieved within our framework. Furthermore, since our method places a strong emphasis on preserving the structural integrity of the source 3DGS, it is less suitable for tasks that require substantial geometric transformations. As result, editing scenarios involving large pose changes, object repositioning, major structural alterations remain challenging. Addressing these limitations, either by incorporating more flexible editing modules or by enabling geometry-awan transformations, will be important direction for our future work.
Conclusion
We introduced DFFSplat, novel framework for text-based 3D editing that significantly improves multi-view and semantic consistency by incorporating 3D-consistent diffusion feature field. By leveraging dual-encoder architecture disentangle structural and appearance information, along with enforcing time-invann invariance across diffusion steps, our method effectively addresses prevalent artifacts such as the Janus problem. Experimental results confirm that our approach outperforms prior methods in terms of CLIP similarity and semantic coherence, highlighting its potential for high-quality 3D content editing.
Acknowledgements
This work was supported in part by MSIT/IITP (No. RS-2022-11220680, RS-2020-11201821, RS- 2019-1119042,, RS-2024-00459611 RS-2024-003362227 03-2034-00433633, RS-2024-00437102. RS- RS-2025-254425599 MSIT/NRF (No. RS-2024-003377729,, and KNPA/KIPoT (No. RS-2025- 25393280).
10

---

References
[1] T. Brooks, A. Holynski, and A. A. Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 18392-18402, 2023.
[2] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-9660, 2021.
[3] M. Chen, I. Laina, and A. Vedaldi. Dge: Direct gaussian 3d editing by consistent multi-view editing. In European Conference on Computer Vision, pages 74 92. Springer, 2024.
[4] Y. Chen, Z. Chen, C. Zhang, F. Wang, X. Yang, Y. Wang, Z. Cai, L. Yang. H. Liu, and G. Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pages 21476-21485, 2024.
[5] J. Dong and Y.-X. Wang. Vica-nerf: View-consistency--ne
3d editing of neural radiance fields. Advances in Neural Information Processing Systems, 36:61466-61177, 2023.
[6] Z. Fan, P. Wang, Y. Jiang, X. Gong, D. Xu, and Z. Wang. Nerf-sos: viy self-supervised object segmentation complex scenes. arXiv preprint arXiv:2209.08776 2022.
[7] R. Gal, O. Patashnik, H. Maron, A. H. Bermano, G. Chechik, and D. Cohen-Or Or. Stylegan Stylegan-nada: nada: Clip-guided domain adaptation of image generators. ACM Transactions on Graphics (TOG), 1133 2022.
[8] R. Goel, D. Sirikonda, S. Saini, and P. Narayanan. Interactive segmentation of radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4201-4211, 2023.
[9] A. Haque, M. Tancik, 3d scenes with instructions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19740-19750, 2023.
Efros, A. Holynski, and A. Kanazawa. Instruct-ner/Znerf: Editing
[10] A. Hertz, R. Mokady. J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or. Prompt-to- prompt image editing with cross attention control. arXiv preprint Til::2008.01626 2022.
[11] J. Ho and T. Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
[12] Hong, D. Ahn, and S. Kim. Debiasing scores and prompts of 2d diffusion for view-consistent text-to-3d generation. Advances in Neural Information Processing Systems, 36:11970-11987, 2023.
[13] H. Kamata, Sakuma, A. Hayakawa, M. Ishii, and T. Narihira. Instruct 3d-to-3d: Text instruction guided 3d-to-3d conversion. arXiv preprint arXiv:2303.1 15780, 2023.
[14] B. Kawar, S. Zada, O. Lang, O. Tov, H. Chang, T. Dekel, I. Mosseri, and M. Irani. Imagic: Text- based real image editing with diffusion models. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 6007-6017, 2023.
[15] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42((4):139 209,,,00233
[16] Kerr, M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19729-19739, 2023.
[17) G. Kim, T. Kwon, and J. C. Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2426-2435, 2022.
[18] S. Kobayashi, Matsumoto, and V. Sitzmann. Decomposing nerf for editing via feature field distillation. Advances in neural information processing systems, 35:233 11 1-23330, 2022.
11

---

[19] K. Liu, F. Zhan, J. Zhang, M. Xu, Y. Yu, A. El Saddik, C. Theobalt, E. Xing, and S. Lu. Weakly supervised 3d open-vocabulary segmentation. Advances in Neural Information Processing Systems, 36:53433 (6:53433-53456, 2023.
[20] K. Mazur, E. Sucar, and A. J. Davison. Feature-realittic neural fusion for real-time, open set scene understanding. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 8201-820) IEFE, 2023.
[21] B. Mildenhall, P. P. Srinivasan, M. Tancik, T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commumications of the ACM, 65(1):99-106,22 2021.
[22] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint
2021.
[23] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07199, 2023.
[24] B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.1 14988, 2022.
[25] M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20051-20060, 2024.
[26] R.-Z. Qiu, G. Yang, W. Zeng, and X. Wang. Feature splatting: Language- physics-basee scene synthesis and editing. arXiv preprint arXiv:2404.0122. 2024.
[27] A. Ramesh, Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint ar arXiv:2204.06125, 1(2):3, 2022.
(28) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[29] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gon- tijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion mod- els with deep language understanding. Advances in neural information processing systems, 35:36479- 33494, 2022.
[30] E. Sella, G. Fiehelman, P. Hedman, and H. Averbuch-Elor Vox-e: Text-guided voxel editing of 3d objects. In Proceedings of the IEEE/CVF international conference on computer vision, pages 430 440, 2023.
[31] N. M. M. Shafiullah, C. Paxton, L. Pinto, S. Chintala, and A. Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory. arXiv preprint arXiv:2210.0566,, 2022.
[32] L. Song, L. Cao, J. Gu, Y. Jiang, J. Yuan, and H. Tang. Efficient-ner/2nne streamlining text- driven 3d editing with multiview correspondence--nhannee diffusion models. arXiv preprint arXiv:2312.08563, 2023.
[33] N. Tsagkas, O. Mac Aodha, and X. Lu. VI-fields: Towards Language-groundee neural implicit spatial representations. arXiv preprint arXiv:2305. 2023.
[34] N. Tumanyan, O. Bar- Bar-Tal, S. Bagon, and T. Dekel. Splicing vit features for semantic appear- ance transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10748-10757, 2022.
[35] N. Tumanyan, M. Geyer, S. Bagon, and T. Dekel. Plug-and-play diffusion features for text- driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921 2023.
12

---

[36] T. Ukarapol and K. Pruvost. Gradeadreamer: Enhanced text-to-3d generation using gaussian splatting and multi-view diffusion. arXiv preprint arXiv:2406.09850, 2024.
[37] Y. Wang, X. Yi, Z. Wu, N. Zhao, L. Chen, and H. Zhang. View-consistent 3d editing with gaussian splatting. In European Conference on Computer Vision, pages 404-420. Springer, 2024.
[38] J. Wu, J.-W. Bian, X. Li, G. Wang. Reid, P. Torr, and V. A. Prisacariu. Gaussetrl: Multi-view consistent text-driven 3d gaussian splatting editing. In European Conference Computer Vision, pages 55-71. Springer, 2024.
[39] J. Ye, N. Wang, and X. Wang. Featurenerf: Learning generalizable nerfs by distilling foundation models. Proceedings the IEEE/CVF International Conference on Computer Vision, pages 8962 8973, 2023.
[40] I.. Yu, W. Xiang, and K. Han. Edit-diffnerf: Editing 3d neural radiance fields using 2d diffusion model. arXiv preprint arXiv:2306.0044 09551, 2023.
[41] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 586-595, 2018.
[42] S. Zhou, H. Chang, S. Jiang, Z. Fan, Z. Zhu, D. Xu, P. Chari, S. You, Z. Wang, and A. Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting enable distilled feature fields. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 21676-21685, 2024.
[43] Zhuang, C. Wang, L. Lin, L. Liu, and Li. Dreameditor: Text-driven 3d scene editing with neural fields. In SIGGR APH Asia 2023 Conference Papers, pages 1-10, 2023.
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency. research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduce well.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitation of this work would be in Supplementary.
14

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We include quantitative and qualitative comparison with previous works.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The implementation details of our main experimental results would be in Supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
15

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will release the code and data after accept.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips. /tippscc/pppii httpccccccpppccccccceeeeo httpci//gggiiee///eeuuuut
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We provide implementation details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification:
Guidelines: Could not report due to computation cost. Will report next time.
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification:
The experiments compute resources would be in supplementary.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
17

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics ftttps//neeurie httpps.cc/pp/e cc/public//eite httplliic/tticciiiellllliieiee
Answer: [Yes]
Justification: The research conducted in the paper conform with NeurIPS code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss this in the supplementary.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: We will include this in the supplementary.
Guidelines:
The answer NA means that the paper poses such risks.
18

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e-g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We attribute the original oweners of the data and code.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released. the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. dataett has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: We will provide an anonymized URL for the required content after accept.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing experiments.
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the core method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: battps://neurrs //tt:rries...///ee ce/ fec///nneerencee22255///////// for what should or should be described.
20


Reviews:
[{"review_id": "Yc1Ys8YlOV", "reviewer": "Reviewer_FBKe", "summary": "This paper proposes DFFSplat, a method designed to address multi-view inconsistency in the InstructGS2GS 3D editing pipeline, particularly the Janus problem. The core idea is to disentangle structural and appearance features from the intermediate features of a diffusion model using an autoencoder architecture. The structural feature is regularized by enforcing its self-similarity map to match that extracted by DINOv2. These structural features are then used to construct a 3D feature field, which enforces consistency across different views. At inference, the rendered structural feature is combined with the appearance feature and passed to the diffusion model for final image generation. Experiments show that this approach mitigates the Janus problem caused by view inconsistency.", "questions": "1. Please provide a more thorough ablation study to justify the role of the 3D feature field, including results averaged over five different random seeds with standard deviations.\n2. Include an ablation study where both the 3D feature field and the appearance encoder $E_{\\text{app}}$ are removed. This would help clarify the necessity and contribution of each component.\n3. Please discuss the relevance of GaussCtrl in relation to your work, and provide a direct comparison or justification for its exclusion as a baseline.\n4. Consider proofreading the manuscript to correct grammatical errors and clarify confusing sentences, particularly in Lines 52–56 and other areas mentioned in the review.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper identifies the root cause of the Janus problem as the frontal-view bias in diffusion models and proposes to address this by disentangling the structural feature map and regularizing it using the self-similarity map from DINOv2. This design encourages the structural representation of rear views to differ from the frontal-view representation, thus helping to avoid Janus artifacts. The motivation and intuition behind this approach are well articulated and logically sound.\n2. The paper is generally well-written and easy to follow, aside from a few grammatical errors and unclear sentences, which I outline in the Weaknesses section.\n\n**Weaknesses**\\\nI have two main concerns: one regarding the necessity and justification of the 3D feature field, and the other about the omission of an important baseline, GaussCtrl [1].\n\nNecessity of the 3D Feature Field:\n1. As discussed in the Strengths section, I believe the regularization of the structural feature via DINOv2 is the key factor in addressing the Janus problem. Once the structural features are regularized, the additional step of constructing a 3D feature field seems redundant. In fact, as acknowledged by the authors in Line 150, the feature field performs an averaging operation, which may dilute the fine-grained structural information. This weakens the justification for its inclusion in the pipeline.\n2. The paper attempts to justify the 3D feature field via an ablation study in Table 2. However, the differences in CLIP and CLIP Dir metrics are minimal and within a range that could easily be attributed to randomness. Given that the InstructGS2GS pipeline is known to be highly stochastic, these results do not provide convincing evidence. A more robust experiment would be to repeat the evaluations across five different random seeds and report the mean and standard deviation of the metrics.\n3. The plot on the right side of Figure 4 shows increased feature distortion when using the 3D feature field, which further reinforces my concern that this module may be doing more harm than good.\n4. I also question the role of the appearance encoder $E_{\\text{app}}$. Typically, disentangling tasks, especially in the context of appearance vs. structure, require explicit supervision. An example is [2], where pose and shape attributes are disentangled with appropriate supervision. In contrast, $E_{\\text{app}}$ is trained without supervision, raising doubts about whether it genuinely captures appearance information. Furthermore, since $E_{\\text{str}}$ is supervised only via the DINOv2 self-similarity map (which may already encode appearance to some extent), it is unclear whether $E_{\\text{app}}$ is necessary at all. It may be possible for a single encoder to learn a sufficiently disentangled representation on its own.\n\nMissing Baseline:\n1. In Line 35 of the introduction, the paper suggests that existing methods struggle with multi-view consistency, potentially implying that this work is the first to identify or address the issue. However, multi-view inconsistency is a well-known and long-standing challenge in 3D editing. A particularly relevant and recent work is GaussCtrl [1], which addresses geometric and appearance consistency by conditioning editing on both a reference view and a depth map. Its motivation is closely aligned with that of this paper, and it specifically claims to reduce the Janus effect. Despite its relevance, GaussCtrl is neither cited nor used as a baseline in this work, which is a significant omission.\n\nMinor Errors:\n1. Line 52: \"related\" → \"relates\"\n2. Lines 54–56: The sentence is awkward and grammatically incorrect. \"Isolating\" should be paired with \"from,\" not \"into.\" The sentence should be rephrased for clarity.\n3. Equation 4 indicates that the encoder takes a timestep $t$ as input, but this dependency is not reflected in Figure 2.\n\n**Conclusion**\\\nDue to the concerns outlined above, particularly the unclear necessity of the 3D feature field and the omission of a key related method (GaussCtrl), I currently recommend Reject. However, I am open to revisiting this recommendation if the authors can adequately address the points raised in this review.\n\n[1] Wu, Jing, Jia-Wang Bian, Xinghui Li, Guangrun Wang, Ian Reid, Philip Torr, and Victor Adrian Prisacariu. \"Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting editing.\" In European Conference on Computer Vision, pp. 55-71. Cham: Springer Nature Switzerland, 2024.\\\n[2] Zhou, Keyang, Bharat Lal Bhatnagar, and Gerard Pons-Moll. \"Unsupervised shape and pose disentanglement for 3d meshes.\" In European conference on computer vision, pp. 341-357. Cham: Springer International Publishing, 2020.", "was_revised": true, "final_justification": "Thanks to the additional results and information provided by the authors, my concern about the necessity of the 3D feature field and appearance encoder has been largely resolved. Therefore, I would raise my score to \"borderline accept\".", "rebuttal": "## [@FBKe] - On the Necessity of the 3D Feature Field\n\nWe appreciate Reviewer FBKe's thoughtful comments and would like to clarify the motivation and necessity of the proposed 3D feature field beyond the regularization of structural features using DINOv2.\n\nWhile the DINO-based regularization indeed helps extract semantically meaningful and spatially coherent structural features, we argue that the 3D feature field plays an indispensable role in enforcing structural consistency across views, which cannot be achieved by regularization alone. Specifically, we highlight three key justifications:\n\n### Viewpoint-Agnostic Semantic Fusion:\nAs stated in Sec. 4.1–4.2 of our paper, the 3D feature field aggregates diffusion features across multiple views into a unified representation. This operation not only removes view-specific noise but also harmonizes structural discrepancies caused by the diffusion model’s frontal-view bias (see Line 38–43). While this averaging process may appear lossy, it effectively neutralizes conflicting semantic cues across views and ensures geometric alignment at the feature level.\n\nTo evaluate this point, we conducted an ablation study assessing multi-view and semantic consistency via correspondence matching. Specifically, as mentioned in the paper, we compute pixel-level correspondence matching maps among multi-view images rendered from both the original and edited 3D scenes, and then measure the differences between these maps. A smaller difference indicates better preservation of the source scene’s geometry and improved multi-view consistency due to accurate pixel-level matching across views. As demonstrated in the table below, incorporating the 3D feature field consistently yields improved multi-view and semantic consistency. Please note that these results are computed as the mean across five random seeds.\n\n| Method | Difference in correspondence matching |\n| --- | --- |\n| Ours | **3.19** |\n| - 3D feature field | 3.66 |\n\n### Interpretation of Feature Distortion in Fig. 4:\nReviewer FBKe points out the increased distortion in Fig. 4 when using the 3D feature field. However, we argue that this result does not reflect degradation, but rather a necessary correction. The increased distortion stems from our 3D field overriding incorrect features (e.g., Janus artifacts caused by frontal bias) to enforce consistent multi-view structure. As further evidence, this distortion trend remains stable across time steps (if substantial fine-grained information had been lost, we would expect a larger distortion at low-noise timesteps), where appearance cues dominate (see Fig. 4 Right). The consistency of the distortion across timesteps supports our claim that the 3D field preserves fine-grained appearance via $E_{app}$ while prioritizing semantic alignment.\n\n### Empirical Improvements & Visual Coherence:\n\nAlthough the numerical gain in CLIP metrics shown in Table 2 may appear modest, the visual improvements are pronounced, especially in challenging cases (e.g., the panda face on the back in Fig. 1 & Fig. 3). These qualitative results highlight that the 3D feature field mitigates Janus-like inconsistencies that neither the structure encoder nor appearance encoder alone can address. We also acknowledge the stochasticity of the editing process and are preparing multi-seed results as the reviewer suggested, which will be included in the revised supplementary.\n\n---\nIn summary, the 3D feature field is not simply a redundant addition to the DINO-regularized encoder but rather a necessary mechanism to fuse multi-view semantic structure and eliminate artifacts rooted in the inherently 2D nature of diffusion editing. Without this explicit 3D fusion, rear-view inconsistencies and view conflicts persist (as confirmed in our ablations in Fig. 4 and Table 2).\n\n## [@FBKe] - Role and Necessity of the Appearance Encoder\n\nWe appreciate the reviewer’s careful concern of $E_{app}$. Below we clarify its motivation, training signal, and empirical contribution.\n\n### Separating What to Keep vs. What to Restore\n\nOur structural branch ($E_{str}$) is *aggressively* regularised: (1) DINOv2 self-similarity aligns global geometry, and (2) the time-invariance loss collapses features across *all* timesteps into a single 3D field. This design intentionally *filters out* view-dependent, high-frequency cues that will be edited (i.e., removed). $E_{app}$ is therefore indispensable for *re-injecting* those cues within the editing process; otherwise the edited outputs become over-smoothed (Fig. 4. left) and appearance information from prompt-conditions cannot be properly applied.\n\nHere we would like to re-emphasize that this is enabled by our carefully designed method that jointly learns $E_{app}$ (via a dual-encoder scheme) and the 3D feature field, allowing precise and controllable manipulation over 3D visual attributes. Importantly, such disentanglement is not optional but rather mandatory for effective 3D editing. For these reasons, we respectfully disagree that $E_{app}$ is redundant. It is a lightweight yet crucial component that (1) safeguards fine details, (2) enlarges the editing expressiveness of DFFSplat through disentangled control.\n\nWe sincerely appreciate the time and effort for the insightful review.\n\n\n### Synergy with the 3D Feature Field\n\nAbove, we have discussed the significant benefits and necessity raised by 3D field.\nHowever, this only holds with the assumption that our $E_{app}$ is accompanied.\nWithout this, Fig.4 shows that it indeed leads to non-neglectable information loss.\nThus, in order to fully retain the advantages of 3D field while not suffer from significant info loss, it must be jointly utilized.\n\nBelow, we provide more specified discussion.\n\nFig. 4 (right) shows that the apparent distortion introduced by 3D fusion *plateaus* when $E_{app}$ is present, whereas it *grows* with timestep if $E_{app}$ is removed. \nThis indicates that the view-specific noise/texture are reconstructed via the $E_{app}$-relevant path, which enables the 3D field to purposefully discard these information (beneficial in editing).\n\n \n### Single-Encoder Variant Fails to Disentangle\n\nThe dual-encoder shares a common decoder and is trained with an overall reconstruction loss. Because $E_{str}$ is tightly constrained, any residual information needed to minimize that loss *must* flow through $E_{app}$. This “information-bottleneck” effect acts as an implicit appearance supervision.\n\nAs discussed above the informational bottlneck designed via the _dual_-encoder is necessary.\nWe implemented the reviewer’s suggestion (one encoder + DINO loss). Across **5 seeds** the average score show significant drop. Also visually, Janus artifacts resurfaced (panda eyes on bear back; Batman double-ear). \n\nUnfortunately, visual examples are not allowed to be presented here.\nThe ablation analysis and visual results will be added in our final manuscript.\n\n| Method | CLIP (mean / std) | clip_dir (mean / std) |\n| --- | --- | --- |\n| Ours (Full Method) | 25.13 / 0.034 | 17.94 / 0.078 |\n| - 3D feature field | 25.09 / 0.037 | 17.87 / 0.226 |\n| - $E_{app}$ | 24.42 / 0.254 | 16.79 / 0.491 |\n\n[1] Yue, Zhongqi, et al. \"Exploring diffusion time-steps for unsupervised representation learning.\" *arXiv preprint arXiv:2401.11430* (2024).\n\n[2] Tumanyan, Narek, et al. \"Splicing vit features for semantic appearance transfer.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n## [@FBKe] - Missing Baseline\n\nThank you for highlighting the omission of the GaussCtrl baseline. GaussCtrl addresses geometric and appearance consistency on multi-view by conditioning edits on a reference view and a corresponding depth map. However, we observed that relying solely on depth maps often fails to capture sufficient semantic information, leading to semantic inconsistencies with the source scene. For instance, in our \"bear-to-panda\" experimental scenario, GaussCtrl generated inappropriate semantic artifacts, such as facial features incorrectly appearing on the panda's backside, thus exacerbating the Janus problem. This indicates that depth maps alone are insufficient for robust semantic guidance. In contrast, our method effectively maintains semantic consistency by the guidance of structure features from source scene.\n\nAdditionally, quantitative comparisons demonstrated that our approach achieves higher editing fidelity and faster inference speeds compared to GaussCtrl. We will include these detailed comparative experiments and an expanded discussion of GaussCtrl in our revised paper to clarify its relation to our proposed method.\n\n| Method | CLIP | CLIP Dir. | Time |\n| --- | --- | --- | --- |\n| GaussCtrl | 23.77 | 15.25 | ~8min |\n| Ours  | **25.29** | **17.69** | ~4min |\n\n[1] Wu, Jing, et al. \"Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting editing.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024."}, {"review_id": "cF9Ws6LVnU", "reviewer": "Reviewer_MgqZ", "summary": "The paper introduces DFFSplat, a pipeline for text-driven 3D scene editing in which both the source scene and the edited result are represented with 3D Gaussian Splatting (3DGS). The core idea is to learn a 3D-consistent diffusion feature field from multi-view diffusion features of the source scene and to inject these structural features back into a 2D diffusion editor (InstructPix2Pix) during per-view editing. A dual-encoder disentangles view-invariant structural features from view-dependent appearance, and a time-invariance regularizer makes the structural branch usable as a static 3D field. Editing is further steered by an additional view-consistency guidance term.", "questions": "## Questions & Suggestions\n\n### Time Budget\nI am confused by the claimed runtime of 4 minutes in Table 1, as a different number is mentioned in the supplemental material. Could you please clearly list all steps involved in your method—such as 3DGS training, feature field creation, dual-encoder inference, and editing—and indicate how much time each one takes in total?\n\n### Limitations\nYou marked \"Yes\" in the checklist regarding a discussion of limitations, but I could not find any such section in the paper. Please explicitly list your method’s limitations, either in a separate section or clearly within the main text.\n\n### Qualitative Consistency\nThe results in the main paper and supplemental differ significantly in visual quality. Could you clarify whether the same models, training setup, and hyperparameters were used for both? If not, what changed? Also are the results all single shots or multiple tries?", "limitations": "I could not find any explicit discussion of the limitations of the proposed method in the main paper, even though the authors marked “Yes” in the checklist. Before attempting to list these myself, I would like the authors to clearly address this point and explicitly state the key limitations of their approach.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Methodology\n\n### Strengths\n- Novel 3D diffusion feature field that explicitly enforces multi-view consistency. \n- Clean dual-encoder design with separate structural and appearance branches.\n\n### Weaknesses\n- Feature field training and the auto-encoder introduce additional complexity. A detailed compute breakdown is missing.\n\n---\n\n## Evaluation\n\n> Note: I initially considered giving a much lower evaluation score, as the results shown in the main paper did not appear very significant to me.\n\n### Strengths\n- Supplementary results look visually pleasing. \n- In some cases, the Janus problem appears to be mitigated.\n\n### Weaknesses\n- The main paper's results are not promising and only marginally better than prior work. For example, the panda remains partly grey, Batman has double ears (Fig. 3), and the Spiderman mask is on par with previous methods. Overall benefits remain subtle.\n- I would have expected some video material in the supplemental to illustrate view-dependent effects and multi-view consistency.\n\n---\n\n## Writing & Clarity\n\n### Strengths\n- The paper is generally well-structured.\n- Mathematical formulations are explicit.\n\n### Weaknesses\n- The related work section is short and lacks depth.\n\n---\n\n## Impact\n\n### Strengths\n- Tackles a high-value problem: reliable text-driven 3D editing.\n- The idea of merging diffusion features in 3D is promising and could inspire other use cases.\n\n### Weaknesses\n- The actual total compute time is unclear; see related question.\n\n---\n\n## Additional Comment\n\nI appreciate the core insight and find the idea of merging diffusion features in 3D highly interesting. Before reviewing the supplemental material, I was not convinced that DFFSplat delivers meaningfully better results than state-of-the-art NeRF or GS editors. The improvements shown in the main paper are small and occasionally even negative.\n\nHowever, the supplemental results are notably stronger, and I urge the authors to revise the main paper to match this quality. Currently, the main paper and supplemental feel disconnected. I also find the related work section too brief—it omits relevant methods that project features into 3D, especially from the scene understanding literature.\n\nFinally, although the authors use GS as their representation, they should still compare against other recent methods such as IN2N and SIGNeRF.\n\nI will give a **weak accept** for now because I like the core idea and want to acknowledge the promising results shown in the supplemental.", "was_revised": true, "final_justification": "I recognize the concerns raised by my fellow reviewers. Judging a paper that emphasizes qualitative performance without another round of qualitative results is challenging. However, given that the authors are not responsible due to the NeurIPS technical issues and the rebuttal addresses the critiques convincingly, I am inclined to give them the benefit of the doubt.\n\nI particularly like the base injection idea it seems both novel and potentially valuable for future work. The results, especially those in the supplementary materials, appear strong, and the runtime improvements are noteworthy. While the quantitative CLIP results remain somewhat inconclusive, they do not detract significantly from the paper’s overall contributions.\n\nI would be willing to change my rating to accept, provided no other reviewer raises substantial concerns. That said, I strongly recommend the authors revise the Limitations, Related Work, and Runtime Analysis sections, and, if possible, include better qualitative results and a video demonstration.", "rebuttal": "## [@MgqZ] - Clarity in Runtime and Pipeline Complexity\n\nThank you for highlighting the confusion regarding runtime complexities in reported times. The average runtime indicated in Table 1 specifically refers to the editing phase itself and does not include the feature field creation step. We apologize for any confusion this may have caused.\n\nOur methodology involves several preliminary steps prior to the editing process. First, a dual-encoder is trained, which takes approximately 30 seconds. Subsequently, training the 3D diffusion feature field requires an additional 4 minutes. Importantly, this 3D-consistent diffusion feature field needs to be generated only once for a given source scene and does not have to be recreated for each editing task. After these steps are completed, the actual text-driven 3D editing operation takes approximately 4 minutes per task.\n\nTo summarize, the complete breakdown of the computational steps involved is as follows:\n\n- Dual-encoder training: ~30 seconds (one-time)\n- 3D diffusion feature field training: ~4 minutes (one-time)\n- Text-based editing step: ~4 minutes (per editing instance)\n\nWe will clearly specify these details in the paper to avoid any future confusion.\n\n## [@MgqZ] - About Qualitative Improvements\n\nRegarding the observed limitations and subtle improvements compared to prior work, we appreciate your critical observations. To clearly illustrate the resolution of the Janus problem and semantic inconsistency issues with the source image, our main paper deliberately presented challenging cases. Most baseline methods failed to effectively address these challenging scenarios, whereas our proposed method successfully resolved them.\n\nAdditionally, to demonstrate the capability of our approach to address these challenges within a limited number of training iterations, we intentionally restricted the training iterations for editing 3DGS to approximately 1000 iterations during our experiments. Consequently, some results in paper might appear less promising. However, allocating additional training time resolves issues such as \"the panda remaining partly grey\" and \"Batman having double ears\". Furthermore, increasing the text guidance scale beyond our experimental settings resulted in significantly improved performance.\n\nAdditionally, we agree that including video materials illustrating multi-view consistency would substantially strengthen our evaluation. Therefore, we intend to incorporate comprehensive video examples in the supplementary material to clearly demonstrate these aspects and further validate the advantages of our approach.\n\n## [@MgqZ] - Missing Discussion of Method Limitations\n\nWe appreciate your comment regarding the explicit discussion of limitations. Initially, the limitations were provided in the supplementary material, but we agree that clearly listing them within the main text would improve readability and transparency. We will incorporate a dedicated limitations section into the main paper in our subsequent revision.\n\n## [@MgqZ] - Inconsistency Between Main Paper and Supplemental Results\n\nWe appreciate your careful observation regarding the visual quality differences between the main paper and supplementary results. For generating the supplementary results, we used the same dual-encoder and 3D feature field models as in the main paper. However, during the editing phase, we adjusted the number of iterations and the text guidance scale according to each scene's complexity. Typically, more complex scenes benefit significantly from a higher number of training iterations, resulting in enhanced editing performance. Additionally, increasing the text guidance scale leads to improved fidelity to the input text prompt.\n\nInterestingly, while increasing the guidance scale tends to exacerbate the Janus problem in baseline methods due to the property of stable diffusion model to generate predominantly front-view content, our method consistently demonstrates improved results without encountering the Janus issue.\n\nWe agree with your suggestion to improve the visual quality presented in the main paper, as this will strengthen our paper. Thank you for highlighting this valuable point.\n\n## [@MgqZ] - Lack of Related Works\n\nThank you for highlighting the brevity and limited depth of our related work section. We agree that expanding and deepening this section will better position our contribution within the existing literature. We will enhance the related work section by thoroughly discussing additional relevant methods, clearly articulating our method's differences, and better contextualizing its novelty in the revised paper.\n\n## [@MgqZ] - More Comparisons with Recent Baselines\n\nWe have conducted additional comparisons and observed that our method significantly outperforms these NeRF-based baselines. While these experiments were performed under the same conditions as described in our main paper, we note that due to time constraints, our evaluations were limited to three scenes (\"bear,\" \"face,\" and \"person\"). We chose these specific scenes since they were originally employed in the DGE[1] paper and are thus well-known.\n\n| **Method** | **CLIP** | **CLIP Dir.** | **Time** |\n| --- | --- | --- | --- |\n| IN2N | 21.77 | 9.08 | ~15min |\n| SIGNeRF | 21.42 | 11.72 | ~15min |\n| Ours (Full method) | **25.22**  | **17.75**  | **~4min** |\n\n[1] Chen, Minghao, Iro Laina, and Andrea Vedaldi. \"Dge: Direct gaussian 3d editing by consistent multi-view editing.\" *European Conference on Computer Vision*. Cham: Springer Nature Switzerland, 2024."}, {"review_id": "93E7oLMXuW", "reviewer": "Reviewer_LEXJ", "summary": "Diffusion Feature Field for Text-based 3D Editing with Gaussian Splatting presents a novel approach for multi-view consistent editing of 3DGS. It particularly focuses on the multi-head Janus problem when using image based editing and proposes to disentangle structure and appearance for more faithful editing. It takes a common approach for rendering 3DGS with diffusion features to make a diffusion feature field of the scene. Its primary contribution is a dual encoder scheme where structure and appearcne of the features is encoded separately. It applies self similarity maps of DINO derived from [32] to force the encoder to learn structural information which in turn forces the other encoder to encode apperance as the complete feature set is expected during the fused decoding process. These encodings are carefully injected at specific layers of the editing pipeline (InstructPix2Pix) to ensure structural consistency. The authors further propose to ensure consistency of the diffusion features at different time steps. The method is compared quantitaively against 3 baselines using CLIP similarity and direction metrics and an ablation is conducted.", "questions": "* Can the authors clarify when forcing DINO structural features to be similar to diffusion features, how these 2 features relate to each other and if the authors handle varying shapes of the feature maps from the 2 models?\n* How do the authors select the specific layers for injecting diffusion features into IP2P?\n* Please particularly address points 2 and 3 of the weaknesses", "limitations": "While limitations have been discussed from an instructpix2pix perspective, can the authors also discuss limitations of the dual encoding stratergy?", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths \n* The paper is quite well-written and well-explained. \n* The paper proposes a novel and simple solution for disentangling geometry and appearance in 3DGS using the dual encoding stratergy employing self similarity maps.\n* The resuls show the method addresses the problem of multi-head Janus well. The quantitaive results show reasonable gains compared to baselines. \n\n## Weaknesses\n* The related work is missing several papers-- Decomposing nerf for editing via feature field distillation, ProteusNeRF, GSEditPro, etc. \n* The need for separate DINO structural features. The method forces the diffusion structural features to be similar to DINO structural features (self similarity maps). Why can't the diffusion features be forced to learn structural information directly by utilizing self similarity maps? Moreover, this forcing of diffusion features to be more like DINO can lose out on the geometrical improvements which diffusion features bring over DINO and act as complementary [1]. And if DINO is needed, why not render DINO features directly rather than diffusion features?\n* Diffuion features can be extracted from images using a single timestep but this paper utilizes multiple time steps. While this can have some improvements, the paper also forces the diffusion features to be coherent across time steps which seems a bit orthogonal. Can an ablation be conducted to show the need for multi time step diffusion features which is computationally more expensive?\n* The variety of scenes shown is quite limited and I find the omission of videos in the supplementary a big limitation to clearly judge the quality of the edited 3DGS (such as floaters). \n* Adding GSEDitPro to the baseline would make the evauation stronger\n\n\n## Suggested improvements:\n* The figure captions do not properly explain the figures. For example, in Figure 1, it's not understandable at first glance what the feature maps are trying to convey. \n* A few sentences are not grammatically correct such as L125-127.\n\n[1] https://arxiv.org/pdf/2305.15347", "was_revised": true, "final_justification": "My concerns have been largely addressed and I think this paper has many merits that would be useful to the community but unofortunately I cannot wholeheartedly recommend acceptance (only weak accept) due to the omission of videos in the supplementary to clearly judge the quality of the edited 3DGS", "rebuttal": "## [@LEXJ] - Incomplete Coverage of Related Work\n\nThank you for pointing out the important omissions in our related work section. We acknowledge that including references such as \"Decomposing NeRF for Editing via Feature Field Distillation\", \"ProteusNeRF\", and \"GSEditPro\" will enhance the comprehensiveness of our discussion. We will incorporate and carefully discuss these additional works, clearly highlighting how our methodology relates to and differs from them, in the revised paper.\n\n## [@LEXJ] - Redundancy and Drawbacks of Aligning Diffusion Features with DINO\n\nThank you for raising insightful questions about the necessity of separate DINO structural features. As noted in the [1], \"DINO features can capture high-level semantic information and excel at obtaining sparse but accurate matches. In contrast, Stable Diffusion features focus on low-level spatial information and ensure the spatial coherence of the correspondence.\" Consequently, Stable Diffusion features predominantly capture low-level spatial layout details, whereas DINO features more effectively encode higher-level semantic content.\n\nGiven our primary goal of addressing the Janus problem in 3D editing, maintaining consistent semantic information from the source scene is critical. While Diffusion features indeed contain sufficient spatial structure information, their heavy reliance on spatial positioning within images makes them less effective for mitigating semantic inconsistencies such as the Janus issue. In practical tests, PCA visualizations of Diffusion features from different views of an object (e.g., the left and right sides of a bear) revealed that spatial positions within the image significantly influenced feature similarity. This spatial dependency makes Diffusion features inherently less suited for addressing the Janus problem.\n\nTo empirically verify the benefit of using DINO features for regularization, we conducted experiments comparing the training of structure features to match self-similarity maps derived from both Diffusion and DINO structural features. The outcomes demonstrated notably superior performance when regularizing with the self-similarity maps of DINO structural features.\n\nWe agree your point that using DINO features for regularization might reduce the benefits of the spatial information inherently present in Diffusion features. Additionally, while rendering DINO features directly could be a potential alternative, integrating them directly into the Diffusion generation process poses practical challenges. Exploring methods to effectively combine the complementary strengths of both Diffusion and DINO features represents an intriguing and valuable avenue for future research.\n\n[1] Zhang, Junyi, et al. \"A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence.\" *Advances in Neural Information Processing Systems* 36 (2023): 45533-45547.\n\n## [@LEXJ] - Justification for Multi-Timestep Usage\n\nThank you for your valuable observation regarding our use of multi-timestep diffusion features. We agree that diffusion features could indeed be extracted from images using a single timestep. However, directly substituting features from one timestep to another introduces a problem arising from differing noise levels at each timestep. Injecting features extracted at different noise levels can significantly impact the generative capability of the diffusion model. Therefore, instead of relying solely on single-timestep features, we propose an approach that injects features specifically matched to the appropriate noise level at each timestep.\n\nRecognizing the computational cost of generating separate 3D feature fields for multiple timesteps, we introduced the dual-encoder strategy to address this challenge. Our structure encoder is explicitly designed to extract noise-invariant structural information from the features, while our appearance encoder captures information that includes noise. Thus, the appearance encoder effectively maintains noise-level information across multiple denoising steps during editing. The structure encoder’s noise-invariant features are then rendered through our 3D feature field, allowing consistent structural information to be provided across various timesteps irrespective of their noise levels.\n\nWe provide an experimental comparison between training our dual encoder and 3D feature field using single-timestep features (extracted at the last denoising step) versus multi-timestep features, employing the same experimental setup and scenes as described in our main paper. Our results clearly indicate lower CLIP scores and higher MSE values when trained with single-timestep features. The higher MSE specifically suggests increased feature distortion, which we attribute to the appearance encoder’s inability to generalize across different noise levels due to being trained exclusively at a single timestep.\n\n| Method | CLIP | CLIP Dir. | MSE |\n| --- | --- | --- | --- |\n| single timestep | 25.08 | 17.98 | 1.7144 |\n| multi timesteps | 25.29  | 17.69  | 1.0931 |\n\n## [@LEXJ] - Evaluation Limitations\n\nWe agree that incorporating a broader range of scenes and providing video examples will greatly enhance the clarity and assessment of our results. We will address these points by adding more diverse scenes and supplementary videos in future. Additionally, we attempted to evaluate GSEDitPro as a baseline; however, we could not include it due to the unavailability of its official implementation. We intend to include GSEDitPro as a baseline in our evaluation once the official code is publicly released.\n\n## [@LEXJ] - Explanation on Aligning Differently Shaped Feature Maps\n\nTo address the varying feature shapes between DINO and diffusion features, we interpolate the ViT-based DINO features to match the dimensions of diffusion features. We will explicitly detail this interpolation procedure in the revised paper.\n\n## [@LEXJ] - Clarification on Injection Layer Selection\n\nOur selection of specific layers for injecting diffusion features into Instruct-Pix2Pix was guided by insights from the original Plug-and-Play (PnP) [1] paper. Specifically, the PnP study conducted PCA visualizations across various layers within the Stable Diffusion v1 UNet architecture, demonstrating that the fourth layer of the UNet up-block effectively captures spatial information. Following this methodology, we verified through similar PCA visualizations that the fourth layer of the fine-tuned Instruct-Pix2Pix model similarly contains meaningful spatial information. Consequently, we chose this particular diffusion feature layer for feature injection, ensuring effective preservation and transfer of spatial structure during editing. We will clarify this selection process explicitly in the paper.\n\n[1] Tumanyan, Narek, et al. \"Plug-and-play diffusion features for text-driven image-to-image translation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n## [@LEXJ] - Limitation of Dual Encoding\n\nIndeed, our method requires training an additional dual encoder and feature field specifically for each source scene prior to editing. This introduces extra computational overhead, with approximately 4 minutes required for feature field training. However, this step only needs to be conducted once per source scene and does not need repetition for subsequent editing tasks.\n\nAdditionally, the complexity and structural details of scenes vary significantly across different samples, necessitating careful tuning of hyperparameters for optimal performance. Therefore, hyperparameter adjustments might be required on a per-sample basis to ensure the structure encoder accurately captures the appropriate level of structural detail. We acknowledge these limitations clearly and will explicitly include them in our paper."}, {"review_id": "39hiLTIwLQ", "reviewer": "Reviewer_WiaQ", "summary": "This paper presents a method for 3D editing using 3D diffusion features. The authors mainly want to deal with the view consistent problem during editing, i.e., current methods use single-view diffusion model for editing and lack 3D constraint and may lead to Janus problem. The authors deal with this issue by training a 3D consistent feature field that disentangles the structure information (view independent) and appearance information (view dependent). During editing, the diffusion process is conditioned on the rendered structure features to ensure 3D consistency. The authors perform comparison and proved that their methods can outperform baselines.", "questions": "I think the paper is limited on structural editing, which is hard to address in the current framework. I would like to see authors how to address this during rebuttal and adjust my score accordingly.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- This paper is well motivated and technically sound. The authors started by analyzing the Janus issue of 3D editing methods, then proposed use 3D consistent features to deal with the issue, following a feature disentanglement training strategy. \n- The proposed method is easy to understand and follow. The authors provided relevant equations, figures and details for readers. With the clear motivation, the reviewers are able to understand the whole process of the proposed method.\n- The ablation study and detailed analysis of the proposed method is valuable and improves the understanding, including Fig. 4 the diffusion timesteps part, the correspondence matching metrics. This helps the readers understand the effectiveness of this method and its underlying reasons.\n- The proposed method, although is optimization based, is quite efficient and only takes 4 minutes and one GPU.\n\nWeaknesses:\n- One major and very important limitation of this method is that it cannot perform editing with structure changes, such as deletion, adding etc, since it relies on the 3D structure features during editing that are trained from input multi-view images. The authors also haven't show any examples on this. Therefore, this limits the functionality of 3D editing.\n- I would also like to see more results of video comparisons, which can help readers better spot the differences between different methods.", "was_revised": true, "final_justification": "I am still negative towards this paper due to lack of video results (which is crucial for 3D and video tasks) and missing experiments of structural editing. The overall idea is great, but lack of these prevents me from accepting.", "rebuttal": "## [@WiaQ] - Is 3D editing with structure changes possible?\n\nWe appreciate your valuable feedback regarding the limitations related to structural editing, such as object removal and incorporation. Our method builds upon the GaussianEditor [1] framework, which inherently supports these types of structural changes.\n\nFor object removal, GaussianEditor identifies the Gaussian representations for removal using masks and subsequently removes these Gaussians. However, simply removing these Gaussians creates empty regions, necessitating additional steps to fill these areas appropriately. GaussianEditor addresses this by employing a 2D inpainting technique guided by precise masks. To accurately determine these masks, GaussianEditor leverages a K-Nearest Neighbor (KNN) algorithm to identify Gaussians at the boundaries near the removed objects, thereby achieving precise 3D inpainting regions. These boundary Gaussians are projected across multiple views to obtain accurate multi-view 2D masks. Consequently, the method can effectively fill the removed regions with appropriate content guided by these masks.\n\nHowever, relying solely on single-view masking can result in inaccurate masks, especially in regions affected by occlusion from that particular viewpoint. To address this issue, we leverage our 3D feature field, which effectively captures rich semantic information, enabling clearer distinctions between foreground objects and background regions. PCA visualizations of these semantic-rich features demonstrate their effectiveness in accurately separating foreground and background areas. Utilizing these features would enhance the precision of identifying foreground Gaussians and accurately determining regions requiring inpainting across multiple views. We will include detailed evidence supporting this improvement in the supplementary materials. Nevertheless, we agree with your insight that directly employing structural features during editing may inherently constrain scene structure preservation and complicate object removal; thus, we avoid their direct use for this specific task.\n\nRegarding object adding, GaussianEditor inputs a prompt (P) and a mask (M) corresponding to the target region in an image (I) rendered from a specific viewpoint. Using a 2D inpainting diffusion model, it generates the desired object within the masked region according to the provided prompt. Then, the generated object's foreground is then extracted and transformed into a coarse 3D mesh through an Image-to-3D method, which is subsequently converted into a Gaussian representation and integrated into the scene. However, this direct Gaussian integration often results in unnatural boundaries between the original scene and the added object.\n\nWe can address this challenge by utilizing our 3D feature field. Specifically, after generating a scene incorporating the new object, we obtain a diffusion-based feature field for the scene. Since this field encodes semantic information about the added object, the derived structural features effectively guide the editing process, resulting in the addition of the new object without boundaries.\n\nIn summary, our method can indeed extend effectively to object removal and incorporation tasks, leveraging our 3D feature field to significantly enhance performance compared to existing methods.\n\n[1] GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting\n\n## [@WiaQ] - More Results of Video Comparisons\n\nRegarding your request for additional video comparison results, we agree that further video examples would greatly aid readers in visually distinguishing differences between methods. We will include more comprehensive video comparisons in the supplementary materials of our final manuscript to clearly illustrate the advantages and limitations of our approach relative to existing methods."}]

Author rebuttal:

Meta-review: Majority of the reviewers are leaning towards accept for this paper. Reviewers appreciated the well-written paper with interesting technique to disentangle the appearance and structure to mitigate multi-view inconsistency issues. On the other hand, multiple reviewers raised concerns related to missing comparisons, experiments and visual results. AC carefully looked at the reviews and the author response. Authors addressed several of the concerns in the rebuttal. It is felt that the strengths outweigh the weaknesses. The reviewers did raise some valuable concerns that should be addressed in the final camera-ready version of the paper, which include adding the relevant rebuttal discussions in the main paper. The authors are encouraged to make the necessary changes to the best of their ability, especially it is important to present video results with the work.
Decision: Accept (poster)