3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
Authors: Qihang Zhang
Yinghao Xu
Chaoyang Wang
Hsin-Ying Lee
Gordon Wetzstein
Bolei Zhou
Ceyuan Yang
Venue: ICLR 2025 Poster (2025)
Keywords: image editting
gaussian splatting
3D
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Scene image editing is crucial for entertainment, photography, and advertising design. Existing methods solely focus on either 2D individual object or 3D global scene editing. This results in a lack of a unified approach to effectively control and manipulate scenes at the 3D level with different levels of granularity. In this work, we propose 3DitScene, a novel and unified scene editing framework leveraging language-guided disentangled Gaussian Splatting that enables seamless editing from 2D to 3D, allowing precise control over scene composition and individual objects. We first incorporate 3D Gaussians that are refined through generative priors and optimization techniques. Language features from CLIP then introduce semantics into 3D geometry for object disentanglement. With the disentangled Gaussians, 3DitScene allows for manipulation at both the global and individual levels, revolutionizing creative expression and empowering control over scenes and objects. Experimental results demonstrate the effectiveness and versatility of 3DitScene in scene image editing.

Published as a conference paper at ICLR 2025
3D ITSCNNE:: EDITING ANY SCENE VIA LANGUAGE-
GUIDED DISENTANGLED GAUSSIAN SPLATTING
Qihang Zhang
Yinghao Xu Stanford
Chaoyang Wang
CUHK qhzhang@ linnk www.:eilnnkkc.kk....mmmmm edu.hk .hk
Snap Inc.
yhxu@stanford.edd
Hsin-Ying Lee
Gordon Wetzstein
Snap Inc. hiee5@snapchat.com
Stanford
gordon. wtw.:ewwwwttttteeeeettnnrrr.r... edu
Bolei Zhou UCLA boleiêcs.
Ceyuan Yang
ByteDance
wwciiisssucla.... ucla.edu
limbo 00 600mmi11..... com
"Move the girl, then delete the govl and Rottate the camera"
"Rotate the camera then delete the dag
"Rotate the dog
"Remoove the headsoar then WLOVE the cawera"
"Move the boy outside"
Figure 1:
Image pairs edited by 3DitScene. Our method is capable of simultancously handling various types of edits in both 2D and 3D spaces.
ABSTRACT
Scene image editing is crucial for entertainment, photography, and advertising design. Existing methods solely focus on either 2D individual object or 3D global scene editing. This results in a lack of unified approach to effectively control and manipulate scenes at the 3D level with different levels of granularity. In this work, we propose 3Di Scene, a novel and unified scene editing framework leveraging language-guided disentangled Gaussian Splatting that enables seamless editing from 2D to 3D, allowing precise control over scene composition and individual objects. We first incorporate 3D Gaussians that are refined through generative priors and optimization techniques. Language features from CLIP then introduce semanties into 3D geometry for object disentanglement. With the disentangled

---

Published as a conference paper at ICLR 2025
Gaussians, 3DitScene allows for manipulation at both the global and individual levels, revolutionizing creative expression and empowering control over scenes and objects. Experimental results demonstrate the effectiveness and versatility of 3Dit.ceene scene image editing. Code is available at https: / / thub. com/zghh255333it
INTRODUCTION
Editing scene images is of great importance in various fields, ranging from entertainment, professional photography and advertising design. Content editing allows to create immersive and captivating experiences for audiences, convey the artistic vision effectively and achieve the desired aesthetic outcomes. With the rapid development of deep generative modeling, many attempts have been made to edit image effectively. However, they have encountered limitations that hindered their potential.
Previous methods primarily concentrate scene editing in 2D image space. They commonly rely on generative priors, such as GANs and Diffusion Models (DM), and employ techniques like modification of cross-attention mechanisms (Hertz et al., 2022; 2023), and optimization of network parameters (Kim et al., 2022; Kawar et al., 2023; Ruiz et al., 2023; Gal et al., 2022; Chen et al., 2023b) edit the appearance and object identity within scene images. While some efforts have been made to extend these methods to 3D editing, they ignore 3D cues and pose challenge in maintaining 3D consistency, especially when changing the camera pose. Moreover, these approaches typically focus on global scenes and lack the ability to disentangle objects accurately, resulting in limited control over individual objects at the 3D level.
In order to edit any scene images and enable 3D control over both scene and its individual objects, we propose 3D Scene, a novel scene editing framework which leverage a new scene representation, language-guided disentangled Gaussian Splatting. Concretely, the given image is first projected into 3D Gaussians which are further refined and enriched through 2D generative prior (Rombach et al., 2022; Poole et al., 2022). We thus obtain a comprehensive 3D scene representation that naturally enables novel view synthesis for a given image. In addition, language features from CLIP distilled into the corresponding 3D Gaussians to introduce semantics into 3D geometry. These semantic 3D Gaussians help disentangle individual objects out of the entire scene representation, resulting in language-guided disentangled Gaussians for scene decomposition. They also allow for more friendri interaction i.e., users could query specific objects or interest via text. To this end, our 3Dit t Scene enables seamless editing from 2D 3D and allow for modifications both the global and individual levels, empowering creators to have precise control over scene composition, and object-level edits.
We dub our pipeline as 3DitScene. Different from previous works that focus on addressing a single type of editing, 3Dit integrates editing requirements within unified framework. Our teaser figure demonstrates the versatility of 3 t.Scene by showcasing its application to diverse scene images. We have conducted evaluations of 3D SDDittt t.Scene under various settings, and the results demonstrate significant improvements over baseline methods.
2 RELATED WORK
Image Editing with Generative Models. The field of 2D image synthesis has advanced significantly with the development of generative models such as GANs (Karras et al., 2021; 2019) and diffusion models (Rombach et al., 2022; Song et al., 2020; Ho et al., 2020). Many studies capitalize on the rich prior knowledge embedded in generative models for image editing. Some endeavors utilize GANs for various image editing tasks, including image-to-image translation, latent manipulation (Shen et al., 2020; Yang et al., 2021; Zhu et al., 2020; Xu et al., 2021; Jahanian et al., 2019), and text-guided manipulation (Patashnik et al., 2021). However, due to limitations in training on large-scale data, GANs often struggle to perform well on real-world scene images. As diffusion models make notable progress, the community is increasingly focusing harnessing the potent text-to-image diffusion model for real image editing (Kim et al., 2022; Kawar et al., 2023; Ruiz et al., 2023; Gal et al., 2022; Chen et al., 2023b; Hertz al., 2022; 2023; Meng et al., 2021b;
2

---

Published as a conference paper at ICLR 2025
Input View
RGB branch
raference view
Leeen
Initialize
Novel View
Pender
Stable Diftusion
Lans
Stable Diffusion
Semantic branch
Depth
Expand
Render
CLIP
Laieill
DM
Language-guiden disentangled 3DGS
3DitScene training pipeline. Given input view, first initialize 3DGS by lifting pixels to 3D space and then expand it over novel views by RGB and depth inpainting. Semantic features then distilled into 3D Gaussians achieve object-level disentanglement.
Figure 2:
1. Query langusge
Disentangled representation
Diverse editing
corgi"
CLIP
Figure 3: 3DitScene Inference pipeline. User can query object of interest via language prompt. Enabled by the disentangled 3D representation, user can change camera viewpoint, and manipulate the object of interest in a flexible manner.
Su et al., 2022). However, these methods are confined to the 2D domain and are limited in editing objects within 3D space. Concurrently, other research efforts (Yenphraphai al., 2024a) attempt to address 3D-aware image editing, but they introduces inconsistency in the editing process, and cannot change the camera perspective of the entire scene. Wang al. (2024a); Chen al. (2024a); Ye al. (2023); Palandra et al. (2024); Wu et al. (2024a); Wang et al. (2024b); Jaganathan et al. (2024) focus on editing with a given 3DGS scene, but is limited in the types edits they support. In contrast, our model leverages explicit 3D Gaussian to convert 2D images into 3D space while disentangling objects with language guidance. This approach enables our model not only to consistently perform 3D-aware object editing but also facilitates scene-level novel-view synthesis.
Single-view 3D Scene Synthesis. Among 3D scenes generation (Zhang et al., 2023b; Höllein al., 2023; Chung et al., 2023; Chen et al., 2023a;e; Mao al., 2023; Epstein al., 2024), conditional generation on a single-view presents an unique challenge. Previous approaches address this challenge by training versatile model capable of inferring a 3D representation of scene based on a single input image (Wiles et al., 2020; Tucker & Snavely, 2020; Hu et al., 2021; Han al., 2022; Flynn al., 2019; Li al., 2021; Hong et al., 2023; Yu et al., 2021). However, these methods demand extensive datasets for training and tend to produce blurry textures when confronted with significant changes camera viewpoints. Recently, several works have embraced diffusion priors (Liu et al., 2023; Chan et al., 2023; Xu et al., 2023; Gu et al., 2023; Tang et al., 2023; Qian al., 2023; Chen al., 2024b) to acquire probabilistic distribution for unseen views, leading to better synthesis results. Nevertheless, these methods often concentrate on object-centric scenes or lack 3D consistency. Our approach connect 2D images and 3D scenes with explicit 3D Gaussians and incorporate diffusion knowledge, which overcome the mentioned challenges.
METHOD
Our target is to propose 3D-aware scene image editing framework (Fig. 2) that allows simultaneous control over the camera and objects. To accomplish this, Sec. 3.1 introduces novel scene representation called language-guidn -uidedd disentangled Gaussian splatting. In order to achieve object- level control, Sec. 3.2 further distills language features into the Gaussian splatting representation,
3

---

Published as a conference paper at ICLR 2025
achieving disentanglement at the object level. We elaborate the optimization process in Sec. 3.3 and demonstrate the flexible user control enabled by our framework during inference in Sec. 3.4.
3.1 3D GAUSSIAN SPLATTING FROM SINGLE IMAGE
Preliminary. 3D Gaussian Splatting (3DGS) (Kerbl et al., 2023) has been proved effective in both reconstructive (Luiten et al., 2023; Yang et al., 2023) and generative setting (Zou et al., 2023; Tang et alll, 2023). It represents 3D scene via set of explicit 3D Gaussians. Each 3D Gaussian describes its location by center vector x E R scaling factor s E R a rotation quaternion q E R and also stores opacity value a E RR and spherical harmonics (SH) coefficients the degrees of freedom of SH) for volumetric rendering. All the above parameters can be denoted as O X; St, 9i Oie, cali [0....... ,N I]}, where N is the number of 3D Gaussians. A tile-based rasterizer is used to render these Gaussians into 2D image.
R$ (k represents
Image-to-3DGS initialization. Given an input image I R3xHxW an off-the-shelf depth prediction model is applied to estimate its depth map D E R#xx, Then, we could transform image pixels into 3D space, forming the corresponding 3D point clouds:
P= o1 D,,
where K and T are camera intrinsic and extrinsic matrices respectively. Such point clouds P are then used to initialize the 3DGS by directly copying the location and color values, with other GS- related parameters randomly initialized. To refine the 3DGS's s appearance, we adopt reconstruction loss:
Lrccon = -II- f(P,K, T)
where f is the rendering function.
We further enhance the rendered quality by leveraging prior knowledge from image generative foundation model, namely Stable Diffusion (Rombach al., 2022). It provides update direction to the images rendered by the current 3DGS in the form of Score Distillation Sampling (Poole et al., 2022) loss, denoted Lsns-
3DGS expansion by inpainting. When camera perspectives changes, rendered views will contain holes due to occlusion new region outside the original view frustum. We use Stable Diffusion to inpaint the uncovered regions. Then, the newly added pixels need to be accurately transformed into 3D space align seamlessly with the existing 3D Gaussians.
Previous methods (Chung et al., 2023; Höllein et al., 2023; Yu et al., 2023) first predict the depth values, and then use heuristic methods to adjust the values to align with the existing 3D structure. However, relying on heuristic methods often overlooked various scenarios, leading to artifacts such as depth discontinuities or shape deformations.
Instead, we propose a novel method to lifted novel contents to 3D while ensuring seamless alignment without any heuristic procedures. The key insight treat the problem as an image inpainting task, and utilize state-of -the-artt diffusion-based depth estimation models (Ke et al., 2024; Fu et al., 2024; Yang et al., 2024) as a prior to solve the task. During denoising steps, rather than using models to predict the noise over the entire image, employ the forward diffusion process to determine the value fixed areas (Meng et al., 2021a). This approach guarantees the final result, after denoising. adheres to the depth of original fixed parts, ensuring smooth expansion.
After smooth 3DGS expansion via depth impainting, we take the imagined novel views as reference views, and apply reconstruction loss Lissn to supervise the updated 3DGS. SDS loss LSDS is adopted for views rendered from camera perspectives that are interpolated between the user-provided viewpoint and the newly imagined views.
3.2 LANGUAGE-GUIDED DISENTANGLED GAUSSIAN SPLATTING
Based on the 3DGS built from single input image, users can generate novel views. In this section, further distill CLIP (Radford et al., 2021) language feature to 3D Gaussians. This introduce semantics into 3D geometry, which helps disentangle individual objects out of the entire scene representation.
4

---

Published as a conference paper at ICLR 2025
Language feature distillation. We augment each 3D Gaussian with language embedding E RC; where denotes the number of the channels. Similar to RGB image I, a 2D semantic feature map first use Segment
E IR CC can also be rendered by the rasterizer. To learn the embedding, Anything Model (SAM) (Kirillov et al., 2023; Zhang et al., 2023a) to get semantic masks M, Then, can obtain embedding of each object I M, and supervise the corresponding region rendered feature map E, according to the distillation loss:
Edistill
-EI 9 g(00 @ M
(3)
where is the CLIP's image encoder, and @ denotes element-wise multiplication. Following LangSplat (Qin et al., 2024), additionally train autoencoder to compress the embedding space to optimize the memory consumption of language embedding e.
Scene decomposition. After distillation, enables user to query and ground specific object, and perform editing over single object (e.g. translation, rotation, removal, re-stylizing).
decompose the scene into different objects. This
is worth noting that such scene decomposition property not only enables more flexible edits during inference stage, but also provides augmentation over scene layouts during the optimization process. Since now we can query and render each object independently, we apply random translation, rotation, and removal over objects. This augmentation the scene layout leads significant improvement in the appearance of occluded regions, ultimately enhancing the overall quality of the edited views (see Sec. 4.4).
3.3 TRAINING
The overall training objective be expressed
L Areonerecon + ASDS + Adisun
(4)
where Arecoms ASDS and Aissil are coefficients that balance each loss term.
3.4 INFERENCE
Due to the disentangled nature of our representation, users can now interact with and manipulate objects a flexible manner. Here, we mainly discuss prompting objects via different modalities:
Text prompt. Users can query an object through text prompts as shown in Fig. 3. Following LERF (Kerr et al., 2023) and LangSplat (Qin al., 2024), we calculate the relevancy score score between the language embedding e in the 3D Gaussians and the embedding of the text prompt as:
exp(e ee)
score = min
(5)
exp(e e,) exple (
where c'amon is the CLIP embeddings of canonical phrases including "object", "things", "stuff" and "lexture". Gaussians that have relevance scores below a predefined threshold are excluded. The remaining part is identified the object of user interest.
Bounding box. Users can also select an object by drawing an approximate bounding box around on the input image. We group 3D Gaussians within the bounding box by K-Means clustering, and discard clusters whose number of Gaussians does not exceed a threshold proportion.
In the meantime, user can also adjust the camera by specifying intrinsic and extrinsic parameters.
EXPERIMENTS
4.1 SETTINGS
Implementation details. To lift image to 3D, use GeoWizard (Fu al., 2024) estimate its relative depth. Stable Diffusion (Rombach et al., 2022)'s inpainting pipeline is adopted to generate new content for 3DGS's expansion. We leverage MobileSAM (Zhang al., 2023a) and
5

---

Published as a conference paper at ICLR 2025
User study result. We report the percentage of favorite users for the consisteney and quality of images edited by each method
Table 1:
AnyDoor
Ojbect 3DIT
Image Scuplting
Ours
Human
5.1
16.8
12.7
65.4
Consistency
GPT4-v
0.0
6.7
31.3
62.0
Human
10.4
0.5
25.1
64.0
Quality
GPT4-v
13.3
40.8
view 1
view 2
view 3
disentangled object
Figure 4: Visualization of rendered images and feature maps. For each sample, show three
views of rendered images and feature maps. To demonstrate the disentangled scene representation, use the language embedding select foreground object and render it exclusively.
OpenCLIP (Ilharco et al., 2021) to segment and compute rendered views' feature maps, which are further leveraged to supervise the language embedding of 3D Gaussians. We use Stable Diffusion to perform Score Distillation Sampling (Poole et al., 2022) during optimization. Given the already decent image quality the start of optimization benefited from explicit 3DGS initialization, adopt a low classifier-free guidance (Ho & Salimans, 2022) scale.
Baselines. We compare our method with following scene image editing works: (1) AnyDoor (Chen et al., 2023b) is 2D diffusion- based model that can teleport target objects into given scene images. leverages Stable Diffusion's powerful image generative prior by finetuning upon it. (2) Object 3DIT (Michel et al., 2024) is designed for 3D-aware object-centric image editing via language instructions. finetunes Stable Diffusion over a synthetic dataset containing pairs of original image, language instruction, and edited image.
(3) Image Sculpting (Yenphraphai et al., 2024b) is also designed for 3D-aware object-centric image editing. estimates a 3D model from an object in the input image enable precise 3D control over the geometry. It also uses Stable Diffusion refine the edited image quality. (4) AdaMPI (Han et al., 2022) focuses on the control over camera perspective. leverages monocular depth estimation and color inpainting with learned adaptive layered depth representations. (5) LucidDreamer (Chung et al., 2023) tackles novel view synthesis by querying Stable Diffusion's inpainting pipeline with dense camera trajectories.
4.2 QUANTITATIVE RESULTS
We conduct study to compare the edited results by method with the established baselines. We generate 20 samples for each method and request users to vote for their preferred method based consistency with the original image and quality for each sample. We collect feedback from

---

Published as a conference paper at ICLR 2025
Input
AnyDoor
Object 3DIT
Image Sculpting
Ours
Figure 5: Comparison results of object-centric manipulation. We apply translation, resizing, and removal over foreground objects.
Input
AdaMPI
LucidDreamer
Ours
Figure 6: Comparison results of camera control. We show two views with different camera perspectives for each method.
25 users, and report the result in Tab. 1. Our method consistently outperforms previous basclines in terms of both consistency and image quality. As recommended in previous study (Wu et al., 2024b), GPT4-v has the ability to evaluate 3D consistency and image quality. Therefore, we include GPT-4v additional criterion. The preference of GPT4-v is well aligned with human preference, which once again demonstrates the superiority of 3Di Scene.
4.3 QUALITATIVE RESULTS
Fig. 4 showcases the generated novel views with their respective feature maps produced by our framework. The feature maps demonstrate remarkable accuracy in capturing the semantic content of the images. This ability to distinctly separate semantic information plays crucial role in achieving precise object-level control In the following. we demonstrate flexible editing over scene images enabled by framework, and also compare with baseline methods.

---

Published as a conference paper at ICLR 2025
Imput
MOO a00
we aaee
Figure 7: Ablation results for layout augmentation during optimization. To evaluate the degree of object-level disentanglement, we conduct object removal for each sample. The top row displays the input image, while the
showcase the edited scene
Input
w/o recon
w/o Leecon (CFG=50)
w/o LSDS
Ours
Figure 8: Ablation results for loss terms. We show rendered novel views under different loss settings. The left column lists the input image. In right columns, two views are shown for each configuration. The quality degrades when reconstruction SDS loss term is discarded
Object manipulation. Since different methods define object manipulation, particularly translation operations, in different coordinate systems', it becomes challenging to evaluate them under a unified and fair setting. Therefore, we evaluate each method under its own specific setting achieve the best possible result. As shown in Fig. 5, AnyDoor struggles to maintain object identity and 3D consistency when manipulating object layouts, primarily due to the absence of 3D cues. Object 3DIT, trained on synthetic datasets, exhibits limited generalization ability to real images. By leveraging a 3D model derived from the input image, Image Sculpting achieves better results. Nonetheless, it encounters issues with inconsistency when manipulating objects.
In contrast, our method delivers satisfactory 3D-aware object-level editing results. maintains accurate 3D consistency of edited objects after rearranging their layout. Additionally, it preserves occlusion relationships within the scene, such moving the girl be partially occluded by foreground object in the last row example.
Camera control. We compare our methods with AdaMPI and LucidDreamer for camera control. As illustrated in Fig. 6, AdaMPI only focuses on scenarios where the camera zooms in, and does not consider novel view synthesis. Therefore, this approach is not suitable for 3D-aware image editing when large camera control is required. LucidDreamer also leverages Stable Diffusion's 's inpainting capacity for novel view synthesis. However, it suffers from sudden transitions in the content within the frame (see sample in the bottom line). It also requires dense camera poses. In contrast, our method only needs as few as three camera poses and enables smooth transitions from the input view to novel views, enhancing control over the camera perspective.
Door, Object 3DI and Image Sculpting respectively employs 2D masks, language prompts, and image coordinates for control. We coordinates in 3D space instead.
8

---

Published as a conference paper at ICLR 2025
RGB with partial depth
Heuristic alignment
Depth inpainting (Ours)
Figure 9:
Ablation results for depth inpainting. The first row shows images with their corresponding depth maps (available on the left half). The second and third row display the depth map predicted by heuristic alignment, and our depth inpainting method respectively.
4.4 ABLATION STUDY
Layout augmentation during optimization. As representation disentangles object level, we could perform layout augmentation during optimization. Here, we investigate whether disentanglement property benefits the optimization process. We use the task of removing objects to evaluate the degree of disentanglement.
As illustrated in Fig. 7, when layout augmentation is disabled during optimization, floating artifacts can be observed. We discover that these Gaussians lie inside the object. They are occluded by Gaussians at the surface. As they do not contribute to the rendering result, they are consequently not updated by gradient descent during optimization, leaving their language embeddings unsupervised.
contrast, when applying layout augmentation during optimization, such Gaussians will be exposed when the foreground object is moved away, and hence updated. With this ablation, it is concluded that the disentanglement property the proposed representation not only enables more flexible inference. but also contributes to the optimization process.
Loss terms. During optimization, we adopt three loss terms: Lreenr LsDs and Ldistll- Cassill plays a critical role in distilling language embedding into 3D. The remaining terms focus enhancing the visual quality images. As illustrated in Fig. 8, the image quality degrades severely without freamm Lsns- Without Lrecon: the image is only refined by the SDS loss, which creates discrepancies with the original image. When the CFG value is set low, as default, the image appears lacking in details and exhibits unusual texture patterns. Increasing the CFG value introduces more details, yet leads to inconsistencies with the original image, while the issue of strange texture patterns persists. Additionally, only applying frecom results floating artifacts and blurriness across the entire image. In conclusion, both SDS and reconstruction loss are crucial for achieving decent image quality.
Depth inpainting. When expanding 3DGS at novel views, we need to estimate the depth map of unseen regions. Here, we compare our inpainting- based depth estimation with heuristic-basee method. Fig. show images with depth map available in the left part. The task to predict the depth map of the right part. Method relying on heuristic alignment results to artifacts like depth discontinuity. In contrast, our proposed method is capable producing accurate depth maps that align well with the left known part.
CONCLUSION AND DISCUSSION
We present a novel framework, 3Dit Scene for scene image editing. Our primary objective is facilitate 3D-aware editing of both objects and the entire scene within unified framework.

---

Published as a conference paper at ICLR 2025
We achieve this by leveraging representation. This representation is learnt by distilling CLIP's language feature into 3D Gaussians. The semantic 3D Gaussians effectively disentangle individual objects out of the entire scene, thereby enabling localized object editing. We test 3Di SScene under different settings and prove its superiority compared to previous methods.
representation, language-guided disentangled scene
REFERENCES
Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. ICCV, 2023.
Dave Zhenyu Chen, Haoxuan Li, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießnnrr Scenetex: High-quality texture synthesis for indoor scenes via diffusion priors. arXiv preprint arXiv:2311.172611 2023a.
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero- shot object-level image customization. arXiv preprint arXiv:2307.0942 2023b.
Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 21476-22 -14455, 2024a.
Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Scenedreamer: Unbounded 3d scene generation from 2d image collections. arXiv preprint arXiv:2302.01330,
2023c. 2023c.
Zilong Chen, Feng Wang, Yikai Wang, and Huaping Liu. Text-to-3d using gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 21401-21412, 2024b.
Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free generation of 3d gaussian splatting scenes. arXiv preprint arXiv:2311.1338. 2023.
Dave Epstein, Ben Poole. Ben Mildenhall, Alexei A Efros, and Aleksander Holynski. Disentangled 3d scene generation with layout learning. arXiv preprint arXiv:2402.16936, 2024.
John Flynn, Michael Broxton, Paul Debevec, Matthew Vall, Graham Fyffc, Ryan Overbeck, Noah Snavely, and Richard Tucker. Deepview: View synthesis with learned gradient descent. CVPR, 2019.
Xiao Fu, Wei Yin. Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from single image. arXiv preprint arXiv:2403. 12013, 2024.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth word: Personalizing text-to-image generation using textual inversion, arXiv preprint arxiv:2208..166
2022.
Jiatao Gu, Alex Trevithick, Kai-En Lin, Joshua M Susskind, Christian Theobalt, Lingjic Liu, and Ravi Ramamoorthi. Nerfdiff: Single-image view synthesis with nerf-guided distillation from 3d-aware diffusion. In ICML, 2023.
Yuxuan Han, Ruicheng Wang, and Jiaolong Yang. Single-view view synthesis in the wild with learned adaptive multiplane images. In ACM SIGGRAPH Conference Proceedings, 2022.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfrr Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-promy image editing with cross attention control, arXiv preprint arXiv:2208. 220.:21666.0666 2022.
Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. arXiv preprint arXiv:2312.02/33,, 2023.
10

---

Published as a conference paper at ICLR 2025
Jonathan Ho and Tim Salimans. arXiv:2207.12598, 2022.
Classifier-free diffusion
guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from text-to-image models. arXiv preprint SiV::2303.11889 2023.
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:231 1.04400, 2023.
Ronghang Hu, Nikhila Ravi, Alexander C Berg, and Deepak Pathak. Worldsheet: Wrapping the world in a 3d sheet for view synthesis from a single image. In ICCV, 2021.
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip. July 2021. URL https:/ /ttpi.oorg///////.//////ooooooo. 5281/ zeroodo.. 5143774473 If you this software, please cite it below.
Vishnu Jaganathan, Hannah Hanyun Huang, Muhammad Zubair Irshad, Varun Jampani, Amit Raj, and Zsolt Kira. Ice-g: Image conditional editing of 3d gaussian splats. arXiv preprint arXiv:2406. 08488, 2024.
Ali Jahanian, Lucy Chai, and Phillip Isola. On the" steerability" of generative adversarial networks. arXiv preprint 1907077007 2019.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. NewrIPS, 2021.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Trani. Imagic: Text-based real image editing with diffusion models. In CVPR, 2023.
Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp- 9492-9502, 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions Graphics, 42(4), 2023.
Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In CVPR, pp. 19729-19739, 2023.
Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. In CVPR, 2022.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan- Yen Lo, et al. Segment anything. arXiv preprint arXiv:2304.0266 2023.
Jiaxin Li, Zijian Feng. Qi She, Henghui Ding, Changhu Wang, and Gim Hee Lee. Mine: Towards continuous depth mpi with nerf for novel view synthesis. ICCV, 2021.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3; Zero-shot one image 3d object. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 0298-9309, 2023.
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint 2Xiv:2308.097/3 2023.
11

---

Published as a conference paper at ICLR 2025
Weijia Mao, Yan-Pei Cao, Wei Liu, Zhongcong Xu, and Mike Zheng Shou. Showroom3d: Text to high-quality 3d room generation using 3d priors. arXiv preprint arXiv:2312.13324. 2023.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2/08.01073, 2021a.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Jun Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.00 01007, 2021b.
Oscar Michel, Anand Bhattad, Eli VanderBilt, Ranjay Krishna, Aniruddha Kembhavi, and Tanmay Gupta. Object 3dit: Language-guided 3d-aware image editing. Advances in Neural Information Processing Systems, 36, 2024.
Francesco Palandra, Andrea Sanchietti, Daniele Baieri, and Emanuele Rodolà. Gsedit: Efficient text-guided editing of 3d objects via gaussian splatting. arXiv preprint arXiv:2403.05154 2024.
Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen- -r, and Dani Lischinski. Styleclip: Text- driven manipulation of stylegan imagery. In CVPR, 2021.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion, arXiv preprint arXiv:2209. 148888 2022.
Guocheng Qian, Jinjic Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin- Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, et al. Magic123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors. arXiv preprint arXiv:2306. 17843, 2023.
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. In CVPR, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pp- 8748 8763. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning -ame-e diffusion models for subject-driven generation. CVPR, 2023.
Yujun Shen, Ceyuan Yang. Xiaoou Tang, and Bolei Zhou. Interfacegan: Interpreting the disentangled face representation learned by gans. IEEE TPAMI, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint 2rXiv:2011.13456. 2020.
Xuan Su, Jiaming Song, Chenlin Meng, and Stefano Ermon. Dual diffusion implicit bridges for image-to-image translation. arXiv preprint arXiv:2203.08388, 2022.
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309.1666 2023.
Richard Tucker and Noah Snavely. Single-view view synthesis with multiplane images. In CVPR, 2020.
Junjie Wang, Jiemin Fang, Xiaopeng Zhang, Lingxi Xie, and Qi Tian. Gaussianeditor: Editing 3d gaussians delicately with text instructions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20902-20911 2024a.
12

---

Published as a conference paper at ICLR 2025
Yuxuan Wang, Xuanyu Yi, Zike Wu, Na Zhao, Long Chen, and Hanwang Zhang. View-consistent 3d editing with gaussian splatting. arXiv preprint arXi:24433.1188, 20, 2024b.
Olivia Wiles, Georgia Gkioxari, Richard Szeliski, and Justin Johnson. Synsin: End-to-end view
synthesis from a single image. In CVPR, 2020.
Jing Wu, Jia- Ha-Wang Bian, Xinghui Li, Guangrun Wang, Ian Reid, Philip Torr, and Victor Adrian Prisacariu. Gaussctrl: multi-view consistent text-driven 3d gaussian splatting editing. arXiv preprint arXiv:2403.0m 08733, 2024a.
Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gordon Wetzstein. Gpt-4v (ision) is human-aligned evaluator for text-to-3d generation. arXiv preprint arXiv:2401.04094 2024b.
Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, and Bolei Zhou. Generative hicrarchical features from synthesizing images. In CVPR, 2021.
Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model. arXiv preprint arXiv:311.00221 2023.
Ceyuan Yang, Yujun Shen, and Bolei Zhou. Semantic hierarchy emerges in deep generative representations for scene synthesis. ICV,, 2021.
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. arXiv preprint arXiv:2401108891, 2024.
Ziyi Yang. Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv preprint arXiv:2309. 13101, 2023.
Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. arXiv preprint arXiv:2312.007333 2023.
Jiraphon Yenphraphai. Xichen Pan, Sainan Liu, Daniele Panozzo, and Saining Xie. Image sculpting: Precise object editing with 3d geometry control. arXiv preprint arXiv:2401.01702, 2024a.
Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, and Saining Xie. Image sculpting: Precise object editing with 3d geometry control. arXiv preprint arXiv:2401.01702, 2024b.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one few images. In CVPR, 2021.
Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from anywhere to everywhere. arXiv preprint arXiv:2312.038844 2023.
Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim, Sung-Ho Bac, Seungkyu Lec, and Choong Seon Hong. Faster segment anything: Towards lightweight sam for mobile applications. arXiv preprint arXiv:2306. 14289, 2023a.
Qihang Zhang, Chaoyang Wang, Aliaksandr Siarohin, Peiye Zhuang, Yinghao Xu, Ceyuan Yang, Dahua Lin, Bolei Zhou, Sergey Tulyakov, and Hsin-Ying Lee. Scenewiz3d: Towards text-guided 3d scene composition. arXiv preprint arXiv:2312.0888 2023b.
Jiapeng Zhu, Yujun Shen, Deli Zhao, and Bolei Zhou. In-domain gan inversion for real image editing. In ECCV, 2020.
Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song- Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. arXiv preprint 27X1v.211..09147, 2023.
13

---

Published as a conference paper at ICLR 2025
APPENDIX
A IMPLEMENTATION DETAILS
Scene initialization. We first utilize GeoWizard (Fu et al., 2024) estimate the relative depth of the input image. Next, we lift the image into 3D space based on this depth and perform 300 SDS steps. The camera azimuth angle is randomly sampled from [-15%, 15 After 300 steps, we employ Stable Diffusion (Rombach et al., 2022)'s inpainting pipeline to generate new content for the expansion 3DGS. Specifically, we inpaint the rendered views at azimuth angles -150 and
15". Finally, regions and lift them into 3D space.
Geo (Fu et al., 2024) again to estimate the depth of the newly added
SDS optimization. We perform 1500 SDS steps optimize the whole scene. We randomly sample the diffusion time step from r, where / = 0.02, and r starts at 0.5 and gradually decreases 0.2 by the 1000th step. We use guidance strength of 5 for classifier-free guidance.
Coefficients.
In Eq. (4), we choose Apecon 1000, ASDS 0.01, and Adisüll 1.
B EDITING PIPELINE AND USER INTERFACE
To enhance the user experience, we explore two types of editing frameworks:
1. Large language model (LLM) based. We prompt the LLM to take the user's editing request as input and parse it into predefined components, including camera movement angles, descriptions of the object of interest, and the transformation matrix for that object. Our editing algorithm then uses these components as input to optimize the 3D scene and сarry out the necessary edits.
However, relying solely on the LLM the interface for our editing algorithm has its drawbacks; it makes it difficult for users to interactively manipulate and edit the optimized 3D scene. To address this, have also developed user interface that facilitates these interactions.
2. User interface (UT) based. We developed an user interface for editing. We have implemented web-based interactive panel that allows users control the editing process. Fig. Al provides an overview the user interface. To use it, users need to upload an image, specify the object of interest, and provide text prompts, illustrated in Fig. A2. They can then visualize the optimization process real time. After the optimization complete, draggable sliders for various edits, including moving, removing, and rotating objects, as shown in Fig. A3, Fig. A4, and Fig. A5.
Both types of editing workflows do not require users to interact with the code during the entire process.
14

---

Published as a conference paper at ICLR 2025
Uppood imoge
select the drow tod stort
aloted
offert
Recition cver
mmyode
inference single froms
infernnce Video
mmpide
Seed
vathing
1000
Doangles
eeferenc imagee
Desombe the Image
Procege the and right creas
Pronge the aves botbel the faregrenn enjert
Seeei
Purnber tranting sees
Keliy Sevare
Sqoors
Timo Stuare
made
glon skanding thesddele Toverbe
therrdom Touembc National
iHorabl in Youamis Natorr Pork
1000
Figure A1: Overview of the user interface. To provide better experience, have developed a web-based interface. Users simply need to upload the input image and enter a text prompt. They can then visualize the optimization process in real time and edit the scene using sliders.
Upload image, and epecify object interest
Optional text prompt
Visualize the optimization
Figure A2: Procedure to optimize a scene via the user interface. To optimize 3D scene for 3D-aware image editing, users need to upload an image, specify the object of interest, and provide text prompts. They can then visualize the optimization process in real time.
15

---

Published as a conference paper at ICLR 2025
Figure A3: Interactive editing via the user interface. In this sample, we demonstrate that by sliding the bar, users can adjust the object's offset along the Y coordinate.
Figure A4: Interactive editing via the interface. In this sample, demonstrate that by sliding the bar, users can adjust the object's 's offset along the X coordinate.
Figure A5: Interactive editing via the user interface. In this sample, we demonstrate that by sliding the bar, users can adjust the object's rotation.
16


Reviews:
[{"review_id": "LQzCN6xGus", "reviewer": "Reviewer_txJu", "summary": "The authors present a new image editing pipeline that lift 2D data priors, including CLIP, mono-depth, SAM and Stable Diffusion, into a 3D Gaussian Splatting representation for manipulation. This decomposes the scene into a camera and queryable objects that can be rotated and translated in 3D space. The process introduces discontinuities at novel views, which the authors propose to fill in following an image inpainting procedure and using the SDS loss from Dreamfusion. The resulting scene can be rendered to produce an edited image.", "questions": "* What is the runtime of the proposed method and other alternatives? While the 3D representation seems to improve quality, it seems unlikely that this multi-stage method would be fast enough for real use.\n* The teaser figure seems to suggest that this process is \"hands-off,\" taking in a text prompt about manipulating the scene and then generating, but I'm having trouble telling if this is actually true. Could the authors please clarify this?\n* To what extent does the method begin to fail if there are large baseline camera movements? SDS at low CFG has very limited generation ability, for instance.\n* Are there other downstream tasks where this pipeline could be useful? Maybe other instances of image editing?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Strengths: \n* The authors leverage a broad variety of methods to extend image features to 3D, all of which are well-justified inclusions.\n* Authors ablate the main technical decisions in the paper, specifically the different loss terms, CFG scale and object removal.\n* Uses GPT4o LLM-as-critic evaluations from past works to automatically evaluate 3D generation quality.\n* Writing is focused and practical, highlighting different elements of the design space and clearly explaining each decision the authors made.\n* Method is training-free, so it should improve over time as base models improve.\n* Figures are well-presented and understandable.\n* The importance of the proposed task is clear and the method seems to be a reasonable improvement over existing works.", "weaknesses": "Weaknesses:\n* The main weakness of this paper is that it mostly combines existing techniques in a pipeline to improve an application rather than introducing new technical insights. For instance:\n  * LucidDreamer used monocular depth to unproject an image into a Gaussian Splatting scene\n  * LERF introduced language-embedded 3D representations\n  * Dreamfusion introduced the SDS objective\n  * Inpainting techniques are already broadly explored in the image diffusion model literature\n* Evaluations are very limited, with the user study not being well-explained and including a small number of participants (25) and generated samples (20). This is a known challenge in this problem space, but it would have been nice to see more discussion or contribution in this direction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ci9kqhvjBF", "reviewer": "Reviewer_d3t4", "summary": "This paper provides a method for highly precise 3d scene editing.\nThe author starts from a single image input, first using estimated depth to init the 3dgs. Use inpainting for both novel view and its depth map. Authors also introduce a language feature distillation loss, so that we will be able to use text to query the gs during inference", "questions": "How long does it take for training one scene?\nHow long does it take for rendering one image?\nI don't understand why “Remove the headscarf then move the camera” case in the teaser can generate the hair of the woman. cause in inference you just manipulate the gs, is ther anything generative that enables this capability?\n\nI pretty much love the paper, if the author can provide more details, I am happy to raise my score.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The quality is very well. The proposed methods are solid\n\nAuthors also conduct a lot of  experiments to compare with multiple baselines.", "weaknesses": "I am afraid that the SAM mask is not accurate, the distill loss may cause some artifacts. Like some of the gaussians in the edge may not be able to match the image prompt. Is it true?\nCurrent method cannot deal with shadow and lightning, like the Rotate the camera then delete the dog case in the teaser\nNeed more visual results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xaMPOkRimE", "reviewer": "Reviewer_gw3n", "summary": "This paper presents a scene editing framework that leverages language-guided disentangled Gaussian Splatting. Initially, the input image is projected into 3D Gaussians, which are subsequently refined and enriched through a 2D generative prior. Language features from CLIP are then embedded into the corresponding 3D Gaussians. This representation enables view changes and text-guided editing of the scene.", "questions": "Please refer to the weaknesses section.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The proposed technique is well-founded, and the experimental results are impressive.", "weaknesses": "The novelty appears limited, as the proposed method largely combines existing techniques. Projecting a 2D image into 3D Gaussians based on depth information has been demonstrated in works like LucidDreamer, among others. Additionally, embedding language features into 3D Gaussian Splatting (3DGS) and enabling text-guided editing have been explored in methods such as Grouping Gaussian, LangSplat, and GaussianEditor.\n\nIt is unclear how the proposed method outperforms prior techniques, such as LucidDreamer and AdaMPI, in Figure 6. While LucidDreamer and AdaMPI exhibit some out-of-range areas, simple inpainting could potentially address these issues.\n\nSince the method relies on depth maps to generate 3D Gaussian splats, it appears limited in its ability to handle significant changes in view angle.\n\nThe method seems to only function with generated images rather than real-world inputs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8qg5A46fQi", "reviewer": "Reviewer_Fas1", "summary": "The authors introduce 3DitScene, a unified framework that employs language-guided disentangled Gaussian Splatting, allowing seamless transitions from 2D to 3D editing. This approach utilizes refined 3D Gaussians informed by CLIP’s language features, enabling precise control over both individual objects and overall scene composition. Experimental results validate the effectiveness and versatility of 3DitScene, demonstrating its superiority over existing methods.", "questions": "See what is written in the weaknesses section.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. 3DitScene effectively integrates various editing requirements into a single framework, facilitating comprehensive manipulation of both scenes and objects.\n2. The framework employs language-guided disentangled Gaussian Splatting to produce a detailed 3D scene representation, which enables novel view synthesis and improves object decomposition.", "weaknesses": "The implementation details are not clearly articulated. Specifically, the training and optimization specifics for 3D Gaussian Splatting (3DGS) are lacking, including the number of iterations during initialization and for novel view optimization. Furthermore, the parameters of the loss function, such as λrecon, are not specified. Additionally, the experimental results do not appear to be satisfactory.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lxYbtmdMsm", "reviewer": "Reviewer_fiMW", "summary": "The paper proposed a unified framework, 3DitScene, for 3D-aware 2D image editing with different level of granulairies and diverse editing tasks, such as object level editing and camera adjustments.\nBy converting a single image into 3D Gaussian splatting (3DGS) representation, 3DitScene generates additional 3D content in previously unseen areas via RGB and depth inpainting from diffussion priors. Additionally, it distills CLIP language semantics into the expanded 3D. With the recontructed 3D scene with disentangled 3D semantics, the framework enables editing of both viewing camera and objects in the the image by camera, bounding box and textual controls.", "questions": "1. The paper lacks clarity on the editing process, despite mentioning text, bounding boxes, and camera controls in Sec.3.4.\nThe detailed operation steps are still ambiguous, and there is no clear editing procedure outlined in paper or video demo.\n\n    From Sec.3.4, it appears that editing primarily involves \"text-guided\" editing by querying specific object + object manipulation?\n\n    Is \"camera control\" achieved always by changing camera instrincs and extrinsics parameters? But in Fig.1, it seems that text prompts are also used for control.\n\n    In addition, there's no examples of bounding box usage in the paper.\n\n    I also suggest more editing details such as text prompts in Fig.5 and 6 and video, similar to what is shown in Fig.1.\n\n2. Since the 3D-aware editing is based on explicit 3DGS, and the disenangled object manipulation does not involve whole image re-generation. Accordingly, lighting inconsistencies arise after editng, e.g., shadows do not change appropriately as object moves. Can the lifted semantics help resolve this problem, or do you have any ideas can handle this obvious artifact?\n\n3. Language-semantics 2D-to-3D lifting is increasingly common nowadays, e.g., LERF, LangSplat, Gaussian Grouping. What are the advantages of proposed semantics lifting schemes over these existing works? Additionally, would other language-embedded 2D-to-3D lifting methods should be compatible with this proposed editing framework?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The work proposed a 3D-aware image editing pipeline by establishing semantic 3D representation via 2D-to-3D inpainting for novel view completion and language semantics lifting for semantic disenanglements. Under this language-guided disenangled 3D representation, the editing supports object-level maipulatiion and camera controls.\n\n2. Evaluation experiments are extensive for performance and comparisons with baselines. The method designs are validated through conducting ablation studies, exploring various alternative designs, including layout augmentation, losses, depth inpainting strategies.", "weaknesses": "1. While the paper proposed a practical and accurate image editing system, it does not demonstrate the editing procedure for practical use. Also see Questions 1 for further details.\n\n2. Some details are missing for method reproduction and training, such as trainining parameter configuration. It is also unclear about camera configuration, e.g. whether K and T in Eq.1 are always preset and fixed for given single image? More details are needed to facilitate reproduction and editing controls.\n\nAdditional concerns can be found in the Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)