3DGS-Drag: Dragging Gaussians for Intuitive Point-Based 3D Editing
Authors: Jiahua Dong
Yu-Xiong Wang
Venue: ICLR 2025 Poster (2025)
Keywords: 3D Editing
Diffusion Model
3D Vision
Primary area: generative models
TL;DR: An intuitive point-based 3D editing pipeline, which enables geometry-related content editing on 3D Gaussians.

Abstract: The transformative potential of 3D content creation has been progressively unlocked through advancements in generative models. Recently, intuitive drag editing with geometric changes has attracted significant attention in 2D editing yet remains challenging for 3D scenes. In this paper, we introduce 3DGS-Drag, a point-based 3D editing framework that provides efficient, intuitive drag manipulation of real 3D scenes. Our approach bridges the gap between deformation-based and 2D-editing-based 3D editing methods, addressing their limitations to geometry-related content editing. We leverage two key innovations: deformation guidance utilizing 3D Gaussian Splatting for consistent geometric modifications and diffusion guidance for content correction and visual quality enhancement. A progressive editing strategy further supports aggressive 3D drag edits. Our method enables a wide range of edits, including motion change, shape adjustment, inpainting, and content extension. Experimental results demonstrate the effectiveness of 3DGS-Drag in various scenes, achieving state-of-the-art performance in geometry-related 3D content editing. Notably, the editing is efficient, taking 10 to 20 minutes on a single RTX 4090 GPU.

Published as a conference paper at ICLR 2025
3DGS-DRAG: DRAGGING GAUSSIANS FOR INTUITIVE POINT-BASED 3D EDITING
Jiahua Dong
Yu-Xiong Wang
University of Illinois Urbana-Cham Champaain {j (jiahuad2, yxw} iiliooiii edu
User Edit
Rendered Drag Result
User Edit
Rendered Drag Result
User Edit
Rendered Drag Result
Figure I: Our proposed 3DGS-Drag framework enables high-quality 3D drag editing: Users
only need to input 3D handle points (circle) and target points (triangle). Our method precisely moves the handle points to match the target points while preserving the overall content and details.
ABSTRACT
The transformative potential of 3D content creation has been progressively un- locked through advancements generative models. Recently, intuitive drag edit- ing with geometric changes has attracted significant attention in 2D editing yet remains challenging for 3D scenes. In this paper, we introduce 3DGS-Drag point-based 3D editing framework that provides efficient, intuitive drag manip- ulation real 3D scenes. Our approach bridges the gap between deformation- based and 2D-editing- based 3D editing methods, addressing their limitations geometry-related content editing. We leverage two key innovations: deforma- tion guidance utilizing 3D Gaussian Splatting for consistent geometric modifica- tions and diffusion guidance for content correction and visual quality enhance- ment. progressive editing strategy further supports aggressive 3D drag edits. Our method enables wide range of edits, including motion change, shape ad- justment, inpainting, and content extension. Experimental results demonstrate the effectiveness of 3DGS-Drag in various scenes, achieving state-of-the-e art perfor- mance geometry-related 3D content editing. Notably, the editing efficient, taking 10 to 20 minutes on single RTX 4090 GPU. Our code is available at http://ggihuu..ooo/Dnnnaaaaee
INTRODUCTION
Recent years have witnessed remarkable advancements 3D scene representation techniques, such as Neural Radiance Fields (NeRF) (Mildenhall al., 2021) and 3D Gaussian Splatting (3DGS) (Kerbl et al., 2023). These methods have revolutionized the way we capture, represent, and synthesize 3D content, offering unprecedented levels of detail and realism. Inspired by their success

---

Published as a conference paper at ICLR 2025
and the blooming development of 2D generative models (Rombach et al., 2022). recent works in 3D generation (Tang et al., 2024; Poole et al., 2023) can now generate 3D content with high quality and efficiency. However, precise and intuitive editing of 3D scenes remains challenge, particularly in contrast to the sophisticated editing capabilities available for 2D images. While 2D editing methods like DragGAN (Pan et al., 2023) offer point-based manipulation, extending such functionalities to 3D scenes presents substantial technical hurdles.
Specifically, the underexplored capability behind is to achieve intuitive content editing with geomet- ric change. The recent progress in 3D editing can be roughly grouped into two classes: deformation- based and 2D-editing- based. The deformation-base primarily focus on motion editing, assuming strong geometry prior (Xie et al., 2024) or relying on video to learn motion pattern (Huang et al., 2024). Besides the requirement for sufficient prior infor- mation, they naturally cannot intuitively edit unseen content. For 2D-editing-baae basnnd methods, recent works (Haque et al., 2023; Dong & Wang, 2023; Chen et al., 2024a) have attempted to distill the editing ability from 2D diffusion models (Brooks et al., 2023) by editing the dataset of different view images with the 2D diffusion model. These approaches remain limited to appearance modifications and minor geometric adjustments, since larger 2D geometric edits fail to converge to 3D. The text guidance they used also sometimes causes incorrect edits, because the diffusion model fails to un- derstand the text prompt. Bridging the geometric editing ability from deformation and the content editing ability from 2D-editing models has not yet been well studied.
methods (Huang et al., 2024; Xie et al., 2024)
Motivated by these observations, introduce 3DGS-Drag an intuitive 3D drag editing method for real scenes. Extending the flexible editing format of DragGAN (Pan et al., 2023), we take 3D handle points and target points inputs, aiming for geometry-related 3D content editing. Our key insight is to fully leverage two sources of guidance for 3D content editing, which explicitly reg- ularize the edits from different views to be consistent and optimized toward the target 3D points. The first guidance is deformation guidance. Benefiting from the explicit representation of 3D Gaus- sian Splitting (Kerbl et al., 2023), we propose a simple but effective deformation strategy without the requirement for prior information. With such strategy, we directly deform the 31) Gaussians and leverage them as guidance for different views. Morcover, the deformation of the Gaussians fa- cilitates optimization around the deformed space, simplifying the generation of detailed geometry. The second one is diffusion guidance. Notably, since there is no prior information in our setting, the deformed Gaussians always have incorrect content and artifacts. We use the diffusion model to correct the content and improve the visual quality. This guidance is grounded in our observation that fine-tuned diffusion model serves view-consistent editor for 3D scene. Consequently, it achieves better consistency given previous deformation guidance.
To support more challenging 3D drag edits, we further propose a progressive editing strategy. Specif- ically, we divide the drag operation into several intervals and proceed with editing step by step. The continuity of editing is guaranteed by 3D relocation strategy. In the end, our experimental results demonstrate the effectiveness of our 3DGS- in various scenes and editing. We resolve the chal- lenges of a 3D drag operation and indicate an enhancement in multi-view consistency compared to prior techniques.
Our major contributions can be summarized as follows: 1) We present a novel framework for editing 3D scenes, featuring point-based drag editing approach. 2) We propose an effective method to bridge 3D deformation guidance and diffusion guidance for conducting scometry-relater 3D content editing. 3) We further propose progressive drag editing method to improve editing results. 4) Extensive evaluations show our method achieves state-of-the-art results such setting, which implicitly includes motion change, shape adjustment, inpainting, and content extension.
2 RELATED WORK
2.1 2D IMAGE EDITING
Initially, the image generation methods rise from generative adversarial networks (GAN) (Good- fellow et al., 2014; Karras et al., 2019). Based on its latent representation, early works tried to modify the latent to adjust certain attributes contents of the image (Abdal et al., 2021; Endo, 2022: Harkönen et al., 2020; Leimkühler & Drettakis, 2021). However, due to the limited capability of the GAN model and the implicit representation of the latent code, it is hard to achieve high-
2

---

Published as a conference paper at ICLR 2025
quality and detailed edits. Recently, diffusion models have shown great potential for text-to-image tasks (Rombach et al., 2022). Its feature map representation and the large-scale data empower lots of image editing methods (Kawar et al., 2023; Ramesh et al., 2022; Meng et al., 2022; Brooks et al., 2023). SDEdit (Meng et al., 2022) performs a nosing and denoising procedure to keep the structural information and change the details. Instruct-Pix2Pix (Brooks et al., 2023) builds instruction edit- ing dataset and train the diffusion model to edit the image following the instruction. Compared with previous methods, Instruct-Pix2Pix shows better editing consistency.
Although text-based image editing can generate high-fidelity results, it cannot reach fine-grained editing. DragGAN (Pan et al., 2023) proposed a point-based interactive editing method. The user inputs several handle points and target points; then, the latent will be optimized to move the han- dle points to the target. To improve the generality, DragDiffusion (Shi et al., 2024) transfers this technique to diffusion models (Rombach et al., 2022). Later, SDE-Drag (Nie et al., 2024) and RegionDrag (Lu et al., 2024) further improve the performance. An forward process is nec- essary for these diffusion- based methods, making this operation time-consuming. In addition, there is no 3D consistency guaranteed in such 2D models, thus not available to be directly applied to 3D.
In this paper, we adopt the 2D diffusion model to perform 3D-consistent view correction. Our editing not only generates intuitive new content but also removes potential 3D artifacts.
2.2 ED-EDITING-BANEE 3D EDITING
Previous to 3DGS (Kerbl et al., 2023). Neural Radiance Fields (NeRF) (Mildenhall et al., 2021) used as a common connector from 3D representation to 2D models. Early works on NeRF can only deal with color and shape adjustment (Chiang et al., 2022: Huang et al., 2021; 2022; Wu et al., 2023; Bao et al., 2023; Zhang et al., 2022; Jambon et al., 2023). SNeRF (Nguyen-Phone 2022) proposes to use an image stylization model, achieving high-quality stylization results. Later, NeRF-Art (Wang et al., 2023) uses CLIP (Radford et al., 2021) to distill the knowledge to NeRF. However, since the CLIP is not generative model and is highly semantic-based, such an approach cannot get results with high fidelity. Instruct-NeRF2Nede Instruct-Pix2Pix model to Iteratively edit the dataset. They can edit various scenes with broad range of instructions. ViCA-NeRF (Dong & Wang. 2023) proposes to directly edit the dataset with- out fine-tuning NeRF Specifically, they make multi-view consistent edits by utilizing the depth of information. DreamEditor (Zhuang et al., 2023) proposes to use a fine-tuned Dreambooth (Ruiz al., 2023) to help with editing. ConsistentDreamer (Chen al., 2024a) further fine-tune Con- trollNet to give more detailed edits. However, all these methods are limited by the 3D consistency from different views, thus only available to make subtle geometric changes. PDS (Koo et al., 2024) propose a new distillation loss to help improve the result but suffer from degeneration of rendering quality and the ability for sufficient geometric editing.
et al.,
(Haque et al., 2023) proposes to use the
Inspired by the efficiency of 3DGS, recent approaches (Fang et al., 2024; Chen et al., 2024b; Chen & Wang, 2024) propose to migrate the success of NeRF editing to 3G Gaussians. However, they are mainly following the idea of Instreut-NeRF2NeeR. (Haque et al., 2023) by changing the 3D representation, thus having similar limitations. Some approaches (Xie et al., 2023; Shen et al., 2024; Yoo et al., 2024; Dong et al., 2024) have attempted to extend the drag operation to 3D; however, they are limited to handling single objects. In contrast, our approach leverages the explicit representation of 3DGS and focuses real scenes.
2.3 DEFORMATION- EITORMATIION-BAEDD BASED 3D EDTING
3D deformation challenging task since the target is generate unseen motions. Traditional methods (Sorkine-Hornung & Alexa, 2007; Sorkine, 2005) apply certain Laplacian coordinates for mesh deformation. Recently, people have focused on deformation in 3D representations like NeRF and 3DGS. Specifically, Xu & Harada (2022) proposes to build 3D cages as the motion prior to guide deformation. Yuan et al. (2022) reconstruct the mesh from NeRF and deform the mesh instead. NeuralEditor (Chen et al., 2023) requires dense point cloud deformation as input and applies point- like NeRF structure for deformation. All these methods need strong geometry prior to editing, which is hard and inconvenient in practice. PhysGaussian (Xie et al., 2024) considers Gaussian ellipsoids as a Continuum and integrates physics. SC-GS (Huang et al., 2024) samples control points stracture-rpprseenting graph to guide motion. However, the physics simulation and continuum
3

---

Published as a conference paper at ICLR 2025
Multi-Step Editing Scheduler (Step
Handle Point Relocation & Target Points Scheduling
Image Buffer
Update
P'x(u), P':(0)
Handle Ph.
LoRA
Fine-tune
3D Target Point
Deform
Render
Diffusion
Deformation Guidance
Diffusion Guidance
Dutaset
User Input
Train Gaussians
Figure 2: Overview of 3DGS-Drag Given trained 3D Gaussian splatting model and the dataset, we use the multi-step editing scheduler to calculate the intermediate handle points P/, (x) and target points vi( for step i. In each step, first deform the 3DD Gaussians using handle points and target points. Then, we render the image for each view and correct with a diffusion model. The final corrected images will be used train 3D Gaussians improve quality. The diffusion model is fine-tuned with LoRA for more consistent edits.
assumption make PhysGaussian less flexible and limited to continuous scenes. SC-GS's 's control points are an approximation of dense points, thus also relying on sufficient capture of the object's geometry. It also takes dynamic scenes as input to build prior motion knowledge.
The hard prior knowledge requirements or strict assumptions make these methods not suitable for large real scenes, where there is often only part-view information and complex layouts. In addition, they not have the ability to create new parts. Rather than building better deformation method, propose simpler deformation strategy for 3DGS to give rough deformations. Since the 2D generative models (Rombach et al., 2022) already have the sense of normal motions and contents, borrow such knowledge to provide more flexible 3D edits
METHOD
3.1 PRELIMINARY
3D Gaussian Splatting. 3D Gaussian splatting (Kerbl al., 2023) uses a collection of 3D Gaussians to represent 3D information, demonstrating effectiveness in object and scene reconstruction tasks. Each Gaussian is characterized by center μ RR33 scaling factor E TR The model also incorporates an opacity value a E R and a color feature e E Rd for volumetric rendering, where d indicates the degrees of freedom. The full set of parameters denoted as I', where 1 [u', s', , a', c²} represents the parameters for the i-th Gaussian.
R and rotation quaternion
3.2 FRAMEWORK OVERVIEW
Our framework is illustrated in Figure 2. It takes a pretrained 3D Gaussian splatting model and several handle points along with their corresponding target points input. Specifically, the handle points are denoted as Ps- xx3 and the target points are denoted as pt xx3 where is the number of handle points. We aim to move the handle part the target position while preserving similar content. Depending on the input points, this process may entail appearance and geometric changes, allowing more challenging edits with user- aer-ffiendly inputs.
Different from the idea of 2D drag editing techniques (Pan et al., 2023; Shi et al., 2024; Lu et al., 2024), which either optimize or operate the inverse feature of 2D image, we use deformation- based geometric guidance and diflusion-based appearance guidance for 3D editing. For a single step of drag operation, we first deform the 3D Gaussians with the provided handle and target points (Sec. 3.3). Such deformation is conducted in aapy-and manner to allow more editing flex- ibility. Due to the sparsity and long-disg distance challenge of the drag operation, the rendering result from the deformed Gaussians have poor visual quality and incorrect content. Thus, we propose to use diffusion- seffuuiion-uuided image correction on the rendered images (Sec. 3.4), which efficiently corrects the contents and removes artifacts. To resolve editing with aggressive changes, propose
4

---

Published as a conference paper at ICLR 2025
multi-step editing scheduler to progressively edit the scene (Sec. 3.5). As the whole process is divided into intervals, the user can stop at any intermediate step when achieving a satisfactory out- come.
3.3 DEFORMATION GUIDANCE FOR GEOMETRIC MODIFICATION
As we aim to deform the 3D scenes to provide geometry guidance, we leverage 3DGS to benefit from its explicit representations and efficiency. The deformation involves two challenges in our task: (1) How to approximately deform the 3D Gaussians given sparse handle points and long-distance drag target, without structural modification to standard 3DGS; (2) How avoid degeneration to direct deformation, allowing more flexibility to edits like moving, extending, and others. The proposed solution is described as follows. As result, we achieve reliable deformation to 3DGS given limited point information.
Drag Deformation. The explicit representation of 3DGS enables efficient 3D deformation and ad- justment. However, the real deformation function cannot be precisely computed, given only handle points and target points. Thus, we approximate it to give a rough geometry guidance. For the ith handle point Pi. we assign the Gaussians P/ within certain distance in 3DD to this point. These Gaussians are considered to be deformed and guided by this handle point. The union of (Pili = 1, n} is denoted as Pa = -
Firstly, we calculate the translation and rotation for each handle point. For the translation, we simply calculate it Apj, = p ph. For the rotation, it not to further change the position of handle points but to represent the potential orientation change. Since the 3D Gaussians are also parame- terized by rotation q, such a parameter crucial to guide the Gaussian deformation. However, our handle points are just coordinates without information on the orientation. To approximate the rota- tion, we calculate its relative rotation with top-K K (K = 2) nearest handle points {p), lk N where Nj, are the indices of top-K nearest handle points. Linear weight is used due the sparsity of the points. Specifically, the weight is calculated as:
1p%
wht =
ΣjeNE |0o - p//
Then, calculate the relative rotation quaternion Aath between ph and p (Details in Sec. D), and the quaternion Aqj, of pair (pi. p() is calculated as Agj. EKEN, wi Ag
After calculating each handle point's translation and rotation quaternion, we can interpolate the entire 3D Gaussians" deformation. Specifically for each Gaussian D E Ph. the deformed Gaussian interpolated from the transformation of top-K (K 2) nearby handle points {pi,1 N,) where N; are the indices of top-K nearest handle points. The deformed center n² and rotation quaternion da are:
11
p1l12
w
EjeN. 1r -pih
11a 1¹ Σ with Apti
KeN:
da
Σ (w*
Agh @
REN;
where 2² and q¹ are the original center and rotation quaternion. @ is the quaternion production. When there is only one handle point, no quaternion change will be applied. We do not directly update the old Gaussians to the deformed Gaussians since this limits deformation and is not suitable for tasks like "make his sleeves longer." Inspired by SDE-Drag (Nie et al., 2024), we use a copy- and-paste manner to place the deformed Gaussians and keep the old ones. To offer more flexibility for optimization, we adjust the opacity of the original Gaussians P, to a smaller value, allowing the 2D updates to determine whether to keep remove the Gaussians.
5

---

Published as a conference paper at ICLR 2025
Deformed Gaussions
Consistent 2D Edits
Edited Rendering
Figure 3: Multi-view consistent 2D edits: With the deformed rendering as input, the fine-tuned diffusion model can perform multi-view consistent edits, and the artifacts and incorrect parts (shoes) are fixed.
Local Editing Mask. Since drag operations mainly focus on a part of the entire scene, local edit- ing is necessary to maintain the background information. Following Gaussian Editor (Chen et al., 2024b), we assign a mask M to the Gaussians of Ph. which are considered changeable. Different from Gaussian Editor, our work builds both 3D and 2D local editing masks to work with more com- plex scenes and geometry edits. For the 3D mask, we inherit the mask from the original Gaussians when deforming new Gaussians or during the densification procedure. These Gaussians outside of the mask are not changed in the optimization. For the 2D mask, we render the mask for each view and round it to (0, 1) with threshold, resulting in masks (m") where denotes the eth view. Note that the mask rendering is after the deformation, so the original region and the target region will both be covered. The mask further dilated to change the context of the nearby area.
3.4 DIFFUSION GUIDANCE FOR APPEARANCE CORRECTION
The direct deformation of Gaussians often creates notable artifacts and cannot generate correct se- mantic content. Inspired by recent successes in 3D editing (Haque et al., 2023), we update the dataset to edit 3D scenes. However, integrating the concept of 2D dragging into a 3D context is non-trivial. Previous 2D drag methods often necessitate time-consuming forward and backward process (Shi al., 2024; Nie al., 2024). Moreover, during the training process, the inconsistent 2D edits from different views make the final result deviate from expectations and full of artifacts. To address these issues, we propose to use inverse-free 2D image editing that achieves stronger consistency, efficiency, and quality, relying consistent renderings from the deformed 3D content. As shown in Figure 3, our method generates multi-view consistent 2D edits. In detail, given the rendered image from the deformed 3D Gaussians, introduce Image2lmage view correction to obtain corrected 2D edits. To overcome the challenge of dataset editing with geometry change, we update the dataset an annealed dataset editing way.
Imageemmage View Correction. Although the deformed Gaussian gives better 3D consistency, cannot benefit from latent-based drag methods (Pan et al., 2023). This because the 3D consistency is ensured with newly rendered images. In contrast, latent-based methods heavily rely on operating the feature map of the same image. Inspired by the common approach for image editing (Meng et al., 2022), we add noise and then denoise it through the Dreambooth (Ruiz et al., 2023) model. By changing the image to a sketch level and denoising it, the diffusion model can partially understand and complete the deformed part.
To mitigate the influence of randomness from the diffusion model, the Dreambooth model fine- tuned on each scene with LoRA (Hu et al., 2022). We find that after fine-tuning, the diffusion model becomes multi-view consistent editor. The experiment results in Figure 3 show that the diffusion model can successfully understand the deformed image and generate image with the correct content even without the inverse process. However, such corrections still cannot fully converge in update, requiring better dataset editing strategy follows.
Annealed Dataset Editing. Iterative dataset editing has been common approach for 3D appear- ance editing (Haque et al., 2023). The idea is to progressively change the appearance of 3D and use the rendering to guide consistent 2D editing further. However, such strategy does not work well with gelaetry-related edits because it is harder converge given inconsistent geometry. In addition,

---

Published as a conference paper at ICLR 2025
User Edit
Step
Step 2
Step
Dragging Steps
Figure 4:
Intermediate dragging steps and tracked mask: Our method conducts progressive editing toward the target point. The dragged Gaussians are tracked to achieve aggressive edits.
long-term iterative updates also accumulate serious blurriness (Haque al., 2023). To address this, we propose to update the dataset with limited A times, and each time anneals the strength (Meng al., 2022) for Image2Image view correction. The annealing function is follows:
S(a) Smit a- (Smit Sama), 1,2,33
(5)
where Smitt and Stnal are the initial strength and final strength respectively. S(a) denotes the strength for the ath updates. Note that lower strength means that diffusion starts from later timesteps, resulting in finer detail correction. Our strategy performs editing in a coarse-to-fine manner. Each time, all the views updated to prevent accumulated errors.
Loss Function. With the rendered image IN from 3D Gaussians, the corresponding edited image T the editing area's s groundtruth, the original image I background groundtruth and mask m for view v, our loss function for training 3D Gaussians is formulated as:
L Σ (T, Assiim (Tr, (e)) @ (1-m") Xipips (",, Ie) m"),
(6)
where Li and Lasim ensure local editing. Lipips the LPIPS (Zhang al., 2018) loss function to correct the editing area. \ι Assim and Aipips are the weighting coefficient for each loss.
3.5 FROM ONE-STEP TO MULTI-STER DRAG EDITING
The previous sections introduce the one-step drag editing using our method. As the long-distance drag operation often requires more than one step to avoid corruption, we propose multi-ster step editing scheduler to solve such problems. Specifically, we split the drag operation into T intervals and set the progressive target points (p](u)]u 1, 1.2....T) T). corresponding target points:
each interval, we perform drag toward the
" (w) = Ph ( Ph),
However, the actual handle point position usually changes when training 3D Gaussians. We propose relocating the handle points at the end of every interval to make the next interval's 's deformation more precise. In addition, we further conduct history-awar diffusion fine-tuning improve the ability for more aggressive editing.
The handle point relocation is performed after each interval's training process. To keep track of the handle points, we use the Gaussians associated with each handle point. Specifically for handle point p/,, we update it with the averaged position change of Gaussians P/. As shown in Figure 4, the dragged part can be successfully relocated. Note that the assigned Gaussians Pý are updated to newly deformed Gaussians during deformation and inherited from parents during the densification process of training. The local mask is updated as the union with the mask.
Handle Point Relocation.
History-Awan Diffusion Fine-Tuning, For can shift out of the diffusion model's domain since it is fine-tuned on the original images, resulting in degeneration back to the original images. We build an image buffer to fine-tune the diffusion model. The diffusion model will be fine-tuned with the image buffer every interval. Initially, the buffer only contains original images, and the newly edited result will be added to the buffer during intervals.
drag operations, the edited 2D images

---

Published as a conference paper at ICLR 2025
Drug
Edits
Rendered Results
Figure 5: Qualitative results in various scenes: Our method can handle complex scenes and gen- crate highly detailed results. With a simple drag input, 3DGS-Drag can identify the 3D context and perform edits like moving objects, inpainting the background, adjusting appearance, modifying object shape, and adjusting motion. The orange bounding boxes highlight the modified regions.
4 EXPERIMENT
4.1 IMPLEMENTATION DETAILS
User Input. Our input is multiple handle points and corresponding target points. The input points are in 3D space. The user can specify the sphere radius of that handle point to adjust the editing scale. We automatically perform local editing by applying the mask rendered from assigned Gaussians. The mask is dilated to change the necessary context.
Drag Editing. The pretrained 3D Gaussians trained with original 3D Gaussian Splatting (Kerbl et al., 2023). During editing, 50 views are selected to enable efficient editing by default. Specifically, we choose the views with a larger visible area the handle points' Gaussians, which is determined by the local editing mask on each view. We fine-tune the Dreambooth model (Ruiz et al., 2023) with fine-tuned on the selected views. We use batch size 4 and train for 200 iterations. After each dragging step, we continue fine-tuning the diffusion model for 50 iterations with the updated image buffer in each interval. Each time, the newly edited image will be enqueued. The loss weight of YY Assim and Alpips set to 8, 2, respectively, Note that the À and Assim are 10 times bigger than normal to ensure the background.
LoRA (Hu et al., 2022). Initially,
Datasets. experiments include edits on eight scenes, using the published datasets from Instruct- NeRF2NeRF Haque et al. (2023), PDS Koo et al. (2024), Mip-NeRF360 Barron et al. (2022), and Tank and Temple Knapitsch al. (2017).
4.2 QUALITATIVE EVALUATION
Editing Results in Various Scenes. We show editing results from different views in Figure 1 and Figure 5. Since the handle and target points are 3D, we plot them 2D for illustration. Each drag is represented by red arrow where the start is the handle point, and the end is the target point. In the standing- personn-ppoonnn scene in Figure when we raise one hand, this is very challenging since the arm only observed partially, and the part under the arm is unknown. Our method also shows the ability to generate new poses and fix the texture on the pants below the arm. We are also able to change the leg motion and extend the sleeves. When dealing with more complex scenes, such as the bamboo scene in Figure 1, 3DGS-Draa -Drag can understand the texture of the plant and extend it to be taller or wider. We can also easily change part of the background, like the wall. When the drag operation is to move the football, we can separate this object from the background and inpaint the texture at the original position instead of empty region. In short, our drag operation understand different operations
8

---

Published as a conference paper at ICLR 2025
Soxced apicading Rari
Figure Baseline comparisons: Compared with baselines, 3DGS-Drag achieves high-quality, fine-grained editing by correctly modifying different parts and in terms of efficiency. Specifically, Destruct-eeR2NeeR (Haque et al., 2023) and PDS (Koo et al., 2024) cannot correctly edit. De- formation results in incomplete edits, and SDE-Drag (Nie et al., 2024) sometimes fails to make changes.
User Edit
W/O Local Editing
1-Step Drag
5-Step Drag
20-Step Drag
Figure 7: Ablation study the local mask and drag steps: Without the local mask, the scene will be blurred, resulting in failed edits. Using very few steps makes hard to achieve aggressive edits. More steps will slightly improve the performance.
in front-view 360-degree scenes, such moving objects and extending objects, demonstrating the ability to identify the 3D context.
Baseline Comparison. Since there is no directly comparable work intuitive 3D drag operation in real scenes, we extend and re-purpose representative baselines. The results are shown Figure 6. Specifically, the comparison with baselines is listed as follows:
Nessrect-eRF2NeRF (Haque et al., 2023): We manually create text descriptions for drag opera- tions in this baseline. Then, give edits for the 'person' scene. For the more complex 'garden' scene, Instruct-NeRF2NeRR just blurs the rendering. This demonstrates its insufficient ability to perform geometric modification. Deformation: We use our deformation to represent the previous deformation-bass approaches since have different input settings. Notably, the geometry is moved, which results lot of incorrect content and artifacts.
Instruct-NeRF2NeRE edit the scene. The model fails to
PDS: PDS (Koo al., 2024) claims to able change the geometry, but this method struggles in all three editing scenarios. In addition, PDS tends to create noisy and blurred editing results compared with others.
SDE-Drag One alternative solution is simply use the 2D drag method each view. Here we choose SDE-Drag (Nie et al., 2024) in comparison. However, such a strategy cannot reach consistent edits, resulting in flawed results failure cases in editing.
Compared with these baselines, our methods achieve significantly better editing results, with better details and correct content. Remarkably, for the "lower his hairline" text prompt, both Instruct- NeRF2NeRF and PDS misunderstand the text and make the hairline higher, which further empha- sizes the importance of intuitive 3DD editing.

---

Published as a conference paper at ICLR 2025
Ablation Study The diffusion guidance's effectiveness is validated when compared with the de- formation approach (Figure 6). Here, We further ablate the local mask and multi-step strategies in Figure 7. (I) When local editing is not applied, the entire scene is blurred, and the edits fail. This is due to the optimization issue: inconsistent edits will create large floats in 3D Gaussians. (2) For drag steps, compare three different drag steps from [1, 5, 20), finding that more fewer steps lead to different insights. When using single step, the deformed Gaussians cannot give enough guidance to the diffusion model, resulting broken edits. Thus, one-step drag editing usually meets challenges when we have more aggressive edits. When applying more steps (20 steps), the editing quality is slightly improved. This illustrates that 3DGS- 3DGS-Drag is robust when updated more times. However, since more steps will slow the execution, choosing an appropriate number of steps is better.
4.3 QUANTITATIVE EVALUATION
Quantitatively evaluating 3D editing results often challenging since there lacks ground truth. Here, two metrics for evaluation: user preference and GPT score, shown in Figure 8. For user preference, we conducted user study across 19 subjects and collected their preference for each edit. For the GPT score, since GPT with vision has been proven to be human-aligned evaluator (Wu et al., 2024), we use gpt4-o to evaluate each editing, rating in 5 levels. Specifically, we measure (1) whether the content is correctly edited and (2) the rendered image quality for each method. Our method achieves the best results on all these metrics.
User Preference
GPT Evaluation
R1.89%
11,40%
387
1.41
35
Beedering Qualit
70.9%%
61.81%
3.5
51.0%
31.0%
21.8%
10.0%%
081%
30G2-Brrag (Dzo)
Insicec--dBD2Natt
PDS
3008 30G8-Brag
Figure 8: Quantitative evaluation: We conduct both user study and GPT evaluation on the editing results. Compared with Iestreet-NeRFNNeRF (Haque et al., 2023) and PDS (Koo et al., 2024), 3DGS-Drag performs significantly better.
4.4 DISCUSSION
Limitations. Similar previous diffusion-based 3D editing methods (Chen et al., 2024b; Haque et al., 2023), our approach relies on the diffusion model to provide accurate guidance. Thus, our method may yield suboptimal results when the target object is too small within the field of view or when the scene exhibits considerable size and complexity. We also cannot deal with drag operations that are too aggressive. such cases, the object may be relocated to areas with restricted visibility, which is out of vision for most views.
Running Time. When using 50 views for editing. our method needs 15 minutes. Specifically, about 2 minutes are needed for initial diffusion model fine-tuning, and 13 minutes needed for the rest of the editing process. In comparison, Instruct-NeRF2Nee (Haque et al., 2023) needs one hour. The running time is tested single RTX 4090 GPU.
5 CONCLUSION
In this paper, we introduced 3DGS-Drag, an intuitive drag editing approach for 3D scenes. In con- trast to previous work (Haque et al., 2023; Dong Wang, 2023; Wang et al., 2023), which mainly focuses on appearance, we address the challenge of geometry-related content editing. Empirical experiments show that our method can achieve highly detailed edits across various scenes. Such an advantage stems primarily from our key contributions: the copy-and-paste Gaussian defor- mation and the diffusion correction. We showcase that our method enables previously challenging edits, paving the way for exploring new possibilities in 3D editing.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported in part by NSF Grant 2106825, NIFA Award 2020-67021-32799 the Toy- ota Research Institute, the IBM-IIlinois Discovery Accelerator Institute, the Amazon-lliinois Center on AI for Interactive Conversational Experiences, Snap Inc., and the Jump ARCHES endowment through the Health Care Engineering Systems Center at Illinois and the OSF Foundation. This work used computational resources, including the NCSA Delta and DeltaAl supercomputers through al- locations CIS220014 and CIS230012 from the Advanced Cyberinfraatrructure Coordination Ecosys- tem: Services Support (ACCESS) program, as well as the TACC Frontera supereomputer, Ama- zon Web Services (AWS), and OpenAl API through the National Artificial Intelligence Research Resource (NAIRR) Pilot.
REPRODUCIRILITY STATEMENT
Our code is released https://gtthub.com/ com/DonggiahuuaAAGGS.... -rag. For the implementation details, we have covered our mathematical details in Sec. 3.3 and training details in Sec. 4.1. The framework architecture fully introduced Sec. All the datasets we used are publicly available, as explained in Sec. 4.1.
REFERENCES
Rameen Abdal, Peihao Zhu, Niloy Mitra, and Peter Wonka. Styleflow: Attribute-conditioned exploration of gyylegan-generane images using conditional continuous normalizing flows. ACM Transactions Graphics, 2021.
Chong Bao, Yinda Zhang, Bangbang Yang. Tianxing Fan, Zesong Yang. Hujun Bao, Guofeng Zhang, and Zhaopeng Cui. SINE: Semantic-driven image-based NeRF editing with prior-guided editing field. In CVPR, 2023.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022.
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In CVPR, 2023.
Jun-Kun Chen and Yu-Xiong Wang. Proedit: Simple progression is all you need for high-quality 3d scene editing. NeurlPS, 2024.
Jun-Kun Chen, Jipeng Lyu, and Yu-Xiong Wang. Neuraleditor: Editing neural radiance fields via manipulating point clouds. In CVPR, 2023.
Jun-Kun Chen, Samuel Rota Bulð, Norman Müller, Lorenzo Porzi, Peter Kontschieder, and Yu- Xiong Wang. Consistdreamer: 3d-consistent 2d diffusion for high-fidelity scene editing, In CVPR, 2024a.
Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang. Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In CVPR, 2024b.
Pei-Ze Chiang, Meng-Shium Tsai, Hung- Yu Tseng, Wei-Sheng Lai, and Wei-Chen Chiu. Stylizing 3D scene via implicit representation and hypernetwork. In WACV, 2022.
Jiahua Dong and Yu-Xiong Wang. Vica-nerf: View-consistency-aware 3d editing of neural radiance fields. In NeurIPS, 2023.
Shaocong Dong, Lihe Ding, Zhanpeng Huang, Zibin Wang, Tianfan Xue, and Dan Xu. Interac- tive3d: Create what you want by interactive 3d generation. In CVPR, pp- 4999-5008, 2024.
Yuki Endo. User-controllable latent transformer for stylegan image layout editing. In Computer Graphics Forum, 2022.
Jiemin Fang, Junjie Wang. Xiaopeng Zhang, Lingxi Xie, and Qi Tian. Gaussianeditor: Editing 3d gaussians delicately with text instructions. In CVPR, 2024.
11

---

Published as a conference paper at ICLR 2025
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 2014.
Ayaan Haque, Matthew Tancik, Alexei A Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2neff: Editing 3d scenes with instructions. In ICCV, 2023.
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan controls. In NeurlPS, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022.
Hsin-Ping Huang, Hung-Yo Yu Tseng, Saurabh Saini, Maneesh Singh, and Ming-Hsuan Yang. Learning to stylize novel views. In ICCV, 2021.
Yi-Hua Huang, Yue He, Yu-Jie Yuan, Yu-Kun Lai, and Lin Gao. StylizedNeRF: Consistent 3D scene stylization as stylized NeRF via 2D-3D mutual learning. In CVPR, 2022.
Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: Sparse-controlled gaussian splatring for editable dynamic scenes. In CVPR, 2024.
Clément Jambon, Bernhard Kerbl, Georgios Kopanas, Stavros Diolatzis, Thomas Leimkühler, and George Drettakis. NeRFshop: Interactive editing of neural radiance fields. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2023.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. CVPR, 2019.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics, 2023.
Arno Knapitsch, Jaesik Park, Qian-Yi Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics, 2017.
Juil Koo, Chanho Park, and Minhyuk Sung. Posterior distillation sampling. In CVPR, 2024.
Thomas Leimkühler and George Drettakis. Freestylegan: Free-view editable portrait rendering with the camera manifold. In SIGGRAPH Asía, 2021.
Jingyi Lu, Xinghui Li, and Kai Han. Regiondrag: Fast region- based image editing with diffusion models. In ECCV, 2024.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In ICLR, 2022.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 2021.
Thu Nguyen-Phuoe Feng Liu, and Lei Xiao. SNeRF: Stylized neural implicit representations for 3D scenes. In WACV, 2022.
Shen Nic, Hanzhong Allan Guo, Cheng Lu, Yuhao Zhou, Chenyu Zheng, and Chongxuan Li. The blessing of randomness: Sde beats ode in general diffusion--aasee based image editing, In ICLR, 2024.
Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your gan: Interactive point-based manipulation on the generative image mani- fold. In SIGGRAPH, 2023.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. In ICLR, 2023.
12

---

Published as a conference paper at ICLR 2025
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with CLIP latents. In arXiv preprint arXiv:2204.06/25, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. CVPR, 2023.
Sitian Shen, Jing Xu, Yuheng Yuan, Xingyi Yang, Qiuhong Shen, and Xinchao Wang. Draggaussian: Enabling drag-style manipulation on 3d gaussian representation. In CVPR, 2024.
Yujun Shi, Chuhui Xue, Jiachun Pan, Wenqing Zhang, Vincent YF Tan, and Song Bai. Dragdiffu- sion: Harnessing diffusion models for interactive point-based image editing. In CVPR, 2024.
Olga Sorkine. Laplacian mesh processing. Eurographics (State of the Art Reports), 2005.
Olga Sorkine-Homung and Mare Alexa. As-rigid-as-possible surface modeling. In Eurographics Symposium on Geometry Processing, 2007.
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. In ICLR, 2024.
Can Wang, Ruixiang Jiang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. Nerf-art: Text-driven neural radiance fields stylization. IEEE Transactions on Visualization and Computer Graphics, 2023.
Qiling Wu, Jianchao Tan, and Kun Xu. PaletteNeRF: Palette-based color editing for NeRFs. In CVPR, 2023.
Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gordon Wetzstein. Gpt-4v (ision) human-aligned evaluator for text-to-3d generation. In CVPR, 2024.
Tianhao Xie, Eugene Belilovsky, Sudhir Mudur, and Tiberiu Popa. Dragd3d: Vertex- editing for realistic mesh deformations using 2d diffusion priors. In arXiv preprint arXiv:2310.0456,, 2023.
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang, Phys- gaussian: Physies-integrated 3d gaussians for generative dynamics. In CVPR, pp. 4399-4398, 2024.
Tianhan Xu and Tatsuya Harada. Deforming radiance fields with cages. In ECCV, 2022.
Seungwoo Yoo, Kunho Kim, Vladimir Kim, and Minhyuk Sung. As-plausible-as possibbbe: Plausibility-wware mesh deformation using 2d diffusion priors. In CVPR, pp. 4315 4324, 2024.
Yu-Jie Yuan, Yang-Tian Sun, Yu-Kun Lai, Yuewen Ma, Rongfei Jia, and Lin Gao. Nerf-editing: geometry editing of neural radiance fields. In CVPR, 2022.
Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, and Noah Snavely. ARF: Artistic radiance fields. In ECCV, 2022.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In CVPR, 2018
Jingyu Zhuang, Chen Wang, Liang Lin, Lingjie Liu, and Guanbin Li. Dreameditor: Text-driven 3d scene editing with neural fields. In SIGGRAPH Asia, 2023.
13

---

Published as a conference paper at ICLR 2025
A DEMO VIDEO
A demo video of our framework description and editing results is included in the supplementary
material.
B ADDITIONAL EXPERIMENTS
B. QUALITATIVE RESULTS LARGE OBJECT MOVEMENTS
We further conduct experiments object movements, specifically including large movements of both regular-sized and large objects. As shown in Figure 9, our method succeeds in handling long- distance movements, such repositioning the flowerpot. Furthermore, for very large objects like the truck and the table, our approach effectively moves them in a specified direction while minimizing artifacts in the removed and placed regions. These results showcase the generalizability of method.
User Edits
Rendered Dmag Results
Figure 9: Additional qualitative results for larger movements and larger objects: Our method succeeds in longer-range movements like moving the flowerpot and large object movements like moving the table.
B.2 ABLATION ON DATASET EDITING STRATEGY
To validate the importance of our dataset editing strategy, we conduct comparison between using our annealed dataset editing and the iterative dataset editing (Haque et al., 2023). As shown in Figure 10, editing one frame each time cannot change the geometry due to inconsistent constraints from other unedited views, leading to degenerated result in the original scene. Instead, our method can successfully make the edits.
B.3 QUANTITATIVE ABLATION ON LOCAL EDITING
To further demonstrate the effectivvnees of our local editing, conduct quantitative evaluation on the "extend sleeves" edit. Specifically, we calculate the similarity between the rendered edited result and the originally rendered image in the unedited pixels. As shown in Table 1, our local editing strategy demonstrates a strong capability to preserve the unedited regions and backgrounds effectively.
14

---

Published as a conference paper at ICLR 2025
User Edit
Annealed Datasct Editing (Ours)
Instruct-NcRF2NeRF Datasct Editing
Figure 10: Ablation on dataset editing strategies: Iterative dataset editing from Instruct-
NeRF2NeRF (Haque al., 2023) leads degenerated results. In contrast, annealed dataset editing maintains the geometry change.
SSIM
PSNR1
LPIPSJ
Local Editing
0.995
43.43
0.004
Non-Local Editing
0.901
24.44
0.158
Table 1: Quantitative ablation local editing: Our local editing strategy demonstrates strong capability to preserve the unedited regions and backgrounds effectively.
B.4 SCALED USER STUDY
To improve the generalizability of user study, increased the number of participants from 19 to 99. As shown in Figure 11, the key conclusion that our method surpasses previous baselines remains the same.
User Preference
80,0%
70.3%
60.0%
50,0%
40,0%
30.0%
20.0%
17.0%
12.7%
10,0%
0.0%
3DGS-Drag (Ours)
Instruc-NcRF2NoRF
PDS
Figure 11: Scaled user study with 99 participants: Our method still achieves significantly better preference over the baselines.
B.5 COMPARISON WITH 2D DRAG METHODS
We conduct qualitative comparisons on 2D drag edits to verify our method's effectiveness. Specif- ically, we focus on our deformation-guided diffusion editing quality, comparing the result with the previous 2D drag methods, Dragdiffusion (Shi et al., 2024) and SDE-Drag (Nie et al., 2024). As shown in Figure 12, DragDiffusion tends to create many more artifacts and unrelated textures, while result is cleaner and matches better with the edit. Considering SDE- DDag (Nie al., 2024), it fails to move the leg correctly and instead generates an object at the location. The results show that method can achieve better consistency by directly operating the image level.
15

---

Published as a conference paper at ICLR 2025
User Edit
DragDiffusion
SDE-Drag
Ours
Figure 12: Comparison on 2D drag results: Compared with recent 2D drag methods, our method
does not need a time-consuming pwerse-forward process and produces more consistent results. In comparison, the baseline DragDiffusion generates noisy legs and floor. SDE-Drag succeeds in main- taining the background but inserts objects in the hand and does not correctly move the leg.
Rendered Drag Results
Figure 13: Limitation on generating unseen side: When dragging background objects with an unseen side the foreground, the results from other views incorrect.
C IN-DEPTH DISCUSSION OF LIMITATIONS
Our method encounters two specific failure cases: generating the unseen side of an object and drag- ging objects outside the border area. Here, we present qualitative results to further elucidate these limitations.
C.I GENERATING UNSEEN SIDE
As shown in Figure 13, when moving the wooden support the foreground, the portions (e.g., the back of the object) are rendered incorrectly and exhibit noticeable artifacts. This occurs because there are 3D Gassians to represent the unseen back side. Consequently, the deformed result contains meaningless patterns that cannot be corrected through the diffusion process.
C.2 DRAGGING OBJECTS OUTSIDE THE BORDER
As shown in Figure 14, we struggle to refine the artifacts when only a portion of the object is visible the border. This because the diffusion model has less capability to correct the boundary part due to the ambiguity in interpolating the part to the whole. In addition, since is observed by sparser views, raises further challenges for optimizing 3D Gaussians.
C.3 POTENTIAL BIASES IN QUANTITATIVE EVALUATIONS
While employ both human preference scores and automated metrics from GPT evaluation, there could still be potential biases, such as those arising from the participant selection process or the specific version and training data of the GPT models used. This challenge is common in generative
16

---

Published as a conference paper at ICLR 2025
Figure 14: Limitation on dragging objects outside the border: Refining and optimizing become challenging when dragging towards the unseen or border area.
modeling, where there no ground truth. Future work would benefit from conducting larger-scale human studies with more diverse participant pools and developing comprehensive evaluation protocols that can better assess both geometric accuracy and visual fidelity 3D edits.
D ADDITIONAL DEFORMATION DETAILS
The calculation relative rotation Aghh is briefly described as follows. Given handle points pA and P and their corresponding target points pi and oÊ respectively, we firstly calculate the unit vectors
Ph
p P
and
pill
ll - rill
Next, we compute the cross product and the dot product of these two unit vectors:
x Ut
(9)
We then construct the quaternion Agh te by combining the dot product and cross product:
Aqh
tk [s, Fg.
(10)
Finally, standardize and normalize the quaternion ensure it has unit length.
E SOCIAL IMPACT AND FUTURE WORK
Future Work. For future work, plan extend current progressive editing capabilities generate 3D animations. As 3DGS-Drag is able to move or modify objects progressively, it is possible to generate long-term trajectories and human motions. In addition, model's scalability to accommodate larger scenes with dynamic objects and shadow effects.
will focus improving the
Potential Social Impact. The potential societal impact of 3DGS- SoGS-Drag spans across multiple dimen- sions. Designed as a fine-grained editing model, our 3DGS-Dra offers convenient manipulation of 3D scenes and robust support for AR applications. In addition, given the rapid development and widespread adoption of 3D Gaussians, our method seamlessly integrates with this ecosystem. With its user-friendly interface requiring only the selection of handle and target points, our model ac- cessible even to untrained individuals.
17


Reviews:
[{"review_id": "H7xBrB7d2s", "reviewer": "Reviewer_SwqC", "summary": "This paper proposes a drag-based 3D Gaussian editing method. The pipeline is divided into two steps. It first binds 3D Gaussians to nearby handle points, and copy-and-paste the handle points to the target position. Secondly, it leverage a diffusion model to correct artifacts caused by Gaussian movements. Specifically, it converts the rendered image to sketch level to help the diffusion model understand the complete the deformed part. It also use Dreambooth and iterative dataset updating to better ensure 3D consistency. The paper also extends from one-step editing to multi-step editing, providing a stable and reliable 3D Gaussian editing method.", "questions": "No.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a drag-based 3D Gaussian editing method, while previous methods focus on text-guided editing. It uses a fine-tuned diffusion model to provide view-consistent correction to the edited scene and unsure rendering quality through iterative dataset updates. It also introduces multi-step drag editing to allow long-distance editing operations.", "weaknesses": "The examples shown in the paper have small movements. Will the method fail if we move a large object to a large range of movement? I suggest the author to provide some failure cases, which will better illustrate the upper bound of the method and make the work more solid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UrEf9HCCUx", "reviewer": "Reviewer_zAUe", "summary": "This paper introduces a drag-based 3D editing framework for 3D Gaussian representations. The approach employs deformation guidance to deform 3D Gaussians (3DGS) from a specified handle point to a target point, followed by diffusion guidance to enhance visual quality. Experimental results demonstrate that this method outperforms existing baselines.", "questions": "1. Can this method reposition the entire flowerpot (as in Fig. 2) rather than only elongating it?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents an innovative drag-based approach for editing 3DGS.\n\n2. The experiments are thorough and provide convincing evidence of the method’s effectiveness.\n\n3. The paper is well-structured.", "weaknesses": "1. In the case where the wall is widened (bottom right in Fig. 1), there is a noticeable color discrepancy between the original and the widened sections. Additionally, leaves near the edited boundary on the wall appear blurry.\n\n2. In the baseline comparison, authors claim that \"there’s no exact previous work on intuitive 3D drag operation\". However, this is inaccurate.  ARSP [1] implements drag-based 3D editing using mesh-based deformation techniques. Interactive3D [2] offers a set of deformable 3D point operations on 3DGS and utilizes SDS to optimize the deformed 3DGS.\n\n[1] Yoo, Seungwoo, et al. \"As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priors.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. \n\n[2] Dong, Shaocong, et al. \"Interactive3D: Create What You Want by Interactive 3D Generation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "neB4bebRyq", "reviewer": "Reviewer_zn4P", "summary": "This paper presents a method for editing gaussian-splatting 3D representation of scenes. The GS-represented 3D objects can be moved by geometrical 3D arrows designated in 3D scenes. To compensate for degradation of the rendering quality of the deformed objects, the method incorporates image correction technique based on diffusion model and  LoRA methods.", "questions": "I would like more detailed explanation for Annealed Dataset Editing, because it should be the center of the contribution. Specifically, please explain how the method differs from Instruct-NeRF2NeRF [Haque et al. 2023].\n\nSmall comments:\n\nLine 197: $c \\in R^d$:     What is $d$?\n\nLines 240-241: Here, $k$ is used for two different meanings. In \"top-k (k=2)\"    and  as a \"temporal\" index in $\\{p_h^k| k \\n N_h^i\\}$. It is misleading. \n\nEquations (1) and (2):  Minus (\"-\" ) symbols are used right after sigma symbols (summation). It seems odd. I think the minus symbols normally goes right before the sigma symbols. \n\nFigure 4: Maybe figure 4 is not referenced from the text.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method deals with editing of GS-represented scenes. From the demo examples, the results of the method seem reasonable. \n\nThe combination of GS-editing and diffusion-based image correction may be a practical solution. \n\nThe method seems to work better than Instruct-NeRF2NeRF.", "weaknesses": "Explanation of the diffusion guidance step in section 3.4 was not clear to me.  The difference from [Haque et al. 2023] is written as equation (5). Since all the information is in equation (5) and Figure 2, I'm not sure whether the proposed method is very similar to [Haque et al. 2023] or not (although there are of course differences between NeRF and GS).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bRhQVFtWox", "reviewer": "Reviewer_Yk2w", "summary": "This work introduces a novel framework for point-based 3D content editing, addressing the limitations of existing 2D editing tools when applied to 3D scenes. The method enables users to intuitively edit 3D scenes by specifying 3D handle and target points, with the system automatically adjusting the geometry while preserving visual details. The approach combines two key innovations: deformation guidance, using 3D Gaussian splatting for geometric changes, and diffusion guidance for correcting content and enhancing visual quality. A multi-step editing strategy further ensures smooth and consistent results. Experiments show that 3DGS-Drag outperforms existing methods in various tasks, including motion changes, shape adjustments, and inpainting, achieving state-of-the-art results with significant efficiency, requiring only 10-20 minutes per edit on an RTX 4090 GPU.", "questions": "The results presented in this work primarily demonstrate edits on small objects or minor scene adjustments. It would be beneficial to evaluate the method’s ability to handle larger object edits within the scene, such as moving a table or a truck.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper is well-written and easy to follow.\n\n* The proposed multi-step progressive editing strategy enables higher-quality edits while maintaining detail and multi-view consistency, addressing limitations found in previous 3D Gaussian Splatting (3DGS)-based editing methods like GaussianEditor and viCA-NeRF.\n\n* The framework uniquely integrates 3D Gaussian Splatting for geometric deformation with diffusion models for content correction. This combination helps maintain visual consistency across views and supports versatile editing capabilities.\n\n* Through qualitative and quantitative comparisons with baseline methods, the paper demonstrates that the proposed method achieves superior results in terms of editing quality, user preference, and performance across different scenarios.", "weaknesses": "1. The study introduces user preference and GPT evaluations for quantitative assessment. However, it does not clearly address potential biases in the user study, such as participant selection. Additionally, the sample size of 19 subjects may be insufficient to support the evaluation's conclusions. Recognizing that quantitative evaluation of diffusion-based methods can be challenging, as is the case with GPT evaluations, it is important to acknowledge these limitations.\n\n2. The qualitative figures do not clearly demonstrate the effectiveness of local editing. The proposed method appears to perform relatively well when editing small objects or areas. It is recommended to highlight the specific changes in the figures to better illustrate the differences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)